Concurrency Constraints for Flash on SPI0/1
The SPI0/1 bus is shared between the cache and the SPI1 peripheral (controlled by the drivers including this SPI Flash driver). Operations to SPI1 may cause significant influence to the cache and hence the whole system. There are no such constraints and impacts for flash chips connected to other SPI buses, which are not covered in this document.
There are three kinds of activities that can happen on SPI0/1 bus:
Flash writing operations (via SPI1). For example, erasing, page programming, or status register writing commands (e.g.,
SE,PP, andWRSR). During these commands, the flash is in a unreadable state. The CPU and the cache have to wait until the writing command is completed. APIs below can trigger writing commands:Calling non_encrypted SPI flash write API (
esp_flash_write(),esp_flash_erase_region(), etc.)Calling
esp_flash_write_encrypted()
Short operations (via SPI1, includes non-writing flash commands). APIs below can trigger short operations:
Calling non_encrypted SPI flash read API (
esp_flash_read(), etc.)
Cache read (via SPI0). Following API and operations can trigger cache read:
Code execution from SPI Flash or PSRAM
Fetch static data of .data/.rodata/.bss segment from SPI Flash or PSRAM
All other read/write operation to the PSRAM via the heap or esp_himem
Read from area mapped to SPI Flash, includes:
mmap-like functions:
spi_flash_mmap(),spi_flash_mmap_pages(),esp_mmu_map(),bootloader_mmap(), andesp_partition_mmap().Functions relying on
spi_flash_mmap():esp_partition_find(),esp_partition_register_external().Encrypted flash read/write APIs
esp_flash_read_encrypted()andesp_flash_write_encrypted()(on esp32, or for data validation).
All SPI flash APIs are exclusive to each other by some internal mutex provided by the driver.
For all SPI1 operations (read/write), caches are disabled during these operations by default. Most tasks will be disabled, and access to Flash/PSRAM is forbidden. See Cache Disabled (Default) for more details.
Some options help reduce the impact of cache disabling. The impact of write operations differs between modes.
Auto Suspend: In this mode, when cache access to flash misses during flash erase/write operations, it is allowed to suspend the flash writing to read from it transparently with some latency. As a result, caches are kept enabled and code execution won't be affected so much during writing operations.
This is an optional feature that depends on special SPI Flash models, hence disabled by default. See Optional Features for Flash and Flash Auto Suspend Feature for more details.
See OS Functions and SPI Bus Lock for the detailed information of software implementation.
Cache Disabled (Default)
By default, caches are disabled during SPI1 operations (read/write). All SPI1 operations will automatically and transparently disable the caches.
When the caches are disabled, all non-IRAM-safe interrupts will be disabled, and all other tasks are suspended. Only IRAM-safe interrupt handlers will be executed. These will be restored when the Flash operation completes.
See IRAM-Safe Interrupt Handlers for information on how to prevent an interrupt handler from being disabled when the cache is disabled.
When the cache is disabled, all CPUs should execute code and access data only from internal RAM. For differences between internal RAM (e.g., IRAM, DRAM) and flash cache, please refer to the application memory layout documentation.
IRAM-Safe Interrupt Handlers
For interrupt handlers which need to execute when the cache is disabled (e.g., for low latency operations), set the ESP_INTR_FLAG_IRAM flag when the interrupt handler is registered.
You must ensure that all data and functions accessed by these interrupt handlers, including the ones that handlers call, are located in IRAM or DRAM. See How to Place Code in IRAM.
If a function or symbol is not correctly put into IRAM/DRAM, and the interrupt handler reads from the flash cache during a flash operation, it will cause a crash. This may be due to an Illegal Instruction exception (for code which should be in IRAM) or garbage data being read (for constant data which should be in DRAM).
Note
When working with strings in ISRs, it is not advised to use printf and other output functions. For debugging purposes, use ESP_DRAM_LOGE() and similar macros when logging from ISRs. Make sure that both TAG and format string are placed into DRAM in that case.
Non-IRAM-Safe Interrupt Handlers
If the ESP_INTR_FLAG_IRAM flag is not set when registering, the interrupt handler will not be executed when the caches are disabled. Once the caches are restored, the non-IRAM-safe interrupts will be re-enabled. After this moment, the interrupt handler will run normally again. This means that as long as caches are disabled, the corresponding hardware events will not occur.
Flash Auto Suspend Feature
Important
The flash chip you are using should have a suspend/resume feature.
The MSPI hardware should support the auto-suspend feature, i.e., hardware can send suspend command automatically.
If you use suspend feature on an unsupported chip, it may cause a severe crash. Therefore, we strongly suggest you reading the flash chip datasheets first. Ensure the flash chip satisfies the following conditions at minimum.
With the current software implementation, SUS bit in status registers should in SR2 bit7 or SR bit15.
With the current software implementation, suspend command should be 75H, with resume command being 7AH.
When the flash is successfully suspended, all address of the flash, except from the section/block being erased, can be read correctly. At this state，resume can be sent immediately as well.
When the flash is successfully resumed, another suspend can be sent immediately at this state.
When CONFIG_SPI_FLASH_AUTO_SUSPEND is enabled, the caches will be kept enabled. They would be disabled if CONFIG_SPI_FLASH_AUTO_SUSPEND is disabled. The hardware handles the arbitration between SPI0 and SPI1. If the SPI1 operation is short, such as a reading operation, the CPU and the cache will wait until the SPI1 operation is completed. However, during processes like erasing, page programming, or status register writing (e.g., SE, PP, and WRSR), an auto suspend will happen, interrupting the ongoing flash operation. This allows the CPU to access data from the cache and flash within limited time.
This approach allows certain code/variables to be stored in flash/PSRAM instead of IRAM/DRAM, while still being executable during flash erasing. This reduces the usage of IRAM/DRAM.
Please note that this feature comes with the overhead of flash suspend/resume. Frequent interruptions during flash erasing can significantly prolong the erasing process. To avoid this, you may use FreeRTOS task priorities to ensure that only real-time critical tasks are executed at a higher priority than flash erasing, allowing the flash erasing to complete in reasonable time.
There are three kinds of code:
Critical code: inside IRAM/DRAM. This kind of code usually has high performance requirements, related to cache/flash/PSRAM, or is called very often.
Cached code: inside flash/PSRAM. This kind of code has lower performance requirements or is called less often. They will execute during erasing, with some overhead.
Low-priority code: inside flash/PSRAM and disabled during erasing. This kind of code should be forbidden from being executed to avoid affecting the flash erasing, by setting a lower task priority than the erasing task.
Regarding the flash suspend feature usage and corresponding response time delay, please also see the system/flash_suspend example.
Note
The flash auto suspend feature relies heavily on strict timing. You can see it as a kind of optimization plan, which means that you cannot use it in every situation, like high requirement of real-time system or triggering interrupt very frequently (e.g., LCD flush, bluetooth, Wi-Fi, etc.). You should take following steps to evaluate what kind of ISR can be used together with flash suspend.
As you can see from the diagram, two key values should be noted:
ISR time: The ISR time cannot be very long, at least not larger than the value you set in
IWDT. But it will significantly lengthen the erasing/writing completion time.ISR interval: ISR cannot be triggered very often. The most important time is the ISR interval minus ISR time (from point b to point c in the diagram). During this time, SPI1 will send resume command to restart the operation. However, it needs a time
tsusfor preparation, and the typical value oftsusis about 40 us. If SPI1 cannot resume the operation but another suspend command comes, it will cause CPU starve andTWDTmay be triggered.
The tsus time mentioned in point 2 can be found by looking through the flash datasheets, usually in the AC CHARACTERISTICS section. Users needs to make sure that the tsus value obtained from the datasheets is not greater than the CONFIG_SPI_FLASH_SUSPEND_TSUS_VAL_US value in Kconfig.
Furthermore, the flash suspend might be delayed. If both the CPU and the cache access the flash via SPI0 frequently and SPI1 sends the suspend command frequently as well, the efficiency of MSPI data transfer will be influenced. So, we have a lock inside to prevent this. When SPI1 sends the suspend command, SPI0 will take over memory SPI bus and take the lock. After SPI0 finishes sending data, it will retain control of the memory SPI bus until the lock delay period time finishes. During this lock delay period, if there is any other SPI0 transaction, then the SPI0 transaction will be proceeded and a new lock delay period will start. Otherwise, SPI0 will release the memory bus and start SPI0/1 arbitration.