# Concurrency Constraints for Flash on SPI0/1

[中文]

The SPI0/1 bus is shared between the cache and the SPI1 peripheral (controlled by the drivers including this SPI Flash driver). Operations to SPI1 may cause significant influence to the cache and hence the whole system. There are no such constraints and impacts for flash chips connected to other SPI buses, which are not covered in this document.

There are three kinds of activities that can happen on SPI0/1 bus:

- Flash writing operations (via SPI1). For example, erasing, page programming, or status register writing commands (e.g., `SE`, `PP`, and `WRSR`). During these commands, the flash is in a unreadable state. The CPU and the cache have to wait until the writing command is completed. APIs below can trigger writing commands:

  > - Calling non_encrypted SPI flash write API (`esp_flash_write()`, `esp_flash_erase_region()`, etc.)

  > - Calling `esp_flash_write_encrypted()`

- Short operations (via SPI1, includes non-writing flash commands). APIs below can trigger short operations:

  > - Calling non_encrypted SPI flash read API (`esp_flash_read()`, etc.)

- Cache read (via SPI0). Following API and operations can trigger cache read:

  > - Code execution from SPI Flash or PSRAM

  > - Fetch static data of .data/.rodata/.bss segment from SPI Flash or PSRAM

  > - All other read/write operation to the PSRAM via the heap or esp_himem

  > - Read from area mapped to SPI Flash, includes:

  >   > - mmap-like functions: `spi_flash_mmap()`, `spi_flash_mmap_pages()`, `esp_mmu_map()`, `bootloader_mmap()`, and `esp_partition_mmap()`.

  >   > - Functions relying on `spi_flash_mmap()`: `esp_partition_find()`, `esp_partition_register_external()`.

  >   > - Encrypted flash read/write APIs `esp_flash_read_encrypted()` and `esp_flash_write_encrypted()` (on esp32, or for data validation).

All SPI flash APIs are exclusive to each other by some internal mutex provided by the driver.

For all SPI1 operations (read/write), caches are disabled during these operations by default. Most tasks will be disabled, and access to Flash/PSRAM is forbidden. See Cache Disabled (Default) for more details.

Some options help reduce the impact of cache disabling. The impact of write operations differs between modes.

- XIP from PSRAM: In this mode, all segments that were previously executed from Flash are loaded and executed from PSRAM instead. As a result, the cache can remain enabled while the flash is being erased or written, and code execution is not affected by write operations in most cases. See Executing Code from PSRAM for more details.

- Auto Suspend: In this mode, when cache access to flash misses during flash erase/write operations, it is allowed to suspend the flash writing to read from it transparently with some latency. As a result, caches are kept enabled and code execution won't be affected so much during writing operations.

  > This is an optional feature that depends on special SPI Flash models, hence disabled by default. See Optional Features for Flash and Flash Auto Suspend Feature for more details.

See OS Functions and SPI Bus Lock for the detailed information of software implementation.

## Cache Disabled (Default)

By default, caches are disabled during SPI1 operations (read/write). All SPI1 operations will automatically and transparently disable the caches.

When the caches are disabled, all non-IRAM-safe interrupts will be disabled, and all other tasks are suspended. The other core will be polling in a busy loop. Only IRAM-safe interrupt handlers will be executed. These will be restored when the Flash operation completes.

See IRAM-Safe Interrupt Handlers for information on how to prevent an interrupt handler from being disabled when the cache is disabled.

When the cache is disabled, all CPUs should execute code and access data only from internal RAM. For differences between internal RAM (e.g., IRAM, DRAM) and flash cache, please refer to the application memory layout documentation.

### IRAM-Safe Interrupt Handlers

For interrupt handlers which need to execute when the cache is disabled (e.g., for low latency operations), set the `ESP_INTR_FLAG_IRAM` flag when the interrupt handler is registered.

You must ensure that all data and functions accessed by these interrupt handlers, including the ones that handlers call, are located in IRAM or DRAM. See How to Place Code in IRAM.

If a function or symbol is not correctly put into IRAM/DRAM, and the interrupt handler reads from the flash cache during a flash operation, it will cause a crash. This may be due to an Illegal Instruction exception (for code which should be in IRAM) or garbage data being read (for constant data which should be in DRAM).

Note

When working with strings in ISRs, it is not advised to use `printf` and other output functions. For debugging purposes, use `ESP_DRAM_LOGE()` and similar macros when logging from ISRs. Make sure that both `TAG` and format string are placed into `DRAM` in that case.

### Non-IRAM-Safe Interrupt Handlers

If the `ESP_INTR_FLAG_IRAM` flag is not set when registering, the interrupt handler will not be executed when the caches are disabled. Once the caches are restored, the non-IRAM-safe interrupts will be re-enabled. After this moment, the interrupt handler will run normally again. This means that as long as caches are disabled, the corresponding hardware events will not occur.

## When DMA Read Data from Flash

The Flash device doesn't allow reading while it is being erased/programmed, even when the data is not in the region being erased/programmed.

When the flash is being erased/programmed, the Flash data read by DMA is unpredictable. It is recommended to stop DMA access to Flash before erasing or writing to it. If DMA cannot be stopped (for example, the LCD needs to continuously refresh image data stored in Flash), it is advisable to copy such data to PSRAM or internal SRAM.

## Flash Auto Suspend Feature

Important

1. The flash chip you are using should have a suspend/resume feature.

2. The MSPI hardware should support the auto-suspend feature, i.e., hardware can send suspend command automatically.

If you use suspend feature on an unsupported chip, it may cause a severe crash. Therefore, we strongly suggest you reading the flash chip datasheets first. Ensure the flash chip satisfies the following conditions at minimum.

1. With the current software implementation, SUS bit in status registers should in SR2 bit7 or SR bit15.

2. With the current software implementation, suspend command should be 75H, with resume command being 7AH.

3. When the flash is successfully suspended, all address of the flash, except from the section/block being erased, can be read correctly. At this state，resume can be sent immediately as well.

4. When the flash is successfully resumed, another suspend can be sent immediately at this state.

When CONFIG_SPI_FLASH_AUTO_SUSPEND is enabled, the caches will be kept enabled. They would be disabled if CONFIG_SPI_FLASH_AUTO_SUSPEND is disabled. The hardware handles the arbitration between SPI0 and SPI1. If the SPI1 operation is short, such as a reading operation, the CPU and the cache will wait until the SPI1 operation is completed. However, during processes like erasing, page programming, or status register writing (e.g., `SE`, `PP`, and `WRSR`), an auto suspend will happen, interrupting the ongoing flash operation. This allows the CPU to access data from the cache and flash within limited time.

This approach allows certain code/variables to be stored in flash/PSRAM instead of IRAM/DRAM, while still being executable during flash erasing. This reduces the usage of IRAM/DRAM.

Please note that this feature comes with the overhead of flash suspend/resume. Frequent interruptions during flash erasing can significantly prolong the erasing process. To avoid this, you may use FreeRTOS task priorities to ensure that only real-time critical tasks are executed at a higher priority than flash erasing, allowing the flash erasing to complete in reasonable time.

There are three kinds of code:

1. Critical code: inside IRAM/DRAM. This kind of code usually has high performance requirements, related to cache/flash/PSRAM, or is called very often.

2. Cached code: inside flash/PSRAM. This kind of code has lower performance requirements or is called less often. They will execute during erasing, with some overhead.

3. Low-priority code: inside flash/PSRAM and disabled during erasing. This kind of code should be forbidden from being executed to avoid affecting the flash erasing, by setting a lower task priority than the erasing task.

Regarding the flash suspend feature usage and corresponding response time delay, please also see the system/flash_suspend example.

Note

The flash auto suspend feature relies heavily on strict timing. You can see it as a kind of optimization plan, which means that you cannot use it in every situation, like high requirement of real-time system or triggering interrupt very frequently (e.g., LCD flush, bluetooth, Wi-Fi, etc.). You should take following steps to evaluate what kind of ISR can be used together with flash suspend.

As you can see from the diagram, two key values should be noted:

1. ISR time: The ISR time cannot be very long, at least not larger than the value you set in `IWDT`. But it will significantly lengthen the erasing/writing completion time.

2. ISR interval: ISR cannot be triggered very often. The most important time is the ISR interval minus ISR time (from point b to point c in the diagram). During this time, SPI1 will send resume command to restart the operation. However, it needs a time `tsus` for preparation, and the typical value of `tsus` is about 40 us. If SPI1 cannot resume the operation but another suspend command comes, it will cause CPU starve and `TWDT` may be triggered.

The `tsus` time mentioned in point 2 can be found by looking through the flash datasheets, usually in the AC CHARACTERISTICS section. Users needs to make sure that the `tsus` value obtained from the datasheets is not greater than the CONFIG_SPI_FLASH_SUSPEND_TSUS_VAL_US value in Kconfig.

Furthermore, the flash suspend might be delayed. If both the CPU and the cache access the flash via SPI0 frequently and SPI1 sends the suspend command frequently as well, the efficiency of MSPI data transfer will be influenced. So, we have a lock inside to prevent this. When SPI1 sends the suspend command, SPI0 will take over memory SPI bus and take the lock. After SPI0 finishes sending data, it will retain control of the memory SPI bus until the lock delay period time finishes. During this lock delay period, if there is any other SPI0 transaction, then the SPI0 transaction will be proceeded and a new lock delay period will start. Otherwise, SPI0 will release the memory bus and start SPI0/1 arbitration.

### Advanced Suspend-resume Usage

In the default configuration, flash suspend is issued automatically by hardware, and the resume is also handled by hardware at an appropriate moment, while a fixed time delay is used to wait for the suspend command to take effect. This implementation already covers the majority of use cases. However, in scenarios that have stricter requirements on real-time response, performance, or suspend granularity, ESP-IDF also provides additional advanced configuration options for fine-tuning the suspend-resume behavior.

1. Detect Whether Suspend Has Taken Effect via Flash Register

Normally, after issuing the suspend command, the hardware waits for a fixed period of time defined by `tsus` before handing the memory bus back to SPI0/CPU. Because this delay must be configured according to the worst case given in the datasheet, it is conservative in most situations.

By enabling CONFIG_SPI_FLASH_AUTO_CHECK_SUSPEND_STATUS, the hardware will instead poll the `WIP` bit in the flash status register to determine whether the suspend command has actually taken effect, rather than waiting for the fixed time given by CONFIG_SPI_FLASH_SUSPEND_TSUS_VAL_US. Since the actual suspend setup time is usually much shorter than the maximum value listed in the datasheet, this approach can significantly reduce the overhead during suspend and improve overall performance.

Important

Before enabling this option, please confirm that the behavior of the `WIP` bit after suspend on your flash chip matches the expectation. Most flash chips that support suspend/resume satisfy this requirement, but it is still recommended to consult the corresponding datasheet to verify it.

2. Software-controlled Flash Resume

Normally, after the flash is suspended, the hardware will automatically arrange an appropriate moment to send the resume command, allowing the flash to continue its original erase/write operation. This implementation is transparent to software, but it has a side effect: while a high-priority task or interrupt is still running, the hardware may again initiate a suspend/resume sequence, which interrupts those tasks and affects the continuity and timing of their execution.

By enabling CONFIG_SPI_FLASH_SOFTWARE_RESUME, the hardware auto-resume feature is disabled, and the flash resume operation is instead issued by software at an appropriate point. With this option enabled, once the flash is suspended it will stay suspended until software explicitly resumes it. In the SPI1 wait-idle flow, software actively checks the flash's suspend state, and if it finds the flash in a suspended state, it calls the driver's resume interface to let the flash continue the original operation. This means that the resume command is only issued after the high-priority task or interrupt has truly finished and software has returned to the SPI1 operation context, avoiding repeated bus preemption caused by suspend-resume on high-priority paths.

Because this mechanism relies on the software layer to perform resume at well-defined execution points, and the current implementation does not provide corresponding protection for multi-core scenarios, this option has the following limitations:

- It is only supported on single-core scenarios. CONFIG_FREERTOS_UNICORE must be enabled.

- It is an experimental feature and is only visible after CONFIG_IDF_EXPERIMENTAL_FEATURES is enabled.

- This feature improves the continuity of interrupt response and the overall application performance. However, it also increases the time needed to complete a single flash operation.

## Executing Code from PSRAM

Select CONFIG_SPIRAM_XIP_FROM_PSRAM config to enable this mode. In this mode, code is executed from PSRAM, and the cache will not be disabled during write APIs in most cases.

In this mode, the flash `.text` sections (used for instructions) and the flash `.rodata` sections (used for read-only data) will be loaded into PSRAM at startup. The corresponding virtual addresses will be mapped to PSRAM. You do not need to ensure that code and data executed while the flash is being erased or programmed reside in IRAM.

### Exception: Cache-Mapped Regions in Flash

Due to the restriction from SPI Nor Flash parts, access to cache mapped regions in flash (mapped via APIs like spi_flash_mmap) is still not allowed while the flash is being erased/written, regardless of whether the erase/write region and the mapped region overlap. In this case, cache should still be disabled to prevent reading corrupted data from the cache.

To prevent cache disabling, a lock is implemented inside the SPI Flash driver to ensure mutual exclusion between cache mapping and flash writing, and most ESP-IDF APIs that perform flash mapping use this flag. If mmap-like APIs are called by yourself, you can specify this flag `SPI_FLASH_MMAP_FLAG_BLOCKS_WRITE` to prevent cache disabling. You cannot use this flag in a task that uses `esp_flash_erase_*` or `esp_flash_write` between `spi_flash_mmap` and `spi_flash_munmap` (regardless of whether the write region and mapped region overlap), otherwise it will cause a deadlock. See About the SPI_FLASH_MMAP_FLAG_BLOCKS_WRITE flag for more details about the flag.

If mmap-like APIs are called without this flag, the cache will still be disabled when flash erasing or writing happens.

Was this page helpful?

- Thank you! We received your feedback.
- If you have any comments, fill in Espressif Documentation Feedback Form.

- We value your feedback.
- Let us know how we can improve this page by filling in Espressif Documentation Feedback Form.