Memory Synchronization

Introduction

ESP32-S2 can access its connected PSRAM via these ways:

  • CPU

  • DMA

By default, CPU accesses the above mentioned memory via cache. Whereas DMA accesses the memory directly, without going through cache.

This leads to potential cache data coherence issue:

  • When a DMA transaction changes the content of a piece of memory, and the content has been cached already. Under this condition:

    • CPU may read stale data.

    • the stale data in the cache may be written back to the memory. The new data updated by the previous DMA transaction will be overwritten.

  • CPU changes the content of an address. The content is in the cache, but not in the memory yet (cache will write back the content to the memory according to its own strategy). Under this condition:

    • The next DMA transactions to read this content from the memory will get stale data.

There are three common methods to address such cache data coherence issue:

  1. Hardware based cache Coherent Interconnect, ESP32-S2 does not have such ability.

  2. Use the DMA buffer from non-cacheable memory. Memory that CPU access it without going through cache is called non-cacheable memory.

  3. Explicitly call a memory synchronization API to writeback the content in the cache back to the memory, or invalidate the content in the cache.

Memory Synchronisation Driver

The suggested way to deal with such cache data coherence issue is by using the memory synchronization API esp_cache_msync() provided by ESP-IDF esp_mm component.

Driver Concept

Direction of the cache memory synchronization:

Type of the cache memory synchronization:

Driver Behaviour

Calling esp_cache_msync() will do a synchronization between cache and memory. The first parameter addr and the second parameter size together describe the memory region that is to be synchronized. About the third parameter flags:

  • ESP_CACHE_MSYNC_FLAG_DIR_C2M. With this flag, content in the specified address region is written back to the memory. This direction is usually used after the content of an address is updated by the CPU, e.g. a memset to the address. Operation in this direction should happen before a DMA operation to the same address.

  • ESP_CACHE_MSYNC_FLAG_DIR_M2C. With this flag, content in the specified address region is invalidated from the cache. This direction is usually used after the content of an address is updated by the DMA. Operation in this direction should happen before a CPU read operation to the same address.

The above two flags help select the synchronization direction. Specially, if neither of these two flags are used, esp_cache_msync() will by default select the ESP_CACHE_MSYNC_FLAG_DIR_C2M direction. Users are not allowed to set both of the two flags at the same time.

The above two flags help select the type of the synchronization address. Specially, if neither of these two flags are used, esp_cache_msync() will by default select the ESP_CACHE_MSYNC_FLAG_TYPE_DATA direction. Users are not allowed to set both of the two flags at the same time.

Address Alignment Requirement

There is address and size alignment requirement (in bytes) for using esp_cache_msync(). The alignment requirement comes from cache.

  • An address region whose start address and size both meet the cache memory synchronization alignment requirement is defined as an aligned address region.

  • An address region whose start address or size does not meet the cache memory synchronization alignment requirement is defined as an unaligned address region.

By default, if you specify an unaligned address region, esp_cache_msync() will return an ESP_ERR_INVALID_ARG error, together with the required alignment.

Memory Allocation Helper

cache memory synchronization is usually considered when DMA is involved. ESP-IDF provides an API to do memory allocation that can meet the alignment requirement from both the cache and the DMA.

  • esp_dma_capable_malloc(), this API allocates a chunk of memory that meets the alignment requirement from both the cache and the DMA.

  • esp_dma_capable_calloc(), this API allocates a chunk of memory that meets the alignment requirement from both the cache and the DMA. The initialized value in the memory is set to zero.

You can also use ESP_DMA_MALLOC_FLAG_PSRAM to allocate from the PSRAM.

Warning for Address Alignment Requirement

You can set the ESP_CACHE_MSYNC_FLAG_UNALIGNED flag to bypass such check. Note you should be very careful about using this flag. cache memory synchronization to an unaligned address region may silently corrupt the memory.

For example, assume:

  • alignment requirement is 0x40 bytes.

  • a call to esp_cache_msync(), with ESP_CACHE_MSYNC_FLAG_DIR_M2C | ESP_CACHE_MSYNC_FLAG_UNALIGNED flags, the specified address region is 0x4000_0020 ~ 0x4000_0060 (see data C in below graph).

Above settings will trigger a cache invalidation to the address region 0x4000_0000 ~ 0x4000_0080, see sync item0 and sync item1 in the below graph.

If the content in 0x4000_0000 ~ 0x4000_0020 (data A in the below graph) or 0x4000_0060 ~ 0x4000_0080 (data B in the below graph) are not written back to the memory yet, then these data A and data B will be discarded.

../../_images/cache_align_issue.png

API Reference

API Reference - ESP Msync Driver

Header File

  • components/esp_mm/include/esp_cache.h

  • This header file can be included with:

    #include "esp_cache.h"

  • This header file is a part of the API provided by the esp_mm component. To declare that your component depends on esp_mm, add the following to your CMakeLists.txt:

    REQUIRES esp_mm

    or

    PRIV_REQUIRES esp_mm

Functions

esp_err_t esp_cache_msync(void *addr, size_t size, int flags)

Memory sync between Cache and storage memory.

For cache-to-memory (C2M) direction:

  • For cache writeback supported chips (you can refer to SOC_CACHE_WRITEBACK_SUPPORTED in soc_caps.h)

    • This API will do a writeback to synchronise between cache and storage memory

    • With ESP_CACHE_MSYNC_FLAG_INVALIDATE, this API will also invalidate the values that just written

    • Note: although ESP32 is with PSRAM, but cache writeback isn't supported, so this API will do nothing on ESP32

  • For other chips, this API will do nothing. The out-of-sync should be already dealt by the SDK

For memory-to-cache (M2C) direction:

  • This API will by default do an invalidation

This API is cache-safe and thread-safe

Note

If you don't set direction (ESP_CACHE_MSYNC_FLAG_DIR_x flags), this API is by default C2M direction

Note

If you don't set type (ESP_CACHE_MSYNC_FLAG_TYPE_x flags), this API is by default doing msync for data

Note

You should not call this during any Flash operations (e.g. esp_flash APIs, nvs and some other APIs that are based on esp_flash APIs)

Note

If XIP_From_PSRAM is enabled (by enabling both CONFIG_SPIRAM_FETCH_INSTRUCTIONS and CONFIG_SPIRAM_RODATA), you can call this API during Flash operations

Parameters

  • addr -- [in] Starting address to do the msync

  • size -- [in] Size to do the msync

  • flags -- [in] Flags, see ESP_CACHE_MSYNC_FLAG_x

Returns

  • ESP_OK:

    • Successful msync

    • For C2M direction, if this chip doesn't support cache writeback, if the input addr is a cache supported one, this API will return ESP_OK

  • ESP_ERR_INVALID_ARG: Invalid argument, not cache supported addr, see printed logs

Macros

ESP_CACHE_MSYNC_FLAG_INVALIDATE

Do an invalidation.

Cache msync flags

  • For cache-to-memory (C2M) direction: setting this flag will start an invalidation after the cache writeback operation

  • For memory-to-cache (M2C) direction: setting / unsetting this flag will behave similarly, trigger an invalidation

ESP_CACHE_MSYNC_FLAG_UNALIGNED

Allow msync to a address block that are not aligned to the data cache line size.

ESP_CACHE_MSYNC_FLAG_DIR_C2M

Cache msync direction: from Cache to memory.

Note

If you don't set direction (ESP_CACHE_MSYNC_FLAG_DIR_x flags), it is by default cache-to-memory (C2M) direction

ESP_CACHE_MSYNC_FLAG_DIR_M2C

Cache msync direction: from memory to Cache.

ESP_CACHE_MSYNC_FLAG_TYPE_DATA

Cache msync type: data.

Note

If you don't set type (ESP_CACHE_MSYNC_FLAG_TYPE_x flags), it is by default data type

ESP_CACHE_MSYNC_FLAG_TYPE_INST

Cache msync type: instruction.

API Reference - ESP DMA Utils

Header File

Functions

esp_err_t esp_dma_capable_malloc(size_t size, const esp_dma_mem_info_t *dma_mem_info, void **out_ptr, size_t *actual_size)

Helper function for malloc a DMA capable memory buffer.

Note

This API will take care of the cache alignment internally, you will need to set esp_dma_mem_info_t: dma_alignment_bytes with either the custom alignment or DMA alignment of used peripheral driver.

Parameters

  • size -- [in] Size in bytes, the amount of memory to allocate

  • dma_mem_info -- [in] DMA and memory info, see esp_dma_mem_info_t

  • out_ptr -- [out] A pointer to the memory allocated successfully

  • actual_size -- [out] Actual size for allocation in bytes, when the size you specified doesn't meet the DMA alignment requirements, this value might be bigger than the size you specified. Set null if you don't care this value.

Returns

  • ESP_OK:

  • ESP_ERR_INVALID_ARG: Invalid argument

  • ESP_ERR_NO_MEM: No enough memory for allocation

esp_err_t esp_dma_capable_calloc(size_t calloc_num, size_t size, const esp_dma_mem_info_t *dma_mem_info, void **out_ptr, size_t *actual_size)

Helper function for calloc a DMA capable memory buffer.

Parameters

  • calloc_num -- [in] Number of elements to allocate

  • size -- [in] Size in bytes, the amount of memory to allocate

  • dma_mem_info -- [in] DMA and memory info, see esp_dma_mem_info_t

  • out_ptr -- [out] A pointer to the memory allocated successfully

  • actual_size -- [out] Actual size for allocation in bytes, when the size you specified doesn't meet the DMA alignment requirements, this value might be bigger than the size you specified. Set null if you don't care this value.

Returns

  • ESP_OK:

  • ESP_ERR_INVALID_ARG: Invalid argument

  • ESP_ERR_NO_MEM: No enough memory for allocation

bool esp_dma_is_buffer_alignment_satisfied(const void *ptr, size_t size, esp_dma_mem_info_t dma_mem_info)

Helper function to check if a DMA buffer pointer and size meet both hardware alignment requirements and custom alignment requirements.

Parameters

  • ptr -- [in] Pointer to the buffer

  • size -- [in] Size of the buffer

  • dma_mem_info -- [in] DMA and memory info, see esp_dma_mem_info_t

Returns

  • True: Buffer is aligned

  • False: Buffer is not aligned, or buffer is not DMA capable

esp_err_t esp_dma_malloc(size_t size, uint32_t flags, void **out_ptr, size_t *actual_size)

Note

This API will use MAX alignment requirement

esp_err_t esp_dma_calloc(size_t n, size_t size, uint32_t flags, void **out_ptr, size_t *actual_size)

Note

This API will use MAX alignment requirement

bool esp_dma_is_buffer_aligned(const void *ptr, size_t size, esp_dma_buf_location_t location)

Note

This API will use MAX alignment requirement

Structures

struct esp_dma_mem_info_t

DMA Mem info.

Public Members

int extra_heap_caps

extra heap caps based on MALLOC_CAP_DMA

size_t dma_alignment_bytes

DMA alignment.

struct dma_alignment_info_t

Needed info to get GDMA alignment.

Public Members

bool is_desc

allocate DMA descriptor

bool on_psram

allocate DMA from the PSRAM

Macros

ESP_DMA_MALLOC_FLAG_PSRAM

Memory is in PSRAM.

DMA malloc flags

Enumerations

enum esp_dma_buf_location_t

DMA buffer location.

Values:

enumerator ESP_DMA_BUF_LOCATION_INTERNAL

DMA buffer is in internal memory.

enumerator ESP_DMA_BUF_LOCATION_PSRAM

DMA buffer is in PSRAM.

enumerator ESP_DMA_BUF_LOCATION_AUTO

Auto detect buffer location, under this condition API will loop to search the buffer location.

Provide feedback about this document