Heap Memory Allocation

Overview

The ESP32 has multiple types of RAM. Internally, there’s IRAM, DRAM as well as RAM that can be used as both. It’s also possible to connect external SPI RAM to the ESP32 - external RAM can be integrated into the ESP32’s memory map using the flash cache.

For most purposes, the standard libc malloc() and free() functions can be used for heap allocation without any issues.

However, in order to fully make use of all of the memory types and their characteristics, esp-idf also has a capabilities-based heap memory allocator. If you want to have memory with certain properties (for example, DMA-capable memory, or executable memory), you can create an OR-mask of the required capabilities and pass that to heap_caps_malloc(). For instance, the standard malloc() implementation internally allocates memory via heap_caps_malloc(size, MALLOC_CAP_8BIT) in order to get data memory that is byte-addressable.

Because malloc uses this allocation system as well, memory allocated using heap_caps_malloc() can be freed by calling the standard free() function.

The “soc” component contains a list of memory regions for the chip, along with the type of each memory (aka its tag) and the associated capabilities for that memory type. On startup, a separate heap is initialised for each contiguous memory region. The capabilities-based allocator chooses the best heap for each allocation, based on the requested capabilities.

Special Uses

DMA-Capable Memory

Use the MALLOC_CAP_DMA flag to allocate memory which is suitable for use with hardware DMA engines (for example SPI and I2S). This capability flag excludes any external PSRAM.

32-Bit Accessible Memory

If a certain memory structure is only addressed in 32-bit units, for example an array of ints or pointers, it can be useful to allocate it with the MALLOC_CAP_32BIT flag. This also allows the allocator to give out IRAM memory; something which it can’t do for a normal malloc() call. This can help to use all the available memory in the ESP32.

Memory allocated with MALLOC_CAP_32BIT can only be accessed via 32-bit reads and writes, any other type of access will generate a fatal LoadStoreError exception.

API Reference - Heap Allocation

Functions

void *heap_caps_malloc(size_t size, uint32_t caps)

Allocate a chunk of memory which has the given capabilities.

Equivalent semantics to libc malloc(), for capability-aware memory.

In IDF, malloc(p) is equivalent to heap_caps_malloc(p, MALLOC_CAP_8BIT).

Return
A pointer to the memory allocated on success, NULL on failure
Parameters
  • size: Size, in bytes, of the amount of memory to allocate
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory to be returned

void heap_caps_free(void *ptr)

Free memory previously allocated via heap_caps_malloc() or heap_caps_realloc().

Equivalent semantics to libc free(), for capability-aware memory.

In IDF, free(p) is equivalent to heap_caps_free(p).

Parameters
  • ptr: Pointer to memory previously returned from heap_caps_malloc() or heap_caps_realloc(). Can be NULL.

void *heap_caps_realloc(void *ptr, size_t size, int caps)

Reallocate memory previously allocated via heap_caps_malloc() or heap_caps_realloc().

Equivalent semantics to libc realloc(), for capability-aware memory.

In IDF, realloc(p, s) is equivalent to heap_caps_realloc(p, s, MALLOC_CAP_8BIT).

‘caps’ parameter can be different to the capabilities that any original ‘ptr’ was allocated with. In this way, realloc can be used to “move” a buffer if necessary to ensure it meets a new set of capabilities.

Return
Pointer to a new buffer of size ‘size’ with capabilities ‘caps’, or NULL if allocation failed.
Parameters
  • ptr: Pointer to previously allocated memory, or NULL for a new allocation.
  • size: Size of the new buffer requested, or 0 to free the buffer.
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory desired for the new allocation.

void *heap_caps_calloc(size_t n, size_t size, uint32_t caps)

Allocate a chunk of memory which has the given capabilities. The initialized value in the memory is set to zero.

Equivalent semantics to libc calloc(), for capability-aware memory.

In IDF, calloc(p) is equivalent to heap_caps_calloc(p, MALLOC_CAP_8BIT).

Return
A pointer to the memory allocated on success, NULL on failure
Parameters
  • n: Number of continuing chunks of memory to allocate
  • size: Size, in bytes, of a chunk of memory to allocate
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory to be returned

size_t heap_caps_get_free_size(uint32_t caps)

Get the total free size of all the regions that have the given capabilities.

This function takes all regions capable of having the given capabilities allocated in them and adds up the free space they have.

Note that because of heap fragmentation it is probably not possible to allocate a single block of memory of this size. Use heap_caps_get_largest_free_block() for this purpose.

Return
Amount of free bytes in the regions
Parameters
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory

size_t heap_caps_get_minimum_free_size(uint32_t caps)

Get the total minimum free memory of all regions with the given capabilities.

This adds all the low water marks of the regions capable of delivering the memory with the given capabilities.

Note the result may be less than the global all-time minimum available heap of this kind, as “low water marks” are tracked per-region. Individual regions’ heaps may have reached their “low water marks” at different points in time. However this result still gives a “worst case” indication for all-time minimum free heap.

Return
Amount of free bytes in the regions
Parameters
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory

size_t heap_caps_get_largest_free_block(uint32_t caps)

Get the largest free block of memory able to be allocated with the given capabilities.

Returns the largest value of s for which heap_caps_malloc(s, caps) will succeed.

Return
Size of largest free block in bytes.
Parameters
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory

void heap_caps_get_info(multi_heap_info_t *info, uint32_t caps)

Get heap info for all regions with the given capabilities.

Calls multi_heap_info() on all heaps which share the given capabilities. The information returned is an aggregate across all matching heaps. The meanings of fields are the same as defined for multi_heap_info_t, except that minimum_free_bytes has the same caveats described in heap_caps_get_minimum_free_size().

Parameters
  • info: Pointer to a structure which will be filled with relevant heap metadata.
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory

void heap_caps_print_heap_info(uint32_t caps)

Print a summary of all memory with the given capabilities.

Calls multi_heap_info on all heaps which share the given capabilities, and prints a two-line summary for each, then a total summary.

Parameters
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory

bool heap_caps_check_integrity_all(bool print_errors)

Check integrity of all heap memory in the system.

Calls multi_heap_check on all heaps. Optionally print errors if heaps are corrupt.

Calling this function is equivalent to calling heap_caps_check_integrity with the caps argument set to MALLOC_CAP_INVALID.

Return
True if all heaps are valid, False if at least one heap is corrupt.
Parameters
  • print_errors: Print specific errors if heap corruption is found.

bool heap_caps_check_integrity(uint32_t caps, bool print_errors)

Check integrity of all heaps with the given capabilities.

Calls multi_heap_check on all heaps which share the given capabilities. Optionally print errors if the heaps are corrupt.

See also heap_caps_check_integrity_all to check all heap memory in the system and heap_caps_check_integrity_addr to check memory around a single address.

Return
True if all heaps are valid, False if at least one heap is corrupt.
Parameters
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory
  • print_errors: Print specific errors if heap corruption is found.

bool heap_caps_check_integrity_addr(intptr_t addr, bool print_errors)

Check integrity of heap memory around a given address.

This function can be used to check the integrity of a single region of heap memory, which contains the given address.

This can be useful if debugging heap integrity for corruption at a known address, as it has a lower overhead than checking all heap regions. Note that if the corrupt address moves around between runs (due to timing or other factors) then this approach won’t work and you should call heap_caps_check_integrity or heap_caps_check_integrity_all instead.

Note
The entire heap region around the address is checked, not only the adjacent heap blocks.
Return
True if the heap containing the specified address is valid, False if at least one heap is corrupt or the address doesn’t belong to a heap region.
Parameters
  • addr: Address in memory. Check for corruption in region containing this address.
  • print_errors: Print specific errors if heap corruption is found.

void heap_caps_malloc_extmem_enable(size_t limit)

Enable malloc() in external memory and set limit below which malloc() attempts are placed in internal memory.

When external memory is in use, the allocation strategy is to initially try to satisfy smaller allocation requests with internal memory and larger requests with external memory. This sets the limit between the two, as well as generally enabling allocation in external memory.

Parameters
  • limit: Limit, in bytes.

void *heap_caps_malloc_prefer(size_t size, size_t num, ...)

Allocate a chunk of memory as preference in decreasing order.

Attention
The variable parameters are bitwise OR of MALLOC_CAP_* flags indicating the type of memory. This API prefers to allocate memory with the first parameter. If failed, allocate memory with the next parameter. It will try in this order until allocating a chunk of memory successfully or fail to allocate memories with any of the parameters.
Return
A pointer to the memory allocated on success, NULL on failure
Parameters
  • size: Size, in bytes, of the amount of memory to allocate
  • num: Number of variable paramters

void *heap_caps_realloc_prefer(void *ptr, size_t size, size_t num, ...)

Allocate a chunk of memory as preference in decreasing order.

Return
Pointer to a new buffer of size ‘size’, or NULL if allocation failed.
Parameters
  • ptr: Pointer to previously allocated memory, or NULL for a new allocation.
  • size: Size of the new buffer requested, or 0 to free the buffer.
  • num: Number of variable paramters

void *heap_caps_calloc_prefer(size_t n, size_t size, size_t num, ...)

Allocate a chunk of memory as preference in decreasing order.

Return
A pointer to the memory allocated on success, NULL on failure
Parameters
  • n: Number of continuing chunks of memory to allocate
  • size: Size, in bytes, of a chunk of memory to allocate
  • num: Number of variable paramters

void heap_caps_dump(uint32_t caps)

Dump the full structure of all heaps with matching capabilities.

Prints a large amount of output to serial (because of locking limitations, the output bypasses stdout/stderr). For each (variable sized) block in each matching heap, the following output is printed on a single line:

  • Block address (the data buffer returned by malloc is 4 bytes after this if heap debugging is set to Basic, or 8 bytes otherwise).
  • Data size (the data size may be larger than the size requested by malloc, either due to heap fragmentation or because of heap debugging level).
  • Address of next block in the heap.
  • If the block is free, the address of the next free block is also printed.

Parameters
  • caps: Bitwise OR of MALLOC_CAP_* flags indicating the type of memory

void heap_caps_dump_all()

Dump the full structure of all heaps.

Covers all registered heaps. Prints a large amount of output to serial.

Output is the same as for heap_caps_dump.

Macros

MALLOC_CAP_EXEC

Flags to indicate the capabilities of the various memory systems.

Memory must be able to run executable code

MALLOC_CAP_32BIT

Memory must allow for aligned 32-bit data accesses.

MALLOC_CAP_8BIT

Memory must allow for 8/16/…-bit data accesses.

MALLOC_CAP_DMA

Memory must be able to accessed by DMA.

MALLOC_CAP_PID2

Memory must be mapped to PID2 memory space (PIDs are not currently used)

MALLOC_CAP_PID3

Memory must be mapped to PID3 memory space (PIDs are not currently used)

MALLOC_CAP_PID4

Memory must be mapped to PID4 memory space (PIDs are not currently used)

MALLOC_CAP_PID5

Memory must be mapped to PID5 memory space (PIDs are not currently used)

MALLOC_CAP_PID6

Memory must be mapped to PID6 memory space (PIDs are not currently used)

MALLOC_CAP_PID7

Memory must be mapped to PID7 memory space (PIDs are not currently used)

MALLOC_CAP_SPIRAM

Memory must be in SPI RAM.

MALLOC_CAP_INTERNAL

Memory must be internal; specifically it should not disappear when flash/spiram cache is switched off.

MALLOC_CAP_DEFAULT

Memory can be returned in a non-capability-specific memory allocation (e.g. malloc(), calloc()) call.

MALLOC_CAP_INVALID

Memory can’t be used / list end marker.

Heap Tracing & Debugging

The following features are documented on the Heap Memory Debugging page:

API Reference - Initialisation

Functions

void heap_caps_init()

Initialize the capability-aware heap allocator.

This is called once in the IDF startup code. Do not call it at other times.

void heap_caps_enable_nonos_stack_heaps()

Enable heap(s) in memory regions where the startup stacks are located.

On startup, the pro/app CPUs have a certain memory region they use as stack, so we cannot do allocations in the regions these stack frames are. When FreeRTOS is completely started, they do not use that memory anymore and heap(s) there can be enabled.

esp_err_t heap_caps_add_region(intptr_t start, intptr_t end)

Add a region of memory to the collection of heaps at runtime.

Most memory regions are defined in soc_memory_layout.c for the SoC, and are registered via heap_caps_init(). Some regions can’t be used immediately and are later enabled via heap_caps_enable_nonos_stack_heaps().

Call this function to add a region of memory to the heap at some later time.

This function does not consider any of the “reserved” regions or other data in soc_memory_layout, caller needs to consider this themselves.

All memory within the region specified by start & end parameters must be otherwise unused.

The capabilities of the newly registered memory will be determined by the start address, as looked up in the regions specified in soc_memory_layout.c.

Use heap_caps_add_region_with_caps() to register a region with custom capabilities.

Return
ESP_OK on success, ESP_ERR_INVALID_ARG if a parameter is invalid, ESP_ERR_NOT_FOUND if the specified start address doesn’t reside in a known region, or any error returned by heap_caps_add_region_with_caps().
Parameters
  • start: Start address of new region.
  • end: End address of new region.

esp_err_t heap_caps_add_region_with_caps(const uint32_t caps[], intptr_t start, intptr_t end)

Add a region of memory to the collection of heaps at runtime, with custom capabilities.

Similar to heap_caps_add_region(), only custom memory capabilities are specified by the caller.

Return
  • ESP_OK on success
  • ESP_ERR_INVALID_ARG if a parameter is invalid
  • ESP_ERR_NO_MEM if no memory to register new heap.
  • ESP_FAIL if region overlaps the start and/or end of an existing region
Parameters
  • caps: Ordered array of capability masks for the new region, in order of priority. Must have length SOC_MEMORY_TYPE_NO_PRIOS. Does not need to remain valid after the call returns.
  • start: Start address of new region.
  • end: End address of new region.

Implementation Notes

Knowledge about the regions of memory in the chip comes from the “soc” component, which contains memory layout information for the chip.

Each contiguous region of memory contains its own memory heap. The heaps are created using the multi_heap functionality. multi_heap allows any contiguous region of memory to be used as a heap.

The heap capabilities allocator uses knowledge of the memory regions to initialize each individual heap. When you call a function in the heap capabilities API, it will find the most appropriate heap for the allocation (based on desired capabilities, available space, and preferences for each region’s use) and then call the multi_heap function to use the heap situation in that particular region.

API Reference - Multi Heap API

(Note: The multi heap API is used internally by the heap capabilities allocator. Most IDF programs will never need to call this API directly.)

Functions

void *multi_heap_malloc(multi_heap_handle_t heap, size_t size)

malloc() a buffer in a given heap

Semantics are the same as standard malloc(), only the returned buffer will be allocated in the specified heap.

Return
Pointer to new memory, or NULL if allocation fails.
Parameters
  • heap: Handle to a registered heap.
  • size: Size of desired buffer.

void multi_heap_free(multi_heap_handle_t heap, void *p)

free() a buffer in a given heap.

Semantics are the same as standard free(), only the argument ‘p’ must be NULL or have been allocated in the specified heap.

Parameters
  • heap: Handle to a registered heap.
  • p: NULL, or a pointer previously returned from multi_heap_malloc() or multi_heap_realloc() for the same heap.

void *multi_heap_realloc(multi_heap_handle_t heap, void *p, size_t size)

realloc() a buffer in a given heap.

Semantics are the same as standard realloc(), only the argument ‘p’ must be NULL or have been allocated in the specified heap.

Return
New buffer of ‘size’ containing contents of ‘p’, or NULL if reallocation failed.
Parameters
  • heap: Handle to a registered heap.
  • p: NULL, or a pointer previously returned from multi_heap_malloc() or multi_heap_realloc() for the same heap.
  • size: Desired new size for buffer.

size_t multi_heap_get_allocated_size(multi_heap_handle_t heap, void *p)

Return the size that a particular pointer was allocated with.

Return
Size of the memory allocated at this block. May be more than the original size argument, due to padding and minimum block sizes.
Parameters
  • heap: Handle to a registered heap.
  • p: Pointer, must have been previously returned from multi_heap_malloc() or multi_heap_realloc() for the same heap.

multi_heap_handle_t multi_heap_register(void *start, size_t size)

Register a new heap for use.

This function initialises a heap at the specified address, and returns a handle for future heap operations.

There is no equivalent function for deregistering a heap - if all blocks in the heap are free, you can immediately start using the memory for other purposes.

Return
Handle of a new heap ready for use, or NULL if the heap region was too small to be initialised.
Parameters
  • start: Start address of the memory to use for a new heap.
  • size: Size (in bytes) of the new heap.

void multi_heap_set_lock(multi_heap_handle_t heap, void *lock)

Associate a private lock pointer with a heap.

The lock argument is supplied to the MULTI_HEAP_LOCK() and MULTI_HEAP_UNLOCK() macros, defined in multi_heap_platform.h.

The lock in question must be recursive.

When the heap is first registered, the associated lock is NULL.

Parameters
  • heap: Handle to a registered heap.
  • lock: Optional pointer to a locking structure to associate with this heap.

void multi_heap_dump(multi_heap_handle_t heap)

Dump heap information to stdout.

For debugging purposes, this function dumps information about every block in the heap to stdout.

Parameters
  • heap: Handle to a registered heap.

bool multi_heap_check(multi_heap_handle_t heap, bool print_errors)

Check heap integrity.

Walks the heap and checks all heap data structures are valid. If any errors are detected, an error-specific message can be optionally printed to stderr. Print behaviour can be overriden at compile time by defining MULTI_CHECK_FAIL_PRINTF in multi_heap_platform.h.

Return
true if heap is valid, false otherwise.
Parameters
  • heap: Handle to a registered heap.
  • print_errors: If true, errors will be printed to stderr.

size_t multi_heap_free_size(multi_heap_handle_t heap)

Return free heap size.

Returns the number of bytes available in the heap.

Equivalent to the total_free_bytes member returned by multi_heap_get_heap_info().

Note that the heap may be fragmented, so the actual maximum size for a single malloc() may be lower. To know this size, see the largest_free_block member returned by multi_heap_get_heap_info().

Return
Number of free bytes.
Parameters
  • heap: Handle to a registered heap.

size_t multi_heap_minimum_free_size(multi_heap_handle_t heap)

Return the lifetime minimum free heap size.

Equivalent to the minimum_free_bytes member returned by multi_heap_get_info().

Returns the lifetime “low water mark” of possible values returned from multi_free_heap_size(), for the specified heap.

Return
Number of free bytes.
Parameters
  • heap: Handle to a registered heap.

void multi_heap_get_info(multi_heap_handle_t heap, multi_heap_info_t *info)

Return metadata about a given heap.

Fills a multi_heap_info_t structure with information about the specified heap.

Parameters
  • heap: Handle to a registered heap.
  • info: Pointer to a structure to fill with heap metadata.

Structures

struct multi_heap_info_t

Structure to access heap metadata via multi_heap_get_info.

Public Members

size_t total_free_bytes

Total free bytes in the heap. Equivalent to multi_free_heap_size().

size_t total_allocated_bytes

Total bytes allocated to data in the heap.

size_t largest_free_block

Size of largest free block in the heap. This is the largest malloc-able size.

size_t minimum_free_bytes

Lifetime minimum free heap size. Equivalent to multi_minimum_free_heap_size().

size_t allocated_blocks

Number of (variable size) blocks allocated in the heap.

size_t free_blocks

Number of (variable size) free blocks in the heap.

size_t total_blocks

Total number of (variable size) blocks in the heap.

Type Definitions

typedef struct multi_heap_info *multi_heap_handle_t

Opaque handle to a registered heap.