Model API Reference

This section covers model loading and static memory planning, making it convenient for users to directly load and run ESPDL models.

Header File

Macros

DL_LOG_INFER_LATENCY_INIT_WITH_SIZE(size)
DL_LOG_INFER_LATENCY_INIT()
DL_LOG_INFER_LATENCY_START()
DL_LOG_INFER_LATENCY_END()
DL_LOG_INFER_LATENCY_PRINT(prefix, key)
DL_LOG_INFER_LATENCY_END_PRINT(prefix, key)
DL_LOG_INFER_LATENCY_ARRAY_INIT_WITH_SIZE(n, size)
DL_LOG_INFER_LATENCY_ARRAY_INIT(n)
DL_LOG_INFER_LATENCY_ARRAY_START(i)
DL_LOG_INFER_LATENCY_ARRAY_END(i)
DL_LOG_INFER_LATENCY_ARRAY_PRINT(i, prefix, key)
DL_LOG_INFER_LATENCY_ARRAY_END_PRINT(i, prefix, key)

Classes

class Model

Neural Network Model.

Public Functions

inline bool is_built() const

Whether the current execution plan has successfully bound its variable arenas. A failed rebuild leaves the model safely non-runnable.

Returns:

true when the model is ready to run, otherwise false.

Model(const char *rodata_address_or_partition_label_or_path, fbs::model_location_type_t location = fbs::MODEL_LOCATION_IN_FLASH_RODATA, int max_internal_size = 0, memory_manager_t mm_type = MEMORY_MANAGER_GREEDY, const uint8_t *key = nullptr, bool param_copy = true, const std::map<std::string, std::vector<int>> &input_shapes = {})

Create the Model object by rodata address or partition label.

Parameters:

  • rodata_address_or_partition_label_or_path – The address of model data while location is MODEL_LOCATION_IN_FLASH_RODATA. The label of partition while location is MODEL_LOCATION_IN_FLASH_PARTITION. The path of model while location is MODEL_LOCATION_IN_SDCARD.

  • location – The model location.

  • max_internal_size – In bytes. Limit the max internal size usage. Only take effect when there’s a PSRAM, and you want to alloc memory on internal RAM first.

  • mm_type – Type of memory manager

  • key – The key of encrypted model.

  • param_copy – Set to false to avoid copy model parameters from FLASH to PSRAM. Only set this param to false when your PSRAM resource is very tight. This saves PSRAM and sacrifices the performance of model inference because the frequency of PSRAM is higher than FLASH. Only takes effect when MODEL_LOCATION_IN_FLASH_RODATA(CONFIG_SPIRAM_RODATA not set) or MODEL_LOCATION_IN_FLASH_PARTITION.

  • input_shapes – Optional map to override graph input shapes. The key is the graph input name and the value is the shape. If empty, the shapes stored in the model are used.

Model(const char *rodata_address_or_partition_label_or_path, int model_index, fbs::model_location_type_t location = fbs::MODEL_LOCATION_IN_FLASH_RODATA, int max_internal_size = 0, memory_manager_t mm_type = MEMORY_MANAGER_GREEDY, const uint8_t *key = nullptr, bool param_copy = true, const std::map<std::string, std::vector<int>> &input_shapes = {})

Create the Model object by rodata address or partition label.

Parameters:

  • rodata_address_or_partition_label_or_path – The address of model data while location is MODEL_LOCATION_IN_FLASH_RODATA. The label of partition while location is MODEL_LOCATION_IN_FLASH_PARTITION. The path of model while location is MODEL_LOCATION_IN_SDCARD.

  • model_index – The model index of packed models.

  • location – The model location.

  • max_internal_size – In bytes. Limit the max internal size usage. Only take effect when there’s a PSRAM, and you want to alloc memory on internal RAM first.

  • mm_type – Type of memory manager

  • key – The key of encrypted model.

  • param_copy – Set to false to avoid copy model parameters from FLASH to PSRAM. Only set this param to false when your PSRAM resource is very tight. This saves PSRAM and sacrifices the performance of model inference because the frequency of PSRAM is higher than FLASH. Only takes effect when MODEL_LOCATION_IN_FLASH_RODATA(CONFIG_SPIRAM_RODATA not set) or MODEL_LOCATION_IN_FLASH_PARTITION.

  • input_shapes – Optional map to override graph input shapes. The key is the graph input name and the value is the shape. If empty, the shapes stored in the model are used.

Model(const char *rodata_address_or_partition_label_or_path, const char *model_name, fbs::model_location_type_t location = fbs::MODEL_LOCATION_IN_FLASH_RODATA, int max_internal_size = 0, memory_manager_t mm_type = MEMORY_MANAGER_GREEDY, const uint8_t *key = nullptr, bool param_copy = true, const std::map<std::string, std::vector<int>> &input_shapes = {})

Create the Model object by rodata address or partition label.

Parameters:

  • rodata_address_or_partition_label_or_path – The address of model data while location is MODEL_LOCATION_IN_FLASH_RODATA. The label of partition while location is MODEL_LOCATION_IN_FLASH_PARTITION. The path of model while location is MODEL_LOCATION_IN_SDCARD.

  • model_name – The model name of packed models.

  • location – The model location.

  • max_internal_size – In bytes. Limit the max internal size usage. Only take effect when there’s a PSRAM, and you want to alloc memory on internal RAM first.

  • mm_type – Type of memory manager

  • key – The key of encrypted model.

  • param_copy – Set to false to avoid copy model parameters from FLASH to PSRAM. Only set this param to false when your PSRAM resource is very tight. This saves PSRAM and sacrifices the performance of model inference because the frequency of PSRAM is higher than FLASH. Only takes effect when MODEL_LOCATION_IN_FLASH_RODATA(CONFIG_SPIRAM_RODATA not set) or MODEL_LOCATION_IN_FLASH_PARTITION.

  • input_shapes – Optional map to override graph input shapes. The key is the graph input name and the value is the shape. If empty, the shapes stored in the model are used.

Model(fbs::FbsModel *fbs_model, int internal_size = 0, memory_manager_t mm_type = MEMORY_MANAGER_GREEDY, const std::map<std::string, std::vector<int>> &input_shapes = {})

Create the Model object by fbs_model.

Parameters:

  • fbs_model – The fbs model.

  • internal_size – Internal ram size, in bytes

  • mm_type – Type of memory manager

  • input_shapes – Optional map to override graph input shapes. The key is the graph input name and the value is the shape. If empty, the shapes stored in the model are used.

virtual ~Model()

Destroy the Model object.

virtual esp_err_t load(const char *rodata_address_or_partition_label_or_path, fbs::model_location_type_t location = fbs::MODEL_LOCATION_IN_FLASH_RODATA, const uint8_t *key = nullptr, bool param_copy = true)

Load model graph and parameters from FLASH or sdcard.

Parameters:

  • rodata_address_or_partition_label_or_path – The address of model data while location is MODEL_LOCATION_IN_FLASH_RODATA. The label of partition while location is MODEL_LOCATION_IN_FLASH_PARTITION. The path of model while location is MODEL_LOCATION_IN_SDCARD.

  • location – The model location.

  • key – The key of encrypted model.

  • param_copy – Set to false to avoid copy model parameters from FLASH to PSRAM. Only set this param to false when your PSRAM resource is very tight. This saves PSRAM and sacrifices the performance of model inference because the frequency of PSRAM is higher than FLASH. Only takes effect when MODEL_LOCATION_IN_FLASH_RODATA(CONFIG_SPIRAM_RODATA not set) or MODEL_LOCATION_IN_FLASH_PARTITION.

Returns:

  • ESP_OK Success

  • ESP_FAIL Failed

virtual esp_err_t load(const char *rodata_address_or_partition_label_or_path, fbs::model_location_type_t location = fbs::MODEL_LOCATION_IN_FLASH_RODATA, int model_index = 0, const uint8_t *key = nullptr, bool param_copy = true)

Load model graph and parameters from FLASH or sdcard.

Parameters:

  • rodata_address_or_partition_label_or_path – The address of model data while location is MODEL_LOCATION_IN_FLASH_RODATA. The label of partition while location is MODEL_LOCATION_IN_FLASH_PARTITION. The path of model while location is MODEL_LOCATION_IN_SDCARD.

  • location – The model location.

  • model_index – The model index of packed models.

  • key – The key of encrypted model.

  • param_copy – Set to false to avoid copy model parameters from FLASH to PSRAM. Only set this param to false when your PSRAM resource is very tight. This saves PSRAM and sacrifices the performance of model inference because the frequency of PSRAM is higher than FLASH. Only takes effect when MODEL_LOCATION_IN_FLASH_RODATA(CONFIG_SPIRAM_RODATA not set) or MODEL_LOCATION_IN_FLASH_PARTITION.

Returns:

  • ESP_OK Success

  • ESP_FAIL Failed

virtual esp_err_t load(const char *rodata_address_or_partition_label_or_path, fbs::model_location_type_t location = fbs::MODEL_LOCATION_IN_FLASH_RODATA, const char *model_name = nullptr, const uint8_t *key = nullptr, bool param_copy = true)

Load model graph and parameters from FLASH or sdcard.

Parameters:

  • rodata_address_or_partition_label_or_path – The address of model data while location is MODEL_LOCATION_IN_FLASH_RODATA. The label of partition while location is MODEL_LOCATION_IN_FLASH_PARTITION. The path of model while location is MODEL_LOCATION_IN_SDCARD.

  • location – The model location.

  • model_name – The model name of packed models.

  • key – The key of encrypted model.

  • param_copy – Set to false to avoid copy model parameters from FLASH to PSRAM. Only set this param to false when your PSRAM resource is very tight. This saves PSRAM and sacrifices the performance of model inference because the frequency of PSRAM is higher than FLASH. Only takes effect when MODEL_LOCATION_IN_FLASH_RODATA(CONFIG_SPIRAM_RODATA not set) or MODEL_LOCATION_IN_FLASH_PARTITION.

Returns:

  • ESP_OK Success

  • ESP_FAIL Failed

virtual esp_err_t load(fbs::FbsModel *fbs_model)

Load model graph and parameters from Flatbuffers model.

Parameters:

fbs_model – The FlatBuffers model

Returns:

  • ESP_OK Success

  • ESP_FAIL Failed

virtual void build(size_t max_internal_size, memory_manager_t mm_type = MEMORY_MANAGER_GREEDY, bool preload = false, const std::map<std::string, std::vector<int>> &input_shapes = {})

Allocate memory for the model.

Parameters:

  • max_internal_size – In bytes. Limit the max internal size usage. Only take effect when there’s a PSRAM, and you want to alloc memory on internal RAM first.

  • mm_type – Type of memory manager

  • preload – Whether to preload the model’s parameters to internal ram (not implemented yet)

  • input_shapes – Optional map to override graph input shapes. The key is the graph input name and the value is the shape. If empty, the shapes stored in the model are used.

virtual void run(runtime_mode_t mode = RUNTIME_MODE_SINGLE_CORE)

Run the model module by module.

Parameters:

mode – Runtime mode.

virtual void run(TensorBase *input, runtime_mode_t mode = RUNTIME_MODE_SINGLE_CORE)

Run the model module by module.

Parameters:

  • input – The model input.

  • mode – Runtime mode.

virtual void run(std::map<std::string, TensorBase*> &user_inputs, runtime_mode_t mode = RUNTIME_MODE_SINGLE_CORE, std::map<std::string, TensorBase*> user_outputs = {})

Run the model module by module.

Parameters:

  • user_inputs – The model inputs.

  • mode – Runtime mode.

  • user_outputs – It’s for debug to specify the output of the intermediate layer; Under normal use, there is no need to pass a value to this parameter. If no parameter is passed, the default is the graphical output, which can be obtained through Model::get_outputs().

virtual void run(int stage_num, int stage_index, runtime_mode_t mode = RUNTIME_MODE_SINGLE_CORE)

Run a single stage of the model.

The execution plan is split into stage_num near-equal segments and only the segment indexed by stage_index is executed. The stage progress is managed by the caller (no state is kept inside the model), so a full inference is done by calling this for stage_index from 0 to stage_num - 1. Inputs/outputs are handled the same way as the other run() overloads (assign inputs before running stage 0, read outputs after running the last stage).

Parameters:

  • stage_num – Total number of stages to split the execution plan into (>= 1).

  • stage_index – Index of the stage to execute, in range [0, stage_num).

  • mode – Runtime mode.

void minimize()

Minimize the model.

esp_err_t test()

Test whether the model inference result is correct. The model should contain test_inputs and test_outputs. Enable export_test_values option in esp-ppq to use this api. The comparison epsilon is read from model metadata test_max_error (default 5e-4).

Returns:

esp_err_t

std::map<std::string, mem_info_t> get_memory_info()

Get memory info.

Returns:

Memory usage statistics on internal and PSRAM.

std::map<std::string, module_info> get_module_info()

Get module info.

Returns:

return Type and latency of each module.

void print_module_info(const std::map<std::string, module_info> &info, bool sort_module_by_latency = false)

Print the module info obtained by get_module_info function.

Parameters:

  • info –

  • sort_module_by_latency –

void profile_memory()

Print model memory summary.

void profile_module(bool sort_module_by_latency = false)

Print module info summary. (Name, Type, Latency)

Parameters:

sort_module_by_latency – True The module is printed in latency decreasing sort. False The module is printed in ONNX topological sort.

void profile(bool sort_module_by_latency = false)

Combination of profile_memory & profile_module.

Parameters:

sort_module_by_latency – True The module is printed in latency decreasing sort. False The module is printed in ONNX topological sort.

virtual std::map<std::string, TensorBase*> &get_inputs()

Get inputs of model.

Returns:

The map of model input’s name and TensorBase*

virtual TensorBase *get_input()

Get the only input of model.

Returns:

TensorBase*

virtual TensorBase *get_input(const std::string &name)

Get input of model by name.

Parameters:

name – input name

Returns:

TensorBase*

virtual TensorBase *get_intermediate(const std::string &name)

Get intermediate TensorBase of model.

Note

When using memory manager, the content of TensorBase’s data may be overwritten by the outputs of other

Parameters:

name – The name of intermediate Tensor. operators.

Returns:

The intermediate TensorBase*.

virtual std::map<std::string, TensorBase*> &get_outputs()

Get outputs of model.

Returns:

The map of model output’s name and TensorBase*

virtual TensorBase *get_output()

Get the only output of model.

Returns:

TensorBase*

virtual TensorBase *get_output(const std::string &name)

Get output of model by name.

Parameters:

name – output name

Returns:

TensorBase*

std::string get_metadata_prop(const std::string &key)

Get the model’s metadata prop.

Parameters:

key – The key of metadata prop

Returns:

The value of metadata prop

virtual void print()

Print the model.

virtual void reset()

Reset the model. This function resets the model context and all modules, but does not reload the model or rebuild the execution plan.

inline virtual fbs::FbsModel *get_fbs_model()

Get the fbs model instance.

Returns:

fbs::FbsModel *

Header File

Macros

CONTEXT_PARAMETER_OFFSET

Offset for parameter tensors

Classes

class ModelContext

Model Context class including variable tensors and parameters.

Public Functions

inline ModelContext()

Constructor for ModelContext. Initializes the PSRAM and internal root pointers to nullptr.

inline ~ModelContext()

Destructor for ModelContext. Clears all resources and tensors.

int add_tensor(const std::string name, bool is_paramter = false, TensorBase *tensor = nullptr)

Adds a tensor to the parameter or variable list.

Parameters:

  • name – The name of the tensor.

  • is_paramter – Whether the tensor is a parameter (default: false).

  • tensor – Pointer to the TensorBase object (default: nullptr).

Returns:

int Returns the index of the added tensor.

int push_back_tensor(TensorBase *tensor, bool is_paramter = false)

Push back a tensor.

Parameters:

  • tensor – Pointer to the TensorBase object.

  • is_paramter – Whether the tensor is a parameter (default: false).

Returns:

int Returns the index of the added tensor.

void update_tensor(int index, TensorBase *tensor)

Updates the tensor at the specified index.

Parameters:

  • index – The index of the tensor to update.

  • tensor – Pointer to the new TensorBase object.

WorkspaceBindingId create_workspace_binding(const std::string &name)

Creates an empty native workspace binding.

Workspace bindings are not tensors and do not affect variable tensor indices or counts.

Parameters:

name – Workspace name, used for memory reports.

Returns:

WorkspaceBindingId Stable ID for the binding.

void bind_workspace(WorkspaceBindingId binding_id, void *data, size_t size)

Binds a workspace ID to a region in a planner-owned root arena.

Parameters:

  • binding_id – Workspace binding ID.

  • data – Workspace data pointer.

  • size – Workspace capacity in bytes.

void *get_bound_workspace(WorkspaceBindingId binding_id, size_t &size)

Gets the arena region currently bound to a workspace ID.

Parameters:

  • binding_id – Workspace binding ID.

  • size – Receives the workspace capacity, or zero for an invalid or unbound binding.

Returns:

void* Workspace data pointer, or nullptr for an invalid or unbound binding.

inline int get_workspace_binding_count() const

Gets the number of native workspace bindings.

Returns:

Number of workspace bindings.

inline const std::vector<WorkspaceBinding> &get_workspace_bindings() const

Gets all native workspace bindings. Bytes are already part of the variable arena.

Returns:

Read-only workspace binding list.

inline void clear_workspace_bindings()

Drops workspace binding metadata without freeing planner root arenas.

inline void reset_variable_arena()

Drops variable tensor views and their planner-owned arenas while preserving graph indices, parameter tensors, and name mappings.

Used before a model is rebuilt for a new set of input shapes. Each variable slot remains in place as nullptr until the memory manager binds its replacement view.

TensorBase *get_tensor(int index)

Gets the tensor by its index.

Parameters:

index – The index of the tensor.

Returns:

TensorBase* Returns the pointer to the TensorBase object, or nullptr if the index is invalid.

TensorBase *get_tensor(const std::string &name)

Gets the tensor by its name.

Parameters:

name – The name of the tensor.

Returns:

TensorBase* Returns the pointer to the TensorBase object, or nullptr if the name is not found.

int get_tensor_index(const std::string &name)

Gets the tensor index by its name.

Parameters:

name – The name of the tensor.

Returns:

int Returns index if the name is found, else -1

inline int find_tensor_index(const std::string &name) const

Silent variant of get_tensor_index: returns -1 without logging an error when the tensor name is not yet registered. Use this on hot paths (e.g. per-op input loop) where a miss is expected.

Parameters:

name – The name of the tensor.

Returns:

int Returns index if the name is found, else -1

bool has_tensor(const std::string &name) const

Checks if a tensor exists by its name.

Parameters:

name – The name of the tensor.

Returns:

bool Returns true if the tensor exists, false otherwise.

int get_variable_index(const std::string &name)

Gets the variable tensor index by its name.

Parameters:

name – The name of the tensor.

Returns:

int Returns index if the name is found and is variable tensor, else -1

inline int get_variable_count()

Gets the count of variable tensors.

Returns:

int Returns the number of variable tensors.

inline int get_parameter_count()

Gets the count of parameter tensors.

Returns:

int Returns the number of parameter tensors.

bool root_alloc(size_t internal_size, size_t psram_size, int alignment = 16)

Allocates memory for PSRAM and internal roots.

Parameters:

  • internal_size – The size of the internal memory in bytes.

  • psram_size – The size of the PSRAM memory in bytes.

  • alignment – The alignment of the memory in bytes.

Returns:

Bool Return true if the allocation is successful, false otherwise.

inline void *get_psram_root()

Gets the pointer to the PSRAM root.

Returns:

Void* Returns the pointer to the PSRAM root.

inline void *get_internal_root()

Gets the pointer to the internal root.

Returns:

Void* Returns the pointer to the internal root.

size_t get_parameter_memory_size(mem_info_t &mem_info, bool copy)

Gets the size of the parameters in bytes.

Parameters:

  • mem_info – The size of the memory used by the parameters in bytes, filtered by copy option.

  • copy – Filter the parameters by auto_free.

Returns:

size_t Returns the total size of the parameters memory in bytes.

size_t get_variable_memory_size(mem_info_t &mem_info)

Get the variable memory size object.

Parameters:

mem_info – The size of the memory used by the variables in bytes.

Returns:

size_t Returns the total size of the variables memory in bytes.

inline void root_free()

Frees the memory allocated for PSRAM and internal roots. This function ensures proper cleanup of allocated memory.

inline void root_reset()

Resets the context by clearing variables, parameters, and name-to-index map. This is used to reset the state of the context for a new inference or after an inference.

inline void minimize()

Minimizes the context by clearing the name-to-index map. This is used to free unnecessary intermediate variables during the inference.

inline void release_tensors()

Drops all tensor references without deleting the tensors. Used by contexts that only borrow tensors owned by the caller.

inline void clear()

Clears all resources and tensors in the context. This includes clearing variables, parameters, name-to-index map, and freeing memory.

Public Members

std::vector<TensorBase*> m_variables

Variable tensors of model, the first one is nullptr

std::vector<TensorBase*> m_parameters

Parameters of model, the first one is nullptr

struct WorkspaceBinding

Non-owning view of a planner-managed native workspace.

Public Members

std::string name

“<node>:workspace<j>”

void *data

Pointer into a planner-owned root arena

size_t size

Workspace capacity in bytes

Header File

Classes

class MemoryManagerBase

Memory manager base class, each model has its own memory manager TODO: share memory manager with different models.

Subclassed by dl::MemoryManagerGreedy

Public Functions

inline MemoryManagerBase(int alignment = 16)

Construct a new Memory Manager Base object.

Parameters:

alignment – Memory address alignment

inline virtual ~MemoryManagerBase()

Destroy the MemoryManager object. Return resource.

virtual bool alloc(fbs::FbsModel *fbs_model, std::vector<dl::module::Module*> &execution_plan, ModelContext *context, const std::map<std::string, std::vector<int>> &input_shapes = {}) = 0

Allocate memory for each tensor, include all input and output tensors.

Parameters:

  • fbs_model – FlatBuffer’s Model

  • execution_plan – Topological sorted module list

  • context – Model context

  • input_shapes – Optional map to override graph input shapes. The key is the graph input name and the value is the shape. If empty, the shapes stored in fbs_model are used.

Returns:

Bool Return true if the allocation is successful, false otherwise.

Public Members

int alignment

The root pointer needs to be aligned must be a power of two

class MemoryInfo

Common lifetime and placement metadata for planner-managed memory.

Both tensors and native workspaces use this representation so they can share the same greedy arena and lifetime-reuse algorithm.

Subclassed by dl::TensorInfo, dl::WorkspaceInfo

Public Functions

MemoryInfo(const std::string &name, int time_begin, int time_end, size_t size, bool is_internal = false)

Construct planner metadata for a tensor or native workspace.

Parameters:

  • name – Allocation name used in planner diagnostics.

  • time_begin – First execution step that uses the allocation.

  • time_end – Last execution step that uses the allocation.

  • size – Allocation size in bytes.

  • is_internal – Whether the allocation is initially assigned to internal RAM.

void update_time(int new_time)

Extend the allocation lifetime to include a new execution step.

Parameters:

new_time – New lifetime end; -1 marks a persistent allocation.

inline bool is_inplaced()

Check whether this allocation shares storage with a leader.

Returns:

true when an in-place leader is assigned, otherwise false.

inline virtual bool is_psram_only() const

Check whether the allocation must stay in PSRAM.

Returns:

true when internal RAM placement is forbidden, otherwise false.

inline uint32_t get_offset()

Get the offset in the planner’s PSRAM arena.

Returns:

PSRAM arena offset in bytes.

inline void set_offset(uint32_t offset)

Set the offset in the planner’s PSRAM arena.

Parameters:

offset – PSRAM arena offset in bytes.

inline uint32_t get_internal_offset()

Get the offset in the planner’s internal-RAM arena.

Returns:

Internal-RAM arena offset in bytes.

inline bool get_internal_state()

Check whether the allocation is assigned to internal RAM.

Returns:

true for internal RAM, otherwise false.

inline void set_internal_state(bool is_internal)

Set whether the allocation is assigned to internal RAM.

Parameters:

is_internal – true for internal RAM, false for PSRAM.

inline void set_internal_offset(uint32_t offset)

Assign an offset in the internal-RAM arena.

Parameters:

offset – Internal-RAM arena offset in bytes.

inline int get_time_end()

Get the last execution step in the allocation lifetime.

Returns:

Lifetime end, or -1 for a persistent allocation.

inline int get_time_begin()

Get the first execution step in the allocation lifetime.

Returns:

Lifetime begin.

inline size_t get_size()

Get the allocation size.

Returns:

Allocation size in bytes.

inline std::string get_name()

Get the planner allocation name.

Returns:

Allocation name.

inline void print()

Print lifetime, size, and arena offsets for diagnostics.

class TensorInfo : public dl::MemoryInfo

Tensor-specific planner metadata.

Public Functions

TensorInfo(std::string &name, int time_begin, int time_end, std::vector<int> shape, dtype_t dtype, int exponent, bool is_internal = false)

Construct a new Tensor Info object.

Parameters:

  • name – Tensor name

  • time_begin – Tensor lifetime begin

  • time_end – Tensor lifetime end

  • shape – Tensor shape

  • dtype – Tensor dtype

  • exponent – Tensor exponent

  • is_internal – Is tensor in internal RAM or not

inline ~TensorInfo()

Destroy the Tensor Info object.

void set_inplace_leader_tensor(TensorInfo *tensor)

Set the inplace leader tensor object.

Parameters:

tensor – Inplace leader tensor

inline void set_inplace_follower_dirty_tensor(TensorInfo *tensor)

Set the inplace follower dirty tensor object.

Parameters:

tensor – Inplace follower dirty tensor

inline void set_inplace_follower_clean_tensor(TensorInfo *tensor)

Set the inplace follower clean tensor object.

Parameters:

tensor – Inplace follower clean tensor

inline std::pair<TensorInfo*, TensorInfo*> get_inplace_follower_tensor()

Get the inplace follower tensor object.

Returns:

std::pair<TensorInfo *, TensorInfo *>

TensorBase *create_tensor(void *internal_root, void *psram_root)

Create a TensorBase view over the planner-owned tensor storage.

Parameters:

  • internal_root – Base address of the internal-RAM arena.

  • psram_root – Base address of the PSRAM arena.

Returns:

TensorBase view bound to the selected planner arena.

inline std::vector<int> get_shape()

Get the tensor shape.

Returns:

std::vector<int>

class WorkspaceInfo : public dl::MemoryInfo

Native workspace planner metadata.

Public Functions

inline WorkspaceInfo(const std::string &name, int time_begin, int time_end, size_t size, WorkspaceBindingId workspace_binding_id, bool psram_only)

Construct metadata for a module-native workspace.

Parameters:

  • name – Workspace name used in planner diagnostics.

  • time_begin – First execution step that uses the workspace.

  • time_end – Last execution step that uses the workspace.

  • size – Workspace size in bytes.

  • workspace_binding_id – Binding used to publish the planned address.

  • psram_only – Whether internal RAM placement is forbidden.

inline WorkspaceBindingId get_workspace_binding_id() const

Get the model-context binding for this workspace.

Returns:

Workspace binding identifier.

inline virtual bool is_psram_only() const override

Check whether the workspace must stay in PSRAM.

Returns:

true when internal RAM placement is forbidden, otherwise false.

void *get_data(void *internal_root, void *psram_root)

Resolve the workspace address from the planner root arenas.

Parameters:

  • internal_root – Base address of the internal-RAM arena.

  • psram_root – Base address of the PSRAM arena.

Returns:

Pointer to the planned workspace storage.

class MemoryChunk

Memory chunk used to simulate allocation of tensors and workspaces.

Public Functions

MemoryChunk(size_t size, int is_free, int alignment = 16)

Construct a new Memory Chunk object.

Parameters:

  • size – Memory chunk size

  • is_free – Whether free or not

  • alignment – Memory chunk alignment

MemoryChunk(MemoryInfo *memory, int alignment = 16)

Construct a new Memory Chunk object.

Parameters:

  • memory – Planner allocation metadata

  • alignment – Memory chunk alignment

inline ~MemoryChunk()

Destroy the Memory Chunk object.

MemoryChunk *merge_free_chunk(MemoryChunk *chunk)

Merge continuous free chunk.

Parameters:

chunk –

Returns:

MemoryChunk*

MemoryChunk *insert(MemoryInfo *memory)

Insert a planner allocation into a free chunk.

Parameters:

memory –

Returns:

MemoryChunk*

MemoryChunk *extend(MemoryInfo *memory)

Extend free chunk and insert a planner allocation.

Parameters:

memory –

Returns:

MemoryChunk*

inline void free()

Free memory chunk.

Public Members

size_t size

Memory chunk size

bool is_free

Whether memory chunk is free or not

int offset

Offset relative to root pointer

int alignment

Memory address alignment

MemoryInfo *memory

Info of the planner allocation occupying this chunk

Header File

Classes

class MemoryManagerGreedy : public dl::MemoryManagerBase

Greedy memory manager that allocates memory for tensors in execution order, prioritizing internal RAM allocation first.

Public Functions

inline MemoryManagerGreedy(int max_internal_size, int alignment = 16)

Constructs a greedy memory manager with specified constraints.

Parameters:

  • max_internal_size – Maximum allowed internal RAM usage in bytes

  • alignment – Memory address alignment requirement (default: 16 bytes)

inline ~MemoryManagerGreedy()

Destructor that releases all managed memory resources.

virtual bool alloc(fbs::FbsModel *fbs_model, std::vector<dl::module::Module*> &execution_plan, ModelContext *context, const std::map<std::string, std::vector<int>> &input_shapes = {}) override

Allocates memory for all network tensors following greedy strategy.

Parameters:

  • fbs_model – FlatBuffer model containing network architecture

  • execution_plan – Execution graph ordered by computation dependencies

  • context – Device-specific runtime configuration

  • input_shapes – Optional map to override shapes of graph inputs for allocation

Returns:

bool True if successful allocation, false if memory insufficient

void free()

Releases all allocated memory including tensor buffers and memory pools.