Tensor API Reference

Tensor is the fundamental data type in esp-dl, used for storing multi-type data such as int8, int16, float, etc., similar to the tensor in PyTorch. We have implemented some common tensor operations. Please refer to the following APIs for details.

Header File

esp-dl/dl/tensor/include/dl_tensor_base.hpp

Classes

class ExponentInfo

Exponent info for per-tensor / per-channel quantization.

Per-tensor: m_exponent only, zero heap allocation. Per-channel: heap-allocated m_exponents array. Provides operator int() so existing DL_SCALE(tensor->exponent) code works unchanged.

Public Functions

inline ExponentInfo(int exponent = 0)

Construct a per-tensor ExponentInfo with a single exponent value.

Parameters:: exponent – The exponent value for per-tensor quantization. Default is 0.

inline ExponentInfo(const std::vector<int> &exponents)

Construct an ExponentInfo from a vector of exponents.

Parameters:: exponents – Vector of exponent values. If size <= 1, uses per-tensor mode; otherwise uses per-channel mode with heap allocation.

inline ~ExponentInfo(): Destroy the ExponentInfo object and free heap-allocated memory.

inline ExponentInfo(const ExponentInfo &other)

Copy constructor.

Parameters:: other – The ExponentInfo object to copy from.

inline ExponentInfo &operator=(const ExponentInfo &other)

Copy assignment operator.

Parameters:: other – The ExponentInfo object to assign from.
Returns:: Reference to this object.

inline ExponentInfo(ExponentInfo &&other) noexcept

Move constructor.

Parameters:: other – The ExponentInfo object to move from.

inline ExponentInfo &operator=(ExponentInfo &&other) noexcept

Move assignment operator.

Parameters:: other – The ExponentInfo object to move from.
Returns:: Reference to this object.

inline ExponentInfo &operator=(int value)

Assign a single integer value as per-tensor exponent.

Parameters:: value – The exponent value to assign.
Returns:: Reference to this object.

inline int get(int ch = -1) const

Get exponent value.

Parameters:: ch – Channel index. -1 (default) returns per-tensor value. ch >= 0 returns per-channel value if available, otherwise per-tensor.
Returns:: The exponent value for the specified channel or per-tensor exponent.

inline operator int() const

Implicit conversion to int, returns the per-tensor exponent value.

Returns:: The per-tensor exponent value.

inline bool is_per_channel() const

Check if using per-channel quantization.

Returns:: true if per-channel exponents are stored, false otherwise.

inline int channel_size() const

Get the number of channels.

Returns:: Number of channels for per-channel mode, or 1 for per-tensor mode.

inline const int *data() const

Get pointer to exponent data.

Returns:: Pointer to per-channel exponents array if available, otherwise pointer to per-tensor exponent.

inline bool operator==(const ExponentInfo &other) const

Compare two ExponentInfo objects for equality.

Parameters:: other – The ExponentInfo object to compare with.
Returns:: true if both have the same exponent values, false otherwise.

inline bool operator!=(const ExponentInfo &other) const

Compare two ExponentInfo objects for inequality.

Parameters:: other – The ExponentInfo object to compare with.
Returns:: true if exponent values differ, false if equal.

class TensorBase

This class is designed according to PyTorch Tensor. TensorBase is required to ensure that the first address are aligned to 16 bytes and the memory size should be a multiple of 16 bytes.

TODO:: Implement more functions

Public Functions

TensorBase(std::vector<int> shape, const void *element, int exponent = 0, dtype_t dtype = DATA_TYPE_FLOAT, bool deep = true, uint32_t caps = MALLOC_CAP_DEFAULT)

Construct a TensorBase object.

Parameters:

shape – Shape of tensor
element – Pointer of data
exponent – Exponent of tensor, default is 0
dtype – Data type of element, default is float
deep – True: malloc memory and copy data, false: use the pointer directly
caps – Bitwise OR of MALLOC_CAP_* flags indicating the type of memory to be returned

TensorBase(std::vector<int> shape, const void *element, const std::vector<int> &exponents, dtype_t dtype = DATA_TYPE_FLOAT, bool deep = true, uint32_t caps = MALLOC_CAP_DEFAULT)

Construct a TensorBase object with per-channel exponents.

Parameters:

shape – Shape of tensor
element – Pointer of data
exponents – Per-channel exponents
dtype – Data type of element, default is float
deep – True: malloc memory and copy data, false: use the pointer directly
caps – Bitwise OR of MALLOC_CAP_* flags indicating the type of memory to be returned

inline virtual ~TensorBase(): Destroy the TensorBase object.

bool assign(TensorBase *tensor)

Assign tensor to this tensor.

Parameters:: tensor –
Returns:: true if assign successfully, otherwise false.

bool assign(std::vector<int> shape, const void *element, int exponent, dtype_t dtype)

Assign data to this tensor.

Parameters:

shape –
element –
exponent –
dtype –

Returns:

true if assign successfully, otherwise false.

inline int get_size()

Get the size of Tensor.

Returns:: the size of Tensor.

inline int get_aligned_size()

Get the aligned size of Tensor.

Returns:: the aligned size of Tensor.

inline size_t get_dtype_bytes()

Get the dtype size, in bytes.

Returns:: the size of dtype.

inline const char *get_dtype_string()

Get the dtype string of Tensor.

Returns:: the string of Tensor’s dtype.

inline int get_bytes()

Get the bytes of Tensor.

Returns:: the bytes of Tensor.

inline int get_aligned_bytes()

Get the bytes of Tensor.

Returns:: the bytes of Tensor.

inline virtual void *get_element_ptr()

Get data pointer. If cache(preload data pointer) is not null, return cache pointer, otherwise return data pointer.

Returns:: the pointer of Tensor’s data

template<typename T> inline T *get_element_ptr()

Get data pointer by the specified template. If cache(preload data pointer) is not null, return cache pointer, otherwise return data pointer.

Returns:: the pointer of Tensor’s data

TensorBase &set_element_ptr(void *data)

Set the data pointer of Tensor.

Parameters:: data – point to data memory
Returns:: TensorBase& self

inline std::vector<int> get_shape()

Get the shape of Tensor.

Returns:: std::vector<int> the shape of Tensor

TensorBase &set_shape(const std::vector<int> shape)

Set the shape of Tensor.

Parameters:: shape – the shape of Tensor.
Returns:: Tensor.

inline int get_exponent()

Get the exponent of Tensor.

Returns:: int the exponent of Tensor

inline dtype_t get_dtype()

Get the data type of Tensor.

Returns:: dtype_t the data type of Tensor

inline uint32_t get_caps()

Get the memory flags of Tensor.

Returns:: uint32_t the memory flags of Tensor

TensorBase *reshape(std::vector<int> shape)

Change a new shape to the Tensor without changing its data.

Parameters:: shape – the target shape
Returns:: TensorBase *self

template<typename T> TensorBase *flip(const std::vector<int> &axes)

Flip the input Tensor along the specified axes.

Parameters:: axes – the specified axes
Returns:: TensorBase& self

TensorBase *transpose(TensorBase *input, std::vector<int> perm = {})

Reverse or permute the axes of the input Tensor.

Parameters:

input – the input Tensor
perm – the new arrangement of the dims. if perm == {}, the dims arrangement will be reversed.

Returns:

TensorBase *self

template<typename T> TensorBase *transpose(T *input_element, std::vector<int> &input_shape, std::vector<int> &input_axis_offset, std::vector<int> &perm)

Reverse or permute the axes of the input Tensor.

Parameters:

input_element – the input data pointer
input_shape – the input data shape
input_axis_offset – the input data axis offset
perm – the new arrangement of the dims. if perm == {}, the dims arrangement will be reversed.

Returns:

TensorBase *self

bool is_same_shape(TensorBase *tensor)

Check the shape is the same as the shape of input.

Parameters:

tensor – Input tensor pointer

Returns:

true: same shape
false: not

bool equal(TensorBase *tensor, float epsilon = 1e-6, bool verbose = false)

Compare the shape and data of two Tensor.

Parameters:

tensor – Input tensor
epsilon – The max error of two element
verbose – If true, print the detail of results

Returns:

true if two tensor is equal otherwise false

TensorBase *slice(const std::vector<int> &start, const std::vector<int> &end, const std::vector<int> &axes = {}, const std::vector<int> &step = {})

Produces a slice of the this tensor along multiple axes.

Warning

The length of start, end and step must be same as the shape of input tensor

Parameters:

start – Starting indicesd
end – Ending indices
axes – Axes that starts and ends apply to.
step – Slice step, step = 1 if step is not specified

Returns:

TensorBase* Output tensor pointer, created by this slice function

template<typename T> TensorBase *pad(T *input_element, const std::vector<int> &input_shape, const std::vector<int> &pads, const padding_mode_t mode, TensorBase *const_value = nullptr)

Pad input tensor.

Parameters:

input_element – Data pointer of input tensor
input_shape – Shape of input tensor
pads – The number of padding elements to add, pads format should be: [x1_begin, x2_begin, …, x1_end, x2_end,…]
mode – Supported modes: constant(default), reflect, edge
const_value – (Optional) A scalar value to be used if the mode chosen is constant

Returns:

Output tensor pointer

TensorBase *pad(TensorBase *input, const std::vector<int> &pads, const padding_mode_t mode, TensorBase *const_value = nullptr)

Pad input tensor.

Parameters:

input – Input tensor pointer
pads – Padding elements to add, pads format should be: [x1_begin, x2_begin, …, x1_end, x2_end,…]
mode – Supported modes: constant(default), reflect, edge
const_value – (Optional) A scalar value to be used if the mode chosen is constant

Returns:

Output tensor pointer

template<typename T> bool compare_elements(const T *gt_elements, float epsilon = 1e-6, bool verbose = false)

Compare the elements of two Tensor.

Parameters:

gt_elements – The ground truth elements
epsilon – The max error of two element
verbose – If true, print the detail of results

Returns:

true if all elements are equal otherwise false

int get_element_index(const std::vector<int> &axis_index)

Get the index of element.

Parameters:: axis_index – The coordinates of element
Returns:: int the index of element

std::vector<int> get_element_coordinates(int index)

Get the coordinates of element.

Parameters:: index – The index of element
Returns:: The coordinates of element

template<typename T> T get_element(int index)

Get a element of Tensor by index.

Parameters:: index – The index of element
Returns:: The element of tensor

template<typename T> T get_element(const std::vector<int> &axis_index)

Get a element of Tensor.

Parameters:: axis_index – The index of element
Returns:: The element of tensor

void memset(int value)

Set a element of Tensor by index.

Parameters:: value – The value of element

void rand()

Fill tensor data with random bytes from hardware RNG.

Uses esp_fill_random() to fill the tensor’s internal buffer with random bytes. Requires ESP-IDF (esp_random.h).

size_t set_preload_addr(void *addr, size_t size)

Set preload address of Tensor.

Parameters:

addr – The address of preload data
size – Size of preload data

Returns:

The size of preload data

inline virtual void preload(): Preload the data of Tensor.

void reset_bias_layout(quant_type_t op_quant_type, bool is_depthwise)

Reset the layout of Tensor.

Warning

Only available for Convolution. Don’t use it unless you know exactly what it does.

Parameters:

op_quant_type – The quant type of operation
is_depthwise – Whether is depthwise convolution

void push(TensorBase *new_tensor, int dim)

Push new_tensor to current tensor. The time series dimension size of new tensor must is lesser or equal than that of the current tensor.”.

Parameters:

new_tensor – The new tensor will be pushed
dim – Specify the dimension on which to perform streaming stack pushes

virtual void print(bool print_data = false)

print the information of TensorBase

Parameters:: print_data – Whether print the data

Public Members

int size: size of element including padding

std::vector<int> shape: shape of Tensor

dtype_t dtype: data type of element

ExponentInfo exponent: exponent of element (per-tensor or per-channel)

bool auto_free: free element when object destroy

std::vector<int> axis_offset: element offset of each axis

void *data: data pointer

void *cache: cache pointer， used for preload and do not need to free

uint32_t caps: flags indicating the type of memory

Public Static Functions

static void slice(TensorBase *input, TensorBase *output, const std::vector<int> &start, const std::vector<int> &end, const std::vector<int> &axes = {}, const std::vector<int> &step = {})

Produces a slice along multiple axes.

Warning

The length of start, end and step must be same as the shape of input tensor

Parameters:

input – Input Tensor
output – Output Tensor
start – Starting indicesd
end – Ending indices
axes – Axes that starts and ends apply to.
step – Slice step, step = 1 if step is not specified