Getting Started

[中文]

Hardware Requirements

  • An ESP32-S3 or ESP32-P4 development board. Recommended: ESP32-S3-EYE or ESP32-P4-Function-EV-Board

  • PC (Linux)

Note

  • Some boards currently use Type C connectors. Make sure you use the right cable to connect the board!

  • ESP-DL also supports ESP32, but its operator implementations are written in C, so the execution speed on ESP32 will be significantly slower than on ESP32-S3 or ESP32-P4. If needed, you can manually add compilation configuration files to your project—the function interface calls in ESP-DL remain identical. Note:

    • When quantizing ESP32 platform models using ESP-PPQ, set the target to c.

    • When deploying ESP32 platform models using ESP-DL, set the project compilation target to esp32.

Software Requirements

ESP-IDF

ESP-DL runs based on ESP-IDF. For detailed instructions on how to get ESP-IDF, see the ESP-IDF Programming Guide.

Note

Please use release/v5.3 or higher version of ESP-IDF.

ESP-PPQ

ESP-PPQ is a quantization tool based on ppq, and its source code is fully open-sourced. ESP-PPQ adds Espressif’s customized quantizer and exporter based on PPQ, which makes it convenient for users to select quantization rules that match ESP-DL according to different chip selections, and export them to standard model files that can be directly loaded by ESP-DL. ESP-PPQ is compatible with all PPQ APIs and quantization scripts. For more details, please refer to PPQ documents and videos. If you want to quantize your model, you can install esp-ppq using the following method:

Method 1: Install the package using pip

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install esp-ppq

Method 2: Install from source with pip to stay synchronized with the master branch

git clone https://github.com/espressif/esp-ppq.git
cd esp-ppq
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -e .

Method 3: Install the package using uv

uv pip install "esp-ppq[cpu]" --torch-backend=cpu
# GPU
# uv pip install "esp-ppq[cpu]" --torch-backend=cu124
# AMD GPU
# uv pip install "esp-ppq[cpu]" --torch-backend=rocm6.2
# Intel XPU
# uv pip install "esp-ppq[cpu]" --torch-backend=xpu

Method 4: Install from source using uv to stay in sync with the master branch

git clone https://github.com/espressif/esp-ppq.git
cd esp-ppq
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
uv pip install -e .

Method 5: Use esp-ppq with docker

docker build -t esp-ppq:your_tag https://github.com/espressif/esp-ppq.git

Note

  • The example code installs the Linux PyTorch CPU version. Please install the appropriate PyTorch version based on your actual needs.

  • If installing the package with uv, simply modify the --torch-backend parameter, which will override the PyTorch URLs index configured in the project.

Quick Start

ESP-DL provides some out-of-the-box examples

Example Compile & Flash

idf.py set-target [Soc]
idf.py flash monitor -p [PORT]

Replace [Soc] with the specific chip, currently supports esp32s3 and esp32p4. The example does not yet include the model and compilation configuration files for esp32.

Example Configuration

idf.py menuconfig

Some examples contain configurable options that can be configured using idf.py menuconfig after specifying the chip using idf.py set-target.

Trouble shooting

Check ESP-IDF doc

See ESP-IDF DOC

Erase FLASH & Clear Example

idf.py erase-flash -p [PORT]

Delete build/, sdkconfig, dependencies.lock, managed_components/ and try again.

Model Quantization

First, please refer to operator_support_state.md to ensure that the operators in your model are supported.

ESP-DL must use the proprietary format .espdl for model deployment. Deep learning models need to be quantized and converted to the format before they can be used. ESP-PPQ provides two interfaces, espdl_quantize_onnx and espdl_quantize_torch, to support ONNX models and PyTorch models to be exported as .espdl models. Other deep learning frameworks, such as TensorfFlow, PaddlePaddle, etc., need to convert the model to ONNX first. So make sure your model can be converted to ONNX model. For more details, please refer to:

Model deployment

ESP-DL provides a series of APIs to quickly load and run models. For more details, see: