espdl – Model Inference

[中文]

The espdl module runs ESP-DL .espdl models on captured images. It provides task-specific wrappers for object detection (ESPDet, YOLO11), pose estimation (YOLO11nPose), and image classification (ImageNetCls), plus Model for exposing raw ESP-DL output tensors to Python.

Raw Output Tensors

import sensor, espdl

model = espdl.Model("/sdcard/custom.espdl", mean=(0, 0, 0), std=(255, 255, 255), letterbox=True)
try:
    print("inputs:", model.inputs())
    print("outputs:", model.outputs())
    img = sensor.snapshot()
    outputs = model.predict(img)
    for name, tensor in outputs.items():
        _, shape, dtype, exponent, raw = tensor
        print(name, shape, dtype, exponent, len(raw))
finally:
    model.deinit()

Use Model when the built-in task wrappers do not match a model’s output layout. ESP-DL still performs image preprocessing and inference; predict() returns the raw output tensors so Python code can run the model-specific decode. inputs() and outputs() return TensorInfo tuples (name, shape, dtype, exponent). predict() returns RawTensor tuples (name, shape, dtype, exponent, bytes) keyed by output tensor name. Decode the raw bytes according to dtype, apply the ESP-DL power-of-two exponent scale, and run logic such as sigmoid, box decode, NMS, softmax, or top-k selection. When letterbox=True, remove the model-input padding before mapping coordinates back to the source image.

Object Detection

import sensor, espdl

det = espdl.ESPDet("/sdcard/espdet_pico_224_224_face.espdl", score=0.5, nms=0.7)
try:
    img = sensor.snapshot()
    for x, y, w, h, score, category in det.detect(img):
        img.draw_rectangle(x, y, w, h, color=(255, 0, 0), thickness=2)
        img.draw_string(x, max(0, y - 12), "%.2f:%d" % (score, category))
finally:
    det.deinit()

score removes low-confidence candidates, while nms controls suppression of overlapping boxes. Detection coordinates are mapped back to the input image and can be used directly for drawing. Load a model once and reuse it across frames; deinit() releases model weights and intermediate buffers.

Image Classification

import espdl, image

img = image.Image("/sdcard/cat.jpg").to_rgb565(copy=True)
classifier = espdl.ImageNetCls(
    "/sdcard/imagenet_cls_mobilenetv2_s8_v1.espdl",
    topk=5,
    score=0.0,
)
try:
    for label, score in classifier.classify(img):
        print(label, "%.4f" % score)
finally:
    classifier.deinit()

Classification returns up to topk (label, score) pairs ordered by the model output. Convert file images to RGB565 when necessary so the input format is accepted by the wrapper.

Pose Estimation

pose = espdl.YOLO11nPose("/sdcard/espdet_yolo11n_pose_160_160_coco.espdl", score=0.35, topk=5)
try:
    img = sensor.snapshot()
    for x, y, w, h, score, category, keypoints in pose.detect(img):
        img.draw_rectangle(x, y, w, h, color=(255, 0, 0))
        for px, py in keypoints:
            if px > 0 and py > 0:
                img.draw_circle(px, py, 2, color=(0, 255, 0), fill=True)
finally:
    pose.deinit()

Each pose result contains 17 COCO keypoints. A missing or low-confidence point is returned as (0, 0) and should be skipped before drawing or computing joint geometry.

Adjust Thresholds at Runtime

det.set_thresholds(score=0.65, nms=0.6)

Thresholds can be changed without reloading the model, which is useful when adapting an application to lighting, distance, or scene-density changes.

Result tuples

TensorInfo: (name, shape, dtype, exponent)
RawTensor: (name, shape, dtype, exponent, bytes)
Detection: (x, y, w, h, score, category)
Pose: (x, y, w, h, score, category, keypoints) with 17 COCO keypoints
Classification: (label, score)

Functions

espdl.load_model(path, *, profile=...)

Preload an ESP-DL model file.

Parameters:

path – model path, for example “/sdcard/model.espdl” or “/flash/model.espdl”.
profile – True enables ESP-DL profiling output when supported.

Classes

class espdl.Model(path, *, mean=..., std=..., letterbox=..., pad=...)

Generic ESP-DL model runner for examples that implement model-specific decoding in Python.

Create a raw model runner from an .espdl model.

Parameters:

path – model path.
mean – optional RGB mean values for image preprocessing.
std – optional RGB standard deviation values for image preprocessing.
letterbox – True keeps aspect ratio and pads the model input.
pad – optional RGB padding values used when letterbox is enabled.

deinit(): Release model resources.

inputs(): Return model input metadata keyed by tensor name.

outputs(): Return model output metadata keyed by tensor name.

predict(image)

Run model inference on an image and return raw output tensors keyed by tensor name.

Parameters:: image – RGB565 or grayscale image.

class espdl.ESPDet(path, *, score=..., nms=..., mean=..., std=...)

ESP-DL object detection wrapper for ESPDet models.

Create a detector from an .espdl model.

Parameters:

path – model path.
score – optional confidence threshold.
nms – optional non-maximum suppression threshold.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.

deinit(): Release model resources.

detect(image)

Run object detection on an image.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, score=..., nms=...)

Update detector thresholds.

Parameters:

score – new confidence threshold, or None to keep current value.
nms – new NMS threshold, or None to keep current value.

class espdl.YOLO11(path, *, score=..., nms=..., topk=..., mean=..., std=...)

ESP-DL YOLO11 object detection wrapper.

Create a YOLO11 detector from an .espdl model.

Parameters:

path – model path.
score – optional confidence threshold.
nms – optional non-maximum suppression threshold.
topk – maximum number of detections returned per frame.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.

deinit(): Release model resources.

detect(image)

Run object detection on an image.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, score=..., nms=...)

Update detector thresholds.

Parameters:

score – new confidence threshold, or None to keep current value.
nms – new NMS threshold, or None to keep current value.

class espdl.YOLO11nPose(path, *, score=..., nms=..., topk=..., mean=..., std=...)

ESP-DL YOLO11n COCO pose wrapper.

Create a YOLO11n pose detector from an .espdl model.

Parameters:

path – model path.
score – optional confidence threshold.
nms – optional non-maximum suppression threshold.
topk – maximum number of pose results returned per frame.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.

deinit(): Release model resources.

detect(image)

Run COCO pose detection on an image. Returns 17 COCO keypoints for each person.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, score=..., nms=...)

Update detector thresholds.

Parameters:

score – new confidence threshold, or None to keep current value.
nms – new NMS threshold, or None to keep current value.

class espdl.ImageNetCls(path, *, topk=..., score=..., mean=..., std=..., softmax=...)

ESP-DL ImageNet classification wrapper.

Create a classifier from an .espdl model.

Parameters:

path – model path.
topk – maximum number of classes returned.
score – optional minimum score threshold.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.
softmax – True applies softmax to output scores.

deinit(): Release model resources.

classify(image)

Run image classification on an image.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, topk=..., score=...)

Update classifier thresholds.

Parameters:

topk – new maximum result count, or None to keep current value.
score – new minimum score, or None to keep current value.