espdl – Model Inference

[中文]

The espdl module runs ESP-DL .espdl models on captured images. It provides task-specific wrappers for object detection (ESPDet, YOLO11), pose estimation (YOLO11nPose), and image classification (ImageNetCls).

Object Detection

import sensor, espdl

det = espdl.ESPDet("/sdcard/espdet_pico_224_224_face.espdl", score=0.5, nms=0.7)
try:
    img = sensor.snapshot()
    for x, y, w, h, score, category in det.detect(img):
        img.draw_rectangle(x, y, w, h, color=(255, 0, 0), thickness=2)
        img.draw_string(x, max(0, y - 12), "%.2f:%d" % (score, category))
finally:
    det.deinit()

score removes low-confidence candidates, while nms controls suppression of overlapping boxes. Detection coordinates are mapped back to the input image and can be used directly for drawing. Load a model once and reuse it across frames; deinit() releases model weights and intermediate buffers.

Image Classification

import espdl, image

img = image.Image("/sdcard/cat.jpg").to_rgb565(copy=True)
classifier = espdl.ImageNetCls(
    "/sdcard/imagenet_cls_mobilenetv2_s8_v1.espdl",
    topk=5,
    score=0.0,
)
try:
    for label, score in classifier.classify(img):
        print(label, "%.4f" % score)
finally:
    classifier.deinit()

Classification returns up to topk (label, score) pairs ordered by the model output. Convert file images to RGB565 when necessary so the input format is accepted by the wrapper.

Pose Estimation

pose = espdl.YOLO11nPose("/sdcard/espdet_yolo11n_pose_160_160_coco.espdl", score=0.35, topk=5)
try:
    img = sensor.snapshot()
    for x, y, w, h, score, category, keypoints in pose.detect(img):
        img.draw_rectangle(x, y, w, h, color=(255, 0, 0))
        for px, py in keypoints:
            if px > 0 and py > 0:
                img.draw_circle(px, py, 2, color=(0, 255, 0), fill=True)
finally:
    pose.deinit()

Each pose result contains 17 COCO keypoints. A missing or low-confidence point is returned as (0, 0) and should be skipped before drawing or computing joint geometry.

Adjust Thresholds at Runtime

det.set_thresholds(score=0.65, nms=0.6)

Thresholds can be changed without reloading the model, which is useful when adapting an application to lighting, distance, or scene-density changes.

Result tuples

Detection: (x, y, w, h, score, category)
Pose: (x, y, w, h, score, category, keypoints) with 17 COCO keypoints
Classification: (label, score)

Functions

espdl.load_model(path, *, profile=...)

Preload an ESP-DL model file.

Parameters:

path – model path, for example “/sdcard/model.espdl” or “/flash/model.espdl”.
profile – True enables ESP-DL profiling output when supported.

Classes

class espdl.ESPDet(path, *, score=..., nms=..., mean=..., std=...)

ESP-DL object detection wrapper for ESPDet models.

Create a detector from an .espdl model.

Parameters:

path – model path.
score – optional confidence threshold.
nms – optional non-maximum suppression threshold.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.

deinit(): Release model resources.

detect(image)

Run object detection on an image.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, score=..., nms=...)

Update detector thresholds.

Parameters:

score – new confidence threshold, or None to keep current value.
nms – new NMS threshold, or None to keep current value.

class espdl.YOLO11(path, *, score=..., nms=..., topk=..., mean=..., std=...)

ESP-DL YOLO11 object detection wrapper.

Create a YOLO11 detector from an .espdl model.

Parameters:

path – model path.
score – optional confidence threshold.
nms – optional non-maximum suppression threshold.
topk – maximum number of detections returned per frame.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.

deinit(): Release model resources.

detect(image)

Run object detection on an image.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, score=..., nms=...)

Update detector thresholds.

Parameters:

score – new confidence threshold, or None to keep current value.
nms – new NMS threshold, or None to keep current value.

class espdl.YOLO11nPose(path, *, score=..., nms=..., topk=..., mean=..., std=...)

ESP-DL YOLO11n COCO pose wrapper.

Create a YOLO11n pose detector from an .espdl model.

Parameters:

path – model path.
score – optional confidence threshold.
nms – optional non-maximum suppression threshold.
topk – maximum number of pose results returned per frame.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.

deinit(): Release model resources.

detect(image)

Run COCO pose detection on an image. Returns 17 COCO keypoints for each person.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, score=..., nms=...)

Update detector thresholds.

Parameters:

score – new confidence threshold, or None to keep current value.
nms – new NMS threshold, or None to keep current value.

class espdl.ImageNetCls(path, *, topk=..., score=..., mean=..., std=..., softmax=...)

ESP-DL ImageNet classification wrapper.

Create a classifier from an .espdl model.

Parameters:

path – model path.
topk – maximum number of classes returned.
score – optional minimum score threshold.
mean – optional RGB mean values for preprocessing.
std – optional RGB standard deviation values for preprocessing.
softmax – True applies softmax to output scores.

deinit(): Release model resources.

classify(image)

Run image classification on an image.

Parameters:: image – RGB565 or grayscale image.

set_thresholds(*, topk=..., score=...)

Update classifier thresholds.

Parameters:

topk – new maximum result count, or None to keep current value.
score – new minimum score, or None to keep current value.