PicoDet

Component registry: espressif/brookesia_service_picodet
Helper header: #include "brookesia/service_helper/media/picodet.hpp"
Helper class: esp_brookesia::service::helper::PicoDet

Overview

brookesia_service_picodet is an on-device object-detection service based on esp-dl ESPDet-Pico models. It provides:

Model lifecycle: Load one model with Open, reuse it for inference, and release it with Close.
File inference: Detect objects in JPEG or BMP files with Detect.
Frame-stream inference: Subscribe to a caller-selected frame event with Attach.
Annotated display output: Draw cached boxes onto incoming frames and present them through the Display service.

Features

Model Package

A model package is a filesystem directory containing an ESP-DL model and a manifest.json file. The manifest controls the following fields:

Field	Description
`model_file`	ESP-DL model filename relative to the package directory.
`input`	Model input width and height.
`preprocess`	Mean, standard deviation, channel swap, and letterbox color.
`postprocess`	Score threshold, NMS threshold, top-K limit, and anchor-point strides.
`labels`	Class names indexed by the category values returned from inference.

Supporting another compatible ESPDet-Pico model normally requires replacing the model package rather than changing the service implementation.

Model Lifecycle and File Inference

The normal file-inference sequence is:

Call Open with model_dir; optional score_thr and nms_thr values override the manifest.
Call Detect with a .jpg, .jpeg, or .bmp path.
Consume the returned {category, label, score, box} objects; box is [x_min, y_min, x_max, y_max].
Call Close to release the detector and model memory.

GetInfo reports whether a model is open together with its input size and labels. The service keeps only one model open at a time.

Frame-Stream Attachment

Attach dynamically subscribes to frame_event on frame_source. Defaults are VideoEncoder0 and StreamSinkFrameReady. The source service remains owned by the caller.

The subscribed event must contain:

Item	Type	Description
`Frame`	`RawBuffer`	Writable RGB frame data.
`SinkInfo`	`Object`	`{format, width, height}`; format supports RGB565 and RGB888.

PicoDet runs inference every detect_every_n_frames frames, caches the latest boxes, and publishes DetectionUpdated after each inference. For example, a value of 5 means inference runs on every fifth frame while cached boxes are used on intermediate frames.

Call Detach to disconnect the frame event and release any Display output binding. Detach is idempotent.

Display Output

When display_output is non-empty, PicoDet registers a Display source, requests the named output, draws boxes in place, and presents frames synchronously with Display::present_frame_sync().

The incoming frame format must match the selected Display output pixel format. Because detection, drawing, and presentation share the frame callback path, inference latency affects the effective stream frame rate.

Camera Integration

For camera input:

Open VideoEncoder0 with enable_stream_mode: true so it publishes frame buffers.
Do not enable the Video service's own Display preview when PicoDet owns annotated display output.
Start and stop the Video service separately; Attach does not own the camera-service lifecycle.
Increase detect_every_n_frames when lower inference frequency is preferable to blocking every frame.

Model Deployment

PicoDet accepts a filesystem path and does not generate, download, or install model packages. Common deployment methods are:

Pre-provisioned: Add model.espdl and manifest.json to a LittleFS or SD-card image.
Runtime download: Use the HTTP service's download_path support to store both files, then pass their directory to Open.

Standard Include / Helper Class

Standard include: #include \"brookesia/service_helper/media/picodet.hpp\"
Helper class: esp_brookesia::service::helper::PicoDet

Service Interfaces

Functions

`Open`

Description

Load a PicoDet model. Example config: {"model_dir":"/littlefs/models/picodet_cat","score_thr":6E-1,"nms_thr":7E-1}

Execution

Requires scheduler: Required

Parameters

Config
- Type: Object
- Required: required
- Description: Model load configuration. 'model_dir' is required; thresholds are optional overrides.

Return Value

Type: Object
Description: Loaded model info. Example: {"opened":true,"width":224,"height":224,"labels":["cat"]}

Schema JSON

Show raw JSON


{
  "name": "Open",
  "description": "Load a PicoDet model. Example config: {\"model_dir\":\"/littlefs/models/picodet_cat\",\"score_thr\":6E-1,\"nms_thr\":7E-1}",
  "require_scheduler": true,
  "default_timeout_ms": 5000,
  "parameters": [
    {
      "name": "Config",
      "description": "Model load configuration. 'model_dir' is required; thresholds are optional overrides.",
      "type": "Object",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": {
    "type": "Object",
    "description": "Loaded model info. Example: {\"opened\":true,\"width\":224,\"height\":224,\"labels\":[\"cat\"]}"
  }
}

CLI Command

svc_call PicoDet Open {"Config":null}

`Detect`

Description

Run detection on one image file and return the detected boxes.

Execution

Requires scheduler: Required

Parameters

ImagePath
- Type: String
- Required: required
- Description: Filesystem path to a .jpg/.jpeg or .bmp image.

Return Value

Type: Array
Description: Array of boxes. Example: {"category":0,"label":"cat","score":8.7E-1,"box":[12,34,120,210]}

Schema JSON

Show raw JSON


{
  "name": "Detect",
  "description": "Run detection on one image file and return the detected boxes.",
  "require_scheduler": true,
  "default_timeout_ms": 5000,
  "parameters": [
    {
      "name": "ImagePath",
      "description": "Filesystem path to a .jpg/.jpeg or .bmp image.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": {
    "type": "Array",
    "description": "Array of boxes. Example: {\"category\":0,\"label\":\"cat\",\"score\":8.7E-1,\"box\":[12,34,120,210]}"
  }
}

CLI Command

svc_call PicoDet Detect {"ImagePath":null}

`Close`

Description

Unload the model and reclaim memory.

Execution

Requires scheduler: Required

Parameters

No parameters.

Schema JSON

Show raw JSON


{
  "name": "Close",
  "description": "Unload the model and reclaim memory.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}

CLI Command

svc_call PicoDet Close

`GetInfo`

Description

Query loaded state, model input size and labels.

Execution

Requires scheduler: Not required

Parameters

No parameters.

Return Value

Type: Object
Description: Loaded model info. Example: {"opened":true,"width":224,"height":224,"labels":["cat"]}

Schema JSON

Show raw JSON


{
  "name": "GetInfo",
  "description": "Query loaded state, model input size and labels.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Object",
    "description": "Loaded model info. Example: {\"opened\":true,\"width\":224,\"height\":224,\"labels\":[\"cat\"]}"
  }
}

CLI Command

svc_call PicoDet GetInfo

`Attach`

Description

Wire the detector into the pipeline: subscribe to 'frame_event' on the 'frame_source' service (the caller owns that service's lifecycle) and run inference every N frames, publishing DetectionUpdated. When 'display_output' is set, also draw the boxes onto each frame and present it to that display output. Requires Open first. Example config: {"frame_source":"VideoEncoder0","frame_event":"StreamSinkFrameReady","display_output":"Output0","detect_every_n_frames":5}

Execution

Requires scheduler: Required

Parameters

Config
- Type: Object
- Required: optional
- Default: {}
- Description: Attach configuration; all fields are optional.

Schema JSON

Show raw JSON


{
  "name": "Attach",
  "description": "Wire the detector into the pipeline: subscribe to 'frame_event' on the 'frame_source' service (the caller owns that service's lifecycle) and run inference every N frames, publishing DetectionUpdated. When 'display_output' is set, also draw the boxes onto each frame and present it to that display output. Requires Open first. Example config: {\"frame_source\":\"VideoEncoder0\",\"frame_event\":\"StreamSinkFrameReady\",\"display_output\":\"Output0\",\"detect_every_n_frames\":5}",
  "require_scheduler": true,
  "default_timeout_ms": 5000,
  "parameters": [
    {
      "name": "Config",
      "description": "Attach configuration; all fields are optional.",
      "type": "Object",
      "required": false,
      "default_value": {}
    }
  ],
  "return_value": null
}

CLI Command

svc_call PicoDet Attach {"Config":null}

`Detach`

Description

Unwire the detector from the frame source and release the display output.

Execution

Requires scheduler: Required

Parameters

No parameters.

Schema JSON

Show raw JSON


{
  "name": "Detach",
  "description": "Unwire the detector from the frame source and release the display output.",
  "require_scheduler": true,
  "default_timeout_ms": 5000,
  "parameters": [],
  "return_value": null
}

CLI Command

svc_call PicoDet Detach

Events

`DetectionUpdated`

Description

Published after each inference while attached, with the fresh detection boxes.

Execution

Requires scheduler: Required

Items

Boxes
- Type: Array
- Description: Array of boxes. Example item: {"category":0,"label":"cat","score":8.7E-1,"box":[12,34,120,210]}

Schema JSON

Show raw JSON


{
  "name": "DetectionUpdated",
  "description": "Published after each inference while attached, with the fresh detection boxes.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Boxes",
      "description": "Array of boxes. Example item: {\"category\":0,\"label\":\"cat\",\"score\":8.7E-1,\"box\":[12,34,120,210]}",
      "type": "Array"
    }
  ]
}

CLI Command

svc_subscribe PicoDet DetectionUpdated

PicoDet

Overview

Features

Model Package

Model Lifecycle and File Inference

Frame-Stream Attachment

Display Output

Camera Integration

Model Deployment

Standard Include / Helper Class

Service Interfaces

Functions

Open

Description

Execution

Parameters

Return Value

Schema JSON

CLI Command

Detect

Description

Execution

Parameters

Return Value

Schema JSON

CLI Command

Close

Description

Execution

Parameters

Schema JSON

CLI Command

GetInfo

Description

Execution

Parameters

Return Value

Schema JSON

CLI Command

Attach

Description

Execution

Parameters

Schema JSON

CLI Command

Detach

Description

Execution

Parameters

Schema JSON

CLI Command

Events

DetectionUpdated

Description

Execution

Items

Schema JSON

CLI Command

Related Types and Configs

`Open`

`Detect`

`Close`

`GetInfo`

`Attach`

`Detach`

`DetectionUpdated`