ESP Media Service

[English]

简介

ESP Media Service 为 ESP-ADF 中的音视频服务定义了一套通用媒体接口，以 ESP Service 为基类。应用程序创建服务、配置媒体流，再把源流链接到接收端流。链接并启动后，媒体帧通过读取接口与写入接口自动流动，应用程序不必手动转发帧数据。音视频采集与播放见 ESP-GMF 通用多媒体框架。

功能清单

  • 统一的音视频服务模型，源、接收端、源接收一体三种角色（ESP_MEDIA_ROLE_SRC / SINK / SRC_SINK）

  • 基于 stream ID（esp_media_stream_id_t）的多媒体端点，一个服务可以同时暴露多个 stream

  • 链接时的请求协商：接收端可以通过 esp_media_service_request_t 向源提出诉求（如是否需要 global cache）

  • Provider 读取接口（esp_media_provider_t）与 track manager 写入接口相互解耦

  • 内置默认的内存 track manager（esp_media_track_mngr_t），支持按 track 独立缓存或全局到达顺序缓存

  • 统一通过 ESP Service 管理服务生命周期，媒体相关操作作为独立的 vtable（esp_media_service_ops_t）叠加在基类之上

技术拆解

服务角色与链接

ESP Media Service 通过 esp_media_service_ops_t 中的 get_role 声明自己是源、接收端还是二者兼具；esp_media_service_link() 在链接时校验所选源和接收端的角色是否兼容，再把源的 provider 传给接收端。

        flowchart TD
    Create["创建服务"] --> Config["配置服务"]
    Config --> Link["链接源/接收端 stream"]
    Link --> Start["启动服务"]
    Start --> Flow["媒体帧流动"]
    Flow --> Stop["停止服务"]
    Stop --> Unlink["取消链接并销毁"]
    
esp_media_stream_id_t stream = ESP_MEDIA_DEFAULT_STREAM;

esp_media_service_link(src_service, stream, sink_service, stream);

esp_service_start(sink_service);
esp_service_start(src_service);

链接建立后，媒体数据通过源服务导出的 provider 流向接收端；取消链接需要调用 esp_media_service_unlink()，会清除接收端 stream 上当前设置的 provider。

Provider 与 Track Manager

媒体数据的传递依赖两个互相解耦的接口：接收端在链接时从源服务获取只读的 esp_media_provider_t，通过 esp_media_provider_acquire_frame() / esp_media_provider_release_frame() 获取和释放帧；源服务侧则持有一个 esp_media_track_mngr_t，通过写入 API 产生帧，并把它导出的 provider 句柄暴露给下游。

/* 源服务侧：创建 track manager，注册 track，导出 provider */
esp_media_track_mngr_cfg_t cfg = { .max_track_num = 2 };
esp_media_track_mngr_create(&cfg, &svc->mngr);
esp_media_track_mngr_add_track(svc->mngr, &audio_track);
esp_media_track_mngr_get_provider(svc->mngr, &svc->provider);

/* 接收端侧：读取并释放帧 */
esp_media_frame_t frame = {0};
if (esp_media_provider_acquire_frame(&sink->provider, &frame, timeout_ms) == ESP_OK) {
    process_frame(&frame);
    esp_media_provider_release_frame(&sink->provider, &frame);
}

警告

通过 esp_media_provider_acquire_frame() 获取的帧必须调用 esp_media_provider_release_frame() 释放，释放之后不能再访问 frame.data。

Track Manager 的缓存模式

默认的 esp_media_track_mngr_t 支持两种 payload 所有权模式：ESP_MEDIA_TRACK_CACHE_INTERNAL 由 manager 复制并持有帧数据；ESP_MEDIA_TRACK_CACHE_USER 只缓存帧元数据，payload 仍由用户持有，消费后通过 frame_release 回调归还。此外还支持 global cache，让多个 track 共用一个按到达顺序排列的队列，适合 RTMP 等音视频交错传输场景；启用 global cache 需要在添加 track 之前完成配置。

停止与中止顺序

媒体接口对停止顺序做了容错设计：源服务停止时应调用 esp_media_track_write_abort()，通过 ESP_MEDIA_PROVIDER_EVENT_TRACKS_ABORT 事件通知下游；接收端停止时应先置本地停止标志，再调用 esp_media_provider_abort() 唤醒阻塞的读取，等待任务退出、释放已获取的帧，最后才取消链接。若 track manager 被多个服务通过链接共享，需要先取消链接，再对它执行 reset 或 destroy；只要还有任务持有已获取的帧或阻塞在队列上，就不能对 track manager 执行 reset 或 destroy。

应用示例

源/接收端的完整示例见 esp_media_service 组件仓库 的 examples 目录。服务基类用法见 ESP Service。

FAQ

Q1：一个服务只能实现 ``get_provider`` 或 ``set_provider`` 中的一个吗？

不是必须的。角色为 ESP_MEDIA_ROLE_SRC_SINK 的服务可以同时实现两者，既作为下游的源，也作为上游的接收端；esp_media_service_link() 只按角色位校验所选的一对源/接收端服务是否兼容。

API 参考

Header File

Functions

esp_err_t esp_media_service_init(esp_media_service_t *service, const esp_media_service_config_t *config)

Initialize a media service base object. 

    Derived services call this after allocating or embedding
    esp_media_service_t, before exposing the base esp_service_t
参数:

  • service – [inout] Media service object to initialize

  • config – [in] Media service configuration

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or config is NULL

  • Others Error returned by esp_service_init()

esp_err_t esp_media_service_deinit(esp_service_t *service)

Deinitialize a media service base object.

参数:

service – [in] Base service pointer returned by ESP_SERVICE_BASE()

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service is NULL

  • Others Error returned by esp_service_deinit()

esp_err_t esp_media_service_get_role(esp_service_t *service, esp_media_role_t *out_role)

Query the media role of a service.

参数:

  • service – [in] Base media service handle

  • out_role – [out] Output media role

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or out_role is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_get_provider(esp_service_t *service, esp_media_stream_id_t stream, esp_media_provider_t *out_provider)

Get a provider from a source stream.

参数:

  • service – [in] Base media service handle

  • stream – [in] Source stream ID

  • out_provider – [out] Output provider

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Service or provider is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_set_provider(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_provider_t *provider)

Set a provider on a sink stream. 

    Pass NULL provider to disconnect the sink stream
参数:

  • service – [in] Base media service handle

  • stream – [in] Sink stream ID

  • provider – [in] Provider handle to consume, or NULL to clear

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service is NULL, or provider has no ops

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_get_request(esp_service_t *service, esp_media_stream_id_t stream, esp_media_service_request_t *out_request)

Get link request hints for a media stream.

参数:

  • service – [in] Base media service handle

  • stream – [in] Stream ID

  • out_request – [out] Output request hints

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or out_request is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_set_request(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_service_request_t *request)

Set link request hints for a media stream.

参数:

  • service – [in] Base media service handle

  • stream – [in] Stream ID

  • request – [in] Request hints to apply

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or request is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_link(esp_service_t *src_service, esp_media_stream_id_t src_stream, esp_service_t *sink_service, esp_media_stream_id_t sink_stream)

Link a source stream to a sink stream. 

    The sink request hints are applied to the source when supported, then
    the source provider is passed to the sink
参数:

  • src_service – [in] Source media service handle

  • src_stream – [in] Source stream ID

  • sink_service – [in] Sink media service handle

  • sink_stream – [in] Sink stream ID

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG src_service or sink_service is NULL

  • ESP_ERR_NOT_SUPPORTED Role check failed or required operation is missing

  • Others Error returned by source or sink implementation

esp_err_t esp_media_service_unlink(esp_service_t *src_service, esp_media_stream_id_t src_stream, esp_service_t *sink_service, esp_media_stream_id_t sink_stream)

Unlink a source stream from a sink stream. 

    Clears the provider currently set on the sink stream
参数:

  • src_service – [in] Source media service handle

  • src_stream – [in] Source stream ID

  • sink_service – [in] Sink media service handle

  • sink_stream – [in] Sink stream ID

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG src_service or sink_service is NULL

  • ESP_ERR_NOT_SUPPORTED Role check failed or sink operation is missing

  • Others Error returned by source or sink implementation

Structures

struct esp_media_service_request_t

Media-service link request/capability hints. 

    The structure is intentionally extensible so future services can add
    link-time requests without changing the media op shape

Public Members

bool need_global_cache

Sink requests source frames in one arrival-order cache

struct esp_media_service_ops_t

Media-service virtual operations.

Public Members

esp_err_t (*get_role)(esp_service_t *service, esp_media_role_t *out_role)

Query source/sink role

esp_err_t (*get_provider)(esp_service_t *service, esp_media_stream_id_t stream, esp_media_provider_t *out_provider)

Get provider exported by a source stream

esp_err_t (*set_provider)(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_provider_t *provider)

Set provider consumed by a sink stream

esp_err_t (*get_request)(esp_service_t *service, esp_media_stream_id_t stream, esp_media_service_request_t *request)

Get sink link requests

esp_err_t (*set_request)(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_service_request_t *request)

Apply sink requests to a source

struct esp_media_service_config_t

Media service configuration.

Public Members

const char *name

Service instance name

void *user_data

User data passed to esp_service

const esp_service_ops_t *service_ops

Optional esp_service lifecycle ops

const esp_media_service_ops_t *media_ops

Optional media ops

struct esp_media_service

Base media service structure Derived services embed this as the first member.

Public Members

esp_service_t base

Base service, must stay first

const esp_media_service_ops_t *media_ops

Media virtual operations

Macros

ESP_MEDIA_SERVICE_CONFIG_DEFAULT()

SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

See LICENSE file for details. Default media service configuration

ESP_MEDIA_DEFAULT_STREAM

Default media stream ID

Type Definitions

typedef uint16_t esp_media_stream_id_t

Instance-aware media stream address.

typedef struct esp_media_service esp_media_service_t

Base media service structure Derived services embed this as the first member.

Enumerations

enum esp_media_role_t

Media service data direction.

Values:

enumerator ESP_MEDIA_ROLE_NONE
enumerator ESP_MEDIA_ROLE_SRC

Service produces media frames

enumerator ESP_MEDIA_ROLE_SINK

Service consumes media frames

enumerator ESP_MEDIA_ROLE_SRC_SINK

Service produces and consumes frames

Header File

Functions

esp_err_t esp_media_provider_get_track_num(const esp_media_provider_t *provider, uint16_t *out_num)

Get the number of tracks exposed by a provider.

SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

See LICENSE file for details.

参数:

  • provider – [in] Provider handle

  • out_num – [out] Output track count

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_num is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_get_track_info(const esp_media_provider_t *provider, uint16_t index, esp_media_track_info_t *out_info)

Get track metadata by provider track index.

参数:

  • provider – [in] Provider handle

  • index – [in] Track index in the provider

  • out_info – [out] Output track metadata

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_info is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_set_event_cb(const esp_media_provider_t *provider, esp_media_provider_event_cb_t cb, void *event_ctx)

Register a provider event callback.

参数:

  • provider – [in] Provider handle

  • cb – [in] Event callback, or NULL to clear it

  • event_ctx – [in] User context passed to cb

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_abort(const esp_media_provider_t *provider)

Abort provider-side blocking read operations.

参数:

provider – [in] Provider handle

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_acquire_frame(const esp_media_provider_t *provider, esp_media_frame_t *out_frame, uint32_t timeout_ms)

Acquire the next frame from a provider. 

    The returned frame must be released with esp_media_provider_release_frame().
    Set fields such as type or track_id in out_frame before the call to
    select a specific track when the provider supports it
参数:

  • provider – [in] Provider handle

  • out_frame – [inout] Input track selector, output acquired frame

  • timeout_ms – [in] Timeout in milliseconds; 0 means no wait, UINT32_MAX means wait forever

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_frame is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_read_frame(const esp_media_provider_t *provider, esp_media_frame_t *out_frame, uint32_t timeout_ms)

Read the next frame into caller-provided storage. 

    Set out_frame->data and out_frame->size before calling. On success,
    out_frame->size is updated to the actual payload size
参数:

  • provider – [in] Provider handle

  • out_frame – [inout] Input buffer descriptor, output frame descriptor

  • timeout_ms – [in] Timeout in milliseconds; 0 means no wait, UINT32_MAX means wait forever

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_frame is NULL

  • ESP_ERR_INVALID_SIZE Provided buffer is too small

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_release_frame(const esp_media_provider_t *provider, esp_media_frame_t *frame)

Release a frame acquired from a provider.

参数:

  • provider – [in] Provider handle

  • frame – [in] Frame returned by esp_media_provider_acquire_frame()

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or frame is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

Header File

Functions

esp_err_t esp_media_track_mngr_create(const esp_media_track_mngr_cfg_t *cfg, esp_media_track_mngr_t **out_mngr)

Create a media track manager.

参数:

  • cfg – [in] Provider configuration

  • out_mngr – [out] Output track manager handle

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG cfg is NULL, max_track_num is 0, or out_provider is NULL

  • ESP_ERR_NO_MEM Allocation failed

esp_err_t esp_media_track_mngr_destroy(esp_media_track_mngr_t *mngr)

Destroy a media track manager. 

    Wakes and destroys internal queues, releases pending user-owned
    frames through their release callbacks, and frees the track manager
参数:

mngr – [in] Track manager handle returned by esp_media_track_mngr_create()

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

esp_err_t esp_media_track_mngr_reset(esp_media_track_mngr_t *mngr)

Reset tracks and queued data. 

    Removes all tracks and clears abort state. Global-cache mode is kept.
    User must re-add tracks after reset
参数:

mngr – [in] Track manager handle

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

esp_err_t esp_media_track_mngr_set_global_cache(esp_media_track_mngr_t *mngr, bool enable, size_t cache_size)

Configure global cache mode before tracks are added.

参数:

  • mngr – [in] Track manager handle

  • enable – [in] true to use one arrival-order cache shared by all tracks

  • cache_size – [in] Shared queue byte size, 0 uses default

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

  • ESP_ERR_INVALID_STATE Tracks have already been added

  • ESP_ERR_NO_MEM Allocation failed

esp_err_t esp_media_track_mngr_add_track(esp_media_track_mngr_t *mngr, const esp_media_track_mngr_track_cfg_t *cfg)

Add a track to a track manager.

参数:

  • mngr – [in] Track manager handle

  • cfg – [in] Track configuration

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager or cfg is NULL

  • ESP_ERR_NO_MEM Track limit reached or queue allocation failed

esp_err_t esp_media_track_mngr_update_track(esp_media_track_mngr_t *mngr, uint16_t index, const esp_media_track_info_t *info)

Queue a track metadata update. 

    The update is committed when provider acquire/read reaches the queued
    zero-size frame with ESP_MEDIA_FRAME_FLAG_TRACK_CHANGED. The provider
    event callback is invoked synchronously at that point
参数:

  • mngr – [in] Track manager handle

  • index – [in] Track index to update

  • info – [in] New track metadata

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager or info is NULL

  • ESP_ERR_NOT_FOUND index is out of range

  • ESP_ERR_NO_MEM Track queue is unavailable

  • ESP_ERR_TIMEOUT Failed to queue update frame

  • ESP_FAIL Failed to commit update frame

esp_err_t esp_media_track_mngr_remove_track(esp_media_track_mngr_t *mngr, uint16_t index)

Queue a track removal notification. 

    The removal is committed when provider acquire/read reaches the queued
    zero-size frame with ESP_MEDIA_FRAME_FLAG_TRACK_REMOVED. The provider
    event callback is invoked synchronously at that point
参数:

  • mngr – [in] Track manager handle

  • index – [in] Track index to remove

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

  • ESP_ERR_NOT_FOUND index is out of range

  • ESP_ERR_NO_MEM Track queue is unavailable

  • ESP_ERR_TIMEOUT Failed to queue removal frame

  • ESP_FAIL Failed to commit removal frame

esp_err_t esp_media_track_mngr_get_provider(esp_media_track_mngr_t *mngr, esp_media_provider_t *provider)

Get the media provider handle exported by a track manager.

参数:

  • mngr – [in] Track manager handle

  • provider – [out] Output media provider

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager or provider is NULL

Structures

struct esp_media_track_mngr_cfg_t

Media track manager configuration.

备注

The global cache used or not can be reconfigured by API esp_media_track_mngr_set_global_cache

Public Members

uint16_t max_track_num

Maximum number of tracks

bool use_global_cache

Use one arrival-order cache shared by all tracks

size_t cache_size

Shared queue byte size, 0 uses default

struct esp_media_track_mngr_cache_cfg_t

Media provider track buffer configuration.

Public Members

esp_media_track_cache_type_t cache_type

Track cache type

size_t cache_size

Queue byte size for track manager-owned payload mode

uint16_t addr_align

Track manager-owned frame data alignment, 0 uses pointer size

uint16_t size_align

Cached frame size alignment, 0 no special request

uint16_t queue_num

Metadata queue depth

esp_media_frame_release_cb_t frame_release

Release callback for user-owned frames

void *release_ctx

Context for frame_release

struct esp_media_track_mngr_track_cfg_t

Media track manager track configuration.

Public Members

esp_media_track_info_t info

Track metadata

esp_media_track_mngr_cache_cfg_t cache_cfg

Track buffer/cache settings

Type Definitions

typedef struct esp_media_track_mngr esp_media_track_mngr_t

Definition of media track manager.

SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

See LICENSE file for details.

typedef void (*esp_media_frame_release_cb_t)(const esp_media_frame_t *frame, void *ctx)

Release callback for user-owned frames after consumed.

Param frame:

[in] Frame whose payload can be released by the owner

Param ctx:

[in] User context from esp_media_track_mngr_cache_cfg_t

Enumerations

enum esp_media_track_cache_type_t

Media track cache type.

Values:

enumerator ESP_MEDIA_TRACK_CACHE_INTERNAL

Track manager owns queued frame payload data

enumerator ESP_MEDIA_TRACK_CACHE_USER

Track manager queues frame metadata only, user owns payload data

提供有关此文档的反馈