ESP Media Service

[English]

简介

ESP Media Service 为 ESP-ADF 中的音视频服务定义了一套通用媒体接口，以 ESP Service 为基类。应用程序创建服务、配置媒体流，再把源流链接到接收端流。链接并启动后，媒体帧通过读取接口与写入接口自动流动，应用程序不必手动转发帧数据。音视频采集与播放见 ESP-GMF 通用多媒体框架。

功能清单

  • 统一的音视频服务模型，源、接收端、源接收一体三种角色（ESP_MEDIA_ROLE_SRC / SINK / SRC_SINK）

  • 基于 stream ID（esp_media_stream_id_t）的多媒体端点，一个服务可以同时暴露多个 stream

  • 链接时的请求协商：接收端可以通过 esp_media_service_request_t 向源提出诉求（如是否需要 global cache）

  • Provider 读取接口（esp_media_provider_t）与 track manager 写入接口相互解耦

  • 内置默认的内存 track manager（esp_media_track_mngr_t），支持按 track 独立缓存或全局到达顺序缓存

  • 统一通过 ESP Service 管理服务生命周期，媒体相关操作作为独立的 vtable（esp_media_service_ops_t）叠加在基类之上

技术拆解

服务角色与链接

ESP Media Service 通过 esp_media_service_ops_t 中的 get_role 声明自己是源、接收端还是二者兼具；esp_media_service_link() 在链接时校验所选源和接收端的角色是否兼容，再把源的 provider 传给接收端。

        flowchart TD
    Create["创建服务"] --> Config["配置服务"]
    Config --> Link["链接源/接收端 stream"]
    Link --> Start["启动服务"]
    Start --> Flow["媒体帧流动"]
    Flow --> Stop["停止服务"]
    Stop --> Unlink["取消链接并销毁"]
    
esp_media_stream_id_t stream = ESP_MEDIA_DEFAULT_STREAM;

esp_media_service_link(src_service, stream, sink_service, stream);

esp_service_start(sink_service);
esp_service_start(src_service);

链接建立后，媒体数据通过源服务导出的 provider 流向接收端；取消链接需要调用 esp_media_service_unlink()，会清除接收端 stream 上当前设置的 provider。

Provider 与 Track Manager

媒体数据的传递依赖两个互相解耦的接口：接收端在链接时从源服务获取只读的 esp_media_provider_t，通过 esp_media_provider_acquire_frame() / esp_media_provider_release_frame() 获取和释放帧；源服务侧则持有一个 esp_media_track_mngr_t，通过写入 API 产生帧，并把它导出的 provider 句柄暴露给下游。

/* 源服务侧：创建 track manager，注册 track，导出 provider */
esp_media_track_mngr_cfg_t cfg = { .max_track_num = 2 };
esp_media_track_mngr_create(&cfg, &svc->mngr);
esp_media_track_mngr_add_track(svc->mngr, &audio_track);
esp_media_track_mngr_get_provider(svc->mngr, &svc->provider);

/* 接收端侧：读取并释放帧 */
esp_media_frame_t frame = {0};
if (esp_media_provider_acquire_frame(&sink->provider, &frame, timeout_ms) == ESP_OK) {
    process_frame(&frame);
    esp_media_provider_release_frame(&sink->provider, &frame);
}

警告

通过 esp_media_provider_acquire_frame() 获取的帧必须调用 esp_media_provider_release_frame() 释放，释放之后不能再访问 frame.data。

Track Manager 的缓存模式

默认的 esp_media_track_mngr_t 支持两种 payload 所有权模式：ESP_MEDIA_TRACK_CACHE_INTERNAL 由 manager 复制并持有帧数据；ESP_MEDIA_TRACK_CACHE_USER 只缓存帧元数据，payload 仍由用户持有，消费后通过 frame_release 回调归还。此外还支持 global cache，让多个 track 共用一个按到达顺序排列的队列，适合 RTMP 等音视频交错传输场景；启用 global cache 需要在添加 track 之前完成配置。

停止与中止顺序

媒体接口对停止顺序做了容错设计：源服务停止时应调用 esp_media_track_write_abort()，通过 ESP_MEDIA_PROVIDER_EVENT_TRACKS_ABORT 事件通知下游；接收端停止时应先置本地停止标志，再调用 esp_media_provider_abort() 唤醒阻塞的读取，等待任务退出、释放已获取的帧，最后才取消链接。若 track manager 被多个服务通过链接共享，需要先取消链接，再对它执行 reset 或 destroy；只要还有任务持有已获取的帧或阻塞在队列上，就不能对 track manager 执行 reset 或 destroy。

应用示例

源/接收端的完整示例见 esp_media_service 组件仓库 的 examples 目录。服务基类用法见 ESP Service。SIP / RTSP / RTMP 服务例程见 多媒体例程 中的 sip_cli、rtsp_cli、rtsp_push、rtmp_cli：应用程序 link 采集、协议与播放，不搬运帧。

FAQ

Q1：一个服务只能实现 ``get_provider`` 或 ``set_provider`` 中的一个吗？

不是必须的。角色为 ESP_MEDIA_ROLE_SRC_SINK 的服务可以同时实现两者，既作为下游的源，也作为上游的接收端；esp_media_service_link() 只按角色位校验所选的一对源/接收端服务是否兼容。

API 参考

Header File

Functions

esp_err_t esp_media_service_init(esp_media_service_t *service, const esp_media_service_config_t *config)

Initialize a media service base object. 

    Derived services call this after allocating or embedding
    esp_media_service_t, before exposing the base esp_service_t
参数:

  • service – [inout] Media service object to initialize

  • config – [in] Media service configuration

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or config is NULL

  • Others Error returned by esp_service_init()

esp_err_t esp_media_service_deinit(esp_service_t *service)

Deinitialize a media service base object.

参数:

service – [in] Base service pointer returned by ESP_SERVICE_BASE()

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service is NULL

  • Others Error returned by esp_service_deinit()

esp_err_t esp_media_service_get_role(esp_service_t *service, esp_media_role_t *out_role)

Query the media role of a service.

参数:

  • service – [in] Base media service handle

  • out_role – [out] Output media role

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or out_role is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_get_provider(esp_service_t *service, esp_media_stream_id_t stream, esp_media_provider_t *out_provider)

Get a provider from a source stream.

参数:

  • service – [in] Base media service handle

  • stream – [in] Source stream ID

  • out_provider – [out] Output provider

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Service or provider is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_set_provider(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_provider_t *provider)

Set a provider on a sink stream. 

    Pass NULL provider to disconnect the sink stream
参数:

  • service – [in] Base media service handle

  • stream – [in] Sink stream ID

  • provider – [in] Provider handle to consume, or NULL to clear

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service is NULL, or provider has no ops

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_get_request(esp_service_t *service, esp_media_stream_id_t stream, esp_media_service_request_t *out_request)

Get link request hints for a media stream.

参数:

  • service – [in] Base media service handle

  • stream – [in] Stream ID

  • out_request – [out] Output request hints

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or out_request is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_set_request(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_service_request_t *request)

Set link request hints for a media stream.

参数:

  • service – [in] Base media service handle

  • stream – [in] Stream ID

  • request – [in] Request hints to apply

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG service or request is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

  • Others Error returned by the service implementation

esp_err_t esp_media_service_link(esp_service_t *src_service, esp_media_stream_id_t src_stream, esp_service_t *sink_service, esp_media_stream_id_t sink_stream)

Link a source stream to a sink stream. 

    The sink request hints are applied to the source when supported, then
    the source provider is passed to the sink
参数:

  • src_service – [in] Source media service handle

  • src_stream – [in] Source stream ID

  • sink_service – [in] Sink media service handle

  • sink_stream – [in] Sink stream ID

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG src_service or sink_service is NULL

  • ESP_ERR_NOT_SUPPORTED Role check failed or required operation is missing

  • Others Error returned by source or sink implementation

esp_err_t esp_media_service_unlink(esp_service_t *src_service, esp_media_stream_id_t src_stream, esp_service_t *sink_service, esp_media_stream_id_t sink_stream)

Unlink a source stream from a sink stream. 

    Clears the provider currently set on the sink stream
参数:

  • src_service – [in] Source media service handle

  • src_stream – [in] Source stream ID

  • sink_service – [in] Sink media service handle

  • sink_stream – [in] Sink stream ID

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG src_service or sink_service is NULL

  • ESP_ERR_NOT_SUPPORTED Role check failed or sink operation is missing

  • Others Error returned by source or sink implementation

Structures

struct esp_media_service_request_t

Media-service link request/capability hints. 

    The structure is intentionally extensible so future services can add
    link-time requests without changing the media op shape

Public Members

bool need_global_cache

Sink requests source frames in one arrival-order cache

struct esp_media_service_ops_t

Media-service virtual operations.

Public Members

esp_err_t (*get_role)(esp_service_t *service, esp_media_role_t *out_role)

Query source/sink role

esp_err_t (*get_provider)(esp_service_t *service, esp_media_stream_id_t stream, esp_media_provider_t *out_provider)

Get provider exported by a source stream

esp_err_t (*set_provider)(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_provider_t *provider)

Set provider consumed by a sink stream

esp_err_t (*get_request)(esp_service_t *service, esp_media_stream_id_t stream, esp_media_service_request_t *request)

Get sink link requests

esp_err_t (*set_request)(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_service_request_t *request)

Apply sink requests to a source

struct esp_media_service_config_t

Media service configuration.

Public Members

const char *name

Service instance name

void *user_data

User data passed to esp_service

const esp_service_ops_t *service_ops

Optional esp_service lifecycle ops

const esp_media_service_ops_t *media_ops

Optional media ops

struct esp_media_service

Base media service structure Derived services embed this as the first member.

Public Members

esp_service_t base

Base service, must stay first

const esp_media_service_ops_t *media_ops

Media virtual operations

Macros

ESP_MEDIA_SERVICE_CONFIG_DEFAULT()

SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

See LICENSE file for details. Default media service configuration

ESP_MEDIA_DEFAULT_STREAM

Default media stream ID

Type Definitions

typedef uint16_t esp_media_stream_id_t

Instance-aware media stream address.

typedef struct esp_media_service esp_media_service_t

Base media service structure Derived services embed this as the first member.

Enumerations

enum esp_media_role_t

Media service data direction.

Values:

enumerator ESP_MEDIA_ROLE_NONE
enumerator ESP_MEDIA_ROLE_SRC

Service produces media frames

enumerator ESP_MEDIA_ROLE_SINK

Service consumes media frames

enumerator ESP_MEDIA_ROLE_SRC_SINK

Service produces and consumes frames

Header File

Functions

esp_err_t esp_media_provider_get_track_num(const esp_media_provider_t *provider, uint16_t *out_num)

Get the number of tracks exposed by a provider.

SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

See LICENSE file for details.

参数:

  • provider – [in] Provider handle

  • out_num – [out] Output track count

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_num is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_get_track_info(const esp_media_provider_t *provider, uint16_t index, esp_media_track_info_t *out_info)

Get track metadata by provider track index.

参数:

  • provider – [in] Provider handle

  • index – [in] Track index in the provider

  • out_info – [out] Output track metadata

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_info is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_set_event_cb(const esp_media_provider_t *provider, esp_media_provider_event_cb_t cb, void *event_ctx)

Register a provider event callback.

参数:

  • provider – [in] Provider handle

  • cb – [in] Event callback, or NULL to clear it

  • event_ctx – [in] User context passed to cb

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_abort(const esp_media_provider_t *provider)

Abort provider-side blocking read operations. 

    Wakes blocked readers/writers. USER-cache payloads still held or queued
    are returned via frame_release before waiters are woken.
参数:

provider – [in] Provider handle

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_acquire_frame(const esp_media_provider_t *provider, esp_media_frame_t *out_frame, uint32_t timeout_ms)

Acquire the next frame from a provider. 

    The returned frame must be released with esp_media_provider_release_frame().
    Set fields such as type or track_id in out_frame before the call to
    select a specific track when the provider supports it
参数:

  • provider – [in] Provider handle

  • out_frame – [inout] Input track selector, output acquired frame

  • timeout_ms – [in] Timeout in milliseconds; 0 means no wait, UINT32_MAX means wait forever

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_frame is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_read_frame(const esp_media_provider_t *provider, esp_media_frame_t *out_frame, uint32_t timeout_ms)

Read the next frame into caller-provided storage. 

    Set out_frame->data and out_frame->size before calling. On success,
    out_frame->size is updated to the actual payload size
参数:

  • provider – [in] Provider handle

  • out_frame – [inout] Input buffer descriptor, output frame descriptor

  • timeout_ms – [in] Timeout in milliseconds; 0 means no wait, UINT32_MAX means wait forever

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or out_frame is NULL

  • ESP_ERR_INVALID_SIZE Provided buffer is too small

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

esp_err_t esp_media_provider_release_frame(const esp_media_provider_t *provider, esp_media_frame_t *frame)

Release a frame acquired from a provider.

参数:

  • provider – [in] Provider handle

  • frame – [in] Frame returned by esp_media_provider_acquire_frame()

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG provider or frame is NULL

  • ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

  • Others Error returned by the provider implementation

Header File

Functions

esp_err_t esp_media_track_mngr_create(const esp_media_track_mngr_cfg_t *cfg, esp_media_track_mngr_t **out_mngr)

Create a media track manager.

参数:

  • cfg – [in] Provider configuration

  • out_mngr – [out] Output track manager handle

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG cfg is NULL, max_track_num is 0, or out_provider is NULL

  • ESP_ERR_NO_MEM Allocation failed

esp_err_t esp_media_track_mngr_destroy(esp_media_track_mngr_t *mngr)

Destroy a media track manager. 

    Wakes and destroys internal queues, releases pending user-owned
    frames through their release callbacks, and frees the track manager
参数:

mngr – [in] Track manager handle returned by esp_media_track_mngr_create()

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

esp_err_t esp_media_track_mngr_reset(esp_media_track_mngr_t *mngr)

Reset tracks and queued data. 

    Removes all tracks and clears abort state. Global-cache mode is kept.
    User must re-add tracks after reset
参数:

mngr – [in] Track manager handle

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

esp_err_t esp_media_track_mngr_set_global_cache(esp_media_track_mngr_t *mngr, bool enable, size_t cache_size)

Configure global cache mode (may be called after tracks exist, e.g. on re-link) 

    When mode and cache size are unchanged, queues are drained, abort is cleared,
    and in-flight read/write nodes are dropped so a stopped manager can restart.
    When mode or global cache size changes, old queues are destroyed first
    then recreated (avoids holding two large caches at peak). If recreate
    fails, abort is set so blocked readers/writers wake with invalid state.
参数:

  • mngr – [in] Track manager handle

  • enable – [in] true to use one arrival-order cache shared by all tracks

  • cache_size – [in] Shared queue byte size when enable is true; 0 uses default

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

  • ESP_ERR_NO_MEM Allocation failed

esp_err_t esp_media_track_mngr_add_track(esp_media_track_mngr_t *mngr, const esp_media_track_mngr_track_cfg_t *cfg)

Add a track to a track manager.

参数:

  • mngr – [in] Track manager handle

  • cfg – [in] Track configuration

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager or cfg is NULL

  • ESP_ERR_NO_MEM Track limit reached or queue allocation failed

esp_err_t esp_media_track_mngr_update_track(esp_media_track_mngr_t *mngr, uint16_t index, const esp_media_track_info_t *info)

Queue a track metadata update. 

    The update is committed when provider acquire/read reaches the queued
    zero-size frame with ESP_MEDIA_FRAME_FLAG_TRACK_CHANGED. The provider
    event callback is invoked synchronously at that point.
    If typical metadata (codec, layout) matches the existing track, this
    is a no-op. Bitrate and unused union padding are ignored.
参数:

  • mngr – [in] Track manager handle

  • index – [in] Track index to update

  • info – [in] New track metadata

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager or info is NULL

  • ESP_ERR_NOT_FOUND index is out of range

  • ESP_ERR_NO_MEM Track queue is unavailable

  • ESP_ERR_TIMEOUT Failed to queue update frame

  • ESP_FAIL Failed to commit update frame

esp_err_t esp_media_track_mngr_remove_track(esp_media_track_mngr_t *mngr, uint16_t index)

Queue a track removal notification. 

    The removal is committed when provider acquire/read reaches the queued
    zero-size frame with ESP_MEDIA_FRAME_FLAG_TRACK_REMOVED. The provider
    event callback is invoked synchronously at that point
参数:

  • mngr – [in] Track manager handle

  • index – [in] Track index to remove

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager is NULL

  • ESP_ERR_NOT_FOUND index is out of range

  • ESP_ERR_NO_MEM Track queue is unavailable

  • ESP_ERR_TIMEOUT Failed to queue removal frame

  • ESP_FAIL Failed to commit removal frame

esp_err_t esp_media_track_mngr_get_provider(esp_media_track_mngr_t *mngr, esp_media_provider_t *provider)

Get the media provider handle exported by a track manager.

参数:

  • mngr – [in] Track manager handle

  • provider – [out] Output media provider

返回:

  • ESP_OK On success

  • ESP_ERR_INVALID_ARG Track manager or provider is NULL

Structures

struct esp_media_track_mngr_cfg_t

Media track manager configuration.

备注

The global cache used or not can be reconfigured by API esp_media_track_mngr_set_global_cache

Public Members

uint16_t max_track_num

Maximum number of tracks

bool use_global_cache

Use one arrival-order cache shared by all tracks

size_t cache_size

Shared queue byte size, 0 uses default

struct esp_media_track_mngr_cache_cfg_t

Media provider track buffer configuration.

Public Members

esp_media_track_cache_type_t cache_type

Track cache type

size_t cache_size

Queue byte size for track manager-owned payload mode

uint16_t addr_align

Track manager-owned frame data alignment, 0 uses pointer size

uint16_t size_align

Cached frame size alignment, 0 no special request

uint16_t queue_num

Metadata queue depth

esp_media_frame_release_cb_t frame_release

Release callback for user-owned frames

void *release_ctx

Context for frame_release

struct esp_media_track_mngr_track_cfg_t

Media track manager track configuration.

Public Members

esp_media_track_info_t info

Track metadata

esp_media_track_mngr_cache_cfg_t cache_cfg

Track buffer/cache settings

Type Definitions

typedef struct esp_media_track_mngr esp_media_track_mngr_t

Definition of media track manager.

SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

See LICENSE file for details.

typedef void (*esp_media_frame_release_cb_t)(const esp_media_frame_t *frame, void *ctx)

Release callback for user-owned frames after consumed.

Param frame:

[in] Frame whose payload can be released by the owner

Param ctx:

[in] User context from esp_media_track_mngr_cache_cfg_t

Enumerations

enum esp_media_track_cache_type_t

Media track cache type.

Values:

enumerator ESP_MEDIA_TRACK_CACHE_INTERNAL

Track manager owns queued frame payload data

enumerator ESP_MEDIA_TRACK_CACHE_USER

Track manager queues frame metadata only, user owns payload data

提供有关此文档的反馈