# ESP Media Service

[English]

## 简介

ESP Media Service 为 ESP-ADF 中的音视频服务定义了一套通用媒体接口，以 ESP Service 为基类。应用程序创建服务、配置媒体流，再把源流链接到接收端流。链接并启动后，媒体帧通过读取接口与写入接口自动流动，应用程序不必手动转发帧数据。音视频采集与播放见 ESP-GMF 通用多媒体框架。

## 功能清单

- 统一的音视频服务模型，源、接收端、源接收一体三种角色（`ESP_MEDIA_ROLE_SRC` / `SINK` / `SRC_SINK`）

- 基于 stream ID（`esp_media_stream_id_t`）的多媒体端点，一个服务可以同时暴露多个 stream

- 链接时的请求协商：接收端可以通过 `esp_media_service_request_t` 向源提出诉求（如是否需要 global cache）

- Provider 读取接口（`esp_media_provider_t`）与 track manager 写入接口相互解耦

- 内置默认的内存 track manager（`esp_media_track_mngr_t`），支持按 track 独立缓存或全局到达顺序缓存

- 统一通过 ESP Service 管理服务生命周期，媒体相关操作作为独立的 vtable（`esp_media_service_ops_t`）叠加在基类之上

## 技术拆解

### 服务角色与链接

ESP Media Service 通过 `esp_media_service_ops_t` 中的 `get_role` 声明自己是源、接收端还是二者兼具；`esp_media_service_link()` 在链接时校验所选源和接收端的角色是否兼容，再把源的 provider 传给接收端。

            flowchart TD
        Create["创建服务"] --> Config["配置服务"]
        Config --> Link["链接源/接收端 stream"]
        Link --> Start["启动服务"]
        Start --> Flow["媒体帧流动"]
        Flow --> Stop["停止服务"]
        Stop --> Unlink["取消链接并销毁"]

    esp_media_stream_id_t stream = ESP_MEDIA_DEFAULT_STREAM;

    esp_media_service_link(src_service, stream, sink_service, stream);

    esp_service_start(sink_service);
    esp_service_start(src_service);

链接建立后，媒体数据通过源服务导出的 provider 流向接收端；取消链接需要调用 `esp_media_service_unlink()`，会清除接收端 stream 上当前设置的 provider。

### Provider 与 Track Manager

媒体数据的传递依赖两个互相解耦的接口：接收端在链接时从源服务获取只读的 `esp_media_provider_t`，通过 `esp_media_provider_acquire_frame()` / `esp_media_provider_release_frame()` 获取和释放帧；源服务侧则持有一个 `esp_media_track_mngr_t`，通过写入 API 产生帧，并把它导出的 provider 句柄暴露给下游。

    /* 源服务侧：创建 track manager，注册 track，导出 provider */
    esp_media_track_mngr_cfg_t cfg = { .max_track_num = 2 };
    esp_media_track_mngr_create(&cfg, &svc->mngr);
    esp_media_track_mngr_add_track(svc->mngr, &audio_track);
    esp_media_track_mngr_get_provider(svc->mngr, &svc->provider);

    /* 接收端侧：读取并释放帧 */
    esp_media_frame_t frame = {0};
    if (esp_media_provider_acquire_frame(&sink->provider, &frame, timeout_ms) == ESP_OK) {
        process_frame(&frame);
        esp_media_provider_release_frame(&sink->provider, &frame);
    }

警告

通过 `esp_media_provider_acquire_frame()` 获取的帧必须调用 `esp_media_provider_release_frame()` 释放，释放之后不能再访问 `frame.data`。

### Track Manager 的缓存模式

默认的 `esp_media_track_mngr_t` 支持两种 payload 所有权模式：`ESP_MEDIA_TRACK_CACHE_INTERNAL` 由 manager 复制并持有帧数据；`ESP_MEDIA_TRACK_CACHE_USER` 只缓存帧元数据，payload 仍由用户持有，消费后通过 `frame_release` 回调归还。此外还支持 global cache，让多个 track 共用一个按到达顺序排列的队列，适合 RTMP 等音视频交错传输场景；启用 global cache 需要在添加 track 之前完成配置。

### 停止与中止顺序

媒体接口对停止顺序做了容错设计：源服务停止时应调用 `esp_media_track_write_abort()`，通过 `ESP_MEDIA_PROVIDER_EVENT_TRACKS_ABORT` 事件通知下游；接收端停止时应先置本地停止标志，再调用 `esp_media_provider_abort()` 唤醒阻塞的读取，等待任务退出、释放已获取的帧，最后才取消链接。若 track manager 被多个服务通过链接共享，需要先取消链接，再对它执行 reset 或 destroy；只要还有任务持有已获取的帧或阻塞在队列上，就不能对 track manager 执行 reset 或 destroy。

## 应用示例

源/接收端的完整示例见 esp_media_service 组件仓库 的 examples 目录。服务基类用法见 ESP Service。

## FAQ

Q1：一个服务只能实现 ``get_provider`` 或 ``set_provider`` 中的一个吗？

不是必须的。角色为 `ESP_MEDIA_ROLE_SRC_SINK` 的服务可以同时实现两者，既作为下游的源，也作为上游的接收端；`esp_media_service_link()` 只按角色位校验所选的一对源/接收端服务是否兼容。

## API 参考

### Header File

- components/esp_media_service/include/esp_media_service.h

### Functions

esp_err_t esp_media_service_init(esp_media_service_t *service, const esp_media_service_config_t *config)

  Initialize a media service base object.

          Derived services call this after allocating or embedding
          esp_media_service_t, before exposing the base esp_service_t

  参数:

    - service – [inout] Media service object to initialize

    - config – [in] Media service configuration

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG service or config is NULL

    - Others Error returned by esp_service_init()

esp_err_t esp_media_service_deinit(esp_service_t *service)

  Deinitialize a media service base object.

  参数:

    service – [in] Base service pointer returned by ESP_SERVICE_BASE()

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG service is NULL

    - Others Error returned by esp_service_deinit()

esp_err_t esp_media_service_get_role(esp_service_t *service, esp_media_role_t *out_role)

  Query the media role of a service.

  参数:

    - service – [in] Base media service handle

    - out_role – [out] Output media role

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG service or out_role is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

    - Others Error returned by the service implementation

esp_err_t esp_media_service_get_provider(esp_service_t *service, esp_media_stream_id_t stream, esp_media_provider_t *out_provider)

  Get a provider from a source stream.

  参数:

    - service – [in] Base media service handle

    - stream – [in] Source stream ID

    - out_provider – [out] Output provider

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Service or provider is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

    - Others Error returned by the service implementation

esp_err_t esp_media_service_set_provider(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_provider_t *provider)

  Set a provider on a sink stream.

          Pass NULL provider to disconnect the sink stream

  参数:

    - service – [in] Base media service handle

    - stream – [in] Sink stream ID

    - provider – [in] Provider handle to consume, or NULL to clear

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG service is NULL, or provider has no ops

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

    - Others Error returned by the service implementation

esp_err_t esp_media_service_get_request(esp_service_t *service, esp_media_stream_id_t stream, esp_media_service_request_t *out_request)

  Get link request hints for a media stream.

  参数:

    - service – [in] Base media service handle

    - stream – [in] Stream ID

    - out_request – [out] Output request hints

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG service or out_request is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

    - Others Error returned by the service implementation

esp_err_t esp_media_service_set_request(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_service_request_t *request)

  Set link request hints for a media stream.

  参数:

    - service – [in] Base media service handle

    - stream – [in] Stream ID

    - request – [in] Request hints to apply

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG service or request is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the service

    - Others Error returned by the service implementation

esp_err_t esp_media_service_link(esp_service_t *src_service, esp_media_stream_id_t src_stream, esp_service_t *sink_service, esp_media_stream_id_t sink_stream)

  Link a source stream to a sink stream.

          The sink request hints are applied to the source when supported, then
          the source provider is passed to the sink

  参数:

    - src_service – [in] Source media service handle

    - src_stream – [in] Source stream ID

    - sink_service – [in] Sink media service handle

    - sink_stream – [in] Sink stream ID

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG src_service or sink_service is NULL

    - ESP_ERR_NOT_SUPPORTED Role check failed or required operation is missing

    - Others Error returned by source or sink implementation

esp_err_t esp_media_service_unlink(esp_service_t *src_service, esp_media_stream_id_t src_stream, esp_service_t *sink_service, esp_media_stream_id_t sink_stream)

  Unlink a source stream from a sink stream.

          Clears the provider currently set on the sink stream

  参数:

    - src_service – [in] Source media service handle

    - src_stream – [in] Source stream ID

    - sink_service – [in] Sink media service handle

    - sink_stream – [in] Sink stream ID

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG src_service or sink_service is NULL

    - ESP_ERR_NOT_SUPPORTED Role check failed or sink operation is missing

    - Others Error returned by source or sink implementation

### Structures

struct esp_media_service_request_t

  Media-service link request/capability hints.

          The structure is intentionally extensible so future services can add
          link-time requests without changing the media op shape

  Public Members

  bool need_global_cache

    Sink requests source frames in one arrival-order cache

struct esp_media_service_ops_t

  Media-service virtual operations.

  Public Members

  esp_err_t (*get_role)(esp_service_t *service, esp_media_role_t *out_role)

    Query source/sink role

  esp_err_t (*get_provider)(esp_service_t *service, esp_media_stream_id_t stream, esp_media_provider_t *out_provider)

    Get provider exported by a source stream

  esp_err_t (*set_provider)(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_provider_t *provider)

    Set provider consumed by a sink stream

  esp_err_t (*get_request)(esp_service_t *service, esp_media_stream_id_t stream, esp_media_service_request_t *request)

    Get sink link requests

  esp_err_t (*set_request)(esp_service_t *service, esp_media_stream_id_t stream, const esp_media_service_request_t *request)

    Apply sink requests to a source

struct esp_media_service_config_t

  Media service configuration.

  Public Members

  const char *name

    Service instance name

  void *user_data

    User data passed to esp_service

  const esp_service_ops_t *service_ops

    Optional esp_service lifecycle ops

  const esp_media_service_ops_t *media_ops

    Optional media ops

struct esp_media_service

  Base media service structure Derived services embed this as the first member.

  Public Members

  esp_service_t base

    Base service, must stay first

  const esp_media_service_ops_t *media_ops

    Media virtual operations

### Macros

ESP_MEDIA_SERVICE_CONFIG_DEFAULT()

  SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

  See LICENSE file for details. Default media service configuration

ESP_MEDIA_DEFAULT_STREAM

  Default media stream ID

### Type Definitions

typedef uint16_t esp_media_stream_id_t

  Instance-aware media stream address.

typedef struct esp_media_service esp_media_service_t

  Base media service structure Derived services embed this as the first member.

### Enumerations

enum esp_media_role_t

  Media service data direction.

  Values:

  enumerator ESP_MEDIA_ROLE_NONE
  enumerator ESP_MEDIA_ROLE_SRC

    Service produces media frames

  enumerator ESP_MEDIA_ROLE_SINK

    Service consumes media frames

  enumerator ESP_MEDIA_ROLE_SRC_SINK

    Service produces and consumes frames

### Header File

- components/esp_media_service/include/esp_media_provider.h

### Functions

esp_err_t esp_media_provider_get_track_num(const esp_media_provider_t *provider, uint16_t *out_num)

  Get the number of tracks exposed by a provider.

  SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

  See LICENSE file for details.

  参数:

    - provider – [in] Provider handle

    - out_num – [out] Output track count

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG provider or out_num is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

    - Others Error returned by the provider implementation

esp_err_t esp_media_provider_get_track_info(const esp_media_provider_t *provider, uint16_t index, esp_media_track_info_t *out_info)

  Get track metadata by provider track index.

  参数:

    - provider – [in] Provider handle

    - index – [in] Track index in the provider

    - out_info – [out] Output track metadata

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG provider or out_info is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

    - Others Error returned by the provider implementation

esp_err_t esp_media_provider_set_event_cb(const esp_media_provider_t *provider, esp_media_provider_event_cb_t cb, void *event_ctx)

  Register a provider event callback.

  参数:

    - provider – [in] Provider handle

    - cb – [in] Event callback, or NULL to clear it

    - event_ctx – [in] User context passed to cb

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG provider is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

    - Others Error returned by the provider implementation

esp_err_t esp_media_provider_abort(const esp_media_provider_t *provider)

  Abort provider-side blocking read operations.

  参数:

    provider – [in] Provider handle

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG provider is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

    - Others Error returned by the provider implementation

esp_err_t esp_media_provider_acquire_frame(const esp_media_provider_t *provider, esp_media_frame_t *out_frame, uint32_t timeout_ms)

  Acquire the next frame from a provider.

          The returned frame must be released with esp_media_provider_release_frame().
          Set fields such as type or track_id in out_frame before the call to
          select a specific track when the provider supports it

  参数:

    - provider – [in] Provider handle

    - out_frame – [inout] Input track selector, output acquired frame

    - timeout_ms – [in] Timeout in milliseconds; 0 means no wait, UINT32_MAX means wait forever

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG provider or out_frame is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

    - Others Error returned by the provider implementation

esp_err_t esp_media_provider_read_frame(const esp_media_provider_t *provider, esp_media_frame_t *out_frame, uint32_t timeout_ms)

  Read the next frame into caller-provided storage.

          Set out_frame->data and out_frame->size before calling. On success,
          out_frame->size is updated to the actual payload size

  参数:

    - provider – [in] Provider handle

    - out_frame – [inout] Input buffer descriptor, output frame descriptor

    - timeout_ms – [in] Timeout in milliseconds; 0 means no wait, UINT32_MAX means wait forever

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG provider or out_frame is NULL

    - ESP_ERR_INVALID_SIZE Provided buffer is too small

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

    - Others Error returned by the provider implementation

esp_err_t esp_media_provider_release_frame(const esp_media_provider_t *provider, esp_media_frame_t *frame)

  Release a frame acquired from a provider.

  参数:

    - provider – [in] Provider handle

    - frame – [in] Frame returned by esp_media_provider_acquire_frame()

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG provider or frame is NULL

    - ESP_ERR_NOT_SUPPORTED Operation is not implemented by the provider

    - Others Error returned by the provider implementation

### Header File

- components/esp_media_service/include/esp_media_track_mngr.h

### Functions

esp_err_t esp_media_track_mngr_create(const esp_media_track_mngr_cfg_t *cfg, esp_media_track_mngr_t **out_mngr)

  Create a media track manager.

  参数:

    - cfg – [in] Provider configuration

    - out_mngr – [out] Output track manager handle

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG cfg is NULL, max_track_num is 0, or out_provider is NULL

    - ESP_ERR_NO_MEM Allocation failed

esp_err_t esp_media_track_mngr_destroy(esp_media_track_mngr_t *mngr)

  Destroy a media track manager.

          Wakes and destroys internal queues, releases pending user-owned
          frames through their release callbacks, and frees the track manager

  参数:

    mngr – [in] Track manager handle returned by esp_media_track_mngr_create()

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Track manager is NULL

esp_err_t esp_media_track_mngr_reset(esp_media_track_mngr_t *mngr)

  Reset tracks and queued data.

          Removes all tracks and clears abort state. Global-cache mode is kept.
          User must re-add tracks after reset

  参数:

    mngr – [in] Track manager handle

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Track manager is NULL

esp_err_t esp_media_track_mngr_set_global_cache(esp_media_track_mngr_t *mngr, bool enable, size_t cache_size)

  Configure global cache mode before tracks are added.

  参数:

    - mngr – [in] Track manager handle

    - enable – [in] true to use one arrival-order cache shared by all tracks

    - cache_size – [in] Shared queue byte size, 0 uses default

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Track manager is NULL

    - ESP_ERR_INVALID_STATE Tracks have already been added

    - ESP_ERR_NO_MEM Allocation failed

esp_err_t esp_media_track_mngr_add_track(esp_media_track_mngr_t *mngr, const esp_media_track_mngr_track_cfg_t *cfg)

  Add a track to a track manager.

  参数:

    - mngr – [in] Track manager handle

    - cfg – [in] Track configuration

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Track manager or cfg is NULL

    - ESP_ERR_NO_MEM Track limit reached or queue allocation failed

esp_err_t esp_media_track_mngr_update_track(esp_media_track_mngr_t *mngr, uint16_t index, const esp_media_track_info_t *info)

  Queue a track metadata update.

          The update is committed when provider acquire/read reaches the queued
          zero-size frame with ESP_MEDIA_FRAME_FLAG_TRACK_CHANGED. The provider
          event callback is invoked synchronously at that point

  参数:

    - mngr – [in] Track manager handle

    - index – [in] Track index to update

    - info – [in] New track metadata

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Track manager or info is NULL

    - ESP_ERR_NOT_FOUND index is out of range

    - ESP_ERR_NO_MEM Track queue is unavailable

    - ESP_ERR_TIMEOUT Failed to queue update frame

    - ESP_FAIL Failed to commit update frame

esp_err_t esp_media_track_mngr_remove_track(esp_media_track_mngr_t *mngr, uint16_t index)

  Queue a track removal notification.

          The removal is committed when provider acquire/read reaches the queued
          zero-size frame with ESP_MEDIA_FRAME_FLAG_TRACK_REMOVED. The provider
          event callback is invoked synchronously at that point

  参数:

    - mngr – [in] Track manager handle

    - index – [in] Track index to remove

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Track manager is NULL

    - ESP_ERR_NOT_FOUND index is out of range

    - ESP_ERR_NO_MEM Track queue is unavailable

    - ESP_ERR_TIMEOUT Failed to queue removal frame

    - ESP_FAIL Failed to commit removal frame

esp_err_t esp_media_track_mngr_get_provider(esp_media_track_mngr_t *mngr, esp_media_provider_t *provider)

  Get the media provider handle exported by a track manager.

  参数:

    - mngr – [in] Track manager handle

    - provider – [out] Output media provider

  返回:

    - ESP_OK On success

    - ESP_ERR_INVALID_ARG Track manager or provider is NULL

### Structures

struct esp_media_track_mngr_cfg_t

  Media track manager configuration.

  备注

  The global cache used or not can be reconfigured by API `esp_media_track_mngr_set_global_cache`

  Public Members

  uint16_t max_track_num

    Maximum number of tracks

  bool use_global_cache

    Use one arrival-order cache shared by all tracks

  size_t cache_size

    Shared queue byte size, 0 uses default

struct esp_media_track_mngr_cache_cfg_t

  Media provider track buffer configuration.

  Public Members

  esp_media_track_cache_type_t cache_type

    Track cache type

  size_t cache_size

    Queue byte size for track manager-owned payload mode

  uint16_t addr_align

    Track manager-owned frame data alignment, 0 uses pointer size

  uint16_t size_align

    Cached frame size alignment, 0 no special request

  uint16_t queue_num

    Metadata queue depth

  esp_media_frame_release_cb_t frame_release

    Release callback for user-owned frames

  void *release_ctx

    Context for frame_release

struct esp_media_track_mngr_track_cfg_t

  Media track manager track configuration.

  Public Members

  esp_media_track_info_t info

    Track metadata

  esp_media_track_mngr_cache_cfg_t cache_cfg

    Track buffer/cache settings

### Type Definitions

typedef struct esp_media_track_mngr esp_media_track_mngr_t

  Definition of media track manager.

  SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO., LTD SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT

  See LICENSE file for details.

typedef void (*esp_media_frame_release_cb_t)(const esp_media_frame_t *frame, void *ctx)

  Release callback for user-owned frames after consumed.

  Param frame:

    [in] Frame whose payload can be released by the owner

  Param ctx:

    [in] User context from esp_media_track_mngr_cache_cfg_t

### Enumerations

enum esp_media_track_cache_type_t

  Media track cache type.

  Values:

  enumerator ESP_MEDIA_TRACK_CACHE_INTERNAL

    Track manager owns queued frame payload data

  enumerator ESP_MEDIA_TRACK_CACHE_USER

    Track manager queues frame metadata only, user owns payload data

提供有关此文档的反馈