# GMF-Video

[English]

gmf_video 是 ESP-GMF 的视频处理组件，提供 9 个处理单元（element）覆盖编解码、像素加速器、软件图像效果与帧率 / 叠加工具。所有处理单元都继承自 `esp_gmf_video_element_t`，对外暴露统一的生命周期功能函数、acquire-release 数据接口和运行时方法（method）调用；底层算法分别由 ESP32-P4 的硬件 PPA / 2D-DMA、`esp_video_codec` 编解码库与 `esp_image_effects` 软件图像效果库实现。本篇按四类展开每个处理单元的用途、配置与运行时控制；处理单元基类与运行时方法机制见 GMF 处理单元，数据通路见 数据流转。

## 功能清单

- vid_dec：视频解码，支持 H.264 与 MJPEG，可指定输出像素格式（如 YUV420P、RGB565LE）

- vid_enc：视频编码，支持 H.264 与 MJPEG，可运行中调整 bitrate、GOP、QP 范围（仅 H.264）

- vid_ppa：ESP32-P4 像素加速器复合处理单元，把颜色转换、缩放、裁剪、旋转合并为一次硬件处理

- vid_fps_cvt：帧率转换，按 PTS 丢帧把输入帧率降到指定输出帧率

- vid_overlay：叠加混合器，把额外画面（水印、UI、时间戳）按 alpha 混合或透明色（colorkey）叠加到原始视频

- vid_color_cvt：软件颜色转换，覆盖 YUYV、RGB565、RGB888、YUV420P 等常见格式互转

- vid_crop：软件裁剪，从原始帧中提取指定矩形区域

- vid_scale：软件缩放，按目标分辨率重采样视频帧

- vid_rotate：软件旋转，支持任意角度（单位为度）

- 共性：旁路（bypass）优化、自动对齐查询、`need_recfg` 触发的运行时重配置、PTS 透传与丢帧逻辑

## 技术拆解

### 元素层次与共性机制

每个具体处理单元都按”包装一个底层视频算法 / 硬件句柄”的方式构造：派生结构以 `esp_gmf_video_element_t` 为首字段，绑定 codec / PPA / imgfx handle 与本处理单元的配置；`open` 创建底层 handle 并向处理链（pipeline）上报输出 `esp_gmf_info_video_t`（宽、高、帧率、像素格式 FourCC）；`process` 在每次调度时从输入数据端口（port）取帧、执行转换并写出输出数据端口；`close` 释放硬件资源。

            classDiagram
        direction TD

        class esp_gmf_video_element_t
        class Codec {
            vid_dec
            vid_enc
        }
        class HwAccel {
            vid_ppa
        }
        class SwEffects {
            vid_color_cvt
            vid_crop
            vid_scale
            vid_rotate
        }
        class Utility {
            vid_fps_cvt
            vid_overlay
        }

        esp_gmf_video_element_t <|-- Codec
        esp_gmf_video_element_t <|-- HwAccel
        esp_gmf_video_element_t <|-- SwEffects
        esp_gmf_video_element_t <|-- Utility

四条公共机制：

- 旁路优化：vid_dec 在源 / 目标格式一致时进入零拷贝直通；软件效果处理单元在源 / 目标参数一致时同样置旁路，输入数据载体（payload）直接透传输出数据端口。

- need_recfg 重配置：所有 setter（`set_dst_format` / `set_dst_resolution` / `set_angle` 等）改动后置 `need_recfg` 标志，下一次 `process` 边界销毁旧 handle、重建新 handle，运行中切换分辨率或像素格式不需要应用层重建处理单元。

- 缓冲对齐自动查询：底层硬件（PPA、H.264 编码器）通常对输入 / 输出缓冲区有对齐要求，处理单元在 `open` 阶段调用硬件驱动查询对齐参数，写入数据端口 attr 让框架按对齐分配数据载体，避免硬件直接拒绝处理。

- 格式信息上报：上游处理单元解析到当前帧的宽高 / 像素格式后通过 `REPORT_INFO` 事件上报，处理链把信息投递给下游处理单元，下游据此 open 或触发重配置。机制详情见 GMF 处理链与任务调度 的 REPORT_INFO 说明。

### 视频编解码

vid_dec 与 vid_enc 是视频处理链的起点或终点处理单元，典型组装如下。

            flowchart LR
        subgraph rec ["录制链"]
            direction LR
            CapIn(("camera/raw")) --> CC1["vid_color_cvt"]
            CC1 --> Scale["vid_scale"]
            Scale --> Enc["vid_enc"]
            Enc --> FileOut(("file/http"))
        end
        subgraph play ["播放链"]
            direction LR
            FileIn(("file/http")) --> Dec["vid_dec"]
            Dec --> PPA["vid_ppa"]
            PPA --> Disp(("display"))
        end

vid_dec：把压缩流解码成原始像素。`esp_gmf_video_dec_init()` 接受 `esp_gmf_video_dec_cfg_t`，`codec_cc` 字段用 FourCC 指定解码器实现；为零时框架按可用硬件 / 软件解码器自动选择。

通过 `esp_gmf_video_dec_get_dst_formats()` 查询指定源编码（H.264 / MJPEG 等）对应的可输出像素格式列表，再用 `esp_gmf_video_dec_set_dst_format()` 在处理单元启动前固定输出格式。如果源格式与目标格式一致（罕见但有用，例如纯转封装链），vid_dec 自动进入旁路模式直接转发数据载体。

    esp_gmf_video_dec_cfg_t cfg = { .codec_cc = 0 };  /* 自动选 */
    esp_gmf_element_handle_t dec = NULL;
    esp_gmf_video_dec_init(&cfg, &dec);

    const uint32_t *fmts = NULL;
    uint8_t fmt_num = 0;
    esp_gmf_video_dec_get_dst_formats(dec, ESP_FOURCC_H264, &fmts, &fmt_num);
    esp_gmf_video_dec_set_dst_format(dec, fmts[0]);

vid_enc：把原始像素编码成压缩流。`esp_gmf_video_enc_init()` 接受 `esp_gmf_video_enc_cfg_t`，`codec_cc` 指定编码器实现。

正常工作流是处理单元启动后从上游获取源信息（`esp_gmf_info_video_t`），按 `set_dst_codec` 指定的目标 codec 自动选具体实现并 open。如果要在启动前查询编码器能力（例如查支持的源像素格式），先用 `esp_gmf_video_enc_preset()` 传入源信息与目标 codec，再调用 `esp_gmf_video_enc_get_src_formats()` 与 `esp_gmf_video_enc_get_out_size()`。

运行中可调三个参数（H.264 编码器额外支持后两个）：

    esp_gmf_video_enc_set_bitrate(enc, 2 * 1000 * 1000);  /* 2 Mbps */
    esp_gmf_video_enc_set_gop(enc, 30);                    /* 每 30 帧一个 I 帧 */
    esp_gmf_video_enc_set_qp(enc, 20, 40);                 /* QP 区间 */

输出缓冲区大小按 `esp_gmf_video_enc_get_out_size()` 估算，MJPEG 按 10:1、H.264 按 2:1 的压缩比给上限，框架按此尺寸分配输出数据载体。

### 硬件加速 PPA

vid_ppa 仅支持 ESP32-P4/ESP32S31，把颜色转换、缩放、裁剪、旋转合并到一次硬件处理中。`esp_gmf_video_ppa_init()` 不需要传配置；四个 setter 分别配置目标格式、目标分辨率、裁剪区域、旋转角度（仅支持 0 / 90 / 180 / 270 度）。这些 setter 都只能在处理单元启动前调用，`open` 时一次性向硬件下发组合参数。

PPA 内部根据当前请求选择硬件路径：单纯的颜色格式转换可以由 2D-DMA 完成，吞吐更高；涉及缩放、旋转或 2D-DMA 不支持的格式组合时回退到 PPA 主路径。路径选择由框架内部完成，处理单元仅暴露统一接口。

    esp_gmf_element_handle_t ppa = NULL;
    esp_gmf_video_ppa_init(NULL, &ppa);

    esp_gmf_video_resolution_t dst_res = { .width = 480, .height = 320 };
    esp_gmf_video_ppa_set_dst_resolution(ppa, &dst_res);
    esp_gmf_video_ppa_set_dst_format(ppa, ESP_FOURCC_RGB16);
    esp_gmf_video_ppa_set_rotation(ppa, 90);

    esp_gmf_video_rgn_t crop = { .x = 0, .y = 0, .width = 800, .height = 600 };
    esp_gmf_video_ppa_set_cropped_rgn(ppa, &crop);

非 ESP32-P4 芯片上 vid_ppa 不可用，需要等价功能时用下文 4 个软件效果处理单元串接（vid_color_cvt → vid_crop → vid_scale → vid_rotate），代价是 CPU 占用上升。

### 软件图像效果

四个软件效果处理单元包装 `esp_image_effects` 库的对应算法，对外只暴露各自的 setter。所有效果都遵循同一条路径：`acquire_in` 取输入帧、按配置生成输出帧大小、`acquire_out` 取输出帧、调用 `esp_imgfx_*_process` 处理、双双 release。

vid_color_cvt：像素格式转换，例如 RGB565 → YUV420P、YUYV → RGB888。颜色空间标准（BT601 / BT709）通过初始化时的 `esp_imgfx_color_convert_cfg_t.color_space_std` 字段选择。`esp_gmf_video_color_convert_dst_format()` 在运行中切换目标格式，置 `need_recfg`。

vid_crop：裁剪元素。初始化用 `DEFAULT_ESP_GMF_CROP_CONFIG` 给出默认 320×240 输入、160×120 裁剪区。运行中用 `esp_gmf_video_crop_rgn()` 切换裁剪矩形（`esp_gmf_video_rgn_t` 给出 x / y / width / height）。

vid_scale：缩放元素。`filter_type` 字段决定算法（如 `ESP_IMGFX_SCALE_FILTER_TYPE_DOWN_RESAMPLE`）。运行中用 `esp_gmf_video_scale_dst_resolution()` 修改目标分辨率。

vid_rotate：旋转元素。与 vid_ppa 只支持四个角度不同，软件旋转支持任意角度（单位为度），适合需要倾斜显示或图像校正的场景。`esp_gmf_video_rotate_set_rotation()` 在运行中切换角度。

    esp_imgfx_scale_cfg_t cfg = DEFAULT_ESP_GMF_SCALE_CONFIG();
    cfg.in_res       = (esp_imgfx_res_t){ .width = 640, .height = 480 };
    cfg.in_pixel_fmt = ESP_IMGFX_PIXEL_FMT_RGB565_LE;
    cfg.scale_res    = (esp_imgfx_res_t){ .width = 320, .height = 240 };

    esp_gmf_element_handle_t scale = NULL;
    esp_gmf_video_scale_init(&cfg, &scale);

### 帧率与叠加

vid_fps_cvt：通过丢帧把输入帧率降低到指定输出帧率。处理单元维护起始 PTS 与累计已输出帧数，按 `1 / fps` 步长计算 `expected_pts`。每一帧到来时：当前帧 PTS 大于等于 `expected_pts` 才透传到下游并累计计数；否则直接 `release_in` 丢弃该帧。

由于纯丢帧不能升采样，`set_fps` 设定值必须低于上游帧率，否则等于无效。

    esp_gmf_element_handle_t fps = NULL;
    esp_gmf_video_fps_cvt_init(NULL, &fps);
    esp_gmf_video_fps_cvt_set_fps(fps, 15);  /* 从 30 fps 降到 15 fps */

vid_overlay：把额外画面叠加到主视频上。处理单元是多输入：主视频从默认 in 数据端口进入，叠加层从用户通过 `esp_gmf_video_overlay_set_overlay_port()` 注册的额外数据端口进入。合成支持 alpha 混合与透明色（colorkey）两种模式。alpha 混合公式：

    Output = Original × (255 − alpha) + Overlay × alpha

`set_rgn` 指定叠加层在主画面上的位置与格式（`esp_gmf_overlay_rgn_info_t` 含 `format_id`、`dst_rgn`，以及可选的 `has_trans_color` / `trans_color` 用于 colorkey 合成，与指定 RGB 接近的像素视为透明）；`set_alpha` 在运行中改透明度（0 全透明、255 全不透明）；`esp_gmf_video_overlay_enable()`(false) 临时禁用叠加，处理单元不再 acquire 叠加数据端口，主画面原样透传。

    esp_gmf_element_handle_t ovl = NULL;
    esp_gmf_video_overlay_init(NULL, &ovl);

    esp_gmf_video_overlay_set_overlay_port(ovl, overlay_port);
    esp_gmf_overlay_rgn_info_t rgn = {
        .format_id = ESP_FOURCC_RGB16,
        .dst_rgn   = { .x = 16, .y = 16, .width = 128, .height = 32 },
    };
    esp_gmf_video_overlay_set_rgn(ovl, &rgn);
    esp_gmf_video_overlay_set_alpha(ovl, 200);

## 性能

视频处理的吞吐主要由底层硬件 / 算法决定，处理单元调度与 acquire-release 的开销相对较小。按瓶颈分类：

类别 | 主要瓶颈 | 调优方向
--- | --- | ---
vid_dec / vid_enc | 硬件 codec 或软件库的吞吐 | ESP32-P4 支持 H.264 硬件编码；提前 `preset` 让框架预分配缓冲
vid_ppa | PPA / 2D-DMA 时钟与缓冲带宽 | 合并相邻像素操作到同一 vid_ppa 实例减少中间 buffer
软件效果 | CPU 与内存带宽 | 用 ESP32-P4 时优先 vid_ppa；其余 SoC 控制分辨率不要超过 SoC 处理能力
vid_fps_cvt | 仅做 PTS 比较与丢帧，CPU 开销极小 | 适合放在编码或显示前减少下游负载
vid_overlay | alpha 混合或 colorkey 合成的 CPU 周期 | 叠加区域越小、像素格式 byte 数越少越快

## 应用示例

- `elements/test_apps/main/elements/gmf_video_el_test.c`：典型 video 处理单元用例，包含 `gen_pattern_color_bar` 合成测试帧、open 阶段格式上报验证

- `elements/test_apps/main/elements/gmf_image_effects_test.c`：图像效果链测试，覆盖软件效果元素的串接

上层 `esp_video_render` 与 `esp_capture` 等应用组件也通过本组件连接视频处理链，可作进阶参考。

## SoC 兼容性

各处理单元在乐鑫 SoC 上的支持矩阵：

element | ESP32 | ESP32-S2 | ESP32-S3 | ESP32-P4
--- | --- | --- | --- | ---
vid_ppa | 不支持 | 不支持 | 不支持 | 支持
vid_fps_cvt | 支持 | 支持 | 支持 | 支持
vid_overlay | 支持 | 支持 | 支持 | 支持
vid_dec | MJPEG | MJPEG | SW H.264 / MJPEG | SW H.264 / HW MJPEG
vid_enc | MJPEG | MJPEG | SW H.264 / MJPEG | HW H.264 / HW MJPEG
vid_color_cvt / vid_crop / vid_scale / vid_rotate | 支持 | 支持 | 支持 | 支持

ESP32 与 ESP32-S2 不支持 H.264。ESP32-S3 的 H.264 编解码均通过软件实现；ESP32-P4 支持 H.264 硬件编码与 MJPEG 硬件编解码，H.264 解码通过软件实现。若需 ESP32-P4 的 PPA 加速，可省略软件效果处理单元，改用 vid_ppa。

## FAQ

Q： vid_dec 输出格式如何选择？

先用 `esp_gmf_video_dec_get_dst_formats()` 查询当前源编码可支持的输出格式列表，再选择符合下游处理单元输入要求的格式（例如 vid_ppa 需要 RGB565、显示需要 RGB888），并在处理单元启动前调用 `esp_gmf_video_dec_set_dst_format()` 固定输出格式。需要查询输出帧大小时，才要求先设置目标格式或在处理单元启动后调用对应查询接口。

Q： vid_enc 启动前查不到输出帧大小？

正常情况编码器要在获取上游 `esp_gmf_info_video_t` 后才能确定输出大小。若希望在启动前查询，先调 `esp_gmf_video_enc_preset()` 传入预期源信息与目标 codec，`esp_gmf_video_enc_get_out_size()` 才能返回值。

Q： vid_ppa 在 ESP32 上是否可用？

不支持。vid_ppa 依赖 ESP32-P4 的硬件 PPA / 2D-DMA。其他 SoC 上用 vid_color_cvt / vid_scale / vid_rotate / vid_crop 软件实现替代，分辨率与帧率需要按 CPU 实际能力收敛。

Q： vid_fps_cvt 设置的帧率高于上游会怎样？

处理单元不做插帧。若 `set_fps` 高于上游帧率，每一帧的 PTS 都大于等于 `expected_pts`，全部透传，输出帧率被上游限定。该配置仅在降低帧率时生效，设定值不得高于上游帧率。

Q： vid_overlay 的两个输入如何同步？

主输入是普通 GMF 输入数据端口，叠加输入是用户通过 `set_overlay_port` 注册的额外数据端口。处理单元每次 `process` 先 acquire 主输入再 acquire 叠加输入，应用方负责保证两路按帧对齐（例如同一个处理链内的两个上游处理单元同步产帧），否则可能出现叠加帧落后或抖动。

Q： 软件效果处理单元与 vid_ppa 在 ESP32-P4 上如何取舍？

需要做颜色转换 + 缩放 + 旋转 / 裁剪组合时优先用 vid_ppa：一次硬件处理完成全部转换，CPU 占用低、延迟稳。仅需单一操作或需要 PPA 不支持的角度 / 像素格式时用对应软件元素。

## API 参考

本组件涉及的头文件：

- `esp_gmf_video_dec.h` / `esp_gmf_video_enc.h`：视频编解码处理单元

- `esp_gmf_video_ppa.h`：像素加速器

- `esp_gmf_video_fps_cvt.h`：帧率转换

- `esp_gmf_video_overlay.h`：叠加混合器

- `esp_gmf_video_color_convert.h` / `esp_gmf_video_crop.h` / `esp_gmf_video_scale.h` / `esp_gmf_video_rotate.h`：软件图像效果

- `esp_gmf_video_types.h`：视频分辨率 / 区域 / 叠加配置等共享类型

- `esp_gmf_video_param.h`：视频参数辅助接口

处理单元基类 `esp_gmf_video_element_t` 的接口位于 GMF 处理单元。

### Header File

- elements/gmf_video/include/esp_gmf_video_dec.h

### Functions

esp_gmf_err_t esp_gmf_video_dec_init(esp_gmf_video_dec_cfg_t *cfg, esp_gmf_element_handle_t *handle)

  Initializes the GMF Video decoder with the provided configuration.

  参数:

    - cfg – [in] Pointer to the Video decoder configuration (optional)

    - handle – [out] Pointer to the Video decoder handle to be initialized

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_dec_get_out_size(esp_gmf_element_handle_t handle, uint32_t *frame_size)

  Get Video decoder output frame size.

  备注

  This API should only called after element running or `esp_gmf_video_dec_set_dst_format` is called

  参数:

    - handle – [in] Video decoder element handle

    - frame_size – [out] Output frame size to store

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_dec_set_dst_format(esp_gmf_element_handle_t handle, uint32_t dst_fmt)

  Setting for video decoder destination format (e.g., YUV420P, RGB565LE)

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video decoder element handle

    - dst_fmt – [in] Destination format (GMF FourCC representation of video codec)

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_dec_get_dst_formats(esp_gmf_element_handle_t handle, uint32_t src_codec, const uint32_t **dst_fmts, uint8_t *dst_fmts_num)

  Get supported output formats for a given video decoder type This function retrieves the list of output formats (e.g., YUV420P, RGB565LE) that the decoder can produce when decoding from the specified source codec.

  参数:

    - handle – [in] Video decoder element handle

    - src_codec – [in] Decode type (GMF FourCC representation of video codec)

    - dst_fmts – [out] Output video formats to store (GMF FourCC representation of video format)

    - dst_fmts_num – [out] Output video format number to store

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

### Structures

struct esp_gmf_video_dec_cfg_t

  Video decoder configuration (optional)

  Public Members

  uint32_t codec_cc

    FourCC code to force a specific decoder implementation See `esp_video_codec_desc_t` in `esp_video_codec_types.h` for details If zero, the system will select a suitable decoder (HW preferred)

### Header File

- elements/gmf_video/include/esp_gmf_video_enc.h

### Functions

esp_gmf_err_t esp_gmf_video_enc_init(esp_gmf_video_enc_cfg_t *cfg, esp_gmf_element_handle_t *handle)

  Initializes the GMF video encoder with the provided configuration.

  参数:

    - cfg – [in] Pointer to the video encoder configuration

    - handle – [out] Pointer to the video encoder handle to be initialized

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_enc_set_dst_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec)

  Setting for video encoder destination codec (e.g., H.264, MJPEG)

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video encoder element handle

    - dst_codec – [in] Video encode destination codec (use GMF FourCC presentation)

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_preset(esp_gmf_element_handle_t handle, esp_gmf_info_video_t *src_info, uint32_t dst_codec)

  Do presetting for video encoder.

  备注

  This function pre-configures the video encoder with both source and destination information This allows querying encoder capabilities before runtime Normally, source information is reported from dependent element in the running state Call this function only before element running

  参数:

    - handle – [in] Video encoder element handle

    - src_info – [in] Video encode input video information

    - dst_codec – [in] Destination video codec (use GMF FourCC presentation)

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_get_out_size(esp_gmf_element_handle_t handle, uint32_t *frame_size)

  Get video encoder output frame size.

  备注

  This API should only be called after `esp_gmf_video_enc_preset` set or after element running

  参数:

    - handle – [in] Video encoder element handle

    - frame_size – [out] Output frame size to store

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_get_src_formats(esp_gmf_element_handle_t handle, uint32_t dst_codec, const uint32_t **src_fmts, uint8_t *src_fmts_num)

  Get supported source formats (e.g., YUV420P, RGB565LE etc) for a given video encoder type This function retrieves the list of supported input formats that can be encoded into the specified destination codec.

  参数:

    - handle – [in] Video encoder element handle

    - dst_codec – [in] Video encode destination codec (GMF FourCC representation of video codec)

    - src_fmts – [out] Source formats to stored (GMF FourCC representation of video format)

    - src_fmts_num – [out] Source format number to store

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_bitrate(esp_gmf_element_handle_t handle, uint32_t bitrate)

  Setting for video encoder bitrate.

  备注

  Support setting both before and after element running

  参数:

    - handle – [in] Video encoder element handle

    - bitrate – [in] Output bitrate

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_gop(esp_gmf_element_handle_t handle, uint32_t gop)

  Set the GOP (Group of Pictures) for video encoder.

  备注

  This setting applies specifically to the H.264 encoder The H.264 encoder generates an I-frame at the start of each GOP Can be configured both before and after the encoder element is running

  参数:

    - handle – [in] Handle to the video encoder element

    - gop – [in] GOP size for H.264 (in number of frames)

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_qp(esp_gmf_element_handle_t handle, uint32_t min_qp, uint32_t max_qp)

  Set the QP (Quantization Parameter) range for video encoder.

  备注

  This setting applies specifically to the H.264 encoder Higher QP values generally result in lower image quality Can be configured both before and after the encoder element is running

  参数:

    - handle – [in] Handle to the video encoder element

    - min_qp – [in] Minimum QP value

    - max_qp – [in] Maximum QP value

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

### Structures

struct esp_gmf_video_enc_cfg_t

  Video encoder configuration.

  Public Members

  uint32_t codec_cc

    FourCC code to force a specific encoder implementation See `esp_video_codec_desc_t` in `esp_video_codec_types.h` for details If zero, the system will select a suitable encoder (HW preferred)

### Header File

- elements/gmf_video/include/esp_gmf_video_ppa.h

### Functions

esp_gmf_err_t esp_gmf_video_ppa_init(void *config, esp_gmf_element_handle_t *handle)

  Video PPA (Pixel Processing Accelerator) is a compact module Currently it only support on ESP32P4 and including following video processing: 1) Color convert 2) Scale 3) Rotate 5) Crop These operations can be mixed so that can do them all together for better performance.

  Initializes the GMF Video PPA

  参数:

    - config – [in] No need to set

    - handle – [out] Video PPA handle to store

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

  Set video PPA destination resolution.

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video PPA handle

    - res – [in] Video PPA output resolution

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_dst_format(esp_gmf_element_handle_t handle, uint32_t format)

  Set video PPA destination format.

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video PPA handle

    - format – [in] Video PPA destination format (GMF FourCC representation of video format)

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_cropped_rgn(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

  Set video PPA cropped region.

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video PPA handle

    - rgn – [in] Region to be kept in original video frame

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_rotation(esp_gmf_element_handle_t handle, uint16_t degree)

  Set video PPA rotation degree.

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video PPA handle

    - degree – [in] Currently only support (0, 90, 180, 270) unit(one degree)

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

### Header File

- elements/gmf_video/include/esp_gmf_video_overlay.h

### Functions

esp_gmf_err_t esp_gmf_video_overlay_init(void *config, esp_gmf_element_handle_t *handle)

  Initializes the GMF overlay mixer with the provided configuration.

  Diagram: +————————-&#8212;+ | Original Plane | | (Background) | | +——————&#8212;+ | | | Overlay Plane | | | | (Foreground) | | | +——————&#8212;+ | +————————-&#8212;+

  备注

  This module mixes the overlay plane (foreground) into the original plane (background) using alpha blending The formula is: Output_Plane_Pixel = Original_Plane_Pixel * (255 - alpha) + Overlay_Plane_Pixel * alpha where `alpha` is the transparency level (0 = fully transparent, 255 = fully opaque)

  参数:

    - config – [in] No need to set

    - handle – [out] Pointer to the overlay mixer handle to be initialized

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_rgn(esp_gmf_element_handle_t handle, esp_gmf_overlay_rgn_info_t *rgn_info)

  Set the output region for video overlay.

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video overlay element handle

    - rgn_info – [out] Region information

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_overlay_port(esp_gmf_element_handle_t handle, esp_gmf_port_handle_t port)

  Set the GMF port for video overlay input.

  备注

  The video overlay element will use this port to acquire overlay data, mix it into the original video plane, and then release the port The port is managed by the user, the overlay element only accesses it This API can be called before element running and allow reset

  参数:

    - handle – [in] Video overlay element handle

    - port – [in] GMF port for video overlay data

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_alpha(esp_gmf_element_handle_t handle, uint8_t alpha)

  Set the alpha value of the overlay window.

  备注

  Allow to set both before and after element running

  参数:

    - handle – [in] Video overlay element handle

    - alpha – [in] Overlay window alpha setting: 0 means fully transparent, 255 means fully opaque

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_overlay_enable(esp_gmf_element_handle_t handle, bool enable)

  Enable overlay mixer or not.

  备注

  When overlay mixer is disabled will not acquire data from overlay port and do mixer anymore Original plane data leave unchanged Allow to set both before and after element running

  参数:

    - handle – [in] Video overlay mixer handle

    - enable – [in] Enable mixer or not

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

### Header File

- elements/gmf_video/include/esp_gmf_video_fps_cvt.h

### Functions

esp_gmf_err_t esp_gmf_video_fps_cvt_init(void *config, esp_gmf_element_handle_t *handle)

  Initializes the GMF video frame rate convert.

  参数:

    - config – [in] No need to set

    - handle – [out] Pointer to the video frame rate convert handle to be initialized

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_fps_cvt_set_fps(esp_gmf_element_handle_t handle, uint16_t fps)

  Set GMF video frame rate converter output frame rate.

  备注

  This API should only called before element running

  参数:

    - handle – [in] Video frame rate converter handle

    - fps – [out] Destination frame rate to set (unit frame per second)

  返回:

    - ESP_GMF_ERR_OK Success

    - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

### Header File

- elements/gmf_video/include/esp_gmf_video_color_convert.h

### Functions

esp_gmf_err_t esp_gmf_video_color_convert_init(esp_imgfx_color_convert_cfg_t *config, esp_gmf_element_handle_t *handle)

  Initialize the video color convert element with the specified configuration.

  参数:

    - config – [in] Pointer to a color convert configuration

    - handle – [out] Pointer to a `esp_gmf_element_handle_t` where the initialized video color convert handle will be stored

  返回:

    - ESP_GMF_ERR_OK Initialization successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

    - ESP_GMF_ERR_MEMORY_LACK No memory for video color convert element

esp_gmf_err_t esp_gmf_video_color_convert_dst_format(esp_gmf_element_handle_t handle, uint32_t format)

  Set video color convert destination format.

  备注

  This API have no special timing request, user can call it freely

  参数:

    - handle – [in] Video color convert handle

    - format – [in] Video color convert destination format (GMF FourCC representation of video format)

  返回:

    - ESP_GMF_ERR_OK Successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

### Macros

DEFAULT_ESP_GMF_COLOR_CONVERT_CONFIG()

### Header File

- elements/gmf_video/include/esp_gmf_video_crop.h

### Functions

esp_gmf_err_t esp_gmf_video_crop_init(esp_imgfx_crop_cfg_t *config, esp_gmf_element_handle_t *handle)

  Initialize the video crop element with the specified configuration.

  参数:

    - config – [in] Pointer to a `esp_imgfx_crop_cfg_t` structure that contains the configuration settings for the video crop

    - handle – [out] Pointer to a `esp_gmf_element_handle_t` where the initialized video crop handle will be stored

  返回:

    - ESP_GMF_ERR_OK Initialization successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

    - ESP_GMF_ERR_MEMORY_LACK No memory for video crop element

esp_gmf_err_t esp_gmf_video_crop_rgn(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

  Set video cropped region.

  备注

  This API have no special timing request, user can call it freely

  参数:

    - handle – [in] Video video crop handle

    - rgn – [in] Region to be kept in original video frame

  返回:

    - ESP_GMF_ERR_OK Successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

### Macros

DEFAULT_ESP_GMF_CROP_CONFIG()

### Header File

- elements/gmf_video/include/esp_gmf_video_scale.h

### Functions

esp_gmf_err_t esp_gmf_video_scale_init(esp_imgfx_scale_cfg_t *config, esp_gmf_element_handle_t *handle)

  Initialize the video scale element with the specified configuration.

  参数:

    - config – [in] Pointer to a `esp_imgfx_scale_cfg_t` structure that contains the configuration settings for the video scale

    - handle – [out] Pointer to a `esp_gmf_element_handle_t` where the initialized video scale handle will be stored

  返回:

    - ESP_GMF_ERR_OK Initialization successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

    - ESP_GMF_ERR_MEMORY_LACK No memory for video scale element

esp_gmf_err_t esp_gmf_video_scale_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

  Set video scale destination resolution.

  备注

  This API have no special timing request, user can call it freely

  参数:

    - handle – [in] Video video scale handle

    - res – [in] Video video scale output resolution

  返回:

    - ESP_GMF_ERR_OK Successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

### Macros

DEFAULT_ESP_GMF_SCALE_CONFIG()

### Header File

- elements/gmf_video/include/esp_gmf_video_rotate.h

### Functions

esp_gmf_err_t esp_gmf_video_rotate_init(esp_imgfx_rotate_cfg_t *config, esp_gmf_element_handle_t *handle)

  Initialize the video rotate element with the specified configuration.

  参数:

    - config – [in] Pointer to a `esp_imgfx_rotate_cfg_t` structure that contains the configuration settings for the video rotate

    - handle – [out] Pointer to a `esp_gmf_element_handle_t` where the initialized video rotate handle will be stored

  返回:

    - ESP_GMF_ERR_OK Initialization successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

    - ESP_GMF_ERR_MEMORY_LACK No memory for video rotate element

esp_gmf_err_t esp_gmf_video_rotate_set_rotation(esp_gmf_element_handle_t handle, uint16_t degree)

  Set video rotate rotation degree.

  备注

  This API have no special timing request, user can call it freely

  参数:

    - handle – [in] Video video rotate handle

    - degree – [in] Any angle, unit(one degree)

  返回:

    - ESP_GMF_ERR_OK Successful

    - ESP_GMF_ERR_INVALID_ARG Invalid arguments

### Macros

DEFAULT_ESP_GMF_ROTATE_CONFIG()

### Header File

- elements/gmf_video/include/esp_gmf_video_types.h

### Structures

struct esp_gmf_video_resolution_t

  Video resolution definition.

  Public Members

  uint16_t width

    Width of the video in pixels

  uint16_t height

    Height of the video in pixels

struct esp_gmf_video_rgn_t

  Video region definition Represents a rectangular region of the video frame.

  Public Members

  uint16_t x

    X-coordinate of the top-left corner

  uint16_t y

    Y-coordinate of the top-left corner

  uint16_t width

    Width of the region in pixels

  uint16_t height

    Height of the region in pixels

struct esp_gmf_overlay_rgn_info_t

  Specifies the format and destination region for a video overlay.

  备注

  This structure defines the parameters needed to position and format an overlay on a video frame

  Public Members

  uint32_t format_id

    FourCC of overlay region frame

  esp_gmf_video_rgn_t dst_rgn

    Region position to put the overlay

  uint8_t rgn_index

    Region index start from 0 (first region)

  bool has_trans_color

    Whether region has transparent color (if near it will become transparent)

  uint8_t trans_color[3]

    Transparent color RGB order

### Header File

- elements/gmf_video/include/esp_gmf_video_param.h

### Functions

esp_gmf_err_t esp_gmf_video_param_set_dst_format(esp_gmf_element_handle_t handle, uint32_t dst_fmt)

  This file provides general parameter setting functions for common video processing using GMF method.

          These functions simplify setting parameters for video processing elements,
          handling the underlying method calls. They do not enforce a specific order
          or argument count, but apply the basic settings where supported. Refer to
          the individual element APIs for details on supported settings and timing.

  Set destination format for video element (e.g., color converter, video decoder)

  参数:

    - handle – [in] Video element handle

    - dst_fmt – [in] Destination format

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_fps(esp_gmf_element_handle_t handle, uint16_t fps)

  Set frame rate for video element (mainly for FPS converter)

  参数:

    - handle – [in] Video element handle

    - fps – [in] Destination frame rate

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

  Set destination resolution for video element (mainly for video scaler)

  参数:

    - handle – [in] Video element handle

    - res – [in] Destination resolution

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_dst_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec)

  Set destination codec for video element (e.g., video encoder)

  参数:

    - handle – [in] Video element handle

    - dst_codec – [in] Destination codec

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_venc_preset(esp_gmf_element_handle_t handle, esp_gmf_info_video_t *vid_info, uint32_t dst_codec)

  Preset source video information and destination codec for video encoder.

  备注

  In special cases call this API then query some information from video encoder before element running

  参数:

    - handle – [in] Video element handle

    - vid_info – [in] Source video information

    - dst_codec – [in] Destination codec

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_get_src_fmts_by_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec, const uint32_t **src_fmts, uint8_t *src_fmts_num)

  Get supported source formats by codec for video element (mainly for video encoder)

  参数:

    - handle – [in] Video element handle

    - dst_codec – [in] Destination codec

    - src_fmts – [in] Pointer to store source formats array

    - src_fmts_num – [in] Pointer to store source formats number

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_src_codec(esp_gmf_element_handle_t handle, uint32_t src_codec)

  Set source codec for video element (e.g., video decoder)

  参数:

    - handle – [in] Video element handle

    - src_codec – [in] Source codec

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_rotate_angle(esp_gmf_element_handle_t handle, uint16_t degree)

  Set the rotation angle for the video element (mainly for video rotator)

  参数:

    - handle – [in] Video element handle

    - degree – [in] Rotation angle in degrees (e.g., 0, 90, 180, 270)

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_cropped_region(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

  Set the cropped region for video element (mainly for video cropper)

  参数:

    - handle – [in] Video element handle

    - rgn – [in] Kept region after crop

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_overlay_enable(esp_gmf_element_handle_t self, bool enable)

  Enable overlay for video element (mainly for video overlay)

  参数:

    - handle – [in] Video element handle

    - enable – [in] Whether enable video overlay or not

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_overlay_port(esp_gmf_element_handle_t self, void *port)

  Set overlay port for video element (mainly for video overlay)

  参数:

    - handle – [in] Video element handle

    - port – [in] Handle to the input GMF port providing overlay data

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_overlay_rgn(esp_gmf_element_handle_t self, esp_gmf_overlay_rgn_info_t *rgn)

  Set overlay region information for video element (mainly for video overlay)

  参数:

    - handle – [in] Video element handle

    - rgn – [in] Overlay region information

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_get_dst_fmts_by_codec(esp_gmf_element_handle_t handle, uint32_t src_codec, const uint32_t **dst_fmts, uint8_t *dst_fmts_num)

  Get supported destination formats by codec for video element (mainly for video decoder)

  参数:

    - handle – [in] Video element handle

    - src_codec – [in] Source codec

    - src_fmts – [in] Pointer to store source formats array

    - src_fmts_num – [in] Pointer to store source formats number

  返回:

    - ESP_GMF_ERR_OK On success

    - ESP_GMF_ERR_NOT_FOUND Not found the method

    - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

    - Others Failed to apply method

此文档对您有帮助吗？

- 反馈已收到，谢谢！
- 如果您有其他意见，欢迎填写乐鑫文档反馈表。

- 我们重视您的反馈。
- 您可以填写乐鑫文档反馈表告诉我们如何改进该文档。