GMF-Video

[English]

gmf_video 是 ESP-GMF 的视频处理组件，提供 9 个处理单元（element）覆盖编解码、像素加速器、软件图像效果与帧率 / 叠加工具。所有处理单元都继承自 esp_gmf_video_element_t，对外暴露统一的生命周期功能函数、acquire-release 数据接口和运行时方法（method）调用；底层算法分别由 ESP32-P4 的硬件 PPA / 2D-DMA、esp_video_codec 编解码库与 esp_image_effects 软件图像效果库实现。本篇按四类展开每个处理单元的用途、配置与运行时控制；处理单元基类与运行时方法机制见 GMF 处理单元，数据通路见 数据流转。

功能清单

• vid_dec：视频解码，支持 H.264 与 MJPEG，可指定输出像素格式（如 YUV420P、RGB565LE）

• vid_enc：视频编码，支持 H.264 与 MJPEG，可运行中调整 bitrate、GOP、QP 范围（仅 H.264）

• vid_ppa：ESP32-P4 像素加速器复合处理单元，把颜色转换、缩放、裁剪、旋转合并为一次硬件处理

• vid_fps_cvt：帧率转换，按 PTS 丢帧把输入帧率降到指定输出帧率

• vid_overlay：叠加混合器，把额外画面（水印、UI、时间戳）按 alpha 混合或透明色（colorkey）叠加到原始视频

• vid_color_cvt：软件颜色转换，覆盖 YUYV、RGB565、RGB888、YUV420P 等常见格式互转

• vid_crop：软件裁剪，从原始帧中提取指定矩形区域

• vid_scale：软件缩放，按目标分辨率重采样视频帧

• vid_rotate：软件旋转，支持任意角度（单位为度）

• 共性：旁路（bypass）优化、自动对齐查询、need_recfg 触发的运行时重配置、PTS 透传与丢帧逻辑

技术拆解

元素层次与共性机制

每个具体处理单元都按”包装一个底层视频算法 / 硬件句柄”的方式构造：派生结构以 esp_gmf_video_element_t 为首字段，绑定 codec / PPA / imgfx handle 与本处理单元的配置；open 创建底层 handle 并向处理链（pipeline）上报输出 esp_gmf_info_video_t（宽、高、帧率、像素格式 FourCC）；process 在每次调度时从输入数据端口（port）取帧、执行转换并写出输出数据端口；close 释放硬件资源。

        classDiagram
    direction TD

    class esp_gmf_video_element_t
    class Codec {
        vid_dec
        vid_enc
    }
    class HwAccel {
        vid_ppa
    }
    class SwEffects {
        vid_color_cvt
        vid_crop
        vid_scale
        vid_rotate
    }
    class Utility {
        vid_fps_cvt
        vid_overlay
    }

    esp_gmf_video_element_t <|-- Codec
    esp_gmf_video_element_t <|-- HwAccel
    esp_gmf_video_element_t <|-- SwEffects
    esp_gmf_video_element_t <|-- Utility
    

四条公共机制：

• 旁路优化：vid_dec 在源 / 目标格式一致时进入零拷贝直通；软件效果处理单元在源 / 目标参数一致时同样置旁路，输入数据载体（payload）直接透传输出数据端口。

• need_recfg 重配置：所有 setter（set_dst_format / set_dst_resolution / set_angle 等）改动后置 need_recfg 标志，下一次 process 边界销毁旧 handle、重建新 handle，运行中切换分辨率或像素格式不需要应用层重建处理单元。

• 缓冲对齐自动查询：底层硬件（PPA、H.264 编码器）通常对输入 / 输出缓冲区有对齐要求，处理单元在 open 阶段调用硬件驱动查询对齐参数，写入数据端口 attr 让框架按对齐分配数据载体，避免硬件直接拒绝处理。

• 格式信息上报：上游处理单元解析到当前帧的宽高 / 像素格式后通过 REPORT_INFO 事件上报，处理链把信息投递给下游处理单元，下游据此 open 或触发重配置。机制详情见 GMF 处理链与任务调度 的 REPORT_INFO 说明。

视频编解码

vid_dec 与 vid_enc 是视频处理链的起点或终点处理单元，典型组装如下。

        flowchart LR
    subgraph rec ["录制链"]
        direction LR
        CapIn(("camera/raw")) --> CC1["vid_color_cvt"]
        CC1 --> Scale["vid_scale"]
        Scale --> Enc["vid_enc"]
        Enc --> FileOut(("file/http"))
    end
    subgraph play ["播放链"]
        direction LR
        FileIn(("file/http")) --> Dec["vid_dec"]
        Dec --> PPA["vid_ppa"]
        PPA --> Disp(("display"))
    end
    

vid_dec：把压缩流解码成原始像素。esp_gmf_video_dec_init() 接受 esp_gmf_video_dec_cfg_t，codec_cc 字段用 FourCC 指定解码器实现；为零时框架按可用硬件 / 软件解码器自动选择。

通过 esp_gmf_video_dec_get_dst_formats() 查询指定源编码（H.264 / MJPEG 等）对应的可输出像素格式列表，再用 esp_gmf_video_dec_set_dst_format() 在处理单元启动前固定输出格式。如果源格式与目标格式一致（罕见但有用，例如纯转封装链），vid_dec 自动进入旁路模式直接转发数据载体。

esp_gmf_video_dec_cfg_t cfg = { .codec_cc = 0 };  /* 自动选 */
esp_gmf_element_handle_t dec = NULL;
esp_gmf_video_dec_init(&cfg, &dec);

const uint32_t *fmts = NULL;
uint8_t fmt_num = 0;
esp_gmf_video_dec_get_dst_formats(dec, ESP_FOURCC_H264, &fmts, &fmt_num);
esp_gmf_video_dec_set_dst_format(dec, fmts[0]);

vid_enc：把原始像素编码成压缩流。esp_gmf_video_enc_init() 接受 esp_gmf_video_enc_cfg_t，codec_cc 指定编码器实现。

正常工作流是处理单元启动后从上游获取源信息（esp_gmf_info_video_t），按 set_dst_codec 指定的目标 codec 自动选具体实现并 open。如果要在启动前查询编码器能力（例如查支持的源像素格式），先用 esp_gmf_video_enc_preset() 传入源信息与目标 codec，再调用 esp_gmf_video_enc_get_src_formats() 与 esp_gmf_video_enc_get_out_size()。

运行中可调三个参数（H.264 编码器额外支持后两个）：

esp_gmf_video_enc_set_bitrate(enc, 2 * 1000 * 1000);  /* 2 Mbps */
esp_gmf_video_enc_set_gop(enc, 30);                    /* 每 30 帧一个 I 帧 */
esp_gmf_video_enc_set_qp(enc, 20, 40);                 /* QP 区间 */

输出缓冲区大小按 esp_gmf_video_enc_get_out_size() 估算，MJPEG 按 10:1、H.264 按 2:1 的压缩比给上限，框架按此尺寸分配输出数据载体。

硬件加速 PPA

vid_ppa 仅支持 ESP32-P4/ESP32S31，把颜色转换、缩放、裁剪、旋转合并到一次硬件处理中。esp_gmf_video_ppa_init() 不需要传配置；四个 setter 分别配置目标格式、目标分辨率、裁剪区域、旋转角度（仅支持 0 / 90 / 180 / 270 度）。这些 setter 都只能在处理单元启动前调用，open 时一次性向硬件下发组合参数。

PPA 内部根据当前请求选择硬件路径：单纯的颜色格式转换可以由 2D-DMA 完成，吞吐更高；涉及缩放、旋转或 2D-DMA 不支持的格式组合时回退到 PPA 主路径。路径选择由框架内部完成，处理单元仅暴露统一接口。

esp_gmf_element_handle_t ppa = NULL;
esp_gmf_video_ppa_init(NULL, &ppa);

esp_gmf_video_resolution_t dst_res = { .width = 480, .height = 320 };
esp_gmf_video_ppa_set_dst_resolution(ppa, &dst_res);
esp_gmf_video_ppa_set_dst_format(ppa, ESP_FOURCC_RGB16);
esp_gmf_video_ppa_set_rotation(ppa, 90);

esp_gmf_video_rgn_t crop = { .x = 0, .y = 0, .width = 800, .height = 600 };
esp_gmf_video_ppa_set_cropped_rgn(ppa, &crop);

非 ESP32-P4 芯片上 vid_ppa 不可用，需要等价功能时用下文 4 个软件效果处理单元串接（vid_color_cvt → vid_crop → vid_scale → vid_rotate），代价是 CPU 占用上升。

软件图像效果

四个软件效果处理单元包装 esp_image_effects 库的对应算法，对外只暴露各自的 setter。所有效果都遵循同一条路径：acquire_in 取输入帧、按配置生成输出帧大小、acquire_out 取输出帧、调用 esp_imgfx_*_process 处理、双双 release。

vid_color_cvt：像素格式转换，例如 RGB565 → YUV420P、YUYV → RGB888。颜色空间标准（BT601 / BT709）通过初始化时的 esp_imgfx_color_convert_cfg_t.color_space_std 字段选择。esp_gmf_video_color_convert_dst_format() 在运行中切换目标格式，置 need_recfg。

vid_crop：裁剪元素。初始化用 DEFAULT_ESP_GMF_CROP_CONFIG 给出默认 320×240 输入、160×120 裁剪区。运行中用 esp_gmf_video_crop_rgn() 切换裁剪矩形（esp_gmf_video_rgn_t 给出 x / y / width / height）。

vid_scale：缩放元素。filter_type 字段决定算法（如 ESP_IMGFX_SCALE_FILTER_TYPE_DOWN_RESAMPLE）。运行中用 esp_gmf_video_scale_dst_resolution() 修改目标分辨率。

vid_rotate：旋转元素。与 vid_ppa 只支持四个角度不同，软件旋转支持任意角度（单位为度），适合需要倾斜显示或图像校正的场景。esp_gmf_video_rotate_set_rotation() 在运行中切换角度。

esp_imgfx_scale_cfg_t cfg = DEFAULT_ESP_GMF_SCALE_CONFIG();
cfg.in_res       = (esp_imgfx_res_t){ .width = 640, .height = 480 };
cfg.in_pixel_fmt = ESP_IMGFX_PIXEL_FMT_RGB565_LE;
cfg.scale_res    = (esp_imgfx_res_t){ .width = 320, .height = 240 };

esp_gmf_element_handle_t scale = NULL;
esp_gmf_video_scale_init(&cfg, &scale);

帧率与叠加

vid_fps_cvt：通过丢帧把输入帧率降低到指定输出帧率。处理单元维护起始 PTS 与累计已输出帧数，按 1 / fps 步长计算 expected_pts。每一帧到来时：当前帧 PTS 大于等于 expected_pts 才透传到下游并累计计数；否则直接 release_in 丢弃该帧。

由于纯丢帧不能升采样，set_fps 设定值必须低于上游帧率，否则等于无效。

esp_gmf_element_handle_t fps = NULL;
esp_gmf_video_fps_cvt_init(NULL, &fps);
esp_gmf_video_fps_cvt_set_fps(fps, 15);  /* 从 30 fps 降到 15 fps */

vid_overlay：把额外画面叠加到主视频上。处理单元是多输入：主视频从默认 in 数据端口进入，叠加层从用户通过 esp_gmf_video_overlay_set_overlay_port() 注册的额外数据端口进入。合成支持 alpha 混合与透明色（colorkey）两种模式。alpha 混合公式：

Output = Original × (255 − alpha) + Overlay × alpha

set_rgn 指定叠加层在主画面上的位置与格式（esp_gmf_overlay_rgn_info_t 含 format_id、dst_rgn，以及可选的 has_trans_color / trans_color 用于 colorkey 合成，与指定 RGB 接近的像素视为透明）；set_alpha 在运行中改透明度（0 全透明、255 全不透明）；esp_gmf_video_overlay_enable()(false) 临时禁用叠加，处理单元不再 acquire 叠加数据端口，主画面原样透传。

esp_gmf_element_handle_t ovl = NULL;
esp_gmf_video_overlay_init(NULL, &ovl);

esp_gmf_video_overlay_set_overlay_port(ovl, overlay_port);
esp_gmf_overlay_rgn_info_t rgn = {
    .format_id = ESP_FOURCC_RGB16,
    .dst_rgn   = { .x = 16, .y = 16, .width = 128, .height = 32 },
};
esp_gmf_video_overlay_set_rgn(ovl, &rgn);
esp_gmf_video_overlay_set_alpha(ovl, 200);

性能

视频处理的吞吐主要由底层硬件 / 算法决定，处理单元调度与 acquire-release 的开销相对较小。按瓶颈分类：

类别	主要瓶颈	调优方向
vid_dec / vid_enc	硬件 codec 或软件库的吞吐	ESP32-P4 支持 H.264 硬件编码；提前 preset 让框架预分配缓冲
vid_ppa	PPA / 2D-DMA 时钟与缓冲带宽	合并相邻像素操作到同一 vid_ppa 实例减少中间 buffer
软件效果	CPU 与内存带宽	用 ESP32-P4 时优先 vid_ppa；其余 SoC 控制分辨率不要超过 SoC 处理能力
vid_fps_cvt	仅做 PTS 比较与丢帧，CPU 开销极小	适合放在编码或显示前减少下游负载
vid_overlay	alpha 混合或 colorkey 合成的 CPU 周期	叠加区域越小、像素格式 byte 数越少越快

应用示例

• elements/test_apps/main/elements/gmf_video_el_test.c：典型 video 处理单元用例，包含 gen_pattern_color_bar 合成测试帧、open 阶段格式上报验证

• elements/test_apps/main/elements/gmf_image_effects_test.c：图像效果链测试，覆盖软件效果元素的串接

上层 esp_video_render 与 esp_capture 等应用组件也通过本组件连接视频处理链，可作进阶参考。

SoC 兼容性

各处理单元在乐鑫 SoC 上的支持矩阵：

element	ESP32	ESP32-S2	ESP32-S3	ESP32-P4
vid_ppa	不支持	不支持	不支持	支持
vid_fps_cvt	支持	支持	支持	支持
vid_overlay	支持	支持	支持	支持
vid_dec	MJPEG	MJPEG	SW H.264 / MJPEG	SW H.264 / HW MJPEG
vid_enc	MJPEG	MJPEG	SW H.264 / MJPEG	HW H.264 / HW MJPEG
vid_color_cvt / vid_crop / vid_scale / vid_rotate	支持	支持	支持	支持

ESP32 与 ESP32-S2 不支持 H.264。ESP32-S3 的 H.264 编解码均通过软件实现；ESP32-P4 支持 H.264 硬件编码与 MJPEG 硬件编解码，H.264 解码通过软件实现。若需 ESP32-P4 的 PPA 加速，可省略软件效果处理单元，改用 vid_ppa。

FAQ

Q： vid_dec 输出格式如何选择？

先用 esp_gmf_video_dec_get_dst_formats() 查询当前源编码可支持的输出格式列表，再选择符合下游处理单元输入要求的格式（例如 vid_ppa 需要 RGB565、显示需要 RGB888），并在处理单元启动前调用 esp_gmf_video_dec_set_dst_format() 固定输出格式。需要查询输出帧大小时，才要求先设置目标格式或在处理单元启动后调用对应查询接口。

Q： vid_enc 启动前查不到输出帧大小？

正常情况编码器要在获取上游 esp_gmf_info_video_t 后才能确定输出大小。若希望在启动前查询，先调 esp_gmf_video_enc_preset() 传入预期源信息与目标 codec，esp_gmf_video_enc_get_out_size() 才能返回值。

Q： vid_ppa 在 ESP32 上是否可用？

不支持。vid_ppa 依赖 ESP32-P4 的硬件 PPA / 2D-DMA。其他 SoC 上用 vid_color_cvt / vid_scale / vid_rotate / vid_crop 软件实现替代，分辨率与帧率需要按 CPU 实际能力收敛。

Q： vid_fps_cvt 设置的帧率高于上游会怎样？

处理单元不做插帧。若 set_fps 高于上游帧率，每一帧的 PTS 都大于等于 expected_pts，全部透传，输出帧率被上游限定。该配置仅在降低帧率时生效，设定值不得高于上游帧率。

Q： vid_overlay 的两个输入如何同步？

主输入是普通 GMF 输入数据端口，叠加输入是用户通过 set_overlay_port 注册的额外数据端口。处理单元每次 process 先 acquire 主输入再 acquire 叠加输入，应用方负责保证两路按帧对齐（例如同一个处理链内的两个上游处理单元同步产帧），否则可能出现叠加帧落后或抖动。

Q： 软件效果处理单元与 vid_ppa 在 ESP32-P4 上如何取舍？

需要做颜色转换 + 缩放 + 旋转 / 裁剪组合时优先用 vid_ppa：一次硬件处理完成全部转换，CPU 占用低、延迟稳。仅需单一操作或需要 PPA 不支持的角度 / 像素格式时用对应软件元素。

API 参考

本组件涉及的头文件：

• esp_gmf_video_dec.h / esp_gmf_video_enc.h：视频编解码处理单元

• esp_gmf_video_ppa.h：像素加速器

• esp_gmf_video_fps_cvt.h：帧率转换

• esp_gmf_video_overlay.h：叠加混合器

• esp_gmf_video_color_convert.h / esp_gmf_video_crop.h / esp_gmf_video_scale.h / esp_gmf_video_rotate.h：软件图像效果

• esp_gmf_video_types.h：视频分辨率 / 区域 / 叠加配置等共享类型

• esp_gmf_video_param.h：视频参数辅助接口

处理单元基类 esp_gmf_video_element_t 的接口位于 GMF 处理单元。

Header File

• elements/gmf_video/include/esp_gmf_video_dec.h

Functions

esp_gmf_err_t esp_gmf_video_dec_init(esp_gmf_video_dec_cfg_t *cfg, esp_gmf_element_handle_t *handle)

Initializes the GMF Video decoder with the provided configuration.

参数:

• cfg – [in] Pointer to the Video decoder configuration (optional)

• handle – [out] Pointer to the Video decoder handle to be initialized

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_dec_get_out_size(esp_gmf_element_handle_t handle, uint32_t *frame_size)

Get Video decoder output frame size.

备注

This API should only called after element running or esp_gmf_video_dec_set_dst_format is called

参数:

• handle – [in] Video decoder element handle

• frame_size – [out] Output frame size to store

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_dec_set_dst_format(esp_gmf_element_handle_t handle, uint32_t dst_fmt)

Setting for video decoder destination format (e.g., YUV420P, RGB565LE)

备注

This API should only called before element running

参数:

• handle – [in] Video decoder element handle

• dst_fmt – [in] Destination format (GMF FourCC representation of video codec)

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_dec_get_dst_formats(esp_gmf_element_handle_t handle, uint32_t src_codec, const uint32_t **dst_fmts, uint8_t *dst_fmts_num)

Get supported output formats for a given video decoder type This function retrieves the list of output formats (e.g., YUV420P, RGB565LE) that the decoder can produce when decoding from the specified source codec.

参数:

• handle – [in] Video decoder element handle

• src_codec – [in] Decode type (GMF FourCC representation of video codec)

• dst_fmts – [out] Output video formats to store (GMF FourCC representation of video format)

• dst_fmts_num – [out] Output video format number to store

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Structures

struct esp_gmf_video_dec_cfg_t

Video decoder configuration (optional)

Public Members

uint32_t codec_cc

FourCC code to force a specific decoder implementation See esp_video_codec_desc_t in esp_video_codec_types.h for details If zero, the system will select a suitable decoder (HW preferred)

Header File

• elements/gmf_video/include/esp_gmf_video_enc.h

Functions

esp_gmf_err_t esp_gmf_video_enc_init(esp_gmf_video_enc_cfg_t *cfg, esp_gmf_element_handle_t *handle)

Initializes the GMF video encoder with the provided configuration.

参数:

• cfg – [in] Pointer to the video encoder configuration

• handle – [out] Pointer to the video encoder handle to be initialized

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_enc_set_dst_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec)

Setting for video encoder destination codec (e.g., H.264, MJPEG)

备注

This API should only called before element running

参数:

• handle – [in] Video encoder element handle

• dst_codec – [in] Video encode destination codec (use GMF FourCC presentation)

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_preset(esp_gmf_element_handle_t handle, esp_gmf_info_video_t *src_info, uint32_t dst_codec)

Do presetting for video encoder.

备注

This function pre-configures the video encoder with both source and destination information This allows querying encoder capabilities before runtime Normally, source information is reported from dependent element in the running state Call this function only before element running

参数:

• handle – [in] Video encoder element handle

• src_info – [in] Video encode input video information

• dst_codec – [in] Destination video codec (use GMF FourCC presentation)

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_get_out_size(esp_gmf_element_handle_t handle, uint32_t *frame_size)

Get video encoder output frame size.

备注

This API should only be called after esp_gmf_video_enc_preset set or after element running

参数:

• handle – [in] Video encoder element handle

• frame_size – [out] Output frame size to store

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_get_src_formats(esp_gmf_element_handle_t handle, uint32_t dst_codec, const uint32_t **src_fmts, uint8_t *src_fmts_num)

Get supported source formats (e.g., YUV420P, RGB565LE etc) for a given video encoder type This function retrieves the list of supported input formats that can be encoded into the specified destination codec.

参数:

• handle – [in] Video encoder element handle

• dst_codec – [in] Video encode destination codec (GMF FourCC representation of video codec)

• src_fmts – [out] Source formats to stored (GMF FourCC representation of video format)

• src_fmts_num – [out] Source format number to store

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_bitrate(esp_gmf_element_handle_t handle, uint32_t bitrate)

Setting for video encoder bitrate.

备注

Support setting both before and after element running

参数:

• handle – [in] Video encoder element handle

• bitrate – [in] Output bitrate

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_gop(esp_gmf_element_handle_t handle, uint32_t gop)

Set the GOP (Group of Pictures) for video encoder.

备注

This setting applies specifically to the H.264 encoder The H.264 encoder generates an I-frame at the start of each GOP Can be configured both before and after the encoder element is running

参数:

• handle – [in] Handle to the video encoder element

• gop – [in] GOP size for H.264 (in number of frames)

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_qp(esp_gmf_element_handle_t handle, uint32_t min_qp, uint32_t max_qp)

Set the QP (Quantization Parameter) range for video encoder.

备注

This setting applies specifically to the H.264 encoder Higher QP values generally result in lower image quality Can be configured both before and after the encoder element is running

参数:

• handle – [in] Handle to the video encoder element

• min_qp – [in] Minimum QP value

• max_qp – [in] Maximum QP value

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Structures

struct esp_gmf_video_enc_cfg_t

Video encoder configuration.

Public Members

uint32_t codec_cc

FourCC code to force a specific encoder implementation See esp_video_codec_desc_t in esp_video_codec_types.h for details If zero, the system will select a suitable encoder (HW preferred)

Header File

• elements/gmf_video/include/esp_gmf_video_ppa.h

Functions

esp_gmf_err_t esp_gmf_video_ppa_init(void *config, esp_gmf_element_handle_t *handle)

Video PPA (Pixel Processing Accelerator) is a compact module Currently it only support on ESP32P4 and including following video processing: 1) Color convert 2) Scale 3) Rotate 5) Crop These operations can be mixed so that can do them all together for better performance.

Initializes the GMF Video PPA

参数:

• config – [in] No need to set

• handle – [out] Video PPA handle to store

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

Set video PPA destination resolution.

备注

This API should only called before element running

参数:

• handle – [in] Video PPA handle

• res – [in] Video PPA output resolution

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_dst_format(esp_gmf_element_handle_t handle, uint32_t format)

Set video PPA destination format.

备注

This API should only called before element running

参数:

• handle – [in] Video PPA handle

• format – [in] Video PPA destination format (GMF FourCC representation of video format)

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_cropped_rgn(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

Set video PPA cropped region.

备注

This API should only called before element running

参数:

• handle – [in] Video PPA handle

• rgn – [in] Region to be kept in original video frame

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_rotation(esp_gmf_element_handle_t handle, uint16_t degree)

Set video PPA rotation degree.

备注

This API should only called before element running

参数:

• handle – [in] Video PPA handle

• degree – [in] Currently only support (0, 90, 180, 270) unit(one degree)

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Header File

• elements/gmf_video/include/esp_gmf_video_overlay.h

Functions

esp_gmf_err_t esp_gmf_video_overlay_init(void *config, esp_gmf_element_handle_t *handle)

Initializes the GMF overlay mixer with the provided configuration.

Diagram: +————————-—+ | Original Plane | | (Background) | | +———————+ | | | Overlay Plane | | | | (Foreground) | | | +———————+ | +————————-—+

备注

This module mixes the overlay plane (foreground) into the original plane (background) using alpha blending The formula is: Output_Plane_Pixel = Original_Plane_Pixel * (255 - alpha) + Overlay_Plane_Pixel * alpha where alpha is the transparency level (0 = fully transparent, 255 = fully opaque)

参数:

• config – [in] No need to set

• handle – [out] Pointer to the overlay mixer handle to be initialized

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_rgn(esp_gmf_element_handle_t handle, esp_gmf_overlay_rgn_info_t *rgn_info)

Set the output region for video overlay.

备注

This API should only called before element running

参数:

• handle – [in] Video overlay element handle

• rgn_info – [out] Region information

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_overlay_port(esp_gmf_element_handle_t handle, esp_gmf_port_handle_t port)

Set the GMF port for video overlay input.

备注

The video overlay element will use this port to acquire overlay data, mix it into the original video plane, and then release the port The port is managed by the user, the overlay element only accesses it This API can be called before element running and allow reset

参数:

• handle – [in] Video overlay element handle

• port – [in] GMF port for video overlay data

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_alpha(esp_gmf_element_handle_t handle, uint8_t alpha)

Set the alpha value of the overlay window.

备注

Allow to set both before and after element running

参数:

• handle – [in] Video overlay element handle

• alpha – [in] Overlay window alpha setting: 0 means fully transparent, 255 means fully opaque

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_overlay_enable(esp_gmf_element_handle_t handle, bool enable)

Enable overlay mixer or not.

备注

When overlay mixer is disabled will not acquire data from overlay port and do mixer anymore Original plane data leave unchanged Allow to set both before and after element running

参数:

• handle – [in] Video overlay mixer handle

• enable – [in] Enable mixer or not

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Header File

• elements/gmf_video/include/esp_gmf_video_fps_cvt.h

Functions

esp_gmf_err_t esp_gmf_video_fps_cvt_init(void *config, esp_gmf_element_handle_t *handle)

Initializes the GMF video frame rate convert.

参数:

• config – [in] No need to set

• handle – [out] Pointer to the video frame rate convert handle to be initialized

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_fps_cvt_set_fps(esp_gmf_element_handle_t handle, uint16_t fps)

Set GMF video frame rate converter output frame rate.

备注

This API should only called before element running

参数:

• handle – [in] Video frame rate converter handle

• fps – [out] Destination frame rate to set (unit frame per second)

返回:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Header File

• elements/gmf_video/include/esp_gmf_video_color_convert.h

Functions

esp_gmf_err_t esp_gmf_video_color_convert_init(esp_imgfx_color_convert_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video color convert element with the specified configuration.

参数:

• config – [in] Pointer to a color convert configuration

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video color convert handle will be stored

返回:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video color convert element

esp_gmf_err_t esp_gmf_video_color_convert_dst_format(esp_gmf_element_handle_t handle, uint32_t format)

Set video color convert destination format.

备注

This API have no special timing request, user can call it freely

参数:

• handle – [in] Video color convert handle

• format – [in] Video color convert destination format (GMF FourCC representation of video format)

返回:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_COLOR_CONVERT_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_crop.h

Functions

esp_gmf_err_t esp_gmf_video_crop_init(esp_imgfx_crop_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video crop element with the specified configuration.

参数:

• config – [in] Pointer to a esp_imgfx_crop_cfg_t structure that contains the configuration settings for the video crop

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video crop handle will be stored

返回:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video crop element

esp_gmf_err_t esp_gmf_video_crop_rgn(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

Set video cropped region.

备注

This API have no special timing request, user can call it freely

参数:

• handle – [in] Video video crop handle

• rgn – [in] Region to be kept in original video frame

返回:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_CROP_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_scale.h

Functions

esp_gmf_err_t esp_gmf_video_scale_init(esp_imgfx_scale_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video scale element with the specified configuration.

参数:

• config – [in] Pointer to a esp_imgfx_scale_cfg_t structure that contains the configuration settings for the video scale

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video scale handle will be stored

返回:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video scale element

esp_gmf_err_t esp_gmf_video_scale_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

Set video scale destination resolution.

备注

This API have no special timing request, user can call it freely

参数:

• handle – [in] Video video scale handle

• res – [in] Video video scale output resolution

返回:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_SCALE_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_rotate.h

Functions

esp_gmf_err_t esp_gmf_video_rotate_init(esp_imgfx_rotate_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video rotate element with the specified configuration.

参数:

• config – [in] Pointer to a esp_imgfx_rotate_cfg_t structure that contains the configuration settings for the video rotate

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video rotate handle will be stored

返回:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video rotate element

esp_gmf_err_t esp_gmf_video_rotate_set_rotation(esp_gmf_element_handle_t handle, uint16_t degree)

Set video rotate rotation degree.

备注

This API have no special timing request, user can call it freely

参数:

• handle – [in] Video video rotate handle

• degree – [in] Any angle, unit(one degree)

返回:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_ROTATE_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_types.h

Structures

struct esp_gmf_video_resolution_t

Video resolution definition.

Public Members

uint16_t width

Width of the video in pixels

uint16_t height

Height of the video in pixels

struct esp_gmf_video_rgn_t

Video region definition Represents a rectangular region of the video frame.

Public Members

uint16_t x

X-coordinate of the top-left corner

uint16_t y

Y-coordinate of the top-left corner

uint16_t width

Width of the region in pixels

uint16_t height

Height of the region in pixels

struct esp_gmf_overlay_rgn_info_t

Specifies the format and destination region for a video overlay.

备注

This structure defines the parameters needed to position and format an overlay on a video frame

Public Members

uint32_t format_id

FourCC of overlay region frame

esp_gmf_video_rgn_t dst_rgn

Region position to put the overlay

uint8_t rgn_index

Region index start from 0 (first region)

bool has_trans_color

Whether region has transparent color (if near it will become transparent)

uint8_t trans_color[3]

Transparent color RGB order

Header File

• elements/gmf_video/include/esp_gmf_video_param.h

Functions

esp_gmf_err_t esp_gmf_video_param_set_dst_format(esp_gmf_element_handle_t handle, uint32_t dst_fmt)

This file provides general parameter setting functions for common video processing using GMF method.

    These functions simplify setting parameters for video processing elements,
    handling the underlying method calls. They do not enforce a specific order
    or argument count, but apply the basic settings where supported. Refer to
    the individual element APIs for details on supported settings and timing.

Set destination format for video element (e.g., color converter, video decoder)

参数:

• handle – [in] Video element handle

• dst_fmt – [in] Destination format

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_fps(esp_gmf_element_handle_t handle, uint16_t fps)

Set frame rate for video element (mainly for FPS converter)

参数:

• handle – [in] Video element handle

• fps – [in] Destination frame rate

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

Set destination resolution for video element (mainly for video scaler)

参数:

• handle – [in] Video element handle

• res – [in] Destination resolution

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_dst_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec)

Set destination codec for video element (e.g., video encoder)

参数:

• handle – [in] Video element handle

• dst_codec – [in] Destination codec

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_venc_preset(esp_gmf_element_handle_t handle, esp_gmf_info_video_t *vid_info, uint32_t dst_codec)

Preset source video information and destination codec for video encoder.

备注

In special cases call this API then query some information from video encoder before element running

参数:

• handle – [in] Video element handle

• vid_info – [in] Source video information

• dst_codec – [in] Destination codec

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_get_src_fmts_by_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec, const uint32_t **src_fmts, uint8_t *src_fmts_num)

Get supported source formats by codec for video element (mainly for video encoder)

参数:

• handle – [in] Video element handle

• dst_codec – [in] Destination codec

• src_fmts – [in] Pointer to store source formats array

• src_fmts_num – [in] Pointer to store source formats number

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_src_codec(esp_gmf_element_handle_t handle, uint32_t src_codec)

Set source codec for video element (e.g., video decoder)

参数:

• handle – [in] Video element handle

• src_codec – [in] Source codec

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_rotate_angle(esp_gmf_element_handle_t handle, uint16_t degree)

Set the rotation angle for the video element (mainly for video rotator)

参数:

• handle – [in] Video element handle

• degree – [in] Rotation angle in degrees (e.g., 0, 90, 180, 270)

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_cropped_region(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

Set the cropped region for video element (mainly for video cropper)

参数:

• handle – [in] Video element handle

• rgn – [in] Kept region after crop

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_overlay_enable(esp_gmf_element_handle_t self, bool enable)

Enable overlay for video element (mainly for video overlay)

参数:

• handle – [in] Video element handle

• enable – [in] Whether enable video overlay or not

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_overlay_port(esp_gmf_element_handle_t self, void *port)

Set overlay port for video element (mainly for video overlay)

参数:

• handle – [in] Video element handle

• port – [in] Handle to the input GMF port providing overlay data

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_overlay_rgn(esp_gmf_element_handle_t self, esp_gmf_overlay_rgn_info_t *rgn)

Set overlay region information for video element (mainly for video overlay)

参数:

• handle – [in] Video element handle

• rgn – [in] Overlay region information

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_get_dst_fmts_by_codec(esp_gmf_element_handle_t handle, uint32_t src_codec, const uint32_t **dst_fmts, uint8_t *dst_fmts_num)

Get supported destination formats by codec for video element (mainly for video decoder)

参数:

• handle – [in] Video element handle

• src_codec – [in] Source codec

• src_fmts – [in] Pointer to store source formats array

• src_fmts_num – [in] Pointer to store source formats number

返回:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

---

此文档对您有帮助吗？

• 反馈已收到，谢谢！
• 如果您有其他意见，欢迎填写乐鑫文档反馈表。

• 我们重视您的反馈。
• 您可以填写乐鑫文档反馈表告诉我们如何改进该文档。