GMF-Video

[中文]

gmf_video is the video processing component of ESP-GMF, providing 9 elements covering codec, pixel accelerator, software image effects, and frame rate / overlay tools. All elements inherit from esp_gmf_video_element_t, exposing unified lifecycle functions, acquire-release data interfaces, and runtime method calls. The underlying algorithms are implemented by ESP32-P4’s hardware PPA / 2D-DMA, the esp_video_codec library, and the esp_image_effects software image effects library. This document covers the purpose, configuration, and runtime control of each element by category. For the element base class and runtime method mechanism, see GMF Elements; for the data path, see Data Flow.

Feature List

• vid_dec: video decoding, supporting H.264 and MJPEG; output pixel format can be specified (e.g., YUV420P, RGB565LE)

• vid_enc: video encoding, supporting H.264 and MJPEG; bitrate, GOP and QP range(H.264 only) can be adjusted at runtime

• vid_ppa: ESP32-P4 pixel processing accelerator composite element, merging color conversion, scaling, cropping, and rotation into a single hardware pass

• vid_fps_cvt: frame rate conversion, dropping frames by PTS to reduce input frame rate to a specified output frame rate

• vid_overlay: overlay mixer, blending additional content (watermarks, UI, timestamps) onto the original video via alpha blending or transparent-color (colorkey) compositing

• vid_color_cvt: software color conversion, covering common format conversions such as YUYV, RGB565, RGB888, and YUV420P

• vid_crop: software cropping, extracting a specified rectangular region from the original frame

• vid_scale: software scaling, resampling video frames to a target resolution

• vid_rotate: software rotation, supporting arbitrary angles (in degrees)

• Common: bypass optimization, alignment auto-query, need_recfg-triggered runtime reconfiguration, PTS pass-through and frame drop logic

Technical Details

Element Hierarchy and Common Mechanisms

Each specific element is constructed by “wrapping an underlying video algorithm / hardware handle”: the derived struct uses esp_gmf_video_element_t as its first field, binding the codec / PPA / imgfx handle and this element’s configuration; open creates the underlying handle and reports the output esp_gmf_info_video_t (width, height, frame rate, pixel format FourCC) to the pipeline; process fetches a frame from the input port at each scheduling interval, performs the conversion, and writes to the output port; close releases hardware resources.

        classDiagram
    direction TD

    class esp_gmf_video_element_t
    class Codec {
        vid_dec
        vid_enc
    }
    class HwAccel {
        vid_ppa
    }
    class SwEffects {
        vid_color_cvt
        vid_crop
        vid_scale
        vid_rotate
    }
    class Utility {
        vid_fps_cvt
        vid_overlay
    }

    esp_gmf_video_element_t <|-- Codec
    esp_gmf_video_element_t <|-- HwAccel
    esp_gmf_video_element_t <|-- SwEffects
    esp_gmf_video_element_t <|-- Utility
    

Four common mechanisms:

• Bypass optimization: vid_dec enters zero-copy pass-through when source and destination formats are identical; software effect elements also set bypass when source and destination parameters are identical, directly passing the input payload to the output port.

• need_recfg reconfiguration: All setters (set_dst_format / set_dst_resolution / set_angle, etc.) set the need_recfg flag after a change; at the next process boundary, the old handle is destroyed and a new one is rebuilt, allowing resolution or pixel format changes at runtime without the application rebuilding the element.

• Buffer alignment auto-query: Underlying hardware (PPA, H.264 encoder) typically has alignment requirements for input/output buffers; the element queries alignment parameters from the hardware driver at open and writes them to the port attr so the framework allocates payloads with the correct alignment, preventing hardware from rejecting processing.

• Format info reporting: After the upstream element resolves the width/height / pixel format of the current frame, it reports via the REPORT_INFO event; the pipeline delivers the info to downstream elements, which open or trigger reconfiguration accordingly. For details see the REPORT_INFO description in GMF Pipeline and Task Scheduling.

Video Codec

vid_dec and vid_enc are the start or end elements of a video pipeline; a typical assembly is as follows.

        flowchart LR
    subgraph rec ["Recording Pipeline"]
        direction LR
        CapIn(("camera/raw")) --> CC1["vid_color_cvt"]
        CC1 --> Scale["vid_scale"]
        Scale --> Enc["vid_enc"]
        Enc --> FileOut(("file/http"))
    end
    subgraph play ["Playback Pipeline"]
        direction LR
        FileIn(("file/http")) --> Dec["vid_dec"]
        Dec --> PPA["vid_ppa"]
        PPA --> Disp(("display"))
    end
    

vid_dec: Decodes compressed streams to raw pixels. esp_gmf_video_dec_init() accepts esp_gmf_video_dec_cfg_t; the codec_cc field uses FourCC to specify the decoder implementation; when zero, the framework automatically selects based on available hardware / software decoders.

Use esp_gmf_video_dec_get_dst_formats() to query the list of available output pixel formats for a given source codec (H.264 / MJPEG, etc.), then use esp_gmf_video_dec_set_dst_format() to fix the output format before starting the element. If the source and destination formats are identical (rare but useful, e.g., a remux-only pipeline), vid_dec automatically enters bypass mode and forwards payloads directly.

esp_gmf_video_dec_cfg_t cfg = { .codec_cc = 0 };  /* Auto select */
esp_gmf_element_handle_t dec = NULL;
esp_gmf_video_dec_init(&cfg, &dec);

const uint32_t *fmts = NULL;
uint8_t fmt_num = 0;
esp_gmf_video_dec_get_dst_formats(dec, ESP_FOURCC_H264, &fmts, &fmt_num);
esp_gmf_video_dec_set_dst_format(dec, fmts[0]);

vid_enc: Encodes raw pixels to a compressed stream. esp_gmf_video_enc_init() accepts esp_gmf_video_enc_cfg_t; codec_cc specifies the encoder implementation.

The normal workflow is for the element to obtain source info (esp_gmf_info_video_t) from upstream after startup, then auto-select the specific implementation based on the target codec specified by set_dst_codec and open. To query encoder capabilities before startup (e.g., query supported source pixel formats), first call esp_gmf_video_enc_preset() with the source info and target codec, then call esp_gmf_video_enc_get_src_formats() and esp_gmf_video_enc_get_out_size().

Three parameters can be adjusted at runtime (H.264 encoder additionally supports the last two):

esp_gmf_video_enc_set_bitrate(enc, 2 * 1000 * 1000);  /* 2 Mbps */
esp_gmf_video_enc_set_gop(enc, 30);                    /* I-frame every 30 frames */
esp_gmf_video_enc_set_qp(enc, 20, 40);                 /* QP range */

Output buffer size is estimated by esp_gmf_video_enc_get_out_size(), using a 10:1 compression ratio upper bound for MJPEG and 2:1 for H.264; the framework allocates output payloads at this size.

Hardware-Accelerated PPA

vid_ppa is supported on ESP32-P4/ESP32S31 only, merging color conversion, scaling, cropping, and rotation into a single hardware pass. esp_gmf_video_ppa_init() requires no configuration; four setters configure the target format, target resolution, crop region, and rotation angle (only 0 / 90 / 180 / 270 degrees are supported). These setters can only be called before the element starts; open submits the combined parameters to hardware in one shot.

PPA internally selects the hardware path based on the current request: pure pixel format conversion can be completed by 2D-DMA with higher throughput; scaling, rotation, or format combinations not supported by 2D-DMA fall back to the main PPA path. Path selection is handled internally; the element only exposes a unified interface.

esp_gmf_element_handle_t ppa = NULL;
esp_gmf_video_ppa_init(NULL, &ppa);

esp_gmf_video_resolution_t dst_res = { .width = 480, .height = 320 };
esp_gmf_video_ppa_set_dst_resolution(ppa, &dst_res);
esp_gmf_video_ppa_set_dst_format(ppa, ESP_FOURCC_RGB16);
esp_gmf_video_ppa_set_rotation(ppa, 90);

esp_gmf_video_rgn_t crop = { .x = 0, .y = 0, .width = 800, .height = 600 };
esp_gmf_video_ppa_set_cropped_rgn(ppa, &crop);

On non-ESP32-P4 chips, vid_ppa is unavailable; use the four software effect elements in series (vid_color_cvt → vid_crop → vid_scale → vid_rotate) for equivalent functionality, at the cost of increased CPU usage.

Software Image Effects

The four software effect elements wrap the corresponding algorithms in the esp_image_effects library, exposing only their respective setters. All effects follow the same path: acquire_in to get the input frame, generate output frame size based on configuration, acquire_out to get the output frame, call esp_imgfx_*_process to process, then release both.

vid_color_cvt: Pixel format conversion, e.g., RGB565 → YUV420P, YUYV → RGB888. The color space standard (BT601 / BT709) is selected via the esp_imgfx_color_convert_cfg_t.color_space_std field at initialization. esp_gmf_video_color_convert_dst_format() switches the target format at runtime, setting need_recfg.

vid_crop: Crop element. Initialized with DEFAULT_ESP_GMF_CROP_CONFIG for a default 320×240 input and 160×120 crop region. Use esp_gmf_video_crop_rgn() at runtime to switch the crop rectangle (esp_gmf_video_rgn_t provides x / y / width / height).

vid_scale: Scale element. The filter_type field determines the algorithm (e.g., ESP_IMGFX_SCALE_FILTER_TYPE_DOWN_RESAMPLE). Use esp_gmf_video_scale_dst_resolution() at runtime to change the target resolution.

vid_rotate: Rotate element. Unlike vid_ppa which only supports four angles, software rotation supports arbitrary angles (in degrees), suitable for tilted display or image correction scenarios. esp_gmf_video_rotate_set_rotation() switches the angle at runtime.

esp_imgfx_scale_cfg_t cfg = DEFAULT_ESP_GMF_SCALE_CONFIG();
cfg.in_res       = (esp_imgfx_res_t){ .width = 640, .height = 480 };
cfg.in_pixel_fmt = ESP_IMGFX_PIXEL_FMT_RGB565_LE;
cfg.scale_res    = (esp_imgfx_res_t){ .width = 320, .height = 240 };

esp_gmf_element_handle_t scale = NULL;
esp_gmf_video_scale_init(&cfg, &scale);

Frame Rate and Overlay

vid_fps_cvt: Reduces input frame rate to a specified output frame rate by dropping frames. The element maintains a start PTS and accumulated output frame count, computing expected_pts at 1 / fps steps. For each arriving frame: the frame is passed to downstream and the count is incremented only if its PTS is >= expected_pts; otherwise it is dropped via release_in.

Since pure frame dropping cannot upsample, the value set by set_fps must be lower than the upstream frame rate; otherwise it has no effect.

esp_gmf_element_handle_t fps = NULL;
esp_gmf_video_fps_cvt_init(NULL, &fps);
esp_gmf_video_fps_cvt_set_fps(fps, 15);  /* Reduce from 30 fps to 15 fps */

vid_overlay: Overlays additional content onto the main video. The element has multiple inputs: the main video enters through the default in port, and the overlay layer enters through the extra port registered by the user via esp_gmf_video_overlay_set_overlay_port(). Compositing supports alpha blending and transparent-color (colorkey) modes. Alpha blending formula:

Output = Original × (255 − alpha) + Overlay × alpha

set_rgn specifies the position and format of the overlay layer on the main frame (esp_gmf_overlay_rgn_info_t contains format_id, dst_rgn, and optional has_trans_color / trans_color for colorkey compositing—pixels near the specified RGB value are treated as transparent); set_alpha changes transparency at runtime (0 fully transparent, 255 fully opaque); esp_gmf_video_overlay_enable()(false) temporarily disables the overlay, the element no longer acquires the overlay port, and the main frame passes through unchanged.

esp_gmf_element_handle_t ovl = NULL;
esp_gmf_video_overlay_init(NULL, &ovl);

esp_gmf_video_overlay_set_overlay_port(ovl, overlay_port);
esp_gmf_overlay_rgn_info_t rgn = {
    .format_id = ESP_FOURCC_RGB16,
    .dst_rgn   = { .x = 16, .y = 16, .width = 128, .height = 32 },
};
esp_gmf_video_overlay_set_rgn(ovl, &rgn);
esp_gmf_video_overlay_set_alpha(ovl, 200);

Performance

Video processing throughput is primarily determined by the underlying hardware / algorithm; the overhead of element scheduling and acquire-release is relatively small. By bottleneck category:

Category	Main Bottleneck	Optimization Direction
vid_dec / vid_enc	Throughput of hardware codec or software library	ESP32-P4 supports H.264 hardware encoding; call preset early to let the framework pre-allocate buffers
vid_ppa	PPA / 2D-DMA clock and buffer bandwidth	Merge adjacent pixel operations into the same vid_ppa instance to reduce intermediate buffers
Software effects	CPU and memory bandwidth	Prefer vid_ppa on ESP32-P4; control resolution to within SoC processing capacity for other SoCs
vid_fps_cvt	Only PTS comparison and frame dropping; very low CPU overhead	Suitable for placement before encoding or display to reduce downstream load
vid_overlay	CPU cycles for alpha blending or colorkey compositing	Smaller overlay region and fewer bytes per pixel format means faster processing

Application Examples

• elements/test_apps/main/elements/gmf_video_el_test.c: Typical video element test cases, including gen_pattern_color_bar for synthesizing test frames and format reporting validation at the open stage

• elements/test_apps/main/elements/gmf_image_effects_test.c: Image effects pipeline test, covering chaining of software effect elements

The upper-level application components esp_video_render and esp_capture also connect video pipelines through this component and can serve as advanced references.

SoC Compatibility

Support matrix of elements on Espressif SoCs:

element	ESP32	ESP32-S2	ESP32-S3	ESP32-P4
vid_ppa	Not supported	Not supported	Not supported	Supported
vid_fps_cvt	Supported	Supported	Supported	Supported
vid_overlay	Supported	Supported	Supported	Supported
vid_dec	MJPEG	MJPEG	SW H.264 / MJPEG	SW H.264 / HW MJPEG
vid_enc	MJPEG	MJPEG	SW H.264 / MJPEG	HW H.264 / HW MJPEG
vid_color_cvt / vid_crop / vid_scale / vid_rotate	Supported	Supported	Supported	Supported

ESP32 and ESP32-S2 do not support H.264. On ESP32-S3, H.264 encode and decode are implemented in software. On ESP32-P4, H.264 hardware encoding and MJPEG hardware encode/decode are supported; H.264 decoding is implemented in software. For ESP32-P4 PPA acceleration, omit the software effect elements and use vid_ppa instead.

FAQ

Q: How do I select the output format for vid_dec?

First use esp_gmf_video_dec_get_dst_formats() to query the list of output formats supported by the current source codec, then select a format that meets the input requirements of the downstream element (e.g., vid_ppa needs RGB565, display needs RGB888), and call esp_gmf_video_dec_set_dst_format() to fix the output format before starting the element. Output frame size queries require the target format to be set first, or the corresponding query interface to be called after the element starts.

Q: Cannot query output frame size before vid_enc starts?

Normally the encoder can only determine the output size after obtaining esp_gmf_info_video_t from upstream. To query before startup, first call esp_gmf_video_enc_preset() with the expected source info and target codec; then esp_gmf_video_enc_get_out_size() can return a value.

Q: Is vid_ppa supported on ESP32?

No. vid_ppa depends on ESP32-P4 hardware PPA / 2D-DMA. On other SoCs, use the software implementations vid_color_cvt / vid_scale / vid_rotate / vid_crop instead; resolution and frame rate should be constrained according to actual CPU capability.

Q: What happens if the frame rate set in vid_fps_cvt is higher than upstream?

The element does not insert frames. If set_fps is higher than the upstream frame rate, every frame’s PTS will be >= expected_pts, all frames pass through, and the output frame rate is limited by upstream. The configuration takes effect only when reducing the frame rate; it must not exceed the upstream rate.

Q: How are the two inputs of vid_overlay synchronized?

The main input is a regular GMF input port; the overlay input is the extra port registered by the user via set_overlay_port. The element acquires the main input first then the overlay input in each process; the application is responsible for ensuring the two streams are frame-aligned (e.g., two upstream elements in the same pipeline producing frames synchronously); otherwise overlay frames may lag or jitter.

Q: How to choose between software effect elements and vid_ppa on ESP32-P4?

When combining color conversion + scaling + rotation / cropping, prefer vid_ppa: a single hardware pass completes all conversions with low CPU usage and stable latency. Use the corresponding software elements when only a single operation is needed, or when the angle / pixel format required is not supported by PPA.

API Reference

Header files for this component:

• esp_gmf_video_dec.h / esp_gmf_video_enc.h: Video codec elements

• esp_gmf_video_ppa.h: Pixel processing accelerator

• esp_gmf_video_fps_cvt.h: Frame rate conversion

• esp_gmf_video_overlay.h: Overlay mixer

• esp_gmf_video_color_convert.h / esp_gmf_video_crop.h / esp_gmf_video_scale.h / esp_gmf_video_rotate.h: Software image effects

• esp_gmf_video_types.h: Shared types for video resolution / region / overlay configuration

• esp_gmf_video_param.h: Video parameter helper interface

The interface for the element base class esp_gmf_video_element_t is located in GMF Elements.

Header File

• elements/gmf_video/include/esp_gmf_video_dec.h

Functions

esp_gmf_err_t esp_gmf_video_dec_init(esp_gmf_video_dec_cfg_t *cfg, esp_gmf_element_handle_t *handle)

Initializes the GMF Video decoder with the provided configuration.

Parameters:

• cfg – [in] Pointer to the Video decoder configuration (optional)

• handle – [out] Pointer to the Video decoder handle to be initialized

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_dec_get_out_size(esp_gmf_element_handle_t handle, uint32_t *frame_size)

Get Video decoder output frame size.

Note

This API should only called after element running or esp_gmf_video_dec_set_dst_format is called

Parameters:

• handle – [in] Video decoder element handle

• frame_size – [out] Output frame size to store

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_dec_set_dst_format(esp_gmf_element_handle_t handle, uint32_t dst_fmt)

Setting for video decoder destination format (e.g., YUV420P, RGB565LE)

Note

This API should only called before element running

Parameters:

• handle – [in] Video decoder element handle

• dst_fmt – [in] Destination format (GMF FourCC representation of video codec)

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_dec_get_dst_formats(esp_gmf_element_handle_t handle, uint32_t src_codec, const uint32_t **dst_fmts, uint8_t *dst_fmts_num)

Get supported output formats for a given video decoder type This function retrieves the list of output formats (e.g., YUV420P, RGB565LE) that the decoder can produce when decoding from the specified source codec.

Parameters:

• handle – [in] Video decoder element handle

• src_codec – [in] Decode type (GMF FourCC representation of video codec)

• dst_fmts – [out] Output video formats to store (GMF FourCC representation of video format)

• dst_fmts_num – [out] Output video format number to store

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Structures

struct esp_gmf_video_dec_cfg_t

Video decoder configuration (optional)

Public Members

uint32_t codec_cc

FourCC code to force a specific decoder implementation See esp_video_codec_desc_t in esp_video_codec_types.h for details If zero, the system will select a suitable decoder (HW preferred)

Header File

• elements/gmf_video/include/esp_gmf_video_enc.h

Functions

esp_gmf_err_t esp_gmf_video_enc_init(esp_gmf_video_enc_cfg_t *cfg, esp_gmf_element_handle_t *handle)

Initializes the GMF video encoder with the provided configuration.

Parameters:

• cfg – [in] Pointer to the video encoder configuration

• handle – [out] Pointer to the video encoder handle to be initialized

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_enc_set_dst_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec)

Setting for video encoder destination codec (e.g., H.264, MJPEG)

Note

This API should only called before element running

Parameters:

• handle – [in] Video encoder element handle

• dst_codec – [in] Video encode destination codec (use GMF FourCC presentation)

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_preset(esp_gmf_element_handle_t handle, esp_gmf_info_video_t *src_info, uint32_t dst_codec)

Do presetting for video encoder.

Note

This function pre-configures the video encoder with both source and destination information This allows querying encoder capabilities before runtime Normally, source information is reported from dependent element in the running state Call this function only before element running

Parameters:

• handle – [in] Video encoder element handle

• src_info – [in] Video encode input video information

• dst_codec – [in] Destination video codec (use GMF FourCC presentation)

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_get_out_size(esp_gmf_element_handle_t handle, uint32_t *frame_size)

Get video encoder output frame size.

Note

This API should only be called after esp_gmf_video_enc_preset set or after element running

Parameters:

• handle – [in] Video encoder element handle

• frame_size – [out] Output frame size to store

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_get_src_formats(esp_gmf_element_handle_t handle, uint32_t dst_codec, const uint32_t **src_fmts, uint8_t *src_fmts_num)

Get supported source formats (e.g., YUV420P, RGB565LE etc) for a given video encoder type This function retrieves the list of supported input formats that can be encoded into the specified destination codec.

Parameters:

• handle – [in] Video encoder element handle

• dst_codec – [in] Video encode destination codec (GMF FourCC representation of video codec)

• src_fmts – [out] Source formats to stored (GMF FourCC representation of video format)

• src_fmts_num – [out] Source format number to store

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_bitrate(esp_gmf_element_handle_t handle, uint32_t bitrate)

Setting for video encoder bitrate.

Note

Support setting both before and after element running

Parameters:

• handle – [in] Video encoder element handle

• bitrate – [in] Output bitrate

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_gop(esp_gmf_element_handle_t handle, uint32_t gop)

Set the GOP (Group of Pictures) for video encoder.

Note

This setting applies specifically to the H.264 encoder The H.264 encoder generates an I-frame at the start of each GOP Can be configured both before and after the encoder element is running

Parameters:

• handle – [in] Handle to the video encoder element

• gop – [in] GOP size for H.264 (in number of frames)

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_enc_set_qp(esp_gmf_element_handle_t handle, uint32_t min_qp, uint32_t max_qp)

Set the QP (Quantization Parameter) range for video encoder.

Note

This setting applies specifically to the H.264 encoder Higher QP values generally result in lower image quality Can be configured both before and after the encoder element is running

Parameters:

• handle – [in] Handle to the video encoder element

• min_qp – [in] Minimum QP value

• max_qp – [in] Maximum QP value

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Structures

struct esp_gmf_video_enc_cfg_t

Video encoder configuration.

Public Members

uint32_t codec_cc

FourCC code to force a specific encoder implementation See esp_video_codec_desc_t in esp_video_codec_types.h for details If zero, the system will select a suitable encoder (HW preferred)

Header File

• elements/gmf_video/include/esp_gmf_video_ppa.h

Functions

esp_gmf_err_t esp_gmf_video_ppa_init(void *config, esp_gmf_element_handle_t *handle)

Video PPA (Pixel Processing Accelerator) is a compact module Currently it only support on ESP32P4 and including following video processing: 1) Color convert 2) Scale 3) Rotate 5) Crop These operations can be mixed so that can do them all together for better performance.

Initializes the GMF Video PPA

Parameters:

• config – [in] No need to set

• handle – [out] Video PPA handle to store

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

Set video PPA destination resolution.

Note

This API should only called before element running

Parameters:

• handle – [in] Video PPA handle

• res – [in] Video PPA output resolution

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_dst_format(esp_gmf_element_handle_t handle, uint32_t format)

Set video PPA destination format.

Note

This API should only called before element running

Parameters:

• handle – [in] Video PPA handle

• format – [in] Video PPA destination format (GMF FourCC representation of video format)

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_cropped_rgn(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

Set video PPA cropped region.

Note

This API should only called before element running

Parameters:

• handle – [in] Video PPA handle

• rgn – [in] Region to be kept in original video frame

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_ppa_set_rotation(esp_gmf_element_handle_t handle, uint16_t degree)

Set video PPA rotation degree.

Note

This API should only called before element running

Parameters:

• handle – [in] Video PPA handle

• degree – [in] Currently only support (0, 90, 180, 270) unit(one degree)

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Header File

• elements/gmf_video/include/esp_gmf_video_overlay.h

Functions

esp_gmf_err_t esp_gmf_video_overlay_init(void *config, esp_gmf_element_handle_t *handle)

Initializes the GMF overlay mixer with the provided configuration.

Diagram: +————————-—+ | Original Plane | | (Background) | | +———————+ | | | Overlay Plane | | | | (Foreground) | | | +———————+ | +————————-—+

Note

This module mixes the overlay plane (foreground) into the original plane (background) using alpha blending The formula is: Output_Plane_Pixel = Original_Plane_Pixel * (255 - alpha) + Overlay_Plane_Pixel * alpha where alpha is the transparency level (0 = fully transparent, 255 = fully opaque)

Parameters:

• config – [in] No need to set

• handle – [out] Pointer to the overlay mixer handle to be initialized

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_rgn(esp_gmf_element_handle_t handle, esp_gmf_overlay_rgn_info_t *rgn_info)

Set the output region for video overlay.

Note

This API should only called before element running

Parameters:

• handle – [in] Video overlay element handle

• rgn_info – [out] Region information

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_overlay_port(esp_gmf_element_handle_t handle, esp_gmf_port_handle_t port)

Set the GMF port for video overlay input.

Note

The video overlay element will use this port to acquire overlay data, mix it into the original video plane, and then release the port The port is managed by the user, the overlay element only accesses it This API can be called before element running and allow reset

Parameters:

• handle – [in] Video overlay element handle

• port – [in] GMF port for video overlay data

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_overlay_set_alpha(esp_gmf_element_handle_t handle, uint8_t alpha)

Set the alpha value of the overlay window.

Note

Allow to set both before and after element running

Parameters:

• handle – [in] Video overlay element handle

• alpha – [in] Overlay window alpha setting: 0 means fully transparent, 255 means fully opaque

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

esp_gmf_err_t esp_gmf_video_overlay_enable(esp_gmf_element_handle_t handle, bool enable)

Enable overlay mixer or not.

Note

When overlay mixer is disabled will not acquire data from overlay port and do mixer anymore Original plane data leave unchanged Allow to set both before and after element running

Parameters:

• handle – [in] Video overlay mixer handle

• enable – [in] Enable mixer or not

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Header File

• elements/gmf_video/include/esp_gmf_video_fps_cvt.h

Functions

esp_gmf_err_t esp_gmf_video_fps_cvt_init(void *config, esp_gmf_element_handle_t *handle)

Initializes the GMF video frame rate convert.

Parameters:

• config – [in] No need to set

• handle – [out] Pointer to the video frame rate convert handle to be initialized

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

esp_gmf_err_t esp_gmf_video_fps_cvt_set_fps(esp_gmf_element_handle_t handle, uint16_t fps)

Set GMF video frame rate converter output frame rate.

Note

This API should only called before element running

Parameters:

• handle – [in] Video frame rate converter handle

• fps – [out] Destination frame rate to set (unit frame per second)

Returns:

• ESP_GMF_ERR_OK Success

• ESP_GMF_ERR_INVALID_ARG Invalid configuration provided

Header File

• elements/gmf_video/include/esp_gmf_video_color_convert.h

Functions

esp_gmf_err_t esp_gmf_video_color_convert_init(esp_imgfx_color_convert_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video color convert element with the specified configuration.

Parameters:

• config – [in] Pointer to a color convert configuration

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video color convert handle will be stored

Returns:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video color convert element

esp_gmf_err_t esp_gmf_video_color_convert_dst_format(esp_gmf_element_handle_t handle, uint32_t format)

Set video color convert destination format.

Note

This API have no special timing request, user can call it freely

Parameters:

• handle – [in] Video color convert handle

• format – [in] Video color convert destination format (GMF FourCC representation of video format)

Returns:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_COLOR_CONVERT_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_crop.h

Functions

esp_gmf_err_t esp_gmf_video_crop_init(esp_imgfx_crop_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video crop element with the specified configuration.

Parameters:

• config – [in] Pointer to a esp_imgfx_crop_cfg_t structure that contains the configuration settings for the video crop

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video crop handle will be stored

Returns:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video crop element

esp_gmf_err_t esp_gmf_video_crop_rgn(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

Set video cropped region.

Note

This API have no special timing request, user can call it freely

Parameters:

• handle – [in] Video video crop handle

• rgn – [in] Region to be kept in original video frame

Returns:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_CROP_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_scale.h

Functions

esp_gmf_err_t esp_gmf_video_scale_init(esp_imgfx_scale_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video scale element with the specified configuration.

Parameters:

• config – [in] Pointer to a esp_imgfx_scale_cfg_t structure that contains the configuration settings for the video scale

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video scale handle will be stored

Returns:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video scale element

esp_gmf_err_t esp_gmf_video_scale_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

Set video scale destination resolution.

Note

This API have no special timing request, user can call it freely

Parameters:

• handle – [in] Video video scale handle

• res – [in] Video video scale output resolution

Returns:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_SCALE_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_rotate.h

Functions

esp_gmf_err_t esp_gmf_video_rotate_init(esp_imgfx_rotate_cfg_t *config, esp_gmf_element_handle_t *handle)

Initialize the video rotate element with the specified configuration.

Parameters:

• config – [in] Pointer to a esp_imgfx_rotate_cfg_t structure that contains the configuration settings for the video rotate

• handle – [out] Pointer to a esp_gmf_element_handle_t where the initialized video rotate handle will be stored

Returns:

• ESP_GMF_ERR_OK Initialization successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

• ESP_GMF_ERR_MEMORY_LACK No memory for video rotate element

esp_gmf_err_t esp_gmf_video_rotate_set_rotation(esp_gmf_element_handle_t handle, uint16_t degree)

Set video rotate rotation degree.

Note

This API have no special timing request, user can call it freely

Parameters:

• handle – [in] Video video rotate handle

• degree – [in] Any angle, unit(one degree)

Returns:

• ESP_GMF_ERR_OK Successful

• ESP_GMF_ERR_INVALID_ARG Invalid arguments

Macros

DEFAULT_ESP_GMF_ROTATE_CONFIG()

Header File

• elements/gmf_video/include/esp_gmf_video_types.h

Structures

struct esp_gmf_video_resolution_t

Video resolution definition.

Public Members

uint16_t width

Width of the video in pixels

uint16_t height

Height of the video in pixels

struct esp_gmf_video_rgn_t

Video region definition Represents a rectangular region of the video frame.

Public Members

uint16_t x

X-coordinate of the top-left corner

uint16_t y

Y-coordinate of the top-left corner

uint16_t width

Width of the region in pixels

uint16_t height

Height of the region in pixels

struct esp_gmf_overlay_rgn_info_t

Specifies the format and destination region for a video overlay.

Note

This structure defines the parameters needed to position and format an overlay on a video frame

Public Members

uint32_t format_id

FourCC of overlay region frame

esp_gmf_video_rgn_t dst_rgn

Region position to put the overlay

uint8_t rgn_index

Region index start from 0 (first region)

bool has_trans_color

Whether region has transparent color (if near it will become transparent)

uint8_t trans_color[3]

Transparent color RGB order

Header File

• elements/gmf_video/include/esp_gmf_video_param.h

Functions

esp_gmf_err_t esp_gmf_video_param_set_dst_format(esp_gmf_element_handle_t handle, uint32_t dst_fmt)

This file provides general parameter setting functions for common video processing using GMF method.

    These functions simplify setting parameters for video processing elements,
    handling the underlying method calls. They do not enforce a specific order
    or argument count, but apply the basic settings where supported. Refer to
    the individual element APIs for details on supported settings and timing.

Set destination format for video element (e.g., color converter, video decoder)

Parameters:

• handle – [in] Video element handle

• dst_fmt – [in] Destination format

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_fps(esp_gmf_element_handle_t handle, uint16_t fps)

Set frame rate for video element (mainly for FPS converter)

Parameters:

• handle – [in] Video element handle

• fps – [in] Destination frame rate

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_dst_resolution(esp_gmf_element_handle_t handle, esp_gmf_video_resolution_t *res)

Set destination resolution for video element (mainly for video scaler)

Parameters:

• handle – [in] Video element handle

• res – [in] Destination resolution

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_dst_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec)

Set destination codec for video element (e.g., video encoder)

Parameters:

• handle – [in] Video element handle

• dst_codec – [in] Destination codec

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_venc_preset(esp_gmf_element_handle_t handle, esp_gmf_info_video_t *vid_info, uint32_t dst_codec)

Preset source video information and destination codec for video encoder.

Note

In special cases call this API then query some information from video encoder before element running

Parameters:

• handle – [in] Video element handle

• vid_info – [in] Source video information

• dst_codec – [in] Destination codec

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_get_src_fmts_by_codec(esp_gmf_element_handle_t handle, uint32_t dst_codec, const uint32_t **src_fmts, uint8_t *src_fmts_num)

Get supported source formats by codec for video element (mainly for video encoder)

Parameters:

• handle – [in] Video element handle

• dst_codec – [in] Destination codec

• src_fmts – [in] Pointer to store source formats array

• src_fmts_num – [in] Pointer to store source formats number

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_src_codec(esp_gmf_element_handle_t handle, uint32_t src_codec)

Set source codec for video element (e.g., video decoder)

Parameters:

• handle – [in] Video element handle

• src_codec – [in] Source codec

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_rotate_angle(esp_gmf_element_handle_t handle, uint16_t degree)

Set the rotation angle for the video element (mainly for video rotator)

Parameters:

• handle – [in] Video element handle

• degree – [in] Rotation angle in degrees (e.g., 0, 90, 180, 270)

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_cropped_region(esp_gmf_element_handle_t handle, esp_gmf_video_rgn_t *rgn)

Set the cropped region for video element (mainly for video cropper)

Parameters:

• handle – [in] Video element handle

• rgn – [in] Kept region after crop

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_overlay_enable(esp_gmf_element_handle_t self, bool enable)

Enable overlay for video element (mainly for video overlay)

Parameters:

• handle – [in] Video element handle

• enable – [in] Whether enable video overlay or not

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_overlay_port(esp_gmf_element_handle_t self, void *port)

Set overlay port for video element (mainly for video overlay)

Parameters:

• handle – [in] Video element handle

• port – [in] Handle to the input GMF port providing overlay data

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_set_overlay_rgn(esp_gmf_element_handle_t self, esp_gmf_overlay_rgn_info_t *rgn)

Set overlay region information for video element (mainly for video overlay)

Parameters:

• handle – [in] Video element handle

• rgn – [in] Overlay region information

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

esp_gmf_err_t esp_gmf_video_param_get_dst_fmts_by_codec(esp_gmf_element_handle_t handle, uint32_t src_codec, const uint32_t **dst_fmts, uint8_t *dst_fmts_num)

Get supported destination formats by codec for video element (mainly for video decoder)

Parameters:

• handle – [in] Video element handle

• src_codec – [in] Source codec

• src_fmts – [in] Pointer to store source formats array

• src_fmts_num – [in] Pointer to store source formats number

Returns:

• ESP_GMF_ERR_OK On success

• ESP_GMF_ERR_NOT_FOUND Not found the method

• ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory

• Others Failed to apply method

---

Was this page helpful?

• Thank you! We received your feedback.
• If you have any comments, fill in Espressif Documentation Feedback Form.

• We value your feedback.
• Let us know how we can improve this page by filling in Espressif Documentation Feedback Form.