Multimedia Examples

[中文]

This page categorizes multimedia examples by function, covering playback, recording and capture, audio algorithms, display and rendering, Bluetooth audio, cloud agents, communication protocols, checks, and system services.

Playback Examples

Playback examples demonstrate a complete media playback flow: reading and decoding from an SD card, Flash, HTTP, HLS, or MIDI, then outputting to the speaker or display, with touchscreen or serial CLI control.

Example	Description	Component Link
pipeline_play_sdcard_music	Plays music from an SD card, with decode and audio effects. Supports MP3/WAV/FLAC/AAC and other formats.	gmf_examples
pipeline_play_multi_source_music	Switches between HTTP and SD card sources and can insert a Flash tone. Playback of the original track resumes after the tone finishes.	gmf_examples
pipeline_loop_play_no_gap	Uses a GMF task policy to loop multiple SD card tracks with no noticeable gap when switching.	gmf_examples
pipeline_play_embed_music	Plays Flash audio embedded in firmware. It does not require an SD card or network and is suitable for boot tones.	gmf_examples
pipeline_play_http_music	Connects to Wi-Fi, fetches an HTTP URL, then decodes and plays the stream. m3u8/HLS is not supported.	gmf_examples
audio_player	Implements full audio playback with esp_player: demux, decode, and playback control. It can mix 2–4 streams to one DAC.	esp_player
video_player (esp_player)	Plays local audio and video with esp_player: demux, decode, and A/V sync. Video goes to the LCD and audio to the speaker.	esp_player
play_music_control	Controls multi-source playback from a serial CLI: SD card, HTTP/HTTPS, and Flash tones, with playlist support.	adf_examples
music_player	Scans MP3/AAC/WAV files on the SD card and shows a playlist and playback controls on a touchscreen. Supports single-track, list, and shuffle modes.	adf_examples
hls_live_stream	Parses an HLS (m3u8) live stream and decodes it for playback. Suitable for internet radio and live audio.	esp_hls_stream
midi_play	Plays MIDI files from the SD card.	esp_midi
midi_keyboard	Receives MIDI messages in real time and synthesizes sound, implementing a simple electronic keyboard.	esp_midi
play_embed_music	Board example: plays a WAV file embedded in firmware. Used to verify board audio output.	esp_board_manager
play_sdcard_music	Board example: plays WAV from the SD card.	esp_board_manager

Recording and Capture Examples

Recording and capture examples demonstrate microphone and camera capture. Captured data can be encoded and written to an SD card, uploaded over HTTP, or previewed on a local display.

Example	Description	Component Link
pipeline_record_sdcard	Records from the microphone to the SD card. Supports PCM/MP3/AAC and other formats, with optional low-power recording.	gmf_examples
pipeline_record_http	Captures and encodes microphone audio, then uploads it to a server over HTTP. Suitable for cloud recording or a speech-recognition frontend.	gmf_examples
pipeline_record_audio_muxer	Encodes a recording and muxes it into an MP4/TS container before writing it to the SD card, instead of saving a raw audio stream only.	gmf_examples
aec_rec	Plays an SD card MP3 on one path while the microphone path runs AEC, then encodes and saves the result for echo-cancellation comparison.	gmf_ai_audio
audio_capture	Implements audio capture with esp_capture, covering basic recording, AEC, file writing, and custom processing without building a pipeline manually.	esp_capture
video_capture	Implements video capture with esp_capture: single frame, recording, overlay, and dual-stream, with optional SD card storage.	esp_capture
av_record_live_display	Captures camera and microphone data, shows a live LCD preview, and records audio and video as MP4 on the SD card.	adf_examples
live_photo_capture	Captures a short video, picks a cover frame, and writes a Motion Photo style file to the SD card.	esp_live_photo
video_muxer	Muxes camera and microphone streams with ESP Muxer and saves the result to the SD card.	esp_muxer
record_to_sdcard	Board example: records from the microphone to the SD card.	esp_board_manager
record_and_play	Board example: plays captured microphone audio on the speaker in real time to verify the audio path.	esp_board_manager

Audio Algorithm Examples

Audio algorithm examples demonstrate common audio processing features, including effects, howling suppression, wake-word and command recognition, sample-rate conversion, and spectrum analysis.

Example	Description	Component Link
pipeline_audio_effects	Demonstrates resampling, Sonic time-stretch/pitch-shift, at least 4-way mixing, EQ, and DRC on multiple GMF pipelines.	gmf_examples
pipeline_howl	Karaoke setup with three pipelines: SD accompaniment and microphone are processed separately, then mixed after howling suppression and played from the DAC.	gmf_examples
wwe (wake word and commands)	Implements wake-word, VAD, and command recognition with AFE and can run offline.	gmf_ai_audio
asrc_demo	Converts sample rate, channel count, and bit depth with ASRC so record or playback formats match the target specification.	esp_asrc
fft_spectrum_print	Generates a multi-tone cosine signal, runs a 512-point Q15 real FFT, prints a spectrum bar graph over UART, then checks round-trip error with IFFT.	gmf_fft
ae_howl	Applies esp_ae_howl howling suppression to the microphone, then mixes it with SD card background music.	esp_audio_effects
esp_audio_effects_demo	Demonstrates ALC, Fade, EQ, Sonic, DRC, MBC, Mixer, and basic format conversion one by one.	esp_audio_effects

Display and Render Examples

Display and render examples demonstrate how to send video frames and PCM audio to the LCD and DAC, covering dual-eye synchronized display, overlays, multi-stream mixing, and LVGL UI verification on the board.

Example	Description	Component Link
dual_eyes	Renders left and right eye streams in sync. Supports side-by-side on one screen or dual-screen output.	esp_video_render
video_player (esp_video_render)	Builds a lightweight local player with esp_video_render: extract, decode, and render from the SD card, with an overlay UI.	esp_video_render
video_render	Reads MJPEG from the SD card and demonstrates single-stream/multi-stream render, scaling, and a progress-bar overlay.	esp_video_render
audio_render	Implements single-stream PCM playback and multi-stream mixing with esp_audio_render. It can also play network-decoded audio.	esp_audio_render
simple_piano	Implements a polyphonic piano with esp_audio_render: multiple tracks sound together and render to the DAC in real time.	esp_audio_render
display_lvgl	Initializes the display with board manager and runs an LVGL test UI to verify display and touch.	esp_board_manager
av_render_test	Captures camera and microphone data and outputs them to the LCD and I2S together to verify the A/V render path.	av_render

Bluetooth Audio Examples

Bluetooth audio examples demonstrate classic Bluetooth audio, covering A2DP playback, HFP calls, and AVRCP control, with optional LE Audio.

Example	Description	Component Link
bt_audio	One project covers A2DP Source/Sink, HFP calls, and AVRCP control. LE Audio, BIS, and an LVGL touch UI are optional.	esp_bt_audio

Cloud Agent Examples

Cloud agent examples demonstrate connecting a device to a cloud voice agent, covering real-time voice interaction with Coze, Baidu RTC, and OpenAI Realtime.

Example	Description	Component Link
coze_ws_app	Connects to the Coze WebSocket OpenAPI for two-way streaming voice, with direct talk, wake word, and button barge-in.	adf_examples
voice_assistant_app	Implements a Baidu RTC voice assistant: conversation, network music playback, and two-way audio mixing.	adf_examples
openai_demo	Connects to OpenAI Realtime with esp_webrtc for live voice conversation and demonstrates function-call device control.	esp-webrtc-solution

Protocol Examples

Protocol examples demonstrate media transport over the network, including HTTP download and real-time streaming or calls over WebRTC, RTSP, and RTMP.

Example	Description	Component Link
pipeline_http_download_to_sdcard	Downloads a file from a URL to the SD card and prints overall download and write speed when finished.	gmf_examples
peer_demo	Establishes a connection between two ESP boards with esp_peer and implements a simple chat over dummy audio and a data channel.	esp-webrtc-solution
esp_peer_demo	Component-level esp_peer demo: two-role real-time communication with an audio stream and data-channel messages.	esp_peer
whip_demo	Pushes device audio and video to a WHIP server.	esp-webrtc-solution
kvs_master	Runs as an Amazon KVS MASTER: receives an SDP offer from a browser VIEWER and returns an answer.	esp-webrtc-solution
kms_demo	Publishes WebRTC audio and video to a Kurento Media Server so a browser can watch through KMS.	esp-webrtc-solution
janus_demo	Publishes media as a Janus VideoRoom publisher using Janus HTTP signaling.	esp-webrtc-solution
doorbell_demo	Smart doorbell over AppRTC WebSocket signaling: remote control, live video, and two-way audio.	esp-webrtc-solution
doorbell_local	Local doorbell with HTTPS signaling hosted on the device. Includes live video, two-way audio, and optional pedestrian detection.	esp-webrtc-solution
videocall_demo	Video call between two devices over a modified AppRTC signaling path. Media uses an esp_webrtc data channel.	esp-webrtc-solution
webrtc_usb_camera	The browser sends a WebRTC stream; the host enumerates the device as a USB UVC camera.	esp-webrtc-solution
local_jpeg_stream	Hosts HTTPS signaling on the device and sends camera JPEG to a browser over a WebRTC data channel, with two-way audio.	esp-webrtc-solution
rtsp_demo	After Wi-Fi connects, starts an RTSP server or pusher on the device for LAN streaming.	esp-webrtc-solution
rtmp_demo	Captures device audio and video and pushes them to an RTMP server through esp_media_protocols.	esp-webrtc-solution

Check and Benchmark Examples

Check and benchmark examples verify recording quality, extract media frames, and benchmark components such as playlists.

Example	Description	Component Link
esp_audio_analyzer_app	Captures MIC and loopback together, then saves to the SD card or uploads over Wi-Fi for hardware recording quality and acoustic checks.	adf_examples
extractor_test	Extracts audio and video frames from media files with esp_extractor to verify demux and frame extraction.	esp_extractor
playlist_benchmark	Benchmarks esp_playlist operations on the SD card and prints average time per operation over UART.	esp_playlist

System Service Examples

System service examples demonstrate device-side system features, including low-power sleep, service orchestration, provisioning, configuration management, and OTA updates.

Example	Description	Component Link
audio_power_save	Enters light sleep automatically after connecting, using MQTT keepalive. UART, MQTT, GPIO, or a timer can wake the device, and a sleep tone is played.	adf_examples
services_hub	Orchestrates Wi-Fi, OTA, CLI, and button services through the event hub and service manager. MCP is optional.	adf_examples
adf_event_hub_example	Connects five independent services through the event hub and prints the full event stream over UART. The same sources also build on PC.	adf_event_hub
button_svc_example	Combines a single GPIO button and an ADC button group, and demonstrates full event subscription.	esp_button_service
esp_cli_service_example	Controls the device from the serial console.	esp_cli_service
mock_services	Six mock services demonstrate esp_service create, start, pause/resume, destroy, and event subscription.	esp_service
wifi_service_example	Creates a Wi-Fi service, stores profiles encrypted, starts provisioning, and controls the connection from the CLI.	esp_wifi_service
config_manager_example	Saves multiple application config groups to persistent storage, demonstrating dual slots, default merge, and CRC checks.	esp_config_manager
ota_http	Pulls firmware over HTTP with manifest version checks, SHA-256 verification, and resumable download.	esp_ota_service
ota_fs	Performs offline OTA from an SD card or USB drive, including multi-partition batch upgrades.	esp_ota_service
ota_ble	Pushes firmware over BLE GATT when Wi-Fi is unavailable, using the official ESP BLE OTA app.	esp_ota_service

Provide feedback about this document