Audio

[中文]

  • Component registry: espressif/brookesia_service_audio

  • Helper header: #include "brookesia/service_helper/media/audio.hpp"

  • Helper classes: esp_brookesia::service::helper::AudioPlayback, AudioEncoder<0>, AudioDecoder<0>

Overview

brookesia_service_audio provides:

  • Playback: Stream from URL; pause, resume, stop.

  • Codecs: PCM, OPUS, G711A encode/decode.

  • Playback state: Idle / playing / paused with events.

  • Encoder: Start/stop/configure; configurable read size.

  • Decoder: Start/stop and feed compressed data; streaming decode.

Features

Codec Formats

The audio service supports the following codec formats:

Format

Encode

Decode

Notes

PCM

Yes

Yes

Lossless

OPUS

Yes

Yes

VBR and fixed bitrate

G711A

Yes

Yes

Telephony quality

Playback

  • URL playback.

  • Pause, resume, stop.

  • State events for UI sync.

Encoder Configuration

  • Codecs: PCM, OPUS, G711A.

  • Channels: 1–4.

  • Bits per sample: 8, 16, 24, 32.

  • Sample rates: 8000, 16000, 24000, 32000, 44100, 48000 Hz.

  • Frame duration (ms).

  • OPUS: VBR and bitrate.

Decoder Configuration

  • Codecs: PCM, OPUS, G711A.

  • Channels: 1–4.

  • Bits per sample: 8, 16, 24, 32.

  • Sample rates: 8000, 16000, 24000, 32000, 44100, 48000 Hz.

  • Frame duration (ms).

Events

  • Playback state changes (Idle, Playing, Paused).

  • Encoder events.

  • Encoded data ready.

Standard Include / Helper Class

  • Standard include: #include \"brookesia/service_helper/media/audio.hpp\"

  • Helper class: esp_brookesia::service::helper::Audio

Service Interfaces

Functions

Playback

Play
Description

Play audio from a URL. Supports loop and interrupt playback.

Execution
  • Requires scheduler: Required

Parameters
  • Url

  • Config

    • Type: Object

    • Required: optional

    • Default: {"interrupt":true,"delay_ms":0,"loop_count":0,"loop_interval_ms":0,"timeout_ms":0}

    • Description: Playback config. Example: {"interrupt":true,"delay_ms":0,"loop_count":0,"loop_interval_ms":0,"timeout_ms":0}

Schema JSON
Show raw JSON

{
  "name": "Play",
  "description": "Play audio from a URL. Supports loop and interrupt playback.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Url",
      "description": "Audio URL, for example: \"file://littlefs/example.mp3\".",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "Config",
      "description": "Playback config. Example: {\"interrupt\":true,\"delay_ms\":0,\"loop_count\":0,\"loop_interval_ms\":0,\"timeout_ms\":0}",
      "type": "Object",
      "required": false,
      "default_value": {
        "interrupt": true,
        "delay_ms": 0,
        "loop_count": 0,
        "loop_interval_ms": 0,
        "timeout_ms": 0
      }
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioPlayback Play {"Url":null,"Config":null}
PlayUrls
Description

Play audio from multiple URLs. Supports loop and interrupt playback.

Execution
  • Requires scheduler: Required

Parameters
  • Urls

  • Config

    • Type: Object

    • Required: optional

    • Default: {"interrupt":true,"delay_ms":0,"loop_count":0,"loop_interval_ms":0,"timeout_ms":0}

    • Description: Playback config. Example: {"interrupt":true,"delay_ms":0,"loop_count":0,"loop_interval_ms":0,"timeout_ms":0}

Schema JSON
Show raw JSON

{
  "name": "PlayUrls",
  "description": "Play audio from multiple URLs. Supports loop and interrupt playback.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Urls",
      "description": "Audio URL list. Example: [\"file://littlefs/example1.mp3\",\"file://littlefs/example2.mp3\"]",
      "type": "Array",
      "required": true,
      "default_value": null
    },
    {
      "name": "Config",
      "description": "Playback config. Example: {\"interrupt\":true,\"delay_ms\":0,\"loop_count\":0,\"loop_interval_ms\":0,\"timeout_ms\":0}",
      "type": "Object",
      "required": false,
      "default_value": {
        "interrupt": true,
        "delay_ms": 0,
        "loop_count": 0,
        "loop_interval_ms": 0,
        "timeout_ms": 0
      }
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioPlayback PlayUrls {"Urls":null,"Config":null}
Pause
Description

Pause playback.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "Pause",
  "description": "Pause playback.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioPlayback Pause
Resume
Description

Resume playback.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "Resume",
  "description": "Resume playback.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioPlayback Resume
Stop
Description

Stop playback.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "Stop",
  "description": "Stop playback.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioPlayback Stop
SetVolume
Description

Set target playback volume percentage.

Execution
  • Requires scheduler: Required

Parameters
  • Volume

    • Type: Number

    • Required: required

    • Description: Playback volume percentage in range [0, 100].

Schema JSON
Show raw JSON

{
  "name": "SetVolume",
  "description": "Set target playback volume percentage.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Volume",
      "description": "Playback volume percentage in range [0, 100].",
      "type": "Number",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioPlayback SetVolume {"Volume":null}
GetVolume
Description

Get target playback volume percentage [0, 100].

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Return Value
  • Type: Number

  • Description: Playback volume percentage in range [0, 100].

Schema JSON
Show raw JSON

{
  "name": "GetVolume",
  "description": "Get target playback volume percentage [0, 100].",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Number",
    "description": "Playback volume percentage in range [0, 100]."
  }
}
CLI Command
svc_call AudioPlayback GetVolume
SetMute
Description

Set whether audio playback is muted.

Execution
  • Requires scheduler: Required

Parameters
  • Enable

    • Type: Boolean

    • Required: required

    • Description: True to mute playback, false to unmute it.

Schema JSON
Show raw JSON

{
  "name": "SetMute",
  "description": "Set whether audio playback is muted.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Enable",
      "description": "True to mute playback, false to unmute it.",
      "type": "Boolean",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioPlayback SetMute {"Enable":null}
GetMute
Description

Get whether audio playback is muted.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Return Value
  • Type: Boolean

  • Description: True when muted, false when unmuted.

Schema JSON
Show raw JSON

{
  "name": "GetMute",
  "description": "Get whether audio playback is muted.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Boolean",
    "description": "True when muted, false when unmuted."
  }
}
CLI Command
svc_call AudioPlayback GetMute
LoadData
Description

Load persisted audio playback state, including volume and mute.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "LoadData",
  "description": "Load persisted audio playback state, including volume and mute.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioPlayback LoadData
ResetData
Description

Reset persisted audio playback state, including volume and mute.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "ResetData",
  "description": "Reset persisted audio playback state, including volume and mute.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioPlayback ResetData

Encoder

Start
Description

Start audio encoder.

Execution
  • Requires scheduler: Required

Parameters
  • Config

    • Type: Object

    • Required: required

    • Description: Audio encoder config. Example: {"type":"PCM","general":{"channels":0,"sample_bits":0,"sample_rate":0,"frame_duration":0},"extra":null,"fetch_interval_ms":10,"fetch_data_size":4096,"enable_afe":false,"afe_wake_start_timeout_ms":30000,"afe_wake_end_timeout_ms":10000}

Schema JSON
Show raw JSON

{
  "name": "Start",
  "description": "Start audio encoder.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Config",
      "description": "Audio encoder config. Example: {\"type\":\"PCM\",\"general\":{\"channels\":0,\"sample_bits\":0,\"sample_rate\":0,\"frame_duration\":0},\"extra\":null,\"fetch_interval_ms\":10,\"fetch_data_size\":4096,\"enable_afe\":false,\"afe_wake_start_timeout_ms\":30000,\"afe_wake_end_timeout_ms\":10000}",
      "type": "Object",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioEncoder0 Start {"Config":null}
Stop
Description

Stop audio encoder.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "Stop",
  "description": "Stop audio encoder.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioEncoder0 Stop
Pause
Description

Pause audio encoder.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "Pause",
  "description": "Pause audio encoder.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioEncoder0 Pause
Resume
Description

Resume audio encoder.

Execution
  • Requires scheduler: Required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "Resume",
  "description": "Resume audio encoder.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioEncoder0 Resume
PauseWakeEnd
Description

Pause pending AFE WakeEnd event without pausing audio capture.

Execution
  • Requires scheduler: Not required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "PauseWakeEnd",
  "description": "Pause pending AFE WakeEnd event without pausing audio capture.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioEncoder0 PauseWakeEnd
ResumeWakeEnd
Description

Resume pending AFE WakeEnd event without resuming audio capture.

Execution
  • Requires scheduler: Not required

Parameters
  • No parameters.

Schema JSON
Show raw JSON

{
  "name": "ResumeWakeEnd",
  "description": "Resume pending AFE WakeEnd event without resuming audio capture.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AudioEncoder0 ResumeWakeEnd
GetAFEWakeWords
Description

Get AFE wake words.

Execution
  • Requires scheduler: Not required

Parameters
  • No parameters.

Return Value
  • Type: Array

  • Description: Example: ["ni hao xiao zhi","hello brookesia"]

Schema JSON
Show raw JSON

{
  "name": "GetAFEWakeWords",
  "description": "Get AFE wake words.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Array",
    "description": "Example: [\"ni hao xiao zhi\",\"hello brookesia\"]"
  }
}
CLI Command
svc_call AudioEncoder0 GetAFEWakeWords

Decoder

GetOutputs
Description

Get audio output list.

Execution
  • Requires scheduler: Not required

Parameters
  • No parameters.

Return Value
  • Type: Array

  • Description: Example: [{"id":0,"name":"Speaker0","role":"speaker","sample_rates":[16000,22050,44100,48000],"channels":[1,2],"sample_bits":[16]}]

Schema JSON
Show raw JSON

{
  "name": "GetOutputs",
  "description": "Get audio output list.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Array",
    "description": "Example: [{\"id\":0,\"name\":\"Speaker0\",\"role\":\"speaker\",\"sample_rates\":[16000,22050,44100,48000],\"channels\":[1,2],\"sample_bits\":[16]}]"
  }
}
CLI Command
svc_call AudioDecoder0 GetOutputs
GetSources
Description

Get registered audio sources.

Execution
  • Requires scheduler: Not required

Parameters
  • No parameters.

Return Value
  • Type: Array

  • Description: Example: [{"id":1,"name":"TTS","role":"speech","preferred_outputs":["Speaker0"],"priority":10}]

Schema JSON
Show raw JSON

{
  "name": "GetSources",
  "description": "Get registered audio sources.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Array",
    "description": "Example: [{\"id\":1,\"name\":\"TTS\",\"role\":\"speech\",\"preferred_outputs\":[\"Speaker0\"],\"priority\":10}]"
  }
}
CLI Command
svc_call AudioDecoder0 GetSources
RegisterSource
Description

Register an audio source.

Execution
  • Requires scheduler: Not required

Parameters
  • Source

    • Type: Object

    • Required: required

    • Description: Source info. Example: {"id":0,"name":"TTS","role":"speech","preferred_outputs":["Speaker0"],"priority":0}

Return Value
  • Type: Number

  • Description: Registered source id.

Schema JSON
Show raw JSON

{
  "name": "RegisterSource",
  "description": "Register an audio source.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Source",
      "description": "Source info. Example: {\"id\":0,\"name\":\"TTS\",\"role\":\"speech\",\"preferred_outputs\":[\"Speaker0\"],\"priority\":0}",
      "type": "Object",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": {
    "type": "Number",
    "description": "Registered source id."
  }
}
CLI Command
svc_call AudioDecoder0 RegisterSource {"Source":null}
UnregisterSource
Description

Unregister an audio source by name.

Execution
  • Requires scheduler: Not required

Parameters
  • SourceName

    • Type: String

    • Required: required

    • Description: Source name.

Schema JSON
Show raw JSON

{
  "name": "UnregisterSource",
  "description": "Unregister an audio source by name.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "SourceName",
      "description": "Source name.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioDecoder0 UnregisterSource {"SourceName":null}
RequestOutput
Description

Request an output for an audio source.

Execution
  • Requires scheduler: Not required

Parameters
  • SourceName

    • Type: String

    • Required: required

    • Description: Source name.

  • OutputName

    • Type: String

    • Required: required

    • Description: Output name.

Schema JSON
Show raw JSON

{
  "name": "RequestOutput",
  "description": "Request an output for an audio source.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "SourceName",
      "description": "Source name.",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "OutputName",
      "description": "Output name.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioDecoder0 RequestOutput {"SourceName":null,"OutputName":null}
ReleaseOutput
Description

Release an output from an audio source.

Execution
  • Requires scheduler: Not required

Parameters
  • SourceName

    • Type: String

    • Required: required

    • Description: Source name.

  • OutputName

    • Type: String

    • Required: required

    • Description: Output name.

Schema JSON
Show raw JSON

{
  "name": "ReleaseOutput",
  "description": "Release an output from an audio source.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "SourceName",
      "description": "Source name.",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "OutputName",
      "description": "Output name.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioDecoder0 ReleaseOutput {"SourceName":null,"OutputName":null}
SetActiveSource
Description

Set the active audio source for an output.

Execution
  • Requires scheduler: Not required

Parameters
  • OutputName

    • Type: String

    • Required: required

    • Description: Output name.

  • SourceName

    • Type: String

    • Required: required

    • Description: Source name.

Schema JSON
Show raw JSON

{
  "name": "SetActiveSource",
  "description": "Set the active audio source for an output.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "OutputName",
      "description": "Output name.",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "SourceName",
      "description": "Source name.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioDecoder0 SetActiveSource {"OutputName":null,"SourceName":null}
GetActiveSource
Description

Get the active audio source name for an output.

Execution
  • Requires scheduler: Not required

Parameters
  • OutputName

    • Type: String

    • Required: required

    • Description: Output name.

Return Value
  • Type: String

  • Description: Active source name, or an empty string when no source is active.

Schema JSON
Show raw JSON

{
  "name": "GetActiveSource",
  "description": "Get the active audio source name for an output.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "OutputName",
      "description": "Output name.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": {
    "type": "String",
    "description": "Active source name, or an empty string when no source is active."
  }
}
CLI Command
svc_call AudioDecoder0 GetActiveSource {"OutputName":null}
OpenStream
Description

Open a direct audio stream for a source and output.

Execution
  • Requires scheduler: Not required

Parameters
  • SourceName

    • Type: String

    • Required: required

    • Description: Source name.

  • OutputName

    • Type: String

    • Required: required

    • Description: Output name.

  • Config

    • Type: Object

    • Required: required

    • Description: Stream config. Example: {"type":"PCM","general":{"channels":0,"sample_bits":0,"sample_rate":0,"frame_duration":0},"queue_size_bytes":32768,"queue_policy":"DropNewest"}

Schema JSON
Show raw JSON

{
  "name": "OpenStream",
  "description": "Open a direct audio stream for a source and output.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "SourceName",
      "description": "Source name.",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "OutputName",
      "description": "Output name.",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "Config",
      "description": "Stream config. Example: {\"type\":\"PCM\",\"general\":{\"channels\":0,\"sample_bits\":0,\"sample_rate\":0,\"frame_duration\":0},\"queue_size_bytes\":32768,\"queue_policy\":\"DropNewest\"}",
      "type": "Object",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioDecoder0 OpenStream {"SourceName":null,"OutputName":null,"Config":null}
CloseStream
Description

Close a direct audio stream for a source and output.

Execution
  • Requires scheduler: Not required

Parameters
  • SourceName

    • Type: String

    • Required: required

    • Description: Source name.

  • OutputName

    • Type: String

    • Required: required

    • Description: Output name.

Schema JSON
Show raw JSON

{
  "name": "CloseStream",
  "description": "Close a direct audio stream for a source and output.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "SourceName",
      "description": "Source name.",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "OutputName",
      "description": "Output name.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AudioDecoder0 CloseStream {"SourceName":null,"OutputName":null}

Events

Playback

PlayStateChanged
Description

Emitted when playback state changes.

Execution
  • Requires scheduler: Required

Items
  • State

    • Type: String

    • Description: Playback state. Allowed values: [Idle, Playing, Paused]

Schema JSON
Show raw JSON

{
  "name": "PlayStateChanged",
  "description": "Emitted when playback state changes.",
  "require_scheduler": true,
  "items": [
    {
      "name": "State",
      "description": "Playback state. Allowed values: [Idle, Playing, Paused]",
      "type": "String"
    }
  ]
}
CLI Command
svc_subscribe AudioPlayback PlayStateChanged
VolumeChanged
Description

Emitted when the target audio playback volume changes.

Execution
  • Requires scheduler: Required

Items
  • Volume

    • Type: Number

    • Description: Current target playback volume percentage [0, 100].

Schema JSON
Show raw JSON

{
  "name": "VolumeChanged",
  "description": "Emitted when the target audio playback volume changes.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Volume",
      "description": "Current target playback volume percentage [0, 100].",
      "type": "Number"
    }
  ]
}
CLI Command
svc_subscribe AudioPlayback VolumeChanged
MuteChanged
Description

Emitted when the target audio playback mute state changes.

Execution
  • Requires scheduler: Required

Items
  • IsMuted

    • Type: Boolean

    • Description: Whether audio playback is currently muted. True if muted, false if unmuted.

Schema JSON
Show raw JSON

{
  "name": "MuteChanged",
  "description": "Emitted when the target audio playback mute state changes.",
  "require_scheduler": true,
  "items": [
    {
      "name": "IsMuted",
      "description": "Whether audio playback is currently muted. True if muted, false if unmuted.",
      "type": "Boolean"
    }
  ]
}
CLI Command
svc_subscribe AudioPlayback MuteChanged

Encoder

AFEEventHappened
Description

Emitted when an AFE event occurs.

Execution
  • Requires scheduler: Required

Items
  • Event

    • Type: String

    • Description: AFE event. Allowed values: [VAD_Start, VAD_End, WakeStart, WakeEnd]

Schema JSON
Show raw JSON

{
  "name": "AFEEventHappened",
  "description": "Emitted when an AFE event occurs.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Event",
      "description": "AFE event. Allowed values: [VAD_Start, VAD_End, WakeStart, WakeEnd]",
      "type": "String"
    }
  ]
}
CLI Command
svc_subscribe AudioEncoder0 AFEEventHappened

Decoder

This contract does not publish standard event schemas.