Agent Manager

[中文]

  • Component registry: espressif/brookesia_agent_manager

  • Helper header: #include "brookesia/service_helper/agent/manager.hpp"

  • Helper class: esp_brookesia::service::helper::Manager

Overview

brookesia_agent_manager is the core agent framework:

  • Lifecycle: Plugin-based init, activate, start, stop, sleep, wake, and dynamic switching.

  • State machine: Correct, consistent transitions.

  • Control: Pause/resume, interrupt, status queries.

  • Dialog modes: RealTime and Manual (manual listen start/stop).

  • Events: Operations, state, speech/listen, text, emotes—decoupled from apps.

  • Extensions: Function calling, text, interrupt, emotes via agent properties.

  • AFE (optional): Wake begin/end handling.

  • Services: Audio service and Device time-sync integration.

  • Persistence: Optional brookesia_service_storage for active agent and mode.

State Machine Architecture

The manager uses a state machine for agent lifecycle.

State Diagram

flowchart TD
    A["**TimeSyncing**<br/>· Waits for time sync<br/>· Can be skipped by config"]
    B["**Ready**<br/>· Base environment is ready<br/>· Waits for activation"]
    C["**Activating**<br/>· Initializes plugin resources<br/>· Prepares agent context"]
    D["**Activated**<br/>· Plugin is activated<br/>· Session can start"]
    E["**Starting**<br/>· Starts session flow<br/>· Opens required services"]
    F["**Started**<br/>· Agent is running<br/>· Handles interactions"]
    G["**Sleeping**<br/>· Enters sleep flow<br/>· Pauses active interaction"]
    H["**Slept**<br/>· Agent is sleeping<br/>· Waits for wake or stop"]
    I["**WakingUp**<br/>· Restores session resources<br/>· Returns to running state"]
    J["**Stopping**<br/>· Stops session<br/>· Releases runtime resources"]

    A -->|"time synced / skipped"| B
    B -->|"Activate"| C
    C -->|"activated"| D
    D -->|"Start"| E
    E -->|"started"| F
    F -->|"Sleep"| G
    G -->|"slept"| H
    H -->|"Wake"| I
    I -->|"woke"| F
    F -->|"Stop"| J
    H -->|"Stop"| J
    J -->|"stopped"| D

State Descriptions

State

Type

Description

TimeSyncing

Transient

Waiting for time sync.

Ready

Stable

Time OK (or skipped); waiting for activate.

Activating

Transient

Activating agent.

Activated

Stable

Activated; waiting for start.

Starting

Transient

Starting agent.

Started

Stable

Running; audio in/out enabled.

Sleeping

Transient

Entering sleep.

Slept

Stable

Asleep; can wake or stop.

WakingUp

Transient

Waking from sleep.

Stopping

Transient

Stopping agent.

Standard Include / Helper Class

  • Standard include: #include \"brookesia/service_helper/agent/manager.hpp\"

  • Helper class: esp_brookesia::service::helper::Manager

Service Interfaces

Functions

SetAgentInfo

Description

Set agent info.

Execution

  • Requires scheduler: Not required

Parameters

  • Name

    • Type: String

    • Required: required

    • Description: Agent name.

  • Info

    • Type: Object

    • Required: required

    • Description: Agent info as a JSON object. Example: {"api_key":"api_key_value","model":"model_name"}

Schema JSON
Show raw JSON 

{
  "name": "SetAgentInfo",
  "description": "Set agent info.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Name",
      "description": "Agent name.",
      "type": "String",
      "required": true,
      "default_value": null
    },
    {
      "name": "Info",
      "description": "Agent info as a JSON object. Example: {\"api_key\":\"api_key_value\",\"model\":\"model_name\"}",
      "type": "Object",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AgentManager SetAgentInfo {"Name":null,"Info":null}

SetChatMode

Description

Set chat mode.

Execution

  • Requires scheduler: Required

Parameters

  • Mode

    • Type: String

    • Required: required

    • Description: Chat mode. Allowed values: [Manual, RealTime]

Schema JSON
Show raw JSON 

{
  "name": "SetChatMode",
  "description": "Set chat mode.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Mode",
      "description": "Chat mode. Allowed values: [Manual, RealTime]",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AgentManager SetChatMode {"Mode":null}

SetTargetAgent

Description

Set target agent. The agent will be activated by triggering Activate action.

Execution

  • Requires scheduler: Required

Parameters

  • Name

    • Type: String

    • Required: required

    • Description: Agent name to set as target.

Schema JSON
Show raw JSON 

{
  "name": "SetTargetAgent",
  "description": "Set target agent. The agent will be activated by triggering `Activate` action.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Name",
      "description": "Agent name to set as target.",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AgentManager SetTargetAgent {"Name":null}

GetTargetAgent

Description

Get target agent name.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: String

  • Description: Example: "Coze"

Schema JSON
Show raw JSON 

{
  "name": "GetTargetAgent",
  "description": "Get target agent name.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "String",
    "description": "Example: \"Coze\""
  }
}
CLI Command
svc_call AgentManager GetTargetAgent

GetAgentNames

Description

Get agent names.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: Array

  • Description: Example: ["Agent1", "Agent2"]

Schema JSON
Show raw JSON 

{
  "name": "GetAgentNames",
  "description": "Get agent names.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Array",
    "description": "Example: [\"Agent1\", \"Agent2\"]"
  }
}
CLI Command
svc_call AgentManager GetAgentNames

GetAgentAttributes

Description

Get agent attributes.

Execution

  • Requires scheduler: Not required

Parameters

  • Name

    • Type: String

    • Required: optional

    • Default: ""

    • Description: Agent name (optional). Returns all agents when omitted.

Return Value

  • Type: Array

  • Description: Example: [{"name":"Agent","operation_timeout":{"activate":1000,"start":1000,"sleep":1000,"wake_up":1000,"stop":1000},"support_general_functions":[],"support_general_events":[],"require_time_sync":false}]

Schema JSON
Show raw JSON 

{
  "name": "GetAgentAttributes",
  "description": "Get agent attributes.",
  "require_scheduler": false,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Name",
      "description": "Agent name (optional). Returns all agents when omitted.",
      "type": "String",
      "required": false,
      "default_value": ""
    }
  ],
  "return_value": {
    "type": "Array",
    "description": "Example: [{\"name\":\"Agent\",\"operation_timeout\":{\"activate\":1000,\"start\":1000,\"sleep\":1000,\"wake_up\":1000,\"stop\":1000},\"support_general_functions\":[],\"support_general_events\":[],\"require_time_sync\":false}]"
  }
}
CLI Command
svc_call AgentManager GetAgentAttributes {"Name":null}

GetChatMode

Description

Get chat mode.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: String

  • Description: Allowed values: [Manual, RealTime]. Example: "Manual"

Schema JSON
Show raw JSON 

{
  "name": "GetChatMode",
  "description": "Get chat mode.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "String",
    "description": "Allowed values: [Manual, RealTime]. Example: \"Manual\""
  }
}
CLI Command
svc_call AgentManager GetChatMode

GetActiveAgent

Description

Get active agent name.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: String

  • Description: Example: "Coze"

Schema JSON
Show raw JSON 

{
  "name": "GetActiveAgent",
  "description": "Get active agent name.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "String",
    "description": "Example: \"Coze\""
  }
}
CLI Command
svc_call AgentManager GetActiveAgent

TriggerGeneralAction

Description

Trigger a general action.

Execution

  • Requires scheduler: Required

Parameters

  • Action

    • Type: String

    • Required: required

    • Description: General action. Allowed values: [TimeSync, Activate, Start, Stop, Sleep, WakeUp]

Schema JSON
Show raw JSON 

{
  "name": "TriggerGeneralAction",
  "description": "Trigger a general action.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [
    {
      "name": "Action",
      "description": "General action. Allowed values: [TimeSync, Activate, Start, Stop, Sleep, WakeUp]",
      "type": "String",
      "required": true,
      "default_value": null
    }
  ],
  "return_value": null
}
CLI Command
svc_call AgentManager TriggerGeneralAction {"Action":null}

Suspend

Description

Suspend the agent.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Schema JSON
Show raw JSON 

{
  "name": "Suspend",
  "description": "Suspend the agent.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AgentManager Suspend

Resume

Description

Resume the agent.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Schema JSON
Show raw JSON 

{
  "name": "Resume",
  "description": "Resume the agent.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AgentManager Resume

InterruptSpeaking

Description

Interrupt speaking; the agent stops and keeps listening.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Schema JSON
Show raw JSON 

{
  "name": "InterruptSpeaking",
  "description": "Interrupt speaking; the agent stops and keeps listening.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AgentManager InterruptSpeaking

ManualStartListening

Description

Manually start listening. Only in Manual mode.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Schema JSON
Show raw JSON 

{
  "name": "ManualStartListening",
  "description": "Manually start listening. Only in `Manual` mode.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AgentManager ManualStartListening

ManualStopListening

Description

Manually stop listening. Only in Manual mode.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Schema JSON
Show raw JSON 

{
  "name": "ManualStopListening",
  "description": "Manually stop listening. Only in `Manual` mode.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AgentManager ManualStopListening

GetGeneralState

Description

Get general state.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: String

  • Description: Allowed values: [TimeSyncing, Ready, Activating, Activated, Starting, Stopping, Started, Sleeping, WakingUp, Slept]. Example: "Started"

Schema JSON
Show raw JSON 

{
  "name": "GetGeneralState",
  "description": "Get general state.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "String",
    "description": "Allowed values: [TimeSyncing, Ready, Activating, Activated, Starting, Stopping, Started, Sleeping, WakingUp, Slept]. Example: \"Started\""
  }
}
CLI Command
svc_call AgentManager GetGeneralState

GetSuspendStatus

Description

Get suspend status.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: Boolean

  • Description: Example: false

Schema JSON
Show raw JSON 

{
  "name": "GetSuspendStatus",
  "description": "Get suspend status.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Boolean",
    "description": "Example: false"
  }
}
CLI Command
svc_call AgentManager GetSuspendStatus

GetSpeakingStatus

Description

Get speaking status.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: Boolean

  • Description: Example: true

Schema JSON
Show raw JSON 

{
  "name": "GetSpeakingStatus",
  "description": "Get speaking status.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Boolean",
    "description": "Example: true"
  }
}
CLI Command
svc_call AgentManager GetSpeakingStatus

GetListeningStatus

Description

Get listening status.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Return Value

  • Type: Boolean

  • Description: Example: true

Schema JSON
Show raw JSON 

{
  "name": "GetListeningStatus",
  "description": "Get listening status.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": {
    "type": "Boolean",
    "description": "Example: true"
  }
}
CLI Command
svc_call AgentManager GetListeningStatus

LoadData

Description

Load persisted manager data.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Schema JSON
Show raw JSON 

{
  "name": "LoadData",
  "description": "Load persisted manager data.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AgentManager LoadData

ResetData

Description

Reset manager and agent data.

Execution

  • Requires scheduler: Required

Parameters

  • No parameters.

Schema JSON
Show raw JSON 

{
  "name": "ResetData",
  "description": "Reset manager and agent data.",
  "require_scheduler": true,
  "default_timeout_ms": null,
  "parameters": [],
  "return_value": null
}
CLI Command
svc_call AgentManager ResetData

Events

GeneralActionTriggered

Description

Emitted when a general action is triggered.

Execution

  • Requires scheduler: Required

Items

  • Action

    • Type: String

    • Description: General action. Allowed values: [TimeSync, Activate, Start, Stop, Sleep, WakeUp]

Schema JSON
Show raw JSON 

{
  "name": "GeneralActionTriggered",
  "description": "Emitted when a general action is triggered.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Action",
      "description": "General action. Allowed values: [TimeSync, Activate, Start, Stop, Sleep, WakeUp]",
      "type": "String"
    }
  ]
}
CLI Command
svc_subscribe AgentManager GeneralActionTriggered

GeneralEventHappened

Description

Emitted when a general event occurs.

Execution

  • Requires scheduler: Required

Items

  • Event

    • Type: String

    • Description: General event. Allowed values: [TimeSynced, Activated, Started, Stopped, Slept, Awake]

  • IsUnexpected

    • Type: Boolean

    • Description: Whether the event was unexpected.

Schema JSON
Show raw JSON 

{
  "name": "GeneralEventHappened",
  "description": "Emitted when a general event occurs.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Event",
      "description": "General event. Allowed values: [TimeSynced, Activated, Started, Stopped, Slept, Awake]",
      "type": "String"
    },
    {
      "name": "IsUnexpected",
      "description": "Whether the event was unexpected.",
      "type": "Boolean"
    }
  ]
}
CLI Command
svc_subscribe AgentManager GeneralEventHappened

SuspendStatusChanged

Description

Emitted when suspend status changes.

Execution

  • Requires scheduler: Required

Items

  • IsSuspended

    • Type: Boolean

    • Description: Whether the agent is suspended.

Schema JSON
Show raw JSON 

{
  "name": "SuspendStatusChanged",
  "description": "Emitted when suspend status changes.",
  "require_scheduler": true,
  "items": [
    {
      "name": "IsSuspended",
      "description": "Whether the agent is suspended.",
      "type": "Boolean"
    }
  ]
}
CLI Command
svc_subscribe AgentManager SuspendStatusChanged

SpeakingStatusChanged

Description

Emitted when speaking status changes.

Execution

  • Requires scheduler: Required

Items

  • IsSpeaking

    • Type: Boolean

    • Description: Whether the agent is speaking.

Schema JSON
Show raw JSON 

{
  "name": "SpeakingStatusChanged",
  "description": "Emitted when speaking status changes.",
  "require_scheduler": true,
  "items": [
    {
      "name": "IsSpeaking",
      "description": "Whether the agent is speaking.",
      "type": "Boolean"
    }
  ]
}
CLI Command
svc_subscribe AgentManager SpeakingStatusChanged

ListeningStatusChanged

Description

Emitted when listening status changes.

Execution

  • Requires scheduler: Required

Items

  • IsListening

    • Type: Boolean

    • Description: Whether the agent is listening.

Schema JSON
Show raw JSON 

{
  "name": "ListeningStatusChanged",
  "description": "Emitted when listening status changes.",
  "require_scheduler": true,
  "items": [
    {
      "name": "IsListening",
      "description": "Whether the agent is listening.",
      "type": "Boolean"
    }
  ]
}
CLI Command
svc_subscribe AgentManager ListeningStatusChanged

AgentSpeakingTextGot

Description

Emitted when the agent speaks text.

Execution

  • Requires scheduler: Required

Items

  • Text

    • Type: String

    • Description: Spoken text.

Schema JSON
Show raw JSON 

{
  "name": "AgentSpeakingTextGot",
  "description": "Emitted when the agent speaks text.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Text",
      "description": "Spoken text.",
      "type": "String"
    }
  ]
}
CLI Command
svc_subscribe AgentManager AgentSpeakingTextGot

UserSpeakingTextGot

Description

Emitted when the user speaks text.

Execution

  • Requires scheduler: Required

Items

  • Text

    • Type: String

    • Description: User text.

Schema JSON
Show raw JSON 

{
  "name": "UserSpeakingTextGot",
  "description": "Emitted when the user speaks text.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Text",
      "description": "User text.",
      "type": "String"
    }
  ]
}
CLI Command
svc_subscribe AgentManager UserSpeakingTextGot

EmoteGot

Description

Emitted when the agent shows an emote.

Execution

  • Requires scheduler: Required

Items

  • Emote

    • Type: String

    • Description: Emote name.

Schema JSON
Show raw JSON 

{
  "name": "EmoteGot",
  "description": "Emitted when the agent shows an emote.",
  "require_scheduler": true,
  "items": [
    {
      "name": "Emote",
      "description": "Emote name.",
      "type": "String"
    }
  ]
}
CLI Command
svc_subscribe AgentManager EmoteGot

API Reference

Agent Base Class

Public header: #include "brookesia/agent_manager/base.hpp"

Header File

Classes

class Base : public esp_brookesia::service::ServiceBase

Common base class for all agent implementations.

The class extends service::ServiceBase with agent lifecycle hooks, audio pipeline coordination, and shared state tracked by Manager and StateMachine.

Subclassed by esp_brookesia::agent::Coze, esp_brookesia::agent::Openai, esp_brookesia::agent::XiaoZhi

Public Functions

inline const AgentAttributes &get_attributes() const

Get immutable metadata describing the agent.

Returns

const AgentAttributes& Agent attributes passed at construction time.

inline const AudioConfig &get_audio_config() const

Get the audio configuration used by the agent.

Returns

const AudioConfig& Current encoder and decoder configuration.

Agent Manager

Public header: #include "brookesia/agent_manager/manager.hpp"

Header File

Classes

class Manager : public esp_brookesia::service::ServiceBase

Service responsible for managing agent selection and shared agent state.

Public Types

enum class DataType

Persistent keys managed by the agent manager.

Values:

enumerator TargetAgent
enumerator ChatMode
enumerator Max

Public Static Functions

static inline Manager &get_instance()

Get the global agent-manager singleton.

Returns

Manager& Singleton instance.