ESP-APA-DOA Component

Note

This document is automatically translated using AI. Please excuse any detailed errors. The official English version is still in progress.

Introduction to ESP-APA-DOA

ESP-APA-DOA is a component for estimating the direction of arrival (DOA) of sound sources in ESP-IDF projects. It estimates the azimuth of the speaker on the horizontal plane based on multichannel microphone PCM data. The component currently mainly covers dual-mic linear arrays and three-mic triangular arrays, using GCC-PHAT to estimate the TDOA (Time Difference Of Arrival) between microphones, and then outputs the angle and direction bin in combination with the microphone geometry.

Unlike a complete voice frontend, ESP-APA-DOA itself is only responsible for DOA calculation and does not have built-in wake-up, AEC, NS, voice recognition capabilities. Applications can place it after existing recording links, VAD links, or voice interaction links, and only call DOA calculation when valid voice is detected, thereby reducing CPU usage and reducing misjudgment of noise segments.

The typical input and output of the component are as follows:

Item	Description	Remarks
Input	Multichannel `int16_t` PCM	Supports packed or planar layout
Output	`azimuth` and `direction_bin`	`azimuth` is the smoothed direction angle
Typical sampling rate	16 kHz	Suitable for voice DOA scenarios
Core algorithm	GCC-PHAT + geometric solution + gate + smoothing	Engineering-oriented, easy to debug and port

Dual-mic DOA

The dual-mic mode is suitable for devices with two microphones such as ESP32-S3-Korvo-2, ESP-VoCat, ESP32-S31-Korvo-1. The two microphones form a linear array and can only get the TDOA of one pair of microphones, so it is naturally more suitable for judging the left and right directions in the front half plane.

The angle output definition of ESP-APA-DOA dual-mic is:

Angle	Meaning
`0°`	Towards the direction of the left microphone
`90°`	The direction of the perpendicular line between the two mics, that is, straight ahead
`180°`	Towards the direction of the right microphone

The core limitation of dual-mic is front-back ambiguity: the same TDOA can correspond to two symmetrical directions in the front and back. Therefore, dual-mic generally does not output a complete 0..360°, but outputs a user angle of 0..180°. For scenarios such as smart speakers, toys, desktop screens, robot heads, etc., if you mainly care about the user’s left and right positions in front of the device, the dual-mic solution is usually sufficient.

The current ESP-APA-DOA dual-mic path, in addition to the basic GCC-PHAT, also includes TDOA filtering, confidence judgment, weak tail sound protection, channel energy balance check, direction smoothing, and bin hysteresis. The goal of these processes is not to break the physical limit of dual-mic, but to make the output more continuous, easier to debug, and reduce the direction jitter caused by the tail of the voice, reverberation, and weak energy frames.

In the actual evaluation and product interaction of dual-mic, it is recommended to use the three typical angles of 45°, 90°, and 135° first:

45°: Left front, can verify whether the left direction is distinguishable.
90°: Straight ahead, usually the most stable position of dual-mic and most suitable as the center reference.
135°: Right front, can verify whether the right direction is distinguishable.

It is not recommended to use 0° or 180° as the main acceptance angle at the beginning. They are close to the endfire endpoints of dual-mic, and are very sensitive to mic distance errors, left-right mic gain differences, reflections, and test station positions. They are prone to phenomena where the endpoints are not full or return to 90°. The endpoints can be used as supplementary test items, but they are more suitable for separate debugging after 45°, 90°, 135° are already stable.

Comparison with ESP-SR Dual Mic DOA

DOA capability is also included in ESP-SR, typically used as part of the front end of voice recognition. ESP-APA-DOA is more of an independent component, suitable for scenarios that only require the direction of the sound source, or wish to debug and integrate DOA separately.

Based on the dual mic test comparison records in project_example/esp_doa, ESP-APA-DOA and ESP-SR dual mic DOA can be understood as follows:

Comparison Dimension	ESP-SR Dual Mic DOA	ESP-APA-DOA Dual Mic
Component Positioning	Part of the voice front-end capabilities, usually used with AFE, wake-up, VAD, etc.	Independent DOA component, not strongly dependent on ESP-SR or GMF
Output Style	More discrete results, boundary angles sometimes dare to give full values	Output is more continuous, with TDOA, confidence, level, etc. for debugging basis
Stability	Central angle is usable, but there may be noticeable jumps within the same speech	Smoother at test angles such as `90°`, `135°`
Boundary Performance	`180°` is more likely to report full, but the tail section may also scatter	More conservative at the boundary, `0°` and `180°` may not be full
Debuggability	Mainly look at the final DOA result	Can analyze problems with `raw_tdoa`, `filt_tdoa`, `conf`, `level`, `balance`

Therefore, if the product is already using ESP-SR AFE, and DOA is just an additional capability of the voice front end, you can prefer to continue using ESP-SR. If you need a more independent, more interpretable direction estimation module, or need to adjust the mic distance, channel order, TDOA filtering and direction smoothing separately for different boards, ESP-APA-DOA will be more convenient.

It should be noted that the dual mic endpoints themselves are the most difficult positions. The effects of 0° and 180° are not only affected by the algorithm, but also by the mic distance, microphone sensitivity differences, shell openings, desktop reflections, test positions and recording channel order. When encountering endpoints that are not full or direction returning to the center, you should prioritize saving multi-channel original PCM/WAV, and check the sampling rate, channel number, left and right mic order and energy balance.

Three Mic DOA

Three mic mode is suitable for three mic array devices such as ESP32-S3-Korvo-1. The current three mic model of ESP-APA-DOA uses an equilateral triangle array by default, with three microphones forming three pairs, yielding three sets of TDOA.

The main advantages of three mics over two mics are:

Can output a complete 0..360° azimuth, instead of only outputting the front half plane 0..180°.
There is geometric consistency between the three TDOAs, and the current direction can be judged by the residual.
More suitable for scenarios that require omnidirectional positioning, such as ring lights, robot turning, screen orientation, conference equipment, etc.

The output angle of the three mics is defined as:

Angle	Meaning
`0°`	Straight ahead
`90°`	Right side
`180°`	Rear
`270°`	Left side

The three mics still first use GCC-PHAT to estimate the TDOA of each pair of microphones, and then use the least squares solution based on the three mic geometric matrix to obtain the two-dimensional direction vector. Since there are three baselines, three mics can better distinguish the front and back directions than two mics. But three mics also rely more on hardware consistency: microphone position, channel order, sampling synchronization and channel level all need to be correct, otherwise there may be angle rotation, mirroring, fixed on a certain axis, or the residual becomes larger leading to unstable results.

Usage and Integration Suggestions

The integration chain of ESP-APA-DOA can generally be understood as: the application first completes multi-mic collection and necessary VAD judgment, then sends the multi-channel PCM of the valid voice segment into the DOA component, and finally uses the component’s output azimuth and direction_bin to drive UI, light effects, screen orientation or robot turning. The following parameters are recommended for confirmation during integration:

Parameter	Description
`num_mics`	Number of microphones. Set to `2` for dual mics, and `3` for triple mics.
`mic_side_m`	Center distance between adjacent microphones. For dual mics, it’s the distance between the two mics, and for triple mics, it’s the side length of the triangle.
`sample_rate_hz`	Must be consistent with the actual PCM sampling rate.
`window_samples`	The length of the historical window used for each DOA analysis. The larger the window, the more stable it usually is, but the delay and CPU usage are higher.
`chunk_samples`	The number of new samples input each time. The smaller it is, the more frequent the updates, but the number of calls and CPU usage are higher.
`input_layout`	PCM channel layout, must be consistent with the actual recording data.
`azimuth_offset_deg`	Used for board-level installation direction calibration.

Example Projects

The ESP-APA-DOA component includes the following examples:

Example	Device	Number of Microphones
`examples/basic_doa`	No hardware dependency, use synthetic PCM to verify API	3
`examples/2_mic_example`	ESP32-S3-Korvo-2	2
`examples/2_mic_s31_example`	ESP32-S31-Korvo-1	2
`examples/2_mic_vocat`	ESP-VoCat	2
`examples/3_mic_example`	ESP32-S3-Korvo-1	3

Debugging Suggestions

If the DOA result does not change with the speaking angle, it is recommended to check the hardware and recording link first, rather than directly modifying the algorithm:

Save the original multi-channel PCM/WAV, confirm that the sampling rate, channel number in the file header and the actual data are consistent.
Check whether each microphone channel has sound, and whether there is silence, clipping, severe background noise or DC bias.
Check whether the channel order is consistent with num_mics, input_layout, and slot configuration.
For dual mics, pay special attention to whether the left and right channels are reversed, and whether there are explainable changes at 0°, 90°, and 180°.
For triple mics, pay special attention to whether the TDOA of the three pairs changes with direction, and whether it meets geometric consistency.
For weak tail sound and reverberation scenes, you can combine with VAD, and only call DOA output during valid voice segments.

Overall, ESP-APA-DOA is more suitable for products that need independent sound source direction estimation, need dual/triple mic unified API, and need to retain debuggable information. The dual mic solution is low cost and easy to integrate, but has front and back ambiguity and endpoint limitations; the triple mic solution has slightly higher hardware cost, but can provide a complete 360° azimuth, which is more suitable for devices that need omnidirectional interaction.