Flashing Models

[中文]

In the AI industry, a model refers to a mathematical representation of a system or process. It is used to make predictions or decisions based on input data. There are many types of models, such as decision trees, neural networks, and support vector machines, each with their own strengths and weaknesses. Esprssif also provides our trained models such as WakeNet and MultiNet (see the model data used in model)

To use our models in your project, you need to flash these models. Currently, ESP-SR supports the following methods to flash models:

ESP32: Load directly from Flash

Configuration

Run idf.py menuconfig to navigate to ESP Speech Recognition:

overview

overview

Use AFE

This option is enabled by default. Users do not need to modify it. Please keep the default configuration.

Use WakeNet

This option is enabled by default. When the user only uses AEC or BSS, etc., and does not need WakeNet or MultiNet, please disable this option, which reduces the size of the project firmware.

Select wake words by via menuconfig by navigating to ESP Speech Recognition > Select wake words. The model name of wake word in parentheses must be used to initialize WakeNet handle.

select wake wake

If you want to select multiple wake words, please select Load Multiple Wake Words

multi wake wake

Then you can select multiple wake words at the same time:

image1

Note

ESP32 doesn’t support multiple wake words.

For more details, please refer to WakeNet .

Use Multinet

This option is enabled by default. When users only use WakeNet or other algorithm modules, please disable this option, which reduces the size of the project firmware in some cases.

Chinese Speech Commands Model

ESP32 only supports command words in Chinese:

  • None

  • Chinese single recognition (MultiNet2)

For more details, please refer to Section MultiNet .

How To Use

After the above-mentioned configuration, users can initialize and start using the models following the examples described in the ESP-Skainet repo.

Here, we only introduce the code implementation, which can also be found in src/model_path.c.

ESP32 can only load model data from flash. Therefore, the model data in the code will automatically read the required data from the Flash according to the address. Note that, ESP32 and ESP32-S3 APIs are compatible.

Load Model Data from flash

  1. Write a partition table:

    model,  data, spiffs,         , SIZE,

    Among them, SIZE can refer to the recommended size when the user uses idf.py build to compile, for example: Recommended model partition size: 500K

  2. Initialize the flash partition: User can use esp_srmodel_init(partition_label) API to initialize flash and return all loaded models.

    • base_path: The model storage base_path is srmodel and cannot be changed

    • partition_label: The partition label of the model is model, which needs to be consistent with the Name in the above partition table

After completing the above configuration, the project will automatically generate model.bin after the project is compiled, and flash it to the flash partition.

Model initialization and Usage

 //
 // step1: return models in flash or in sdcard
 //
 char *model_path = your_model_path: // partition_label or model_path in sdcard;
 models = esp_srmodel_init(model_path);

//
// step2: select the specific model by keywords
//
char *wn_name = esp_srmodel_filter(models, ESP_WN_PREFIX, NULL); // select WakeNet model
char *nm_name = esp_srmodel_filter(models, ESP_MN_PREFIX, NULL); // select MultiNet model
char *alexa_wn_name = esp_srmodel_filter(models, ESP_WN_PREFIX, "alexa"); // select WakeNet with "alexa" wake word.
char *en_mn_name = esp_srmodel_filter(models, ESP_MN_PREFIX, ESP_MN_ENGLISH); // select english MultiNet model
char *cn_mn_name = esp_srmodel_filter(models, ESP_MN_PREFIX, ESP_MN_CHINESE); // select english MultiNet model

// It also works if you use the model name directly in your code.
char *my_wn_name = "wn9_hilexin"
// we recommend you to check that it is loaded correctly
 if (!esp_srmodel_exists(models, my_wn_name))
     printf("%s can not be loaded correctly\n")

//
// step3: initialize model
//
esp_wn_iface_t *wakenet = esp_wn_handle_from_name(wn_name);
model_iface_data_t *wn_model_data = wakenet->create(wn_name, DET_MODE_2CH_90);

esp_mn_iface_t *multinet = esp_mn_handle_from_name(mn_name);
model_iface_data_t *mn_model_data = multinet->create(mn_name, 6000);

Provide feedback about this document