How to deploy YOLO11n
In this tutorial, we will introduce how to quantize a pre-trained YOLO11n model using ESP-PPQ and deploy the quantized YOLO11n model using ESP-DL.
Preparation
Model quantization
Pre-trained Model
You can download pre-trained yolo11n model from Ultralytics release.
Currently, ESP-PPQ supports ONNX, PyTorch, and TensorFlow models. During the quantization process, PyTorch and TensorFlow models are first converted to ONNX models, so the pre-trained yolo11n model needs to be converted to an ONNX model.
Specifically, refer to the script export_onnx.py to convert the pre-trained yolo11n model to an ONNX model.
In the script, we have overridden the forward method of the Detect class, which offers following advantages:
Faster inference. Compared to the original yolo11n model, operations related to decoding bounding boxes in Detect head are moved from the inference pass to the post-processing phase, resulting in a significant reduction in inference latency. On one hand, operations like
Conv,Transpose,Slice,SplitandConcatare time-consuming when applied during inference pass. On the other hand, the inference outputs are first filtered using a score threshold before decoding the boxes in the post-processing pass, which significantly reduces the number of calculations, thereby accelerating the overall inference speed.Lower quantization Error. The
ConcatandAddoperators adopt joint quantization in ESP-PPQ. To reduce quantization errors, the box and score are output by separate branches, rather than being concatenated, due to the significant difference in their ranges. Similarly, since the ranges of the two inputs ofAddandSubdiffer significantly, the calculations are performed in the post-processing phase to avoid quantization errors.
Calibration Dataset
The calibration dataset needs to match the input format of the model. The calibration dataset should cover all possible input scenarios to better quantize the model. Here, the calibration dataset used in this example is calib_yolo11n.
8bit Post-Training Quantization
ESP32-P4 Quantization settings
The following quantization settings were generated by AutoQuant. To use AutoQuant, please update esp-ppq to the latest version and refer to the tutorial.
quant_setting = QuantizationSettingFactory.espdl_setting()
quant_setting.quantize_activation_setting.calib_algorithm = 'percentile'
quant_setting.bias_correct = True
quant_setting.bias_correct_setting.interested_layers = []
quant_setting.bias_correct_setting.block_size = 4
quant_setting.bias_correct_setting.steps = 32
quant_setting.tqt_optimization = True
tqt_setting = quant_setting.tqt_optimization_setting
tqt_setting.lr = 1e-5
tqt_setting.steps = 500
tqt_setting.block_size = 4
tqt_setting.is_scale_trainable = True
tqt_setting.gamma = 0.0
tqt_setting.int_lambda = 0.0
tqt_setting.collecting_device = 'cuda'
ESP32-P4 Quantization error
Analysing Graphwise Quantization Error(Phrase 2):: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 4.59it/s]
Layer | NOISE:SIGNAL POWER RATIO
/model.3/conv/Conv: | ████████████████████ | 6.440%
/model.2/cv2/conv/Conv: | ███████████████████ | 6.035%
/model.2/m.0/cv2/conv/Conv: | █████████████████ | 5.410%
/model.6/m.0/cv2/conv/Conv: | █████████████████ | 5.329%
/model.10/m/m.0/attn/proj/conv/Conv: | ████████████████ | 5.175%
/model.23/cv2.0/cv2.0.1/conv/Conv: | ██████████████ | 4.425%
/model.4/m.0/cv1/conv/Conv: | ██████████████ | 4.371%
/model.10/m/m.0/ffn/ffn.1/conv/Conv: | █████████████ | 4.270%
/model.4/cv2/conv/Conv: | █████████████ | 4.266%
/model.6/cv1/conv/Conv: | █████████████ | 4.050%
/model.8/m.0/cv2/conv/Conv: | ███████████ | 3.640%
/model.6/m.0/cv3/conv/Conv: | ███████████ | 3.530%
/model.5/conv/Conv: | ███████████ | 3.420%
/model.4/cv1/conv/Conv: | ██████████ | 3.341%
/model.6/cv2/conv/Conv: | ██████████ | 3.242%
/model.6/m.0/m/m.0/cv1/conv/Conv: | ██████████ | 3.111%
/model.23/cv2.1/cv2.1.1/conv/Conv: | █████████ | 3.045%
/model.2/cv1/conv/Conv: | █████████ | 2.928%
/model.23/cv3.0/cv3.0.1/cv3.0.1.1/conv/Conv: | ████████ | 2.712%
/model.8/cv1/conv/Conv: | ████████ | 2.700%
/model.2/m.0/cv1/conv/Conv: | ████████ | 2.677%
/model.1/conv/Conv: | ████████ | 2.523%
/model.23/cv2.1/cv2.1.0/conv/Conv: | ████████ | 2.514%
/model.6/m.0/m/m.0/cv2/conv/Conv: | ████████ | 2.474%
/model.23/cv2.0/cv2.0.0/conv/Conv: | ███████ | 2.404%
/model.6/m.0/m/m.1/cv1/conv/Conv: | ███████ | 2.325%
/model.10/m/m.0/attn/pe/conv/Conv: | ███████ | 2.323%
/model.7/conv/Conv: | ███████ | 2.322%
/model.23/cv3.1/cv3.1.1/cv3.1.1.1/conv/Conv: | ███████ | 2.317%
/model.6/m.0/cv1/conv/Conv: | ███████ | 2.279%
/model.4/m.0/cv2/conv/Conv: | ███████ | 2.273%
/model.19/m.0/cv2/conv/Conv: | ███████ | 2.273%
/model.16/m.0/cv2/conv/Conv: | ███████ | 2.259%
/model.16/m.0/cv1/conv/Conv: | ███████ | 2.122%
/model.8/m.0/cv1/conv/Conv: | ██████ | 2.061%
/model.23/cv3.0/cv3.0.0/cv3.0.0.1/conv/Conv: | ██████ | 2.031%
/model.13/m.0/cv2/conv/Conv: | ██████ | 1.982%
/model.10/m/m.0/attn/qkv/conv/Conv: | ██████ | 1.969%
/model.10/cv1/conv/Conv: | ██████ | 1.939%
/model.19/cv2/conv/Conv: | ██████ | 1.879%
/model.8/m.0/m/m.0/cv1/conv/Conv: | ██████ | 1.868%
/model.23/cv3.2/cv3.2.1/cv3.2.1.1/conv/Conv: | ██████ | 1.861%
/model.8/cv2/conv/Conv: | ██████ | 1.835%
/model.17/conv/Conv: | ██████ | 1.819%
/model.19/cv1/conv/Conv: | █████ | 1.773%
/model.16/cv2/conv/Conv: | █████ | 1.750%
/model.23/cv3.1/cv3.1.0/cv3.1.0.1/conv/Conv: | █████ | 1.740%
/model.22/m.0/cv2/conv/Conv: | █████ | 1.730%
/model.19/m.0/cv1/conv/Conv: | █████ | 1.706%
/model.23/cv2.0/cv2.0.2/Conv: | █████ | 1.661%
/model.13/cv1/conv/Conv: | █████ | 1.635%
/model.20/conv/Conv: | █████ | 1.635%
/model.13/m.0/cv1/conv/Conv: | █████ | 1.613%
/model.13/cv2/conv/Conv: | █████ | 1.594%
/model.23/cv2.2/cv2.2.0/conv/Conv: | █████ | 1.560%
/model.23/cv3.2/cv3.2.0/cv3.2.0.1/conv/Conv: | █████ | 1.538%
/model.6/m.0/m/m.1/cv2/conv/Conv: | █████ | 1.513%
/model.23/cv3.2/cv3.2.0/cv3.2.0.0/conv/Conv: | █████ | 1.511%
/model.22/cv1/conv/Conv: | ████ | 1.456%
/model.8/m.0/m/m.0/cv2/conv/Conv: | ████ | 1.453%
/model.22/cv2/conv/Conv: | ████ | 1.384%
/model.16/cv1/conv/Conv: | ████ | 1.345%
/model.8/m.0/m/m.1/cv1/conv/Conv: | ████ | 1.336%
/model.8/m.0/cv3/conv/Conv: | ████ | 1.329%
/model.23/cv2.2/cv2.2.1/conv/Conv: | ████ | 1.288%
/model.10/m/m.0/attn/MatMul: | ████ | 1.260%
/model.22/m.0/cv3/conv/Conv: | ███ | 1.117%
/model.10/cv2/conv/Conv: | ███ | 1.094%
/model.23/cv2.1/cv2.1.2/Conv: | ███ | 1.045%
/model.22/m.0/m/m.0/cv1/conv/Conv: | ███ | 1.025%
/model.22/m.0/m/m.0/cv2/conv/Conv: | ███ | 0.999%
/model.10/m/m.0/attn/MatMul_1: | ███ | 0.984%
/model.22/m.0/m/m.1/cv2/conv/Conv: | ███ | 0.889%
/model.23/cv3.1/cv3.1.0/cv3.1.0.0/conv/Conv: | ███ | 0.864%
/model.8/m.0/m/m.1/cv2/conv/Conv: | ███ | 0.839%
/model.9/cv2/conv/Conv: | ██ | 0.800%
/model.23/cv3.2/cv3.2.1/cv3.2.1.0/conv/Conv: | ██ | 0.761%
/model.23/cv3.1/cv3.1.1/cv3.1.1.0/conv/Conv: | ██ | 0.759%
/model.22/m.0/cv1/conv/Conv: | ██ | 0.750%
/model.22/m.0/m/m.1/cv1/conv/Conv: | ██ | 0.705%
/model.23/cv3.0/cv3.0.1/cv3.0.1.0/conv/Conv: | ██ | 0.677%
/model.10/m/m.0/ffn/ffn.0/conv/Conv: | ██ | 0.669%
/model.23/cv2.2/cv2.2.2/Conv: | █ | 0.434%
/model.9/cv1/conv/Conv: | █ | 0.390%
/model.23/cv3.0/cv3.0.0/cv3.0.0.0/conv/Conv: | █ | 0.341%
/model.0/conv/Conv: | | 0.157%
/model.23/cv3.2/cv3.2.2/Conv: | | 0.079%
/model.23/cv3.1/cv3.1.2/Conv: | | 0.068%
/model.23/cv3.0/cv3.0.2/Conv: | | 0.038%
Analysing Layerwise quantization error:: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 89/89 [00:12<00:00, 7.30it/s]
Layer | NOISE:SIGNAL POWER RATIO
/model.1/conv/Conv: | ████████████████████ | 0.712%
/model.0/conv/Conv: | ███████████████ | 0.521%
/model.3/conv/Conv: | ██████████ | 0.369%
/model.2/cv2/conv/Conv: | █████ | 0.181%
/model.4/m.0/cv1/conv/Conv: | ███ | 0.120%
/model.2/cv1/conv/Conv: | ███ | 0.101%
/model.4/m.0/cv2/conv/Conv: | ███ | 0.095%
/model.2/m.0/cv1/conv/Conv: | ██ | 0.071%
/model.4/cv1/conv/Conv: | ██ | 0.069%
/model.5/conv/Conv: | █ | 0.052%
/model.4/cv2/conv/Conv: | █ | 0.045%
/model.7/conv/Conv: | █ | 0.036%
/model.10/cv2/conv/Conv: | █ | 0.029%
/model.9/cv2/conv/Conv: | █ | 0.029%
/model.6/cv1/conv/Conv: | █ | 0.027%
/model.2/m.0/cv2/conv/Conv: | █ | 0.026%
/model.13/cv1/conv/Conv: | █ | 0.024%
/model.6/cv2/conv/Conv: | █ | 0.022%
/model.10/cv1/conv/Conv: | █ | 0.021%
/model.10/m/m.0/attn/pe/conv/Conv: | | 0.017%
/model.8/cv2/conv/Conv: | | 0.017%
/model.16/cv1/conv/Conv: | | 0.016%
/model.13/cv2/conv/Conv: | | 0.015%
/model.23/cv2.2/cv2.2.0/conv/Conv: | | 0.015%
/model.19/m.0/cv1/conv/Conv: | | 0.012%
/model.23/cv2.0/cv2.0.0/conv/Conv: | | 0.012%
/model.9/cv1/conv/Conv: | | 0.011%
/model.16/cv2/conv/Conv: | | 0.011%
/model.13/m.0/cv1/conv/Conv: | | 0.010%
/model.19/cv2/conv/Conv: | | 0.010%
/model.23/cv2.1/cv2.1.0/conv/Conv: | | 0.010%
/model.22/m.0/m/m.1/cv1/conv/Conv: | | 0.009%
/model.10/m/m.0/attn/proj/conv/Conv: | | 0.009%
/model.10/m/m.0/ffn/ffn.0/conv/Conv: | | 0.008%
/model.23/cv2.2/cv2.2.1/conv/Conv: | | 0.008%
/model.8/cv1/conv/Conv: | | 0.008%
/model.22/m.0/cv1/conv/Conv: | | 0.007%
/model.19/m.0/cv2/conv/Conv: | | 0.007%
/model.19/cv1/conv/Conv: | | 0.007%
/model.16/m.0/cv1/conv/Conv: | | 0.006%
/model.16/m.0/cv2/conv/Conv: | | 0.006%
/model.10/m/m.0/attn/qkv/conv/Conv: | | 0.006%
/model.13/m.0/cv2/conv/Conv: | | 0.006%
/model.22/cv1/conv/Conv: | | 0.006%
/model.23/cv2.2/cv2.2.2/Conv: | | 0.006%
/model.23/cv2.0/cv2.0.2/Conv: | | 0.005%
/model.22/m.0/m/m.0/cv2/conv/Conv: | | 0.005%
/model.8/m.0/cv1/conv/Conv: | | 0.005%
/model.23/cv2.1/cv2.1.2/Conv: | | 0.005%
/model.6/m.0/m/m.0/cv1/conv/Conv: | | 0.005%
/model.6/m.0/m/m.1/cv1/conv/Conv: | | 0.005%
/model.8/m.0/m/m.1/cv1/conv/Conv: | | 0.004%
/model.8/m.0/m/m.0/cv1/conv/Conv: | | 0.004%
/model.6/m.0/m/m.0/cv2/conv/Conv: | | 0.004%
/model.20/conv/Conv: | | 0.004%
/model.10/m/m.0/ffn/ffn.1/conv/Conv: | | 0.004%
/model.22/cv2/conv/Conv: | | 0.004%
/model.23/cv2.1/cv2.1.1/conv/Conv: | | 0.004%
/model.23/cv2.0/cv2.0.1/conv/Conv: | | 0.004%
/model.17/conv/Conv: | | 0.004%
/model.8/m.0/m/m.0/cv2/conv/Conv: | | 0.004%
/model.6/m.0/cv3/conv/Conv: | | 0.003%
/model.6/m.0/cv1/conv/Conv: | | 0.003%
/model.23/cv3.1/cv3.1.1/cv3.1.1.1/conv/Conv: | | 0.003%
/model.22/m.0/m/m.0/cv1/conv/Conv: | | 0.003%
/model.6/m.0/m/m.1/cv2/conv/Conv: | | 0.003%
/model.22/m.0/m/m.1/cv2/conv/Conv: | | 0.002%
/model.22/m.0/cv3/conv/Conv: | | 0.002%
/model.23/cv3.2/cv3.2.0/cv3.2.0.1/conv/Conv: | | 0.002%
/model.8/m.0/m/m.1/cv2/conv/Conv: | | 0.002%
/model.8/m.0/cv3/conv/Conv: | | 0.002%
/model.10/m/m.0/attn/MatMul_1: | | 0.002%
/model.23/cv3.2/cv3.2.1/cv3.2.1.0/conv/Conv: | | 0.001%
/model.23/cv3.1/cv3.1.1/cv3.1.1.0/conv/Conv: | | 0.001%
/model.23/cv3.1/cv3.1.2/Conv: | | 0.001%
/model.23/cv3.0/cv3.0.1/cv3.0.1.0/conv/Conv: | | 0.001%
/model.23/cv3.2/cv3.2.2/Conv: | | 0.001%
/model.23/cv3.0/cv3.0.0/cv3.0.0.1/conv/Conv: | | 0.001%
/model.23/cv3.2/cv3.2.1/cv3.2.1.1/conv/Conv: | | 0.001%
/model.23/cv3.2/cv3.2.0/cv3.2.0.0/conv/Conv: | | 0.001%
/model.23/cv3.0/cv3.0.2/Conv: | | 0.001%
/model.23/cv3.1/cv3.1.0/cv3.1.0.1/conv/Conv: | | 0.000%
/model.10/m/m.0/attn/MatMul: | | 0.000%
/model.23/cv3.1/cv3.1.0/cv3.1.0.0/conv/Conv: | | 0.000%
/model.8/m.0/cv2/conv/Conv: | | 0.000%
/model.23/cv3.0/cv3.0.0/cv3.0.0.0/conv/Conv: | | 0.000%
/model.6/m.0/cv2/conv/Conv: | | 0.000%
/model.23/cv3.0/cv3.0.1/cv3.0.1.1/conv/Conv: | | 0.000%
/model.22/m.0/cv2/conv/Conv: | | 0.000%
ESP32-P4 Quantization results
With the same inputs, The mAP50-95 on COCO val2017 after quantization is 0.373, slightly lower than the float model.
ESP32-S3 Quantization settings
The following quantization settings were generated by AutoQuant. To use AutoQuant, please update esp-ppq to the latest version and refer to the tutorial.
quant_setting = QuantizationSettingFactory.espdl_setting()
quant_setting.quantize_activation_setting.calib_algorithm = 'percentile'
quant_setting.tqt_optimization = True
tqt_setting = quant_setting.tqt_optimization_setting
tqt_setting.lr = 1e-5
tqt_setting.steps = 800
tqt_setting.block_size = 4
tqt_setting.is_scale_trainable = True
tqt_setting.gamma = 0.0
tqt_setting.int_lambda = 0.0
tqt_setting.collecting_device = 'cuda'
ESP32-S3 Quantization error
Analysing Graphwise Quantization Error(Phrase 2):: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 4.25it/s]
Layer | NOISE:SIGNAL POWER RATIO
/model.3/conv/Conv: | ████████████████████ | 7.039%
/model.2/cv2/conv/Conv: | █████████████████ | 5.965%
/model.10/m/m.0/attn/proj/conv/Conv: | █████████████████ | 5.887%
/model.2/m.0/cv2/conv/Conv: | ████████████████ | 5.726%
/model.6/m.0/cv2/conv/Conv: | ████████████████ | 5.483%
/model.4/m.0/cv1/conv/Conv: | ██████████████ | 4.815%
/model.23/cv2.0/cv2.0.1/conv/Conv: | ██████████████ | 4.815%
/model.10/m/m.0/ffn/ffn.1/conv/Conv: | █████████████ | 4.569%
/model.4/cv2/conv/Conv: | ████████████ | 4.377%
/model.6/cv1/conv/Conv: | ███████████ | 4.048%
/model.6/m.0/cv3/conv/Conv: | ███████████ | 3.755%
/model.8/m.0/cv2/conv/Conv: | ██████████ | 3.517%
/model.4/cv1/conv/Conv: | ██████████ | 3.507%
/model.5/conv/Conv: | ██████████ | 3.495%
/model.23/cv3.0/cv3.0.1/cv3.0.1.1/conv/Conv: | ██████████ | 3.472%
/model.23/cv3.1/cv3.1.1/cv3.1.1.1/conv/Conv: | ██████████ | 3.460%
/model.6/cv2/conv/Conv: | ██████████ | 3.369%
/model.23/cv2.1/cv2.1.1/conv/Conv: | █████████ | 3.262%
/model.6/m.0/m/m.0/cv1/conv/Conv: | █████████ | 3.243%
/model.2/cv1/conv/Conv: | ████████ | 2.904%
/model.23/cv2.1/cv2.1.0/conv/Conv: | ████████ | 2.770%
/model.8/cv1/conv/Conv: | ████████ | 2.730%
/model.23/cv3.0/cv3.0.0/cv3.0.0.1/conv/Conv: | ████████ | 2.706%
/model.10/m/m.0/attn/pe/conv/Conv: | ████████ | 2.671%
/model.23/cv2.0/cv2.0.0/conv/Conv: | ███████ | 2.626%
/model.6/m.0/m/m.0/cv2/conv/Conv: | ███████ | 2.595%
/model.7/conv/Conv: | ███████ | 2.547%
/model.1/conv/Conv: | ███████ | 2.519%
/model.19/m.0/cv2/conv/Conv: | ███████ | 2.476%
/model.6/m.0/m/m.1/cv1/conv/Conv: | ███████ | 2.471%
/model.4/m.0/cv2/conv/Conv: | ███████ | 2.422%
/model.16/m.0/cv2/conv/Conv: | ███████ | 2.412%
/model.6/m.0/cv1/conv/Conv: | ███████ | 2.371%
/model.16/m.0/cv1/conv/Conv: | ███████ | 2.334%
/model.10/m/m.0/attn/qkv/conv/Conv: | ██████ | 2.159%
/model.23/cv3.1/cv3.1.0/cv3.1.0.1/conv/Conv: | ██████ | 2.150%
/model.10/cv1/conv/Conv: | ██████ | 2.135%
/model.23/cv2.2/cv2.2.0/conv/Conv: | ██████ | 2.123%
/model.13/m.0/cv2/conv/Conv: | ██████ | 2.073%
/model.22/m.0/cv2/conv/Conv: | ██████ | 2.059%
/model.8/m.0/cv1/conv/Conv: | ██████ | 2.053%
/model.19/cv2/conv/Conv: | ██████ | 2.049%
/model.23/cv3.2/cv3.2.1/cv3.2.1.1/conv/Conv: | ██████ | 2.041%
/model.13/cv2/conv/Conv: | ██████ | 2.027%
/model.17/conv/Conv: | ██████ | 2.001%
/model.16/cv2/conv/Conv: | █████ | 1.968%
/model.8/m.0/m/m.0/cv1/conv/Conv: | █████ | 1.916%
/model.8/cv2/conv/Conv: | █████ | 1.888%
/model.23/cv2.0/cv2.0.2/Conv: | █████ | 1.870%
/model.19/m.0/cv1/conv/Conv: | █████ | 1.847%
/model.19/cv1/conv/Conv: | █████ | 1.846%
/model.22/cv1/conv/Conv: | █████ | 1.725%
/model.23/cv3.2/cv3.2.0/cv3.2.0.1/conv/Conv: | █████ | 1.716%
/model.20/conv/Conv: | █████ | 1.710%
/model.23/cv3.2/cv3.2.0/cv3.2.0.0/conv/Conv: | █████ | 1.681%
/model.13/cv1/conv/Conv: | █████ | 1.669%
/model.2/m.0/cv1/conv/Conv: | █████ | 1.646%
/model.13/m.0/cv1/conv/Conv: | █████ | 1.641%
/model.6/m.0/m/m.1/cv2/conv/Conv: | █████ | 1.639%
/model.22/cv2/conv/Conv: | ████ | 1.524%
/model.8/m.0/m/m.0/cv2/conv/Conv: | ████ | 1.489%
/model.16/cv1/conv/Conv: | ████ | 1.457%
/model.8/m.0/cv3/conv/Conv: | ████ | 1.422%
/model.10/m/m.0/attn/MatMul: | ████ | 1.387%
/model.8/m.0/m/m.1/cv1/conv/Conv: | ████ | 1.369%
/model.23/cv2.2/cv2.2.1/conv/Conv: | ████ | 1.275%
/model.22/m.0/cv3/conv/Conv: | ███ | 1.259%
/model.10/m/m.0/attn/MatMul_1: | ███ | 1.195%
/model.10/cv2/conv/Conv: | ███ | 1.190%
/model.23/cv2.1/cv2.1.2/Conv: | ███ | 1.143%
/model.9/cv2/conv/Conv: | ███ | 1.026%
/model.22/m.0/m/m.1/cv2/conv/Conv: | ███ | 0.995%
/model.22/m.0/m/m.0/cv2/conv/Conv: | ███ | 0.994%
/model.22/m.0/m/m.0/cv1/conv/Conv: | ███ | 0.990%
/model.23/cv3.1/cv3.1.0/cv3.1.0.0/conv/Conv: | ███ | 0.977%
/model.8/m.0/m/m.1/cv2/conv/Conv: | ███ | 0.966%
/model.23/cv3.1/cv3.1.1/cv3.1.1.0/conv/Conv: | ███ | 0.964%
/model.23/cv3.0/cv3.0.1/cv3.0.1.0/conv/Conv: | ██ | 0.910%
/model.22/m.0/m/m.1/cv1/conv/Conv: | ██ | 0.768%
/model.10/m/m.0/ffn/ffn.0/conv/Conv: | ██ | 0.763%
/model.23/cv3.2/cv3.2.1/cv3.2.1.0/conv/Conv: | ██ | 0.752%
/model.22/m.0/cv1/conv/Conv: | ██ | 0.725%
/model.23/cv3.0/cv3.0.0/cv3.0.0.0/conv/Conv: | █ | 0.564%
/model.9/cv1/conv/Conv: | █ | 0.381%
/model.23/cv2.2/cv2.2.2/Conv: | █ | 0.363%
/model.0/conv/Conv: | | 0.172%
/model.23/cv3.1/cv3.1.2/Conv: | | 0.083%
/model.23/cv3.2/cv3.2.2/Conv: | | 0.079%
/model.23/cv3.0/cv3.0.2/Conv: | | 0.047%
Analysing Layerwise quantization error:: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 89/89 [00:12<00:00, 7.40it/s]
Layer | NOISE:SIGNAL POWER RATIO
/model.0/conv/Conv: | ████████████████████ | 0.451%
/model.1/conv/Conv: | ████████████████████ | 0.450%
/model.3/conv/Conv: | ██████████████████ | 0.402%
/model.2/cv2/conv/Conv: | ██████████████ | 0.312%
/model.9/cv2/conv/Conv: | ██████████ | 0.222%
/model.4/cv2/conv/Conv: | ████████ | 0.178%
/model.10/cv1/conv/Conv: | ███████ | 0.155%
/model.2/cv1/conv/Conv: | ██████ | 0.145%
/model.4/m.0/cv1/conv/Conv: | ██████ | 0.139%
/model.4/cv1/conv/Conv: | ██████ | 0.137%
/model.22/cv1/conv/Conv: | █████ | 0.119%
/model.5/conv/Conv: | █████ | 0.118%
/model.16/cv1/conv/Conv: | █████ | 0.118%
/model.4/m.0/cv2/conv/Conv: | █████ | 0.117%
/model.22/m.0/cv1/conv/Conv: | █████ | 0.104%
/model.2/m.0/cv1/conv/Conv: | ███ | 0.068%
/model.7/conv/Conv: | ███ | 0.057%
/model.13/cv2/conv/Conv: | ██ | 0.053%
/model.13/cv1/conv/Conv: | ██ | 0.047%
/model.16/cv2/conv/Conv: | ██ | 0.044%
/model.10/cv2/conv/Conv: | ██ | 0.043%
/model.6/cv2/conv/Conv: | ██ | 0.042%
/model.19/cv2/conv/Conv: | ██ | 0.041%
/model.6/cv1/conv/Conv: | ██ | 0.039%
/model.6/m.0/m/m.1/cv1/conv/Conv: | █ | 0.030%
/model.2/m.0/cv2/conv/Conv: | █ | 0.029%
/model.10/m/m.0/attn/qkv/conv/Conv: | █ | 0.029%
/model.10/m/m.0/attn/pe/conv/Conv: | █ | 0.027%
/model.8/cv2/conv/Conv: | █ | 0.027%
/model.19/cv1/conv/Conv: | █ | 0.026%
/model.19/m.0/cv1/conv/Conv: | █ | 0.025%
/model.22/m.0/m/m.1/cv1/conv/Conv: | █ | 0.025%
/model.10/m/m.0/ffn/ffn.0/conv/Conv: | █ | 0.024%
/model.23/cv2.1/cv2.1.0/conv/Conv: | █ | 0.023%
/model.16/m.0/cv1/conv/Conv: | █ | 0.021%
/model.23/cv2.0/cv2.0.0/conv/Conv: | █ | 0.021%
/model.13/m.0/cv2/conv/Conv: | █ | 0.020%
/model.9/cv1/conv/Conv: | █ | 0.020%
/model.23/cv2.2/cv2.2.0/conv/Conv: | █ | 0.019%
/model.10/m/m.0/attn/proj/conv/Conv: | █ | 0.018%
/model.8/cv1/conv/Conv: | █ | 0.017%
/model.16/m.0/cv2/conv/Conv: | █ | 0.017%
/model.22/cv2/conv/Conv: | █ | 0.015%
/model.13/m.0/cv1/conv/Conv: | █ | 0.014%
/model.6/m.0/cv3/conv/Conv: | █ | 0.013%
/model.20/conv/Conv: | █ | 0.012%
/model.22/m.0/cv3/conv/Conv: | | 0.011%
/model.23/cv2.2/cv2.2.1/conv/Conv: | | 0.011%
/model.17/conv/Conv: | | 0.011%
/model.19/m.0/cv2/conv/Conv: | | 0.010%
/model.23/cv2.2/cv2.2.2/Conv: | | 0.010%
/model.23/cv2.1/cv2.1.2/Conv: | | 0.009%
/model.8/m.0/cv3/conv/Conv: | | 0.009%
/model.8/m.0/cv1/conv/Conv: | | 0.009%
/model.23/cv2.0/cv2.0.2/Conv: | | 0.009%
/model.22/m.0/m/m.0/cv2/conv/Conv: | | 0.008%
/model.10/m/m.0/ffn/ffn.1/conv/Conv: | | 0.008%
/model.6/m.0/m/m.0/cv2/conv/Conv: | | 0.008%
/model.23/cv2.1/cv2.1.1/conv/Conv: | | 0.007%
/model.8/m.0/m/m.0/cv1/conv/Conv: | | 0.007%
/model.6/m.0/m/m.0/cv1/conv/Conv: | | 0.006%
/model.8/m.0/m/m.1/cv1/conv/Conv: | | 0.006%
/model.23/cv2.0/cv2.0.1/conv/Conv: | | 0.006%
/model.8/m.0/m/m.0/cv2/conv/Conv: | | 0.006%
/model.22/m.0/m/m.0/cv1/conv/Conv: | | 0.006%
/model.23/cv3.1/cv3.1.1/cv3.1.1.0/conv/Conv: | | 0.005%
/model.23/cv3.1/cv3.1.2/Conv: | | 0.004%
/model.6/m.0/m/m.1/cv2/conv/Conv: | | 0.004%
/model.6/m.0/cv1/conv/Conv: | | 0.004%
/model.23/cv3.0/cv3.0.0/cv3.0.0.1/conv/Conv: | | 0.004%
/model.23/cv3.2/cv3.2.0/cv3.2.0.1/conv/Conv: | | 0.004%
/model.23/cv3.1/cv3.1.0/cv3.1.0.1/conv/Conv: | | 0.004%
/model.8/m.0/m/m.1/cv2/conv/Conv: | | 0.003%
/model.23/cv3.1/cv3.1.1/cv3.1.1.1/conv/Conv: | | 0.003%
/model.22/m.0/m/m.1/cv2/conv/Conv: | | 0.003%
/model.23/cv3.2/cv3.2.1/cv3.2.1.0/conv/Conv: | | 0.003%
/model.23/cv3.0/cv3.0.0/cv3.0.0.0/conv/Conv: | | 0.002%
/model.10/m/m.0/attn/MatMul_1: | | 0.002%
/model.23/cv3.2/cv3.2.1/cv3.2.1.1/conv/Conv: | | 0.001%
/model.23/cv3.2/cv3.2.2/Conv: | | 0.001%
/model.23/cv3.0/cv3.0.1/cv3.0.1.0/conv/Conv: | | 0.001%
/model.23/cv3.0/cv3.0.2/Conv: | | 0.001%
/model.23/cv3.0/cv3.0.1/cv3.0.1.1/conv/Conv: | | 0.001%
/model.23/cv3.2/cv3.2.0/cv3.2.0.0/conv/Conv: | | 0.001%
/model.6/m.0/cv2/conv/Conv: | | 0.001%
/model.22/m.0/cv2/conv/Conv: | | 0.000%
/model.8/m.0/cv2/conv/Conv: | | 0.000%
/model.23/cv3.1/cv3.1.0/cv3.1.0.0/conv/Conv: | | 0.000%
/model.10/m/m.0/attn/MatMul: | | 0.000%
ESP32-S3 Quantization results
With the same inputs, The mAP50-95 on COCO val2017 after quantization is 0.37, marginally lower than ESP32-P4 quantization results.
Note
If the model inference speed is a higher priority and a certain degree of accuracy loss is acceptable, you may consider quantizing the model with an input size of 320x320 for the YOLO11N model. The model inference speed of different input resolutions can be found in README.md .
Model deployment
Object detection base class
Pre-process
ImagePreprocessor class contains the common pre-precoess pipeline, color conversion, crop, resize, normalization, quantize。