Note

The version corresponding to the current document is ESP-IDF v5.5.0

HTTP Client Example

[中文]

Note

This document is automatically translated using AI. Please excuse any detailed errors. The official English version is still in progress.

Example Description

This example demonstrates how to configure an HTTP client on ESP32, establish a connection with the server via HTTP, and shows the basic operations of request sending, response receiving, and event callback. Through this example, developers can learn about the initialization process of the HTTP client under different configuration methods, the usage of structure members, and the event callback handling mechanism, providing a reference for subsequent implementation of specific HTTP requests and response processing.

Running Method

The complete code of the example can be found at http client example. Detailed configuration and running steps can be found in the README.md file under the examples directory. Before running, you need to correctly set the Wi-Fi SSID and password through menuconfig to enable the device to successfully connect to the network.

Header File Description

The header files used in this example cover the system, network, FreeRTOS, and HTTP/TLS modules, building the core functions of task scheduling and secure network communication.

The header files are classified by function as follows:

  1. Standard C library functions: Provide basic functions such as string processing, memory management, character type judgment, and system parameter macros.

#include <string.h>
#include <sys/param.h>
#include <stdlib.h>
#include <ctype.h>

  1. System and log: Provide system control and log functions for printing logs, obtaining system information, and performing system operations.

#include "esp_log.h"
#include "esp_system.h"

  1. NVS storage: Provides non-volatile storage functions for initializing and managing NVS data.

#include "nvs_flash.h"

  1. Network and event: Provide network interface initialization and event loop mechanism for initializing the network, handling network events, and event loop management.

#include "esp_event.h"
#include "esp_netif.h"

  1. TLS/HTTP and HTTP client: Provide HTTP/HTTPS client and secure connection functions for sending requests, receiving responses, and certificate verification. Certificate verification is an optional header file, only used when HTTPS certificate verification is needed to ensure connection security.

#include "esp_tls.h"
#if CONFIG_MBEDTLS_CERTIFICATE_BUNDLE
#include "esp_crt_bundle.h"
#endif
#include "esp_http_client.h"

  1. FreeRTOS related: Provide real-time operating system functions for task creation, scheduling, and kernel operations.

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

  1. Example auxiliary: Provide example auxiliary functions for quickly initializing the network and handling common operations in the example.

#include "protocol_examples_common.h"
#include "protocol_examples_utils.h"

Macro Definition Description

This example sets the size of the HTTP data reception and output buffer through macro definition, in bytes:

#define MAX_HTTP_RECV_BUFFER 512      // Define the buffer size for receiving HTTP response data each time
#define MAX_HTTP_OUTPUT_BUFFER 2048   // Define the buffer size for storing processed HTTP response data

External Constant Description

This example defines related data of the root certificate through external constants, which is used for server certificate verification in HTTPS requests to ensure a secure connection.

  1. howsmyssl.com root certificate address: Points to the start and end positions of the howsmyssl.com root certificate embedded in the program, used to verify the server identity when ESP32 accesses the corresponding website, ensuring a secure connection. This website is mainly used to test the HTTPS request function of ESP32.

extern const char howsmyssl_com_root_cert_pem_start[] asm("_binary_howsmyssl_com_root_cert_pem_start");
extern const char howsmyssl_com_root_cert_pem_end[]   asm("_binary_howsmyssl_com_root_cert_pem_end");

  1. postman root certificate address: Points to the start and end positions of the postman root certificate embedded in the program, used to verify the identity when ESP32 accesses Postman Echo API or other test servers provided by Postman, facilitating the demonstration of sending HTTP requests and receiving responses.

extern const char postman_root_cert_pem_start[] asm("_binary_postman_root_cert_pem_start");
extern const char postman_root_cert_pem_end[]   asm("_binary_postman_root_cert_pem_end");

Note

The two websites used in the example are for demonstration purposes only. In actual projects, if you need to access other HTTPS servers, you should provide the root certificate of the corresponding server.

Task Function Description

In this example, except for the event callback function, all other task functions show the complete process of the HTTP/S client, including the configuration of the esp_http_client_config_t structure and the execution of request operations. The example demonstrates how to complete client configuration and request operations according to specific application requirements by configuring different members of the structure:

  • HTTP URL configuration

  • Basic authentication configuration

  • Digest authentication configuration

  • HTTPS URL configuration

  • Query parameter encoding

  • Different redirect type configuration

  • Cross-protocol redirect configuration

  • Chunked data transmission

  • Stream data reading

  • Asynchronous request handling

  • Accessing invalid URL

  • Using low-level API

  • Partial content download

The specific structure members of esp_http_client_config_t can refer to ESP HTTP API. Configuration instructions can refer to HTTP structure configuration instructions. The execution logic of each request operation can refer to Request operation instructions. The overall function logic is relatively simple, and its core purpose is to demonstrate the complete process from configuration to request, helping developers understand how to choose the appropriate configuration method in different scenarios and verify its effect.

Event Callback Function

_http_event_handler() is used to handle various events triggered by the HTTP client during the request process. Through this function, the application layer can obtain status information, receive data, or perform necessary resource management operations at different stages. The events that the client may trigger can be referred to Event Handling.

The example code only shows some common handling methods, some of which only print logs, mainly for demonstrating the event triggering process. In actual applications, it can be expanded into data parsing, error recovery, or custom resource management operations according to needs.

Its execution logic is:

  1. Create variables:

  • Create the variable output_buffer to store the HTTP response body data, that is, the actual data returned by the server in the response.

  • Create the variable output_len to record the length of the received data, which is used to control the position of data copying and prevent out-of-bounds access.

  1. Event handling: After receiving the event, enter the event handling process and perform corresponding operations according to the event type.

  • HTTP_EVENT_ERROR indicates that an error occurred during the request process. It is usually triggered when the connection fails, read/write errors, or underlying TLS/transport errors occur.

  • HTTP_EVENT_ON_CONNECTED is triggered after successfully establishing a TCP or TLS connection.

  • HTTP_EVENT_HEADER_SENT is triggered when the request header has been sent.

  • HTTP_EVENT_ON_HEADER is triggered when a single response header field is received.

  • HTTP_EVENT_ON_DATA is triggered when a data fragment of the response body is received. For larger or chunked responses, this event will be triggered multiple times, providing a piece of data each time.

    • Clear the buffer: If this is the first write and the caller (i.e., application layer code) provides a user_data buffer, clear this buffer to avoid residual data and ensure clean space for new requests.

    • chunk encoding judgment, the example only provides the processing logic when non-chunk encoding.

    • Select the target buffer and calculate the copyable length:

      • If the caller provides a user_data buffer, use MIN() to calculate the maximum number of bytes that can be written to user_data this time to prevent out-of-bounds, and copy the current fragment to the appropriate position of user_data.

      • If the caller does not provide a user_data buffer, apply for a dynamic buffer output_buffer:

        • Call esp_http_client_get_content_length() to get the total length of the response, which is used to allocate enough heap memory at once. For related usage and parameter description, please refer to ESP HTTP Client API.

        • Allocate buffer memory dynamically according to the response length, and ensure that the memory allocation is successful.

        • Use MIN() to calculate the maximum number of bytes that can be written to output_buffer this time to prevent out-of-bounds, and copy the current fragment to the appropriate position of output_buffer.

    • Update the received length.

  • HTTP_EVENT_ON_FINISH is triggered when the response is received and the request processing is finished, which is suitable for final cleanup or processing of the complete response.

    • If dynamic memory is allocated, release it.

    • Reset the received length to prepare for the next request.

  • HTTP_EVENT_DISCONNECTED is triggered when the connection is disconnected, which may be a normal disconnection or caused by an error. It is suitable for recording underlying errors and performing resource cleanup.

    • Invoke esp_tls_get_and_clear_last_error() to obtain and clear the last error code from the underlying TLS layer, separate out ESP error codes and MbedTLS error codes, and print detailed error information for troubleshooting connection or certificate issues. For usage and parameter explanations, refer to ESP TLS API.

    • If dynamic memory has been allocated, release it and reset the received length to prepare for the next request.

  • HTTP_EVENT_REDIRECT is triggered when a 3xx redirect response status code is received and set to manual redirection. You can modify the request header or explicitly allow/deny redirection operations in the callback.

HTTP Client Configuration and Execution

The execution logic of the remaining task functions is the same, so they are not explained one by one. Only the differences in structure configuration and usage scenarios of each function are explained. The standard execution process of the HTTP client can be referred to HTTP client execution process.

HTTP URL Configuration

  • http_rest_with_url() demonstrates the process of initiating various HTTP requests and handling server responses after configuration, through the complete URL.

  • http_rest_with_hostname_path() demonstrates the process of initiating various HTTP requests and handling server responses after configuration, through partial fields (host, path, etc.).

In the above examples, because the resources or interfaces accessed by each request are different, you need to call esp_http_client_set_url() to reset the access address before setting the request method each time. For usage and parameter explanations, refer to ESP HTTP Client API.

The flexibility of URL configuration methods is demonstrated by two ways of resetting the access address:

  • The complete URL method is intuitive and suitable for directly specifying the access target (including protocol, host, port, and path) in the application, which is convenient for quick verification or access to fixed resources.

  • The host and path method is more flexible. When the host part remains unchanged, you can access different resources by simply modifying the path. This is suitable for scenarios where you need to frequently switch interfaces or resources under the same server.

This design shows that the ESP HTTP client can support both simple and direct configuration and meet the needs of scenario reuse and dynamic resource switching.

Basic Authentication Configuration: Execute when CONFIG_ESP_HTTP_CLIENT_ENABLE_BASIC_AUTH is enabled.

  • http_auth_basic() demonstrates the standard Basic authentication process.

  • http_auth_basic_redirect() demonstrates the handling of Basic authentication in redirection scenarios. Without additional authentication type or retry count configuration, the client can still handle the authentication challenge returned by the server.

For explanations about Basic authentication, refer to Basic Authentication. For explanations about structure configuration, refer to the Authentication Mechanism Configuration section in HTTP Authentication and Security Configuration.

Digest Authentication Configuration: Execute when CONFIG_ESP_HTTP_CLIENT_ENABLE_DIGEST_AUTH is enabled.

  • http_auth_digest_md5() demonstrates Digest authentication through the MD5 algorithm.

  • http_auth_digest_sha256() demonstrates Digest authentication using the SHA-256 algorithm.

For information on Digest authentication, refer to Digest Authentication.

In the above example, to demonstrate Digest authentication, the client triggers the server to return the corresponding authentication challenge through different URLs.

However, it should be noted that the URL path set by the client cannot determine whether the server returns an authentication challenge, nor can it specify the type of digest algorithm. The Digest authentication process is completed by the client and the server together, and developers only need to judge the status according to the response code and perform corresponding processing. At the same time, Digest authentication does not depend on the specific member configuration of esp_http_client_config_t, but if SHA-256 algorithm is used for authentication, because the authentication header of this algorithm is longer than MD5, the default sending buffer may not be enough to accommodate the complete request header, so the structure member buffer_size_tx needs to be modified to increase the sending buffer to ensure that the authentication request can be sent correctly.

HTTPS URL Configuration

  • https_with_url() demonstrates the process of initiating an HTTPS request through a complete URL and handling the server’s response.

  • https_with_hostname_path() demonstrates the process of initiating an HTTPS request through partial fields (host, path, etc.) and handling the server’s response.

In the above examples, different methods of configuring URLs are demonstrated. For specific instructions on configuration methods, refer to the URL Configuration section in HTTP Authentication and Security Configuration.

When accessing HTTPS services, it is necessary to perform a TLS handshake and verify the server certificate. The above examples also show various ways to configure the root certificate, among which https_with_url() uses the system precompiled certificate bundle to verify the server certificate, which is only executed when the CONFIG_MBEDTLS_CERTIFICATE_BUNDLE macro is enabled. For specific instructions, refer to the Server Certificate Verification section in HTTP Authentication and Security Configuration.

The PEM format certificate in this example is obtained through an external variable, please check External Constant Description.

Query Parameter Encoding

  • http_encoded_query() demonstrates how to URL encode query parameters with special characters and use them correctly in HTTP GET requests.

For specific instructions on URL encoding, refer to URL Encoding

The execution flow of this example is as follows:

  1. Configure the HTTP client through host and path.

  2. Create two string variables to store the original query parameters and the URL-encoded query parameters.

  3. Call example_uri_encode() to URL encode the original query parameters and return the length of the encoded string. A return value of 0 or negative indicates encoding failure.

  4. If the encoding is successful, print the encoding result to help with debugging, and store the encoded string in the query member of the configuration structure. For detailed instructions on this member, refer to the URL Configuration section in HTTP Authentication and Security Configuration.

  5. Use the configuration structure to initiate a request and get the server’s response.

Note

In the example, the encoding result is stored in a fixed-length buffer, so it is necessary to ensure that the buffer is large enough to accommodate the potentially expanded string after encoding, to avoid the risk of buffer overflow or data truncation.

Configuration of Different Redirect Types

  • http_relative_redirect() demonstrates the automatic handling process of relative path redirection.

  • http_absolute_redirect() demonstrates the automatic handling process of absolute path redirection.

  • http_absolute_redirect_manual() demonstrates the manual handling process of absolute path redirection.

For information on redirect types, refer to Redirect Types. For information on manual handling of redirects and structure configuration, refer to Automatic Redirect Behavior.

In the above examples, to demonstrate different types of redirection, the client uses different URLs, where the path part is set according to the type of redirection to trigger the server to return the corresponding Location, thereby stably reproducing the processing flow of relative and absolute path redirection.

However, it should be noted that the URL path set by the client itself cannot determine the type of redirection returned by the server. Whether the server returns a relative or absolute path depends entirely on its interface implementation. The paths used in the examples (i.e., /relative-redirect/3, /absolute-redirect/3) actually correspond to the test interfaces pre-designed on the server, which are used to consistently return the Location of relative or absolute paths, thereby ensuring the stable demonstration of different redirection scenarios. At the same time, the type of redirection does not depend on the configuration of the esp_http_client_config_t structure member, and the client is only responsible for generating and initiating new requests according to the Location returned by the server.

Cross-protocol redirection configuration

  • http_relative_redirect() demonstrates the scenario of redirecting from HTTP to HTTPS by specifying the target as an HTTPS address.

In this example, the client triggers the server to return a Location that redirects to HTTPS through the URL, thereby demonstrating how the client handles cross-protocol redirection. It should be noted that the redirection target itself does not depend on the client’s URL, but is determined by the Location field in the server’s response header. The specific URL set in the example is only to utilize the test interface provided by the server to ensure that the returned redirection target is HTTPS, thereby stably demonstrating the processing flow of cross-protocol redirection, and it is not that the client can directly control the server to return HTTPS.

When the client receives a Location pointing to HTTPS and initiates a new request, it must perform a TLS handshake and verify the server certificate. If a trusted root certificate is not pre-configured, the TLS verification will fail, causing the request to be incomplete. Therefore, to support possible cross-protocol redirection, the client needs to set the certificate information in advance when configuring the esp_http_client_config_t structure to ensure that the server’s identity can be correctly verified when automatically following redirection. For specific instructions, please refer to the Server Certificate Verification section in the HTTP Authentication and Security Configuration subsection. This example is configured through cert_pem, and the certificate is obtained through an external variable, please refer to the External Constant Description.

Chunked data transmission

  • http_download_chunk() demonstrates the basic process of ESP32 using HTTP client for chunked transmission download.

Chunked data transmission is a transmission method where the server sends the response body in several chunks step by step, and the client receives and processes it chunk by chunk. For further explanation, please refer to Chunked Transfer Encoding.

In this example, the client triggers the server to return a chunked transfer response through the URL. Whenever the server sends a data chunk, the HTTP client triggers the HTTP_EVENT_ON_DATA event callback and processes the currently received data chunk in the callback.

It should be noted that the URL path set by the client itself cannot determine whether the server returns a chunked transfer response. Also, the chunked transfer response does not depend on the configuration of the esp_http_client_config_t structure member, and the client only needs to set the processing logic of the chunked data through the event callback.

Streamed data reading

  • http_perform_as_stream_reader() demonstrates the process of ESP32 performing streamed reading through the underlying API to obtain a complete response.

For explanations and execution flow of streamed transmission, please refer to the Streamed Transmission section in the Request Operation Instructions subsection.

Asynchronous request handling

  • https_async() demonstrates how to initiate asynchronous HTTPS requests in ESP-IDF using esp_http_client, including POST and HEAD methods.

For explanations on asynchronous requests, please refer to Asynchronous Requests.

The main configurations in the structure include:

During the execution of the program, it is necessary to loop call esp_http_client_perform() to ensure the request is completed. For related usage and parameter description, refer to ESP HTTP Client API.

In this example, after sending the POST request, the program calls esp_http_client_cleanup() to release the resources occupied by the HTTPS client, so before executing subsequent GET requests, it is necessary to reinitialize the client and apply the configuration. For related usage and parameter description, refer to ESP HTTP Client API.

Access Invalid URL

  • https_with_invalid_url() demonstrates the process of handling when accessing an invalid URL.

This example tests and demonstrates the error handling and log output of the ESP32 HTTP client when the URL is invalid or inaccessible by setting the URL to a non-existent HTTPS address, which helps to understand the types of errors that may be encountered when an HTTPS request fails and how to capture them.

The execution result is as follows:

I (4430) HTTP_CLIENT: HTTPS request with invalid url =>
E (4500) esp-tls: couldn't get hostname for :not.existent.url: getaddrinfo() returns 202, addrinfo=0x0
E (4500) esp-tls: Failed to open new connection
E (4510) transport_base: Failed to open a new connection
E (4510) HTTP_CLIENT: Connection failed, sock < 0
I (4520) HTTP_CLIENT: HTTP_EVENT_ERROR
E (4520) HTTP_CLIENT: Error perform http request ESP_ERR_HTTP_CONNECT
I (4530) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED
I (4530) HTTP_CLIENT: Last esp error code: 0x8001
I (4530) HTTP_CLIENT: Last mbedtls failure: 0x0
I (4540) HTTP_CLIENT: Finish http example

The log print shows that when trying to access a non-existent URL, the client cannot resolve the hostname, the connection establishment fails, and triggers HTTP_EVENT_ERROR and HTTP_EVENT_DISCONNECTED events, but there is no TLS layer error.

Using Low-Level API

  • http_native_request() demonstrates the process of ESP32 performing a one-time read through the low-level API to get a complete response.

This example follows the same logic as the Stream Reading Data example, but uses esp_http_client_read_response() to read the response body into the buffer all at once. For usage and parameter explanations, refer to ESP HTTP Client API.

The example demonstrates GET and POST requests. After the GET request is completed, you need to call esp_http_client_close() to disconnect, so that subsequent requests can be made; before initiating a new request, you need to reinitialize the client and apply the configuration. For usage and parameter explanations, refer to ESP HTTP Client API.

Partial Content Download

  • http_partial_download() demonstrates how to download part of a remote file through the HTTPS client, rather than the entire file.

This method utilizes the HTTP/S protocol’s Range request header, allowing the client to request only a specified byte range of the file. You only need to call esp_http_client_set_header() and pass in the Range request header and specify the download method. For usage and parameter explanations, refer to ESP HTTP Client API. The example shows three common ways of partial download:

  • Exclude the first 10 bytes at the beginning of the file: Range: bytes=10-, download the content from the 11th byte to the end of the file, which can be used to skip the file header or implement breakpoint continuation.

  • Only download the last 10 bytes of the file: Range: bytes=-10, download a small segment of data at the end of the file, which can be used to verify file integrity or obtain information at the end of the file.

  • Download the 11~20 bytes in the middle of the file: Range: bytes=11-20, precisely download a segment of data in the middle of the file, suitable for block processing of large files or stream parsing.

In execution, note that:

  • Not all servers support Range requests, ensure that the server returns a 206 Partial Content response.

  • The content length is the length of the response body, not the total length of the file, you need to judge the total size of the file in conjunction with the HTTP/S header’s Content-Range information.

Centralized Function Calls

http_test_task() is a main entry function, calling it will sequentially execute each HTTP task function in the example, without the need to call each task individually.

Main Function Description

This function serves as the program entry point, used to complete system initialization, network configuration, and execute each task function.

Its execution flow is as follows:

  1. Initialize Non-Volatile Storage NVS.

  2. Call esp_netif_init() to initialize the network interface layer ESP-NETIF, preparing for Wi-Fi or Ethernet functionality.

  3. Call esp_event_loop_create_default() to create the default event loop, providing a foundation for system and MQTT client event handling. For API introduction and parameter explanation, refer to event loop library.

  4. Call example_connect() to configure and connect to the network (Wi-Fi or Ethernet), ensuring the device has networking capabilities.

  5. Call xTaskCreate() to create the main entry function http_test_task(). For further explanation and usage examples, refer to dynamically creating tasks.