ESP-TLS¶

Overview¶

The ESP-TLS component provides a simplified API interface for accessing the commonly used TLS functionality. It supports common scenarios like CA certification validation, SNI, ALPN negotiation, non-blocking connection among others. All the configuration can be specified in the esp_tls_cfg_t data structure. Once done, TLS communication can be conducted using the following APIs:

esp_tls_conn_new(): for opening a new TLS connection.

esp_tls_conn_read(): for reading from the connection.

esp_tls_conn_write(): for writing into the connection.

esp_tls_conn_delete(): for freeing up the connection.

Any application layer protocol like HTTP1, HTTP2 etc can be executed on top of this layer.

Application Example¶

Simple HTTPS example that uses ESP-TLS to establish a secure socket connection: protocols/https_request.

Tree structure for ESP-TLS component¶

├── esp_tls.c
├── esp_tls.h
├── esp_tls_mbedtls.c
├── esp_tls_wolfssl.c
└── private_include
    ├── esp_tls_mbedtls.h
    └── esp_tls_wolfssl.h

The ESP-TLS component has a file esp-tls/esp_tls.h which contain the public API headers for the component. Internally ESP-TLS component uses one of the two SSL/TLS Libraries between mbedtls and wolfssl for its operation. API specific to mbedtls are present in esp-tls/private_include/esp_tls_mbedtls.h and API specific to wolfssl are present in esp-tls/private_include/esp_tls_wolfssl.h.

TLS Server verification¶

The ESP-TLS provides multiple options for TLS server verification on the client side. The ESP-TLS client can verify the server by validating the peer’s server certificate or with the help of pre-shared keys. The user should select only one of the following options in the esp_tls_cfg_t structure for TLS server verification. If no option is selected then client will return a fatal error by default at the time of the TLS connection setup.

cacert_buf and cacert_bytes: The CA certificate can be provided in a buffer to the esp_tls_cfg_t structure. The ESP-TLS will use the CA certificate present in the buffer to verify the server. The following variables in esp_tls_cfg_t structure must be set.

cacert_buf - pointer to the buffer which contains the CA cert.

cacert_bytes - size of the CA certificate in bytes.

use_global_ca_store: The global_ca_store can be initialized and set at once. Then it can be used to verify the server for all the ESP-TLS connections which have set use_global_ca_store = true in their respective esp_tls_cfg_t structure. See API Reference section below on information regarding different API used for initializing and setting up the global_ca_store.

crt_bundle_attach: The ESP x509 Certificate Bundle API provides an easy way to include a bundle of custom x509 root certificates for TLS server verification. More details can be found at ESP x509 Certificate Bundle

psk_hint_key: To use pre-shared keys for server verification, CONFIG_ESP_TLS_PSK_VERIFICATION should be enabled in the ESP-TLS menuconfig. Then the pointer to PSK hint and key should be provided to the esp_tls_cfg_t structure. The ESP-TLS will use the PSK for server verification only when no other option regarding the server verification is selected.

skip server verification: This is an insecure option provided in the ESP-TLS for testing purpose. The option can be set by enabling CONFIG_ESP_TLS_INSECURE and CONFIG_ESP_TLS_SKIP_SERVER_CERT_VERIFY in the ESP-TLS menuconfig. When this option is enabled the ESP-TLS will skip server verification by default when no other options for server verification are selected in the esp_tls_cfg_t structure. WARNING:Enabling this option comes with a potential risk of establishing a TLS connection with a server which has a fake identity, provided that the server certificate is not provided either through API or other mechanism like ca_store etc.

Underlying SSL/TLS Library Options¶

The ESP-TLS component has an option to use mbedtls or wolfssl as their underlying SSL/TLS library. By default only mbedtls is available and is used, wolfssl SSL/TLS library is available publicly at https://github.com/espressif/esp-wolfssl. The repository provides wolfssl component in binary format, it also provides few examples which are useful for understanding the API. Please refer the repository README.md for information on licensing and other options. Please see below option for using wolfssl in your project.

Note

As the library options are internal to ESP-TLS, switching the libries will not change ESP-TLS specific code for a project.

How to use wolfssl with ESP-IDF¶

There are two ways to use wolfssl in your project

Directly add wolfssl as a component in your project with following three commands.:

(First change directory (cd) to your project directory)
mkdir components
cd components
git clone https://github.com/espressif/esp-wolfssl.git

Add wolfssl as an extra component in your project.

Download wolfssl with:

git clone https://github.com/espressif/esp-wolfssl.git

Include esp-wolfssl in ESP-IDF with setting EXTRA_COMPONENT_DIRS in CMakeLists.txt/Makefile of your project as done in wolfssl/examples. For reference see Optional Project variables in build-system.

After above steps, you will have option to choose wolfssl as underlying SSL/TLS library in configuration menu of your project as follows:

idf.py/make menuconfig -> ESP-TLS -> choose SSL/TLS Library -> mbedtls/wolfssl

Comparison between mbedtls and wolfssl¶

The following table shows a typical comparison between wolfssl and mbedtls when protocols/https_request example (which has server authentication) was run with both SSL/TLS libraries and with all respective configurations set to default. (mbedtls IN_CONTENT length and OUT_CONTENT length were set to 16384 bytes and 4096 bytes respectively)

Property

Wolfssl

Mbedtls

Total Heap Consumed

~19 Kb

~37 Kb

Task Stack Used

~2.2 Kb

~3.6 Kb

Bin size

~858 Kb

~736 Kb

Note

These values are subject to change with change in configuration options and version of respective libraries.

Digital Signature with ESP-TLS¶

ESP-TLS provides support for using the Digital Signature (DS) with ESP32-S3. Use of the DS for TLS is supported only when ESP-TLS is used with mbedTLS (default stack) as its underlying SSL/TLS stack. For more details on Digital Signature, please refer to the Digital Signature Documentation. The technical details of Digital Signature such as how to calculate private key parameters can be found in ESP32-S3 Technical Reference Manual > Digital Signature (DS) [PDF]. The DS peripheral must be configured before it can be used to perform Digital Signature, see Configure the DS Peripheral in Digital Signature.

The DS peripheral must be initlized with the required encrypted private key parameters (obtained when the DS peripheral is configured). ESP-TLS internally initializes the DS peripheral when provided with the required DS context (DS parameters). Please see the below code snippet for passing the DS context to esp-tls context. The DS context passed to the esp-tls context should not be freed till the TLS connection is deleted.

#include "esp_tls.h"
esp_ds_data_ctx_t *ds_ctx;
/* initialize ds_ctx with encrypted private key parameters, which can be read from the nvs or
provided through the application code */
esp_tls_cfg_t cfg = {
    .clientcert_buf = /* the client cert */,
    .clientcert_bytes = /* length of the client cert */,
    /* other configurations options */
    .ds_data = (void *)ds_ctx,
};

Note

When using Digital Signature for the TLS connection, along with the other required params, only the client cert (clientcert_buf) and the DS params (ds_data) are required and the client key (clientkey_buf) can be set to NULL.

An example of mutual authentication with the DS peripheral can be found at ssl mutual auth which internally uses (ESP-TLS) for the TLS connection.

API Reference¶

Header File¶

components/esp-tls/esp_tls.h

Functions¶

esp_tls_t *esp_tls_init(void)¶

Create TLS connection.

This function allocates and initializes esp-tls structure handle.

Return: tls Pointer to esp-tls as esp-tls handle if successfully initialized, NULL if allocation error

esp_tls_t *esp_tls_conn_new(const char *hostname, int hostlen, int port, const esp_tls_cfg_t *cfg)¶

Create a new blocking TLS/SSL connection.

This function establishes a TLS/SSL connection with the specified host in blocking manner.

Note: This API is present for backward compatibility reasons. Alternative function with the same functionality is esp_tls_conn_new_sync (and its asynchronous version esp_tls_conn_new_async)

Return

pointer to esp_tls_t, or NULL if connection couldn’t be opened.

Parameters

[in] hostname: Hostname of the host.
[in] hostlen: Length of hostname.
[in] port: Port number of the host.
[in] cfg: TLS configuration as esp_tls_cfg_t. If you wish to open non-TLS connection, keep this NULL. For TLS connection, a pass pointer to esp_tls_cfg_t. At a minimum, this structure should be zero-initialized.

int esp_tls_conn_new_sync(const char *hostname, int hostlen, int port, const esp_tls_cfg_t *cfg, esp_tls_t *tls)¶

Create a new blocking TLS/SSL connection.

This function establishes a TLS/SSL connection with the specified host in blocking manner.

Return

-1 If connection establishment fails.
1 If connection establishment is successful.
0 If connection state is in progress.

Parameters

[in] hostname: Hostname of the host.
[in] hostlen: Length of hostname.
[in] port: Port number of the host.
[in] cfg: TLS configuration as esp_tls_cfg_t. If you wish to open non-TLS connection, keep this NULL. For TLS connection, a pass pointer to esp_tls_cfg_t. At a minimum, this structure should be zero-initialized.
[in] tls: Pointer to esp-tls as esp-tls handle.

esp_tls_t *esp_tls_conn_http_new(const char *url, const esp_tls_cfg_t *cfg)¶

Create a new blocking TLS/SSL connection with a given “HTTP” url.

The behaviour is same as esp_tls_conn_new() API. However this API accepts host’s url.

Return

pointer to esp_tls_t, or NULL if connection couldn’t be opened.

Parameters

[in] url: url of host.
[in] cfg: TLS configuration as esp_tls_cfg_t. If you wish to open non-TLS connection, keep this NULL. For TLS connection, a pass pointer to ‘esp_tls_cfg_t’. At a minimum, this structure should be zero-initialized.

int esp_tls_conn_new_async(const char *hostname, int hostlen, int port, const esp_tls_cfg_t *cfg, esp_tls_t *tls)¶

Create a new non-blocking TLS/SSL connection.

This function initiates a non-blocking TLS/SSL connection with the specified host, but due to its non-blocking nature, it doesn’t wait for the connection to get established.

Return

-1 If connection establishment fails.
0 If connection establishment is in progress.
1 If connection establishment is successful.

Parameters

[in] hostname: Hostname of the host.
[in] hostlen: Length of hostname.
[in] port: Port number of the host.
[in] cfg: TLS configuration as esp_tls_cfg_t. non_block member of this structure should be set to be true.
[in] tls: pointer to esp-tls as esp-tls handle.

int esp_tls_conn_http_new_async(const char *url, const esp_tls_cfg_t *cfg, esp_tls_t *tls)¶

Create a new non-blocking TLS/SSL connection with a given “HTTP” url.

The behaviour is same as esp_tls_conn_new() API. However this API accepts host’s url.

Return

-1 If connection establishment fails.
0 If connection establishment is in progress.
1 If connection establishment is successful.

Parameters

[in] url: url of host.
[in] cfg: TLS configuration as esp_tls_cfg_t.
[in] tls: pointer to esp-tls as esp-tls handle.

static ssize_t esp_tls_conn_write(esp_tls_t *tls, const void *data, size_t datalen)¶

Write from buffer ‘data’ into specified tls connection.

Return

>=0 if write operation was successful, the return value is the number of bytes actually written to the TLS/SSL connection.
<0 if write operation was not successful, because either an error occured or an action must be taken by the calling process.
ESP_TLS_ERR_SSL_WANT_READ/ ESP_TLS_ERR_SSL_WANT_WRITE. if the handshake is incomplete and waiting for data to be available for reading. In this case this functions needs to be called again when the underlying transport is ready for operation.

Parameters

[in] tls: pointer to esp-tls as esp-tls handle.
[in] data: Buffer from which data will be written.
[in] datalen: Length of data buffer.

static ssize_t esp_tls_conn_read(esp_tls_t *tls, void *data, size_t datalen)¶

Read from specified tls connection into the buffer ‘data’.

Return

>0 if read operation was successful, the return value is the number of bytes actually read from the TLS/SSL connection.
0 if read operation was not successful. The underlying connection was closed.
<0 if read operation was not successful, because either an error occured or an action must be taken by the calling process.

Parameters

[in] tls: pointer to esp-tls as esp-tls handle.
[in] data: Buffer to hold read data.
[in] datalen: Length of data buffer.

void esp_tls_conn_delete(esp_tls_t *tls)¶

Compatible version of esp_tls_conn_destroy() to close the TLS/SSL connection.

Note

This API will be removed in IDFv5.0

Parameters

[in] tls: pointer to esp-tls as esp-tls handle.

int esp_tls_conn_destroy(esp_tls_t *tls)¶

Close the TLS/SSL connection and free any allocated resources.

This function should be called to close each tls connection opened with esp_tls_conn_new() or esp_tls_conn_http_new() APIs.

Return

- 0 on success

-1 if socket error or an invalid argument

Parameters

[in] tls: pointer to esp-tls as esp-tls handle.

ssize_t esp_tls_get_bytes_avail(esp_tls_t *tls)¶

Return the number of application data bytes remaining to be read from the current record.

This API is a wrapper over mbedtls’s mbedtls_ssl_get_bytes_avail() API.

Return

-1 in case of invalid arg
bytes available in the application data record read buffer

Parameters

[in] tls: pointer to esp-tls as esp-tls handle.

esp_err_t esp_tls_get_conn_sockfd(esp_tls_t *tls, int *sockfd)¶

Returns the connection socket file descriptor from esp_tls session.

Return

- ESP_OK on success and value of sockfd will be updated with socket file descriptor for connection

ESP_ERR_INVALID_ARG if (tls == NULL || sockfd == NULL)

Parameters

[in] tls: handle to esp_tls context
[out] sockfd: int pointer to sockfd value.

esp_err_t esp_tls_init_global_ca_store(void)¶

Create a global CA store, initially empty.

This function should be called if the application wants to use the same CA store for multiple connections. This function initialises the global CA store which can be then set by calling esp_tls_set_global_ca_store(). To be effective, this function must be called before any call to esp_tls_set_global_ca_store().

Return

ESP_OK if creating global CA store was successful.
ESP_ERR_NO_MEM if an error occured when allocating the mbedTLS resources.

esp_err_t esp_tls_set_global_ca_store(const unsigned char *cacert_pem_buf, const unsigned int cacert_pem_bytes)¶

Set the global CA store with the buffer provided in pem format.

This function should be called if the application wants to set the global CA store for multiple connections i.e. to add the certificates in the provided buffer to the certificate chain. This function implicitly calls esp_tls_init_global_ca_store() if it has not already been called. The application must call this function before calling esp_tls_conn_new().

Return

ESP_OK if adding certificates was successful.
Other if an error occured or an action must be taken by the calling process.

Parameters

[in] cacert_pem_buf: Buffer which has certificates in pem format. This buffer is used for creating a global CA store, which can be used by other tls connections.
[in] cacert_pem_bytes: Length of the buffer.

void esp_tls_free_global_ca_store(void)¶

Free the global CA store currently being used.

The memory being used by the global CA store to store all the parsed certificates is freed up. The application can call this API if it no longer needs the global CA store.

esp_err_t esp_tls_get_and_clear_last_error(esp_tls_error_handle_t h, int *esp_tls_code, int *esp_tls_flags)¶

Returns last error in esp_tls with detailed mbedtls related error codes. The error information is cleared internally upon return.

Return

ESP_ERR_INVALID_STATE if invalid parameters
ESP_OK (0) if no error occurred
specific error code (based on ESP_ERR_ESP_TLS_BASE) otherwise

Parameters

[in] h: esp-tls error handle.
[out] esp_tls_code: last error code returned from mbedtls api (set to zero if none) This pointer could be NULL if caller does not care about esp_tls_code
[out] esp_tls_flags: last certification verification flags (set to zero if none) This pointer could be NULL if caller does not care about esp_tls_code

esp_err_t esp_tls_get_and_clear_error_type(esp_tls_error_handle_t h, esp_tls_error_type_t err_type, int *error_code)¶

Returns the last error captured in esp_tls of a specific type The error information is cleared internally upon return.

Return

ESP_ERR_INVALID_STATE if invalid parameters
ESP_OK if a valid error returned and was cleared

Parameters

[in] h: esp-tls error handle.
[in] err_type: specific error type
[out] error_code: last error code returned from mbedtls api (set to zero if none) This pointer could be NULL if caller does not care about esp_tls_code

mbedtls_x509_crt *esp_tls_get_global_ca_store(void)¶

Get the pointer to the global CA store currently being used.

The application must first call esp_tls_set_global_ca_store(). Then the same CA store could be used by the application for APIs other than esp_tls.

Note

Modifying the pointer might cause a failure in verifying the certificates.

Return

Pointer to the global CA store currently being used if successful.
NULL if there is no global CA store set.

esp_err_t esp_tls_plain_tcp_connect(const char *host, int hostlen, int port, const esp_tls_cfg_t *cfg, esp_tls_error_handle_t error_handle, int *sockfd)¶

Creates a plain TCP connection, returning a valid socket fd on success or an error handle.

Return

ESP_OK on success ESP_ERR_INVALID_ARG if invalid output parameters ESP-TLS based error codes on failure

Parameters

[in] host: Hostname of the host.
[in] hostlen: Length of hostname.
[in] port: Port number of the host.
[in] cfg: ESP-TLS configuration as esp_tls_cfg_t.
[out] error_handle: ESP-TLS error handle holding potential errors occurred during connection
[out] sockfd: Socket descriptor if successfully connected on TCP layer

Structures¶

struct psk_key_hint¶

ESP-TLS preshared key and hint structure.

Public Members

const uint8_t *key¶: key in PSK authentication mode in binary format

const size_t key_size¶: length of the key

const char *hint¶: hint in PSK authentication mode in string format

struct tls_keep_alive_cfg¶

esp-tls client session ticket ctx

Keep alive parameters structure

Public Members

bool keep_alive_enable¶: Enable keep-alive timeout

int keep_alive_idle¶: Keep-alive idle time (second)

int keep_alive_interval¶: Keep-alive interval time (second)

int keep_alive_count¶: Keep-alive packet retry send count

struct esp_tls_cfg¶

ESP-TLS configuration parameters.

Note

Note about format of certificates:

This structure includes certificates of a Certificate Authority, of client or server as well as private keys, which may be of PEM or DER format. In case of PEM format, the buffer must be NULL terminated (with NULL character included in certificate size).
Certificate Authority’s certificate may be a chain of certificates in case of PEM format, but could be only one certificate in case of DER format
Variables names of certificates and private key buffers and sizes are defined as unions providing backward compatibility for legacy *_pem_buf and *_pem_bytes names which suggested only PEM format was supported. It is encouraged to use generic names such as cacert_buf and cacert_bytes.

Public Members

const char **alpn_protos¶

Application protocols required for HTTP2. If HTTP2/ALPN support is required, a list of protocols that should be negotiated. The format is length followed by protocol name. For the most common cases the following is ok: const char **alpn_protos = { “h2”, NULL };

where ‘h2’ is the protocol name

const unsigned char *cacert_buf¶: Certificate Authority’s certificate in a buffer. Format may be PEM or DER, depending on mbedtls-support This buffer should be NULL terminated in case of PEM

const unsigned char *cacert_pem_buf¶: CA certificate buffer legacy name

unsigned int cacert_bytes¶: Size of Certificate Authority certificate pointed to by cacert_buf (including NULL-terminator in case of PEM format)

unsigned int cacert_pem_bytes¶: Size of Certificate Authority certificate legacy name

const unsigned char *clientcert_buf¶: Client certificate in a buffer Format may be PEM or DER, depending on mbedtls-support This buffer should be NULL terminated in case of PEM

const unsigned char *clientcert_pem_buf¶: Client certificate legacy name

unsigned int clientcert_bytes¶: Size of client certificate pointed to by clientcert_pem_buf (including NULL-terminator in case of PEM format)

unsigned int clientcert_pem_bytes¶: Size of client certificate legacy name

const unsigned char *clientkey_buf¶: Client key in a buffer Format may be PEM or DER, depending on mbedtls-support This buffer should be NULL terminated in case of PEM

const unsigned char *clientkey_pem_buf¶: Client key legacy name

unsigned int clientkey_bytes¶: Size of client key pointed to by clientkey_pem_buf (including NULL-terminator in case of PEM format)

unsigned int clientkey_pem_bytes¶: Size of client key legacy name

const unsigned char *clientkey_password¶: Client key decryption password string

unsigned int clientkey_password_len¶: String length of the password pointed to by clientkey_password

bool non_block¶: Configure non-blocking mode. If set to true the underneath socket will be configured in non blocking mode after tls session is established

bool use_secure_element¶: Enable this option to use secure element or atecc608a chip ( Integrated with ESP32-WROOM-32SE )

int timeout_ms¶: Network timeout in milliseconds

bool use_global_ca_store¶: Use a global ca_store for all the connections in which this bool is set.

const char *common_name¶: If non-NULL, server certificate CN must match this name. If NULL, server certificate CN must match hostname.

bool skip_common_name¶: Skip any validation of server certificate CN field

tls_keep_alive_cfg_t *keep_alive_cfg¶: Enable TCP keep-alive timeout for SSL connection

const psk_hint_key_t *psk_hint_key¶: Pointer to PSK hint and key. if not NULL (and certificates are NULL) then PSK authentication is enabled with configured setup. Important note: the pointer must be valid for connection

esp_err_t (*crt_bundle_attach)(void *conf)¶: Function pointer to esp_crt_bundle_attach. Enables the use of certification bundle for server verification, must be enabled in menuconfig

void *ds_data¶: Pointer for digital signature peripheral context

bool is_plain_tcp¶: Use non-TLS connection: When set to true, the esp-tls uses plain TCP transport rather then TLS/SSL connection. Note, that it is possible to connect using a plain tcp transport directly with esp_tls_plain_tcp_connect() API

struct ifreq *if_name¶: The name of interface for data to go through. Use the default interface without setting

struct esp_tls¶

ESP-TLS Connection Handle.

Public Members

mbedtls_ssl_context ssl¶: TLS/SSL context

mbedtls_entropy_context entropy¶: mbedTLS entropy context structure

mbedtls_ctr_drbg_context ctr_drbg¶: mbedTLS ctr drbg context structure. CTR_DRBG is deterministic random bit generation based on AES-256

mbedtls_ssl_config conf¶: TLS/SSL configuration to be shared between mbedtls_ssl_context structures

mbedtls_net_context server_fd¶: mbedTLS wrapper type for sockets

mbedtls_x509_crt cacert¶: Container for the X.509 CA certificate

mbedtls_x509_crt *cacert_ptr¶: Pointer to the cacert being used.

mbedtls_x509_crt clientcert¶: Container for the X.509 client certificate

mbedtls_pk_context clientkey¶: Container for the private key of the client certificate

int sockfd¶: Underlying socket file descriptor.

ssize_t (*read)(struct esp_tls *tls, char *data, size_t datalen)¶: Callback function for reading data from TLS/SSL connection.

ssize_t (*write)(struct esp_tls *tls, const char *data, size_t datalen)¶: Callback function for writing data to TLS/SSL connection.

esp_tls_conn_state_t conn_state¶: ESP-TLS Connection state

fd_set rset¶: read file descriptors

fd_set wset¶: write file descriptors

bool is_tls¶: indicates connection type (TLS or NON-TLS)

esp_tls_role_t role¶

esp-tls role

ESP_TLS_CLIENT
ESP_TLS_SERVER

esp_tls_error_handle_t error_handle¶: handle to error descriptor

Type Definitions¶

typedef enum esp_tls_conn_state esp_tls_conn_state_t¶: ESP-TLS Connection State.

typedef enum esp_tls_role esp_tls_role_t¶

typedef struct psk_key_hint psk_hint_key_t¶: ESP-TLS preshared key and hint structure.

typedef struct tls_keep_alive_cfg tls_keep_alive_cfg_t¶

esp-tls client session ticket ctx

Keep alive parameters structure

typedef struct esp_tls_cfg esp_tls_cfg_t¶

ESP-TLS configuration parameters.

Note

Note about format of certificates:

This structure includes certificates of a Certificate Authority, of client or server as well as private keys, which may be of PEM or DER format. In case of PEM format, the buffer must be NULL terminated (with NULL character included in certificate size).
Certificate Authority’s certificate may be a chain of certificates in case of PEM format, but could be only one certificate in case of DER format
Variables names of certificates and private key buffers and sizes are defined as unions providing backward compatibility for legacy *_pem_buf and *_pem_bytes names which suggested only PEM format was supported. It is encouraged to use generic names such as cacert_buf and cacert_bytes.

typedef struct esp_tls esp_tls_t¶: ESP-TLS Connection Handle.

Enumerations¶

enum esp_tls_conn_state¶

ESP-TLS Connection State.

Values:

ESP_TLS_INIT = 0¶

ESP_TLS_CONNECTING¶

ESP_TLS_HANDSHAKE¶

ESP_TLS_FAIL¶

ESP_TLS_DONE¶

enum esp_tls_role¶

Values:

ESP_TLS_CLIENT = 0¶

ESP_TLS_SERVER¶

Provide feedback about this document