ESP-MQTT

Overview

ESP-MQTT is an implementation of MQTT protocol client (MQTT is a lightweight publish/subscribe messaging protocol).

Features

  • Supports MQTT over TCP, SSL with mbedtls, MQTT over Websocket, MQTT over Websocket Secure.

  • Easy to setup with URI

  • Multiple instances (Multiple clients in one application)

  • Support subscribing, publishing, authentication, last will messages, keep alive pings and all 3 QoS levels (it should be a fully functional client).

Application Example

Configuration

URI

  • Curently support mqtt, mqtts, ws, wss schemes

  • MQTT over TCP samples:

    • mqtt://mqtt.eclipseprojects.io: MQTT over TCP, default port 1883:

    • mqtt://mqtt.eclipseprojects.io:1884 MQTT over TCP, port 1884:

    • mqtt://username:password@mqtt.eclipseprojects.io:1884 MQTT over TCP, port 1884, with username and password

  • MQTT over SSL samples:

    • mqtts://mqtt.eclipseprojects.io: MQTT over SSL, port 8883

    • mqtts://mqtt.eclipseprojects.io:8884: MQTT over SSL, port 8884

  • MQTT over Websocket samples:

    • ws://mqtt.eclipseprojects.io:80/mqtt

  • MQTT over Websocket Secure samples:

    • wss://mqtt.eclipseprojects.io:443/mqtt

  • Minimal configurations:

const esp_mqtt_client_config_t mqtt_cfg = {
    .uri = "mqtt://mqtt.eclipseprojects.io",
    // .user_context = (void *)your_context
};
esp_mqtt_client_handle_t client = esp_mqtt_client_init(&mqtt_cfg);
esp_mqtt_client_register_event(client, ESP_EVENT_ANY_ID, mqtt_event_handler, client);
esp_mqtt_client_start(client);

  • Note: By default mqtt client uses event loop library to post related mqtt events (connected, subscribed, published, etc.)

SSL

  • Get certificate from server, example: mqtt.eclipseprojects.io openssl s_client -showcerts -connect mqtt.eclipseprojects.io:8883 </dev/null 2>/dev/null|openssl x509 -outform PEM >mqtt_eclipse_org.pem

  • Check the sample application: examples/mqtt_ssl

  • Configuration:

const esp_mqtt_client_config_t mqtt_cfg = {
    .uri = "mqtts://mqtt.eclipseprojects.io:8883",
    .event_handle = mqtt_event_handler,
    .cert_pem = (const char *)mqtt_eclipse_org_pem_start,
};

If the certificate is not null-terminated then cert_len should also be set. Other SSL related configuration parameters are:

  • use_global_ca_store: use the global certificate store to verify server certificate, see esp-tls.h for more information

  • client_cert_pem: pointer to certificate data in PEM or DER format for SSL mutual authentication, default is NULL, not required if mutual authentication is not needed.

  • client_cert_len: length of the buffer pointed to by client_cert_pem. May be 0 for null-terminated pem.

  • client_key_pem: pointer to private key data in PEM or DER format for SSL mutual authentication, default is NULL, not required if mutual authentication is not needed.

  • client_key_len: length of the buffer pointed to by client_key_pem. May be 0 for null-terminated pem.

  • psk_hint_key: pointer to PSK struct defined in esp_tls.h to enable PSK authentication (as alternative to certificate verification). If not NULL and server/client certificates are NULL, PSK is enabled

  • alpn_protos: NULL-terminated list of protocols to be used for ALPN.

Last Will and Testament

MQTT allows for a last will and testament (LWT) message to notify other clients when a client ungracefully disconnects. This is configured by the following fields in the esp_mqtt_client_config_t-struct.

  • lwt_topic: pointer to the LWT message topic

  • lwt_msg: pointer to the LWT message

  • lwt_msg_len: length of the LWT message, required if lwt_msg is not null-terminated

  • lwt_qos: quality of service for the LWT message

  • lwt_retain: specifies the retain flag of the LWT message

Other Configuration Parameters

  • disable_clean_session: determines the clean session flag for the connect message, defaults to a clean session

  • keepalive: determines how many seconds the client will wait for a ping response before disconnecting, default is 120 seconds.

  • disable_auto_reconnect: enable to stop the client from reconnecting to server after errors or disconnects

  • user_context: custom context that will be passed to the event handler

  • task_prio: MQTT task priority, defaults to 5

  • task_stack: MQTT task stack size, defaults to 6144 bytes, setting this will override setting from menuconfig

  • buffer_size: size of MQTT send/receive buffer, default is 1024 bytes

  • username: pointer to the username used for connecting to the broker

  • password: pointer to the password used for connecting to the broker

  • client_id: pointer to the client id, defaults to ESP32_%CHIPID% where %CHIPID% are the last 3 bytes of MAC address in hex format

  • host: MQTT broker domain (ipv4 as string), setting the uri will override this

  • port: MQTT broker port, specifying the port in the uri will override this

  • transport: sets the transport protocol, setting the uri will override this

  • refresh_connection_after_ms: refresh connection after this value (in milliseconds)

  • event_handle: handle for MQTT events as a callback in legacy mode

  • event_loop_handle: handle for MQTT event loop library

For more options on esp_mqtt_client_config_t, please refer to API reference below

Change settings in Project Configuration Menu

The settings for MQTT can be found using idf.py menuconfig, under Component config -> ESP-MQTT Configuration

The following settings are available:

Events

The following events may be posted by the MQTT client:

  • MQTT_EVENT_BEFORE_CONNECT: The client is initialized and about to start connecting to the broker.

  • MQTT_EVENT_CONNECTED: The client has successfully established a connection to the broker. The client is now ready to send and receive data.

  • MQTT_EVENT_DISCONNECTED: The client has aborted the connection due to being unable to read or write data, e.g. because the server is unavailable.

  • MQTT_EVENT_SUBSCRIBED: The broker has acknowledged the client’s subscribe request. The event data will contain the message ID of the subscribe message.

  • MQTT_EVENT_UNSUBSCRIBED: The broker has acknowledged the client’s unsubscribe request. The event data will contain the message ID of the unsubscribe message.

  • MQTT_EVENT_PUBLISHED: The broker has acknowledged the client’s publish message. This will only be posted for Quality of Service level 1 and 2, as level 0 does not use acknowledgements. The event data will contain the message ID of the publish message.

  • MQTT_EVENT_DATA: The client has received a publish message. The event data contains: message ID, name of the topic it was published to, received data and its length. For data that exceeds the internal buffer multiple MQTT_EVENT_DATA will be posted and current_data_offset and total_data_len from event data updated to keep track of the fragmented message.

  • MQTT_EVENT_ERROR: The client has encountered an error. esp_mqtt_error_type_t from error_handle in the event data can be used to further determine the type of the error. The type of error will determine which parts of the error_handle struct is filled.

API Reference

Functions

esp_mqtt_client_handle_t esp_mqtt_client_init(const esp_mqtt_client_config_t *config)

Creates mqtt client handle based on the configuration.

Parameters

config – mqtt configuration structure

Returns

mqtt_client_handle if successfully created, NULL on error

esp_err_t esp_mqtt_client_set_uri(esp_mqtt_client_handle_t client, const char *uri)

Sets mqtt connection URI. This API is usually used to overrides the URI configured in esp_mqtt_client_init.

Parameters

  • client – mqtt client handle

  • uri –

Returns

ESP_FAIL if URI parse error, ESP_OK on success

esp_err_t esp_mqtt_client_start(esp_mqtt_client_handle_t client)

Starts mqtt client with already created client handle.

Parameters

client – mqtt client handle

Returns

ESP_OK on success ESP_ERR_INVALID_ARG on wrong initialization ESP_FAIL on other error

esp_err_t esp_mqtt_client_reconnect(esp_mqtt_client_handle_t client)

This api is typically used to force reconnection upon a specific event.

Parameters

client – mqtt client handle

Returns

ESP_OK on success ESP_ERR_INVALID_ARG on wrong initialization ESP_FAIL if client is in invalid state

esp_err_t esp_mqtt_client_disconnect(esp_mqtt_client_handle_t client)

This api is typically used to force disconnection from the broker.

Parameters

client – mqtt client handle

Returns

ESP_OK on success ESP_ERR_INVALID_ARG on wrong initialization

esp_err_t esp_mqtt_client_stop(esp_mqtt_client_handle_t client)

Stops mqtt client tasks.

  • Notes:

  • Cannot be called from the mqtt event handler

Parameters

client – mqtt client handle

Returns

ESP_OK on success ESP_ERR_INVALID_ARG on wrong initialization ESP_FAIL if client is in invalid state

int esp_mqtt_client_subscribe(esp_mqtt_client_handle_t client, const char *topic, int qos)

Subscribe the client to defined topic with defined qos.

Notes:

  • Client must be connected to send subscribe message

  • This API is could be executed from a user task or from a mqtt event callback i.e. internal mqtt task (API is protected by internal mutex, so it might block if a longer data receive operation is in progress.

Parameters

  • client – mqtt client handle

  • topic –

  • qos –

Returns

message_id of the subscribe message on success -1 on failure

int esp_mqtt_client_unsubscribe(esp_mqtt_client_handle_t client, const char *topic)

Unsubscribe the client from defined topic.

Notes:

  • Client must be connected to send unsubscribe message

  • It is thread safe, please refer to esp_mqtt_client_subscribe for details

Parameters

  • client – mqtt client handle

  • topic –

Returns

message_id of the subscribe message on success -1 on failure

int esp_mqtt_client_publish(esp_mqtt_client_handle_t client, const char *topic, const char *data, int len, int qos, int retain)

Client to send a publish message to the broker.

Notes:

  • This API might block for several seconds, either due to network timeout (10s) or if publishing payloads longer than internal buffer (due to message fragmentation)

  • Client doesn’t have to be connected for this API to work, enqueueing the messages with qos>1 (returning -1 for all the qos=0 messages if disconnected). If MQTT_SKIP_PUBLISH_IF_DISCONNECTED is enabled, this API will not attempt to publish when the client is not connected and will always return -1.

  • It is thread safe, please refer to esp_mqtt_client_subscribe for details

Parameters

  • client – mqtt client handle

  • topic – topic string

  • data – payload string (set to NULL, sending empty payload message)

  • len – data length, if set to 0, length is calculated from payload string

  • qos – qos of publish message

  • retain – retain flag

Returns

message_id of the publish message (for QoS 0 message_id will always be zero) on success. -1 on failure.

int esp_mqtt_client_enqueue(esp_mqtt_client_handle_t client, const char *topic, const char *data, int len, int qos, int retain, bool store)

Enqueue a message to the outbox, to be sent later. Typically used for messages with qos>0, but could be also used for qos=0 messages if store=true.

This API generates and stores the publish message into the internal outbox and the actual sending to the network is performed in the mqtt-task context (in contrast to the esp_mqtt_client_publish() which sends the publish message immediately in the user task’s context). Thus, it could be used as a non blocking version of esp_mqtt_client_publish().

Parameters

  • client – mqtt client handle

  • topic – topic string

  • data – payload string (set to NULL, sending empty payload message)

  • len – data length, if set to 0, length is calculated from payload string

  • qos – qos of publish message

  • retain – retain flag

  • store – if true, all messages are enqueued; otherwise only qos1 and qos 2 are enqueued

Returns

message_id if queued successfully, -1 otherwise

esp_err_t esp_mqtt_client_destroy(esp_mqtt_client_handle_t client)

Destroys the client handle.

Notes:

  • Cannot be called from the mqtt event handler

Parameters

client – mqtt client handle

Returns

ESP_OK ESP_ERR_INVALID_ARG on wrong initialization

esp_err_t esp_mqtt_set_config(esp_mqtt_client_handle_t client, const esp_mqtt_client_config_t *config)

Set configuration structure, typically used when updating the config (i.e. on “before_connect” event.

Parameters

  • client – mqtt client handle

  • config – mqtt configuration structure

Returns

ESP_ERR_NO_MEM if failed to allocate ESP_ERR_INVALID_ARG if conflicts on transport configuration. ESP_OK on success

esp_err_t esp_mqtt_client_register_event(esp_mqtt_client_handle_t client, esp_mqtt_event_id_t event, esp_event_handler_t event_handler, void *event_handler_arg)

Registers mqtt event.

Parameters

  • client – mqtt client handle

  • event – event type

  • event_handler – handler callback

  • event_handler_arg – handlers context

Returns

ESP_ERR_NO_MEM if failed to allocate ESP_ERR_INVALID_ARG on wrong initialization ESP_OK on success

int esp_mqtt_client_get_outbox_size(esp_mqtt_client_handle_t client)

Get outbox size.

Parameters

client – mqtt client handle

Returns

outbox size 0 on wrong initialization

Structures

struct esp_mqtt_error_codes

MQTT error code structure to be passed as a contextual information into ERROR event.

Important: This structure extends esp_tls_last_error error structure and is backward compatible with it (so might be down-casted and treated as esp_tls_last_error error, but recommended to update applications if used this way previously)

Use this structure directly checking error_type first and then appropriate error code depending on the source of the error:

| error_type | related member variables | note | | MQTT_ERROR_TYPE_TCP_TRANSPORT | esp_tls_last_esp_err, esp_tls_stack_err, esp_tls_cert_verify_flags, sock_errno | Error reported from tcp_transport/esp-tls | | MQTT_ERROR_TYPE_CONNECTION_REFUSED | connect_return_code | Internal error reported from MQTT broker on connection |

Public Members

esp_err_t esp_tls_last_esp_err

last esp_err code reported from esp-tls component

int esp_tls_stack_err

tls specific error code reported from underlying tls stack

int esp_tls_cert_verify_flags

tls flags reported from underlying tls stack during certificate verification

esp_mqtt_error_type_t error_type

error type referring to the source of the error

esp_mqtt_connect_return_code_t connect_return_code

connection refused error code reported from MQTT broker on connection

int esp_transport_sock_errno

errno from the underlying socket

struct esp_mqtt_event_t

MQTT event configuration structure

Public Members

esp_mqtt_event_id_t event_id

MQTT event type

esp_mqtt_client_handle_t client

MQTT client handle for this event

void *user_context

User context passed from MQTT client config

char *data

Data associated with this event

int data_len

Length of the data for this event

int total_data_len

Total length of the data (longer data are supplied with multiple events)

int current_data_offset

Actual offset for the data associated with this event

char *topic

Topic associated with this event

int topic_len

Length of the topic for this event associated with this event

int msg_id

MQTT messaged id of message

int session_present

MQTT session_present flag for connection event

esp_mqtt_error_codes_t *error_handle

esp-mqtt error handle including esp-tls errors as well as internal mqtt errors

bool retain

Retained flag of the message associated with this event

int qos

qos of the messages associated with this event

bool dup

dup flag of the message associated with this event

struct esp_mqtt_client_config_t

MQTT client configuration structure

Public Members

mqtt_event_callback_t event_handle

handle for MQTT events as a callback in legacy mode

esp_event_loop_handle_t event_loop_handle

handle for MQTT event loop library

const char *host

MQTT server domain (ipv4 as string)

const char *uri

Complete MQTT broker URI

uint32_t port

MQTT server port

bool set_null_client_id

Selects a NULL client id

const char *client_id

Set client id. Ignored if set_null_client_id == true If NULL set the default client id. Default client id is ESP32_CHIPID% where CHIPID% are last 3 bytes of MAC address in hex format

const char *username

MQTT username

const char *password

MQTT password

const char *lwt_topic

LWT (Last Will and Testament) message topic (NULL by default)

const char *lwt_msg

LWT message (NULL by default)

int lwt_qos

LWT message qos

int lwt_retain

LWT retained message flag

int lwt_msg_len

LWT message length

int disable_clean_session

mqtt clean session, default clean_session is true

int keepalive

mqtt keepalive, default is 120 seconds

bool disable_auto_reconnect

this mqtt client will reconnect to server (when errors/disconnect). Set disable_auto_reconnect=true to disable

void *user_context

pass user context to this option, then can receive that context in event->user_context

int task_prio

MQTT task priority, default is 5, can be changed in make menuconfig

int task_stack

MQTT task stack size, default is 6144 bytes, can be changed in make menuconfig

int buffer_size

size of MQTT send/receive buffer, default is 1024 (only receive buffer size if out_buffer_size defined)

const char *cert_pem

Pointer to certificate data in PEM or DER format for server verify (with SSL), default is NULL, not required to verify the server. PEM-format must have a terminating NULL-character. DER-format requires the length to be passed in cert_len.

size_t cert_len

Length of the buffer pointed to by cert_pem. May be 0 for null-terminated pem

const char *client_cert_pem

Pointer to certificate data in PEM or DER format for SSL mutual authentication, default is NULL, not required if mutual authentication is not needed. If it is not NULL, also client_key_pem has to be provided. PEM-format must have a terminating NULL-character. DER-format requires the length to be passed in client_cert_len.

size_t client_cert_len

Length of the buffer pointed to by client_cert_pem. May be 0 for null-terminated pem

const char *client_key_pem

Pointer to private key data in PEM or DER format for SSL mutual authentication, default is NULL, not required if mutual authentication is not needed. If it is not NULL, also client_cert_pem has to be provided. PEM-format must have a terminating NULL-character. DER-format requires the length to be passed in client_key_len

size_t client_key_len

Length of the buffer pointed to by client_key_pem. May be 0 for null-terminated pem

esp_mqtt_transport_t transport

overrides URI transport

int refresh_connection_after_ms

Refresh connection after this value (in milliseconds)

const struct psk_key_hint *psk_hint_key

Pointer to PSK struct defined in esp_tls.h to enable PSK authentication (as alternative to certificate verification). If not NULL and server/client certificates are NULL, PSK is enabled

bool use_global_ca_store

Use a global ca_store for all the connections in which this bool is set.

esp_err_t (*crt_bundle_attach)(void *conf)

Pointer to ESP x509 Certificate Bundle attach function for the usage of certification bundles in mqtts

int reconnect_timeout_ms

Reconnect to the broker after this value in miliseconds if auto reconnect is not disabled (defaults to 10s)

const char **alpn_protos

NULL-terminated list of supported application protocols to be used for ALPN

const char *clientkey_password

Client key decryption password string

int clientkey_password_len

String length of the password pointed to by clientkey_password

esp_mqtt_protocol_ver_t protocol_ver

MQTT protocol version used for connection, defaults to value from menuconfig

int out_buffer_size

size of MQTT output buffer. If not defined, both output and input buffers have the same size defined as buffer_size

bool skip_cert_common_name_check

Skip any validation of server certificate CN field, this reduces the security of TLS and makes the mqtt client susceptible to MITM attacks

bool use_secure_element

enable secure element for enabling SSL connection

void *ds_data

carrier of handle for digital signature parameters

int network_timeout_ms

Abort network operation if it is not completed after this value, in milliseconds (defaults to 10s)

bool disable_keepalive

Set disable_keepalive=true to turn off keep-alive mechanism, false by default (keepalive is active by default). Note: setting the config value keepalive to 0 doesn’t disable keepalive feature, but uses a default keepalive period

const char *path

Path in the URI

int message_retransmit_timeout

timeout for retansmit of failded packet

Macros

MQTT_ERROR_TYPE_ESP_TLS

MQTT_ERROR_TYPE_TCP_TRANSPORT error type hold all sorts of transport layer errors, including ESP-TLS error, but in the past only the errors from MQTT_ERROR_TYPE_ESP_TLS layer were reported, so the ESP-TLS error type is re-defined here for backward compatibility

Type Definitions

typedef struct esp_mqtt_client *esp_mqtt_client_handle_t
typedef struct esp_mqtt_error_codes esp_mqtt_error_codes_t

MQTT error code structure to be passed as a contextual information into ERROR event.

Important: This structure extends esp_tls_last_error error structure and is backward compatible with it (so might be down-casted and treated as esp_tls_last_error error, but recommended to update applications if used this way previously)

Use this structure directly checking error_type first and then appropriate error code depending on the source of the error:

| error_type | related member variables | note | | MQTT_ERROR_TYPE_TCP_TRANSPORT | esp_tls_last_esp_err, esp_tls_stack_err, esp_tls_cert_verify_flags, sock_errno | Error reported from tcp_transport/esp-tls | | MQTT_ERROR_TYPE_CONNECTION_REFUSED | connect_return_code | Internal error reported from MQTT broker on connection |

typedef esp_mqtt_event_t *esp_mqtt_event_handle_t
typedef esp_err_t (*mqtt_event_callback_t)(esp_mqtt_event_handle_t event)

Enumerations

enum esp_mqtt_event_id_t

MQTT event types.

User event handler receives context data in esp_mqtt_event_t structure with

  • user_context - user data from esp_mqtt_client_config_t

  • client - mqtt client handle

  • various other data depending on event type

Values:

enumerator MQTT_EVENT_ANY
enumerator MQTT_EVENT_ERROR

on error event, additional context: connection return code, error handle from esp_tls (if supported)

enumerator MQTT_EVENT_CONNECTED

connected event, additional context: session_present flag

enumerator MQTT_EVENT_DISCONNECTED

disconnected event

enumerator MQTT_EVENT_SUBSCRIBED

subscribed event, additional context:

  • msg_id message id

  • data pointer to the received data

  • data_len length of the data for this event

enumerator MQTT_EVENT_UNSUBSCRIBED

unsubscribed event

enumerator MQTT_EVENT_PUBLISHED

published event, additional context: msg_id

enumerator MQTT_EVENT_DATA

data event, additional context:

  • msg_id message id

  • topic pointer to the received topic

  • topic_len length of the topic

  • data pointer to the received data

  • data_len length of the data for this event

  • current_data_offset offset of the current data for this event

  • total_data_len total length of the data received

  • retain retain flag of the message

  • qos qos level of the message

  • dup dup flag of the message Note: Multiple MQTT_EVENT_DATA could be fired for one message, if it is longer than internal buffer. In that case only first event contains topic pointer and length, other contain data only with current data length and current data offset updating.

enumerator MQTT_EVENT_BEFORE_CONNECT

The event occurs before connecting

enumerator MQTT_EVENT_DELETED

Notification on delete of one message from the internal outbox, if the message couldn’t have been sent and acknowledged before expiring defined in OUTBOX_EXPIRED_TIMEOUT_MS. (events are not posted upon deletion of successfully acknowledged messages)

  • This event id is posted only if MQTT_REPORT_DELETED_MESSAGES==1

  • Additional context: msg_id (id of the deleted message).

enum esp_mqtt_connect_return_code_t

MQTT connection error codes propagated via ERROR event

Values:

enumerator MQTT_CONNECTION_ACCEPTED

Connection accepted

enumerator MQTT_CONNECTION_REFUSE_PROTOCOL

MQTT connection refused reason: Wrong protocol

enumerator MQTT_CONNECTION_REFUSE_ID_REJECTED

MQTT connection refused reason: ID rejected

enumerator MQTT_CONNECTION_REFUSE_SERVER_UNAVAILABLE

MQTT connection refused reason: Server unavailable

enumerator MQTT_CONNECTION_REFUSE_BAD_USERNAME

MQTT connection refused reason: Wrong user

enumerator MQTT_CONNECTION_REFUSE_NOT_AUTHORIZED

MQTT connection refused reason: Wrong username or password

enum esp_mqtt_error_type_t

MQTT connection error codes propagated via ERROR event

Values:

enumerator MQTT_ERROR_TYPE_NONE
enumerator MQTT_ERROR_TYPE_TCP_TRANSPORT
enumerator MQTT_ERROR_TYPE_CONNECTION_REFUSED
enum esp_mqtt_transport_t

Values:

enumerator MQTT_TRANSPORT_UNKNOWN
enumerator MQTT_TRANSPORT_OVER_TCP

MQTT over TCP, using scheme: mqtt

enumerator MQTT_TRANSPORT_OVER_SSL

MQTT over SSL, using scheme: mqtts

enumerator MQTT_TRANSPORT_OVER_WS

MQTT over Websocket, using scheme:: ws

enumerator MQTT_TRANSPORT_OVER_WSS

MQTT over Websocket Secure, using scheme: wss

enum esp_mqtt_protocol_ver_t

MQTT protocol version used for connection

Values:

enumerator MQTT_PROTOCOL_UNDEFINED
enumerator MQTT_PROTOCOL_V_3_1
enumerator MQTT_PROTOCOL_V_3_1_1

Provide feedback about this document