Camera Controller Driver

[中文]

Introduction

ESP32-P4 has the following hardware designed for communication with external camera sensors:

MIPI Camera Serial Interface (MIPI CSI)
Digital Video Port via ISP module (ISP DVP)
Digital Video Port via LCD_CAM module (LCD_CAM DVP)

The camera controller driver provides an interface for these hardware peripherals.

Functional Overview

Resource Allocation – Describes how to allocate and configure camera controller instances, and how to release resources when no longer needed.
Enable and Disable the Camera Controller Driver – Explains how to enable and disable a camera controller.
Start and Stop the Camera Controller Driver – Details how to start and stop a camera controller.
Receive Data from a Camera Sensor – Explains how to receive signals from a camera sensor.
Register Event Callbacks – Describes how to register user-defined event callback functions.
Thread Safety – Lists APIs that are thread-safe.
Kconfig Options – Lists supported Kconfig options that affect driver behavior.
IRAM Safety – Provides guidance on making CSI interrupt and control functions work reliably when the cache is disabled.

Resource Allocation

Install the Camera Controller Driver

You can implement the camera controller driver using one of the following methods:

esp_cam_new_csi_ctlr()
esp_cam_new_isp_dvp_ctlr()
esp_cam_new_lcd_cam_ctlr()

You can implement a camera controller driver using the CSI peripheral. This requires configuration via esp_cam_ctlr_csi_config_t.

After specifying esp_cam_ctlr_csi_config_t, call esp_cam_new_csi_ctlr() to allocate and initialize a CSI camera controller handle. If successful, this function returns a CSI camera controller handle. See the example below.

#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_types.h"
#include "esp_cam_ctlr_csi.h"

void app_main(void)
{
    esp_cam_ctlr_csi_config_t csi_config = {
        .ctlr_id = 0,
        .h_res = MIPI_CSI_DISP_HSIZE,
        .v_res = MIPI_CSI_DISP_VSIZE_640P,
        .lane_bit_rate_mbps = MIPI_CSI_LANE_BITRATE_MBPS,
        .input_data_color_type = CAM_CTLR_COLOR_RAW8,
        .output_data_color_type = CAM_CTLR_COLOR_RGB565,
        .data_lane_num = 2,
        .byte_swap_en = false,
        .queue_items = 1,
    };

    esp_cam_ctlr_handle_t handle = NULL;
    ESP_ERROR_CHECK(esp_cam_new_csi_ctlr(&csi_config, &handle));
}

The CSI peripheral in ESP32-P4 requires a stable 2.5 V power supply. Refer to the schematic diagram to ensure the power supply pins are connected to 2.5 V before using the MIPI CSI driver.

In ESP32-P4, you can power the CSI using the internal adjustable LDO. Connect the LDO channel output pin to the CSI power supply pin. Before initializing the CSI driver, use the API in Low Dropout Voltage Regulator (LDO) to configure the LDO to output 2.5 V.

You can implement a camera controller driver using the ISP DVP peripheral. This requires configuration via esp_cam_ctlr_isp_dvp_cfg_t.

After specifying esp_cam_ctlr_isp_dvp_cfg_t, call esp_cam_new_isp_dvp_ctlr() to allocate and initialize an ISP DVP camera controller handle. If successful, this function returns an ISP DVP camera controller handle. See the example below.

Before calling esp_cam_new_isp_dvp_ctlr(), create an ISP handle using esp_isp_new_processor().

#include "esp_err.h"
#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_isp_dvp.h"
#include "driver/isp.h"

#define MIPI_CSI_DISP_HSIZE   800   // example value, replace with actual resolution
#define MIPI_CSI_DISP_VSIZE   600   // example value, replace with actual resolution

void app_main(void)
{
    isp_proc_handle_t isp_proc = NULL;
    esp_isp_processor_cfg_t isp_config = {
        .clk_hz = 120 * 1000 * 1000,
        .input_data_source = ISP_INPUT_DATA_SOURCE_DVP,
        .input_data_color_type = ISP_COLOR_RAW8,
        .output_data_color_type = ISP_COLOR_RGB565,
        .has_line_start_packet = false,
        .has_line_end_packet = false,
        .h_res = MIPI_CSI_DISP_HSIZE,
        .v_res = MIPI_CSI_DISP_VSIZE,
    };
    ESP_ERROR_CHECK(esp_isp_new_processor(&isp_config, &isp_proc));

    esp_cam_ctlr_handle_t cam_handle = NULL;
    esp_cam_ctlr_isp_dvp_cfg_t dvp_ctlr_config = {
        .data_width = 8,
        .data_io = {53, 54, 52, 0, 1, 45, 46, 47, -1, -1, -1, -1, -1, -1, -1, -1},
        .pclk_io = 21,
        .hsync_io = 5,
        .vsync_io = 23,
        .de_io = 22,
        .io_flags.vsync_invert = 1,
        .queue_items = 10,
    };
    ESP_ERROR_CHECK(esp_cam_new_isp_dvp_ctlr(isp_proc, &dvp_ctlr_config, &cam_handle));
}

You can implement a camera controller driver using the DVP port of LCD_CAM. This requires configuration via esp_cam_ctlr_dvp_config_t.

esp_cam_ctlr_dvp_config_t::exexternal_xtal: Set this to use an externally generated xclk. Otherwise, the camera driver generates it internally.

After specifying esp_cam_ctlr_lcd_cam_cfg_t, call esp_cam_new_lcd_cam_ctlr() to allocate and initialize a DVP camera controller handle. If successful, this function returns a DVP camera controller handle. See the example below.

After calling esp_cam_new_dvp_ctlr(), allocate a camera buffer that meets alignment constraints, or call esp_cam_ctlr_alloc_buffer() to allocate automatically.

To configure format conversion, call esp_cam_ctlr_format_conversion(). The driver supports the following conversion types:

YUV to RGB
RGB to YUV
YUV to YUV

Supported color ranges: * Full range: 0-255 for both RGB and YUV * Limited range: RGB 16-240, YUV Y:16-240, U-V:16-235

#include "esp_err.h"
#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_types.h"
#include "esp_cam_ctlr_isp_dvp.h"

void app_main(void)
{
    esp_cam_ctlr_handle_t cam_handle = NULL;

    esp_cam_ctlr_dvp_pin_config_t pin_cfg = {
        .data_width = EXAMPLE_DVP_CAM_DATA_WIDTH,
        .data_io = {
            EXAMPLE_DVP_CAM_D0_IO,
            EXAMPLE_DVP_CAM_D1_IO,
            EXAMPLE_DVP_CAM_D2_IO,
            EXAMPLE_DVP_CAM_D3_IO,
            EXAMPLE_DVP_CAM_D4_IO,
            EXAMPLE_DVP_CAM_D5_IO,
            EXAMPLE_DVP_CAM_D6_IO,
            EXAMPLE_DVP_CAM_D7_IO,
        },
        .vsync_io = EXAMPLE_DVP_CAM_VSYNC_IO,
        .de_io = EXAMPLE_DVP_CAM_DE_IO,
        .pclk_io = EXAMPLE_DVP_CAM_PCLK_IO,
        .xclk_io = EXAMPLE_DVP_CAM_XCLK_IO, // Set XCLK pin to generate XCLK signal
    };

    esp_cam_ctlr_dvp_config_t dvp_config = {
        .ctlr_id = 0,
        .clk_src = CAM_CLK_SRC_DEFAULT,
        .h_res = CONFIG_EXAMPLE_CAM_HRES,
        .v_res = CONFIG_EXAMPLE_CAM_VRES,
        .input_data_color_type = CAM_CTLR_COLOR_RGB565,
        .dma_burst_size = 128,
        .pin = &pin_cfg,
        .bk_buffer_dis = 1,
        .xclk_freq = EXAMPLE_DVP_CAM_XCLK_FREQ_HZ,
    };

    ESP_ERROR_CHECK(esp_cam_new_dvp_ctlr(&dvp_config, &cam_handle));
}

Uninstall the Camera Controller Driver

To release resources for a camera controller driver that is no longer needed, call esp_cam_ctlr_del(). This releases the underlying hardware.

Enable and Disable the Camera Controller Driver

Before operating the camera controller, enable the driver by calling esp_cam_ctlr_enable(). This switches the driver state from init to enable.

#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_types.h"
#include "esp_err.h"

void app_main(void)
{
    esp_cam_ctlr_handle_t handle;
    ESP_ERROR_CHECK(esp_cam_ctlr_enable(handle));
}

To disable the driver and return to the init state, call esp_cam_ctlr_disable().

#include "esp_err.h"
#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_types.h"

void app_main(void)
{
    esp_cam_ctlr_handle_t handle;
    ESP_ERROR_CHECK(esp_cam_ctlr_disable(handle));
}

Start and Stop the Camera Controller Driver

Before receiving signals from a camera sensor, start the driver by calling esp_cam_ctlr_start(). This switches the driver state from enable to start.

#include "esp_err.h"
#include "esp_log.h"
#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_types.h"

void app_main(void)
{
    esp_cam_ctlr_handle_t handle = NULL;
    ESP_ERROR_CHECK(esp_cam_ctlr_start(handle));
    ESP_LOGI("CAM", "Camera controller started successfully");
}

To stop the driver and return to the enable state, call esp_cam_ctlr_stop().

#include "esp_err.h"
#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_types.h"

void app_main(void)
{
    esp_cam_ctlr_handle_t handle = NULL;
    ESP_ERROR_CHECK(esp_cam_ctlr_stop(handle));
}

Receive Data from a Camera Sensor

To receive data from a camera sensor, call esp_cam_ctlr_receive().

#include "esp_err.h"
#include "esp_cam_ctlr.h"
#include "esp_cam_ctlr_types.h"

ESP_ERROR_CHECK(esp_cam_ctlr_receive(handle, &my_trans, ESP_CAM_CTLR_MAX_DELAY));

Register Event Callbacks

After the camera controller driver starts receiving data, it can generate events dynamically. To execute user-defined functions when events occur, register your callback function using esp_cam_ctlr_register_event_callbacks(). Supported event callbacks are listed in esp_cam_ctlr_evt_cbs_t:

esp_cam_ctlr_evt_cbs_t::on_get_new_trans – Called after the driver finishes a transaction and attempts to get a new transaction descriptor. Also called in s_ctlr_csi_start(). If this callback does not provide a new transaction descriptor, the driver uses the internal backup buffer if the bk_buffer_dis flag is set.
esp_cam_ctlr_evt_cbs_t::on_trans_finished – Called when the driver finishes a transaction. This function runs in the ISR context. Ensure that it does not block (for example, only use FreeRTOS APIs with the ISR suffix).

Thread Safety

The following factory functions are thread-safe and can be called from different RTOS tasks without additional locking:

esp_cam_new_csi_ctlr()
esp_cam_new_isp_dvp_ctlr()
esp_cam_ctlr_del()

Kconfig Options

The following Kconfig options affect interrupt handler behavior when the cache is disabled:

CONFIG_CAM_CTLR_MIPI_CSI_ISR_CACHE_SAFE, see Thread Safety for details.
CONFIG_CAM_CTLR_ISP_DVP_ISR_CACHE_SAFE, see Thread Safety for details.

IRAM Safety

By default, CSI interrupts are delayed when the cache is disabled during flash write or erase operations. They are handled after the cache is enabled again.

The following Kconfig options:

Enable the interrupt being serviced even when the cache is disabled.
Place all functions used by the ISR into IRAM.
Place driver object into DRAM (to avoid mapping to PSRAM).

This allows interrupts to run while the cache is disabled, but increases IRAM usage. Ensure that user callbacks and related code/data are IRAM-safe or DRAM-safe when the cache is disabled.

Application Examples

peripherals/camera/mipi_isp_dsi – Demonstrates how to use the esp_driver_cam component to capture signals from a MIPI CSI camera sensor via the ISP module and display them on an LCD screen via a DSI interface.
peripherals/camera/dvp_isp_dsi – Demonstrates how to use the esp_driver_cam component to capture signals from a DVP camera sensor via the ISP module and display them on an LCD screen via a DSI interface.

API Reference

Header File

components/esp_driver_cam/include/esp_cam_ctlr.h
This header file can be included with:
#include "esp_cam_ctlr.h"
This header file is a part of the API provided by the esp_driver_cam component. To declare that your component depends on esp_driver_cam, add the following to your CMakeLists.txt:
REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam

Functions

esp_err_t esp_cam_ctlr_enable(esp_cam_ctlr_handle_t handle)

Enable the ESP CAM controller.

Parameters:

handle -- [in] ESP CAM controller handle.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid state.

esp_err_t esp_cam_ctlr_start(esp_cam_ctlr_handle_t handle)

Start the ESP CAM controller.

Parameters:

handle -- [in] ESP CAM controller handle.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid state.

esp_err_t esp_cam_ctlr_stop(esp_cam_ctlr_handle_t handle)

Stop the ESP CAM controller.

Parameters:

handle -- [in] ESP CAM controller handle.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid state.

esp_err_t esp_cam_ctlr_disable(esp_cam_ctlr_handle_t handle)

Disable the ESP CAM controller.

Parameters:

handle -- [in] ESP CAM controller handle.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid state.

esp_err_t esp_cam_ctlr_receive(esp_cam_ctlr_handle_t handle, esp_cam_ctlr_trans_t *trans, uint32_t timeout_ms)

Receive data for the given transaction.

Parameters:

handle -- [in] ESP CAM controller handle.
trans -- [in] ESP CAM controller transaction type.
timeout_ms -- [in] Timeout. Measurement unit: milliseconds.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid state.

esp_err_t esp_cam_ctlr_del(esp_cam_ctlr_handle_t handle)

Delete the ESP CAM controller handle.

Parameters:

handle -- [in] ESP CAM controller handle.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid state.

esp_err_t esp_cam_ctlr_register_event_callbacks(esp_cam_ctlr_handle_t handle, const esp_cam_ctlr_evt_cbs_t *cbs, void *user_data)

Register ESP CAM controller event callbacks.

Parameters:

handle -- [in] ESP CAM controller handle.
cbs -- [in] ESP CAM controller event callbacks.
user_data -- [in] User data.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid state.

esp_err_t esp_cam_ctlr_get_frame_buffer(esp_cam_ctlr_handle_t handle, uint32_t fb_num, const void **fb0, ...)

Get the ESP CAM controller internal allocated backup buffer(s) address.

Note

Data in the internal buffer is generally ready when the on_trans_finished event occurs.

Parameters:

handle -- [in] ESP CAM controller handle.
fb_num -- [in] Number of frame buffers to get. This value must match the number of subsequent fbN parameters.
fb0 -- [out] Address of the first frame buffer.
... -- [out] List of other frame buffers, if any.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_INVALID_STATE: Invalid driver state.

esp_err_t esp_cam_ctlr_get_frame_buffer_len(esp_cam_ctlr_handle_t handle, size_t *ret_fb_len)

Get the length of the ESP CAM controller internal backup buffer.

Parameters:

handle -- [in] ESP CAM controller handle.
ret_fb_len -- [out] Optional. The size of each frame buffer. Measurement unit: bytes.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: NULL pointer.
ESP_ERR_INVALID_STATE: Invalid driver state.

void *esp_cam_ctlr_alloc_buffer(esp_cam_ctlr_handle_t handle, size_t size, uint32_t buf_caps)

Allocate a camera buffer for the ESP CAM controller.

Note

This function must be called after esp_cam_new_*_ctlr.

Parameters:

handle -- [in] ESP CAM controller handle.
size -- [in] Buffer size. Measurement unit: bytes.
buf_caps -- [in] Buffer allocation capabilities:
- MALLOC_CAP_SPIRAM || MALLOC_CAP_DMA: Memory in external SPI RAM.
- MALLOC_CAP_INTERNAL || MALLOC_CAP_DMA: Memory in internal SRAM.

Returns:

Buffer pointer on success.
NULL on failure.

esp_err_t esp_cam_ctlr_format_conversion(esp_cam_ctlr_handle_t handle, const cam_ctlr_format_conv_config_t *conv_cfg)

Configure format conversion.

Parameters:

handle -- [in] ESP CAM controller handle.
conv_cfg -- [in] Color conversion configuration, containing source and destination formats.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_NOT_SUPPORTED: Format conversion not supported by this controller.

Header File

components/esp_driver_cam/include/esp_cam_ctlr_types.h
This header file can be included with:
#include "esp_cam_ctlr_types.h"
This header file is a part of the API provided by the esp_driver_cam component. To declare that your component depends on esp_driver_cam, add the following to your CMakeLists.txt:
REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam

Structures

struct esp_cam_ctlr_trans_t

Transaction type for the ESP CAM controller.

Public Members

void *buffer: Transaction buffer.

size_t buflen: Length of the transaction buffer.

size_t received_size: Received size. This value is written by the driver and indicates the actual received size.

struct esp_cam_ctlr_evt_cbs_t

Event callbacks for the ESP CAM controller.

Public Members

bool (*on_get_new_trans)(esp_cam_ctlr_handle_t handle, esp_cam_ctlr_trans_t *trans, void *user_data)

Callback for receiving a new transaction.

Param handle:: [in] ESP CAM controller handle.
Param trans:: [in] New transaction.
Param user_data:: [in] User-registered data.
Return:: Whether a high-priority task is woken up by this function.

bool (*on_trans_finished)(esp_cam_ctlr_handle_t handle, esp_cam_ctlr_trans_t *trans, void *user_data)

Callback for a finished transaction.

Param handle:: [in] ESP CAM controller handle.
Param trans:: [out] Finished transaction.
Param user_data:: [in] User-registered data.
Return:: Whether a high-priority task is woken up by this function.

Macros

ESP_CAM_CTLR_MAX_DELAY: Maximum timeout value in ticks for the ESP CAM controller.

Type Definitions

typedef struct esp_cam_ctlr_t *esp_cam_ctlr_handle_t: Handle for the ESP CAM controller.

Header File

components/esp_driver_cam/csi/include/esp_cam_ctlr_csi.h
This header file can be included with:
#include "esp_cam_ctlr_csi.h"
This header file is a part of the API provided by the esp_driver_cam component. To declare that your component depends on esp_driver_cam, add the following to your CMakeLists.txt:
REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam

Functions

esp_err_t esp_cam_new_csi_ctlr(const esp_cam_ctlr_csi_config_t *config, esp_cam_ctlr_handle_t *ret_handle)

Create a new ESP CAM CSI controller.

Parameters:

config -- [in] CSI controller configurations.
ret_handle -- [out] Returned ESP CAM controller handle.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_NO_MEM: Out of memory.
ESP_ERR_NOT_SUPPORTED: Currently unsupported modes or types.
ESP_ERR_NOT_FOUND: CSI is already registered.

Structures

struct esp_cam_ctlr_csi_config_t

ESP CAM CSI controller configurations.

Public Members

int ctlr_id: CSI controller ID.

mipi_csi_phy_clock_source_t clk_src: CSI PHY clock source.

uint32_t h_res: Input horizontal resolution. Measurement unit: pixels per line.

uint32_t v_res: Input vertical resolution. Measurement unit: lines per frame.

uint8_t data_lane_num: Number of data lanes.

int lane_bit_rate_mbps: Lane bit rate. Measurement unit: Mbps.

cam_ctlr_color_t input_data_color_type: Input color type.

cam_ctlr_color_t output_data_color_type: Output color type.

int queue_items: Number of queue items.

uint32_t byte_swap_en: Set to 1 to enable byte swap.

uint32_t bk_buffer_dis: Set to 1 to disable backup buffer.

struct esp_cam_ctlr_csi_config_t: Boolean flags.

Header File

components/esp_driver_cam/isp_dvp/include/esp_cam_ctlr_isp_dvp.h
This header file can be included with:
#include "esp_cam_ctlr_isp_dvp.h"
This header file is a part of the API provided by the esp_driver_cam component. To declare that your component depends on esp_driver_cam, add the following to your CMakeLists.txt:
REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam

Functions

esp_err_t esp_cam_new_isp_dvp_ctlr(isp_proc_handle_t isp_proc, const esp_cam_ctlr_isp_dvp_cfg_t *ctlr_config, esp_cam_ctlr_handle_t *ret_handle)

Create a new ESP CAM ISP DVP controller.

Parameters:

isp_proc -- [in] Processor handle.
ctlr_config -- [in] ISP DVP controller configurations.
ret_handle -- [out] Returned ESP CAM controller handle.

Returns:

ESP_OK: Success.
ESP_ERR_INVALID_ARG: Invalid argument.
ESP_ERR_NO_MEM: Out of memory.
ESP_ERR_NOT_SUPPORTED: Currently not supported modes or types.
ESP_ERR_NOT_FOUND: ISP DVP is already registered.

Structures

struct esp_cam_ctlr_isp_dvp_cfg_t

ESP CAM ISP DVP controller configurations.

Public Members

cam_ctlr_data_width_t data_width: Number of data lines.

int data_io[ISP_DVP_DATA_SIG_NUM]: ISP DVP data-in IO numbers.

int pclk_io: ISP DVP pclk IO number.

int hsync_io: ISP DVP hsync IO number.

int vsync_io: ISP DVP vsync IO number.

int de_io: ISP DVP de IO number.

uint32_t pclk_invert: Set to 1 to invert the pclk signal.

uint32_t hsync_invert: Set to 1 to invert the hsync signal (i.e., active low).

uint32_t vsync_invert: Set to 1 to invert the vsync signal (i.e., active high).

uint32_t de_invert: Set to 1 to invert the de signal (i.e., active low).

struct esp_cam_ctlr_isp_dvp_cfg_t io_flags: ISP DVP IO flags.

int queue_items: Queue items.

uint32_t byte_swap_en: Set to 1 to enable byte swap.

uint32_t bk_buffer_dis: Set to 1 to disable backup buffer.

Was this page helpful?

Thank you! We received your feedback.
If you have any comments, fill in Espressif Documentation Feedback Form.

We value your feedback.
Let us know how we can improve this page by filling in Espressif Documentation Feedback Form.