Camera Controller Driver
Introduction
ESP32-P4 has the following hardware that is intended for communication with external camera sensor:
MIPI Camera Serial Interface (CSI)
ISP Digital Video Port (ISP DVP)
The camera controller driver is designed for this hardware peripheral.
Functional Overview
Resource Allocation - covers how to allocate camera controller instances with properly set of configurations. It also covers how to recycle the resources when they are no longer needed.
Enable and Disable Camera Controller Driver - covers how to enable and disable a camera controller.
Start and Stop Camera Controller Driver - covers how to start and stop a camera controller.
Receive from a Camera Sensor- covers how to receive camera signal from a sensor or something else.
Register Event Callbacks- covers how to hook user specific code to camera controller driver event callback function.
Thread Safety - lists which APIs are guaranteed to be thread safe by the driver.
Kconfig Options - lists the supported Kconfig options that can bring different effects to the driver.
IRAM Safe - describes tips on how to make the CSI interrupt and control functions work better along with a disabled cache.
Resource Allocation
Install Camera Controller Driver
A camera controller driver can be implemented by the CSI peripheral, which requires the configuration that specified by esp_cam_ctlr_csi_config_t
.
If the configurations in esp_cam_ctlr_csi_config_t
is specified, users can call esp_cam_new_csi_ctlr()
to allocate and initialize a CSI camera controller handle. This function will return an CSI camera controller handle if it runs correctly. You can take following code as reference.
esp_cam_ctlr_csi_config_t csi_config = {
.ctlr_id = 0,
.h_res = MIPI_CSI_DISP_HSIZE,
.v_res = MIPI_CSI_DISP_VSIZE_640P,
.lane_bit_rate_mbps = MIPI_CSI_LANE_BITRATE_MBPS,
.input_data_color_type = CAM_CTLR_COLOR_RAW8,
.output_data_color_type = CAM_CTLR_COLOR_RGB565,
.data_lane_num = 2,
.byte_swap_en = false,
.queue_items = 1,
};
esp_cam_ctlr_handle_t handle = NULL;
ESP_ERROR_CHECK(esp_cam_new_csi_ctlr(&csi_config, &handle));
A camera controller driver can be implemented by the ISP DVP peripheral, which requires the configuration that specified by esp_cam_ctlr_isp_dvp_cfg_t
.
If the configurations in esp_cam_ctlr_isp_dvp_cfg_t
is specified, users can call esp_cam_new_isp_dvp_ctlr()
to allocate and initialize a ISP DVP camera controller handle. This function will return an ISP DVP camera controller handle if it runs correctly. You can take following code as reference.
Before calling esp_cam_new_isp_dvp_ctlr()
, you should also call esp_isp_new_processor()
to create an ISP handle.
isp_proc_handle_t isp_proc = NULL;
esp_isp_processor_cfg_t isp_config = {
.clk_hz = 120 * 1000 * 1000,
.input_data_source = ISP_INPUT_DATA_SOURCE_DVP,
.input_data_color_type = ISP_COLOR_RAW8,
.output_data_color_type = ISP_COLOR_RGB565,
.has_line_start_packet = false,
.has_line_end_packet = false,
.h_res = MIPI_CSI_DISP_HSIZE,
.v_res = MIPI_CSI_DISP_VSIZE,
};
ESP_ERROR_CHECK(esp_isp_new_processor(&isp_config, &isp_proc));
esp_cam_ctlr_isp_dvp_cfg_t dvp_ctlr_config = {
.data_width = 8,
.data_io = {53, 54, 52, 0, 1, 45, 46, 47, -1, -1, -1, -1, -1, -1, -1, -1},
.pclk_io = 21,
.hsync_io = 5,
.vsync_io = 23,
.de_io = 22,
.io_flags.vsync_invert = 1,
.queue_items = 10,
};
ESP_ERROR_CHECK(esp_cam_new_isp_dvp_ctlr(isp_proc, &dvp_ctlr_config, &cam_handle));
Uninstall Camera Controller Driver
If a previously installed camera controller driver is no longer needed, it's recommended to recycle the resource by calling esp_cam_ctlr_del()
, so that to release the underlying hardware.
Enable and Disable Camera Controller Driver
Before starting camera controller operation, you need to enable the camera controller driver first, by calling esp_cam_ctlr_enable()
. This function:
Switches the driver state from init to enable.
ESP_ERROR_CHECK(esp_cam_ctlr_enable(handle));
Calling esp_cam_ctlr_disable()
does the opposite, that is, put the driver back to the init state.
ESP_ERROR_CHECK(esp_cam_ctlr_disable(handle));
Start and Stop Camera Controller Driver
Before receiving camera signal from camera sensor, you need to start the camera controller driver first, by calling esp_cam_ctlr_start()
. This function:
Switches the driver state from enable to start.
ESP_ERROR_CHECK(esp_cam_ctlr_start(handle));
Calling esp_cam_ctlr_stop()
does the opposite, that is, put the driver back to the enable state.
ESP_ERROR_CHECK(esp_cam_ctlr_stop(handle));
Receive from a Camera Sensor
Now you can call esp_cam_ctlr_receive()
to receive from a camera sensor or something else.
ESP_ERROR_CHECK(esp_cam_ctlr_receive(handle, &my_trans, ESP_CAM_CTLR_MAX_DELAY));
Register Event Callbacks
After the camera controller driver starts receiving, it can generate a specific event dynamically. If you have some functions that should be called when the event happens, please hook your function to the interrupt service routine by calling esp_cam_ctlr_register_event_callbacks()
. All supported event callbacks are listed in esp_cam_ctlr_evt_cbs_t
:
esp_cam_ctlr_evt_cbs_t::on_get_new_trans
sets a callback function which will be called after the camera controller driver finishes previous transaction, and tries to get a new transaction descriptor. It will also be called when ins_ctlr_csi_start()
. If this callback does not get a new transaction descriptor, the camera controller driver will use the internal backup buffer ifbk_buffer_dis
flag is set.esp_cam_ctlr_evt_cbs_t::on_trans_finished
sets a callback function when the camera controller driver finishes a transaction. As this function is called within the ISR context, you must ensure that the function does not attempt to block (e.g., by making sure that only FreeRTOS APIs withISR
suffix are called from within the function).
Thread Safety
The factory functions:
are guaranteed to be thread safe by the driver, which means, they can be called from different RTOS tasks without protection by extra locks.
Kconfig Options
The following Kconfig options affect the behavior of the interrupt handler when cache is disabled:
CONFIG_CAM_CTLR_MIPI_CSI_ISR_IRAM_SAFE, see Thread Safety for more details.
CONFIG_CAM_CTLR_ISP_DVP_ISR_IRAM_SAFE, see Thread Safety for more details.
IRAM Safe
By default, the CSI interrupt will be deferred when the cache is disabled because of writing or erasing the flash.
There are Kconfig options
that
Enables the interrupt being serviced even when the cache is disabled
Places all functions that used by the ISR into IRAM
Places driver object into DRAM (in case it is mapped to PSRAM by accident)
This allows the interrupt to run while the cache is disabled, but comes at the cost of increased IRAM consumption. So user callbacks need to notice that the code and data inside (the callback) should be IRAM-safe or DRAM-safe, when cache is disabled.
Application Examples
peripherals/camera/camera_dsi demonstrates how to use the
esp_driver_cam
component to capture signals from a camera sensor and display it on an ILI9881C LCD screen via a DSI interface.
API Reference
Header File
This header file can be included with:
#include "esp_cam_ctlr.h"
This header file is a part of the API provided by the
esp_driver_cam
component. To declare that your component depends onesp_driver_cam
, add the following to your CMakeLists.txt:REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam
Functions
-
esp_err_t esp_cam_ctlr_enable(esp_cam_ctlr_handle_t handle)
Enable ESP CAM controller.
- Parameters
handle -- [in] ESP CAM controller handle
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid state
-
esp_err_t esp_cam_ctlr_start(esp_cam_ctlr_handle_t handle)
Start ESP CAM controller.
- Parameters
handle -- [in] ESP CAM controller handle
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid state
-
esp_err_t esp_cam_ctlr_stop(esp_cam_ctlr_handle_t handle)
Stop ESP CAM controller.
- Parameters
handle -- [in] ESP CAM controller handle
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid state
-
esp_err_t esp_cam_ctlr_disable(esp_cam_ctlr_handle_t handle)
Disable ESP CAM controller.
- Parameters
handle -- [in] ESP CAM controller handle
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid state
-
esp_err_t esp_cam_ctlr_receive(esp_cam_ctlr_handle_t handle, esp_cam_ctlr_trans_t *trans, uint32_t timeout_ms)
Receive data to the given transaction.
- Parameters
handle -- [in] ESP CAM controller handle
trans -- [in] ESP CAM controller transaction type
timeout_ms -- [in] Timeout in ms
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid state
-
esp_err_t esp_cam_ctlr_del(esp_cam_ctlr_handle_t handle)
Delete ESP CAM controller handle.
- Parameters
handle -- [in] ESP CAM controller handle
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid state
-
esp_err_t esp_cam_ctlr_register_event_callbacks(esp_cam_ctlr_handle_t handle, const esp_cam_ctlr_evt_cbs_t *cbs, void *user_data)
Register ESP CAM controller event callbacks.
- Parameters
handle -- [in] ESP CAM controller handle
cbs -- [in] ESP CAM controller event callbacks
user_data -- [in] User data
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid state
-
esp_err_t esp_cam_ctlr_get_frame_buffer(esp_cam_ctlr_handle_t handle, uint32_t fb_num, const void **fb0, ...)
Get ESP CAM controller internal malloced backup buffer(s) addr.
Note
Generally, data in internal buffer is ready when
on_trans_finished
event- Parameters
handle -- [in] ESP CAM controller handle
fb_num -- [in] Number of frame buffer(s) to get. This value must be the same as the number of the followed fbN parameters
fb0 -- [out] Address of the frame buffer 0 (first frame buffer)
... -- [out] List of other frame buffers if any
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_INVALID_STATE: Invalid driver state
-
esp_err_t esp_cam_ctlr_get_frame_buffer_len(esp_cam_ctlr_handle_t handle, size_t *ret_fb_len)
Get ESP CAM controller internal backup buffer length.
- Parameters
handle -- [in] ESP CAM controller handle
ret_fb_len -- [out] Optional, The size of each frame buffer, in bytes.
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: NULL ptr
ESP_ERR_INVALID_STATE: Invalid driver state
Header File
This header file can be included with:
#include "esp_cam_ctlr_types.h"
This header file is a part of the API provided by the
esp_driver_cam
component. To declare that your component depends onesp_driver_cam
, add the following to your CMakeLists.txt:REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam
Structures
-
struct esp_cam_ctlr_trans_t
ESP CAM controller transaction type.
-
struct esp_cam_ctlr_evt_cbs_t
ESP CAM controller event callbacks.
Public Members
-
bool (*on_get_new_trans)(esp_cam_ctlr_handle_t handle, esp_cam_ctlr_trans_t *trans, void *user_data)
On get new transaction callback.
- Param handle
[in] ESP CAM controller handle
- Param trans
[in] New transaction
- Param user_data
[in] User registered data
- Return
Whether a high priority task is woken up by this function
-
bool (*on_trans_finished)(esp_cam_ctlr_handle_t handle, esp_cam_ctlr_trans_t *trans, void *user_data)
On transaction finish callback.
- Param handle
[in] ESP CAM controller handle
- Param trans
[out] Finished transaction
- Param user_data
[in] User registered data
- Return
Whether a high priority task is woken up by this function
-
bool (*on_get_new_trans)(esp_cam_ctlr_handle_t handle, esp_cam_ctlr_trans_t *trans, void *user_data)
Macros
-
ESP_CAM_CTLR_MAX_DELAY
ESP CAM controller max timeout value.
Type Definitions
-
typedef struct esp_cam_ctlr_t *esp_cam_ctlr_handle_t
ESP CAM controller handle.
Header File
This header file can be included with:
#include "esp_cam_ctlr_csi.h"
This header file is a part of the API provided by the
esp_driver_cam
component. To declare that your component depends onesp_driver_cam
, add the following to your CMakeLists.txt:REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam
Functions
-
esp_err_t esp_cam_new_csi_ctlr(const esp_cam_ctlr_csi_config_t *config, esp_cam_ctlr_handle_t *ret_handle)
New ESP CAM CSI controller.
- Parameters
config -- [in] CSI controller configurations
ret_handle -- [out] Returned ESP CAM controller handle
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_NO_MEM: Out of memory
ESP_ERR_NOT_SUPPORTED: Currently not support modes or types
ESP_ERR_NOT_FOUND: CSI is registered already
Structures
-
struct esp_cam_ctlr_csi_config_t
ESP CAM CSI controller configurations.
Public Members
-
int ctlr_id
CSI controller ID.
-
mipi_csi_phy_clock_source_t clk_src
CSI phy clock source.
-
uint32_t h_res
Input horizontal resolution, i.e. the number of pixels in a line.
-
uint32_t v_res
Input vertical resolution, i.e. the number of lines in a frame.
-
uint8_t data_lane_num
Data lane num.
-
int lane_bit_rate_mbps
Lane bit rate in Mbps.
-
cam_ctlr_color_t input_data_color_type
Input color type.
-
cam_ctlr_color_t output_data_color_type
Output color type.
-
int queue_items
Queue items.
-
uint32_t byte_swap_en
Enable byte swap.
-
uint32_t bk_buffer_dis
Disable backup buffer.
-
struct esp_cam_ctlr_csi_config_t::[anonymous] [anonymous]
Boolean Flags.
-
int ctlr_id
Header File
components/esp_driver_cam/isp_dvp/include/esp_cam_ctlr_isp_dvp.h
This header file can be included with:
#include "esp_cam_ctlr_isp_dvp.h"
This header file is a part of the API provided by the
esp_driver_cam
component. To declare that your component depends onesp_driver_cam
, add the following to your CMakeLists.txt:REQUIRES esp_driver_cam
or
PRIV_REQUIRES esp_driver_cam
Functions
-
esp_err_t esp_cam_new_isp_dvp_ctlr(isp_proc_handle_t isp_proc, const esp_cam_ctlr_isp_dvp_cfg_t *ctlr_config, esp_cam_ctlr_handle_t *ret_handle)
New ESP CAM ISP DVP controller.
- Parameters
isp_proc -- [in] Processor handle
ctlr_config -- [in] ISP DVP controller configurations
ret_handle -- [out] Returned ESP CAM controller handle
- Returns
ESP_OK
ESP_ERR_INVALID_ARG: Invalid argument
ESP_ERR_NO_MEM: Out of memory
ESP_ERR_NOT_SUPPORTED: Currently not support modes or types
ESP_ERR_NOT_FOUND: ISP DVP is registered already
Structures
-
struct esp_cam_ctlr_isp_dvp_cfg_t
ESP CAM ISP DVP controller configurations.
Public Members
-
cam_ctlr_data_width_t data_width
Number of data lines.
-
int data_io[ISP_DVP_DATA_SIG_NUM]
ISP DVP data-in IO numbers.
-
int pclk_io
ISP DVP pclk IO numbers.
-
int hsync_io
ISP DVP hsync IO numbers.
-
int vsync_io
ISP DVP vsync IO numbers.
-
int de_io
ISP DVP de IO numbers.
-
uint32_t pclk_invert
The pclk is inverted.
-
uint32_t hsync_invert
The hsync signal is inverted.
-
uint32_t vsync_invert
The vsync signal is inverted.
-
uint32_t de_invert
The de signal is inverted.
-
struct esp_cam_ctlr_isp_dvp_cfg_t::[anonymous] io_flags
ISP DVP IO flags.
-
int queue_items
Queue items.
-
uint32_t byte_swap_en
Enable byte swap.
-
uint32_t bk_buffer_dis
Disable backup buffer.
-
cam_ctlr_data_width_t data_width