sensor – Camera

[中文]

The sensor module controls the camera and captures frames. It mirrors the OpenMV sensor API so existing OpenMV scripts port with little change.

Camera Initialization and Continuous Capture

A typical program resets the sensor, selects a pixel format and frame size, lets automatic exposure and white balance settle, and then captures frames continuously:

import sensor

sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time=2000)

while True:
    img = sensor.snapshot()

sensor.reset() applies the board’s default camera configuration. RGB565 is suitable for display, drawing, and most AI workflows, while GRAYSCALE reduces memory and processing cost for many detection algorithms. sensor.snapshot() returns an image.Image backed by the reusable frame buffer, so copy the image when it must remain unchanged after the next capture.

Image Orientation and Camera Status

Use horizontal mirror and vertical flip to correct the image orientation imposed by the physical sensor installation. status() provides the active dimensions, pixel format, sensor ID, orientation, and crop information for diagnostics:

import sensor

sensor.reset()
sensor.set_hmirror(True)
sensor.set_vflip(False)

info = sensor.status()
print("sensor id:", info["id"])
print("output:", info["width"], "x", info["height"])
print("ready:", info["ready"])

Mirror and flip settings affect subsequently captured frames. Product code can call status() after initialization to verify that the camera is ready and that the negotiated output size matches the processing pipeline.

Temporarily Stop Capture

The camera stream can be stopped when capture is not required and restarted before the next frame:

sensor.shutdown(True)
# Perform work that does not require the camera.
sensor.shutdown(False)
sensor.skip_frames(n=3)
img = sensor.snapshot()

After restarting the stream, discard a few frames if exposure or the incoming frame queue needs to stabilize.

IPA Runtime Control

IPA and ISP control methods are optional firmware features. Check hasattr(sensor, "get_ipa") or hasattr(sensor, "get_isp") before using them; these methods are absent when the feature is not built in.

Use sensor.set_ipa(False) to stop the image processing algorithm (IPA) controller and sensor.set_ipa(True) to start it again. Initialize the camera before using these methods. sensor.get_ipa() returns the controller’s current state, or None when runtime control is unsupported. Setting IPA on an unsupported camera raises OSError.

sensor.reset()
sensor.skip_frames(time=2000)
if hasattr(sensor, "get_ipa") and sensor.get_ipa() is not None:
    sensor.set_ipa(False)
    print("IPA:", sensor.get_ipa())
    img = sensor.snapshot()
    img.flush()
    sensor.set_ipa(True)

Changing IPA state briefly stops and restarts capture while preserving the output format, frame size, mirror, and flip settings. Setting the current state again has no effect. sensor.reset() and restarting after sensor.shutdown(True) restore the default IPA state.

A failed switch leaves the camera shut down rather than in its previous state: the capture buffers, the scaler, and the camera device itself are all released before OSError is raised. Call sensor.reset() to bring the camera back.

Disabling IPA stops its automatic parameter updates; it does not bypass the ISP or clear existing tuning. The image may initially look unchanged. To compare the effect, change the illumination or light source color while previewing, then enable IPA and allow its algorithms to settle again. The algorithms controlled by this switch depend on the active IPA configuration.

Manual ISP Control

After initialization, sensor.get_isp(block) returns a dictionary of the current driver parameters, or None when the block is unsupported. A block is unsupported when the camera driver has no getter for it or when the firmware was built for a chip revision that lacks the hardware: lsc needs ESP32-P4 v1.0 or newer and blc needs v3.0 or newer, while exposure is write-only on sensors such as SC2336, so sensor.get_isp("exposure") normally returns None. Call sensor.set_ipa(False) before writing manual settings. sensor.set_isp(block, **params) updates one block, keeping unspecified fields from its current configuration. Writes while IPA is running raise OSError to prevent the algorithms from overwriting manual settings.

sensor.reset()
sensor.skip_frames(time=2000)
sensor.set_ipa(False)

print(sensor.get_isp("wb"))
sensor.set_isp("wb", enable=False)
sensor.set_isp("ccm", enable=False)
sensor.set_isp("gamma", enable=False)
sensor.snapshot().flush()

sensor.set_isp("wb", enable=True, rg=1.8, bg=1.4)
sensor.set_isp("exposure", value=10000)  # Microseconds, rounded to 100 us.
sensor.set_isp("pixel_gain", value=2.0)  # Gain multiplier.

sensor.set_ipa(True)

The supported block names and fields follow the tuning parameter vocabulary:

ISP parameters
Block	Fields
`exposure`	`value`: exposure in microseconds, positive. The device control counts 100-microsecond units, so a write is rounded to the nearest 100 microseconds and must then fit the range and step the sensor reports, which is where the usable limits come from. SC2336 at 1280x720 30fps, for example, accepts roughly 200 microseconds through 33.2 milliseconds in 100-microsecond steps.
`pixel_gain`	`value`: positive gain multiplier, selected from the nearest supported gain menu entry within the sensor’s range.
`demosaic`	`enable`, `gradient_ratio`: nonnegative gradient ratio within the driver’s fixed-point range.
`wb`	`enable`, `rg`, `bg`: red and blue gain multipliers, 0 through 1023/256.
`lsc`	`enable`, `lsc_gain_size`, `gain_r`, `gain_gr`, `gain_gb`, `gain_b`: four gain lists, each with the length returned by `get_isp("lsc")`; values are 0 through 1023/256. The size is determined by capture dimensions.
`blc`	`enable`, `stretch_enable`, `top_left_offset`, `top_right_offset`, `bottom_left_offset`, `bottom_right_offset`: black level offsets, integers from 0 through 255.
`bf`	`enable`, `level` (2–20), `matrix`: nine integers from 0 through 15 in row order.
`sharpen`	`enable`, `h_thresh`, `l_thresh` (0–255, low ≤ high), `h_coeff`, `m_coeff` (0 through 255/32), `matrix`: nine integers from 0 through 31 in row order.
`ccm`	`enable`, `matrix`: nine finite numbers strictly between -4 and 4 in row order.
`gamma`	`enable`, `x`, `y`: two lists of 16 integers from 0 through 255. X must increase from an implicit zero by powers of two; the last point must be 255 and is treated as 256 for the final interval.

Set enable=True when applying new parameters to a disabled block. Passing only enable=False disables processing without replacing its stored tables. White balance gains are kept across that disable and come back when the block is enabled again. Gamma readback reports the red channel; setting x or y applies the resulting curve to all three channels, while an enable-only update preserves the separate channel curves. LSC tables are copied into camera-owned storage and remain valid across garbage collection and IPA switches; the camera keeps the previously installed table intact until the driver accepts the new one, so a rejected write leaves the image unchanged. Fixed-point values and sensor settings may be quantized by the driver.

Unknown blocks, invalid values, and array length mismatches are rejected before applying the update. An unsupported write raises OSError. A device error may leave a partially applied configuration; use sensor.reset() to reinitialize after such a failure.

The example/01-Camera/04-IPA/isp_preview.py example configures the output format, disables IPA, then continuously previews while randomizing exposure, gain, white balance gains, and CCM every 2 seconds until Ctrl-C. Each update prints the requested exposure and the driver readback for gain, white balance gains, and CCM. Adjust UPDATE_INTERVAL_MS and the parameter range constants to change the experiment; a value the active sensor rejects is printed and skipped rather than stopping the preview. Ctrl-C leaves IPA disabled and preserves the last settings.

Constants

sensor.GRAYSCALE: Pixel format constants accepted by set_pixformat().

sensor.RGB565

sensor.QQVGA: Frame size constants accepted by set_framesize().

sensor.QVGA

sensor.VGA

Functions

sensor.reset(): Reset and initialize the camera with the board default sensor configuration.

sensor.shutdown(enable=...)

Stop or restart the camera stream.

Parameters:: enable – True shuts the camera down; False starts it again.

sensor.set_pixformat(pixformat)

Select the output pixel format.

Parameters:: pixformat – sensor.GRAYSCALE or sensor.RGB565.

sensor.get_pixformat(): Return the current pixel format constant.

sensor.set_framesize(framesize)

Select the output frame size.

Parameters:: framesize – sensor.QQVGA, sensor.QVGA, or sensor.VGA when supported by the board.

sensor.get_framesize(): Return the current frame size constant.

sensor.width(): Return the current output image width in pixels.

sensor.height(): Return the current output image height in pixels.

sensor.get_id(): Return the camera sensor ID.

sensor.set_hmirror(enable)

Mirror the camera image horizontally.

Parameters:: enable – True enables horizontal mirror.

sensor.get_hmirror(): Return whether horizontal mirror is enabled.

sensor.set_vflip(enable)

Flip the camera image vertically.

Parameters:: enable – True enables vertical flip.

sensor.get_vflip(): Return whether vertical flip is enabled.

sensor.set_ipa(enable): Optional API: present only in firmware with IPA/ISP control enabled. Start or stop the IPA controller after camera initialization; briefly pauses capture. Existing ISP tuning is retained when disabled. Unsupported cameras raise OSError.

sensor.get_ipa(): Optional API: present only in firmware with IPA/ISP control enabled. Return the IPA controller state, or None when runtime control is unsupported. A supported camera must be initialized first.

sensor.get_isp(block): Optional API: present only in firmware with IPA/ISP control enabled. Read current ISP block parameters, or None if unsupported. Initialize the camera first. Blocks the camera driver or chip revision does not implement return None, including “exposure” on sensors whose driver makes it write-only.

sensor.set_isp(block, **params): Optional API: present only in firmware with IPA/ISP control enabled. Update one ISP block using keyword parameters; unspecified fields retain their values. Disable IPA before writing. Invalid parameters raise ValueError; device errors raise OSError.

sensor.skip_frames(time=..., n=...)

Drop frames while camera exposure and processing settle.

Parameters:: time – milliseconds to wait. n: number of frames to skip.

sensor.snapshot(buffer=...)

Capture one image from the camera.

Parameters:: buffer – reserved for compatibility; pass None in current ESP-VISION builds.

sensor.status(): Return camera readiness, size, format, mirror, flip, and crop status.

Classes

class sensor.SensorStatus: Dictionary returned by status().