Video

The video driver API offers a generic interface to video devices.

Basic Operation

Video Device

A video device is the abstraction of a hardware or software video function, which can produce, process, consume or transform video data. The video API is designed to offer flexible way to create, handle and combine various video devices.

Endpoint

Each video device can have one or more endpoints. Output endpoints configure video output function and generate data. Input endpoints configure video input function and consume data.

Video Buffer

A video buffer provides the transport mechanism for the data. There is no particular requirement on the content. The requirement for the content is defined by the endpoint format. A video buffer can be queued to a device endpoint for filling (input ep) or consuming (output ep) operation, once the operation is achieved, buffer can be dequeued for post-processing, release or reuse.

Controls

A video control is accessed and identified by a CID (control identifier). It represents a video control property. Different devices will have different controls available which can be generic, related to a device class or vendor specific. The set/get control functions provide a generic scalable interface to handle and create controls.

Configuration Options

Related configuration options:

API Reference

Related code samples

Video TCP server sink

Capture video frames and send them over the network to a TCP client.

Video capture

Use the video API to retrieve video frames from a capture device.

group video_interface

Video Interface.

Since

2.1

Version

1.0.0

Defines

video_fourcc(a, b, c, d)

Typedefs

typedef int (*video_api_set_format_t)(const struct device *dev, enum video_endpoint_id ep, struct video_format *fmt)

Set video format.

See video_set_format() for argument descriptions.

typedef int (*video_api_get_format_t)(const struct device *dev, enum video_endpoint_id ep, struct video_format *fmt)

Get current video format.

See video_get_format() for argument descriptions.

typedef int (*video_api_enqueue_t)(const struct device *dev, enum video_endpoint_id ep, struct video_buffer *buf)

Enqueue a buffer in the driver’s incoming queue.

See video_enqueue() for argument descriptions.

typedef int (*video_api_dequeue_t)(const struct device *dev, enum video_endpoint_id ep, struct video_buffer **buf, k_timeout_t timeout)

Dequeue a buffer from the driver’s outgoing queue.

See video_dequeue() for argument descriptions.

typedef int (*video_api_flush_t)(const struct device *dev, enum video_endpoint_id ep, bool cancel)

Flush endpoint buffers, buffer are moved from incoming queue to outgoing queue.

See video_flush() for argument descriptions.

typedef int (*video_api_stream_start_t)(const struct device *dev)

Start the capture or output process.

See video_stream_start() for argument descriptions.

typedef int (*video_api_stream_stop_t)(const struct device *dev)

Stop the capture or output process.

See video_stream_stop() for argument descriptions.

typedef int (*video_api_set_ctrl_t)(const struct device *dev, unsigned int cid, void *value)

Set a video control value.

See video_set_ctrl() for argument descriptions.

typedef int (*video_api_get_ctrl_t)(const struct device *dev, unsigned int cid, void *value)

Get a video control value.

See video_get_ctrl() for argument descriptions.

typedef int (*video_api_get_caps_t)(const struct device *dev, enum video_endpoint_id ep, struct video_caps *caps)

Get capabilities of a video endpoint.

See video_get_caps() for argument descriptions.

typedef int (*video_api_set_signal_t)(const struct device *dev, enum video_endpoint_id ep, struct k_poll_signal *signal)

Register/Unregister poll signal for buffer events.

See video_set_signal() for argument descriptions.

Enums

enum video_endpoint_id

video_endpoint_id enum

Identify the video device endpoint.

Values:

enumerator VIDEO_EP_NONE
enumerator VIDEO_EP_ANY
enumerator VIDEO_EP_IN
enumerator VIDEO_EP_OUT
enum video_signal_result

video_event enum

Identify video event.

Values:

enumerator VIDEO_BUF_DONE
enumerator VIDEO_BUF_ABORTED
enumerator VIDEO_BUF_ERROR

Functions

static inline int video_set_format(const struct device *dev, enum video_endpoint_id ep, struct video_format *fmt)

Set video format.

Configure video device with a specific format.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • ep – Endpoint ID.

  • fmt – Pointer to a video format struct.

Return values:

  • 0 – Is successful.

  • -EINVAL – If parameters are invalid.

  • -ENOTSUP – If format is not supported.

  • -EIO – General input / output error.

static inline int video_get_format(const struct device *dev, enum video_endpoint_id ep, struct video_format *fmt)

Get video format.

Get video device current video format.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • ep – Endpoint ID.

  • fmt – Pointer to video format struct.

Return values:

pointer – to video format

static inline int video_enqueue(const struct device *dev, enum video_endpoint_id ep, struct video_buffer *buf)

Enqueue a video buffer.

Enqueue an empty (capturing) or filled (output) video buffer in the driver’s endpoint incoming queue.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • ep – Endpoint ID.

  • buf – Pointer to the video buffer.

Return values:

  • 0 – Is successful.

  • -EINVAL – If parameters are invalid.

  • -EIO – General input / output error.

static inline int video_dequeue(const struct device *dev, enum video_endpoint_id ep, struct video_buffer **buf, k_timeout_t timeout)

Dequeue a video buffer.

Dequeue a filled (capturing) or displayed (output) buffer from the driver’s endpoint outgoing queue.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • ep – Endpoint ID.

  • buf – Pointer a video buffer pointer.

  • timeout – Timeout

Return values:

  • 0 – Is successful.

  • -EINVAL – If parameters are invalid.

  • -EIO – General input / output error.

static inline int video_flush(const struct device *dev, enum video_endpoint_id ep, bool cancel)

Flush endpoint buffers.

A call to flush finishes when all endpoint buffers have been moved from incoming queue to outgoing queue. Either because canceled or fully processed through the video function.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • ep – Endpoint ID.

  • cancel – If true, cancel buffer processing instead of waiting for completion.

Return values:

0 – Is successful, -ERRNO code otherwise.

static inline int video_stream_start(const struct device *dev)

Start the video device function.

video_stream_start is called to enter ‘streaming’ state (capture, output…). The driver may receive buffers with video_enqueue() before video_stream_start is called. If driver/device needs a minimum number of buffers before being able to start streaming, then driver set the min_vbuf_count to the related endpoint capabilities.

Return values:

  • 0 – Is successful.

  • -EIO – General input / output error.

static inline int video_stream_stop(const struct device *dev)

Stop the video device function.

On video_stream_stop, driver must stop any transactions or wait until they finish.

Return values:

  • 0 – Is successful.

  • -EIO – General input / output error.

static inline int video_get_caps(const struct device *dev, enum video_endpoint_id ep, struct video_caps *caps)

Get the capabilities of a video endpoint.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • ep – Endpoint ID.

  • caps – Pointer to the video_caps struct to fill.

Return values:

0 – Is successful, -ERRNO code otherwise.

static inline int video_set_ctrl(const struct device *dev, unsigned int cid, void *value)

Set the value of a control.

This set the value of a video control, value type depends on control ID, and must be interpreted accordingly.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • cid – Control ID.

  • value – Pointer to the control value.

Return values:

  • 0 – Is successful.

  • -EINVAL – If parameters are invalid.

  • -ENOTSUP – If format is not supported.

  • -EIO – General input / output error.

static inline int video_get_ctrl(const struct device *dev, unsigned int cid, void *value)

Get the current value of a control.

This retrieve the value of a video control, value type depends on control ID, and must be interpreted accordingly.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • cid – Control ID.

  • value – Pointer to the control value.

Return values:

  • 0 – Is successful.

  • -EINVAL – If parameters are invalid.

  • -ENOTSUP – If format is not supported.

  • -EIO – General input / output error.

static inline int video_set_signal(const struct device *dev, enum video_endpoint_id ep, struct k_poll_signal *signal)

Register/Unregister k_poll signal for a video endpoint.

Register a poll signal to the endpoint, which will be signaled on frame completion (done, aborted, error). Registering a NULL poll signal unregisters any previously registered signal.

Parameters:

  • dev – Pointer to the device structure for the driver instance.

  • ep – Endpoint ID.

  • signal – Pointer to k_poll_signal

Return values:

0 – Is successful, -ERRNO code otherwise.

struct video_buffer *video_buffer_aligned_alloc(size_t size, size_t align)

Allocate aligned video buffer.

Parameters:

  • size – Size of the video buffer (in bytes).

  • align – Alignment of the requested memory, must be a power of two.

Return values:

pointer – to allocated video buffer

struct video_buffer *video_buffer_alloc(size_t size)

Allocate video buffer.

Parameters:

  • size – Size of the video buffer (in bytes).

Return values:

pointer – to allocated video buffer

void video_buffer_release(struct video_buffer *buf)

Release a video buffer.

Parameters:

  • buf – Pointer to the video buffer to release.

struct video_format
#include <video.h>

Video format structure.

Used to configure frame format.

Public Members

uint32_t pixelformat

FourCC pixel format value (Video pixel formats)

uint32_t width

frame width in pixels.

uint32_t height

frame height in pixels.

uint32_t pitch

line stride.

This is the number of bytes that needs to be added to the address in the first pixel of a row in order to go to the address of the first pixel of the next row (>=width).

struct video_format_cap
#include <video.h>

Video format capability.

Used to describe a video endpoint format capability.

Public Members

uint32_t pixelformat

FourCC pixel format value (Video pixel formats).

uint32_t width_min

minimum supported frame width in pixels.

uint32_t width_max

maximum supported frame width in pixels.

uint32_t height_min

minimum supported frame height in pixels.

uint32_t height_max

maximum supported frame height in pixels.

uint16_t width_step

width step size in pixels.

uint16_t height_step

height step size in pixels.

struct video_caps
#include <video.h>

Video format capabilities.

Used to describe video endpoint capabilities.

Public Members

const struct video_format_cap *format_caps

list of video format capabilities (zero terminated).

uint8_t min_vbuf_count

minimal count of video buffers to enqueue before being able to start the stream.

struct video_buffer
#include <video.h>

Video buffer structure.

Represent a video frame.

Public Members

void *driver_data

pointer to driver specific data.

uint8_t *buffer

pointer to the start of the buffer.

uint32_t size

size of the buffer in bytes.

uint32_t bytesused

number of bytes occupied by the valid data in the buffer.

uint32_t timestamp

time reference in milliseconds at which the last data byte was actually received for input endpoints or to be consumed for output endpoints.

struct video_driver_api
#include <video.h>
group video_controls

Video controls.

Control classes

VIDEO_CTRL_CLASS_GENERIC

Generic class controls.

VIDEO_CTRL_CLASS_CAMERA

Camera class controls.

VIDEO_CTRL_CLASS_MPEG

MPEG-compression controls.

VIDEO_CTRL_CLASS_JPEG

JPEG-compression controls.

VIDEO_CTRL_CLASS_VENDOR

Vendor-specific class controls.

Generic class control IDs

VIDEO_CID_HFLIP

Mirror the picture horizontally.

VIDEO_CID_VFLIP

Mirror the picture vertically.

Camera class control IDs

VIDEO_CID_CAMERA_EXPOSURE
VIDEO_CID_CAMERA_GAIN
VIDEO_CID_CAMERA_ZOOM
VIDEO_CID_CAMERA_BRIGHTNESS
VIDEO_CID_CAMERA_SATURATION
VIDEO_CID_CAMERA_WHITE_BAL
VIDEO_CID_CAMERA_CONTRAST
VIDEO_CID_CAMERA_COLORBAR
VIDEO_CID_CAMERA_QUALITY