The R-Car SoCs contain a large number of peripheral devices. SoC variants within a family usually differ by the number and configuration of the peripherals, while new SoC generations introduce more drastic changes in the peripherals themselves, both in the form of new peripherals and significative architectural changes to existing peripherals.
This page describes Linux support for the R-Car family peripherals. It puts the focus on usage of the peripherals from a userspace point of view. Implementation details internal to the Linux kernel are considered out of scope here, and differences between versions of a peripheral are only described if they matter for applications.
The VSP is the successor of the VIO6 video processing engine found in the first generation R-Car SoCs. I provides per-frame video processing such as color management, up/down scaling and blending. The first version of the VSP, named VSP1, is available on the second generation R-Car SoCs. The third generation R-Car SoCs introduced an updated VSP named VSP2 with support for more video processing functions.
The VSP operates in memory-to-memory mode, with some instances supporting direct output to the Display Unit (DU). All instances have one or more input channels to read data from memory, one or more output channels to write data to memory, and a variable number of video processing blocks that can be arranged freely to create processing pipelines.
- The Read Pixel Formatter (RPF) reads images from memory and unpacks them with optional pixel encoding conversion and alpha channel support.
- The Write Pixel Formatter (WPF) packs and writes images to memory with optional dithering, mirroring/flipping and pixel encoding conversion support.
- The Blend ROP Unit (BRU) blends up to four (Gen2) or five (Gen3) input frames into a single output using ROP and/or alpha blending.
- The Blend ROP Sub unit (BRS) is an additional two inputs blend/ROP unit.
- The Super Resolution Unit (SRU) performs super-resolution processing with optional x2 upscaling.
- The Up/Down Scaler (SRU) scales the input frame up or down by a factor of x1/4 to x4.
- The Look Up Table (LUT) converts the input frame using a per-component look-up table.
- The Cubic Look-up Table (CLU) converts the input frame using a 2-dimensional or 3-dimensional look-up table that takes all pixel components into account simultaneously.
- The Hue Saturation value Transform (HST) converts a frame from RGB to HSV.
- The Hue Saturation value Inverse transform (HSI) converts a frame from HSV to RGB.
- The Histogram Generator One dimension (HGO) computes per-component one-dimensional histograms on a frame.
- The Histogram Generator Two dimension (HGT) computes a two-dimensional histogram on a frame.
- The Lcdc InterFace (LIF) transfers frames to the Display Unit (DU).
- The SHarPness (SHP) sharpens or unsharpens frames.
The following table shows the blocks available in each VSP variant.
|# RPF||# WPF||Other Blocks|
|VSPD||4||1||BRU, HGO, HSI, HST, LIF, LUT, UDS (x1)|
|VSPR||5||4||BRU, HSI, HST, SRU, UDS (x3)|
|VSPS (H2)||5||4||BRU, CLU, HGO, HGT, HSI, HST, LUT, SRU, UDS (x3)|
|VSPS (M2)||5||4||BRU, CLU, HGO, HGT, HSI, HST, LUT, SRU, UDS (x1)|
|VSPBC (H3, M3)||5||1||BRU, CLU, HGO, LUT|
|VSPD (H3, M3)||5||2||BRU, LIF|
|VSPD (V3)||5||1||BRS, BRU, LIF|
|VSPI (H3, M3)||1||1||CLU, HGO, HGT, HSI, HST, LUT, SHP, SRU, UDS (x1)|
Both the VSP1 and VSP2 are supported in mainline Linux by a V4L2 driver named vsp1 (the BRS, HGT and SHP blocks are currently not supported). The driver uses the Media Controller API. It creates one video device node for each input and output, and exposes one V4L2 subdev device node to userspace for each hardware processing block. Applications use the Media Controller link setup API to connect the blocks into a video pipelines, the V4L2 subdev API to configure formats on the pipeline and the V4L2 API to stream video in and out of the VSP.
The following list describes the API exposed by each supported hardware block.
- The RPF exposes one subdev node and one video device node to userspace. The video node is a video output device named rpf.[0-9] input implementing the V4L2 multiplane video API with V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE buffers. It reads data from memory and provides it to the RPF. The subdev is named rpf.[0-9] and has two pads, a sink pad connected to the RPF video node and a source pad connectable to any of the other non-RPF subdevs.
- Similarly to the RPF, the WPF exposes one subdev node and one video device node to userspace. The video node is a video capture device named wpf.[0-9] implementing the V4L2 multiplane video API with V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE buffers. It stores data provided by the WPF to memory. The subdev is named wpf.[0-9] and has two pads, a sink pad connectable to any of the other non-WPF subdevs and a source pad connected to the WPF video node.
- The HGO exposes one subdev node and one video device node to userspace. The video node is a video capture device named hgo output implementing the V4L2 metadata API with V4L2_BUF_TYPE_META_CAPTURE buffers. It stores histogram data computed by the HGO to memory and operates with a V4L2_BUF_TYPE_META_CAPTURE buffers queue. The subdev is named hgo and has two pads, a sink pad connectable to any of the non-WPF subdevs and a source pad connected to the HGO video node.
- The HGO subdev supports the crop and compose selection rectangles on its sink pad. The crop rectangle controls the portion of the input frame that is used for histogram computation. The compose rectangle controls subsampling through skipping, with supported skipping values of no skipping, 1/2 and 1/4 skipping, independently in the horizontal and vertical directions. The compose rectangle top left corner is thus hardwired to (0,0), only the rectangle size can be modified.
- The HGO subdev exposes two controls to configure the Maximum RGB Mode (Gen2 and Gen3) and the number of bins (Gen3 only). When Maximum RGB Mode is enabled the HGO computes the minimum, maximum, sums and histogram values on the maximum of the R, G and B components for each pixel. Otherwise it computes the statistics on the R, G and B components separately. Maximum RGB Mode can only be enabled when operating on RGB pixels. The number of bins can be configured to 64 or 256 on Gen3 and is fixed to 64 on Gen2.
The VSP unit tests suite is available in the following git tree.
Instructions to build, install and run the test scripts are available in the README file.