R-Car/Tests:rcar-vin

From eLinux.org
Jump to: navigation, search

This document describes how to test the VIN and CSI-2 (only on Gen3) functionality on Renesas R-Car Gen2 and Gen3.

Get the code

Gen2 from v4.10, Gen3 from v4.17.

Kernel Options

The following kernel configuration options should be added (Gen3)

CONFIG_V4L_PLATFORM_DRIVERS=y
CONFIG_VIDEO_RCAR_VIN=y
CONFIG_VIDEO_RCAR_CSI2=y
CONFIG_VIDEO_ADV748X=y

Boot the kernel

You should see this on successful probe (this is from Gen3):

rcar-vin e6ef0000.video: Device registered as video0
rcar-vin e6ef1000.video: Device registered as video1
rcar-vin e6ef2000.video: Device registered as video2
rcar-vin e6ef3000.video: Device registered as video3
rcar-vin e6ef4000.video: Device registered as video4
rcar-vin e6ef5000.video: Device registered as video5
rcar-vin e6ef6000.video: Device registered as video6
rcar-vin e6ef7000.video: Device registered as video7

Media Controller (Gen3)

The hardware available on Gen3 is much more complex then on Gen2. For this reason it's no longer enough to interact with only the video device to control the hardware, the entire media pipeline needs to configured. To facilitate this the Media Controller (MC) framework is used exclusivity on Gen3. By using the MC framework it is up to user-space to first configure the media pipeline and then setup the correct format for each entity involved in the pipeline before streaming can be started on the video device.

The pipeline will consist of a video sensor (CSI-2 Transmitter, ADV7482 on Salvator-X) a CSI-2 receiver (R-Car CSI-2) and a dma engine (R-Car VIN). The video sensor is immutably connected to a single R-Car CSI-2 while the routing between the R-Car CSI-2 and R-Car VIN can be configured at runtime. There are limitations on which channel from the R-Car CSI-2 node can be connected to which R-Car VIN node, this is a hardware limitation that is different for each SoC in the Gen3 series. All links between these different nodes and all possible routing configurations are exposed as a media device to user-space. After the links in the pipeline have been setup by the user the proper format needs to be set on all nodes involved in the pipeline before the user can start a stream at the video device.

To control the routing and format from user-space the user needs to interact with the media device, the subdevices for video sensor and R-Car CSI-2, the video node for R-Car VIN. For configuring the media device and subdevices 'media-ctl' can be used and to configure the video device itself a grabber can be used, such as 'yavta' or 'qv4l2'. The following sections will briefly describe how to use these tools to configure the pipeline for streaming.

Show current routing

Examine the current routing setup with 'media-ctl -d /dev/mediaX -p'.

$ media-ctl -d /dev/media0 -p
Media controller API version 4.17.0

Media device information
------------------------
driver          rcar_vin
model           renesas,vin-r8a77965
serial          
bus info        platform:e6ef0000.video
hw revision     0x0
driver version  4.17.0

Device topology
- entity 1: rcar_csi2 feaa0000.csi2 (5 pads, 9 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev19
	pad0: Sink
		[fmt:unknown/0x0]
		<- "adv748x 4-0070 txa":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/0x0]
		-> "VIN0 output":0 []
		-> "VIN1 output":0 []
		-> "VIN2 output":0 []
	pad2: Source
		[fmt:unknown/0x0]
		-> "VIN0 output":0 []
		-> "VIN1 output":0 []
		-> "VIN3 output":0 []
	pad3: Source
		[fmt:unknown/0x0]
		-> "VIN2 output":0 []
	pad4: Source
		[fmt:unknown/0x0]
		-> "VIN3 output":0 []

- entity 7: adv748x 4-0070 txa (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev21
	pad0: Sink
		[fmt:unknown/0x0]
		<- "adv748x 4-0070 hdmi":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/0x0]
		-> "rcar_csi2 feaa0000.csi2":0 [ENABLED,IMMUTABLE]

- entity 10: adv748x 4-0070 hdmi (2 pads, 1 link)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev20
	pad0: Sink
		[dv.caps:BT.656/1120 min:640x480@13000000 max:1920x1200@162000000 stds:CEA-861,DMT caps:progressive]
	pad1: Source
		[fmt:RGB888_1X24/1280x720 field:none colorspace:srgb]
		[dv.caps:BT.656/1120 min:640x480@13000000 max:1920x1200@162000000 stds:CEA-861,DMT caps:progressive]
		[dv.query:no-link]
		[dv.current:BT.656/1120 1280x720p30 (3300x750) stds:CEA-861 flags:can-reduce-fps,CE-video,has-cea861-vic]
		-> "adv748x 4-0070 txa":0 [ENABLED,IMMUTABLE]

- entity 17: rcar_csi2 fea80000.csi2 (5 pads, 17 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev22
	pad0: Sink
		[fmt:unknown/0x0]
		<- "adv748x 4-0070 txb":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/0x0]
		-> "VIN0 output":0 []
		-> "VIN1 output":0 []
		-> "VIN2 output":0 []
		-> "VIN4 output":0 []
		-> "VIN5 output":0 []
		-> "VIN6 output":0 []
	pad2: Source
		[fmt:unknown/0x0]
		-> "VIN1 output":0 []
		-> "VIN2 output":0 []
		-> "VIN3 output":0 []
		-> "VIN5 output":0 []
		-> "VIN6 output":0 []
		-> "VIN7 output":0 []
	pad3: Source
		[fmt:unknown/0x0]
		-> "VIN2 output":0 []
		-> "VIN6 output":0 []
	pad4: Source
		[fmt:unknown/0x0]
		-> "VIN3 output":0 []
		-> "VIN7 output":0 []

- entity 23: adv748x 4-0070 txb (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev24
	pad0: Sink
		[fmt:unknown/0x0]
		<- "adv748x 4-0070 afe":8 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:unknown/0x0]
		-> "rcar_csi2 fea80000.csi2":0 [ENABLED,IMMUTABLE]

- entity 26: adv748x 4-0070 afe (9 pads, 1 link)
             type V4L2 subdev subtype Decoder flags 0
             device node name /dev/v4l-subdev23
	pad0: Sink
	pad1: Sink
	pad2: Sink
	pad3: Sink
	pad4: Sink
	pad5: Sink
	pad6: Sink
	pad7: Sink
	pad8: Source
		[fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
		-> "adv748x 4-0070 txb":0 [ENABLED,IMMUTABLE]

- entity 52: VIN0 output (1 pad, 3 links)
             type Node subtype V4L flags 0
             device node name /dev/video10
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":1 []

- entity 56: VIN1 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video11
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":2 []

- entity 60: VIN2 output (1 pad, 5 links)
             type Node subtype V4L flags 0
             device node name /dev/video12
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":3 []
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":3 []

- entity 64: VIN3 output (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video13
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 feaa0000.csi2":4 []
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":4 []

- entity 68: VIN4 output (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video14
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":1 []

- entity 72: VIN5 output (1 pad, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/video15
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":2 []

- entity 76: VIN6 output (1 pad, 3 links)
             type Node subtype V4L flags 0
             device node name /dev/video16
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":3 []

- entity 80: VIN7 output (1 pad, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/video17
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":4 []

Activate/Deactivate a link

The Media Controller framework allows user-space to enable/disable a link and that way control the routing of video data from a R-Car CSI-2 to a R-Car VIN instance. Not all CSI-2 channels are possible to route to any and all VIN instances, this is limited by the SoC hardware and is specific for each SoC version. Look at the previous section on how to show which links are possible on your particular SoC. Further more not all links can be activated independent of other links, this is also a limitation set by the hardware. If you try to enable a link which would not be possible with regard to other already active links the operation will fail with a EMLINK error. Try deactivating some other link to crate more routing possibilities and try again, all possible routing setups for a specific SoC are documented in the datasheet.

A link is always configured from a R-Car CSI-2 entity to a R-Car VIN entity, the same way the video data is flowing.

To enable a link use the media-ctl utility from v4l-utils package:

# media-ctl -d /dev/media2 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"

To disable the same link use:

# media-ctl -d /dev/media2 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [0]"

Deactivate all active links

This is a useful command to reset all links before you start enabling new links to make sure you got the biggest possible routing space to start out with.

# media-ctl -d /dev/media0 -r

Example configuring the pipeline and propagate format

Once the user have configured a pipeline using 'media-ctl' as described above the format needs to be propagated in the pipeline before streaming can start. In this example we use a R-Car H3 board where the HDMI input will be configured to output on VIN2 and CVBS configured to VIN4. If we look at the media graph as show above we can identify which nodes are involved in each of these two pipelines:

HDMI: 'adv748x 4-0070 hdmi' -> 'rcar_csi2 feaa0000.csi2' -> 'VIN2 output'
CVBS: 'adv748x 4-0070 afe' -> 'rcar_csi2 fea80000.csi2' -> 'VIN4 output'

Start by clearing any previous links and configure the ones we want, remember that the link between the adv7482 and the R-Car CSI-2 entities are immutable and won't be cleared and therefore do not need to be set again.

media-ctl -d /dev/media0 -r
media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN2 output':0 [1]"
media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN4 output':0 [1]"

When the pipelines are configured it's time to propagate the format in the pipeline, start by examine which format is present at the video sources sinks.

# HDMI
$ media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"
	[fmt:RGB888_1X24/640x480 field:none colorspace:srgb]

# CVBS
$ media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"
	[fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]

Now we know the formats so we can propagate them:

# HDMI
media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/640x480 field:none]"
media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/640x480 field:none]"

# CVBS
media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"
media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"

Once the pipeline format is setup we can then grab frames using for example 'yavta' or look use a GUI like 'qv4l2'

# HDMI
yavta -f RGB565 -s 640x480 --field none -n 4 --capture=10 -F /dev/video2

# CVBS
yavta -f RGB565 -s 720x480 --field interlaced -n 4 --capture=10 -F -F /dev/video4

Multi-channel CSI-2 tests

Multi-channel CSI-2 testing uses an expansion board for the R-Car H3 Salvator-X based on a Maxim MAX9286 quad-channel GMSL (Gigabit Multimedia Serial Link) deserializer. Up to four cameras can be connected to the deserializer, which outputs the four video streams on separate virtual channels over one CSI-2 connections.

The cameras used in this setup are IMI RDACM20 camera modules based on an Omnivision 10635 camera sensor and a Maxim MAX9271 GMSL serializer. The GMSL link carries power, control (I2C) and video data over a single coax cable.

At this time only capture a single virtual channel at a time is supported.

On the hardware side, make sure to set SW29 to OFF-OFF on the Salvator-X board to select the external CSI-2 devices, to remove the J19 jumper from the MAX9286 expansion board to disable the second MAX9286, and to connect RDACM20 cameras to the first four GMSL ports (J1 to J4) and/or the last GMSL ports (J5, J10, J12 and J13).

On the software side, the latest version of the work-in-progress kernel code is available from

       git://git.ragnatech.se/linux v4l2/mux+gms


The kernel must be compiled with the following options.

CONFIG_VIDEO_MAX9286=y
CONFIG_VIDEO_RDACM20=y
CONFIG_VIDEO_RCAR_CSI2=y
CONFIG_VIDEO_RCAR_VIN=y
# CONFIG_VIDEO_RENESAS_FDP1 is not set
# CONFIG_VIDEO_RENESAS_JPU is not set
# CONFIG_VIDEO_RENESAS_VSP1 is not set

Disabling FDP1, JPU and VSP1 ensure that the media controller and video device nodes will have known names. You can enable those drivers if you update the device names accordingly in the commands below.

Capture from a single camera on J1

You need recent versions of the media-ctl and yavta tools on your system. After booting the system, simply run

# media-ctl -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"
# media-ctl -V "'rcar_csi2 feaa0000.csi2':1 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feaa0000.csi2':2 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feaa0000.csi2':3 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feaa0000.csi2':4 [fmt:UYVY8_2X8/1280x800 field:none]"
# yavta -f YUYV -s 1280x800 -c10 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' is a video capture device.
Video format set: YUYV (56595559) 1280x800 (stride 2560) buffer size 2048000
Video format: YUYV (56595559) 1280x800 (stride 2560) buffer size 2048000
8 buffers requested.
length: 2048000 offset: 0
Buffer 0 mapped at address 0x7f80c94000.
length: 2048000 offset: 2048000
Buffer 1 mapped at address 0x7f80aa0000.
length: 2048000 offset: 4096000
Buffer 2 mapped at address 0x7f808ac000.
length: 2048000 offset: 6144000
Buffer 3 mapped at address 0x7f806b8000.
length: 2048000 offset: 8192000
Buffer 4 mapped at address 0x7f804c4000.
length: 2048000 offset: 10240000
Buffer 5 mapped at address 0x7f802d0000.
length: 2048000 offset: 12288000
Buffer 6 mapped at address 0x7f800dc000.
length: 2048000 offset: 14336000
Buffer 7 mapped at address 0x7f7fee8000.
0 (0) [-] 0 2048000 bytes 182.333337 182.333348 8.889 fps
1 (1) [-] 1 2048000 bytes 182.366871 182.366878 29.820 fps
2 (2) [-] 2 2048000 bytes 182.400405 182.400412 29.820 fps
3 (3) [-] 3 2048000 bytes 182.433939 182.433946 29.820 fps
4 (4) [-] 4 2048000 bytes 182.467473 182.467480 29.820 fps
5 (5) [-] 5 2048000 bytes 182.501007 182.501014 29.820 fps
6 (6) [-] 6 2048000 bytes 182.534541 182.534548 29.820 fps
7 (7) [-] 7 2048000 bytes 182.568076 182.568082 29.820 fps
8 (0) [-] 8 2048000 bytes 182.601611 182.601622 29.820 fps
9 (1) [-] 9 2048000 bytes 182.635145 182.635157 29.820 fps
Captured 10 frames in 0.414323 seconds (24.135744 fps, 49430003.394668 B/s).
8 buffers released.

This will capture three images from the first camera (J1) to files frame-000007.bin, frame-000008.bin and frame-000009.bin.

Capture from two cameras using different Virtual Channels

You need recent versions of the media-ctl and yavta tools on your system. After booting the system, simply use the following commands to fist capture from camera J5 (CSI41/VC0) and then from camera J10 (CSI41/VC1).

Note: at this time it's not possible to capture from the two different VC in parallel, the two yavta commands needs to be run in sequence.

# media-ctl -l "'rcar_csi2 feab0000.csi2':1 -> 'VIN4 output':0 [1]"
# media-ctl -l "'rcar_csi2 feab0000.csi2':4 -> 'VIN7 output':0 [1]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':1 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':2 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':3 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':4 [fmt:UYVY8_2X8/1280x800 field:none]"
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin4-#.bin' /dev/video4
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin7-#.bin' /dev/video7

Example:

# media-ctl -l "'rcar_csi2 feab0000.csi2':1 -> 'VIN4 output':0 [1]"
# media-ctl -l "'rcar_csi2 feab0000.csi2':4 -> 'VIN7 output':0 [1]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':1 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':2 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':3 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':4 [fmt:UYVY8_2X8/1280x800 field:none]"
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin4-#.bin' /dev/video4
Device /dev/video4 opened.
Device `R_Car_VIN' on `platform:e6ef4000.video' (driver 'rcar_vin') is a video capture (without mplanes) device.
Video format set: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
Video format: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
8 buffers requested.
length: 2048000 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffffafa0c000.
lengt[  956.789621] max9286 4-006c: max9286_write(0x0a, 0xf1)
h: 2048000 offset: 2048000 timestamp type/source: mono/EoF
Buffer 1/0 mapped at[  956.802085] max9286 4-006c: max9286_read(0x31)
 address 0xffffaf818000.
length: 2048000 offset: 4096000 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffffaf624000.
length: 2048000 offset: 6144000 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffffaf430000.
length: 2048000 offset: 8192000 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffffaf23c000.
length: 2048000 offset: 10240000 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffffaf048000.
length: 2048000 offset: 12288000 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffffaee54000.
length: 2048000 o[  956.858450] max9286 4-006c: max9286_read(0x31)
ffset: 14336000 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffffaec60000.
[  956.915090] max9286 4-006c: max9286_read(0x31)
[  956.970377] max9286 4-006c: max9286_read(0x31)
[  957.025661] max9286 4-006c: max9286_read(0x31)
[  957.030940] max9286 4-006c: max9286_write(0x15, 0x9b)
0 (1) [-] none 0 2048000 B 957.156820 957.156873 8.216 fps ts mono/EoF
1 (2) [-] none 1 2048000 B 957.190355 957.190389 29.820 fps ts mono/EoF
2 (0) [-] none 2 2048000 B 957.223891 957.223908 29.819 fps ts mono/EoF
3 (3) [-] none 3 2048000 B 957.257427 957.257450 29.819 fps ts mono/EoF
4 (4) [-] none 4 2048000 B 957.290964 957.290986 29.818 fps ts mono/EoF
5 (5) [-] none 5 2048000 B 957.324500 957.324527 29.819 fps ts mono/EoF
6 (6) [-] none 6 2048000 B 957.358037 957.358057 29.818 fps ts mono/EoF
7 (7) [-] none 7 2048000 B 957.391573 957.391599 29.819 fps ts mono/EoF
8 (2) [-] none 8 2048000 B 957.458652 958.522813 14.908 fps ts mono/EoF
9 (1) [-] none 9 2048000 B 957.592800 959.653125 7.454 fps ts mono/EoF
[  960.764509] max9286 4-006c: max9286_write(0x15, 0x13)
Captured 10 frames in 2.618015 seconds (3.819688 fps, 7822720.374142 B/s).
8 buffers released.
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin7-#.bin' /dev/video7
Device /dev/video7 opened.
Device `R_Car_VIN' on `platform:e6ef7000.video' (driver 'rcar_vin') is a video capture (without mplanes) device.
Video format set: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
Video format: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
8 buffers requested.
length: 2048000 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffffba7ec000.
length: 20480[  980.717553] max9286 4-006c: max9286_write(0x0a, 0xf2)
00 offset: 2048000 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address[  980.730066] max9286 4-006c: max9286_read(0x31)
 0xffffba5f8000.
length: 2048000 offset: 4096000 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffffba404000.
length: 2048000 offset: 6144000 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffffba210000.
length: 2048000 offset: 8192000 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffffba01c000.
length: 2048000 offset: 10240000 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffffb9e28000.
length: 2048000 offset: 12288000 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffffb9c34000.
length: 2048000 offset: 1[  980.786173] max9286 4-006c: max9286_read(0x31)
4336000 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffffb9a40000.
[  980.842695] max9286 4-006c: max9286_read(0x31)
[  980.889524] max9286 4-006c: max9286_read(0x31)
[  980.944683] max9286 4-006c: max9286_read(0x31)
[  980.949837] max9286 4-006c: max9286_write(0x15, 0x9b)
0 (1) [-] none 0 2048000 B 981.088147 981.088163 7.466 fps ts mono/EoF
1 (2) [-] none 1 2048000 B 981.121681 981.121693 29.820 fps ts mono/EoF
2 (0) [-] none 2 2048000 B 981.155219 981.155228 29.817 fps ts mono/EoF
3 (3) [-] none 3 2048000 B 981.188755 981.188763 29.819 fps ts mono/EoF
4 (4) [-] none 4 2048000 B 981.222292 981.222301 29.818 fps ts mono/EoF
5 (5) [-] none 5 2048000 B 981.255828 981.255837 29.819 fps ts mono/EoF
6 (6) [-] none 6 2048000 B 981.289365 981.289373 29.818 fps ts mono/EoF
7 (7) [-] none 7 2048000 B 981.322902 981.322912 29.818 fps ts mono/EoF
8 (2) [-] none 8 2048000 B 981.389978 982.445497 14.908 fps ts mono/EoF
9 (1) [-] none 9 2048000 B 981.524124 983.585483 7.455 fps ts mono/EoF
[  984.697546] max9286 4-006c: max9286_write(0x15, 0x13)
Captured 10 frames in 2.631267 seconds (3.800449 fps, 7783319.958064 B/s).
8 buffers released.

This will capture three images from the fifth (J5) camera to files vin4-000007.bin, vin4-000008.bin and vin4-000009.bin and sixth (J10) camera to files vin7-000007.bin, vin7-000008.bin and vin7-000009.bin

Convert BIN to PNM

You can convert the captured bin frames to pnm with the raw2rgbpnm tool (which doesn't have to be run on the target system, you can use it in your development environment).

$ raw2rgbpnm -f YUYV -s 1280x800 frame-000007.bin frame-000007.pnm
0 padding bytes detected at end of line
Image size: 1280x800, bytes per pixel: 16, format: YVYU (16  YUV 4:2:2)
Writing to file `frame.pnm'...

Examine multiplexed pads routes

A vital part of multiplexed pad is each sub-device internal routing between its sink and source pads. This information is used by the kernel to perform link format validation and it can both be viewed and changed from user-space. However in the MAX9286 use-case this page describes the routes are all static and can't be changed.

To view the routes on both side of the multiplexed link between MAX9286_B and CSI41 start by finding each of the two sub-devices device nodes (using media-ctl -p). In the example bellow MAX9286_B is /dev/v4l-subdev55 and CSI41 is /dev/v4l-subdev51.

# ./v4l2-ctl -d /dev/v4l-subdev55 --get-routing 
0/0 -> 4/0 [ENABLED,IMMUTABLE]
1/0 -> 4/1 [ENABLED,IMMUTABLE]
2/0 -> 4/2 [ENABLED,IMMUTABLE]
3/0 -> 4/3 [ENABLED,IMMUTABLE]
# ./v4l2-ctl -d /dev/v4l-subdev51 --get-routing 
0/0 -> 1/0 [ENABLED,IMMUTABLE]
0/1 -> 2/0 [ENABLED,IMMUTABLE]
0/2 -> 3/0 [ENABLED,IMMUTABLE]
0/3 -> 4/0 [ENABLED,IMMUTABLE]

Use v4l-utils to test

The v4l2-utils (git://linuxtv.org/v4l-utils.git) provide two tools which are helpful in testing rcar-vin.

qv4l2

A GUI tool which lets you view the video from a rcar-vin device and test a wide range of different input parameters. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.

# qv4l2 -d 23

v4l2-compliance

A compliance test tool that verifies the drivers user-space interface. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.

There are a few other options worth testing

  • On Gen2 --stream-all-formats (-f) and --streaming=<count> (-s) will test that the driver actually can grab frames in different formats.
  • On Gen2 --stream-all-formats (-f) will test that the driver actually can grab frames. The --stream-all-formats option can't be used on Gen3 since the format needs to be changed in the entire pipeline and not just at the video device.
$ v4l2-compliance -d 2 -s
v4l2-compliance SHA   : 098aa386ea6cf08cacb44e0982431c5e837e63d0

Driver Info:
	Driver name   : rcar_vin
	Card type     : R_Car_VIN
	Bus info      : platform:e6ef2000.video
	Driver version: 4.11.0
	Capabilities  : 0x85200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x05200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format

Compliance test for device /dev/video2 (not using libv4l2):

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second video open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Test input 0:

	Control ioctls:
		test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
		test VIDIOC_QUERYCTRL: OK (Not Supported)
		test VIDIOC_G/S_CTRL: OK (Not Supported)
		test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
		test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
		test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
		Standard Controls: 0 Private Controls: 0

	Format ioctls:
		test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
		test VIDIOC_G/S_PARM: OK (Not Supported)
		test VIDIOC_G_FBUF: OK (Not Supported)
		test VIDIOC_G_FMT: OK
		test VIDIOC_TRY_FMT: OK
		test VIDIOC_S_FMT: OK
		test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
		test Cropping: OK (Not Supported)
		test Composing: OK (Not Supported)
		test Scaling: OK

	Codec ioctls:
		test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
		test VIDIOC_G_ENC_INDEX: OK (Not Supported)
		test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

	Buffer ioctls:
		test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
		test VIDIOC_EXPBUF: OK

Test input 0:

Streaming ioctls:
	test read/write: OK
	test MMAP: OK
	test USERPTR: OK (Not Supported)
	test DMABUF: Cannot test, specify --expbuf-device


Total: 46, Succeeded: 46, Failed: 0, Warnings: 0

Test V4L2_FIELD_SEQ_TB and V4L2_FIELD_SEQ_BT

The hardware it self don't support capturing frames in this field mode however it is possible to do two captures to the same buffer and provide user-space with sequential frames containing both top and bottom fields.

To test this please follow the example script below which is designed to run on Renesas Salvator-X R-Car H3 using the CVBS input connected to CSI20 which in turn is routed to the VIN1 instance. Please not that in the example the media device /dev/media5 is used, this might be different on your system depending on configuration.

#!/bin/bash

# Set media device used for VIN (this might be different on your system)
mdev=/dev/media5

# Configure pipeline CVBS -> CSI20 -> VIN1
cam="'adv748x 4-0070 afe':8"
atx="'adv748x 4-0070 txb':0"
csi="'rcar_csi2 fea80000.csi2':1"
vin="VIN1 output"
media-ctl -d $mdev -r
media-ctl -d $mdev -l "$csi-> '$vin':0 [1]"

# Auto propagate format in pipeline
format=$(media-ctl -d $mdev --get-v4l2 "$cam" | sed 's|.*fmt:\([^/]*\).*|\1|')
size=$(media-ctl -d $mdev --get-v4l2 "$cam" | sed 's|.*fmt:[^/]*/\([^ ]*\).*|\1|')
field=$(media-ctl -d $mdev --get-v4l2 "$cam" | sed 's|.*field:\([^] ]*\).*|\1|')
vdev=$(media-ctl -d $mdev -e "$vin" )
media-ctl -d $mdev -V "$cam [fmt:$format/$size field:$field]"
media-ctl -d $mdev -V "$atx [fmt:$format/$size field:$field]"
media-ctl -d $mdev -V "$csi [fmt:$format/$size field:$field]"
echo "format: $format size: $size field: $field vdev: $vdev"

# Capture frames using seq-tb
yavta -f RGB565 -s $size --field seq-tb $vdev
yavta -n 4 --capture=10 --file=seq-tb-#.rgb $vdev

# Convert frames to png
for f in seq-tb-*.rgb; do
    ffmpeg -s $size -pix_fmt rgb565le -i $f -f image2 -pix_fmt rgb24 $(basename $f .rgb).png
done

Test Up Down Scaler (UDS)

Not all VINs have access to a UDS so first make sure to test with a VIN that have a UDS. In this example we will work on M3-N where VIN0, VIN1, VIN4 and VIN5 have access to a UDS. We will capture HDMI and capture using VIN0. Please note that /dev/media0 and /dev/video0 might be different on your system based on your kernel configuration.

HDMI: 'adv748x 4-0070 hdmi' -> 'rcar_csi2 feaa0000.csi2' -> 'VIN0 output'

First step is to setup the pipeline:

media-ctl -d /dev/media0 -r
media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"

Check what format is detected on the HDMI input

$ media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"
	[fmt:RGB888_1X24/640x480 field:none colorspace:srgb]

Now we know the formats so we can propagate them:

media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/640x480 field:none]"
media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/640x480 field:none]"

Last step is to grab frames requesting a format different from the one previously propagated in the pipeline (640x480). Here we double the size when capturing:

yavta -f RGB565 -s 1280x960 --field none -n 4 --capture=10 -F /dev/video0

If all went as expected the captured frames is scaled to twice the size of the original.

Test Suspend and Resume

  • Start a video capture using any method outlined in this wiki page which fits your platform.
  • Configure your system for a suitable wake-up source and suspend your system according to the Salvator-X or Salvator-XS procedure.
  • Wake up the sytem.
  • Observer the suspended video capture resuming.