Difference between revisions of "R-Car/Tests:rcar-vin"

From eLinux.org
Jump to: navigation, search
m
(Multi-channel CSI-2 tests)
Line 308: Line 308:
 
The cameras used in this setup are IMI RDACM20 camera modules based on an Omnivision 10635 camera sensor and a Maxim MAX9271 GMSL serializer. The GMSL link carries power, control (I2C) and video data over a single coax cable.
 
The cameras used in this setup are IMI RDACM20 camera modules based on an Omnivision 10635 camera sensor and a Maxim MAX9271 GMSL serializer. The GMSL link carries power, control (I2C) and video data over a single coax cable.
  
At this time only capture a single virtual channel is supported, connected to the first GMSL camera.
+
At this time only capture a single virtual channel at a time is supported.
  
On the hardware side, make sure to set SW29 to OFF-OFF on the Salvator-X board to select the external CSI-2 devices, to remove the J19 jumper from the MAX9286 expansion board to disable the second MAX9286, and to connect RDACM20 cameras to the first four GMSL ports (J1 to J4).
+
On the hardware side, make sure to set SW29 to OFF-OFF on the Salvator-X board to select the external CSI-2 devices, to remove the J19 jumper from the MAX9286 expansion board to disable the second MAX9286, and to connect RDACM20 cameras to the first four GMSL ports (J1 to J4) and/or the last GMSL ports (J5, J10, J12 and J13).
  
 
On the software side, the latest version of the work-in-progress kernel code is available from
 
On the software side, the latest version of the work-in-progress kernel code is available from
  
git://linuxtv.org/pinchartl/media.git vin-gmsl-20170515
+
git://git.ragnatech.se/linux vin/mux+gmsl
  
 
The kernel must be compiled with the following options.
 
The kernel must be compiled with the following options.
Line 330: Line 330:
 
Disabling FDP1, JPU and VSP1 ensure that the media controller and video device nodes will have known names. You can enable those drivers if you update the device names accordingly in the commands below.
 
Disabling FDP1, JPU and VSP1 ensure that the media controller and video device nodes will have known names. You can enable those drivers if you update the device names accordingly in the commands below.
  
 +
=== Capture from a single camera on J1 ===
 
You need recent versions of the media-ctl and yavta tools on your system. After booting the system, simply run
 
You need recent versions of the media-ctl and yavta tools on your system. After booting the system, simply run
  
Line 370: Line 371:
 
</pre>
 
</pre>
  
This will capture three images from the first camera to files frame-000007.bin, frame-000008.bin and frame-000009.bin. You can convert them to pnm with the raw2rgbpnm tool (which doesn't have to be run on the target system, you can use it in your development environment).
+
This will capture three images from the first camera (J1) to files frame-000007.bin, frame-000008.bin and frame-000009.bin.
 +
 
 +
=== Capture from two cameras using different Virtual Channels ===
 +
You need recent versions of the media-ctl and yavta tools on your system. After booting the system,
 +
simply use the following commands to fist capture from camera J5 (CSI41/VC0) and then from camera J10 (CSI41/VC1).
 +
 
 +
'''Note:''' at this time it's not possible to capture from the two different VC in parallel, the two yavta commands needs to be run in sequence.
 +
 
 +
<pre>
 +
# media-ctl -V "'rcar_csi2 feab0000.csi2':1 [fmt:UYVY8_2X8/1280x800 field:none]"
 +
# media-ctl -V "'rcar_csi2 feab0000.csi2':2 [fmt:UYVY8_2X8/1280x800 field:none]"
 +
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin4-#.bin' /dev/video4
 +
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin7-#.bin' /dev/video7
 +
</pre>
 +
 
 +
Example:
 +
 
 +
<pre>
 +
# media-ctl -V "'rcar_csi2 feab0000.csi2':1 [fmt:UYVY8_2X8/1280x800 field:none]"
 +
# media-ctl -V "'rcar_csi2 feab0000.csi2':2 [fmt:UYVY8_2X8/1280x800 field:none]"
 +
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin4-#.bin' /dev/video4
 +
Device /dev/video4 opened.
 +
Device `R_Car_VIN' on `platform:e6ef4000.video' (driver 'rcar_vin') is a video capture (without mplanes) device.
 +
Video format set: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
 +
Video format: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
 +
8 buffers requested.
 +
length: 2048000 offset: 0 timestamp type/source: mono/EoF
 +
Buffer 0/0 mapped at address 0xffffafa0c000.
 +
lengt[  956.789621] max9286 4-006c: max9286_write(0x0a, 0xf1)
 +
h: 2048000 offset: 2048000 timestamp type/source: mono/EoF
 +
Buffer 1/0 mapped at[  956.802085] max9286 4-006c: max9286_read(0x31)
 +
address 0xffffaf818000.
 +
length: 2048000 offset: 4096000 timestamp type/source: mono/EoF
 +
Buffer 2/0 mapped at address 0xffffaf624000.
 +
length: 2048000 offset: 6144000 timestamp type/source: mono/EoF
 +
Buffer 3/0 mapped at address 0xffffaf430000.
 +
length: 2048000 offset: 8192000 timestamp type/source: mono/EoF
 +
Buffer 4/0 mapped at address 0xffffaf23c000.
 +
length: 2048000 offset: 10240000 timestamp type/source: mono/EoF
 +
Buffer 5/0 mapped at address 0xffffaf048000.
 +
length: 2048000 offset: 12288000 timestamp type/source: mono/EoF
 +
Buffer 6/0 mapped at address 0xffffaee54000.
 +
length: 2048000 o[  956.858450] max9286 4-006c: max9286_read(0x31)
 +
ffset: 14336000 timestamp type/source: mono/EoF
 +
Buffer 7/0 mapped at address 0xffffaec60000.
 +
[  956.915090] max9286 4-006c: max9286_read(0x31)
 +
[  956.970377] max9286 4-006c: max9286_read(0x31)
 +
[  957.025661] max9286 4-006c: max9286_read(0x31)
 +
[  957.030940] max9286 4-006c: max9286_write(0x15, 0x9b)
 +
0 (1) [-] none 0 2048000 B 957.156820 957.156873 8.216 fps ts mono/EoF
 +
1 (2) [-] none 1 2048000 B 957.190355 957.190389 29.820 fps ts mono/EoF
 +
2 (0) [-] none 2 2048000 B 957.223891 957.223908 29.819 fps ts mono/EoF
 +
3 (3) [-] none 3 2048000 B 957.257427 957.257450 29.819 fps ts mono/EoF
 +
4 (4) [-] none 4 2048000 B 957.290964 957.290986 29.818 fps ts mono/EoF
 +
5 (5) [-] none 5 2048000 B 957.324500 957.324527 29.819 fps ts mono/EoF
 +
6 (6) [-] none 6 2048000 B 957.358037 957.358057 29.818 fps ts mono/EoF
 +
7 (7) [-] none 7 2048000 B 957.391573 957.391599 29.819 fps ts mono/EoF
 +
8 (2) [-] none 8 2048000 B 957.458652 958.522813 14.908 fps ts mono/EoF
 +
9 (1) [-] none 9 2048000 B 957.592800 959.653125 7.454 fps ts mono/EoF
 +
[  960.764509] max9286 4-006c: max9286_write(0x15, 0x13)
 +
Captured 10 frames in 2.618015 seconds (3.819688 fps, 7822720.374142 B/s).
 +
8 buffers released.
 +
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin7-#.bin' /dev/video7
 +
Device /dev/video7 opened.
 +
Device `R_Car_VIN' on `platform:e6ef7000.video' (driver 'rcar_vin') is a video capture (without mplanes) device.
 +
Video format set: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
 +
Video format: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
 +
8 buffers requested.
 +
length: 2048000 offset: 0 timestamp type/source: mono/EoF
 +
Buffer 0/0 mapped at address 0xffffba7ec000.
 +
length: 20480[  980.717553] max9286 4-006c: max9286_write(0x0a, 0xf2)
 +
00 offset: 2048000 timestamp type/source: mono/EoF
 +
Buffer 1/0 mapped at address[  980.730066] max9286 4-006c: max9286_read(0x31)
 +
0xffffba5f8000.
 +
length: 2048000 offset: 4096000 timestamp type/source: mono/EoF
 +
Buffer 2/0 mapped at address 0xffffba404000.
 +
length: 2048000 offset: 6144000 timestamp type/source: mono/EoF
 +
Buffer 3/0 mapped at address 0xffffba210000.
 +
length: 2048000 offset: 8192000 timestamp type/source: mono/EoF
 +
Buffer 4/0 mapped at address 0xffffba01c000.
 +
length: 2048000 offset: 10240000 timestamp type/source: mono/EoF
 +
Buffer 5/0 mapped at address 0xffffb9e28000.
 +
length: 2048000 offset: 12288000 timestamp type/source: mono/EoF
 +
Buffer 6/0 mapped at address 0xffffb9c34000.
 +
length: 2048000 offset: 1[  980.786173] max9286 4-006c: max9286_read(0x31)
 +
4336000 timestamp type/source: mono/EoF
 +
Buffer 7/0 mapped at address 0xffffb9a40000.
 +
[  980.842695] max9286 4-006c: max9286_read(0x31)
 +
[  980.889524] max9286 4-006c: max9286_read(0x31)
 +
[  980.944683] max9286 4-006c: max9286_read(0x31)
 +
[  980.949837] max9286 4-006c: max9286_write(0x15, 0x9b)
 +
0 (1) [-] none 0 2048000 B 981.088147 981.088163 7.466 fps ts mono/EoF
 +
1 (2) [-] none 1 2048000 B 981.121681 981.121693 29.820 fps ts mono/EoF
 +
2 (0) [-] none 2 2048000 B 981.155219 981.155228 29.817 fps ts mono/EoF
 +
3 (3) [-] none 3 2048000 B 981.188755 981.188763 29.819 fps ts mono/EoF
 +
4 (4) [-] none 4 2048000 B 981.222292 981.222301 29.818 fps ts mono/EoF
 +
5 (5) [-] none 5 2048000 B 981.255828 981.255837 29.819 fps ts mono/EoF
 +
6 (6) [-] none 6 2048000 B 981.289365 981.289373 29.818 fps ts mono/EoF
 +
7 (7) [-] none 7 2048000 B 981.322902 981.322912 29.818 fps ts mono/EoF
 +
8 (2) [-] none 8 2048000 B 981.389978 982.445497 14.908 fps ts mono/EoF
 +
9 (1) [-] none 9 2048000 B 981.524124 983.585483 7.455 fps ts mono/EoF
 +
[  984.697546] max9286 4-006c: max9286_write(0x15, 0x13)
 +
Captured 10 frames in 2.631267 seconds (3.800449 fps, 7783319.958064 B/s).
 +
8 buffers released.
 +
</pre>
 +
 
 +
This will capture three images from the fifth (J5) camera to files vin4-000007.bin, vin4-000008.bin and vin4-000009.bin and sixth (J10) camera to files vin7-000007.bin, vin7-000008.bin and vin7-000009.bin
 +
 
 +
=== Convert BIN to PNM ===
 +
You can convert the captured bin frames to pnm with the raw2rgbpnm tool (which doesn't have to be run on the target system, you can use it in your development environment).
  
 
<pre>
 
<pre>

Revision as of 16:14, 10 August 2017

This document describes how to test the VIN and CSI-2 (only on Gen3) functionality on Renesas R-Car Gen2 and Gen3.

Get the code

Ideally all the code to test VIN should be available in the mainline kernel, unfortunately this is not the case yet. Renesas driver parts not yet picked-up are available in renesas-drivers tree while none Renesas drivers (e.g. the ADV7482 driver for Gen3 testing) are only available in the out-of-tree repository bellow.

There are also some patches needed to facilitate testing that are not part of renesas-drivers such as kernel configurations and DT changes. To ease testing a tag is available where all patches are included which can be build and used to test at:

https://git.ragnatech.se/linux rcar-vin-elinux-v12

Gen2

All that is needed is included in v4.10.

Gen3

  • VIN, CSI2, and ADV748x driver support is included in renesas-drivers.
  • To test CVBS or HDMI input the RCAR_VIN, RCAR_CSI2 and ADV7482 needs to be enabled (not default in defconfig) and added to DT, see example in out-of-tree repository mentioned above.

Kernel Options

The following kernel configuration options should be added (Gen3)

CONFIG_V4L_PLATFORM_DRIVERS=y
CONFIG_VIDEO_RCAR_VIN=y
CONFIG_VIDEO_RCAR_CSI2=y
CONFIG_VIDEO_ADV748X=y

Boot the kernel

You should see this on successful probe (this is from Gen3):

rcar-vin e6ef0000.video: Device registered as video0
rcar-vin e6ef1000.video: Device registered as video1
rcar-vin e6ef2000.video: Device registered as video2
rcar-vin e6ef3000.video: Device registered as video3
rcar-vin e6ef4000.video: Device registered as video4
rcar-vin e6ef5000.video: Device registered as video5
rcar-vin e6ef6000.video: Device registered as video6
rcar-vin e6ef7000.video: Device registered as video7

Media Controller (Gen3)

The hardware available on Gen3 is much more complex then on Gen2. For this reason it's no longer enough to interact with only the video device to control the hardware, the entire media pipeline needs to configured. To facilitate this the Media Controller (MC) framework is used exclusivity on Gen3. By using the MC framework it is up to user-space to first configure the media pipeline and then setup the correct format for each entity involved in the pipeline before streaming can be started on the video device.

The pipeline will consist of a video sensor (CSI-2 Transmitter, ADV7482 on Salvator-X) a CSI-2 receiver (R-Car CSI-2) and a dma engine (R-Car VIN). The video sensor is immutably connected to a single R-Car CSI-2 while the routing between the R-Car CSI-2 and R-Car VIN can be configured at runtime. There are limitations on which channel from the R-Car CSI-2 node can be connected to which R-Car VIN node, this is a hardware limitation that is different for each SoC in the Gen3 series. All links between these different nodes and all possible routing configurations are exposed as a media device to user-space. After the links in the pipeline have been setup by the user the proper format needs to be set on all nodes involved in the pipeline before the user can start a stream at the video device.

To control the routing and format from user-space the user needs to interact with the media device, the subdevices for video sensor and R-Car CSI-2, the video node for R-Car VIN. For configuring the media device and subdevices 'media-ctl' can be used and to configure the video device itself a grabber can be used, such as 'yavta' or 'qv4l2'. The following sections will briefly describe how to use these tools to configure the pipeline for streaming.

Show current routing

Examine the current routing setup with 'media-ctl -d /dev/mediaX -p'.

$ media-ctl -d /dev/media0 -p
Media controller API version 0.1.0

Media device information
------------------------
driver          Renesas VIN
model           video
serial          
bus info        /soc/video@e6ef0000
hw revision     0x0
driver version  4.12.0

Device topology
- entity 1: rcar_vin e6ef0000.video (1 pad, 2 links)
            type Node subtype V4L flags 0
            device node name /dev/video3
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 [ENABLED]
		<- "rcar_csi2 fea80000.csi2":1 []

- entity 5: rcar_csi2 feaa0000.csi2 (5 pads, 8 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
	pad0: Sink
		[fmt:RGB888_1X24/1280x720 field:none]
		<- "adv748x 4-0070 txa":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:RGB888_1X24/1280x720 field:none]
		-> "rcar_vin e6ef0000.video":0 [ENABLED]
		-> "rcar_vin e6ef1000.video":0 []
		-> "rcar_vin e6ef2000.video":0 []
	pad2: Source
		[fmt:RGB888_1X24/1280x720 field:none]
		-> "rcar_vin e6ef1000.video":0 []
		-> "rcar_vin e6ef3000.video":0 []
	pad3: Source
		[fmt:RGB888_1X24/1280x720 field:none]
		-> "rcar_vin e6ef2000.video":0 []
	pad4: Source
		[fmt:RGB888_1X24/1280x720 field:none]
		-> "rcar_vin e6ef3000.video":0 []

- entity 11: adv748x 4-0070 txa (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
	pad0: Sink
		[fmt:RGB888_1X24/1280x720 field:none]
		<- "adv748x 4-0070 hdmi":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:RGB888_1X24/1280x720 field:none]
		-> "rcar_csi2 feaa0000.csi2":0 [ENABLED,IMMUTABLE]

- entity 14: adv748x 4-0070 hdmi (2 pads, 1 link)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev0
	pad0: Sink
		[dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive]
	pad1: Source
		[fmt:RGB888_1X24/1280x720 field:none]
		[dv.caps:BT.656/1120 min:640x480@0 max:1920x1200@162000000 stds:CEA-861,DMT caps:interlaced,progressive]
		[dv.detect:BT.656/1120 1280x720p60 (1650x750) stds:CEA-861 flags:can-reduce-fps,CE-video,has-cea861-vic]
		[dv.current:BT.656/1120 1280x720p60 (1650x750) stds:CEA-861 flags:can-reduce-fps,CE-video,has-cea861-vic]
		-> "adv748x 4-0070 txa":0 [ENABLED,IMMUTABLE]

- entity 25: rcar_csi2 fea80000.csi2 (5 pads, 15 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev5
	pad0: Sink
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		<- "adv748x 4-0070 txb":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		-> "rcar_vin e6ef0000.video":0 []
		-> "rcar_vin e6ef1000.video":0 [ENABLED]
		-> "rcar_vin e6ef2000.video":0 []
		-> "rcar_vin e6ef4000.video":0 []
		-> "rcar_vin e6ef5000.video":0 []
		-> "rcar_vin e6ef6000.video":0 []
	pad2: Source
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		-> "rcar_vin e6ef1000.video":0 []
		-> "rcar_vin e6ef3000.video":0 []
		-> "rcar_vin e6ef5000.video":0 []
		-> "rcar_vin e6ef7000.video":0 []
	pad3: Source
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		-> "rcar_vin e6ef2000.video":0 []
		-> "rcar_vin e6ef6000.video":0 []
	pad4: Source
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		-> "rcar_vin e6ef3000.video":0 []
		-> "rcar_vin e6ef7000.video":0 []

- entity 31: adv748x 4-0070 txb (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev4
	pad0: Sink
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		<- "adv748x 4-0070 afe":8 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		-> "rcar_csi2 fea80000.csi2":0 [ENABLED,IMMUTABLE]

- entity 34: adv748x 4-0070 afe (9 pads, 1 link)
             type V4L2 subdev subtype Decoder flags 0
             device node name /dev/v4l-subdev3
	pad0: Sink
	pad1: Sink
	pad2: Sink
	pad3: Sink
	pad4: Sink
	pad5: Sink
	pad6: Sink
	pad7: Sink
	pad8: Source
		[fmt:UYVY8_2X8/720x576 field:interlaced]
		-> "adv748x 4-0070 txb":0 [ENABLED,IMMUTABLE]

- entity 60: rcar_vin e6ef1000.video (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video4
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":1 [ENABLED]
		<- "rcar_csi2 fea80000.csi2":2 []

- entity 72: rcar_vin e6ef2000.video (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video5
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":1 []
		<- "rcar_csi2 feaa0000.csi2":3 []
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":3 []

- entity 84: rcar_vin e6ef3000.video (1 pad, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/video6
	pad0: Sink
		<- "rcar_csi2 feaa0000.csi2":2 []
		<- "rcar_csi2 feaa0000.csi2":4 []
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":4 []

- entity 96: rcar_vin e6ef4000.video (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video7
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":1 []

- entity 102: rcar_vin e6ef5000.video (1 pad, 2 links)
              type Node subtype V4L flags 0
              device node name /dev/video8
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":2 []

- entity 110: rcar_vin e6ef6000.video (1 pad, 2 links)
              type Node subtype V4L flags 0
              device node name /dev/video9
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":1 []
		<- "rcar_csi2 fea80000.csi2":3 []

- entity 118: rcar_vin e6ef7000.video (1 pad, 2 links)
              type Node subtype V4L flags 0
              device node name /dev/video10
	pad0: Sink
		<- "rcar_csi2 fea80000.csi2":2 []
		<- "rcar_csi2 fea80000.csi2":4 []

Activate/Deactivate a link

The Media Controller framework allows user-space to enable/disable a link and that way control the routing of video data from a R-Car CSI-2 to a R-Car VIN instance. Not all CSI-2 channels are possible to route to any and all VIN instances, this is limited by the SoC hardware and is specific for each SoC version. Look at the previous section on how to show which links are possible on your particular SoC. Further more not all links can be activated independent of other links, this is also a limitation set by the hardware. If you try to enable a link which would not be possible with regard to other already active links the operation will fail with a EMLINK error. Try deactivating some other link to crate more routing possibilities and try again, all possible routing setups for a specific SoC are documented in the datasheet.

A link is always configured from a R-Car CSI-2 entity to a R-Car VIN entity, the same way the video data is flowing.

To enable a link use the media-ctl utility from v4l-utils package:

# media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin e6ef0000.video':0 [1]"

To disable the same link use:

# media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin.e6ef0000.video':0 [0]"

Deactivate all active links

This is a useful command to reset all links before you start enabling new links to make sure you got the biggest possible routing space to start out with.

# media-ctl -d /dev/media0 -r

Example configuring the pipeline and propagate format

Once the user have configured a pipeline using 'media-ctl' as described above the format needs to be propagated in the pipeline before streaming can start. In this example we use a R-Car H3 board where the HDMI input will be configured to output on VIN2 and CVBS configured to VIN4. If we look at the media graph as show above we can identify which nodes are involved in each of these two pipelines:

HDMI: 'adv748x 4-0070 hdmi' -> 'rcar_csi2 feaa0000.csi2' -> 'rcar_vin e6ef2000.video'
CVBS: 'adv748x 4-0070 afe' -> 'rcar_csi2 fea80000.csi2' -> 'rcar_vin e6ef4000.video'

Start by clearing any previous links and configure the ones we want, remember that the link between the adv7482 and the R-Car CSI-2 entities are immutable and won't be cleared and therefore do not need to be set again.

media-ctl -d /dev/media0 -r
media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'rcar_vin e6ef2000.video':0 [1]"
media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'rcar_vin e6ef4000.video':0 [1]"

When the pipelines are configured it's time to propagate the format in the pipeline, start by examine which format is present at the video sources sinks.

# HDMI
$ media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"
	[fmt:RGB888_1X24/640x480 field:none]

# CVBS
$ media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"
	[fmt:UYVY8_2X8/720x576 field:interlaced]

Now we know the formats so we can propagate them:

# HDMI
media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/640x480 field:none]"
media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/640x480 field:none]"

# CVBS
media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x576 field:interlaced]"
media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x576 field:interlaced]"

Once the pipeline format is setup we can then grab frames using for example 'yavta' or look use a GUI like 'qv4l2'

# HDMI
yavta -f RGB565 -s 640x480 --field none -n 4 --capture=10 -F /dev/video2

# CVBS
yavta -f RGB565 -s 720x576 --field interlaced -n 4 --capture=10 -F /dev/video4

Multi-channel CSI-2 tests

Multi-channel CSI-2 testing uses an expansion board for the R-Car H3 Salvator-X based on a Maxim MAX9286 quad-channel GMSL (Gigabit Multimedia Serial Link) deserializer. Up to four cameras can be connected to the deserializer, which outputs the four video streams on separate virtual channels over one CSI-2 connections.

The cameras used in this setup are IMI RDACM20 camera modules based on an Omnivision 10635 camera sensor and a Maxim MAX9271 GMSL serializer. The GMSL link carries power, control (I2C) and video data over a single coax cable.

At this time only capture a single virtual channel at a time is supported.

On the hardware side, make sure to set SW29 to OFF-OFF on the Salvator-X board to select the external CSI-2 devices, to remove the J19 jumper from the MAX9286 expansion board to disable the second MAX9286, and to connect RDACM20 cameras to the first four GMSL ports (J1 to J4) and/or the last GMSL ports (J5, J10, J12 and J13).

On the software side, the latest version of the work-in-progress kernel code is available from

git://git.ragnatech.se/linux vin/mux+gmsl

The kernel must be compiled with the following options.

CONFIG_VIDEO_MAX9286=y
CONFIG_VIDEO_RDACM20=y
CONFIG_VIDEO_RCAR_CSI2=y
CONFIG_VIDEO_RCAR_VIN=y
# CONFIG_VIDEO_RENESAS_FDP1 is not set
# CONFIG_VIDEO_RENESAS_JPU is not set
# CONFIG_VIDEO_RENESAS_VSP1 is not set

Disabling FDP1, JPU and VSP1 ensure that the media controller and video device nodes will have known names. You can enable those drivers if you update the device names accordingly in the commands below.

Capture from a single camera on J1

You need recent versions of the media-ctl and yavta tools on your system. After booting the system, simply run

# media-ctl -V "'rcar_csi2 feaa0000.csi2':0 [fmt:UYVY8_2X8/1280x800 field:none]"
# yavta -f YUYV -s 1280x800 -c10 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' is a video capture device.
Video format set: YUYV (56595559) 1280x800 (stride 2560) buffer size 2048000
Video format: YUYV (56595559) 1280x800 (stride 2560) buffer size 2048000
8 buffers requested.
length: 2048000 offset: 0
Buffer 0 mapped at address 0x7f80c94000.
length: 2048000 offset: 2048000
Buffer 1 mapped at address 0x7f80aa0000.
length: 2048000 offset: 4096000
Buffer 2 mapped at address 0x7f808ac000.
length: 2048000 offset: 6144000
Buffer 3 mapped at address 0x7f806b8000.
length: 2048000 offset: 8192000
Buffer 4 mapped at address 0x7f804c4000.
length: 2048000 offset: 10240000
Buffer 5 mapped at address 0x7f802d0000.
length: 2048000 offset: 12288000
Buffer 6 mapped at address 0x7f800dc000.
length: 2048000 offset: 14336000
Buffer 7 mapped at address 0x7f7fee8000.
0 (0) [-] 0 2048000 bytes 182.333337 182.333348 8.889 fps
1 (1) [-] 1 2048000 bytes 182.366871 182.366878 29.820 fps
2 (2) [-] 2 2048000 bytes 182.400405 182.400412 29.820 fps
3 (3) [-] 3 2048000 bytes 182.433939 182.433946 29.820 fps
4 (4) [-] 4 2048000 bytes 182.467473 182.467480 29.820 fps
5 (5) [-] 5 2048000 bytes 182.501007 182.501014 29.820 fps
6 (6) [-] 6 2048000 bytes 182.534541 182.534548 29.820 fps
7 (7) [-] 7 2048000 bytes 182.568076 182.568082 29.820 fps
8 (0) [-] 8 2048000 bytes 182.601611 182.601622 29.820 fps
9 (1) [-] 9 2048000 bytes 182.635145 182.635157 29.820 fps
Captured 10 frames in 0.414323 seconds (24.135744 fps, 49430003.394668 B/s).
8 buffers released.

This will capture three images from the first camera (J1) to files frame-000007.bin, frame-000008.bin and frame-000009.bin.

Capture from two cameras using different Virtual Channels

You need recent versions of the media-ctl and yavta tools on your system. After booting the system, simply use the following commands to fist capture from camera J5 (CSI41/VC0) and then from camera J10 (CSI41/VC1).

Note: at this time it's not possible to capture from the two different VC in parallel, the two yavta commands needs to be run in sequence.

# media-ctl -V "'rcar_csi2 feab0000.csi2':1 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':2 [fmt:UYVY8_2X8/1280x800 field:none]"
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin4-#.bin' /dev/video4
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin7-#.bin' /dev/video7

Example:

# media-ctl -V "'rcar_csi2 feab0000.csi2':1 [fmt:UYVY8_2X8/1280x800 field:none]"
# media-ctl -V "'rcar_csi2 feab0000.csi2':2 [fmt:UYVY8_2X8/1280x800 field:none]"
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin4-#.bin' /dev/video4
Device /dev/video4 opened.
Device `R_Car_VIN' on `platform:e6ef4000.video' (driver 'rcar_vin') is a video capture (without mplanes) device.
Video format set: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
Video format: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
8 buffers requested.
length: 2048000 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffffafa0c000.
lengt[  956.789621] max9286 4-006c: max9286_write(0x0a, 0xf1)
h: 2048000 offset: 2048000 timestamp type/source: mono/EoF
Buffer 1/0 mapped at[  956.802085] max9286 4-006c: max9286_read(0x31)
 address 0xffffaf818000.
length: 2048000 offset: 4096000 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffffaf624000.
length: 2048000 offset: 6144000 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffffaf430000.
length: 2048000 offset: 8192000 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffffaf23c000.
length: 2048000 offset: 10240000 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffffaf048000.
length: 2048000 offset: 12288000 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffffaee54000.
length: 2048000 o[  956.858450] max9286 4-006c: max9286_read(0x31)
ffset: 14336000 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffffaec60000.
[  956.915090] max9286 4-006c: max9286_read(0x31)
[  956.970377] max9286 4-006c: max9286_read(0x31)
[  957.025661] max9286 4-006c: max9286_read(0x31)
[  957.030940] max9286 4-006c: max9286_write(0x15, 0x9b)
0 (1) [-] none 0 2048000 B 957.156820 957.156873 8.216 fps ts mono/EoF
1 (2) [-] none 1 2048000 B 957.190355 957.190389 29.820 fps ts mono/EoF
2 (0) [-] none 2 2048000 B 957.223891 957.223908 29.819 fps ts mono/EoF
3 (3) [-] none 3 2048000 B 957.257427 957.257450 29.819 fps ts mono/EoF
4 (4) [-] none 4 2048000 B 957.290964 957.290986 29.818 fps ts mono/EoF
5 (5) [-] none 5 2048000 B 957.324500 957.324527 29.819 fps ts mono/EoF
6 (6) [-] none 6 2048000 B 957.358037 957.358057 29.818 fps ts mono/EoF
7 (7) [-] none 7 2048000 B 957.391573 957.391599 29.819 fps ts mono/EoF
8 (2) [-] none 8 2048000 B 957.458652 958.522813 14.908 fps ts mono/EoF
9 (1) [-] none 9 2048000 B 957.592800 959.653125 7.454 fps ts mono/EoF
[  960.764509] max9286 4-006c: max9286_write(0x15, 0x13)
Captured 10 frames in 2.618015 seconds (3.819688 fps, 7822720.374142 B/s).
8 buffers released.
# yavta -f YUYV -s 1280x800 -c10 --skip 7 --file='vin7-#.bin' /dev/video7
Device /dev/video7 opened.
Device `R_Car_VIN' on `platform:e6ef7000.video' (driver 'rcar_vin') is a video capture (without mplanes) device.
Video format set: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
Video format: YUYV (56595559) 1280x800 (stride 2560) field none buffer size 2048000
8 buffers requested.
length: 2048000 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffffba7ec000.
length: 20480[  980.717553] max9286 4-006c: max9286_write(0x0a, 0xf2)
00 offset: 2048000 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address[  980.730066] max9286 4-006c: max9286_read(0x31)
 0xffffba5f8000.
length: 2048000 offset: 4096000 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffffba404000.
length: 2048000 offset: 6144000 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffffba210000.
length: 2048000 offset: 8192000 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffffba01c000.
length: 2048000 offset: 10240000 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffffb9e28000.
length: 2048000 offset: 12288000 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffffb9c34000.
length: 2048000 offset: 1[  980.786173] max9286 4-006c: max9286_read(0x31)
4336000 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffffb9a40000.
[  980.842695] max9286 4-006c: max9286_read(0x31)
[  980.889524] max9286 4-006c: max9286_read(0x31)
[  980.944683] max9286 4-006c: max9286_read(0x31)
[  980.949837] max9286 4-006c: max9286_write(0x15, 0x9b)
0 (1) [-] none 0 2048000 B 981.088147 981.088163 7.466 fps ts mono/EoF
1 (2) [-] none 1 2048000 B 981.121681 981.121693 29.820 fps ts mono/EoF
2 (0) [-] none 2 2048000 B 981.155219 981.155228 29.817 fps ts mono/EoF
3 (3) [-] none 3 2048000 B 981.188755 981.188763 29.819 fps ts mono/EoF
4 (4) [-] none 4 2048000 B 981.222292 981.222301 29.818 fps ts mono/EoF
5 (5) [-] none 5 2048000 B 981.255828 981.255837 29.819 fps ts mono/EoF
6 (6) [-] none 6 2048000 B 981.289365 981.289373 29.818 fps ts mono/EoF
7 (7) [-] none 7 2048000 B 981.322902 981.322912 29.818 fps ts mono/EoF
8 (2) [-] none 8 2048000 B 981.389978 982.445497 14.908 fps ts mono/EoF
9 (1) [-] none 9 2048000 B 981.524124 983.585483 7.455 fps ts mono/EoF
[  984.697546] max9286 4-006c: max9286_write(0x15, 0x13)
Captured 10 frames in 2.631267 seconds (3.800449 fps, 7783319.958064 B/s).
8 buffers released.

This will capture three images from the fifth (J5) camera to files vin4-000007.bin, vin4-000008.bin and vin4-000009.bin and sixth (J10) camera to files vin7-000007.bin, vin7-000008.bin and vin7-000009.bin

Convert BIN to PNM

You can convert the captured bin frames to pnm with the raw2rgbpnm tool (which doesn't have to be run on the target system, you can use it in your development environment).

$ raw2rgbpnm -f YUYV -s 1280x800 frame-000007.bin frame-000007.pnm
0 padding bytes detected at end of line
Image size: 1280x800, bytes per pixel: 16, format: YVYU (16  YUV 4:2:2)
Writing to file `frame.pnm'...

Use v4l-utils to test

The v4l2-utils (git://linuxtv.org/v4l-utils.git) provide two tools which are helpful in testing rcar-vin.

qv4l2

A GUI tool which lets you view the video from a rcar-vin device and test a wide range of different input parameters. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.

# qv4l2 -d 23

v4l2-compliance

A compliance test tool that verifies the drivers user-space interface. Start the tool and specifying which video device you wish to operate on using the -d parameter, the video device number for the different video nodes can be observed in the boot log as shown above.

There are a few other options worth testing

  • On Gen2 --stream-all-formats (-f) and --streaming=<count> (-s) will test that the driver actually can grab frames in different formats.
  • On Gen2 --stream-all-formats (-f) will test that the driver actually can grab frames. The --stream-all-formats option can't be used on Gen3 since the format needs to be changed in the entire pipeline and not just at the video device.
$ v4l2-compliance -d 2 -s
v4l2-compliance SHA   : 098aa386ea6cf08cacb44e0982431c5e837e63d0

Driver Info:
	Driver name   : rcar_vin
	Card type     : R_Car_VIN
	Bus info      : platform:e6ef2000.video
	Driver version: 4.11.0
	Capabilities  : 0x85200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x05200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format

Compliance test for device /dev/video2 (not using libv4l2):

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second video open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Test input 0:

	Control ioctls:
		test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
		test VIDIOC_QUERYCTRL: OK (Not Supported)
		test VIDIOC_G/S_CTRL: OK (Not Supported)
		test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
		test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
		test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
		Standard Controls: 0 Private Controls: 0

	Format ioctls:
		test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
		test VIDIOC_G/S_PARM: OK (Not Supported)
		test VIDIOC_G_FBUF: OK (Not Supported)
		test VIDIOC_G_FMT: OK
		test VIDIOC_TRY_FMT: OK
		test VIDIOC_S_FMT: OK
		test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
		test Cropping: OK (Not Supported)
		test Composing: OK (Not Supported)
		test Scaling: OK

	Codec ioctls:
		test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
		test VIDIOC_G_ENC_INDEX: OK (Not Supported)
		test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

	Buffer ioctls:
		test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
		test VIDIOC_EXPBUF: OK

Test input 0:

Streaming ioctls:
	test read/write: OK
	test MMAP: OK
	test USERPTR: OK (Not Supported)
	test DMABUF: Cannot test, specify --expbuf-device


Total: 46, Succeeded: 46, Failed: 0, Warnings: 0

Test V4L2_FIELD_SEQ_TB and V4L2_FIELD_SEQ_BT

The hardware it self don't support capturing frames in this field mode however it is possible to do two captures to the same buffer and provide user-space with sequential frames containing both top and bottom fields.

To test this please follow the example script below which is designed to run on Renesas Salvator-X R-Car H3 using the CVBS input connected to CSI20 which in turn is routed to the VIN1 instance. Please not that in the example the media device /dev/media5 is used, this might be different on your system depending on configuration.

#!/bin/bash

# Set media device used for VIN (this might be different on your system)
mdev=/dev/media5

# Configure pipeline CVBS -> CSI20 -> VIN1
cam="'adv748x 4-0070 afe':8"
atx="'adv748x 4-0070 txb':0"
csi="'rcar_csi2 fea80000.csi2':1"
vin="rcar_vin e6ef1000.video"
media-ctl -d $mdev -r
media-ctl -d $mdev -l "$csi-> '$vin':0 [1]"

# Auto propagate format in pipeline
format=$(media-ctl -d $mdev --get-v4l2 "$cam" | sed 's|.*fmt:\([^/]*\).*|\1|')
size=$(media-ctl -d $mdev --get-v4l2 "$cam" | sed 's|.*fmt:[^/]*/\([^ ]*\).*|\1|')
field=$(media-ctl -d $mdev --get-v4l2 "$cam" | sed 's|.*field:\([^] ]*\).*|\1|')
vdev=$(media-ctl -d $mdev -e "$vin" )
media-ctl -d $mdev -V "$cam [fmt:$format/$size field:$field]"
media-ctl -d $mdev -V "$atx [fmt:$format/$size field:$field]"
media-ctl -d $mdev -V "$csi [fmt:$format/$size field:$field]"
echo "format: $format size: $size field: $field vdev: $vdev"

# Capture frames using seq-tb
yavta -f RGB565 -s $size --field seq-tb $vdev
yavta -n 4 --capture=10 --file=seq-tb-#.rgb $vdev

# Convert frames to png
for f in seq-tb-*.rgb; do
    ffmpeg -s $size -pix_fmt rgb565le -i $f -f image2 -pix_fmt rgb24 $(basename $f .rgb).png
done