V4L2 + V4l-utils, Capture for multi-planar #1

      Comments Off on V4L2 + V4l-utils, Capture for multi-planar #1

It was very hard to find out how it works until I made it by myself in multiple times. As we maybe already knows – Alaways comes mind peace when I have the answer, right – It was maybe very easy but very hard to know, not kindly explained anywhere to get any example about multi-planar camera models.

Here I write some ideas let to know for everyone as like searching knowledge about capturing any image from multi-planar model on Embedded Linux.


My testing environments based on Rockchip RK3399 ARM64 processors with my own build Linaro debian linux AARCH64. You can search how to build your own 64bit linux for your RK3399. And my hardware bundle Rock960B was belong to 96Board – 4GB lpDDR3 and 32GB eMMC 5.1 bundle.

And tested camera was “AR0521” greyscaled GRBG bayer array formatted sensor, it means my tested module may not formal one as like easy to buy anywhere on the internet – some hardware manufacturer supplies AR0521 but almost color formatted.

And you can understand how MIPI CSI2 camera works on your RK3399. It is not kind of USB camera.

About Multi-planar

 Some devices require data for each input or output video frame to be placed in discontiguous memory buffers. In such cases, one video frame has to be addressed using more than one memory address, i.e. one pointer per “plane”. A plane is a sub-buffer of the current frame. For examples of such formats see Image Formats. Initially, V4L2 API did not support multi-planar buffers and a set of extensions has been introduced to handle them. Those extensions constitute what is being referred to as the “multi-planar API”.

  Legacy 4L2 API calls and structures are interpreted differently, depending on whether single- or multi-planar API is being used. An application can choose whether to use one or the other by passing a corresponding buffer type to its ioctl calls. Multi-planar versions of buffer types are suffixed with an _MPLANE string.

OnSemi AR0521

The AR0521 is a 1/2.5−inch CMOS digital image sensor with an active−pixel array of 2592 (H) x 1944 (V). It captures images in either linear or high dynamic range modes with a rolling−shutter readout, and includes sophisticated camera functions such as binning, windowing and both video and single frame modes. It is designed for both low light and high dynamic range performance, with line interleaved T1/T2 read out to support off chip HDR in an ISP chip. The AR0521 produces extraordinarily clear, sharp digital pictures, and its ability to capture both continuous video and single frames makes it the perfect choice for a security applications.
You can refer to more detailed information : https://www.onsemi.com/PowerSolutions/product.do?id=AR0521

96Board Rock960B

Rock960b is a higher performanced model form Rock960a. Size almost same as like Raspberry Pi3B but got more deeply better performance than Raspberry, and easy to use for AARCH64 with 4GB memory with built-in eMMC 5.1. You can get more information here: https://www.96rocks.com/.

Embedded Linux

My embedded linux was belong to 96Board release, you can refer to build or download my private built system image here: http://rageworx.info/?s=rock960&submit=Search

Get the gears

Before you start in developing software works, you need your gears as like GCC. Now update your system and upgrade for latest

Then, now get some package for build your source codes and V4L-utils, cmake and camke-curse-gui are option for build v4-utils.

And you may need get “git”.

Get V4L-Utils

 v4l-utils availed to get thru apt-get but it’s not contains all headers. Recommend to clone a git repository from Linux-TV on your source code collection directory.

Being started with one header

 v4l-utils provides their own wrapped headers, and one header “v4l-helpers.h” makes contol v4l device easily. Just include this one line at your starting C/C++ code.

 And you need to make your Makefile can refer to previous clonned path of v4l-utils as like this:

Open camera

Just open camera with this code.

v4l_fd is a structure of v4l-helpers, and you need initialize with v4l_fd_init(); instead = {NULL}; initializing. And just open with v4l_open(); returns real file descriptor as like used open();.

Check camera capture type and format.

You can check what capture type is selected for this camera, is this camera using multi-plane or not.

ar0521 need to be configured as V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE because it designed to Multi-Planar. Better to stop code here if it recognized with MPLANE flag.

Now time to get resolutions of camera.

pixelfmt (format) is one of these:

  • [ 0] format = ”YUYV”, description = ”YUYV 4:2:2”
  • [ 1] format = ”YVYU”, description = ”YVYU 4:2:2”
  • [ 2] format = ”VYUY”, description = ”VYUY 4:2:2”
  • [ 3] format = ”422P”, description = ”Planar YVU 4:2:2”
  • [ 4] format = ”NV16”, description = ”Y/CbCr 4:2:2”
  • [ 5] format = ”NV61”, description = ”Y/CrCb 4:2:2”
  • [ 6] format = ”YM61”, description = ”Planar YVU 4:2:2 (N-C)”
  • [ 7] format = ”NV21”, description = ”Y/CrCb 4:2:0”
  • [ 8] format = ”NV12”, description = ”Y/CbCr 4:2:0”
  • [ 9] format = ”NM21”, description = ”Y/CrCb 4:2:0 (N-C)”
  • [10] format = ”NM12”, description = ”Y/CbCr 4:2:0 (N-C)”
  • [11] format = ”YU12”, description = ”Planar YUV 4:2:0”
  • [12] format = ”YV12”, description = ”Planar YVU 4:2:0”
  • [13] format = ”YM24”, description = ”Planar YUV 4:4:4 (N-C)”
  • [14] format = ”GREY”, description = ”8-bit Greyscale”
  • [15] format = ”RGGB”, description = ”8-bit Bayer RGRG/GBGB”
  • [16] format = ”GRBG”, description = ”8-bit Bayer GRGR/BGBG”
  • [17] format = ”GBRG”, description = ”8-bit Bayer GBGB/RGRG”
  • [18] format = ”BA81”, description = ”8-bit Bayer BGBG/GRGR”
  • [19] format = ”RGGB”, description = ”8-bit Bayer RGRG/GBGB”
  • [20] format = ”BA10”, description = ”10-bit Bayer GRGR/BGBG”
  • [21] format = ”GB10”, description = ”10-bit Bayer GBGB/RGRG”
  • [22] format = ”BG10”, description = ”10-bit Bayer BGBG/GRGR”
  • [23] format = ”RG12”, description = ”12-bit Bayer RGRG/GBGB”
  • [24] format = ”BA12”, description = ”12-bit Bayer GRGR/BGBG”
  • [25] format = ”GB12”, description = ”12-bit Bayer GBGB/RGRG”
  • [26] format = ”BG12”, description = ”12-bit Bayer BGBG/GRGR”

And it can recognized as string with this code:

.. this stories will be continued ..