V4L2 + V4l-utils, Capture for multi-planar #1

      Comments Off on V4L2 + V4l-utils, Capture for multi-planar #1

It was very hard to find out how it works until I made it by myself in multiple times. As we maybe already knows – Always comes mind peace when I have the answer, right – It was maybe very easy but very hard to know, not kindly explained anywhere to get any example about multi-planar camera models.

Here I write some ideas let to know for everyone as like searching knowledge about capturing any image from multi-planar model on Embedded Linux.


My testing environments based on Rockchip RK3399 ARM64 processors with my own build Linaro Debian Linux AARCH64. You can search how to build your own 64bit Linux for your RK3399. And my hardware bundle Rock960B was belonged to 96Board – 4 GB lpDDR3 and 32 GB eMMC 5.1 bundle.

And tested camera was “AR0521” grayscale GRBG Bayer array formatted sensor, it means my tested module may not formal one as like easy to buy anywhere on the internet – some hardware manufacturer supplies AR0521 but almost color formatted.

And, you can understand how MIPI CSI2 camera works on your RK3399. It is not a kind of USB camera.

About Multi-planar

 Some devices require data for each input or output video frame to be placed in discontiguous memory buffers. In such cases, one video frame has to be addressed using more than one memory address, i.e. one pointer per “plane”. A plane is a sub-buffer of the current frame. For examples of such formats see Image Formats. Initially, V4L2 API did not support multi-planar buffers and a set of extensions has been introduced to handle them. Those extensions constitute what is being referred to as the “multi-planar API”.

  Legacy 4L2 API calls and structures are interpreted differently, depending on whether single- or multi-planar API is being used. An application can choose whether to use one or the other by passing a corresponding buffer type to its ioctl calls. Multi-planar versions of buffer types are suffixed with an _MPLANE string.

OnSemi AR0521

The AR0521 is a 1/2.5−inch CMOS digital image sensor with an active−pixel array of 2592 (H) x 1944 (V). It captures images in either linear or high dynamic range modes with a rolling−shutter readout, and includes sophisticated camera functions such as binning, windowing and both video and single frame modes. It is designed for both low light and high dynamic range performance, with line interleaved T1/T2 read out to support off chip HDR in an ISP chip. The AR0521 produces extraordinarily clear, sharp digital pictures, and its ability to capture both continuous video and single frames makes it the perfect choice for a security application.
You can refer to more detailed information : https://www.onsemi.com/PowerSolutions/product.do?id=AR0521

96Board Rock960B

Rock960b is a higher performance model from Rock960a. Size almost same as like Raspberry Pi3B but got more deeply better performance than Raspberry, and easy to use for AARCH64 with 4 GB memory with built-in eMMC 5.1. You can get more information here: https://www.96rocks.com/.

Embedded Linux

My embedded linux was belonged to 96Board release, you can refer to build or download my private built system image here: http://rageworx.info/?s=rock960&submit=Search


Before you start in developing software works, you need your gears as like GCC. Now update your system and upgrade for latest,

$ sudo apt-get update
$ sudo apt-get upgrade

Then, now get some package for building your source codes and V4L-utils, cmake and camke-curse-gui are option for build v4-utils.

$ sudo apt-get install g++
$ sudo apt-get install make automake 
$ sudo apt-get install cmake cmake-curse-gui

And you may need to get “git”.

$ sudo apt-get install git

Get V4L-Utils

 v4l-utils availed to get through apt-get, but it’s not containing all headers. Recommend cloning a git repository from Linux-TV on your source code collection directory.

$ git clone git://linuxtv.org/v4l-utils.git

Being started with one header

 v4l-utils provides their own wrapped headers, and one header “v4l-helpers.h” makes control v4l device easily. Just include this one line at your starting C/C++ code.

#include "v4l-helpers.h"

 And you need to make your Makefile can refer to previous cloned path of v4l-utils as like this:

# V4LPATH is your clonned path of v4l-utils
V4LPATH = ../../v4l-utils
V4LINCS = ${V4LPATH}/include
V4LUTILS = ${V4LPATH}/utils


Open camera

Just open camera with this code.

v4l_fd cam1;
char cam_devpath[] = "/dev/video0";

v4l_fd_init( &cam1 );
int real_fd = v4l_open( &cam1, cam_devpath, true );
if ( real_fd < 0 )
    printf( "Your %s cannot be open.\n", cam_devpath );

v4l_fd is a structure of v4l-helpers, and you require to initialize with v4l_fd_init(); instead = {NULL}; initializing. And just open with v4l_open(); returns real file descriptor as like used open();.

Check camera capture type and format.

You can check what capture type is selected for this camera, is this camera using multi-plane or not.

struct v4l2_format vfmt = {0};
unsigned captype = v4l_determine_type( &cam1 );

if ( ( captpye & V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE ) == 0 )
    printf( "WARNING: this camera not supports Multiplane.\n" );

ar0521 need to be configured as V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE because it designed to Multi-Planar. Better to stop code here if it recognized with MPLANE flag.

Now time to get resolutions of camera.

unsigned pixelfmt = v4l_format_g_pixelformat( &vfmt );
unsigned planes   = v4l_format_g_num_plaens( &vfmt );
unsigned i_width  = v4l_format_g_width( &vfmt );
unsigned i_height = v4l_format_g_height( &vfmt );

pixelfmt (format) is one of these:

  • [ 0] format = ”YUYV”, description = ”YUYV 4:2:2”
  • [ 1] format = ”YVYU”, description = ”YVYU 4:2:2”
  • [ 2] format = ”VYUY”, description = ”VYUY 4:2:2”
  • [ 3] format = ”422P”, description = ”Planar YVU 4:2:2”
  • [ 4] format = ”NV16”, description = ”Y/CbCr 4:2:2”
  • [ 5] format = ”NV61”, description = ”Y/CrCb 4:2:2”
  • [ 6] format = ”YM61”, description = ”Planar YVU 4:2:2 (N-C)”
  • [ 7] format = ”NV21”, description = ”Y/CrCb 4:2:0”
  • [ 8] format = ”NV12”, description = ”Y/CbCr 4:2:0”
  • [ 9] format = ”NM21”, description = ”Y/CrCb 4:2:0 (N-C)”
  • [10] format = ”NM12”, description = ”Y/CbCr 4:2:0 (N-C)”
  • [11] format = ”YU12”, description = ”Planar YUV 4:2:0”
  • [12] format = ”YV12”, description = ”Planar YVU 4:2:0”
  • [13] format = ”YM24”, description = ”Planar YUV 4:4:4 (N-C)”
  • [14] format = ”GREY”, description = ”8-bit Greyscale”
  • [15] format = ”RGGB”, description = ”8-bit Bayer RGRG/GBGB”
  • [16] format = ”GRBG”, description = ”8-bit Bayer GRGR/BGBG”
  • [17] format = ”GBRG”, description = ”8-bit Bayer GBGB/RGRG”
  • [18] format = ”BA81”, description = ”8-bit Bayer BGBG/GRGR”
  • [19] format = ”RGGB”, description = ”8-bit Bayer RGRG/GBGB”
  • [20] format = ”BA10”, description = ”10-bit Bayer GRGR/BGBG”
  • [21] format = ”GB10”, description = ”10-bit Bayer GBGB/RGRG”
  • [22] format = ”BG10”, description = ”10-bit Bayer BGBG/GRGR”
  • [23] format = ”RG12”, description = ”12-bit Bayer RGRG/GBGB”
  • [24] format = ”BA12”, description = ”12-bit Bayer GRGR/BGBG”
  • [25] format = ”GB12”, description = ”12-bit Bayer GBGB/RGRG”
  • [26] format = ”BG12”, description = ”12-bit Bayer BGBG/GRGR”

And it can recognize as string with this code:

const char* fmt2str( unsigned fmt )
    static char retstr[5] = {0};
    memset( retstr, 0, 5 );

    retstr[0] = fmt & 0xFF;
    retstr[1] = (fmt >> 8) & 0xFF;
    retstr[2] = (fmt >> 16) & 0xFF;
    retstr[3] = (fmt >> 24) & 0xFF;

    return retstr;


.. it will be continued to next post ..