[meta-freescale] IMX6 Video processing build

[meta-freescale] IMX6 Video processing build

[Terry Barnaby]

I wonder if anyone has knowledge of how to build a Yocto image with full 
iMX video processing support for an iMX6 ? I am going around in circles 
of failed Yocto builds and/or images that don't appear to support the 
full video processing abilities.

I am trying to test the video processing abilities of the iMX6, and 
later the iMX8 SOC's. The video processing pipeline needs to use the 
MIPI-CSI 1920x1080p30 camera input, the IPU for overlaying a generated 
image and sending the video stream between processing blocks to reduce 
memory access, the VPU for H264 encoding and the GPU for rendering the 
camera video stream with overlay onto a LCD and/or HDMI display with GUI.

It looks like the iMX6DL and some of the newer iMX8's like iMX8M-Plus 
and iMX8DualXPlus should be able to do most of this in hardware with 
little CPU usage and without too much memory bandwidth used. But it is 
difficult to know for sure, hence the testing before we choose a 
suitable SOC and design our hardware boards.

I am trying to use gstreamer1.0 to perform these tests.

One of our test boards is a Wandboard WB-IMX6U-BW iMX6DL. There was a 
binary Yocto 2.0 build for this that we have been using, but it is quite 
old and there appears to be video processing issues so I have been 
trying to build a more recent Yocto build using 
http://freescale.github.io 3.1 dunfell. I am still gaining knowledge of 
the iMX6 with Yocto builds but after many many builds I am still not 
getting anywhere to obtaining a build with a fully working video 
processing pipeline. The Yocto 2.0 does allow the use of the iMX IPU/VPU 
and GPU via gstreamer with commands like:

gst-launch-1.0 -v compositor name=comp ! queue ! vpuenc_h264 ! qtmux ! 
filesink location=temp.mp4 \

     overlaysink imxv4l2src device=/dev/video0 ! 
video/x-raw,width=1920,height=1080,framerate=30/1 ! c.sink_0 \

     multifilesrc location=./test1.png caps=image/png,framerate=1/1 ! 
pngdec ! imxvideoconvert_ipu ! c.sink_1

Which uses around 11% of one of the CPU's.

I have built (amongst many other tries):

BRANCH=dunfel, MACHINE=wandboard, DISTRO=fslc-x11, 
IMX_DEFAULT_BSP="mainline", kernel-fslc, bitbake core-image-sato

This builds and runs (Need to use a SPL and uboot.img built manually 
from uboot git master branch) and displays a GUI.

This does have hardware h264 encode via the gstreamer v4l2h264enc 
element, but the ov56545 camera, although found, does not work with the 
v4l2src gstreamer element producing a Buffer error messages. Also there 
are no imxcompositor_ipu, imxcompositor_g2d or other such iMX video 
elements available.

It appears that with this IMX_DEFAULT_BSP="mainline" build, 
MACHINEOVERRIDES loses the "imx6dl" and other such compatible "machines" 
so that I cannot build gstreamer1.0-plugins-imx.

I have tried:

BRANCH=dunfel, MACHINE=wandboard, DISTRO=fslc-x11, 
IMX_DEFAULT_BSP="nxp", kernel-fslc-imx, bitbake core-image-sato

This failed to build mesa. I added "libdrm" to X11_DEPS to allow this to 
build, not sure this is correct.

This builds and runs (Need to use a SPL and uboot.img built manually 
from uboot git master branch) but no GUI. There doesn't seem to be a 
/dev/fb0 device and no mention in dmesg of the HDMI interface.

I have tried many many other options including the linux-fslc kernel 
with IMX_DEFAULT_BSP="nxp", fails to build, linux-imx kernel no GUI, 
DISTRO='fslc-xwayland', bitbake core-image-weston, bitbake 
fsl-image-multimedia-full etc etc. But am getting nowhere.

1. I presume I do need gstreamer1.0-plugins-imx to support the iMX6 
IPU's image conversion/composing and internal data paths ?

2. I am unclear what the IMX_DEFAULT_BSP="nxp" actually does and why 
"imx6dl" is removed from MACHINEOVERRIDES when building with 
IMX_DEFAULT_BSP="mainline" ?

3. Any pointers on what I should be doing to get a Yocto build with full 
video processing support ?

Any ideas gratefully received !