Gstreamer pipeline problem

Gstreamer pipeline problem

[Chris Tapp]

I've got an application which uses playbin2 to capture video. The pipeline is of the form:

playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb, pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> ! fakesink"

I then get the "frame" property from the pipeline and use this to grab the latest frame.

This works on my development system (Ubuntu 11.10) and a Cedar Trail / Yocto system, but the pipeline fails on the Wandboard Quad. I think this is related to:

0:00:13.028151336  1349 0x4442d520 WARN           basetransform /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetransform.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform could not transform video/x-raw-yuv, width=(int)854, height=(int)480, framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false in anything we support

I added an ffmpegcolorspace element betwween the queue2 and the videoscale to get round this and the pipeline now builds, but only a few frames are captured. There are different diagnostics showing:

0:00:02.881403000  1361   0x28da60 WARN                  vpudec vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate Internal framebuffers!!!!
Message Callback : Element playbin0x250b68 changed state from READY to PAUSED.
0:00:03.237675000  1361   0x28da60 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89, 8 frames in displaying queue!!
0:00:03.242324334  1361   0x28da60 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89, 8 frames in displaying queue!!

<lots of repeats>

0:00:08.499914334  1382   0x28d860 WARN                  vpudec vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
0:00:08.500784667  1382   0x28d860 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88, 8 frames in displaying queue!!

<lots of repeats>

Message Callback : Element playbin0x250aa0 changed state from PAUSED to PLAYING.
0:00:09.253202667  1382   0x28d860 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88, 8 frames in displaying queue!!

0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed, ret 2 No such file or directory queued 0


The "Message Callback" events are my own logging to try and see what's happening in my app.

Is this something I'm doing wrong, or are these messages a real issue somewhere?

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[Chris Tapp]

On 10 Jul 2013, at 20:19, Chris Tapp wrote:

> I've got an application which uses playbin2 to capture video. The pipeline is of the form:
> 
> playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb, pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> ! fakesink"
> 
> I then get the "frame" property from the pipeline and use this to grab the latest frame.
> 
> This works on my development system (Ubuntu 11.10) and a Cedar Trail / Yocto system, but the pipeline fails on the Wandboard Quad. I think this is related to:
> 
> 0:00:13.028151336  1349 0x4442d520 WARN           basetransform /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetransform.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform could not transform video/x-raw-yuv, width=(int)854, height=(int)480, framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false in anything we support
> 
> I added an ffmpegcolorspace element betwween the queue2 and the videoscale to get round this and the pipeline now builds, but only a few frames are captured. There are different diagnostics showing:
> 
> 0:00:02.881403000  1361   0x28da60 WARN                  vpudec vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate Internal framebuffers!!!!
> Message Callback : Element playbin0x250b68 changed state from READY to PAUSED.
> 0:00:03.237675000  1361   0x28da60 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89, 8 frames in displaying queue!!
> 0:00:03.242324334  1361   0x28da60 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89, 8 frames in displaying queue!!
> 
> <lots of repeats>
> 
> 0:00:08.499914334  1382   0x28d860 WARN                  vpudec vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
> 0:00:08.500784667  1382   0x28d860 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88, 8 frames in displaying queue!!
> 
> <lots of repeats>
> 
> Message Callback : Element playbin0x250aa0 changed state from PAUSED to PLAYING.
> 0:00:09.253202667  1382   0x28d860 WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88, 8 frames in displaying queue!!
> 
> 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed, ret 2 No such file or directory queued 0
> 
> 
> The "Message Callback" events are my own logging to try and see what's happening in my app.
> 
> Is this something I'm doing wrong, or are these messages a real issue somewhere?

This is when playing a .webm. The results for an .flv are as expected.

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[Thomas Senyk]

On Wednesday, 10 July, 2013 20:53:58 Chris Tapp wrote:
> On 10 Jul 2013, at 20:19, Chris Tapp wrote:
> > I've got an application which uses playbin2 to capture video. The pipeline
> > is of the form:
> > 
> > playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb,
> > pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> !
> > fakesink"
> > 
> > I then get the "frame" property from the pipeline and use this to grab the
> > latest frame.
> > 
> > This works on my development system (Ubuntu 11.10) and a Cedar Trail /
> > Yocto system, but the pipeline fails on the Wandboard Quad. I think this
> > is related to:
> > 
> > 0:00:13.028151336  1349 0x4442d520 WARN           basetransform
> > /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linux
> > -gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetrans
> > form.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform
> > could not transform video/x-raw-yuv, width=(int)854, height=(int)480,
> > framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false
> > in anything we support
> > 
> > I added an ffmpegcolorspace element betwween the queue2 and the videoscale
> > to get round this and the pipeline now builds, but only a few frames are
> > captured. There are different diagnostics showing:
> > 
> > 0:00:02.881403000  1361   0x28da60 WARN                  vpudec
> > vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate
> > Internal framebuffers!!!! Message Callback : Element playbin0x250b68
> > changed state from READY to PAUSED. 0:00:03.237675000  1361   0x28da60
> > WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame
> > buffer message, return 0x89, 8 frames in displaying queue!!
> > 0:00:03.242324334  1361   0x28da60 WARN                  vpudec
> > vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89,
> > 8 frames in displaying queue!!
> > 
> > <lots of repeats>
> > 
> > 0:00:08.499914334  1382   0x28d860 WARN                  vpudec
> > vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
> > 0:00:08.500784667  1382   0x28d860 WARN                  vpudec
> > vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88,
> > 8 frames in displaying queue!!
> > 
> > <lots of repeats>
> > 
> > Message Callback : Element playbin0x250aa0 changed state from PAUSED to
> > PLAYING. 0:00:09.253202667  1382   0x28d860 WARN                  vpudec
> > vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88,
> > 8 frames in displaying queue!!
> > 
> > 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink
> > mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed,
> > ret 2 No such file or directory queued 0
> > 
> > 
> > The "Message Callback" events are my own logging to try and see what's
> > happening in my app.
> > 
> > Is this something I'm doing wrong, or are these messages a real issue
> > somewhere?
> This is when playing a .webm. The results for an .flv are as expected.

is it the same when you use v4l2 sink instead of fakesink?
Is playbin (without defining the pipeline) behaving the same?

> 
> Chris Tapp
> 
> opensource@keylevel.com
> www.keylevel.com
> 
> 
> 
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale

Re: Gstreamer pipeline problem

[Chris Tapp]

On 11 Jul 2013, at 09:39, Thomas Senyk wrote:

> On Wednesday, 10 July, 2013 20:53:58 Chris Tapp wrote:
>> On 10 Jul 2013, at 20:19, Chris Tapp wrote:
>>> I've got an application which uses playbin2 to capture video. The pipeline
>>> is of the form:
>>> 
>>> playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb,
>>> pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> !
>>> fakesink"
>>> 
>>> I then get the "frame" property from the pipeline and use this to grab the
>>> latest frame.
>>> 
>>> This works on my development system (Ubuntu 11.10) and a Cedar Trail /
>>> Yocto system, but the pipeline fails on the Wandboard Quad. I think this
>>> is related to:
>>> 
>>> 0:00:13.028151336  1349 0x4442d520 WARN           basetransform
>>> /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linux
>>> -gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetrans
>>> form.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform
>>> could not transform video/x-raw-yuv, width=(int)854, height=(int)480,
>>> framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false
>>> in anything we support
>>> 
>>> I added an ffmpegcolorspace element betwween the queue2 and the videoscale
>>> to get round this and the pipeline now builds, but only a few frames are
>>> captured. There are different diagnostics showing:
>>> 
>>> 0:00:02.881403000  1361   0x28da60 WARN                  vpudec
>>> vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate
>>> Internal framebuffers!!!! Message Callback : Element playbin0x250b68
>>> changed state from READY to PAUSED. 0:00:03.237675000  1361   0x28da60
>>> WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame
>>> buffer message, return 0x89, 8 frames in displaying queue!!
>>> 0:00:03.242324334  1361   0x28da60 WARN                  vpudec
>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89,
>>> 8 frames in displaying queue!!
>>> 
>>> <lots of repeats>
>>> 
>>> 0:00:08.499914334  1382   0x28d860 WARN                  vpudec
>>> vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
>>> 0:00:08.500784667  1382   0x28d860 WARN                  vpudec
>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88,
>>> 8 frames in displaying queue!!
>>> 
>>> <lots of repeats>
>>> 
>>> Message Callback : Element playbin0x250aa0 changed state from PAUSED to
>>> PLAYING. 0:00:09.253202667  1382   0x28d860 WARN                  vpudec
>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88,
>>> 8 frames in displaying queue!!
>>> 
>>> 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink
>>> mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed,
>>> ret 2 No such file or directory queued 0
>>> 
>>> 
>>> The "Message Callback" events are my own logging to try and see what's
>>> happening in my app.
>>> 
>>> Is this something I'm doing wrong, or are these messages a real issue
>>> somewhere?
>> This is when playing a .webm. The results for an .flv are as expected.
> 
> is it the same when you use v4l2 sink instead of fakesink?
> Is playbin (without defining the pipeline) behaving the same?

I've not got that in the build at the moment - I'll add it in and give it a try.

I was going to try v4l2 out on my development system as I've not used it before, but I can't even get:

gst-launch videotestsrc ! v4l2sink

to work as there's no /dev/video1 ! Does anyone know where I can find instructions on how to get v4l2 working under Ubuntu 12.04 for a radeon HD 5450 card? Google has not been my friend ;-)

>> 
>> Chris Tapp
>> 
>> opensource@keylevel.com
>> www.keylevel.com
>> 
>> 
>> 
>> _______________________________________________
>> meta-freescale mailing list
>> meta-freescale@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-freescale

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[John Weber]

Hi Chris,

> gst-launch videotestsrc ! v4l2sink
>
> to work as there's no /dev/video1 ! Does anyone know where I can find instructions on how to get v4l2 working under Ubuntu 12.04 for a radeon HD 5450 card? Google has not been my friend ;-)
>

/dev/video1 is created when you have an actual video input like a video decoder 
or camera or UVC camera and it would be used with v4l2src (or in the case of 
i.MX6 CSI, mfw_v4lsrc).

videotestsrc should not need a video input device to work though as it is 
creates its own video data.  What is the error you are seeing?

What is your setup?  Board? Yocto version?  Yocto image?

John

Re: Gstreamer pipeline problem

[Chris Tapp]

Hi John,

On 12 Jul 2013, at 01:22, John Weber wrote:

> Hi Chris,
> 
>> gst-launch videotestsrc ! v4l2sink
>> 
>> to work as there's no /dev/video1 ! Does anyone know where I can find instructions on how to get v4l2 working under Ubuntu 12.04 for a radeon HD 5450 card? Google has not been my friend ;-)
>> 
> 
> /dev/video1 is created when you have an actual video input like a video decoder or camera or UVC camera and it would be used with v4l2src (or in the case of i.MX6 CSI, mfw_v4lsrc).
> 
> videotestsrc should not need a video input device to work though as it is creates its own video data.  What is the error you are seeing?
> 
> What is your setup?  Board? Yocto version?  Yocto image?

Sorry, I don't make myself clear here. I wanted to try v4l on my Ubuntu development system first - it's this that isn't working:

gst-launch-0.10 videotestsrc ! v4l2sink
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0: Cannot identify device "/dev/video1".
Additional debug info:
v4l2_calls.c(493): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0:
system error: No such file or directory
Setting pipeline to NULL ...
Freeing pipeline ...



> John
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[Philip Craig]

On Fri, Jul 12, 2013 at 5:50 PM, Chris Tapp <opensource@keylevel.com> wrote:
> Hi John,
>
> On 12 Jul 2013, at 01:22, John Weber wrote:
>
>> Hi Chris,
>>
>>> gst-launch videotestsrc ! v4l2sink
>>>
>>> to work as there's no /dev/video1 ! Does anyone know where I can find instructions on how to get v4l2 working under Ubuntu 12.04 for a radeon HD 5450 card? Google has not been my friend ;-)
>>>
>>
>> /dev/video1 is created when you have an actual video input like a video decoder or camera or UVC camera and it would be used with v4l2src (or in the case of i.MX6 CSI, mfw_v4lsrc).
>>
>> videotestsrc should not need a video input device to work though as it is creates its own video data.  What is the error you are seeing?
>>
>> What is your setup?  Board? Yocto version?  Yocto image?
>
> Sorry, I don't make myself clear here. I wanted to try v4l on my Ubuntu development system first - it's this that isn't working:

If you don't already have a v4l2 output device on your dev system,
then you're unlikely to be able to use v4l2sink. I had a quick look
around for virtual v4l2 output devices, but didn't see any (I only
found virtual input devices). I'm not sure that testing v4l2sink on
your dev system is going to tell you much though. It would be roughly
equivalent to testing with xvimagesink or autovideosink.

Re: Gstreamer pipeline problem

[Chris Tapp]

On 12 Jul 2013, at 09:03, Philip Craig wrote:

> On Fri, Jul 12, 2013 at 5:50 PM, Chris Tapp <opensource@keylevel.com> wrote:
>> Hi John,
>> 
>> On 12 Jul 2013, at 01:22, John Weber wrote:
>> 
>>> Hi Chris,
>>> 
>>>> gst-launch videotestsrc ! v4l2sink
>>>> 
>>>> to work as there's no /dev/video1 ! Does anyone know where I can find instructions on how to get v4l2 working under Ubuntu 12.04 for a radeon HD 5450 card? Google has not been my friend ;-)
>>>> 
>>> 
>>> /dev/video1 is created when you have an actual video input like a video decoder or camera or UVC camera and it would be used with v4l2src (or in the case of i.MX6 CSI, mfw_v4lsrc).
>>> 
>>> videotestsrc should not need a video input device to work though as it is creates its own video data.  What is the error you are seeing?
>>> 
>>> What is your setup?  Board? Yocto version?  Yocto image?
>> 
>> Sorry, I don't make myself clear here. I wanted to try v4l on my Ubuntu development system first - it's this that isn't working:
> 
> If you don't already have a v4l2 output device on your dev system,
> then you're unlikely to be able to use v4l2sink. I had a quick look
> around for virtual v4l2 output devices, but didn't see any (I only
> found virtual input devices). I'm not sure that testing v4l2sink on
> your dev system is going to tell you much though. It would be roughly
> equivalent to testing with xvimagesink or autovideosink.

Thanks for the feedback.

That's unfortunate as I need a pipeline which can run on many different platforms, including 'standard pc'. This was one reason why I was using fakesink to capture the images - v4l looked like a way to go so that I could stream the video direct to the screen and get the full benefits of any acceleration supported by a platform (zero copy, etc.).

I've not found a lot of helpful information on the internet about v4l2 (less about output devices), so I've not even been able to identify one I can put in my system.

I'll have to have a look at using gst-plugins-gl to see if I can use the texture upload aspects (there was a thread on this recently I need to look at in more detail).

Mean time I'll also try v4l on the Wandboard to see if that gives any info on the problem I'm having with my fakesink pipeline.

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[John Weber]

Hi Chris,

> That's unfortunate as I need a pipeline which can run on many different platforms, including 'standard pc'. This was one reason why I was using fakesink to capture the images - v4l looked like a way to go so that I could stream the video direct to the screen and get the full benefits of any acceleration supported by a platform (zero copy, etc.).
>

I think you will probably need to build an application that creates a pipeline 
based on the host.  For example, a desktop system probably will not have much 
use for V4L output devices, so they might not use v4l2sink but autovideosink. 
On a i.MX6 for example, you'll want to use mfw_v4lsink, which as I understand it 
does take advantage of some of the built-in acceleration.

Also, for video decoding (H.264 decompression I mean), you'll want to change the 
element you use there as well.  For i.MX6 this is vpudec, but for general 
purpose there is x264dec (think that is the name but not sure).

Regards,
John

Re: Gstreamer pipeline problem

[Chris Tapp]

Hi Thomas,

Some more info below:

On 11 Jul 2013, at 09:39, Thomas Senyk wrote:

> On Wednesday, 10 July, 2013 20:53:58 Chris Tapp wrote:
>> On 10 Jul 2013, at 20:19, Chris Tapp wrote:
>>> I've got an application which uses playbin2 to capture video. The pipeline
>>> is of the form:
>>> 
>>> playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb,
>>> pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> !
>>> fakesink"
>>> 
>>> I then get the "frame" property from the pipeline and use this to grab the
>>> latest frame.
>>> 
>>> This works on my development system (Ubuntu 11.10) and a Cedar Trail /
>>> Yocto system, but the pipeline fails on the Wandboard Quad. I think this
>>> is related to:
>>> 
>>> 0:00:13.028151336  1349 0x4442d520 WARN           basetransform
>>> /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linux
>>> -gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetrans
>>> form.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform
>>> could not transform video/x-raw-yuv, width=(int)854, height=(int)480,
>>> framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false
>>> in anything we support
>>> 
>>> I added an ffmpegcolorspace element betwween the queue2 and the videoscale
>>> to get round this and the pipeline now builds, but only a few frames are
>>> captured. There are different diagnostics showing:
>>> 
>>> 0:00:02.881403000  1361   0x28da60 WARN                  vpudec
>>> vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate
>>> Internal framebuffers!!!! Message Callback : Element playbin0x250b68
>>> changed state from READY to PAUSED. 0:00:03.237675000  1361   0x28da60
>>> WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame
>>> buffer message, return 0x89, 8 frames in displaying queue!!
>>> 0:00:03.242324334  1361   0x28da60 WARN                  vpudec
>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89,
>>> 8 frames in displaying queue!!
>>> 
>>> <lots of repeats>
>>> 
>>> 0:00:08.499914334  1382   0x28d860 WARN                  vpudec
>>> vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
>>> 0:00:08.500784667  1382   0x28d860 WARN                  vpudec
>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88,
>>> 8 frames in displaying queue!!
>>> 
>>> <lots of repeats>
>>> 
>>> Message Callback : Element playbin0x250aa0 changed state from PAUSED to
>>> PLAYING. 0:00:09.253202667  1382   0x28d860 WARN                  vpudec
>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x88,
>>> 8 frames in displaying queue!!
>>> 
>>> 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink
>>> mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed,
>>> ret 2 No such file or directory queued 0
>>> 
>>> 
>>> The "Message Callback" events are my own logging to try and see what's
>>> happening in my app.
>>> 
>>> Is this something I'm doing wrong, or are these messages a real issue
>>> somewhere?
>> This is when playing a .webm. The results for an .flv are as expected.
> 
> is it the same when you use v4l2 sink instead of fakesink?
> Is playbin (without defining the pipeline) behaving the same?

I've run some tests using pipelines of the form:

   gst-launch playbin2=file://trailer.webm video-sink="<a-sink-to-test>"

GST_DEBUG was set to "*:2"

1) When the sink is set to "mfw_4vlsink" I get full screen video playback;
2) When the sink is set to "fakesink" I get nothing on the screen (as expected), and no debug of interest;
3) When the sink is set to "queue2 ! mfw_4vlsink" I get very choppy full screen video playback (which eventually stalls) and debug output as originally reported;
4) When the sink is set to "queue2 ! fakesink" I get nothing on the screen (as expected), and no debug of interest;
5) When the sink is set to "queue2 ! fakesink sync=1" I get nothing on the screen and debug output as originally reported.

These were all run from the command line, no 'X' running.

It looks like queue2 + sync=1 (default for v4lsink) combination causes the problem... Setting sync=0 for v4lsink makes no difference and looks to be ignored.

Things also 'get confused' at times and 1) doesn't work anymore without a restart.

Does any of that help? ;-)

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[Chris Tapp]

Hi John,

On 12 Jul 2013, at 15:12, John Weber wrote:

> Hi Chris,
> 
>> That's unfortunate as I need a pipeline which can run on many different platforms, including 'standard pc'. This was one reason why I was using fakesink to capture the images - v4l looked like a way to go so that I could stream the video direct to the screen and get the full benefits of any acceleration supported by a platform (zero copy, etc.).
>> 
> 
> I think you will probably need to build an application that creates a pipeline based on the host.  For example, a desktop system probably will not have much use for V4L output devices, so they might not use v4l2sink but autovideosink. On a i.MX6 for example, you'll want to use mfw_v4lsink, which as I understand it does take advantage of some of the built-in acceleration.
> 
> Also, for video decoding (H.264 decompression I mean), you'll want to change the element you use there as well.  For i.MX6 this is vpudec, but for general purpose there is x264dec (think that is the name but not sure).


Yes, that's one of the reasons I've been using playbin2 and fakesink. I'm hoping that playbin will build a decent decode pipeline and I can then use the frames in fakesink to create an OpenGLES texture which I can render in my app. This also gives a portable way of rendering the video where I want, at the size I need and without and decoration.

I was hoping that glupload may help make this more efficient, but I can't even get gst-plugins-gl to build at the moment.

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[Thomas Senyk]

On Friday, 12 July, 2013 17:44:41 Chris Tapp wrote:
> Hi John,
> 
> On 12 Jul 2013, at 15:12, John Weber wrote:
> > Hi Chris,
> > 
> >> That's unfortunate as I need a pipeline which can run on many different
> >> platforms, including 'standard pc'. This was one reason why I was using
> >> fakesink to capture the images - v4l looked like a way to go so that I
> >> could stream the video direct to the screen and get the full benefits of
> >> any acceleration supported by a platform (zero copy, etc.).> 
> > I think you will probably need to build an application that creates a
> > pipeline based on the host.  For example, a desktop system probably will
> > not have much use for V4L output devices, so they might not use v4l2sink
> > but autovideosink. On a i.MX6 for example, you'll want to use
> > mfw_v4lsink, which as I understand it does take advantage of some of the
> > built-in acceleration.
> > 
> > Also, for video decoding (H.264 decompression I mean), you'll want to
> > change the element you use there as well.  For i.MX6 this is vpudec, but
> > for general purpose there is x264dec (think that is the name but not
> > sure).
> Yes, that's one of the reasons I've been using playbin2 and fakesink. I'm
> hoping that playbin will build a decent decode pipeline and I can then use
> the frames in fakesink to create an OpenGLES texture which I can render in
> my app. This also gives a portable way of rendering the video where I want,
> at the size I need and without and decoration.
> 
> I was hoping that glupload may help make this more efficient, but I can't
> even get gst-plugins-gl to build at the moment.

I'm not sure if glupload will help you. 
I tried last week to integrate into Qt and failed.

It heavily depends on the usecase and whether your using X11 or not.
The reason why gst-plugins-gl is applicable for me/QtMultimedia:
https://lists.yoctoproject.org/pipermail/meta-freescale/2013-July/003563.html

Maybe your usecase is different.
Maybe it's good enough for you to render into another framebuffer or you're 
using X11.



> 
> Chris Tapp
> 
> opensource@keylevel.com
> www.keylevel.com
> 
> 
> 
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale

Re: Gstreamer pipeline problem

[Thomas Senyk]

On Friday, 12 July, 2013 17:35:01 Chris Tapp wrote:
> Hi Thomas,
> 
> Some more info below:
> 
> On 11 Jul 2013, at 09:39, Thomas Senyk wrote:
> > On Wednesday, 10 July, 2013 20:53:58 Chris Tapp wrote:
> >> On 10 Jul 2013, at 20:19, Chris Tapp wrote:
> >>> I've got an application which uses playbin2 to capture video. The
> >>> pipeline
> >>> is of the form:
> >>> 
> >>> playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb,
> >>> pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> !
> >>> fakesink"
> >>> 
> >>> I then get the "frame" property from the pipeline and use this to grab
> >>> the
> >>> latest frame.
> >>> 
> >>> This works on my development system (Ubuntu 11.10) and a Cedar Trail /
> >>> Yocto system, but the pipeline fails on the Wandboard Quad. I think this
> >>> is related to:
> >>> 
> >>> 0:00:13.028151336  1349 0x4442d520 WARN           basetransform
> >>> /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linu
> >>> x
> >>> -gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetra
> >>> ns
> >>> form.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform
> >>> could not transform video/x-raw-yuv, width=(int)854, height=(int)480,
> >>> framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false
> >>> in anything we support
> >>> 
> >>> I added an ffmpegcolorspace element betwween the queue2 and the
> >>> videoscale
> >>> to get round this and the pipeline now builds, but only a few frames are
> >>> captured. There are different diagnostics showing:
> >>> 
> >>> 0:00:02.881403000  1361   0x28da60 WARN                  vpudec
> >>> vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate
> >>> Internal framebuffers!!!! Message Callback : Element playbin0x250b68
> >>> changed state from READY to PAUSED. 0:00:03.237675000  1361   0x28da60
> >>> WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no
> >>> frame
> >>> buffer message, return 0x89, 8 frames in displaying queue!!
> >>> 0:00:03.242324334  1361   0x28da60 WARN                  vpudec
> >>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>> 0x89,
> >>> 8 frames in displaying queue!!
> >>> 
> >>> <lots of repeats>
> >>> 
> >>> 0:00:08.499914334  1382   0x28d860 WARN                  vpudec
> >>> vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
> >>> 0:00:08.500784667  1382   0x28d860 WARN                  vpudec
> >>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>> 0x88,
> >>> 8 frames in displaying queue!!
> >>> 
> >>> <lots of repeats>
> >>> 
> >>> Message Callback : Element playbin0x250aa0 changed state from PAUSED to
> >>> PLAYING. 0:00:09.253202667  1382   0x28d860 WARN                  vpudec
> >>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>> 0x88,
> >>> 8 frames in displaying queue!!
> >>> 
> >>> 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink
> >>> mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed,
> >>> ret 2 No such file or directory queued 0
> >>> 
> >>> 
> >>> The "Message Callback" events are my own logging to try and see what's
> >>> happening in my app.
> >>> 
> >>> Is this something I'm doing wrong, or are these messages a real issue
> >>> somewhere?
> >> 
> >> This is when playing a .webm. The results for an .flv are as expected.
> > 
> > is it the same when you use v4l2 sink instead of fakesink?
> > Is playbin (without defining the pipeline) behaving the same?
> 
> I've run some tests using pipelines of the form:
> 
>    gst-launch playbin2=file://trailer.webm video-sink="<a-sink-to-test>"
> 
> GST_DEBUG was set to "*:2"
> 
> 1) When the sink is set to "mfw_4vlsink" I get full screen video playback;
> 2) When the sink is set to "fakesink" I get nothing on the screen (as
> expected), and no debug of interest; 3) When the sink is set to "queue2 !
> mfw_4vlsink" I get very choppy full screen video playback (which eventually
> stalls) and debug output as originally reported; 4) When the sink is set to
> "queue2 ! fakesink" I get nothing on the screen (as expected), and no debug
> of interest; 5) When the sink is set to "queue2 ! fakesink sync=1" I get
> nothing on the screen and debug output as originally reported.
> 
> These were all run from the command line, no 'X' running.
> 
> It looks like queue2 + sync=1 (default for v4lsink) combination causes the
> problem... Setting sync=0 for v4lsink makes no difference and looks to be
> ignored.

Do you actually need queue2 or sync=1?

Your end goal is to get everything into your opengl context and render it, 
right?
Shouldn't you be good with setting up a minimal gstreamer pipeline and let 
gstreamer handle the rest?


Don't get me wrong. I'm not an gstreamer expert :)
I'm just trying to give any (possibly) useful hint I have :)


> 
> Things also 'get confused' at times and 1) doesn't work anymore without a
> restart.

I've seen this as well .. the framebuffer gets black and nothing seems to be 
able to render to it anymore ... I think for me it's something with vpu vs. 
gpu memory (which is the same 'allocation' on the unified-memory)

See:
https://community.freescale.com/docs/DOC-93591

Do you see any allocation errors?
I'm going to play with 'gpumem' today to see if I can get better/stable 
results.

> 
> Does any of that help? ;-)


Another thing I spotted:
I just looked at your original mail and from the work I've done last week I 
can tell that I've never seen* or therefor used  x-raw-rgb
* 'seen' as in: source file having rgb format

For me there was no need ... I'm having a sink (Qt/c++ code) which takes x-
raw-yuv ... this code give the frame over to the opengl-scenegraph and then I 
map this memory with code like:

void *bits = (void*)mFrame.bits();
GLuint physical = ~0U;	
glTexDirectVIVMap_LOCALE(GL_TEXTURE_2D, w, h, GL_VIV_YV12, &bits, &physical);


This is then directly used in the shaders without any YUV->RGB code. The 
texture units knows YUV and does the texture-coordinate/color-lookup 
accordingly.
 ... what I'm trying to say: you don't have to convert to GL_RGB(A).



> 
> Chris Tapp
> 
> opensource@keylevel.com
> www.keylevel.com

Re: Gstreamer pipeline problem

[Chris Tapp]

On 15 Jul 2013, at 09:24, Thomas Senyk wrote:

> On Friday, 12 July, 2013 17:35:01 Chris Tapp wrote:
>> Hi Thomas,
>> 
>> Some more info below:
>> 
>> On 11 Jul 2013, at 09:39, Thomas Senyk wrote:
>>> On Wednesday, 10 July, 2013 20:53:58 Chris Tapp wrote:
>>>> On 10 Jul 2013, at 20:19, Chris Tapp wrote:
>>>>> I've got an application which uses playbin2 to capture video. The
>>>>> pipeline
>>>>> is of the form:
>>>>> 
>>>>> playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb,
>>>>> pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height> !
>>>>> fakesink"
>>>>> 
>>>>> I then get the "frame" property from the pipeline and use this to grab
>>>>> the
>>>>> latest frame.
>>>>> 
>>>>> This works on my development system (Ubuntu 11.10) and a Cedar Trail /
>>>>> Yocto system, but the pipeline fails on the Wandboard Quad. I think this
>>>>> is related to:
>>>>> 
>>>>> 0:00:13.028151336  1349 0x4442d520 WARN           basetransform
>>>>> /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-linu
>>>>> x
>>>>> -gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbasetra
>>>>> ns
>>>>> form.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform
>>>>> could not transform video/x-raw-yuv, width=(int)854, height=(int)480,
>>>>> framerate=(fraction)24/1, format=(fourcc)I420, interlaced=(boolean)false
>>>>> in anything we support
>>>>> 
>>>>> I added an ffmpegcolorspace element betwween the queue2 and the
>>>>> videoscale
>>>>> to get round this and the pipeline now builds, but only a few frames are
>>>>> captured. There are different diagnostics showing:
>>>>> 
>>>>> 0:00:02.881403000  1361   0x28da60 WARN                  vpudec
>>>>> vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate
>>>>> Internal framebuffers!!!! Message Callback : Element playbin0x250b68
>>>>> changed state from READY to PAUSED. 0:00:03.237675000  1361   0x28da60
>>>>> WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no
>>>>> frame
>>>>> buffer message, return 0x89, 8 frames in displaying queue!!
>>>>> 0:00:03.242324334  1361   0x28da60 WARN                  vpudec
>>>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
>>>>> 0x89,
>>>>> 8 frames in displaying queue!!
>>>>> 
>>>>> <lots of repeats>
>>>>> 
>>>>> 0:00:08.499914334  1382   0x28d860 WARN                  vpudec
>>>>> vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
>>>>> 0:00:08.500784667  1382   0x28d860 WARN                  vpudec
>>>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
>>>>> 0x88,
>>>>> 8 frames in displaying queue!!
>>>>> 
>>>>> <lots of repeats>
>>>>> 
>>>>> Message Callback : Element playbin0x250aa0 changed state from PAUSED to
>>>>> PLAYING. 0:00:09.253202667  1382   0x28d860 WARN                  vpudec
>>>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
>>>>> 0x88,
>>>>> 8 frames in displaying queue!!
>>>>> 
>>>>> 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink
>>>>> mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer failed,
>>>>> ret 2 No such file or directory queued 0
>>>>> 
>>>>> 
>>>>> The "Message Callback" events are my own logging to try and see what's
>>>>> happening in my app.
>>>>> 
>>>>> Is this something I'm doing wrong, or are these messages a real issue
>>>>> somewhere?
>>>> 
>>>> This is when playing a .webm. The results for an .flv are as expected.
>>> 
>>> is it the same when you use v4l2 sink instead of fakesink?
>>> Is playbin (without defining the pipeline) behaving the same?
>> 
>> I've run some tests using pipelines of the form:
>> 
>>   gst-launch playbin2=file://trailer.webm video-sink="<a-sink-to-test>"
>> 
>> GST_DEBUG was set to "*:2"
>> 
>> 1) When the sink is set to "mfw_4vlsink" I get full screen video playback;
>> 2) When the sink is set to "fakesink" I get nothing on the screen (as
>> expected), and no debug of interest; 3) When the sink is set to "queue2 !
>> mfw_4vlsink" I get very choppy full screen video playback (which eventually
>> stalls) and debug output as originally reported; 4) When the sink is set to
>> "queue2 ! fakesink" I get nothing on the screen (as expected), and no debug
>> of interest; 5) When the sink is set to "queue2 ! fakesink sync=1" I get
>> nothing on the screen and debug output as originally reported.
>> 
>> These were all run from the command line, no 'X' running.
>> 
>> It looks like queue2 + sync=1 (default for v4lsink) combination causes the
>> problem... Setting sync=0 for v4lsink makes no difference and looks to be
>> ignored.
> 
> Do you actually need queue2 or sync=1?
> 
> Your end goal is to get everything into your opengl context and render it, 
> right?
> Shouldn't you be good with setting up a minimal gstreamer pipeline and let 
> gstreamer handle the rest?

I normally run a basic playbin2 pipeline to 'fakesink'. When it's time to render I grab the latest frame via playbin2, upload this into a texture and render.

I seem to need queue2 on the video-sink and audio-sink to keep them in sync. Video sinks generally set sync=1 by default, but fakesink doesn't. If this isn't set, then the video plays too fast and loses sync with the audio.

> 
> Don't get me wrong. I'm not an gstreamer expert :)
> I'm just trying to give any (possibly) useful hint I have :)

Me neither. Any hints are more than welcome ;-)

> 
> 
>> 
>> Things also 'get confused' at times and 1) doesn't work anymore without a
>> restart.
> 
> I've seen this as well .. the framebuffer gets black and nothing seems to be 
> able to render to it anymore ... I think for me it's something with vpu vs. 
> gpu memory (which is the same 'allocation' on the unified-memory)
> 
> See:
> https://community.freescale.com/docs/DOC-93591
> 
> Do you see any allocation errors?
> I'm going to play with 'gpumem' today to see if I can get better/stable 
> results.
> 
>> 
>> Does any of that help? ;-)
> 
> 
> Another thing I spotted:
> I just looked at your original mail and from the work I've done last week I 
> can tell that I've never seen* or therefor used  x-raw-rgb
> * 'seen' as in: source file having rgb format
> 
> For me there was no need ... I'm having a sink (Qt/c++ code) which takes x-
> raw-yuv ... this code give the frame over to the opengl-scenegraph and then I 
> map this memory with code like:
> 
> void *bits = (void*)mFrame.bits();
> GLuint physical = ~0U;	
> glTexDirectVIVMap_LOCALE(GL_TEXTURE_2D, w, h, GL_VIV_YV12, &bits, &physical);
> 
> 
> This is then directly used in the shaders without any YUV->RGB code. The 
> texture units knows YUV and does the texture-coordinate/color-lookup 
> accordingly.
> ... what I'm trying to say: you don't have to convert to GL_RGB(A).
> 
> 
> 
>> 
>> Chris Tapp
>> 
>> opensource@keylevel.com
>> www.keylevel.com

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[Chris Tapp]

On 15 Jul 2013, at 09:02, Thomas Senyk wrote:

> On Friday, 12 July, 2013 17:44:41 Chris Tapp wrote:
>> Hi John,
>> 
>> On 12 Jul 2013, at 15:12, John Weber wrote:
>>> Hi Chris,
>>> 
>>>> That's unfortunate as I need a pipeline which can run on many different
>>>> platforms, including 'standard pc'. This was one reason why I was using
>>>> fakesink to capture the images - v4l looked like a way to go so that I
>>>> could stream the video direct to the screen and get the full benefits of
>>>> any acceleration supported by a platform (zero copy, etc.).> 
>>> I think you will probably need to build an application that creates a
>>> pipeline based on the host.  For example, a desktop system probably will
>>> not have much use for V4L output devices, so they might not use v4l2sink
>>> but autovideosink. On a i.MX6 for example, you'll want to use
>>> mfw_v4lsink, which as I understand it does take advantage of some of the
>>> built-in acceleration.
>>> 
>>> Also, for video decoding (H.264 decompression I mean), you'll want to
>>> change the element you use there as well.  For i.MX6 this is vpudec, but
>>> for general purpose there is x264dec (think that is the name but not
>>> sure).
>> Yes, that's one of the reasons I've been using playbin2 and fakesink. I'm
>> hoping that playbin will build a decent decode pipeline and I can then use
>> the frames in fakesink to create an OpenGLES texture which I can render in
>> my app. This also gives a portable way of rendering the video where I want,
>> at the size I need and without and decoration.
>> 
>> I was hoping that glupload may help make this more efficient, but I can't
>> even get gst-plugins-gl to build at the moment.
> 
> I'm not sure if glupload will help you. 
> I tried last week to integrate into Qt and failed.
> 
> It heavily depends on the usecase and whether your using X11 or not.
> The reason why gst-plugins-gl is applicable for me/QtMultimedia:
> https://lists.yoctoproject.org/pipermail/meta-freescale/2013-July/003563.html
> 
> Maybe your usecase is different.
> Maybe it's good enough for you to render into another framebuffer or you're 
> using X11.

I saw the thread you started on this. I am using X, which is why I was think it was worth trying to see if I got a performance boost.

> 
> 
> 
>> 
>> Chris Tapp
>> 
>> opensource@keylevel.com
>> www.keylevel.com
>> 
>> 
>> 
>> _______________________________________________
>> meta-freescale mailing list
>> meta-freescale@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-freescale

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: Gstreamer pipeline problem

[Thomas Senyk]

On Monday, 15 July, 2013 17:41:17 Chris Tapp wrote:
> On 15 Jul 2013, at 09:24, Thomas Senyk wrote:
> > On Friday, 12 July, 2013 17:35:01 Chris Tapp wrote:
> >> Hi Thomas,
> >> 
> >> Some more info below:
> >> 
> >> On 11 Jul 2013, at 09:39, Thomas Senyk wrote:
> >>> On Wednesday, 10 July, 2013 20:53:58 Chris Tapp wrote:
> >>>> On 10 Jul 2013, at 20:19, Chris Tapp wrote:
> >>>>> I've got an application which uses playbin2 to capture video. The
> >>>>> pipeline
> >>>>> is of the form:
> >>>>> 
> >>>>> playbin2 uri=... video-sink="queue2 ! videoscale ! video/x-raw-rgb,
> >>>>> pixel-aspect-ratio=1/1, width=<capture-width>, height=<capture-height>
> >>>>> !
> >>>>> fakesink"
> >>>>> 
> >>>>> I then get the "frame" property from the pipeline and use this to grab
> >>>>> the
> >>>>> latest frame.
> >>>>> 
> >>>>> This works on my development system (Ubuntu 11.10) and a Cedar Trail /
> >>>>> Yocto system, but the pipeline fails on the Wandboard Quad. I think
> >>>>> this
> >>>>> is related to:
> >>>>> 
> >>>>> 0:00:13.028151336  1349 0x4442d520 WARN           basetransform
> >>>>> /media/SSD-RAID/build-danny-wandboard/tmp/work/armv7a-vfp-neon-poky-li
> >>>>> nu
> >>>>> x
> >>>>> -gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/libs/gst/base/gstbaset
> >>>>> ra
> >>>>> ns
> >>>>> form.c:1304:gst_base_transform_setcaps:<videoscale0x2ab820> transform
> >>>>> could not transform video/x-raw-yuv, width=(int)854, height=(int)480,
> >>>>> framerate=(fraction)24/1, format=(fourcc)I420,
> >>>>> interlaced=(boolean)false
> >>>>> in anything we support
> >>>>> 
> >>>>> I added an ffmpegcolorspace element betwween the queue2 and the
> >>>>> videoscale
> >>>>> to get round this and the pipeline now builds, but only a few frames
> >>>>> are
> >>>>> captured. There are different diagnostics showing:
> >>>>> 
> >>>>> 0:00:02.881403000  1361   0x28da60 WARN                  vpudec
> >>>>> vpudec.c:914:gst_vpudec_core_create_and_register_frames: Allocate
> >>>>> Internal framebuffers!!!! Message Callback : Element playbin0x250b68
> >>>>> changed state from READY to PAUSED. 0:00:03.237675000  1361   0x28da60
> >>>>> WARN                  vpudec vpudec.c:1578:gst_vpudec_chain: Got no
> >>>>> frame
> >>>>> buffer message, return 0x89, 8 frames in displaying queue!!
> >>>>> 0:00:03.242324334  1361   0x28da60 WARN                  vpudec
> >>>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>>>> 0x89,
> >>>>> 8 frames in displaying queue!!
> >>>>> 
> >>>>> <lots of repeats>
> >>>>> 
> >>>>> 0:00:08.499914334  1382   0x28d860 WARN                  vpudec
> >>>>> vpudec.c:1655:gst_vpudec_chain: Retry too many times, maybe BUG!!
> >>>>> 0:00:08.500784667  1382   0x28d860 WARN                  vpudec
> >>>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>>>> 0x88,
> >>>>> 8 frames in displaying queue!!
> >>>>> 
> >>>>> <lots of repeats>
> >>>>> 
> >>>>> Message Callback : Element playbin0x250aa0 changed state from PAUSED
> >>>>> to
> >>>>> PLAYING. 0:00:09.253202667  1382   0x28d860 WARN                 
> >>>>> vpudec
> >>>>> vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return
> >>>>> 0x88,
> >>>>> 8 frames in displaying queue!!
> >>>>> 
> >>>>> 0:00:13.364523335  1460   0x142ec0 WARN             mfw_v4lsink
> >>>>> mfw_gst_v4l_buffer.c:435:mfw_gst_v4l2_new_buffer: Try new buffer
> >>>>> failed,
> >>>>> ret 2 No such file or directory queued 0
> >>>>> 
> >>>>> 
> >>>>> The "Message Callback" events are my own logging to try and see what's
> >>>>> happening in my app.
> >>>>> 
> >>>>> Is this something I'm doing wrong, or are these messages a real issue
> >>>>> somewhere?
> >>>> 
> >>>> This is when playing a .webm. The results for an .flv are as expected.
> >>> 
> >>> is it the same when you use v4l2 sink instead of fakesink?
> >>> Is playbin (without defining the pipeline) behaving the same?
> >> 
> >> I've run some tests using pipelines of the form:
> >>   gst-launch playbin2=file://trailer.webm video-sink="<a-sink-to-test>"
> >> 
> >> GST_DEBUG was set to "*:2"
> >> 
> >> 1) When the sink is set to "mfw_4vlsink" I get full screen video
> >> playback;
> >> 2) When the sink is set to "fakesink" I get nothing on the screen (as
> >> expected), and no debug of interest; 3) When the sink is set to "queue2 !
> >> mfw_4vlsink" I get very choppy full screen video playback (which
> >> eventually
> >> stalls) and debug output as originally reported; 4) When the sink is set
> >> to
> >> "queue2 ! fakesink" I get nothing on the screen (as expected), and no
> >> debug
> >> of interest; 5) When the sink is set to "queue2 ! fakesink sync=1" I get
> >> nothing on the screen and debug output as originally reported.
> >> 
> >> These were all run from the command line, no 'X' running.
> >> 
> >> It looks like queue2 + sync=1 (default for v4lsink) combination causes
> >> the
> >> problem... Setting sync=0 for v4lsink makes no difference and looks to be
> >> ignored.
> > 
> > Do you actually need queue2 or sync=1?
> > 
> > Your end goal is to get everything into your opengl context and render it,
> > right?
> > Shouldn't you be good with setting up a minimal gstreamer pipeline and let
> > gstreamer handle the rest?
> 
> I normally run a basic playbin2 pipeline to 'fakesink'. When it's time to
> render I grab the latest frame via playbin2, upload this into a texture and
> render.

ah ok, then I see two possible setups.

either: 
<decoder pipeline> ! glupload ! fakesink ! <your application using the tex ids 
from glupload coming in with GstBuffer>

or: 
<decoder pipeline> ! fakesink ! <your application maps the memory, coming in 
with GstBuffer, using glTexDirectVIVMap>


... using fakesink 'in front' of your application is a means to get access to 
the GstBuffer objects, right? (rather then haven to implement the pad and 
everything for a actually sink, you just have to implement one call-back 
function for 'handoff'-signal).


Do I get you right?

If so, I advice you to do the 'or', it's not much code and you got all the 
opengl parts in your hand.
There shouldn't be any trade-off for this .. besides having to write hardware-
specific code.



> 
> I seem to need queue2 on the video-sink and audio-sink to keep them in sync.
> Video sinks generally set sync=1 by default, but fakesink doesn't. If this
> isn't set, then the video plays too fast and loses sync with the audio.

ok, never looked any lipsync so far (probably should have...) so can't tell, 
therefor never tried to fix it either.

> > Don't get me wrong. I'm not an gstreamer expert :)
> > I'm just trying to give any (possibly) useful hint I have :)
> 
> Me neither. Any hints are more than welcome ;-)
> 
> >> Things also 'get confused' at times and 1) doesn't work anymore without a
> >> restart.
> > 
> > I've seen this as well .. the framebuffer gets black and nothing seems to
> > be able to render to it anymore ... I think for me it's something with
> > vpu vs. gpu memory (which is the same 'allocation' on the unified-memory)
> > 
> > See:
> > https://community.freescale.com/docs/DOC-93591
> > 
> > Do you see any allocation errors?
> > I'm going to play with 'gpumem' today to see if I can get better/stable
> > results.
> > 
> >> Does any of that help? ;-)
> > 
> > Another thing I spotted:
> > I just looked at your original mail and from the work I've done last week
> > I
> > can tell that I've never seen* or therefor used  x-raw-rgb
> > * 'seen' as in: source file having rgb format
> > 
> > For me there was no need ... I'm having a sink (Qt/c++ code) which takes
> > x-
> > raw-yuv ... this code give the frame over to the opengl-scenegraph and
> > then I map this memory with code like:
> > 
> > void *bits = (void*)mFrame.bits();
> > GLuint physical = ~0U;
> > glTexDirectVIVMap_LOCALE(GL_TEXTURE_2D, w, h, GL_VIV_YV12, &bits,
> > &physical);
> > 
> > 
> > This is then directly used in the shaders without any YUV->RGB code. The
> > texture units knows YUV and does the texture-coordinate/color-lookup
> > accordingly.
> > ... what I'm trying to say: you don't have to convert to GL_RGB(A).
> > 
> >> Chris Tapp
> >> 
> >> opensource@keylevel.com
> >> www.keylevel.com
> 
> Chris Tapp
> 
> opensource@keylevel.com
> www.keylevel.com

Re: Gstreamer pipeline problem

[Chris Tapp]

On 15 Jul 2013, at 18:11, Thomas Senyk wrote:

<SNIP>

>> 
>> I normally run a basic playbin2 pipeline to 'fakesink'. When it's time to
>> render I grab the latest frame via playbin2, upload this into a texture and
>> render.
> 
> ah ok, then I see two possible setups.
> 
> either: 
> <decoder pipeline> ! glupload ! fakesink ! <your application using the tex ids 
> from glupload coming in with GstBuffer>
> 
> or: 
> <decoder pipeline> ! fakesink ! <your application maps the memory, coming in 
> with GstBuffer, using glTexDirectVIVMap>
> 
> 
> ... using fakesink 'in front' of your application is a means to get access to 
> the GstBuffer objects, right? (rather then haven to implement the pad and 
> everything for a actually sink, you just have to implement one call-back 
> function for 'handoff'-signal).
> 
> Do I get you right?

Nearly. What I've got is basically just:

playbin2 uri="..." video-sink="fakesink"

So, not a lot going on! I then use the "frame" property of playbin2 to get the most recently rendered frame, which I then glTexImage2D.

I really do it this way so I can easily control where the frame is rendered and get it in sync with my rendering loop (which has lots of other animation taking place).

> 
> If so, I advice you to do the 'or', it's not much code and you got all the 
> opengl parts in your hand.
> There shouldn't be any trade-off for this .. besides having to write hardware-
> specific code.

That's the real catch - I'm trying to write code which I can use on multiple platforms.

>> 
>> I seem to need queue2 on the video-sink and audio-sink to keep them in sync.
>> Video sinks generally set sync=1 by default, but fakesink doesn't. If this
>> isn't set, then the video plays too fast and loses sync with the audio.
> 
> ok, never looked any lipsync so far (probably should have...) so can't tell, 
> therefor never tried to fix it either.

I missed this bit earlier:

>>> Another thing I spotted:
>>> I just looked at your original mail and from the work I've done last week
>>> I
>>> can tell that I've never seen* or therefor used  x-raw-rgb
>>> * 'seen' as in: source file having rgb format
>>> 
>>> For me there was no need ... I'm having a sink (Qt/c++ code) which takes
>>> x-
>>> raw-yuv ... this code give the frame over to the opengl-scenegraph and
>>> then I map this memory with code like:
>>> 
>>> void *bits = (void*)mFrame.bits();
>>> GLuint physical = ~0U;
>>> glTexDirectVIVMap_LOCALE(GL_TEXTURE_2D, w, h, GL_VIV_YV12, &bits,
>>> &physical);
>>> 
>>> This is then directly used in the shaders without any YUV->RGB code. The
>>> texture units knows YUV and does the texture-coordinate/color-lookup
>>> accordingly.
>>> ... what I'm trying to say: you don't have to convert to GL_RGB(A).

That would be nice if it can be done portably.

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: vpudec doe not like "queue(2)" - Was: Gstreamer pipeline problem

[Chris Tapp]

Ok, I've now got this easily reproducible:

export GST_DEBUG="*:2"

1) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm
2) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="multiqueue ! mfw_v4lsink"
3) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="queue ! mfw_v4lsink"
4) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="queue2 ! mfw_v4lsink"

Playback for 1) and 2) is good, where as playback for 3) and 4) it starts of choppy, stalls and gets lots of messages of the form:

0:00:00.472157023  7566 0x40d60150 WARN    vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89, 10 frames in displaying queue!!

GST_DEBUG=GST_ELEMENT_FACTORY:3 shows that playbin2 without the video-sink property uses 'multiqueue'.

So, it looks as if there's an issue with 'vpudec' when the pipeline includes a 'queue' or 'queue2' element.

Is this the best place to report this, or does it need to go somewhere else?

Chris Tapp

opensource@keylevel.com
www.keylevel.com

Re: vpudec doe not like "queue(2)" - Was: Gstreamer pipeline problem

[John Weber]

Hi Chris,

I would try posting it on community.freescale.com.

Regards,
John

On 7/16/13 2:15 PM, Chris Tapp wrote:
> Ok, I've now got this easily reproducible:
>
> export GST_DEBUG="*:2"
>
> 1) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm
> 2) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="multiqueue ! mfw_v4lsink"
> 3) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="queue ! mfw_v4lsink"
> 4) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="queue2 ! mfw_v4lsink"
>
> Playback for 1) and 2) is good, where as playback for 3) and 4) it starts of choppy, stalls and gets lots of messages of the form:
>
> 0:00:00.472157023  7566 0x40d60150 WARN    vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89, 10 frames in displaying queue!!
>
> GST_DEBUG=GST_ELEMENT_FACTORY:3 shows that playbin2 without the video-sink property uses 'multiqueue'.
>
> So, it looks as if there's an issue with 'vpudec' when the pipeline includes a 'queue' or 'queue2' element.
>
> Is this the best place to report this, or does it need to go somewhere else?
>
> Chris Tapp
>
> opensource@keylevel.com
> www.keylevel.com
>
>
>
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale
>

Re: vpudec doe not like "queue(2)" - Was: Gstreamer pipeline problem

[Chris Tapp]

Hi John,

On 16 Jul 2013, at 21:18, John Weber wrote:

> Hi Chris,
> 
> I would try posting it on community.freescale.com.

Thanks. Post is at https://community.freescale.com/message/340471

> Regards,
> John
> 
> On 7/16/13 2:15 PM, Chris Tapp wrote:
>> Ok, I've now got this easily reproducible:
>> 
>> export GST_DEBUG="*:2"
>> 
>> 1) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm
>> 2) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="multiqueue ! mfw_v4lsink"
>> 3) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="queue ! mfw_v4lsink"
>> 4) gst-launch playbin2 uri=http://media.w3.org/2010/05/sintel/trailer.webm video-sink="queue2 ! mfw_v4lsink"
>> 
>> Playback for 1) and 2) is good, where as playback for 3) and 4) it starts of choppy, stalls and gets lots of messages of the form:
>> 
>> 0:00:00.472157023  7566 0x40d60150 WARN    vpudec vpudec.c:1578:gst_vpudec_chain: Got no frame buffer message, return 0x89, 10 frames in displaying queue!!
>> 
>> GST_DEBUG=GST_ELEMENT_FACTORY:3 shows that playbin2 without the video-sink property uses 'multiqueue'.
>> 
>> So, it looks as if there's an issue with 'vpudec' when the pipeline includes a 'queue' or 'queue2' element.
>> 
>> Is this the best place to report this, or does it need to go somewhere else?
>> 
>> Chris Tapp
>> 
>> opensource@keylevel.com
>> www.keylevel.com
>> 
>> 
>> 
>> _______________________________________________
>> meta-freescale mailing list
>> meta-freescale@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-freescale
>> 
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale

Chris Tapp

opensource@keylevel.com
www.keylevel.com