coda: not generating EOS event

coda: not generating EOS event

[Fabio Estevam]

Hi,

I am running Gstreamer 1.4.4 with on a imx6q-sabresd board and I am
able to decode a video through the coda driver.

The pipeline I use is:
gst-launch-1.0 filesrc
location=/home/H264_test1_Talkinghead_mp4_480x360.mp4 ! qtdemux !
h264parse ! v4l2video1dec ! videoconvert ! fbdevsink

It plays the video fine, but it does not handle the end of stream
correctly: when the video reaches its end, then I get a flood of the
following messages:
[  144.049830] DEC_PIC_SUCCESS = 524289
[  144.064928] DEC_PIC_SUCCESS = 524289
[  144.085516] DEC_PIC_SUCCESS = 524289
[  144.108056] DEC_PIC_SUCCESS = 524289
[  144.130351] DEC_PIC_SUCCESS = 524289
[  144.152852] DEC_PIC_SUCCESS = 524289
[  144.176114] DEC_PIC_SUCCESS = 524289
[  144.198165] DEC_PIC_SUCCESS = 524289
[  144.220579] DEC_PIC_SUCCESS = 524289
[  144.242607] DEC_PIC_SUCCESS = 524289

These same DEC_PIC_SUCCESS messages appear if I use fakesink:
gst-launch-1.0 filesrc
location=/home/H264_test1_Talkinghead_mp4_480x360.mp4 ! qtdemux !
h264parse ! v4l2video1dec ! fakesink

In coda_dqbuf() we have:

    /* If this is the last capture buffer, emit an end-of-stream event */
    if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE &&
        coda_buf_is_end_of_stream(ctx, buf)) {
        const struct v4l2_event eos_event = {
            .type = V4L2_EVENT_EOS
        };

        v4l2_event_queue_fh(&ctx->fh, &eos_event);
    }

Seems we should also check for V4L2_BUF_TYPE_VIDEO_OUPUT. I tried it,
but still not able to fix the end of stream problem.

Any ideas?

Thanks

Re: coda: not generating EOS event

[Nicolas Dufresne]

Le jeudi 11 décembre 2014 à 11:00 -0200, Fabio Estevam a écrit :
> Hi,
> 
> I am running Gstreamer 1.4.4 with on a imx6q-sabresd board and I am
> able to decode a video through the coda driver.
> 
> The pipeline I use is:
> gst-launch-1.0 filesrc
> location=/home/H264_test1_Talkinghead_mp4_480x360.mp4 ! qtdemux !
> h264parse ! v4l2video1dec ! videoconvert ! fbdevsink

This is a known issue. The handling of EOS and draining is ill defined
in V4L2 specification. We should be clarifying this soon. Currently
GStreamer implements what was originally done in Exynos MFC driver. This
consist of sending empty buffer to the V4L2 Output (the input), until we
get an empty buffer (bytesused = 0) on the V4L2 Capture (output).

The CODA driver uses the new method to initiate the drain, which is to
send V4L2_DEC_CMD_STOP, this was not implemented by any driver when GST
v4l2 support was added. This is the right thing to do. This is tracked
at (contribution welcome of course):

https://bugzilla.gnome.org/show_bug.cgi?id=733864

Finally, CODA indicate that all buffer has been sent through an event,
V4L2_EVENT_EOS. This event was designed for another use case, it should
in fact be sent when the decoder is done with the encoded buffers,
rather then when the last decoded buffer has been queued. When correctly
implemented, this event cannot be used to figure-out when the last
decoded buffer has been dequeued.

During last workshop, it has been proposed to introduce a flag on the
last decoded buffer, _LAST. This flag could also be put on an empty
buffer to accommodate driver like MFC that only know it's EOS after the
last buffer has been delivered. This all need to be SPECed and
implemented. If you need to get that to run now, the easiest might be to
hack CODA to mimic MFC behavior. Implement DEC_CMD_STOP in GStreamer
shall not be very hard though, it simply need someone to do it.

cheers,
Nicolas

Re: coda: not generating EOS event

[Hans Verkuil]

Hi Pawel,

On 12/11/14 15:42, Nicolas Dufresne wrote:
> Le jeudi 11 décembre 2014 à 11:00 -0200, Fabio Estevam a écrit :
>> Hi,
>>
>> I am running Gstreamer 1.4.4 with on a imx6q-sabresd board and I am
>> able to decode a video through the coda driver.
>>
>> The pipeline I use is:
>> gst-launch-1.0 filesrc
>> location=/home/H264_test1_Talkinghead_mp4_480x360.mp4 ! qtdemux !
>> h264parse ! v4l2video1dec ! videoconvert ! fbdevsink
> 
> This is a known issue. The handling of EOS and draining is ill defined
> in V4L2 specification. We should be clarifying this soon. Currently
> GStreamer implements what was originally done in Exynos MFC driver. This
> consist of sending empty buffer to the V4L2 Output (the input), until we
> get an empty buffer (bytesused = 0) on the V4L2 Capture (output).
> 
> The CODA driver uses the new method to initiate the drain, which is to
> send V4L2_DEC_CMD_STOP, this was not implemented by any driver when GST
> v4l2 support was added. This is the right thing to do. This is tracked
> at (contribution welcome of course):
> 
> https://bugzilla.gnome.org/show_bug.cgi?id=733864
> 
> Finally, CODA indicate that all buffer has been sent through an event,
> V4L2_EVENT_EOS. This event was designed for another use case, it should
> in fact be sent when the decoder is done with the encoded buffers,
> rather then when the last decoded buffer has been queued. When correctly
> implemented, this event cannot be used to figure-out when the last
> decoded buffer has been dequeued.
> 
> During last workshop, it has been proposed to introduce a flag on the
> last decoded buffer, _LAST. This flag could also be put on an empty

Are you planning to work on this? That was my assumption, but it's probably
a good idea to check in case we are waiting for one another :-)

Regards,

	Hans

> buffer to accommodate driver like MFC that only know it's EOS after the
> last buffer has been delivered. This all need to be SPECed and
> implemented. If you need to get that to run now, the easiest might be to
> hack CODA to mimic MFC behavior. Implement DEC_CMD_STOP in GStreamer
> shall not be very hard though, it simply need someone to do it.
> 
> cheers,
> Nicolas
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: coda: not generating EOS event

[Philipp Zabel]

Hi,

Am Donnerstag, den 11.12.2014, 15:47 +0100 schrieb Hans Verkuil:
> Hi Pawel,
> 
> On 12/11/14 15:42, Nicolas Dufresne wrote:
> > Le jeudi 11 décembre 2014 à 11:00 -0200, Fabio Estevam a écrit :
> >> Hi,
> >>
> >> I am running Gstreamer 1.4.4 with on a imx6q-sabresd board and I am
> >> able to decode a video through the coda driver.
> >>
> >> The pipeline I use is:
> >> gst-launch-1.0 filesrc
> >> location=/home/H264_test1_Talkinghead_mp4_480x360.mp4 ! qtdemux !
> >> h264parse ! v4l2video1dec ! videoconvert ! fbdevsink
> > 
> > This is a known issue. The handling of EOS and draining is ill defined
> > in V4L2 specification. We should be clarifying this soon. Currently
> > GStreamer implements what was originally done in Exynos MFC driver. This
> > consist of sending empty buffer to the V4L2 Output (the input), until we
> > get an empty buffer (bytesused = 0) on the V4L2 Capture (output).
> > 
> > The CODA driver uses the new method to initiate the drain, which is to
> > send V4L2_DEC_CMD_STOP, this was not implemented by any driver when GST
> > v4l2 support was added. This is the right thing to do. This is tracked
> > at (contribution welcome of course):
> > 
> > https://bugzilla.gnome.org/show_bug.cgi?id=733864
> > 
> > Finally, CODA indicate that all buffer has been sent through an event,
> > V4L2_EVENT_EOS. This event was designed for another use case, it should
> > in fact be sent when the decoder is done with the encoded buffers,
> > rather then when the last decoded buffer has been queued. When correctly
> > implemented, this event cannot be used to figure-out when the last
> > decoded buffer has been dequeued.
> > 
> > During last workshop, it has been proposed to introduce a flag on the
> > last decoded buffer, _LAST. This flag could also be put on an empty
> 
> Are you planning to work on this? That was my assumption, but it's probably
> a good idea to check in case we are waiting for one another :-)

I had another look at the ELC-E 2014 V4L2 codec API document and have
tried to implement the proposed decoder draining flow in an RFC:
"Signalling last decoded frame by V4L2_BUF_FLAG_LAST and -EPIPE":

http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/87152

Are you planning to pour the workshop's codec API document into a V4L2
documentation patch?

regards
Philipp

Re: coda: not generating EOS event

[Pawel Osciak]

Hi Philipp,

On Thu, Jan 22, 2015 at 11:58 PM, Philipp Zabel <p.zabel@pengutronix.de> wrote:
> Hi,
>
> Are you planning to pour the workshop's codec API document into a V4L2
> documentation patch?

Yes, definitely, I am a bit delayed, but will do this eventually, this
is in progress. Sorry for the delay. Please feel free to submit
anything related to EOS if you decide to work on this.

-- 
Best regards,
Pawel

Re: coda: not generating EOS event

[Jean-Michel Hautbois]

Hi all,

2015-01-22 15:58 GMT+01:00 Philipp Zabel <p.zabel@pengutronix.de>:
> Hi,
>
> Am Donnerstag, den 11.12.2014, 15:47 +0100 schrieb Hans Verkuil:
>> Hi Pawel,
>>
>> On 12/11/14 15:42, Nicolas Dufresne wrote:
>> > Le jeudi 11 décembre 2014 à 11:00 -0200, Fabio Estevam a écrit :
>> >> Hi,
>> >>
>> >> I am running Gstreamer 1.4.4 with on a imx6q-sabresd board and I am
>> >> able to decode a video through the coda driver.
>> >>
>> >> The pipeline I use is:
>> >> gst-launch-1.0 filesrc
>> >> location=/home/H264_test1_Talkinghead_mp4_480x360.mp4 ! qtdemux !
>> >> h264parse ! v4l2video1dec ! videoconvert ! fbdevsink
>> >
>> > This is a known issue. The handling of EOS and draining is ill defined
>> > in V4L2 specification. We should be clarifying this soon. Currently
>> > GStreamer implements what was originally done in Exynos MFC driver. This
>> > consist of sending empty buffer to the V4L2 Output (the input), until we
>> > get an empty buffer (bytesused = 0) on the V4L2 Capture (output).
>> >
>> > The CODA driver uses the new method to initiate the drain, which is to
>> > send V4L2_DEC_CMD_STOP, this was not implemented by any driver when GST
>> > v4l2 support was added. This is the right thing to do. This is tracked
>> > at (contribution welcome of course):
>> >
>> > https://bugzilla.gnome.org/show_bug.cgi?id=733864
>> >
>> > Finally, CODA indicate that all buffer has been sent through an event,
>> > V4L2_EVENT_EOS. This event was designed for another use case, it should
>> > in fact be sent when the decoder is done with the encoded buffers,
>> > rather then when the last decoded buffer has been queued. When correctly
>> > implemented, this event cannot be used to figure-out when the last
>> > decoded buffer has been dequeued.
>> >
>> > During last workshop, it has been proposed to introduce a flag on the
>> > last decoded buffer, _LAST. This flag could also be put on an empty
>>
>> Are you planning to work on this? That was my assumption, but it's probably
>> a good idea to check in case we are waiting for one another :-)
>
> I had another look at the ELC-E 2014 V4L2 codec API document and have
> tried to implement the proposed decoder draining flow in an RFC:
> "Signalling last decoded frame by V4L2_BUF_FLAG_LAST and -EPIPE":
>
> http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/87152
>
> Are you planning to pour the workshop's codec API document into a V4L2
> documentation patch?

I am now working on the encoder part (for coda) which should do the same.
But everything is documented from a decoder POV, not the encoder's...
Is the mecanism already there, and the only thing which should be
added is the ENC_CMD_STOP command ?

Thanks,
JM