[RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Hans Verkuil]

It turns out that we have both JPEG and Motion-JPEG pixel formats defined.

Furthermore, some drivers support one, some the other and some both.

These pixelformats both mean the same.

I propose that we settle on JPEG (since it seems to be used most often) and
add JPEG support to those drivers that currently only use MJPEG.

We also need to update the V4L2_PIX_FMT_JPEG documentation since it just says
TBD:

https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compressed.html

$ git grep -l V4L2_PIX_FMT_MJPEG
drivers/media/pci/meye/meye.c
drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
drivers/media/platform/sti/delta/delta-cfg.h
drivers/media/platform/sti/delta/delta-mjpeg-dec.c
drivers/media/usb/cpia2/cpia2_v4l.c
drivers/media/usb/go7007/go7007-driver.c
drivers/media/usb/go7007/go7007-fw.c
drivers/media/usb/go7007/go7007-v4l2.c
drivers/media/usb/s2255/s2255drv.c
drivers/media/usb/uvc/uvc_driver.c
drivers/staging/media/zoran/zoran_driver.c
drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
drivers/usb/gadget/function/uvc_v4l2.c

It looks like s2255 and cpia2 support both already, so that would leave
8 drivers that need to be modified, uvc being the most important of the
lot.

Any comments?

Regards,

	Hans

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Laurent Pinchart]

Hi Hans,

On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote:
> It turns out that we have both JPEG and Motion-JPEG pixel formats defined.
> 
> Furthermore, some drivers support one, some the other and some both.
> 
> These pixelformats both mean the same.

Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were not 
included in the JPEG headers.

> I propose that we settle on JPEG (since it seems to be used most often) and
> add JPEG support to those drivers that currently only use MJPEG.
> 
> We also need to update the V4L2_PIX_FMT_JPEG documentation since it just
> says TBD:
> 
> https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compresse
> d.html
> 
> $ git grep -l V4L2_PIX_FMT_MJPEG
> drivers/media/pci/meye/meye.c
> drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
> drivers/media/platform/sti/delta/delta-cfg.h
> drivers/media/platform/sti/delta/delta-mjpeg-dec.c
> drivers/media/usb/cpia2/cpia2_v4l.c
> drivers/media/usb/go7007/go7007-driver.c
> drivers/media/usb/go7007/go7007-fw.c
> drivers/media/usb/go7007/go7007-v4l2.c
> drivers/media/usb/s2255/s2255drv.c
> drivers/media/usb/uvc/uvc_driver.c
> drivers/staging/media/zoran/zoran_driver.c
> drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
> drivers/usb/gadget/function/uvc_v4l2.c
> 
> It looks like s2255 and cpia2 support both already, so that would leave
> 8 drivers that need to be modified, uvc being the most important of the
> lot.
> 
> Any comments?

-- 
Regards,

Laurent Pinchart

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Hans Verkuil]

On 10/01/2018 01:48 PM, Laurent Pinchart wrote:
> Hi Hans,
> 
> On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote:
>> It turns out that we have both JPEG and Motion-JPEG pixel formats defined.
>>
>> Furthermore, some drivers support one, some the other and some both.
>>
>> These pixelformats both mean the same.
> 
> Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were not 
> included in the JPEG headers.

I'm not aware of any difference. If there is one, then it is certainly not
documented.

Ezequiel, since you've been working with this recently, do you know anything
about this?

Regards,

	Hans

> 
>> I propose that we settle on JPEG (since it seems to be used most often) and
>> add JPEG support to those drivers that currently only use MJPEG.
>>
>> We also need to update the V4L2_PIX_FMT_JPEG documentation since it just
>> says TBD:
>>
>> https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compresse
>> d.html
>>
>> $ git grep -l V4L2_PIX_FMT_MJPEG
>> drivers/media/pci/meye/meye.c
>> drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
>> drivers/media/platform/sti/delta/delta-cfg.h
>> drivers/media/platform/sti/delta/delta-mjpeg-dec.c
>> drivers/media/usb/cpia2/cpia2_v4l.c
>> drivers/media/usb/go7007/go7007-driver.c
>> drivers/media/usb/go7007/go7007-fw.c
>> drivers/media/usb/go7007/go7007-v4l2.c
>> drivers/media/usb/s2255/s2255drv.c
>> drivers/media/usb/uvc/uvc_driver.c
>> drivers/staging/media/zoran/zoran_driver.c
>> drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
>> drivers/usb/gadget/function/uvc_v4l2.c
>>
>> It looks like s2255 and cpia2 support both already, so that would leave
>> 8 drivers that need to be modified, uvc being the most important of the
>> lot.
>>
>> Any comments?
> 

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Laurent Pinchart]

Hi Hans,

On Monday, 1 October 2018 14:54:29 EEST Hans Verkuil wrote:
> On 10/01/2018 01:48 PM, Laurent Pinchart wrote:
> > On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote:
> >> It turns out that we have both JPEG and Motion-JPEG pixel formats
> >> defined.
> >> 
> >> Furthermore, some drivers support one, some the other and some both.
> >> 
> >> These pixelformats both mean the same.
> > 
> > Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were
> > not included in the JPEG headers.
> 
> I'm not aware of any difference. If there is one, then it is certainly not
> documented.

What I can tell for sure is that many UVC devices don't include Huffman tables 
in their JPEG headers.

> Ezequiel, since you've been working with this recently, do you know anything
> about this?

-- 
Regards,

Laurent Pinchart

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Nicolas Dufresne]

Hello Hans,

Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit :
> It turns out that we have both JPEG and Motion-JPEG pixel formats defined.
> 
> Furthermore, some drivers support one, some the other and some both.
> 
> These pixelformats both mean the same.
> 
> I propose that we settle on JPEG (since it seems to be used most often) and
> add JPEG support to those drivers that currently only use MJPEG.

Thanks for looking into this. As per GStreamer code, I see 3 alias for
JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code
was written before I knew GStreamer existed. It's possible there is a
subtle difference, I have never looked at it, but clearly all our JPEG
decoder handle these as being the same.

https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2object.c#n956

> 
> We also need to update the V4L2_PIX_FMT_JPEG documentation since it just says
> TBD:
> 
> https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compressed.html
> 
> $ git grep -l V4L2_PIX_FMT_MJPEG
> drivers/media/pci/meye/meye.c
> drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
> drivers/media/platform/sti/delta/delta-cfg.h
> drivers/media/platform/sti/delta/delta-mjpeg-dec.c
> drivers/media/usb/cpia2/cpia2_v4l.c
> drivers/media/usb/go7007/go7007-driver.c
> drivers/media/usb/go7007/go7007-fw.c
> drivers/media/usb/go7007/go7007-v4l2.c
> drivers/media/usb/s2255/s2255drv.c
> drivers/media/usb/uvc/uvc_driver.c
> drivers/staging/media/zoran/zoran_driver.c
> drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
> drivers/usb/gadget/function/uvc_v4l2.c
> 
> It looks like s2255 and cpia2 support both already, so that would leave
> 8 drivers that need to be modified, uvc being the most important of the
> lot.
> 
> Any comments?
> 
> Regards,
> 
> 	Hans

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Laurent Pinchart]

Hello,

On Monday, 1 October 2018 15:42:56 EEST Nicolas Dufresne wrote:
> Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit :
> > It turns out that we have both JPEG and Motion-JPEG pixel formats defined.
> > 
> > Furthermore, some drivers support one, some the other and some both.
> > 
> > These pixelformats both mean the same.
> > 
> > I propose that we settle on JPEG (since it seems to be used most often)
> > and add JPEG support to those drivers that currently only use MJPEG.
> 
> Thanks for looking into this. As per GStreamer code, I see 3 alias for
> JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code
> was written before I knew GStreamer existed. It's possible there is a
> subtle difference, I have never looked at it, but clearly all our JPEG
> decoder handle these as being the same.
> 
> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l
> 2object.c#n956

Some old code to give a bit of context:

https://github.com/TimSC/mjpeg/

It should be feasible to implement a decoder that inserts the right Huffman 
table when none exists, but it has to be explicitly handled. An out-of-band 
mechanism to convey the information seems potentially useful to me.

> > We also need to update the V4L2_PIX_FMT_JPEG documentation since it just
> > says TBD:
> > 
> > https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compres
> > sed.html
> > 
> > $ git grep -l V4L2_PIX_FMT_MJPEG
> > drivers/media/pci/meye/meye.c
> > drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
> > drivers/media/platform/sti/delta/delta-cfg.h
> > drivers/media/platform/sti/delta/delta-mjpeg-dec.c
> > drivers/media/usb/cpia2/cpia2_v4l.c
> > drivers/media/usb/go7007/go7007-driver.c
> > drivers/media/usb/go7007/go7007-fw.c
> > drivers/media/usb/go7007/go7007-v4l2.c
> > drivers/media/usb/s2255/s2255drv.c
> > drivers/media/usb/uvc/uvc_driver.c
> > drivers/staging/media/zoran/zoran_driver.c
> > drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
> > drivers/usb/gadget/function/uvc_v4l2.c
> > 
> > It looks like s2255 and cpia2 support both already, so that would leave
> > 8 drivers that need to be modified, uvc being the most important of the
> > lot.
> > 
> > Any comments?

-- 
Regards,

Laurent Pinchart

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Ezequiel Garcia]

On Mon, 2018-10-01 at 08:42 -0400, Nicolas Dufresne wrote:
> Hello Hans,
> 
> Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit :
> > It turns out that we have both JPEG and Motion-JPEG pixel formats defined.
> > 
> > Furthermore, some drivers support one, some the other and some both.
> > 
> > These pixelformats both mean the same.
> > 
> > I propose that we settle on JPEG (since it seems to be used most often) and
> > add JPEG support to those drivers that currently only use MJPEG.
> 
> Thanks for looking into this. As per GStreamer code, I see 3 alias for
> JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code
> was written before I knew GStreamer existed. It's possible there is a
> subtle difference, I have never looked at it, but clearly all our JPEG
> decoder handle these as being the same.
> 
> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2object.c#n956
> 

To add more data points on the gstreamer side, there's really no difference
between gstreamer's types image/jpeg and video/x-jpeg.

Notably, jpegdec element just stuffs a huffman table if one is missing,
for any jpeg:

https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/ext/jpeg/gstjpegdec.c#n584

> > 
> > We also need to update the V4L2_PIX_FMT_JPEG documentation since it just says
> > TBD:
> > 
> > https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compressed.html
> > 
> > $ git grep -l V4L2_PIX_FMT_MJPEG
> > drivers/media/pci/meye/meye.c
> > drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
> > drivers/media/platform/sti/delta/delta-cfg.h
> > drivers/media/platform/sti/delta/delta-mjpeg-dec.c
> > drivers/media/usb/cpia2/cpia2_v4l.c
> > drivers/media/usb/go7007/go7007-driver.c
> > drivers/media/usb/go7007/go7007-fw.c
> > drivers/media/usb/go7007/go7007-v4l2.c
> > drivers/media/usb/s2255/s2255drv.c
> > drivers/media/usb/uvc/uvc_driver.c
> > drivers/staging/media/zoran/zoran_driver.c
> > drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
> > drivers/usb/gadget/function/uvc_v4l2.c
> > 
> > It looks like s2255 and cpia2 support both already, so that would leave
> > 8 drivers that need to be modified, uvc being the most important of the
> > lot.
> > 
> > Any comments?
> > 
> > Regards,
> > 
> > 	Hans
> 
> 

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Hans Verkuil]

On 10/01/2018 06:12 PM, Ezequiel Garcia wrote:
> On Mon, 2018-10-01 at 08:42 -0400, Nicolas Dufresne wrote:
>> Hello Hans,
>>
>> Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit :
>>> It turns out that we have both JPEG and Motion-JPEG pixel formats defined.
>>>
>>> Furthermore, some drivers support one, some the other and some both.
>>>
>>> These pixelformats both mean the same.
>>>
>>> I propose that we settle on JPEG (since it seems to be used most often) and
>>> add JPEG support to those drivers that currently only use MJPEG.
>>
>> Thanks for looking into this. As per GStreamer code, I see 3 alias for
>> JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code
>> was written before I knew GStreamer existed. It's possible there is a
>> subtle difference, I have never looked at it, but clearly all our JPEG
>> decoder handle these as being the same.
>>
>> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2object.c#n956
>>
> 
> To add more data points on the gstreamer side, there's really no difference
> between gstreamer's types image/jpeg and video/x-jpeg.
> 
> Notably, jpegdec element just stuffs a huffman table if one is missing,
> for any jpeg:
> 
> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/ext/jpeg/gstjpegdec.c#n584

lib/libv4lconvert/libv4lconvert.c also treats JPEG and MJPEG the same.

It looks like JPEG and MJPEG are randomly used and I don't think you can assume
that one will have a huffman table and not the other.

Regards,

	Hans

> 
>>>
>>> We also need to update the V4L2_PIX_FMT_JPEG documentation since it just says
>>> TBD:
>>>
>>> https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compressed.html
>>>
>>> $ git grep -l V4L2_PIX_FMT_MJPEG
>>> drivers/media/pci/meye/meye.c
>>> drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c
>>> drivers/media/platform/sti/delta/delta-cfg.h
>>> drivers/media/platform/sti/delta/delta-mjpeg-dec.c
>>> drivers/media/usb/cpia2/cpia2_v4l.c
>>> drivers/media/usb/go7007/go7007-driver.c
>>> drivers/media/usb/go7007/go7007-fw.c
>>> drivers/media/usb/go7007/go7007-v4l2.c
>>> drivers/media/usb/s2255/s2255drv.c
>>> drivers/media/usb/uvc/uvc_driver.c
>>> drivers/staging/media/zoran/zoran_driver.c
>>> drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
>>> drivers/usb/gadget/function/uvc_v4l2.c
>>>
>>> It looks like s2255 and cpia2 support both already, so that would leave
>>> 8 drivers that need to be modified, uvc being the most important of the
>>> lot.
>>>
>>> Any comments?
>>>
>>> Regards,
>>>
>>> 	Hans
>>
>>
> 

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Ezequiel Garcia]

Hi Hans,

Thanks for looking into. I remember MJPEG vs. JPEG being a source
of confusion for me a few years ago, so clarification is greatly
welcome :-)

On Mon, 2018-10-01 at 15:03 +0300, Laurent Pinchart wrote:
> Hi Hans,
> 
> On Monday, 1 October 2018 14:54:29 EEST Hans Verkuil wrote:
> > On 10/01/2018 01:48 PM, Laurent Pinchart wrote:
> > > On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote:
> > > > It turns out that we have both JPEG and Motion-JPEG pixel formats
> > > > defined.
> > > > 
> > > > Furthermore, some drivers support one, some the other and some both.
> > > > 
> > > > These pixelformats both mean the same.
> > > 
> > > Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were
> > > not included in the JPEG headers.
> > 
> > I'm not aware of any difference. If there is one, then it is certainly not
> > documented.
> 
> What I can tell for sure is that many UVC devices don't include Huffman tables 
> in their JPEG headers.
> 
> > Ezequiel, since you've been working with this recently, do you know anything
> > about this?
> 
> 

JPEG frames must include huffman and quantization tables, as per the standard.

AFAIK, there's no MJPEG specification per-se and vendors specify its own
way of conveying a Motion JPEG stream.

For instance, omiting the huffman table seems to be a vendor thing. Microsoft
explicitly omits the huffman tables from each frame:

https://www.fileformat.info/format/bmp/spec/b7c72ebab8064da48ae5ed0c053c67a4/view.htm

Others could be following the same things.

Like I mentioned before, Gstreamer always check for missing huffman table
and adds one if missing. Gstreamer has other quirks for missing markers,
e.g. deal with a missing EOI:

https://github.com/GStreamer/gst-plugins-good/commit/10ff3c8e14e8fba9e0a5d696dce0bea27de644d7

I think Hans suggestion of settling on JPEG makes sense and it would
be consistent with Gstreamer. Otherwise, we should specify exactly what we
understand by MJPEG, but I don't think it's worth it.

Thanks,
Ezequiel

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Laurent Pinchart]

Hi Hans,

On Monday, 1 October 2018 19:28:58 EEST Hans Verkuil wrote:
> On 10/01/2018 06:12 PM, Ezequiel Garcia wrote:
> > On Mon, 2018-10-01 at 08:42 -0400, Nicolas Dufresne wrote:
> >> Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit :
> >>> It turns out that we have both JPEG and Motion-JPEG pixel formats
> >>> defined.
> >>> 
> >>> Furthermore, some drivers support one, some the other and some both.
> >>> 
> >>> These pixelformats both mean the same.
> >>> 
> >>> I propose that we settle on JPEG (since it seems to be used most often)
> >>> and add JPEG support to those drivers that currently only use MJPEG.
> >> 
> >> Thanks for looking into this. As per GStreamer code, I see 3 alias for
> >> JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code
> >> was written before I knew GStreamer existed. It's possible there is a
> >> subtle difference, I have never looked at it, but clearly all our JPEG
> >> decoder handle these as being the same.
> >> 
> >> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gst
> >> v4l2object.c#n956
> > 
> > To add more data points on the gstreamer side, there's really no
> > difference between gstreamer's types image/jpeg and video/x-jpeg.
> > 
> > Notably, jpegdec element just stuffs a huffman table if one is missing,
> > for any jpeg:
> > 
> > https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/ext/jpeg/gstj
> > pegdec.c#n584
> 
> lib/libv4lconvert/libv4lconvert.c also treats JPEG and MJPEG the same.
> 
> It looks like JPEG and MJPEG are randomly used and I don't think you can
> assume that one will have a huffman table and not the other.

That at least should be fixed. If we decide that whether the frames will 
contain a Huffman table or not is useful information for userspace, then we 
should convey it, either through the current mechanism (JPEG vs. MJPEG) or 
through a different mechanism. Otherwise, we can merge JPEG and MJPEG (as long 
as it doesn't break userspace).

-- 
Regards,

Laurent Pinchart

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Dave Stevenson]

Hi All,

On Mon, 1 Oct 2018 at 17:32, Ezequiel Garcia <ezequiel@collabora.com> wrote:
>
> Hi Hans,
>
> Thanks for looking into. I remember MJPEG vs. JPEG being a source
> of confusion for me a few years ago, so clarification is greatly
> welcome :-)
>
> On Mon, 2018-10-01 at 15:03 +0300, Laurent Pinchart wrote:
> > Hi Hans,
> >
> > On Monday, 1 October 2018 14:54:29 EEST Hans Verkuil wrote:
> > > On 10/01/2018 01:48 PM, Laurent Pinchart wrote:
> > > > On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote:
> > > > > It turns out that we have both JPEG and Motion-JPEG pixel formats
> > > > > defined.
> > > > >
> > > > > Furthermore, some drivers support one, some the other and some both.
> > > > >
> > > > > These pixelformats both mean the same.
> > > >
> > > > Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were
> > > > not included in the JPEG headers.
> > >
> > > I'm not aware of any difference. If there is one, then it is certainly not
> > > documented.
> >
> > What I can tell for sure is that many UVC devices don't include Huffman tables
> > in their JPEG headers.
> >
> > > Ezequiel, since you've been working with this recently, do you know anything
> > > about this?
> >
> >
>
> JPEG frames must include huffman and quantization tables, as per the standard.
>
> AFAIK, there's no MJPEG specification per-se and vendors specify its own
> way of conveying a Motion JPEG stream.

There is the specfication for MJPEG in Quicktime containers, which
defines the MJPEG-A and MJPEG-B variants [1].
MJPEG-B is not a concatenation of JPEG frames as the framing is
different, so can't really be combined into V4L2_PIX_FMT_JPEG.
Have people encountered devices that produce MJPEG-A or MJPEG-B via
V4L2? I haven't, but I have been forced to support both variants on
decode.

On that thought, whilst capture devices generally don't care, is there
a need to differentiate for M2M codec devices which can support
encoding the variants? Or likewise on M2M decoders that support only
JPEG, how do they tell userspace that they don't support MJPEG-A or
MJPEG-B?

  Dave

[1] https://developer.apple.com/standards/qtff-2001.pdf

> For instance, omiting the huffman table seems to be a vendor thing. Microsoft
> explicitly omits the huffman tables from each frame:
>
> https://www.fileformat.info/format/bmp/spec/b7c72ebab8064da48ae5ed0c053c67a4/view.htm
>
> Others could be following the same things.
>
> Like I mentioned before, Gstreamer always check for missing huffman table
> and adds one if missing. Gstreamer has other quirks for missing markers,
> e.g. deal with a missing EOI:
>
> https://github.com/GStreamer/gst-plugins-good/commit/10ff3c8e14e8fba9e0a5d696dce0bea27de644d7
>
> I think Hans suggestion of settling on JPEG makes sense and it would
> be consistent with Gstreamer. Otherwise, we should specify exactly what we
> understand by MJPEG, but I don't think it's worth it.
>
> Thanks,
> Ezequiel

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Mauro Carvalho Chehab]

Em Mon, 01 Oct 2018 08:42:56 -0400
Nicolas Dufresne <nicolas@ndufresne.ca> escreveu:

> Hello Hans,
> 
> Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit :
> > It turns out that we have both JPEG and Motion-JPEG pixel formats defined.
> > 
> > Furthermore, some drivers support one, some the other and some both.
> > 
> > These pixelformats both mean the same.
> > 
> > I propose that we settle on JPEG (since it seems to be used most often) and
> > add JPEG support to those drivers that currently only use MJPEG.  
> 
> Thanks for looking into this. As per GStreamer code, I see 3 alias for
> JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code
> was written before I knew GStreamer existed. It's possible there is a
> subtle difference, I have never looked at it, but clearly all our JPEG
> decoder handle these as being the same.
> 
> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2object.c#n956

The code at libv4l handles both MJPEG and JPEG the same way. PJPG is
handled somewhat differently (although it uses the same code). There is a
code there that cleanups some Pixart-specific headers.

Thanks,
Mauro

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Mauro Carvalho Chehab]

Em Mon, 1 Oct 2018 18:19:21 +0100
Dave Stevenson <dave.stevenson@raspberrypi.org> escreveu:

> Hi All,
> 
> On Mon, 1 Oct 2018 at 17:32, Ezequiel Garcia <ezequiel@collabora.com> wrote:
> >
> > Hi Hans,
> >
> > Thanks for looking into. I remember MJPEG vs. JPEG being a source
> > of confusion for me a few years ago, so clarification is greatly
> > welcome :-)
> >
> > On Mon, 2018-10-01 at 15:03 +0300, Laurent Pinchart wrote:  
> > > Hi Hans,
> > >
> > > On Monday, 1 October 2018 14:54:29 EEST Hans Verkuil wrote:  
> > > > On 10/01/2018 01:48 PM, Laurent Pinchart wrote:  
> > > > > On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote:  
> > > > > > It turns out that we have both JPEG and Motion-JPEG pixel formats
> > > > > > defined.
> > > > > >
> > > > > > Furthermore, some drivers support one, some the other and some both.
> > > > > >
> > > > > > These pixelformats both mean the same.  
> > > > >
> > > > > Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were
> > > > > not included in the JPEG headers.  
> > > >
> > > > I'm not aware of any difference. If there is one, then it is certainly not
> > > > documented.  
> > >
> > > What I can tell for sure is that many UVC devices don't include Huffman tables
> > > in their JPEG headers.
> > >  
> > > > Ezequiel, since you've been working with this recently, do you know anything
> > > > about this?  
> > >
> > >  
> >
> > JPEG frames must include huffman and quantization tables, as per the standard.
> >
> > AFAIK, there's no MJPEG specification per-se and vendors specify its own
> > way of conveying a Motion JPEG stream.  
> 
> There is the specfication for MJPEG in Quicktime containers, which
> defines the MJPEG-A and MJPEG-B variants [1].
> MJPEG-B is not a concatenation of JPEG frames as the framing is
> different, so can't really be combined into V4L2_PIX_FMT_JPEG.
> Have people encountered devices that produce MJPEG-A or MJPEG-B via
> V4L2? I haven't, but I have been forced to support both variants on
> decode.

Checking it is not an easy task. I *suspect* that those cameras are all
MJPEG-A, as the libv4l decoder uses tinyjpeg library to handle both
JPEG and MJPEG.

Maybe Hans de Goede knows more about that, and may have actually tested
it with different camera models.

> 
> On that thought, whilst capture devices generally don't care, is there
> a need to differentiate for M2M codec devices which can support
> encoding the variants? Or likewise on M2M decoders that support only
> JPEG, how do they tell userspace that they don't support MJPEG-A or
> MJPEG-B?
> 
>   Dave
> 
> [1] https://developer.apple.com/standards/qtff-2001.pdf
> 
> > For instance, omiting the huffman table seems to be a vendor thing. Microsoft
> > explicitly omits the huffman tables from each frame:
> >
> > https://www.fileformat.info/format/bmp/spec/b7c72ebab8064da48ae5ed0c053c67a4/view.htm
> >
> > Others could be following the same things.
> >
> > Like I mentioned before, Gstreamer always check for missing huffman table
> > and adds one if missing. Gstreamer has other quirks for missing markers,
> > e.g. deal with a missing EOI:
> >
> > https://github.com/GStreamer/gst-plugins-good/commit/10ff3c8e14e8fba9e0a5d696dce0bea27de644d7
> >
> > I think Hans suggestion of settling on JPEG makes sense and it would
> > be consistent with Gstreamer. Otherwise, we should specify exactly what we
> > understand by MJPEG, but I don't think it's worth it.
> >
> > Thanks,
> > Ezequiel  



Thanks,
Mauro

Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG

[Hans de Goede]

Hi,

On 05-10-18 13:55, Mauro Carvalho Chehab wrote:
> Em Mon, 1 Oct 2018 18:19:21 +0100
> Dave Stevenson <dave.stevenson@raspberrypi.org> escreveu:
> 
>> Hi All,
>>
>> On Mon, 1 Oct 2018 at 17:32, Ezequiel Garcia <ezequiel@collabora.com> wrote:
>>>
>>> Hi Hans,
>>>
>>> Thanks for looking into. I remember MJPEG vs. JPEG being a source
>>> of confusion for me a few years ago, so clarification is greatly
>>> welcome :-)
>>>
>>> On Mon, 2018-10-01 at 15:03 +0300, Laurent Pinchart wrote:
>>>> Hi Hans,
>>>>
>>>> On Monday, 1 October 2018 14:54:29 EEST Hans Verkuil wrote:
>>>>> On 10/01/2018 01:48 PM, Laurent Pinchart wrote:
>>>>>> On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote:
>>>>>>> It turns out that we have both JPEG and Motion-JPEG pixel formats
>>>>>>> defined.
>>>>>>>
>>>>>>> Furthermore, some drivers support one, some the other and some both.
>>>>>>>
>>>>>>> These pixelformats both mean the same.
>>>>>>
>>>>>> Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were
>>>>>> not included in the JPEG headers.
>>>>>
>>>>> I'm not aware of any difference. If there is one, then it is certainly not
>>>>> documented.
>>>>
>>>> What I can tell for sure is that many UVC devices don't include Huffman tables
>>>> in their JPEG headers.
>>>>   
>>>>> Ezequiel, since you've been working with this recently, do you know anything
>>>>> about this?
>>>>
>>>>   
>>>
>>> JPEG frames must include huffman and quantization tables, as per the standard.
>>>
>>> AFAIK, there's no MJPEG specification per-se and vendors specify its own
>>> way of conveying a Motion JPEG stream.
>>
>> There is the specfication for MJPEG in Quicktime containers, which
>> defines the MJPEG-A and MJPEG-B variants [1].
>> MJPEG-B is not a concatenation of JPEG frames as the framing is
>> different, so can't really be combined into V4L2_PIX_FMT_JPEG.
>> Have people encountered devices that produce MJPEG-A or MJPEG-B via
>> V4L2? I haven't, but I have been forced to support both variants on
>> decode.
> 
> Checking it is not an easy task. I *suspect* that those cameras are all
> MJPEG-A, as the libv4l decoder uses tinyjpeg library to handle both
> JPEG and MJPEG.
> 
> Maybe Hans de Goede knows more about that, and may have actually tested
> it with different camera models.

I've tested the JPG path in libv4l with quite a lot of cameras and
sofar it has worked for all of them. There are some non UVC cameras where
the hardware produces raw JPG data, but in that case the kernel driver
prefixes a JPG header to each frame so that it looks like a regular JPG.

Regards,

Hans



> 
>>
>> On that thought, whilst capture devices generally don't care, is there
>> a need to differentiate for M2M codec devices which can support
>> encoding the variants? Or likewise on M2M decoders that support only
>> JPEG, how do they tell userspace that they don't support MJPEG-A or
>> MJPEG-B?
>>
>>    Dave
>>
>> [1] https://developer.apple.com/standards/qtff-2001.pdf
>>
>>> For instance, omiting the huffman table seems to be a vendor thing. Microsoft
>>> explicitly omits the huffman tables from each frame:
>>>
>>> https://www.fileformat.info/format/bmp/spec/b7c72ebab8064da48ae5ed0c053c67a4/view.htm
>>>
>>> Others could be following the same things.
>>>
>>> Like I mentioned before, Gstreamer always check for missing huffman table
>>> and adds one if missing. Gstreamer has other quirks for missing markers,
>>> e.g. deal with a missing EOI:
>>>
>>> https://github.com/GStreamer/gst-plugins-good/commit/10ff3c8e14e8fba9e0a5d696dce0bea27de644d7
>>>
>>> I think Hans suggestion of settling on JPEG makes sense and it would
>>> be consistent with Gstreamer. Otherwise, we should specify exactly what we
>>> understand by MJPEG, but I don't think it's worth it.
>>>
>>> Thanks,
>>> Ezequiel
> 
> 
> 
> Thanks,
> Mauro
>