Re: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension

Re: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension

[Maarten Lankhorst]

Hey,

Bit late reply, hopefully not too late.

Op 30-08-12 16:00, Tom Cooksey schreef:
> Hi All,
>
> Over the last few months I've been working on & off with a few people from
> Linaro on a new EGL extension. The extension allows constructing an EGLImage
> from a (set of) dma_buf file descriptors, including support for multi-plane
> YUV. I envisage the primary use-case of this extension to be importing video
> frames from v4l2 into the EGL/GLES graphics driver to texture from.
> Originally the intent was to develop this as a Khronos-ratified extension.
> However, this is a little too platform-specific to be an officially
> sanctioned Khronos extension. It also goes against the general "EGLStream"
> direction the EGL working group is going in. As such, the general feeling
> was to make this an EXT "multi-vendor" extension with no official stamp of
> approval from Khronos. As this is no-longer intended to be a Khronos
> extension, I've re-written it to be a lot more Linux & dma_buf specific. It
> also allows me to circulate the extension more widely (I.e. To those outside
> Khronos membership).
>
> ARM are implementing this extension for at least our Mali-T6xx driver and
> likely earlier drivers too. I am sending this e-mail to solicit feedback,
> both from other vendors who might implement this extension (Mesa3D?) and
> from potential users of the extension. However, any feedback is welcome.
> Please find the extension text as it currently stands below. There several
> open issues which I've proposed solutions for, but I'm not really happy with
> those proposals and hoped others could chip-in with better ideas. There are
> likely other issues I've not thought about which also need to be added and
> addressed.
>
> Once there's a general consensus or if no-one's interested, I'll update the
> spec, move it out of Draft status and get it added to the Khronos registry,
> which includes assigning values for the new symbols.
>
>
> Cheers,
>
> Tom
>
>
> ---------8<---------
>
>
> Name
>
>     EXT_image_dma_buf_import
>
> Name Strings
>
>     EGL_EXT_image_dma_buf_import
>
> Contributors
>
>     Jesse Barker
>     Rob Clark
>     Tom Cooksey
>
> Contacts
>
>     Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
>     Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)
>
> Status
>
>     DRAFT
>
> Version
>
>     Version 3, August 16, 2012
>
> Number
>
>     EGL Extension ???
>
> Dependencies
>
>     EGL 1.2 is required.
>
>     EGL_KHR_image_base is required.
>
>     The EGL implementation must be running on a Linux kernel supporting the
>     dma_buf buffer sharing mechanism.
>
>     This extension is written against the wording of the EGL 1.2
> Specification.
>
> Overview
>
>     This extension allows creating an EGLImage from a Linux dma_buf file
>     descriptor or multiple file descriptors in the case of multi-plane YUV
>     images.
>
> New Types
>
>     None
>
> New Procedures and Functions
>
>     None
>
> New Tokens
>
>     Accepted by the <target> parameter of eglCreateImageKHR:
>
>         EGL_LINUX_DMA_BUF_EXT
>
>     Accepted as an attribute in the <attrib_list> parameter of
>     eglCreateImageKHR:
>
>         EGL_LINUX_DRM_FOURCC_EXT
>         EGL_DMA_BUF_PLANE0_FD_EXT
>         EGL_DMA_BUF_PLANE0_OFFSET_EXT
>         EGL_DMA_BUF_PLANE0_PITCH_EXT
>         EGL_DMA_BUF_PLANE1_FD_EXT
>         EGL_DMA_BUF_PLANE1_OFFSET_EXT
>         EGL_DMA_BUF_PLANE1_PITCH_EXT
>         EGL_DMA_BUF_PLANE2_FD_EXT
>         EGL_DMA_BUF_PLANE2_OFFSET_EXT
>         EGL_DMA_BUF_PLANE2_PITCH_EXT
You might want to add PLANE3 just in case someone wants to import a AYUV image.
> Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)
>
>     Add to section 2.5.1 "EGLImage Specification" (as defined by the
>     EGL_KHR_image_base specification), in the description of
>     eglCreateImageKHR:
>
>    "Values accepted for <target> are listed in Table aaa, below.
>
>  
> +-------------------------+--------------------------------------------+
>       |  <target>               |  Notes
> |
>  
> +-------------------------+--------------------------------------------+
>       |  EGL_LINUX_DMA_BUF_EXT  |   Used for EGLImages imported from Linux
> |
>       |                         |   dma_buf file descriptors
> |
>  
> +-------------------------+--------------------------------------------+
>        Table aaa.  Legal values for eglCreateImageKHR <target> parameter
>
>     ...
>
>     If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display,
> <ctx>
>     must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
>     EGLClientBuffer. The details of the image is specified by the attributes
>     passed into eglCreateImageKHR. Required attributes and their values are
> as
>     follows:
>
>         * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in
> pixels
>
>         * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as
> specified
>           by drm_fourcc.h and used as the pixel_format parameter of the
>           drm_mode_fb_cmd2 ioctl.
>
>         * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0
> of
>           the image.
>
>         * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
>           dma_buf of the first sample in plane 0, in bytes.
>  
>         * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the
> start of
>           subsequent rows of samples in plane 0. May have special meaning
> for
>           non-linear formats.
>
>     For images in an RGB color-space or those using a single-plane YUV
> format,
>     only the first plane's file descriptor, offset & pitch should be
> specified.
>     For semi-planar YUV formats, the chroma samples are stored in plane 1
> and
>     for fully planar formats, U-samples are stored in plane 1 and V-samples
> are
>     stored in plane 2. Planes 1 & 2 are specified by the following
> attributes,
>     which have the same meanings as defined above for plane 0:
>
Nitpick, Y'CbCr not YUV.

How do you want to deal with the case where Y' and CbCr are different hardware buffers?
Could some support for 2d arrays be added in case Y' and CbCr are separated into top/bottom fields?
How are semi-planar/planar formats handled that have a different width/height for Y' and CbCr? (YUV420)

~Maarten

Re: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension

[Tom Cooksey]

Hi Maarten,

Thanks for taking a look at this! Responses in-line...


Cheers,

Tom


> -----Original Message-----
> From: Maarten Lankhorst [mailto:m.b.lankhorst@gmail.com]
> Sent: 02 October 2012 13:10
> To: Tom Cooksey
> Cc: mesa-dev@lists.freedesktop.org; linaro-mm-sig@lists.linaro.org; dri-
> devel@lists.freedesktop.org; 'Jesse Barker'; linux-media@vger.kernel.org
> Subject: Re: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension
> 
> Hey,
> 
> Bit late reply, hopefully not too late.
> 
> Op 30-08-12 16:00, Tom Cooksey schreef:
> > Hi All,
> >
> > Over the last few months I've been working on & off with a few people from
> > Linaro on a new EGL extension. The extension allows constructing an EGLImage
> > from a (set of) dma_buf file descriptors, including support for multi-plane
> > YUV. I envisage the primary use-case of this extension to be importing video
> > frames from v4l2 into the EGL/GLES graphics driver to texture from.
> > Originally the intent was to develop this as a Khronos-ratified extension.
> > However, this is a little too platform-specific to be an officially
> > sanctioned Khronos extension. It also goes against the general "EGLStream"
> > direction the EGL working group is going in. As such, the general feeling
> > was to make this an EXT "multi-vendor" extension with no official stamp of
> > approval from Khronos. As this is no-longer intended to be a Khronos
> > extension, I've re-written it to be a lot more Linux & dma_buf specific. It
> > also allows me to circulate the extension more widely (I.e. To those outside
> > Khronos membership).
> >
> > ARM are implementing this extension for at least our Mali-T6xx driver and
> > likely earlier drivers too. I am sending this e-mail to solicit feedback,
> > both from other vendors who might implement this extension (Mesa3D?) and
> > from potential users of the extension. However, any feedback is welcome.
> > Please find the extension text as it currently stands below. There several
> > open issues which I've proposed solutions for, but I'm not really happy with
> > those proposals and hoped others could chip-in with better ideas. There are
> > likely other issues I've not thought about which also need to be added and
> > addressed.
> >
> > Once there's a general consensus or if no-one's interested, I'll update the
> > spec, move it out of Draft status and get it added to the Khronos registry,
> > which includes assigning values for the new symbols.
> >
> >
> > Cheers,
> >
> > Tom
> >
> >
> > ---------8<---------
> >
> >
> > Name
> >
> >     EXT_image_dma_buf_import
> >
> > Name Strings
> >
> >     EGL_EXT_image_dma_buf_import
> >
> > Contributors
> >
> >     Jesse Barker
> >     Rob Clark
> >     Tom Cooksey
> >
> > Contacts
> >
> >     Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
> >     Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)
> >
> > Status
> >
> >     DRAFT
> >
> > Version
> >
> >     Version 3, August 16, 2012
> >
> > Number
> >
> >     EGL Extension ???
> >
> > Dependencies
> >
> >     EGL 1.2 is required.
> >
> >     EGL_KHR_image_base is required.
> >
> >     The EGL implementation must be running on a Linux kernel supporting the
> >     dma_buf buffer sharing mechanism.
> >
> >     This extension is written against the wording of the EGL 1.2
> > Specification.
> >
> > Overview
> >
> >     This extension allows creating an EGLImage from a Linux dma_buf file
> >     descriptor or multiple file descriptors in the case of multi-plane YUV
> >     images.
> >
> > New Types
> >
> >     None
> >
> > New Procedures and Functions
> >
> >     None
> >
> > New Tokens
> >
> >     Accepted by the <target> parameter of eglCreateImageKHR:
> >
> >         EGL_LINUX_DMA_BUF_EXT
> >
> >     Accepted as an attribute in the <attrib_list> parameter of
> >     eglCreateImageKHR:
> >
> >         EGL_LINUX_DRM_FOURCC_EXT
> >         EGL_DMA_BUF_PLANE0_FD_EXT
> >         EGL_DMA_BUF_PLANE0_OFFSET_EXT
> >         EGL_DMA_BUF_PLANE0_PITCH_EXT
> >         EGL_DMA_BUF_PLANE1_FD_EXT
> >         EGL_DMA_BUF_PLANE1_OFFSET_EXT
> >         EGL_DMA_BUF_PLANE1_PITCH_EXT
> >         EGL_DMA_BUF_PLANE2_FD_EXT
> >         EGL_DMA_BUF_PLANE2_OFFSET_EXT
> >         EGL_DMA_BUF_PLANE2_PITCH_EXT
>
> You might want to add PLANE3 just in case someone wants to import a AYUV image.

At least on fourcc.org and in drm_fourcc.h, there only seems to be a single AYUV format
and that is a packed format, so everything would be in the first plane. I don't think
it's worth adding a PLANE3 just in case someone invents a new planar AYUV format?



> > Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)
> >
> >     Add to section 2.5.1 "EGLImage Specification" (as defined by the
> >     EGL_KHR_image_base specification), in the description of
> >     eglCreateImageKHR:
> >
> >    "Values accepted for <target> are listed in Table aaa, below.
> >
> >
> > +-------------------------+--------------------------------------------+
> >       |  <target>               |  Notes
> > |
> >
> > +-------------------------+--------------------------------------------+
> >       |  EGL_LINUX_DMA_BUF_EXT  |   Used for EGLImages imported from Linux
> > |
> >       |                         |   dma_buf file descriptors
> > |
> >
> > +-------------------------+--------------------------------------------+
> >        Table aaa.  Legal values for eglCreateImageKHR <target> parameter
> >
> >     ...
> >
> >     If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display,
> > <ctx>
> >     must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
> >     EGLClientBuffer. The details of the image is specified by the attributes
> >     passed into eglCreateImageKHR. Required attributes and their values are
> > as
> >     follows:
> >
> >         * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in
> > pixels
> >
> >         * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as
> > specified
> >           by drm_fourcc.h and used as the pixel_format parameter of the
> >           drm_mode_fb_cmd2 ioctl.
> >
> >         * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0
> > of
> >           the image.
> >
> >         * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
> >           dma_buf of the first sample in plane 0, in bytes.
> >
> >         * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the
> > start of
> >           subsequent rows of samples in plane 0. May have special meaning
> > for
> >           non-linear formats.
> >
> >     For images in an RGB color-space or those using a single-plane YUV
> > format,
> >     only the first plane's file descriptor, offset & pitch should be
> > specified.
> >     For semi-planar YUV formats, the chroma samples are stored in plane 1
> > and
> >     for fully planar formats, U-samples are stored in plane 1 and V-samples
> > are
> >     stored in plane 2. Planes 1 & 2 are specified by the following
> > attributes,
> >     which have the same meanings as defined above for plane 0:
> >
> Nitpick, Y'CbCr not YUV.

Fair enough. :-)


> How do you want to deal with the case where Y' and CbCr are different hardware buffers?

I'm not sure I completely understand your question? Do you mean semi-planar formats like
NV12? For those formats, Y-samples would be stored in the first plane and YV/CbCr samples
stored in the second plane. From a dma_buf point of view, each plane can be a completely
different buffer with its own dma_buf fd. Is that what you meant?


> Could some support for 2d arrays be added in case Y' and CbCr are separated into top/bottom
> fields?

When you say top/bottom fields, I assume you're talking about interlaced content? From
what I have seen, interlaced frames are usually stored with the top & bottom fields
interleaved in a single buffer. As the fields would need to be displayed as at different
times, I would assume the application would create two EGLImages, one for the top field
and another for the bottom. That can be handled pretty easily by setting the pitch to 2x
the buffer width and adding 1 scanline's worth of offset for the bottom field's EGLImage.
This should work regardless of weather the data is packed in a single plane, semi-planar
or multi-planar. If each field is stored in a separate buffer then it should be trivial to
create two EGLImages, one for each buffer.



> How are semi-planar/planar formats handled that have a different width/height for Y' and CbCr?
> (YUV420)

The spec says EGL_WIDTH & EGL_HEIGHT specify the *logical* width and height of the buffer
in pixels. For pixel formats with sub-sampled Chroma values, it should be trivial for the
EGL implementation to calculate the width/height of the Chroma sample buffers using the
logical width & height and by inspecting the pixel format passed as the
EGL_LINUX_DRM_FOURCC_EXT attribute. I.e. If the pixel format says it's YUV420, the Chroma
buffer's width = EGL_WIDTH/2 & height =EGL_HEIGHT/2.


Maartin - do you have any opinions regarding the open issues in the spec? Specifically,
issues 4, 5 & 6? I am having second thoughts about leaving the BT.709/BT.601 color space
and Chroma-siting undefined - while they don't need to be specified when negotiating how
the data is stored in memory, they are kinda needed when you come to interpret the data,
which a GPU's texture unit is likely to do if it's been imported into EGL.



Cheers,

Tom

RE: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension

[Tom Cooksey]

Hi Maarten,

Thanks for taking a look at this! Responses in-line...


Cheers,

Tom


> -----Original Message-----
> From: Maarten Lankhorst [mailto:m.b.lankhorst@gmail.com]
> Sent: 02 October 2012 13:10
> To: Tom Cooksey
> Cc: mesa-dev@lists.freedesktop.org; linaro-mm-sig@lists.linaro.org; dri-
> devel@lists.freedesktop.org; 'Jesse Barker'; linux-media@vger.kernel.org
> Subject: Re: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension
> 
> Hey,
> 
> Bit late reply, hopefully not too late.
> 
> Op 30-08-12 16:00, Tom Cooksey schreef:
> > Hi All,
> >
> > Over the last few months I've been working on & off with a few people from
> > Linaro on a new EGL extension. The extension allows constructing an EGLImage
> > from a (set of) dma_buf file descriptors, including support for multi-plane
> > YUV. I envisage the primary use-case of this extension to be importing video
> > frames from v4l2 into the EGL/GLES graphics driver to texture from.
> > Originally the intent was to develop this as a Khronos-ratified extension.
> > However, this is a little too platform-specific to be an officially
> > sanctioned Khronos extension. It also goes against the general "EGLStream"
> > direction the EGL working group is going in. As such, the general feeling
> > was to make this an EXT "multi-vendor" extension with no official stamp of
> > approval from Khronos. As this is no-longer intended to be a Khronos
> > extension, I've re-written it to be a lot more Linux & dma_buf specific. It
> > also allows me to circulate the extension more widely (I.e. To those outside
> > Khronos membership).
> >
> > ARM are implementing this extension for at least our Mali-T6xx driver and
> > likely earlier drivers too. I am sending this e-mail to solicit feedback,
> > both from other vendors who might implement this extension (Mesa3D?) and
> > from potential users of the extension. However, any feedback is welcome.
> > Please find the extension text as it currently stands below. There several
> > open issues which I've proposed solutions for, but I'm not really happy with
> > those proposals and hoped others could chip-in with better ideas. There are
> > likely other issues I've not thought about which also need to be added and
> > addressed.
> >
> > Once there's a general consensus or if no-one's interested, I'll update the
> > spec, move it out of Draft status and get it added to the Khronos registry,
> > which includes assigning values for the new symbols.
> >
> >
> > Cheers,
> >
> > Tom
> >
> >
> > ---------8<---------
> >
> >
> > Name
> >
> >     EXT_image_dma_buf_import
> >
> > Name Strings
> >
> >     EGL_EXT_image_dma_buf_import
> >
> > Contributors
> >
> >     Jesse Barker
> >     Rob Clark
> >     Tom Cooksey
> >
> > Contacts
> >
> >     Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
> >     Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)
> >
> > Status
> >
> >     DRAFT
> >
> > Version
> >
> >     Version 3, August 16, 2012
> >
> > Number
> >
> >     EGL Extension ???
> >
> > Dependencies
> >
> >     EGL 1.2 is required.
> >
> >     EGL_KHR_image_base is required.
> >
> >     The EGL implementation must be running on a Linux kernel supporting the
> >     dma_buf buffer sharing mechanism.
> >
> >     This extension is written against the wording of the EGL 1.2
> > Specification.
> >
> > Overview
> >
> >     This extension allows creating an EGLImage from a Linux dma_buf file
> >     descriptor or multiple file descriptors in the case of multi-plane YUV
> >     images.
> >
> > New Types
> >
> >     None
> >
> > New Procedures and Functions
> >
> >     None
> >
> > New Tokens
> >
> >     Accepted by the <target> parameter of eglCreateImageKHR:
> >
> >         EGL_LINUX_DMA_BUF_EXT
> >
> >     Accepted as an attribute in the <attrib_list> parameter of
> >     eglCreateImageKHR:
> >
> >         EGL_LINUX_DRM_FOURCC_EXT
> >         EGL_DMA_BUF_PLANE0_FD_EXT
> >         EGL_DMA_BUF_PLANE0_OFFSET_EXT
> >         EGL_DMA_BUF_PLANE0_PITCH_EXT
> >         EGL_DMA_BUF_PLANE1_FD_EXT
> >         EGL_DMA_BUF_PLANE1_OFFSET_EXT
> >         EGL_DMA_BUF_PLANE1_PITCH_EXT
> >         EGL_DMA_BUF_PLANE2_FD_EXT
> >         EGL_DMA_BUF_PLANE2_OFFSET_EXT
> >         EGL_DMA_BUF_PLANE2_PITCH_EXT
>
> You might want to add PLANE3 just in case someone wants to import a AYUV image.

At least on fourcc.org and in drm_fourcc.h, there only seems to be a single AYUV format
and that is a packed format, so everything would be in the first plane. I don't think
it's worth adding a PLANE3 just in case someone invents a new planar AYUV format?



> > Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)
> >
> >     Add to section 2.5.1 "EGLImage Specification" (as defined by the
> >     EGL_KHR_image_base specification), in the description of
> >     eglCreateImageKHR:
> >
> >    "Values accepted for <target> are listed in Table aaa, below.
> >
> >
> > +-------------------------+--------------------------------------------+
> >       |  <target>               |  Notes
> > |
> >
> > +-------------------------+--------------------------------------------+
> >       |  EGL_LINUX_DMA_BUF_EXT  |   Used for EGLImages imported from Linux
> > |
> >       |                         |   dma_buf file descriptors
> > |
> >
> > +-------------------------+--------------------------------------------+
> >        Table aaa.  Legal values for eglCreateImageKHR <target> parameter
> >
> >     ...
> >
> >     If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display,
> > <ctx>
> >     must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
> >     EGLClientBuffer. The details of the image is specified by the attributes
> >     passed into eglCreateImageKHR. Required attributes and their values are
> > as
> >     follows:
> >
> >         * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in
> > pixels
> >
> >         * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as
> > specified
> >           by drm_fourcc.h and used as the pixel_format parameter of the
> >           drm_mode_fb_cmd2 ioctl.
> >
> >         * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0
> > of
> >           the image.
> >
> >         * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
> >           dma_buf of the first sample in plane 0, in bytes.
> >
> >         * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the
> > start of
> >           subsequent rows of samples in plane 0. May have special meaning
> > for
> >           non-linear formats.
> >
> >     For images in an RGB color-space or those using a single-plane YUV
> > format,
> >     only the first plane's file descriptor, offset & pitch should be
> > specified.
> >     For semi-planar YUV formats, the chroma samples are stored in plane 1
> > and
> >     for fully planar formats, U-samples are stored in plane 1 and V-samples
> > are
> >     stored in plane 2. Planes 1 & 2 are specified by the following
> > attributes,
> >     which have the same meanings as defined above for plane 0:
> >
> Nitpick, Y'CbCr not YUV.

Fair enough. :-)


> How do you want to deal with the case where Y' and CbCr are different hardware buffers?

I'm not sure I completely understand your question? Do you mean semi-planar formats like
NV12? For those formats, Y-samples would be stored in the first plane and YV/CbCr samples
stored in the second plane. From a dma_buf point of view, each plane can be a completely
different buffer with its own dma_buf fd. Is that what you meant?


> Could some support for 2d arrays be added in case Y' and CbCr are separated into top/bottom
> fields?

When you say top/bottom fields, I assume you're talking about interlaced content? From
what I have seen, interlaced frames are usually stored with the top & bottom fields
interleaved in a single buffer. As the fields would need to be displayed as at different
times, I would assume the application would create two EGLImages, one for the top field
and another for the bottom. That can be handled pretty easily by setting the pitch to 2x
the buffer width and adding 1 scanline's worth of offset for the bottom field's EGLImage.
This should work regardless of weather the data is packed in a single plane, semi-planar
or multi-planar. If each field is stored in a separate buffer then it should be trivial to
create two EGLImages, one for each buffer.



> How are semi-planar/planar formats handled that have a different width/height for Y' and CbCr?
> (YUV420)

The spec says EGL_WIDTH & EGL_HEIGHT specify the *logical* width and height of the buffer
in pixels. For pixel formats with sub-sampled Chroma values, it should be trivial for the
EGL implementation to calculate the width/height of the Chroma sample buffers using the
logical width & height and by inspecting the pixel format passed as the
EGL_LINUX_DRM_FOURCC_EXT attribute. I.e. If the pixel format says it's YUV420, the Chroma
buffer's width = EGL_WIDTH/2 & height =EGL_HEIGHT/2.


Maartin - do you have any opinions regarding the open issues in the spec? Specifically,
issues 4, 5 & 6? I am having second thoughts about leaving the BT.709/BT.601 color space
and Chroma-siting undefined - while they don't need to be specified when negotiating how
the data is stored in memory, they are kinda needed when you come to interpret the data,
which a GPU's texture unit is likely to do if it's been imported into EGL.



Cheers,

Tom

Re: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension

[Rob Clark]

On Tue, Oct 2, 2012 at 2:10 PM, Maarten Lankhorst
<m.b.lankhorst@gmail.com> wrote:
> How do you want to deal with the case where Y' and CbCr are different hardware buffers?
> Could some support for 2d arrays be added in case Y' and CbCr are separated into top/bottom fields?
> How are semi-planar/planar formats handled that have a different width/height for Y' and CbCr? (YUV420)

The API works (AFAIU) like drm addfb2 ioctl, take I420 for example,
you could either do:

  single buffer:
     fd0 = fd
     offset0 = 0
     pitch0 = width
     fd1 = fd
     offset1 = width * height
     pitch1 = width / 2
     fd2 = fd
     offset2 = offset1 + (width / height / 4)
     pitch2 = width / 2

  multiple buffers:
     offset0 = offset1 = offset2 = 0
     fd0 = fd_luma
     fd1 = fd_u
     fd2 = fd_v
     ... and so on

for interlaced/stereo.. is sticking our heads in sand an option?  :-P

You could get lots of permutations for data layout of fields between
interlaced and stereo.  One option might be to ignore and let the user
create two egl-images and deal with blending in the shader?

BR,
-R

RE: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension

[Tom Cooksey]

Hi Rob,

> -----Original Message-----
> From: robdclark@gmail.com [mailto:robdclark@gmail.com] On Behalf Of Rob Clark
> Sent: 03 October 2012 13:39
> To: Maarten Lankhorst
> Cc: Tom Cooksey; mesa-dev@lists.freedesktop.org; linaro-mm-sig@lists.linaro.org; dri-
> devel@lists.freedesktop.org; Jesse Barker; linux-media@vger.kernel.org
> Subject: Re: [Linaro-mm-sig] [RFC] New dma_buf -> EGLImage EGL extension
>
> On Tue, Oct 2, 2012 at 2:10 PM, Maarten Lankhorst
> <m.b.lankhorst@gmail.com> wrote:
> > How do you want to deal with the case where Y' and CbCr are different hardware buffers?
> > Could some support for 2d arrays be added in case Y' and CbCr are separated into top/bottom
> fields?
> > How are semi-planar/planar formats handled that have a different width/height for Y' and
> CbCr? (YUV420)
>
> The API works (AFAIU) like drm addfb2 ioctl, take I420 for example,
> you could either do:
>
>   single buffer:
>      fd0 = fd
>      offset0 = 0
>      pitch0 = width
>      fd1 = fd
>      offset1 = width * height
>      pitch1 = width / 2
>      fd2 = fd
>      offset2 = offset1 + (width / height / 4)
>      pitch2 = width / 2
>
>   multiple buffers:
>      offset0 = offset1 = offset2 = 0
>      fd0 = fd_luma
>      fd1 = fd_u
>      fd2 = fd_v
>      ... and so on

Yup, that's pretty much how I'd envisaged it.


> for interlaced/stereo.. is sticking our heads in sand an option?  :-P
>
> You could get lots of permutations for data layout of fields between
> interlaced and stereo.  One option might be to ignore and let the user
> create two egl-images and deal with blending in the shader?

I think for interlaced video the only option really is to create two EGLImages as the two fields have to be displayed at different times. If the application wanted to display them progressively they'd have to run a de-interlacing filter over the two images. Perhaps writing such a filter as a GLSL shader might not be such a bad idea, but it's kinda the app's problem. Same deal with stereo.


Cheers,

Tom

PS: I've updated the spec and sent out a new draft.


-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.