linux-imx-headers and ioctl mismatches

linux-imx-headers and ioctl mismatches

[Carlos Rafael Giani]

The linux-imx-headers recipe gathers certain headers from the kernel 
source so that other recipes which do access these headers don't have to 
mark the entire kernel as a dependency.

However, if a platform doesn't actually use linux-imx as its kernel, or 
at least uses a kernel release version that differs from that of 
linux-imx, ioctl mismatches can occur.

Current example: linux-imx is at version 4.19 , while linux-boundary is 
at 4.14. The DMA_BUF_IOCTL_PHYS ioctl is defined in the uapi/dma-buf.h 
header in 4.14 as:

   #define DMA_BUF_IOCTL_PHYS      _IOW(DMA_BUF_BASE, 1, struct 
dma_buf_phys)

but in 4.19 as:

   #define DMA_BUF_IOCTL_PHYS      _IOW(DMA_BUF_BASE, 10, struct 
dma_buf_phys)

This leads to Boundary Devices platforms that can't handle this ioctl 
properly because of the definition mismatch.

Just mentioning this here to discuss ideas. Perhaps it would make sense 
to create a virtual/imx-headers package, or place the git repo URL as a 
configurable variable?

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Otavio Salvador]

On Fri, Apr 10, 2020 at 7:19 AM Carlos Rafael Giani <crg7475@mailbox.org> wrote:
...
> Just mentioning this here to discuss ideas. Perhaps it would make sense
> to create a virtual/imx-headers package, or place the git repo URL as a
> configurable variable?

This is not an easy problem. The issue is kinda similar of Linux ABI
and we can't use two different versions on same binary feed so we'd
need to mark it all as MACHINE_ARCH.

The easiest solution I can think of is do a backport of this change to
linux-boundary or they bump their kernel version.


-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Carlos Rafael Giani]

Hm I thought there was some sort of imx-kernel base class. Apparently 
there isn't. If there were, it could then be merged with the 
linux-imx-headers recipe, and perhaps create something like a -dev package.

But then again, a -dev package from a kernel recipe ...? Kind of weird.

On 10.04.20 16:12, Otavio Salvador wrote:
> On Fri, Apr 10, 2020 at 7:19 AM Carlos Rafael Giani <crg7475@mailbox.org> wrote:
> ...
>> Just mentioning this here to discuss ideas. Perhaps it would make sense
>> to create a virtual/imx-headers package, or place the git repo URL as a
>> configurable variable?
> This is not an easy problem. The issue is kinda similar of Linux ABI
> and we can't use two different versions on same binary feed so we'd
> need to mark it all as MACHINE_ARCH.
>
> The easiest solution I can think of is do a backport of this change to
> linux-boundary or they bump their kernel version.
>
>

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Otavio Salvador]

On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani
<crg7475@mailbox.org> wrote:
> Hm I thought there was some sort of imx-kernel base class. Apparently
> there isn't. If there were, it could then be merged with the
> linux-imx-headers recipe, and perhaps create something like a -dev package.
>
> But then again, a -dev package from a kernel recipe ...? Kind of weird.

The problem is that you cannot mix packages / apps with different
versions of this. This ends being part of the binary ABI.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Otavio Salvador]

On Fri, Apr 10, 2020 at 11:46 AM Gary Bisson
<gary.bisson@boundarydevices.com> wrote:
> On Fri, Apr 10, 2020 at 11:22:45AM -0300, Otavio Salvador wrote:
> > On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani
> > <crg7475@mailbox.org> wrote:
> > > Hm I thought there was some sort of imx-kernel base class. Apparently
> > > there isn't. If there were, it could then be merged with the
> > > linux-imx-headers recipe, and perhaps create something like a -dev package.
> > >
> > > But then again, a -dev package from a kernel recipe ...? Kind of weird.
> >
> > The problem is that you cannot mix packages / apps with different
> > versions of this. This ends being part of the binary ABI.
>
> I understand it's a tricky topic as some packages won't build if the
> kernel headers are not aligned. But I'm not sure it is better to have
> packages building but having runtime issues because of mismatch...
>
> Since our kernel is also used on other OSes we will not update its
> headers. It also feels wrong to do so.

Add a patch to the layer for it.

> Otavio, do you expect all the vendors to have their kernel matching
> exactly the linux-imx-headers version?
> Because looking at meta-freescale-3rdparty master branch [1], some
> kernels are still based on 3.14...

and it is vendors consideration for their users. If they don't care,
who should care? We are starting removing broken boards and vendors
should pay attention to users.

There are two routes here:

- use mainline BSP
- use NXP BSP

for NXP BSP it comes with the need to keep upgrading it.


> I personnaly think that the headers should be installed from the kernel
> being built, then it's up to the vendor to check the compatibility but
> at least that would fix the issue Carlos is seeing.

This does not help; those headers should not exists in first case but
as we do, a common base need to be used for all distro. Their symbols
goes to other libraries and apps so it cannot change between machines.

> FYI, our next kernel release will be 5.4.3_1.0.0 as it is out since last
> week. So if master stays with 4.19 we'll still have a mismatch I guess.

Will be going to 5.4 soon and people can help, patches welcome.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Kevin Lannen]

Hi Otavio,

I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?

Regards,


Kevin Lannen

Embedded Systems Engineer
kevin@luci.com
970-690-8619



-----Original Message-----
From: meta-freescale@lists.yoctoproject.org <meta-freescale@lists.yoctoproject.org> On Behalf Of Otavio Salvador via lists.yoctoproject.org
Sent: Friday, April 10, 2020 11:52 AM
To: Gary Bisson <gary.bisson@boundarydevices.com>
Cc: Carlos Rafael Giani <crg7475@mailbox.org>; meta-freescale <meta-freescale@yoctoproject.org>; Chris Dimich <chris.dimich@boundarydevices.com>
Subject: Re: [meta-freescale] linux-imx-headers and ioctl mismatches

On Fri, Apr 10, 2020 at 11:46 AM Gary Bisson <gary.bisson@boundarydevices.com> wrote:
> On Fri, Apr 10, 2020 at 11:22:45AM -0300, Otavio Salvador wrote:
> > On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani 
> > <crg7475@mailbox.org> wrote:
> > > Hm I thought there was some sort of imx-kernel base class. 
> > > Apparently there isn't. If there were, it could then be merged 
> > > with the linux-imx-headers recipe, and perhaps create something like a -dev package.
> > >
> > > But then again, a -dev package from a kernel recipe ...? Kind of weird.
> >
> > The problem is that you cannot mix packages / apps with different 
> > versions of this. This ends being part of the binary ABI.
>
> I understand it's a tricky topic as some packages won't build if the 
> kernel headers are not aligned. But I'm not sure it is better to have 
> packages building but having runtime issues because of mismatch...
>
> Since our kernel is also used on other OSes we will not update its 
> headers. It also feels wrong to do so.

Add a patch to the layer for it.

> Otavio, do you expect all the vendors to have their kernel matching 
> exactly the linux-imx-headers version?
> Because looking at meta-freescale-3rdparty master branch [1], some 
> kernels are still based on 3.14...

and it is vendors consideration for their users. If they don't care, who should care? We are starting removing broken boards and vendors should pay attention to users.

There are two routes here:

- use mainline BSP
- use NXP BSP

for NXP BSP it comes with the need to keep upgrading it.


> I personnaly think that the headers should be installed from the 
> kernel being built, then it's up to the vendor to check the 
> compatibility but at least that would fix the issue Carlos is seeing.

This does not help; those headers should not exists in first case but as we do, a common base need to be used for all distro. Their symbols goes to other libraries and apps so it cannot change between machines.

> FYI, our next kernel release will be 5.4.3_1.0.0 as it is out since 
> last week. So if master stays with 4.19 we'll still have a mismatch I guess.

Will be going to 5.4 soon and people can help, patches welcome.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Otavio Salvador]

On Fri, Apr 10, 2020 at 3:25 PM Kevin Lannen <kevin@patroness.com> wrote:
> I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?

It all depends how long it takes. Dunfell is an important release and
as such we ought to try to have it using new components.

We need someone to help updating NXP components; I am focusing more on
mainline BSP support since I've been doing this on our free time.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Kevin Lannen]

>> I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?

>It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.

>We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.

Is there a github issue or a list somewhere tracking what needs to be done for this?


Kevin Lannen

Embedded Systems Engineer
kevin@luci.com
970-690-8619

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Otavio Salvador]

On Fri, Apr 10, 2020 at 3:43 PM Kevin Lannen <kevin@patroness.com> wrote:
> >> I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
>
> >It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.
>
> >We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.
>
> Is there a github issue or a list somewhere tracking what needs to be done for this?

No, even though we can open. Generally we need to update:

- linux-imx
- u-boot-imx
- headers
- gpu drivers
- media codecs
- gst

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Carlos Rafael Giani]

[-- Attachment #1: Type: text/plain, Size: 1658 bytes --]

This issue came up when I noticed that the ION and DWL allocators in 
libimxdmabuffer failed. (DWL is an API from the Hantro VPU libraries, 
and it fails, because internally it itself uses ION.)

This is the relevant code: 
https://github.com/Freescale/libimxdmabuffer/blob/master/imxdmabuffer/imxdmabuffer_ion_allocator.c#L428

I am now wondering if I can add some workaround to also support older 
kernels that are subject to such a header mismatch. An ugly workaround 
would be to manually define the IOCTL, like this:

   #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 19, 0)
   #define CUSTOM_DMA_BUF_IOCTL_PHYS      _IOW(DMA_BUF_BASE, 1, struct 
dma_buf_phys)
   #else
   #define CUSTOM_DMA_BUF_IOCTL_PHYS      DMA_BUF_IOCTL_PHYS
   #endif

And then use the CUSTOM_DMA_BUF_IOCTL_PHYS instead of the regular 
DMA_BUF_IOCTL_PHYS in the code.

Pro: This would make libimxdmabuffer work with existing setups, and 
could even be backported to zeus and warrior.

Con: Defining IOCTLs in userspace like this is usually not a good idea.

Comments?


On 10.04.20 16:22, Otavio Salvador wrote:
> On Fri, Apr 10, 2020 at 11:20 AM Carlos Rafael Giani
> <crg7475@mailbox.org> wrote:
>> Hm I thought there was some sort of imx-kernel base class. Apparently
>> there isn't. If there were, it could then be merged with the
>> linux-imx-headers recipe, and perhaps create something like a -dev package.
>>
>> But then again, a -dev package from a kernel recipe ...? Kind of weird.
> The problem is that you cannot mix packages / apps with different
> versions of this. This ends being part of the binary ABI.
>
>
> 

[-- Attachment #2: Type: text/html, Size: 2636 bytes --]

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Otavio Salvador]

Carlos,

On Sat, Apr 11, 2020 at 7:32 AM Carlos Rafael Giani <crg7475@mailbox.org> wrote:
> This issue came up when I noticed that the ION and DWL allocators in libimxdmabuffer failed. (DWL is an API from the Hantro VPU libraries, and it fails, because internally it itself uses ION.)
>
> This is the relevant code: https://github.com/Freescale/libimxdmabuffer/blob/master/imxdmabuffer/imxdmabuffer_ion_allocator.c#L428
>
> I am now wondering if I can add some workaround to also support older kernels that are subject to such a header mismatch. An ugly workaround would be to manually define the IOCTL, like this:
>
>   #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 19, 0)
>   #define CUSTOM_DMA_BUF_IOCTL_PHYS      _IOW(DMA_BUF_BASE, 1, struct dma_buf_phys)
>   #else
>   #define CUSTOM_DMA_BUF_IOCTL_PHYS      DMA_BUF_IOCTL_PHYS
>   #endif
>
> And then use the CUSTOM_DMA_BUF_IOCTL_PHYS instead of the regular DMA_BUF_IOCTL_PHYS in the code.

This does not work as the libimxdmabuffer is shared (and it should be)
across multiple machines of same SoC family. Considering that, the
best choice is to apply a patch to old Linux kernels to make it to use
the newer API/ABI with this backport.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Andrey Zhizhikin]

On Fri, Apr 10, 2020 at 8:46 PM Otavio Salvador
<otavio.salvador@ossystems.com.br> wrote:
>
> On Fri, Apr 10, 2020 at 3:43 PM Kevin Lannen <kevin@patroness.com> wrote:
> > >> I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
> >
> > >It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.
> >
> > >We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.
> >
> > Is there a github issue or a list somewhere tracking what needs to be done for this?
>
> No, even though we can open. Generally we need to update:
>
> - linux-imx
> - u-boot-imx
> - headers
> - gpu drivers
> - media codecs
> - gst

I'm currently working on this one, primarily for imx8mm and imx8mn
machines. So far, I have the following list I've noted for myself:
- linux-imx (with mfgtool)
- u-boot-imx
- linux-imx-headers
- kernel-module-imx-gpu-viv (gpu drivers)
- kernel-module-qca9377 (wlan drivers)
- imx-atf
- imx-mkimage
- imx-test
- optee-imx (client, os, test)
- media codecs
- gst (base, good, bad, gstreamer and imx-gst)

I'm almost through with it for the Mini, then would cross-check if
everything compiles for Nano. I would send a PR once both validations
are done.

Otavio,
One general question: Do we keep the 4.19.35 in the layer, or I can
simply drop it and introduce the 5.4.3? So far, I did just that and it
seems to be going OK.

>
> --
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750
> 



-- 
Regards,
Andrey.

Re: [meta-freescale] linux-imx-headers and ioctl mismatches

[Otavio Salvador]

On Sat, Apr 11, 2020 at 7:05 PM Andrey Zhizhikin <andrey.z@gmail.com> wrote:
> On Fri, Apr 10, 2020 at 8:46 PM Otavio Salvador
> <otavio.salvador@ossystems.com.br> wrote:
> >
> > On Fri, Apr 10, 2020 at 3:43 PM Kevin Lannen <kevin@patroness.com> wrote:
> > > >> I would be happy to help out some with moving linux-imx to the 5.4 kernel as we are very interested in using it. Is this planned to be in for the Dunfell branch when that is released?
> > >
> > > >It all depends how long it takes. Dunfell is an important release and as such we ought to try to have it using new components.
> > >
> > > >We need someone to help updating NXP components; I am focusing more on mainline BSP support since I've been doing this on our free time.
> > >
> > > Is there a github issue or a list somewhere tracking what needs to be done for this?
> >
> > No, even though we can open. Generally we need to update:
> >
> > - linux-imx
> > - u-boot-imx
> > - headers
> > - gpu drivers
> > - media codecs
> > - gst
>
> I'm currently working on this one, primarily for imx8mm and imx8mn
> machines. So far, I have the following list I've noted for myself:
> - linux-imx (with mfgtool)
> - u-boot-imx
> - linux-imx-headers
> - kernel-module-imx-gpu-viv (gpu drivers)
> - kernel-module-qca9377 (wlan drivers)
> - imx-atf
> - imx-mkimage
> - imx-test
> - optee-imx (client, os, test)
> - media codecs
> - gst (base, good, bad, gstreamer and imx-gst)
>
> I'm almost through with it for the Mini, then would cross-check if
> everything compiles for Nano. I would send a PR once both validations
> are done.
>
> Otavio,
> One general question: Do we keep the 4.19.35 in the layer, or I can
> simply drop it and introduce the 5.4.3? So far, I did just that and it
> seems to be going OK.

Please drop.


-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750