Custom kernel headers and SDK

Custom kernel headers and SDK

[Longchamp, Valentin]

[-- Attachment #1: Type: text/plain, Size: 3092 bytes --]

Hello,

We are currently migrating our Embedded Linux software and the corresponding
small in house distribution to Yocto. We are small Linux team and we provide and
maintain the toolchain, kernel and base distribution for a broader team of
software developers that build the applications for our hardware.

So far, we have been able to use Yocto to build our whole software (kernel,
minimal distribution and application software) by adding our minimal meta-layer
and defining stuff there and it works great.

Since the application devs are "not interested" in our distro nor in the
compiler, we want to make them available for them thanks to the SDK which is a
great fit for this use case. So far it works very well and we are currently
working on adapting the build system to the SDK.

And here comes my question. We have however our own kernel sources that extend
the headers for a few "in-house" drivers. From the warning in linux-libc-headers
(http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc),
I understand that it should be avoided to recreate a recipe for the kernel
headers to avoid having the libc "machinified". From yocto, I could get the
custom headers thanks to STAGING_KERNEL_DIR. However, we need these custom
headers from "outside" of yocto, when using the SDK.

I have found 2 ways to install them into the sysroot and thus SDK:
- either by adding the kernel-devsrc to the SDK that installs them into
/usr/src/kernel by default.
- or by adding something similar to what is proposed in this mailing-list
discussion to our kernel recipe:
https://lists.yoctoproject.org/pipermail/linux-yocto/2014-April/002178.html

The problem with these 2 solutions is that sysroot contains:
1) the sanitized kernel headers at in sysroot/usr/include/linux
2) our custom headers at sysroot/usr/src/kernel/include/linux

and when I add a -I option in the CFLAGS for 2), I get conflicts since the SDK
automatically looks first in sysroot/usr/include.

Here is an example of the current errors I get with 1) and 2) in both the -I path:

>                  from plat/icndrv/hdlc/mpc8xxx_scc/src/libhdlc-km.c:2:
> /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/src/kernel/include/linux/types.h:14:26: error: conflicting types for 'fd_set'
>  typedef __kernel_fd_set  fd_set;
>                           ^
> In file included from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/types.h:219:0,
>                  from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/uio.h:23,
>                  from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/socket.h:26,
>                  from plat/icndrv/hdlc/mpc8xxx_scc/src/libhdlc-km.c:1:
> /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/select.h:75:5: note: previous declaration of 'fd_set' was here
>    } fd_set;

What would be the correct "yocto" way of solving our current problem with our
custom headers to be added to the SDK ?

Thanks

Valentin

[-- Attachment #2: Type: text/html, Size: 4225 bytes --]

Re: Custom kernel headers and SDK

[Vuille, Martin (Martin)]

[-- Attachment #1: Type: text/plain, Size: 4722 bytes --]

Hi Valentin,

Not sure whether this the “canonical” Yocto way, but here’s how we solved
the same issue.

First, let me say that we use a BSP from our technology partner, and have
our own meta-XXX layer for customization.


-          Created a .bbappend in our layer for the kernel recipe

-          In the .bbappend

o   add a do_install_append task that creates directory ${D}${includedir}/XXX
and installs the required headers from kernel source into that
directory

o   add a new package: PACKAGES += “ XXX-kernel-headers”

o   specify the files for that package: FILES_XXX-kernel-headers = “${includedir}/XXX/*”

-          In the recipe for the SDK, include the package XXX-kernel-headers

MV

From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Longchamp, Valentin
Sent: December 08, 2015 4:06 PM
To: yocto@yoctoproject.org
Cc: Brunck, Holger
Subject: [yocto] Custom kernel headers and SDK


Hello,

We are currently migrating our Embedded Linux software and the corresponding
small in house distribution to Yocto. We are small Linux team and we provide and
maintain the toolchain, kernel and base distribution for a broader team of
software developers that build the applications for our hardware.

So far, we have been able to use Yocto to build our whole software (kernel,
minimal distribution and application software) by adding our minimal meta-layer
and defining stuff there and it works great.

Since the application devs are "not interested" in our distro nor in the
compiler, we want to make them available for them thanks to the SDK which is a
great fit for this use case. So far it works very well and we are currently
working on adapting the build system to the SDK.

And here comes my question. We have however our own kernel sources that extend
the headers for a few "in-house" drivers. From the warning in linux-libc-headers
(http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc<https://urldefense.proofpoint.com/v2/url?u=http-3A__git.yoctoproject.org_cgit_cgit.cgi_poky_tree_meta_recipes-2Dkernel_linux-2Dlibc-2Dheaders_linux-2Dlibc-2Dheaders.inc&d=BQQGaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=SbT_XjBLTrjzWy1iRhblzG3bKRhcodBp7tD_mnp1dpg&m=aecjOhK90BxgIsO0Z0SxTTAYA24E0E2rNHYdLiaPdEM&s=NsNJ4rNpfxO5fD-mwfYiZL8m1FwdIcozCxbta8sS7lA&e=>),
I understand that it should be avoided to recreate a recipe for the kernel
headers to avoid having the libc "machinified". From yocto, I could get the
custom headers thanks to STAGING_KERNEL_DIR. However, we need these custom
headers from "outside" of yocto, when using the SDK.

I have found 2 ways to install them into the sysroot and thus SDK:
- either by adding the kernel-devsrc to the SDK that installs them into
/usr/src/kernel by default.
- or by adding something similar to what is proposed in this mailing-list
discussion to our kernel recipe:
https://lists.yoctoproject.org/pipermail/linux-yocto/2014-April/002178.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.yoctoproject.org_pipermail_linux-2Dyocto_2014-2DApril_002178.html&d=BQQGaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=SbT_XjBLTrjzWy1iRhblzG3bKRhcodBp7tD_mnp1dpg&m=aecjOhK90BxgIsO0Z0SxTTAYA24E0E2rNHYdLiaPdEM&s=utjVbEgHLBYm4zWbGYyBQ0jFBfRz43KePurEZ1meevs&e=>

The problem with these 2 solutions is that sysroot contains:
1) the sanitized kernel headers at in sysroot/usr/include/linux
2) our custom headers at sysroot/usr/src/kernel/include/linux

and when I add a -I option in the CFLAGS for 2), I get conflicts since the SDK
automatically looks first in sysroot/usr/include.

Here is an example of the current errors I get with 1) and 2) in both the -I path:

>                  from plat/icndrv/hdlc/mpc8xxx_scc/src/libhdlc-km.c:2:
> /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/src/kernel/include/linux/types.h:14:26: error: conflicting types for 'fd_set'
>  typedef __kernel_fd_set  fd_set;
>                           ^
> In file included from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/types.h:219:0,
>                  from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/uio.h:23,
>                  from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/socket.h:26,
>                  from plat/icndrv/hdlc/mpc8xxx_scc/src/libhdlc-km.c:1:
> /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/select.h:75:5: note: previous declaration of 'fd_set' was here
>    } fd_set;

What would be the correct "yocto" way of solving our current problem with our
custom headers to be added to the SDK ?

Thanks

Valentin

[-- Attachment #2: Type: text/html, Size: 15006 bytes --]

Re: Custom kernel headers and SDK

[Longchamp, Valentin]

Hi Martin,

Thanks a lot for the suggestion. I had in the meantime come to a simliar solution: define a linux-XXX-headers recipe in our own meta layer where I install the missing kernel headers and then I add this linux-XXX-headers-dev package to our SDK's sysroot thanks to TARGET_TOOLCHAIN_TASK.
Your implementation looks fine and I am going to pick some ideas from it.

However, I have stumbled upon another problem with this approach: it works well if you add new files to the kernel headers. We unforntunately have at least one uapi file (i2c.h to name it) that is present in the standard kernel headers where we have our own version with some additions/extensions. If I install our version from ou linux-XXX-headers recipe, bitbake (correctly) complains that this file is provided by 2 packages (linux-libc-headers and linux-XXX-headers). And here I really don't know how to solve this issue. Any ideas/suggestions ?

Thanks

Valentin
________________________________________
From: Vuille, Martin (Martin) [vmartin@avaya.com]
Sent: Wednesday, December 09, 2015 5:40 PM
To: Longchamp, Valentin; yocto@yoctoproject.org
Cc: Brunck, Holger
Subject: RE: Custom kernel headers and SDK

Hi Valentin,

Not sure whether this the “canonical” Yocto way, but here’s how we solved
the same issue.

First, let me say that we use a BSP from our technology partner, and have
our own meta-XXX layer for customization.


-          Created a .bbappend in our layer for the kernel recipe

-          In the .bbappend

o   add a do_install_append task that creates directory ${D}${includedir}/XXX
and installs the required headers from kernel source into that
directory

o   add a new package: PACKAGES += “ XXX-kernel-headers”

o   specify the files for that package: FILES_XXX-kernel-headers = “${includedir}/XXX/*”

-          In the recipe for the SDK, include the package XXX-kernel-headers

MV

From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Longchamp, Valentin
Sent: December 08, 2015 4:06 PM
To: yocto@yoctoproject.org
Cc: Brunck, Holger
Subject: [yocto] Custom kernel headers and SDK


Hello,

We are currently migrating our Embedded Linux software and the corresponding
small in house distribution to Yocto. We are small Linux team and we provide and
maintain the toolchain, kernel and base distribution for a broader team of
software developers that build the applications for our hardware.

So far, we have been able to use Yocto to build our whole software (kernel,
minimal distribution and application software) by adding our minimal meta-layer
and defining stuff there and it works great.

Since the application devs are "not interested" in our distro nor in the
compiler, we want to make them available for them thanks to the SDK which is a
great fit for this use case. So far it works very well and we are currently
working on adapting the build system to the SDK.

And here comes my question. We have however our own kernel sources that extend
the headers for a few "in-house" drivers. From the warning in linux-libc-headers
(http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc<https://urldefense.proofpoint.com/v2/url?u=http-3A__git.yoctoproject.org_cgit_cgit.cgi_poky_tree_meta_recipes-2Dkernel_linux-2Dlibc-2Dheaders_linux-2Dlibc-2Dheaders.inc&d=BQQGaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=SbT_XjBLTrjzWy1iRhblzG3bKRhcodBp7tD_mnp1dpg&m=aecjOhK90BxgIsO0Z0SxTTAYA24E0E2rNHYdLiaPdEM&s=NsNJ4rNpfxO5fD-mwfYiZL8m1FwdIcozCxbta8sS7lA&e=>),
I understand that it should be avoided to recreate a recipe for the kernel
headers to avoid having the libc "machinified". From yocto, I could get the
custom headers thanks to STAGING_KERNEL_DIR. However, we need these custom
headers from "outside" of yocto, when using the SDK.

I have found 2 ways to install them into the sysroot and thus SDK:
- either by adding the kernel-devsrc to the SDK that installs them into
/usr/src/kernel by default.
- or by adding something similar to what is proposed in this mailing-list
discussion to our kernel recipe:
https://lists.yoctoproject.org/pipermail/linux-yocto/2014-April/002178.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.yoctoproject.org_pipermail_linux-2Dyocto_2014-2DApril_002178.html&d=BQQGaQ&c=BFpWQw8bsuKpl1SgiZH64Q&r=SbT_XjBLTrjzWy1iRhblzG3bKRhcodBp7tD_mnp1dpg&m=aecjOhK90BxgIsO0Z0SxTTAYA24E0E2rNHYdLiaPdEM&s=utjVbEgHLBYm4zWbGYyBQ0jFBfRz43KePurEZ1meevs&e=>

The problem with these 2 solutions is that sysroot contains:
1) the sanitized kernel headers at in sysroot/usr/include/linux
2) our custom headers at sysroot/usr/src/kernel/include/linux

and when I add a -I option in the CFLAGS for 2), I get conflicts since the SDK
automatically looks first in sysroot/usr/include.

Here is an example of the current errors I get with 1) and 2) in both the -I path:

>                  from plat/icndrv/hdlc/mpc8xxx_scc/src/libhdlc-km.c:2:
> /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/src/kernel/include/linux/types.h:14:26: error: conflicting types for 'fd_set'
>  typedef __kernel_fd_set  fd_set;
>                           ^
> In file included from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/types.h:219:0,
>                  from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/uio.h:23,
>                  from /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/socket.h:26,
>                  from plat/icndrv/hdlc/mpc8xxx_scc/src/libhdlc-km.c:1:
> /opt/work/ppc-toolchain/sysroots/powerpc-oe-linux/usr/include/sys/select.h:75:5: note: previous declaration of 'fd_set' was here
>    } fd_set;

What would be the correct "yocto" way of solving our current problem with our
custom headers to be added to the SDK ?

Thanks

Valentin

Re: Custom kernel headers and SDK

[Vuille, Martin (Martin)]

Hi Valentin,

When we first encountered the need for custom kernel headers, I asked
a similar question to yours on this list. And what I learned from some of the
responses was that modifying the standard headers was fraught with
problems, including the issue that you raise.

Fortunately, we got away without having to modify and export any standard
headers, so I dodged that bullet.

Some ideas that come to mind:

- Can you install your customized version in ${includedir}/XXX and use the
  include path search order to make sure the compiler sees it first?

- Can you include the standard header in a "wrapper" header and make your
  customizations in the wrapper instead?

Regards,
MV

> -----Original Message-----
> From: Longchamp, Valentin [mailto:Valentin.Longchamp@keymile.com]
> Sent: December 10, 2015 4:47 AM
> To: Vuille, Martin (Martin); yocto@yoctoproject.org
> Cc: Brunck, Holger
> Subject: RE: Custom kernel headers and SDK
> 
> Hi Martin,
> 
> Thanks a lot for the suggestion. I had in the meantime come to a simliar
> solution: define a linux-XXX-headers recipe in our own meta layer where I
> install the missing kernel headers and then I add this linux-XXX-headers-dev
> package to our SDK's sysroot thanks to TARGET_TOOLCHAIN_TASK.
> Your implementation looks fine and I am going to pick some ideas from it.
> 
> However, I have stumbled upon another problem with this approach: it
> works well if you add new files to the kernel headers. We unforntunately
> have at least one uapi file (i2c.h to name it) that is present in the standard
> kernel headers where we have our own version with some
> additions/extensions. If I install our version from ou linux-XXX-headers
> recipe, bitbake (correctly) complains that this file is provided by 2 packages
> (linux-libc-headers and linux-XXX-headers). And here I really don't know how
> to solve this issue. Any ideas/suggestions ?
> 
> Thanks
> 
> Valentin

Re: Custom kernel headers and SDK

[William Mills]

On 12/10/2015 09:40 AM, Vuille, Martin (Martin) wrote:
> Hi Valentin,
> 
> When we first encountered the need for custom kernel headers, I asked
> a similar question to yours on this list. And what I learned from some of the
> responses was that modifying the standard headers was fraught with
> problems, including the issue that you raise.
> 
> Fortunately, we got away without having to modify and export any standard
> headers, so I dodged that bullet.
> 
> Some ideas that come to mind:
> 
> - Can you install your customized version in ${includedir}/XXX and use the
>   include path search order to make sure the compiler sees it first?

This was the approach we used in a Debian based system.  Applications
that needed to be dependent on the custom capabilities of our kernel
depended on our XXX-kernel-header package and adjusted their include
path to use it first.  All other apps continue to use the standard
kernel headers.  (busybox or tar does not really care about our new
wizbang feature and the Debian distro sure was not going to rebuild them
just for us.)

I see no reason this could not work as well in an OE based distro.

> 
> - Can you include the standard header in a "wrapper" header and make your
>   customizations in the wrapper instead?
> 
> Regards,
> MV
> 
>> -----Original Message-----
>> From: Longchamp, Valentin [mailto:Valentin.Longchamp@keymile.com]
>> Sent: December 10, 2015 4:47 AM
>> To: Vuille, Martin (Martin); yocto@yoctoproject.org
>> Cc: Brunck, Holger
>> Subject: RE: Custom kernel headers and SDK
>>
>> Hi Martin,
>>
>> Thanks a lot for the suggestion. I had in the meantime come to a simliar
>> solution: define a linux-XXX-headers recipe in our own meta layer where I
>> install the missing kernel headers and then I add this linux-XXX-headers-dev
>> package to our SDK's sysroot thanks to TARGET_TOOLCHAIN_TASK.
>> Your implementation looks fine and I am going to pick some ideas from it.
>>
>> However, I have stumbled upon another problem with this approach: it
>> works well if you add new files to the kernel headers. We unforntunately
>> have at least one uapi file (i2c.h to name it) that is present in the standard
>> kernel headers where we have our own version with some
>> additions/extensions. If I install our version from ou linux-XXX-headers
>> recipe, bitbake (correctly) complains that this file is provided by 2 packages
>> (linux-libc-headers and linux-XXX-headers). And here I really don't know how
>> to solve this issue. Any ideas/suggestions ?
>>
>> Thanks
>>
>> Valentin

Re: Custom kernel headers and SDK

[Valentin Longchamp]

Hello William and Martin,

On 10/12/2015 15:53, William Mills wrote:
> 
> On 12/10/2015 09:40 AM, Vuille, Martin (Martin) wrote:
>> Hi Valentin,
>>
>> When we first encountered the need for custom kernel headers, I asked
>> a similar question to yours on this list. And what I learned from some of the
>> responses was that modifying the standard headers was fraught with
>> problems, including the issue that you raise.
>>
>> Fortunately, we got away without having to modify and export any standard
>> headers, so I dodged that bullet.
>>
>> Some ideas that come to mind:
>>
>> - Can you install your customized version in ${includedir}/XXX and use the
>>   include path search order to make sure the compiler sees it first?
> 
> This was the approach we used in a Debian based system.  Applications
> that needed to be dependent on the custom capabilities of our kernel
> depended on our XXX-kernel-header package and adjusted their include
> path to use it first.  All other apps continue to use the standard
> kernel headers.  (busybox or tar does not really care about our new
> wizbang feature and the Debian distro sure was not going to rebuild them
> just for us.)
> 
> I see no reason this could not work as well in an OE based distro.
> 
>>
>> - Can you include the standard header in a "wrapper" header and make your
>>   customizations in the wrapper instead?
>>
>> Regards,
>> MV

I just wanted to thank you for your answers and suggestion.

I have now used this approach and it works fine:
- make a new package that installs our changed linux-headers into another
directory than the linux-libc-headers
- added this package to TOOLCHAIN_TARGET_TASK so that it gets installed to the
SDK's sysroot
- changed the include path of the userspace applications that require these
changed headers and use the SDK so that they find them ($SDKTARGETSYSROOT helps
a lot here).

Valentin

>>
>>> -----Original Message-----
>>> From: Longchamp, Valentin [mailto:Valentin.Longchamp@keymile.com]
>>> Sent: December 10, 2015 4:47 AM
>>> To: Vuille, Martin (Martin); yocto@yoctoproject.org
>>> Cc: Brunck, Holger
>>> Subject: RE: Custom kernel headers and SDK
>>>
>>> Hi Martin,
>>>
>>> Thanks a lot for the suggestion. I had in the meantime come to a simliar
>>> solution: define a linux-XXX-headers recipe in our own meta layer where I
>>> install the missing kernel headers and then I add this linux-XXX-headers-dev
>>> package to our SDK's sysroot thanks to TARGET_TOOLCHAIN_TASK.
>>> Your implementation looks fine and I am going to pick some ideas from it.
>>>
>>> However, I have stumbled upon another problem with this approach: it
>>> works well if you add new files to the kernel headers. We unforntunately
>>> have at least one uapi file (i2c.h to name it) that is present in the standard
>>> kernel headers where we have our own version with some
>>> additions/extensions. If I install our version from ou linux-XXX-headers
>>> recipe, bitbake (correctly) complains that this file is provided by 2 packages
>>> (linux-libc-headers and linux-XXX-headers). And here I really don't know how
>>> to solve this issue. Any ideas/suggestions ?
>>>
>>> Thanks
>>>
>>> Valentin