All of lore.kernel.org
 help / color / mirror / Atom feed
* Need suggestions: in case where linux-lib-headers and the kernel versions differ
@ 2021-04-23 15:03 Sourabh Banerjee
  2021-04-23 15:12 ` [OE-core] " Quentin Schulz
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Sourabh Banerjee @ 2021-04-23 15:03 UTC (permalink / raw)
  To: openembedded-core

[-- Attachment #1: Type: text/plain, Size: 1630 bytes --]

Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different?
For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:

1) Machine is supported on 5.4 kernel
*Kernel Recipe:* linux-yocto_5.4.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
There is no conflict in this case as libc-headers and linux-yocto are on same version.

2) Machine is supported on a higher kernel (let's say 5.10)
*Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports  linux-libc-headers_5.4 completely.

3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"

I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3
-  videodev2.h: This header is enhanced in 5.4
-  fsverity.h : this header is not present in 4.19

--
Regards,
Sourabh

[-- Attachment #2: Type: text/html, Size: 2137 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] Need suggestions: in case where linux-lib-headers and the kernel versions differ
  2021-04-23 15:03 Need suggestions: in case where linux-lib-headers and the kernel versions differ Sourabh Banerjee
@ 2021-04-23 15:12 ` Quentin Schulz
  2021-04-23 15:35 ` Bruce Ashfield
  2021-04-23 18:01 ` Khem Raj
  2 siblings, 0 replies; 8+ messages in thread
From: Quentin Schulz @ 2021-04-23 15:12 UTC (permalink / raw)
  To: Sourabh Banerjee; +Cc: openembedded-core

Hi Sourabh,

On Fri, Apr 23, 2021 at 08:03:08AM -0700, Sourabh Banerjee wrote:
> Hi All,
> I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different?
> For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
> With Dunfell let's consider following 3 cases:
> 
> 1) Machine is supported on 5.4 kernel
> *Kernel Recipe:* linux-yocto_5.4.bb
> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> There is no conflict in this case as libc-headers and linux-yocto are on same version.
> 
> 2) Machine is supported on a higher kernel (let's say 5.10)
> *Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb
> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports  linux-libc-headers_5.4 completely.
> 
> 3) Machine is supported on a lower kernel (let's say 4.19)
> Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
> libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
> What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
> 

The issue is that linux-libc-headers is not machine-specific, only
distro-specific, so you'd need this PREFERRED_VERSION_linux-libc-headers
specified in a distro (or local.conf for that matter, but... NO, please
NO :) ).

You could also technically provide your own linux-libc-headers in a
layer of a higher priority and then PREFERRED_VERSION_linux-libc-headers
won't matter as the highest prio layer's recipe will take precedence.
But that is IMO bad practice as it means your distro has a different
linux-libc-headers depending on whether a layer is included or not.

I don't have a solution personally, just wanted to share why the last
suggestion does not seem correct to me.

Cheers,
Quentin

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] Need suggestions: in case where linux-lib-headers and the kernel versions differ
  2021-04-23 15:03 Need suggestions: in case where linux-lib-headers and the kernel versions differ Sourabh Banerjee
  2021-04-23 15:12 ` [OE-core] " Quentin Schulz
@ 2021-04-23 15:35 ` Bruce Ashfield
  2021-04-23 16:06   ` Yi Fan Yu
  2021-04-23 18:01 ` Khem Raj
  2 siblings, 1 reply; 8+ messages in thread
From: Bruce Ashfield @ 2021-04-23 15:35 UTC (permalink / raw)
  To: Sourabh Banerjee; +Cc: Patches and discussions about the oe-core layer

On Fri, Apr 23, 2021 at 11:03 AM Sourabh Banerjee
<sbanerje@codeaurora.org> wrote:
>
> Hi All,
> I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different?
> For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
> With Dunfell let's consider following 3 cases:
>
> 1) Machine is supported on 5.4 kernel
>     Kernel Recipe: linux-yocto_5.4.bb
>     libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>     There is no conflict in this case as libc-headers and linux-yocto are on same version.
>
> 2) Machine is supported on a higher kernel (let's say 5.10)
>     Kernel Recipe: Let's assume provided by a BSP layer, linux_5.10.bb
>     libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>     The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports  linux-libc-headers_5.4 completely.
>
> 3) Machine is supported on a lower kernel (let's say 4.19)
>     Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
>     libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>     The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
>     What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
>

This is not something that we've really seen happen.  When the
libc-headers are property used, they'll check/probe for functionality
and use what is available.

If you have drivers/applications going directly at the libc-headers,
versus the toolchain use. That is a different question. In this case,
you'd be better of dealing with these on a per-application basis, and
pointing them at alternate headers.

You can dream up all sorts of issues, and most of them don't happen in
practice. I've run hundreds of kernels that are older than the
libc-headers, and have never run into an issue with standard,
non-machine specific applications.

Just look at master of oe-core. We continue to support 5.4, after I've
bumped the headers to 5.10.

Cheers,

Bruce

>     I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3
>       -  videodev2.h: This header is enhanced in 5.4
>       -  fsverity.h : this header is not present in 4.19
>
> --
> Regards,
> Sourabh
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] Need suggestions: in case where linux-lib-headers and the kernel versions differ
  2021-04-23 15:35 ` Bruce Ashfield
@ 2021-04-23 16:06   ` Yi Fan Yu
  2021-04-23 16:49     ` Bruce Ashfield
  0 siblings, 1 reply; 8+ messages in thread
From: Yi Fan Yu @ 2021-04-23 16:06 UTC (permalink / raw)
  To: Bruce Ashfield, Sourabh Banerjee
  Cc: Patches and discussions about the oe-core layer

On 4/23/21 11:35 AM, Bruce Ashfield wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
> 
> On Fri, Apr 23, 2021 at 11:03 AM Sourabh Banerjee
> <sbanerje@codeaurora.org> wrote:
>>
>> Hi All,
>> I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different?
>> For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
>> With Dunfell let's consider following 3 cases:
>>
>> 1) Machine is supported on 5.4 kernel
>>      Kernel Recipe: linux-yocto_5.4.bb
>>      libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>>      There is no conflict in this case as libc-headers and linux-yocto are on same version.
>>
>> 2) Machine is supported on a higher kernel (let's say 5.10)
>>      Kernel Recipe: Let's assume provided by a BSP layer, linux_5.10.bb
>>      libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>>      The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports  linux-libc-headers_5.4 completely.
>>
>> 3) Machine is supported on a lower kernel (let's say 4.19)
>>      Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
>>      libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>>      The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
>>      What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
>>
> 
> This is not something that we've really seen happen.  When the
> libc-headers are property used, they'll check/probe for functionality
> and use what is available.
> 
> If you have drivers/applications going directly at the libc-headers,
> versus the toolchain use. That is a different question. In this case,
> you'd be better of dealing with these on a per-application basis, and
> pointing them at alternate headers.
> 
> You can dream up all sorts of issues, and most of them don't happen in
> practice. I've run hundreds of kernels that are older than the
> libc-headers, and have never run into an issue with standard,
> non-machine specific applications.
> 
> Just look at master of oe-core. We continue to support 5.4, after I've
> bumped the headers to 5.10.
> 
maybe unrelated and anecdotal:

I looked at a windriver bug related to a test in libnl called `test-genl`

The failure only occurred in 5.4.X kernel.
and I was able to narrow it down to the 5.10 header update oe-core 
causing the failure.

(if you are interested in details, talk to randy...)

> Cheers,
> 
> Bruce
> 
>>      I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3
>>        -  videodev2.h: This header is enhanced in 5.4
>>        -  fsverity.h : this header is not present in 4.19
>>
>> --
>> Regards,
>> Sourabh
>>
>>
> 
> 
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
> 
> 
> 
> 
> 

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] Need suggestions: in case where linux-lib-headers and the kernel versions differ
  2021-04-23 16:06   ` Yi Fan Yu
@ 2021-04-23 16:49     ` Bruce Ashfield
  0 siblings, 0 replies; 8+ messages in thread
From: Bruce Ashfield @ 2021-04-23 16:49 UTC (permalink / raw)
  To: Yi Fan Yu
  Cc: Sourabh Banerjee, Patches and discussions about the oe-core layer

On Fri, Apr 23, 2021 at 12:06 PM Yi Fan Yu <yifan.yu@windriver.com> wrote:
>
> On 4/23/21 11:35 AM, Bruce Ashfield wrote:
> > [Please note: This e-mail is from an EXTERNAL e-mail address]
> >
> > On Fri, Apr 23, 2021 at 11:03 AM Sourabh Banerjee
> > <sbanerje@codeaurora.org> wrote:
> >>
> >> Hi All,
> >> I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different?
> >> For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
> >> With Dunfell let's consider following 3 cases:
> >>
> >> 1) Machine is supported on 5.4 kernel
> >>      Kernel Recipe: linux-yocto_5.4.bb
> >>      libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> >>      There is no conflict in this case as libc-headers and linux-yocto are on same version.
> >>
> >> 2) Machine is supported on a higher kernel (let's say 5.10)
> >>      Kernel Recipe: Let's assume provided by a BSP layer, linux_5.10.bb
> >>      libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> >>      The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports  linux-libc-headers_5.4 completely.
> >>
> >> 3) Machine is supported on a lower kernel (let's say 4.19)
> >>      Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
> >>      libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> >>      The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
> >>      What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
> >>
> >
> > This is not something that we've really seen happen.  When the
> > libc-headers are property used, they'll check/probe for functionality
> > and use what is available.
> >
> > If you have drivers/applications going directly at the libc-headers,
> > versus the toolchain use. That is a different question. In this case,
> > you'd be better of dealing with these on a per-application basis, and
> > pointing them at alternate headers.
> >
> > You can dream up all sorts of issues, and most of them don't happen in
> > practice. I've run hundreds of kernels that are older than the
> > libc-headers, and have never run into an issue with standard,
> > non-machine specific applications.
> >
> > Just look at master of oe-core. We continue to support 5.4, after I've
> > bumped the headers to 5.10.
> >
> maybe unrelated and anecdotal:
>
> I looked at a windriver bug related to a test in libnl called `test-genl`
>
> The failure only occurred in 5.4.X kernel.
> and I was able to narrow it down to the 5.10 header update oe-core
> causing the failure.
>

libnl is known for this. It's basically the ioctl of the networking world.

So that's consistent with what I said, it is rare, and is usually in
tightly coupled applications. When that happens, we almost always just
carry a header or two along with the application itself.

Bruce

> (if you are interested in details, talk to randy...)
>
> > Cheers,
> >
> > Bruce
> >
> >>      I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3
> >>        -  videodev2.h: This header is enhanced in 5.4
> >>        -  fsverity.h : this header is not present in 4.19
> >>
> >> --
> >> Regards,
> >> Sourabh
> >>
> >>
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > 
> >



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] Need suggestions: in case where linux-lib-headers and the kernel versions differ
  2021-04-23 15:03 Need suggestions: in case where linux-lib-headers and the kernel versions differ Sourabh Banerjee
  2021-04-23 15:12 ` [OE-core] " Quentin Schulz
  2021-04-23 15:35 ` Bruce Ashfield
@ 2021-04-23 18:01 ` Khem Raj
  2021-04-23 19:31   ` Sourabh Banerjee
  2 siblings, 1 reply; 8+ messages in thread
From: Khem Raj @ 2021-04-23 18:01 UTC (permalink / raw)
  To: Sourabh Banerjee, openembedded-core



On 4/23/21 8:03 AM, Sourabh Banerjee wrote:
> Hi All,
> I need your suggestion on how to reconcile if the linux-libc-headers and 
> kernel versions are different?
> For this discussion let's consider we are using the LTS branch YP-3.1 
> (Dunfell).
> With Dunfell let's consider following 3 cases:
> 
> 1) Machine is supported on 5.4 kernel
> *Kernel Recipe:* linux-yocto_5.4.bb
> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>      There is no conflict in this case as libc-headers and linux-yocto 
> are on same version.
> 
> 2) Machine is supported on a higher kernel (let's say 5.10)
> *Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb
> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>      The kernel version is higher in this case. But, should be okay as 
> 5.10 UAPI is backward compatible and supports  
> linux-libc-headers_5.4 completely.
> 
> 3) Machine is supported on a lower kernel (let's say 4.19)
> Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
> libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>      The kernel version is lower in this case. I suppose, this may lead 
> to some runtime issues? As the kernel built from 4.19 will not be able 
> to support 5.4 UAPI fully.
>      What's the recommendation here? Should the BSP provider port 
> linux-libc-header_4.19.bb in BSP layer and 
> set PREFERRED_VERSION_linux-libc-headers = "4.19%"
> 
>      I suspect this as UAPI headers such as following differ, and what 
> if this leads to runtime issues in case #3
>        -  videodev2.h: This header is enhanced in 5.4
>        -  fsverity.h : this header is not present in 4.19
> 

right. all this combos should work fine in general case, however they 
all are not tested with equal coverage. So best is always to try have 
UAPIs older or equal to the kernel you intend to use. Generally releases 
support supported LTS kernels and that should be good enough, however if 
you have very old kernel and want to use latest yocto release, be aware 
that we are not testing it with those old UAPIs so all testing falls on 
your in this regard.

> -- 
> Regards,
> Sourabh
> 
> 
> 
> 

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] Need suggestions: in case where linux-lib-headers and the kernel versions differ
  2021-04-23 18:01 ` Khem Raj
@ 2021-04-23 19:31   ` Sourabh Banerjee
  2021-04-23 19:52     ` Bruce Ashfield
  0 siblings, 1 reply; 8+ messages in thread
From: Sourabh Banerjee @ 2021-04-23 19:31 UTC (permalink / raw)
  To: Khem Raj; +Cc: openembedded-core


Thank you for sharing your thoughts!

Let me focus a bit on the case where the BSP kernel is lower than libc.
I.e., BSP layer comes with kernel 4.19 support for the machine in 
question.

 From your replies I see following options:

Option #1)
Use linux-libc-headers_5.4 as it is from poky. BSP layer provides the 
lower version of kernel.
There may be selective bugs for application (Such as the libnl case 
discussed on thread).

Deal with such applications case-by-case. By case-by-case, I mean, with 
a linux-vendor-headers.
And make the application depend on linux-vendor-headers and look for the 
header under custom path?

Option #2)
Implement a linux-libc-headers_4.19 (or the lower version you are at).
This way seemingly, the headers and kernel coming from BSP layer are 
reconciled.

But, doing so violates the "Poky" distro, as "Poky" originally came with 
linux-libc-headers_5.4 on Dunfell.
And, that is bad idea, hence create your own higher distro layer that 
chooses linux-libc-headers_4.19.


All done, this approach still does not make things right!

As all testing/validation of 3rd-Party applications/3rd-Party layers 
that Yocto Project
and different contributors have done using Poky and with the original 
linux-libc-headers_5.4, are lost.

The lower version of headers are exposing those 3rd-Party 
applications/3rd-Party to unknown issues.

It's sort of a dilemma!
The customer I am working with is worried about "Option #1" 
(linux-libc-headers_5.4 + lower BSP kernel).
As discovery of 5.4 vs 4.19 bugs may not be easy to uncover. And then 
for for a case-by-case fix they will
have to deviate away from linux-libc-headers, and make application 
recipes, header path specific fixes.

"Option #2" (linux-libc-headers_4.19 from a higher distro layer), seems 
like a way out, but at the cost of upending all
the testing and validation done with linux-libc-headers_5.4 for upstream 
recipes/3rd-party layer.

Upgrading the machine from 4.19 to 5.4 is also off the table at this 
time.


Option #3)
Selective Backport.

I am trying to get a feel if this is even the right path?
Let linux-libc-headers_5.4 be as it is from poky. BSP layer provides the 
lower version of kernel.
Identify and freeze on list of the UAPI headers that customer plans to 
use from linux-libc-headers_5.4.

Determine what is the gap between 4.19 and 5.4, backport those from 5.4 
to 4.19.
Do a similar exercise with 3rd-party OSS applications/frameworks/layers 
to be used on project and
weed out cases like libnl by way of backporting from 5.4 to 4.19.


Does this seem to keep things aligned with linux-libc-headers from 
"Poky" and still addresses the holes that lower
BSP kernel creates when working with linux-libc-headers_5.4?

Thanks for your time!
-- 
Regards,
Sourabh


On 2021-04-23 23:31, Khem Raj wrote:
> On 4/23/21 8:03 AM, Sourabh Banerjee wrote:
>> Hi All,
>> I need your suggestion on how to reconcile if the linux-libc-headers 
>> and kernel versions are different?
>> For this discussion let's consider we are using the LTS branch YP-3.1 
>> (Dunfell).
>> With Dunfell let's consider following 3 cases:
>> 
>> 1) Machine is supported on 5.4 kernel
>> *Kernel Recipe:* linux-yocto_5.4.bb
>> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>>      There is no conflict in this case as libc-headers and linux-yocto 
>> are on same version.
>> 
>> 2) Machine is supported on a higher kernel (let's say 5.10)
>> *Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb
>> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>>      The kernel version is higher in this case. But, should be okay as 
>> 5.10 UAPI is backward compatible and supports  
>> linux-libc-headers_5.4 completely.
>> 
>> 3) Machine is supported on a lower kernel (let's say 4.19)
>> Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
>> libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
>>      The kernel version is lower in this case. I suppose, this may 
>> lead to some runtime issues? As the kernel built from 4.19 will not be 
>> able to support 5.4 UAPI fully.
>>      What's the recommendation here? Should the BSP provider port 
>> linux-libc-header_4.19.bb in BSP layer and 
>> set PREFERRED_VERSION_linux-libc-headers = "4.19%"
>> 
>>      I suspect this as UAPI headers such as following differ, and what 
>> if this leads to runtime issues in case #3
>>        -  videodev2.h: This header is enhanced in 5.4
>>        -  fsverity.h : this header is not present in 4.19
>> 
> 
> right. all this combos should work fine in general case, however they
> all are not tested with equal coverage. So best is always to try have
> UAPIs older or equal to the kernel you intend to use. Generally
> releases support supported LTS kernels and that should be good enough,
> however if you have very old kernel and want to use latest yocto
> release, be aware that we are not testing it with those old UAPIs so
> all testing falls on your in this regard.
> 
>> -- Regards,
>> Sourabh
>> 
>> 
>> 
>> 


-- 
Regards,
Sourabh

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] Need suggestions: in case where linux-lib-headers and the kernel versions differ
  2021-04-23 19:31   ` Sourabh Banerjee
@ 2021-04-23 19:52     ` Bruce Ashfield
  0 siblings, 0 replies; 8+ messages in thread
From: Bruce Ashfield @ 2021-04-23 19:52 UTC (permalink / raw)
  To: Sourabh Banerjee
  Cc: Khem Raj, Patches and discussions about the oe-core layer

On Fri, Apr 23, 2021 at 3:31 PM Sourabh Banerjee
<sbanerje@codeaurora.org> wrote:
>
>
> Thank you for sharing your thoughts!
>
> Let me focus a bit on the case where the BSP kernel is lower than libc.
> I.e., BSP layer comes with kernel 4.19 support for the machine in
> question.
>
>  From your replies I see following options:
>
> Option #1)
> Use linux-libc-headers_5.4 as it is from poky. BSP layer provides the
> lower version of kernel.
> There may be selective bugs for application (Such as the libnl case
> discussed on thread).
>
> Deal with such applications case-by-case. By case-by-case, I mean, with
> a linux-vendor-headers.
> And make the application depend on linux-vendor-headers and look for the
> header under custom path?

To be clear, you don't have to do this for everything. Everything in oe-ore
(as an example) has already been tested against the libc-headers in the
release, at both a lower and matching kernel version level.

It is good to be concerned, but this is not a common occurrence, and really
isn't the case to optimize for (i.e. that there is an issue, most applications
simply don't care, and it isn't impossible to identify ones that do).

>
> Option #2)
> Implement a linux-libc-headers_4.19 (or the lower version you are at).
> This way seemingly, the headers and kernel coming from BSP layer are
> reconciled.
>
> But, doing so violates the "Poky" distro, as "Poky" originally came with
> linux-libc-headers_5.4 on Dunfell.
> And, that is bad idea, hence create your own higher distro layer that
> chooses linux-libc-headers_4.19.
>
>
> All done, this approach still does not make things right!
>
> As all testing/validation of 3rd-Party applications/3rd-Party layers
> that Yocto Project
> and different contributors have done using Poky and with the original
> linux-libc-headers_5.4, are lost.
>
> The lower version of headers are exposing those 3rd-Party
> applications/3rd-Party to unknown issues.
>
> It's sort of a dilemma!
> The customer I am working with is worried about "Option #1"
> (linux-libc-headers_5.4 + lower BSP kernel).
> As discovery of 5.4 vs 4.19 bugs may not be easy to uncover. And then
> for for a case-by-case fix they will
> have to deviate away from linux-libc-headers, and make application
> recipes, header path specific fixes.
>
> "Option #2" (linux-libc-headers_4.19 from a higher distro layer), seems
> like a way out, but at the cost of upending all
> the testing and validation done with linux-libc-headers_5.4 for upstream
> recipes/3rd-party layer.
>
> Upgrading the machine from 4.19 to 5.4 is also off the table at this
> time.
>
>
> Option #3)
> Selective Backport.
>
> I am trying to get a feel if this is even the right path?
> Let linux-libc-headers_5.4 be as it is from poky. BSP layer provides the
> lower version of kernel.
> Identify and freeze on list of the UAPI headers that customer plans to
> use from linux-libc-headers_5.4.

This really isn't any different than option #1. If you can identify the
UAPI headers you are interested in, and can identify the applications
that use them .. then simply provide those headers to those applications
as you would in #1.

I wouldn't recommend it.

If you are really concerned about the mismatch, then you can/should
define your own distro and provide the matching libc-headers. That's
the less error prone way to do it,  if it is something you want to do.

Bruce

>
> Determine what is the gap between 4.19 and 5.4, backport those from 5.4
> to 4.19.
> Do a similar exercise with 3rd-party OSS applications/frameworks/layers
> to be used on project and
> weed out cases like libnl by way of backporting from 5.4 to 4.19.
>
>
> Does this seem to keep things aligned with linux-libc-headers from
> "Poky" and still addresses the holes that lower
> BSP kernel creates when working with linux-libc-headers_5.4?
>
> Thanks for your time!
> --
> Regards,
> Sourabh
>
>
> On 2021-04-23 23:31, Khem Raj wrote:
> > On 4/23/21 8:03 AM, Sourabh Banerjee wrote:
> >> Hi All,
> >> I need your suggestion on how to reconcile if the linux-libc-headers
> >> and kernel versions are different?
> >> For this discussion let's consider we are using the LTS branch YP-3.1
> >> (Dunfell).
> >> With Dunfell let's consider following 3 cases:
> >>
> >> 1) Machine is supported on 5.4 kernel
> >> *Kernel Recipe:* linux-yocto_5.4.bb
> >> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> >>      There is no conflict in this case as libc-headers and linux-yocto
> >> are on same version.
> >>
> >> 2) Machine is supported on a higher kernel (let's say 5.10)
> >> *Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb
> >> *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> >>      The kernel version is higher in this case. But, should be okay as
> >> 5.10 UAPI is backward compatible and supports
> >> linux-libc-headers_5.4 completely.
> >>
> >> 3) Machine is supported on a lower kernel (let's say 4.19)
> >> Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
> >> libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
> >>      The kernel version is lower in this case. I suppose, this may
> >> lead to some runtime issues? As the kernel built from 4.19 will not be
> >> able to support 5.4 UAPI fully.
> >>      What's the recommendation here? Should the BSP provider port
> >> linux-libc-header_4.19.bb in BSP layer and
> >> set PREFERRED_VERSION_linux-libc-headers = "4.19%"
> >>
> >>      I suspect this as UAPI headers such as following differ, and what
> >> if this leads to runtime issues in case #3
> >>        -  videodev2.h: This header is enhanced in 5.4
> >>        -  fsverity.h : this header is not present in 4.19
> >>
> >
> > right. all this combos should work fine in general case, however they
> > all are not tested with equal coverage. So best is always to try have
> > UAPIs older or equal to the kernel you intend to use. Generally
> > releases support supported LTS kernels and that should be good enough,
> > however if you have very old kernel and want to use latest yocto
> > release, be aware that we are not testing it with those old UAPIs so
> > all testing falls on your in this regard.
> >
> >> -- Regards,
> >> Sourabh
> >>
> >>
> >>
> >>
>
>
> --
> Regards,
> Sourabh
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-04-23 19:52 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-23 15:03 Need suggestions: in case where linux-lib-headers and the kernel versions differ Sourabh Banerjee
2021-04-23 15:12 ` [OE-core] " Quentin Schulz
2021-04-23 15:35 ` Bruce Ashfield
2021-04-23 16:06   ` Yi Fan Yu
2021-04-23 16:49     ` Bruce Ashfield
2021-04-23 18:01 ` Khem Raj
2021-04-23 19:31   ` Sourabh Banerjee
2021-04-23 19:52     ` Bruce Ashfield

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.