* linux-libc-headers - how to handle for older kernels? @ 2019-12-01 21:57 Peter Bergin 2019-12-02 9:01 ` [yocto] " Mikko Rapeli 2019-12-02 9:13 ` Mike Looijmans 0 siblings, 2 replies; 12+ messages in thread From: Peter Bergin @ 2019-12-01 21:57 UTC (permalink / raw) To: yocto Hi, I'm currently working in a project using Yocto 2.6 (thud) release. It has default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my project we will use kernel v4.1. I would like advice how to handle the linux-libc-headers package for my project, should I use the v4.18 headers or should I use the v4.1 header files which matches the running kernel? From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: "Kernel headers are backwards compatible, but not forwards compatible. This means that a program built against a C library using older kernel headers should run on a newer kernel (although it may not have access to new features), but a program built against newer kernel headers may not work on an older kernel." With the information from the quote above I would directly use v4.1 headers as my linux-libc-headers. But then reading the information in the file meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think another round. It states: " # You're probably looking here thinking you need to create some new copy # of linux-libc-headers since you have your own custom kernel. To put # this simply, you DO NOT. ... # There can also be a case where your kernel extremely old and you want # an older libc ABI for that old kernel. The headers installed by this # recipe should still be a standard mainline kernel, not your own custom # one. " The first part states that I should not change linux-libc-headers. But when I read the last part I'm not sure about the interpretation and it could be for my case. Just a matter of definition if v4.1 is extremely old compared to v4.18. Then another thing comes in to the equation; the LIBC ABI. When I look into the configuration of the glibc package it uses the configure switch "--enable-kernel=3.2" which means it shall be compatible with all kernel newer than v3.2. Then probably glibc is fine if it is compiled with v4.18 and run on v4.1? If building all applications against v4.18 headers but run on v4.1 kernel. I have a feeling that there potentially can be problems here. Please help me with some information about this and share your opinions? Are there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? The only drawback I see is that it will be a new configuration not well tested by the community. Are there other risks or drawbacks using your own version of linux-libc-headers? Best regards, /Peter ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-01 21:57 linux-libc-headers - how to handle for older kernels? Peter Bergin @ 2019-12-02 9:01 ` Mikko Rapeli 2019-12-02 9:13 ` Mike Looijmans 1 sibling, 0 replies; 12+ messages in thread From: Mikko Rapeli @ 2019-12-02 9:01 UTC (permalink / raw) To: peter; +Cc: yocto Hi, On Sun, Dec 01, 2019 at 10:57:15PM +0100, Peter Bergin wrote: > Hi, > > I'm currently working in a project using Yocto 2.6 (thud) release. It has > default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my > project we will use kernel v4.1. I would like advice how to handle the > linux-libc-headers package for my project, should I use the v4.18 headers or > should I use the v4.1 header files which matches the running kernel? > > > From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: > "Kernel headers are backwards compatible, but not forwards compatible. This > means that a program built against a C library using older kernel headers > should run on a newer kernel (although it may not have access to new > features), but a program built against newer kernel headers may not work on > an older kernel." > > With the information from the quote above I would directly use v4.1 headers > as my linux-libc-headers. But then reading the information in the file > meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think > another round. It states: > > " > # You're probably looking here thinking you need to create some new copy > # of linux-libc-headers since you have your own custom kernel. To put > # this simply, you DO NOT. > ... > # There can also be a case where your kernel extremely old and you want > # an older libc ABI for that old kernel. The headers installed by this > # recipe should still be a standard mainline kernel, not your own custom > # one. > " > > The first part states that I should not change linux-libc-headers. But when > I read the last part I'm not sure about the interpretation and it could be > for my case. Just a matter of definition if v4.1 is extremely old compared > to v4.18. > > Then another thing comes in to the equation; the LIBC ABI. When I look into > the configuration of the glibc package it uses the configure switch > "--enable-kernel=3.2" which means it shall be compatible with all kernel > newer than v3.2. Then probably glibc is fine if it is compiled with v4.18 > and run on v4.1? > > If building all applications against v4.18 headers but run on v4.1 kernel. I > have a feeling that there potentially can be problems here. > > Please help me with some information about this and share your opinions? Are > there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? > The only drawback I see is that it will be a new configuration not well > tested by the community. Are there other risks or drawbacks using your own > version of linux-libc-headers? I would fork linux-libc-headers to the kernel you are actually using despite the comment in the file. That's the only way to export headers from the correct kernel to everyone who needs them. There is no other documented way of exporting the actual kernel headers to users in bitbake builds and SDK. One optimization I would do is to fork the uapi headers from the actual kernel recipe so that full system does not need to be recompiled when kernel changes. One needs to manually track for kernel uapi header changes then. Cheers, -Mikko ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-01 21:57 linux-libc-headers - how to handle for older kernels? Peter Bergin 2019-12-02 9:01 ` [yocto] " Mikko Rapeli @ 2019-12-02 9:13 ` Mike Looijmans 2019-12-02 9:19 ` Mikko Rapeli 2019-12-02 13:44 ` Bruce Ashfield 1 sibling, 2 replies; 12+ messages in thread From: Mike Looijmans @ 2019-12-02 9:13 UTC (permalink / raw) To: peter, yocto On 01-12-19 22:57, Peter Bergin via Lists.Yoctoproject.Org wrote: > Hi, > > I'm currently working in a project using Yocto 2.6 (thud) release. It has > default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my > project we will use kernel v4.1. I would like advice how to handle the > linux-libc-headers package for my project, should I use the v4.18 headers or > should I use the v4.1 header files which matches the running kernel? > > From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: > "Kernel headers are backwards compatible, but not forwards compatible. This > means that a program built against a C library using older kernel headers > should run on a newer kernel (although it may not have access to new > features), but a program built against newer kernel headers may not work on an > older kernel." > > With the information from the quote above I would directly use v4.1 headers as > my linux-libc-headers. But then reading the information in the file > meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think > another round. It states: > > " > # You're probably looking here thinking you need to create some new copy > # of linux-libc-headers since you have your own custom kernel. To put > # this simply, you DO NOT. > ... > # There can also be a case where your kernel extremely old and you want > # an older libc ABI for that old kernel. The headers installed by this > # recipe should still be a standard mainline kernel, not your own custom > # one. > " > > The first part states that I should not change linux-libc-headers. But when I > read the last part I'm not sure about the interpretation and it could be for > my case. Just a matter of definition if v4.1 is extremely old compared to v4.18. > > Then another thing comes in to the equation; the LIBC ABI. When I look into > the configuration of the glibc package it uses the configure switch > "--enable-kernel=3.2" which means it shall be compatible with all kernel newer > than v3.2. Then probably glibc is fine if it is compiled with v4.18 and run on > v4.1? > > If building all applications against v4.18 headers but run on v4.1 kernel. I > have a feeling that there potentially can be problems here. > > Please help me with some information about this and share your opinions? Are > there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? > The only drawback I see is that it will be a new configuration not well tested > by the community. Are there other risks or drawbacks using your own version of > linux-libc-headers? It is not broken, so please don't fix it. OpenPLi has been using kernels way older than 4.1 with the kernel-headers generated by OE/yocto and did not experience any problems with that. There's about 50+ machines in there that have pre-built binary drivers that only work with a particular kernel config and hence the old stuff. There are some corner-cases with exotic kernels and exotic exports and exotic boot executables that use the kernel compiler, but I doubt that you're in there... If you have a kernel that exports something that's not in the regular headers, it's way better to solve that using a syscall than trying to poke in low level libc stuff. So again, if you don't experience problems, please don't try to fix it... ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-02 9:13 ` Mike Looijmans @ 2019-12-02 9:19 ` Mikko Rapeli 2019-12-02 9:28 ` Mike Looijmans 2019-12-02 13:44 ` Bruce Ashfield 1 sibling, 1 reply; 12+ messages in thread From: Mikko Rapeli @ 2019-12-02 9:19 UTC (permalink / raw) To: mike.looijmans; +Cc: peter, yocto Hi, On Mon, Dec 02, 2019 at 09:13:47AM +0000, Mike Looijmans wrote: > On 01-12-19 22:57, Peter Bergin via Lists.Yoctoproject.Org wrote: > > Hi, > > > > I'm currently working in a project using Yocto 2.6 (thud) release. It has > > default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my > > project we will use kernel v4.1. I would like advice how to handle the > > linux-libc-headers package for my project, should I use the v4.18 headers or > > should I use the v4.1 header files which matches the running kernel? > > > > From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: > > "Kernel headers are backwards compatible, but not forwards compatible. This > > means that a program built against a C library using older kernel headers > > should run on a newer kernel (although it may not have access to new > > features), but a program built against newer kernel headers may not work on an > > older kernel." > > > > With the information from the quote above I would directly use v4.1 headers as > > my linux-libc-headers. But then reading the information in the file > > meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think > > another round. It states: > > > > " > > # You're probably looking here thinking you need to create some new copy > > # of linux-libc-headers since you have your own custom kernel. To put > > # this simply, you DO NOT. > > ... > > # There can also be a case where your kernel extremely old and you want > > # an older libc ABI for that old kernel. The headers installed by this > > # recipe should still be a standard mainline kernel, not your own custom > > # one. > > " > > > > The first part states that I should not change linux-libc-headers. But when I > > read the last part I'm not sure about the interpretation and it could be for > > my case. Just a matter of definition if v4.1 is extremely old compared to v4.18. > > > > Then another thing comes in to the equation; the LIBC ABI. When I look into > > the configuration of the glibc package it uses the configure switch > > "--enable-kernel=3.2" which means it shall be compatible with all kernel newer > > than v3.2. Then probably glibc is fine if it is compiled with v4.18 and run on > > v4.1? > > > > If building all applications against v4.18 headers but run on v4.1 kernel. I > > have a feeling that there potentially can be problems here. > > > > Please help me with some information about this and share your opinions? Are > > there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? > > The only drawback I see is that it will be a new configuration not well tested > > by the community. Are there other risks or drawbacks using your own version of > > linux-libc-headers? > > > It is not broken, so please don't fix it. > > OpenPLi has been using kernels way older than 4.1 with the kernel-headers > generated by OE/yocto and did not experience any problems with that. There's > about 50+ machines in there that have pre-built binary drivers that only work > with a particular kernel config and hence the old stuff. > > There are some corner-cases with exotic kernels and exotic exports and exotic > boot executables that use the kernel compiler, but I doubt that you're in there... > > If you have a kernel that exports something that's not in the regular headers, > it's way better to solve that using a syscall than trying to poke in low level > libc stuff. > > So again, if you don't experience problems, please don't try to fix it... How to find a Linux kernel uapi header file with custom ioctl() etc definitions which are missing form the default linux-libc-headers? Other recipes and SDK builds need these. Copying the needed header files into the recipes who needs them is not ok. Cheers, -Mikko ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-02 9:19 ` Mikko Rapeli @ 2019-12-02 9:28 ` Mike Looijmans 2019-12-02 9:33 ` Mikko Rapeli 0 siblings, 1 reply; 12+ messages in thread From: Mike Looijmans @ 2019-12-02 9:28 UTC (permalink / raw) To: Mikko.Rapeli; +Cc: peter, yocto On 02-12-19 10:19, Mikko.Rapeli@bmw.de wrote: > Hi, > > On Mon, Dec 02, 2019 at 09:13:47AM +0000, Mike Looijmans wrote: >> On 01-12-19 22:57, Peter Bergin via Lists.Yoctoproject.Org wrote: >>> Hi, >>> >>> I'm currently working in a project using Yocto 2.6 (thud) release. It has >>> default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my >>> project we will use kernel v4.1. I would like advice how to handle the >>> linux-libc-headers package for my project, should I use the v4.18 headers or >>> should I use the v4.1 header files which matches the running kernel? >>> >>> From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: >>> "Kernel headers are backwards compatible, but not forwards compatible. This >>> means that a program built against a C library using older kernel headers >>> should run on a newer kernel (although it may not have access to new >>> features), but a program built against newer kernel headers may not work on an >>> older kernel." >>> >>> With the information from the quote above I would directly use v4.1 headers as >>> my linux-libc-headers. But then reading the information in the file >>> meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think >>> another round. It states: >>> >>> " >>> # You're probably looking here thinking you need to create some new copy >>> # of linux-libc-headers since you have your own custom kernel. To put >>> # this simply, you DO NOT. >>> ... >>> # There can also be a case where your kernel extremely old and you want >>> # an older libc ABI for that old kernel. The headers installed by this >>> # recipe should still be a standard mainline kernel, not your own custom >>> # one. >>> " >>> >>> The first part states that I should not change linux-libc-headers. But when I >>> read the last part I'm not sure about the interpretation and it could be for >>> my case. Just a matter of definition if v4.1 is extremely old compared to v4.18. >>> >>> Then another thing comes in to the equation; the LIBC ABI. When I look into >>> the configuration of the glibc package it uses the configure switch >>> "--enable-kernel=3.2" which means it shall be compatible with all kernel newer >>> than v3.2. Then probably glibc is fine if it is compiled with v4.18 and run on >>> v4.1? >>> >>> If building all applications against v4.18 headers but run on v4.1 kernel. I >>> have a feeling that there potentially can be problems here. >>> >>> Please help me with some information about this and share your opinions? Are >>> there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? >>> The only drawback I see is that it will be a new configuration not well tested >>> by the community. Are there other risks or drawbacks using your own version of >>> linux-libc-headers? >> >> >> It is not broken, so please don't fix it. >> >> OpenPLi has been using kernels way older than 4.1 with the kernel-headers >> generated by OE/yocto and did not experience any problems with that. There's >> about 50+ machines in there that have pre-built binary drivers that only work >> with a particular kernel config and hence the old stuff. >> >> There are some corner-cases with exotic kernels and exotic exports and exotic >> boot executables that use the kernel compiler, but I doubt that you're in there... >> >> If you have a kernel that exports something that's not in the regular headers, >> it's way better to solve that using a syscall than trying to poke in low level >> libc stuff. >> >> So again, if you don't experience problems, please don't try to fix it... > > How to find a Linux kernel uapi header file with custom ioctl() etc definitions which > are missing form the default linux-libc-headers? > > Other recipes and SDK builds need these. > > Copying the needed header files into the recipes who needs them is not ok. One solution I can think of is to put the header into it's own recipe/repository and then refer to it like any other library would. Refer to that recipe from the module (or kernel) recipe that needs it. This way you have your header in a single maintainable location and dependencies properly taken care of. If that's not something you could live with, share your recipes, since vague problem descriptions will only get you vague solutions... ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-02 9:28 ` Mike Looijmans @ 2019-12-02 9:33 ` Mikko Rapeli 2019-12-02 14:01 ` Adrian Bunk 0 siblings, 1 reply; 12+ messages in thread From: Mikko Rapeli @ 2019-12-02 9:33 UTC (permalink / raw) To: mike.looijmans; +Cc: peter, yocto Hi, On Mon, Dec 02, 2019 at 09:28:03AM +0000, Mike Looijmans wrote: > One solution I can think of is to put the header into it's own > recipe/repository and then refer to it like any other library would. Refer to > that recipe from the module (or kernel) recipe that needs it. This way you > have your header in a single maintainable location and dependencies properly > taken care of. > > If that's not something you could live with, share your recipes, since vague > problem descriptions will only get you vague solutions... This is the problem I see with multiple BSPs. In the end for every one of them the best solution is to fork linux-libc-headers. Adding custom recipes to export headers for every BSP kernel is not scaling so well. Sharing the BSPs and hacks around them would be nice but they are not public sadly :( Cheers, -Mikko ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-02 9:33 ` Mikko Rapeli @ 2019-12-02 14:01 ` Adrian Bunk 2019-12-02 14:09 ` Mikko Rapeli 0 siblings, 1 reply; 12+ messages in thread From: Adrian Bunk @ 2019-12-02 14:01 UTC (permalink / raw) To: Mikko Rapeli; +Cc: mike.looijmans, peter, yocto On Mon, Dec 02, 2019 at 09:33:33AM +0000, Mikko Rapeli wrote: >... > On Mon, Dec 02, 2019 at 09:28:03AM +0000, Mike Looijmans wrote: > > One solution I can think of is to put the header into it's own > > recipe/repository and then refer to it like any other library would. Refer to > > that recipe from the module (or kernel) recipe that needs it. This way you > > have your header in a single maintainable location and dependencies properly > > taken care of. > > > > If that's not something you could live with, share your recipes, since vague > > problem descriptions will only get you vague solutions... > > This is the problem I see with multiple BSPs. In the end for every one of them > the best solution is to fork linux-libc-headers. >... The fork can be a .bbappend with a patch adding the required definitions to linux-libc-headers. > Cheers, > > -Mikko cu Adrian ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-02 14:01 ` Adrian Bunk @ 2019-12-02 14:09 ` Mikko Rapeli 0 siblings, 0 replies; 12+ messages in thread From: Mikko Rapeli @ 2019-12-02 14:09 UTC (permalink / raw) To: bunk; +Cc: mike.looijmans, peter, yocto On Mon, Dec 02, 2019 at 04:01:41PM +0200, Adrian Bunk wrote: > On Mon, Dec 02, 2019 at 09:33:33AM +0000, Mikko Rapeli wrote: > >... > > On Mon, Dec 02, 2019 at 09:28:03AM +0000, Mike Looijmans wrote: > > > One solution I can think of is to put the header into it's own > > > recipe/repository and then refer to it like any other library would. Refer to > > > that recipe from the module (or kernel) recipe that needs it. This way you > > > have your header in a single maintainable location and dependencies properly > > > taken care of. > > > > > > If that's not something you could live with, share your recipes, since vague > > > problem descriptions will only get you vague solutions... > > > > This is the problem I see with multiple BSPs. In the end for every one of them > > the best solution is to fork linux-libc-headers. > >... > > The fork can be a .bbappend with a patch adding the required > definitions to linux-libc-headers. Yes, this is what I meant. Basically just adjust the SRC_URI and possibly PV through a linux-libc-headers_%.bbappend in product or BSP specific meta layer. Cheers, -Mikko ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-02 9:13 ` Mike Looijmans 2019-12-02 9:19 ` Mikko Rapeli @ 2019-12-02 13:44 ` Bruce Ashfield 2019-12-03 7:59 ` Peter Bergin 1 sibling, 1 reply; 12+ messages in thread From: Bruce Ashfield @ 2019-12-02 13:44 UTC (permalink / raw) To: Mike Looijmans; +Cc: peter, yocto On Mon, Dec 2, 2019 at 4:14 AM Mike Looijmans <mike.looijmans@topic.nl> wrote: > > On 01-12-19 22:57, Peter Bergin via Lists.Yoctoproject.Org wrote: > > Hi, > > > > I'm currently working in a project using Yocto 2.6 (thud) release. It has > > default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my > > project we will use kernel v4.1. I would like advice how to handle the > > linux-libc-headers package for my project, should I use the v4.18 headers or > > should I use the v4.1 header files which matches the running kernel? > > > > From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: > > "Kernel headers are backwards compatible, but not forwards compatible. This > > means that a program built against a C library using older kernel headers > > should run on a newer kernel (although it may not have access to new > > features), but a program built against newer kernel headers may not work on an > > older kernel." > > > > With the information from the quote above I would directly use v4.1 headers as > > my linux-libc-headers. But then reading the information in the file > > meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think > > another round. It states: > > > > " > > # You're probably looking here thinking you need to create some new copy > > # of linux-libc-headers since you have your own custom kernel. To put > > # this simply, you DO NOT. > > ... > > # There can also be a case where your kernel extremely old and you want > > # an older libc ABI for that old kernel. The headers installed by this > > # recipe should still be a standard mainline kernel, not your own custom > > # one. > > " > > > > The first part states that I should not change linux-libc-headers. But when I > > read the last part I'm not sure about the interpretation and it could be for > > my case. Just a matter of definition if v4.1 is extremely old compared to v4.18. > > > > Then another thing comes in to the equation; the LIBC ABI. When I look into > > the configuration of the glibc package it uses the configure switch > > "--enable-kernel=3.2" which means it shall be compatible with all kernel newer > > than v3.2. Then probably glibc is fine if it is compiled with v4.18 and run on > > v4.1? > > > > If building all applications against v4.18 headers but run on v4.1 kernel. I > > have a feeling that there potentially can be problems here. > > > > Please help me with some information about this and share your opinions? Are > > there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? > > The only drawback I see is that it will be a new configuration not well tested > > by the community. Are there other risks or drawbacks using your own version of > > linux-libc-headers? > > > It is not broken, so please don't fix it. > > OpenPLi has been using kernels way older than 4.1 with the kernel-headers > generated by OE/yocto and did not experience any problems with that. There's > about 50+ machines in there that have pre-built binary drivers that only work > with a particular kernel config and hence the old stuff. > > There are some corner-cases with exotic kernels and exotic exports and exotic > boot executables that use the kernel compiler, but I doubt that you're in there... > > If you have a kernel that exports something that's not in the regular headers, > it's way better to solve that using a syscall than trying to poke in low level > libc stuff. > > So again, if you don't experience problems, please don't try to fix it... This has been my experience as well. I've run a really wide set of BSP kernel's against the various "much newer" yocto/OE libc-headers over the years, and I've never hit an ABI or otherwise incompatibility. Like Mike is saying, if you probably have a feeling if your BSP is doing anything exotic and can explicitly test those problem areas, or put in place some specific workarounds if there are some changed or different interfaces. Cheers, Bruce > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > > View/Reply Online (#47483): https://lists.yoctoproject.org/g/yocto/message/47483 > Mute This Topic: https://lists.yoctoproject.org/mt/64460477/1050810 > Group Owner: yocto+owner@lists.yoctoproject.org > Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-02 13:44 ` Bruce Ashfield @ 2019-12-03 7:59 ` Peter Bergin 2019-12-03 15:28 ` Bruce Ashfield 0 siblings, 1 reply; 12+ messages in thread From: Peter Bergin @ 2019-12-03 7:59 UTC (permalink / raw) To: Bruce Ashfield; +Cc: Mike Looijmans, yocto On 2019-12-02 14:44, Bruce Ashfield wrote: > On Mon, Dec 2, 2019 at 4:14 AM Mike Looijmans <mike.looijmans@topic.nl> wrote: >> On 01-12-19 22:57, Peter Bergin via Lists.Yoctoproject.Org wrote: >>> Hi, >>> >>> I'm currently working in a project using Yocto 2.6 (thud) release. It has >>> default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my >>> project we will use kernel v4.1. I would like advice how to handle the >>> linux-libc-headers package for my project, should I use the v4.18 headers or >>> should I use the v4.1 header files which matches the running kernel? >>> >>> From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: >>> "Kernel headers are backwards compatible, but not forwards compatible. This >>> means that a program built against a C library using older kernel headers >>> should run on a newer kernel (although it may not have access to new >>> features), but a program built against newer kernel headers may not work on an >>> older kernel." >>> >>> With the information from the quote above I would directly use v4.1 headers as >>> my linux-libc-headers. But then reading the information in the file >>> meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think >>> another round. It states: >>> >>> " >>> # You're probably looking here thinking you need to create some new copy >>> # of linux-libc-headers since you have your own custom kernel. To put >>> # this simply, you DO NOT. >>> ... >>> # There can also be a case where your kernel extremely old and you want >>> # an older libc ABI for that old kernel. The headers installed by this >>> # recipe should still be a standard mainline kernel, not your own custom >>> # one. >>> " >>> >>> The first part states that I should not change linux-libc-headers. But when I >>> read the last part I'm not sure about the interpretation and it could be for >>> my case. Just a matter of definition if v4.1 is extremely old compared to v4.18. >>> >>> Then another thing comes in to the equation; the LIBC ABI. When I look into >>> the configuration of the glibc package it uses the configure switch >>> "--enable-kernel=3.2" which means it shall be compatible with all kernel newer >>> than v3.2. Then probably glibc is fine if it is compiled with v4.18 and run on >>> v4.1? >>> >>> If building all applications against v4.18 headers but run on v4.1 kernel. I >>> have a feeling that there potentially can be problems here. >>> >>> Please help me with some information about this and share your opinions? Are >>> there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? >>> The only drawback I see is that it will be a new configuration not well tested >>> by the community. Are there other risks or drawbacks using your own version of >>> linux-libc-headers? >> >> It is not broken, so please don't fix it. >> >> OpenPLi has been using kernels way older than 4.1 with the kernel-headers >> generated by OE/yocto and did not experience any problems with that. There's >> about 50+ machines in there that have pre-built binary drivers that only work >> with a particular kernel config and hence the old stuff. >> >> There are some corner-cases with exotic kernels and exotic exports and exotic >> boot executables that use the kernel compiler, but I doubt that you're in there... >> >> If you have a kernel that exports something that's not in the regular headers, >> it's way better to solve that using a syscall than trying to poke in low level >> libc stuff. >> >> So again, if you don't experience problems, please don't try to fix it... > This has been my experience as well. That's appealing approach. The part with the custom kernel headers is in my case solved that we have separate kernel headers that we add to the SDK. It is how I have understood the recommendation to solve this. > I've run a really wide set of BSP kernel's against the various "much > newer" yocto/OE libc-headers over the years, and I've never hit an ABI > or otherwise incompatibility. Actually there are issues with Yocto 2.6 together with v4.1 headers. I did an experiment to use v4.1 as linux-libc-headers. For my image i hit build failures in two packages, systemd and iptables. Those were easily fixed by upstream commits referenced below. I can not by a quick review say that those should cause run-time issues, probably not, but just showing that there are cases where packages not is fully aligned with all versions of kernel headers. systemd: https://github.com/systemd/systemd/commit/9c869d08d82c73f62ab3527567858ce4b0cf1257 iptables: https://git.netfilter.org/iptables/commit/?id=5beb1582d13d3bfdd0d2b277f5f3154b2fbf4a8e If you want the most deterministic build for one system and one setup I can not see any other option than using the same version for the kernel headers in both virtual/kernel and linux-libc-headers. Then if you want flexibility and possibility to support many builds it is better to have one common linux-libc-header version. Also from a test and validation point of view it is better if many people using the same version of linux-libc-headers instead of their own combination as we have a situation where this works almost all the time. Can anyone comment on this analyze, is it valid? Thanks, /Peter ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-03 7:59 ` Peter Bergin @ 2019-12-03 15:28 ` Bruce Ashfield 2019-12-04 7:46 ` Peter Bergin 0 siblings, 1 reply; 12+ messages in thread From: Bruce Ashfield @ 2019-12-03 15:28 UTC (permalink / raw) To: Peter Bergin; +Cc: Mike Looijmans, yocto On Tue, Dec 3, 2019 at 2:59 AM Peter Bergin <peter@berginkonsult.se> wrote: > > On 2019-12-02 14:44, Bruce Ashfield wrote: > > On Mon, Dec 2, 2019 at 4:14 AM Mike Looijmans <mike.looijmans@topic.nl> wrote: > >> On 01-12-19 22:57, Peter Bergin via Lists.Yoctoproject.Org wrote: > >>> Hi, > >>> > >>> I'm currently working in a project using Yocto 2.6 (thud) release. It has > >>> default kernel v4.18 and also linux-libc-headers from kernel v4.18. In my > >>> project we will use kernel v4.1. I would like advice how to handle the > >>> linux-libc-headers package for my project, should I use the v4.18 headers or > >>> should I use the v4.1 header files which matches the running kernel? > >>> > >>> From https://www.kernel.org/doc/html/latest/kbuild/headers_install.html: > >>> "Kernel headers are backwards compatible, but not forwards compatible. This > >>> means that a program built against a C library using older kernel headers > >>> should run on a newer kernel (although it may not have access to new > >>> features), but a program built against newer kernel headers may not work on an > >>> older kernel." > >>> > >>> With the information from the quote above I would directly use v4.1 headers as > >>> my linux-libc-headers. But then reading the information in the file > >>> meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc makes me think > >>> another round. It states: > >>> > >>> " > >>> # You're probably looking here thinking you need to create some new copy > >>> # of linux-libc-headers since you have your own custom kernel. To put > >>> # this simply, you DO NOT. > >>> ... > >>> # There can also be a case where your kernel extremely old and you want > >>> # an older libc ABI for that old kernel. The headers installed by this > >>> # recipe should still be a standard mainline kernel, not your own custom > >>> # one. > >>> " > >>> > >>> The first part states that I should not change linux-libc-headers. But when I > >>> read the last part I'm not sure about the interpretation and it could be for > >>> my case. Just a matter of definition if v4.1 is extremely old compared to v4.18. > >>> > >>> Then another thing comes in to the equation; the LIBC ABI. When I look into > >>> the configuration of the glibc package it uses the configure switch > >>> "--enable-kernel=3.2" which means it shall be compatible with all kernel newer > >>> than v3.2. Then probably glibc is fine if it is compiled with v4.18 and run on > >>> v4.1? > >>> > >>> If building all applications against v4.18 headers but run on v4.1 kernel. I > >>> have a feeling that there potentially can be problems here. > >>> > >>> Please help me with some information about this and share your opinions? Are > >>> there any risks at all to use v4.1 as linux-libc-headers in my Yocto build? > >>> The only drawback I see is that it will be a new configuration not well tested > >>> by the community. Are there other risks or drawbacks using your own version of > >>> linux-libc-headers? > >> > >> It is not broken, so please don't fix it. > >> > >> OpenPLi has been using kernels way older than 4.1 with the kernel-headers > >> generated by OE/yocto and did not experience any problems with that. There's > >> about 50+ machines in there that have pre-built binary drivers that only work > >> with a particular kernel config and hence the old stuff. > >> > >> There are some corner-cases with exotic kernels and exotic exports and exotic > >> boot executables that use the kernel compiler, but I doubt that you're in there... > >> > >> If you have a kernel that exports something that's not in the regular headers, > >> it's way better to solve that using a syscall than trying to poke in low level > >> libc stuff. > >> > >> So again, if you don't experience problems, please don't try to fix it... > > This has been my experience as well. > That's appealing approach. The part with the custom kernel headers is in > my case solved that we have separate kernel headers that we add to the > SDK. It is how I have understood the recommendation to solve this. > I'm assuming that you mean those headers are in a separate package and are installed into the dependent packages sysroot (or the SDK in your descriptin) via other means that linux-libc-headers .. if so, yes, that is the right thing to do. > > I've run a really wide set of BSP kernel's against the various "much > > newer" yocto/OE libc-headers over the years, and I've never hit an ABI > > or otherwise incompatibility. > Actually there are issues with Yocto 2.6 together with v4.1 headers. I > did an experiment to use v4.1 as linux-libc-headers. For my image i hit > build failures in two packages, systemd and iptables. Those were easily > fixed by upstream commits referenced below. I can not by a quick review > say that those should cause run-time issues, probably not, but just > showing that there are cases where packages not is fully aligned with > all versions of kernel headers. These are one of the cases that do pop up, and yes, we normally fix by updating (or patching) the packages. The same thing happens to me in master each time I introduce the updated libc-headers for the release. "They are normally just factoring/shuffling/churn on the header files requiring some different includes (or dropped ones), and don't impact the userspace -> kernel interfaces (i.e. the interface stability). Looking at the ones you referenced below, I'd agree that they won't impact the runtime. > > systemd: > https://github.com/systemd/systemd/commit/9c869d08d82c73f62ab3527567858ce4b0cf1257 > iptables: > https://git.netfilter.org/iptables/commit/?id=5beb1582d13d3bfdd0d2b277f5f3154b2fbf4a8e > > If you want the most deterministic build for one system and one setup I > can not see any other option than using the same version for the kernel > headers in both virtual/kernel and linux-libc-headers. Then if you want I wouldn't say deterministic, but yes, if you are only ever concerned about one kernel, one userspace, etc, then a more consistent build would be against the 1:1 matching headers. > flexibility and possibility to support many builds it is better to have > one common linux-libc-header version. Also from a test and validation The reality is that most builds run against any number of kernels (in particular if you are getting into binary builds), so you do need a stake in the ground for consistent headers -> many kernels, so yes, more flexibility and reuse by not varying the libc-heades .. since again, the are the libc-headers and the c-library does most of the abstraction as well. > point of view it is better if many people using the same version of > linux-libc-headers instead of their own combination as we have a > situation where this works almost all the time. Can anyone comment on > this analyze, is it valid? That's the premise around bumping the libc-headers to a reference version as I bump the various reference kernels. That we are getting more coverage and collective benefits by using a common base across many different userspace and kernel combinations. Cheers, Bruce > > Thanks, > /Peter > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [yocto] linux-libc-headers - how to handle for older kernels? 2019-12-03 15:28 ` Bruce Ashfield @ 2019-12-04 7:46 ` Peter Bergin 0 siblings, 0 replies; 12+ messages in thread From: Peter Bergin @ 2019-12-04 7:46 UTC (permalink / raw) To: Bruce Ashfield; +Cc: Mike Looijmans, yocto On 2019-12-03 16:28, Bruce Ashfield wrote: > I'm assuming that you mean those headers are in a separate package and > are installed into the dependent packages sysroot (or the SDK in your > descriptin) via other means that linux-libc-headers .. if so, yes, > that is the right thing to do. Thanks Bruce! Yes my plan is to create a new package in the 'virtual/kernel' recipe where I add the additional files to '/usr/include/linux/' and then add that new package to my SDK (or the sysroot if needed). Best regards, /Peter ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-12-04 7:46 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-12-01 21:57 linux-libc-headers - how to handle for older kernels? Peter Bergin 2019-12-02 9:01 ` [yocto] " Mikko Rapeli 2019-12-02 9:13 ` Mike Looijmans 2019-12-02 9:19 ` Mikko Rapeli 2019-12-02 9:28 ` Mike Looijmans 2019-12-02 9:33 ` Mikko Rapeli 2019-12-02 14:01 ` Adrian Bunk 2019-12-02 14:09 ` Mikko Rapeli 2019-12-02 13:44 ` Bruce Ashfield 2019-12-03 7:59 ` Peter Bergin 2019-12-03 15:28 ` Bruce Ashfield 2019-12-04 7:46 ` Peter Bergin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.