[PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[bkylerussell]

By having multiple uninative feeds based on host gcc version (rather than a
single uninative feed for all possible host gcc versions), we avoid a general
class of problems (as first observed in the bug below) where changes in native
gcc don't cause native sstate to regenerate.  Below is another example where
this problem is encountered across multiple supported host distros for a given
poky version.

gcc 6 supports default generation of PIE, which gets enabled between Ubuntu
16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros
of warrior.  Consider a native package dependency chain as follows:

     A
   / | \
  B  C  D

If a complete set of native sstate for the above packages is built prior
to the default PIE change, but a workstation upgrades its distro to a version
that natively compiles PIE by default, the sstate hash is not currently tainted,
so the newer compiler version will pull from sstate not compiled for PIE.

For example, if a source change to C causes part of the above chain to rebuild
(C -> A), then this may result in non-PIE sstate (B and D) giving link errors
with A.

ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
ld: final link failed: Nonrepresentable section on output

This appears to have been considered previously as an option to address
the following bug, but appears to have been set aside only because of
proximity to the end of a release cycle.

[YOCTO #10441]

Signed-off-by: Kyle Russell <bkylerussell@gmail.com>
---
 meta/lib/oe/utils.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index a84039f585..8afe19fd99 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
         bb.fatal("Can't get compiler version from %s --version output" % compiler)
 
     version = match.group(1)
-    return "-%s" % version if version in ("4.8", "4.9") else ""
+    return "-%s" % version
 
 
 def get_multilib_datastore(variant, d):
-- 
2.25.1

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[Richard Purdie]

On Mon, 2021-06-07 at 17:25 -0400, bkylerussell@gmail.com wrote:
> By having multiple uninative feeds based on host gcc version (rather than a
> single uninative feed for all possible host gcc versions), we avoid a general
> class of problems (as first observed in the bug below) where changes in native
> gcc don't cause native sstate to regenerate.  Below is another example where
> this problem is encountered across multiple supported host distros for a given
> poky version.
> 
> gcc 6 supports default generation of PIE, which gets enabled between Ubuntu
> 16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros
> of warrior.  Consider a native package dependency chain as follows:
> 
>      A
>    / | \
>   B  C  D
> 
> If a complete set of native sstate for the above packages is built prior
> to the default PIE change, but a workstation upgrades its distro to a version
> that natively compiles PIE by default, the sstate hash is not currently tainted,
> so the newer compiler version will pull from sstate not compiled for PIE.
> 
> For example, if a source change to C causes part of the above chain to rebuild
> (C -> A), then this may result in non-PIE sstate (B and D) giving link errors
> with A.
> 
> ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
> ld: final link failed: Nonrepresentable section on output
> 
> This appears to have been considered previously as an option to address
> the following bug, but appears to have been set aside only because of
> proximity to the end of a release cycle.
> 
> [YOCTO #10441]

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441

This doesn't look quite right?

> Signed-off-by: Kyle Russell <bkylerussell@gmail.com>
> ---
>  meta/lib/oe/utils.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
> index a84039f585..8afe19fd99 100644
> --- a/meta/lib/oe/utils.py
> +++ b/meta/lib/oe/utils.py
> @@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
>          bb.fatal("Can't get compiler version from %s --version output" % compiler)
>  
> 
> 
> 
> 
> 
> 
> 
>      version = match.group(1)
> -    return "-%s" % version if version in ("4.8", "4.9") else ""
> +    return "-%s" % version

This is a huge performance penalty to put onto the build unfortunately and I really
don't want to do that by default on the autobuilder. We haven't seen many of these
errors on the autobuilder and I'm reluctant to take the performance hit on our testing.

There were specific issues with gcc 4.8 and 4.9 which required separate sstate. Do we
know which gcc version switched the default and can we force older gcc's to use the 
same default or does it not work? I'm wondering if we could split off a "nopic" verison
of sstate for example for the older gccs?

Cheers,

Richard

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[Phil Blundell]

[-- Attachment #1: Type: text/plain, Size: 2603 bytes --]

Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.

p.


On 7 June 2021 22:25:58 BST, bkylerussell@gmail.com wrote:
>By having multiple uninative feeds based on host gcc version (rather
>than a
>single uninative feed for all possible host gcc versions), we avoid a
>general
>class of problems (as first observed in the bug below) where changes in
>native
>gcc don't cause native sstate to regenerate.  Below is another example
>where
>this problem is encountered across multiple supported host distros for
>a given
>poky version.
>
>gcc 6 supports default generation of PIE, which gets enabled between
>Ubuntu
>16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported
>distros
>of warrior.  Consider a native package dependency chain as follows:
>
>     A
>   / | \
>  B  C  D
>
>If a complete set of native sstate for the above packages is built
>prior
>to the default PIE change, but a workstation upgrades its distro to a
>version
>that natively compiles PIE by default, the sstate hash is not currently
>tainted,
>so the newer compiler version will pull from sstate not compiled for
>PIE.
>
>For example, if a source change to C causes part of the above chain to
>rebuild
>(C -> A), then this may result in non-PIE sstate (B and D) giving link
>errors
>with A.
>
>ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used
>when making a PIE object; recompile with -fPIC
>ld: final link failed: Nonrepresentable section on output
>
>This appears to have been considered previously as an option to address
>the following bug, but appears to have been set aside only because of
>proximity to the end of a release cycle.
>
>[YOCTO #10441]
>
>Signed-off-by: Kyle Russell <bkylerussell@gmail.com>
>---
> meta/lib/oe/utils.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
>index a84039f585..8afe19fd99 100644
>--- a/meta/lib/oe/utils.py
>+++ b/meta/lib/oe/utils.py
>@@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
>bb.fatal("Can't get compiler version from %s --version output" %
>compiler)
> 
>     version = match.group(1)
>-    return "-%s" % version if version in ("4.8", "4.9") else ""
>+    return "-%s" % version
> 
> 
> def get_multilib_datastore(variant, d):
>-- 
>2.25.1

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2868 bytes --]

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[bkylerussell]

[-- Attachment #1: Type: text/plain, Size: 1705 bytes --]

On Mon, Jun 7, 2021 at 5:43 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441
>
> This doesn't look quite right?
>

Yes, that's the one.  I assumed this was what you meant by, "Another option
would be to have multiple uninative feeds based on gcc
version rather than a single uninative one."

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441#c1

This is a huge performance penalty to put onto the build unfortunately and
> I really
> don't want to do that by default on the autobuilder. We haven't seen many
> of these
> errors on the autobuilder and I'm reluctant to take the performance hit on
> our testing.
>

I realize this is not ideal, but I guess the trade-off is dealing with
these issues as they arise on
a case-by-case basis.

There were specific issues with gcc 4.8 and 4.9 which required separate
> sstate. Do we
> know which gcc version switched the default and can we force older gcc's
> to use the
> same default or does it not work? I'm wondering if we could split off a
> "nopic" verison
> of sstate for example for the older gccs?
>

gcc 6 introduced an --enable-default-pie configure option, but I'm not
exactly sure where
between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option
up.  Since packages
based on the older gcc already exist in sstate, I think we would have to
force the newer
behavior on each affected package.

I realize we can just swizzle the compiler flags on the affected packages
to make things happy,
but it seemed like this approach had at least been considered before, so I
thought I would at least
get some feedback.

[-- Attachment #2: Type: text/html, Size: 2694 bytes --]

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[Andrea Adami]

On Tue, Jun 8, 2021 at 12:02 AM Phil Blundell via
lists.openembedded.org <pb=pbcl.net@lists.openembedded.org> wrote:
>
> Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.
>
> p.
>

I was exactly wondering about distro/include/uninative-flags.inc which
at the time was supposed to fix the changes btw ubuntu 16.x and 18.x.
But this doesn't seem included nowhere.

A.A.

>
> On 7 June 2021 22:25:58 BST, bkylerussell@gmail.com wrote:
>>
>> By having multiple uninative feeds based on host gcc version (rather than a
>> single uninative feed for all possible host gcc versions), we avoid a general
>> class of problems (as first observed in the bug below) where changes in native
>> gcc don't cause native sstate to regenerate.  Below is another example where
>> this problem is encountered across multiple supported host distros for a given
>> poky version.
>>
>> gcc 6 supports default generation of PIE, which gets enabled between Ubuntu
>> 16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros
>> of warrior.  Consider a native package dependency chain as follows:
>>
>>      A
>>    / | \
>>   B  C  D
>>
>> If a complete set of native sstate for the above packages is built prior
>> to the default PIE change, but a workstation upgrades its distro to a version
>> that natively compiles PIE by default, the sstate hash is not currently tainted,
>> so the newer compiler version will pull from sstate not compiled for PIE.
>>
>> For example, if a source change to C causes part of the above chain to rebuild
>> (C -> A), then this may result in non-PIE sstate (B and D) giving link errors
>> with A.
>>
>> ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
>> ld: final link failed: Nonrepresentable section on output
>>
>> This appears to have been considered previously as an option to address
>> the following bug, but appears to have been set aside only because of
>> proximity to the end of a release cycle.
>>
>> [YOCTO #10441]
>>
>> Signed-off-by: Kyle Russell <bkylerussell@gmail.com>
>> ________________________________
>>  meta/lib/oe/utils.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
>> index a84039f585..8afe19fd99 100644
>> --- a/meta/lib/oe/utils.py
>> +++ b/meta/lib/oe/utils.py
>> @@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
>>          bb.fatal("Can't get compiler version from %s --version output" % compiler)
>>
>>      version = match.group(1)
>> -    return "-%s" % version if version in ("4.8", "4.9") else ""
>> +    return "-%s" % version
>>
>>
>>  def get_multilib_datastore(variant, d):
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> 
>

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[Richard Purdie]

On Tue, 2021-06-08 at 11:20 +0200, Andrea Adami wrote:
> On Tue, Jun 8, 2021 at 12:02 AM Phil Blundell via
> lists.openembedded.org <pb=pbcl.net@lists.openembedded.org> wrote:
> > 
> > Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.
> > 
> > p.
> > 
> 
> I was exactly wondering about distro/include/uninative-flags.inc which
> at the time was supposed to fix the changes btw ubuntu 16.x and 18.x.
> But this doesn't seem included nowhere.

Well remembered, I'd fogotten that. It is included by default:

$ grep uninative-flags * -R

meta/conf/distro/defaultsetup.conf:require conf/distro/include/uninative-flags.inc

Cheers,

Richard

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[Andrea Adami]

On Tue, Jun 8, 2021 at 11:48 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2021-06-08 at 11:20 +0200, Andrea Adami wrote:
> > On Tue, Jun 8, 2021 at 12:02 AM Phil Blundell via
> > lists.openembedded.org <pb=pbcl.net@lists.openembedded.org> wrote:
> > >
> > > Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.
> > >
> > > p.
> > >
> >
> > I was exactly wondering about distro/include/uninative-flags.inc which
> > at the time was supposed to fix the changes btw ubuntu 16.x and 18.x.
> > But this doesn't seem included nowhere.
>
> Well remembered, I'd fogotten that. It is included by default:
>
> $ grep uninative-flags * -R
>
> meta/conf/distro/defaultsetup.conf:require conf/distro/include/uninative-flags.inc
>
> Cheers,
>
> Richard
>
Oops, right, I was looking at distro includes but this is included by
bitbake.conf
A.A.

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[Richard Purdie]

On Mon, 2021-06-07 at 18:49 -0400, Kyle Russell wrote:
> On Mon, Jun 7, 2021 at 5:43 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> > 
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441
> > 
> > This doesn't look quite right?
> > 
> 
> 
> Yes, that's the one.  I assumed this was what you meant by, "Another option would 
> be to have multiple uninative feeds based on gcc version rather than a single uninative one."

That is effectively what the code ended up doing and why 4.8 and 4.9
are split out. We only wanted to separate out the specific versions 
with issues as we wanted to allow reuse where we could.

> https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441#c1
> 
> > This is a huge performance penalty to put onto the build unfortunately and I really
> > don't want to do that by default on the autobuilder. We haven't seen many of these
> > errors on the autobuilder and I'm reluctant to take the performance hit on our testing.
> > 
> 
> I realize this is not ideal, but I guess the trade-off is dealing with these issues 
> as they arise on a case-by-case basis.

If they were many issues arriving that might be what we'd have to do but so
far I've not seen reports of that many...

> 
> > There were specific issues with gcc 4.8 and 4.9 which required separate sstate. Do we
> > know which gcc version switched the default and can we force older gcc's to use the 
> > same default or does it not work? I'm wondering if we could split off a "nopic" verison
> > of sstate for example for the older gccs?
> > 
> 
> 
> gcc 6 introduced an --enable-default-pie configure option, but I'm not exactly sure where
> between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option up.  Since packages
> based on the older gcc already exist in sstate, I think we would have to force the newer
> behavior on each affected package.
> 
> I realize we can just swizzle the compiler flags on the affected packages to make things happy,
> but it seemed like this approach had at least been considered before, so I thought I would at least
> get some feedback.

As Andrea points out, we did fix this with a global include file which should have
handled the issue. Are you using a release which doesn't have that or not using
the include through your own custom distro?

Cheers,

Richard

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[bkylerussell]

[-- Attachment #1: Type: text/plain, Size: 3134 bytes --]

On Tue, Jun 8, 2021 at 7:16 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> > > There were specific issues with gcc 4.8 and 4.9 which required
> separate sstate. Do we
> > > know which gcc version switched the default and can we force older
> gcc's to use the
> > > same default or does it not work? I'm wondering if we could split off
> a "nopic" verison
> > > of sstate for example for the older gccs?
> > >
> > gcc 6 introduced an --enable-default-pie configure option, but I'm not
> exactly sure where
> > between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option
> up.  Since packages
> > based on the older gcc already exist in sstate, I think we would have to
> force the newer
> > behavior on each affected package.
> >
> > I realize we can just swizzle the compiler flags on the affected
> packages to make things happy,
> > but it seemed like this approach had at least been considered before, so
> I thought I would at least
> > get some feedback.
>
> As Andrea points out, we did fix this with a global include file which
> should have
> handled the issue. Are you using a release which doesn't have that or not
> using
> the include through your own custom distro?
>

Yes indeed, nice find.  bitbake -e confirms I'm including
uninative-flags.inc.  I do think this is exactly the
same issue, but there's a different set of dependencies involved in my
chain.  As someone suggested earlier,
I can force -fPIE on the packages where this is a problem, so I could of
course do something similar for those
packages.

But I still feel like there's an underlying uninative limitation that's not
being addressed, or highlighted.  For example,
the SSTATE_MIRRORS documentation seems to imply that differentiating
between uninative based on native gcc
version is the normal/correct behavior, even though it's clear to me now
that the example was chosen specifically
for the 4.8/4.9 issue, not just the general case.

"When pointing to sstate build artifacts on another machine that uses a
different GCC version for native builds,
you must configure SSTATE_MIRROR with a regular expression that maps local
search paths to server paths.
The paths need to take into account NATIVELSBSTRING set by the uninative
class."
 -
https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-SSTATE_MIRRORS

But I suppose even similar versions of gcc across different supported
distros could be configured differently.
For example, warrior claims support for Ubuntu 16.04, 18.04, and Fedora 28,
but only Ubuntu 18.04 seems to
use --enable-default-pie based on gcc -v.  (Even newer Fedoras seem to not
use this flag.)

I guess we could have a "nopic" split like you suggested.  Or maybe
uninative itself should be hashed against
the native gcc configure args and version, since version by itself also
doesn't seem to be sufficient based on the
Fedora example above.  Neither one of those sounds particularly appealing
the more I think about it, so maybe
I'll just opt for fixing the packages like we've done for binutils-native.

[-- Attachment #2: Type: text/html, Size: 4289 bytes --]