[PATCH] gcc-sanitizers: Move content from gcclibdir into libdir

[PATCH] gcc-sanitizers: Move content from gcclibdir into libdir

[Mike Crowe]

In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> OE does not use the traditional /usr/lib/gcc prefix to store
> gcc-runtime it basically is moved into libdir, however some newer
> files were installed by newer versions of gcc especially libgomp (
> omp.h openacc.h ) into gcclibdir, so we have content in both
> directories, this confuses other tools which are trying to guess the
> gcc installation and its runtime location, since now we have two
> directories, the tools either choose one or other and we get
> inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> clang for riscv64 chose /usr/lib/gcc

> This change ensures that OE ends up with single valid location for gcc
> runtime files

I think that the same thing needs to happen in gcc-sanitizers.inc,
otherwise I get errors like:

| .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error: sanitizer/lsan_interface.h: No such file or directory

when attempting to compile with sanitizers enabled.

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Cc: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-devtools/gcc/gcc-sanitizers.inc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
index 668e14a59f..67b755edf8 100644
--- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
+++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
@@ -35,6 +35,11 @@ do_compile () {
 do_install () {
     cd ${B}/${TARGET_SYS}/libsanitizer/
     oe_runmake 'DESTDIR=${D}' MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/ install
+    if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
+	install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
+	mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/* ${D}${libdir}/${TARGET_SYS}/${BINV}/include
+	rmdir --ignore-fail-on-non-empty -p ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
+    fi
     if [ -d ${D}${infodir} ]; then
         rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
     fi
-- 
2.20.1

Re: [PATCH] gcc-sanitizers: Move content from gcclibdir into libdir

[Khem Raj]

On Thu, Feb 25, 2021 at 9:37 AM Mike Crowe <mac@mcrowe.com> wrote:
>
> In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> > OE does not use the traditional /usr/lib/gcc prefix to store
> > gcc-runtime it basically is moved into libdir, however some newer
> > files were installed by newer versions of gcc especially libgomp (
> > omp.h openacc.h ) into gcclibdir, so we have content in both
> > directories, this confuses other tools which are trying to guess the
> > gcc installation and its runtime location, since now we have two
> > directories, the tools either choose one or other and we get
> > inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> > clang for riscv64 chose /usr/lib/gcc
>
> > This change ensures that OE ends up with single valid location for gcc
> > runtime files
>
> I think that the same thing needs to happen in gcc-sanitizers.inc,
> otherwise I get errors like:
>

right. Seems a good fix thanks

> | .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error: sanitizer/lsan_interface.h: No such file or directory
>
> when attempting to compile with sanitizers enabled.
>
> Signed-off-by: Mike Crowe <mac@mcrowe.com>
> Cc: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/recipes-devtools/gcc/gcc-sanitizers.inc | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> index 668e14a59f..67b755edf8 100644
> --- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> +++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> @@ -35,6 +35,11 @@ do_compile () {
>  do_install () {
>      cd ${B}/${TARGET_SYS}/libsanitizer/
>      oe_runmake 'DESTDIR=${D}' MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/ install
> +    if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
> +       install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> +       mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/* ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> +       rmdir --ignore-fail-on-non-empty -p ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
> +    fi
>      if [ -d ${D}${infodir} ]; then
>          rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
>      fi
> --
> 2.20.1
>

Re: [OE-core] [PATCH] gcc-sanitizers: Move content from gcclibdir into libdir

[Alexandre Belloni]

Hello,

On 25/02/2021 17:36:53+0000, Mike Crowe via lists.openembedded.org wrote:
> In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> > OE does not use the traditional /usr/lib/gcc prefix to store
> > gcc-runtime it basically is moved into libdir, however some newer
> > files were installed by newer versions of gcc especially libgomp (
> > omp.h openacc.h ) into gcclibdir, so we have content in both
> > directories, this confuses other tools which are trying to guess the
> > gcc installation and its runtime location, since now we have two
> > directories, the tools either choose one or other and we get
> > inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> > clang for riscv64 chose /usr/lib/gcc
> 
> > This change ensures that OE ends up with single valid location for gcc
> > runtime files
> 
> I think that the same thing needs to happen in gcc-sanitizers.inc,
> otherwise I get errors like:
> 
> | .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error: sanitizer/lsan_interface.h: No such file or directory
> 
> when attempting to compile with sanitizers enabled.
> 
> Signed-off-by: Mike Crowe <mac@mcrowe.com>
> Cc: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/recipes-devtools/gcc/gcc-sanitizers.inc | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> index 668e14a59f..67b755edf8 100644
> --- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> +++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> @@ -35,6 +35,11 @@ do_compile () {
>  do_install () {
>      cd ${B}/${TARGET_SYS}/libsanitizer/
>      oe_runmake 'DESTDIR=${D}' MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/ install
> +    if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
> +	install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> +	mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/* ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> +	rmdir --ignore-fail-on-non-empty -p ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
> +    fi
>      if [ -d ${D}${infodir} ]; then
>          rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
>      fi

This seems to result in the following erro on the autobuilders:

ERROR: gcc-sanitizers-10.2.0-r0 do_package: QA Issue: gcc-sanitizers: Files/directories were installed but not shipped in any package:
  /usr/lib/i686-poky-linux
  /usr/lib/i686-poky-linux/10.2.0
  /usr/lib/i686-poky-linux/10.2.0/include
  /usr/lib/i686-poky-linux/10.2.0/include/sanitizer
  /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/common_interface_defs.h
  /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/lsan_interface.h
  /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/asan_interface.h
  /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/tsan_interface.h
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
gcc-sanitizers: 8 installed and not shipped files. [installed-vs-shipped]
ERROR: gcc-sanitizers-10.2.0-r0 do_package: Fatal QA errors found, failing task.

It seems to be an easy one to fix.

Regards,

-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] gcc-sanitizers: Move content from gcclibdir into libdir

[Khem Raj]

thanks Alexandre

I think new dir needs to be added to FILES directive as well.

On Sat, Feb 27, 2021 at 7:02 AM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> Hello,
>
> On 25/02/2021 17:36:53+0000, Mike Crowe via lists.openembedded.org wrote:
> > In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> > > OE does not use the traditional /usr/lib/gcc prefix to store
> > > gcc-runtime it basically is moved into libdir, however some newer
> > > files were installed by newer versions of gcc especially libgomp (
> > > omp.h openacc.h ) into gcclibdir, so we have content in both
> > > directories, this confuses other tools which are trying to guess the
> > > gcc installation and its runtime location, since now we have two
> > > directories, the tools either choose one or other and we get
> > > inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> > > clang for riscv64 chose /usr/lib/gcc
> >
> > > This change ensures that OE ends up with single valid location for gcc
> > > runtime files
> >
> > I think that the same thing needs to happen in gcc-sanitizers.inc,
> > otherwise I get errors like:
> >
> > | .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error: sanitizer/lsan_interface.h: No such file or directory
> >
> > when attempting to compile with sanitizers enabled.
> >
> > Signed-off-by: Mike Crowe <mac@mcrowe.com>
> > Cc: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/recipes-devtools/gcc/gcc-sanitizers.inc | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > index 668e14a59f..67b755edf8 100644
> > --- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > @@ -35,6 +35,11 @@ do_compile () {
> >  do_install () {
> >      cd ${B}/${TARGET_SYS}/libsanitizer/
> >      oe_runmake 'DESTDIR=${D}' MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/ install
> > +    if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
> > +     install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> > +     mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/* ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> > +     rmdir --ignore-fail-on-non-empty -p ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
> > +    fi
> >      if [ -d ${D}${infodir} ]; then
> >          rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
> >      fi
>
> This seems to result in the following erro on the autobuilders:
>
> ERROR: gcc-sanitizers-10.2.0-r0 do_package: QA Issue: gcc-sanitizers: Files/directories were installed but not shipped in any package:
>   /usr/lib/i686-poky-linux
>   /usr/lib/i686-poky-linux/10.2.0
>   /usr/lib/i686-poky-linux/10.2.0/include
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/common_interface_defs.h
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/lsan_interface.h
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/asan_interface.h
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/tsan_interface.h
> Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
> gcc-sanitizers: 8 installed and not shipped files. [installed-vs-shipped]
> ERROR: gcc-sanitizers-10.2.0-r0 do_package: Fatal QA errors found, failing task.
>
> It seems to be an easy one to fix.
>
> Regards,
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

Re: [OE-core] [PATCH] gcc-sanitizers: Move content from gcclibdir into libdir

[Mike Crowe]

On Saturday 27 February 2021 at 16:02:18 +0100, Alexandre Belloni wrote:
> Hello,
> 
> On 25/02/2021 17:36:53+0000, Mike Crowe via lists.openembedded.org wrote:
> > In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> > > OE does not use the traditional /usr/lib/gcc prefix to store
> > > gcc-runtime it basically is moved into libdir, however some newer
> > > files were installed by newer versions of gcc especially libgomp (
> > > omp.h openacc.h ) into gcclibdir, so we have content in both
> > > directories, this confuses other tools which are trying to guess the
> > > gcc installation and its runtime location, since now we have two
> > > directories, the tools either choose one or other and we get
> > > inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> > > clang for riscv64 chose /usr/lib/gcc
> > 
> > > This change ensures that OE ends up with single valid location for gcc
> > > runtime files
> > 
> > I think that the same thing needs to happen in gcc-sanitizers.inc,
> > otherwise I get errors like:
> > 
> > | .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error: sanitizer/lsan_interface.h: No such file or directory
> > 
> > when attempting to compile with sanitizers enabled.
> > 
> > Signed-off-by: Mike Crowe <mac@mcrowe.com>
> > Cc: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/recipes-devtools/gcc/gcc-sanitizers.inc | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > index 668e14a59f..67b755edf8 100644
> > --- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > @@ -35,6 +35,11 @@ do_compile () {
> >  do_install () {
> >      cd ${B}/${TARGET_SYS}/libsanitizer/
> >      oe_runmake 'DESTDIR=${D}' MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/ install
> > +    if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
> > +	install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> > +	mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/* ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> > +	rmdir --ignore-fail-on-non-empty -p ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
> > +    fi
> >      if [ -d ${D}${infodir} ]; then
> >          rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
> >      fi
> 
> This seems to result in the following erro on the autobuilders:
> 
> ERROR: gcc-sanitizers-10.2.0-r0 do_package: QA Issue: gcc-sanitizers: Files/directories were installed but not shipped in any package:
>   /usr/lib/i686-poky-linux
>   /usr/lib/i686-poky-linux/10.2.0
>   /usr/lib/i686-poky-linux/10.2.0/include
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/common_interface_defs.h
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/lsan_interface.h
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/asan_interface.h
>   /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/tsan_interface.h
> Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
> gcc-sanitizers: 8 installed and not shipped files. [installed-vs-shipped]
> ERROR: gcc-sanitizers-10.2.0-r0 do_package: Fatal QA errors found, failing task.
> 
> It seems to be an easy one to fix.

Removing the gcc/ part of the path added to FILES_${PN} seems to solve the
problem for me. I'm not sure why I didn't see that in the dunfell/usrmerge
tree that I originally wrote the fix for.

Updated patch at:
https://lists.openembedded.org/g/openembedded-core/topic/patch_v2_gcc_sanitizers/80970998?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,80970998

Thanks.

Mike.