All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
@ 2018-09-25 10:28 Mikko Rapeli
  2018-09-25 10:44 ` Mikko.Rapeli
  0 siblings, 1 reply; 8+ messages in thread
From: Mikko Rapeli @ 2018-09-25 10:28 UTC (permalink / raw)
  To: openembedded-core

And enable them by default as ERROR_QA. Reason is that
absolute build directory paths in CMake .cmake modules
and in pkg-config .pc files cause recipe builds to escape
their recipe specific sysroots and triggers hard to debug
and timing sensitive build failures. It's better to fail
early.

A failure from sumo version of libical looks like:

ERROR: libical-2.0.0-r0 do_package_qa: QA Issue: CMake module /work/i586-poky-linux/libical/2.0.0-r0/packages-split/libical-dev/usr/lib/cmake/LibIcal/LibIcalTargets-noconfig.cmake contains reference to tmpdir which causes build raceconditions between recipes [buildpaths_cmake]
ERROR: libical-2.0.0-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
ERROR: libical-2.0.0-r0 do_package_qa: Function failed: do_package_qa
ERROR: Logfile of failure stored in: /home/builder/src/yocto/poky/build/tmp/work/i586-poky-linux/libical/2.0.0-r0/temp/log.do_package_qa.4934
NOTE: recipe libical-2.0.0-r0: task do_package_qa: Failed
ERROR: Task (/home/builder/src/yocto/poky/meta/recipes-support/libical/libical_2.0.0.bb:do_package_qa) failed with exit code '1'

For some reason libical from poky master branch is not affected.

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
---
 meta/classes/insane.bbclass | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index eb2d967..a69e3f8 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -33,7 +33,8 @@ ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
             perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
             split-strip packages-list pkgv-undefined var-undefined \
             version-going-backwards expanded-d invalid-chars \
-            license-checksum dev-elf file-rdeps \
+            license-checksum dev-elf file-rdeps buildpaths_cmake \
+            buildpaths_pkgconfig \
             "
 # Add usrmerge QA check based on distro feature
 ERROR_QA_append = "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', ' usrmerge', '', d)}"
@@ -516,9 +517,30 @@ def package_qa_hash_style(path, name, d, elf, messages):
     if has_syms and not sane:
         package_qa_add_message(messages, "ldflags", "No GNU_HASH in the elf binary: '%s'" % path)
 
+QAPATHTEST[buildpaths_cmake] = "package_qa_check_buildpaths_cmake"
+def package_qa_check_buildpaths_cmake(path, name, d, elf, messages):
+    """
+    Check for build paths inside target CMake files which cause build
+    race conditions between recipes sysroots.
+    """
+    if path.lower().endswith(".cmake"):
+        message = "CMake module %s contains reference to tmpdir which causes build raceconditions between recipes" % package_qa_clean_path(path,d)
+        package_qa_check_buildpaths(path, name, d, elf, messages, qacheck="buildpaths_cmake", qamessage=message)
+
+
+QAPATHTEST[buildpaths_pkgconfig] = "package_qa_check_buildpaths_pkgconfig"
+def package_qa_check_buildpaths_pkgconfig(path, name, d, elf, messages):
+    """
+    Check for build paths inside target pkg-config files which cause build
+    race conditions between recipe sysroots.
+    """
+    if path.lower().endswith(".pc"):
+        message = "pkg-config file %s contains reference to tmpdir which causes build raceconditions between recipes" % package_qa_clean_path(path,d)
+        package_qa_check_buildpaths(path, name, d, elf, messages, qacheck="buildpaths_pkgconfig", qamessage=message)
+
 
 QAPATHTEST[buildpaths] = "package_qa_check_buildpaths"
-def package_qa_check_buildpaths(path, name, d, elf, messages):
+def package_qa_check_buildpaths(path, name, d, elf, messages, qacheck="buildpaths", qamessage=None):
     """
     Check for build paths inside target files and error if not found in the whitelist
     """
@@ -538,7 +560,9 @@ def package_qa_check_buildpaths(path, name, d, elf, messages):
     with open(path, 'rb') as f:
         file_content = f.read()
         if tmpdir in file_content:
-            package_qa_add_message(messages, "buildpaths", "File %s in package contained reference to tmpdir" % package_qa_clean_path(path,d))
+            if not qamessage:
+                qamessage = "File %s in package contained reference to tmpdir" % package_qa_clean_path(path,d)
+            package_qa_add_message(messages, qacheck, qamessage)
 
 
 QAPATHTEST[xorg-driver-abi] = "package_qa_check_xorg_driver_abi"
-- 
1.9.1



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
  2018-09-25 10:28 [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks Mikko Rapeli
@ 2018-09-25 10:44 ` Mikko.Rapeli
  2018-09-25 11:08   ` Richard Purdie
  0 siblings, 1 reply; 8+ messages in thread
From: Mikko.Rapeli @ 2018-09-25 10:44 UTC (permalink / raw)
  To: openembedded-core

On Tue, Sep 25, 2018 at 01:28:13PM +0300, Mikko Rapeli wrote:
> And enable them by default as ERROR_QA. Reason is that
> absolute build directory paths in CMake .cmake modules
> and in pkg-config .pc files cause recipe builds to escape
> their recipe specific sysroots and triggers hard to debug
> and timing sensitive build failures. It's better to fail
> early.
> 
> A failure from sumo version of libical looks like:
> 
> ERROR: libical-2.0.0-r0 do_package_qa: QA Issue: CMake module /work/i586-poky-linux/libical/2.0.0-r0/packages-split/libical-dev/usr/lib/cmake/LibIcal/LibIcalTargets-noconfig.cmake contains reference to tmpdir which causes build raceconditions between recipes [buildpaths_cmake]
> ERROR: libical-2.0.0-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
> ERROR: libical-2.0.0-r0 do_package_qa: Function failed: do_package_qa
> ERROR: Logfile of failure stored in: /home/builder/src/yocto/poky/build/tmp/work/i586-poky-linux/libical/2.0.0-r0/temp/log.do_package_qa.4934
> NOTE: recipe libical-2.0.0-r0: task do_package_qa: Failed
> ERROR: Task (/home/builder/src/yocto/poky/meta/recipes-support/libical/libical_2.0.0.bb:do_package_qa) failed with exit code '1'
> 
> For some reason libical from poky master branch is not affected.

The reason why master branch is not affected is:

commit 26cccb93059b8963651b7d17cea2ee95f52633b7
Author: Juro Bystricky <juro.bystricky@intel.com>
Date:   Tue Mar 20 15:36:36 2018 -0700

    libical-dev_2.0: improve reproducibility
    
    Remove build host references from distributed files.
    
    (From OE-Core rev: 20f2670e755bcbf90b2b6c08192c022fe7e7eaad)
    
    Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
    Signed-off-by: Ross Burton <ross.burton@intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/meta/recipes-support/libical/libical_2.0.0.bb b/meta/recipes-support/libical/libical_2.0.0.
index dcc21cc..daa47ab 100644
--- a/meta/recipes-support/libical/libical_2.0.0.bb
+++ b/meta/recipes-support/libical/libical_2.0.0.bb
@@ -17,3 +17,10 @@ SRC_URI[sha256sum] = "654c11f759c19237be39f6ad401d917e5a05f36f1736385ed958e60cf2
 UPSTREAM_CHECK_URI = "https://github.com/libical/libical/releases"
 
 inherit cmake pkgconfig
+
+do_install_append_class-target () {
+    # Remove build host references
+    sed -i \
+       -e 's,${STAGING_LIBDIR},${libdir},g' \
+       ${D}${libdir}/cmake/LibIcal/LibIcalTargets-noconfig.cmake
+}

Now I'm struggling horribly with the same problem in various custom
CMake modules, so how about doing that same STAGING_LIBDIR to libdir
switch for all CMake modules automatically in cmake.bbclass?

I tried to correctly fix the libical CMake files too with help from
#cmake, but eventually had to give up. libical has an embedded
FindICU.cmake module but it seems like the upstream FindICU.cmake ends
up producing the same absolute paths while it does fix some other
bugs, and of course introduces new ones like no longer providing
ICU_I18N_FOUND variable when i18n ICU module is found.

-Mikko

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
  2018-09-25 10:44 ` Mikko.Rapeli
@ 2018-09-25 11:08   ` Richard Purdie
  2018-09-25 11:21     ` Mikko.Rapeli
  0 siblings, 1 reply; 8+ messages in thread
From: Richard Purdie @ 2018-09-25 11:08 UTC (permalink / raw)
  To: Mikko.Rapeli, openembedded-core

On Tue, 2018-09-25 at 10:44 +0000, Mikko.Rapeli@bmw.de wrote:
> On Tue, Sep 25, 2018 at 01:28:13PM +0300, Mikko Rapeli wrote:
> > And enable them by default as ERROR_QA. Reason is that
> > absolute build directory paths in CMake .cmake modules
> > and in pkg-config .pc files cause recipe builds to escape
> > their recipe specific sysroots and triggers hard to debug
> > and timing sensitive build failures. It's better to fail
> > early.
> > 
> > A failure from sumo version of libical looks like:
> > 
> > ERROR: libical-2.0.0-r0 do_package_qa: QA Issue: CMake module
> > /work/i586-poky-linux/libical/2.0.0-r0/packages-split/libical-
> > dev/usr/lib/cmake/LibIcal/LibIcalTargets-noconfig.cmake contains
> > reference to tmpdir which causes build raceconditions between
> > recipes [buildpaths_cmake]
> > ERROR: libical-2.0.0-r0 do_package_qa: QA run found fatal errors.
> > Please consider fixing them.
> > ERROR: libical-2.0.0-r0 do_package_qa: Function failed:
> > do_package_qa
> > ERROR: Logfile of failure stored in:
> > /home/builder/src/yocto/poky/build/tmp/work/i586-poky-
> > linux/libical/2.0.0-r0/temp/log.do_package_qa.4934
> > NOTE: recipe libical-2.0.0-r0: task do_package_qa: Failed
> > ERROR: Task (/home/builder/src/yocto/poky/meta/recipes-
> > support/libical/libical_2.0.0.bb:do_package_qa) failed with exit
> > code '1'
> > 
> > For some reason libical from poky master branch is not affected.
> 
> The reason why master branch is not affected is:
> 
> commit 26cccb93059b8963651b7d17cea2ee95f52633b7
> Author: Juro Bystricky <juro.bystricky@intel.com>
> Date:   Tue Mar 20 15:36:36 2018 -0700
> 
>     libical-dev_2.0: improve reproducibility
>     
>     Remove build host references from distributed files.
>     
>     (From OE-Core rev: 20f2670e755bcbf90b2b6c08192c022fe7e7eaad)
>     
>     Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
>     Signed-off-by: Ross Burton <ross.burton@intel.com>
>     Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org
> >
> 
> diff --git a/meta/recipes-support/libical/libical_2.0.0.bb
> b/meta/recipes-support/libical/libical_2.0.0.
> index dcc21cc..daa47ab 100644
> --- a/meta/recipes-support/libical/libical_2.0.0.bb
> +++ b/meta/recipes-support/libical/libical_2.0.0.bb
> @@ -17,3 +17,10 @@ SRC_URI[sha256sum] =
> "654c11f759c19237be39f6ad401d917e5a05f36f1736385ed958e60cf2
>  UPSTREAM_CHECK_URI = "https://github.com/libical/libical/releases"
>  
>  inherit cmake pkgconfig
> +
> +do_install_append_class-target () {
> +    # Remove build host references
> +    sed -i \
> +       -e 's,${STAGING_LIBDIR},${libdir},g' \
> +       ${D}${libdir}/cmake/LibIcal/LibIcalTargets-noconfig.cmake
> +}
> 
> Now I'm struggling horribly with the same problem in various custom
> CMake modules, so how about doing that same STAGING_LIBDIR to libdir
> switch for all CMake modules automatically in cmake.bbclass?
> 
> I tried to correctly fix the libical CMake files too with help from
> #cmake, but eventually had to give up. libical has an embedded
> FindICU.cmake module but it seems like the upstream FindICU.cmake
> ends
> up producing the same absolute paths while it does fix some other
> bugs, and of course introduces new ones like no longer providing
> ICU_I18N_FOUND variable when i18n ICU module is found.

I suspect we need to talk to cmake upstream about fixing this properly
but adding something in the class may be an option until a better
upstream solution can be found.

I am puzzled by the need to add a .pc file path check since I thought
there was already a test for that in insane.bbclass?

package_qa_check_staged maybe? "Check staged la and pc files for common
problems like references to the work directory."

Cheers,

Richard










^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
  2018-09-25 11:08   ` Richard Purdie
@ 2018-09-25 11:21     ` Mikko.Rapeli
  2018-09-25 15:19       ` richard.purdie
  0 siblings, 1 reply; 8+ messages in thread
From: Mikko.Rapeli @ 2018-09-25 11:21 UTC (permalink / raw)
  To: richard.purdie; +Cc: openembedded-core

On Tue, Sep 25, 2018 at 12:08:24PM +0100, Richard Purdie wrote:
> On Tue, 2018-09-25 at 10:44 +0000, Mikko.Rapeli@bmw.de wrote:
> > On Tue, Sep 25, 2018 at 01:28:13PM +0300, Mikko Rapeli wrote:
> > > And enable them by default as ERROR_QA. Reason is that
> > > absolute build directory paths in CMake .cmake modules
> > > and in pkg-config .pc files cause recipe builds to escape
> > > their recipe specific sysroots and triggers hard to debug
> > > and timing sensitive build failures. It's better to fail
> > > early.
> > > 
> > > A failure from sumo version of libical looks like:
> > > 
> > > ERROR: libical-2.0.0-r0 do_package_qa: QA Issue: CMake module
> > > /work/i586-poky-linux/libical/2.0.0-r0/packages-split/libical-
> > > dev/usr/lib/cmake/LibIcal/LibIcalTargets-noconfig.cmake contains
> > > reference to tmpdir which causes build raceconditions between
> > > recipes [buildpaths_cmake]
> > > ERROR: libical-2.0.0-r0 do_package_qa: QA run found fatal errors.
> > > Please consider fixing them.
> > > ERROR: libical-2.0.0-r0 do_package_qa: Function failed:
> > > do_package_qa
> > > ERROR: Logfile of failure stored in:
> > > /home/builder/src/yocto/poky/build/tmp/work/i586-poky-
> > > linux/libical/2.0.0-r0/temp/log.do_package_qa.4934
> > > NOTE: recipe libical-2.0.0-r0: task do_package_qa: Failed
> > > ERROR: Task (/home/builder/src/yocto/poky/meta/recipes-
> > > support/libical/libical_2.0.0.bb:do_package_qa) failed with exit
> > > code '1'
> > > 
> > > For some reason libical from poky master branch is not affected.
> > 
> > The reason why master branch is not affected is:
> > 
> > commit 26cccb93059b8963651b7d17cea2ee95f52633b7
> > Author: Juro Bystricky <juro.bystricky@intel.com>
> > Date:   Tue Mar 20 15:36:36 2018 -0700
> > 
> >     libical-dev_2.0: improve reproducibility
> >     
> >     Remove build host references from distributed files.
> >     
> >     (From OE-Core rev: 20f2670e755bcbf90b2b6c08192c022fe7e7eaad)
> >     
> >     Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
> >     Signed-off-by: Ross Burton <ross.burton@intel.com>
> >     Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org
> > >
> > 
> > diff --git a/meta/recipes-support/libical/libical_2.0.0.bb
> > b/meta/recipes-support/libical/libical_2.0.0.
> > index dcc21cc..daa47ab 100644
> > --- a/meta/recipes-support/libical/libical_2.0.0.bb
> > +++ b/meta/recipes-support/libical/libical_2.0.0.bb
> > @@ -17,3 +17,10 @@ SRC_URI[sha256sum] =
> > "654c11f759c19237be39f6ad401d917e5a05f36f1736385ed958e60cf2
> >  UPSTREAM_CHECK_URI = "https://github.com/libical/libical/releases"
> >  
> >  inherit cmake pkgconfig
> > +
> > +do_install_append_class-target () {
> > +    # Remove build host references
> > +    sed -i \
> > +       -e 's,${STAGING_LIBDIR},${libdir},g' \
> > +       ${D}${libdir}/cmake/LibIcal/LibIcalTargets-noconfig.cmake
> > +}
> > 
> > Now I'm struggling horribly with the same problem in various custom
> > CMake modules, so how about doing that same STAGING_LIBDIR to libdir
> > switch for all CMake modules automatically in cmake.bbclass?
> > 
> > I tried to correctly fix the libical CMake files too with help from
> > #cmake, but eventually had to give up. libical has an embedded
> > FindICU.cmake module but it seems like the upstream FindICU.cmake
> > ends
> > up producing the same absolute paths while it does fix some other
> > bugs, and of course introduces new ones like no longer providing
> > ICU_I18N_FOUND variable when i18n ICU module is found.
> 
> I suspect we need to talk to cmake upstream about fixing this properly
> but adding something in the class may be an option until a better
> upstream solution can be found.
> 
> I am puzzled by the need to add a .pc file path check since I thought
> there was already a test for that in insane.bbclass?
> 
> package_qa_check_staged maybe? "Check staged la and pc files for common
> problems like references to the work directory."

That check is not enabled by default. At least bash is producing a broken
bash.pc (and several other files like Makefile.in and bashbug) in sumo
with embedded absolute paths to build sysroot.

-Mikko

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
  2018-09-25 11:21     ` Mikko.Rapeli
@ 2018-09-25 15:19       ` richard.purdie
  2018-09-25 17:11         ` Mikko.Rapeli
  0 siblings, 1 reply; 8+ messages in thread
From: richard.purdie @ 2018-09-25 15:19 UTC (permalink / raw)
  To: Mikko.Rapeli; +Cc: openembedded-core

On Tue, 2018-09-25 at 11:21 +0000, Mikko.Rapeli@bmw.de wrote:
> On Tue, Sep 25, 2018 at 12:08:24PM +0100, Richard Purdie wrote:
> > I suspect we need to talk to cmake upstream about fixing this
> > properly
> > but adding something in the class may be an option until a better
> > upstream solution can be found.
> > 
> > I am puzzled by the need to add a .pc file path check since I
> > thought
> > there was already a test for that in insane.bbclass?
> > 
> > package_qa_check_staged maybe? "Check staged la and pc files for
> > common
> > problems like references to the work directory."
> 
> That check is not enabled by default. At least bash is producing a
> broken
> bash.pc (and several other files like Makefile.in and bashbug) in
> sumo
> with embedded absolute paths to build sysroot.

I still felt I was missing something:

ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la 

which sets "pkgconfig" and "la". package_qa_check_staged calls
package_qa_handle_error("la",...) and
package_qa_handle_error("pkgconfig"...).

do_qa_staging calls package_qa_check_staged() and is triggered by:

do_populate_sysroot[postfuncs] += "do_qa_staging "

which is set for everything and should be running on master?

I had a look at bash.pc in a random local build and there is no
hardcoded path in it for master. I then found a sumo build and looked
at bash.pc there and also no hardcoded paths.

The issues would have appeared to have been fixed by:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dabbfe23de0615a958fac31b83684ade024cf17d 
which should be in sumo.

What may be the real issue is that sanity checker is quite specific
about what its looking for and does do:

file_content = file_content.replace(recipesysroot, "")

which may make sense for .la files but perhaps not .pc files, I'd guess
its to stop compiler flags triggering errors. 

So the real fix here may be to remove that line from the .pc check, at
least in the target case (native case it would make sense)?

Cheers,

Richard



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
  2018-09-25 15:19       ` richard.purdie
@ 2018-09-25 17:11         ` Mikko.Rapeli
  2018-09-25 18:29           ` richard.purdie
  0 siblings, 1 reply; 8+ messages in thread
From: Mikko.Rapeli @ 2018-09-25 17:11 UTC (permalink / raw)
  To: richard.purdie; +Cc: openembedded-core

On Tue, Sep 25, 2018 at 04:19:39PM +0100, richard.purdie@linuxfoundation.org wrote:
> On Tue, 2018-09-25 at 11:21 +0000, Mikko.Rapeli@bmw.de wrote:
> > On Tue, Sep 25, 2018 at 12:08:24PM +0100, Richard Purdie wrote:
> > > I suspect we need to talk to cmake upstream about fixing this
> > > properly
> > > but adding something in the class may be an option until a better
> > > upstream solution can be found.
> > > 
> > > I am puzzled by the need to add a .pc file path check since I
> > > thought
> > > there was already a test for that in insane.bbclass?
> > > 
> > > package_qa_check_staged maybe? "Check staged la and pc files for
> > > common
> > > problems like references to the work directory."
> > 
> > That check is not enabled by default. At least bash is producing a
> > broken
> > bash.pc (and several other files like Makefile.in and bashbug) in
> > sumo
> > with embedded absolute paths to build sysroot.
> 
> I still felt I was missing something:
> 
> ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la 
> 
> which sets "pkgconfig" and "la". package_qa_check_staged calls
> package_qa_handle_error("la",...) and
> package_qa_handle_error("pkgconfig"...).
> 
> do_qa_staging calls package_qa_check_staged() and is triggered by:
> 
> do_populate_sysroot[postfuncs] += "do_qa_staging "
> 
> which is set for everything and should be running on master?
> 
> I had a look at bash.pc in a random local build and there is no
> hardcoded path in it for master. I then found a sumo build and looked
> at bash.pc there and also no hardcoded paths.
> 
> The issues would have appeared to have been fixed by:
> 
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dabbfe23de0615a958fac31b83684ade024cf17d 
> which should be in sumo.

Cool, but this ignores nativesdk, which is where I saw the problems with
this test. Somehow missed that nativesdk part in my reply, possibly because
of plain recipe name in the error message.

> What may be the real issue is that sanity checker is quite specific
> about what its looking for and does do:
> 
> file_content = file_content.replace(recipesysroot, "")
> 
> which may make sense for .la files but perhaps not .pc files, I'd guess
> its to stop compiler flags triggering errors. 
> 
> So the real fix here may be to remove that line from the .pc check, at
> least in the target case (native case it would make sense)?

And .cmake files?

I've fixed issues found by this test in my tree by following the pattern from
these "improve reproducibility" fixes. Some recipes were installing generated
.cmake files from build tree (bad, bad) and several recipes were somehow
generating .cmake files with absolute paths for libraries. It's completely
unclear to me when and why CMake decides to fill in absolute paths, like with
libical.

-Mikko

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
  2018-09-25 17:11         ` Mikko.Rapeli
@ 2018-09-25 18:29           ` richard.purdie
  2018-09-25 20:28             ` Mikko.Rapeli
  0 siblings, 1 reply; 8+ messages in thread
From: richard.purdie @ 2018-09-25 18:29 UTC (permalink / raw)
  To: Mikko.Rapeli; +Cc: openembedded-core

On Tue, 2018-09-25 at 17:11 +0000, Mikko.Rapeli@bmw.de wrote:
> On Tue, Sep 25, 2018 at 04:19:39PM +0100, richard.purdie@linuxfoundat
> ion.org wrote:
> > On Tue, 2018-09-25 at 11:21 +0000, Mikko.Rapeli@bmw.de wrote:
> > > On Tue, Sep 25, 2018 at 12:08:24PM +0100, Richard Purdie wrote:
> > > > I suspect we need to talk to cmake upstream about fixing this
> > > > properly
> > > > but adding something in the class may be an option until a
> > > > better
> > > > upstream solution can be found.
> > > > 
> > > > I am puzzled by the need to add a .pc file path check since I
> > > > thought
> > > > there was already a test for that in insane.bbclass?
> > > > 
> > > > package_qa_check_staged maybe? "Check staged la and pc files
> > > > for
> > > > common
> > > > problems like references to the work directory."
> > > 
> > > That check is not enabled by default. At least bash is producing
> > > a
> > > broken
> > > bash.pc (and several other files like Makefile.in and bashbug) in
> > > sumo
> > > with embedded absolute paths to build sysroot.
> > 
> > I still felt I was missing something:
> > 
> > ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig
> > la 
> > 
> > which sets "pkgconfig" and "la". package_qa_check_staged calls
> > package_qa_handle_error("la",...) and
> > package_qa_handle_error("pkgconfig"...).
> > 
> > do_qa_staging calls package_qa_check_staged() and is triggered by:
> > 
> > do_populate_sysroot[postfuncs] += "do_qa_staging "
> > 
> > which is set for everything and should be running on master?
> > 
> > I had a look at bash.pc in a random local build and there is no
> > hardcoded path in it for master. I then found a sumo build and
> > looked
> > at bash.pc there and also no hardcoded paths.
> > 
> > The issues would have appeared to have been fixed by:
> > 
> > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dabbfe23de0615
> > a958fac31b83684ade024cf17d 
> > which should be in sumo.
> 
> Cool, but this ignores nativesdk, which is where I saw the problems
> with this test. Somehow missed that nativesdk part in my reply,
> possibly because of plain recipe name in the error message.

With the addition of nativesdk, this starts to make more sense...

> > What may be the real issue is that sanity checker is quite specific
> > about what its looking for and does do:
> > 
> > file_content = file_content.replace(recipesysroot, "")
> > 
> > which may make sense for .la files but perhaps not .pc files, I'd
> > guess
> > its to stop compiler flags triggering errors. 
> > 
> > So the real fix here may be to remove that line from the .pc check,
> > at
> > least in the target case (native case it would make sense)?
> 
> And .cmake files?

We should add something to cover those. I was just worried about why
tests I thought were there weren't working. I do think we should
fix/improve the existing pkgconfig test rather than add another similar
one.

> I've fixed issues found by this test in my tree by following the
> pattern from these "improve reproducibility" fixes. Some recipes were
> installing generated .cmake files from build tree (bad, bad) and
> several recipes were somehow generating .cmake files with absolute
> paths for libraries. It's completely unclear to me when and why CMake
> decides to fill in absolute paths, like with libical.

I'm afraid you probably know more about cmake than I do at this
point...

Cheers,

Richard


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks
  2018-09-25 18:29           ` richard.purdie
@ 2018-09-25 20:28             ` Mikko.Rapeli
  0 siblings, 0 replies; 8+ messages in thread
From: Mikko.Rapeli @ 2018-09-25 20:28 UTC (permalink / raw)
  To: richard.purdie; +Cc: openembedded-core

On Tue, Sep 25, 2018 at 07:29:22PM +0100, richard.purdie@linuxfoundation.org wrote:
> On Tue, 2018-09-25 at 17:11 +0000, Mikko.Rapeli@bmw.de wrote:
> > On Tue, Sep 25, 2018 at 04:19:39PM +0100, richard.purdie@linuxfoundat
> > ion.org wrote:
> > > On Tue, 2018-09-25 at 11:21 +0000, Mikko.Rapeli@bmw.de wrote:
> > > > On Tue, Sep 25, 2018 at 12:08:24PM +0100, Richard Purdie wrote:
> > > > > I suspect we need to talk to cmake upstream about fixing this
> > > > > properly
> > > > > but adding something in the class may be an option until a
> > > > > better
> > > > > upstream solution can be found.
> > > > > 
> > > > > I am puzzled by the need to add a .pc file path check since I
> > > > > thought
> > > > > there was already a test for that in insane.bbclass?
> > > > > 
> > > > > package_qa_check_staged maybe? "Check staged la and pc files
> > > > > for
> > > > > common
> > > > > problems like references to the work directory."
> > > > 
> > > > That check is not enabled by default. At least bash is producing
> > > > a
> > > > broken
> > > > bash.pc (and several other files like Makefile.in and bashbug) in
> > > > sumo
> > > > with embedded absolute paths to build sysroot.
> > > 
> > > I still felt I was missing something:
> > > 
> > > ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig
> > > la 
> > > 
> > > which sets "pkgconfig" and "la". package_qa_check_staged calls
> > > package_qa_handle_error("la",...) and
> > > package_qa_handle_error("pkgconfig"...).
> > > 
> > > do_qa_staging calls package_qa_check_staged() and is triggered by:
> > > 
> > > do_populate_sysroot[postfuncs] += "do_qa_staging "
> > > 
> > > which is set for everything and should be running on master?
> > > 
> > > I had a look at bash.pc in a random local build and there is no
> > > hardcoded path in it for master. I then found a sumo build and
> > > looked
> > > at bash.pc there and also no hardcoded paths.
> > > 
> > > The issues would have appeared to have been fixed by:
> > > 
> > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=dabbfe23de0615
> > > a958fac31b83684ade024cf17d 
> > > which should be in sumo.
> > 
> > Cool, but this ignores nativesdk, which is where I saw the problems
> > with this test. Somehow missed that nativesdk part in my reply,
> > possibly because of plain recipe name in the error message.
> 
> With the addition of nativesdk, this starts to make more sense...
> 
> > > What may be the real issue is that sanity checker is quite specific
> > > about what its looking for and does do:
> > > 
> > > file_content = file_content.replace(recipesysroot, "")
> > > 
> > > which may make sense for .la files but perhaps not .pc files, I'd
> > > guess
> > > its to stop compiler flags triggering errors. 
> > > 
> > > So the real fix here may be to remove that line from the .pc check,
> > > at
> > > least in the target case (native case it would make sense)?
> > 
> > And .cmake files?
> 
> We should add something to cover those. I was just worried about why
> tests I thought were there weren't working. I do think we should
> fix/improve the existing pkgconfig test rather than add another similar
> one.

Ok, I'll see what I can do about the tests...

> > I've fixed issues found by this test in my tree by following the
> > pattern from these "improve reproducibility" fixes. Some recipes were
> > installing generated .cmake files from build tree (bad, bad) and
> > several recipes were somehow generating .cmake files with absolute
> > paths for libraries. It's completely unclear to me when and why CMake
> > decides to fill in absolute paths, like with libical.
> 
> I'm afraid you probably know more about cmake than I do at this
> point...

I wonder what would be exposed if recipe builds would really be confined
in their sysroots with seccomp or filesystem namespaces..

-Mikko

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2018-09-25 20:28 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-25 10:28 [PATCH RFC] insane.bbclass: add buildpaths_cmake and buildpaths_pkgconfig checks Mikko Rapeli
2018-09-25 10:44 ` Mikko.Rapeli
2018-09-25 11:08   ` Richard Purdie
2018-09-25 11:21     ` Mikko.Rapeli
2018-09-25 15:19       ` richard.purdie
2018-09-25 17:11         ` Mikko.Rapeli
2018-09-25 18:29           ` richard.purdie
2018-09-25 20:28             ` Mikko.Rapeli

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.