ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Robert P. J. Day]

  (warning: i've been away from YP for well over a year and am now
studiously trying to catch up, so i have some work to do.)

  currently trying to build a "core-image-minimal" for a zedboard on
my wholly unsupported fedora 30 (branched) system, so i'm well aware
there will almost certainly be breakage as i try to resolve version
numbers and so on, but working off the "thud" branch, the first issue
i had was trying to configure cmake-native -- the error message is
exactly what you can read someone asking about here:

https://stackoverflow.com/questions/52663287/glibcxx-3-4-26-not-found

/home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-native/3.12.2-r0/build/Bootstrap.cmk/cmake:
/home/rpjday/oe/builds/zedboard/tmp/sysroots-uninative/x86_64-linux/usr/lib/libstdc++.so.6:
version `GLIBCXX_3.4.26' not found (required by
/home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-native/3.12.2-r0/build/Bootstrap.cmk/cmake)
| ---------------------------------------------
| Error when bootstrapping CMake:
| Problem while running initial CMake
| ---------------------------------------------

  i'm unsure how to resolve that easily, but my first reaction was,
"if i already have cmake installed on the host, why not just take
advantage of ASSUME_PROVIDED"? i recall from some time back asking why
more host utils were not, by default, included in ASSUME_PROVIDED, and
the answer (quite reasonably) was that one wants to have as few
variables as possible for reliably replicated builds.

  fair enough, but in my case, until i figure out how to fix that, i
thought, why not just:

  ASSUME_PROVIDED += "cmake-native"

so i did, and that got me past the cmake-native build issue, until i
hit this trying to build libsolv-native:

DEBUG: Executing shell function do_configure
/home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/libsolv-native/0.6.35-r0/temp/run.do_configure.16705:
line 130: cmake: command not found

  hang on ... why would a subsequent recipe not be able to locate
/usr/bin/cmake on my host after i explicitly identified cmake-native
as assumed to be provided? is there something about the libsolv-native
recipe that does not take that possibility into account? i'm just
about to check, but if someone has an explanation for that, i'm
all ears.

  finally, regarding ASSUME_PROVIDED, given that i'm already living
life dangerously with an unsupported distro, is there any harm in just
loading up on ASSUME_PROVIDED packages, as long as they work? in many
cases, i can see that the version that would be built by YP is close
to, if not identical to, my host version. so other than risking
breakage down the line as versions change, as long as the host
packages work, is there any harm in just taking advantage of them?

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                         http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Bach, Pascal]

>   currently trying to build a "core-image-minimal" for a zedboard on my
> wholly unsupported fedora 30 (branched) system, so i'm well aware there
> will almost certainly be breakage as i try to resolve version numbers and so
> on, but working off the "thud" branch, the first issue i had was trying to
> configure cmake-native -- the error message is exactly what you can read
> someone asking about here:
> 
> https://stackoverflow.com/questions/52663287/glibcxx-3-4-26-not-found
> 
> /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> native/3.12.2-r0/build/Bootstrap.cmk/cmake:
> /home/rpjday/oe/builds/zedboard/tmp/sysroots-uninative/x86_64-
> linux/usr/lib/libstdc++.so.6:
> version `GLIBCXX_3.4.26' not found (required by
> /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> native/3.12.2-r0/build/Bootstrap.cmk/cmake)
> | ---------------------------------------------
> | Error when bootstrapping CMake:
> | Problem while running initial CMake
> | ---------------------------------------------
> 
>   i'm unsure how to resolve that easily, but my first reaction was, "if i already
> have cmake installed on the host, why not just take advantage of
> ASSUME_PROVIDED"? i recall from some time back asking why more host
> utils were not, by default, included in ASSUME_PROVIDED, and the answer
> (quite reasonably) was that one wants to have as few variables as possible
> for reliably replicated builds.
> 
>   fair enough, but in my case, until i figure out how to fix that, i thought, why
> not just:
> 
>   ASSUME_PROVIDED += "cmake-native"
> 
> so i did, and that got me past the cmake-native build issue, until i hit this
> trying to build libsolv-native:
> 
> DEBUG: Executing shell function do_configure
> /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/libsolv-
> native/0.6.35-r0/temp/run.do_configure.16705:
> line 130: cmake: command not found
> 
>   hang on ... why would a subsequent recipe not be able to locate
> /usr/bin/cmake on my host after i explicitly identified cmake-native as
> assumed to be provided? is there something about the libsolv-native recipe
> that does not take that possibility into account? i'm just about to check, but if
> someone has an explanation for that, i'm all ears.

There is an issue that CMake is not able to find binaries in hosttools. It might be the case that this is the issue you are seeing.

I submitted a patch for that but I'm not sure it
Ever made it into master or a release. I need to follow up on this, the patch is here: https://patchwork.openembedded.org/series/14429/#

>   finally, regarding ASSUME_PROVIDED, given that i'm already living life
> dangerously with an unsupported distro, is there any harm in just loading up
> on ASSUME_PROVIDED packages, as long as they work? in many cases, i can
> see that the version that would be built by YP is close to, if not identical to,
> my host version. so other than risking breakage down the line as versions
> change, as long as the host packages work, is there any harm in just taking
> advantage of them?
> 

Generally it's find to use ASSUME_PROVIDED if you don't mind the dependency between the build host and Yocto. 

Pascal

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Robert P. J. Day]

On Thu, 25 Apr 2019, Bach, Pascal wrote:

> >   currently trying to build a "core-image-minimal" for a zedboard on my
> > wholly unsupported fedora 30 (branched) system, so i'm well aware there
> > will almost certainly be breakage as i try to resolve version numbers and so
> > on, but working off the "thud" branch, the first issue i had was trying to
> > configure cmake-native -- the error message is exactly what you can read
> > someone asking about here:
> >
> > https://stackoverflow.com/questions/52663287/glibcxx-3-4-26-not-found
> >
> > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> > native/3.12.2-r0/build/Bootstrap.cmk/cmake:
> > /home/rpjday/oe/builds/zedboard/tmp/sysroots-uninative/x86_64-
> > linux/usr/lib/libstdc++.so.6:
> > version `GLIBCXX_3.4.26' not found (required by
> > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> > native/3.12.2-r0/build/Bootstrap.cmk/cmake)
> > | ---------------------------------------------
> > | Error when bootstrapping CMake:
> > | Problem while running initial CMake
> > | ---------------------------------------------
> >
> >   i'm unsure how to resolve that easily, but my first reaction
> > was, "if i already have cmake installed on the host, why not just
> > take advantage of ASSUME_PROVIDED"? i recall from some time back
> > asking why more host utils were not, by default, included in
> > ASSUME_PROVIDED, and the answer (quite reasonably) was that one
> > wants to have as few variables as possible for reliably replicated
> > builds.
> >
> >   fair enough, but in my case, until i figure out how to fix that,
> > i thought, why not just:
> >
> >   ASSUME_PROVIDED += "cmake-native"
> >
> > so i did, and that got me past the cmake-native build issue, until
> > i hit this trying to build libsolv-native:
> >
> > DEBUG: Executing shell function do_configure
> > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/libsolv-
> > native/0.6.35-r0/temp/run.do_configure.16705:
> > line 130: cmake: command not found
> >
> >   hang on ... why would a subsequent recipe not be able to locate
> > /usr/bin/cmake on my host after i explicitly identified
> > cmake-native as assumed to be provided? is there something about
> > the libsolv-native recipe that does not take that possibility into
> > account? i'm just about to check, but if someone has an
> > explanation for that, i'm all ears.
>
> There is an issue that CMake is not able to find binaries in
> hosttools. It might be the case that this is the issue you are
> seeing.
>
> I submitted a patch for that but I'm not sure it Ever made it into
> master or a release. I need to follow up on this, the patch is here:
> https://patchwork.openembedded.org/series/14429/#

  that *sort of* sounds like what is happening, but to be pedantic,
it's not that cmake can't find binaries in hosttools, it's that
*other* packages can't locate cmake on the host even after setting

  ASSUME_PROVIDED += "cmake-native"

i am assuming that whatever search path is used for "native" tools is
extended by the use of ASSUME_PROVIDED -- is that what your patch is
addressing? in any event, i've now run into a couple other packages
with the same issue -- "cmake: command not found" -- libcomps-native
and librepo-native, so i'll just ASSUME_PROVIDED them too for the
moment, as they're both installed on my host.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                         http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Richard Purdie]

On Thu, 2019-04-25 at 08:05 -0400, Robert P. J. Day wrote:
>   (warning: i've been away from YP for well over a year and am now
> studiously trying to catch up, so i have some work to do.)
> 
>   currently trying to build a "core-image-minimal" for a zedboard on
> my wholly unsupported fedora 30 (branched) system, so i'm well aware
> there will almost certainly be breakage as i try to resolve version
> numbers and so on, but working off the "thud" branch, the first issue
> i had was trying to configure cmake-native -- the error message is
> exactly what you can read someone asking about here:
> 
> https://stackoverflow.com/questions/52663287/glibcxx-3-4-26-not-found
> 
> /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> native/3.12.2-r0/build/Bootstrap.cmk/cmake:
> /home/rpjday/oe/builds/zedboard/tmp/sysroots-uninative/x86_64-
> linux/usr/lib/libstdc++.so.6:
> version `GLIBCXX_3.4.26' not found (required by
> /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> native/3.12.2-r0/build/Bootstrap.cmk/cmake)
> > ---------------------------------------------
> > Error when bootstrapping CMake:
> > Problem while running initial CMake
> > ---------------------------------------------

This probably means that your system c library is newer than the one in
uninative and things aren't working because of that. Either try:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5f6156b32c9d17bdb06d67199373433b0e470cc7

or turning off uninative.

>   i'm unsure how to resolve that easily, but my first reaction was,
> "if i already have cmake installed on the host, why not just take
> advantage of ASSUME_PROVIDED"? i recall from some time back asking
> why
> more host utils were not, by default, included in ASSUME_PROVIDED,
> and
> the answer (quite reasonably) was that one wants to have as few
> variables as possible for reliably replicated builds.

That is one reason, there is a second. We patch some of the utilities
to do what we need. We rely on our patches being present.

>   fair enough, but in my case, until i figure out how to fix that, i
> thought, why not just:
> 
>   ASSUME_PROVIDED += "cmake-native"

This is a bad idea as we patch cmake iirc.

> 
> so i did, and that got me past the cmake-native build issue, until i
> hit this trying to build libsolv-native:
> 
> DEBUG: Executing shell function do_configure
> /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/libsolv-
> native/0.6.35-r0/temp/run.do_configure.16705:
> line 130: cmake: command not found
> 
>   hang on ... why would a subsequent recipe not be able to locate
> /usr/bin/cmake on my host after i explicitly identified cmake-native
> as assumed to be provided? is there something about the libsolv-
> native
> recipe that does not take that possibility into account? i'm just
> about to check, but if someone has an explanation for that, i'm
> all ears.

You would also have to do:

HOSTTOOLS += "cmake"

to allow cmake to be visible from the host system. Its a host
contamination protection mechanisn that has been there since pyro.

>   finally, regarding ASSUME_PROVIDED, given that i'm already living
> life dangerously with an unsupported distro, is there any harm in
> just loading up on ASSUME_PROVIDED packages, as long as they work? in
> many cases, i can see that the version that would be built by YP is
> close to, if not identical to, my host version. so other than risking
> breakage down the line as versions change, as long as the host
> packages work, is there any harm in just taking advantage of them?

I think it will be a world of pain ;-)

Cheers,

Richard

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Richard Purdie]

On Thu, 2019-04-25 at 08:32 -0400, Robert P. J. Day wrote:
> On Thu, 25 Apr 2019, Bach, Pascal wrote:
> 
> > >   currently trying to build a "core-image-minimal" for a zedboard
> > > on my
> > > wholly unsupported fedora 30 (branched) system, so i'm well aware
> > > there
> > > will almost certainly be breakage as i try to resolve version
> > > numbers and so
> > > on, but working off the "thud" branch, the first issue i had was
> > > trying to
> > > configure cmake-native -- the error message is exactly what you
> > > can read
> > > someone asking about here:
> > > 
> > > https://stackoverflow.com/questions/52663287/glibcxx-3-4-26-not-found
> > > 
> > > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> > > native/3.12.2-r0/build/Bootstrap.cmk/cmake:
> > > /home/rpjday/oe/builds/zedboard/tmp/sysroots-uninative/x86_64-
> > > linux/usr/lib/libstdc++.so.6:
> > > version `GLIBCXX_3.4.26' not found (required by
> > > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> > > native/3.12.2-r0/build/Bootstrap.cmk/cmake)
> > > > ---------------------------------------------
> > > > Error when bootstrapping CMake:
> > > > Problem while running initial CMake
> > > > ---------------------------------------------
> > > 
> > >   i'm unsure how to resolve that easily, but my first reaction
> > > was, "if i already have cmake installed on the host, why not just
> > > take advantage of ASSUME_PROVIDED"? i recall from some time back
> > > asking why more host utils were not, by default, included in
> > > ASSUME_PROVIDED, and the answer (quite reasonably) was that one
> > > wants to have as few variables as possible for reliably
> > > replicated
> > > builds.
> > > 
> > >   fair enough, but in my case, until i figure out how to fix
> > > that,
> > > i thought, why not just:
> > > 
> > >   ASSUME_PROVIDED += "cmake-native"
> > > 
> > > so i did, and that got me past the cmake-native build issue,
> > > until
> > > i hit this trying to build libsolv-native:
> > > 
> > > DEBUG: Executing shell function do_configure
> > > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/libsolv-
> > > native/0.6.35-r0/temp/run.do_configure.16705:
> > > line 130: cmake: command not found
> > > 
> > >   hang on ... why would a subsequent recipe not be able to locate
> > > /usr/bin/cmake on my host after i explicitly identified
> > > cmake-native as assumed to be provided? is there something about
> > > the libsolv-native recipe that does not take that possibility
> > > into
> > > account? i'm just about to check, but if someone has an
> > > explanation for that, i'm all ears.
> > 
> > There is an issue that CMake is not able to find binaries in
> > hosttools. It might be the case that this is the issue you are
> > seeing.
> > 
> > I submitted a patch for that but I'm not sure it Ever made it into
> > master or a release. I need to follow up on this, the patch is
> > here:
> > https://patchwork.openembedded.org/series/14429/#
> 
>   that *sort of* sounds like what is happening, but to be pedantic,
> it's not that cmake can't find binaries in hosttools, it's that
> *other* packages can't locate cmake on the host even after setting
> 
>   ASSUME_PROVIDED += "cmake-native"

Setting that "fixes" dependencies but it doesn't make cmake visible in
our HOSTTOOLS.

> i am assuming that whatever search path is used for "native" tools is
> extended by the use of ASSUME_PROVIDED -- is that what your patch is
> addressing?

How would it know in general terms which binaries a given X would
provide?

>  in any event, i've now run into a couple other packages
> with the same issue -- "cmake: command not found" -- libcomps-native
> and librepo-native, so i'll just ASSUME_PROVIDED them too for the
> moment, as they're both installed on my host.

This is a bad idea and you're on a path to madness ;-)

Cheers,

Richard

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Robert P. J. Day]

On Thu, 25 Apr 2019, Bach, Pascal wrote:

> rday wrote:

> >   finally, regarding ASSUME_PROVIDED, given that i'm already
> > living life dangerously with an unsupported distro, is there any
> > harm in just loading up on ASSUME_PROVIDED packages, as long as
> > they work? in many cases, i can see that the version that would be
> > built by YP is close to, if not identical to, my host version. so
> > other than risking breakage down the line as versions change, as
> > long as the host packages work, is there any harm in just taking
> > advantage of them?
>
> Generally it's fine to use ASSUME_PROVIDED if you don't mind the
> dependency between the build host and Yocto.

  as i mentioned, i'm already living life on the edge doing all this
on fedora 30 branched (the unofficial next release), so i'm quite
prepared to trip over versioning issues, and testing all this against
F30 at least warns me about upcoming issues that i might be able to do
bug reports or patches against ahead of time.

  and it's easy to see there's a *ton* of packages i can assume
natively that should be fairly safe; in many cases, the version that
would be built by YP is exactly the same version i already have
natively. in other cases, the difference might be a micro-version off.

  anyway, i probably wouldn't have bothered with this if it weren't
for that cmake issue. but let's see where this goes.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                         http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Robert P. J. Day]

On Thu, 25 Apr 2019, Richard Purdie wrote:

   ... big snip of warning me about what i'm trying to do ...

> This is a bad idea and you're on a path to madness ;-)

  i already knew that ... i did admit i've been out of YP for a while
and have a lot of catching up to do. i suspect my next few days will
be full of reading.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                         http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Robert P. J. Day]

On Thu, 25 Apr 2019, Richard Purdie wrote:

> On Thu, 2019-04-25 at 08:05 -0400, Robert P. J. Day wrote:

... snip ...

> >   ASSUME_PROVIDED += "cmake-native"
>
> This is a bad idea as we patch cmake iirc.

... snip ...
>
> You would also have to do:
>
> HOSTTOOLS += "cmake"
>
> to allow cmake to be visible from the host system. Its a host
> contamination protection mechanisn that has been there since pyro.

  just to summarize, if one wanted to (perhaps injudiciously) take
advantage of host tools, that requires:

1) ASSUME_PROVIDED to specify the native package that no longer needs
   to be built, and

2) HOSTTOOLS to identify the binary (or binaries) that can be picked
   up from the host

  i can see an obvious example here:

https://patchwork.openembedded.org/patch/140375/

but if this is the case, then it seems that the YP reference manual
should be clarified to explain the association between these
variables. for example, the current ref manual reads:

  ASSUME_PROVIDED

  Lists recipe names (PN values) BitBake does not attempt to build.
  Instead, BitBake assumes these recipes have already been built.

  In OpenEmbedded-Core, ASSUME_PROVIDED mostly specifies native
  tools that should not be built. An example is git-native, which when
  specified, allows for the Git binary from the host to be used rather
  than building git-native.

reading that, a reader would not realize the necessity(?) of setting
HOSTTOOLS, yes?

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                         http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: ASSUME_PROVIDED, cmake-native and why cmake is subsequently not found?

[Robert P. J. Day]

On Thu, 25 Apr 2019, Richard Purdie wrote:

> On Thu, 2019-04-25 at 08:05 -0400, Robert P. J. Day wrote:

> > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> > native/3.12.2-r0/build/Bootstrap.cmk/cmake:
> > /home/rpjday/oe/builds/zedboard/tmp/sysroots-uninative/x86_64-
> > linux/usr/lib/libstdc++.so.6:
> > version `GLIBCXX_3.4.26' not found (required by
> > /home/rpjday/oe/builds/zedboard/tmp/work/x86_64-linux/cmake-
> > native/3.12.2-r0/build/Bootstrap.cmk/cmake)
> > > ---------------------------------------------
> > > Error when bootstrapping CMake:
> > > Problem while running initial CMake
> > > ---------------------------------------------
>
> This probably means that your system c library is newer than the one
> in uninative and things aren't working because of that. Either try:
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5f6156b32c9d17bdb06d67199373433b0e470cc7

  that patch is already in thud branch of poky i'm using ...

> or turning off uninative.

  yes, that seems to have solved things for now:

    INHERIT_remove = "uninative"

moving on to the next head-scratcher ...

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                         http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================