RDEPENDS_${PN} and virtclass-native

RDEPENDS_${PN} and virtclass-native

[Phil Blundell]

By way of displacement activity to avoid actually fixing my perl
compilation problem, it occurred to me to investigate why perl was
getting dragged into a micro-base-image build in the first place.  The
culprit turns out to be imake, which does:

RDEPENDS_${PN} = "perl xproto"

and is then BBCLASSEXTENDed to imake-native (which in turn is pulled in
by way of prelink-native and transfig-native).

Now, leaving aside the question of whether it is reasonable for prelink
to be depending on transfig, it is clearly wrong for the -native version
of imake to be depending on perl.  It seems that native.bbclass makes
some effort to rewrite plain RDEPENDS to the -native version, but it
doesn't apply the same tactics to RDEPENDS_${PN} or any such.  (And, in
fact, rewriting plain RDEPENDS is probably futile since few if any
recipes are going to be setting it.)

Obviously I can fix this by just setting RDEPENDS_virtclass-native in
the recipe, and that's what I've done in my local tree.  But I wonder if
a better solution would be for native.bbclass to be slightly more
adventurous about rewriting these things for itself.

p.

Re: RDEPENDS_${PN} and virtclass-native

[Chris Larson]

On Wed, May 25, 2011 at 10:01 AM, Phil Blundell <pb@pbcl.net> wrote:
> By way of displacement activity to avoid actually fixing my perl
> compilation problem, it occurred to me to investigate why perl was
> getting dragged into a micro-base-image build in the first place.  The
> culprit turns out to be imake, which does:
>
> RDEPENDS_${PN} = "perl xproto"
>
> and is then BBCLASSEXTENDed to imake-native (which in turn is pulled in
> by way of prelink-native and transfig-native).
>
> Now, leaving aside the question of whether it is reasonable for prelink
> to be depending on transfig, it is clearly wrong for the -native version
> of imake to be depending on perl.  It seems that native.bbclass makes
> some effort to rewrite plain RDEPENDS to the -native version, but it
> doesn't apply the same tactics to RDEPENDS_${PN} or any such.  (And, in
> fact, rewriting plain RDEPENDS is probably futile since few if any
> recipes are going to be setting it.)
>
> Obviously I can fix this by just setting RDEPENDS_virtclass-native in
> the recipe, and that's what I've done in my local tree.  But I wonder if
> a better solution would be for native.bbclass to be slightly more
> adventurous about rewriting these things for itself.

Huh, I knew it messed with rdepends, but I would have assumed that it
would alter the package specific vars for all packages in PACKAGES, at
the least..
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: RDEPENDS_${PN} and virtclass-native

[Richard Purdie]

On Wed, 2011-05-25 at 18:01 +0100, Phil Blundell wrote:
> By way of displacement activity to avoid actually fixing my perl
> compilation problem, it occurred to me to investigate why perl was
> getting dragged into a micro-base-image build in the first place.  The
> culprit turns out to be imake, which does:
> 
> RDEPENDS_${PN} = "perl xproto"
> 
> and is then BBCLASSEXTENDed to imake-native (which in turn is pulled in
> by way of prelink-native and transfig-native).
> 
> Now, leaving aside the question of whether it is reasonable for prelink
> to be depending on transfig, it is clearly wrong for the -native version
> of imake to be depending on perl.  It seems that native.bbclass makes
> some effort to rewrite plain RDEPENDS to the -native version, but it
> doesn't apply the same tactics to RDEPENDS_${PN} or any such.  (And, in
> fact, rewriting plain RDEPENDS is probably futile since few if any
> recipes are going to be setting it.)
> 
> Obviously I can fix this by just setting RDEPENDS_virtclass-native in
> the recipe, and that's what I've done in my local tree.  But I wonder if
> a better solution would be for native.bbclass to be slightly more
> adventurous about rewriting these things for itself.

I think we need to fix native.bbclass. I dread to think what this is
doing to the dependency tree at present. We did remove most references
to RDEPENDS directly so I think that makes this a high priority to fix.

I'd note that the rewriting code is fraught with ordering/expansion
issues which is why that code plays games with the variables like it
does.

DEPENDS is horrible and it looks like we needed to do something similar
for RDEPENDS. Better solutions to the ordering problems would be very
welcome. It problem was the use of DEPENDS_prepend in places iirc.

Cheers,

Richard

Re: RDEPENDS_${PN} and virtclass-native

[Khem Raj]

On Wed, May 25, 2011 at 4:02 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2011-05-25 at 18:01 +0100, Phil Blundell wrote:
>> By way of displacement activity to avoid actually fixing my perl
>> compilation problem, it occurred to me to investigate why perl was
>> getting dragged into a micro-base-image build in the first place.  The
>> culprit turns out to be imake, which does:
>>
>> RDEPENDS_${PN} = "perl xproto"
>>
>> and is then BBCLASSEXTENDed to imake-native (which in turn is pulled in
>> by way of prelink-native and transfig-native).
>>
>> Now, leaving aside the question of whether it is reasonable for prelink
>> to be depending on transfig, it is clearly wrong for the -native version
>> of imake to be depending on perl.  It seems that native.bbclass makes
>> some effort to rewrite plain RDEPENDS to the -native version, but it
>> doesn't apply the same tactics to RDEPENDS_${PN} or any such.  (And, in
>> fact, rewriting plain RDEPENDS is probably futile since few if any
>> recipes are going to be setting it.)
>>
>> Obviously I can fix this by just setting RDEPENDS_virtclass-native in
>> the recipe, and that's what I've done in my local tree.  But I wonder if
>> a better solution would be for native.bbclass to be slightly more
>> adventurous about rewriting these things for itself.
>
> I think we need to fix native.bbclass. I dread to think what this is
> doing to the dependency tree at present. We did remove most references
> to RDEPENDS directly so I think that makes this a high priority to fix.
>
> I'd note that the rewriting code is fraught with ordering/expansion
> issues which is why that code plays games with the variables like it
> does.
>
> DEPENDS is horrible and it looks like we needed to do something similar
> for RDEPENDS. Better solutions to the ordering problems would be very
> welcome. It problem was the use of DEPENDS_prepend in places iirc.
>


while we are talking about fixing native.bbclass on a different
tangent. I think overrides is another thing we need to fix
specially libc overrides. When a recipe inherits
native,cross, or nativesdk then we have to change
the libc override to be always libc-glibc (unless one
is building on a box which is running uclibc)

otherwise we end up with slightly different native
packages and sstate can not use them effectively
it keeps rebuilding and refreshing everytime.

> Cheers,
>
> Richard
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

Re: RDEPENDS_${PN} and virtclass-native

[Richard Purdie]

On Thu, 2011-05-26 at 00:02 +0100, Richard Purdie wrote:
> I think we need to fix native.bbclass. I dread to think what this is
> doing to the dependency tree at present. We did remove most references
> to RDEPENDS directly so I think that makes this a high priority to fix.
> 
> I'd note that the rewriting code is fraught with ordering/expansion
> issues which is why that code plays games with the variables like it
> does.
> 
> DEPENDS is horrible and it looks like we needed to do something similar
> for RDEPENDS. Better solutions to the ordering problems would be very
> welcome. It problem was the use of DEPENDS_prepend in places iirc.

I just reminded myself of the problem with DEPENDS.

Imagine the recipe says:

DEPENDS = "foo"
inherit autotools

Behind the scenes, autotools (amongst other thinsg) does a:

DEPENDS_prepend = "autoconf-native"

The trouble comes when you try:

DEPENDS_virtclass-native = ""

since you want to drop the foo dependency. The dependencies being added
by autotools.bbclass are still wanted but they get overwritten. At this
point I had to remember bitbake's finalize ordering. It does:

finalize()
<call anonymous python>
finalize()

The first call to finalize() expands the _prepend/_append before we see
them. We then lose the ability to see what was added though the classes
vs. what was in the recipe. The second finalize can then only clobber
the data.

If we could get our function called before the first finalize(), we
would remove the need to play the games we do which is looking very very
attractive.

I suspect its time to extend the anonymous python to indicate more
specifically when it would like to be run...

Cheers,

Richard

Re: RDEPENDS_${PN} and virtclass-native

[Chris Larson]

On Wed, May 25, 2011 at 5:00 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2011-05-26 at 00:02 +0100, Richard Purdie wrote:
>> I think we need to fix native.bbclass. I dread to think what this is
>> doing to the dependency tree at present. We did remove most references
>> to RDEPENDS directly so I think that makes this a high priority to fix.
>>
>> I'd note that the rewriting code is fraught with ordering/expansion
>> issues which is why that code plays games with the variables like it
>> does.
>>
>> DEPENDS is horrible and it looks like we needed to do something similar
>> for RDEPENDS. Better solutions to the ordering problems would be very
>> welcome. It problem was the use of DEPENDS_prepend in places iirc.
>
> I just reminded myself of the problem with DEPENDS.
>
> Imagine the recipe says:
>
> DEPENDS = "foo"
> inherit autotools
>
> Behind the scenes, autotools (amongst other thinsg) does a:
>
> DEPENDS_prepend = "autoconf-native"
>
> The trouble comes when you try:
>
> DEPENDS_virtclass-native = ""
>
> since you want to drop the foo dependency. The dependencies being added
> by autotools.bbclass are still wanted but they get overwritten. At this
> point I had to remember bitbake's finalize ordering. It does:
>
> finalize()
> <call anonymous python>
> finalize()
>
> The first call to finalize() expands the _prepend/_append before we see
> them. We then lose the ability to see what was added though the classes
> vs. what was in the recipe. The second finalize can then only clobber
> the data.
>
> If we could get our function called before the first finalize(), we
> would remove the need to play the games we do which is looking very very
> attractive.
>
> I suspect its time to extend the anonymous python to indicate more
> specifically when it would like to be run...

Just need to add another event at the appropriate place. I wonder
where RecipeParsed fits into that ordering.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: RDEPENDS_${PN} and virtclass-native

[Richard Purdie]

On Wed, 2011-05-25 at 17:11 -0700, Chris Larson wrote:
> On Wed, May 25, 2011 at 5:00 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Thu, 2011-05-26 at 00:02 +0100, Richard Purdie wrote:
> >> I think we need to fix native.bbclass. I dread to think what this is
> >> doing to the dependency tree at present. We did remove most references
> >> to RDEPENDS directly so I think that makes this a high priority to fix.
> >>
> >> I'd note that the rewriting code is fraught with ordering/expansion
> >> issues which is why that code plays games with the variables like it
> >> does.
> >>
> >> DEPENDS is horrible and it looks like we needed to do something similar
> >> for RDEPENDS. Better solutions to the ordering problems would be very
> >> welcome. It problem was the use of DEPENDS_prepend in places iirc.
> >
> > I just reminded myself of the problem with DEPENDS.
> >
> > Imagine the recipe says:
> >
> > DEPENDS = "foo"
> > inherit autotools
> >
> > Behind the scenes, autotools (amongst other thinsg) does a:
> >
> > DEPENDS_prepend = "autoconf-native"
> >
> > The trouble comes when you try:
> >
> > DEPENDS_virtclass-native = ""
> >
> > since you want to drop the foo dependency. The dependencies being added
> > by autotools.bbclass are still wanted but they get overwritten. At this
> > point I had to remember bitbake's finalize ordering. It does:
> >
> > finalize()
> > <call anonymous python>
> > finalize()
> >
> > The first call to finalize() expands the _prepend/_append before we see
> > them. We then lose the ability to see what was added though the classes
> > vs. what was in the recipe. The second finalize can then only clobber
> > the data.
> >
> > If we could get our function called before the first finalize(), we
> > would remove the need to play the games we do which is looking very very
> > attractive.
> >
> > I suspect its time to extend the anonymous python to indicate more
> > specifically when it would like to be run...
> 
> Just need to add another event at the appropriate place. I wonder
> where RecipeParsed fits into that ordering.

Of course its not that simple. I just discovered:

OVERRIDES = "z"
DEPENDS_prepend = "a "
DEPENDS = "b"
DEPENDS_z = "c"

d.update_data()
d.getVar("DEPENDS") gives "a c"
d.update_data()
d.getVar("DEPENDS") gives "c"

since the _prepend is lost after the update_data(), the _z override is
not and so the second call overwrites all of DEPENDS.

We've talked before how overrides should really renameVar, not just copy
contents. Obviously there are a lot of subtleties to this though and we
can't just switch due to backwards compatibility :/

Depending on the number of update_data() calls is truly horrible
behaviour though :(

Cheers,

Richard

Re: RDEPENDS_${PN} and virtclass-native

[Phil Blundell]

On Wed, 2011-05-25 at 16:08 -0700, Khem Raj wrote:
> while we are talking about fixing native.bbclass on a different
> tangent. I think overrides is another thing we need to fix
> specially libc overrides. When a recipe inherits
> native,cross, or nativesdk then we have to change
> the libc override to be always libc-glibc (unless one
> is building on a box which is running uclibc)

Just forcing "libc-glibc" on for native packages doesn't seem like the
right answer.  As well as the (perhaps unlikely) situation where the
build host is running uclibc, it might be a Darwin or Cygwin host which
doesn't use glibc at all.

I guess this is another reason why libc-* overrides should be used
sparingly :-}

p.

Re: RDEPENDS_${PN} and virtclass-native

[Phil Blundell]

On Wed, 2011-05-25 at 18:01 +0100, Phil Blundell wrote:
> Now, leaving aside the question of whether it is reasonable for prelink
> to be depending on transfig,

This also seemed worthy of a bit of further investigation.  Here's what
I found out:

The dependency on transfig-native was added by commit
2fbcf1419909cf2840d56bf461ab04f8298f14c6, on the grounds that fig2dev is
needed to build the prelink documentation.  Which seems reasonable
enough on the face of it, except that (a) we probably don't want to
build documentation (or depend on transfig-native) for the -native
package; and (b) the documentation build had already been disabled, for
both native and non-native prelink, three days previously by commit
66e6a9798e5462a6cd8dce22a400b81f1dcae88b.

So, right now, it seems that the dependency on transfig-native is
entirely redundant and I think commit
66e6a9798e5462a6cd8dce22a400b81f1dcae88b can/should be reverted.  In the
longer term it seems like switching the documentation back on for the
non-native prelink would be a good thing to do.

p.

[PATCH] prelink: remove dependency on transfig-native

[Phil Blundell]

Per http://lists.linuxtogo.org/pipermail/openembedded-core/2011-May/003179.html

Signed-off-by: Phil Blundell <philb@gnu.org>
---
 meta/recipes-devtools/prelink/prelink_git.bb |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/meta/recipes-devtools/prelink/prelink_git.bb
b/meta/recipes-devtools/prelink/prelink_git.bb
index 44aac2d..1eea30a 100644
--- a/meta/recipes-devtools/prelink/prelink_git.bb
+++ b/meta/recipes-devtools/prelink/prelink_git.bb
@@ -1,6 +1,7 @@
 SECTION = "devel"
 # Need binutils for libiberty.a
-DEPENDS = "elfutils binutils transfig-native"
+# Would need transfig-native for documentation if it wasn't disabled
+DEPENDS = "elfutils binutils"
 SUMMARY = "An ELF prelinking utility"
 DESCRIPTION = "The prelink package contains a utility which modifies
ELF shared libraries \
 and executables, so that far fewer relocations need to be resolved at \
-- 
1.7.1

Re: [PATCH] prelink: remove dependency on transfig-native

[Richard Purdie]

On Tue, 2011-05-31 at 12:29 +0100, Phil Blundell wrote:
> Per http://lists.linuxtogo.org/pipermail/openembedded-core/2011-May/003179.html
> 
> Signed-off-by: Phil Blundell <philb@gnu.org>
> ---
>  meta/recipes-devtools/prelink/prelink_git.bb |    3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/meta/recipes-devtools/prelink/prelink_git.bb
> b/meta/recipes-devtools/prelink/prelink_git.bb
> index 44aac2d..1eea30a 100644
> --- a/meta/recipes-devtools/prelink/prelink_git.bb
> +++ b/meta/recipes-devtools/prelink/prelink_git.bb
> @@ -1,6 +1,7 @@
>  SECTION = "devel"
>  # Need binutils for libiberty.a
> -DEPENDS = "elfutils binutils transfig-native"
> +# Would need transfig-native for documentation if it wasn't disabled
> +DEPENDS = "elfutils binutils"
>  SUMMARY = "An ELF prelinking utility"
>  DESCRIPTION = "The prelink package contains a utility which modifies
> ELF shared libraries \
>  and executables, so that far fewer relocations need to be resolved at \

Applied to master, thanks.

Please don't send line wrapped patches though ;-)

Cheers,

Richard

Re: RDEPENDS_${PN} and virtclass-native

[Richard Purdie]

On Thu, 2011-05-26 at 01:59 +0100, Richard Purdie wrote:
> Of course its not that simple. I just discovered:
> 
> OVERRIDES = "z"
> DEPENDS_prepend = "a "
> DEPENDS = "b"
> DEPENDS_z = "c"
> 
> d.update_data()
> d.getVar("DEPENDS") gives "a c"
> d.update_data()
> d.getVar("DEPENDS") gives "c"
> 
> since the _prepend is lost after the update_data(), the _z override is
> not and so the second call overwrites all of DEPENDS.
> 
> We've talked before how overrides should really renameVar, not just copy
> contents. Obviously there are a lot of subtleties to this though and we
> can't just switch due to backwards compatibility :/
> 
> Depending on the number of update_data() calls is truly horrible
> behaviour though :(

I've looked into this and started an RFC thread over on the
bitbake-devel list:

http://lists.linuxtogo.org/pipermail/bitbake-devel/2011-June/000903.html

Cheers,

Richard