* Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? @ 2017-02-13 0:26 Andreas Müller 2017-02-13 13:47 ` Andreas Müller 0 siblings, 1 reply; 17+ messages in thread From: Andreas Müller @ 2017-02-13 0:26 UTC (permalink / raw) To: Patches and discussions about the oe-core layer Hi I try to get meta-qt5-extra fit for latest oe-core (RSS). One thing that causes my attention is that many recipes's configure can be fixed by adding gettext-native. This seems odd to me because all of these recipes depend upon kdoctools which depends on gettext-native. For instance for kconfigwidgets depending on kdoctools: * in kdoctool'sworkdir/recipe-sysroot-native/usr/bin there is a gettext binary * but in kconfigwidgets'sworkdir/recipe-sysroot-native/usr/bin there isn't a gettext binary Is it possible that recipe specific sysroot broke native dependency chain? One note which might be important for this issue: In meta-qt5-extra I chose a the following design decision: If there is a pair of native/cross recipes * each cross recipe depends on native recipe * all other recipes depend on cross recipes This reduced maintenance efforts (I don't have to care if a recipe depends on cross libs or native executables) and avoids race trouble with cmake's toolchain path sequence 1. cross 2. native So what goes wrong here - or where am I mistaken? Andreas ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 0:26 Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? Andreas Müller @ 2017-02-13 13:47 ` Andreas Müller 2017-02-13 14:36 ` Martin Jansa 0 siblings, 1 reply; 17+ messages in thread From: Andreas Müller @ 2017-02-13 13:47 UTC (permalink / raw) To: Patches and discussions about the oe-core layer On Mon, Feb 13, 2017 at 1:26 AM, Andreas Müller <schnitzeltony@googlemail.com> wrote: > Hi > > I try to get meta-qt5-extra fit for latest oe-core (RSS). One thing > that causes my attention is that many recipes's configure can be fixed > by adding gettext-native. This seems odd to me because all of these > recipes depend upon kdoctools which depends on gettext-native. > For instance for kconfigwidgets depending on kdoctools: > > * in kdoctool'sworkdir/recipe-sysroot-native/usr/bin there is a gettext binary > * but in kconfigwidgets'sworkdir/recipe-sysroot-native/usr/bin there > isn't a gettext binary > > Is it possible that recipe specific sysroot broke native dependency chain? > > One note which might be important for this issue: In meta-qt5-extra I > chose a the following design decision: If there is a pair of > native/cross recipes > * each cross recipe depends on native recipe > * all other recipes depend on cross recipes > > This reduced maintenance efforts (I don't have to care if a recipe > depends on cross libs or native executables) and avoids race trouble > with cmake's toolchain path sequence 1. cross 2. native > > So what goes wrong here - or where am I mistaken? > Another more common examples: * in meta-qt5-extra I could fix many recipes by depending on qtbase-native although they already depended on qtbase (qtbase depends on qtbase-native) * right after recipe specific sysroot a dependency on qtbase-native was added in cmake_qt5.bbclass although it depended on qtbase already and qtbase depends on qtbase-native Before we carry on adding dozens of dependencies and blacklisting world: Are missing native dependencies a bug or a feature - or where am I going wrong? What scares me most is that my way of depending always on cross recipes in meta-qt5-extra will not work anymore (mentioned in my previous mail) and turn into a unmaintainable burden. Andreas ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 13:47 ` Andreas Müller @ 2017-02-13 14:36 ` Martin Jansa 2017-02-13 15:15 ` Andreas Müller 2017-02-13 15:24 ` Patrick Ohly 0 siblings, 2 replies; 17+ messages in thread From: Martin Jansa @ 2017-02-13 14:36 UTC (permalink / raw) To: Andreas Müller; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 2582 bytes --] Hi Andreas, I think it's feature which was already there, but almost never triggered (even in test-dependencies.sh tests), but with RSS it fails reliably. See: http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html On Mon, Feb 13, 2017 at 2:47 PM, Andreas Müller < schnitzeltony@googlemail.com> wrote: > On Mon, Feb 13, 2017 at 1:26 AM, Andreas Müller > <schnitzeltony@googlemail.com> wrote: > > Hi > > > > I try to get meta-qt5-extra fit for latest oe-core (RSS). One thing > > that causes my attention is that many recipes's configure can be fixed > > by adding gettext-native. This seems odd to me because all of these > > recipes depend upon kdoctools which depends on gettext-native. > > For instance for kconfigwidgets depending on kdoctools: > > > > * in kdoctool'sworkdir/recipe-sysroot-native/usr/bin there is a gettext > binary > > * but in kconfigwidgets'sworkdir/recipe-sysroot-native/usr/bin there > > isn't a gettext binary > > > > Is it possible that recipe specific sysroot broke native dependency > chain? > > > > One note which might be important for this issue: In meta-qt5-extra I > > chose a the following design decision: If there is a pair of > > native/cross recipes > > * each cross recipe depends on native recipe > > * all other recipes depend on cross recipes > > > > This reduced maintenance efforts (I don't have to care if a recipe > > depends on cross libs or native executables) and avoids race trouble > > with cmake's toolchain path sequence 1. cross 2. native > > > > So what goes wrong here - or where am I mistaken? > > > Another more common examples: > > * in meta-qt5-extra I could fix many recipes by depending on > qtbase-native although they already depended on qtbase (qtbase depends > on qtbase-native) > * right after recipe specific sysroot a dependency on qtbase-native > was added in cmake_qt5.bbclass although it depended on qtbase already > and qtbase depends on qtbase-native > > Before we carry on adding dozens of dependencies and blacklisting > world: Are missing native dependencies a bug or a feature - or where > am I going wrong? What scares me most is that my way of depending > always on cross recipes in meta-qt5-extra will not work anymore > (mentioned in my previous mail) and turn into a unmaintainable burden. > > Andreas > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > [-- Attachment #2: Type: text/html, Size: 3557 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 14:36 ` Martin Jansa @ 2017-02-13 15:15 ` Andreas Müller 2017-02-13 15:24 ` Patrick Ohly 1 sibling, 0 replies; 17+ messages in thread From: Andreas Müller @ 2017-02-13 15:15 UTC (permalink / raw) To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer On Mon, Feb 13, 2017 at 3:36 PM, Martin Jansa <martin.jansa@gmail.com> wrote: > Hi Andreas, > > I think it's feature which was already there, but almost never triggered > (even in test-dependencies.sh tests), but with RSS it fails reliably. > > See: > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html > Yeah now I remember failing on this too - thanks for the pointer. Andreas ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 14:36 ` Martin Jansa 2017-02-13 15:15 ` Andreas Müller @ 2017-02-13 15:24 ` Patrick Ohly 2017-02-13 15:32 ` Martin Jansa ` (2 more replies) 1 sibling, 3 replies; 17+ messages in thread From: Patrick Ohly @ 2017-02-13 15:24 UTC (permalink / raw) To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: > Hi Andreas, > > > I think it's feature which was already there, but almost never > triggered (even in test-dependencies.sh tests), but with RSS it fails > reliably. > > > See: > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html That's not quite the same, if I understand it correctly. In that email, Richard was talking about "dependencies of that target are not needed and not installed" and used "quilt-native" and "compiler/toolchain" as example. In other words, if recipe foo DEPENDS on bar for getting foo compiled, that dependency on bar gets ignored when installing "foo" into the recipe specific sysroot because it shouldn't be needed anymore. But the example here is a recipe foo which has a runtime dependency on bar, so bar must be installed in addition to foo, otherwise foo will not work. This is where it gets tricky: do native recipes have RDEPENDS? They are not getting packaged, so I suppose not. One could collect all RDEPENDS_* values (regardless of the actual package), but that might be too broad (for example, when "packaging" the native recipe wouldn't even produce that package). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 15:24 ` Patrick Ohly @ 2017-02-13 15:32 ` Martin Jansa 2017-02-13 15:37 ` Patrick Ohly 2017-02-13 15:45 ` Andreas Müller 2017-02-13 17:03 ` Patrick Ohly 2 siblings, 1 reply; 17+ messages in thread From: Martin Jansa @ 2017-02-13 15:32 UTC (permalink / raw) To: Patrick Ohly; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 1721 bytes --] On Mon, Feb 13, 2017 at 04:24:15PM +0100, Patrick Ohly wrote: > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: > > Hi Andreas, > > > > > > I think it's feature which was already there, but almost never > > triggered (even in test-dependencies.sh tests), but with RSS it fails > > reliably. > > > > > > See: > > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html > > That's not quite the same, if I understand it correctly. In that email, > Richard was talking about "dependencies of that target are not needed > and not installed" and used "quilt-native" and "compiler/toolchain" as > example. In other words, if recipe foo DEPENDS on bar for getting foo > compiled, that dependency on bar gets ignored when installing "foo" into > the recipe specific sysroot because it shouldn't be needed anymore. I'm not sure if it's the same or not, but back then I got very rarely missing dependency on wayland-native or qtwayland-native, now with RSS I got 20-30 recipes failing because of missing dependency on wayland-native, qtwayland-native or qtbase-native to stage the scanner, moc and other native tools in recipe-sysroot-native. And Andreas issues look similar to that. > But the example here is a recipe foo which has a runtime dependency on > bar, so bar must be installed in addition to foo, otherwise foo will not > work. > > This is where it gets tricky: do native recipes have RDEPENDS? They are > not getting packaged, so I suppose not. One could collect all RDEPENDS_* > values (regardless of the actual package), but that might be too broad > (for example, when "packaging" the native recipe wouldn't even produce > that package). [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 201 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 15:32 ` Martin Jansa @ 2017-02-13 15:37 ` Patrick Ohly 2017-02-13 15:52 ` Max Krummenacher 0 siblings, 1 reply; 17+ messages in thread From: Patrick Ohly @ 2017-02-13 15:37 UTC (permalink / raw) To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer On Mon, 2017-02-13 at 16:32 +0100, Martin Jansa wrote: > On Mon, Feb 13, 2017 at 04:24:15PM +0100, Patrick Ohly wrote: > > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: > > > Hi Andreas, > > > > > > > > > I think it's feature which was already there, but almost never > > > triggered (even in test-dependencies.sh tests), but with RSS it fails > > > reliably. > > > > > > > > > See: > > > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html > > > > That's not quite the same, if I understand it correctly. In that email, > > Richard was talking about "dependencies of that target are not needed > > and not installed" and used "quilt-native" and "compiler/toolchain" as > > example. In other words, if recipe foo DEPENDS on bar for getting foo > > compiled, that dependency on bar gets ignored when installing "foo" into > > the recipe specific sysroot because it shouldn't be needed anymore. > > I'm not sure if it's the same or not, but back then I got very rarely > missing dependency on wayland-native or qtwayland-native, now with RSS I > got 20-30 recipes failing because of missing dependency on > wayland-native, qtwayland-native or qtbase-native to stage the scanner, > moc and other native tools in recipe-sysroot-native. > > And Andreas issues look similar to that. I'm sure Andreas can clarify, but "kdoctools which depends on gettext-native" sounds like a runtime-dependency to me (but I'm just guessing that kdoctools wraps gettext, I don't really know). -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 15:37 ` Patrick Ohly @ 2017-02-13 15:52 ` Max Krummenacher 0 siblings, 0 replies; 17+ messages in thread From: Max Krummenacher @ 2017-02-13 15:52 UTC (permalink / raw) To: Patrick Ohly; +Cc: Patches and discussions about the oe-core layer Hi I got the same issue, formulated in the following question: http://lists.openembedded.org/pipermail/openembedded-core/2017-February/132604.html Here a recipe failed because a native package which were in DEPENDS did not get the RDEPENDS of those native package installed. Something I would have expected to happen. Assuming such native package A gets updated and newly RDEPENDS on something additional we will have to hunt all packages which DEPEND on A and a the additional RDEPENDS... Max 2017-02-13 16:37 GMT+01:00 Patrick Ohly <patrick.ohly@intel.com>: > On Mon, 2017-02-13 at 16:32 +0100, Martin Jansa wrote: >> On Mon, Feb 13, 2017 at 04:24:15PM +0100, Patrick Ohly wrote: >> > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: >> > > Hi Andreas, >> > > >> > > >> > > I think it's feature which was already there, but almost never >> > > triggered (even in test-dependencies.sh tests), but with RSS it fails >> > > reliably. >> > > >> > > >> > > See: >> > > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html >> > >> > That's not quite the same, if I understand it correctly. In that email, >> > Richard was talking about "dependencies of that target are not needed >> > and not installed" and used "quilt-native" and "compiler/toolchain" as >> > example. In other words, if recipe foo DEPENDS on bar for getting foo >> > compiled, that dependency on bar gets ignored when installing "foo" into >> > the recipe specific sysroot because it shouldn't be needed anymore. >> >> I'm not sure if it's the same or not, but back then I got very rarely >> missing dependency on wayland-native or qtwayland-native, now with RSS I >> got 20-30 recipes failing because of missing dependency on >> wayland-native, qtwayland-native or qtbase-native to stage the scanner, >> moc and other native tools in recipe-sysroot-native. >> >> And Andreas issues look similar to that. > > I'm sure Andreas can clarify, but "kdoctools which depends on > gettext-native" sounds like a runtime-dependency to me (but I'm just > guessing that kdoctools wraps gettext, I don't really know). > > -- > Best Regards, Patrick Ohly > > The content of this message is my personal opinion only and although > I am an employee of Intel, the statements I make here in no way > represent Intel's position on the issue, nor am I authorized to speak > on behalf of Intel on this matter. > > > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 15:24 ` Patrick Ohly 2017-02-13 15:32 ` Martin Jansa @ 2017-02-13 15:45 ` Andreas Müller 2017-02-13 18:05 ` Richard Purdie 2017-02-13 17:03 ` Patrick Ohly 2 siblings, 1 reply; 17+ messages in thread From: Andreas Müller @ 2017-02-13 15:45 UTC (permalink / raw) To: Patrick Ohly; +Cc: Patches and discussions about the oe-core layer On Mon, Feb 13, 2017 at 4:24 PM, Patrick Ohly <patrick.ohly@intel.com> wrote: > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: >> Hi Andreas, >> >> >> I think it's feature which was already there, but almost never >> triggered (even in test-dependencies.sh tests), but with RSS it fails >> reliably. >> >> >> See: >> http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html Richard wrote | What it does mean is that any recipe needing a -native recipe to build | should list it in DEPENDS directly, not rely on other dependencies to | pull it in for them. This applies to pkgconfig-native, intltool-native | and also to wayland-native. This answers my question and leaves me unhappy - I am out for a while.. Andreas ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 15:45 ` Andreas Müller @ 2017-02-13 18:05 ` Richard Purdie 2017-02-13 18:17 ` Andreas Müller 2017-03-05 0:55 ` Andreas Müller 0 siblings, 2 replies; 17+ messages in thread From: Richard Purdie @ 2017-02-13 18:05 UTC (permalink / raw) To: Andreas Müller, Patrick Ohly Cc: Patches and discussions about the oe-core layer On Mon, 2017-02-13 at 16:45 +0100, Andreas Müller wrote: > On Mon, Feb 13, 2017 at 4:24 PM, Patrick Ohly <patrick.ohly@intel.com > > wrote: > > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: > > > I think it's feature which was already there, but almost never > > > triggered (even in test-dependencies.sh tests), but with RSS it > > > fails > > > reliably. > > > > > > See: > > > http://lists.openembedded.org/pipermail/openembedded-core/2016-Ju > > > ly/124435.html > Richard wrote > > > > What it does mean is that any recipe needing a -native recipe to > > build > > should list it in DEPENDS directly, not rely on other dependencies > > to > > pull it in for them. This applies to pkgconfig-native, intltool- > > native > > and also to wayland-native. > This answers my question and leaves me unhappy - I am out for a > while.. To be clear, you can't depend on the build dependencies of some other recipe to be available to your recipe just because you DEPEND on it. The RDEPENDS issue that Patrick mentions is a different one, its a valid recipe markup problem we have processing RDEPENDS information in the native case. The kind of indirection that Andreas sounds like he's been relying on may have happened to work most of the time but I'd bet there were ways that a semi populated sstate cache could break it. With RSS, we encoded the sstate behaviour in a deterministic way so the races that might have happened now do happen all the time. The simple solution for repetitive dependencies and shortcuts is just to create a class, list all the things you need in there and then inherit the class in the recipes as appropriate. I'd hope that isn't too hard to sort out... Cheers, Richard ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 18:05 ` Richard Purdie @ 2017-02-13 18:17 ` Andreas Müller 2017-03-05 0:55 ` Andreas Müller 1 sibling, 0 replies; 17+ messages in thread From: Andreas Müller @ 2017-02-13 18:17 UTC (permalink / raw) To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer On Mon, Feb 13, 2017 at 7:05 PM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > To be clear, you can't depend on the build dependencies of some other > recipe to be available to your recipe just because you DEPEND on it. > Yes I expected that my fault. Together with my hack on sysroot alignments I think this is the end of kf5-kde in meta-qt5-extra. Andreas ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 18:05 ` Richard Purdie 2017-02-13 18:17 ` Andreas Müller @ 2017-03-05 0:55 ` Andreas Müller 1 sibling, 0 replies; 17+ messages in thread From: Andreas Müller @ 2017-03-05 0:55 UTC (permalink / raw) To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 857 bytes --] On Mon, Feb 13, 2017 at 7:05 PM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > To be clear, you can't depend on the build dependencies of some other > recipe to be available to your recipe just because you DEPEND on it. Sorry me again - but is it possible that this behavior has changed? With recent oe-core I see * many configure logs finding pkg-config from recipe-sysroot-native although the recipes do not inherit pkgconfig and do not have pkgconfig-native in depends * adding cmake to inherit adds "bzip2-replacement-native expat-native xz-native zlib-native curl-native" to recipe-sysroot-recipe-sysroot-native/sysroot-providers. I reduced the recipe I was currently working on to no dependencies and inherit cmake only (for test attached). Can somebody tell me what was changed or what I am missing? Andreas [-- Attachment #2: supertuxkart_0.9.2.bb --] [-- Type: application/octet-stream, Size: 423 bytes --] LICENSE = "GPLv2 & GPLv3+ " LIC_FILES_CHKSUM = "file://COPYING;md5=a71cb78659d60f2ced58a594cb65bfba" inherit cmake S = "${WORKDIR}/supertuxkart-${PV}" SRC_URI = "http://sourceforge.net/projects/supertuxkart/files/SuperTuxKart/${PV}/supertuxkart-${PV}-src.tar.xz;protocol=https" SRC_URI[md5sum] = "f1f5081fd41b8eeb310b4edc07b9ee12" SRC_URI[sha256sum] = "0b080bb098a26adb552d6fd48905bcb6b1e873ef1567457d7268d7d3aaa48282" ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 15:24 ` Patrick Ohly 2017-02-13 15:32 ` Martin Jansa 2017-02-13 15:45 ` Andreas Müller @ 2017-02-13 17:03 ` Patrick Ohly 2017-02-13 17:06 ` Patrick Ohly 2017-02-14 10:26 ` Patrick Ohly 2 siblings, 2 replies; 17+ messages in thread From: Patrick Ohly @ 2017-02-13 17:03 UTC (permalink / raw) To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer On Mon, 2017-02-13 at 16:24 +0100, Patrick Ohly wrote: > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: > > Hi Andreas, > > > > > > I think it's feature which was already there, but almost never > > triggered (even in test-dependencies.sh tests), but with RSS it fails > > reliably. > > > > > > See: > > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html > > That's not quite the same, if I understand it correctly. In that email, > Richard was talking about "dependencies of that target are not needed > and not installed" and used "quilt-native" and "compiler/toolchain" as > example. In other words, if recipe foo DEPENDS on bar for getting foo > compiled, that dependency on bar gets ignored when installing "foo" into > the recipe specific sysroot because it shouldn't be needed anymore. > > But the example here is a recipe foo which has a runtime dependency on > bar, so bar must be installed in addition to foo, otherwise foo will not > work. > > This is where it gets tricky: do native recipes have RDEPENDS? They are > not getting packaged, so I suppose not. One could collect all RDEPENDS_* > values (regardless of the actual package), but that might be too broad > (for example, when "packaging" the native recipe wouldn't even produce > that package). Apparently RDEPENDS do work, also for native recipes. Based on some testing, it seems that all RDEPENDS are considered, even those that refer to packages that would normally be empty, i.e. the sysroot potentially contains more than strictly needed, but that shouldn't be a problem. Andreas, does adding RDEPENDS instead of (or, where needed, in addition to) DEPENDS fix you problem? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 17:03 ` Patrick Ohly @ 2017-02-13 17:06 ` Patrick Ohly 2017-02-14 10:26 ` Patrick Ohly 1 sibling, 0 replies; 17+ messages in thread From: Patrick Ohly @ 2017-02-13 17:06 UTC (permalink / raw) To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer On Mon, 2017-02-13 at 18:03 +0100, Patrick Ohly wrote: > On Mon, 2017-02-13 at 16:24 +0100, Patrick Ohly wrote: > > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: > > > Hi Andreas, > > > > > > > > > I think it's feature which was already there, but almost never > > > triggered (even in test-dependencies.sh tests), but with RSS it fails > > > reliably. > > > > > > > > > See: > > > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html > > > > That's not quite the same, if I understand it correctly. In that email, > > Richard was talking about "dependencies of that target are not needed > > and not installed" and used "quilt-native" and "compiler/toolchain" as > > example. In other words, if recipe foo DEPENDS on bar for getting foo > > compiled, that dependency on bar gets ignored when installing "foo" into > > the recipe specific sysroot because it shouldn't be needed anymore. > > > > But the example here is a recipe foo which has a runtime dependency on > > bar, so bar must be installed in addition to foo, otherwise foo will not > > work. > > > > This is where it gets tricky: do native recipes have RDEPENDS? They are > > not getting packaged, so I suppose not. One could collect all RDEPENDS_* > > values (regardless of the actual package), but that might be too broad > > (for example, when "packaging" the native recipe wouldn't even produce > > that package). > > Apparently RDEPENDS do work, also for native recipes. Based on some > testing, it seems that all RDEPENDS are considered, even those that > refer to packages that would normally be empty, i.e. the sysroot > potentially contains more than strictly needed, but that shouldn't be a > problem. > > Andreas, does adding RDEPENDS instead of (or, where needed, in addition > to) DEPENDS fix you problem? ... and to avoid confusion: I meant adding gettext to RDEPEND_${PN} in kdoctools, not adding it to every recipe which uses kdoctools. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-13 17:03 ` Patrick Ohly 2017-02-13 17:06 ` Patrick Ohly @ 2017-02-14 10:26 ` Patrick Ohly 2017-02-19 22:26 ` Richard Purdie 1 sibling, 1 reply; 17+ messages in thread From: Patrick Ohly @ 2017-02-14 10:26 UTC (permalink / raw) To: Martin Jansa, Richard Purdie Cc: Patches and discussions about the oe-core layer On Mon, 2017-02-13 at 18:03 +0100, Patrick Ohly wrote: > On Mon, 2017-02-13 at 16:24 +0100, Patrick Ohly wrote: > > On Mon, 2017-02-13 at 15:36 +0100, Martin Jansa wrote: > > > Hi Andreas, > > > > > > > > > I think it's feature which was already there, but almost never > > > triggered (even in test-dependencies.sh tests), but with RSS it fails > > > reliably. > > > > > > > > > See: > > > http://lists.openembedded.org/pipermail/openembedded-core/2016-July/124435.html > > > > That's not quite the same, if I understand it correctly. In that email, > > Richard was talking about "dependencies of that target are not needed > > and not installed" and used "quilt-native" and "compiler/toolchain" as > > example. In other words, if recipe foo DEPENDS on bar for getting foo > > compiled, that dependency on bar gets ignored when installing "foo" into > > the recipe specific sysroot because it shouldn't be needed anymore. > > > > But the example here is a recipe foo which has a runtime dependency on > > bar, so bar must be installed in addition to foo, otherwise foo will not > > work. > > > > This is where it gets tricky: do native recipes have RDEPENDS? They are > > not getting packaged, so I suppose not. One could collect all RDEPENDS_* > > values (regardless of the actual package), but that might be too broad > > (for example, when "packaging" the native recipe wouldn't even produce > > that package). > > Apparently RDEPENDS do work, also for native recipes. Based on some > testing, it seems that all RDEPENDS are considered, even those that > refer to packages that would normally be empty, i.e. the sysroot > potentially contains more than strictly needed, but that shouldn't be a > problem. My testing was flawed: in addition to the RDEPENDS there also was a DEPENDS with the same entry, and despite what was said earlier about build dependencies, that entry did have an effect. So it is a bit more complicated. The way I'd expect this to work for native tools is this: 1. DEPENDS should be ignored (build dependencies like compiler which are not needed when using the resulting tool) 2. RDEPENDS_${PN} needs to be used (essential runtime dependencies, set the same way as for the target recipes, because then BBCLASSEXTEND native mostly works automatically) 3. RDEPENDS_foo for foo != ${PN} is *not* used - that's basically a convention, and addresses the issue I mentioned where it is unclear for a native recipe whether it really has such a package But the actual implementation isn't quite doing this. Instead, extend_recipe_sysroot() in staging.bbclass and setscene_depvalid() in sstate.bbclass look at the task dependencies. Here's an example, using Poky e758547db = current master: $ bitbake wic-tools ... $ grep gettext tmp/work/i586-poky-linux/wic-tools/1.0-r0/temp/log.do_prepare_recipe_sysroot Considering setscene task: ['gettext-native', 'do_populate_sysroot'] considering dependency: ['gettext-native', 'do_populate_sysroot'] Skipping setscene dependency virtual:native:/work/poky/meta/recipes-core/gettext/gettext_0.19.8.1.bb:do_populate_sysroot for installation into the sysroot ... $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/ -name gettext tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/usr/share/gettext tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/installeddeps/gettext "gettext" is not installed. Let's add it to RDEPENDS_${PN} of dosfstools-native, one of the dependencies of wic-tools: $ echo 'RDEPENDS_dosfstools-native_append_pn-dosfstools-native = " gettext-native"' >conf/auto.conf $ bitbake -e dosfstools-native | grep ^RDEPENDS RDEPENDS_dosfstools-native="gettext-native" RDEPENDS="" RDEPENDS_kernel-base="" RDEPENDS_dosfstools-native-staticdev="dosfstools-native-dev (= 4.1-r0)" RDEPENDS_dosfstools-native-dev="dosfstools-native (= 4.1-r0)" First observation: "bitbake wic-tools:do_prepare_recipe_sysroot" does not run again. But even forcing it to run again doesn't change anything, i.e. either "gettext-native" in RDEPENDS_dosfstools-native or RDEPENDS_dosfstools-native are ignored. Let's compare adding something new to RDEPENDS vs. DEPENDS: $ echo 'DEPENDS_append_pn-dosfstools-native = " socat-native"' >conf/auto.conf $ bitbake -g wic-tools $ grep -e '->.*socat-native.do_populate_sysroot' task-depends.dot "dosfstools-native.do_prepare_recipe_sysroot" -> "socat-native.do_populate_sysroot" $ echo 'RDEPENDS_dosfstools-native_append_pn-dosfstools-native = " gettext-native"' >conf/auto.conf $ bitbake -g wic-tools $ grep -e '->.*socat-native.do_populate_sysroot' task-depends.dot Nothing. RDEPENDS_dosfstools-native was completely ignored. To me that looks like a missing feature or even a bug, if it was meant to work already. I'm leaning towards the "bug" theory. There are examples where RDEPENDS is set explicitly for native recipes, albeit inconsistently: meta/recipes-devtools/xmlto/xmlto_0.0.28.bb:RDEPENDS_class-native = "libxslt-native" meta/recipes-graphics/xorg-app/mkfontdir_1.0.7.bb:RDEPENDS_${PN}_class-native += "mkfontscale-native" Of these two, only setting RDEPENDS_class-native actually has an effect (tested by adding socat-native to the assignment). mkfontscale-native gets installed through some other, indirect dependency. Digging deeper, it seems that it also depends on whether a recipe sets PACKAGES. The insane.bbclass pkgvarcheck allows RDEPENDS (without suffix) if PACKAGES is empty, which is the case for native recipes - unless a recipe sets PACKAGES explicitly, as libnewt does with PACKAGES_append. It gets more confusing. Based on the observation about xmlto, I'd expect that this change to libnewt should have an effect: $ git diff meta/recipes-extended/newt diff --git a/meta/recipes-extended/newt/libnewt_0.52.19.bb b/meta/recipes-extended/newt/libnewt_0.52.19.bb index a26ce1fbe75..0a1d693e110 100644 --- a/meta/recipes-extended/newt/libnewt_0.52.19.bb +++ b/meta/recipes-extended/newt/libnewt_0.52.19.bb @@ -38,7 +38,8 @@ CLEANBROKEN = "1" export CPPFLAGS -PACKAGES_prepend = "whiptail " +PACKAGES_prepend_class-target = "whiptail " +RDEPENDS_class-native = "socat-native" do_configure_prepend() { sh autogen.sh It changes PACKAGES and RDEPENDS as expected, but tasks dependencies do not: $ bitbake -e libnewt-native | grep -e PACKAGES= -e ^RDEPENDS= RDEPENDS="socat-native" PACKAGES="" $ bitbake -g libnewt-native $ grep -e '->.*socat-native' task-depends.dot Nothing. Here's the same test for xmlto: $ git diff meta/recipes-devtools/xmlto diff --git a/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb b/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb index ce5d1e0c502..9e995fe5e9d 100644 --- a/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb +++ b/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb @@ -13,7 +13,7 @@ SRC_URI[md5sum] = "a1fefad9d83499a15576768f60f847c6" SRC_URI[sha256sum] = "2f986b7c9a0e9ac6728147668e776d405465284e13c74d4146c9cbc51fd8aad3" inherit autotools -RDEPENDS_class-native = "libxslt-native" +RDEPENDS_class-native = "libxslt-native socat-native" # xmlto needs getopt/xmllint/xsltproc/bash/tail at runtime RDEPENDS_${PN} = "docbook-xml-dtd4 \ $ bitbake -e xmlto-native | grep -e PACKAGES= -e ^RDEPENDS= RDEPENDS="libxslt-native socat-native" PACKAGES="" $ bitbake -g xmlto-native $ grep -e '->.*socat-native' task-depends.dot "socat-native.do_populate_sysroot" -> "socat-native.do_install" "socat-native.do_patch" -> "socat-native.do_unpack" "socat-native.do_unpack" -> "socat-native.do_fetch" "socat-native.do_compile" -> "socat-native.do_configure" "socat-native.do_prepare_recipe_sysroot" -> "socat-native.do_fetch" "socat-native.do_configure" -> "socat-native.do_prepare_recipe_sysroot" "socat-native.do_configure" -> "socat-native.do_patch" "socat-native.do_install" -> "socat-native.do_compile" "xmlto-native.do_populate_sysroot" -> "socat-native.do_populate_sysroot" If this email sounds confused, then that's because I am - sorry for that ;-} Can someone clarify how RDEPENDS really works at the moment for native recipes? Now regarding DEPENDS being ignored: that's not quite the case either. $ echo 'DEPENDS_append_pn-dosfstools-native = " socat-native"' >conf/auto.conf $ bitbake wic-tools ... $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/ -name socat tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/usr/bin/socat In other words, a build dependency got installed although it shouldn't really be needed. Finally, re-running do_populate_sysroot_setscene does not remove entries which no longer should be there. Test case: $ echo 'DEPENDS_append_pn-dosfstools-native = " socat-native"' >conf/auto.conf $ bitbake wic-tools:do_prepare_recipe_sysroot | cat ... NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: Started NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: Succeeded NOTE: Tasks Summary: Attempted 682 tasks of which 681 didn't need to be rerun and all succeeded. $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/ -name socat tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/usr/bin/socat $ echo >conf/auto.conf $ bitbake wic-tools:do_prepare_recipe_sysroot | cat ... NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: Started NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: Succeeded NOTE: Tasks Summary: Attempted 674 tasks of which 673 didn't need to be rerun and all succeeded. $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/ -name socat tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/usr/bin/socat $ bitbake -c clean wic-tools ... $ bitbake wic-tools:do_prepare_recipe_sysroot | cat ... NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: Started NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: Succeeded NOTE: Tasks Summary: Attempted 674 tasks of which 672 didn't need to be rerun and all succeeded. $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native/ -name socat Is there a missing [cleandirs] for the recipe-sysroot-native or is this reuse of the existing content intentional? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-14 10:26 ` Patrick Ohly @ 2017-02-19 22:26 ` Richard Purdie 2017-02-20 14:14 ` Patrick Ohly 0 siblings, 1 reply; 17+ messages in thread From: Richard Purdie @ 2017-02-19 22:26 UTC (permalink / raw) To: Patrick Ohly, Martin Jansa Cc: Patches and discussions about the oe-core layer On Tue, 2017-02-14 at 11:26 +0100, Patrick Ohly wrote: > My testing was flawed: in addition to the RDEPENDS there also was a > DEPENDS with the same entry, and despite what was said earlier about > build dependencies, that entry did have an effect. So it is a bit > more > complicated. > > The way I'd expect this to work for native tools is this: > 1. DEPENDS should be ignored (build dependencies like compiler > which are not needed when using the resulting tool) Firstly. this is not true at all. Native recipes have build dependencies just the same as target recipes. They depend on the native compiler but you could say that is in ASSUME_PROVIDED. A recipe which uses gettext macros would DEPENDS on gettext-native, it gets autoreconf'd just like a target recipe and hence has the same kind of dependency requirements. Obviously native recipe only depend on other native recipes though. qemu-native DEPENDing on things like libsdl- native is another example. By comparison RDEPENDS of a native recipe are only needed after its been compiled and we're about to run it (at least by definition). > 2. RDEPENDS_${PN} needs to be used (essential runtime > dependencies, > set the same way as for the target recipes, because then > BBCLASSEXTEND native mostly works automatically) > 3. RDEPENDS_foo for foo != ${PN} is *not* used - that's > basically a > convention, and addresses the issue I mentioned where it is > unclear for a native recipe whether it really has such a > package > > But the actual implementation isn't quite doing this. Instead, > extend_recipe_sysroot() in staging.bbclass and setscene_depvalid() in > sstate.bbclass look at the task dependencies. Effectively staging.bbclass calls into sstate.bbclass and they both run off exactly the same base dependency data which is represented by task- depends.dot. > Here's an example, using Poky e758547db = current master: > $ bitbake wic-tools > ... > $ grep gettext tmp/work/i586-poky-linux/wic-tools/1.0- > r0/temp/log.do_prepare_recipe_sysroot > Considering setscene task: ['gettext-native', 'do_populate_sysroot'] > considering dependency: ['gettext-native', 'do_populate_sysroot'] > Skipping setscene dependency virtual:native:/work/poky/meta/recipes- > core/gettext/gettext_0.19.8.1.bb:do_populate_sysroot for installation > into the sysroot > ... > $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/ > -name gettext > tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/usr/share/gettext > tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/installeddeps/gettext > > "gettext" is not installed. wic-tools is a target recipe (whether it should be or not is a different question). From an sstate perspective, execution of a target recipe will not need any indirect native build tools so it is correctly ignoring gettext-native. > Let's add it to RDEPENDS_${PN} of dosfstools-native, one of the > dependencies of wic-tools: > $ echo 'RDEPENDS_dosfstools-native_append_pn-dosfstools-native = " > gettext-native"' >conf/auto.conf > $ bitbake -e dosfstools-native | grep ^RDEPENDS > RDEPENDS_dosfstools-native="gettext-native" > RDEPENDS="" > RDEPENDS_kernel-base="" > RDEPENDS_dosfstools-native-staticdev="dosfstools-native-dev (= 4.1- > r0)" > RDEPENDS_dosfstools-native-dev="dosfstools-native (= 4.1-r0)" I checked the bitbake code and it will only see RDEPENDS, not RDEPENDS-_${PN} since PACKAGES is empty in the -native case. That explains what you see above however I tried dropping the _${PN} piece and it still doesn't find the gettext-native dependency. With a little more digging, I realised that base.bbclass does: do_prepare_recipe_sysroot[deptask] = "do_populate_sysroot" Note that this is "deptask", not "rdeptask". The lack of the "r" means it will work off DEPENDS, not RDEPENDS. [...] > Nothing. RDEPENDS_dosfstools-native was completely ignored. To me > that looks like a missing feature or even a bug, if it was meant to > work already. > > I'm leaning towards the "bug" theory. There are examples where > RDEPENDS > is set explicitly for native recipes, albeit inconsistently: > meta/recipes-devtools/xmlto/xmlto_0.0.28.bb:RDEPENDS_class-native = > "libxslt-native" > meta/recipes-graphics/xorg- > app/mkfontdir_1.0.7.bb:RDEPENDS_${PN}_class-native += "mkfontscale- > native" I think the system is working as designed however the design is suboptimal as I've hinted at before and the metadata is confused and mainly appears to work by luck. > Of these two, only setting RDEPENDS_class-native actually has an > effect > (tested by adding socat-native to the assignment). mkfontscale-native > gets installed through some other, indirect dependency. > > Digging deeper, it seems that it also depends on whether a recipe > sets > PACKAGES. The insane.bbclass pkgvarcheck allows RDEPENDS (without > suffix) if PACKAGES is empty, which is the case for native recipes - > unless a recipe sets PACKAGES explicitly, as libnewt does with > PACKAGES_append. I think my explanation above does clarify that? > It gets more confusing. Based on the observation about xmlto, I'd > expect > that this change to libnewt should have an effect: > > $ git diff meta/recipes-extended/newt > diff --git a/meta/recipes-extended/newt/libnewt_0.52.19.bb > b/meta/recipes-extended/newt/libnewt_0.52.19.bb > index a26ce1fbe75..0a1d693e110 100644 > --- a/meta/recipes-extended/newt/libnewt_0.52.19.bb > +++ b/meta/recipes-extended/newt/libnewt_0.52.19.bb > @@ -38,7 +38,8 @@ CLEANBROKEN = "1" > > export CPPFLAGS > > -PACKAGES_prepend = "whiptail " > +PACKAGES_prepend_class-target = "whiptail " > +RDEPENDS_class-native = "socat-native" > > do_configure_prepend() { > sh autogen.sh > > It changes PACKAGES and RDEPENDS as expected, but tasks dependencies > do > not: > > $ bitbake -e libnewt-native | grep -e PACKAGES= -e ^RDEPENDS= > RDEPENDS="socat-native" > PACKAGES="" > $ bitbake -g libnewt-native > $ grep -e '->.*socat-native' task-depends.dot > > Nothing. This is because of the deptask verses rdeptask. > Here's the same test for xmlto: > > $ git diff meta/recipes-devtools/xmlto > diff --git a/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb > b/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb > index ce5d1e0c502..9e995fe5e9d 100644 > --- a/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb > +++ b/meta/recipes-devtools/xmlto/xmlto_0.0.28.bb > @@ -13,7 +13,7 @@ SRC_URI[md5sum] = > "a1fefad9d83499a15576768f60f847c6" > SRC_URI[sha256sum] = > "2f986b7c9a0e9ac6728147668e776d405465284e13c74d4146c9cbc51fd8aad3" > > inherit autotools > -RDEPENDS_class-native = "libxslt-native" > +RDEPENDS_class-native = "libxslt-native socat-native" > > # xmlto needs getopt/xmllint/xsltproc/bash/tail at runtime > RDEPENDS_${PN} = "docbook-xml-dtd4 \ > > $ bitbake -e xmlto-native | grep -e PACKAGES= -e ^RDEPENDS= > RDEPENDS="libxslt-native socat-native" > PACKAGES="" > $ bitbake -g xmlto-native > $ grep -e '->.*socat-native' task-depends.dot > "socat-native.do_populate_sysroot" -> "socat-native.do_install" > "socat-native.do_patch" -> "socat-native.do_unpack" > "socat-native.do_unpack" -> "socat-native.do_fetch" > "socat-native.do_compile" -> "socat-native.do_configure" > "socat-native.do_prepare_recipe_sysroot" -> "socat-native.do_fetch" > "socat-native.do_configure" -> "socat- > native.do_prepare_recipe_sysroot" > "socat-native.do_configure" -> "socat-native.do_patch" > "socat-native.do_install" -> "socat-native.do_compile" > "xmlto-native.do_populate_sysroot" -> "socat- > native.do_populate_sysroot" > > If this email sounds confused, then that's because I am - sorry for > that ;-} > > Can someone clarify how RDEPENDS really works at the moment for > native > recipes? > > Now regarding DEPENDS being ignored: that's not quite the case > either. > > $ echo 'DEPENDS_append_pn-dosfstools-native = " socat-native"' > >conf/auto.conf > $ bitbake wic-tools > ... > $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/ -name socat > tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/usr/bin/socat > > In other words, a build dependency got installed although it > shouldn't really be needed. We can't tell the difference between a build dependency which may contain a library we link to and need at runtime and one that does not. > Finally, re-running do_populate_sysroot_setscene does not remove > entries > which no longer should be there. Test case: > > $ echo 'DEPENDS_append_pn-dosfstools-native = " socat-native"' > >conf/auto.conf > $ bitbake wic-tools:do_prepare_recipe_sysroot | cat > ... > NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: > Started > NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: > Succeeded > NOTE: Tasks Summary: Attempted 682 tasks of which 681 didn't need to > be rerun and all succeeded. > $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/ -name socat > tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/usr/bin/socat > $ echo >conf/auto.conf > $ bitbake wic-tools:do_prepare_recipe_sysroot | cat > ... > NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: > Started > NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: > Succeeded > NOTE: Tasks Summary: Attempted 674 tasks of which 673 didn't need to > be rerun and all succeeded. > $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/ -name socat > tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/usr/bin/socat > $ bitbake -c clean wic-tools > ... > $ bitbake wic-tools:do_prepare_recipe_sysroot | cat > ... > NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: > Started > NOTE: recipe wic-tools-1.0-r0: task do_prepare_recipe_sysroot: > Succeeded > NOTE: Tasks Summary: Attempted 674 tasks of which 672 didn't need to > be rerun and all succeeded. > $ find tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot- > native/ -name socat > > Is there a missing [cleandirs] for the recipe-sysroot-native or is > this > reuse of the existing content intentional? It is "intentional" since you can have two tasks running in parallel, each with different dependencies which both have to be installed into the sysroot. Neither will know about the dependencies the other has. What does happen is that any changed/unreachable dependency will be removed from the sysroot though since we can detect those. I wish we could do better but since its recipe specific, not task specific, we're stuck with this problem as far as I can tell. Hopefully that at least explains some of what is happening. As I've said, I do think we need work/cleanup in this area. Cheers, Richard ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? 2017-02-19 22:26 ` Richard Purdie @ 2017-02-20 14:14 ` Patrick Ohly 0 siblings, 0 replies; 17+ messages in thread From: Patrick Ohly @ 2017-02-20 14:14 UTC (permalink / raw) To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer On Sun, 2017-02-19 at 14:26 -0800, Richard Purdie wrote: > On Tue, 2017-02-14 at 11:26 +0100, Patrick Ohly wrote: > > My testing was flawed: in addition to the RDEPENDS there also was a > > DEPENDS with the same entry, and despite what was said earlier about > > build dependencies, that entry did have an effect. So it is a bit > > more > > complicated. > > > > The way I'd expect this to work for native tools is this: > > 1. DEPENDS should be ignored (build dependencies like compiler > > which are not needed when using the resulting tool) > > Firstly. this is not true at all. Native recipes have build > dependencies just the same as target recipes. Yes, of course. But here I was talking about installing an already built tool in the RSS, and in that case the build dependencies are not always necessary anymore. What I hadn't considered is that there's no packaging and thus also no analysis of ELF files to detect library dependencies. Without that, one has no choice and must install all DEPENDS, even those that only provide build tools that aren't needed anymore. I'm not suggesting to add such packaging. It's another big change and it is not clear whether the reduced size of the resulting sysroots really would outweigh the extra costs and complexity (for example, boot-strapping the ELF analysis). > By comparison RDEPENDS of a native recipe are only needed after its > been compiled and we're about to run it (at least by definition). Let's focus on RDEPENDS. > With a little more digging, I realised that base.bbclass does: > > do_prepare_recipe_sysroot[deptask] = "do_populate_sysroot" > > Note that this is "deptask", not "rdeptask". > > The lack of the "r" means it will work off DEPENDS, not RDEPENDS. So just to be clear, all RDEPENDS* variables are currently ignored when populating the RSS. I think that's what surprised me. Some sort of support for it also in the native case would make sense, because it is cleaner and avoids rebuilding tools when only they runtime dependencies change. We just need to agree on something and then make it work. I propose we use RDEPENDS_$i for i in PACKAGES, because unsetting PACKAGES in native.bbclass when used via BBCLASSEXTEND is a hack that already fails in some cases (like the PACKAGES_prepend that I mentioned earlier) and because it is more consistent with target recipes. To handle that case that a RDEPENDS_foobar pulls in a dependency that is irrelevant because the foobar package would have been empty and skipped if packaging was done, PACKAGES_class-native can be used to set a smaller list. What about RDEPENDS for native-sdk? Does it matter there? Can do_prepare_recipe_sysroot be extended to consider both DEPENDS and RDEPENDS_$i? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2017-03-05 0:55 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-02-13 0:26 Does recipe specific sysrooot (or whatelse in current oe) break native dependencies? Andreas Müller 2017-02-13 13:47 ` Andreas Müller 2017-02-13 14:36 ` Martin Jansa 2017-02-13 15:15 ` Andreas Müller 2017-02-13 15:24 ` Patrick Ohly 2017-02-13 15:32 ` Martin Jansa 2017-02-13 15:37 ` Patrick Ohly 2017-02-13 15:52 ` Max Krummenacher 2017-02-13 15:45 ` Andreas Müller 2017-02-13 18:05 ` Richard Purdie 2017-02-13 18:17 ` Andreas Müller 2017-03-05 0:55 ` Andreas Müller 2017-02-13 17:03 ` Patrick Ohly 2017-02-13 17:06 ` Patrick Ohly 2017-02-14 10:26 ` Patrick Ohly 2017-02-19 22:26 ` Richard Purdie 2017-02-20 14:14 ` Patrick Ohly
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.