Changing DISTRO_FEATURES / build directory

Changing DISTRO_FEATURES / build directory

[Chris Laplante]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

There is a familiar message in sstate.bbclass that explains: "... It may be you changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning those recipes should again resolve this error, however switching DISTRO_FEATURES on an existing build directory is not supported - you should really clean out tmp and rebuild (reusing sstate should be safe)..."

But the release notes for the 2.0 release suggest that it should work: https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#migration-2.0-automatic-stale-sysroot-file-cleanup.

So, is changing DISTRO_FEATURES but not changing the build directory expected to work?

Thanks,
Chris

[-- Attachment #2: Type: text/html, Size: 2748 bytes --]

Re: [OE-core] Changing DISTRO_FEATURES / build directory

[Richard Purdie]

On Thu, 2020-04-23 at 19:12 +0000, Chris Laplante via
lists.openembedded.org wrote:
> There is a familiar message in sstate.bbclass that explains: “… It
> may be you changed DISTRO_FEATURES from systemd to udev or vice
> versa. Cleaning those recipes should again resolve this error,
> however switching DISTRO_FEATURES on an existing build directory is
> not supported – you should really clean out tmp and rebuild (reusing
> sstate should be safe)…”

We should prune that message down as there are a few ways you could hit
it and not touch DISTRO_FEATURES, most of those are more likely now.

> But the release notes for the 2.0 release suggest that it should
> work: 
> https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#migration-2.0-automatic-stale-sysroot-file-cleanup
> .
>  
> So, is changing DISTRO_FEATURES but not changing the build directory
> expected to work?

Should work but has bugs.

There were some fixed around the thud/zeus timeframe.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13702

is a recently reported one that has patches in master-next (both to fix
and add a test case).

Is there a failure with reproducer you have? Which release?

Cheers,

Richard

Re: [OE-core] Changing DISTRO_FEATURES / build directory

[Chris Laplante]

> > So, is changing DISTRO_FEATURES but not changing the build directory
> > expected to work?
> 
> Should work but has bugs.
> 
> There were some fixed around the thud/zeus timeframe.
> 
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13702
> 
> is a recently reported one that has patches in master-next (both to fix and
> add a test case).
> 
> Is there a failure with reproducer you have? Which release?

I haven't encountered any bugs yet, no. I am working on adding some custom DISTRO_FEATURES to our distro. I was trying to convince myself whether vardeps are necessary for DISTRO_FEATURES, and couldn't decide. For instance, if I have:

    do_unpack_append() {
        echo "${@bb.utils.contains("DISTRO_FEATURES", "my-feature", "true", "false", d)}" >> ${S}/whatever
    }

bitbake-dumpsigs on the do_unpack sigdata reveals that DISTRO_FEATURES is *not* part of the task dependencies. So I was wondering if I needed to do:

    do_unpack[vardeps] += "DISTRO_FEATURES"

but looking around poky and meta-openembedded, I don't see anyone else doing that kind of thing. 

The interesting thing is that the bitbake-dumpsig does have some special looking DISTRO_FEATURES output, such as:

    DISTRO_FEATURES{usrmerge} = Unset
    DISTRO_FEATURES{my-feature} = Set

So it seems there is some DISTRO_FEATURES specific processing?

It's also unclear to me whether the signature code is able to understand bb.utils.contains and automatically pluck out the fact that DISTRO_FEATURES should be accounted for. 

Thanks,
Chris

Re: [OE-core] Changing DISTRO_FEATURES / build directory

[Richard Purdie]

On Fri, 2020-04-24 at 13:59 +0000, chris.laplante@agilent.com wrote:
> > > So, is changing DISTRO_FEATURES but not changing the build
> > > directory
> > > expected to work?
> > 
> > Should work but has bugs.
> > 
> > There were some fixed around the thud/zeus timeframe.
> > 
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=13702
> > 
> > is a recently reported one that has patches in master-next (both to
> > fix and
> > add a test case).
> > 
> > Is there a failure with reproducer you have? Which release?
> 
> I haven't encountered any bugs yet, no. I am working on adding some
> custom DISTRO_FEATURES to our distro. I was trying to convince myself
> whether vardeps are necessary for DISTRO_FEATURES, and couldn't
> decide. For instance, if I have:
> 
>     do_unpack_append() {
>         echo "${@bb.utils.contains("DISTRO_FEATURES", "my-feature",
> "true", "false", d)}" >> ${S}/whatever
>     }
> 
> bitbake-dumpsigs on the do_unpack sigdata reveals that
> DISTRO_FEATURES is *not* part of the task dependencies. So I was
> wondering if I needed to do:
> 
>     do_unpack[vardeps] += "DISTRO_FEATURES"
> 
> but looking around poky and meta-openembedded, I don't see anyone
> else doing that kind of thing. 
>
> The interesting thing is that the bitbake-dumpsig does have some
> special looking DISTRO_FEATURES output, such as:
> 
>     DISTRO_FEATURES{usrmerge} = Unset
>     DISTRO_FEATURES{my-feature} = Set
> 
> So it seems there is some DISTRO_FEATURES specific processing?
>
> It's also unclear to me whether the signature code is able to
> understand bb.utils.contains and automatically pluck out the fact
> that DISTRO_FEATURES should be accounted for. 

"contains" has special magic where it caches the specific value
presence rather than the whole string. I'd therefore recommend using
the contains functions where at all possible for that reason.

Cheers,

Richard

Re: [OE-core] Changing DISTRO_FEATURES / build directory

[Chris Laplante]

> "contains" has special magic where it caches the specific value presence
> rather than the whole string. I'd therefore recommend using the contains
> functions where at all possible for that reason.

So then regarding my other points, it is correctly handled by the signature code? If so, why doesn’t it show up in task dependencies?

Thanks,
Chris 

Re: [OE-core] Changing DISTRO_FEATURES / build directory

[Richard Purdie]

On Fri, 2020-04-24 at 14:20 +0000, chris.laplante@agilent.com wrote:
> > "contains" has special magic where it caches the specific value
> > presence
> > rather than the whole string. I'd therefore recommend using the
> > contains
> > functions where at all possible for that reason.
> 
> So then regarding my other points, it is correctly handled by the
> signature code? If so, why doesn’t it show up in task dependencies?

It should be handled correctly and it does show up.

For example, 

$ bitbake-dumpsig ./tmp/stamps/core2-64-poky-linux/dbus-test/1.12.16-r0.do_configure.sig*
Variable root_prefix value is ${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', '${exec_prefix}', '${base_prefix}', d)}
DISTRO_FEATURES{usrmerge} = Unset

which means DISTRO_FEATURES didn't contain "usrmerge". It would only
change hash if DISTRO_FEATURES then did contain it, there is no
absolute value dependency.

Cheers,

Richard