Re: [OE-Core][PATCH v4 00/11] Add rust runtime tests

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Wed, 2022-12-28 at 14:51 +0000, Alex Kiernan wrote:
> On Wed, Dec 28, 2022 at 12:01 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > 
> > On Tue, 2022-12-27 at 18:09 +0000, Alex Kiernan wrote:
> > > On Mon, Dec 26, 2022 at 6:33 PM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > > 
> > > > On Sun, 2022-12-25 at 21:59 +0000, Alex Kiernan wrote:
> > > > > This series adds runtime tests for rust and cargo, merges (most of) the
> > > > > include files into the base recipes and fixes target cargo builds.
> > > > > 
> > > > > Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part
> > > > > of packagegroup-core-sdk.
> > > > > 
> > > > > Changes in v4:
> > > > > - Fixup `__stack_chk_fail_local` in musl-x86
> > > > > - Add crossbeam atomics fixes for ppc/mips
> > > > > 
> > > > > Changes in v3:
> > > > > - Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS
> > > > >   (which defaults to SDK_TOOLCHAIN_LANGS)
> > > > > 
> > > > > Changes in v2:
> > > > > - Drop rust.inc/rust.bb merge, not actually made use of in the change
> > > > >   set
> > > > > - Reorder so cargo test goes after build fixes
> > > > > 
> > > > > Alex Kiernan (11):
> > > > >   rust-llvm: Merge .inc into .bb
> > > > >   rust-llvm: Update LLVM_VERSION to match embedded version
> > > > >   packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
> > > > >   packagegroup-core-sdk: Add SDK toolchain language selection support
> > > > >   cargo: Merge .inc into .bb
> > > > >   cargo: Extend DEBUG_PREFIX_MAP to cover vendor
> > > > >   rust: Merge .inc into .bb
> > > > >   rust: Move musl-x86 fix for `__stack_chk_fail_local` to rust-source
> > > > >   cargo: Override crossbeam-utils with 0.8.14
> > > > >   packagegroup-rust-sdk-target: Add cargo
> > > > >   oeqa/runtime/rust: Add cargo test
> > > > 
> > > > Looks much better thanks. There was one remaining issue which showed up
> > > > in testing with a reproducibility issue:
> > > > 
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio
> > > > 
> > > > We did already have an open bug for a rust reproducibility issue and we
> > > > excluded it from the tests until we could get to the bottom of it, I
> > > > suspect cargo may have the same issue. It frustrates me a bit as it is
> > > > the one reproducibility exclusion for core.
> > > > 
> > > > Ideally we would fix it but excluding it from the tests and noting it
> > > > in the bug may be an option if it is the same issue. It may be simpler
> > > > to reproduce and fix the issue with cargo too.
> > > > 
> > > > Cc: Sundeep who as been looking at it.
> > > > 
> > > 
> > > Just reading the ticket and the upstream ones... the upstream ticket
> > > is closed as remap path fixes it (which it looks like it does to me),
> > > there are build paths embedded in the build directory which don't get
> > > caught by that, but they're in thing like bootstrap which isn't
> > > packaged (which may or may not be an issue, but I'm suspecting not);
> > > it kinda feels like we're missing something.
> > > 
> > > Can I get hold of the cargo packages generated here to have a poke at them?
> > 
> > Yes. The key line in the log is:
> > 
> > 022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to /srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk
> > 
> > i.e.
> > 
> > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk
> > 
> > where you'll find both the package files and diffoscope output:
> > 
> > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/
> > 
> > which seems to be saying the symbol table changed size which is
> > unusual. I didn't look very deeply.
> > 
> 
> Upstream has this ticket which, given how I did the patching for
> cargo, feels like the right kind of thing (rust has something similar
> out of the box):
> 
> https://github.com/rust-lang/rust/issues/98185
> 
> I've switched it around so that it uses local patching rather than
> cargo patching, which if it is this, seems like it might resolve it.

It was a different issue, thanks for tracking that down and fixing it!

The tests all passed so I merged the v5 series. Thanks for working
through things, I think this makes our rust situation much more robust
and should help future proof it :)

Cheers,

Richard