Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs

["Richard Purdie" <richard.purdie@linuxfoundation.org>]

On Tue, 2021-01-12 at 09:21 +0000, Richard Purdie via
lists.openembedded.org wrote:
> On Mon, 2021-01-11 at 13:45 +0100, Michael Ho wrote:
> > From: Michael Ho <Michael.Ho@bmw.de>
> > 
> > Switch do_populate_sdk for the ipk package manager to use a separate target
> > opkg config file and separate the lockfiles restricting do_rootfs and
> > do_populate_sdk from running in parallel.
> > 
> > This way if an image recipe includes a dependency to do_populate_sdk by
> > default then it will run in parallel to do_rootfs saving time compared to the
> > sequential execution.
> > 
> > Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
> > ---
> >  meta/classes/package_ipk.bbclass       | 1 +
> >  meta/classes/rootfs_ipk.bbclass        | 4 ++--
> >  meta/lib/oe/package_manager/ipk/sdk.py | 6 ++++++
> >  3 files changed, 9 insertions(+), 2 deletions(-)
> 
> I have to admit I'm very nervous about this change. The races we've
> seen betweem rootfs and sdk can be quite unusual.
> 
> I did put this in for testing and we saw:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/2938
> 
> which in the pseudo.log shows:
> 
> path mismatch [2 links]: ino 372706913 db '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/tmp-wic' req '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/sdk/image/opt/poky/3.2+snapshot/sysroots/cortexa8hf-neon-poky-linux-gnueabi/usr/src/debug/puzzles'.
> path mismatch [2 links]: ino 372706913 db '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/tmp-wic' req '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/sdk/image/opt/poky/3.2+snapshot/sysroots/cortexa8hf-neon-poky-linux-gnueabi/usr/src/debug/puzzles'.
> 
> its hard to know if this is due to this patch or possibly Paul's wic
> changes as both were in this test series. I suspect it won't reproduce
> every time since its a race.

Just to confirm, this is Paul's patch as it was seen with rpm packaging
and therefore couldn't be the ipk change.

I am worried we don't test ipk extensively enough to spot the races in
the ipk code though, I know I've had to fix quite a few over the years
:(. 

Cheers,

Richard