[PATCH] bitbake.conf: Add sstate-build-populate_sdk and intercept_scripts- to PSEUDO_IGNORE_PATHS

[PATCH] bitbake.conf: Add sstate-build-populate_sdk and intercept_scripts- to PSEUDO_IGNORE_PATHS

[Tomasz Dziendzielski]

Recipes using populate_sdk.bbclass are failing with pseudo abort due to
path mismatch on these paths.

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 375467d155..c234c75823 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -683,7 +683,7 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
-PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,/run/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${CCACHE_DIR}"
+PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,/run/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${WORKDIR}/sstate-build-populate_sdk,${WORKDIR}/intercept_scripts-,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${CCACHE_DIR}"
 
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
-- 
2.30.1

Re: [OE-core] [PATCH] bitbake.conf: Add sstate-build-populate_sdk and intercept_scripts- to PSEUDO_IGNORE_PATHS

[Richard Purdie]

On Wed, 2021-02-17 at 23:49 +0100, Tomasz Dziendzielski wrote:
> Recipes using populate_sdk.bbclass are failing with pseudo abort due to
> path mismatch on these paths.
> 
> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> ---
>  meta/conf/bitbake.conf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

We build SDKs on the autobuilder and don't see this. Any idea what you 
need to do to trigger it?

Cheers,

Richard

Re: [OE-core] [PATCH] bitbake.conf: Add sstate-build-populate_sdk and intercept_scripts- to PSEUDO_IGNORE_PATHS

[Tomasz Dziendzielski]

[-- Attachment #1: Type: text/plain, Size: 1165 bytes --]

 >> Recipes using populate_sdk.bbclass are failing with pseudo abort due to
>> path mismatch on these paths.
>>
>> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
>> ---
>>  meta/conf/bitbake.conf | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>
>We build SDKs on the autobuilder and don't see this. Any idea what you
>need to do to trigger it?

I was able to reproduce abort on sstate-build-populate_sdk on the latest
master without any old build/ directory.
$ . ./oe-init-build-env
$ bitbake meta-go-toolchain
$ bitbake meta-go-toolchain -c populate_sdk -f

Please note that above commands might not fail every time, it seems to be a
bit random. It occurs not only on rebuilds but also on the first run. If we
build multiple SDKs at one bitbake execution the build is unlikely to
finish successfully, because each time different (or the same) SDKs will
fail.

In our environment I also had to add ${WORKDIR}/oe-sdk-repo,${SDK_OUTPUT}
to PSEUDO_IGNORE_PATHS to make SDKs work but I'm not sure if failures on
these two are not related to some of our custom mechanisms.

Best regards,
Tomasz Dziendzielski

[-- Attachment #2: Type: text/html, Size: 1530 bytes --]

Re: [OE-core] [PATCH] bitbake.conf: Add sstate-build-populate_sdk and intercept_scripts- to PSEUDO_IGNORE_PATHS

[Richard Purdie]

On Thu, 2021-02-18 at 11:30 +0100, Tomasz Dziendzielski wrote:
> > > Recipes using populate_sdk.bbclass are failing with pseudo abort due to
> > > path mismatch on these paths.
> > > 
> > > Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> > > ---
> > >  meta/conf/bitbake.conf | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > We build SDKs on the autobuilder and don't see this. Any idea what you 
> > need to do to trigger it?
> 
> I was able to reproduce abort on sstate-build-populate_sdk on the latest master without any old build/
> directory.
> $ . ./oe-init-build-env
> $ bitbake meta-go-toolchain
> $ bitbake meta-go-toolchain -c populate_sdk -f

Which packaging backend are you using? I suspect this is package 
backend specific.

> Please note that above commands might not fail every time, it seems to 
> be a bit random. It occurs not only on rebuilds but also on the first run. 
> If we build multiple SDKs at one bitbake execution the build is unlikely 
> to finish successfully, because each time different (or the same) SDKs will fail.

The "better" way to test this is to look inside WORKDIR/pseudo/files.db 
with sqlite3 and then see which paths are in the database which you'd 
not expect. That way you'll see the paths regardless of whether there
was an inode reuse issue or not.

> In our environment I also had to add ${WORKDIR}/oe-sdk-repo,${SDK_OUTPUT} 
> to PSEUDO_IGNORE_PATHS to make SDKs work but I'm not sure if failures on 
> these two are not related to some of our custom mechanisms.

In my local build I can see WORKDIR/oe-sdk-repo, WORKDIR/apt and 
WORKDIR/oe-rootfs-repo in files.db which look suspicious.

I don't see sstate-build-populate_sdk or intercept_scripts
paths though :/.

At the very least I think we should append these to PSEUDO_IGNORE_PATHS 
in the classes which define them rather than globally.

Cheers,

Richard