[poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Anton Kachalov]

[-- Attachment #1: Type: text/plain, Size: 1642 bytes --]

Allow the Yocto's building tree to be placed under /usr.

Relevant issue: https://github.com/openbmc/openbmc/issues/3724

Signed-off-by: Anton D. Kachalov <rnouse@google.com>
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index a04005b0f5..77c8148aa1 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -685,7 +685,7 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
-PSEUDO_IGNORE_PATHS =
"/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${CCACHE_DIR}"
+PSEUDO_IGNORE_PATHS =
"/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${CCACHE_DIR}"

 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
-- 
2.29.2.729.g45daf8777d-goog

[-- Attachment #2: Type: text/html, Size: 2096 bytes --]

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Richard Purdie]

On Thu, 2021-01-07 at 22:58 +0100, Anton Kachalov via lists.openembedded.org wrote:
> Allow the Yocto's building tree to be placed under /usr.
> 
> Relevant issue: https://github.com/openbmc/openbmc/issues/3724
> 
> Signed-off-by: Anton D. Kachalov <rnouse@google.com>
> ---
>  meta/conf/bitbake.conf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Do you really need to build under /usr? We probably should make this
more of a hard error if someone attempts it. I commented on the issue,
could you just remove this from PSEUDO_IGNORE_PATHS locally if you
really have to use /usr?

Cheers,

Richard

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Anton Kachalov]

[-- Attachment #1: Type: text/plain, Size: 1029 bytes --]

Hello, Richard.

Yes, for various reasons we can not move out from /usr/local.
Before the change it works smoothly and we didn't notice about any kind of
performance issues.
May be we can explicitly add various standard subdirs under /usr excluding
/usr/local?


On Thu, Jan 7, 2021, 23:16 Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2021-01-07 at 22:58 +0100, Anton Kachalov via
> lists.openembedded.org wrote:
> > Allow the Yocto's building tree to be placed under /usr.
> >
> > Relevant issue: https://github.com/openbmc/openbmc/issues/3724
> >
> > Signed-off-by: Anton D. Kachalov <rnouse@google.com>
> > ---
> >  meta/conf/bitbake.conf | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
>
> Do you really need to build under /usr? We probably should make this
> more of a hard error if someone attempts it. I commented on the issue,
> could you just remove this from PSEUDO_IGNORE_PATHS locally if you
> really have to use /usr?
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 1847 bytes --]

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Richard Purdie]

On Fri, 2021-01-08 at 01:56 +0100, Anton Kachalov wrote:
> Hello, Richard.
> Yes, for various reasons we can not move out from /usr/local.
> Before the change it works smoothly and we didn't notice about any
> kind of performance issues.
> May be we can explicitly add various standard subdirs under /usr
> excluding /usr/local?

Can you add local config like:

PSEUDO_IGONRE_PATHS_remove = "/usr/,"

which should allow things to work for you?

Cheers,

Richard

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 8 januari 2021 09:27
> To: Anton Kachalov <rnouse@google.com>
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo
> ignore paths
> 
> On Fri, 2021-01-08 at 01:56 +0100, Anton Kachalov wrote:
> > Hello, Richard.
> > Yes, for various reasons we can not move out from /usr/local.

Using a build directory anywhere under /usr sounds like a very unusual 
setup and I do not think it is a situation that warrants special 
treatment in the common configuration. Workarounds are better done in 
the local environment.

> > Before the change it works smoothly and we didn't notice about any
> > kind of performance issues.
> > May be we can explicitly add various standard subdirs under /usr
> > excluding /usr/local?

I believe it would be hard to maintain a suitable list of subdirectories 
of /usr (including /usr/local) that pseudo should ignore. However, 
do we actually know if pseudo is normally accessing paths under /usr, 
or was it just added as a "pseudo should never care about anything here 
so we may as well add it"?

> Can you add local config like:
> 
> PSEUDO_IGONRE_PATHS_remove = "/usr/,"
> 
> which should allow things to work for you?

Correct me if I am wrong, but AFAIK you cannot use the _remove operator 
to remove a part of a string. It only works on space separated words. 
You will have to redefine the entire PSEUDO_IGONRE_PATHS to remove any 
parts of it.

This is a drawback of using a list in a bitbake variable that is separated 
using something other than spaces. I guess commas were chosen as separator 
for this variable since spaces are more common (if still rare) in paths 
than commas, but it causes problems like this.

> 
> Cheers,
> 
> Richard

//Peter

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Anton Kachalov]

[-- Attachment #1: Type: text/plain, Size: 2380 bytes --]

Hello,

nope, setting the entire PSEUDO_IGNORE_PATHS without "/usr" doesn't work.
The do_install task fails with error code 134 without any log file. Neither
temp/run.do_install file exists. The pseudo.log doesn't have any errors.

On Mon, 11 Jan 2021 at 19:22, Peter Kjellerstedt <
peter.kjellerstedt@axis.com> wrote:

> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org <openembedded-
> > core@lists.openembedded.org> On Behalf Of Richard Purdie
> > Sent: den 8 januari 2021 09:27
> > To: Anton Kachalov <rnouse@google.com>
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo
> > ignore paths
> >
> > On Fri, 2021-01-08 at 01:56 +0100, Anton Kachalov wrote:
> > > Hello, Richard.
> > > Yes, for various reasons we can not move out from /usr/local.
>
> Using a build directory anywhere under /usr sounds like a very unusual
> setup and I do not think it is a situation that warrants special
> treatment in the common configuration. Workarounds are better done in
> the local environment.
>
> > > Before the change it works smoothly and we didn't notice about any
> > > kind of performance issues.
> > > May be we can explicitly add various standard subdirs under /usr
> > > excluding /usr/local?
>
> I believe it would be hard to maintain a suitable list of subdirectories
> of /usr (including /usr/local) that pseudo should ignore. However,
> do we actually know if pseudo is normally accessing paths under /usr,
> or was it just added as a "pseudo should never care about anything here
> so we may as well add it"?
>
> > Can you add local config like:
> >
> > PSEUDO_IGONRE_PATHS_remove = "/usr/,"
> >
> > which should allow things to work for you?
>
> Correct me if I am wrong, but AFAIK you cannot use the _remove operator
> to remove a part of a string. It only works on space separated words.
> You will have to redefine the entire PSEUDO_IGONRE_PATHS to remove any
> parts of it.
>
> This is a drawback of using a list in a bitbake variable that is separated
> using something other than spaces. I guess commas were chosen as separator
> for this variable since spaces are more common (if still rare) in paths
> than commas, but it causes problems like this.
>
> >
> > Cheers,
> >
> > Richard
>
> //Peter
>
>

[-- Attachment #2: Type: text/html, Size: 3232 bytes --]

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Richard Purdie]

On Fri, 2021-01-15 at 18:04 +0100, Anton Kachalov wrote:
> nope, setting the entire PSEUDO_IGNORE_PATHS without "/usr" doesn't
> work. The do_install task fails with error code 134 without any log
> file. Neither temp/run.do_install file exists. The pseudo.log doesn't
> have any errors.

Right, Peter is correct and _remove is space delimited, the variable is
comma delimited. Sorry, I'm getting confused.

There is the rather uglier:

PSEUDO_IGNORE_PATHS := "${@d.getVar('PSEUDO_IGNORE_PATHS').replace('/usr/,','')}"

which should work (I did test this time!).

Cheers,

Richard

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Richard Purdie]

On Fri, 2021-01-15 at 17:09 +0000, Richard Purdie wrote:
> On Fri, 2021-01-15 at 18:04 +0100, Anton Kachalov wrote:
> > nope, setting the entire PSEUDO_IGNORE_PATHS without "/usr" doesn't
> > work. The do_install task fails with error code 134 without any log
> > file. Neither temp/run.do_install file exists. The pseudo.log doesn't
> > have any errors.
> 
> Right, Peter is correct and _remove is space delimited, the variable is
> comma delimited. Sorry, I'm getting confused.
> 
> There is the rather uglier:
> 
> PSEUDO_IGNORE_PATHS := "${@d.getVar('PSEUDO_IGNORE_PATHS').replace('/usr/,','')}"
> 
> which should work (I did test this time!).

Thinking more, this is safer:

PSEUDO_IGNORE_PATHS := "${@d.getVar('PSEUDO_IGNORE_PATHS', False).replace('/usr/,','')}"

I did test it parses and gave the right value with bitbake -e however I
not expanding it will be much safer as I suspect the first version
won't work in recipe context at build time.

Cheers,

Richard

Re: [OE-core] [poky][PATCH] bitbake: remove /usr from pseudo ignore paths

[Anton Kachalov]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

Perhaps, I'm doing something wrong, but setting this variable in
build/conf/local.conf makes packages fail to build (do_install): only those
which fail with pseudo before (linux-libc-headers and base-files).

On Fri, 15 Jan 2021 at 18:15, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2021-01-15 at 17:09 +0000, Richard Purdie wrote:
> > On Fri, 2021-01-15 at 18:04 +0100, Anton Kachalov wrote:
> > > nope, setting the entire PSEUDO_IGNORE_PATHS without "/usr" doesn't
> > > work. The do_install task fails with error code 134 without any log
> > > file. Neither temp/run.do_install file exists. The pseudo.log doesn't
> > > have any errors.
> >
> > Right, Peter is correct and _remove is space delimited, the variable is
> > comma delimited. Sorry, I'm getting confused.
> >
> > There is the rather uglier:
> >
> > PSEUDO_IGNORE_PATHS :=
> "${@d.getVar('PSEUDO_IGNORE_PATHS').replace('/usr/,','')}"
> >
> > which should work (I did test this time!).
>
> Thinking more, this is safer:
>
> PSEUDO_IGNORE_PATHS := "${@d.getVar('PSEUDO_IGNORE_PATHS',
> False).replace('/usr/,','')}"
>
> I did test it parses and gave the right value with bitbake -e however I
> not expanding it will be much safer as I suspect the first version
> won't work in recipe context at build time.
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 1859 bytes --]