[PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Rasmus Villemoes]

Commit 992cec44 (coreutils: Move stdbuf into an own package
coreutils-stdbuf) breaks package-qa when the single-binary
PACKAGECONFIG is used:

ERROR: coreutils-8.31-r0 do_package_qa: QA Issue: /usr/bin/stdbuf contained in package coreutils-stdbuf requires /usr/bin/coreutils, but no providers found in RDEPENDS_coreutils-stdbuf? [file-rdeps]
ERROR: coreutils-8.31-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
ERROR: Logfile of failure stored in: /mnt/ext4/projects/deif/pdsystems/yocto/tmp-glibc/work/ppce300c3-oe-linux/coreutils/8.31-r0/temp/log.do_package_qa.5594
ERROR: Task (/mnt/ext4/projects/deif/pdsystems/yocto/meta-poky/meta/recipes-core/coreutils/coreutils_8.31.bb:do_package_qa) failed with exit code '1'

With that PACKAGECONFIG, /usr/bin/stdbuf is just a simple "script"
containing the single line

  #!/usr/bin/coreutils --coreutils-prog-shebang=stdbuf

Since there's no point splitting stdbuf to its own package when all
the functionality is in the single big coreutils binary anyway, fix
this by not creating the separate stdbuf package for the single-binary
case.

Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
---
The same issue exists in master, which has been bumped to 8.32, but
other than that the patch should apply cleanly there as well.

 meta/recipes-core/coreutils/coreutils_8.31.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb b/meta/recipes-core/coreutils/coreutils_8.31.bb
index 57b2c1bdba..80e8161367 100644
--- a/meta/recipes-core/coreutils/coreutils_8.31.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
@@ -58,9 +58,9 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo false hostname kill l
 
 sbindir_progs= "chroot"
 
-PACKAGE_BEFORE_PN_class-target += "coreutils-stdbuf"
+PACKAGE_BEFORE_PN_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils-stdbuf', d)}"
 FILES_coreutils-stdbuf = "${bindir}/stdbuf ${libdir}/coreutils/libstdbuf.so"
-RDEPENDS_coreutils_class-target += "coreutils-stdbuf"
+RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils-stdbuf', d)}"
 
 # Let aclocal use the relative path for the m4 file rather than the
 # absolute since coreutils has a lot of m4 files, otherwise there might
-- 
2.23.0

Re: [OE-core] [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 2717 bytes --]

Then this needs to be submitted for master first, we have a strict policy
for that.

Alex

On Fri, 3 Jul 2020 at 09:36, Rasmus Villemoes <rasmus.villemoes@prevas.dk>
wrote:

> Commit 992cec44 (coreutils: Move stdbuf into an own package
> coreutils-stdbuf) breaks package-qa when the single-binary
> PACKAGECONFIG is used:
>
> ERROR: coreutils-8.31-r0 do_package_qa: QA Issue: /usr/bin/stdbuf
> contained in package coreutils-stdbuf requires /usr/bin/coreutils, but no
> providers found in RDEPENDS_coreutils-stdbuf? [file-rdeps]
> ERROR: coreutils-8.31-r0 do_package_qa: QA run found fatal errors. Please
> consider fixing them.
> ERROR: Logfile of failure stored in:
> /mnt/ext4/projects/deif/pdsystems/yocto/tmp-glibc/work/ppce300c3-oe-linux/coreutils/8.31-r0/temp/log.do_package_qa.5594
> ERROR: Task
> (/mnt/ext4/projects/deif/pdsystems/yocto/meta-poky/meta/recipes-core/coreutils/coreutils_8.31.bb:do_package_qa)
> failed with exit code '1'
>
> With that PACKAGECONFIG, /usr/bin/stdbuf is just a simple "script"
> containing the single line
>
>   #!/usr/bin/coreutils --coreutils-prog-shebang=stdbuf
>
> Since there's no point splitting stdbuf to its own package when all
> the functionality is in the single big coreutils binary anyway, fix
> this by not creating the separate stdbuf package for the single-binary
> case.
>
> Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
> ---
> The same issue exists in master, which has been bumped to 8.32, but
> other than that the patch should apply cleanly there as well.
>
>  meta/recipes-core/coreutils/coreutils_8.31.bb | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb
> b/meta/recipes-core/coreutils/coreutils_8.31.bb
> index 57b2c1bdba..80e8161367 100644
> --- a/meta/recipes-core/coreutils/coreutils_8.31.bb
> +++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
> @@ -58,9 +58,9 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd
> echo false hostname kill l
>
>  sbindir_progs= "chroot"
>
> -PACKAGE_BEFORE_PN_class-target += "coreutils-stdbuf"
> +PACKAGE_BEFORE_PN_class-target += "${@bb.utils.contains('PACKAGECONFIG',
> 'single-binary', '', 'coreutils-stdbuf', d)}"
>  FILES_coreutils-stdbuf = "${bindir}/stdbuf
> ${libdir}/coreutils/libstdbuf.so"
> -RDEPENDS_coreutils_class-target += "coreutils-stdbuf"
> +RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG',
> 'single-binary', '', 'coreutils-stdbuf', d)}"
>
>  # Let aclocal use the relative path for the m4 file rather than the
>  # absolute since coreutils has a lot of m4 files, otherwise there might
> --
> 2.23.0
>
> 
>

[-- Attachment #2: Type: text/html, Size: 3692 bytes --]

Re: [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Richard Purdie]

On Fri, 2020-07-03 at 09:36 +0200, Rasmus Villemoes wrote:
> Commit 992cec44 (coreutils: Move stdbuf into an own package
> coreutils-stdbuf) breaks package-qa when the single-binary
> PACKAGECONFIG is used:
> 
> ERROR: coreutils-8.31-r0 do_package_qa: QA Issue: /usr/bin/stdbuf
> contained in package coreutils-stdbuf requires /usr/bin/coreutils,
> but no providers found in RDEPENDS_coreutils-stdbuf? [file-rdeps]
> ERROR: coreutils-8.31-r0 do_package_qa: QA run found fatal errors.
> Please consider fixing them.
> ERROR: Logfile of failure stored in:
> /mnt/ext4/projects/deif/pdsystems/yocto/tmp-glibc/work/ppce300c3-oe-
> linux/coreutils/8.31-r0/temp/log.do_package_qa.5594
> ERROR: Task (/mnt/ext4/projects/deif/pdsystems/yocto/meta-
> poky/meta/recipes-core/coreutils/coreutils_8.31.bb:do_package_qa)
> failed with exit code '1'
> 
> With that PACKAGECONFIG, /usr/bin/stdbuf is just a simple "script"
> containing the single line
> 
>   #!/usr/bin/coreutils --coreutils-prog-shebang=stdbuf
> 
> Since there's no point splitting stdbuf to its own package when all
> the functionality is in the single big coreutils binary anyway, fix
> this by not creating the separate stdbuf package for the single-
> binary
> case.
> 
> Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
> ---
> The same issue exists in master, which has been bumped to 8.32, but
> other than that the patch should apply cleanly there as well.

Does this problem exist for other packages in coreutils? I'd suspect it
does in which case why is stdbuf special? Whilst I realise there is a
problem here is the correct fix not:

RDEPENDS_coreutils-stdbuf_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils', d)}"

?

As Alex says, this would need to be merged in master before we can even
consider it for dunfell.

Cheers,

Richard

Re: [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Rasmus Villemoes]

On 03/07/2020 11.19, Richard Purdie wrote:
> On Fri, 2020-07-03 at 09:36 +0200, Rasmus Villemoes wrote:
>> Commit 992cec44 (coreutils: Move stdbuf into an own package
>> coreutils-stdbuf) breaks package-qa when the single-binary
>> PACKAGECONFIG is used:
>>
>> ERROR: coreutils-8.31-r0 do_package_qa: QA Issue: /usr/bin/stdbuf
>> contained in package coreutils-stdbuf requires /usr/bin/coreutils,
>> but no providers found in RDEPENDS_coreutils-stdbuf? [file-rdeps]
>> ERROR: coreutils-8.31-r0 do_package_qa: QA run found fatal errors.
>> Please consider fixing them.
> 
> Does this problem exist for other packages in coreutils?

What other packages? stdbuf is the only one being split to its own package:

coreutils/8.31-r0$ ls -l packages-split/
total 32
drwxr-xr-x 4 ravi abcdef 4096 Jul  3 09:49 coreutils
drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:50 coreutils-dbg
drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-dev
drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:49 coreutils-doc
drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-locale
-rw-r--r-- 1 ravi abcdef   48 Jul  3 09:49 coreutils.shlibdeps
drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:50 coreutils-src
drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-staticdev


> I'd suspect it
> does in which case why is stdbuf special?

Because it's the only util that gets special packaging treatment. Unless
there's some magic 'please auto-split all utils to their own packages'
that I don't know about and which we don't happen to set.

> Whilst I realise there is a problem here is the correct fix not:
> 
> RDEPENDS_coreutils-stdbuf_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils', d)}"
> 
> ?

[Well, the coreutils should be in the true branch of that.] I dunno, it
creates a cyclic rdepends between coreutils and coreutils-stdbuf. Is
that ok? Seems a bit ugly to me, even if it would work.

> As Alex says, this would need to be merged in master before we can even
> consider it for dunfell.

I'll do it for master once I know which way you want to fix it.

Rasmus

Re: [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Richard Purdie]

On Fri, 2020-07-03 at 11:29 +0200, Rasmus Villemoes wrote:
> On 03/07/2020 11.19, Richard Purdie wrote:
> > On Fri, 2020-07-03 at 09:36 +0200, Rasmus Villemoes wrote:
> > > Commit 992cec44 (coreutils: Move stdbuf into an own package
> > > coreutils-stdbuf) breaks package-qa when the single-binary
> > > PACKAGECONFIG is used:
> > > 
> > > ERROR: coreutils-8.31-r0 do_package_qa: QA Issue: /usr/bin/stdbuf
> > > contained in package coreutils-stdbuf requires /usr/bin/coreutils,
> > > but no providers found in RDEPENDS_coreutils-stdbuf? [file-rdeps]
> > > ERROR: coreutils-8.31-r0 do_package_qa: QA run found fatal errors.
> > > Please consider fixing them.
> > 
> > Does this problem exist for other packages in coreutils?
> 
> What other packages? stdbuf is the only one being split to its own package:
> 
> coreutils/8.31-r0$ ls -l packages-split/
> total 32
> drwxr-xr-x 4 ravi abcdef 4096 Jul  3 09:49 coreutils
> drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:50 coreutils-dbg
> drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-dev
> drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:49 coreutils-doc
> drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-locale
> -rw-r--r-- 1 ravi abcdef   48 Jul  3 09:49 coreutils.shlibdeps
> drwxr-xr-x 3 ravi abcdef 4096 Jul  3 09:50 coreutils-src
> drwxr-xr-x 2 ravi abcdef 4096 Jul  3 09:49 coreutils-staticdev
> 
> 
> > I'd suspect it
> > does in which case why is stdbuf special?
> 
> Because it's the only util that gets special packaging treatment. Unless
> there's some magic 'please auto-split all utils to their own packages'
> that I don't know about and which we don't happen to set.

Sorry, I think I was confusing this with util-linux for some reason!
Just having one package here definitely helps.

> > Whilst I realise there is a problem here is the correct fix not:
> > 
> > RDEPENDS_coreutils-stdbuf_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils', d)}"
> > 
> > ?
> 
> [Well, the coreutils should be in the true branch of that.] I dunno, it
> creates a cyclic rdepends between coreutils and coreutils-stdbuf. Is
> that ok? Seems a bit ugly to me, even if it would work.

I hadn't realised there was already the reverse dependency. I think we
need to let the package exist in both cases so the two dependencies
need to reverse direction depending on whether single-binary is set.

So something like:

RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils-stdbuf', d)}"
RDEPENDS_coreutils-stdbuf_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', 'coreutils', '', d)}"

?

Cheers,

Richard

Re: [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Rasmus Villemoes]

On 03/07/2020 12.48, Richard Purdie wrote:

>>> Whilst I realise there is a problem here is the correct fix not:
>>>
>>> RDEPENDS_coreutils-stdbuf_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils', d)}"
>>>
>>> ?
>>
>> [Well, the coreutils should be in the true branch of that.] I dunno, it
>> creates a cyclic rdepends between coreutils and coreutils-stdbuf. Is
>> that ok? Seems a bit ugly to me, even if it would work.
> 
> I hadn't realised there was already the reverse dependency. I think we
> need to let the package exist in both cases so the two dependencies
> need to reverse direction depending on whether single-binary is set.
> 
> So something like:
> 
> RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils-stdbuf', d)}"
> RDEPENDS_coreutils-stdbuf_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', 'coreutils', '', d)}"
> 
> ?

The argument for creating the coreutils->coreutils-stdbuf RDEPENDS was
that people would get the same stuff in their rootfs as they used to
(but they could also just get stdbuf if they needed that but use busybox
for all the rest).

The above would mean people who set single-binary and include coreutils
in their rootfs no longer get the stdbuf utility (the functionality is
in the binary, but they no longer get the /usr/bin/stdbuf wrapper
script). But I suppose few enough people use single-binary (or the qa
issue would surely have surfaced sometime in the past half year) that we
can live with having to explicitly add coreutils-stdbuf. But then
perhaps we can just unconditionally drop the coreutils->coreutils-stdbuf
dependency, and have anybody who need stdbuf include it explicitly?

In general, I do support splitting things to their own packages without
automatically pulling them back in by the main package; it can be a
little annoying for downstreams, but surely everybody have tests that
ensure the utilities they need are in their images. [I'm not being
sarcastic.]

Rasmus

Re: [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Adrian Bunk]

On Fri, Jul 03, 2020 at 11:48:36AM +0100, Richard Purdie wrote:
>...
> I hadn't realised there was already the reverse dependency. I think we
> need to let the package exist in both cases so the two dependencies
> need to reverse direction depending on whether single-binary is set.
>...

What about no separate coreutils-stdbuf package with single-binary,
instead an RPROVIDES of coreutils-stdbuf?

> Cheers,
> 
> Richard

cu
Adrian

Re: [PATCH][dunfell] coreutils: don't split stdbuf to own package with single-binary

[Rasmus Villemoes]

On 04/07/2020 23.10, Adrian Bunk wrote:
> On Fri, Jul 03, 2020 at 11:48:36AM +0100, Richard Purdie wrote:
>> ...
>> I hadn't realised there was already the reverse dependency. I think we
>> need to let the package exist in both cases so the two dependencies
>> need to reverse direction depending on whether single-binary is set.
>> ...
> 
> What about no separate coreutils-stdbuf package with single-binary,
> instead an RPROVIDES of coreutils-stdbuf?

That would probably work. That's essentially the first patch, plus a

RPROVIDES_coreutils += "${@bb.utils.contains('PACKAGECONFIG',
'single-binary', 'coreutils-stdbuf', '', d)}"

line, right?

Rasmus

[PATCH] [dunfell] coreutils: don't split stdbuf to own package with single-binary

[Rasmus Villemoes]

Commit 992cec44 (coreutils: Move stdbuf into an own package
coreutils-stdbuf) breaks package-qa when the single-binary
PACKAGECONFIG is used:

ERROR: coreutils-8.32-r0 do_package_qa: QA Issue: /usr/bin/stdbuf contained in package coreutils-stdbuf requires /usr/bin/coreutils, but no providers found in RDEPENDS_coreutils-stdbuf? [file-rdeps]
ERROR: coreutils-8.32-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.

With that PACKAGECONFIG, /usr/bin/stdbuf is just a simple "script"
containing the single line

  #!/usr/bin/coreutils --coreutils-prog-shebang=stdbuf

Since there's no point splitting stdbuf to its own package when all
the functionality is in the single big coreutils binary anyway, fix
this by not creating the separate stdbuf package for the single-binary
case. But also make sure that the coreutils-stdbuf item always exists
so recipes can always RDEPEND on coreutils-stdbuf.

Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 74d24b5b895198898944260136d05e991a203c11)
---
 meta/recipes-core/coreutils/coreutils_8.31.bb | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb b/meta/recipes-core/coreutils/coreutils_8.31.bb
index 2b196b7512..0c8452da98 100644
--- a/meta/recipes-core/coreutils/coreutils_8.31.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
@@ -59,9 +59,20 @@ base_bindir_progs = "cat chgrp chmod chown cp date dd echo false hostname kill l
 
 sbindir_progs= "chroot"
 
-PACKAGE_BEFORE_PN_class-target += "coreutils-stdbuf"
+# Split stdbuf into its own package, so one can include
+# coreutils-stdbuf without getting the rest of coreutils, but make
+# coreutils itself pull in stdbuf, so IMAGE_INSTALL += "coreutils"
+# always provides all coreutils
+PACKAGE_BEFORE_PN_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils-stdbuf', d)}"
 FILES_coreutils-stdbuf = "${bindir}/stdbuf ${libdir}/coreutils/libstdbuf.so"
-RDEPENDS_coreutils_class-target += "coreutils-stdbuf"
+RDEPENDS_coreutils_class-target += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', '', 'coreutils-stdbuf', d)}"
+
+# However, when the single-binary PACKAGECONFIG is used, stdbuf
+# functionality is built into the single coreutils binary, so there's
+# no point splitting /usr/bin/stdbuf to its own package. Instead, add
+# an RPROVIDE so that rdepending on coreutils-stdbuf will work
+# regardless of whether single-binary is in effect.
+RPROVIDES_coreutils += "${@bb.utils.contains('PACKAGECONFIG', 'single-binary', 'coreutils-stdbuf', '', d)}"
 
 # Let aclocal use the relative path for the m4 file rather than the
 # absolute since coreutils has a lot of m4 files, otherwise there might
-- 
2.23.0