All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/2 v5] coreutils: add ptest support
@ 2020-03-11 11:32 Trevor Gamblin
  2020-03-11 11:32 ` [PATCH 1/2 v5] coreutils: add ptest Trevor Gamblin
  2020-03-11 11:32 ` [PATCH 2/2 v5] ptest-packagelists.inc: add coreutils to SLOW Trevor Gamblin
  0 siblings, 2 replies; 8+ messages in thread
From: Trevor Gamblin @ 2020-03-11 11:32 UTC (permalink / raw)
  To: openembedded-core

Note that this patch set does not include Richard's "coreutils hack" 
patch, so that will have to be added on top (as discussed in #yocto).

Sample test results:

core-image-minimal:
MACHINE    | PASS | FAIL | SKIP | TOTAL | TIME (m) |
qemux86-64 |  470 |    0 |  144 |   614 |      2.5 |
qemuarm64  |  470 |    0 |  144 |   614 |       51 |

core-image-sato:
MACHINE    | PASS | FAIL | SKIP | TOTAL | Time (m) |
qemux86-64 |  472 |    0 |  142 |   614 |      2.4 |
qemuarm64  |  472 |    0 |  142 |   614 |       52 |

Trevor Gamblin (2):
  coreutils: add ptest
  ptest-packagelists.inc: add coreutils to SLOW

 .../distro/include/ptest-packagelists.inc     |  1 +
 .../coreutils/coreutils/run-ptest             | 17 +++++++
 meta/recipes-core/coreutils/coreutils_8.31.bb | 44 +++++++++++++++++++
 3 files changed, 62 insertions(+)
 create mode 100755 meta/recipes-core/coreutils/coreutils/run-ptest

-- 
2.17.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH 1/2 v5] coreutils: add ptest
  2020-03-11 11:32 [PATCH 0/2 v5] coreutils: add ptest support Trevor Gamblin
@ 2020-03-11 11:32 ` Trevor Gamblin
  2020-04-06  0:26   ` [OE-core] " Khem Raj
  2020-03-11 11:32 ` [PATCH 2/2 v5] ptest-packagelists.inc: add coreutils to SLOW Trevor Gamblin
  1 sibling, 1 reply; 8+ messages in thread
From: Trevor Gamblin @ 2020-03-11 11:32 UTC (permalink / raw)
  To: openembedded-core

coreutils has a large number of tests, including some added by the
Makefile flags RUN_EXPENSIVE_TESTS and RUN_VERY_EXPENSIVE_TESTS that
significantly increase runtime (and that have been disabled). Note
that the coreutils ptest directory is given blanket permissions at
runtime with chmod -R 777 to ensure that the user created for the
tests will be able to run the test scripts and create the necessary
files in the process.

There is still room to improve the results of this ptest without
the aforementioned additions. Of the tests marked SKIP, there are
30 tests that are currently counted as SKIP because they require
sudo permissions, and another 21 that require membership in
multiple user groups. It is important to know that coreutils has
tests for both root and non-root users. Testing showed that 42
tests are skipped when running as root versus 30 when running as a
non-root user, so the decision was made to run the suite as the
latter. Additionally, gdb, valgrind, and strace could be included
in the RDEPENDS list to increase pass rate, but their total
contribution is 13 tests, so they were omitted to reduce image size.

Finally, note that at least one ptest (misc/head-write-error.sh) is
prone to ERROR on builds of core-image-minimal if extra space is
not provided with IMAGE_ROOTFS_EXTRA_SPACE.

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
---
 .../coreutils/coreutils/run-ptest             | 17 ++++++++
 meta/recipes-core/coreutils/coreutils_8.31.bb | 40 +++++++++++++++++++
 2 files changed, 57 insertions(+)
 create mode 100755 meta/recipes-core/coreutils/coreutils/run-ptest

diff --git a/meta/recipes-core/coreutils/coreutils/run-ptest b/meta/recipes-core/coreutils/coreutils/run-ptest
new file mode 100755
index 0000000000..6d4a7b365d
--- /dev/null
+++ b/meta/recipes-core/coreutils/coreutils/run-ptest
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+# remove any stale lock files so that the calls to groupadd/useradd don't stop
+# the ptest if re-using the same image
+rm -rf /etc/passwd.lock /etc/group.lock /etc/gshadow.lock
+
+COREUTILSLIB=@libdir@/coreutils
+LOG="${COREUTILSLIB}/ptest/coreutils_ptest_$(date +%Y%m%d-%H%M%S).log"
+USERNAME="tester"
+groupadd ugroup1
+groupadd ugroup2
+useradd -G ugroup1,ugroup2 $USERNAME || echo "user $USERNAME already exists"
+
+su tester -c "cd ${COREUTILSLIB}/ptest && make check-TESTS top_srcdir=. srcdir=." 2>&1 | tee -a ${LOG}
+userdel $USERNAME 
+groupdel ugroup1
+groupdel ugroup2
diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb b/meta/recipes-core/coreutils/coreutils_8.31.bb
index 57b2c1bdba..cba0bfe15c 100644
--- a/meta/recipes-core/coreutils/coreutils_8.31.bb
+++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
@@ -18,6 +18,7 @@ SRC_URI = "${GNU_MIRROR}/coreutils/${BP}.tar.xz \
            file://0001-uname-report-processor-and-hardware-correctly.patch \
            file://disable-ls-output-quoting.patch \
            file://0001-local.mk-fix-cross-compiling-problem.patch \
+           file://run-ptest \
           "
 
 SRC_URI_append_libc-musl = "file://strtod_fix_clash_with_strtold.patch"
@@ -143,3 +144,42 @@ python __anonymous() {
 }
 
 BBCLASSEXTEND = "native nativesdk"
+
+inherit ptest
+
+RDEPENDS_${PN}-ptest += "bash findutils gawk liberror-perl libmodule-build-perl make perl perl-module-file-stat python3-core sed shadow"
+
+do_install_ptest () {
+    install -d ${D}${PTEST_PATH}/tests
+    cp -r ${S}/tests/* ${D}${PTEST_PATH}/tests
+    sed -i 's/ginstall/install/g'  `grep -R ginstall ${D}${PTEST_PATH}/tests | awk -F: '{print $1}' | uniq`
+    install -d ${D}${PTEST_PATH}/build-aux
+    install ${S}/build-aux/test-driver ${D}${PTEST_PATH}/build-aux/
+    cp ${B}/Makefile ${D}${PTEST_PATH}/
+    cp ${S}/init.cfg ${D}${PTEST_PATH}/
+    cp -r ${B}/src ${D}${PTEST_PATH}/
+    cp -r ${S}/src/*.c ${D}${PTEST_PATH}/src
+    sed -i '/^VPATH/s/= .*$/= ./g' ${D}${PTEST_PATH}/Makefile
+    sed -i '/^PROGRAMS/s/^/#/g' ${D}${PTEST_PATH}/Makefile
+    sed -i '/^Makefile: /s/^.*$/Makefile:/g' ${D}${PTEST_PATH}/Makefile
+    sed -i '/^abs_srcdir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
+    sed -i '/^abs_top_builddir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
+    sed -i '/^abs_top_srcdir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
+    sed -i '/^built_programs/s/ginstall/install/g' ${D}${PTEST_PATH}/Makefile
+    chmod -R 777 ${D}${PTEST_PATH}
+
+    # Disable subcase stty-pairs.sh, it will cause test framework hang
+    sed -i '/stty-pairs.sh/d' ${D}${PTEST_PATH}/Makefile
+
+    # Disable subcase tail-2/assert.sh as it has issues on 32-bit systems
+    sed -i '/assert.sh/d' ${D}${PTEST_PATH}/Makefile
+
+    # Tweak test d_type-check to use python3 instead of python
+    sed -i "1s@.*@#!/usr/bin/python3@" ${D}${PTEST_PATH}/tests/d_type-check
+    install ${B}/src/getlimits ${D}/${bindir}
+    
+    # handle multilib
+    sed -i s:@libdir@:${libdir}:g ${D}${PTEST_PATH}/run-ptest
+}
+
+FILES_${PN}-ptest += "${bindir}/getlimits"
-- 
2.17.1



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 2/2 v5] ptest-packagelists.inc: add coreutils to SLOW
  2020-03-11 11:32 [PATCH 0/2 v5] coreutils: add ptest support Trevor Gamblin
  2020-03-11 11:32 ` [PATCH 1/2 v5] coreutils: add ptest Trevor Gamblin
@ 2020-03-11 11:32 ` Trevor Gamblin
  1 sibling, 0 replies; 8+ messages in thread
From: Trevor Gamblin @ 2020-03-11 11:32 UTC (permalink / raw)
  To: openembedded-core

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
---
 meta/conf/distro/include/ptest-packagelists.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
index d6f3aafc7f..c13ff724b1 100644
--- a/meta/conf/distro/include/ptest-packagelists.inc
+++ b/meta/conf/distro/include/ptest-packagelists.inc
@@ -66,6 +66,7 @@ PTESTS_SLOW = "\
     babeltrace-ptest \
     babeltrace2-ptest \
     busybox-ptest \
+    coreutils-ptest \
     dbus-test-ptest \
     e2fsprogs-ptest \
     glib-2.0-ptest \
-- 
2.17.1



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [OE-core] [PATCH 1/2 v5] coreutils: add ptest
  2020-03-11 11:32 ` [PATCH 1/2 v5] coreutils: add ptest Trevor Gamblin
@ 2020-04-06  0:26   ` Khem Raj
  2020-04-06 11:58     ` Richard Purdie
  0 siblings, 1 reply; 8+ messages in thread
From: Khem Raj @ 2020-04-06  0:26 UTC (permalink / raw)
  To: Trevor Gamblin; +Cc: Patches and discussions about the oe-core layer

On Wed, Mar 11, 2020 at 4:33 AM Trevor Gamblin
<trevor.gamblin@windriver.com> wrote:
>
> coreutils has a large number of tests, including some added by the
> Makefile flags RUN_EXPENSIVE_TESTS and RUN_VERY_EXPENSIVE_TESTS that
> significantly increase runtime (and that have been disabled). Note
> that the coreutils ptest directory is given blanket permissions at
> runtime with chmod -R 777 to ensure that the user created for the
> tests will be able to run the test scripts and create the necessary
> files in the process.
>
> There is still room to improve the results of this ptest without
> the aforementioned additions. Of the tests marked SKIP, there are
> 30 tests that are currently counted as SKIP because they require
> sudo permissions, and another 21 that require membership in
> multiple user groups. It is important to know that coreutils has
> tests for both root and non-root users. Testing showed that 42
> tests are skipped when running as root versus 30 when running as a
> non-root user, so the decision was made to run the suite as the
> latter. Additionally, gdb, valgrind, and strace could be included
> in the RDEPENDS list to increase pass rate, but their total
> contribution is 13 tests, so they were omitted to reduce image size.
>
> Finally, note that at least one ptest (misc/head-write-error.sh) is
> prone to ERROR on builds of core-image-minimal if extra space is
> not provided with IMAGE_ROOTFS_EXTRA_SPACE.
>
> Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
> ---
>  .../coreutils/coreutils/run-ptest             | 17 ++++++++
>  meta/recipes-core/coreutils/coreutils_8.31.bb | 40 +++++++++++++++++++
>  2 files changed, 57 insertions(+)
>  create mode 100755 meta/recipes-core/coreutils/coreutils/run-ptest
>
> diff --git a/meta/recipes-core/coreutils/coreutils/run-ptest b/meta/recipes-core/coreutils/coreutils/run-ptest
> new file mode 100755
> index 0000000000..6d4a7b365d
> --- /dev/null
> +++ b/meta/recipes-core/coreutils/coreutils/run-ptest
> @@ -0,0 +1,17 @@
> +#!/bin/sh
> +
> +# remove any stale lock files so that the calls to groupadd/useradd don't stop
> +# the ptest if re-using the same image
> +rm -rf /etc/passwd.lock /etc/group.lock /etc/gshadow.lock
> +
> +COREUTILSLIB=@libdir@/coreutils
> +LOG="${COREUTILSLIB}/ptest/coreutils_ptest_$(date +%Y%m%d-%H%M%S).log"
> +USERNAME="tester"
> +groupadd ugroup1
> +groupadd ugroup2
> +useradd -G ugroup1,ugroup2 $USERNAME || echo "user $USERNAME already exists"
> +
> +su tester -c "cd ${COREUTILSLIB}/ptest && make check-TESTS top_srcdir=. srcdir=." 2>&1 | tee -a ${LOG}
> +userdel $USERNAME
> +groupdel ugroup1
> +groupdel ugroup2
> diff --git a/meta/recipes-core/coreutils/coreutils_8.31.bb b/meta/recipes-core/coreutils/coreutils_8.31.bb
> index 57b2c1bdba..cba0bfe15c 100644
> --- a/meta/recipes-core/coreutils/coreutils_8.31.bb
> +++ b/meta/recipes-core/coreutils/coreutils_8.31.bb
> @@ -18,6 +18,7 @@ SRC_URI = "${GNU_MIRROR}/coreutils/${BP}.tar.xz \
>             file://0001-uname-report-processor-and-hardware-correctly.patch \
>             file://disable-ls-output-quoting.patch \
>             file://0001-local.mk-fix-cross-compiling-problem.patch \
> +           file://run-ptest \
>            "
>
>  SRC_URI_append_libc-musl = "file://strtod_fix_clash_with_strtold.patch"
> @@ -143,3 +144,42 @@ python __anonymous() {
>  }
>
>  BBCLASSEXTEND = "native nativesdk"
> +
> +inherit ptest
> +
> +RDEPENDS_${PN}-ptest += "bash findutils gawk liberror-perl libmodule-build-perl make perl perl-module-file-stat python3-core sed shadow"
> +

Adding libmodule-build-perl to rdeps here is pulling in whole target
toolchain components now while building core-image-minimal
the dependency chain looks like, total task building for this image are 3000+

"coreutils.do_package_qa" -> "libmodule-build-perl.do_packagedata" ->
"packagegroup-core-buildessential.do_packagedata" ->
"gcc.do_packagedata"

IMO, it's a bad experience for the user when building a simple minimal
image and it ends up building whole toolchain for target that
eventually won't even
be part of the image. perhaps make it dependent on ptest or some such
knob thats not commonly used.

Please revert this patch series until we can find a way to not insert
these build-time dependencies in normal flow.

> +do_install_ptest () {
> +    install -d ${D}${PTEST_PATH}/tests
> +    cp -r ${S}/tests/* ${D}${PTEST_PATH}/tests
> +    sed -i 's/ginstall/install/g'  `grep -R ginstall ${D}${PTEST_PATH}/tests | awk -F: '{print $1}' | uniq`
> +    install -d ${D}${PTEST_PATH}/build-aux
> +    install ${S}/build-aux/test-driver ${D}${PTEST_PATH}/build-aux/
> +    cp ${B}/Makefile ${D}${PTEST_PATH}/
> +    cp ${S}/init.cfg ${D}${PTEST_PATH}/
> +    cp -r ${B}/src ${D}${PTEST_PATH}/
> +    cp -r ${S}/src/*.c ${D}${PTEST_PATH}/src
> +    sed -i '/^VPATH/s/= .*$/= ./g' ${D}${PTEST_PATH}/Makefile
> +    sed -i '/^PROGRAMS/s/^/#/g' ${D}${PTEST_PATH}/Makefile
> +    sed -i '/^Makefile: /s/^.*$/Makefile:/g' ${D}${PTEST_PATH}/Makefile
> +    sed -i '/^abs_srcdir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
> +    sed -i '/^abs_top_builddir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
> +    sed -i '/^abs_top_srcdir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
> +    sed -i '/^built_programs/s/ginstall/install/g' ${D}${PTEST_PATH}/Makefile
> +    chmod -R 777 ${D}${PTEST_PATH}
> +
> +    # Disable subcase stty-pairs.sh, it will cause test framework hang
> +    sed -i '/stty-pairs.sh/d' ${D}${PTEST_PATH}/Makefile
> +
> +    # Disable subcase tail-2/assert.sh as it has issues on 32-bit systems
> +    sed -i '/assert.sh/d' ${D}${PTEST_PATH}/Makefile
> +
> +    # Tweak test d_type-check to use python3 instead of python
> +    sed -i "1s@.*@#!/usr/bin/python3@" ${D}${PTEST_PATH}/tests/d_type-check
> +    install ${B}/src/getlimits ${D}/${bindir}
> +
> +    # handle multilib
> +    sed -i s:@libdir@:${libdir}:g ${D}${PTEST_PATH}/run-ptest
> +}
> +
> +FILES_${PN}-ptest += "${bindir}/getlimits"
> --
> 2.17.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] [PATCH 1/2 v5] coreutils: add ptest
  2020-04-06  0:26   ` [OE-core] " Khem Raj
@ 2020-04-06 11:58     ` Richard Purdie
  2020-04-06 14:32       ` Khem Raj
  0 siblings, 1 reply; 8+ messages in thread
From: Richard Purdie @ 2020-04-06 11:58 UTC (permalink / raw)
  To: Khem Raj, Trevor Gamblin; +Cc: Patches and discussions about the oe-core layer

On Sun, 2020-04-05 at 17:26 -0700, Khem Raj wrote:
> On Wed, Mar 11, 2020 at 4:33 AM Trevor Gamblin
> <trevor.gamblin@windriver.com> wrote:
> > +inherit ptest
> > +
> > +RDEPENDS_${PN}-ptest += "bash findutils gawk liberror-perl libmodule-build-perl make perl perl-module-file-stat python3-core sed shadow"
> > +
> 
> Adding libmodule-build-perl to rdeps here is pulling in whole target
> toolchain components now while building core-image-minimal
> the dependency chain looks like, total task building for this image are 3000+
> 
> "coreutils.do_package_qa" -> "libmodule-build-perl.do_packagedata" ->
> "packagegroup-core-buildessential.do_packagedata" ->
> "gcc.do_packagedata"
> 
> IMO, it's a bad experience for the user when building a simple minimal
> image and it ends up building whole toolchain for target that
> eventually won't even
> be part of the image. perhaps make it dependent on ptest or some such
> knob thats not commonly used.
> 
> Please revert this patch series until we can find a way to not insert
> these build-time dependencies in normal flow.

We were aware of this when it merged. This should already only happen
if ptest is enabled?

(else ${PN}-ptest isn't in PACKAGES and hence the RDEPENDS isn't seen).

I wish we had a better way of handling these dependencies but we sadly
don't.

Cheers,

Richard




^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] [PATCH 1/2 v5] coreutils: add ptest
  2020-04-06 11:58     ` Richard Purdie
@ 2020-04-06 14:32       ` Khem Raj
  2020-04-06 16:47         ` Richard Purdie
  0 siblings, 1 reply; 8+ messages in thread
From: Khem Raj @ 2020-04-06 14:32 UTC (permalink / raw)
  To: Richard Purdie
  Cc: Patches and discussions about the oe-core layer, Trevor Gamblin

[-- Attachment #1: Type: text/plain, Size: 1816 bytes --]

On Mon, Apr 6, 2020 at 4:58 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Sun, 2020-04-05 at 17:26 -0700, Khem Raj wrote:
> > On Wed, Mar 11, 2020 at 4:33 AM Trevor Gamblin
> > <trevor.gamblin@windriver.com> wrote:
> > > +inherit ptest
> > > +
> > > +RDEPENDS_${PN}-ptest += "bash findutils gawk liberror-perl
> libmodule-build-perl make perl perl-module-file-stat python3-core sed
> shadow"
> > > +
> >
> > Adding libmodule-build-perl to rdeps here is pulling in whole target
> > toolchain components now while building core-image-minimal
> > the dependency chain looks like, total task building for this image are
> 3000+
> >
> > "coreutils.do_package_qa" -> "libmodule-build-perl.do_packagedata" ->
> > "packagegroup-core-buildessential.do_packagedata" ->
> > "gcc.do_packagedata"
> >
> > IMO, it's a bad experience for the user when building a simple minimal
> > image and it ends up building whole toolchain for target that
> > eventually won't even
> > be part of the image. perhaps make it dependent on ptest or some such
> > knob thats not commonly used.
> >
> > Please revert this patch series until we can find a way to not insert
> > these build-time dependencies in normal flow.
>
> We were aware of this when it merged. This should already only happen
> if ptest is enabled?
>

I am seeing it with a fresh checkout of poky with no other changes


> (else ${PN}-ptest isn't in PACKAGES and hence the RDEPENDS isn't seen).
>
> I wish we had a better way of handling these dependencies but we sadly
> don't.
>

Such changes should be left to users to enable imo may be a packageconfig
or so

People have reputation of long build times and large disk space for OE this
won’t help that cause

>
> Cheers,
>
> Richard
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2787 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] [PATCH 1/2 v5] coreutils: add ptest
  2020-04-06 14:32       ` Khem Raj
@ 2020-04-06 16:47         ` Richard Purdie
  2020-04-07  2:26           ` Randy MacLeod
  0 siblings, 1 reply; 8+ messages in thread
From: Richard Purdie @ 2020-04-06 16:47 UTC (permalink / raw)
  To: Khem Raj; +Cc: Patches and discussions about the oe-core layer, Trevor Gamblin

On Mon, 2020-04-06 at 07:32 -0700, Khem Raj wrote:
> On Mon, Apr 6, 2020 at 4:58 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> > On Sun, 2020-04-05 at 17:26 -0700, Khem Raj wrote:
> > > On Wed, Mar 11, 2020 at 4:33 AM Trevor Gamblin
> > > <trevor.gamblin@windriver.com> wrote:
> > > > +inherit ptest
> > > > +
> > > > +RDEPENDS_${PN}-ptest += "bash findutils gawk liberror-perl libmodule-build-perl make perl perl-module-file-stat python3-core sed shadow"
> > > > +
> > > 
> > > Adding libmodule-build-perl to rdeps here is pulling in whole target
> > > toolchain components now while building core-image-minimal
> > > the dependency chain looks like, total task building for this image are 3000+
> > > 
> > > "coreutils.do_package_qa" -> "libmodule-build-perl.do_packagedata" ->
> > > "packagegroup-core-buildessential.do_packagedata" ->
> > > "gcc.do_packagedata"
> > > 
> > > IMO, it's a bad experience for the user when building a simple minimal
> > > image and it ends up building whole toolchain for target that
> > > eventually won't even
> > > be part of the image. perhaps make it dependent on ptest or some such
> > > knob thats not commonly used.
> > > 
> > > Please revert this patch series until we can find a way to not insert
> > > these build-time dependencies in normal flow.
> > 
> > We were aware of this when it merged. This should already only happen
> > if ptest is enabled?
> > 
> 
> I am seeing it with a fresh checkout of poky with no other changes 

Poky enables ptest by default:

meta-poky/conf/distro/poky.conf:
POKY_DEFAULT_DISTRO_FEATURES = "largefile opengl ptest multiarch wayland vulkan"

so that is expected.

> > (else ${PN}-ptest isn't in PACKAGES and hence the RDEPENDS isn't seen).
> > 
> > I wish we had a better way of handling these dependencies but we sadly
> > don't.
> > 
> 
> Such changes should be left to users to enable imo may be a packageconfig or so 
> 
> People have reputation of long build times and large disk space for OE this won’t help that cause 

Well, I tried to have this conversation when the patch was being
reviewed and tested. Nobody seemed bothered then. I get tired of
talking to myself in a vacuum and having a reputation about being
difficult over patches when nobody seems to care (as no other voices
came into the conversation). I've tried to minimise the impact which
was the best we can do. We are trying to encourage people to add tests.

Starting to revert things just before release is a really bad idea so
we're not doing that.

We can disable ptest in coreutils specifically if we wanted. Ultimately
we need to find a better way in general of handling some of these
runtime dependencies but at this point that is a 3.2 issue.

Sorry, but if we were to have this discussion it should have been
months ago :(.

I'd also note that 3.1 rc1 is building now.

Cheers,

Richard


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [OE-core] [PATCH 1/2 v5] coreutils: add ptest
  2020-04-06 16:47         ` Richard Purdie
@ 2020-04-07  2:26           ` Randy MacLeod
  0 siblings, 0 replies; 8+ messages in thread
From: Randy MacLeod @ 2020-04-07  2:26 UTC (permalink / raw)
  To: Richard Purdie, Khem Raj, bluelightning
  Cc: Patches and discussions about the oe-core layer, Trevor Gamblin

On 2020-04-06 12:47 p.m., Richard Purdie wrote:
> On Mon, 2020-04-06 at 07:32 -0700, Khem Raj wrote:
>> On Mon, Apr 6, 2020 at 4:58 AM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
>>> On Sun, 2020-04-05 at 17:26 -0700, Khem Raj wrote:
>>>> On Wed, Mar 11, 2020 at 4:33 AM Trevor Gamblin
>>>> <trevor.gamblin@windriver.com> wrote:
>>>>> +inherit ptest
>>>>> +
>>>>> +RDEPENDS_${PN}-ptest += "bash findutils gawk liberror-perl libmodule-build-perl make perl perl-module-file-stat python3-core sed shadow"
>>>>> +
>>>>
>>>> Adding libmodule-build-perl to rdeps here is pulling in whole target
>>>> toolchain components now while building core-image-minimal
>>>> the dependency chain looks like, total task building for this image are 3000+
>>>>
>>>> "coreutils.do_package_qa" -> "libmodule-build-perl.do_packagedata" ->
>>>> "packagegroup-core-buildessential.do_packagedata" ->
>>>> "gcc.do_packagedata"
>>>>
>>>> IMO, it's a bad experience for the user when building a simple minimal
>>>> image and it ends up building whole toolchain for target that
>>>> eventually won't even
>>>> be part of the image. perhaps make it dependent on ptest or some such
>>>> knob thats not commonly used.
>>>>
>>>> Please revert this patch series until we can find a way to not insert
>>>> these build-time dependencies in normal flow.
>>>
>>> We were aware of this when it merged. This should already only happen
>>> if ptest is enabled?
>>>
>>
>> I am seeing it with a fresh checkout of poky with no other changes
> 
> Poky enables ptest by default:
> 
> meta-poky/conf/distro/poky.conf:
> POKY_DEFAULT_DISTRO_FEATURES = "largefile opengl ptest multiarch wayland vulkan"
> 
> so that is expected.

Makes sense to me and I'm glad it's not affecting oe-core by default.
FYI poky has had ptest enabled since at least 2016.

Since poky builds with coreutils will take a bit longer, we should
mention that in the release notes. Paul E added since I hear he
writes much of the release notes.


../Randy

> 
>>> (else ${PN}-ptest isn't in PACKAGES and hence the RDEPENDS isn't seen).
>>>
>>> I wish we had a better way of handling these dependencies but we sadly
>>> don't.
>>>
>>
>> Such changes should be left to users to enable imo may be a packageconfig or so
>>
>> People have reputation of long build times and large disk space for OE this won’t help that cause
> 
> Well, I tried to have this conversation when the patch was being
> reviewed and tested. Nobody seemed bothered then. I get tired of
> talking to myself in a vacuum and having a reputation about being
> difficult over patches when nobody seems to care (as no other voices
> came into the conversation). I've tried to minimise the impact which
> was the best we can do. We are trying to encourage people to add tests.
> 
> Starting to revert things just before release is a really bad idea so
> we're not doing that.
> 
> We can disable ptest in coreutils specifically if we wanted. Ultimately
> we need to find a better way in general of handling some of these
> runtime dependencies but at this point that is a 3.2 issue.
> 
> Sorry, but if we were to have this discussion it should have been
> months ago :(.
> 
> I'd also note that 3.1 rc1 is building now.
> 
> Cheers,
> 
> Richard
> 
> 
> 
> 


-- 
# Randy MacLeod
# Wind River Linux

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2020-04-07  2:27 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-11 11:32 [PATCH 0/2 v5] coreutils: add ptest support Trevor Gamblin
2020-03-11 11:32 ` [PATCH 1/2 v5] coreutils: add ptest Trevor Gamblin
2020-04-06  0:26   ` [OE-core] " Khem Raj
2020-04-06 11:58     ` Richard Purdie
2020-04-06 14:32       ` Khem Raj
2020-04-06 16:47         ` Richard Purdie
2020-04-07  2:26           ` Randy MacLeod
2020-03-11 11:32 ` [PATCH 2/2 v5] ptest-packagelists.inc: add coreutils to SLOW Trevor Gamblin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.