[PATCH RESEND 1/2] glibc: Exclude common code to build tests to glibc-tests.inc

[PATCH RESEND 1/2] glibc: Exclude common code to build tests to glibc-tests.inc

[Lukasz Majewski]

The common code to build tests has been excluded from glibc-testsuite
recipe to glibc-tests.inc

This code will be reused in the recipe necessary for providing glibc
tests executed with ptest framework.

Signed-off-by: Lukasz Majewski <lukma@denx.de>
---
 meta/recipes-core/glibc/glibc-tests.inc       | 32 ++++++++++++++++
 .../glibc/glibc-testsuite_2.34.bb             | 37 ++-----------------
 2 files changed, 36 insertions(+), 33 deletions(-)
 create mode 100644 meta/recipes-core/glibc/glibc-tests.inc

diff --git a/meta/recipes-core/glibc/glibc-tests.inc b/meta/recipes-core/glibc/glibc-tests.inc
new file mode 100644
index 0000000000..a8fe160a54
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc-tests.inc
@@ -0,0 +1,32 @@
+EXCLUDE_FROM_WORLD = "1"
+
+# handle PN differences
+FILESEXTRAPATHS:prepend := "${THISDIR}/glibc:"
+
+# setup depends
+INHIBIT_DEFAULT_DEPS = ""
+
+python () {
+    libc = d.getVar("PREFERRED_PROVIDER_virtual/libc")
+    libclocale = d.getVar("PREFERRED_PROVIDER_virtual/libc-locale")
+    if libc != "glibc" or libclocale != "glibc-locale":
+        raise bb.parse.SkipRecipe("glibc-testsuite requires that virtual/libc is glibc")
+}
+
+DEPENDS += "glibc-locale libgcc gcc-runtime"
+
+# remove the initial depends
+DEPENDS:remove = "libgcc-initial"
+
+do_check[dirs] += "${B}"
+do_check () {
+    # clean out previous test results
+    oe_runmake tests-clean
+    # makefiles don't clean entirely (and also sometimes fails due to too many args)
+    find ${B} -type f -name "*.out" -delete
+    find ${B} -type f -name "*.test-result" -delete
+    find ${B}/catgets -name "*.cat" -delete
+    find ${B}/conform -name "symlist-*" -delete
+    [ ! -e ${B}/timezone/testdata ] || rm -rf ${B}/timezone/testdata
+}
+addtask do_check after do_compile
diff --git a/meta/recipes-core/glibc/glibc-testsuite_2.34.bb b/meta/recipes-core/glibc/glibc-testsuite_2.34.bb
index 72aa332171..e8ad2a938b 100644
--- a/meta/recipes-core/glibc/glibc-testsuite_2.34.bb
+++ b/meta/recipes-core/glibc/glibc-testsuite_2.34.bb
@@ -1,30 +1,12 @@
 require glibc_${PV}.bb
+require glibc-tests.inc
 
-EXCLUDE_FROM_WORLD = "1"
+inherit qemu
 
-# handle PN differences
-FILESEXTRAPATHS:prepend := "${THISDIR}/glibc:"
+SRC_URI += "file://check-test-wrapper"
 
 # strip provides
 PROVIDES = ""
-# setup depends
-INHIBIT_DEFAULT_DEPS = ""
-
-python () {
-    libc = d.getVar("PREFERRED_PROVIDER_virtual/libc")
-    libclocale = d.getVar("PREFERRED_PROVIDER_virtual/libc-locale")
-    if libc != "glibc" or libclocale != "glibc-locale":
-        raise bb.parse.SkipRecipe("glibc-testsuite requires that virtual/libc is glibc")
-}
-
-DEPENDS += "glibc-locale libgcc gcc-runtime"
-
-# remove the initial depends
-DEPENDS:remove = "libgcc-initial"
-
-inherit qemu
-
-SRC_URI += "file://check-test-wrapper"
 
 DEPENDS += "${@'qemu-native' if d.getVar('TOOLCHAIN_TEST_TARGET') == 'user' else ''}"
 
@@ -33,20 +15,10 @@ TOOLCHAIN_TEST_HOST ??= "localhost"
 TOOLCHAIN_TEST_HOST_USER ??= "root"
 TOOLCHAIN_TEST_HOST_PORT ??= "2222"
 
-do_check[dirs] += "${B}"
 do_check[nostamp] = "1"
-do_check () {
+do_check:append () {
     chmod 0755 ${WORKDIR}/check-test-wrapper
 
-    # clean out previous test results
-    oe_runmake tests-clean
-    # makefiles don't clean entirely (and also sometimes fails due to too many args)
-    find ${B} -type f -name "*.out" -delete
-    find ${B} -type f -name "*.test-result" -delete
-    find ${B}/catgets -name "*.cat" -delete
-    find ${B}/conform -name "symlist-*" -delete
-    [ ! -e ${B}/timezone/testdata ] || rm -rf ${B}/timezone/testdata
-
     oe_runmake -i \
         QEMU_SYSROOT="${RECIPE_SYSROOT}" \
         QEMU_OPTIONS="${@qemu_target_binary(d)} ${QEMU_OPTIONS}" \
@@ -56,7 +28,6 @@ do_check () {
         test-wrapper="${WORKDIR}/check-test-wrapper ${TOOLCHAIN_TEST_TARGET}" \
         check
 }
-addtask do_check after do_compile
 
 inherit nopackages
 deltask do_stash_locale
-- 
2.20.1

[PATCH RESEND 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Lukasz Majewski]

This patch introduces new recipe - namely 'glibc-tests', which
builds and installs glibc test suite to OE/Yocto built image.

It reuses code from already available 'glibc-testsuite' recipe,
which is run with 'bitbake glibc-testsuite -c check' and uses qemu
to execute remotely (via SSH) tests on some emulated machine.

This recipe installs eligible tests on some rootfs image.
Afterwards, either all of them or only time related subset, those
tests can be executed on the real hardware, to facilitate validation
of this platform with for example Y2038 problem compliance.

By default all tests are executed, with 'ptest-runner glibc-tests'.
To test only time related subset - one needs to call:
cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \
ln -s run-ptest-time run-ptest; cd -; ptest-runner glibc-tests

To facilitate debugging, source files are provided by default with
the unstripped debugging symbols. Such approach would reduce the
already complex recipe (as it inherits base glibc one), so there
is no need to also install *-dbg and *-src packages.

Signed-off-by: Lukasz Majewski <lukma@denx.de>
---
 .../distro/include/ptest-packagelists.inc     |   1 +
 meta/recipes-core/glibc/glibc-tests_2.34.bb   | 101 ++++++++++++++++++
 meta/recipes-core/glibc/glibc/run-ptest-all   |  25 +++++
 meta/recipes-core/glibc/glibc/run-ptest-time  |  30 ++++++
 4 files changed, 157 insertions(+)
 create mode 100644 meta/recipes-core/glibc/glibc-tests_2.34.bb
 create mode 100755 meta/recipes-core/glibc/glibc/run-ptest-all
 create mode 100755 meta/recipes-core/glibc/glibc/run-ptest-time

diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
index e0ae1dd971..2031c8730a 100644
--- a/meta/conf/distro/include/ptest-packagelists.inc
+++ b/meta/conf/distro/include/ptest-packagelists.inc
@@ -60,6 +60,7 @@ PTESTS_FAST = "\
     slang-ptest \
     wayland-ptest \
     zlib-ptest \
+    glibc-tests-ptest \
 "
 PTESTS_FAST:remove:mips64 = "qemu-ptest"
 PTESTS_PROBLEMS:append:mips64 = "qemu-ptest"
diff --git a/meta/recipes-core/glibc/glibc-tests_2.34.bb b/meta/recipes-core/glibc/glibc-tests_2.34.bb
new file mode 100644
index 0000000000..896809f700
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc-tests_2.34.bb
@@ -0,0 +1,101 @@
+require glibc_${PV}.bb
+require glibc-tests.inc
+
+inherit ptest
+
+SRC_URI:append = " \
+	file://run-ptest-all file://run-ptest-time \
+"
+
+SUMMARY = "glibc tests to be run with ptest"
+
+# Erase some variables already set by glibc_${PV}
+python __anonymous() {
+       # Remove packages provided by glibc build, we only need a subset of them
+       d.setVar("PACKAGES", "${PN}")
+
+       d.setVar("PROVIDES", "${PN}")
+       d.setVar("RPROVIDES", "${PN} glibc-ptest")
+
+       d.setVar("RRECOMMENDS", "")
+}
+
+# Remove any lefovers from original glibc recipe
+RPROVIDES:${PN} = "${PN}"
+RRECOMMENDS:${PN} = ""
+RDEPENDS:${PN} = "sed"
+
+# Just build tests for target - do not run them
+do_check:append () {
+	oe_runmake -i check run-built-tests=no
+}
+addtask do_check after do_compile before do_install_ptest_base
+
+glibc_strip_build_directory () {
+	# Delete all non executable files from build directory
+	find ${B} ! -executable -type f -delete
+
+	# Remove build dynamic libraries and links to them as
+	# those are already installed in the target device
+	find ${B} -type f -name "*.so" -delete
+	find ${B} -type l -name "*.so*" -delete
+
+	# Remove headers (installed with glibc)
+	find ${B} -type f -name "*.h" -delete
+
+	find ${B} -type f -name "isomac" -delete
+	find ${B} -type f -name "annexc" -delete
+}
+
+do_install_ptest_base () {
+	glibc_strip_build_directory
+
+	# Install build test programs to the image
+	install -d ${D}${PTEST_PATH}/tests/glibc-ptest/
+	cp -r ${B}/* ${D}${PTEST_PATH}/tests/glibc-ptest/
+
+	install -d ${D}${PTEST_PATH}
+	cp ${WORKDIR}/run-ptest-* ${D}${PTEST_PATH}/
+
+	# By default run all glibc tests
+	cd ${D}${PTEST_PATH}/
+	ln -s run-ptest-all run-ptest
+}
+
+# The datadir directory is required to allow core (and reused)
+# glibc cleanup function to finish correctly, as this directory
+# is not created for ptests
+stash_locale_package_cleanup:prepend () {
+	mkdir -p ${PKGD}${datadir}
+}
+
+stash_locale_sysroot_cleanup:prepend () {
+	mkdir -p ${SYSROOT_DESTDIR}${datadir}
+}
+
+# Prevent the do_package() task to set 'libc6' prefix
+# for glibc tests related packages
+python populate_packages:prepend () {
+    if d.getVar('DEBIAN_NAMES'):
+        d.setVar('DEBIAN_NAMES', '')
+}
+
+FILES:${PN} = "${PTEST_PATH}/* /usr/src/debug/glibc-tests/*"
+
+EXCLUDE_FROM_SHLIBS = "1"
+
+# Install debug data in .debug and sources in /usr/src/debug
+# It is more handy to have _all_ the sources and symbols in one
+# place (package) as this recipe will be used for validation and
+# debugging.
+PACKAGE_DEBUG_SPLIT_STYLE = "debug"
+
+# glibc test cases violate by default some Yocto/OE checks (staticdev,
+# textrel)
+# 'debug-files' - add everything (including debug) into one package
+#                 (no need to install/build *-src package)
+INSANE_SKIP:${PN} += "staticdev textrel debug-files rpaths"
+
+deltask do_stash_locale
+do_install[noexec] = "1"
+do_populate_sysroot[noexec] = "1"
diff --git a/meta/recipes-core/glibc/glibc/run-ptest-all b/meta/recipes-core/glibc/glibc/run-ptest-all
new file mode 100755
index 0000000000..47bfe5238c
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/run-ptest-all
@@ -0,0 +1,25 @@
+#!/bin/sh
+# ptest script for glibc - to run time related tests to
+# facilitate Y2038 validation
+# Run with 'ptest-runner glibc-tests'
+
+output() {
+  retcode=$?
+  if [ $retcode -eq 0 ]
+    then echo "PASS: $i"
+  elif [ $retcode -eq 77 ]
+    then echo "SKIP: $i"
+  else echo "FAIL: $i"
+  fi
+}
+
+# Allow altering time on the target
+export GLIBC_TEST_ALLOW_TIME_SETTING="1"
+
+tst=$(ls -r ${PWD}/tests/glibc-ptest/*/*)
+
+for i in ${tst}
+do
+	$i >/dev/null 2>&1
+	output
+done
diff --git a/meta/recipes-core/glibc/glibc/run-ptest-time b/meta/recipes-core/glibc/glibc/run-ptest-time
new file mode 100755
index 0000000000..24dedc6e32
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/run-ptest-time
@@ -0,0 +1,30 @@
+#!/bin/sh
+# ptest script for glibc - to run time related tests to
+# facilitate Y2038 validation
+# Run with 'ptest-runner glibc-tests'
+
+output() {
+  retcode=$?
+  if [ $retcode -eq 0 ]
+    then echo "PASS: $i"
+  elif [ $retcode -eq 77 ]
+    then echo "SKIP: $i"
+  else echo "FAIL: $i"
+  fi
+}
+
+# Allow altering time on the target
+export GLIBC_TEST_ALLOW_TIME_SETTING="1"
+
+tst_time64=$(ls -r ${PWD}/tests/glibc-ptest/*/*-time64)
+
+# Remove '-time64' suffix - those tests are also time
+# related
+tst_time_tmp=$(sed -e "s/-time64$//" <<< ${tst_time64})
+tst_time="${tst_time_tmp} ${tst_time64}"
+
+for i in ${tst_time}
+do
+	$i >/dev/null 2>&1
+	output
+done
-- 
2.20.1

Re: [OE-core] [PATCH RESEND 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Alexandre Belloni]

Hello Lukasz,


On 30/09/2021 12:43:57+0200, ?ukasz Majewski wrote:
> This patch introduces new recipe - namely 'glibc-tests', which
> builds and installs glibc test suite to OE/Yocto built image.
> 
> It reuses code from already available 'glibc-testsuite' recipe,
> which is run with 'bitbake glibc-testsuite -c check' and uses qemu
> to execute remotely (via SSH) tests on some emulated machine.
> 
> This recipe installs eligible tests on some rootfs image.
> Afterwards, either all of them or only time related subset, those
> tests can be executed on the real hardware, to facilitate validation
> of this platform with for example Y2038 problem compliance.
> 
> By default all tests are executed, with 'ptest-runner glibc-tests'.
> To test only time related subset - one needs to call:
> cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \
> ln -s run-ptest-time run-ptest; cd -; ptest-runner glibc-tests
> 
> To facilitate debugging, source files are provided by default with
> the unstripped debugging symbols. Such approach would reduce the
> already complex recipe (as it inherits base glibc one), so there
> is no need to also install *-dbg and *-src packages.
> 

This ends up with:

ERROR: Nothing RPROVIDES 'glibc-tests-ptest' (but /home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/recipes-core/images/core-image-ptest-all.bb RDEPENDS on or otherwise requires it)

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/2341/steps/11/logs/stdio

> Signed-off-by: Lukasz Majewski <lukma@denx.de>
> ---
>  .../distro/include/ptest-packagelists.inc     |   1 +
>  meta/recipes-core/glibc/glibc-tests_2.34.bb   | 101 ++++++++++++++++++
>  meta/recipes-core/glibc/glibc/run-ptest-all   |  25 +++++
>  meta/recipes-core/glibc/glibc/run-ptest-time  |  30 ++++++
>  4 files changed, 157 insertions(+)
>  create mode 100644 meta/recipes-core/glibc/glibc-tests_2.34.bb
>  create mode 100755 meta/recipes-core/glibc/glibc/run-ptest-all
>  create mode 100755 meta/recipes-core/glibc/glibc/run-ptest-time
> 
> diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc
> index e0ae1dd971..2031c8730a 100644
> --- a/meta/conf/distro/include/ptest-packagelists.inc
> +++ b/meta/conf/distro/include/ptest-packagelists.inc
> @@ -60,6 +60,7 @@ PTESTS_FAST = "\
>      slang-ptest \
>      wayland-ptest \
>      zlib-ptest \
> +    glibc-tests-ptest \
>  "
>  PTESTS_FAST:remove:mips64 = "qemu-ptest"
>  PTESTS_PROBLEMS:append:mips64 = "qemu-ptest"
> diff --git a/meta/recipes-core/glibc/glibc-tests_2.34.bb b/meta/recipes-core/glibc/glibc-tests_2.34.bb
> new file mode 100644
> index 0000000000..896809f700
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc-tests_2.34.bb
> @@ -0,0 +1,101 @@
> +require glibc_${PV}.bb
> +require glibc-tests.inc
> +
> +inherit ptest
> +
> +SRC_URI:append = " \
> +	file://run-ptest-all file://run-ptest-time \
> +"
> +
> +SUMMARY = "glibc tests to be run with ptest"
> +
> +# Erase some variables already set by glibc_${PV}
> +python __anonymous() {
> +       # Remove packages provided by glibc build, we only need a subset of them
> +       d.setVar("PACKAGES", "${PN}")
> +
> +       d.setVar("PROVIDES", "${PN}")
> +       d.setVar("RPROVIDES", "${PN} glibc-ptest")
> +
> +       d.setVar("RRECOMMENDS", "")
> +}
> +
> +# Remove any lefovers from original glibc recipe
> +RPROVIDES:${PN} = "${PN}"
> +RRECOMMENDS:${PN} = ""
> +RDEPENDS:${PN} = "sed"
> +
> +# Just build tests for target - do not run them
> +do_check:append () {
> +	oe_runmake -i check run-built-tests=no
> +}
> +addtask do_check after do_compile before do_install_ptest_base
> +
> +glibc_strip_build_directory () {
> +	# Delete all non executable files from build directory
> +	find ${B} ! -executable -type f -delete
> +
> +	# Remove build dynamic libraries and links to them as
> +	# those are already installed in the target device
> +	find ${B} -type f -name "*.so" -delete
> +	find ${B} -type l -name "*.so*" -delete
> +
> +	# Remove headers (installed with glibc)
> +	find ${B} -type f -name "*.h" -delete
> +
> +	find ${B} -type f -name "isomac" -delete
> +	find ${B} -type f -name "annexc" -delete
> +}
> +
> +do_install_ptest_base () {
> +	glibc_strip_build_directory
> +
> +	# Install build test programs to the image
> +	install -d ${D}${PTEST_PATH}/tests/glibc-ptest/
> +	cp -r ${B}/* ${D}${PTEST_PATH}/tests/glibc-ptest/
> +
> +	install -d ${D}${PTEST_PATH}
> +	cp ${WORKDIR}/run-ptest-* ${D}${PTEST_PATH}/
> +
> +	# By default run all glibc tests
> +	cd ${D}${PTEST_PATH}/
> +	ln -s run-ptest-all run-ptest
> +}
> +
> +# The datadir directory is required to allow core (and reused)
> +# glibc cleanup function to finish correctly, as this directory
> +# is not created for ptests
> +stash_locale_package_cleanup:prepend () {
> +	mkdir -p ${PKGD}${datadir}
> +}
> +
> +stash_locale_sysroot_cleanup:prepend () {
> +	mkdir -p ${SYSROOT_DESTDIR}${datadir}
> +}
> +
> +# Prevent the do_package() task to set 'libc6' prefix
> +# for glibc tests related packages
> +python populate_packages:prepend () {
> +    if d.getVar('DEBIAN_NAMES'):
> +        d.setVar('DEBIAN_NAMES', '')
> +}
> +
> +FILES:${PN} = "${PTEST_PATH}/* /usr/src/debug/glibc-tests/*"
> +
> +EXCLUDE_FROM_SHLIBS = "1"
> +
> +# Install debug data in .debug and sources in /usr/src/debug
> +# It is more handy to have _all_ the sources and symbols in one
> +# place (package) as this recipe will be used for validation and
> +# debugging.
> +PACKAGE_DEBUG_SPLIT_STYLE = "debug"
> +
> +# glibc test cases violate by default some Yocto/OE checks (staticdev,
> +# textrel)
> +# 'debug-files' - add everything (including debug) into one package
> +#                 (no need to install/build *-src package)
> +INSANE_SKIP:${PN} += "staticdev textrel debug-files rpaths"
> +
> +deltask do_stash_locale
> +do_install[noexec] = "1"
> +do_populate_sysroot[noexec] = "1"
> diff --git a/meta/recipes-core/glibc/glibc/run-ptest-all b/meta/recipes-core/glibc/glibc/run-ptest-all
> new file mode 100755
> index 0000000000..47bfe5238c
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc/run-ptest-all
> @@ -0,0 +1,25 @@
> +#!/bin/sh
> +# ptest script for glibc - to run time related tests to
> +# facilitate Y2038 validation
> +# Run with 'ptest-runner glibc-tests'
> +
> +output() {
> +  retcode=$?
> +  if [ $retcode -eq 0 ]
> +    then echo "PASS: $i"
> +  elif [ $retcode -eq 77 ]
> +    then echo "SKIP: $i"
> +  else echo "FAIL: $i"
> +  fi
> +}
> +
> +# Allow altering time on the target
> +export GLIBC_TEST_ALLOW_TIME_SETTING="1"
> +
> +tst=$(ls -r ${PWD}/tests/glibc-ptest/*/*)
> +
> +for i in ${tst}
> +do
> +	$i >/dev/null 2>&1
> +	output
> +done
> diff --git a/meta/recipes-core/glibc/glibc/run-ptest-time b/meta/recipes-core/glibc/glibc/run-ptest-time
> new file mode 100755
> index 0000000000..24dedc6e32
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc/run-ptest-time
> @@ -0,0 +1,30 @@
> +#!/bin/sh
> +# ptest script for glibc - to run time related tests to
> +# facilitate Y2038 validation
> +# Run with 'ptest-runner glibc-tests'
> +
> +output() {
> +  retcode=$?
> +  if [ $retcode -eq 0 ]
> +    then echo "PASS: $i"
> +  elif [ $retcode -eq 77 ]
> +    then echo "SKIP: $i"
> +  else echo "FAIL: $i"
> +  fi
> +}
> +
> +# Allow altering time on the target
> +export GLIBC_TEST_ALLOW_TIME_SETTING="1"
> +
> +tst_time64=$(ls -r ${PWD}/tests/glibc-ptest/*/*-time64)
> +
> +# Remove '-time64' suffix - those tests are also time
> +# related
> +tst_time_tmp=$(sed -e "s/-time64$//" <<< ${tst_time64})
> +tst_time="${tst_time_tmp} ${tst_time64}"
> +
> +for i in ${tst_time}
> +do
> +	$i >/dev/null 2>&1
> +	output
> +done
> -- 
> 2.20.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#156490): https://lists.openembedded.org/g/openembedded-core/message/156490
> Mute This Topic: https://lists.openembedded.org/mt/85970382/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [PATCH RESEND 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 7839 bytes --]

Dear Community,

> This patch introduces new recipe - namely 'glibc-tests', which
> builds and installs glibc test suite to OE/Yocto built image.
> 
> It reuses code from already available 'glibc-testsuite' recipe,
> which is run with 'bitbake glibc-testsuite -c check' and uses qemu
> to execute remotely (via SSH) tests on some emulated machine.
> 
> This recipe installs eligible tests on some rootfs image.
> Afterwards, either all of them or only time related subset, those
> tests can be executed on the real hardware, to facilitate validation
> of this platform with for example Y2038 problem compliance.
> 
> By default all tests are executed, with 'ptest-runner glibc-tests'.
> To test only time related subset - one needs to call:
> cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \
> ln -s run-ptest-time run-ptest; cd -; ptest-runner glibc-tests
> 
> To facilitate debugging, source files are provided by default with
> the unstripped debugging symbols. Such approach would reduce the
> already complex recipe (as it inherits base glibc one), so there
> is no need to also install *-dbg and *-src packages.
> 

Gentle ping on this patch set.

> Signed-off-by: Lukasz Majewski <lukma@denx.de>
> ---
>  .../distro/include/ptest-packagelists.inc     |   1 +
>  meta/recipes-core/glibc/glibc-tests_2.34.bb   | 101
> ++++++++++++++++++ meta/recipes-core/glibc/glibc/run-ptest-all   |
> 25 +++++ meta/recipes-core/glibc/glibc/run-ptest-time  |  30 ++++++
>  4 files changed, 157 insertions(+)
>  create mode 100644 meta/recipes-core/glibc/glibc-tests_2.34.bb
>  create mode 100755 meta/recipes-core/glibc/glibc/run-ptest-all
>  create mode 100755 meta/recipes-core/glibc/glibc/run-ptest-time
> 
> diff --git a/meta/conf/distro/include/ptest-packagelists.inc
> b/meta/conf/distro/include/ptest-packagelists.inc index
> e0ae1dd971..2031c8730a 100644 ---
> a/meta/conf/distro/include/ptest-packagelists.inc +++
> b/meta/conf/distro/include/ptest-packagelists.inc @@ -60,6 +60,7 @@
> PTESTS_FAST = "\ slang-ptest \
>      wayland-ptest \
>      zlib-ptest \
> +    glibc-tests-ptest \
>  "
>  PTESTS_FAST:remove:mips64 = "qemu-ptest"
>  PTESTS_PROBLEMS:append:mips64 = "qemu-ptest"
> diff --git a/meta/recipes-core/glibc/glibc-tests_2.34.bb
> b/meta/recipes-core/glibc/glibc-tests_2.34.bb new file mode 100644
> index 0000000000..896809f700
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc-tests_2.34.bb
> @@ -0,0 +1,101 @@
> +require glibc_${PV}.bb
> +require glibc-tests.inc
> +
> +inherit ptest
> +
> +SRC_URI:append = " \
> +	file://run-ptest-all file://run-ptest-time \
> +"
> +
> +SUMMARY = "glibc tests to be run with ptest"
> +
> +# Erase some variables already set by glibc_${PV}
> +python __anonymous() {
> +       # Remove packages provided by glibc build, we only need a
> subset of them
> +       d.setVar("PACKAGES", "${PN}")
> +
> +       d.setVar("PROVIDES", "${PN}")
> +       d.setVar("RPROVIDES", "${PN} glibc-ptest")
> +
> +       d.setVar("RRECOMMENDS", "")
> +}
> +
> +# Remove any lefovers from original glibc recipe
> +RPROVIDES:${PN} = "${PN}"
> +RRECOMMENDS:${PN} = ""
> +RDEPENDS:${PN} = "sed"
> +
> +# Just build tests for target - do not run them
> +do_check:append () {
> +	oe_runmake -i check run-built-tests=no
> +}
> +addtask do_check after do_compile before do_install_ptest_base
> +
> +glibc_strip_build_directory () {
> +	# Delete all non executable files from build directory
> +	find ${B} ! -executable -type f -delete
> +
> +	# Remove build dynamic libraries and links to them as
> +	# those are already installed in the target device
> +	find ${B} -type f -name "*.so" -delete
> +	find ${B} -type l -name "*.so*" -delete
> +
> +	# Remove headers (installed with glibc)
> +	find ${B} -type f -name "*.h" -delete
> +
> +	find ${B} -type f -name "isomac" -delete
> +	find ${B} -type f -name "annexc" -delete
> +}
> +
> +do_install_ptest_base () {
> +	glibc_strip_build_directory
> +
> +	# Install build test programs to the image
> +	install -d ${D}${PTEST_PATH}/tests/glibc-ptest/
> +	cp -r ${B}/* ${D}${PTEST_PATH}/tests/glibc-ptest/
> +
> +	install -d ${D}${PTEST_PATH}
> +	cp ${WORKDIR}/run-ptest-* ${D}${PTEST_PATH}/
> +
> +	# By default run all glibc tests
> +	cd ${D}${PTEST_PATH}/
> +	ln -s run-ptest-all run-ptest
> +}
> +
> +# The datadir directory is required to allow core (and reused)
> +# glibc cleanup function to finish correctly, as this directory
> +# is not created for ptests
> +stash_locale_package_cleanup:prepend () {
> +	mkdir -p ${PKGD}${datadir}
> +}
> +
> +stash_locale_sysroot_cleanup:prepend () {
> +	mkdir -p ${SYSROOT_DESTDIR}${datadir}
> +}
> +
> +# Prevent the do_package() task to set 'libc6' prefix
> +# for glibc tests related packages
> +python populate_packages:prepend () {
> +    if d.getVar('DEBIAN_NAMES'):
> +        d.setVar('DEBIAN_NAMES', '')
> +}
> +
> +FILES:${PN} = "${PTEST_PATH}/* /usr/src/debug/glibc-tests/*"
> +
> +EXCLUDE_FROM_SHLIBS = "1"
> +
> +# Install debug data in .debug and sources in /usr/src/debug
> +# It is more handy to have _all_ the sources and symbols in one
> +# place (package) as this recipe will be used for validation and
> +# debugging.
> +PACKAGE_DEBUG_SPLIT_STYLE = "debug"
> +
> +# glibc test cases violate by default some Yocto/OE checks
> (staticdev, +# textrel)
> +# 'debug-files' - add everything (including debug) into one package
> +#                 (no need to install/build *-src package)
> +INSANE_SKIP:${PN} += "staticdev textrel debug-files rpaths"
> +
> +deltask do_stash_locale
> +do_install[noexec] = "1"
> +do_populate_sysroot[noexec] = "1"
> diff --git a/meta/recipes-core/glibc/glibc/run-ptest-all
> b/meta/recipes-core/glibc/glibc/run-ptest-all new file mode 100755
> index 0000000000..47bfe5238c
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc/run-ptest-all
> @@ -0,0 +1,25 @@
> +#!/bin/sh
> +# ptest script for glibc - to run time related tests to
> +# facilitate Y2038 validation
> +# Run with 'ptest-runner glibc-tests'
> +
> +output() {
> +  retcode=$?
> +  if [ $retcode -eq 0 ]
> +    then echo "PASS: $i"
> +  elif [ $retcode -eq 77 ]
> +    then echo "SKIP: $i"
> +  else echo "FAIL: $i"
> +  fi
> +}
> +
> +# Allow altering time on the target
> +export GLIBC_TEST_ALLOW_TIME_SETTING="1"
> +
> +tst=$(ls -r ${PWD}/tests/glibc-ptest/*/*)
> +
> +for i in ${tst}
> +do
> +	$i >/dev/null 2>&1
> +	output
> +done
> diff --git a/meta/recipes-core/glibc/glibc/run-ptest-time
> b/meta/recipes-core/glibc/glibc/run-ptest-time new file mode 100755
> index 0000000000..24dedc6e32
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc/run-ptest-time
> @@ -0,0 +1,30 @@
> +#!/bin/sh
> +# ptest script for glibc - to run time related tests to
> +# facilitate Y2038 validation
> +# Run with 'ptest-runner glibc-tests'
> +
> +output() {
> +  retcode=$?
> +  if [ $retcode -eq 0 ]
> +    then echo "PASS: $i"
> +  elif [ $retcode -eq 77 ]
> +    then echo "SKIP: $i"
> +  else echo "FAIL: $i"
> +  fi
> +}
> +
> +# Allow altering time on the target
> +export GLIBC_TEST_ALLOW_TIME_SETTING="1"
> +
> +tst_time64=$(ls -r ${PWD}/tests/glibc-ptest/*/*-time64)
> +
> +# Remove '-time64' suffix - those tests are also time
> +# related
> +tst_time_tmp=$(sed -e "s/-time64$//" <<< ${tst_time64})
> +tst_time="${tst_time_tmp} ${tst_time64}"
> +
> +for i in ${tst_time}
> +do
> +	$i >/dev/null 2>&1
> +	output
> +done




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RESEND 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Richard Purdie]

On Wed, 2021-10-13 at 10:35 +0200, Lukasz Majewski wrote:
> Dear Community,
> 
> > This patch introduces new recipe - namely 'glibc-tests', which
> > builds and installs glibc test suite to OE/Yocto built image.
> > 
> > It reuses code from already available 'glibc-testsuite' recipe,
> > which is run with 'bitbake glibc-testsuite -c check' and uses qemu
> > to execute remotely (via SSH) tests on some emulated machine.
> > 
> > This recipe installs eligible tests on some rootfs image.
> > Afterwards, either all of them or only time related subset, those
> > tests can be executed on the real hardware, to facilitate validation
> > of this platform with for example Y2038 problem compliance.
> > 
> > By default all tests are executed, with 'ptest-runner glibc-tests'.
> > To test only time related subset - one needs to call:
> > cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \
> > ln -s run-ptest-time run-ptest; cd -; ptest-runner glibc-tests
> > 
> > To facilitate debugging, source files are provided by default with
> > the unstripped debugging symbols. Such approach would reduce the
> > already complex recipe (as it inherits base glibc one), so there
> > is no need to also install *-dbg and *-src packages.
> > 
> 
> Gentle ping on this patch set.

We're in the middle of trying to make a release. This change needs to wait until
after that. We're starting to work through that backlog but easier things will
go first.

FWIW I'm still very reticent to add in two code paths doing similar things so am
not entirely sure we will be able to take this :(

Cheers,

Richard

Re: [PATCH RESEND 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2509 bytes --]

Hi Richard,

> On Wed, 2021-10-13 at 10:35 +0200, Lukasz Majewski wrote:
> > Dear Community,
> >   
> > > This patch introduces new recipe - namely 'glibc-tests', which
> > > builds and installs glibc test suite to OE/Yocto built image.
> > > 
> > > It reuses code from already available 'glibc-testsuite' recipe,
> > > which is run with 'bitbake glibc-testsuite -c check' and uses qemu
> > > to execute remotely (via SSH) tests on some emulated machine.
> > > 
> > > This recipe installs eligible tests on some rootfs image.
> > > Afterwards, either all of them or only time related subset, those
> > > tests can be executed on the real hardware, to facilitate
> > > validation of this platform with for example Y2038 problem
> > > compliance.
> > > 
> > > By default all tests are executed, with 'ptest-runner
> > > glibc-tests'. To test only time related subset - one needs to
> > > call: cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \
> > > ln -s run-ptest-time run-ptest; cd -; ptest-runner glibc-tests
> > > 
> > > To facilitate debugging, source files are provided by default with
> > > the unstripped debugging symbols. Such approach would reduce the
> > > already complex recipe (as it inherits base glibc one), so there
> > > is no need to also install *-dbg and *-src packages.
> > >   
> > 
> > Gentle ping on this patch set.  
> 
> We're in the middle of trying to make a release. This change needs to
> wait until after that. We're starting to work through that backlog
> but easier things will go first.
>

Thanks for the update.
 
> FWIW I'm still very reticent to add in two code paths doing similar
> things so am not entirely sure we will be able to take this :(
> 

As fair as I understand those recipes (glibc related) - there are the
same binaries built once for the target machine.

Then those binaries are either run with QEMU or bundled into separate
package and then installed and run on a _real_ HW.

To even keep things more clear - I can just provide v2 of this patch,
which will only test time related API (as I do know that those
can be executed as standalone binaries) and state that it is meant for
Y2038 compliance testing.

> Cheers,
> 
> Richard
> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [OE-core] [PATCH RESEND 2/2] glibc: ptest: Add support for running glibc test suite with ptest

[Alexandre Belloni]

Hello,

On 13/10/2021 10:35:22+0200, ?ukasz Majewski wrote:
> Dear Community,
> 
> > This patch introduces new recipe - namely 'glibc-tests', which
> > builds and installs glibc test suite to OE/Yocto built image.
> > 
> > It reuses code from already available 'glibc-testsuite' recipe,
> > which is run with 'bitbake glibc-testsuite -c check' and uses qemu
> > to execute remotely (via SSH) tests on some emulated machine.
> > 
> > This recipe installs eligible tests on some rootfs image.
> > Afterwards, either all of them or only time related subset, those
> > tests can be executed on the real hardware, to facilitate validation
> > of this platform with for example Y2038 problem compliance.
> > 
> > By default all tests are executed, with 'ptest-runner glibc-tests'.
> > To test only time related subset - one needs to call:
> > cd /usr/lib/glibc-tests/ptest/; rm run-ptest; \
> > ln -s run-ptest-time run-ptest; cd -; ptest-runner glibc-tests
> > 
> > To facilitate debugging, source files are provided by default with
> > the unstripped debugging symbols. Such approach would reduce the
> > already complex recipe (as it inherits base glibc one), so there
> > is no need to also install *-dbg and *-src packages.
> > 
> 
> Gentle ping on this patch set.
> 

This caused failures on the autobuilders as per my previous email from
the 4th of october:

ERROR: Nothing RPROVIDES 'glibc-tests-ptest' (but
/home/pokybuild/yocto-worker/qemuarm64-ptest/build/meta/recipes-core/images/core-image-ptest-all.bb RDEPENDS on or
otherwise requires it)

https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/2341/steps/11/logs/stdio


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com