[PATCH 00/15] Warrior review request

[PATCH 00/15] Warrior review request

[Armin Kuster]

Here is the next small batch for the next warrior.
This includes " oeqa/target/ssh: Replace suggogatepass with ignoring errors" which was just merged.
( take it up with the owner)

This series was tested on the AB.

The following changes since commit 21079473e0cbd0e43682dc0d15a651e7e64db747:

  oeqa/core/runner: Handle unexpectedSucesses (2019-05-11 08:44:16 -0700)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib stable/warrior-next
  http://cgit.openembedded.org//log/?h=stable/warrior-next

Alejandro Enedino Hernandez Samaniego (1):
  run-postinsts: Fix full execution of scripts at first boot

Alex Kiernan (1):
  shadow: Backport last change reproducibility

Chee Yang Lee (1):
  wic/bootimg-efi: replace hardcoded volume name with label

Hongxu Jia (1):
  oeqa/targetcontrol.py: fix qemuparams not work in runqemu with
    launch_cmd

Kai Kang (1):
  bitbake.conf: set NO_RECOMMENDATIONS with weak assignment

Martin Jansa (2):
  glib-networking: add PACKAGECONFIG for openssl
  perf: make sure that the tools/include/uapi/asm-generic directory
    exists

Mikko Rapeli (2):
  openssh: recommend rng-tools with sshd
  oeqa/sdk: use bash to execute SDK test commands

Ming Liu (1):
  kernel.bbclass: adjust a condition checking

Paul Barker (1):
  gdb: Fix aarch64 build with musl

Richard Purdie (4):
  oeqa/runner: Fix subunit setupClass/setupModule failure handling
  oeqa/concurrenttest: Patch subunit module to handle classSetup
    failures
  oeqa/target/ssh: Replace suggogatepass with ignoring errors
  core-image-sato-sdk-ptest: Tweak size to stay within 4GB limit

 meta/classes/kernel.bbclass                        |  3 +-
 meta/conf/bitbake.conf                             |  2 +-
 meta/lib/oeqa/core/runner.py                       |  4 +-
 meta/lib/oeqa/core/target/ssh.py                   |  4 +-
 meta/lib/oeqa/core/utils/concurrencytest.py        | 27 ++++++-
 meta/lib/oeqa/sdk/case.py                          |  2 +-
 meta/lib/oeqa/sdk/utils/sdkbuildproject.py         |  3 +-
 meta/lib/oeqa/targetcontrol.py                     |  2 +-
 meta/recipes-connectivity/openssh/openssh_7.9p1.bb |  1 +
 .../glib-networking/glib-networking_2.60.1.bb      |  1 +
 meta/recipes-devtools/gdb/gdb-8.2.1.inc            |  1 +
 ...-AArch64-Fix-the-gdb-build-with-musl-libc.patch | 52 +++++++++++++
 .../run-postinsts/run-postinsts/run-postinsts      |  2 +-
 ...p_lstchg-shadow-field-reproducible-re.-71.patch | 89 ++++++++++++++++++++++
 ...002-gettime-Use-secure_getenv-over-getenv.patch | 71 +++++++++++++++++
 meta/recipes-extended/shadow/shadow.inc            |  2 +
 meta/recipes-kernel/perf/perf.bb                   |  2 +-
 .../images/core-image-sato-sdk-ptest.bb            |  4 +-
 scripts/lib/wic/plugins/source/bootimg-efi.py      |  6 +-
 19 files changed, 259 insertions(+), 19 deletions(-)
 create mode 100644 meta/recipes-devtools/gdb/gdb/0012-AArch64-Fix-the-gdb-build-with-musl-libc.patch
 create mode 100644 meta/recipes-extended/shadow/files/0001-Make-the-sp_lstchg-shadow-field-reproducible-re.-71.patch
 create mode 100644 meta/recipes-extended/shadow/files/0002-gettime-Use-secure_getenv-over-getenv.patch

-- 
2.7.4

[PATCH 01/15] gdb: Fix aarch64 build with musl

[Armin Kuster]

From: Paul Barker <paul@betafive.co.uk>

Signed-off-by: Paul Barker <paul@betafive.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/recipes-devtools/gdb/gdb-8.2.1.inc            |  1 +
 ...-AArch64-Fix-the-gdb-build-with-musl-libc.patch | 52 ++++++++++++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 meta/recipes-devtools/gdb/gdb/0012-AArch64-Fix-the-gdb-build-with-musl-libc.patch

diff --git a/meta/recipes-devtools/gdb/gdb-8.2.1.inc b/meta/recipes-devtools/gdb/gdb-8.2.1.inc
index cb8d189..f28b574 100644
--- a/meta/recipes-devtools/gdb/gdb-8.2.1.inc
+++ b/meta/recipes-devtools/gdb/gdb-8.2.1.inc
@@ -18,6 +18,7 @@ SRC_URI = "http://ftp.gnu.org/gnu/gdb/gdb-${PV}.tar.xz \
            file://gdbserver-ctrl-c-handling.patch \
            file://0001-Fix-build-with-latest-GCC-9.0-tree.patch \
            file://CVE-2017-9778.patch \
+           file://0012-AArch64-Fix-the-gdb-build-with-musl-libc.patch \
 "
 SRC_URI[md5sum] = "f8b2562e830a4098dd5b5ea9e9296c70"
 SRC_URI[sha256sum] = "0a6a432907a03c5c8eaad3c3cffd50c00a40c3a5e3c4039440624bae703f2202"
diff --git a/meta/recipes-devtools/gdb/gdb/0012-AArch64-Fix-the-gdb-build-with-musl-libc.patch b/meta/recipes-devtools/gdb/gdb/0012-AArch64-Fix-the-gdb-build-with-musl-libc.patch
new file mode 100644
index 0000000..f35033c
--- /dev/null
+++ b/meta/recipes-devtools/gdb/gdb/0012-AArch64-Fix-the-gdb-build-with-musl-libc.patch
@@ -0,0 +1,52 @@
+From cbbf1a8ba7f037ba3b83f090829edcbab542c285 Mon Sep 17 00:00:00 2001
+From: Szabolcs Nagy <szabolcs.nagy@arm.com>
+Date: Thu, 13 Dec 2018 17:47:17 +0000
+Subject: [PATCH] AArch64: Fix the gdb build with musl libc
+
+Including asm/sigcontext.h together with libc headers is not valid. In
+general linux headers may not work with libc headers, so mixing them
+should be avoided, especially when the linux header defines types that
+are also exposed in libc headers.
+
+In case of asm/sigcontext.h glibc happens to work because glibc signal.h
+directly includes it, but e.g. in musl libc signal.h replicates the
+sigcontext.h definitions in an abi compatible way which are in conflict
+with the linux definitions when both headers are included.
+
+Since old linux headers or old libc headers may not have the necessary
+definitions, gdb has to replicate the definitions it relies on anyway.
+Which is fine since all definitions must be ABI stable. For linux apis
+that are not available via libc headers, replicating the definitions in
+gdb is the most reliable way to use them.
+
+Note: asm/ptrace.h includes asm/sigcontext.h in some versions of linux
+headers, which is just as problematic and should be fixed in linux.
+
+gdb/ChangeLog:
+
+	* nat/aarch64-sve-linux-ptrace.h: Include signal.h instead of
+	asm/sigcontext.h.
+
+[ChangeLog entry stripped so that patch applies cleanly]
+Upstream-Status: Backport
+Signed-off-by: Paul Barker <paul@betafive.co.uk>
+---
+ gdb/nat/aarch64-sve-linux-ptrace.h | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/gdb/nat/aarch64-sve-linux-ptrace.h b/gdb/nat/aarch64-sve-linux-ptrace.h
+index 029e753ffe..172ae39432 100644
+--- a/gdb/nat/aarch64-sve-linux-ptrace.h
++++ b/gdb/nat/aarch64-sve-linux-ptrace.h
+@@ -20,7 +20,7 @@
+ #ifndef AARCH64_SVE_LINUX_PTRACE_H
+ #define AARCH64_SVE_LINUX_PTRACE_H
+ 
+-#include <asm/sigcontext.h>
++#include <signal.h>
+ #include <sys/utsname.h>
+ #include <sys/ptrace.h>
+ #include <asm/ptrace.h>
+-- 
+2.17.1
+
-- 
2.7.4

[PATCH 02/15] shadow: Backport last change reproducibility

[Armin Kuster]

From: Alex Kiernan <alex.kiernan@gmail.com>

The third field in the /etc/shadow file (sp_lstchg) contains the date of
the last password change expressed as the number of days since Jan 1,
1970.

Backport the upstream changes to honour SOURCE_DATE_EPOCH for build
reproducibility.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 ...p_lstchg-shadow-field-reproducible-re.-71.patch | 89 ++++++++++++++++++++++
 ...002-gettime-Use-secure_getenv-over-getenv.patch | 71 +++++++++++++++++
 meta/recipes-extended/shadow/shadow.inc            |  2 +
 3 files changed, 162 insertions(+)
 create mode 100644 meta/recipes-extended/shadow/files/0001-Make-the-sp_lstchg-shadow-field-reproducible-re.-71.patch
 create mode 100644 meta/recipes-extended/shadow/files/0002-gettime-Use-secure_getenv-over-getenv.patch

diff --git a/meta/recipes-extended/shadow/files/0001-Make-the-sp_lstchg-shadow-field-reproducible-re.-71.patch b/meta/recipes-extended/shadow/files/0001-Make-the-sp_lstchg-shadow-field-reproducible-re.-71.patch
new file mode 100644
index 0000000..de0ba3e
--- /dev/null
+++ b/meta/recipes-extended/shadow/files/0001-Make-the-sp_lstchg-shadow-field-reproducible-re.-71.patch
@@ -0,0 +1,89 @@
+From fe34a2a0e44bc80ff213bfd185046a5f10c94997 Mon Sep 17 00:00:00 2001
+From: Chris Lamb <chris@chris-lamb.co.uk>
+Date: Wed, 2 Jan 2019 18:06:16 +0000
+Subject: [PATCH 1/2] Make the sp_lstchg shadow field reproducible (re. #71)
+
+From <https://github.com/shadow-maint/shadow/pull/71>:
+
+```
+The third field in the /etc/shadow file (sp_lstchg) contains the date of
+the last password change expressed as the number of days since Jan 1, 1970.
+As this is a relative time, creating a user today will result in:
+
+username:17238:0:99999:7:::
+whilst creating the same user tomorrow will result in:
+
+username:17239:0:99999:7:::
+This has an impact for the Reproducible Builds[0] project where we aim to
+be independent of as many elements the build environment as possible,
+including the current date.
+
+This patch changes the behaviour to use the SOURCE_DATE_EPOCH[1]
+environment variable (instead of Jan 1, 1970) if valid.
+```
+
+This updated PR adds some missing calls to gettime (). This was originally
+filed by Johannes Schauer in Debian as #917773 [2].
+
+[0] https://reproducible-builds.org/
+[1] https://reproducible-builds.org/specs/source-date-epoch/
+[2] https://bugs.debian.org/917773
+
+Upstream-Status: Backport
+Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
+---
+ libmisc/pwd2spwd.c | 3 +--
+ src/pwck.c         | 2 +-
+ src/pwconv.c       | 2 +-
+ 3 files changed, 3 insertions(+), 4 deletions(-)
+
+diff --git a/libmisc/pwd2spwd.c b/libmisc/pwd2spwd.c
+index c1b9b29ac873..6799dd50d490 100644
+--- a/libmisc/pwd2spwd.c
++++ b/libmisc/pwd2spwd.c
+@@ -40,7 +40,6 @@
+ #include "prototypes.h"
+ #include "defines.h"
+ #include <pwd.h>
+-extern time_t time (time_t *);
+ 
+ /*
+  * pwd_to_spwd - create entries for new spwd structure
+@@ -66,7 +65,7 @@ struct spwd *pwd_to_spwd (const struct passwd *pw)
+ 		 */
+ 		sp.sp_min = 0;
+ 		sp.sp_max = (10000L * DAY) / SCALE;
+-		sp.sp_lstchg = (long) time ((time_t *) 0) / SCALE;
++		sp.sp_lstchg = (long) gettime () / SCALE;
+ 		if (0 == sp.sp_lstchg) {
+ 			/* Better disable aging than requiring a password
+ 			 * change */
+diff --git a/src/pwck.c b/src/pwck.c
+index 0ffb711efb13..f70071b12500 100644
+--- a/src/pwck.c
++++ b/src/pwck.c
+@@ -609,7 +609,7 @@ static void check_pw_file (int *errors, bool *changed)
+ 					sp.sp_inact  = -1;
+ 					sp.sp_expire = -1;
+ 					sp.sp_flag   = SHADOW_SP_FLAG_UNSET;
+-					sp.sp_lstchg = (long) time ((time_t *) 0) / SCALE;
++					sp.sp_lstchg = (long) gettime () / SCALE;
+ 					if (0 == sp.sp_lstchg) {
+ 						/* Better disable aging than
+ 						 * requiring a password change
+diff --git a/src/pwconv.c b/src/pwconv.c
+index 9c69fa131d8e..f932f266c59c 100644
+--- a/src/pwconv.c
++++ b/src/pwconv.c
+@@ -267,7 +267,7 @@ int main (int argc, char **argv)
+ 			spent.sp_flag   = SHADOW_SP_FLAG_UNSET;
+ 		}
+ 		spent.sp_pwdp = pw->pw_passwd;
+-		spent.sp_lstchg = (long) time ((time_t *) 0) / SCALE;
++		spent.sp_lstchg = (long) gettime () / SCALE;
+ 		if (0 == spent.sp_lstchg) {
+ 			/* Better disable aging than requiring a password
+ 			 * change */
+-- 
+2.17.1
+
diff --git a/meta/recipes-extended/shadow/files/0002-gettime-Use-secure_getenv-over-getenv.patch b/meta/recipes-extended/shadow/files/0002-gettime-Use-secure_getenv-over-getenv.patch
new file mode 100644
index 0000000..8c8234d
--- /dev/null
+++ b/meta/recipes-extended/shadow/files/0002-gettime-Use-secure_getenv-over-getenv.patch
@@ -0,0 +1,71 @@
+From 3d921155e0a761f61c8f1ec37328724aee1e2eda Mon Sep 17 00:00:00 2001
+From: Chris Lamb <chris@chris-lamb.co.uk>
+Date: Sun, 31 Mar 2019 15:59:45 +0100
+Subject: [PATCH 2/2] gettime: Use secure_getenv over getenv.
+
+Upstream-Status: Backport
+Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
+---
+ README            | 1 +
+ configure.ac      | 3 +++
+ lib/defines.h     | 6 ++++++
+ libmisc/gettime.c | 2 +-
+ 4 files changed, 11 insertions(+), 1 deletion(-)
+
+diff --git a/README b/README
+index 952ac5787f06..26cfff1e8fa8 100644
+--- a/README
++++ b/README
+@@ -51,6 +51,7 @@ Brian R. Gaeke <brg@dgate.org>
+ Calle Karlsson <ckn@kash.se>
+ Chip Rosenthal <chip@unicom.com>
+ Chris Evans <lady0110@sable.ox.ac.uk>
++Chris Lamb <chris@chris-lamb.co.uk>
+ Cristian Gafton <gafton@sorosis.ro>
+ Dan Walsh <dwalsh@redhat.com>
+ Darcy Boese <possum@chardonnay.niagara.com>
+diff --git a/configure.ac b/configure.ac
+index da236722766b..a738ad662cc3 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -110,6 +110,9 @@ AC_REPLACE_FUNCS(sgetgrent sgetpwent sgetspent)
+ AC_REPLACE_FUNCS(snprintf strcasecmp strdup strerror strstr)
+ 
+ AC_CHECK_FUNC(setpgrp)
++AC_CHECK_FUNC(secure_getenv, [AC_DEFINE(HAS_SECURE_GETENV,
++                                        1,
++                                        [Defined to 1 if you have the declaration of 'secure_getenv'])])
+ 
+ if test "$ac_cv_header_shadow_h" = "yes"; then
+ 	AC_CACHE_CHECK(for working shadow group support,
+diff --git a/lib/defines.h b/lib/defines.h
+index cded1417fd12..2fb1b56eca6b 100644
+--- a/lib/defines.h
++++ b/lib/defines.h
+@@ -382,4 +382,10 @@ extern char *strerror ();
+ # endif
+ #endif
+ 
++#ifdef HAVE_SECURE_GETENV
++#  define shadow_getenv(name) secure_getenv(name)
++# else
++#  define shadow_getenv(name) getenv(name)
++#endif
++
+ #endif				/* _DEFINES_H_ */
+diff --git a/libmisc/gettime.c b/libmisc/gettime.c
+index 53eaf51670bb..0e25a4b75061 100644
+--- a/libmisc/gettime.c
++++ b/libmisc/gettime.c
+@@ -52,7 +52,7 @@
+ 	unsigned long long epoch;
+ 
+ 	fallback = time (NULL);
+-	source_date_epoch = getenv ("SOURCE_DATE_EPOCH");
++	source_date_epoch = shadow_getenv ("SOURCE_DATE_EPOCH");
+ 
+ 	if (!source_date_epoch)
+ 		return fallback;
+-- 
+2.17.1
+
diff --git a/meta/recipes-extended/shadow/shadow.inc b/meta/recipes-extended/shadow/shadow.inc
index 4de21ac..831751d 100644
--- a/meta/recipes-extended/shadow/shadow.inc
+++ b/meta/recipes-extended/shadow/shadow.inc
@@ -11,6 +11,8 @@ DEPENDS = "virtual/crypt"
 UPSTREAM_CHECK_URI = "https://github.com/shadow-maint/shadow/releases"
 SRC_URI = "https://github.com/shadow-maint/shadow/releases/download/${PV}/${BP}.tar.gz \
            file://shadow-4.1.3-dots-in-usernames.patch \
+           file://0001-Make-the-sp_lstchg-shadow-field-reproducible-re.-71.patch  \
+           file://0002-gettime-Use-secure_getenv-over-getenv.patch \
            ${@bb.utils.contains('PACKAGECONFIG', 'pam', '${PAM_SRC_URI}', '', d)} \
            "
 
-- 
2.7.4

[PATCH 03/15] oeqa/runner: Fix subunit setupClass/setupModule failure handling

[Armin Kuster]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

The string format for subunit setupClass/setupModule failures is slightly
different, tweak the regex to correctly handle both cases.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/lib/oeqa/core/runner.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py
index 69743a4..efd981d 100644
--- a/meta/lib/oeqa/core/runner.py
+++ b/meta/lib/oeqa/core/runner.py
@@ -106,13 +106,13 @@ class OETestResult(_TestResult):
 
                 # When fails at module or class level the class name is passed as string
                 # so figure out to see if match
-                m = re.search(r"^setUpModule \((?P<module_name>.*)\)$", scase_str)
+                m = re.search(r"^setUpModule \((?P<module_name>.*)\).*$", scase_str)
                 if m:
                     if case.__class__.__module__ == m.group('module_name'):
                         found = True
                         break
 
-                m = re.search(r"^setUpClass \((?P<class_name>.*)\)$", scase_str)
+                m = re.search(r"^setUpClass \((?P<class_name>.*)\).*$", scase_str)
                 if m:
                     class_name = "%s.%s" % (case.__class__.__module__,
                                             case.__class__.__name__)
-- 
2.7.4

[PATCH 04/15] oeqa/concurrenttest: Patch subunit module to handle classSetup failures

[Armin Kuster]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Currently setupClass errors were not being mapped back to the failing tests
and they were hence being marked as UNKNOWN and the test statistics were
inaccurate.

This is because whilst the errors were being encoded into the test results
stream, the decoder doesn't cope with an error outside a testStart event.

We patch in an addError handler to the outsideTest parser so that this
does get handled in a way similar to the non-concurrent case.

It would be nice if we didn't have to do this but there doesn't seem
to be any other way to fix this other than forking subunit.

We also make a minor change so another of our changes can cope with
tests without a start time.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/lib/oeqa/core/utils/concurrencytest.py | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/core/utils/concurrencytest.py b/meta/lib/oeqa/core/utils/concurrencytest.py
index e050818..df8d5d0 100644
--- a/meta/lib/oeqa/core/utils/concurrencytest.py
+++ b/meta/lib/oeqa/core/utils/concurrencytest.py
@@ -19,6 +19,7 @@ import testtools
 import threading
 import time
 import io
+import subunit
 
 from queue import Queue
 from itertools import cycle
@@ -51,10 +52,11 @@ class BBThreadsafeForwardingResult(ThreadsafeForwardingResult):
     def _add_result_with_semaphore(self, method, test, *args, **kwargs):
         self.semaphore.acquire()
         try:
-            self.result.starttime[test.id()] = self._test_start.timestamp()
-            self.result.threadprogress[self.threadnum].append(test.id())
-            totalprogress = sum(len(x) for x in self.result.threadprogress.values())
-            self.result.progressinfo[test.id()] = "%s: %s/%s %s/%s (%ss) (%s)" % (
+            if self._test_start:
+                self.result.starttime[test.id()] = self._test_start.timestamp()
+                self.result.threadprogress[self.threadnum].append(test.id())
+                totalprogress = sum(len(x) for x in self.result.threadprogress.values())
+                self.result.progressinfo[test.id()] = "%s: %s/%s %s/%s (%ss) (%s)" % (
                     self.threadnum,
                     len(self.result.threadprogress[self.threadnum]),
                     self.totalinprocess,
@@ -67,6 +69,23 @@ class BBThreadsafeForwardingResult(ThreadsafeForwardingResult):
         super(BBThreadsafeForwardingResult, self)._add_result_with_semaphore(method, test, *args, **kwargs)
 
 #
+# We have to patch subunit since it doesn't understand how to handle addError
+# outside of a running test case. This can happen if classSetUp() fails
+# for a class of tests. This unfortunately has horrible internal knowledge.
+#
+def outSideTestaddError(self, offset, line):
+    """An 'error:' directive has been read."""
+    test_name = line[offset:-1].decode('utf8')
+    self.parser._current_test = subunit.RemotedTestCase(test_name)
+    self.parser.current_test_description = test_name
+    self.parser._state = self.parser._reading_error_details
+    self.parser._reading_error_details.set_simple()
+    self.parser.subunitLineReceived(line)
+
+subunit._OutSideTest.addError = outSideTestaddError
+
+
+#
 # A dummy structure to add to io.StringIO so that the .buffer object
 # is available and accepts writes. This allows unittest with buffer=True
 # to interact ok with subunit which wants to access sys.stdout.buffer.
-- 
2.7.4

[PATCH 05/15] bitbake.conf: set NO_RECOMMENDATIONS with weak assignment

[Armin Kuster]

From: Kai Kang <kai.kang@windriver.com>

Set NO_RECOMMENDATIONS with weak assignment then it could be set a
default value somewhere else.

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index a8bafd2..c996301 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -845,7 +845,7 @@ COMBINED_FEATURES[vardeps] += "DISTRO_FEATURES MACHINE_FEATURES"
 SERIAL_CONSOLE ??= ""
 SERIAL_CONSOLES ??= "${@d.getVar('SERIAL_CONSOLE').replace(' ', ';')}"
 
-NO_RECOMMENDATIONS ?= ""
+NO_RECOMMENDATIONS ??= ""
 BAD_RECOMMENDATIONS ?= ""
 
 # Make sure MACHINE isn't exported
-- 
2.7.4

[PATCH 06/15] openssh: recommend rng-tools with sshd

[Armin Kuster]

From: Mikko Rapeli <mikko.rapeli@bmw.de>

Since openssl 1.1.1 and openssh which uses it, sshd
startup is delayed. The delays range from few seconds
to minutes and even to hours. The delays are visible
in host keys generation and when sshd process is started
in response to incoming TCP connection but is failing
to provide SSH version string and clients or tests time out.

In all cases traces show that sshd is waiting for getentropy()
system call to return from Linux kernel, which returns only
after kernel side random number pool is initialized. The pool
is initialized via various entropy source which may be
missing on embedded development boards or via rngd from
rng-tools package from userspace. HW random number generation
and kernel support help but rngd is till needed to feed that data
back to the Linux kernel.

Example from an NXP imx8 board shows that kernel random number pool
initialization can take over 400 seconds without rngd,
and with rngd it is initialized at around 4 seconds after boot.
The completion of initialization is visible in kernel dmesg with line
"random: crng init done".

More details are available from:

 * https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=912087
 * https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=897572
 * https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=43838a23a05fbd13e47d750d3dfd77001536dd33
 * http://www.man7.org/linux/man-pages/man2/getrandom.2.html

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Cc: Mark Hatle <mark.hatle@windriver.com>
Cc: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Cc: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/recipes-connectivity/openssh/openssh_7.9p1.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
index b971b2b..976bcc5 100644
--- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
+++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
@@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
 
 RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
 RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
+RRECOMMENDS_${PN}-sshd += "rng-tools"
 RDEPENDS_${PN}-ptest += "${PN}-sftp ${PN}-misc ${PN}-sftp-server make sed"
 
 RPROVIDES_${PN}-ssh = "ssh"
-- 
2.7.4

[PATCH 07/15] oeqa/sdk: use bash to execute SDK test commands

[Armin Kuster]

From: Mikko Rapeli <mikko.rapeli@bmw.de>

The commands only work with with bash. If /bin/sh is
dash like in Debian, the command execution fails with
errors like:

Standard Output: /bin/sh: 5: export: --sysroot: bad variable name

and all SDK tests fail.

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/lib/oeqa/sdk/case.py                  | 2 +-
 meta/lib/oeqa/sdk/utils/sdkbuildproject.py | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
index d8611c8..5334237 100644
--- a/meta/lib/oeqa/sdk/case.py
+++ b/meta/lib/oeqa/sdk/case.py
@@ -9,7 +9,7 @@ from oeqa.core.case import OETestCase
 class OESDKTestCase(OETestCase):
     def _run(self, cmd):
         return subprocess.check_output(". %s > /dev/null; %s;" % \
-                (self.tc.sdk_env, cmd), shell=True,
+                (self.tc.sdk_env, cmd), shell=True, executable="/bin/bash",
                 stderr=subprocess.STDOUT, universal_newlines=True)
 
     def fetch(self, workdir, dl_dir, url, archive=None):
diff --git a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
index 6fed73e..eafbd7a 100644
--- a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
+++ b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
@@ -42,7 +42,8 @@ class SDKBuildProject(BuildProject):
     def _run(self, cmd):
         self.log("Running . %s; " % self.sdkenv + cmd)
         try:
-            output = subprocess.check_output(". %s; " % self.sdkenv + cmd, shell=True, stderr=subprocess.STDOUT)
+            output = subprocess.check_output(". %s; " % self.sdkenv + cmd, shell=True,
+                                             executable='/bin/bash', stderr=subprocess.STDOUT)
         except subprocess.CalledProcessError as exc:
             print(exc.output.decode('utf-8'))
             return exc.returncode
-- 
2.7.4

[PATCH 08/15] run-postinsts: Fix full execution of scripts at first boot

[Armin Kuster]

From: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>

run-postinsts runs a given set of scripts during the first boot of the
device, when one of these scripts prints something to stdout (isnt
daemonized correctly), since stdout is not available at that time,
the script execution immediately returns with an error (exit_group()),
this error causes the script to terminate all threads within the process,
causing undesired behavior since the script might still had to execute
some other code.

Replace eval built-in with (), since () executes in a subshell,
even if one of the scripts exits, all threads of that process will only
be within that session, this ensures other scripts meant to be run are
still run afterwards.

[YOCTO #13266]

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
index 95eff04..f84a7e1 100755
--- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
+++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
@@ -67,7 +67,7 @@ exec_postinst_scriptlets() {
 		echo "Running postinst $i..."
 		[ "$POSTINST_LOGGING" = "1" ] && eval echo "Running postinst $i..." $append_log
 		if [ -x $i ]; then
-			eval sh -c $i $append_log
+			(sh -c $i $append_log)
 			rm $i
 		else
 			echo "ERROR: postinst $i failed."
-- 
2.7.4

[PATCH 09/15] oeqa/target/ssh: Replace suggogatepass with ignoring errors

[Armin Kuster]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

We continued to see encoding problems with ssh commands run in oeqa. After much
research the conclusion was we should use ignore the errors since some occasional bad
locale encoding is better than the unicode decoding issues we were seeing which crashed
large parts of tests.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/lib/oeqa/core/target/ssh.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/core/target/ssh.py b/meta/lib/oeqa/core/target/ssh.py
index 2adbb30..0c09ddf 100644
--- a/meta/lib/oeqa/core/target/ssh.py
+++ b/meta/lib/oeqa/core/target/ssh.py
@@ -207,7 +207,7 @@ def SSHCall(command, logger, timeout=None, **opts):
                 logger.debug('time: %s, endtime: %s' % (time.time(), endtime))
                 try:
                     if select.select([process.stdout], [], [], 5)[0] != []:
-                        reader = codecs.getreader('utf-8')(process.stdout, 'surrogatepass')
+                        reader = codecs.getreader('utf-8')(process.stdout, 'ignore')
                         data = reader.read(1024, 4096)
                         if not data:
                             process.stdout.close()
@@ -234,7 +234,7 @@ def SSHCall(command, logger, timeout=None, **opts):
                 output += lastline
 
         else:
-            output = process.communicate()[0].decode("utf-8", errors='surrogatepass')
+            output = process.communicate()[0].decode('utf-8', errors='ignore')
             logger.debug('Data from SSH call: %s' % output.rstrip())
 
     options = {
-- 
2.7.4

[PATCH 10/15] oeqa/targetcontrol.py: fix qemuparams not work in runqemu with launch_cmd

[Armin Kuster]

From: Hongxu Jia <hongxu.jia@windriver.com>

As runqemu with launch_cmd means directly run the command, don't need set
rootfs or env vars.

Since commit [a847dd7202 runqemu: Let qemuparams override default settings]
applied in oe-core, if launch_cmd contains "qemuparams='***'", it does not
work, which is overridden by latter qemuparams="-serial tcp:127.0.0.1" in
QemuRunner.launch();

So we set qemuparams as a parameter in runqemu, the fix makes it work

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/lib/oeqa/targetcontrol.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index 1868ad3..b98b183 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -159,7 +159,7 @@ class QemuTarget(BaseTarget):
 
     def start(self, params=None, ssh=True, extra_bootparams='', runqemuparams='', launch_cmd='', discard_writes=True):
         if launch_cmd:
-            start = self.runner.launch(get_ip=ssh, launch_cmd=launch_cmd)
+            start = self.runner.launch(get_ip=ssh, launch_cmd=launch_cmd, qemuparams=params)
         else:
             start = self.runner.start(params, get_ip=ssh, extra_bootparams=extra_bootparams, runqemuparams=runqemuparams, discard_writes=discard_writes)
 
-- 
2.7.4

[PATCH 11/15] wic/bootimg-efi: replace hardcoded volume name with label

[Armin Kuster]

From: Chee Yang Lee <chee.yang.lee@intel.com>

volume name should refer to --label in .wks.
Replace the hardcoded volume name  with label.
set "ESP" as default name when no lable specified.

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 scripts/lib/wic/plugins/source/bootimg-efi.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 83a7e18..19328ce 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -258,8 +258,10 @@ class BootimgEFIPlugin(SourcePlugin):
         # dosfs image, created by mkdosfs
         bootimg = "%s/boot.img" % cr_workdir
 
-        dosfs_cmd = "mkdosfs -n efi -i %s -C %s %d" % \
-                    (part.fsuuid, bootimg, blocks)
+        label = part.label if part.label else "ESP"
+
+        dosfs_cmd = "mkdosfs -n %s -i %s -C %s %d" % \
+                    (label, part.fsuuid, bootimg, blocks)
         exec_native_cmd(dosfs_cmd, native_sysroot)
 
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
-- 
2.7.4

[PATCH 12/15] glib-networking: add PACKAGECONFIG for openssl

[Armin Kuster]

From: Martin Jansa <martin.jansa@gmail.com>

* gnutls from meta-gplv2 isn't new enough for this version of glib-networking:
  meson.build:74:0: ERROR:  Invalid version of dependency, need 'gnutls' ['>= 3.4.6'] found '3.3.30'

* disabling gnutls PACKAGECONFIG isn't an option either:
  meson.build:131:2: ERROR:  Problem encountered: No TLS backends enabled. Please enable at least one TLS backend

* add an option to replace gnutls with openssl

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/recipes-core/glib-networking/glib-networking_2.60.1.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/glib-networking/glib-networking_2.60.1.bb b/meta/recipes-core/glib-networking/glib-networking_2.60.1.bb
index 09af229..e2b4cff 100644
--- a/meta/recipes-core/glib-networking/glib-networking_2.60.1.bb
+++ b/meta/recipes-core/glib-networking/glib-networking_2.60.1.bb
@@ -15,6 +15,7 @@ SRC_URI[archive.sha256sum] = "d71c6b2faa5ac29100314f08a1be020a2afd0291f025614c0a
 PACKAGECONFIG ??= "gnutls"
 
 PACKAGECONFIG[gnutls] = "-Dgnutls=enabled,-Dgnutls=disabled,gnutls"
+PACKAGECONFIG[openssl] = "-Dopenssl=enabled,-Dopenssl=disabled,openssl"
 PACKAGECONFIG[libproxy] = "-Dlibproxy=enabled,-Dlibproxy=disabled,libproxy"
 
 EXTRA_OEMESON = "-Dgnome_proxy=disabled"
-- 
2.7.4

[PATCH 13/15] perf: make sure that the tools/include/uapi/asm-generic directory exists

[Armin Kuster]

From: Martin Jansa <martin.jansa@gmail.com>

* before trying to copy unistd.h into it

* older kernels don't have uapi in tools/include and do_configure fails
  with:
  DEBUG: Executing shell function do_configure
  cp: cannot create regular file '.../perf/1.0-r9/perf-1.0/tools/include/uapi/asm-generic/unistd.h':
  No such file or directory
  WARNING: exit code 1 from a shell command.

* tools/include/uapi was added in kernel 4.8 with
  https://github.com/torvalds/linux/commit/c4b6014e8bb0c8d47fe5c71ebc604f31091e5d3f
  tools: Add copy of perf_event.h to tools/include/linux/

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/recipes-kernel/perf/perf.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 546076f..4e603f1 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -238,7 +238,7 @@ do_configure_prepend () {
 
     # unistd.h can be out of sync between libc-headers and the captured version in the perf source
     # so we copy it from the sysroot unistd.h to the perf unistd.h
-    cp ${STAGING_INCDIR}/asm-generic/unistd.h ${S}/tools/include/uapi/asm-generic/unistd.h
+    install -D -m0644 ${STAGING_INCDIR}/asm-generic/unistd.h ${S}/tools/include/uapi/asm-generic/unistd.h
 }
 
 python do_package_prepend() {
-- 
2.7.4

[PATCH 14/15] core-image-sato-sdk-ptest: Tweak size to stay within 4GB limit

[Armin Kuster]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Adding the valgrind debug symbol information caused the genericx86-64 image to
overflow the 4GB boundary. Tweak the sizes to avoid autobuilder failures yet
leave enough space all the tests still run successfully.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/recipes-sato/images/core-image-sato-sdk-ptest.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-sato/images/core-image-sato-sdk-ptest.bb b/meta/recipes-sato/images/core-image-sato-sdk-ptest.bb
index 5739599..22df057 100644
--- a/meta/recipes-sato/images/core-image-sato-sdk-ptest.bb
+++ b/meta/recipes-sato/images/core-image-sato-sdk-ptest.bb
@@ -6,10 +6,10 @@ IMAGE_FEATURES += "ptest-pkgs"
 
 # This image is sufficiently large (~1.8GB) that we need to be careful that it fits in a live
 # image (which has a 4GB limit), so nullify the overhead factor (1.3x out of the
-# box) and explicitly add just 1300MB.
+# box) and explicitly add just 1200MB.
 # strace-ptest in particular needs more than 500MB
 IMAGE_OVERHEAD_FACTOR = "1.0"
-IMAGE_ROOTFS_EXTRA_SPACE = "1324288"
+IMAGE_ROOTFS_EXTRA_SPACE = "1224288"
 
 # ptests need more memory than standard to avoid the OOM killer
 QB_MEM = "-m 1024"
-- 
2.7.4

[PATCH 15/15] kernel.bbclass: adjust a condition checking

[Armin Kuster]

From: Ming Liu <liu.ming50@gmail.com>

do_bundle_initramfs does not have to depend on
${INITRAMFS_IMAGE}:do_image_complete if INITRAMFS_IMAGE_BUNDLE is not
set.

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 meta/classes/kernel.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index b346a60..437b8c7 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -95,7 +95,8 @@ python __anonymous () {
         d.setVar('ALLOW_EMPTY_%s-image-%s' % (kname, typelower), '1')
 
     image = d.getVar('INITRAMFS_IMAGE')
-    if image:
+    image_bundle = d.getVar('INITRAMFS_IMAGE_BUNDLE')
+    if image and bb.utils.to_boolean(image_bundle, False):
         d.appendVarFlag('do_bundle_initramfs', 'depends', ' ${INITRAMFS_IMAGE}:do_image_complete')
 
     # NOTE: setting INITRAMFS_TASK is for backward compatibility
-- 
2.7.4

Re: [PATCH 01/15] gdb: Fix aarch64 build with musl

[Adrian Bunk]

This is already in warrior.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH 02/15] shadow: Backport last change reproducibility

[Adrian Bunk]

On Mon, May 20, 2019 at 06:43:04PM -0700, Armin Kuster wrote:
>...
> ++#ifdef HAVE_SECURE_GETENV
> ++#  define shadow_getenv(name) secure_getenv(name)
> ++# else
> ++#  define shadow_getenv(name) getenv(name)
> ++#endif
>...

If I understand the upstream discussion correctly,
combined with the other changes this does create
a security vulnerability in musl builds.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed