[PATCH 00/20] kernel/kernel-yocto: consolidated pull request

[PATCH 00/20] kernel/kernel-yocto: consolidated pull request

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Richard,

There's a few routine -stable updates at the start of this, and then some
more painful (and RFC) commits.

As part of this series, we fix one pkg-config issue that was preventing
x86 from doing stack valildation during build. At one point it was
increasing my build times signficantly, but that seems to have gone
away now. If we see something like this on the AB, we'll have to keep
the fix, but disable it by default.

The next big chunk of changes are the ones we talked about during the
tech call today. The 5.16 kernel has introduced mandatory dtschema
checking for device trees built through the kernel.

That checking pulls in a reasonably large amount of dependencies to
the kernel build: libyaml, dtschema, jsonschema, etc.

We weren't able to even run the validation at all, due to the kernel
calling pkg-config directly to check for libyaml, which is the target
pkgconfig and failing on missing dependencies. Rather than patching
the kernel, I've finally given in and made the do_compile task set
the PKG_CONFIG variables to point to our native-sysroot. Since the
kenrel doesn't use pkgconfig for target decisions, and has quite
a few calls directly to pkgconfig, this is the best short term way
to get all those checks working.

Once we are past yaml, we need to create the dtschema recipe, and
bring in the dependencies from meta-python. I've put myself down as
the maintainer for all the new recipes, but otherwise, I haven't
changed or modified them.

BUT! there are scenarios where a non-conformant dts file is being
built, or we just don't want those dependencies in the built. To
work around that, I took the suggestion from this morning and
created a python3-dtschema-wrappers recipe. It contains three
scripts of the same name as the ones from the real package. When
put in the dependency for a kernel build, it can pass the dtschema
steps.

Finally, I created a reference PACKAGECONFIG for linux-yocto-dev
to enable/disable the main package and wrappers (the wrappers are
used by default). I haven't put this into kernel.bbclass yet, since
it is version specific, and it needs to soak for a bit. But if
there's demand, it is something that can be moved to kernel.bclass
(along with a few other version specific dependencies).

Note: I haven't done an AB run with these yet, so I could be
missing some python dependencies, etc, but I wanted to get this
out to the list in case others are running into this with
v5.16+.

Cheers,

Bruce


The following changes since commit 60409917ea3e7353f5b7ee831fe4ffaff0957d94:

  glibc-tests: Do not set SYSTEMD_SERVICE (2021-11-23 14:04:02 +0000)

are available in the Git repository at:

  git://git.yoctoproject.org/poky-contrib zedd/kernel
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (18):
  linux-yocto/5.15: update to v5.15.2
  linux-yocto/5.14: update to v5.14.18
  linux-yocto/5.10: update to v5.10.79
  kernel-yocto: conditionally enable stack protection checking on x86-64
  linux-yocto-dev: bump to v5.16+
  kernel: export native PKGCONFIG variables
  python: introduce python3-dtschema
  python: import jsonpointer from meta-python
  python: import jsonschema from meta-python
  python: import idna from meta-python
  python: import rfc3339-validator from meta-python
  python: import rfc3986-validator from meta-python
  python: import webcolors from meta-python
  python: import ruamel-yaml from meta-python
  maintainers: update for kernel dtschema checking dependencies
  linux-yocto: export pkgconfig variables to devshell
  kernel: introduce python3-dtschema-wrapper
  linux-yocto-dev: introduce dt-validation PACKAGECONFIG

Max Krummenacher (2):
  perf: sort-pmuevents: don't drop elements
  perf: sort-pmuevents: allow for additional type qualifiers and storage
    class

 meta/classes/kernel.bbclass                   |  7 +++
 meta/conf/distro/include/maintainers.inc      |  9 ++++
 ...e-pytest-runner-to-test_requirements.patch | 32 +++++++++++++
 .../python/python3-dtschema_2021.10.bb        | 15 ++++++
 .../python/python3-idna_3.3.bb                | 19 ++++++++
 .../python/python3-jsonpointer/run-ptest      |  3 ++
 .../python/python3-jsonpointer_2.1.bb         | 26 ++++++++++
 .../python/python3-jsonschema_3.2.0.bb        | 48 +++++++++++++++++++
 .../python/python3-rfc3339-validator_0.1.4.bb | 20 ++++++++
 .../python/python3-rfc3986-validator_0.1.1.bb | 23 +++++++++
 .../python/python3-ruamel-yaml_0.17.16.bb     | 23 +++++++++
 .../python/python3-webcolors/run-ptest        |  3 ++
 .../python/python3-webcolors_1.11.1.bb        | 27 +++++++++++
 .../python3-dtschema-wrapper/dt-doc-validate  | 20 ++++++++
 .../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 ++++++++
 .../dtc/python3-dtschema-wrapper/dt-validate  | 20 ++++++++
 .../dtc/python3-dtschema-wrapper_2021.10.bb   | 17 +++++++
 meta/recipes-kernel/linux/linux-yocto-dev.bb  | 10 +++-
 .../linux/linux-yocto-rt_5.10.bb              |  6 +--
 .../linux/linux-yocto-rt_5.14.bb              |  6 +--
 .../linux/linux-yocto-rt_5.15.bb              |  6 +--
 .../linux/linux-yocto-tiny_5.10.bb            |  8 ++--
 .../linux/linux-yocto-tiny_5.14.bb            |  8 ++--
 .../linux/linux-yocto-tiny_5.15.bb            |  8 ++--
 meta/recipes-kernel/linux/linux-yocto.inc     | 13 +++++
 meta/recipes-kernel/linux/linux-yocto_5.10.bb | 24 +++++-----
 meta/recipes-kernel/linux/linux-yocto_5.14.bb | 26 +++++-----
 meta/recipes-kernel/linux/linux-yocto_5.15.bb | 26 +++++-----
 .../perf/perf/sort-pmuevents.py               | 28 ++++++-----
 29 files changed, 429 insertions(+), 72 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
 create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
 create mode 100644 meta/recipes-devtools/python/python3-idna_3.3.bb
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
 create mode 100644 meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
 create mode 100644 meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
 create mode 100644 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
 create mode 100644 meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
 create mode 100644 meta/recipes-devtools/python/python3-webcolors/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb

-- 
2.19.1

[PATCH 01/20] linux-yocto/5.15: update to v5.15.2

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Updating linux-yocto/5.15 to the latest korg -stable release that comprises
the following commits:

    7cc36c3e14ae Linux 5.15.2
    5dbe126056fb rsi: fix control-message timeout
    7d6f8d3bab72 media: staging/intel-ipu3: css: Fix wrong size comparison imgu_css_fw_init
    4787caef521c staging: r8188eu: fix memleak in rtw_wx_set_enc_ext
    8f60f9f6ee24 staging: rtl8192u: fix control-message timeouts
    9033490a3fef staging: r8712u: fix control-message timeout
    3a66e8a661a4 comedi: vmk80xx: fix bulk and interrupt message timeouts
    47b4636ebdbe comedi: vmk80xx: fix bulk-buffer overflow
    06ac746d57e6 comedi: vmk80xx: fix transfer-buffer overflows
    d6a727a681a3 comedi: ni_usb6501: fix NULL-deref in command paths
    20cebb8b620d comedi: dt9812: fix DMA buffers on stack
    e7fb722586a2 isofs: Fix out of bound access for corrupted isofs image
    a65c9afe9f2f staging: rtl8712: fix use-after-free in rtl8712_dl_fw
    a0041453ff9e btrfs: fix lzo_decompress_bio() kmap leakage
    d5dd3b44488b kfence: default to dynamic branch instead of static keys mode
    93ce7441001f kfence: always use static branches to guard kfence_alloc()
    5e57d171e2e6 binder: don't detect sender/target during buffer cleanup
    6e8813eadf8b binder: use cred instead of task for getsecid
    3f3c31dd0f8c binder: use cred instead of task for selinux checks
    ff1bd01f490b binder: use euid from cred instead of using task
    ec7c20d41714 Revert "proc/wchan: use printk format instead of lookup_symbol_name()"
    2e93afda0520 usb-storage: Add compatibility quirk flags for iODD 2531/2541
    ad5df979295b usb: musb: Balance list entry in musb_gadget_queue
    72a9bf9bb16a usb: gadget: Mark USB_FSL_QE broken on 64-bit
    adb1902a1246 usb: ehci: handshake CMD_RUN instead of STS_HALT
    aa8a82d6db0b Revert "x86/kvm: fix vcpu-id indexed array sizes"
    26ac7dec7ff0 KVM: x86: avoid warning with -Wbitwise-instead-of-logical

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../linux/linux-yocto-rt_5.15.bb              |  6 ++---
 .../linux/linux-yocto-tiny_5.15.bb            |  8 +++---
 meta/recipes-kernel/linux/linux-yocto_5.15.bb | 26 +++++++++----------
 3 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_5.15.bb b/meta/recipes-kernel/linux/linux-yocto-rt_5.15.bb
index 879f2be677..d5c11b8d33 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_5.15.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_5.15.bb
@@ -11,13 +11,13 @@ python () {
         raise bb.parse.SkipRecipe("Set PREFERRED_PROVIDER_virtual/kernel to linux-yocto-rt to enable it")
 }
 
-SRCREV_machine ?= "b9ff38638d2398b647629e768713eefb0fc6707a"
-SRCREV_meta ?= "5bcd50d6b0cde22583768dc1acf304259364ddbc"
+SRCREV_machine ?= "ada3579e989bfe75d8598ab993347157dc4611c2"
+SRCREV_meta ?= "e3682eb8fe7512597c75b009f51027534ab2f58d"
 
 SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;branch=${KBRANCH};name=machine \
            git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.15;destsuffix=${KMETA}"
 
-LINUX_VERSION ?= "5.15.1"
+LINUX_VERSION ?= "5.15.2"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_5.15.bb b/meta/recipes-kernel/linux/linux-yocto-tiny_5.15.bb
index 6035dc47f1..49e74252e9 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_5.15.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_5.15.bb
@@ -6,7 +6,7 @@ KCONFIG_MODE = "--allnoconfig"
 
 require recipes-kernel/linux/linux-yocto.inc
 
-LINUX_VERSION ?= "5.15.1"
+LINUX_VERSION ?= "5.15.2"
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
@@ -15,9 +15,9 @@ DEPENDS += "openssl-native util-linux-native"
 KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
-SRCREV_machine:qemuarm ?= "098dbfb0e6d95ae958e1003738fb0dc90a81691b"
-SRCREV_machine ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_meta ?= "5bcd50d6b0cde22583768dc1acf304259364ddbc"
+SRCREV_machine:qemuarm ?= "0daad4eb78782be97ce4d91f2d455ccc2d238991"
+SRCREV_machine ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_meta ?= "e3682eb8fe7512597c75b009f51027534ab2f58d"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto_5.15.bb b/meta/recipes-kernel/linux/linux-yocto_5.15.bb
index 0fd49d78cb..51259617a7 100644
--- a/meta/recipes-kernel/linux/linux-yocto_5.15.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_5.15.bb
@@ -13,17 +13,17 @@ KBRANCH:qemux86  ?= "v5.15/standard/base"
 KBRANCH:qemux86-64 ?= "v5.15/standard/base"
 KBRANCH:qemumips64 ?= "v5.15/standard/mti-malta64"
 
-SRCREV_machine:qemuarm ?= "47b7bd92b84e0ad136d45956e86039ad7307c892"
-SRCREV_machine:qemuarm64 ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_machine:qemumips ?= "e2dd4de2dfebf97237b87c414d0582016f64d5b1"
-SRCREV_machine:qemuppc ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_machine:qemuriscv64 ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_machine:qemuriscv32 ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_machine:qemux86 ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_machine:qemux86-64 ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_machine:qemumips64 ?= "066b9f8175fc7136c15768204ba71d3fddc2f4f8"
-SRCREV_machine ?= "5a8ec126c297dd9e410eb23685003f670f8f3d57"
-SRCREV_meta ?= "5bcd50d6b0cde22583768dc1acf304259364ddbc"
+SRCREV_machine:qemuarm ?= "e185d8ca70e102b1f99736b15f5e8ee1ab62d081"
+SRCREV_machine:qemuarm64 ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_machine:qemumips ?= "51982f22cf688d426ae919380cb19fcde3cd0d1a"
+SRCREV_machine:qemuppc ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_machine:qemuriscv64 ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_machine:qemuriscv32 ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_machine:qemux86 ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_machine:qemux86-64 ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_machine:qemumips64 ?= "95b7f92bb1eec8532be3a5e50726a0643ab2fae9"
+SRCREV_machine ?= "a81c52f3e09edca72d2b3da6cbab5d17769fa8cb"
+SRCREV_meta ?= "e3682eb8fe7512597c75b009f51027534ab2f58d"
 
 # set your preferred provider of linux-yocto to 'linux-yocto-upstream', and you'll
 # get the <version>/base branch, which is pure upstream -stable, and the same
@@ -31,7 +31,7 @@ SRCREV_meta ?= "5bcd50d6b0cde22583768dc1acf304259364ddbc"
 # normal PREFERRED_VERSION settings.
 BBCLASSEXTEND = "devupstream:target"
 DEFAULT_PREFERENCE:class-devupstream = "-1"
-SRCREV_machine:class-devupstream ?= "b6abb62daa5511c4a3eaa30cbdb02544d1f10fa2"
+SRCREV_machine:class-devupstream ?= "7cc36c3e14ae0af800a3a5d20cb17d0c168fc956"
 PN:class-devupstream = "linux-yocto-upstream"
 KBRANCH:class-devupstream = "v5.15/base"
 
@@ -42,7 +42,7 @@ SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRA
            git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.15;destsuffix=${KMETA}"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
-LINUX_VERSION ?= "5.15.1"
+LINUX_VERSION ?= "5.15.2"
 
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
 DEPENDS += "openssl-native util-linux-native"
-- 
2.19.1

[PATCH 02/20] linux-yocto/5.14: update to v5.14.18

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Updating linux-yocto/5.14 to the latest korg -stable release that comprises
the following commits:

    a0265dd8262d Linux 5.14.18
    1379769b0b46 rsi: fix control-message timeout
    eda57a310234 media: staging/intel-ipu3: css: Fix wrong size comparison imgu_css_fw_init
    5013f2aaf959 staging: rtl8192u: fix control-message timeouts
    f3422d1fceba staging: r8712u: fix control-message timeout
    373ac8c59414 comedi: vmk80xx: fix bulk and interrupt message timeouts
    7b0e35618932 comedi: vmk80xx: fix bulk-buffer overflow
    278484ae9329 comedi: vmk80xx: fix transfer-buffer overflows
    4a9d43cb5d5f comedi: ni_usb6501: fix NULL-deref in command paths
    3ac273d154d6 comedi: dt9812: fix DMA buffers on stack
    b2fa1f52d22c isofs: Fix out of bound access for corrupted isofs image
    befd23bd3b17 staging: rtl8712: fix use-after-free in rtl8712_dl_fw
    bec32c40e438 binder: don't detect sender/target during buffer cleanup
    68abe9aefc40 binder: use cred instead of task for getsecid
    46088365bab7 binder: use cred instead of task for selinux checks
    fbb106e79524 binder: use euid from cred instead of using task
    f9b4ef2504bb Revert "proc/wchan: use printk format instead of lookup_symbol_name()"
    4674de4e2734 usb-storage: Add compatibility quirk flags for iODD 2531/2541
    6be11f54f5d4 usb: musb: Balance list entry in musb_gadget_queue
    d7f4ffba71d0 usb: gadget: Mark USB_FSL_QE broken on 64-bit
    d3c7daab289d usb: ehci: handshake CMD_RUN instead of STS_HALT
    a4cdb4c9c453 Revert "x86/kvm: fix vcpu-id indexed array sizes"
    2f63111ab86f KVM: x86: avoid warning with -Wbitwise-instead-of-logical
    f468cbedb2a7 ALSA: pci: cs46xx: Fix set up buffer type properly
    186155ac272e ALSA: pcm: Check mmap capability of runtime dma buffer at first

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../linux/linux-yocto-rt_5.14.bb              |  6 ++---
 .../linux/linux-yocto-tiny_5.14.bb            |  8 +++---
 meta/recipes-kernel/linux/linux-yocto_5.14.bb | 26 +++++++++----------
 3 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_5.14.bb b/meta/recipes-kernel/linux/linux-yocto-rt_5.14.bb
index 7e02f83d51..a328c6211f 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_5.14.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_5.14.bb
@@ -11,13 +11,13 @@ python () {
         raise bb.parse.SkipRecipe("Set PREFERRED_PROVIDER_virtual/kernel to linux-yocto-rt to enable it")
 }
 
-SRCREV_machine ?= "672665c11dd86abd71bbad0537e6388c6a5a84ca"
-SRCREV_meta ?= "b3ac9c40a22d6b00545b7ce51ef6dfb0cb9d2933"
+SRCREV_machine ?= "b9d37d8650cd3787d60516496f60178c51564a59"
+SRCREV_meta ?= "4e79d0b09290482ffb891e6b4acb3c7447b9c94d"
 
 SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;branch=${KBRANCH};name=machine \
            git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.14;destsuffix=${KMETA}"
 
-LINUX_VERSION ?= "5.14.17"
+LINUX_VERSION ?= "5.14.18"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_5.14.bb b/meta/recipes-kernel/linux/linux-yocto-tiny_5.14.bb
index 9ad9549554..98947900a4 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_5.14.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_5.14.bb
@@ -6,7 +6,7 @@ KCONFIG_MODE = "--allnoconfig"
 
 require recipes-kernel/linux/linux-yocto.inc
 
-LINUX_VERSION ?= "5.14.17"
+LINUX_VERSION ?= "5.14.18"
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
@@ -15,9 +15,9 @@ DEPENDS += "openssl-native util-linux-native"
 KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
-SRCREV_machine:qemuarm ?= "566227d5b0913910467a7736d78cad37e60217f8"
-SRCREV_machine ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_meta ?= "b3ac9c40a22d6b00545b7ce51ef6dfb0cb9d2933"
+SRCREV_machine:qemuarm ?= "83004545c9bb5e3ad666b3ffa9ad723eb795881a"
+SRCREV_machine ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_meta ?= "4e79d0b09290482ffb891e6b4acb3c7447b9c94d"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto_5.14.bb b/meta/recipes-kernel/linux/linux-yocto_5.14.bb
index f034f36d45..673c03d82e 100644
--- a/meta/recipes-kernel/linux/linux-yocto_5.14.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_5.14.bb
@@ -13,17 +13,17 @@ KBRANCH:qemux86  ?= "v5.14/standard/base"
 KBRANCH:qemux86-64 ?= "v5.14/standard/base"
 KBRANCH:qemumips64 ?= "v5.14/standard/mti-malta64"
 
-SRCREV_machine:qemuarm ?= "eef94a78151c2227eba9a8468ac611d9507482b5"
-SRCREV_machine:qemuarm64 ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_machine:qemumips ?= "607d94618b35382f38ab2bc0d37494372897ae4d"
-SRCREV_machine:qemuppc ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_machine:qemuriscv64 ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_machine:qemuriscv32 ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_machine:qemux86 ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_machine:qemux86-64 ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_machine:qemumips64 ?= "9ed182ebbcd4c77b8f99439d2feb6c4591148324"
-SRCREV_machine ?= "35888b3a9a222963b856c960e8f0c72c2de18d4a"
-SRCREV_meta ?= "b3ac9c40a22d6b00545b7ce51ef6dfb0cb9d2933"
+SRCREV_machine:qemuarm ?= "992053c4ec826932d9256ea4e1ad8686e4ea5375"
+SRCREV_machine:qemuarm64 ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_machine:qemumips ?= "e362a5e68da1cca6ad77a27cbda02116edc9864d"
+SRCREV_machine:qemuppc ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_machine:qemuriscv64 ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_machine:qemuriscv32 ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_machine:qemux86 ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_machine:qemux86-64 ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_machine:qemumips64 ?= "90b773a3236f213b62d2d4ab23cca9c695bc8383"
+SRCREV_machine ?= "8411a30747a2750c9d4dbe24631958bd00d3157e"
+SRCREV_meta ?= "4e79d0b09290482ffb891e6b4acb3c7447b9c94d"
 
 # set your preferred provider of linux-yocto to 'linux-yocto-upstream', and you'll
 # get the <version>/base branch, which is pure upstream -stable, and the same
@@ -31,7 +31,7 @@ SRCREV_meta ?= "b3ac9c40a22d6b00545b7ce51ef6dfb0cb9d2933"
 # normal PREFERRED_VERSION settings.
 BBCLASSEXTEND = "devupstream:target"
 DEFAULT_PREFERENCE:class-devupstream = "-1"
-SRCREV_machine:class-devupstream ?= "3dfa869cb79d60a2fe66efb4a11517ec7c89de16"
+SRCREV_machine:class-devupstream ?= "a0265dd8262de73457aaa3ce1c5938dc152b8085"
 PN:class-devupstream = "linux-yocto-upstream"
 KBRANCH:class-devupstream = "v5.14/base"
 
@@ -42,7 +42,7 @@ SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRA
            git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.14;destsuffix=${KMETA}"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
-LINUX_VERSION ?= "5.14.17"
+LINUX_VERSION ?= "5.14.18"
 
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
 DEPENDS += "openssl-native util-linux-native"
-- 
2.19.1

[PATCH 03/20] linux-yocto/5.10: update to v5.10.79

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Updating linux-yocto/5.10 to the latest korg -stable release that comprises
the following commits:

    bd816c278316 Linux 5.10.79
    62424fe4c2cf rsi: fix control-message timeout
    8971158af1e0 media: staging/intel-ipu3: css: Fix wrong size comparison imgu_css_fw_init
    1cf43e928954 staging: rtl8192u: fix control-message timeouts
    9963ba5b9d49 staging: r8712u: fix control-message timeout
    844b02496eac comedi: vmk80xx: fix bulk and interrupt message timeouts
    b7fd7f3387f0 comedi: vmk80xx: fix bulk-buffer overflow
    33d7a470730d comedi: vmk80xx: fix transfer-buffer overflows
    ef143dc0c3de comedi: ni_usb6501: fix NULL-deref in command paths
    786f5b034504 comedi: dt9812: fix DMA buffers on stack
    86d4aedcbc69 isofs: Fix out of bound access for corrupted isofs image
    c430094541a8 staging: rtl8712: fix use-after-free in rtl8712_dl_fw
    ab4af56ae250 printk/console: Allow to disable console output by using console="" or console=null
    07d1db141e47 binder: don't detect sender/target during buffer cleanup
    42681b90c4db usb-storage: Add compatibility quirk flags for iODD 2531/2541
    1309753b7841 usb: musb: Balance list entry in musb_gadget_queue
    27409143122f usb: gadget: Mark USB_FSL_QE broken on 64-bit
    94e5305a3816 usb: ehci: handshake CMD_RUN instead of STS_HALT
    a8db6fd04d58 Revert "x86/kvm: fix vcpu-id indexed array sizes"
    ecf58653f1e4 KVM: x86: avoid warning with -Wbitwise-instead-of-logical

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../linux/linux-yocto-rt_5.10.bb              |  6 ++---
 .../linux/linux-yocto-tiny_5.10.bb            |  8 +++----
 meta/recipes-kernel/linux/linux-yocto_5.10.bb | 24 +++++++++----------
 3 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_5.10.bb b/meta/recipes-kernel/linux/linux-yocto-rt_5.10.bb
index f666ac0611..89ef0097fa 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_5.10.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_5.10.bb
@@ -11,13 +11,13 @@ python () {
         raise bb.parse.SkipRecipe("Set PREFERRED_PROVIDER_virtual/kernel to linux-yocto-rt to enable it")
 }
 
-SRCREV_machine ?= "12f6a7187b3c8abab5e139dbfdf7f58f265f4169"
-SRCREV_meta ?= "a0238f7f4f2222d08bb18147bb5e24cc877b0546"
+SRCREV_machine ?= "31ba5af055c8ec4f10e7d830f6303ca2384e345a"
+SRCREV_meta ?= "1a4cd99824c919ba17dc62935532f3748ef18469"
 
 SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;branch=${KBRANCH};name=machine \
            git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.10;destsuffix=${KMETA}"
 
-LINUX_VERSION ?= "5.10.78"
+LINUX_VERSION ?= "5.10.79"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_5.10.bb b/meta/recipes-kernel/linux/linux-yocto-tiny_5.10.bb
index 86010f106f..ef562f42f1 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_5.10.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_5.10.bb
@@ -6,7 +6,7 @@ KCONFIG_MODE = "--allnoconfig"
 
 require recipes-kernel/linux/linux-yocto.inc
 
-LINUX_VERSION ?= "5.10.78"
+LINUX_VERSION ?= "5.10.79"
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
@@ -15,9 +15,9 @@ DEPENDS += "openssl-native util-linux-native"
 KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
-SRCREV_machine:qemuarm ?= "cdec5045c5323846adaf2510e539843d0cfe74ae"
-SRCREV_machine ?= "344c0c38f5b892312b0a1db7f613d2704dd4942f"
-SRCREV_meta ?= "a0238f7f4f2222d08bb18147bb5e24cc877b0546"
+SRCREV_machine:qemuarm ?= "d8f6c9be051a886d4956a7d6195d0e8f10761a33"
+SRCREV_machine ?= "f7838de907e651c3d279e6b5209b7e5d7e12ad21"
+SRCREV_meta ?= "1a4cd99824c919ba17dc62935532f3748ef18469"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto_5.10.bb b/meta/recipes-kernel/linux/linux-yocto_5.10.bb
index 2e81068330..a68744d129 100644
--- a/meta/recipes-kernel/linux/linux-yocto_5.10.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_5.10.bb
@@ -13,17 +13,17 @@ KBRANCH:qemux86  ?= "v5.10/standard/base"
 KBRANCH:qemux86-64 ?= "v5.10/standard/base"
 KBRANCH:qemumips64 ?= "v5.10/standard/mti-malta64"
 
-SRCREV_machine:qemuarm ?= "f98b917d7826304daeecf11cc52be2562a9304ff"
-SRCREV_machine:qemuarm64 ?= "13ff8a3ae368724e008e3bcd77833611de7962b2"
-SRCREV_machine:qemumips ?= "7b94dec2b0f5b582b97cdb3ac97fe153559869e4"
-SRCREV_machine:qemuppc ?= "652531fb0cc8eb3607109bb8d878253be2d3d534"
-SRCREV_machine:qemuriscv64 ?= "2daa192783edd4974da8e900c0dc93186e57a838"
-SRCREV_machine:qemuriscv32 ?= "2daa192783edd4974da8e900c0dc93186e57a838"
-SRCREV_machine:qemux86 ?= "2daa192783edd4974da8e900c0dc93186e57a838"
-SRCREV_machine:qemux86-64 ?= "2daa192783edd4974da8e900c0dc93186e57a838"
-SRCREV_machine:qemumips64 ?= "4c817df0fd06350e18693551699c33361e16a193"
-SRCREV_machine ?= "2daa192783edd4974da8e900c0dc93186e57a838"
-SRCREV_meta ?= "a0238f7f4f2222d08bb18147bb5e24cc877b0546"
+SRCREV_machine:qemuarm ?= "400d6ae8c64508b812dd58ee90106be2aa395bd9"
+SRCREV_machine:qemuarm64 ?= "efc398a5bce6938c24413608381d86e7bf225c91"
+SRCREV_machine:qemumips ?= "358d9326c084f4162b71ed857b16419b4fb3295f"
+SRCREV_machine:qemuppc ?= "0fb273427794cb084265a18b03878a482e438aa9"
+SRCREV_machine:qemuriscv64 ?= "fba683668510aeb475a5fd3fb4b7da5aed402495"
+SRCREV_machine:qemuriscv32 ?= "fba683668510aeb475a5fd3fb4b7da5aed402495"
+SRCREV_machine:qemux86 ?= "fba683668510aeb475a5fd3fb4b7da5aed402495"
+SRCREV_machine:qemux86-64 ?= "fba683668510aeb475a5fd3fb4b7da5aed402495"
+SRCREV_machine:qemumips64 ?= "e1dabeb4b0ea9695dea605a0c91f17c482f49f75"
+SRCREV_machine ?= "fba683668510aeb475a5fd3fb4b7da5aed402495"
+SRCREV_meta ?= "1a4cd99824c919ba17dc62935532f3748ef18469"
 
 # remap qemuarm to qemuarma15 for the 5.8 kernel
 # KMACHINE:qemuarm ?= "qemuarma15"
@@ -32,7 +32,7 @@ SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRA
            git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.10;destsuffix=${KMETA}"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
-LINUX_VERSION ?= "5.10.78"
+LINUX_VERSION ?= "5.10.79"
 
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
 DEPENDS += "openssl-native util-linux-native"
-- 
2.19.1

[PATCH 04/20] kernel-yocto: conditionally enable stack protection checking on x86-64

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

The kernel Makfile uses pkgconfig to check for libelf when enabling
CONFIG_STACK_VALIDATION. We already have libelf in our DEPENDS, but
the kernel Makefiles hardcode 'pkgconfig', so fail to pick up our
pkgconfig-native binary that would report the correct flags and paths
for libelf support.

Rather than patching the kernel Makefile's to use pkgconfig-native,
we can use the KERNEL_EXTRA_ARGS variable to pass the definition of
HOST_LIBELF_LIBS via the kernel build commmand line.

We conditionally set HOST_LIBELF_LIBS based on "stack" being in
a newly introduced variable KERNEL_DEBUG_OPTIONS. The value of
HOST_LIBELF_LIBS is the same as pkgconfig-native would set in a
kernel build (but we cannot call pkgconfig at the point this
variable is set).

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/linux-yocto.inc | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-kernel/linux/linux-yocto.inc b/meta/recipes-kernel/linux/linux-yocto.inc
index 331727d62c..15fc7ff724 100644
--- a/meta/recipes-kernel/linux/linux-yocto.inc
+++ b/meta/recipes-kernel/linux/linux-yocto.inc
@@ -46,6 +46,7 @@ LINUX_VERSION_EXTENSION ??= "-yocto-${LINUX_KERNEL_TYPE}"
 # Pick up shared functions
 inherit kernel
 inherit kernel-yocto
+inherit pkgconfig
 
 B = "${WORKDIR}/linux-${PACKAGE_ARCH}-${LINUX_KERNEL_TYPE}-build"
 
@@ -57,3 +58,6 @@ do_install:append(){
 
 # enable kernel-sample for oeqa/runtime/cases's ksample.py test
 KERNEL_FEATURES:append:qemuall=" features/kernel-sample/kernel-sample.scc"
+
+KERNEL_DEBUG_OPTIONS ?= "stack"
+KERNEL_EXTRA_ARGS:append:x86-64 = "${@bb.utils.contains('KERNEL_DEBUG_OPTIONS', 'stack', 'HOST_LIBELF_LIBS="-L${RECIPE_SYSROOT_NATIVE}/usr/lib/pkgconfig/../../../usr/lib/ -lelf"', '', d)}"
-- 
2.19.1

[PATCH 05/20] linux-yocto-dev: bump to v5.16+

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Bumping the version of -dev ot 5.16. We also pickup two new
dependencies to support kernel dtschema validation. This
requirement was introduced by kernel commit:

   commit 53182e81f47d4ea0c727c49ad23cb782173ab849
   Author: Rob Herring <robh@kernel.org>
   Date:   Mon Sep 13 09:51:46 2021 -0500

       kbuild: Enable DT schema checks for %.dtb targets

       It is possible to build a single dtb, but not with DT schema validation
       enabled. Enable the schema validation to run for %.dtb and %.dtbo
       targets. Anyone building a dtb for a specific platform *should* pay
       attention to schema warnings.

       This could be supported with a separate %.dt.yaml target instead.
       However, the .dt.yaml format is considered an intermediate format and
       could possibly go away at some point if schema checking is integrated
       into dtc. Also, the plan is to enable the schema checks by default once
       platforms are free of warnings, and this is a move in that direction.

       Cc: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
       Cc: Tom Rini <trini@konsulko.com>
       Cc: Masahiro Yamada <masahiroy@kernel.org>
       Cc: linux-kbuild@vger.kernel.org
       Signed-off-by: Rob Herring <robh@kernel.org>
       Acked-by: Masahiro Yamada <masahiroy@kernel.org>
       Link: https://lore.kernel.org/r/20210913145146.766080-1-robh@kernel.org

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/linux-yocto-dev.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb
index 005b688778..ed94ab2517 100644
--- a/meta/recipes-kernel/linux/linux-yocto-dev.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb
@@ -30,7 +30,7 @@ SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name
 SRCREV_machine ?= '${@oe.utils.conditional("PREFERRED_PROVIDER_virtual/kernel", "linux-yocto-dev", "${AUTOREV}", "29594404d7fe73cd80eaa4ee8c43dcc53970c60e", d)}'
 SRCREV_meta ?= '${@oe.utils.conditional("PREFERRED_PROVIDER_virtual/kernel", "linux-yocto-dev", "${AUTOREV}", "29594404d7fe73cd80eaa4ee8c43dcc53970c60e", d)}'
 
-LINUX_VERSION ?= "5.15+"
+LINUX_VERSION ?= "5.16+"
 LINUX_VERSION_EXTENSION ?= "-yoctodev-${LINUX_KERNEL_TYPE}"
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
@@ -39,6 +39,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
 DEPENDS += "openssl-native util-linux-native"
 DEPENDS += "gmp-native libmpc-native"
+# yaml and dtschema are required for 5.16+ device tree validation
+DEPENDS += "libyaml-native python3-dtschema-native"
 
 COMPATIBLE_MACHINE = "(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64|qemuriscv64)"
 
-- 
2.19.1

[PATCH 06/20] kernel: export native PKGCONFIG variables

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

In a similar manner to cml1.bbclass, we export the pkg-config
variables to allow a direct call to pkg-config access to the
native sysroot versus the target sysroot.

The kernel doesn't use pkg-config for target configuration,
and has many explicit calls to pkg-config, without the
possibility of easy override to pkg-config-native.

The calls to pkg-config could be made cross friendly via
replacement with make variables, but until that effort is
undertaken upstream, we need a bridge approach.

In particular, this is required for dtschema validation,
which is a requirement in kernel 5.16+

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/classes/kernel.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index e0b752de19..2d219cb5e5 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -326,6 +326,13 @@ KERNEL_DEBUG_TIMESTAMPS ??= "0"
 
 kernel_do_compile() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
+
+	# setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native)
+	export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig"
+	export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig"
+	export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR"
+	export PKG_CONFIG_SYSROOT_DIR=""
+
 	if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then
 		# kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
 		# be set....
-- 
2.19.1

[PATCH 07/20] python: introduce python3-dtschema

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

kernel commit 53182e81f47d [kbuild: Enable DT schema checks for %.dtb
targets], which is part of v5.16+, makes the checking of dtb schemas
manditory.

To support the checking, we introduce this new recipe dtschema.

Dependencies will be imported in subsequent commits, so we can
track the dependency need on a per-import basis.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-dtschema_2021.10.bb            | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb

diff --git a/meta/recipes-devtools/python/python3-dtschema_2021.10.bb b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
new file mode 100644
index 0000000000..36e572a35c
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
@@ -0,0 +1,15 @@
+DESCRIPTION = "Tooling for devicetree validation using YAML and jsonschema"
+HOMEPAGE = "https://github.com/devicetree-org/dt-schema"
+LICENSE = "BSD-2-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=457495c8fa03540db4a576bf7869e811"
+
+inherit pypi setuptools3
+
+PYPI_PACKAGE = "dtschema"
+
+SRC_URI[md5sum] = "d2f0d847ba2ad4d223317ef7304342b0"
+SRC_URI[sha256sum] = "d09c94d13f46e6674ba11ff31220651ad1b02dae860f5a87905dfac6b8d768d9"
+
+DEPENDS += "${PYTHON_PN}-setuptools-scm ${PYTHON_PN}-ruamel-yaml ${PYTHON_PN}-jsonschema ${PYTHON_PN}-rfc3987"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 08/20] python: import jsonpointer from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

required to support python3-json, which is required for python3-dtschema
and device tree checking by the kernel.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-jsonpointer/run-ptest      |  3 +++
 .../python/python3-jsonpointer_2.1.bb         | 26 +++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer_2.1.bb

diff --git a/meta/recipes-devtools/python/python3-jsonpointer/run-ptest b/meta/recipes-devtools/python/python3-jsonpointer/run-ptest
new file mode 100644
index 0000000000..51e609f4ba
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-jsonpointer/run-ptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3 tests.py
diff --git a/meta/recipes-devtools/python/python3-jsonpointer_2.1.bb b/meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
new file mode 100644
index 0000000000..fb3d9e0da9
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
@@ -0,0 +1,26 @@
+SUMMARY = "Resolve JSON Pointers in Python"
+HOMEPAGE = "https://github.com/stefankoegl/python-json-pointer"
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=32b15c843b7a329130f4e266a281ebb3"
+
+inherit pypi ptest setuptools3
+
+SRC_URI[sha256sum] = "5a34b698db1eb79ceac454159d3f7c12a451a91f6334a4f638454327b7a89962"
+
+RDEPENDS:${PN} += " \
+    ${PYTHON_PN}-json \
+"
+
+BBCLASSEXTEND = "native nativesdk"
+
+SRC_URI += " \
+	file://run-ptest \
+"
+
+RDEPENDS:${PN}-ptest += " \
+	${PYTHON_PN}-pytest \
+"
+
+do_install_ptest() {
+	cp -f ${S}/tests.py ${D}${PTEST_PATH}/
+}
-- 
2.19.1

[PATCH 09/20] python: import jsonschema from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing this package as a dependency to python3-dt-schema,
which is required for kernel device tree validation.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-jsonschema_3.2.0.bb        | 48 +++++++++++++++++++
 1 file changed, 48 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb

diff --git a/meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb b/meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
new file mode 100644
index 0000000000..c7efe33a83
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
@@ -0,0 +1,48 @@
+SUMMARY = "An implementation of JSON Schema validation for Python"
+HOMEPAGE = "https://github.com/Julian/jsonschema"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://COPYING;md5=7a60a81c146ec25599a3e1dabb8610a8 \
+                    file://json/LICENSE;md5=9d4de43111d33570c8fe49b4cb0e01af"
+DEPENDS += "${PYTHON_PN}-vcversioner-native ${PYTHON_PN}-setuptools-scm-native"
+
+SRC_URI[md5sum] = "f1a0b5011f05a02a8dee1070cd10a26d"
+SRC_URI[sha256sum] = "c8a85b28d377cc7737e46e2d9f2b4f44ee3c0e1deac6bf46ddefc7187d30797a"
+
+inherit pypi setuptools3
+
+PACKAGECONFIG ??= "format"
+PACKAGECONFIG[format] = ",,,\
+    ${PYTHON_PN}-idna \
+    ${PYTHON_PN}-jsonpointer \
+    ${PYTHON_PN}-webcolors \
+    ${PYTHON_PN}-rfc3987 \
+    ${PYTHON_PN}-strict-rfc3339 \
+"
+PACKAGECONFIG[nongpl] = ",,,\
+    ${PYTHON_PN}-idna \
+    ${PYTHON_PN}-jsonpointer \
+    ${PYTHON_PN}-webcolors \
+    ${PYTHON_PN}-rfc3986-validator \
+    ${PYTHON_PN}-rfc3339-validator \
+"
+
+RDEPENDS:${PN} += " \
+    ${PYTHON_PN}-attrs \
+    ${PYTHON_PN}-core \
+    ${PYTHON_PN}-datetime \
+    ${PYTHON_PN}-importlib-metadata \
+    ${PYTHON_PN}-io \
+    ${PYTHON_PN}-json \
+    ${PYTHON_PN}-netclient \
+    ${PYTHON_PN}-numbers \
+    ${PYTHON_PN}-pkgutil \
+    ${PYTHON_PN}-pprint \
+    ${PYTHON_PN}-pyrsistent \
+    ${PYTHON_PN}-shell \
+    ${PYTHON_PN}-six \
+    ${PYTHON_PN}-unittest \
+    ${PYTHON_PN}-setuptools-scm \
+    ${PYTHON_PN}-zipp \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 10/20] python: import idna from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing idna from meta-python, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-idna_3.3.bb                | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-idna_3.3.bb

diff --git a/meta/recipes-devtools/python/python3-idna_3.3.bb b/meta/recipes-devtools/python/python3-idna_3.3.bb
new file mode 100644
index 0000000000..a0e6b79a56
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-idna_3.3.bb
@@ -0,0 +1,19 @@
+SUMMARY = "Internationalised Domain Names in Applications"
+HOMEPAGE = "https://github.com/kjd/idna"
+LICENSE = "BSD-3-Clause & Python-2.0 & Unicode"
+LIC_FILES_CHKSUM = "file://LICENSE.md;md5=239668a7c6066d9e0c5382e9c8c6c0e1"
+
+SRC_URI[sha256sum] = "9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d"
+
+inherit pypi setuptools3
+
+# Remove bundled egg-info
+do_compile:prepend() {
+    rm -rf ${S}/idna.egg-info
+}
+
+RDEPENDS:${PN}:class-target = "\
+    ${PYTHON_PN}-codecs \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 11/20] python: import rfc3339-validator from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing rfc3339-validator from meta-python, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-rfc3339-validator_0.1.4.bb | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb

diff --git a/meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb b/meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
new file mode 100644
index 0000000000..7f1bbaf97d
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
@@ -0,0 +1,20 @@
+SUMMARY = "A pure python RFC3339 validator"
+HOMEPAGE = "https://github.com/naimetti/rfc3339-validator"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=a21b13b5a996f08f7e0b088aa38ce9c6"
+
+FILESEXTRAPATHS:prepend := "${THISDIR}/python-rfc3339-validator:"
+
+SRC_URI[sha256sum] = "138a2abdf93304ad60530167e51d2dfb9549521a836871b88d7f4695d0022f6b"
+
+PYPI_PACKAGE = "rfc3339_validator"
+
+inherit pypi setuptools3
+
+RDEPENDS:${PN} += "\
+    ${PYTHON_PN}-core \
+    ${PYTHON_PN}-datetime \
+    ${PYTHON_PN}-six \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 12/20] python: import rfc3986-validator from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing rfc3986-validator, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 ...e-pytest-runner-to-test_requirements.patch | 32 +++++++++++++++++++
 .../python/python3-rfc3986-validator_0.1.1.bb | 23 +++++++++++++
 2 files changed, 55 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
 create mode 100644 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb

diff --git a/meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch b/meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
new file mode 100644
index 0000000000..8a646874e8
--- /dev/null
+++ b/meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
@@ -0,0 +1,32 @@
+From 3531ff73631a0d59234eb4713e7b3a7f5ea57bbb Mon Sep 17 00:00:00 2001
+From: Nicola Lunghi <nicola.lunghi@jci.com>
+Date: Thu, 14 Nov 2019 12:17:51 +0000
+Subject: [PATCH] setup.py: move pytest-runner to test_requirements
+
+This fixes an issue with yocto build.
+pytest-runner is only needed when running tests.
+
+Upstream-Status: Pending
+---
+ setup.py | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index 53ebea7..ebb0de2 100644
+--- a/setup.py
++++ b/setup.py
+@@ -10,9 +10,9 @@ with open('README.md') as readme_file:
+ 
+ requirements = []
+ 
+-setup_requirements = ['pytest-runner', ]
++setup_requirements = []
+ 
+-test_requirements = ['pytest>=3', ]
++test_requirements = ['pytest>=3', 'pytest-runner']
+ 
+ setup(
+     author="Nicolas Aimetti",
+-- 
+2.20.1
+
diff --git a/meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb b/meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
new file mode 100644
index 0000000000..68b3db1f48
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
@@ -0,0 +1,23 @@
+SUMMARY = "Pure python rfc3986 validator"
+HOMEPAGE = "https://github.com/naimetti/rfc3986-validator"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=a21b13b5a996f08f7e0b088aa38ce9c6"
+
+FILESEXTRAPATHS:prepend := "${THISDIR}/python-rfc3986-validator:"
+
+SRC_URI[md5sum] = "47f7657b790aaf6011a1ab3d86c6be95"
+SRC_URI[sha256sum] = "3d44bde7921b3b9ec3ae4e3adca370438eccebc676456449b145d533b240d055"
+
+PYPI_PACKAGE = "rfc3986_validator"
+
+inherit pypi setuptools3
+
+SRC_URI:append = " \
+    file://0001-setup.py-move-pytest-runner-to-test_requirements.patch \
+"
+
+RDEPENDS:${PN} += "\
+    python3-core \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 13/20] python: import webcolors from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing webcolors, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-webcolors/run-ptest        |  3 +++
 .../python/python3-webcolors_1.11.1.bb        | 27 +++++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-webcolors/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-webcolors_1.11.1.bb

diff --git a/meta/recipes-devtools/python/python3-webcolors/run-ptest b/meta/recipes-devtools/python/python3-webcolors/run-ptest
new file mode 100644
index 0000000000..3385d68939
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-webcolors/run-ptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}'
diff --git a/meta/recipes-devtools/python/python3-webcolors_1.11.1.bb b/meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
new file mode 100644
index 0000000000..2ec036ef35
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
@@ -0,0 +1,27 @@
+SUMMARY = "Simple Python module for working with HTML/CSS color definitions."
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=25b90379a52351261c51272e7923d240"
+
+SRC_URI[md5sum] = "54d28a7c80b3e4d974ec2fee86768be9"
+SRC_URI[sha256sum] = "76f360636957d1c976db7466bc71dcb713bb95ac8911944dffc55c01cb516de6"
+
+inherit pypi setuptools3 ptest
+
+RDEPENDS:${PN}:class-target = "\
+    ${PYTHON_PN}-stringold \
+"
+
+SRC_URI += " \
+    file://run-ptest \
+"
+
+RDEPENDS:${PN}-ptest += " \
+    ${PYTHON_PN}-pytest \
+"
+
+do_install_ptest() {
+    install -d ${D}${PTEST_PATH}/tests
+    cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 14/20] python: import ruamel-yaml from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing ruamel-yaml from meta-python. This package is a
dependency of python3-dtschema, which is required to validate
kernel devicetrees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-ruamel-yaml_0.17.16.bb     | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb

diff --git a/meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb b/meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
new file mode 100644
index 0000000000..e64f196004
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
@@ -0,0 +1,23 @@
+SUMMARY = "YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order."
+AUTHOR = "Anthon van der Neut"
+
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=fa0a51dfb461e2f803969e0f3fa71dfe"
+
+PYPI_PACKAGE = "ruamel.yaml"
+
+inherit pypi setuptools3
+
+SRC_URI[sha256sum] = "1a771fc92d3823682b7f0893ad56cb5a5c87c48e62b5399d6f42c8759a583b33"
+
+RDEPENDS:${PN} += "\
+    ${PYTHON_PN}-shell \
+    ${PYTHON_PN}-datetime \
+    ${PYTHON_PN}-netclient \
+"
+
+do_install:prepend() {
+    export RUAMEL_NO_PIP_INSTALL_CHECK=1
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 15/20] maintainers: update for kernel dtschema checking dependencies

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

These packages are required for dtschema checking in the
kernel build. Adding myself as the maintainer.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/conf/distro/include/maintainers.inc | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 5423358e77..3919c5ede6 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -597,16 +597,20 @@ RECIPE_MAINTAINER:pn-python3-cython = "Oleksandr Kravchuk <open.source@oleksandr
 RECIPE_MAINTAINER:pn-python3-dbus = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-dbusmock = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-docutils = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
+RECIPE_MAINTAINER:pn-python3-dtschema = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodome = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodomex = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-extras = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-git = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-gitdb = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-hypothesis = "Tim Orling <timothy.t.orling@intel.com>"
+RECIPE_MAINTAINER:pn-python3-idna = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-importlib-metadata = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-iniconfig = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-iniparse = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-jinja2 = "Richard Purdie <richard.purdie@linuxfoundation.org>"
+RECIPE_MAINTAINER:pn-python3-python3-jsonpointer = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-python3-jsonschema = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-libarchive-c = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-magic = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
@@ -628,6 +632,9 @@ RECIPE_MAINTAINER:pn-python3-pygments = "Oleksandr Kravchuk <open.source@oleksan
 RECIPE_MAINTAINER:pn-python3-pygobject = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-pyparsing = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-pytest = "Tim Orling <timothy.t.orling@intel.com>"
+RECIPE_MAINTAINER:pn-python3-rfc3339-validator = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-rfc3986-validator = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-ruamel-yaml = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-scons = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-scons-native = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-setuptools = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
@@ -642,6 +649,7 @@ RECIPE_MAINTAINER:pn-python3-toml = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-tomli = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-wcwidth = "Tim Orling <timothy.t.orling@intel.com>"
+RECIPE_MAINTAINER:pn-python3-webcolors = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-zipp = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-qemu = "Richard Purdie <richard.purdie@linuxfoundation.org>"
 RECIPE_MAINTAINER:pn-qemu-helper-native = "Richard Purdie <richard.purdie@linuxfoundation.org>"
-- 
2.19.1

[PATCH 16/20] linux-yocto: export pkgconfig variables to devshell

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Not all of the kernel host/build Makefiles allow pkg-config
to be overriden to pkg-config-native. Exporting these variables
allow us to debug host tool issues with the kernel build.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/linux-yocto.inc | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/meta/recipes-kernel/linux/linux-yocto.inc b/meta/recipes-kernel/linux/linux-yocto.inc
index 15fc7ff724..a3b2416367 100644
--- a/meta/recipes-kernel/linux/linux-yocto.inc
+++ b/meta/recipes-kernel/linux/linux-yocto.inc
@@ -61,3 +61,12 @@ KERNEL_FEATURES:append:qemuall=" features/kernel-sample/kernel-sample.scc"
 
 KERNEL_DEBUG_OPTIONS ?= "stack"
 KERNEL_EXTRA_ARGS:append:x86-64 = "${@bb.utils.contains('KERNEL_DEBUG_OPTIONS', 'stack', 'HOST_LIBELF_LIBS="-L${RECIPE_SYSROOT_NATIVE}/usr/lib/pkgconfig/../../../usr/lib/ -lelf"', '', d)}"
+
+do_devshell:prepend() {
+    # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native)
+    d.setVar("PKG_CONFIG_DIR", "${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig")
+    d.setVar("PKG_CONFIG_PATH", "${PKG_CONFIG_DIR}:${STAGING_DATADIR_NATIVE}/pkgconfig")
+    d.setVar("PKG_CONFIG_LIBDIR", "${PKG_CONFIG_DIR}")
+    d.setVarFlag("PKG_CONFIG_SYSROOT_DIR", "unexport", "1")
+    d.appendVar("OE_TERMINAL_EXPORTS", " PKG_CONFIG_DIR PKG_CONFIG_PATH PKG_CONFIG_LIBDIR PKG_CONFIG_SYSROOT_DIR")
+}
-- 
2.19.1

[PATCH 17/20] kernel: introduce python3-dtschema-wrapper

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

The 5.16 kernel introduced mandatory schema checking on any dtb file
built through the kernel.

That funcionality is provided via python3-dt-schema.

The dependencies to enable that functionality is not small, and may
not always be desired (in particular on architectures that do not
support dtbs, or in development cycles). It may also be useful for
allowing a non-conformant dts to be compiled.

This commit introduces a set of wrapper scripts that when added
as a depenency to the kernel, can pass both the validation testing
and validation steps of a dts.

We use the wrapper by default, while more dts file are brought up
to spec and the dt-validation matures upstream.

To enable valiation, simply add 'dt-valiation' to the PACKAGECONFIG
variable and the full dt-schema package will be selected and
validation will occur.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/conf/distro/include/maintainers.inc      |  1 +
 .../python3-dtschema-wrapper/dt-doc-validate  | 20 +++++++++++++++++++
 .../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 +++++++++++++++++++
 .../dtc/python3-dtschema-wrapper/dt-validate  | 20 +++++++++++++++++++
 .../dtc/python3-dtschema-wrapper_2021.10.bb   | 17 ++++++++++++++++
 5 files changed, 78 insertions(+)
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 3919c5ede6..ba7a47c30d 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -598,6 +598,7 @@ RECIPE_MAINTAINER:pn-python3-dbus = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-dbusmock = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-docutils = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-dtschema = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-dtschema-wrapper = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodome = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodomex = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-extras = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
new file mode 100644
index 0000000000..2aa57851c7
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-doc-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield <bruce.ashfield@gmail.com>
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+    case "$arg" in
+        --version)
+            echo "v2021.10"
+            ;;
+    esac
+done
+
+# TBD: left for future consideration
+# exec dt-doc-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
new file mode 100644
index 0000000000..24b89d8619
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-mk-schema wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield <bruce.ashfield@gmail.com>
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+    case "$arg" in
+        --version)
+            echo "v2021.10"
+            ;;
+    esac
+done
+
+# TBD: left for future consideration
+# exec dt-mk-schema.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
new file mode 100644
index 0000000000..8a4710a7ed
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield <bruce.ashfield@gmail.com>
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+    case "$arg" in
+        --version)
+            echo "v2021.10"
+            ;;
+    esac
+done
+
+# TBD: left for future consideration
+# exec dt-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
new file mode 100644
index 0000000000..c869274d09
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
@@ -0,0 +1,17 @@
+DESCRIPTION = "Wrapper for tooling for devicetree validation using YAML and jsonschema"
+HOMEPAGE = "https://yoctoproject.org"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "file://dt-doc-validate \
+           file://dt-mk-schema \
+           file://dt-validate"
+
+do_install() {
+    install -d ${D}${bindir}/
+    install -m 755 ${WORKDIR}/dt-doc-validate ${D}${bindir}/
+    install -m 755 ${WORKDIR}/dt-mk-schema ${D}${bindir}/
+    install -m 755 ${WORKDIR}/dt-validate ${D}${bindir}/
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 18/20] linux-yocto-dev: introduce dt-validation PACKAGECONFIG

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Add a PACKAGECONFIG to control which type of dt-validation
is installed (full, or wrapped).

This is currently in linux-yocto-dev, as 5.16 introduces
the functionality of required dt schema validation.

Other v5.16 providing recipes can use the base functionality
as they desire (with or without PACKAGECONFIG).

Over time, we can migrate the support the the core
kernel classes .. in particular when more providers are at
v5.16+

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/linux-yocto-dev.bb | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb
index ed94ab2517..858772ba41 100644
--- a/meta/recipes-kernel/linux/linux-yocto-dev.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb
@@ -39,8 +39,14 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
 DEPENDS += "openssl-native util-linux-native"
 DEPENDS += "gmp-native libmpc-native"
-# yaml and dtschema are required for 5.16+ device tree validation
-DEPENDS += "libyaml-native python3-dtschema-native"
+
+# yaml and dtschema are required for 5.16+ device tree validation, libyaml is checked
+# via pkgconfig, so must always be present, but we can wrap the others to make them
+# conditional
+DEPENDS += "libyaml-native"
+
+PACKAGECONFIG ??= ""
+DEPENDS += "${@bb.utils.contains('PACKAGECONFIG', 'dt-validation', 'python3-dtschema-native', 'python3-dtschema-wrapper-native', d)}"
 
 COMPATIBLE_MACHINE = "(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64|qemuriscv64)"
 
-- 
2.19.1

[PATCH 19/20] perf: sort-pmuevents: don't drop elements

[bruce.ashfield]

From: Max Krummenacher <max.oss.09@gmail.com>

If a struct element neither has an element cpuid or name it gets silenty
dropped.
Kernel 5.15 for some ARCHs have at least one array of structs matching
this.
e.g. for arm pmu-events.c:

|#include "pmu-events/pmu-events.h"
struct pmu_events_map pmu_events_map[] = {
{
	.cpuid = 0,
	.version = 0,
	.type = 0,
	.table = 0,
},
};

struct pmu_sys_events pmu_sys_event_tables[] = {
	{
		.table = 0
	},
};

Before this patch the second struct is translated to an empty array::

struct pmu_sys_events pmu_sys_event_tables[] = {
};

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/perf/perf/sort-pmuevents.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
index 5ddf0f144f..4f841eb822 100755
--- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
+++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
@@ -61,6 +61,8 @@ for struct in re.findall( struct_block_regex, data ):
             #print( "    name found: %s" % name.group(1) )
             entry_dict[struct[1]]['fields'][name.group(1)] = entry
         
+        if not entry_dict[struct[1]]['fields']:
+            entry_dict[struct[1]]['fields']['0'] = entry
 
 # created ordered dictionaries from the captured values. These are ordered by
 # a sorted() iteration of the keys. We don't care about the order we read
-- 
2.19.1

[PATCH 20/20] perf: sort-pmuevents: allow for additional type qualifiers and storage class

[bruce.ashfield]

From: Max Krummenacher <max.oss.09@gmail.com>

With kernel 5.16 some structs in pmu-events do get a const qualifier, some
a static const storage class and qualifier.

The current sort-pmuevents cannot cope with that and drops all struct
arrays with such additional elements. This then leads to compiler errors.

Allow '^struct', '^const struct', '^static struct', '^static const struct'.

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../perf/perf/sort-pmuevents.py               | 30 ++++++++++---------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
index 4f841eb822..09ba3328a7 100755
--- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
+++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
@@ -33,10 +33,10 @@ if os.path.exists(outfile):
 with open(infile, 'r') as file:
     data = file.read()
 
-preamble_regex = re.compile( '^(.*?)^struct', re.MULTILINE | re.DOTALL )
+preamble_regex = re.compile( '^(.*?)^(struct|const struct|static struct|static const struct)', re.MULTILINE | re.DOTALL )
 
 preamble = re.search( preamble_regex, data )
-struct_block_regex = re.compile( '^struct.*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
+struct_block_regex = re.compile( '^(struct|const struct|static struct|static const struct).*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
 field_regex =  re.compile( '{.*?},', re.MULTILINE | re.DOTALL )
 cpuid_regex = re.compile( '\.cpuid = (.*?),', re.MULTILINE | re.DOTALL )
 name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
@@ -45,24 +45,25 @@ name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
 # types and then their fields.
 entry_dict = {}
 for struct in re.findall( struct_block_regex, data ):
-    # print( "struct: %s %s" % (struct[0],struct[1]) )
-    entry_dict[struct[1]] = {}
-    entry_dict[struct[1]]['type'] = struct[0]
-    entry_dict[struct[1]]['fields'] = {}
-    for entry in re.findall( field_regex, struct[2] ):
+    # print( "struct: %s %s %s" % (struct[0],struct[1],struct[2]) )
+    entry_dict[struct[2]] = {}
+    entry_dict[struct[2]]['type_prefix'] = struct[0]
+    entry_dict[struct[2]]['type'] = struct[1]
+    entry_dict[struct[2]]['fields'] = {}
+    for entry in re.findall( field_regex, struct[3] ):
         #print( "    entry: %s" % entry )
         cpuid = re.search( cpuid_regex, entry )
         if cpuid:
             #print( "    cpuid found: %s" % cpuid.group(1) )
-            entry_dict[struct[1]]['fields'][cpuid.group(1)] = entry
-            
+            entry_dict[struct[2]]['fields'][cpuid.group(1)] = entry
+
         name = re.search( name_regex, entry )
         if name:
             #print( "    name found: %s" % name.group(1) )
-            entry_dict[struct[1]]['fields'][name.group(1)] = entry
-        
-        if not entry_dict[struct[1]]['fields']:
-            entry_dict[struct[1]]['fields']['0'] = entry
+            entry_dict[struct[2]]['fields'][name.group(1)] = entry
+
+        if not entry_dict[struct[2]]['fields']:
+            entry_dict[struct[2]]['fields']['0'] = entry
 
 # created ordered dictionaries from the captured values. These are ordered by
 # a sorted() iteration of the keys. We don't care about the order we read
@@ -74,6 +75,7 @@ for struct in re.findall( struct_block_regex, data ):
 entry_dict_sorted = OrderedDict()
 for i in sorted(entry_dict.keys()):
     entry_dict_sorted[i] = {}
+    entry_dict_sorted[i]['type_prefix'] = entry_dict[i]['type_prefix']
     entry_dict_sorted[i]['type'] = entry_dict[i]['type']
     entry_dict_sorted[i]['fields'] = {}
     for f in sorted(entry_dict[i]['fields'].keys()):
@@ -85,7 +87,7 @@ outf = open( outfile, 'w' )
 print( preamble.group(1) )
 outf.write( preamble.group(1) )
 for d in entry_dict_sorted:
-    outf.write( "struct %s %s[] = {\n" % (entry_dict_sorted[d]['type'],d) )
+    outf.write( "%s %s %s[] = {\n" % (entry_dict_sorted[d]['type_prefix'], entry_dict_sorted[d]['type'],d) )
     for f in entry_dict_sorted[d]['fields']:
         outf.write( entry_dict_sorted[d]['fields'][f] + '\n' )
 
-- 
2.19.1

Re: [OE-core] [PATCH 00/20] kernel/kernel-yocto: consolidated pull request

[Khem Raj]

dont forget to send removal patch for meta-python once they are in core.

On Tue, Nov 23, 2021 at 1:57 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>
> Richard,
>
> There's a few routine -stable updates at the start of this, and then some
> more painful (and RFC) commits.
>
> As part of this series, we fix one pkg-config issue that was preventing
> x86 from doing stack valildation during build. At one point it was
> increasing my build times signficantly, but that seems to have gone
> away now. If we see something like this on the AB, we'll have to keep
> the fix, but disable it by default.
>
> The next big chunk of changes are the ones we talked about during the
> tech call today. The 5.16 kernel has introduced mandatory dtschema
> checking for device trees built through the kernel.
>
> That checking pulls in a reasonably large amount of dependencies to
> the kernel build: libyaml, dtschema, jsonschema, etc.
>
> We weren't able to even run the validation at all, due to the kernel
> calling pkg-config directly to check for libyaml, which is the target
> pkgconfig and failing on missing dependencies. Rather than patching
> the kernel, I've finally given in and made the do_compile task set
> the PKG_CONFIG variables to point to our native-sysroot. Since the
> kenrel doesn't use pkgconfig for target decisions, and has quite
> a few calls directly to pkgconfig, this is the best short term way
> to get all those checks working.
>
> Once we are past yaml, we need to create the dtschema recipe, and
> bring in the dependencies from meta-python. I've put myself down as
> the maintainer for all the new recipes, but otherwise, I haven't
> changed or modified them.
>
> BUT! there are scenarios where a non-conformant dts file is being
> built, or we just don't want those dependencies in the built. To
> work around that, I took the suggestion from this morning and
> created a python3-dtschema-wrappers recipe. It contains three
> scripts of the same name as the ones from the real package. When
> put in the dependency for a kernel build, it can pass the dtschema
> steps.
>
> Finally, I created a reference PACKAGECONFIG for linux-yocto-dev
> to enable/disable the main package and wrappers (the wrappers are
> used by default). I haven't put this into kernel.bbclass yet, since
> it is version specific, and it needs to soak for a bit. But if
> there's demand, it is something that can be moved to kernel.bclass
> (along with a few other version specific dependencies).
>
> Note: I haven't done an AB run with these yet, so I could be
> missing some python dependencies, etc, but I wanted to get this
> out to the list in case others are running into this with
> v5.16+.
>
> Cheers,
>
> Bruce
>
>
> The following changes since commit 60409917ea3e7353f5b7ee831fe4ffaff0957d94:
>
>   glibc-tests: Do not set SYSTEMD_SERVICE (2021-11-23 14:04:02 +0000)
>
> are available in the Git repository at:
>
>   git://git.yoctoproject.org/poky-contrib zedd/kernel
>   http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel
>
> Bruce Ashfield (18):
>   linux-yocto/5.15: update to v5.15.2
>   linux-yocto/5.14: update to v5.14.18
>   linux-yocto/5.10: update to v5.10.79
>   kernel-yocto: conditionally enable stack protection checking on x86-64
>   linux-yocto-dev: bump to v5.16+
>   kernel: export native PKGCONFIG variables
>   python: introduce python3-dtschema
>   python: import jsonpointer from meta-python
>   python: import jsonschema from meta-python
>   python: import idna from meta-python
>   python: import rfc3339-validator from meta-python
>   python: import rfc3986-validator from meta-python
>   python: import webcolors from meta-python
>   python: import ruamel-yaml from meta-python
>   maintainers: update for kernel dtschema checking dependencies
>   linux-yocto: export pkgconfig variables to devshell
>   kernel: introduce python3-dtschema-wrapper
>   linux-yocto-dev: introduce dt-validation PACKAGECONFIG
>
> Max Krummenacher (2):
>   perf: sort-pmuevents: don't drop elements
>   perf: sort-pmuevents: allow for additional type qualifiers and storage
>     class
>
>  meta/classes/kernel.bbclass                   |  7 +++
>  meta/conf/distro/include/maintainers.inc      |  9 ++++
>  ...e-pytest-runner-to-test_requirements.patch | 32 +++++++++++++
>  .../python/python3-dtschema_2021.10.bb        | 15 ++++++
>  .../python/python3-idna_3.3.bb                | 19 ++++++++
>  .../python/python3-jsonpointer/run-ptest      |  3 ++
>  .../python/python3-jsonpointer_2.1.bb         | 26 ++++++++++
>  .../python/python3-jsonschema_3.2.0.bb        | 48 +++++++++++++++++++
>  .../python/python3-rfc3339-validator_0.1.4.bb | 20 ++++++++
>  .../python/python3-rfc3986-validator_0.1.1.bb | 23 +++++++++
>  .../python/python3-ruamel-yaml_0.17.16.bb     | 23 +++++++++
>  .../python/python3-webcolors/run-ptest        |  3 ++
>  .../python/python3-webcolors_1.11.1.bb        | 27 +++++++++++
>  .../python3-dtschema-wrapper/dt-doc-validate  | 20 ++++++++
>  .../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 ++++++++
>  .../dtc/python3-dtschema-wrapper/dt-validate  | 20 ++++++++
>  .../dtc/python3-dtschema-wrapper_2021.10.bb   | 17 +++++++
>  meta/recipes-kernel/linux/linux-yocto-dev.bb  | 10 +++-
>  .../linux/linux-yocto-rt_5.10.bb              |  6 +--
>  .../linux/linux-yocto-rt_5.14.bb              |  6 +--
>  .../linux/linux-yocto-rt_5.15.bb              |  6 +--
>  .../linux/linux-yocto-tiny_5.10.bb            |  8 ++--
>  .../linux/linux-yocto-tiny_5.14.bb            |  8 ++--
>  .../linux/linux-yocto-tiny_5.15.bb            |  8 ++--
>  meta/recipes-kernel/linux/linux-yocto.inc     | 13 +++++
>  meta/recipes-kernel/linux/linux-yocto_5.10.bb | 24 +++++-----
>  meta/recipes-kernel/linux/linux-yocto_5.14.bb | 26 +++++-----
>  meta/recipes-kernel/linux/linux-yocto_5.15.bb | 26 +++++-----
>  .../perf/perf/sort-pmuevents.py               | 28 ++++++-----
>  29 files changed, 429 insertions(+), 72 deletions(-)
>  create mode 100644 meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
>  create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
>  create mode 100644 meta/recipes-devtools/python/python3-idna_3.3.bb
>  create mode 100644 meta/recipes-devtools/python/python3-jsonpointer/run-ptest
>  create mode 100644 meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
>  create mode 100644 meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
>  create mode 100644 meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
>  create mode 100644 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
>  create mode 100644 meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
>  create mode 100644 meta/recipes-devtools/python/python3-webcolors/run-ptest
>  create mode 100644 meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
>
> --
> 2.19.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#158621): https://lists.openembedded.org/g/openembedded-core/message/158621
> Mute This Topic: https://lists.openembedded.org/mt/87269381/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH 00/20] kernel/kernel-yocto: consolidated pull request

[Bruce Ashfield]

On Tue, Nov 23, 2021 at 6:59 PM Khem Raj <raj.khem@gmail.com> wrote:
>
> dont forget to send removal patch for meta-python once they are in core.

Definitely!

This may take a while to sort out, but I have one ready and will send
when required.

Bruce

>
> On Tue, Nov 23, 2021 at 1:57 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > From: Bruce Ashfield <bruce.ashfield@gmail.com>
> >
> > Richard,
> >
> > There's a few routine -stable updates at the start of this, and then some
> > more painful (and RFC) commits.
> >
> > As part of this series, we fix one pkg-config issue that was preventing
> > x86 from doing stack valildation during build. At one point it was
> > increasing my build times signficantly, but that seems to have gone
> > away now. If we see something like this on the AB, we'll have to keep
> > the fix, but disable it by default.
> >
> > The next big chunk of changes are the ones we talked about during the
> > tech call today. The 5.16 kernel has introduced mandatory dtschema
> > checking for device trees built through the kernel.
> >
> > That checking pulls in a reasonably large amount of dependencies to
> > the kernel build: libyaml, dtschema, jsonschema, etc.
> >
> > We weren't able to even run the validation at all, due to the kernel
> > calling pkg-config directly to check for libyaml, which is the target
> > pkgconfig and failing on missing dependencies. Rather than patching
> > the kernel, I've finally given in and made the do_compile task set
> > the PKG_CONFIG variables to point to our native-sysroot. Since the
> > kenrel doesn't use pkgconfig for target decisions, and has quite
> > a few calls directly to pkgconfig, this is the best short term way
> > to get all those checks working.
> >
> > Once we are past yaml, we need to create the dtschema recipe, and
> > bring in the dependencies from meta-python. I've put myself down as
> > the maintainer for all the new recipes, but otherwise, I haven't
> > changed or modified them.
> >
> > BUT! there are scenarios where a non-conformant dts file is being
> > built, or we just don't want those dependencies in the built. To
> > work around that, I took the suggestion from this morning and
> > created a python3-dtschema-wrappers recipe. It contains three
> > scripts of the same name as the ones from the real package. When
> > put in the dependency for a kernel build, it can pass the dtschema
> > steps.
> >
> > Finally, I created a reference PACKAGECONFIG for linux-yocto-dev
> > to enable/disable the main package and wrappers (the wrappers are
> > used by default). I haven't put this into kernel.bbclass yet, since
> > it is version specific, and it needs to soak for a bit. But if
> > there's demand, it is something that can be moved to kernel.bclass
> > (along with a few other version specific dependencies).
> >
> > Note: I haven't done an AB run with these yet, so I could be
> > missing some python dependencies, etc, but I wanted to get this
> > out to the list in case others are running into this with
> > v5.16+.
> >
> > Cheers,
> >
> > Bruce
> >
> >
> > The following changes since commit 60409917ea3e7353f5b7ee831fe4ffaff0957d94:
> >
> >   glibc-tests: Do not set SYSTEMD_SERVICE (2021-11-23 14:04:02 +0000)
> >
> > are available in the Git repository at:
> >
> >   git://git.yoctoproject.org/poky-contrib zedd/kernel
> >   http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel
> >
> > Bruce Ashfield (18):
> >   linux-yocto/5.15: update to v5.15.2
> >   linux-yocto/5.14: update to v5.14.18
> >   linux-yocto/5.10: update to v5.10.79
> >   kernel-yocto: conditionally enable stack protection checking on x86-64
> >   linux-yocto-dev: bump to v5.16+
> >   kernel: export native PKGCONFIG variables
> >   python: introduce python3-dtschema
> >   python: import jsonpointer from meta-python
> >   python: import jsonschema from meta-python
> >   python: import idna from meta-python
> >   python: import rfc3339-validator from meta-python
> >   python: import rfc3986-validator from meta-python
> >   python: import webcolors from meta-python
> >   python: import ruamel-yaml from meta-python
> >   maintainers: update for kernel dtschema checking dependencies
> >   linux-yocto: export pkgconfig variables to devshell
> >   kernel: introduce python3-dtschema-wrapper
> >   linux-yocto-dev: introduce dt-validation PACKAGECONFIG
> >
> > Max Krummenacher (2):
> >   perf: sort-pmuevents: don't drop elements
> >   perf: sort-pmuevents: allow for additional type qualifiers and storage
> >     class
> >
> >  meta/classes/kernel.bbclass                   |  7 +++
> >  meta/conf/distro/include/maintainers.inc      |  9 ++++
> >  ...e-pytest-runner-to-test_requirements.patch | 32 +++++++++++++
> >  .../python/python3-dtschema_2021.10.bb        | 15 ++++++
> >  .../python/python3-idna_3.3.bb                | 19 ++++++++
> >  .../python/python3-jsonpointer/run-ptest      |  3 ++
> >  .../python/python3-jsonpointer_2.1.bb         | 26 ++++++++++
> >  .../python/python3-jsonschema_3.2.0.bb        | 48 +++++++++++++++++++
> >  .../python/python3-rfc3339-validator_0.1.4.bb | 20 ++++++++
> >  .../python/python3-rfc3986-validator_0.1.1.bb | 23 +++++++++
> >  .../python/python3-ruamel-yaml_0.17.16.bb     | 23 +++++++++
> >  .../python/python3-webcolors/run-ptest        |  3 ++
> >  .../python/python3-webcolors_1.11.1.bb        | 27 +++++++++++
> >  .../python3-dtschema-wrapper/dt-doc-validate  | 20 ++++++++
> >  .../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 ++++++++
> >  .../dtc/python3-dtschema-wrapper/dt-validate  | 20 ++++++++
> >  .../dtc/python3-dtschema-wrapper_2021.10.bb   | 17 +++++++
> >  meta/recipes-kernel/linux/linux-yocto-dev.bb  | 10 +++-
> >  .../linux/linux-yocto-rt_5.10.bb              |  6 +--
> >  .../linux/linux-yocto-rt_5.14.bb              |  6 +--
> >  .../linux/linux-yocto-rt_5.15.bb              |  6 +--
> >  .../linux/linux-yocto-tiny_5.10.bb            |  8 ++--
> >  .../linux/linux-yocto-tiny_5.14.bb            |  8 ++--
> >  .../linux/linux-yocto-tiny_5.15.bb            |  8 ++--
> >  meta/recipes-kernel/linux/linux-yocto.inc     | 13 +++++
> >  meta/recipes-kernel/linux/linux-yocto_5.10.bb | 24 +++++-----
> >  meta/recipes-kernel/linux/linux-yocto_5.14.bb | 26 +++++-----
> >  meta/recipes-kernel/linux/linux-yocto_5.15.bb | 26 +++++-----
> >  .../perf/perf/sort-pmuevents.py               | 28 ++++++-----
> >  29 files changed, 429 insertions(+), 72 deletions(-)
> >  create mode 100644 meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
> >  create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> >  create mode 100644 meta/recipes-devtools/python/python3-idna_3.3.bb
> >  create mode 100644 meta/recipes-devtools/python/python3-jsonpointer/run-ptest
> >  create mode 100644 meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
> >  create mode 100644 meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
> >  create mode 100644 meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
> >  create mode 100644 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
> >  create mode 100644 meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
> >  create mode 100644 meta/recipes-devtools/python/python3-webcolors/run-ptest
> >  create mode 100644 meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
> >  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
> >  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
> >  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
> >  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
> >
> > --
> > 2.19.1
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#158621): https://lists.openembedded.org/g/openembedded-core/message/158621
> > Mute This Topic: https://lists.openembedded.org/mt/87269381/1997914
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [PATCH 00/20] kernel/kernel-yocto: consolidated pull request

[Bruce Ashfield]

On Tue, Nov 23, 2021 at 4:57 PM <bruce.ashfield@gmail.com> wrote:
>
> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>
> Richard,
>
> There's a few routine -stable updates at the start of this, and then some
> more painful (and RFC) commits.
>
> As part of this series, we fix one pkg-config issue that was preventing
> x86 from doing stack valildation during build. At one point it was
> increasing my build times signficantly, but that seems to have gone
> away now. If we see something like this on the AB, we'll have to keep
> the fix, but disable it by default.
>
> The next big chunk of changes are the ones we talked about during the
> tech call today. The 5.16 kernel has introduced mandatory dtschema
> checking for device trees built through the kernel.
>
> That checking pulls in a reasonably large amount of dependencies to
> the kernel build: libyaml, dtschema, jsonschema, etc.

I missed some dependencies for the new recipe. I started an AB run,
and will stack any new imports on top of my queue.

I may have also gotten the rdepends -> depends wrong for the -native
recipe, I never get them right for native / host tools the first time :D
I'm looking into that now.

Bruce

>
> We weren't able to even run the validation at all, due to the kernel
> calling pkg-config directly to check for libyaml, which is the target
> pkgconfig and failing on missing dependencies. Rather than patching
> the kernel, I've finally given in and made the do_compile task set
> the PKG_CONFIG variables to point to our native-sysroot. Since the
> kenrel doesn't use pkgconfig for target decisions, and has quite
> a few calls directly to pkgconfig, this is the best short term way
> to get all those checks working.
>
> Once we are past yaml, we need to create the dtschema recipe, and
> bring in the dependencies from meta-python. I've put myself down as
> the maintainer for all the new recipes, but otherwise, I haven't
> changed or modified them.
>
> BUT! there are scenarios where a non-conformant dts file is being
> built, or we just don't want those dependencies in the built. To
> work around that, I took the suggestion from this morning and
> created a python3-dtschema-wrappers recipe. It contains three
> scripts of the same name as the ones from the real package. When
> put in the dependency for a kernel build, it can pass the dtschema
> steps.
>
> Finally, I created a reference PACKAGECONFIG for linux-yocto-dev
> to enable/disable the main package and wrappers (the wrappers are
> used by default). I haven't put this into kernel.bbclass yet, since
> it is version specific, and it needs to soak for a bit. But if
> there's demand, it is something that can be moved to kernel.bclass
> (along with a few other version specific dependencies).
>
> Note: I haven't done an AB run with these yet, so I could be
> missing some python dependencies, etc, but I wanted to get this
> out to the list in case others are running into this with
> v5.16+.
>
> Cheers,
>
> Bruce
>
>
> The following changes since commit 60409917ea3e7353f5b7ee831fe4ffaff0957d94:
>
>   glibc-tests: Do not set SYSTEMD_SERVICE (2021-11-23 14:04:02 +0000)
>
> are available in the Git repository at:
>
>   git://git.yoctoproject.org/poky-contrib zedd/kernel
>   http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel
>
> Bruce Ashfield (18):
>   linux-yocto/5.15: update to v5.15.2
>   linux-yocto/5.14: update to v5.14.18
>   linux-yocto/5.10: update to v5.10.79
>   kernel-yocto: conditionally enable stack protection checking on x86-64
>   linux-yocto-dev: bump to v5.16+
>   kernel: export native PKGCONFIG variables
>   python: introduce python3-dtschema
>   python: import jsonpointer from meta-python
>   python: import jsonschema from meta-python
>   python: import idna from meta-python
>   python: import rfc3339-validator from meta-python
>   python: import rfc3986-validator from meta-python
>   python: import webcolors from meta-python
>   python: import ruamel-yaml from meta-python
>   maintainers: update for kernel dtschema checking dependencies
>   linux-yocto: export pkgconfig variables to devshell
>   kernel: introduce python3-dtschema-wrapper
>   linux-yocto-dev: introduce dt-validation PACKAGECONFIG
>
> Max Krummenacher (2):
>   perf: sort-pmuevents: don't drop elements
>   perf: sort-pmuevents: allow for additional type qualifiers and storage
>     class
>
>  meta/classes/kernel.bbclass                   |  7 +++
>  meta/conf/distro/include/maintainers.inc      |  9 ++++
>  ...e-pytest-runner-to-test_requirements.patch | 32 +++++++++++++
>  .../python/python3-dtschema_2021.10.bb        | 15 ++++++
>  .../python/python3-idna_3.3.bb                | 19 ++++++++
>  .../python/python3-jsonpointer/run-ptest      |  3 ++
>  .../python/python3-jsonpointer_2.1.bb         | 26 ++++++++++
>  .../python/python3-jsonschema_3.2.0.bb        | 48 +++++++++++++++++++
>  .../python/python3-rfc3339-validator_0.1.4.bb | 20 ++++++++
>  .../python/python3-rfc3986-validator_0.1.1.bb | 23 +++++++++
>  .../python/python3-ruamel-yaml_0.17.16.bb     | 23 +++++++++
>  .../python/python3-webcolors/run-ptest        |  3 ++
>  .../python/python3-webcolors_1.11.1.bb        | 27 +++++++++++
>  .../python3-dtschema-wrapper/dt-doc-validate  | 20 ++++++++
>  .../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 ++++++++
>  .../dtc/python3-dtschema-wrapper/dt-validate  | 20 ++++++++
>  .../dtc/python3-dtschema-wrapper_2021.10.bb   | 17 +++++++
>  meta/recipes-kernel/linux/linux-yocto-dev.bb  | 10 +++-
>  .../linux/linux-yocto-rt_5.10.bb              |  6 +--
>  .../linux/linux-yocto-rt_5.14.bb              |  6 +--
>  .../linux/linux-yocto-rt_5.15.bb              |  6 +--
>  .../linux/linux-yocto-tiny_5.10.bb            |  8 ++--
>  .../linux/linux-yocto-tiny_5.14.bb            |  8 ++--
>  .../linux/linux-yocto-tiny_5.15.bb            |  8 ++--
>  meta/recipes-kernel/linux/linux-yocto.inc     | 13 +++++
>  meta/recipes-kernel/linux/linux-yocto_5.10.bb | 24 +++++-----
>  meta/recipes-kernel/linux/linux-yocto_5.14.bb | 26 +++++-----
>  meta/recipes-kernel/linux/linux-yocto_5.15.bb | 26 +++++-----
>  .../perf/perf/sort-pmuevents.py               | 28 ++++++-----
>  29 files changed, 429 insertions(+), 72 deletions(-)
>  create mode 100644 meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
>  create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
>  create mode 100644 meta/recipes-devtools/python/python3-idna_3.3.bb
>  create mode 100644 meta/recipes-devtools/python/python3-jsonpointer/run-ptest
>  create mode 100644 meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
>  create mode 100644 meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
>  create mode 100644 meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
>  create mode 100644 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
>  create mode 100644 meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
>  create mode 100644 meta/recipes-devtools/python/python3-webcolors/run-ptest
>  create mode 100644 meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
>  create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
>
> --
> 2.19.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH 00/20] kernel/kernel-yocto: consolidated pull request

[Ross Burton]

On Tue, 23 Nov 2021 at 21:57, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> We weren't able to even run the validation at all, due to the kernel
> calling pkg-config directly to check for libyaml, which is the target
> pkgconfig and failing on missing dependencies. Rather than patching
> the kernel, I've finally given in and made the do_compile task set
> the PKG_CONFIG variables to point to our native-sysroot.

The correct fix here would be the kernel should use $PKG_CONFIG (or
ideally, BUILD_PKGCONFIG or similar for host lookups) and then you can
override it with export PKGCONFIG=pkgconfig-native.

I see that some bits of the kernel already set PKG_CONFIG ?=
$(CROSS_COMPILE)pkg-config, but others directly use pkg-config.

Ross

Re: [OE-core] [PATCH 00/20] kernel/kernel-yocto: consolidated pull request

[Bruce Ashfield]

On Wed, Nov 24, 2021 at 9:46 AM Ross Burton <ross@burtonini.com> wrote:
>
> On Tue, 23 Nov 2021 at 21:57, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > We weren't able to even run the validation at all, due to the kernel
> > calling pkg-config directly to check for libyaml, which is the target
> > pkgconfig and failing on missing dependencies. Rather than patching
> > the kernel, I've finally given in and made the do_compile task set
> > the PKG_CONFIG variables to point to our native-sysroot.
>
> The correct fix here would be the kernel should use $PKG_CONFIG (or
> ideally, BUILD_PKGCONFIG or similar for host lookups) and then you can
> override it with export PKGCONFIG=pkgconfig-native.
>

Agreed, that's what I was referring to as the medium term solution. Once
I get 5.16 working, I'll look into sending patches upstream to the various
bare calls to pkgconfig

Bruce

> I see that some bits of the kernel already set PKG_CONFIG ?=
> $(CROSS_COMPILE)pkg-config, but others directly use pkg-config.
>
> Ross



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [PATCH 18/20] linux-yocto-dev: introduce dt-validation PACKAGECONFIG

[Bruce Ashfield]

On Tue, Nov 23, 2021 at 4:58 PM <bruce.ashfield@gmail.com> wrote:
>
> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>
> Add a PACKAGECONFIG to control which type of dt-validation
> is installed (full, or wrapped).
>
> This is currently in linux-yocto-dev, as 5.16 introduces
> the functionality of required dt schema validation.
>
> Other v5.16 providing recipes can use the base functionality
> as they desire (with or without PACKAGECONFIG).
>
> Over time, we can migrate the support the the core
> kernel classes .. in particular when more providers are at
> v5.16+
>
> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> ---
>  meta/recipes-kernel/linux/linux-yocto-dev.bb | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb
> index ed94ab2517..858772ba41 100644
> --- a/meta/recipes-kernel/linux/linux-yocto-dev.bb
> +++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb
> @@ -39,8 +39,14 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
>  DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
>  DEPENDS += "openssl-native util-linux-native"
>  DEPENDS += "gmp-native libmpc-native"
> -# yaml and dtschema are required for 5.16+ device tree validation
> -DEPENDS += "libyaml-native python3-dtschema-native"
> +
> +# yaml and dtschema are required for 5.16+ device tree validation, libyaml is checked
> +# via pkgconfig, so must always be present, but we can wrap the others to make them
> +# conditional
> +DEPENDS += "libyaml-native"
> +
> +PACKAGECONFIG ??= ""
> +DEPENDS += "${@bb.utils.contains('PACKAGECONFIG', 'dt-validation', 'python3-dtschema-native', 'python3-dtschema-wrapper-native', d)}"

In my recently updated master, I get the QA warning:

WARNING: linux-yocto-dev-5.16++gitAUTOINC+ac97e62b64_c425b06bc1-r0
do_configure: QA Issue: linux-yocto-dev: invalid PACKAGECONFIG:
dt-validation [invalid-packageconfig]

I can re-work this to use PACKAGECONFIG's rdepends for the full
validation case, and test and add the wrapper rdepends in the
no-validation case, alternatively, I could add a
"dt-validation-inhibit" PACKAGECONFIG, and put the rdepends on the
wrappers there.

I'm open to either, and will pick the first fow now :D

Bruce

>
>  COMPATIBLE_MACHINE = "(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64|qemuriscv64)"
>
> --
> 2.19.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II