[PATCH 00/20 v2] kernel: consolidated pull request

[PATCH 00/20 v2] kernel: consolidated pull request

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Richard,

Here's v2 of the pull request. I see the -stable updates merged, so they
are obviously not in v2.

New in v2:
  - reworked the python-dtschema recipe to properly have depends/rdepends
    to fix warnings
  - imported 4 new dependencies for the recipe
  - fixed up the kernel-dev PACKAGECONFIG to not generate a QA warning

The v1 pull request text follows from here:

There's a few routine -stable updates at the start of this, and then some
more painful (and RFC) commits.

As part of this series, we fix one pkg-config issue that was preventing
x86 from doing stack valildation during build. At one point it was
increasing my build times signficantly, but that seems to have gone
away now. If we see something like this on the AB, we'll have to keep
the fix, but disable it by default.

The next big chunk of changes are the ones we talked about during the
tech call today. The 5.16 kernel has introduced mandatory dtschema
checking for device trees built through the kernel.

That checking pulls in a reasonably large amount of dependencies to
the kernel build: libyaml, dtschema, jsonschema, etc.

We weren't able to even run the validation at all, due to the kernel
calling pkg-config directly to check for libyaml, which is the target
pkgconfig and failing on missing dependencies. Rather than patching
the kernel, I've finally given in and made the do_compile task set
the PKG_CONFIG variables to point to our native-sysroot. Since the
kenrel doesn't use pkgconfig for target decisions, and has quite
a few calls directly to pkgconfig, this is the best short term way
to get all those checks working.

Once we are past yaml, we need to create the dtschema recipe, and
bring in the dependencies from meta-python. I've put myself down as
the maintainer for all the new recipes, but otherwise, I haven't
changed or modified them.

BUT! there are scenarios where a non-conformant dts file is being
built, or we just don't want those dependencies in the built. To
work around that, I took the suggestion from this morning and
created a python3-dtschema-wrappers recipe. It contains three
scripts of the same name as the ones from the real package. When
put in the dependency for a kernel build, it can pass the dtschema
steps.

Finally, I created a reference PACKAGECONFIG for linux-yocto-dev
to enable/disable the main package and wrappers (the wrappers are
used by default). I haven't put this into kernel.bbclass yet, since
it is version specific, and it needs to soak for a bit. But if
there's demand, it is something that can be moved to kernel.bclass
(along with a few other version specific dependencies).

Note: I haven't done an AB run with these yet, so I could be
missing some python dependencies, etc, but I wanted to get this
out to the list in case others are running into this with
v5.16+.

Bruce

The following changes since commit 0ec51bf52a930982ac03976781010a638d4cb0ef:

  logrotate: add a link to upstream review for one of the patch (2021-11-24 10:00:10 +0000)

are available in the Git repository at:

  git://git.yoctoproject.org/poky-contrib zedd/kernel
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (18):
  linux-yocto-dev: bump to v5.16+
  kernel: export native PKGCONFIG variables
  python: introduce python3-dtschema
  python: import jsonpointer from meta-python
  python: import jsonschema from meta-python
  python: import idna from meta-python
  python: import rfc3339-validator from meta-python
  python: import rfc3986-validator from meta-python
  python: import webcolors from meta-python
  python: import ruamel-yaml from meta-python
  python: import pyrsistent from meta-python
  python: import rfc3987 from meta-pyton
  python: import strict-rfc3339 from meta-python
  python: import vcversioner from meta-python
  maintainers: update for kernel dtschema checking dependencies
  linux-yocto: export pkgconfig variables to devshell
  kernel: introduce python3-dtschema-wrapper
  linux-yocto-dev: introduce dt-validation PACKAGECONFIG

Max Krummenacher (2):
  perf: sort-pmuevents: don't drop elements
  perf: sort-pmuevents: allow for additional type qualifiers and storage
    class

 meta/classes/kernel.bbclass                   |  7 +++
 meta/conf/distro/include/maintainers.inc      | 13 +++++
 ...e-pytest-runner-to-test_requirements.patch | 32 +++++++++++++
 .../python/python3-dtschema_2021.10.bb        | 16 +++++++
 .../python/python3-idna_3.3.bb                | 19 ++++++++
 .../python/python3-jsonpointer/run-ptest      |  3 ++
 .../python/python3-jsonpointer_2.1.bb         | 26 ++++++++++
 .../python/python3-jsonschema_3.2.0.bb        | 48 +++++++++++++++++++
 .../python/python3-pyrsistent_0.18.0.bb       | 14 ++++++
 .../python/python3-rfc3339-validator_0.1.4.bb | 20 ++++++++
 .../python/python3-rfc3986-validator_0.1.1.bb | 23 +++++++++
 .../python/python3-rfc3987_1.3.8.bb           | 10 ++++
 .../python/python3-ruamel-yaml_0.17.16.bb     | 23 +++++++++
 .../python/python3-strict-rfc3339_0.7.bb      | 10 ++++
 .../python/python3-vcversioner_2.16.0.0.bb    | 12 +++++
 .../python/python3-webcolors/run-ptest        |  3 ++
 .../python/python3-webcolors_1.11.1.bb        | 27 +++++++++++
 .../python3-dtschema-wrapper/dt-doc-validate  | 20 ++++++++
 .../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 ++++++++
 .../dtc/python3-dtschema-wrapper/dt-validate  | 20 ++++++++
 .../dtc/python3-dtschema-wrapper_2021.10.bb   | 17 +++++++
 meta/recipes-kernel/linux/linux-yocto-dev.bb  | 12 ++++-
 meta/recipes-kernel/linux/linux-yocto.inc     |  9 ++++
 .../perf/perf/sort-pmuevents.py               | 28 ++++++-----
 24 files changed, 419 insertions(+), 13 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
 create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
 create mode 100644 meta/recipes-devtools/python/python3-idna_3.3.bb
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
 create mode 100644 meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
 create mode 100644 meta/recipes-devtools/python/python3-pyrsistent_0.18.0.bb
 create mode 100644 meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
 create mode 100644 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
 create mode 100644 meta/recipes-devtools/python/python3-rfc3987_1.3.8.bb
 create mode 100644 meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
 create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339_0.7.bb
 create mode 100644 meta/recipes-devtools/python/python3-vcversioner_2.16.0.0.bb
 create mode 100644 meta/recipes-devtools/python/python3-webcolors/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb

-- 
2.19.1

[PATCH 01/20] linux-yocto-dev: bump to v5.16+

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Bumping the version of -dev ot 5.16. We also pickup two new
dependencies to support kernel dtschema validation. This
requirement was introduced by kernel commit:

   commit 53182e81f47d4ea0c727c49ad23cb782173ab849
   Author: Rob Herring <robh@kernel.org>
   Date:   Mon Sep 13 09:51:46 2021 -0500

       kbuild: Enable DT schema checks for %.dtb targets

       It is possible to build a single dtb, but not with DT schema validation
       enabled. Enable the schema validation to run for %.dtb and %.dtbo
       targets. Anyone building a dtb for a specific platform *should* pay
       attention to schema warnings.

       This could be supported with a separate %.dt.yaml target instead.
       However, the .dt.yaml format is considered an intermediate format and
       could possibly go away at some point if schema checking is integrated
       into dtc. Also, the plan is to enable the schema checks by default once
       platforms are free of warnings, and this is a move in that direction.

       Cc: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
       Cc: Tom Rini <trini@konsulko.com>
       Cc: Masahiro Yamada <masahiroy@kernel.org>
       Cc: linux-kbuild@vger.kernel.org
       Signed-off-by: Rob Herring <robh@kernel.org>
       Acked-by: Masahiro Yamada <masahiroy@kernel.org>
       Link: https://lore.kernel.org/r/20210913145146.766080-1-robh@kernel.org

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/linux-yocto-dev.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb
index 005b688778..ed94ab2517 100644
--- a/meta/recipes-kernel/linux/linux-yocto-dev.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb
@@ -30,7 +30,7 @@ SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name
 SRCREV_machine ?= '${@oe.utils.conditional("PREFERRED_PROVIDER_virtual/kernel", "linux-yocto-dev", "${AUTOREV}", "29594404d7fe73cd80eaa4ee8c43dcc53970c60e", d)}'
 SRCREV_meta ?= '${@oe.utils.conditional("PREFERRED_PROVIDER_virtual/kernel", "linux-yocto-dev", "${AUTOREV}", "29594404d7fe73cd80eaa4ee8c43dcc53970c60e", d)}'
 
-LINUX_VERSION ?= "5.15+"
+LINUX_VERSION ?= "5.16+"
 LINUX_VERSION_EXTENSION ?= "-yoctodev-${LINUX_KERNEL_TYPE}"
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
@@ -39,6 +39,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
 DEPENDS += "openssl-native util-linux-native"
 DEPENDS += "gmp-native libmpc-native"
+# yaml and dtschema are required for 5.16+ device tree validation
+DEPENDS += "libyaml-native python3-dtschema-native"
 
 COMPATIBLE_MACHINE = "(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64|qemuriscv64)"
 
-- 
2.19.1

[PATCH 02/20] kernel: export native PKGCONFIG variables

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

In a similar manner to cml1.bbclass, we export the pkg-config
variables to allow a direct call to pkg-config access to the
native sysroot versus the target sysroot.

The kernel doesn't use pkg-config for target configuration,
and has many explicit calls to pkg-config, without the
possibility of easy override to pkg-config-native.

The calls to pkg-config could be made cross friendly via
replacement with make variables, but until that effort is
undertaken upstream, we need a bridge approach.

In particular, this is required for dtschema validation,
which is a requirement in kernel 5.16+

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/classes/kernel.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index e0b752de19..2d219cb5e5 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -326,6 +326,13 @@ KERNEL_DEBUG_TIMESTAMPS ??= "0"
 
 kernel_do_compile() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
+
+	# setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native)
+	export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig"
+	export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig"
+	export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR"
+	export PKG_CONFIG_SYSROOT_DIR=""
+
 	if [ "${KERNEL_DEBUG_TIMESTAMPS}" != "1" ]; then
 		# kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
 		# be set....
-- 
2.19.1

[PATCH 03/20] python: introduce python3-dtschema

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

kernel commit 53182e81f47d [kbuild: Enable DT schema checks for %.dtb
targets], which is part of v5.16+, makes the checking of dtb schemas
manditory.

To support the checking, we introduce this new recipe dtschema.

Dependencies will be imported in subsequent commits, so we can
track the dependency need on a per-import basis.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-dtschema_2021.10.bb           | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb

diff --git a/meta/recipes-devtools/python/python3-dtschema_2021.10.bb b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
new file mode 100644
index 0000000000..f16232d610
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
@@ -0,0 +1,16 @@
+DESCRIPTION = "Tooling for devicetree validation using YAML and jsonschema"
+HOMEPAGE = "https://github.com/devicetree-org/dt-schema"
+LICENSE = "BSD-2-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=457495c8fa03540db4a576bf7869e811"
+
+inherit pypi setuptools3
+
+PYPI_PACKAGE = "dtschema"
+
+SRC_URI[md5sum] = "d2f0d847ba2ad4d223317ef7304342b0"
+SRC_URI[sha256sum] = "d09c94d13f46e6674ba11ff31220651ad1b02dae860f5a87905dfac6b8d768d9"
+
+DEPENDS += "python3-setuptools-scm"
+RDEPENDS:${PN} += "python3-ruamel-yaml python3-jsonschema python3-rfc3987"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 04/20] python: import jsonpointer from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

required to support python3-json, which is required for python3-dtschema
and device tree checking by the kernel.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-jsonpointer/run-ptest      |  3 +++
 .../python/python3-jsonpointer_2.1.bb         | 26 +++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-jsonpointer_2.1.bb

diff --git a/meta/recipes-devtools/python/python3-jsonpointer/run-ptest b/meta/recipes-devtools/python/python3-jsonpointer/run-ptest
new file mode 100644
index 0000000000..51e609f4ba
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-jsonpointer/run-ptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3 tests.py
diff --git a/meta/recipes-devtools/python/python3-jsonpointer_2.1.bb b/meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
new file mode 100644
index 0000000000..fb3d9e0da9
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-jsonpointer_2.1.bb
@@ -0,0 +1,26 @@
+SUMMARY = "Resolve JSON Pointers in Python"
+HOMEPAGE = "https://github.com/stefankoegl/python-json-pointer"
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=32b15c843b7a329130f4e266a281ebb3"
+
+inherit pypi ptest setuptools3
+
+SRC_URI[sha256sum] = "5a34b698db1eb79ceac454159d3f7c12a451a91f6334a4f638454327b7a89962"
+
+RDEPENDS:${PN} += " \
+    ${PYTHON_PN}-json \
+"
+
+BBCLASSEXTEND = "native nativesdk"
+
+SRC_URI += " \
+	file://run-ptest \
+"
+
+RDEPENDS:${PN}-ptest += " \
+	${PYTHON_PN}-pytest \
+"
+
+do_install_ptest() {
+	cp -f ${S}/tests.py ${D}${PTEST_PATH}/
+}
-- 
2.19.1

[PATCH 05/20] python: import jsonschema from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing this package as a dependency to python3-dt-schema,
which is required for kernel device tree validation.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-jsonschema_3.2.0.bb        | 48 +++++++++++++++++++
 1 file changed, 48 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb

diff --git a/meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb b/meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
new file mode 100644
index 0000000000..c7efe33a83
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-jsonschema_3.2.0.bb
@@ -0,0 +1,48 @@
+SUMMARY = "An implementation of JSON Schema validation for Python"
+HOMEPAGE = "https://github.com/Julian/jsonschema"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://COPYING;md5=7a60a81c146ec25599a3e1dabb8610a8 \
+                    file://json/LICENSE;md5=9d4de43111d33570c8fe49b4cb0e01af"
+DEPENDS += "${PYTHON_PN}-vcversioner-native ${PYTHON_PN}-setuptools-scm-native"
+
+SRC_URI[md5sum] = "f1a0b5011f05a02a8dee1070cd10a26d"
+SRC_URI[sha256sum] = "c8a85b28d377cc7737e46e2d9f2b4f44ee3c0e1deac6bf46ddefc7187d30797a"
+
+inherit pypi setuptools3
+
+PACKAGECONFIG ??= "format"
+PACKAGECONFIG[format] = ",,,\
+    ${PYTHON_PN}-idna \
+    ${PYTHON_PN}-jsonpointer \
+    ${PYTHON_PN}-webcolors \
+    ${PYTHON_PN}-rfc3987 \
+    ${PYTHON_PN}-strict-rfc3339 \
+"
+PACKAGECONFIG[nongpl] = ",,,\
+    ${PYTHON_PN}-idna \
+    ${PYTHON_PN}-jsonpointer \
+    ${PYTHON_PN}-webcolors \
+    ${PYTHON_PN}-rfc3986-validator \
+    ${PYTHON_PN}-rfc3339-validator \
+"
+
+RDEPENDS:${PN} += " \
+    ${PYTHON_PN}-attrs \
+    ${PYTHON_PN}-core \
+    ${PYTHON_PN}-datetime \
+    ${PYTHON_PN}-importlib-metadata \
+    ${PYTHON_PN}-io \
+    ${PYTHON_PN}-json \
+    ${PYTHON_PN}-netclient \
+    ${PYTHON_PN}-numbers \
+    ${PYTHON_PN}-pkgutil \
+    ${PYTHON_PN}-pprint \
+    ${PYTHON_PN}-pyrsistent \
+    ${PYTHON_PN}-shell \
+    ${PYTHON_PN}-six \
+    ${PYTHON_PN}-unittest \
+    ${PYTHON_PN}-setuptools-scm \
+    ${PYTHON_PN}-zipp \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 06/20] python: import idna from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing idna from meta-python, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-idna_3.3.bb                | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-idna_3.3.bb

diff --git a/meta/recipes-devtools/python/python3-idna_3.3.bb b/meta/recipes-devtools/python/python3-idna_3.3.bb
new file mode 100644
index 0000000000..a0e6b79a56
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-idna_3.3.bb
@@ -0,0 +1,19 @@
+SUMMARY = "Internationalised Domain Names in Applications"
+HOMEPAGE = "https://github.com/kjd/idna"
+LICENSE = "BSD-3-Clause & Python-2.0 & Unicode"
+LIC_FILES_CHKSUM = "file://LICENSE.md;md5=239668a7c6066d9e0c5382e9c8c6c0e1"
+
+SRC_URI[sha256sum] = "9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d"
+
+inherit pypi setuptools3
+
+# Remove bundled egg-info
+do_compile:prepend() {
+    rm -rf ${S}/idna.egg-info
+}
+
+RDEPENDS:${PN}:class-target = "\
+    ${PYTHON_PN}-codecs \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 07/20] python: import rfc3339-validator from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing rfc3339-validator from meta-python, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-rfc3339-validator_0.1.4.bb | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb

diff --git a/meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb b/meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
new file mode 100644
index 0000000000..7f1bbaf97d
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-rfc3339-validator_0.1.4.bb
@@ -0,0 +1,20 @@
+SUMMARY = "A pure python RFC3339 validator"
+HOMEPAGE = "https://github.com/naimetti/rfc3339-validator"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=a21b13b5a996f08f7e0b088aa38ce9c6"
+
+FILESEXTRAPATHS:prepend := "${THISDIR}/python-rfc3339-validator:"
+
+SRC_URI[sha256sum] = "138a2abdf93304ad60530167e51d2dfb9549521a836871b88d7f4695d0022f6b"
+
+PYPI_PACKAGE = "rfc3339_validator"
+
+inherit pypi setuptools3
+
+RDEPENDS:${PN} += "\
+    ${PYTHON_PN}-core \
+    ${PYTHON_PN}-datetime \
+    ${PYTHON_PN}-six \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 08/20] python: import rfc3986-validator from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing rfc3986-validator, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 ...e-pytest-runner-to-test_requirements.patch | 32 +++++++++++++++++++
 .../python/python3-rfc3986-validator_0.1.1.bb | 23 +++++++++++++
 2 files changed, 55 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
 create mode 100644 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb

diff --git a/meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch b/meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
new file mode 100644
index 0000000000..8a646874e8
--- /dev/null
+++ b/meta/recipes-devtools/python/python-rfc3986-validator/0001-setup.py-move-pytest-runner-to-test_requirements.patch
@@ -0,0 +1,32 @@
+From 3531ff73631a0d59234eb4713e7b3a7f5ea57bbb Mon Sep 17 00:00:00 2001
+From: Nicola Lunghi <nicola.lunghi@jci.com>
+Date: Thu, 14 Nov 2019 12:17:51 +0000
+Subject: [PATCH] setup.py: move pytest-runner to test_requirements
+
+This fixes an issue with yocto build.
+pytest-runner is only needed when running tests.
+
+Upstream-Status: Pending
+---
+ setup.py | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index 53ebea7..ebb0de2 100644
+--- a/setup.py
++++ b/setup.py
+@@ -10,9 +10,9 @@ with open('README.md') as readme_file:
+ 
+ requirements = []
+ 
+-setup_requirements = ['pytest-runner', ]
++setup_requirements = []
+ 
+-test_requirements = ['pytest>=3', ]
++test_requirements = ['pytest>=3', 'pytest-runner']
+ 
+ setup(
+     author="Nicolas Aimetti",
+-- 
+2.20.1
+
diff --git a/meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb b/meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
new file mode 100644
index 0000000000..68b3db1f48
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb
@@ -0,0 +1,23 @@
+SUMMARY = "Pure python rfc3986 validator"
+HOMEPAGE = "https://github.com/naimetti/rfc3986-validator"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=a21b13b5a996f08f7e0b088aa38ce9c6"
+
+FILESEXTRAPATHS:prepend := "${THISDIR}/python-rfc3986-validator:"
+
+SRC_URI[md5sum] = "47f7657b790aaf6011a1ab3d86c6be95"
+SRC_URI[sha256sum] = "3d44bde7921b3b9ec3ae4e3adca370438eccebc676456449b145d533b240d055"
+
+PYPI_PACKAGE = "rfc3986_validator"
+
+inherit pypi setuptools3
+
+SRC_URI:append = " \
+    file://0001-setup.py-move-pytest-runner-to-test_requirements.patch \
+"
+
+RDEPENDS:${PN} += "\
+    python3-core \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 09/20] python: import webcolors from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing webcolors, which is a dependency of
python3-jsonschema. These packges are required to support
the validation of kernel device trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-webcolors/run-ptest        |  3 +++
 .../python/python3-webcolors_1.11.1.bb        | 27 +++++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-webcolors/run-ptest
 create mode 100644 meta/recipes-devtools/python/python3-webcolors_1.11.1.bb

diff --git a/meta/recipes-devtools/python/python3-webcolors/run-ptest b/meta/recipes-devtools/python/python3-webcolors/run-ptest
new file mode 100644
index 0000000000..3385d68939
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-webcolors/run-ptest
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+pytest -o log_cli=true -o log_cli_level=INFO | sed -e 's/\[...%\]//g'| sed -e 's/PASSED/PASS/g'| sed -e 's/FAILED/FAIL/g'|sed -e 's/SKIPED/SKIP/g'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS"){printf "%s: %s\n", $NF, $0}else{print}}'| awk '{if ($NF=="PASS" || $NF=="FAIL" || $NF=="SKIP" || $NF=="XFAIL" || $NF=="XPASS") {$NF="";print $0}else{print}}'
diff --git a/meta/recipes-devtools/python/python3-webcolors_1.11.1.bb b/meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
new file mode 100644
index 0000000000..2ec036ef35
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-webcolors_1.11.1.bb
@@ -0,0 +1,27 @@
+SUMMARY = "Simple Python module for working with HTML/CSS color definitions."
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=25b90379a52351261c51272e7923d240"
+
+SRC_URI[md5sum] = "54d28a7c80b3e4d974ec2fee86768be9"
+SRC_URI[sha256sum] = "76f360636957d1c976db7466bc71dcb713bb95ac8911944dffc55c01cb516de6"
+
+inherit pypi setuptools3 ptest
+
+RDEPENDS:${PN}:class-target = "\
+    ${PYTHON_PN}-stringold \
+"
+
+SRC_URI += " \
+    file://run-ptest \
+"
+
+RDEPENDS:${PN}-ptest += " \
+    ${PYTHON_PN}-pytest \
+"
+
+do_install_ptest() {
+    install -d ${D}${PTEST_PATH}/tests
+    cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 10/20] python: import ruamel-yaml from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Importing ruamel-yaml from meta-python. This package is a
dependency of python3-dtschema, which is required to validate
kernel devicetrees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-ruamel-yaml_0.17.16.bb     | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb

diff --git a/meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb b/meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
new file mode 100644
index 0000000000..e64f196004
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-ruamel-yaml_0.17.16.bb
@@ -0,0 +1,23 @@
+SUMMARY = "YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order."
+AUTHOR = "Anthon van der Neut"
+
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=fa0a51dfb461e2f803969e0f3fa71dfe"
+
+PYPI_PACKAGE = "ruamel.yaml"
+
+inherit pypi setuptools3
+
+SRC_URI[sha256sum] = "1a771fc92d3823682b7f0893ad56cb5a5c87c48e62b5399d6f42c8759a583b33"
+
+RDEPENDS:${PN} += "\
+    ${PYTHON_PN}-shell \
+    ${PYTHON_PN}-datetime \
+    ${PYTHON_PN}-netclient \
+"
+
+do_install:prepend() {
+    export RUAMEL_NO_PIP_INSTALL_CHECK=1
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 11/20] python: import pyrsistent from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

pyrsistent is a dependency of python3-jsonschema, which is being
imported to support kernel device tree validation.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-pyrsistent_0.18.0.bb            | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-pyrsistent_0.18.0.bb

diff --git a/meta/recipes-devtools/python/python3-pyrsistent_0.18.0.bb b/meta/recipes-devtools/python/python3-pyrsistent_0.18.0.bb
new file mode 100644
index 0000000000..0b4d2564f6
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-pyrsistent_0.18.0.bb
@@ -0,0 +1,14 @@
+SUMMARY = "Persistent/Immutable/Functional data structures for Python"
+HOMEPAGE = "https://github.com/tobgu/pyrsistent"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE.mit;md5=a4b94c2b800b582a8d3925a9939cbf44"
+
+SRC_URI[sha256sum] = "773c781216f8c2900b42a7b638d5b517bb134ae1acbebe4d1e8f1f41ea60eb4b"
+
+inherit pypi setuptools3
+
+RDEPENDS:${PN} += " \
+    ${PYTHON_PN}-numbers \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 12/20] python: import rfc3987 from meta-pyton

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

rfc3987 is a depenency of jsonschema and dtschema, which are being
added to support kernel device tree validation.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-devtools/python/python3-rfc3987_1.3.8.bb | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-rfc3987_1.3.8.bb

diff --git a/meta/recipes-devtools/python/python3-rfc3987_1.3.8.bb b/meta/recipes-devtools/python/python3-rfc3987_1.3.8.bb
new file mode 100644
index 0000000000..80e2aa5bc2
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-rfc3987_1.3.8.bb
@@ -0,0 +1,10 @@
+SUMMARY = "Parsing and validation of URIs (RFC 3986) and IRIs (RFC 3987)"
+LICENSE = "GPLv3+"
+LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=9;md5=2b723edf67b2f3088bc5e339b1ceda2d"
+
+SRC_URI[md5sum] = "b6c4028acdc788a9ba697e1c1d6b896c"
+SRC_URI[sha256sum] = "d3c4d257a560d544e9826b38bc81db676890c79ab9d7ac92b39c7a253d5ca733"
+
+inherit pypi setuptools3
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 13/20] python: import strict-rfc3339 from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

strict-rfc3339 is a dependency of jsonschema, which is being
imported to support kernel device tree valiation.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-strict-rfc3339_0.7.bb               | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339_0.7.bb

diff --git a/meta/recipes-devtools/python/python3-strict-rfc3339_0.7.bb b/meta/recipes-devtools/python/python3-strict-rfc3339_0.7.bb
new file mode 100644
index 0000000000..52ae9ebe9a
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-strict-rfc3339_0.7.bb
@@ -0,0 +1,10 @@
+SUMMARY = "Strict, simple, lightweight RFC3339 function.s"
+LICENSE = "GPLv3"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=8f0e2cd40e05189ec81232da84bd6e1a"
+
+SRC_URI[md5sum] = "4d9b635b4df885bc37bc1189d66c9abc"
+SRC_URI[sha256sum] = "5cad17bedfc3af57b399db0fed32771f18fc54bbd917e85546088607ac5e1277"
+
+inherit pypi setuptools3
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 14/20] python: import vcversioner from meta-python

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

vcversioner is a dependency of jsonschema, which is being
imported to support kernel device tree validation.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../python/python3-vcversioner_2.16.0.0.bb           | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-vcversioner_2.16.0.0.bb

diff --git a/meta/recipes-devtools/python/python3-vcversioner_2.16.0.0.bb b/meta/recipes-devtools/python/python3-vcversioner_2.16.0.0.bb
new file mode 100644
index 0000000000..149078c51b
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-vcversioner_2.16.0.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "Python vcversioner, automagically update the project's version"
+HOMEPAGE = "https://github.com/habnabit/vcversioner"
+
+LICENSE = "ISC"
+LIC_FILES_CHKSUM = "file://PKG-INFO;md5=827a7a91a8d20d3c666b665cd96db8e3"
+
+SRC_URI[md5sum] = "aab6ef5e0cf8614a1b1140ed5b7f107d"
+SRC_URI[sha256sum] = "dae60c17a479781f44a4010701833f1829140b1eeccd258762a74974aa06e19b"
+
+inherit pypi setuptools3
+
+BBCLASSEXTEND = "native"
-- 
2.19.1

[PATCH 15/20] maintainers: update for kernel dtschema checking dependencies

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

These packages are required for dtschema checking in the
kernel build. Adding myself as the maintainer.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/conf/distro/include/maintainers.inc | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 5423358e77..c717cfbbe1 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -597,16 +597,21 @@ RECIPE_MAINTAINER:pn-python3-cython = "Oleksandr Kravchuk <open.source@oleksandr
 RECIPE_MAINTAINER:pn-python3-dbus = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-dbusmock = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-docutils = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
+RECIPE_MAINTAINER:pn-python3-dtschema = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodome = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodomex = "Joshua Watt <JPEWhacker@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-pyrsistent = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-extras = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-git = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-gitdb = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-hypothesis = "Tim Orling <timothy.t.orling@intel.com>"
+RECIPE_MAINTAINER:pn-python3-idna = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-importlib-metadata = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-iniconfig = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-iniparse = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-jinja2 = "Richard Purdie <richard.purdie@linuxfoundation.org>"
+RECIPE_MAINTAINER:pn-python3-python3-jsonpointer = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-python3-jsonschema = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-libarchive-c = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-magic = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
@@ -628,6 +633,10 @@ RECIPE_MAINTAINER:pn-python3-pygments = "Oleksandr Kravchuk <open.source@oleksan
 RECIPE_MAINTAINER:pn-python3-pygobject = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-pyparsing = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-pytest = "Tim Orling <timothy.t.orling@intel.com>"
+RECIPE_MAINTAINER:pn-python3-rfc3339-validator = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-rfc3986-validator = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-rfc3987 = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-ruamel-yaml = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-scons = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-scons-native = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-setuptools = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
@@ -636,12 +645,15 @@ RECIPE_MAINTAINER:pn-python3-six = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-smartypants = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-smmap = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-sortedcontainers = "Tim Orling <timothy.t.orling@intel.com>"
+RECIPE_MAINTAINER:pn-python3-strict-rfc3339 = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-subunit = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-testtools = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-toml = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-tomli = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander Kanavin <alex.kanavin@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-vcversioner = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-wcwidth = "Tim Orling <timothy.t.orling@intel.com>"
+RECIPE_MAINTAINER:pn-python3-webcolors = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-zipp = "Tim Orling <timothy.t.orling@intel.com>"
 RECIPE_MAINTAINER:pn-qemu = "Richard Purdie <richard.purdie@linuxfoundation.org>"
 RECIPE_MAINTAINER:pn-qemu-helper-native = "Richard Purdie <richard.purdie@linuxfoundation.org>"
-- 
2.19.1

[PATCH 16/20] linux-yocto: export pkgconfig variables to devshell

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Not all of the kernel host/build Makefiles allow pkg-config
to be overriden to pkg-config-native. Exporting these variables
allow us to debug host tool issues with the kernel build.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/linux-yocto.inc | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/meta/recipes-kernel/linux/linux-yocto.inc b/meta/recipes-kernel/linux/linux-yocto.inc
index 15fc7ff724..a3b2416367 100644
--- a/meta/recipes-kernel/linux/linux-yocto.inc
+++ b/meta/recipes-kernel/linux/linux-yocto.inc
@@ -61,3 +61,12 @@ KERNEL_FEATURES:append:qemuall=" features/kernel-sample/kernel-sample.scc"
 
 KERNEL_DEBUG_OPTIONS ?= "stack"
 KERNEL_EXTRA_ARGS:append:x86-64 = "${@bb.utils.contains('KERNEL_DEBUG_OPTIONS', 'stack', 'HOST_LIBELF_LIBS="-L${RECIPE_SYSROOT_NATIVE}/usr/lib/pkgconfig/../../../usr/lib/ -lelf"', '', d)}"
+
+do_devshell:prepend() {
+    # setup native pkg-config variables (kconfig scripts call pkg-config directly, cannot generically be overriden to pkg-config-native)
+    d.setVar("PKG_CONFIG_DIR", "${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig")
+    d.setVar("PKG_CONFIG_PATH", "${PKG_CONFIG_DIR}:${STAGING_DATADIR_NATIVE}/pkgconfig")
+    d.setVar("PKG_CONFIG_LIBDIR", "${PKG_CONFIG_DIR}")
+    d.setVarFlag("PKG_CONFIG_SYSROOT_DIR", "unexport", "1")
+    d.appendVar("OE_TERMINAL_EXPORTS", " PKG_CONFIG_DIR PKG_CONFIG_PATH PKG_CONFIG_LIBDIR PKG_CONFIG_SYSROOT_DIR")
+}
-- 
2.19.1

[PATCH 17/20] kernel: introduce python3-dtschema-wrapper

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

The 5.16 kernel introduced mandatory schema checking on any dtb file
built through the kernel.

That funcionality is provided via python3-dt-schema.

The dependencies to enable that functionality is not small, and may
not always be desired (in particular on architectures that do not
support dtbs, or in development cycles). It may also be useful for
allowing a non-conformant dts to be compiled.

This commit introduces a set of wrapper scripts that when added
as a depenency to the kernel, can pass both the validation testing
and validation steps of a dts.

We use the wrapper by default, while more dts file are brought up
to spec and the dt-validation matures upstream.

To enable valiation, simply add 'dt-valiation' to the PACKAGECONFIG
variable and the full dt-schema package will be selected and
validation will occur.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/conf/distro/include/maintainers.inc      |  1 +
 .../python3-dtschema-wrapper/dt-doc-validate  | 20 +++++++++++++++++++
 .../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 +++++++++++++++++++
 .../dtc/python3-dtschema-wrapper/dt-validate  | 20 +++++++++++++++++++
 .../dtc/python3-dtschema-wrapper_2021.10.bb   | 17 ++++++++++++++++
 5 files changed, 78 insertions(+)
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
 create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index c717cfbbe1..464714a7a9 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -598,6 +598,7 @@ RECIPE_MAINTAINER:pn-python3-dbus = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-dbusmock = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-docutils = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
 RECIPE_MAINTAINER:pn-python3-dtschema = "Bruce Ashfield <bruce.ashfield@gmail.com>"
+RECIPE_MAINTAINER:pn-python3-dtschema-wrapper = "Bruce Ashfield <bruce.ashfield@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodome = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pycryptodomex = "Joshua Watt <JPEWhacker@gmail.com>"
 RECIPE_MAINTAINER:pn-python3-pyrsistent = "Bruce Ashfield <bruce.ashfield@gmail.com>"
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
new file mode 100644
index 0000000000..2aa57851c7
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-doc-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield <bruce.ashfield@gmail.com>
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+    case "$arg" in
+        --version)
+            echo "v2021.10"
+            ;;
+    esac
+done
+
+# TBD: left for future consideration
+# exec dt-doc-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
new file mode 100644
index 0000000000..24b89d8619
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-mk-schema wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield <bruce.ashfield@gmail.com>
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+    case "$arg" in
+        --version)
+            echo "v2021.10"
+            ;;
+    esac
+done
+
+# TBD: left for future consideration
+# exec dt-mk-schema.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
new file mode 100644
index 0000000000..8a4710a7ed
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield <bruce.ashfield@gmail.com>
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+    case "$arg" in
+        --version)
+            echo "v2021.10"
+            ;;
+    esac
+done
+
+# TBD: left for future consideration
+# exec dt-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
new file mode 100644
index 0000000000..c869274d09
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
@@ -0,0 +1,17 @@
+DESCRIPTION = "Wrapper for tooling for devicetree validation using YAML and jsonschema"
+HOMEPAGE = "https://yoctoproject.org"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "file://dt-doc-validate \
+           file://dt-mk-schema \
+           file://dt-validate"
+
+do_install() {
+    install -d ${D}${bindir}/
+    install -m 755 ${WORKDIR}/dt-doc-validate ${D}${bindir}/
+    install -m 755 ${WORKDIR}/dt-mk-schema ${D}${bindir}/
+    install -m 755 ${WORKDIR}/dt-validate ${D}${bindir}/
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.19.1

[PATCH 18/20] linux-yocto-dev: introduce dt-validation PACKAGECONFIG

[bruce.ashfield]

From: Bruce Ashfield <bruce.ashfield@gmail.com>

Add a PACKAGECONFIG to control which type of dt-validation
is installed (full, or wrapped).

This is currently in linux-yocto-dev, as 5.16 introduces
the functionality of required dt schema validation.

Other v5.16 providing recipes can use the base functionality
as they desire (with or without PACKAGECONFIG).

Over time, we can migrate the support the the core
kernel classes .. in particular when more providers are at
v5.16+

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/linux/linux-yocto-dev.bb | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb
index ed94ab2517..6b6ea9a7e8 100644
--- a/meta/recipes-kernel/linux/linux-yocto-dev.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb
@@ -39,8 +39,16 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=6bc538ed5bd9a7fc9398086aedcd7e46"
 DEPENDS += "${@bb.utils.contains('ARCH', 'x86', 'elfutils-native', '', d)}"
 DEPENDS += "openssl-native util-linux-native"
 DEPENDS += "gmp-native libmpc-native"
-# yaml and dtschema are required for 5.16+ device tree validation
-DEPENDS += "libyaml-native python3-dtschema-native"
+
+# yaml and dtschema are required for 5.16+ device tree validation, libyaml is checked
+# via pkgconfig, so must always be present, but we can wrap the others to make them
+# conditional
+DEPENDS += "libyaml-native"
+
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dt-validation] = ",,python3-dtschema-native"
+# we need the wrappers if validation isn't in the packageconfig
+DEPENDS += "${@bb.utils.contains('PACKAGECONFIG', 'dt-validation', '', 'python3-dtschema-wrapper-native', d)}"
 
 COMPATIBLE_MACHINE = "(qemuarm|qemux86|qemuppc|qemumips|qemumips64|qemux86-64|qemuriscv64)"
 
-- 
2.19.1

[PATCH 19/20] perf: sort-pmuevents: don't drop elements

[bruce.ashfield]

From: Max Krummenacher <max.oss.09@gmail.com>

If a struct element neither has an element cpuid or name it gets silenty
dropped.
Kernel 5.15 for some ARCHs have at least one array of structs matching
this.
e.g. for arm pmu-events.c:

|#include "pmu-events/pmu-events.h"
struct pmu_events_map pmu_events_map[] = {
{
	.cpuid = 0,
	.version = 0,
	.type = 0,
	.table = 0,
},
};

struct pmu_sys_events pmu_sys_event_tables[] = {
	{
		.table = 0
	},
};

Before this patch the second struct is translated to an empty array::

struct pmu_sys_events pmu_sys_event_tables[] = {
};

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 meta/recipes-kernel/perf/perf/sort-pmuevents.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
index 5ddf0f144f..4f841eb822 100755
--- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
+++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
@@ -61,6 +61,8 @@ for struct in re.findall( struct_block_regex, data ):
             #print( "    name found: %s" % name.group(1) )
             entry_dict[struct[1]]['fields'][name.group(1)] = entry
         
+        if not entry_dict[struct[1]]['fields']:
+            entry_dict[struct[1]]['fields']['0'] = entry
 
 # created ordered dictionaries from the captured values. These are ordered by
 # a sorted() iteration of the keys. We don't care about the order we read
-- 
2.19.1

[PATCH 20/20] perf: sort-pmuevents: allow for additional type qualifiers and storage class

[bruce.ashfield]

From: Max Krummenacher <max.oss.09@gmail.com>

With kernel 5.16 some structs in pmu-events do get a const qualifier, some
a static const storage class and qualifier.

The current sort-pmuevents cannot cope with that and drops all struct
arrays with such additional elements. This then leads to compiler errors.

Allow '^struct', '^const struct', '^static struct', '^static const struct'.

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../perf/perf/sort-pmuevents.py               | 30 ++++++++++---------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
index 4f841eb822..09ba3328a7 100755
--- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
+++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
@@ -33,10 +33,10 @@ if os.path.exists(outfile):
 with open(infile, 'r') as file:
     data = file.read()
 
-preamble_regex = re.compile( '^(.*?)^struct', re.MULTILINE | re.DOTALL )
+preamble_regex = re.compile( '^(.*?)^(struct|const struct|static struct|static const struct)', re.MULTILINE | re.DOTALL )
 
 preamble = re.search( preamble_regex, data )
-struct_block_regex = re.compile( '^struct.*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
+struct_block_regex = re.compile( '^(struct|const struct|static struct|static const struct).*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
 field_regex =  re.compile( '{.*?},', re.MULTILINE | re.DOTALL )
 cpuid_regex = re.compile( '\.cpuid = (.*?),', re.MULTILINE | re.DOTALL )
 name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
@@ -45,24 +45,25 @@ name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
 # types and then their fields.
 entry_dict = {}
 for struct in re.findall( struct_block_regex, data ):
-    # print( "struct: %s %s" % (struct[0],struct[1]) )
-    entry_dict[struct[1]] = {}
-    entry_dict[struct[1]]['type'] = struct[0]
-    entry_dict[struct[1]]['fields'] = {}
-    for entry in re.findall( field_regex, struct[2] ):
+    # print( "struct: %s %s %s" % (struct[0],struct[1],struct[2]) )
+    entry_dict[struct[2]] = {}
+    entry_dict[struct[2]]['type_prefix'] = struct[0]
+    entry_dict[struct[2]]['type'] = struct[1]
+    entry_dict[struct[2]]['fields'] = {}
+    for entry in re.findall( field_regex, struct[3] ):
         #print( "    entry: %s" % entry )
         cpuid = re.search( cpuid_regex, entry )
         if cpuid:
             #print( "    cpuid found: %s" % cpuid.group(1) )
-            entry_dict[struct[1]]['fields'][cpuid.group(1)] = entry
-            
+            entry_dict[struct[2]]['fields'][cpuid.group(1)] = entry
+
         name = re.search( name_regex, entry )
         if name:
             #print( "    name found: %s" % name.group(1) )
-            entry_dict[struct[1]]['fields'][name.group(1)] = entry
-        
-        if not entry_dict[struct[1]]['fields']:
-            entry_dict[struct[1]]['fields']['0'] = entry
+            entry_dict[struct[2]]['fields'][name.group(1)] = entry
+
+        if not entry_dict[struct[2]]['fields']:
+            entry_dict[struct[2]]['fields']['0'] = entry
 
 # created ordered dictionaries from the captured values. These are ordered by
 # a sorted() iteration of the keys. We don't care about the order we read
@@ -74,6 +75,7 @@ for struct in re.findall( struct_block_regex, data ):
 entry_dict_sorted = OrderedDict()
 for i in sorted(entry_dict.keys()):
     entry_dict_sorted[i] = {}
+    entry_dict_sorted[i]['type_prefix'] = entry_dict[i]['type_prefix']
     entry_dict_sorted[i]['type'] = entry_dict[i]['type']
     entry_dict_sorted[i]['fields'] = {}
     for f in sorted(entry_dict[i]['fields'].keys()):
@@ -85,7 +87,7 @@ outf = open( outfile, 'w' )
 print( preamble.group(1) )
 outf.write( preamble.group(1) )
 for d in entry_dict_sorted:
-    outf.write( "struct %s %s[] = {\n" % (entry_dict_sorted[d]['type'],d) )
+    outf.write( "%s %s %s[] = {\n" % (entry_dict_sorted[d]['type_prefix'], entry_dict_sorted[d]['type'],d) )
     for f in entry_dict_sorted[d]['fields']:
         outf.write( entry_dict_sorted[d]['fields'][f] + '\n' )
 
-- 
2.19.1

Re: [OE-core] [PATCH 03/20] python: introduce python3-dtschema

[Khem Raj]

fails to build for me

https://errors.yoctoproject.org/Errors/Details/618010/

On Wed, Nov 24, 2021 at 1:14 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>
> kernel commit 53182e81f47d [kbuild: Enable DT schema checks for %.dtb
> targets], which is part of v5.16+, makes the checking of dtb schemas
> manditory.
>
> To support the checking, we introduce this new recipe dtschema.
>
> Dependencies will be imported in subsequent commits, so we can
> track the dependency need on a per-import basis.
>
> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> ---
>  .../python/python3-dtschema_2021.10.bb           | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>  create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
>
> diff --git a/meta/recipes-devtools/python/python3-dtschema_2021.10.bb b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> new file mode 100644
> index 0000000000..f16232d610
> --- /dev/null
> +++ b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> @@ -0,0 +1,16 @@
> +DESCRIPTION = "Tooling for devicetree validation using YAML and jsonschema"
> +HOMEPAGE = "https://github.com/devicetree-org/dt-schema"
> +LICENSE = "BSD-2-Clause"
> +LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=457495c8fa03540db4a576bf7869e811"
> +
> +inherit pypi setuptools3
> +
> +PYPI_PACKAGE = "dtschema"
> +
> +SRC_URI[md5sum] = "d2f0d847ba2ad4d223317ef7304342b0"
> +SRC_URI[sha256sum] = "d09c94d13f46e6674ba11ff31220651ad1b02dae860f5a87905dfac6b8d768d9"
> +
> +DEPENDS += "python3-setuptools-scm"
> +RDEPENDS:${PN} += "python3-ruamel-yaml python3-jsonschema python3-rfc3987"
> +
> +BBCLASSEXTEND = "native nativesdk"
> --
> 2.19.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#158749): https://lists.openembedded.org/g/openembedded-core/message/158749
> Mute This Topic: https://lists.openembedded.org/mt/87290455/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH 03/20] python: introduce python3-dtschema

[Bruce Ashfield]

On Thu, Nov 25, 2021 at 12:36 PM Khem Raj <raj.khem@gmail.com> wrote:
>
> fails to build for me
>
> https://errors.yoctoproject.org/Errors/Details/618010/

It was updated in-branch with fixups.  The version in this patch is
known to break on a target build.

Bruce

>
> On Wed, Nov 24, 2021 at 1:14 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > From: Bruce Ashfield <bruce.ashfield@gmail.com>
> >
> > kernel commit 53182e81f47d [kbuild: Enable DT schema checks for %.dtb
> > targets], which is part of v5.16+, makes the checking of dtb schemas
> > manditory.
> >
> > To support the checking, we introduce this new recipe dtschema.
> >
> > Dependencies will be imported in subsequent commits, so we can
> > track the dependency need on a per-import basis.
> >
> > Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> > ---
> >  .../python/python3-dtschema_2021.10.bb           | 16 ++++++++++++++++
> >  1 file changed, 16 insertions(+)
> >  create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> >
> > diff --git a/meta/recipes-devtools/python/python3-dtschema_2021.10.bb b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> > new file mode 100644
> > index 0000000000..f16232d610
> > --- /dev/null
> > +++ b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> > @@ -0,0 +1,16 @@
> > +DESCRIPTION = "Tooling for devicetree validation using YAML and jsonschema"
> > +HOMEPAGE = "https://github.com/devicetree-org/dt-schema"
> > +LICENSE = "BSD-2-Clause"
> > +LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=457495c8fa03540db4a576bf7869e811"
> > +
> > +inherit pypi setuptools3
> > +
> > +PYPI_PACKAGE = "dtschema"
> > +
> > +SRC_URI[md5sum] = "d2f0d847ba2ad4d223317ef7304342b0"
> > +SRC_URI[sha256sum] = "d09c94d13f46e6674ba11ff31220651ad1b02dae860f5a87905dfac6b8d768d9"
> > +
> > +DEPENDS += "python3-setuptools-scm"
> > +RDEPENDS:${PN} += "python3-ruamel-yaml python3-jsonschema python3-rfc3987"
> > +
> > +BBCLASSEXTEND = "native nativesdk"
> > --
> > 2.19.1
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#158749): https://lists.openembedded.org/g/openembedded-core/message/158749
> > Mute This Topic: https://lists.openembedded.org/mt/87290455/1997914
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH 03/20] python: introduce python3-dtschema

[Khem Raj]

On 11/25/21 9:51 AM, Bruce Ashfield wrote:
> On Thu, Nov 25, 2021 at 12:36 PM Khem Raj <raj.khem@gmail.com> wrote:
>>
>> fails to build for me
>>
>> https://errors.yoctoproject.org/Errors/Details/618010/
> 
> It was updated in-branch with fixups.  The version in this patch is
> known to break on a target build.

thanks, my grinder runs slowly since it builds a lot of stuff, so it 
gets to errors a bit late :). Are these fixes already in master-next now ?

> 
> Bruce
> 
>>
>> On Wed, Nov 24, 2021 at 1:14 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>>>
>>> From: Bruce Ashfield <bruce.ashfield@gmail.com>
>>>
>>> kernel commit 53182e81f47d [kbuild: Enable DT schema checks for %.dtb
>>> targets], which is part of v5.16+, makes the checking of dtb schemas
>>> manditory.
>>>
>>> To support the checking, we introduce this new recipe dtschema.
>>>
>>> Dependencies will be imported in subsequent commits, so we can
>>> track the dependency need on a per-import basis.
>>>
>>> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
>>> ---
>>>   .../python/python3-dtschema_2021.10.bb           | 16 ++++++++++++++++
>>>   1 file changed, 16 insertions(+)
>>>   create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
>>>
>>> diff --git a/meta/recipes-devtools/python/python3-dtschema_2021.10.bb b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
>>> new file mode 100644
>>> index 0000000000..f16232d610
>>> --- /dev/null
>>> +++ b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
>>> @@ -0,0 +1,16 @@
>>> +DESCRIPTION = "Tooling for devicetree validation using YAML and jsonschema"
>>> +HOMEPAGE = "https://github.com/devicetree-org/dt-schema"
>>> +LICENSE = "BSD-2-Clause"
>>> +LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=457495c8fa03540db4a576bf7869e811"
>>> +
>>> +inherit pypi setuptools3
>>> +
>>> +PYPI_PACKAGE = "dtschema"
>>> +
>>> +SRC_URI[md5sum] = "d2f0d847ba2ad4d223317ef7304342b0"
>>> +SRC_URI[sha256sum] = "d09c94d13f46e6674ba11ff31220651ad1b02dae860f5a87905dfac6b8d768d9"
>>> +
>>> +DEPENDS += "python3-setuptools-scm"
>>> +RDEPENDS:${PN} += "python3-ruamel-yaml python3-jsonschema python3-rfc3987"
>>> +
>>> +BBCLASSEXTEND = "native nativesdk"
>>> --
>>> 2.19.1
>>>
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#158749): https://lists.openembedded.org/g/openembedded-core/message/158749
>>> Mute This Topic: https://lists.openembedded.org/mt/87290455/1997914
>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
> 
> 
> 

Re: [OE-core] [PATCH 03/20] python: introduce python3-dtschema

[Bruce Ashfield]

On Thu, Nov 25, 2021 at 12:53 PM Khem Raj <raj.khem@gmail.com> wrote:
>
>
>
> On 11/25/21 9:51 AM, Bruce Ashfield wrote:
> > On Thu, Nov 25, 2021 at 12:36 PM Khem Raj <raj.khem@gmail.com> wrote:
> >>
> >> fails to build for me
> >>
> >> https://errors.yoctoproject.org/Errors/Details/618010/
> >
> > It was updated in-branch with fixups.  The version in this patch is
> > known to break on a target build.
>
> thanks, my grinder runs slowly since it builds a lot of stuff, so it
> gets to errors a bit late :). Are these fixes already in master-next now ?

Yup,

Richard picked up the fix after I noticed the failure in my AB run, I see the

DEPENDS += "python3-setuptools-scm-native"

In the master-next version, which was the fix for the target build issue.

Bruce

>
> >
> > Bruce
> >
> >>
> >> On Wed, Nov 24, 2021 at 1:14 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >>>
> >>> From: Bruce Ashfield <bruce.ashfield@gmail.com>
> >>>
> >>> kernel commit 53182e81f47d [kbuild: Enable DT schema checks for %.dtb
> >>> targets], which is part of v5.16+, makes the checking of dtb schemas
> >>> manditory.
> >>>
> >>> To support the checking, we introduce this new recipe dtschema.
> >>>
> >>> Dependencies will be imported in subsequent commits, so we can
> >>> track the dependency need on a per-import basis.
> >>>
> >>> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> >>> ---
> >>>   .../python/python3-dtschema_2021.10.bb           | 16 ++++++++++++++++
> >>>   1 file changed, 16 insertions(+)
> >>>   create mode 100644 meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> >>>
> >>> diff --git a/meta/recipes-devtools/python/python3-dtschema_2021.10.bb b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> >>> new file mode 100644
> >>> index 0000000000..f16232d610
> >>> --- /dev/null
> >>> +++ b/meta/recipes-devtools/python/python3-dtschema_2021.10.bb
> >>> @@ -0,0 +1,16 @@
> >>> +DESCRIPTION = "Tooling for devicetree validation using YAML and jsonschema"
> >>> +HOMEPAGE = "https://github.com/devicetree-org/dt-schema"
> >>> +LICENSE = "BSD-2-Clause"
> >>> +LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=457495c8fa03540db4a576bf7869e811"
> >>> +
> >>> +inherit pypi setuptools3
> >>> +
> >>> +PYPI_PACKAGE = "dtschema"
> >>> +
> >>> +SRC_URI[md5sum] = "d2f0d847ba2ad4d223317ef7304342b0"
> >>> +SRC_URI[sha256sum] = "d09c94d13f46e6674ba11ff31220651ad1b02dae860f5a87905dfac6b8d768d9"
> >>> +
> >>> +DEPENDS += "python3-setuptools-scm"
> >>> +RDEPENDS:${PN} += "python3-ruamel-yaml python3-jsonschema python3-rfc3987"
> >>> +
> >>> +BBCLASSEXTEND = "native nativesdk"
> >>> --
> >>> 2.19.1
> >>>
> >>>
> >>> -=-=-=-=-=-=-=-=-=-=-=-
> >>> Links: You receive all messages sent to this group.
> >>> View/Reply Online (#158749): https://lists.openembedded.org/g/openembedded-core/message/158749
> >>> Mute This Topic: https://lists.openembedded.org/mt/87290455/1997914
> >>> Group Owner: openembedded-core+owner@lists.openembedded.org
> >>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> >>> -=-=-=-=-=-=-=-=-=-=-=-
> >>>
> >
> >
> >



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [PATCH 20/20] perf: sort-pmuevents: allow for additional type qualifiers and storage class

[Ferry Toth]

Hi,

Op 24-11-2021 om 22:14 schreef Bruce Ashfield:
> From: Max Krummenacher <max.oss.09@gmail.com>
> 
> With kernel 5.16 some structs in pmu-events do get a const qualifier, some
> a static const storage class and qualifier.
> 
> The current sort-pmuevents cannot cope with that and drops all struct
> arrays with such additional elements. This then leads to compiler errors.
> 
> Allow '^struct', '^const struct', '^static struct', '^static const struct'.
> 
> Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
> ---
>   .../perf/perf/sort-pmuevents.py               | 30 ++++++++++---------
>   1 file changed, 16 insertions(+), 14 deletions(-)
> 
> diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
> index 4f841eb822..09ba3328a7 100755
> --- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
> +++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
> @@ -33,10 +33,10 @@ if os.path.exists(outfile):
>   with open(infile, 'r') as file:
>       data = file.read()
>   
> -preamble_regex = re.compile( '^(.*?)^struct', re.MULTILINE | re.DOTALL )
> +preamble_regex = re.compile( '^(.*?)^(struct|const struct|static struct|static const struct)', re.MULTILINE | re.DOTALL )
>   
>   preamble = re.search( preamble_regex, data )
> -struct_block_regex = re.compile( '^struct.*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
> +struct_block_regex = re.compile( '^(struct|const struct|static struct|static const struct).*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
>   field_regex =  re.compile( '{.*?},', re.MULTILINE | re.DOTALL )
>   cpuid_regex = re.compile( '\.cpuid = (.*?),', re.MULTILINE | re.DOTALL )
>   name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
> @@ -45,24 +45,25 @@ name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
>   # types and then their fields.
>   entry_dict = {}
>   for struct in re.findall( struct_block_regex, data ):
> -    # print( "struct: %s %s" % (struct[0],struct[1]) )
> -    entry_dict[struct[1]] = {}
> -    entry_dict[struct[1]]['type'] = struct[0]
> -    entry_dict[struct[1]]['fields'] = {}
> -    for entry in re.findall( field_regex, struct[2] ):
> +    # print( "struct: %s %s %s" % (struct[0],struct[1],struct[2]) )
> +    entry_dict[struct[2]] = {}
> +    entry_dict[struct[2]]['type_prefix'] = struct[0]
> +    entry_dict[struct[2]]['type'] = struct[1]
> +    entry_dict[struct[2]]['fields'] = {}
> +    for entry in re.findall( field_regex, struct[3] ):
>           #print( "    entry: %s" % entry )
>           cpuid = re.search( cpuid_regex, entry )
>           if cpuid:
>               #print( "    cpuid found: %s" % cpuid.group(1) )
> -            entry_dict[struct[1]]['fields'][cpuid.group(1)] = entry
> -
> +            entry_dict[struct[2]]['fields'][cpuid.group(1)] = entry
> +
>           name = re.search( name_regex, entry )
>           if name:
>               #print( "    name found: %s" % name.group(1) )
> -            entry_dict[struct[1]]['fields'][name.group(1)] = entry
> -
> -        if not entry_dict[struct[1]]['fields']:
> -            entry_dict[struct[1]]['fields']['0'] = entry
> +            entry_dict[struct[2]]['fields'][name.group(1)] = entry
> +
> +        if not entry_dict[struct[2]]['fields']:
> +            entry_dict[struct[2]]['fields']['0'] = entry
>   
>   # created ordered dictionaries from the captured values. These are ordered by
>   # a sorted() iteration of the keys. We don't care about the order we read
> @@ -74,6 +75,7 @@ for struct in re.findall( struct_block_regex, data ):
>   entry_dict_sorted = OrderedDict()
>   for i in sorted(entry_dict.keys()):
>       entry_dict_sorted[i] = {}
> +    entry_dict_sorted[i]['type_prefix'] = entry_dict[i]['type_prefix']
>       entry_dict_sorted[i]['type'] = entry_dict[i]['type']
>       entry_dict_sorted[i]['fields'] = {}
>       for f in sorted(entry_dict[i]['fields'].keys()):
> @@ -85,7 +87,7 @@ outf = open( outfile, 'w' )
>   print( preamble.group(1) )
>   outf.write( preamble.group(1) )
>   for d in entry_dict_sorted:
> -    outf.write( "struct %s %s[] = {\n" % (entry_dict_sorted[d]['type'],d) )
> +    outf.write( "%s %s %s[] = {\n" % (entry_dict_sorted[d]['type_prefix'], entry_dict_sorted[d]['type'],d) )
>       for f in entry_dict_sorted[d]['fields']:
>           outf.write( entry_dict_sorted[d]['fields'][f] + '\n' )
>   
I have had sort-pmuevents hanging since on hardknott trying to build 
linux 5.16 and disabled perf from my recipe to work around.

However, I was hoping that with honister this patch would resolve that 
issue. But it doesn't in my case.

Is there anything in particular that I need to do to make this work?

Thanks
Ferry
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#158766): https://lists.openembedded.org/g/openembedded-core/message/158766
> Mute This Topic: https://lists.openembedded.org/mt/87290477/4454657
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [gchoc-openembedded-core@m.gmane-mx.org]
> -=-=-=-=-=-=-=-=-=-=-=-
>