[PATCH 0/7] Migration info for gatesgarth + updates

[PATCH 0/7] Migration info for gatesgarth + updates

[Paul Eggleton]

The migration section for the upcoming gatesgarth release as well as
some related updates to other parts of the reference manual. Let me
know if I missed anything.


The following changes since commit 14d0c205c671c4c670d7a887d307d359f70e1b7a:

  dev-manual/dev-manual-common-tasks: fix typos and use extlinks (2020-10-27 13:20:22 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib paule/migration-3.2
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/migration-3.2

Paul Eggleton (7):
  ref-manual: add reference anchors for each QA check
  ref-manual: fix for features_check class change
  ref-manual: QA check updates
  ref-manual: add PSEUDO_IGNORE_PATHS
  ref-manual: add IMAGE_VERSION_SUFFIX variable
  ref-manual: add IMAGE_NAME_SUFFIX variable
  ref-manual: add migration section for 3.2

 documentation/ref-manual/migration-3.2.rst | 261 +++++++++++++++++++++++++++++
 documentation/ref-manual/migration.rst     |   1 +
 documentation/ref-manual/ref-classes.rst   |  41 +++--
 documentation/ref-manual/ref-qa-checks.rst | 243 ++++++++++++++++++++++++++-
 documentation/ref-manual/ref-variables.rst |  61 +++++--
 5 files changed, 571 insertions(+), 36 deletions(-)
 create mode 100644 documentation/ref-manual/migration-3.2.rst

-- 
1.8.3.1

[PATCH 1/7] ref-manual: add reference anchors for each QA check

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

Make it possible to link to the explanation for a particular QA check.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 documentation/ref-manual/ref-qa-checks.rst | 36 ++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/documentation/ref-manual/ref-qa-checks.rst b/documentation/ref-manual/ref-qa-checks.rst
index 5b9f92d..228b4fd 100644
--- a/documentation/ref-manual/ref-qa-checks.rst
+++ b/documentation/ref-manual/ref-qa-checks.rst
@@ -43,6 +43,8 @@ error form along with an explanation.
 Errors and Warnings
 ===================
 
+.. _qa-check-libexec:
+
 -  ``<packagename>: <path> is using libexec please relocate to <libexecdir> [libexec]``
 
    The specified package contains files in ``/usr/libexec`` when the
@@ -51,6 +53,7 @@ Errors and Warnings
    default can be changed (e.g. ``${libdir}``).
 
     
+.. _qa-check-rpaths:
 
 -  ``package <packagename> contains bad RPATH <rpath> in file <file> [rpaths]``
 
@@ -65,6 +68,7 @@ Errors and Warnings
    software.
 
     
+.. _qa-check-useless-rpaths:
 
 -  ``<packagename>: <file> contains probably-redundant RPATH <rpath> [useless-rpaths]``
 
@@ -77,6 +81,7 @@ Errors and Warnings
    the build of the software.
 
     
+.. _qa-check-file-rdeps:
 
 -  ``<packagename> requires <files>, but no providers in its RDEPENDS [file-rdeps]``
 
@@ -88,6 +93,7 @@ Errors and Warnings
    built.
 
     
+.. _qa-check-build-deps:
 
 -  ``<packagename1> rdepends on <packagename2>, but it isn't a build dependency? [build-deps]``
 
@@ -101,6 +107,7 @@ Errors and Warnings
    to add an explicit ``RDEPENDS`` for the dependency.
 
     
+.. _qa-check-dev-so:
 
 -  ``non -dev/-dbg/nativesdk- package contains symlink .so: <packagename> path '<path>' [dev-so]``
 
@@ -112,6 +119,7 @@ Errors and Warnings
    file goes into an appropriate ``-dev`` package.
 
     
+.. _qa-check-staticdev:
 
 -  ``non -staticdev package contains static .a library: <packagename> path '<path>' [staticdev]``
 
@@ -121,6 +129,7 @@ Errors and Warnings
    goes into an appropriate ``-staticdev`` package.
 
     
+.. _qa-check-libdir:
 
 -  ``<packagename>: found library in wrong location [libdir]``
 
@@ -133,6 +142,7 @@ Errors and Warnings
    :term:`INSANE_SKIP` for the package.
 
     
+.. _qa-check-debug-files:
 
 -  ``non debug package contains .debug directory: <packagename> path <path> [debug-files]``
 
@@ -145,6 +155,7 @@ Errors and Warnings
    information on ``FILES``.
 
     
+.. _qa-check-arch:
 
 -  ``Architecture did not match (<machine_arch> to <file_arch>) on <file> [arch]``
 
@@ -199,6 +210,7 @@ Errors and Warnings
    and verify that the compiler options being used are correct.
 
     
+.. _qa-check-textrel:
 
 -  ``ELF binary '<file>' has relocations in .text [textrel]``
 
@@ -218,6 +230,7 @@ Errors and Warnings
    http://www.akkadia.org/drepper/textrelocs.html.
 
     
+.. _qa-check-ldflags:
 
 -  ``No GNU_HASH in the elf binary: '<file>' [ldflags]``
 
@@ -233,6 +246,7 @@ Errors and Warnings
       TARGET_CC_ARCH += "${LDFLAGS}"
 
     
+.. _qa-check-xorg-driver-abi:
 
 -  ``Package <packagename> contains Xorg driver (<driver>) but no xorg-abi- dependencies [xorg-driver-abi]``
 
@@ -245,6 +259,7 @@ Errors and Warnings
    to explicitly add dependencies to binary driver recipes.
 
     
+.. _qa-check-infodir:
 
 -  ``The /usr/share/info/dir file is not meant to be shipped in a particular package. [infodir]``
 
@@ -256,6 +271,8 @@ Errors and Warnings
       rm ${D}${infodir}/dir
    
 
+.. _qa-check-symlink-to-sysroot:
+
 -  ``Symlink <path> in <packagename> points to TMPDIR [symlink-to-sysroot]``
 
    The specified symlink points into :term:`TMPDIR` on the
@@ -264,6 +281,7 @@ Errors and Warnings
    symlink to use a relative path or remove the symlink.
 
     
+.. _qa-check-la:
 
 -  ``<file> failed sanity test (workdir) in path <path> [la]``
 
@@ -273,6 +291,7 @@ Errors and Warnings
    automatically itself.
 
     
+.. _qa-check-pkgconfig:
 
 -  ``<file> failed sanity test (tmpdir) in path <path> [pkgconfig]``
 
@@ -283,6 +302,7 @@ Errors and Warnings
    are accessed.
 
     
+.. _qa-check-debug-deps:
 
 -  ``<packagename> rdepends on <debug_packagename> [debug-deps]``
 
@@ -305,6 +325,7 @@ Errors and Warnings
    manually (e.g. by adding to :term:`RDEPENDS`).
 
     
+.. _qa-check-dev-deps:
 
 -  ``<packagename> rdepends on <dev_packagename> [dev-deps]``
 
@@ -327,6 +348,7 @@ Errors and Warnings
    manually (e.g. by adding to :term:`RDEPENDS`).
 
     
+.. _qa-check-dep-cmp:
 
 -  ``<var>_<packagename> is invalid: <comparison> (<value>)   only comparisons <, =, >, <=, and >= are allowed [dep-cmp]``
 
@@ -341,6 +363,7 @@ Errors and Warnings
    adding to match those listed in the message.
 
     
+.. _qa-check-compile-host-path:
 
 -  ``<recipename>: The compile log indicates that host include and/or library paths were used. Please check the log '<logfile>' for more information. [compile-host-path]``
 
@@ -351,6 +374,7 @@ Errors and Warnings
    file.
 
     
+.. _qa-check-install-host-path:
 
 -  ``<recipename>: The install log indicates that host include and/or library paths were used. Please check the log '<logfile>' for more information. [install-host-path]``
 
@@ -361,6 +385,7 @@ Errors and Warnings
    file.
 
     
+.. _qa-check-configure-unsafe:
 
 -  ``This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities. Rerun configure task after fixing this. The path was '<path>'``
 
@@ -371,6 +396,7 @@ Errors and Warnings
    file.
 
     
+.. _qa-check-pkgname:
 
 -  ``<packagename> doesn't match the [a-z0-9.+-]+ regex [pkgname]``
 
@@ -384,6 +410,7 @@ Errors and Warnings
    change the package name appropriately.
 
     
+.. _qa-check-unknown-configure-option:
 
 -  ``<recipe>: configure was passed unrecognized options: <options> [unknown-configure-option]``
 
@@ -401,6 +428,7 @@ Errors and Warnings
    accordingly.
 
     
+.. _qa-check-pn-overrides:
 
 -  ``Recipe <recipefile> has PN of "<recipename>" which is in OVERRIDES, this can result in unexpected behavior. [pn-overrides]``
 
@@ -416,6 +444,7 @@ Errors and Warnings
    :term:`FILES` for additional information.
 
     
+.. _qa-check-pkgvarcheck:
 
 -  ``<recipefile>: Variable <variable> is set as not being package specific, please fix this. [pkgvarcheck]``
 
@@ -433,6 +462,7 @@ Errors and Warnings
    assignments to these variables within your recipe.
 
     
+.. _qa-check-already-stripped:
 
 -  ``File '<file>' from <recipename> was already stripped, this will prevent future debugging! [already-stripped]``
 
@@ -458,6 +488,7 @@ Errors and Warnings
       strip the symbols from the binaries.
 
     
+.. _qa-check-packages-list:
 
 -  ``<packagename> is listed in PACKAGES multiple times, this leads to packaging errors. [packages-list]``
 
@@ -467,6 +498,7 @@ Errors and Warnings
    already in the variable's value.
 
     
+.. _qa-check-files-invalid:
 
 -  ``FILES variable for package <packagename> contains '//' which is invalid. Attempting to fix this but you should correct the metadata. [files-invalid]``
 
@@ -475,6 +507,7 @@ Errors and Warnings
    that there is only a single "/".
 
     
+.. _qa-check-installed-vs-shipped:
 
 -  ``<recipename>: Files/directories were installed but not shipped in any package [installed-vs-shipped]``
 
@@ -505,6 +538,9 @@ Errors and Warnings
    :term:`PRIVATE_LIBS` in the recipe that provides
    the private version of the library.
 
+
+.. _qa-check-unlisted-pkg-lics:
+
 -  ``LICENSE_<packagename> includes licenses (<licenses>) that are not listed in LICENSE [unlisted-pkg-lics]``
 
    The :term:`LICENSE` of the recipe should be a superset
-- 
1.8.3.1

[PATCH 2/7] ref-manual: fix for features_check class change

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

distro_features_check was renamed to features_check and extended to
support MACHINE_FEATURES, COMBINED_FEATURES and ANY_OF_*_FEATURES in
dunfell, but the documentation still needed to be updated.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 documentation/ref-manual/ref-classes.rst   | 41 +++++++++++++++++++-----------
 documentation/ref-manual/ref-variables.rst | 26 ++++++++++++++-----
 2 files changed, 46 insertions(+), 21 deletions(-)

diff --git a/documentation/ref-manual/ref-classes.rst b/documentation/ref-manual/ref-classes.rst
index 028729f..249b58e 100644
--- a/documentation/ref-manual/ref-classes.rst
+++ b/documentation/ref-manual/ref-classes.rst
@@ -501,21 +501,6 @@ Support for other version control systems such as Subversion is limited
 due to BitBake's automatic fetch dependencies (e.g.
 ``subversion-native``).
 
-.. _ref-classes-distro_features_check:
-
-``distro_features_check.bbclass``
-=================================
-
-The ``distro_features_check`` class allows individual recipes to check
-for required and conflicting
-:term:`DISTRO_FEATURES`.
-
-This class provides support for the
-:term:`REQUIRED_DISTRO_FEATURES` and
-:term:`CONFLICT_DISTRO_FEATURES`
-variables. If any conditions specified in the recipe using the above
-variables are not met, the recipe will be skipped.
-
 .. _ref-classes-distutils:
 
 ``distutils*.bbclass``
@@ -656,6 +641,32 @@ Finally, here is an example that sets the root password to "1876*18":
        usermod -P 1876*18 root; \
        "
 
+.. _ref-classes-features_check:
+
+``features_check.bbclass``
+=================================
+
+The ``features_check`` class allows individual recipes to check
+for required and conflicting
+:term:`DISTRO_FEATURES`, :term:`MACHINE_FEATURES` or :term:`COMBINED_FEATURES`.
+
+This class provides support for the following variables:
+
+- :term:`REQUIRED_DISTRO_FEATURES`
+- :term:`CONFLICT_DISTRO_FEATURES`
+- :term:`ANY_OF_DISTRO_FEATURES`
+- ``REQUIRED_MACHINE_FEATURES``
+- ``CONFLICT_MACHINE_FEATURES``
+- ``ANY_OF_MACHINE_FEATURES``
+- ``REQUIRED_COMBINED_FEATURES``
+- ``CONFLICT_COMBINED_FEATURES``
+- ``ANY_OF_COMBINED_FEATURES``
+
+If any conditions specified in the recipe using the above
+variables are not met, the recipe will be skipped, and if the
+build system attempts to build the recipe then an error will be
+triggered.
+
 .. _ref-classes-fontcache:
 
 ``fontcache.bbclass``
diff --git a/documentation/ref-manual/ref-variables.rst b/documentation/ref-manual/ref-variables.rst
index 0603ba9..a7315cf 100644
--- a/documentation/ref-manual/ref-variables.rst
+++ b/documentation/ref-manual/ref-variables.rst
@@ -132,6 +132,18 @@ system and gives an overview of their function and contents.
       ":ref:`update-alternatives.bbclass <ref-classes-update-alternatives>`"
       section.
 
+   :term:`ANY_OF_DISTRO_FEATURES`
+      When inheriting the
+      :ref:`features_check <ref-classes-features_check>`
+      class, this variable identifies a list of distribution features where
+      at least one must be enabled in the current configuration in order
+      for the OpenEmbedded build system to build the recipe. In other words,
+      if none of the features listed in ``ANY_OF_DISTRO_FEATURES``
+      appear in ``DISTRO_FEATURES`` within the current configuration, then
+      the recipe will be skipped, and if the build system attempts to build
+      the recipe then an error will be triggered.
+      
+
    :term:`APPEND`
       An override list of append strings for each target specified with
       :term:`LABELS`.
@@ -1300,12 +1312,13 @@ system and gives an overview of their function and contents.
 
    :term:`CONFLICT_DISTRO_FEATURES`
       When inheriting the
-      :ref:`distro_features_check <ref-classes-distro_features_check>`
+      :ref:`features_check <ref-classes-features_check>`
       class, this variable identifies distribution features that would be
       in conflict should the recipe be built. In other words, if the
       ``CONFLICT_DISTRO_FEATURES`` variable lists a feature that also
-      appears in ``DISTRO_FEATURES`` within the current configuration, an
-      error occurs and the build stops.
+      appears in ``DISTRO_FEATURES`` within the current configuration, then
+      the recipe will be skipped, and if the build system attempts to build
+      the recipe then an error will be triggered.
 
    :term:`COPYLEFT_LICENSE_EXCLUDE`
       A space-separated list of licenses to exclude from the source
@@ -6122,13 +6135,14 @@ system and gives an overview of their function and contents.
 
    :term:`REQUIRED_DISTRO_FEATURES`
       When inheriting the
-      :ref:`distro_features_check <ref-classes-distro_features_check>`
+      :ref:`features_check <ref-classes-features_check>`
       class, this variable identifies distribution features that must exist
       in the current configuration in order for the OpenEmbedded build
       system to build the recipe. In other words, if the
       ``REQUIRED_DISTRO_FEATURES`` variable lists a feature that does not
-      appear in ``DISTRO_FEATURES`` within the current configuration, an
-      error occurs and the build stops.
+      appear in ``DISTRO_FEATURES`` within the current configuration, then
+      the recipe will be skipped, and if the build system attempts to build
+      the recipe then an error will be triggered.
 
    :term:`RM_WORK_EXCLUDE`
       With ``rm_work`` enabled, this variable specifies a list of recipes
-- 
1.8.3.1

[PATCH 3/7] ref-manual: QA check updates

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

* Update for changes to messages
* Add missing QA checks - some added recently, others several releases
  ago

Some of this was borrowed from commit messages (with editing) - in
particular thanks to Alexander Kanavin for the writeup on patch-fuzz.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 documentation/ref-manual/ref-qa-checks.rst | 207 ++++++++++++++++++++++++++++-
 1 file changed, 200 insertions(+), 7 deletions(-)

diff --git a/documentation/ref-manual/ref-qa-checks.rst b/documentation/ref-manual/ref-qa-checks.rst
index 228b4fd..54977dc 100644
--- a/documentation/ref-manual/ref-qa-checks.rst
+++ b/documentation/ref-manual/ref-qa-checks.rst
@@ -157,7 +157,7 @@ Errors and Warnings
     
 .. _qa-check-arch:
 
--  ``Architecture did not match (<machine_arch> to <file_arch>) on <file> [arch]``
+-  ``Architecture did not match (<file_arch>, expected <machine_arch>) in <file> [arch]``
 
    By default, the OpenEmbedded build system checks the Executable and
    Linkable Format (ELF) type, bit size, and endianness of any binaries
@@ -175,7 +175,7 @@ Errors and Warnings
 
     
 
--  ``Bit size did not match (<machine_bits> to <file_bits>) <recipe> on <file> [arch]``
+-  ``Bit size did not match (<file_bits>, expected <machine_bits>) in <file> [arch]``
 
    By default, the OpenEmbedded build system checks the Executable and
    Linkable Format (ELF) type, bit size, and endianness of any binaries
@@ -193,7 +193,7 @@ Errors and Warnings
 
     
 
--  ``Endianness did not match (<machine_endianness> to <file_endianness>) on <file> [arch]``
+-  ``Endianness did not match (<file_endianness>, expected <machine_endianness>) in <file> [arch]``
 
    By default, the OpenEmbedded build system checks the Executable and
    Linkable Format (ELF) type, bit size, and endianness of any binaries
@@ -232,7 +232,7 @@ Errors and Warnings
     
 .. _qa-check-ldflags:
 
--  ``No GNU_HASH in the elf binary: '<file>' [ldflags]``
+-  ``File '<file>' in package '<package>' doesn't have GNU_HASH (didn't pass LDFLAGS?) [ldflags]``
 
    This indicates that binaries produced when building the recipe have
    not been linked with the :term:`LDFLAGS` options
@@ -387,7 +387,7 @@ Errors and Warnings
     
 .. _qa-check-configure-unsafe:
 
--  ``This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities. Rerun configure task after fixing this. The path was '<path>'``
+-  ``This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities. Rerun configure task after fixing this. [configure-unsafe]``
 
    The log for the :ref:`ref-tasks-configure` task
    indicates that paths on the host were searched for files, which is
@@ -461,7 +461,15 @@ Errors and Warnings
    ``RDEPENDS = "value"``). If you receive this error, correct any
    assignments to these variables within your recipe.
 
-    
+
+- ``recipe uses DEPENDS_${PN}, should use DEPENDS [pkgvarcheck]``
+
+   This check looks for instances of setting ``DEPENDS_${PN}``
+   which is erroneous (:term:`DEPENDS` is a recipe-wide variable and thus
+   it is not correct to specify it for a particular package, nor will such
+   an assignment actually work.) Set ``DEPENDS`` instead.
+
+
 .. _qa-check-already-stripped:
 
 -  ``File '<file>' from <recipename> was already stripped, this will prevent future debugging! [already-stripped]``
@@ -548,7 +556,192 @@ Errors and Warnings
    words, any license in ``LICENSE_*`` should also appear in
    :term:`LICENSE`.
 
-    
+
+.. _qa-check-configure-gettext:
+
+-  ``AM_GNU_GETTEXT used but no inherit gettext [configure-gettext]``
+
+    If a recipe is building something that uses automake and the automake
+    files contain an ``AM_GNU_GETTEXT`` directive then this check will fail
+    if there is no ``inherit gettext`` statement in the recipe to ensure
+    that gettext is available during the build. Add ``inherit gettext`` to
+    remove the warning.
+
+
+.. _qa-check-mime:
+
+- ``package contains mime types but does not inherit mime: <packagename> path '<file>' [mime]``
+
+   The specified package contains mime type files (``.xml`` files in
+   ``${datadir}/mime/packages``) and yet does not inherit the mime
+   class which will ensure that these get properly installed. Either
+   add ``inherit mime`` to the recipe or remove the files at the
+   ``do_install`` step if they are not needed.
+
+
+.. _qa-check-mime-xdg:
+
+- ``package contains desktop file with key 'MimeType' but does not inhert mime-xdg: <packagename> path '<file>' [mime-xdg]``
+
+    The specified package contains a .desktop file with a 'MimeType' key
+    present, but does not inherit the mime-xdg class that is required in
+    order for that to be activated. Either add ``inherit mime`` to the
+    recipe or remove the files at the ``do_install`` step if they are not
+    needed.
+
+
+.. _qa-check-src-uri-bad:
+
+- ``<recipename>: SRC_URI uses unstable GitHub archives [src-uri-bad]``
+
+    GitHub provides "archive" tarballs, however these can be re-generated
+    on the fly and thus the file's signature will not necessarily match that
+    in the SRC_URI checksums in future leading to build failures. It is
+    recommended that you use an official release tarball or switch to
+    pulling the corresponding revision in the actual git repository instead.
+
+
+- ``SRC_URI uses PN not BPN [src-uri-bad]``
+
+    If some part of :term:`SRC_URI` needs to reference the recipe name, it should do
+    so using ${:term:`BPN`} rather than ${:term:`PN`} as the latter will change
+    for different variants of the same recipe e.g. when :term:`BBCLASSEXTEND`
+    or multilib are being used. This check will fail if a reference to ``${PN}``
+    is found within the ``SRC_URI`` value - change it to ``${BPN}`` instead.
+
+
+.. _qa-check-unhandled-features-check:
+
+- ``<recipename>: recipe doesn't inherit features_check [unhandled-features-check]``
+
+    This check ensures that if one of the variables that the :ref:`features_check <ref-classes-features_check>`
+    class supports (e.g. :term:`REQUIRED_DISTRO_FEATURES`) is used, then the recipe
+    inherits ``features_check`` in order for the requirement to actually work. If
+    you are seeing this message, either add ``inherit features_check`` to your recipe
+    or remove the reference to the variable if it is not needed.
+
+
+.. _qa-check-missing-update-alternatives:
+
+- ``<recipename>: recipe defines ALTERNATIVE_<packagename> but doesn't inherit update-alternatives. This might fail during do_rootfs later! [missing-update-alternatives]``
+
+    This check ensures that if a recipe sets the :term:`ALTERNATIVE` variable that the
+    recipe also inherits :ref:`update-alternatives <ref-classes-update-alternatives>` such
+    that the alternative will be correctly set up. If you are seeing this message, either
+    add ``inherit update-alternatives`` to your recipe or remove the reference to the variable
+    if it is not needed.
+
+
+.. _qa-check-shebang-size:
+
+- ``<packagename>: <file> maximum shebang size exceeded, the maximum size is 128. [shebang-size]``
+
+    This check ensures that the shebang line (``#!`` in the first line) for a script
+    is not longer than 128 characters, which can cause an error at runtime depending
+    on the operating system. If you are seeing this message then the specified script
+    may need to be patched to have a shorter in order to avoid runtime problems.
+
+
+.. _qa-check-perllocalpod:
+
+- ``<packagename> contains perllocal.pod (<files>), should not be installed [perllocalpod]``
+
+    ``perllocal.pod`` is an index file of locally installed modules and so shouldn't be
+    installed by any distribution packages. The :ref:`cpan <ref-classes-cpan>` class
+    already sets ``NO_PERLLOCAL`` to stop this file being generated by most Perl recipes,
+    but if a recipe is using ``MakeMaker`` directly then they might not be doing this
+    correctly. This check ensures that perllocal.pod is not in any package in order to
+    avoid multiple packages shipping this file and thus their packages conflicting
+    if installed together.
+
+
+.. _qa-check-usrmerge:
+
+- ``<packagename> package is not obeying usrmerge distro feature. /<path> should be relocated to /usr. [usrmerge]``
+
+    If ``usrmerge`` is in :term:`DISTRO_FEATURES`, this check will ensure that no package
+    installs files to root (``/bin``, ``/sbin``, ``/lib``, ``/lib64``) directories. If you are seeing this
+    message, it indicates that the ``do_install`` step (or perhaps the build process that
+    ``do_install`` is calling into, e.g. ``make install`` is using hardcoded paths instead
+    of the variables set up for this (``bindir``, ``sbindir``, etc.), and should be
+    changed so that it does.
+
+
+.. _qa-check-patch-fuzz:
+
+- ``Fuzz detected: <patch output> [patch-fuzz]``
+
+    This check looks for evidence of "fuzz" when applying patches within the ``do_patch``
+    task. Patch fuzz is a situation when the ``patch`` tool ignores some of the context
+    lines in order to apply the patch. Consider this example:
+
+    Patch to be applied: ::
+
+        --- filename
+        +++ filename
+         context line 1
+         context line 2
+         context line 3
+        +newly added line
+         context line 4
+         context line 5
+         context line 6
+
+    Original source code: ::
+
+        different context line 1
+        different context line 2
+        context line 3
+        context line 4
+        different context line 5
+        different context line 6
+
+    Outcome (after applying patch with fuzz): ::
+
+        different context line 1
+        different context line 2
+        context line 3
+        newly added line
+        context line 4
+        different context line 5
+        different context line 6
+
+    Chances are, the newly added line was actually added in a completely
+    wrong location, or it was already in the original source and was added
+    for the second time. This is especially possible if the context line 3
+    and 4 are blank or have only generic things in them, such as ``#endif`` or ``}``.
+    Depending on the patched code, it is entirely possible for an incorrectly
+    patched file to still compile without errors.
+
+    *How to eliminate patch fuzz warnings*
+
+    Use the ``devtool`` command as explained by the warning. First, unpack the
+    source into devtool workspace: ::
+
+            devtool modify <recipe>
+
+    This will apply all of the patches, and create new commits out of them in
+    the workspace - with the patch context updated.
+
+    Then, replace the patches in the recipe layer: ::
+
+            devtool finish --force-patch-refresh <recipe> <layer_path>
+
+    The patch updates then need be reviewed (preferably with a side-by-side diff
+    tool) to ensure they are indeed doing the right thing i.e.:
+
+    #. they are applied in the correct location within the file;
+    #. they do not introduce duplicate lines, or otherwise do things that
+       are no longer necessary.
+
+    To confirm these things, you can also review the patched source code in
+    devtool's workspace, typically in ``<build_dir>/workspace/sources/<recipe>/``
+
+    Once the review is done, you can create and publish a layer commit with
+    the patch updates that modify the context. Devtool may also refresh
+    other things in the patches, those can be discarded.
+
+
 
 Configuring and Disabling QA Checks
 ===================================
-- 
1.8.3.1

[PATCH 4/7] ref-manual: add PSEUDO_IGNORE_PATHS

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

Add a brief variable glossary entry for the new PSEUDO_IGNORE_PATHS
variable.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 documentation/ref-manual/ref-variables.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/documentation/ref-manual/ref-variables.rst b/documentation/ref-manual/ref-variables.rst
index a7315cf..176c94a 100644
--- a/documentation/ref-manual/ref-variables.rst
+++ b/documentation/ref-manual/ref-variables.rst
@@ -5932,6 +5932,14 @@ system and gives an overview of their function and contents.
       service <dev-manual/dev-manual-common-tasks:working with a pr service>`. You can
       set ``PRSERV_HOST`` to other values to use a remote PR service.
 
+
+   :term:`PSEUDO_IGNORE_PATHS`
+      A comma-separated (without spaces) list of files that should be ignored
+      by pseudo when monitoring and recording file operations, in order to avoid
+      problems with files being written to outside of the pseudo context and
+      reduce pseudo's overhead.
+
+
    :term:`PTEST_ENABLED`
       Specifies whether or not :ref:`Package
       Test <dev-manual/dev-manual-common-tasks:testing packages with ptest>` (ptest)
-- 
1.8.3.1

[PATCH 5/7] ref-manual: add IMAGE_VERSION_SUFFIX variable

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

Add an entry to the variable glossary for IMAGE_VERSION_SUFFIX (which
was added in thud) and update the IMAGE_NAME and KERNEL_ARTIFACT_NAME
entries whose defaults use this variable.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 documentation/ref-manual/ref-variables.rst | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/documentation/ref-manual/ref-variables.rst b/documentation/ref-manual/ref-variables.rst
index 176c94a..319f376 100644
--- a/documentation/ref-manual/ref-variables.rst
+++ b/documentation/ref-manual/ref-variables.rst
@@ -3121,11 +3121,11 @@ system and gives an overview of their function and contents.
    :term:`IMAGE_NAME`
       The name of the output image files minus the extension. This variable
       is derived using the :term:`IMAGE_BASENAME`,
-      :term:`MACHINE`, and :term:`DATETIME`
+      :term:`MACHINE`, and :term:`IMAGE_VERSION_SUFFIX`
       variables:
       ::
 
-         IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${DATETIME}"
+         IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
 
    :term:`IMAGE_OVERHEAD_FACTOR`
       Defines a multiplier that the build system applies to the initial
@@ -3329,6 +3329,14 @@ system and gives an overview of their function and contents.
       For more information about these types of images, see
       ``meta/classes/image_types*.bbclass`` in the :term:`Source Directory`.
 
+   :term:`IMAGE_VERSION_SUFFIX`
+      Version suffix that is part of the default :term:`IMAGE_NAME` and
+      :term:`KERNEL_ARTIFACT_NAME` values.
+      Defaults to ``"-${DATETIME}"``, however you could set this to a
+      version string that comes from your external build environment if
+      desired, and this suffix would then be used consistently across
+      the build artifacts.
+
    :term:`INC_PR`
       Helps define the recipe revision for recipes that share a common
       ``include`` file. You can think of this variable as part of the
@@ -3787,12 +3795,8 @@ system and gives an overview of their function and contents.
 
          KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
 
-      See the :term:`PKGE`, :term:`PKGV`, :term:`PKGR`, and :term:`MACHINE`
-      variables for additional information.
-
-      .. note::
-
-         The ``IMAGE_VERSION_SUFFIX`` variable is set to :term:`DATETIME`.
+      See the :term:`PKGE`, :term:`PKGV`, :term:`PKGR`, :term:`MACHINE`
+      and :term:`IMAGE_VERSION_SUFFIX` variables for additional information.
 
    :term:`KERNEL_CLASSES`
       A list of classes defining kernel image types that the
-- 
1.8.3.1

[PATCH 6/7] ref-manual: add IMAGE_NAME_SUFFIX variable

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

Add a variable glossary entry for IMAGE_NAME_SUFFIX, which was added way
back in krogoth.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 documentation/ref-manual/ref-variables.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/documentation/ref-manual/ref-variables.rst b/documentation/ref-manual/ref-variables.rst
index 319f376..a7fb777 100644
--- a/documentation/ref-manual/ref-variables.rst
+++ b/documentation/ref-manual/ref-variables.rst
@@ -3127,6 +3127,13 @@ system and gives an overview of their function and contents.
 
          IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
 
+   :term:`IMAGE_NAME_SUFFIX`
+      Suffix used for the image output file name - defaults to ``".rootfs"``
+      to distinguish the image file from other files created during image
+      building; however if this suffix is redundant or not desired you can
+      clear the value of this variable (set the value to ""). For example,
+      this is typically cleared in initramfs image recipes.
+
    :term:`IMAGE_OVERHEAD_FACTOR`
       Defines a multiplier that the build system applies to the initial
       image size for cases when the multiplier times the returned disk
-- 
1.8.3.1

[PATCH 7/7] ref-manual: add migration section for 3.2

[Paul Eggleton]

From: Paul Eggleton <paul.eggleton@microsoft.com>

This covers most of the changes that would require action on the part of
the user that I was able to see by scouring the commits. Some of the text
was borrowed from commit messages and edited.

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
---
 documentation/ref-manual/migration-3.2.rst | 261 +++++++++++++++++++++++++++++
 documentation/ref-manual/migration.rst     |   1 +
 2 files changed, 262 insertions(+)
 create mode 100644 documentation/ref-manual/migration-3.2.rst

diff --git a/documentation/ref-manual/migration-3.2.rst b/documentation/ref-manual/migration-3.2.rst
new file mode 100644
index 0000000..a56c5d4
--- /dev/null
+++ b/documentation/ref-manual/migration-3.2.rst
@@ -0,0 +1,261 @@
+Moving to the Yocto Project 3.2 Release
+=======================================
+
+This section provides migration information for moving to the Yocto
+Project 3.2 Release from the prior release.
+
+.. _migration-3.2-minimum-system-requirements:
+
+Minimum system requirements
+---------------------------
+
+``gcc`` version 6.0 is now required at minimum on the build host. For older
+host distributions where this is not available, you can use the
+``buildtools-extended-tarball`` (easily installable using
+``scripts/install-buildtools``).
+
+
+.. _migration-3.2-removed-recipes:
+
+Removed recipes
+---------------
+
+The following recipes have been removed:
+
+- ``bjam-native``: replaced by ``boost-build-native``
+- ``avahi-ui``: folded into the main ``avahi`` recipe - the GTK UI can be disabled using :term:`PACKAGECONFIG` for ``avahi``.
+- ``build-compare``: no longer needed with the removal of the ``packagefeed-stability`` class
+- ``dhcp``: obsolete, functionally replaced by ``dhcpcd`` and ``kea``
+- ``libmodulemd-v1``: replaced by ``libmodulemd``
+- ``packagegroup-core-device-devel``: obsolete
+
+
+.. _migration-3.2-removed-classes:
+
+Removed classes
+---------------
+
+The following classes (.bbclass files) have been removed:
+
+-  ``spdx``: obsolete - the Yocto Project is a strong supporter of SPDX, but this class was old code using a dated approach and had the potential to be misleading. The ``meta-sdpxscanner`` layer is a much more modern and active approach to handling this and is recommended as a replacement.
+
+- ``packagefeed-stability``: this class had become obsolete with the advent of hash equivalence and reproducible builds.
+
+
+pseudo path filtering and mismatch behaviour
+--------------------------------------------
+
+pseudo now operates on a filtered subset of files. This is a significant change
+to the way pseudo operates within OpenEmbedded - by default, pseudo monitors and
+logs (adds to its database) any file created or modified whilst in a ``fakeroot``
+environment. However, there are large numbers of files that we simply don't care
+about the permissions of whilst in that ``fakeroot`` context, for example ${:term:`S`}, ${:term:`B`}, ${:term:`T`},
+${:term:`SSTATE_DIR`}, the central sstate control directories, and others.
+
+As of this release, new functionality in pseudo is enabled to ignore these
+directory trees (controlled using a new :term:`PSEUDO_IGNORE_PATHS` variable)
+resulting in a cleaner database with less chance of "stray" mismatches if files
+are modified outside pseudo context. It also should reduce some overhead from
+pseudo as the interprocess round trip to the server is avoided.
+
+There is a possible complication where some existing recipe may break, for
+example, a recipe was found to be writing to ``${B}/install`` for
+``make install`` in ``do_install`` and since ``${B}`` is listed as not to be tracked,
+there were errors trying to ``chown root`` for files in this location. Another
+example was the ``tcl`` recipe where the source directory ``S`` is set to a
+subdirectory of the source tree but files were written out to the directory
+structure above that subdirectory. For these types of cases in your own recipes,
+extend ``PSEUDO_IGNORE_PATHS`` to cover additional paths that pseudo should not
+be monitoring.
+
+In addition, pseudo's behaviour on mismatches has now been changed - rather
+than doing what turns out to be a rather dangerous "fixup" if it sees a file
+with a different path but the same inode as another file it has previously seen,
+pseudo will throw an ``abort()`` and direct you to a `wiki page <https://wiki.yoctoproject.org/wiki/Pseudo_Abort>`
+that explains how to deal with this.
+
+
+.. _migration-3.2-packagegroup-core-device-devel:
+
+packagegroup-core-device-devel no longer included in images built for qemu* machines
+------------------------------------------------------------------------------------
+
+``packagegroup-core-device-devel`` was previously added automatically to
+images build for ``qemu*`` machines, however the purpose of the group and what
+it should contain is no longer clear, and in general, adding userspace
+development items to images is best done at the image/class level; thus this
+packagegroup was removed.
+
+This packagegroup previously pulled in the following:
+
+- ``distcc-config``
+- ``nfs-export-root``
+- ``bash``
+- ``binutils-symlinks``
+
+If you still need any of these in your image build for a ``qemu*`` machine
+then you will add them explicitly to :term:`IMAGE_INSTALL` or another
+appropriate place in the dependency chain for your image (if you have not
+already done so).
+
+
+.. _migration-3.2-dhcp:
+
+DHCP server/client replaced
+---------------------------
+
+The ``dhcp`` software package has become unmaintained and thus has been
+functionally replaced by ``dhcpcd`` (client) and ``kea`` (server). You will
+need to replace references to the recipe/package names as appropriate - most
+commonly, at the package level ``dhcp-client`` should be replaced by
+``dhcpcd`` and ``dhcp-server`` should be replaced by ``kea``. If you have any
+custom configuration files for these they will need to be adapted - refer to
+the upstream documentation for ``dhcpcd`` and ``kea`` for further details.
+
+
+.. _migration-3.2-packaging-changes:
+
+Packaging changes
+-----------------
+
+- ``python3``: the ``urllib`` python package has now moved into the core package, as it is used more commonly than just netclient (e.g. email, xml, mimetypes, pydoc). In addition, the ``pathlib`` module is now also part of the core package.
+
+- ``iptables``: ``iptables-apply`` and ``ip6tables-apply`` have been split out to their own package to avoid a bash dependency in the main ``iptables`` package
+
+
+.. _migration-3.2-package-qa-checks:
+
+Package QA check changes
+------------------------
+
+Previously, the following package QA checks triggered warnings, however they can
+be indicators of genuine underlying problems and are therefore now treated as
+errors:
+
+- :ref:`already-stripped <qa-check-already-stripped>`
+- :ref:`compile-host-path <qa-check-compile-host-path>`
+- :ref:`installed-vs-shipped <qa-check-installed-vs-shipped>`
+- :ref:`ldflags <qa-check-ldflags>`
+- :ref:`pn-overrides <qa-check-pn-overrides>`
+- :ref:`rpaths <qa-check-rpaths>`
+- :ref:`staticdev <qa-check-staticdev>`
+- :ref:`unknown-configure-option <qa-check-unknown-configure-option>`
+- :ref:`useless-rpaths <qa-check-useless-rpaths>`
+
+In addition, the following new checks were added and default to triggering an error:
+
+- :ref:`shebang-size <qa-check-shebang-size>`: Check for shebang (#!) lines longer than 128 characters, which can give an error at runtime depending on the operating system.
+
+- :ref:`unhandled-features-check <qa-check-unhandled-features-check>`: Check if any of the variables supported by the :ref:`features_check <ref-classes-features_check>` class is set while not inheriting the class itself.
+
+- :ref:`missing-update-alternatives <qa-check-missing-update-alternatives>`: Check if the recipe sets the :term:`ALTERNATIVE` variable for any of its packages, and does not inherit the :ref:`update-alternatives <ref-classes-update-alternatives>` class.
+
+- A trailing slash or duplicated slashes in the value of :term:`S` or :term:`B` will now trigger a warning so that they can be removed and path comparisons can be more reliable - remove any instances of these in your recipes if the warning is displayed.
+
+
+.. _migration-3.2-src-uri-file-globbing:
+
+Globbing no longer supported in ``file://`` entries in ``SRC_URI``
+------------------------------------------------------------------
+
+Globbing (``*`` and ``?`` wildcards) in ``file://`` URLs within :term:`SRC_URI`
+did not properly support file checksums, thus changes to the source files
+would not always change the do_fetch task checksum, and consequently would
+not ensure that the changed files would be incorporated in subsequent builds.
+
+Unfortunately it is not practical to make globbing work generically here, so
+the decision was taken to remove support for globs in ``file://`` URLs.
+If you have any usage of these in your recipes, then you will now need to
+either add each of the files that you expect to match explicitly, or
+alternatively if you still need files to be pulled in dynamically, put the
+files into a subdirectory and reference that instead.
+
+
+.. _migration-3.2-deploydir-clean:
+
+deploy class now cleans ``DEPLOYDIR`` before ``do_deploy``
+----------------------------------------------------------
+
+``do_deploy`` as implemented in the :ref:`deploy <ref-classes-deploy>` class now cleans up ${:term:`DEPLOYDIR`} before running, just as ``do_install`` cleans up ${:term:`D`} before running. This reduces the risk of DEPLOYDIR being accidentally contaminated by files from previous runs, possibly even with different config, in case of incremental builds.
+
+Most recipes and classes that inherit the ``deploy`` class or interact with ``do_deploy`` are unlikely to be affected by this unless they add ``prefuncs`` to ``do_deploy`` *which also* put files into ``${DEPLOYDIR}`` - these should be refactored to use ``do_deploy_prepend`` instead.
+
+
+.. _migration-3.2-nativesdk-sdk-provides-dummy:
+
+Custom SDK / SDK-style recipes need to include ``nativesdk-sdk-provides-dummy``
+-------------------------------------------------------------------------------
+
+All ``nativesdk`` packages require ``/bin/sh`` due to their postinstall scriptlets, thus this package has to be dummy-provided within the SDK and ``nativesdk-sdk-provides-dummy`` now does this. If you have a custom SDK recipe (or your own SDK-style recipe similar to e.g. ``buildtools-tarball``), you will need to ensure ``nativesdk-sdk-provides-dummy`` or an equivalent is included in :term:`TOOLCHAIN_HOST_TASK`.
+
+
+``ld.so.conf`` now moved back to main ``glibc`` package
+-------------------------------------------------------
+
+There are cases where one doesn't want ldconfig on target (e.g. for
+read-only root filesystems, it's rather pointless), yet one still
+needs ``/etc/ld.so.conf`` to be present at image build time:
+
+When some recipe installs libraries to a non-standard location, and
+therefore installs in a file in ``/etc/ld.so.conf.d/foo.conf``, we
+need ``/etc/ld.so.conf`` containing: ::
+
+  include /etc/ld.so.conf.d/*.conf
+
+in order to get those other locations picked up.
+
+Thus ``/etc/ld.so.conf`` is now in the main ``glibc`` package so that
+there's always an ``ld.so.conf`` present when the build-time ``ldconfig``
+runs towards the end of image construction.
+
+The ``ld.so.conf`` and ``ld.so.conf.d/*.conf`` files do not take up
+significant space (at least not compared to the ~700kB ``ldconfig`` binary), and they
+might be needed in case ``ldconfig`` is installable, so they are left
+in place after the image is built. Technically it would be possible to
+remove them if desired, though it would not be trivial if you still
+wanted the build-time ldconfig to function (:term:`ROOTFS_POSTPROCESS_COMMAND`
+will not work as ``ldconfig`` is run after the functions referred to
+by that variable).
+
+
+.. _migration-3.2-virgl:
+
+Host DRI drivers now used for GL support within ``runqemu``
+-----------------------------------------------------------
+
+``runqemu`` now uses the mesa-native libraries everywhere virgl is used
+(i.e. when ``gl``, ``gl-es`` or ``egl-headless`` options are specified),
+but instructs them to load DRI drivers from the host. Unfortunately this
+may not work well with proprietary graphics drivers such as those from
+Nvidia; if you are using such drivers then you may need to switch to an
+alternative (such as Nouveau in the case of Nvidia hardware) or avoid
+using the GL options.
+
+
+.. _migration-3.2-initramfs-suffix:
+
+initramfs images now use a blank suffix
+---------------------------------------
+
+The reference initramfs images (``core-image-minimal-initramfs``,
+``core-image-tiny-initramfs`` and ``core-image-testmaster-initramfs``) now
+set an empty string for :term:`IMAGE_NAME_SUFFIX`, which otherwise defaults
+to ``".rootfs"``. These images aren't root filesystems and thus the rootfs
+label didn't make sense. If you are looking for the output files generated
+by these image recipes directly then you will need to adapt to the new
+naming without the ``.rootfs`` part.
+
+
+.. _migration-3.2-misc:
+
+Miscellaneous changes
+---------------------
+
+- Support for the long-deprecated ``PACKAGE_GROUP`` variable has now been removed - replace any remaining instances with :term:`FEATURE_PACKAGES`.
+- The ``FILESPATHPKG`` variable, having been previously deprecated, has now been removed. Replace any remaining references with appropriate use of :term:`FILESEXTRAPATHS`.
+- Erroneous use of ``inherit +=`` (instead of ``INHERIT +=``) in a configuration file now triggers an error instead of silently being ignored.
+- ptest support has been removed from the ``kbd`` recipe, as upstream has moved to autotest which is difficult to work with in a cross-compilation environment.
+- ``oe.utils.is_machine_specific()`` and ``oe.utils.machine_paths()`` have been removed as their utility was questionable. In the unlikely event that you have references to these in your own code, then the code will need to be reworked.
+- The ``i2ctransfer`` module is now disabled by default when building ``busybox`` in order to be consistent with disabling the other i2c tools there. If you do wish the i2ctransfer module to be built in busybox then add ``CONFIG_I2CTRANSFER=y`` to your custom busybox configuration.
+- In the ``Upstream-Status`` header convention for patches, ``Accepted`` has been replaced with ``Backport`` as these almost always mean the same thing i.e. the patch is already upstream and may need to be removed in a future recipe upgrade. If you are adding these headers to your own patches then use ``Backport`` to indicate that the patch has been sent upstream.
+- The ``tune-supersparc.inc`` tune file has been removed as it does not appear to be widely used and no longer works.
diff --git a/documentation/ref-manual/migration.rst b/documentation/ref-manual/migration.rst
index 20288b0..8d64a7d 100644
--- a/documentation/ref-manual/migration.rst
+++ b/documentation/ref-manual/migration.rst
@@ -27,4 +27,5 @@ notes for a given release.
    migration-2.7
    migration-3.0
    migration-3.1
+   migration-3.2
 
-- 
1.8.3.1

Re: [docs] [PATCH 1/7] ref-manual: add reference anchors for each QA check

[Quentin Schulz]

Hi Paul,

Nice addition :)

On October 28, 2020 2:18:29 AM GMT+01:00, Paul Eggleton <paul.eggleton@linux.microsoft.com> wrote:
>From: Paul Eggleton <paul.eggleton@microsoft.com>
>
>Make it possible to link to the explanation for a particular QA check.
>
>Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
>---
>documentation/ref-manual/ref-qa-checks.rst | 36
>++++++++++++++++++++++++++++++
> 1 file changed, 36 insertions(+)
>
>diff --git a/documentation/ref-manual/ref-qa-checks.rst
>b/documentation/ref-manual/ref-qa-checks.rst
>index 5b9f92d..228b4fd 100644
>--- a/documentation/ref-manual/ref-qa-checks.rst
>+++ b/documentation/ref-manual/ref-qa-checks.rst
>@@ -43,6 +43,8 @@ error form along with an explanation.
> Errors and Warnings
> ===================
> 
>+.. _qa-check-libexec:

If I'm not mistaken, we have ref-classes-<class> and ref-tasks-<task> already. So for consistency, I'd name them ref-qa-checks-<qa-check>.

Nothing else to nitpick on.

Quentin

Re: [docs] [PATCH 7/7] ref-manual: add migration section for 3.2

[Quentin Schulz]

Hi Paul,

On October 28, 2020 2:18:35 AM GMT+01:00, Paul Eggleton <paul.eggleton@linux.microsoft.com> wrote:
>+In addition, pseudo's behaviour on mismatches has now been changed -
>rather
>+than doing what turns out to be a rather dangerous "fixup" if it sees
>a file
>+with a different path but the same inode as another file it has
>previously seen,
>+pseudo will throw an ``abort()`` and direct you to a `wiki page
><https://wiki.yoctoproject.org/wiki/Pseudo_Abort>`

:yocto_wiki:`wiki page </wiki/Pseudo_Abort>`

>+that explains how to deal with this.
>+
>+
>+.. _migration-3.2-packagegroup-core-device-devel:
>+
>+packagegroup-core-device-devel no longer included in images built for
>qemu* machines
>+------------------------------------------------------------------------------------
>+
>+``packagegroup-core-device-devel`` was previously added automatically
>to
>+images build for ``qemu*`` machines, however the purpose of the group

s/build/built/?

>and what
>+it should contain is no longer clear, and in general, adding userspace
>+development items to images is best done at the image/class level;
>thus this
>+packagegroup was removed.
>+
>+This packagegroup previously pulled in the following:
>+
>+- ``distcc-config``
>+- ``nfs-export-root``
>+- ``bash``
>+- ``binutils-symlinks``
>+
>+If you still need any of these in your image build for a ``qemu*``

s/build/built/?

>machine
>+then you will add them explicitly to :term:`IMAGE_INSTALL` or another
>+appropriate place in the dependency chain for your image (if you have
>not
>+already done so).
>+
>+
>+.. _migration-3.2-dhcp:
>+
>+DHCP server/client replaced
>+---------------------------
>+
>+The ``dhcp`` software package has become unmaintained and thus has
>been
>+functionally replaced by ``dhcpcd`` (client) and ``kea`` (server). You
>will
>+need to replace references to the recipe/package names as appropriate
>- most
>+commonly, at the package level ``dhcp-client`` should be replaced by
>+``dhcpcd`` and ``dhcp-server`` should be replaced by ``kea``. If you
>have any
>+custom configuration files for these they will need to be adapted -
>refer to
>+the upstream documentation for ``dhcpcd`` and ``kea`` for further
>details.
>+
>+
>+.. _migration-3.2-packaging-changes:
>+
>+Packaging changes
>+-----------------
>+
>+- ``python3``: the ``urllib`` python package has now moved into the
>core package, as it is used more commonly than just netclient (e.g.
>email, xml, mimetypes, pydoc). In addition, the ``pathlib`` module is
>now also part of the core package.
>+
>+- ``iptables``: ``iptables-apply`` and ``ip6tables-apply`` have been
>split out to their own package to avoid a bash dependency in the main
>``iptables`` package
>+
>+
>+.. _migration-3.2-package-qa-checks:
>+
>+Package QA check changes
>+------------------------
>+
>+Previously, the following package QA checks triggered warnings,
>however they can
>+be indicators of genuine underlying problems and are therefore now
>treated as
>+errors:
>+
>+- :ref:`already-stripped <qa-check-already-stripped>`
>+- :ref:`compile-host-path <qa-check-compile-host-path>`
>+- :ref:`installed-vs-shipped <qa-check-installed-vs-shipped>`
>+- :ref:`ldflags <qa-check-ldflags>`
>+- :ref:`pn-overrides <qa-check-pn-overrides>`
>+- :ref:`rpaths <qa-check-rpaths>`
>+- :ref:`staticdev <qa-check-staticdev>`
>+- :ref:`unknown-configure-option <qa-check-unknown-configure-option>`
>+- :ref:`useless-rpaths <qa-check-useless-rpaths>`
>+
>+In addition, the following new checks were added and default to
>triggering an error:
>+
>+- :ref:`shebang-size <qa-check-shebang-size>`: Check for shebang (#!)
>lines longer than 128 characters, which can give an error at runtime
>depending on the operating system.
>+
>+- :ref:`unhandled-features-check <qa-check-unhandled-features-check>`:
>Check if any of the variables supported by the :ref:`features_check
><ref-classes-features_check>` class is set while not inheriting the
>class itself.
>+
>+- :ref:`missing-update-alternatives
><qa-check-missing-update-alternatives>`: Check if the recipe sets the
>:term:`ALTERNATIVE` variable for any of its packages, and does not
>inherit the :ref:`update-alternatives
><ref-classes-update-alternatives>` class.
>+

If you rename the references to ref-qa-checks-<qa-check> don't forget to change them here too.

>+- A trailing slash or duplicated slashes in the value of :term:`S` or
>:term:`B` will now trigger a warning so that they can be removed and
>path comparisons can be more reliable - remove any instances of these
>in your recipes if the warning is displayed.
>+
>+
>+.. _migration-3.2-src-uri-file-globbing:
>+
>+Globbing no longer supported in ``file://`` entries in ``SRC_URI``
>+------------------------------------------------------------------
>+
>+Globbing (``*`` and ``?`` wildcards) in ``file://`` URLs within
>:term:`SRC_URI`
>+did not properly support file checksums, thus changes to the source
>files
>+would not always change the do_fetch task checksum, and consequently
>would
>+not ensure that the changed files would be incorporated in subsequent
>builds.
>+
>+Unfortunately it is not practical to make globbing work generically
>here, so
>+the decision was taken to remove support for globs in ``file://``
>URLs.
>+If you have any usage of these in your recipes, then you will now need
>to
>+either add each of the files that you expect to match explicitly, or
>+alternatively if you still need files to be pulled in dynamically, put
>the
>+files into a subdirectory and reference that instead.
>+
>+
>+.. _migration-3.2-deploydir-clean:
>+
>+deploy class now cleans ``DEPLOYDIR`` before ``do_deploy``
>+----------------------------------------------------------
>+
>+``do_deploy`` as implemented in the :ref:`deploy <ref-classes-deploy>`
>class now cleans up ${:term:`DEPLOYDIR`} before running, just as
>``do_install`` cleans up ${:term:`D`} before running. This reduces the
>risk of DEPLOYDIR being accidentally contaminated by files from

``DEPLOYDIR``?

>previous runs, possibly even with different config, in case of
>incremental builds.
>+
>+Most recipes and classes that inherit the ``deploy`` class or interact
>with ``do_deploy`` are unlikely to be affected by this unless they add
>``prefuncs`` to ``do_deploy`` *which also* put files into
>``${DEPLOYDIR}`` - these should be refactored to use
>``do_deploy_prepend`` instead.
>+
>+
>+.. _migration-3.2-nativesdk-sdk-provides-dummy:
>+
>+Custom SDK / SDK-style recipes need to include
>``nativesdk-sdk-provides-dummy``
>+-------------------------------------------------------------------------------
>+
>+All ``nativesdk`` packages require ``/bin/sh`` due to their
>postinstall scriptlets, thus this package has to be dummy-provided
>within the SDK and ``nativesdk-sdk-provides-dummy`` now does this. If
>you have a custom SDK recipe (or your own SDK-style recipe similar to
>e.g. ``buildtools-tarball``), you will need to ensure
>``nativesdk-sdk-provides-dummy`` or an equivalent is included in
>:term:`TOOLCHAIN_HOST_TASK`.
>+
>+
>+``ld.so.conf`` now moved back to main ``glibc`` package
>+-------------------------------------------------------
>+
>+There are cases where one doesn't want ldconfig on target (e.g. for

``ldconfig``?

Quentin

Re: [docs] [PATCH 7/7] ref-manual: add migration section for 3.2

[Richard Purdie]

Hi All,

On Wed, 2020-10-28 at 14:12 +0100, Quentin Schulz wrote:
> Hi Paul,
> 
> On October 28, 2020 2:18:35 AM GMT+01:00, Paul Eggleton <
> paul.eggleton@linux.microsoft.com> wrote:
> > +In addition, pseudo's behaviour on mismatches has now been changed
> > -
> > rather
> > +than doing what turns out to be a rather dangerous "fixup" if it
> > sees
> > a file
> > +with a different path but the same inode as another file it has
> > previously seen,
> > +pseudo will throw an ``abort()`` and direct you to a `wiki page
> > <https://wiki.yoctoproject.org/wiki/Pseudo_Abort>`
> 
> :yocto_wiki:`wiki page </wiki/Pseudo_Abort>`

Thanks for the review! Since time is short and I suspect Paul will be
fine with the tweaks, I've squashed the tweaks into the patches in
master-next. I've not make the qa-check changes, those may be easiest
in a follow up patch if someone wants to send one?

Cheers,

Richard

Re: [docs] [PATCH 4/7] ref-manual: add PSEUDO_IGNORE_PATHS

[Peter Kjellerstedt]

> -----Original Message-----
> From: docs@lists.yoctoproject.org <docs@lists.yoctoproject.org> On Behalf
> Of Paul Eggleton
> Sent: den 28 oktober 2020 02:19
> To: docs@lists.yoctoproject.org
> Subject: [docs] [PATCH 4/7] ref-manual: add PSEUDO_IGNORE_PATHS
> 
> From: Paul Eggleton <paul.eggleton@microsoft.com>
> 
> Add a brief variable glossary entry for the new PSEUDO_IGNORE_PATHS
> variable.
> 
> Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
> ---
>  documentation/ref-manual/ref-variables.rst | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/documentation/ref-manual/ref-variables.rst
> b/documentation/ref-manual/ref-variables.rst
> index a7315cf..176c94a 100644
> --- a/documentation/ref-manual/ref-variables.rst
> +++ b/documentation/ref-manual/ref-variables.rst
> @@ -5932,6 +5932,14 @@ system and gives an overview of their function and contents.
>        service <dev-manual/dev-manual-common-tasks:working with a pr service>`. You can
>        set ``PRSERV_HOST`` to other values to use a remote PR service.
> 
> +
> +   :term:`PSEUDO_IGNORE_PATHS`
> +      A comma-separated (without spaces) list of files that should be ignored

Based on the default definition of PSEUDO_IGNORE_PATHS in bitbake.conf, saying 
"files" here seems misleading. It seems to support files, directories and even 
prefixes, e.g., "${WORKDIR}/deploy-".

> +      by pseudo when monitoring and recording file operations, in order to avoid
> +      problems with files being written to outside of the pseudo context and
> +      reduce pseudo's overhead.
> +
> +
>     :term:`PTEST_ENABLED`
>        Specifies whether or not :ref:`Package
>        Test <dev-manual/dev-manual-common-tasks:testing packages with ptest>` (ptest)
> --
> 1.8.3.1

//Peter

Re: [docs] [PATCH 4/7] ref-manual: add PSEUDO_IGNORE_PATHS

[Richard Purdie]

On Wed, 2020-10-28 at 22:18 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: docs@lists.yoctoproject.org <docs@lists.yoctoproject.org> On Behalf
> > Of Paul Eggleton
> > Sent: den 28 oktober 2020 02:19
> > To: docs@lists.yoctoproject.org
> > Subject: [docs] [PATCH 4/7] ref-manual: add PSEUDO_IGNORE_PATHS
> > 
> > From: Paul Eggleton <paul.eggleton@microsoft.com>
> > 
> > Add a brief variable glossary entry for the new PSEUDO_IGNORE_PATHS
> > variable.
> > 
> > Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
> > ---
> >  documentation/ref-manual/ref-variables.rst | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> > 
> > diff --git a/documentation/ref-manual/ref-variables.rst
> > b/documentation/ref-manual/ref-variables.rst
> > index a7315cf..176c94a 100644
> > --- a/documentation/ref-manual/ref-variables.rst
> > +++ b/documentation/ref-manual/ref-variables.rst
> > @@ -5932,6 +5932,14 @@ system and gives an overview of their function and contents.
> >        service <dev-manual/dev-manual-common-tasks:working with a pr service>`. You can
> >        set ``PRSERV_HOST`` to other values to use a remote PR service.
> > 
> > +
> > +   :term:`PSEUDO_IGNORE_PATHS`
> > +      A comma-separated (without spaces) list of files that should be ignored
> 
> Based on the default definition of PSEUDO_IGNORE_PATHS in bitbake.conf, saying 
> "files" here seems misleading. It seems to support files, directories and even 
> prefixes, e.g., "${WORKDIR}/deploy-".

That is correct, I've tweaked the patch queued in -next to reflect
that, thanks.

Cheers,

Richard