[PATCH v3 0/5] Complete upstream version check documentation

[PATCH v3 0/5] Complete upstream version check documentation

[Michael Opdenacker]

This V3 add documentation for the UPSTREAM_CHECK_COMMITS and
UPSTREAM_VERSION_UNKNOWN variables, and takes their usage into account
in the explanations.

There is a little bit of extra text to review if you're interested.

This addresses https://bugzilla.yoctoproject.org/show_bug.cgi?id=14431

Thanks in advance.

Michael Opdenacker (5):
  ref-manual: document UPSTREAM_CHECK_COMMITS and
    UPSTREAM_VERSION_UNKNOWN
  ref-manual: remove checkpkg task
  ref-manual: improve "devtool check-upgrade-status" details
  ref-manual: improve documentation for RECIPE_NO_UPDATE_REASON
  ref-manual: update "devtool check-upgrade-status" output

 .../ref-manual/devtool-reference.rst          | 57 ++++++++++---------
 documentation/ref-manual/tasks.rst            | 31 ----------
 documentation/ref-manual/variables.rst        | 35 +++++++++++-
 3 files changed, 63 insertions(+), 60 deletions(-)

-- 
2.25.1

[PATCH v3 1/5] ref-manual: document UPSTREAM_CHECK_COMMITS and UPSTREAM_VERSION_UNKNOWN

[Michael Opdenacker]

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/ref-manual/variables.rst | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 5caada6260..d06ef5a632 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8483,6 +8483,18 @@ system and gives an overview of their function and contents.
       install initscripts package them in the main package for the recipe,
       you rarely need to set this variable in individual recipes.
 
+   :term:`UPSTREAM_CHECK_COMMITS`
+      You can perform a per-recipe check for what the latest upstream
+      source code version is by calling ``devtool latest-version recipe``. If
+      the recipe source code is provided from Git repositories, but
+      releases are not identified by Git tags, set :term:`UPSTREAM_CHECK_COMMITS`
+      to ``1`` in the recipe, and the OpenEmbedded build system
+      will compare the latest commit with the one currently specified
+      by the recipe (:term:`SRCREV`).
+      ::
+
+         UPSTREAM_CHECK_COMMITS = "1"
+
    :term:`UPSTREAM_CHECK_GITTAGREGEX`
       You can perform a per-recipe check for what the latest upstream
       source code version is by calling ``bitbake -c checkpkg`` recipe. If
@@ -8518,6 +8530,18 @@ system and gives an overview of their function and contents.
 
          UPSTREAM_CHECK_URI = "recipe_url"
 
+   :term:`UPSTREAM_VERSION_UNKNOWN`
+      You can perform a per-recipe check for what the latest upstream
+      source code version is by calling ``devtool latest-version recipe``.
+      If no combination of the :term:`UPSTREAM_CHECK_URI`, :term:`UPSTREAM_CHECK_REGEX`,
+      `UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS` variables in
+      the recipe allows to determine such an latest upstream version,
+      you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe
+      to acknowledge that the check cannot be performed.
+      ::
+
+         UPSTREAM_VERSION_UNKNOWN = "1"
+
    :term:`USE_DEVFS`
       Determines if ``devtmpfs`` is used for ``/dev`` population. The
       default value used for :term:`USE_DEVFS` is "1" when no value is
-- 
2.25.1

[PATCH v3 2/5] ref-manual: remove checkpkg task

[Michael Opdenacker]

It doesn't exist any more and its functionality is now taken care
of by "devtool check-upgrade-status"

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Reviewed-by: Quentin Schulz <foss@0leil.net>
---
 documentation/ref-manual/tasks.rst     | 31 --------------------------
 documentation/ref-manual/variables.rst |  4 ++--
 2 files changed, 2 insertions(+), 33 deletions(-)

diff --git a/documentation/ref-manual/tasks.rst b/documentation/ref-manual/tasks.rst
index ecc701856c..4edae33392 100644
--- a/documentation/ref-manual/tasks.rst
+++ b/documentation/ref-manual/tasks.rst
@@ -438,37 +438,6 @@ Manually Called Tasks
 These tasks are typically manually triggered (e.g. by using the
 ``bitbake -c`` command-line option):
 
-.. _ref-tasks-checkpkg:
-
-``do_checkpkg``
----------------
-
-Provides information about the recipe including its upstream version and
-status. The upstream version and status reveals whether or not a version
-of the recipe exists upstream and a status of not updated, updated, or
-unknown.
-
-To check the upstream version and status of a recipe, use the following
-devtool commands::
-
-   $ devtool latest-version
-   $ devtool check-upgrade-status
-
-See the ":ref:`ref-manual/devtool-reference:\`\`devtool\`\` quick reference`"
-chapter for more information on
-``devtool``. See the ":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`"
-section for information on checking the upgrade status of a recipe.
-
-To build the ``checkpkg`` task, use the ``bitbake`` command with the
-"-c" option and task name::
-
-   $ bitbake core-image-minimal -c checkpkg
-
-By default, the results are stored in :term:`$LOG_DIR <LOG_DIR>` (e.g.
-``$BUILD_DIR/tmp/log``).
-
-.. _ref-tasks-checkuri:
-
 ``do_checkuri``
 ---------------
 
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index d06ef5a632..2342846dc7 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8497,7 +8497,7 @@ system and gives an overview of their function and contents.
 
    :term:`UPSTREAM_CHECK_GITTAGREGEX`
       You can perform a per-recipe check for what the latest upstream
-      source code version is by calling ``bitbake -c checkpkg`` recipe. If
+      source code version is by calling ``devtool latest-version recipe``. If
       the recipe source code is provided from Git repositories, the
       OpenEmbedded build system determines the latest upstream version by
       picking the latest tag from the list of all repository tags.
@@ -8520,7 +8520,7 @@ system and gives an overview of their function and contents.
 
    :term:`UPSTREAM_CHECK_URI`
       You can perform a per-recipe check for what the latest upstream
-      source code version is by calling ``bitbake -c checkpkg`` recipe. If
+      source code version is by calling ``devtool latest-version recipe``. If
       the source code is provided from tarballs, the latest version is
       determined by fetching the directory listing where the tarball is and
       attempting to find a later tarball. When this approach does not work,
-- 
2.25.1

[PATCH v3 3/5] ref-manual: improve "devtool check-upgrade-status" details

[Michael Opdenacker]

- Mention "devtool latest-version recipe" to process a single version
- Explain that the mechanism is controlled by the UPSTREAM_CHECK*
  variables, which are not referred to anywhere else in the manuals,
  except in the variable index.

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 .../ref-manual/devtool-reference.rst          | 29 +++++++++++++------
 documentation/ref-manual/variables.rst        |  4 +--
 2 files changed, 22 insertions(+), 11 deletions(-)

diff --git a/documentation/ref-manual/devtool-reference.rst b/documentation/ref-manual/devtool-reference.rst
index 5fd51569a3..2cf9fd43a3 100644
--- a/documentation/ref-manual/devtool-reference.rst
+++ b/documentation/ref-manual/devtool-reference.rst
@@ -328,12 +328,19 @@ Checking on the Upgrade Status of a Recipe
 Upstream recipes change over time. Consequently, you might find that you
 need to determine if you can upgrade a recipe to a newer version.
 
-To check on the upgrade status of a recipe, use the
-``devtool check-upgrade-status`` command. The command displays a table
-of your current recipe versions, the latest upstream versions, the email
-address of the recipe's maintainer, and any additional information such
-as commit hash strings and reasons you might not be able to upgrade a
-particular recipe.
+To check on the upgrade status of a recipe, you can use the
+``devtool latest-version recipe`` command, which quickly shows the current
+version and the latest version available upstream. To get such information
+for all available recipes, use the ``devtool check-upgrade-status`` command.
+The command displays a table of your current recipe versions, the latest
+upstream versions, the email address of the recipe's maintainer, and any
+additional information such as commit hash strings and reasons you might
+not be able to upgrade a particular recipe.
+
+This upgrade checking mechanism relies on the optional :term:`UPSTREAM_CHECK_URI`,
+:term:`UPSTREAM_CHECK_REGEX`, :term:`UPSTREAM_CHECK_GITTAGREGEX`,
+:term:`UPSTREAM_CHECK_COMMITS` and :term:`UPSTREAM_VERSION_UNKNOWN`
+variables which are defined in package recipes.
 
 .. note::
 
@@ -342,9 +349,10 @@ particular recipe.
       file.
 
    -  If the recipe is using the :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:git fetcher (\`\`git://\`\`)`
-      rather than a
-      tarball, the commit hash points to the commit that matches the
-      recipe's latest version tag.
+      rather than a tarball, the commit hash points to the commit that matches
+      the recipe's latest version tag, or in the absence of suitable tags,
+      to the latest commit (when :term:`UPSTREAM_CHECK_COMMITS` set to ``1``
+      in the recipe).
 
 As with all ``devtool`` commands, you can get help on the individual
 command::
@@ -393,6 +401,9 @@ satisfied.
    NOTE: busybox 1.29.2 1.30.0 Andrej Valek <andrej.valek@siemens.com>
    NOTE: dbus-test 1.12.10 1.12.12 Chen Qi <Qi.Chen@windriver.com>
 
+Last but not least, you may set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1``
+in a recipe when no way is found to determine its latest upstream version.
+
 .. _devtool-upgrading-a-recipe:
 
 Upgrading a Recipe
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 2342846dc7..7545e708c6 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -8534,9 +8534,9 @@ system and gives an overview of their function and contents.
       You can perform a per-recipe check for what the latest upstream
       source code version is by calling ``devtool latest-version recipe``.
       If no combination of the :term:`UPSTREAM_CHECK_URI`, :term:`UPSTREAM_CHECK_REGEX`,
-      `UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS` variables in
+      :term:`UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS` variables in
       the recipe allows to determine such an latest upstream version,
-      you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe
+      you can set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1`` in the recipe
       to acknowledge that the check cannot be performed.
       ::
 
-- 
2.25.1

[PATCH v3 4/5] ref-manual: improve documentation for RECIPE_NO_UPDATE_REASON

[Michael Opdenacker]

This add a description of this variable to the variable index,
and clarifies the explanations given in the devtool reference
section.

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/ref-manual/devtool-reference.rst | 13 ++++---------
 documentation/ref-manual/variables.rst         |  7 +++++++
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/documentation/ref-manual/devtool-reference.rst b/documentation/ref-manual/devtool-reference.rst
index 2cf9fd43a3..8819879d2f 100644
--- a/documentation/ref-manual/devtool-reference.rst
+++ b/documentation/ref-manual/devtool-reference.rst
@@ -377,15 +377,10 @@ Following is a partial example table that reports on all the recipes.
 Notice the reported reason for not upgrading the ``base-passwd`` recipe.
 In this example, while a new version is available upstream, you do not
 want to use it because the dependency on ``cdebconf`` is not easily
-satisfied.
-
-.. note::
-
-   When a reason for not upgrading displays, the reason is usually
-   written into the recipe using the ``RECIPE_NO_UPDATE_REASON``
-   variable. See the
-   :yocto_git:`base-passwd.bb </poky/tree/meta/recipes-core/base-passwd/base-passwd_3.5.29.bb>`
-   recipe for an example.
+satisfied. Maintainers can explicit the reason that is shown by adding
+the :term:`RECIPE_NO_UPDATE_REASON` variable to the corresponding recipe.
+See :yocto_git:`base-passwd.bb </poky/tree/meta/recipes-core/base-passwd/base-passwd_3.5.29.bb>`
+for an example.
 
 ::
 
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 7545e708c6..14ace91a85 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -6132,6 +6132,13 @@ system and gives an overview of their function and contents.
       BitBake User Manual for additional information on tasks and
       dependencies.
 
+   :term:`RECIPE_NO_UPDATE_REASON`
+      If a recipe should not be replaced by a more recent upstream version,
+      putting the reason why in this variable in a recipe allows
+      ``devtool check-upgrade-status`` command to display it, as explained
+      in the ":ref:`ref-manual/devtool-reference:checking on the upgrade status of a recipe`"
+      section.
+
    :term:`REQUIRED_DISTRO_FEATURES`
       When inheriting the
       :ref:`features_check <ref-classes-features_check>`
-- 
2.25.1

[PATCH v3 5/5] ref-manual: update "devtool check-upgrade-status" output

[Michael Opdenacker]

This command now uses "INFO" insted of "NOTES" and output
information in columns. Also good to show more recent
versions of recipes.

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/ref-manual/devtool-reference.rst | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/documentation/ref-manual/devtool-reference.rst b/documentation/ref-manual/devtool-reference.rst
index 8819879d2f..1d2064f77f 100644
--- a/documentation/ref-manual/devtool-reference.rst
+++ b/documentation/ref-manual/devtool-reference.rst
@@ -386,15 +386,12 @@ for an example.
 
    $ devtool check-upgrade-status
    ...
-   NOTE: acpid 2.0.30 2.0.31 Ross Burton <ross.burton@intel.com>
-   NOTE: u-boot-fw-utils 2018.11 2019.01 Marek Vasut <marek.vasut@gmail.com> d3689267f92c5956e09cc7d1baa4700141662bff
-   NOTE: u-boot-tools 2018.11 2019.01 Marek Vasut <marek.vasut@gmail.com> d3689267f92c5956e09cc7d1baa4700141662bff
-   .
-   .
-   .
-   NOTE: base-passwd 3.5.29 3.5.45 Anuj Mittal <anuj.mittal@intel.com> cannot be updated due to: Version 3.5.38 requires cdebconf for update-passwd utility
-   NOTE: busybox 1.29.2 1.30.0 Andrej Valek <andrej.valek@siemens.com>
-   NOTE: dbus-test 1.12.10 1.12.12 Chen Qi <Qi.Chen@windriver.com>
+   INFO: bind                      9.16.20         9.16.21         Armin Kuster <akuster808@gmail.com>
+   INFO: inetutils                 2.1             2.2             Tom Rini <trini@konsulko.com>
+   INFO: iproute2                  5.13.0          5.14.0          Changhyeok Bae <changhyeok.bae@gmail.com>
+   INFO: openssl                   1.1.1l          3.0.0           Alexander Kanavin <alex.kanavin@gmail.com>
+   INFO: base-passwd               3.5.29          3.5.51          Anuj Mittal <anuj.mittal@intel.com>  cannot be updated due to: Version 3.5.38 requires cdebconf for update-passwd utility
+   ...
 
 Last but not least, you may set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1``
 in a recipe when no way is found to determine its latest upstream version.
-- 
2.25.1

Re: [docs] [PATCH v3 3/5] ref-manual: improve "devtool check-upgrade-status" details

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 5026 bytes --]

The functionality overlaps somewhat. 'devtool latest-version' takes a
single recipe as an argument, but 'devtool check-upgrade-status' takes a
list of recipes, or nothing (in which case a check for everything runs),
and prints only the items where there is a new upstream version available
in a table format. The former is intended when you want to see a specific
status of something particular, the latter for an overall picture.

Alex

On Wed, 15 Sept 2021 at 19:58, Michael Opdenacker <
michael.opdenacker@bootlin.com> wrote:

> - Mention "devtool latest-version recipe" to process a single version
> - Explain that the mechanism is controlled by the UPSTREAM_CHECK*
>   variables, which are not referred to anywhere else in the manuals,
>   except in the variable index.
>
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>  .../ref-manual/devtool-reference.rst          | 29 +++++++++++++------
>  documentation/ref-manual/variables.rst        |  4 +--
>  2 files changed, 22 insertions(+), 11 deletions(-)
>
> diff --git a/documentation/ref-manual/devtool-reference.rst
> b/documentation/ref-manual/devtool-reference.rst
> index 5fd51569a3..2cf9fd43a3 100644
> --- a/documentation/ref-manual/devtool-reference.rst
> +++ b/documentation/ref-manual/devtool-reference.rst
> @@ -328,12 +328,19 @@ Checking on the Upgrade Status of a Recipe
>  Upstream recipes change over time. Consequently, you might find that you
>  need to determine if you can upgrade a recipe to a newer version.
>
> -To check on the upgrade status of a recipe, use the
> -``devtool check-upgrade-status`` command. The command displays a table
> -of your current recipe versions, the latest upstream versions, the email
> -address of the recipe's maintainer, and any additional information such
> -as commit hash strings and reasons you might not be able to upgrade a
> -particular recipe.
> +To check on the upgrade status of a recipe, you can use the
> +``devtool latest-version recipe`` command, which quickly shows the current
> +version and the latest version available upstream. To get such information
> +for all available recipes, use the ``devtool check-upgrade-status``
> command.
> +The command displays a table of your current recipe versions, the latest
> +upstream versions, the email address of the recipe's maintainer, and any
> +additional information such as commit hash strings and reasons you might
> +not be able to upgrade a particular recipe.
> +
> +This upgrade checking mechanism relies on the optional
> :term:`UPSTREAM_CHECK_URI`,
> +:term:`UPSTREAM_CHECK_REGEX`, :term:`UPSTREAM_CHECK_GITTAGREGEX`,
> +:term:`UPSTREAM_CHECK_COMMITS` and :term:`UPSTREAM_VERSION_UNKNOWN`
> +variables which are defined in package recipes.
>
>  .. note::
>
> @@ -342,9 +349,10 @@ particular recipe.
>        file.
>
>     -  If the recipe is using the
> :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:git fetcher
> (\`\`git://\`\`)`
> -      rather than a
> -      tarball, the commit hash points to the commit that matches the
> -      recipe's latest version tag.
> +      rather than a tarball, the commit hash points to the commit that
> matches
> +      the recipe's latest version tag, or in the absence of suitable tags,
> +      to the latest commit (when :term:`UPSTREAM_CHECK_COMMITS` set to
> ``1``
> +      in the recipe).
>
>  As with all ``devtool`` commands, you can get help on the individual
>  command::
> @@ -393,6 +401,9 @@ satisfied.
>     NOTE: busybox 1.29.2 1.30.0 Andrej Valek <andrej.valek@siemens.com>
>     NOTE: dbus-test 1.12.10 1.12.12 Chen Qi <Qi.Chen@windriver.com>
>
> +Last but not least, you may set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1``
> +in a recipe when no way is found to determine its latest upstream version.
> +
>  .. _devtool-upgrading-a-recipe:
>
>  Upgrading a Recipe
> diff --git a/documentation/ref-manual/variables.rst
> b/documentation/ref-manual/variables.rst
> index 2342846dc7..7545e708c6 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -8534,9 +8534,9 @@ system and gives an overview of their function and
> contents.
>        You can perform a per-recipe check for what the latest upstream
>        source code version is by calling ``devtool latest-version recipe``.
>        If no combination of the :term:`UPSTREAM_CHECK_URI`,
> :term:`UPSTREAM_CHECK_REGEX`,
> -      `UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS`
> variables in
> +      :term:`UPSTREAM_CHECK_GITTAGREGEX` and
> :term:`UPSTREAM_CHECK_COMMITS` variables in
>        the recipe allows to determine such an latest upstream version,
> -      you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe
> +      you can set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1`` in the recipe
>        to acknowledge that the check cannot be performed.
>        ::
>
> --
> 2.25.1
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 5830 bytes --]

Re: [docs] [PATCH v3 3/5] ref-manual: improve "devtool check-upgrade-status" details

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 4774 bytes --]

Another point to mention is that UPSTREAM_* variables do not need to be
set, most of the time. They are only required when upstream does something
unusual, and defaults do not work.

Alex

On Wed, 15 Sept 2021 at 19:58, Michael Opdenacker <
michael.opdenacker@bootlin.com> wrote:

> - Mention "devtool latest-version recipe" to process a single version
> - Explain that the mechanism is controlled by the UPSTREAM_CHECK*
>   variables, which are not referred to anywhere else in the manuals,
>   except in the variable index.
>
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>  .../ref-manual/devtool-reference.rst          | 29 +++++++++++++------
>  documentation/ref-manual/variables.rst        |  4 +--
>  2 files changed, 22 insertions(+), 11 deletions(-)
>
> diff --git a/documentation/ref-manual/devtool-reference.rst
> b/documentation/ref-manual/devtool-reference.rst
> index 5fd51569a3..2cf9fd43a3 100644
> --- a/documentation/ref-manual/devtool-reference.rst
> +++ b/documentation/ref-manual/devtool-reference.rst
> @@ -328,12 +328,19 @@ Checking on the Upgrade Status of a Recipe
>  Upstream recipes change over time. Consequently, you might find that you
>  need to determine if you can upgrade a recipe to a newer version.
>
> -To check on the upgrade status of a recipe, use the
> -``devtool check-upgrade-status`` command. The command displays a table
> -of your current recipe versions, the latest upstream versions, the email
> -address of the recipe's maintainer, and any additional information such
> -as commit hash strings and reasons you might not be able to upgrade a
> -particular recipe.
> +To check on the upgrade status of a recipe, you can use the
> +``devtool latest-version recipe`` command, which quickly shows the current
> +version and the latest version available upstream. To get such information
> +for all available recipes, use the ``devtool check-upgrade-status``
> command.
> +The command displays a table of your current recipe versions, the latest
> +upstream versions, the email address of the recipe's maintainer, and any
> +additional information such as commit hash strings and reasons you might
> +not be able to upgrade a particular recipe.
> +
> +This upgrade checking mechanism relies on the optional
> :term:`UPSTREAM_CHECK_URI`,
> +:term:`UPSTREAM_CHECK_REGEX`, :term:`UPSTREAM_CHECK_GITTAGREGEX`,
> +:term:`UPSTREAM_CHECK_COMMITS` and :term:`UPSTREAM_VERSION_UNKNOWN`
> +variables which are defined in package recipes.
>
>  .. note::
>
> @@ -342,9 +349,10 @@ particular recipe.
>        file.
>
>     -  If the recipe is using the
> :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:git fetcher
> (\`\`git://\`\`)`
> -      rather than a
> -      tarball, the commit hash points to the commit that matches the
> -      recipe's latest version tag.
> +      rather than a tarball, the commit hash points to the commit that
> matches
> +      the recipe's latest version tag, or in the absence of suitable tags,
> +      to the latest commit (when :term:`UPSTREAM_CHECK_COMMITS` set to
> ``1``
> +      in the recipe).
>
>  As with all ``devtool`` commands, you can get help on the individual
>  command::
> @@ -393,6 +401,9 @@ satisfied.
>     NOTE: busybox 1.29.2 1.30.0 Andrej Valek <andrej.valek@siemens.com>
>     NOTE: dbus-test 1.12.10 1.12.12 Chen Qi <Qi.Chen@windriver.com>
>
> +Last but not least, you may set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1``
> +in a recipe when no way is found to determine its latest upstream version.
> +
>  .. _devtool-upgrading-a-recipe:
>
>  Upgrading a Recipe
> diff --git a/documentation/ref-manual/variables.rst
> b/documentation/ref-manual/variables.rst
> index 2342846dc7..7545e708c6 100644
> --- a/documentation/ref-manual/variables.rst
> +++ b/documentation/ref-manual/variables.rst
> @@ -8534,9 +8534,9 @@ system and gives an overview of their function and
> contents.
>        You can perform a per-recipe check for what the latest upstream
>        source code version is by calling ``devtool latest-version recipe``.
>        If no combination of the :term:`UPSTREAM_CHECK_URI`,
> :term:`UPSTREAM_CHECK_REGEX`,
> -      `UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS`
> variables in
> +      :term:`UPSTREAM_CHECK_GITTAGREGEX` and
> :term:`UPSTREAM_CHECK_COMMITS` variables in
>        the recipe allows to determine such an latest upstream version,
> -      you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe
> +      you can set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1`` in the recipe
>        to acknowledge that the check cannot be performed.
>        ::
>
> --
> 2.25.1
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 5565 bytes --]

Re: [docs] [PATCH v3 0/5] Complete upstream version check documentation

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1134 bytes --]

Other than my two comments, lgtm, thanks!

Alex

On Wed, 15 Sept 2021 at 19:58, Michael Opdenacker <
michael.opdenacker@bootlin.com> wrote:

> This V3 add documentation for the UPSTREAM_CHECK_COMMITS and
> UPSTREAM_VERSION_UNKNOWN variables, and takes their usage into account
> in the explanations.
>
> There is a little bit of extra text to review if you're interested.
>
> This addresses https://bugzilla.yoctoproject.org/show_bug.cgi?id=14431
>
> Thanks in advance.
>
> Michael Opdenacker (5):
>   ref-manual: document UPSTREAM_CHECK_COMMITS and
>     UPSTREAM_VERSION_UNKNOWN
>   ref-manual: remove checkpkg task
>   ref-manual: improve "devtool check-upgrade-status" details
>   ref-manual: improve documentation for RECIPE_NO_UPDATE_REASON
>   ref-manual: update "devtool check-upgrade-status" output
>
>  .../ref-manual/devtool-reference.rst          | 57 ++++++++++---------
>  documentation/ref-manual/tasks.rst            | 31 ----------
>  documentation/ref-manual/variables.rst        | 35 +++++++++++-
>  3 files changed, 63 insertions(+), 60 deletions(-)
>
> --
> 2.25.1
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 1673 bytes --]

Re: [docs] [PATCH v3 1/5] ref-manual: document UPSTREAM_CHECK_COMMITS and UPSTREAM_VERSION_UNKNOWN

[Quentin Schulz]

Hi Michael,

On September 15, 2021 7:58:38 PM GMT+02:00, Michael Opdenacker <michael.opdenacker@bootlin.com> wrote:
>Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>---
> documentation/ref-manual/variables.rst | 24 ++++++++++++++++++++++++
> 1 file changed, 24 insertions(+)
>
>diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
>index 5caada6260..d06ef5a632 100644
>--- a/documentation/ref-manual/variables.rst
>+++ b/documentation/ref-manual/variables.rst
>@@ -8483,6 +8483,18 @@ system and gives an overview of their function and contents.
>       install initscripts package them in the main package for the recipe,
>       you rarely need to set this variable in individual recipes.
> 
>+   :term:`UPSTREAM_CHECK_COMMITS`
>+      You can perform a per-recipe check for what the latest upstream
>+      source code version is by calling ``devtool latest-version recipe``. If
>+      the recipe source code is provided from Git repositories, but
>+      releases are not identified by Git tags, set :term:`UPSTREAM_CHECK_COMMITS`
>+      to ``1`` in the recipe, and the OpenEmbedded build system
>+      will compare the latest commit with the one currently specified
>+      by the recipe (:term:`SRCREV`).
>+      ::
>+
>+         UPSTREAM_CHECK_COMMITS = "1"
>+
>    :term:`UPSTREAM_CHECK_GITTAGREGEX`
>       You can perform a per-recipe check for what the latest upstream
>       source code version is by calling ``bitbake -c checkpkg`` recipe. If
>@@ -8518,6 +8530,18 @@ system and gives an overview of their function and contents.
> 
>          UPSTREAM_CHECK_URI = "recipe_url"
> 
>+   :term:`UPSTREAM_VERSION_UNKNOWN`
>+      You can perform a per-recipe check for what the latest upstream
>+      source code version is by calling ``devtool latest-version recipe``.
>+      If no combination of the :term:`UPSTREAM_CHECK_URI`, :term:`UPSTREAM_CHECK_REGEX`,
>+      `UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS` variables in
>+      the recipe allows to determine such an latest upstream version,

"allows to determine what the latest upstream version is,"
?

>+      you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe

Spurious tick quote after the term reference.

Looks good otherwise, thanks!
Quentin

>+      to acknowledge that the check cannot be performed.
>+      ::
>+
>+         UPSTREAM_VERSION_UNKNOWN = "1"
>+
>    :term:`USE_DEVFS`
>       Determines if ``devtmpfs`` is used for ``/dev`` population. The
>       default value used for :term:`USE_DEVFS` is "1" when no value is

Re: [docs] [PATCH v3 3/5] ref-manual: improve "devtool check-upgrade-status" details

[Quentin Schulz]

Hi Michael,

On September 15, 2021 7:58:40 PM GMT+02:00, Michael Opdenacker <michael.opdenacker@bootlin.com> wrote:
>- Mention "devtool latest-version recipe" to process a single version
>- Explain that the mechanism is controlled by the UPSTREAM_CHECK*
>  variables, which are not referred to anywhere else in the manuals,
>  except in the variable index.
>
>Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>---
> .../ref-manual/devtool-reference.rst          | 29 +++++++++++++------
> documentation/ref-manual/variables.rst        |  4 +--
> 2 files changed, 22 insertions(+), 11 deletions(-)
>
>diff --git a/documentation/ref-manual/devtool-reference.rst b/documentation/ref-manual/devtool-reference.rst
>index 5fd51569a3..2cf9fd43a3 100644
>--- a/documentation/ref-manual/devtool-reference.rst
>+++ b/documentation/ref-manual/devtool-reference.rst
>@@ -328,12 +328,19 @@ Checking on the Upgrade Status of a Recipe
> Upstream recipes change over time. Consequently, you might find that you
> need to determine if you can upgrade a recipe to a newer version.
> 
>-To check on the upgrade status of a recipe, use the
>-``devtool check-upgrade-status`` command. The command displays a table
>-of your current recipe versions, the latest upstream versions, the email
>-address of the recipe's maintainer, and any additional information such
>-as commit hash strings and reasons you might not be able to upgrade a
>-particular recipe.
>+To check on the upgrade status of a recipe, you can use the
>+``devtool latest-version recipe`` command, which quickly shows the current
>+version and the latest version available upstream. To get such information
>+for all available recipes, use the ``devtool check-upgrade-status`` command.
>+The command displays a table of your current recipe versions, the latest
>+upstream versions, the email address of the recipe's maintainer, and any
>+additional information such as commit hash strings and reasons you might
>+not be able to upgrade a particular recipe.
>+

I am always lost with singular/plural for one thing that applies to each and every one of a number of items. I am not sure "recipe versions" is correct here. I was thinking "recipes version" would match better (one version per recipe and there are many recipes) but cannot decide 🙃

"""
Each recipe will have its current version and latest upstream version, as well as the email of the maintainer and any additional information such as the commit hash or reason for not being able to upgrade it, displayed in a table.
"""

is what I could come up with. Hopefully it's a bit clearer to not only me :)


>+This upgrade checking mechanism relies on the optional :term:`UPSTREAM_CHECK_URI`,
>+:term:`UPSTREAM_CHECK_REGEX`, :term:`UPSTREAM_CHECK_GITTAGREGEX`,
>+:term:`UPSTREAM_CHECK_COMMITS` and :term:`UPSTREAM_VERSION_UNKNOWN`
>+variables which are defined in package recipes.
> 
> .. note::
> 
>@@ -342,9 +349,10 @@ particular recipe.
>       file.
> 
>    -  If the recipe is using the :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:git fetcher (\`\`git://\`\`)`
>-      rather than a
>-      tarball, the commit hash points to the commit that matches the
>-      recipe's latest version tag.
>+      rather than a tarball, the commit hash points to the commit that matches
>+      the recipe's latest version tag, or in the absence of suitable tags,
>+      to the latest commit (when :term:`UPSTREAM_CHECK_COMMITS` set to ``1``
>+      in the recipe).
> 
> As with all ``devtool`` commands, you can get help on the individual
> command::
>@@ -393,6 +401,9 @@ satisfied.
>    NOTE: busybox 1.29.2 1.30.0 Andrej Valek <andrej.valek@siemens.com>
>    NOTE: dbus-test 1.12.10 1.12.12 Chen Qi <Qi.Chen@windriver.com>
> 
>+Last but not least, you may set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1``
>+in a recipe when no way is found to determine its latest upstream version.
>+

"""
when there's currently no way to determine...
"""
?
> .. _devtool-upgrading-a-recipe:
> 
> Upgrading a Recipe
>diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
>index 2342846dc7..7545e708c6 100644
>--- a/documentation/ref-manual/variables.rst
>+++ b/documentation/ref-manual/variables.rst
>@@ -8534,9 +8534,9 @@ system and gives an overview of their function and contents.
>       You can perform a per-recipe check for what the latest upstream
>       source code version is by calling ``devtool latest-version recipe``.
>       If no combination of the :term:`UPSTREAM_CHECK_URI`, :term:`UPSTREAM_CHECK_REGEX`,
>-      `UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS` variables in
>+      :term:`UPSTREAM_CHECK_GITTAGREGEX` and :term:`UPSTREAM_CHECK_COMMITS` variables in
>       the recipe allows to determine such an latest upstream version,
>-      you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe
>+      you can set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1`` in the recipe

This one typo fix should be in the commit that introduces it :)

Cheers,
Quentin

>       to acknowledge that the check cannot be performed.
>       ::
> 

Re: [docs] [PATCH v3 1/5] ref-manual: document UPSTREAM_CHECK_COMMITS and UPSTREAM_VERSION_UNKNOWN

[Michael Opdenacker]

Hi Quentin,

Thanks for the reviews!

On 9/16/21 9:14 AM, Quentin Schulz wrote:
>
> "allows to determine what the latest upstream version is,"


Oops, that's what you get when you write text too late in the day.
Fixed, thanks!

>> +      you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe
> Spurious tick quote after the term reference.


Fix too, thanks again.
Cheers,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH v3 3/5] ref-manual: improve "devtool check-upgrade-status" details

[Michael Opdenacker]

Hi Quentin,

On 9/16/21 9:31 AM, Quentin Schulz wrote:
>> +To check on the upgrade status of a recipe, you can use the
>> +``devtool latest-version recipe`` command, which quickly shows the current
>> +version and the latest version available upstream. To get such information
>> +for all available recipes, use the ``devtool check-upgrade-status`` command.
>> +The command displays a table of your current recipe versions, the latest
>> +upstream versions, the email address of the recipe's maintainer, and any
>> +additional information such as commit hash strings and reasons you might
>> +not be able to upgrade a particular recipe.
>> +
> I am always lost with singular/plural for one thing that applies to each and every one of a number of items. I am not sure "recipe versions" is correct here. I was thinking "recipes version" would match better (one version per recipe and there are many recipes) but cannot decide 🙃
>
> """
> Each recipe will have its current version and latest upstream version, as well as the email of the maintainer and any additional information such as the commit hash or reason for not being able to upgrade it, displayed in a table.
> """
>
> is what I could come up with. Hopefully it's a bit clearer to not only me :)


I wasn't the one writing this, but I agree the expression is ambiguous.
I adopted your suggestion, which I agree is clearer. Thanks!

>
>
> +Last but not least, you may set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1``
> +in a recipe when no way is found to determine its latest upstream version.
> +
> """
> when there's currently no way to determine...


Ah, right, that's better, thanks!

> - you can set :term:`UPSTREAM_VERSION_UNKNOWN`` to ``1`` in the recipe
>> +      you can set :term:`UPSTREAM_VERSION_UNKNOWN` to ``1`` in the recipe
> This one typo fix should be in the commit that introduces it :)


Yes, right. I noticed too late and this is embarrassing. My bad!

Thanks again,

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH v3 3/5] ref-manual: improve "devtool check-upgrade-status" details

[Michael Opdenacker]

Hi Alex

Thanks for the reviews!

On 9/15/21 8:39 PM, Alexander Kanavin wrote:
> Another point to mention is that UPSTREAM_* variables do not need to
> be set, most of the time. They are only required when upstream does
> something unusual, and defaults do not work.


Right, I understood that, but this is definitely worth outlining in the
documentation.
I'll share my updates.

Thanks again,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH v3 3/5] ref-manual: improve "devtool check-upgrade-status" details

[Michael Opdenacker]

Hi again Alex,

On 9/15/21 8:37 PM, Alexander Kanavin wrote:
> The functionality overlaps somewhat. 'devtool latest-version' takes a
> single recipe as an argument, but 'devtool check-upgrade-status' takes
> a list of recipes, or nothing (in which case a check for everything
> runs), and prints only the items where there is a new upstream version
> available in a table format. The former is intended when you want to
> see a specific status of something particular, the latter for an
> overall picture.

That's very useful to clarify indeed, so that the users clearly see the
differences between both commands.

In the next message in this thread, I'll share what I tried to write to
merge the extra explanations that both Quentin and you proposed. The
other commits in my branch just had minor fixes, so I don't want to
pollute the list with yet another full iteration.

Thanks again to the reviewers!
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com