[PATCH] auto-generate releases.rst

[PATCH] auto-generate releases.rst

[Quentin Schulz]

From: Quentin Schulz <quentin.schulz@theobroma-systems.com>

In order to maintain one less file, let's auto-generate the releases.rst
file which contains a link for each release ever released.

This gets auto-generated by checking the tags available in this git repo
and adding a link for each that exists.

A few tags are notoriously missing from this git repo and they are
manually listed then, until the tags are pushed for the appropriate
commit.

Cc: Quentin Schulz <foss@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
---
 documentation/.gitignore      |   1 +
 documentation/releases.rst    | 251 ----------------------------------
 documentation/set_versions.py |  75 ++++++++++
 3 files changed, 76 insertions(+), 251 deletions(-)
 delete mode 100644 documentation/releases.rst

diff --git a/documentation/.gitignore b/documentation/.gitignore
index 096b97ec2..4e077d03f 100644
--- a/documentation/.gitignore
+++ b/documentation/.gitignore
@@ -2,6 +2,7 @@ _build/
 Pipfile.lock
 poky.yaml
 sphinx-static/switchers.js
+releases.rst
 .vscode/
 */svg/*.png
 */svg/*.pdf
diff --git a/documentation/releases.rst b/documentation/releases.rst
deleted file mode 100644
index b2b448615..000000000
--- a/documentation/releases.rst
+++ /dev/null
@@ -1,251 +0,0 @@
-.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
-
-..
-   NOTE FOR RELEASE MAINTAINERS:
-   This file only needs updating in the development release ("master" branch)
-   When documentation for stable releases is built,
-   the latest version from "master" is used
-   by https://git.yoctoproject.org/yocto-autobuilder-helper/tree/scripts/run-docs-build
-
-===========================
- Supported Release Manuals
-===========================
-
-******************************
-Release Series 4.0 (kirkstone)
-******************************
-
-- :yocto_docs:`4.0 Documentation </4.0>`
-- :yocto_docs:`4.0.1 Documentation </4.0.1>`
-- :yocto_docs:`4.0.2 Documentation </4.0.2>`
-
-****************************
-Release Series 3.1 (dunfell)
-****************************
-
-- :yocto_docs:`3.1 Documentation </3.1>`
-- :yocto_docs:`3.1.1 Documentation </3.1.1>`
-- :yocto_docs:`3.1.2 Documentation </3.1.2>`
-- :yocto_docs:`3.1.3 Documentation </3.1.3>`
-- :yocto_docs:`3.1.4 Documentation </3.1.4>`
-- :yocto_docs:`3.1.5 Documentation </3.1.5>`
-- :yocto_docs:`3.1.6 Documentation </3.1.6>`
-- :yocto_docs:`3.1.7 Documentation </3.1.7>`
-- :yocto_docs:`3.1.8 Documentation </3.1.8>`
-- :yocto_docs:`3.1.9 Documentation </3.1.9>`
-- :yocto_docs:`3.1.10 Documentation </3.1.10>`
-- :yocto_docs:`3.1.11 Documentation </3.1.11>`
-- :yocto_docs:`3.1.12 Documentation </3.1.12>`
-- :yocto_docs:`3.1.13 Documentation </3.1.13>`
-- :yocto_docs:`3.1.14 Documentation </3.1.14>`
-- :yocto_docs:`3.1.15 Documentation </3.1.15>`
-- :yocto_docs:`3.1.16 Documentation </3.1.16>`
-- :yocto_docs:`3.1.17 Documentation </3.1.17>`
-
-==========================
- Outdated Release Manuals
-==========================
-
-******************************
-Release Series 3.4 (honister)
-******************************
-
-- :yocto_docs:`3.4 Documentation </3.4>`
-- :yocto_docs:`3.4.1 Documentation </3.4.1>`
-- :yocto_docs:`3.4.2 Documentation </3.4.2>`
-- :yocto_docs:`3.4.3 Documentation </3.4.3>`
-- :yocto_docs:`3.4.4 Documentation </3.4.4>`
-
-******************************
-Release Series 3.3 (hardknott)
-******************************
-
-- :yocto_docs:`3.3 Documentation </3.3>`
-- :yocto_docs:`3.3.1 Documentation </3.3.1>`
-- :yocto_docs:`3.3.2 Documentation </3.3.2>`
-- :yocto_docs:`3.3.3 Documentation </3.3.3>`
-- :yocto_docs:`3.3.4 Documentation </3.3.4>`
-- :yocto_docs:`3.3.5 Documentation </3.3.5>`
-- :yocto_docs:`3.3.6 Documentation </3.3.6>`
-
-*******************************
-Release Series 3.2 (gatesgarth)
-*******************************
-
-- :yocto_docs:`3.2 Documentation </3.2>`
-- :yocto_docs:`3.2.1 Documentation </3.2.1>`
-- :yocto_docs:`3.2.2 Documentation </3.2.2>`
-- :yocto_docs:`3.2.3 Documentation </3.2.3>`
-- :yocto_docs:`3.2.4 Documentation </3.2.4>`
-
-*************************
-Release Series 3.0 (zeus)
-*************************
-
-- :yocto_docs:`3.0 Documentation </3.0>`
-- :yocto_docs:`3.0.1 Documentation </3.0.1>`
-- :yocto_docs:`3.0.2 Documentation </3.0.2>`
-- :yocto_docs:`3.0.3 Documentation </3.0.3>`
-- :yocto_docs:`3.0.4 Documentation </3.0.4>`
-
-****************************
-Release Series 2.7 (warrior)
-****************************
-
-- :yocto_docs:`2.7 Documentation </2.7>`
-- :yocto_docs:`2.7.1 Documentation </2.7.1>`
-- :yocto_docs:`2.7.2 Documentation </2.7.2>`
-- :yocto_docs:`2.7.3 Documentation </2.7.3>`
-- :yocto_docs:`2.7.4 Documentation </2.7.4>`
-
-*************************
-Release Series 2.6 (thud)
-*************************
-
-- :yocto_docs:`2.6 Documentation </2.6>`
-- :yocto_docs:`2.6.1 Documentation </2.6.1>`
-- :yocto_docs:`2.6.2 Documentation </2.6.2>`
-- :yocto_docs:`2.6.3 Documentation </2.6.3>`
-- :yocto_docs:`2.6.4 Documentation </2.6.4>`
-
-*************************
-Release Series 2.5 (sumo)
-*************************
-
-- :yocto_docs:`2.5 Documentation </2.5>`
-- :yocto_docs:`2.5.1 Documentation </2.5.1>`
-- :yocto_docs:`2.5.2 Documentation </2.5.2>`
-- :yocto_docs:`2.5.3 Documentation </2.5.3>`
-
-**************************
-Release Series 2.4 (rocko)
-**************************
-
-- :yocto_docs:`2.4 Documentation </2.4>`
-- :yocto_docs:`2.4.1 Documentation </2.4.1>`
-- :yocto_docs:`2.4.2 Documentation </2.4.2>`
-- :yocto_docs:`2.4.3 Documentation </2.4.3>`
-- :yocto_docs:`2.4.4 Documentation </2.4.4>`
-
-*************************
-Release Series 2.3 (pyro)
-*************************
-
-- :yocto_docs:`2.3 Documentation </2.3>`
-- :yocto_docs:`2.3.1 Documentation </2.3.1>`
-- :yocto_docs:`2.3.2 Documentation </2.3.2>`
-- :yocto_docs:`2.3.3 Documentation </2.3.3>`
-- :yocto_docs:`2.3.4 Documentation </2.3.4>`
-
-**************************
-Release Series 2.2 (morty)
-**************************
-
-- :yocto_docs:`2.2 Documentation </2.2>`
-- :yocto_docs:`2.2.1 Documentation </2.2.1>`
-- :yocto_docs:`2.2.2 Documentation </2.2.2>`
-- :yocto_docs:`2.2.3 Documentation </2.2.3>`
-
-****************************
-Release Series 2.1 (krogoth)
-****************************
-
-- :yocto_docs:`2.1 Documentation </2.1>`
-- :yocto_docs:`2.1.1 Documentation </2.1.1>`
-- :yocto_docs:`2.1.2 Documentation </2.1.2>`
-- :yocto_docs:`2.1.3 Documentation </2.1.3>`
-
-***************************
-Release Series 2.0 (jethro)
-***************************
-
-- :yocto_docs:`1.9 Documentation </1.9>`
-- :yocto_docs:`2.0 Documentation </2.0>`
-- :yocto_docs:`2.0.1 Documentation </2.0.1>`
-- :yocto_docs:`2.0.2 Documentation </2.0.2>`
-- :yocto_docs:`2.0.3 Documentation </2.0.3>`
-
-*************************
-Release Series 1.8 (fido)
-*************************
-
-- :yocto_docs:`1.8 Documentation </1.8>`
-- :yocto_docs:`1.8.1 Documentation </1.8.1>`
-- :yocto_docs:`1.8.2 Documentation </1.8.2>`
-
-**************************
-Release Series 1.7 (dizzy)
-**************************
-
-- :yocto_docs:`1.7 Documentation </1.7>`
-- :yocto_docs:`1.7.1 Documentation </1.7.1>`
-- :yocto_docs:`1.7.2 Documentation </1.7.2>`
-- :yocto_docs:`1.7.3 Documentation </1.7.3>`
-
-**************************
-Release Series 1.6 (daisy)
-**************************
-
-- :yocto_docs:`1.6 Documentation </1.6>`
-- :yocto_docs:`1.6.1 Documentation </1.6.1>`
-- :yocto_docs:`1.6.2 Documentation </1.6.2>`
-- :yocto_docs:`1.6.3 Documentation </1.6.3>`
-
-*************************
-Release Series 1.5 (dora)
-*************************
-
-- :yocto_docs:`1.5 Documentation </1.5>`
-- :yocto_docs:`1.5.1 Documentation </1.5.1>`
-- :yocto_docs:`1.5.2 Documentation </1.5.2>`
-- :yocto_docs:`1.5.3 Documentation </1.5.3>`
-- :yocto_docs:`1.5.4 Documentation </1.5.4>`
-
-**************************
-Release Series 1.4 (dylan)
-**************************
-
-- :yocto_docs:`1.4 Documentation </1.4>`
-- :yocto_docs:`1.4.1 Documentation </1.4.1>`
-- :yocto_docs:`1.4.2 Documentation </1.4.2>`
-- :yocto_docs:`1.4.3 Documentation </1.4.3>`
-- :yocto_docs:`1.4.4 Documentation </1.4.4>`
-- :yocto_docs:`1.4.5 Documentation </1.4.5>`
-
-**************************
-Release Series 1.3 (danny)
-**************************
-
-- :yocto_docs:`1.3 Documentation </1.3>`
-- :yocto_docs:`1.3.1 Documentation </1.3.1>`
-- :yocto_docs:`1.3.2 Documentation </1.3.2>`
-
-***************************
-Release Series 1.2 (denzil)
-***************************
-
-- :yocto_docs:`1.2 Documentation </1.2>`
-- :yocto_docs:`1.2.1 Documentation </1.2.1>`
-- :yocto_docs:`1.2.2 Documentation </1.2.2>`
-
-***************************
-Release Series 1.1 (edison)
-***************************
-
-- :yocto_docs:`1.1 Documentation </1.1>`
-- :yocto_docs:`1.1.1 Documentation </1.1.1>`
-- :yocto_docs:`1.1.2 Documentation </1.1.2>`
-
-****************************
-Release Series 1.0 (bernard)
-****************************
-
-- :yocto_docs:`1.0 Documentation </1.0>`
-- :yocto_docs:`1.0.1 Documentation </1.0.1>`
-- :yocto_docs:`1.0.2 Documentation </1.0.2>`
-
-****************************
-Release Series 0.9 (laverne)
-****************************
-
-- :yocto_docs:`0.9 Documentation </0.9>`
diff --git a/documentation/set_versions.py b/documentation/set_versions.py
index a7ceb3455..803d3e4b4 100755
--- a/documentation/set_versions.py
+++ b/documentation/set_versions.py
@@ -14,6 +14,7 @@ import collections
 import sys
 import os
 import itertools
+import re
 
 ourversion = None
 if len(sys.argv) == 2:
@@ -231,3 +232,77 @@ with open("sphinx-static/switchers.js.in", "r") as r, open("sphinx-static/switch
 
 print("switchers.js generated from switchers.js.in")
 
+# generate releases.rst
+
+# list missing tags in yocto-docs
+missing_tags = [
+        'yocto-0.9',
+        'yocto-1.0', 'yocto-1.0.1',
+        'yocto-1.1', 'yocto-1.1.1',
+        'yocto-1.2',
+        'yocto-1.4.4', 'yocto-1.4.5',
+        'yocto-1.5', 'yocto-1.5.2', 'yocto-1.5.3', 'yocto-1.5.4',
+        'yocto-1.6', 'yocto-1.6.1', 'yocto-1.6.2',
+        'yocto-1.7', 'yocto-1.7.1',
+        'yocto-1.9',
+        'yocto-2.5.3',
+        'yocto-3.1', 'yocto-3.1.1', 'yocto-3.1.2', 'yocto-3.1.3',
+        ]
+
+semver = re.compile(r'yocto-(\d+)\.(\d+)(?:\.)?(\d*)')
+
+# git is able to properly order semver versions but not python
+# instead of adding a dependency on semver module, let's convert the version
+# into a decimal number, e.g. 11.23.1 will be 112301 and 1.5 will be 010500 so
+# it can be used as a key for the sorting algorithm.
+# This can be removed once all the old tags are re-created.
+def tag_to_semver_like(v):
+    v_semver = semver.search(v)
+    v_maj, v_min, v_patch = v_semver.groups('0')
+    return int("{:0>2}{:0>2}{:0>2}".format(v_maj, v_min, v_patch), 10)
+
+yocto_tags = subprocess.run(["git", "tag", "--list", "--sort=version:refname", "yocto-*"], capture_output=True, text=True).stdout
+yocto_tags = sorted(yocto_tags.split() + missing_tags, key=tag_to_semver_like)
+tags = [tag[6:] for tag in yocto_tags]
+
+with open('releases.rst', 'w') as f:
+    f.write('===========================\n')
+    f.write(' Supported Release Manuals \n')
+    f.write('===========================\n')
+    f.write('\n')
+
+    for activerelease in activereleases:
+        title = "Release Series %s (%s)" % (release_series[activerelease], activerelease)
+        f.write('*' * len(title) + '\n')
+        f.write(title + '\n')
+        f.write('*' * len(title) + '\n')
+        f.write('\n')
+
+        for tag in tags:
+            if tag == release_series[activerelease] or tag.startswith('%s.' % release_series[activerelease]):
+                f.write('- :yocto_docs:`%s Documentation </%s>`\n' % (tag, tag))
+        f.write('\n')
+
+    f.write('==========================\n')
+    f.write(' Outdated Release Manuals \n')
+    f.write('==========================\n')
+    f.write('\n')
+
+    for series in release_series:
+        if series == devbranch or series in activereleases:
+            continue
+
+        if series == "jethro-pre":
+            continue
+
+        title = "Release Series %s (%s)" % (release_series[series], series)
+        f.write('*' * len(title) + '\n')
+        f.write(title + '\n')
+        f.write('*' * len(title) + '\n')
+        f.write('\n')
+        if series == "jethro":
+            f.write('- :yocto_docs:`1.9 Documentation </1.9>`\n')
+        for tag in tags:
+            if tag == release_series[series] or tag.startswith('%s.' % release_series[series]):
+                f.write('- :yocto_docs:`%s Documentation </%s>`\n' % (tag, tag))
+        f.write('\n')
-- 
2.36.1

[PATCH yocto-autobuilder-helper 1/2] scripts: run-docs-build: use latest version of set_versions.py script

[Quentin Schulz]

From: Quentin Schulz <quentin.schulz@theobroma-systems.com>

`git checkout <pathspec>` without <tree-ish> only undo the changes made
to pathspec compared to HEAD. It is not guaranteed that HEAD is on
latest commit of master branch from upstream when running this script,
therefore let's add the <tree-ish> argument to specify which branch to
use to make sure set_versions.py is the latest, otherwise getlatest
might return the wrong information.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
---
 scripts/run-docs-build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 489fa78..6a71d54 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -97,7 +97,7 @@ cd $ypdocs
 first_sphinx_commit=01dd5af7954e24552aca022917669b27bb0541ed
 first_dunfell_sphinx_commit=c25fe058b88b893b0d146f3ed27320b47cdec236
 
-git checkout set_versions.py
+git checkout origin/master set_versions.py
 #latest_tag=$(git tag --contains "$first_sphinx_commit" --contains "$first_dunfell_sphinx_commit" --sort="version:refname" 'yocto-*' | tail -1 | sed 's/yocto-//')
 latest_tag=$(./set_versions.py getlatest)
 git reset --hard
-- 
2.36.1

[PATCH yocto-autobuilder-helper 2/2] scripts/run-docs-build: do not checkout releases.rst from master anymore

[Quentin Schulz]

Master branch of the yocto-docs now autogenerates the releases.rst file
and the file therefore does not exist anymore.

Signed-off-by: Quentin Schulz <foss+yocto@0leil.net>
---
 scripts/run-docs-build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 6a71d54..c6b3965 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -122,7 +122,7 @@ for branch in dunfell $(git branch --remote --contains "$first_sphinx_commit" --
         git am "${scriptdir}/docs-build-patches/${branch}/"000*
     fi
 
-    git checkout origin/master releases.rst sphinx-static/switchers.js.in set_versions.py
+    git checkout origin/master sphinx-static/switchers.js.in set_versions.py
     if [ -e poky.yaml ]; then
         cp poky.yaml poky.yaml.in
         case $branch in
-- 
2.36.1

Re: [docs] [PATCH] auto-generate releases.rst

[Michael Opdenacker]

Hi Quentin

On 7/19/22 10:12, Quentin Schulz wrote:
> From: Quentin Schulz <quentin.schulz@theobroma-systems.com>
>
> In order to maintain one less file, let's auto-generate the releases.rst
> file which contains a link for each release ever released.
>
> This gets auto-generated by checking the tags available in this git repo
> and adding a link for each that exists.
>
> A few tags are notoriously missing from this git repo and they are
> manually listed then, until the tags are pushed for the appropriate
> commit.

That's brilliant, thanks a lot!
Just a few comments below...

> ")
>   
> +# generate releases.rst


Don't forget to add your name to the authors at the beginning of the 
script and what the script does now.

Thanks again!
Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH yocto-autobuilder-helper 2/2] scripts/run-docs-build: do not checkout releases.rst from master anymore

[Michael Opdenacker]

On 7/19/22 10:12, Quentin Schulz wrote:
> Master branch of the yocto-docs now autogenerates the releases.rst file
> and the file therefore does not exist anymore.
>
> Signed-off-by: Quentin Schulz <foss+yocto@0leil.net>


Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Indeed, it should be applied right after the latest pull request from 
master-next is applied.

Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com