All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH yocto-autobuilder-helper] run-docs-build: build from tags dynamically instead of static list
@ 2021-04-06 22:58 Michael Halstead
  2021-04-07  7:31 ` [yocto] " Quentin Schulz
  0 siblings, 1 reply; 4+ messages in thread
From: Michael Halstead @ 2021-04-06 22:58 UTC (permalink / raw)
  To: yocto; +Cc: Michael Halstead, nicolas.dechesne

All new releases are Sphinx ready so we exclude old tags and build for
all the rest.

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
---
 scripts/run-docs-build | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/scripts/run-docs-build b/scripts/run-docs-build
index 910f03d..13df34a 100755
--- a/scripts/run-docs-build
+++ b/scripts/run-docs-build
@@ -7,6 +7,7 @@ ypdocs=$2/documentation/
 bbdocs=$3/doc/
 docs_buildtools=/srv/autobuilder/autobuilder.yoctoproject.org/pub/buildtools/x86_64-buildtools-docs-nativesdk-standalone-3.2+snapshot-20201105.sh
 outputdir=$builddir/output
+excluded_tags="yocto-3.1.4 yocto-3.1.3 yocto-3.1.2 yocto-3.1.1 yocto-3.1 yocto-3.0.1 yocto-3.0 yocto-2.6.4 yocto-2.6.3 yocto-2.7.1 yocto-2.6.2 yocto-2.7 yocto-2.6.1 yocto-2.6 yocto-2.5.2 yocto-2.5.1 yocto-2.4.4 yocto-2.4.3 yocto-2.5 yocto-2.3.4 yocto-1.0.2 yocto-1.1.2 yocto-1.2.2 yocto-1.2.1 yocto-1.3 yocto-1.3.1 yocto-1.3.2 yocto-1.4.3 yocto-1.4.2 yocto-1.4.1 yocto-1.4 yocto-2.1.3 yocto-2.4.2 yocto-2.1.1 yocto-2.1.2 yocto-2.0.3 yocto-1.8.2 yocto-2.2.3 yocto-2.4.1 yocto-2.3.3 yocto-2.3.2 yocto-2.4 yocto-2.2.2 yocto-2.3.1 yocto-2.3 yocto-2.2.1 yocto-2.0.2 yocto-2.2 yocto-2.1 yocto-2.0.1 yocto-2.0 yocto-1.8.1 yocto-1.7.3 yocto-1.6.3 yocto-1.7.2 yocto-1.8 yocto-1.5.1"
 
 
 cd $builddir
@@ -77,13 +78,18 @@ for branch in dunfell gatesgarth hardknott; do
 done
 
 # Yocto Project releases/tags
-for tag in 3.1.5 3.1.6 3.2 3.2.1 3.2.2 3.2.3; do
+cd $ypdocs
+for tag in $(git tag -l  |grep 'yocto-' |sort); do
+    if [[ $excluded_tags =~ $tag ]]; then
+        continue
+    fi
     cd $ypdocs
-    git checkout yocto-$tag
+    git checkout $tag
     make clean
     make publish
-    mkdir $outputdir/$tag
-    cp -r ./_build/final/* $outputdir/$tag
+    version=$(echo $tag | cut -c7-)
+    mkdir $outputdir/$version
+    cp -r ./_build/final/* $outputdir/$version
 done
 
 # Update switchers.js with the copy from master ypdocs
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [yocto] [PATCH yocto-autobuilder-helper] run-docs-build: build from tags dynamically instead of static list
  2021-04-06 22:58 [PATCH yocto-autobuilder-helper] run-docs-build: build from tags dynamically instead of static list Michael Halstead
@ 2021-04-07  7:31 ` Quentin Schulz
  2021-04-07  8:16   ` Nicolas Dechesne
  0 siblings, 1 reply; 4+ messages in thread
From: Quentin Schulz @ 2021-04-07  7:31 UTC (permalink / raw)
  To: yocto, Michael Halstead; +Cc: nicolas.dechesne

Hi Michael,

On April 6, 2021 10:58:20 PM UTC, Michael Halstead <mhalstead@linuxfoundation.org> wrote:
>All new releases are Sphinx ready so we exclude old tags and build for
>all the rest.
>
>Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
>---
> scripts/run-docs-build | 14 ++++++++++----
> 1 file changed, 10 insertions(+), 4 deletions(-)
>
>diff --git a/scripts/run-docs-build b/scripts/run-docs-build
>index 910f03d..13df34a 100755
>--- a/scripts/run-docs-build
>+++ b/scripts/run-docs-build
>@@ -7,6 +7,7 @@ ypdocs=$2/documentation/
> bbdocs=$3/doc/
> docs_buildtools=/srv/autobuilder/autobuilder.yoctoproject.org/pub/buildtools/x86_64-buildtools-docs-nativesdk-standalone-3.2+snapshot-20201105.sh
> outputdir=$builddir/output
>+excluded_tags="yocto-3.1.4 yocto-3.1.3 yocto-3.1.2 yocto-3.1.1 yocto-3.1 yocto-3.0.1 yocto-3.0 yocto-2.6.4 yocto-2.6.3 yocto-2.7.1 yocto-2.6.2 yocto-2.7 yocto-2.6.1 yocto-2.6 yocto-2.5.2 yocto-2.5.1 yocto-2.4.4 yocto-2.4.3 yocto-2.5 yocto-2.3.4 yocto-1.0.2 yocto-1.1.2 yocto-1.2.2 yocto-1.2.1 yocto-1.3 yocto-1.3.1 yocto-1.3.2 yocto-1.4.3 yocto-1.4.2 yocto-1.4.1 yocto-1.4 yocto-2.1.3 yocto-2.4.2 yocto-2.1.1 yocto-2.1.2 yocto-2.0.3 yocto-1.8.2 yocto-2.2.3 yocto-2.4.1 yocto-2.3.3 yocto-2.3.2 yocto-2.4 yocto-2.2.2 yocto-2.3.1 yocto-2.3 yocto-2.2.1 yocto-2.0.2 yocto-2.2 yocto-2.1 yocto-2.0.1 yocto-2.0 yocto-1.8.1 yocto-1.7.3 yocto-1.6.3 yocto-1.7.2 yocto-1.8 yocto-1.5.1"
> 
> 
> cd $builddir
>@@ -77,13 +78,18 @@ for branch in dunfell gatesgarth hardknott; do
> done
> 
> # Yocto Project releases/tags
>-for tag in 3.1.5 3.1.6 3.2 3.2.1 3.2.2 3.2.3; do
>+cd $ypdocs
>+for tag in $(git tag -l  |grep 'yocto-' |sort); do

IIUC the man page,
git tag --list 'yocto-*' | sort
would be doing the same thing as the one command with grep above.
Discovered it recently so just wanted to share.

I guess this is something we can also do for bitbake Sphinx documentation?

Removed the git context inadvertently but, is =~ some bash built-in? I don't know what's the shebang on top but maybe we want to force it to bash since I'm not sure it's POSIX "compliant" anymore?

Reviewed-by: Quentin Schulz <foss@0leil.net>

Thanks,
Quentin

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [yocto] [PATCH yocto-autobuilder-helper] run-docs-build: build from tags dynamically instead of static list
  2021-04-07  7:31 ` [yocto] " Quentin Schulz
@ 2021-04-07  8:16   ` Nicolas Dechesne
  2021-04-08  5:52     ` Nicolas Dechesne
  0 siblings, 1 reply; 4+ messages in thread
From: Nicolas Dechesne @ 2021-04-07  8:16 UTC (permalink / raw)
  To: Quentin Schulz; +Cc: Yocto-mailing-list, Michael Halstead

[-- Attachment #1: Type: text/plain, Size: 3157 bytes --]

hey!

On Wed, Apr 7, 2021 at 9:31 AM Quentin Schulz <foss@0leil.net> wrote:

> Hi Michael,
>
> On April 6, 2021 10:58:20 PM UTC, Michael Halstead <
> mhalstead@linuxfoundation.org> wrote:
> >All new releases are Sphinx ready so we exclude old tags and build for
> >all the rest.
>

Thanks for starting this!


> >
> >Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
> >---
> > scripts/run-docs-build | 14 ++++++++++----
> > 1 file changed, 10 insertions(+), 4 deletions(-)
> >
> >diff --git a/scripts/run-docs-build b/scripts/run-docs-build
> >index 910f03d..13df34a 100755
> >--- a/scripts/run-docs-build
> >+++ b/scripts/run-docs-build
> >@@ -7,6 +7,7 @@ ypdocs=$2/documentation/
> > bbdocs=$3/doc/
> > docs_buildtools=/srv/autobuilder/
> autobuilder.yoctoproject.org/pub/buildtools/x86_64-buildtools-docs-nativesdk-standalone-3.2+snapshot-20201105.sh
> > outputdir=$builddir/output
> >+excluded_tags="yocto-3.1.4 yocto-3.1.3 yocto-3.1.2 yocto-3.1.1 yocto-3.1
> yocto-3.0.1 yocto-3.0 yocto-2.6.4 yocto-2.6.3 yocto-2.7.1 yocto-2.6.2
> yocto-2.7 yocto-2.6.1 yocto-2.6 yocto-2.5.2 yocto-2.5.1 yocto-2.4.4
> yocto-2.4.3 yocto-2.5 yocto-2.3.4 yocto-1.0.2 yocto-1.1.2 yocto-1.2.2
> yocto-1.2.1 yocto-1.3 yocto-1.3.1 yocto-1.3.2 yocto-1.4.3 yocto-1.4.2
> yocto-1.4.1 yocto-1.4 yocto-2.1.3 yocto-2.4.2 yocto-2.1.1 yocto-2.1.2
> yocto-2.0.3 yocto-1.8.2 yocto-2.2.3 yocto-2.4.1 yocto-2.3.3 yocto-2.3.2
> yocto-2.4 yocto-2.2.2 yocto-2.3.1 yocto-2.3 yocto-2.2.1 yocto-2.0.2
> yocto-2.2 yocto-2.1 yocto-2.0.1 yocto-2.0 yocto-1.8.1 yocto-1.7.3
> yocto-1.6.3 yocto-1.7.2 yocto-1.8 yocto-1.5.1"
> >
> >
> > cd $builddir
> >@@ -77,13 +78,18 @@ for branch in dunfell gatesgarth hardknott; do
> > done
> >
> > # Yocto Project releases/tags
> >-for tag in 3.1.5 3.1.6 3.2 3.2.1 3.2.2 3.2.3; do
> >+cd $ypdocs
> >+for tag in $(git tag -l  |grep 'yocto-' |sort); do
>
> IIUC the man page,
> git tag --list 'yocto-*' | sort
>

sort -V is even better since it does "natural sort of (version) numbers
within text", let's get ready for 3.10 ;)

And using -V, how about something along these lines:

v_sphinx='yocto-3.1.5'
for v in $(git tag --list 'yocto-*'); do
    first=$(printf '%s\n%s' $v $v_sphinx | sort -V | head -n1)
    if [ "$first" = "$v_sphinx" ]; then
        echo "Yocto $v uses Sphinx!"
    fi
done

and it outputs the following when I run it locally:

Yocto yocto-3.1.5 uses Sphinx!
Yocto yocto-3.1.6 uses Sphinx!
Yocto yocto-3.2 uses Sphinx!
Yocto yocto-3.2.1 uses Sphinx!
Yocto yocto-3.2.2 uses Sphinx!
Yocto yocto-3.2.3 uses Sphinx!



would be doing the same thing as the one command with grep above.
> Discovered it recently so just wanted to share.
>
> I guess this is something we can also do for bitbake Sphinx documentation?
>

yes.


>
> Removed the git context inadvertently but, is =~ some bash built-in? I
> don't know what's the shebang on top but maybe we want to force it to bash
> since I'm not sure it's POSIX "compliant" anymore?


It is bash already.


>
> Reviewed-by: Quentin Schulz <foss@0leil.net>
>
> Thanks,
> Quentin
>

[-- Attachment #2: Type: text/html, Size: 4882 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [yocto] [PATCH yocto-autobuilder-helper] run-docs-build: build from tags dynamically instead of static list
  2021-04-07  8:16   ` Nicolas Dechesne
@ 2021-04-08  5:52     ` Nicolas Dechesne
  0 siblings, 0 replies; 4+ messages in thread
From: Nicolas Dechesne @ 2021-04-08  5:52 UTC (permalink / raw)
  To: Quentin Schulz; +Cc: Yocto-mailing-list, Michael Halstead

[-- Attachment #1: Type: text/plain, Size: 5179 bytes --]

On Wed, Apr 7, 2021 at 10:16 AM Nicolas Dechesne <
nicolas.dechesne@linaro.org> wrote:

> hey!
>
> On Wed, Apr 7, 2021 at 9:31 AM Quentin Schulz <foss@0leil.net> wrote:
>
>> Hi Michael,
>>
>> On April 6, 2021 10:58:20 PM UTC, Michael Halstead <
>> mhalstead@linuxfoundation.org> wrote:
>> >All new releases are Sphinx ready so we exclude old tags and build for
>> >all the rest.
>>
>
> Thanks for starting this!
>
>
>> >
>> >Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
>> >---
>> > scripts/run-docs-build | 14 ++++++++++----
>> > 1 file changed, 10 insertions(+), 4 deletions(-)
>> >
>> >diff --git a/scripts/run-docs-build b/scripts/run-docs-build
>> >index 910f03d..13df34a 100755
>> >--- a/scripts/run-docs-build
>> >+++ b/scripts/run-docs-build
>> >@@ -7,6 +7,7 @@ ypdocs=$2/documentation/
>> > bbdocs=$3/doc/
>> > docs_buildtools=/srv/autobuilder/
>> autobuilder.yoctoproject.org/pub/buildtools/x86_64-buildtools-docs-nativesdk-standalone-3.2+snapshot-20201105.sh
>> > outputdir=$builddir/output
>> >+excluded_tags="yocto-3.1.4 yocto-3.1.3 yocto-3.1.2 yocto-3.1.1
>> yocto-3.1 yocto-3.0.1 yocto-3.0 yocto-2.6.4 yocto-2.6.3 yocto-2.7.1
>> yocto-2.6.2 yocto-2.7 yocto-2.6.1 yocto-2.6 yocto-2.5.2 yocto-2.5.1
>> yocto-2.4.4 yocto-2.4.3 yocto-2.5 yocto-2.3.4 yocto-1.0.2 yocto-1.1.2
>> yocto-1.2.2 yocto-1.2.1 yocto-1.3 yocto-1.3.1 yocto-1.3.2 yocto-1.4.3
>> yocto-1.4.2 yocto-1.4.1 yocto-1.4 yocto-2.1.3 yocto-2.4.2 yocto-2.1.1
>> yocto-2.1.2 yocto-2.0.3 yocto-1.8.2 yocto-2.2.3 yocto-2.4.1 yocto-2.3.3
>> yocto-2.3.2 yocto-2.4 yocto-2.2.2 yocto-2.3.1 yocto-2.3 yocto-2.2.1
>> yocto-2.0.2 yocto-2.2 yocto-2.1 yocto-2.0.1 yocto-2.0 yocto-1.8.1
>> yocto-1.7.3 yocto-1.6.3 yocto-1.7.2 yocto-1.8 yocto-1.5.1"
>> >
>> >
>> > cd $builddir
>> >@@ -77,13 +78,18 @@ for branch in dunfell gatesgarth hardknott; do
>> > done
>> >
>> > # Yocto Project releases/tags
>> >-for tag in 3.1.5 3.1.6 3.2 3.2.1 3.2.2 3.2.3; do
>> >+cd $ypdocs
>> >+for tag in $(git tag -l  |grep 'yocto-' |sort); do
>>
>> IIUC the man page,
>> git tag --list 'yocto-*' | sort
>>
>
> sort -V is even better since it does "natural sort of (version) numbers
> within text", let's get ready for 3.10 ;)
>
> And using -V, how about something along these lines:
>
> v_sphinx='yocto-3.1.5'
> for v in $(git tag --list 'yocto-*'); do
>     first=$(printf '%s\n%s' $v $v_sphinx | sort -V | head -n1)
>     if [ "$first" = "$v_sphinx" ]; then
>         echo "Yocto $v uses Sphinx!"
>     fi
> done
>
> and it outputs the following when I run it locally:
>
> Yocto yocto-3.1.5 uses Sphinx!
> Yocto yocto-3.1.6 uses Sphinx!
> Yocto yocto-3.2 uses Sphinx!
> Yocto yocto-3.2.1 uses Sphinx!
> Yocto yocto-3.2.2 uses Sphinx!
> Yocto yocto-3.2.3 uses Sphinx!
>
>
>
> would be doing the same thing as the one command with grep above.
>> Discovered it recently so just wanted to share.
>>
>> I guess this is something we can also do for bitbake Sphinx documentation?
>>
>
> yes.
>

Well, in fact no , we shouldn't, is a short answer ;)

Here is the long answer..

For bitbake and YP, we currently publish: master, master-next and 'stable'
branches (e.g. 1.46, 1.48, 1.50 for bitbake dunfell, gatesgarth, hardknott
for YP). Then we also publish YP releases (and point releases), e.g. 3.1.5
and beyond, but  we don't publish anything for releases/point release of
bitbake, At least just yet. And it's a 'bug'. Since the YP docs includes
links (with intersphinx)  to bitbake docs, then when we look at a master,
master-next or any branches on docs.yp.org, then the links are correct. for
example if you look at

https://docs.yoctoproject.org/dunfell/what-i-wish-id-known.html
vs
https://docs.yoctoproject.org/gatesgarth/what-i-wish-id-known.html

this page has links to bitbake and it points respectively to:
https://docs.yoctoproject.org/bitbake/1.46/bitbake-user-manual/bitbake-user-manual-intro.html#generating-dependency-graphs
and
https://docs.yoctoproject.org/bitbake/1.48/bitbake-user-manual/bitbake-user-manual-intro.html#generating-dependency-graphs

Which is correct.

[Side note: I've noticed that the hardknott page is wrong, since
https://docs.yoctoproject.org/hardknott/what-i-wish-id-known.html
and the following link

>
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#generating-dependency-graphs
I will send a patch for that]

Now if we look at a YP release, such as
https://docs.yoctoproject.org/3.1.6/what-i-wish-id-known.html
it has this link:
https://docs.yoctoproject.org/bitbake/1.46/bitbake-user-manual/bitbake-user-manual-intro.html#generating-dependency-graphs

which is not correct, since it links to a bitbake 'branch' docs build, and
not a bitbake 'release' docs build.



>
>>
>> Removed the git context inadvertently but, is =~ some bash built-in? I
>> don't know what's the shebang on top but maybe we want to force it to bash
>> since I'm not sure it's POSIX "compliant" anymore?
>
>
> It is bash already.
>
>
>>
>> Reviewed-by: Quentin Schulz <foss@0leil.net>
>>
>> Thanks,
>> Quentin
>>
>

[-- Attachment #2: Type: text/html, Size: 8818 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-04-08  5:52 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-06 22:58 [PATCH yocto-autobuilder-helper] run-docs-build: build from tags dynamically instead of static list Michael Halstead
2021-04-07  7:31 ` [yocto] " Quentin Schulz
2021-04-07  8:16   ` Nicolas Dechesne
2021-04-08  5:52     ` Nicolas Dechesne

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.