Variable documentation overlap between Yocto and Bitbake

Variable documentation overlap between Yocto and Bitbake

[Michael Opdenacker]

Greetings,

Following recent discussions, I realize that the "gitsm" fetcher (Git
Submodule) is not documented yet in the description of SRC_URI.

However, shall I add this to both Yocto Project and Bitbake
documentation, which both describe SRC_URI? There's a lot of overlap
between the two, by the way, and this makes me uncomfortable.

Could there be a clean way to document all the common stuff in Bitbake's
documentation, and only OE core's specific options in Yocto docs? I'm
thinking about options such as "striplevel" that are implemented only in
OE core...

Thanks in advance for your advice.

Cheers,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] Variable documentation overlap between Yocto and Bitbake

[Quentin Schulz]

Hi Michael,

On Tue, Nov 02, 2021 at 08:06:21PM +0100, Michael Opdenacker wrote:
> Greetings,
> 
> Following recent discussions, I realize that the "gitsm" fetcher (Git
> Submodule) is not documented yet in the description of SRC_URI.
> 
> However, shall I add this to both Yocto Project and Bitbake
> documentation, which both describe SRC_URI? There's a lot of overlap
> between the two, by the way, and this makes me uncomfortable.
> 
> Could there be a clean way to document all the common stuff in Bitbake's
> documentation, and only OE core's specific options in Yocto docs? I'm
> thinking about options such as "striplevel" that are implemented only in
> OE core...
> 

What I'd like to see personally is removal of duplicates and have
entries in the glossary for terms defined in the bitbake docs. Since the
objects.inv from bitbake docs only does a mapping between references and
URL... I guess we can only add URLs to YP docs variables glossary.

For how to include common stuff, I guess we could give a link saying:
Common stuff defined in Bitbake <link>

Here are Yp/OE specific stuff for this thing:
<specific stuff>

I haven't really looked deeply into Sphinx so maybe there's something
possible but out of the blue, I'd just say we have links to refs to
other sphinx docs but nothing else.

Just my 2¢, not really offering solutions :/

Quentin

Re: [docs] Variable documentation overlap between Yocto and Bitbake

[Nicolas Dechesne]

[-- Attachment #1: Type: text/plain, Size: 3803 bytes --]

hey!

On Wed, Nov 3, 2021 at 11:46 AM Michael Opdenacker <
michael.opdenacker@bootlin.com> wrote:

> Hi Quentin,
>
> Thanks for your suggestions!
>
> On 11/3/21 10:35 AM, Quentin Schulz wrote:
> > Hi Michael,
> >
> > On Tue, Nov 02, 2021 at 08:06:21PM +0100, Michael Opdenacker wrote:
> >> Greetings,
> >>
> >> Following recent discussions, I realize that the "gitsm" fetcher (Git
> >> Submodule) is not documented yet in the description of SRC_URI.
> >>
> >> However, shall I add this to both Yocto Project and Bitbake
> >> documentation, which both describe SRC_URI? There's a lot of overlap
> >> between the two, by the way, and this makes me uncomfortable.
>

This is a good catch.


> >>
> >> Could there be a clean way to document all the common stuff in Bitbake's
> >> documentation, and only OE core's specific options in Yocto docs? I'm
> >> thinking about options such as "striplevel" that are implemented only in
> >> OE core...
> >>
> > What I'd like to see personally is removal of duplicates and have
> > entries in the glossary for terms defined in the bitbake docs. Since the
> > objects.inv from bitbake docs only does a mapping between references and
> > URL... I guess we can only add URLs to YP docs variables glossary.
>

Since we are at the beginning of the dev cycle (and the next release is an
LTS..) how about we discuss again how the bitbake manual is integrated into
yp-docs. I agree that having such duplicate is a bad user experience.

I know we want bitbake and yp-docs to be separate repo, but should we try
to combine them together when we create/publish the YP docs in a more
efficient manner. Without going as far as merging the 2 repos, perhaps we
could use a git submodule in yp-docs, and embed bitbake docs without using
the intersphinx extension, e.g. directly.

e.g. I've tested this, and it seems to work well:

--- a/documentation/index.rst
+++ b/documentation/index.rst
@@ -34,7 +34,7 @@ Welcome to the Yocto Project Documentation
    Application Development and the Extensible SDK (eSDK) <sdk-manual/index>
    Toaster Manual <toaster-manual/index>
    Test Environment Manual <test-manual/index>
-   bitbake
+   bitbake/doc/index

 .. toctree::
    :maxdepth: 1

after cloning bitbake repo in yp-docs/documentation.

I am well aware that bitbake doc can be standalone too, but we could still
publish bitbake docs independently like we do today. Btw, looking at the
new index (using the submodule) I can see 64 duplicate entries ;)


>
> > For how to include common stuff, I guess we could give a link saying:
> > Common stuff defined in Bitbake <link>
> >
> > Here are Yp/OE specific stuff for this thing:
> > <specific stuff>
> >
> > I haven't really looked deeply into Sphinx so maybe there's something
> > possible but out of the blue, I'd just say we have links to refs to
> > other sphinx docs but nothing else.
> >
> > Just my 2¢, not really offering solutions :/
>
>
> Definitely agree with the idea. We can make a reference to
> :term:`bitbake:SRC_URI` in the Yocto docs, and add the OE specific stuff
> there. I checked that this works, and that's not complicated.
> I'll propose a patch for SRC_URI soon.
>
> Thanks again,
> Michael.
>
> --
> Michael Opdenacker, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#2074):
> https://lists.yoctoproject.org/g/docs/message/2074
> Mute This Topic: https://lists.yoctoproject.org/mt/86774485/1279857
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [
> nicolas.dechesne@linaro.org]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 5439 bytes --]

Re: [docs] Variable documentation overlap between Yocto and Bitbake

[Nicolas Dechesne]

[-- Attachment #1: Type: text/plain, Size: 3548 bytes --]

On Wed, Nov 3, 2021 at 12:29 PM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Wed, 2021-11-03 at 12:24 +0100, Nicolas Dechesne wrote:
> > hey!
> >
> > On Wed, Nov 3, 2021 at 11:46 AM Michael Opdenacker
> > <michael.opdenacker@bootlin.com> wrote:
> > > Hi Quentin,
> > >
> > > Thanks for your suggestions!
> > >
> > > On 11/3/21 10:35 AM, Quentin Schulz wrote:
> > > > Hi Michael,
> > > >
> > > > On Tue, Nov 02, 2021 at 08:06:21PM +0100, Michael Opdenacker wrote:
> > > > > Greetings,
> > > > >
> > > > > Following recent discussions, I realize that the "gitsm" fetcher
> (Git
> > > > > Submodule) is not documented yet in the description of SRC_URI.
> > > > >
> > > > > However, shall I add this to both Yocto Project and Bitbake
> > > > > documentation, which both describe SRC_URI? There's a lot of
> overlap
> > > > > between the two, by the way, and this makes me uncomfortable.
> > >
> >
> >
> > This is a good catch.
> >
> > > > >
> > > > > Could there be a clean way to document all the common stuff in
> Bitbake's
> > > > > documentation, and only OE core's specific options in Yocto docs?
> I'm
> > > > > thinking about options such as "striplevel" that are implemented
> only in
> > > > > OE core...
> > > > >
> > > > What I'd like to see personally is removal of duplicates and have
> > > > entries in the glossary for terms defined in the bitbake docs. Since
> the
> > > > objects.inv from bitbake docs only does a mapping between references
> and
> > > > URL... I guess we can only add URLs to YP docs variables glossary.
> > >
> >
> >
> > Since we are at the beginning of the dev cycle (and the next release is
> an
> > LTS..) how about we discuss again how the bitbake manual is integrated
> into
> > yp-docs. I agree that having such duplicate is a bad user experience.
> >
> > I know we want bitbake and yp-docs to be separate repo, but should we
> try to
> > combine them together when we create/publish the YP docs in a more
> efficient
> > manner. Without going as far as merging the 2 repos, perhaps we could
> use a
> > git submodule in yp-docs, and embed bitbake docs without using the
> intersphinx
> > extension, e.g. directly.
> >
> > e.g. I've tested this, and it seems to work well:
> >
> > --- a/documentation/index.rst
> > +++ b/documentation/index.rst
> > @@ -34,7 +34,7 @@ Welcome to the Yocto Project Documentation
> >     Application Development and the Extensible SDK (eSDK)
> <sdk-manual/index>
> >     Toaster Manual <toaster-manual/index>
> >     Test Environment Manual <test-manual/index>
> > -   bitbake
> > +   bitbake/doc/index
> >
> >  .. toctree::
> >     :maxdepth: 1
> >
> > after cloning bitbake repo in yp-docs/documentation.
> >
> > I am well aware that bitbake doc can be standalone too, but we could
> still
> > publish bitbake docs independently like we do today. Btw, looking at the
> new
> > index (using the submodule) I can see 64 duplicate entries ;)
>
> How does the combined poky repo look where it contains both bitbake and
> yocto-
> docs?
>

Right. That's another idea. I think sphinx needs everything in the same top
level folder. let me check. But that could be fixed with a symlink in the
worst case. In fact Poky already combines oe-core, yp-docs and bitbake.. so
pushing the generation of docs.yp.org there might make sense..


>
> I understand why the submodule makes this bit easy, it just pushes a load
> of
> problems to other places (such as the CI, release tarball generation,
> combo-
> layer and probably other issues) :/.
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 4849 bytes --]

Re: [docs] Variable documentation overlap between Yocto and Bitbake

[Michael Opdenacker]

On 11/5/21 10:27 AM, Michael Opdenacker wrote:
>
> In the meantime, the current solution works well. I'll submit a patch
> for SRC_URI, eliminating the duplicates between the two manuals.


This is an interesting exercise because I'm finding features supported
by Bitbake but documented only in Yocto docs!

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com