[Qemu-devel] building rst docs with sphinx

[Qemu-devel] building rst docs with sphinx

[Peter Maydell]

I had another look this afternoon at building our rST docs
with sphinx-build. In particular, we currently have some
docs in rst format, but we're not building them into HTML
or shipping them. (Predictably, this means a few errors and
warnings have crept in...)

I had a play about with adding some makefile runes, but
I'm not sure entirely what I should be aiming for.

(1) configure: My thought is that we should just make
sphinx-build a requirement for the existing --enable-docs
switch (as texinfo and pod2man are currently). The
disadvantage is that we won't support a "build the half
of the docs you have the tools for and leave the others"
setup. The advantage, which I think is significant, is that
distros will naturally be directed to the missing build
dependency (either they're building with --enable-docs
and will get the configure message, or they aren't and
then their build will fail later because of missing docs
files when they try to put the built files into the package).

(2) What do we actually want to ship?
That is, what do we want 'make install-doc' to copy into
the installation directory?
https://wiki.qemu.org/Features/Documentation
has a good suggested breakdown of docs for where we
eventually want to be. I think we probably don't want
to install the "developer's guide" (docs/devel) on
end-user systems. The others are presumably OK.
Currently, we seem to only install manpages and a
few other things in the 'install-doc' makefile target
(we don't install a bunch of plain-text user-facing
docs) so this would be a significant expansion.

(3) Indexes, table-of-contents pages, etc
Are we aiming to ship these?
I think that we probably want to have what from
Sphinx's point of view are multiple separate documents,
so that they each get their own ToC and index. This
means we can for instance ship the ToC/index for
the user docs but not have it contain index entries
for developer docs.

Overall what I'm hoping for is to be able to get some
basic structure/building commands into master so we
have a framework and something we can iterate on to
move forward.

thanks
-- PMM

Re: [Qemu-devel] building rst docs with sphinx

[Philippe Mathieu-Daudé]

Hi Peter,

On 1/24/19 7:56 PM, Peter Maydell wrote:
> I had another look this afternoon at building our rST docs
> with sphinx-build. In particular, we currently have some
> docs in rst format, but we're not building them into HTML
> or shipping them. (Predictably, this means a few errors and
> warnings have crept in...)

Are you talking about the files in docs/ such docs/devel/testing.rst?

Or about documention within the source files?
If so, is that the format you are talking about?

https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#the-c-domain

'''
The C domain (name c) is suited for documentation of C API.

.. c:function:: function prototype

    Describes a C function. The signature should be given as in C, e.g.:

    .. c:function:: PyObject* PyType_GenericAlloc(PyTypeObject *type,
Py_ssize_t nitems)
'''

> 
> I had a play about with adding some makefile runes, but
> I'm not sure entirely what I should be aiming for.
> 
> (1) configure: My thought is that we should just make
> sphinx-build a requirement for the existing --enable-docs
> switch (as texinfo and pod2man are currently). The
> disadvantage is that we won't support a "build the half
> of the docs you have the tools for and leave the others"
> setup. The advantage, which I think is significant, is that
> distros will naturally be directed to the missing build
> dependency (either they're building with --enable-docs
> and will get the configure message, or they aren't and
> then their build will fail later because of missing docs
> files when they try to put the built files into the package).
> 
> (2) What do we actually want to ship?
> That is, what do we want 'make install-doc' to copy into
> the installation directory?
> https://wiki.qemu.org/Features/Documentation
> has a good suggested breakdown of docs for where we
> eventually want to be. I think we probably don't want
> to install the "developer's guide" (docs/devel) on
> end-user systems. The others are presumably OK.
> Currently, we seem to only install manpages and a
> few other things in the 'install-doc' makefile target
> (we don't install a bunch of plain-text user-facing
> docs) so this would be a significant expansion.
> 
> (3) Indexes, table-of-contents pages, etc
> Are we aiming to ship these?
> I think that we probably want to have what from
> Sphinx's point of view are multiple separate documents,
> so that they each get their own ToC and index. This
> means we can for instance ship the ToC/index for
> the user docs but not have it contain index entries
> for developer docs.
> 
> Overall what I'm hoping for is to be able to get some
> basic structure/building commands into master so we
> have a framework and something we can iterate on to
> move forward.
> 
> thanks
> -- PMM
> 

Re: [Qemu-devel] building rst docs with sphinx

[Peter Maydell]

On Thu, 24 Jan 2019 at 21:54, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> Hi Peter,
>
> On 1/24/19 7:56 PM, Peter Maydell wrote:
> > I had another look this afternoon at building our rST docs
> > with sphinx-build. In particular, we currently have some
> > docs in rst format, but we're not building them into HTML
> > or shipping them. (Predictably, this means a few errors and
> > warnings have crept in...)
>
> Are you talking about the files in docs/ such docs/devel/testing.rst?

I mean the .rst files in docs/. (I do want to do some autogen
as well -- we're planning to use the Linux kernel's kerneldoc
sphinx plugin for this, and I have some prototype patches to
plumb that in -- but since that will all be "developer"
docs I'd like to get the basic framework of manuals sorted
before adding that in.)

thanks
-- PMM

Re: [Qemu-devel] building rst docs with sphinx

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1930 bytes --]

On Thu, Jan 24, 2019 at 06:56:09PM +0000, Peter Maydell wrote:
> (1) configure: My thought is that we should just make
> sphinx-build a requirement for the existing --enable-docs
> switch (as texinfo and pod2man are currently). The
> disadvantage is that we won't support a "build the half
> of the docs you have the tools for and leave the others"
> setup. The advantage, which I think is significant, is that
> distros will naturally be directed to the missing build
> dependency (either they're building with --enable-docs
> and will get the configure message, or they aren't and
> then their build will fail later because of missing docs
> files when they try to put the built files into the package).

I'm CCing Cole (Fedora) and the Debian QEMU team so they can give their
input on this point and your next point.

> (2) What do we actually want to ship?
> That is, what do we want 'make install-doc' to copy into
> the installation directory?
> https://wiki.qemu.org/Features/Documentation
> has a good suggested breakdown of docs for where we
> eventually want to be. I think we probably don't want
> to install the "developer's guide" (docs/devel) on
> end-user systems. The others are presumably OK.
> Currently, we seem to only install manpages and a
> few other things in the 'install-doc' makefile target
> (we don't install a bunch of plain-text user-facing
> docs) so this would be a significant expansion.

I agree, developer documentation is not relevant to end-users.

> (3) Indexes, table-of-contents pages, etc
> Are we aiming to ship these?
> I think that we probably want to have what from
> Sphinx's point of view are multiple separate documents,
> so that they each get their own ToC and index. This
> means we can for instance ship the ToC/index for
> the user docs but not have it contain index entries
> for developer docs.

Indexes sound useful for each separate manual (user, developer, etc).

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] building rst docs with sphinx

[Daniel P. Berrangé]

On Thu, Jan 24, 2019 at 06:56:09PM +0000, Peter Maydell wrote:
> I had another look this afternoon at building our rST docs
> with sphinx-build. In particular, we currently have some
> docs in rst format, but we're not building them into HTML
> or shipping them. (Predictably, this means a few errors and
> warnings have crept in...)
> 
> I had a play about with adding some makefile runes, but
> I'm not sure entirely what I should be aiming for.
> 
> (1) configure: My thought is that we should just make
> sphinx-build a requirement for the existing --enable-docs
> switch (as texinfo and pod2man are currently). The
> disadvantage is that we won't support a "build the half
> of the docs you have the tools for and leave the others"
> setup. The advantage, which I think is significant, is that
> distros will naturally be directed to the missing build
> dependency (either they're building with --enable-docs
> and will get the configure message, or they aren't and
> then their build will fail later because of missing docs
> files when they try to put the built files into the package).

Currently in Fedora RPMs we ship the man pages and the
qemu-doc.html, qemu-ga-ref.html & qemu-qmp-ref.html files.
I'm not really seeing stuff in docs/ that grabs me as very
important to ship to users in Fedora RPMs. In fact I rather
doubt anyone even looks at the HTML files we currently
ship in the RPMs, as opposed to just looking in QEMU git.
Aside from man pages, I imagine most users just turn to
Google and/or qemu.org to find QEMU docs, not look inside
the distro packages. 

IOW from a distro POV I think I'd probably find sphinx-build
to be an undesirable mandatory dep.

Of course I do agree that we should be a enabling sphinx-build
during any of our CI builds so that we catch badly formatted
.rst docs. In addition I think much of docs/ could be useful
published on qemu.org.

> (2) What do we actually want to ship?
> That is, what do we want 'make install-doc' to copy into
> the installation directory?
> https://wiki.qemu.org/Features/Documentation
> has a good suggested breakdown of docs for where we
> eventually want to be. I think we probably don't want
> to install the "developer's guide" (docs/devel) on
> end-user systems. The others are presumably OK.
> Currently, we seem to only install manpages and a
> few other things in the 'install-doc' makefile target
> (we don't install a bunch of plain-text user-facing
> docs) so this would be a significant expansion.

I think 'make install-doc' should ship stuff that is useful
to end users of QEMU. This would be man pages, usage guides,
application developer information, etc.

It would not include anything that's related to the internals
of QEMU, as that's targetted at QEMU's developers themselves.

In Fedora we would not care to ship any docs that are targetting
QEMU developers, since those people would not be likely to be
using distro packaged QEMU.

> 
> (3) Indexes, table-of-contents pages, etc
> Are we aiming to ship these?
> I think that we probably want to have what from
> Sphinx's point of view are multiple separate documents,
> so that they each get their own ToC and index. This
> means we can for instance ship the ToC/index for
> the user docs but not have it contain index entries
> for developer docs.
> 
> Overall what I'm hoping for is to be able to get some
> basic structure/building commands into master so we
> have a framework and something we can iterate on to
> move forward.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] building rst docs with sphinx

[Peter Maydell]

On Tue, 29 Jan 2019 at 10:36, Daniel P. Berrangé <berrange@redhat.com> wrote:
> On Thu, Jan 24, 2019 at 06:56:09PM +0000, Peter Maydell wrote:
> > (1) configure: My thought is that we should just make
> > sphinx-build a requirement for the existing --enable-docs
> > switch (as texinfo and pod2man are currently). The
> > disadvantage is that we won't support a "build the half
> > of the docs you have the tools for and leave the others"
> > setup. The advantage, which I think is significant, is that
> > distros will naturally be directed to the missing build
> > dependency (either they're building with --enable-docs
> > and will get the configure message, or they aren't and
> > then their build will fail later because of missing docs
> > files when they try to put the built files into the package).
>
> Currently in Fedora RPMs we ship the man pages and the
> qemu-doc.html, qemu-ga-ref.html & qemu-qmp-ref.html files.
> I'm not really seeing stuff in docs/ that grabs me as very
> important to ship to users in Fedora RPMs. In fact I rather
> doubt anyone even looks at the HTML files we currently
> ship in the RPMs, as opposed to just looking in QEMU git.
> Aside from man pages, I imagine most users just turn to
> Google and/or qemu.org to find QEMU docs, not look inside
> the distro packages.
>
> IOW from a distro POV I think I'd probably find sphinx-build
> to be an undesirable mandatory dep.

The idea is that we would be aiming to transition all the
info that is currently in the other docs over to rST docs
(and also improve on the rST docs), i.e. we would want to
dump the current texinfo tooling eventually. It's true that
as build dep it doesn't bring much extra to the table to
start with, but that should change over time.

Is sphinx-build particularly nasty as a build-dep for distros,
or is it just "don't really want to add another thing" ?

thanks
-- PMM

Re: [Qemu-devel] building rst docs with sphinx

[Cole Robinson]

On 1/28/19 9:18 PM, Stefan Hajnoczi wrote:
> On Thu, Jan 24, 2019 at 06:56:09PM +0000, Peter Maydell wrote:
>> (1) configure: My thought is that we should just make
>> sphinx-build a requirement for the existing --enable-docs
>> switch (as texinfo and pod2man are currently). The
>> disadvantage is that we won't support a "build the half
>> of the docs you have the tools for and leave the others"
>> setup. The advantage, which I think is significant, is that
>> distros will naturally be directed to the missing build
>> dependency (either they're building with --enable-docs
>> and will get the configure message, or they aren't and
>> then their build will fail later because of missing docs
>> files when they try to put the built files into the package).
> 
> I'm CCing Cole (Fedora) and the Debian QEMU team so they can give their
> input on this point and your next point.
>

Peter's assessment sounds correct to me. And sphinx is so widely 
available that adding it as a build dep is no problem at all IMO

>> (2) What do we actually want to ship?
>> That is, what do we want 'make install-doc' to copy into
>> the installation directory?
>> https://wiki.qemu.org/Features/Documentation
>> has a good suggested breakdown of docs for where we
>> eventually want to be. I think we probably don't want
>> to install the "developer's guide" (docs/devel) on
>> end-user systems. The others are presumably OK.
>> Currently, we seem to only install manpages and a
>> few other things in the 'install-doc' makefile target
>> (we don't install a bunch of plain-text user-facing
>> docs) so this would be a significant expansion.
> 
> I agree, developer documentation is not relevant to end-users.
> 

Agreed as well. On my Fedora system I see that some packages install 
HACKING/CONTRIBUTING files but even that seems a little weird. If qemu 
cared about installing developer docs it could be a separate makefile 
target so it's easy to control whether it's handled during package build

Thanks,
Cole