* [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
@ 2020-09-25 16:22 Peter Maydell
2020-09-25 16:22 ` [PATCH v6 01/21] qapi: Fix doc comment indentation again Peter Maydell
` (25 more replies)
0 siblings, 26 replies; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
This series switches all our QAPI doc comments over from texinfo
format to rST. It then removes all the texinfo machinery, because
this was the last user of texinfo.
I think I have now resolved all of Markus' issues raised in his
review of v5, and IMHO this is ready to commit. If there are still
things needing fixing, it would be nice if we were able to commit
patches 1-5, which are the ones which add the new indent-sensitive
code to the QAPI parser. That would put a stop to the steady trickle
of doc-comment changes which break the new rules...
Also available as a git branch at
https://git.linaro.org/people/peter.maydell/qemu-arm.git sphinx-conversions-v6
The basic approach is somewhat similar to how we deal with kerneldoc
and hxtool: we have a custom Sphinx extension which is passed a
filename which is the json file it should run the QAPI parser over and
generate documentation for. Unlike 'kerneldoc' but somewhat like
hxtool, I have chosen to generate documentation by generating a tree
of docutils nodes, rather than by generating rST source that is then
fed to the rST parser to generate docutils nodes. Individual lumps of
doc comment go to the rST parser, but the structured parts we render
directly. This makes it easier to get the structure and heading level
nesting correct.
Changes from v5:
* rebased (in particular, updated to meson build system)
* new patch 1 fixes indent issues that hit master since v5
* new patch 2 makes block-latency-histogram-set's use of Example
sections match everybody else's, instead of special casing it
in the parser
* the .gitignore got pruned after meson conversion so we only
need to change git.orderfile now
* slightly reordered patches to bring the parser.py indent change nearer
the start of the series in the hopes of being able to get at least
that much of the series into master
* we now tell Sphinx about all the json files for dependency info,
so editing a json file correctly rebuilds the docs
* added a test case for the bad-de-indent parser error
* Adopted the various Python scripting suggestions from Markus
* We don't insist on section headings being the only thing in their
doc comment block any more (the existing "must be first line"
requirement is sufficient)
* added a test case for doc-generation that does a compare of
the sphinx plain-text builder output against a reference file
* Added the Python source files for Sphinx extensions (including
the QAPI source files) to the dependency lists for the manuals,
so that changes made to them correctly trigger a docs rebuild
* qemu-ga-ref.rst and qemu-qmp-ref.rst now have a TODO note about
making the manual licensing more visible to readers
* fixed bug in reported file/line info for some errors in rST
in doc comments when using Sphinx 1.6
* don't insist on section headers being in their own freeform doc
comment block; they're (after commit dcdc07a97cbe) always the
first line in the comment block, so just handle the possibility
of having text after that.
There are a few things I have left out of this initial series:
* unlike the texinfo, there is no generation of index entries
or an index in the HTML docs
* although there are HTML anchors on all the command/object/etc
headings, they are not stable but just serial-number based
tags like '#qapidoc-35', so not suitable for trying to link
to from other parts of the docs
My view is that we can add niceties like this later; the series
already seems big enough to me.
thanks
-- PMM
Peter Maydell (21):
qapi: Fix doc comment indentation again
qapi/block.json: Add newline after "Example:" for
block-latency-histogram-set
tests/qapi/doc-good.json: Prepare for qapi-doc Sphinx extension
scripts/qapi: Move doc-comment whitespace stripping to doc.py
scripts/qapi/parser.py: improve doc comment indent handling
qapi/machine.json: Escape a literal '*' in doc comment
docs/sphinx: Add new qapi-doc Sphinx extension
docs/interop: Convert qemu-ga-ref to rST
docs/interop: Convert qemu-qmp-ref to rST
qapi: Use rST markup for literal blocks
qga/qapi-schema.json: Add some headings
tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis
meson.build: Move SPHINX_ARGS to top level meson.build file
meson.build: Make manuals depend on source to Sphinx extensions
tests/qapi-schema: Add test of the rST QAPI doc-comment outputn
scripts/qapi: Remove texinfo generation support
docs/devel/qapi-code-gen.txt: Update to new rST backend conventions
scripts/texi2pod: Delete unused script
Remove Texinfo related line from git.orderfile
configure: Drop texinfo requirement
Remove texinfo dependency from docker and CI configs
docs/conf.py | 6 +-
docs/devel/qapi-code-gen.txt | 106 ++--
docs/index.html.in | 2 -
docs/interop/conf.py | 4 +
docs/interop/index.rst | 2 +
docs/interop/qemu-ga-ref.rst | 9 +
docs/interop/qemu-ga-ref.texi | 80 ---
docs/interop/qemu-qmp-ref.rst | 9 +
docs/interop/qemu-qmp-ref.texi | 80 ---
docs/meson.build | 11 +-
docs/sphinx/qapidoc.py | 550 +++++++++++++++++++++
tests/qapi-schema/doc-good.rst | 5 +
configure | 12 +-
meson.build | 103 +---
qapi/block-core.json | 20 +-
qapi/block.json | 12 +-
qapi/machine.json | 2 +-
qapi/migration.json | 102 ++--
qapi/qapi-schema.json | 6 +-
qga/qapi-schema.json | 8 +-
.travis.yml | 1 -
MAINTAINERS | 3 +-
qapi/meson.build | 4 +-
qga/meson.build | 3 +-
scripts/checkpatch.pl | 2 +-
scripts/coverity-scan/coverity-scan.docker | 1 -
scripts/git.orderfile | 1 -
scripts/qapi-gen.py | 2 -
scripts/qapi/doc.py | 301 -----------
scripts/qapi/gen.py | 7 -
scripts/qapi/parser.py | 105 +++-
scripts/texi2pod.pl | 536 --------------------
tests/docker/dockerfiles/debian10.docker | 1 -
tests/docker/dockerfiles/fedora.docker | 1 -
tests/docker/dockerfiles/ubuntu.docker | 1 -
tests/docker/dockerfiles/ubuntu1804.docker | 1 -
tests/docker/dockerfiles/ubuntu2004.docker | 1 -
tests/qapi-schema/doc-bad-indent.err | 1 +
tests/qapi-schema/doc-bad-indent.json | 8 +
tests/qapi-schema/doc-bad-indent.out | 0
tests/qapi-schema/doc-good.json | 27 +-
tests/qapi-schema/doc-good.out | 24 +-
tests/qapi-schema/doc-good.texi | 319 ------------
tests/qapi-schema/doc-good.txt | 288 +++++++++++
tests/qapi-schema/meson.build | 63 ++-
45 files changed, 1223 insertions(+), 1607 deletions(-)
create mode 100644 docs/interop/qemu-ga-ref.rst
delete mode 100644 docs/interop/qemu-ga-ref.texi
create mode 100644 docs/interop/qemu-qmp-ref.rst
delete mode 100644 docs/interop/qemu-qmp-ref.texi
create mode 100644 docs/sphinx/qapidoc.py
create mode 100644 tests/qapi-schema/doc-good.rst
delete mode 100644 scripts/qapi/doc.py
delete mode 100755 scripts/texi2pod.pl
create mode 100644 tests/qapi-schema/doc-bad-indent.err
create mode 100644 tests/qapi-schema/doc-bad-indent.json
create mode 100644 tests/qapi-schema/doc-bad-indent.out
delete mode 100644 tests/qapi-schema/doc-good.texi
create mode 100644 tests/qapi-schema/doc-good.txt
--
2.20.1
^ permalink raw reply [flat|nested] 69+ messages in thread
* [PATCH v6 01/21] qapi: Fix doc comment indentation again
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
@ 2020-09-25 16:22 ` Peter Maydell
2020-09-28 12:39 ` Markus Armbruster
2020-09-25 16:22 ` [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set Peter Maydell
` (24 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
In commit 26ec4e53f2 and similar commits we fixed the indentation
for doc comments in our qapi json files to follow a new stricter
standard for indentation, which permits only:
@arg: description line 1
description line 2
or:
@arg:
line 1
line 2
but because the script updates that enforce this are not yet in the
tree we have had a steady trickle of subsequent changes which didn't
follow the new rules.
Fix the latest round of mis-indented doc comments.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
qapi/block-core.json | 4 +-
qapi/migration.json | 102 +++++++++++++++++++++----------------------
2 files changed, 53 insertions(+), 53 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 3c16f1e11d6..dd77a91174c 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4316,8 +4316,8 @@
# @data-file-raw: True if the external data file must stay valid as a
# standalone (read-only) raw image without looking at qcow2
# metadata (default: false; since: 4.0)
-# @extended-l2 True to make the image have extended L2 entries
-# (default: false; since 5.2)
+# @extended-l2: True to make the image have extended L2 entries
+# (default: false; since 5.2)
# @size: Size of the virtual disk in bytes
# @version: Compatibility level (default: v3)
# @backing-file: File name of the backing file if a backing file
diff --git a/qapi/migration.json b/qapi/migration.json
index 675f70bb673..b89052c6fd3 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -681,23 +681,23 @@
# Defaults to 1. (Since 5.0)
#
# @block-bitmap-mapping: Maps block nodes and bitmaps on them to
-# aliases for the purpose of dirty bitmap migration. Such
-# aliases may for example be the corresponding names on the
-# opposite site.
-# The mapping must be one-to-one, but not necessarily
-# complete: On the source, unmapped bitmaps and all bitmaps
-# on unmapped nodes will be ignored. On the destination,
-# encountering an unmapped alias in the incoming migration
-# stream will result in a report, and all further bitmap
-# migration data will then be discarded.
-# Note that the destination does not know about bitmaps it
-# does not receive, so there is no limitation or requirement
-# regarding the number of bitmaps received, or how they are
-# named, or on which nodes they are placed.
-# By default (when this parameter has never been set), bitmap
-# names are mapped to themselves. Nodes are mapped to their
-# block device name if there is one, and to their node name
-# otherwise. (Since 5.2)
+# aliases for the purpose of dirty bitmap migration. Such
+# aliases may for example be the corresponding names on the
+# opposite site.
+# The mapping must be one-to-one, but not necessarily
+# complete: On the source, unmapped bitmaps and all bitmaps
+# on unmapped nodes will be ignored. On the destination,
+# encountering an unmapped alias in the incoming migration
+# stream will result in a report, and all further bitmap
+# migration data will then be discarded.
+# Note that the destination does not know about bitmaps it
+# does not receive, so there is no limitation or requirement
+# regarding the number of bitmaps received, or how they are
+# named, or on which nodes they are placed.
+# By default (when this parameter has never been set), bitmap
+# names are mapped to themselves. Nodes are mapped to their
+# block device name if there is one, and to their node name
+# otherwise. (Since 5.2)
#
# Since: 2.4
##
@@ -841,23 +841,23 @@
# Defaults to 1. (Since 5.0)
#
# @block-bitmap-mapping: Maps block nodes and bitmaps on them to
-# aliases for the purpose of dirty bitmap migration. Such
-# aliases may for example be the corresponding names on the
-# opposite site.
-# The mapping must be one-to-one, but not necessarily
-# complete: On the source, unmapped bitmaps and all bitmaps
-# on unmapped nodes will be ignored. On the destination,
-# encountering an unmapped alias in the incoming migration
-# stream will result in a report, and all further bitmap
-# migration data will then be discarded.
-# Note that the destination does not know about bitmaps it
-# does not receive, so there is no limitation or requirement
-# regarding the number of bitmaps received, or how they are
-# named, or on which nodes they are placed.
-# By default (when this parameter has never been set), bitmap
-# names are mapped to themselves. Nodes are mapped to their
-# block device name if there is one, and to their node name
-# otherwise. (Since 5.2)
+# aliases for the purpose of dirty bitmap migration. Such
+# aliases may for example be the corresponding names on the
+# opposite site.
+# The mapping must be one-to-one, but not necessarily
+# complete: On the source, unmapped bitmaps and all bitmaps
+# on unmapped nodes will be ignored. On the destination,
+# encountering an unmapped alias in the incoming migration
+# stream will result in a report, and all further bitmap
+# migration data will then be discarded.
+# Note that the destination does not know about bitmaps it
+# does not receive, so there is no limitation or requirement
+# regarding the number of bitmaps received, or how they are
+# named, or on which nodes they are placed.
+# By default (when this parameter has never been set), bitmap
+# names are mapped to themselves. Nodes are mapped to their
+# block device name if there is one, and to their node name
+# otherwise. (Since 5.2)
#
# Since: 2.4
##
@@ -1037,23 +1037,23 @@
# Defaults to 1. (Since 5.0)
#
# @block-bitmap-mapping: Maps block nodes and bitmaps on them to
-# aliases for the purpose of dirty bitmap migration. Such
-# aliases may for example be the corresponding names on the
-# opposite site.
-# The mapping must be one-to-one, but not necessarily
-# complete: On the source, unmapped bitmaps and all bitmaps
-# on unmapped nodes will be ignored. On the destination,
-# encountering an unmapped alias in the incoming migration
-# stream will result in a report, and all further bitmap
-# migration data will then be discarded.
-# Note that the destination does not know about bitmaps it
-# does not receive, so there is no limitation or requirement
-# regarding the number of bitmaps received, or how they are
-# named, or on which nodes they are placed.
-# By default (when this parameter has never been set), bitmap
-# names are mapped to themselves. Nodes are mapped to their
-# block device name if there is one, and to their node name
-# otherwise. (Since 5.2)
+# aliases for the purpose of dirty bitmap migration. Such
+# aliases may for example be the corresponding names on the
+# opposite site.
+# The mapping must be one-to-one, but not necessarily
+# complete: On the source, unmapped bitmaps and all bitmaps
+# on unmapped nodes will be ignored. On the destination,
+# encountering an unmapped alias in the incoming migration
+# stream will result in a report, and all further bitmap
+# migration data will then be discarded.
+# Note that the destination does not know about bitmaps it
+# does not receive, so there is no limitation or requirement
+# regarding the number of bitmaps received, or how they are
+# named, or on which nodes they are placed.
+# By default (when this parameter has never been set), bitmap
+# names are mapped to themselves. Nodes are mapped to their
+# block device name if there is one, and to their node name
+# otherwise. (Since 5.2)
#
# Since: 2.4
##
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
2020-09-25 16:22 ` [PATCH v6 01/21] qapi: Fix doc comment indentation again Peter Maydell
@ 2020-09-25 16:22 ` Peter Maydell
2020-09-28 12:42 ` Markus Armbruster
2020-09-25 16:22 ` [PATCH v6 03/21] tests/qapi/doc-good.json: Prepare for qapi-doc Sphinx extension Peter Maydell
` (23 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
The block-latency-histogram-set command is the only one which uses
the Example/Examples section with the first line of the documentation
immediately following the ':'. Bring it into line with the rest.
This allows us to avoid special-casing the indentation handling for
"Examples" sections; instead for Examples as for any other section
header these two indentations will be equivalent:
Examples:
Line one
Line two
Examples: Line one
Line two
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
qapi/block.json | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/qapi/block.json b/qapi/block.json
index c54a393cf3a..a009f7d3a2d 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -528,7 +528,8 @@
#
# Since: 4.0
#
-# Example: set new histograms for all io types with intervals
+# Example:
+# set new histograms for all io types with intervals
# [0, 10), [10, 50), [50, 100), [100, +inf):
#
# -> { "execute": "block-latency-histogram-set",
@@ -536,7 +537,8 @@
# "boundaries": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example: set new histogram only for write, other histograms will remain
+# Example:
+# set new histogram only for write, other histograms will remain
# not changed (or not created):
#
# -> { "execute": "block-latency-histogram-set",
@@ -544,7 +546,8 @@
# "boundaries-write": [10, 50, 100] } }
# <- { "return": {} }
#
-# Example: set new histograms with the following intervals:
+# Example:
+# set new histograms with the following intervals:
# read, flush: [0, 10), [10, 50), [50, 100), [100, +inf)
# write: [0, 1000), [1000, 5000), [5000, +inf)
#
@@ -554,7 +557,8 @@
# "boundaries-write": [1000, 5000] } }
# <- { "return": {} }
#
-# Example: remove all latency histograms:
+# Example:
+# remove all latency histograms:
#
# -> { "execute": "block-latency-histogram-set",
# "arguments": { "id": "drive0" } }
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 03/21] tests/qapi/doc-good.json: Prepare for qapi-doc Sphinx extension
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
2020-09-25 16:22 ` [PATCH v6 01/21] qapi: Fix doc comment indentation again Peter Maydell
2020-09-25 16:22 ` [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set Peter Maydell
@ 2020-09-25 16:22 ` Peter Maydell
2020-09-25 16:22 ` [PATCH v6 04/21] scripts/qapi: Move doc-comment whitespace stripping to doc.py Peter Maydell
` (22 subsequent siblings)
25 siblings, 0 replies; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
doc-good.json tests doc comment parser corner cases. We're about to
largely replace it by a Sphinx extension, which will have different
corner cases. Tweak the test so it passes both with the old parser
and the Sphinx extension, by making it match the more restrictive
rST syntax:
* in a single list the bullet types must all match
* lists must have leading and following blank lines
* the rules on when and where indentation matters differ
* the '|' example syntax is going to go away entirely, so stop
testing it
This will avoid the tests spuriously breaking when we tighten up the
parser code in the following commits.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
tests/qapi-schema/doc-good.json | 25 +++++++++++++------------
tests/qapi-schema/doc-good.out | 12 ++++++------
tests/qapi-schema/doc-good.texi | 12 +++---------
3 files changed, 22 insertions(+), 27 deletions(-)
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 9da72a1f556..c6822145c49 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -12,25 +12,25 @@
#
# *strong* _with emphasis_
# @var {in braces}
+#
# * List item one
-# - Two, multiple
+# * Two, multiple
# lines
#
-# 3. Three
-# Still in list
+# * Three
+# Still in list
+#
+# Not in list
#
-# Not in list
# - Second list
-# Note: still in list
+# Note: still in list
#
# Note: not in list
+#
# 1. Third list
# is numbered
#
-# - another item
-#
-# | example
-# | multiple lines
+# 2. another item
#
# Returns: the King
# Since: the first age
@@ -68,7 +68,7 @@
##
# @Base:
# @base1:
-# the first member
+# the first member
##
{ 'struct': 'Base', 'data': { 'base1': 'Enum' } }
@@ -116,7 +116,7 @@
##
# @Alternate:
# @i: an integer
-# @b is undocumented
+# @b is undocumented
#
# Features:
# @alt-feat: a feature
@@ -134,7 +134,7 @@
# @arg1: the first argument
#
# @arg2: the second
-# argument
+# argument
#
# Features:
# @cmd-feat1: a feature
@@ -143,6 +143,7 @@
# Returns: @Object
# TODO: frobnicate
# Notes:
+#
# - Lorem ipsum dolor sit amet
# - Ut enim ad minim veniam
#
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index d78a424cd9d..acb00ade17b 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -75,25 +75,25 @@ doc freeform
*strong* _with emphasis_
@var {in braces}
+
* List item one
-- Two, multiple
+* Two, multiple
lines
-3. Three
+* Three
Still in list
Not in list
+
- Second list
Note: still in list
Note: not in list
+
1. Third list
is numbered
-- another item
-
-| example
-| multiple lines
+2. another item
Returns: the King
Since: the first age
diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
index 7f28fb7a0fb..12808989ffb 100644
--- a/tests/qapi-schema/doc-good.texi
+++ b/tests/qapi-schema/doc-good.texi
@@ -6,6 +6,7 @@
@strong{strong} @emph{with emphasis}
@code{var} @{in braces@}
+
@itemize @bullet
@item
List item one
@@ -20,6 +21,7 @@ Still in list
@end itemize
Not in list
+
@itemize @minus
@item
Second list
@@ -28,6 +30,7 @@ Note: still in list
@end itemize
Note: not in list
+
@enumerate
@item
Third list
@@ -36,15 +39,6 @@ is numbered
@item
another item
-@example
-example
-@end example
-
-@example
-multiple lines
-@end example
-
-
@end enumerate
Returns: the King
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 04/21] scripts/qapi: Move doc-comment whitespace stripping to doc.py
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (2 preceding siblings ...)
2020-09-25 16:22 ` [PATCH v6 03/21] tests/qapi/doc-good.json: Prepare for qapi-doc Sphinx extension Peter Maydell
@ 2020-09-25 16:22 ` Peter Maydell
2020-09-25 16:23 ` [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling Peter Maydell
` (21 subsequent siblings)
25 siblings, 0 replies; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:22 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
As we accumulate lines from doc comments when parsing the JSON, the
QAPIDoc class generally strips leading and trailing whitespace using
line.strip() when it calls _append_freeform(). This is fine for
Texinfo, but for rST leading whitespace is significant. We'd like to
move to having the text in doc comments be rST format rather than a
custom syntax, so move the removal of leading whitespace from the
QAPIDoc class to the texinfo-specific processing code in
texi_format() in qapi/doc.py.
(Trailing whitespace will always be stripped by the rstrip() in
Section::append regardless.)
In a followup commit we will make the whitespace in the lines of doc
comment sections more consistently follow the input source.
There is no change to the generated .texi files before and after this
commit.
Because the qapi-schema test checks the exact values of the
documentation comments against a reference, we need to update that
reference to match the new whitespace. In the first four places this
is now correctly checking that we did put in the amount of whitespace
to pass a rST-formatted list to the backend; in the last two places
the extra whitespace is 'wrong' and will go away again in the
following commit.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
scripts/qapi/doc.py | 1 +
scripts/qapi/parser.py | 12 ++++--------
tests/qapi-schema/doc-good.out | 12 ++++++------
3 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 92f584edcf1..7764de1e4bc 100644
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -79,6 +79,7 @@ def texi_format(doc):
inlist = ''
lastempty = False
for line in doc.split('\n'):
+ line = line.strip()
empty = line == ''
# FIXME: Doing this in a single if / elif chain is
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 165925ca728..04bf10db378 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -427,10 +427,10 @@ class QAPIDoc:
self._append_line = self._append_various_line
self._append_various_line(line)
else:
- self._append_freeform(line.strip())
+ self._append_freeform(line)
else:
# This is a free-form documentation block
- self._append_freeform(line.strip())
+ self._append_freeform(line)
def _append_args_line(self, line):
"""
@@ -463,7 +463,7 @@ class QAPIDoc:
self._append_various_line(line)
return
- self._append_freeform(line.strip())
+ self._append_freeform(line)
def _append_features_line(self, line):
name = line.split(' ', 1)[0]
@@ -482,7 +482,7 @@ class QAPIDoc:
self._append_various_line(line)
return
- self._append_freeform(line.strip())
+ self._append_freeform(line)
def _append_various_line(self, line):
"""
@@ -505,10 +505,6 @@ class QAPIDoc:
line = line[len(name)+1:]
self._start_section(name[:-1])
- if (not self._section.name or
- not self._section.name.startswith('Example')):
- line = line.strip()
-
self._append_freeform(line)
def _start_symbol_section(self, symbols_dict, name):
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index acb00ade17b..9993ffcd89d 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -78,20 +78,20 @@ doc freeform
* List item one
* Two, multiple
-lines
+ lines
* Three
-Still in list
+ Still in list
Not in list
- Second list
-Note: still in list
+ Note: still in list
Note: not in list
1. Third list
-is numbered
+ is numbered
2. another item
@@ -159,7 +159,7 @@ doc symbol=Alternate
arg=i
an integer
-@b is undocumented
+ @b is undocumented
arg=b
feature=alt-feat
@@ -174,7 +174,7 @@ doc symbol=cmd
the first argument
arg=arg2
the second
-argument
+ argument
arg=arg3
feature=cmd-feat1
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (3 preceding siblings ...)
2020-09-25 16:22 ` [PATCH v6 04/21] scripts/qapi: Move doc-comment whitespace stripping to doc.py Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-28 19:15 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 06/21] qapi/machine.json: Escape a literal '*' in doc comment Peter Maydell
` (20 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
Make the handling of indentation in doc comments more sophisticated,
so that when we see a section like:
Notes: some text
some more text
indented line 3
we save it for the doc-comment processing code as:
some text
some more text
indented line 3
and when we see a section with the heading on its own line:
Notes:
some text
some more text
indented text
we also accept that and save it in the same form.
If we detect that the comment document text is not indented as much
as we expect it to be, we throw a parse error. (We don't complain
about over-indented sections, because for rST this can be legitimate
markup.)
The golden reference for the doc comment text is updated to remove
the two 'wrong' indents; these now form a test case that we correctly
stripped leading whitespace from an indented multi-line argument
definition.
We update the documentation in docs/devel/qapi-code-gen.txt to
describe the new indentation rules.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/devel/qapi-code-gen.txt | 23 +++++++
scripts/qapi/parser.py | 93 +++++++++++++++++++++------
tests/qapi-schema/doc-bad-indent.err | 1 +
tests/qapi-schema/doc-bad-indent.json | 8 +++
tests/qapi-schema/doc-bad-indent.out | 0
tests/qapi-schema/doc-good.out | 4 +-
tests/qapi-schema/meson.build | 1 +
7 files changed, 109 insertions(+), 21 deletions(-)
create mode 100644 tests/qapi-schema/doc-bad-indent.err
create mode 100644 tests/qapi-schema/doc-bad-indent.json
create mode 100644 tests/qapi-schema/doc-bad-indent.out
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 9eede44350c..69eaffac376 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -901,6 +901,22 @@ commands and events), member (for structs and unions), branch (for
alternates), or value (for enums), and finally optional tagged
sections.
+Descriptions of arguments can span multiple lines. The description
+text can start on the line following the '@argname:', in which case
+it must not be indented at all. It can also start on the same line
+as the '@argname:'. In this case if it spans multiple lines then
+second and subsequent lines must be indented to line up with the
+first character of the first line of the description:
+
+# @argone:
+# This is a two line description
+# in the first style.
+#
+# @argtwo: This is a two line description
+# in the second style.
+
+The number of spaces between the ':' and the text is not significant.
+
FIXME: the parser accepts these things in almost any order.
FIXME: union branches should be described, too.
@@ -911,6 +927,13 @@ A tagged section starts with one of the following words:
"Note:"/"Notes:", "Since:", "Example"/"Examples", "Returns:", "TODO:".
The section ends with the start of a new section.
+The text of a section can start on a new line, in
+which case it must not be indented at all. It can also start
+on the same line as the 'Note:', 'Returns:', etc tag. In this
+case if it spans multiple lines then second and subsequent
+lines must be indented to match the first, in the same way as
+multiline argument descriptions.
+
A 'Since: x.y.z' tagged section lists the release that introduced the
definition.
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 04bf10db378..6c3455b41f3 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -319,17 +319,32 @@ class QAPIDoc:
"""
class Section:
- def __init__(self, name=None):
+ def __init__(self, parser, name=None, indent=0):
+ # parser, for error messages about indentation
+ self._parser = parser
# optional section name (argument/member or section name)
self.name = name
self.text = ''
+ # the expected indent level of the text of this section
+ self._indent = indent
def append(self, line):
+ # Strip leading spaces corresponding to the expected indent level
+ # Blank lines are always OK.
+ if line:
+ indent = re.match(r'\s*', line).end()
+ if indent < self._indent:
+ raise QAPIParseError(
+ self._parser,
+ "unexpected de-indent (expected at least %d spaces)" %
+ self._indent)
+ line = line[self._indent:]
+
self.text += line.rstrip() + '\n'
class ArgSection(Section):
- def __init__(self, name):
- super().__init__(name)
+ def __init__(self, parser, name, indent=0):
+ super().__init__(parser, name, indent)
self.member = None
def connect(self, member):
@@ -343,7 +358,7 @@ class QAPIDoc:
self._parser = parser
self.info = info
self.symbol = None
- self.body = QAPIDoc.Section()
+ self.body = QAPIDoc.Section(parser)
# dict mapping parameter name to ArgSection
self.args = OrderedDict()
self.features = OrderedDict()
@@ -447,8 +462,21 @@ class QAPIDoc:
name = line.split(' ', 1)[0]
if name.startswith('@') and name.endswith(':'):
- line = line[len(name)+1:]
- self._start_args_section(name[1:-1])
+ # If line is "@arg: first line of description", find the
+ # index of 'f', which is the indent we expect for any
+ # following lines. We then remove the leading "@arg:" from line
+ # and replace it with spaces so that 'f' has the same index
+ # as it did in the original line and can be handled the same
+ # way we will handle following lines.
+ indent = re.match(r'@\S*:\s*', line).end()
+ line = line[indent:]
+ if not line:
+ # Line was just the "@arg:" header; following lines
+ # are not indented
+ indent = 0
+ else:
+ line = ' ' * indent + line
+ self._start_args_section(name[1:-1], indent)
elif self._is_section_tag(name):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -469,8 +497,21 @@ class QAPIDoc:
name = line.split(' ', 1)[0]
if name.startswith('@') and name.endswith(':'):
- line = line[len(name)+1:]
- self._start_features_section(name[1:-1])
+ # If line is "@arg: first line of description", find the
+ # index of 'f', which is the indent we expect for any
+ # following lines. We then remove the leading "@arg:" from line
+ # and replace it with spaces so that 'f' has the same index
+ # as it did in the original line and can be handled the same
+ # way we will handle following lines.
+ indent = re.match(r'@\S*:\s*', line).end()
+ line = line[indent:]
+ if not line:
+ # Line was just the "@arg:" header; following lines
+ # are not indented
+ indent = 0
+ else:
+ line = ' ' * indent + line
+ self._start_features_section(name[1:-1], indent)
elif self._is_section_tag(name):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -502,12 +543,25 @@ class QAPIDoc:
"'%s' can't follow '%s' section"
% (name, self.sections[0].name))
if self._is_section_tag(name):
- line = line[len(name)+1:]
- self._start_section(name[:-1])
+ # If line is "Section: first line of description", find the
+ # index of 'f', which is the indent we expect for any
+ # following lines. We then remove the leading "Section:" from line
+ # and replace it with spaces so that 'f' has the same index
+ # as it did in the original line and can be handled the same
+ # way we will handle following lines.
+ indent = re.match(r'\S*:\s*', line).end()
+ line = line[indent:]
+ if not line:
+ # Line was just the "Section:" header; following lines
+ # are not indented
+ indent = 0
+ else:
+ line = ' ' * indent + line
+ self._start_section(name[:-1], indent)
self._append_freeform(line)
- def _start_symbol_section(self, symbols_dict, name):
+ def _start_symbol_section(self, symbols_dict, name, indent):
# FIXME invalid names other than the empty string aren't flagged
if not name:
raise QAPIParseError(self._parser, "invalid parameter name")
@@ -516,21 +570,21 @@ class QAPIDoc:
"'%s' parameter name duplicated" % name)
assert not self.sections
self._end_section()
- self._section = QAPIDoc.ArgSection(name)
+ self._section = QAPIDoc.ArgSection(self._parser, name, indent)
symbols_dict[name] = self._section
- def _start_args_section(self, name):
- self._start_symbol_section(self.args, name)
+ def _start_args_section(self, name, indent):
+ self._start_symbol_section(self.args, name, indent)
- def _start_features_section(self, name):
- self._start_symbol_section(self.features, name)
+ def _start_features_section(self, name, indent):
+ self._start_symbol_section(self.features, name, indent)
- def _start_section(self, name=None):
+ def _start_section(self, name=None, indent=0):
if name in ('Returns', 'Since') and self.has_section(name):
raise QAPIParseError(self._parser,
"duplicated '%s' section" % name)
self._end_section()
- self._section = QAPIDoc.Section(name)
+ self._section = QAPIDoc.Section(self._parser, name, indent)
self.sections.append(self._section)
def _end_section(self):
@@ -553,7 +607,8 @@ class QAPIDoc:
def connect_member(self, member):
if member.name not in self.args:
# Undocumented TODO outlaw
- self.args[member.name] = QAPIDoc.ArgSection(member.name)
+ self.args[member.name] = QAPIDoc.ArgSection(self._parser,
+ member.name)
self.args[member.name].connect(member)
def connect_feature(self, feature):
diff --git a/tests/qapi-schema/doc-bad-indent.err b/tests/qapi-schema/doc-bad-indent.err
new file mode 100644
index 00000000000..67844539bd2
--- /dev/null
+++ b/tests/qapi-schema/doc-bad-indent.err
@@ -0,0 +1 @@
+doc-bad-indent.json:6:1: unexpected de-indent (expected at least 4 spaces)
diff --git a/tests/qapi-schema/doc-bad-indent.json b/tests/qapi-schema/doc-bad-indent.json
new file mode 100644
index 00000000000..edde8f21dc7
--- /dev/null
+++ b/tests/qapi-schema/doc-bad-indent.json
@@ -0,0 +1,8 @@
+# Multiline doc comments should have consistent indentation
+
+##
+# @foo:
+# @a: line one
+# line two is wrongly indented
+##
+{ 'command': 'foo', 'data': { 'a': 'int' } }
diff --git a/tests/qapi-schema/doc-bad-indent.out b/tests/qapi-schema/doc-bad-indent.out
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 9993ffcd89d..b7e3f4313da 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -159,7 +159,7 @@ doc symbol=Alternate
arg=i
an integer
- @b is undocumented
+@b is undocumented
arg=b
feature=alt-feat
@@ -174,7 +174,7 @@ doc symbol=cmd
the first argument
arg=arg2
the second
- argument
+argument
arg=arg3
feature=cmd-feat1
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index f1449298b07..83a0a68389b 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -53,6 +53,7 @@ schemas = [
'doc-bad-enum-member.json',
'doc-bad-event-arg.json',
'doc-bad-feature.json',
+ 'doc-bad-indent.json',
'doc-bad-section.json',
'doc-bad-symbol.json',
'doc-bad-union-member.json',
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 06/21] qapi/machine.json: Escape a literal '*' in doc comment
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (4 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 4:57 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension Peter Maydell
` (19 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
For rST, '*' is a kind of inline markup (for emphasis), so
"*-softmmu" is a syntax error because of the missing closing '*'.
Escape the '*' with a '\'.
The texinfo document generator will leave the '\' in the
output, which is not ideal, but that generator is going to
go away in a subsequent commit.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
qapi/machine.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index 0ac1880e4a4..9c45b04363c 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -13,7 +13,7 @@
#
# The comprehensive enumeration of QEMU system emulation ("softmmu")
# targets. Run "./configure --help" in the project root directory, and
-# look for the *-softmmu targets near the "--target-list" option. The
+# look for the \*-softmmu targets near the "--target-list" option. The
# individual target constants are not documented here, for the time
# being.
#
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (5 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 06/21] qapi/machine.json: Escape a literal '*' in doc comment Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 6:54 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST Peter Maydell
` (18 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
Some of our documentation is auto-generated from documentation
comments in the JSON schema.
For Sphinx, rather than creating a file to include, the most natural
way to handle this is to have a small custom Sphinx extension which
processes the JSON file and inserts documentation into the rST
file being processed.
This is the same approach that kerneldoc and hxtool use.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/conf.py | 6 +-
docs/sphinx/qapidoc.py | 550 +++++++++++++++++++++++++++++++++++++++++
MAINTAINERS | 1 +
3 files changed, 556 insertions(+), 1 deletion(-)
create mode 100644 docs/sphinx/qapidoc.py
diff --git a/docs/conf.py b/docs/conf.py
index 0dbd90dc112..606f623211d 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -52,7 +52,10 @@ except NameError:
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use an absolute path starting from qemu_docdir.
#
+# Our extensions are in docs/sphinx; the qapidoc extension requires
+# the QAPI modules from scripts/.
sys.path.insert(0, os.path.join(qemu_docdir, "sphinx"))
+sys.path.insert(0, os.path.join(qemu_docdir, "../scripts"))
# -- General configuration ------------------------------------------------
@@ -67,7 +70,7 @@ needs_sphinx = '1.6'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile']
+extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile', 'qapidoc']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -241,3 +244,4 @@ texinfo_documents = [
kerneldoc_bin = os.path.join(qemu_docdir, '../scripts/kernel-doc')
kerneldoc_srctree = os.path.join(qemu_docdir, '..')
hxtool_srctree = os.path.join(qemu_docdir, '..')
+qapidoc_srctree = os.path.join(qemu_docdir, '..')
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
new file mode 100644
index 00000000000..afc4167f8b1
--- /dev/null
+++ b/docs/sphinx/qapidoc.py
@@ -0,0 +1,550 @@
+# coding=utf-8
+#
+# QEMU qapidoc QAPI file parsing extension
+#
+# Copyright (c) 2020 Linaro
+#
+# This work is licensed under the terms of the GNU GPLv2 or later.
+# See the COPYING file in the top-level directory.
+"""qapidoc is a Sphinx extension that implements the qapi-doc directive"""
+
+# The purpose of this extension is to read the documentation comments
+# in QAPI schema files, and insert them all into the current document.
+#
+# It implements one new rST directive, "qapi-doc::".
+# Each qapi-doc:: directive takes one argument, which is the
+# pathname of the schema file to process, relative to the source tree.
+#
+# The docs/conf.py file must set the qapidoc_srctree config value to
+# the root of the QEMU source tree.
+#
+# The Sphinx documentation on writing extensions is at:
+# https://www.sphinx-doc.org/en/master/development/index.html
+import os
+import re
+
+from docutils import nodes
+from docutils.statemachine import ViewList
+from docutils.parsers.rst import directives, Directive
+from sphinx.errors import ExtensionError
+from sphinx.util.nodes import nested_parse_with_titles
+import sphinx
+from qapi.gen import QAPISchemaVisitor
+from qapi.schema import QAPIError, QAPISemError, QAPISchema
+
+# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
+# use switch_source_input. Check borrowed from kerneldoc.py.
+Use_SSI = sphinx.__version__[:3] >= '1.7'
+if Use_SSI:
+ from sphinx.util.docutils import switch_source_input
+else:
+ from sphinx.ext.autodoc import AutodocReporter
+
+
+__version__ = '1.0'
+
+
+# Function borrowed from pydash, which is under the MIT license
+def intersperse(iterable, separator):
+ """Yield the members of *iterable* interspersed with *separator*."""
+ iterable = iter(iterable)
+ yield next(iterable)
+ for item in iterable:
+ yield separator
+ yield item
+
+
+class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
+ """A QAPI schema visitor which generates docutils/Sphinx nodes
+
+ This class builds up a tree of docutils/Sphinx nodes corresponding
+ to documentation for the various QAPI objects. To use it, first create
+ a QAPISchemaGenRSTVisitor object, and call its visit_begin() method.
+ Then you can call one of the two methods 'freeform' (to add documentation
+ for a freeform documentation chunk) or 'symbol' (to add documentation
+ for a QAPI symbol). These will cause the visitor to build up the
+ tree of document nodes. Once you've added all the documentation
+ via 'freeform' and 'symbol' method calls, you can call 'get_document_nodes'
+ to get the final list of document nodes (in a form suitable for returning
+ from a Sphinx directive's 'run' method).
+ """
+ def __init__(self, sphinx_directive):
+ self._cur_doc = None
+ self._sphinx_directive = sphinx_directive
+ self._top_node = nodes.section()
+ self._active_headings = [self._top_node]
+
+ def _serror(self, msg):
+ """Raise an exception giving a user-friendly syntax error message"""
+ file = self._cur_doc.info.fname
+ line = self._cur_doc.info.line
+ raise ExtensionError(
+ '%s line %d: syntax error: %s' % (file, line, msg))
+
+ def _make_dlitem(self, term, defn):
+ """Return a dlitem node with the specified term and definition.
+
+ term should be a list of Text and literal nodes.
+ defn should be one of:
+ - a string, which will be handed to _parse_text_into_node
+ - a list of Text and literal nodes, which will be put into
+ a paragraph node
+ """
+ dlitem = nodes.definition_list_item()
+ dlterm = nodes.term('', '', *term)
+ dlitem += dlterm
+ if defn:
+ dldef = nodes.definition()
+ if isinstance(defn, list):
+ dldef += nodes.paragraph('', '', *defn)
+ else:
+ self._parse_text_into_node(defn, dldef)
+ dlitem += dldef
+ return dlitem
+
+ def _make_section(self, title):
+ """Return a section node with optional title"""
+ section = nodes.section(ids=[self._sphinx_directive.new_serialno()])
+ if title:
+ section += nodes.title(title, title)
+ return section
+
+ def _nodes_for_ifcond(self, ifcond, with_if=True):
+ """Return list of Text, literal nodes for the ifcond
+
+ Return a list which gives text like ' (If: cond1, cond2, cond3)', where
+ the conditions are in literal-text and the commas are not.
+ If with_if is False, we don't return the "(If: " and ")".
+ """
+ condlist = intersperse([nodes.literal('', c) for c in ifcond],
+ nodes.Text(', '))
+ if not with_if:
+ return condlist
+
+ nodelist = [nodes.Text(' ('), nodes.strong('', 'If: ')]
+ nodelist.extend(condlist)
+ nodelist.append(nodes.Text(')'))
+ return nodelist
+
+ def _nodes_for_one_member(self, member):
+ """Return list of Text, literal nodes for this member
+
+ Return a list of doctree nodes which give text like
+ 'name: type (optional) (If: ...)' suitable for use as the
+ 'term' part of a definition list item.
+ """
+ term = [nodes.literal('', member.name)]
+ if member.type.doc_type():
+ term.append(nodes.Text(': '))
+ term.append(nodes.literal('', member.type.doc_type()))
+ if member.optional:
+ term.append(nodes.Text(' (optional)'))
+ if member.ifcond:
+ term.extend(self._nodes_for_ifcond(member.ifcond))
+ return term
+
+ def _nodes_for_variant_when(self, variants, variant):
+ """Return list of Text, literal nodes for variant 'when' clause
+
+ Return a list of doctree nodes which give text like
+ 'when tagname is variant (If: ...)' suitable for use in
+ the 'variants' part of a definition list.
+ """
+ term = [nodes.Text(' when '),
+ nodes.literal('', variants.tag_member.name),
+ nodes.Text(' is '),
+ nodes.literal('', '"%s"' % variant.name)]
+ if variant.ifcond:
+ term.extend(self._nodes_for_ifcond(variant.ifcond))
+ return term
+
+ def _nodes_for_members(self, doc, what, base=None, variants=None):
+ """Return list of doctree nodes for the table of members"""
+ dlnode = nodes.definition_list()
+ for section in doc.args.values():
+ term = self._nodes_for_one_member(section.member)
+ # TODO drop fallbacks when undocumented members are outlawed
+ if section.text:
+ defn = section.text
+ elif (variants and variants.tag_member == section.member
+ and not section.member.type.doc_type()):
+ values = section.member.type.member_names()
+ defn = [nodes.Text('One of ')]
+ defn.extend(intersperse([nodes.literal('', v) for v in values],
+ nodes.Text(', ')))
+ else:
+ defn = [nodes.Text('Not documented')]
+
+ dlnode += self._make_dlitem(term, defn)
+
+ if base:
+ dlnode += self._make_dlitem([nodes.Text('The members of '),
+ nodes.literal('', base.doc_type())],
+ None)
+
+ if variants:
+ for v in variants.variants:
+ if v.type.is_implicit():
+ assert not v.type.base and not v.type.variants
+ for m in v.type.local_members:
+ term = self._nodes_for_one_member(m)
+ term.extend(self._nodes_for_variant_when(variants, v))
+ dlnode += self._make_dlitem(term, None)
+ else:
+ term = [nodes.Text('The members of '),
+ nodes.literal('', v.type.doc_type())]
+ term.extend(self._nodes_for_variant_when(variants, v))
+ dlnode += self._make_dlitem(term, None)
+
+ if not dlnode.children:
+ return []
+
+ section = self._make_section(what)
+ section += dlnode
+ return [section]
+
+ def _nodes_for_enum_values(self, doc):
+ """Return list of doctree nodes for the table of enum values"""
+ seen_item = False
+ dlnode = nodes.definition_list()
+ for section in doc.args.values():
+ termtext = [nodes.literal('', section.member.name)]
+ if section.member.ifcond:
+ termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
+ # TODO drop fallbacks when undocumented members are outlawed
+ if section.text:
+ defn = section.text
+ else:
+ defn = [nodes.Text('Not documented')]
+
+ dlnode += self._make_dlitem(termtext, defn)
+ seen_item = True
+
+ if not seen_item:
+ return []
+
+ section = self._make_section('Values')
+ section += dlnode
+ return [section]
+
+ def _nodes_for_arguments(self, doc, boxed_arg_type):
+ """Return list of doctree nodes for the arguments section"""
+ if boxed_arg_type:
+ assert not doc.args
+ section = self._make_section('Arguments')
+ dlnode = nodes.definition_list()
+ dlnode += self._make_dlitem(
+ [nodes.Text('The members of '),
+ nodes.literal('', boxed_arg_type.name)],
+ None)
+ section += dlnode
+ return [section]
+
+ return self._nodes_for_members(doc, 'Arguments')
+
+ def _nodes_for_features(self, doc):
+ """Return list of doctree nodes for the table of features"""
+ seen_item = False
+ dlnode = nodes.definition_list()
+ for section in doc.features.values():
+ dlnode += self._make_dlitem([nodes.literal('', section.name)],
+ section.text)
+ seen_item = True
+
+ if not seen_item:
+ return []
+
+ section = self._make_section('Features')
+ section += dlnode
+ return [section]
+
+ def _nodes_for_example(self, exampletext):
+ """Return list of doctree nodes for a code example snippet"""
+ return [nodes.literal_block(exampletext, exampletext)]
+
+ def _nodes_for_sections(self, doc):
+ """Return list of doctree nodes for additional sections"""
+ nodelist = []
+ for section in doc.sections:
+ snode = self._make_section(section.name)
+ if section.name and section.name.startswith('Example'):
+ snode += self._nodes_for_example(section.text)
+ else:
+ self._parse_text_into_node(section.text, snode)
+ nodelist.append(snode)
+ return nodelist
+
+ def _nodes_for_if_section(self, ifcond):
+ """Return list of doctree nodes for the "If" section"""
+ nodelist = []
+ if ifcond:
+ snode = self._make_section('If')
+ snode += self._nodes_for_ifcond(ifcond, with_if=False)
+ nodelist.append(snode)
+ return nodelist
+
+ def _add_doc(self, typ, sections):
+ """Add documentation for a command/object/enum...
+
+ We assume we're documenting the thing defined in self._cur_doc.
+ typ is the type of thing being added ("Command", "Object", etc)
+
+ sections is a list of nodes for sections to add to the definition.
+ """
+
+ doc = self._cur_doc
+ snode = nodes.section(ids=[self._sphinx_directive.new_serialno()])
+ snode += nodes.title('', '', *[nodes.literal(doc.symbol, doc.symbol),
+ nodes.Text(' (' + typ + ')')])
+ self._parse_text_into_node(doc.body.text, snode)
+ for s in sections:
+ if s is not None:
+ snode += s
+ self._add_node_to_current_heading(snode)
+
+ def visit_enum_type(self, name, info, ifcond, features, members, prefix):
+ doc = self._cur_doc
+ self._add_doc('Enum',
+ self._nodes_for_enum_values(doc) +
+ self._nodes_for_features(doc) +
+ self._nodes_for_sections(doc) +
+ self._nodes_for_if_section(ifcond))
+
+ def visit_object_type(self, name, info, ifcond, features,
+ base, members, variants):
+ doc = self._cur_doc
+ if base and base.is_implicit():
+ base = None
+ self._add_doc('Object',
+ self._nodes_for_members(doc, 'Members', base, variants) +
+ self._nodes_for_features(doc) +
+ self._nodes_for_sections(doc) +
+ self._nodes_for_if_section(ifcond))
+
+ def visit_alternate_type(self, name, info, ifcond, features, variants):
+ doc = self._cur_doc
+ self._add_doc('Alternate',
+ self._nodes_for_members(doc, 'Members') +
+ self._nodes_for_features(doc) +
+ self._nodes_for_sections(doc) +
+ self._nodes_for_if_section(ifcond))
+
+ def visit_command(self, name, info, ifcond, features, arg_type,
+ ret_type, gen, success_response, boxed, allow_oob,
+ allow_preconfig):
+ doc = self._cur_doc
+ self._add_doc('Command',
+ self._nodes_for_arguments(doc,
+ arg_type if boxed else None) +
+ self._nodes_for_features(doc) +
+ self._nodes_for_sections(doc) +
+ self._nodes_for_if_section(ifcond))
+
+ def visit_event(self, name, info, ifcond, features, arg_type, boxed):
+ doc = self._cur_doc
+ self._add_doc('Event',
+ self._nodes_for_arguments(doc,
+ arg_type if boxed else None) +
+ self._nodes_for_features(doc) +
+ self._nodes_for_sections(doc) +
+ self._nodes_for_if_section(ifcond))
+
+ def symbol(self, doc, entity):
+ """Add documentation for one symbol to the document tree
+
+ This is the main entry point which causes us to add documentation
+ nodes for a symbol (which could be a 'command', 'object', 'event',
+ etc). We do this by calling 'visit' on the schema entity, which
+ will then call back into one of our visit_* methods, depending
+ on what kind of thing this symbol is.
+ """
+ self._cur_doc = doc
+ entity.visit(self)
+ self._cur_doc = None
+
+ def _start_new_heading(self, heading, level):
+ """Start a new heading at the specified heading level
+
+ Create a new section whose title is 'heading' and which is placed
+ in the docutils node tree as a child of the most recent level-1
+ heading. Subsequent document sections (commands, freeform doc chunks,
+ etc) will be placed as children of this new heading section.
+ """
+ if len(self._active_headings) < level:
+ raise QAPISemError(self._cur_doc.info,
+ 'Level %d subheading found outside a '
+ 'level %d heading'
+ % (level, level - 1))
+ snode = self._make_section(heading)
+ self._active_headings[level - 1] += snode
+ self._active_headings = self._active_headings[:level]
+ self._active_headings.append(snode)
+
+ def _add_node_to_current_heading(self, node):
+ """Add the node to whatever the current active heading is"""
+ self._active_headings[-1] += node
+
+ def freeform(self, doc):
+ """Add a piece of 'freeform' documentation to the document tree
+
+ A 'freeform' document chunk doesn't relate to any particular
+ symbol (for instance, it could be an introduction).
+
+ If the freeform document starts with a line of the form
+ '= Heading text', this is a section or subsection heading, with
+ the heading level indicated by the number of '=' signs.
+ """
+
+ # QAPIDoc documentation says free-form documentation blocks
+ # must have only a body section, nothing else.
+ assert not doc.sections
+ assert not doc.args
+ assert not doc.features
+ self._cur_doc = doc
+
+ text = doc.body.text
+ if re.match(r'=+ ', text):
+ # Section/subsection heading (if present, will always be
+ # the first line of the block)
+ (heading, _, text) = text.partition('\n')
+ (leader, _, heading) = heading.partition(' ')
+ self._start_new_heading(heading, len(leader))
+ if text == '':
+ return
+
+ node = self._make_section(None)
+ self._parse_text_into_node(text, node)
+ self._add_node_to_current_heading(node)
+ self._cur_doc = None
+
+ def _parse_text_into_node(self, doctext, node):
+ """Parse a chunk of QAPI-doc-format text into the node
+
+ The doc comment can contain most inline rST markup, including
+ bulleted and enumerated lists.
+ As an extra permitted piece of markup, @var will be turned
+ into ``var``.
+ """
+
+ # Handle the "@var means ``var`` case
+ doctext = re.sub(r'@([\w-]+)', r'``\1``', doctext)
+
+ rstlist = ViewList()
+ for line in doctext.splitlines():
+ # The reported line number will always be that of the start line
+ # of the doc comment, rather than the actual location of the error.
+ # Being more precise would require overhaul of the QAPIDoc class
+ # to track lines more exactly within all the sub-parts of the doc
+ # comment, as well as counting lines here.
+ rstlist.append(line, self._cur_doc.info.fname,
+ self._cur_doc.info.line)
+ # Append a blank line -- in some cases rST syntax errors get
+ # attributed to the line after one with actual text, and if there
+ # isn't anything in the ViewList corresponding to that then Sphinx
+ # 1.6's AutodocReporter will then misidentify the source/line location
+ # in the error message (usually attributing it to the top-level
+ # .rst file rather than the offending .json file). The extra blank
+ # line won't affect the rendered output.
+ rstlist.append("", self._cur_doc.info.fname, self._cur_doc.info.line)
+ self._sphinx_directive.do_parse(rstlist, node)
+
+ def get_document_nodes(self):
+ """Return the list of docutils nodes which make up the document"""
+ return self._top_node.children
+
+
+class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
+ """A QAPI schema visitor which adds Sphinx dependencies each module
+
+ This class calls the Sphinx note_dependency() function to tell Sphinx
+ that the generated documentation output depends on the input
+ schema file associated with each module in the QAPI input.
+ """
+ def __init__(self, env, qapidir):
+ self._env = env
+ self._qapidir = qapidir
+
+ def visit_module(self, name):
+ if name is not None:
+ qapifile = self._qapidir + '/' + name
+ self._env.note_dependency(os.path.abspath(qapifile))
+ super().visit_module(name)
+
+
+class QAPIDocDirective(Directive):
+ """Extract documentation from the specified QAPI .json file"""
+ required_argument = 1
+ optional_arguments = 1
+ option_spec = {
+ 'qapifile': directives.unchanged_required
+ }
+ has_content = False
+
+ def new_serialno(self):
+ """Return a unique new ID string suitable for use as a node's ID"""
+ env = self.state.document.settings.env
+ return 'qapidoc-%d' % env.new_serialno('qapidoc')
+
+ def run(self):
+ env = self.state.document.settings.env
+ qapifile = env.config.qapidoc_srctree + '/' + self.arguments[0]
+ qapidir = os.path.dirname(qapifile)
+
+ try:
+ schema = QAPISchema(qapifile)
+
+ # First tell Sphinx about all the schema files that the
+ # output documentation depends on (including 'qapifile' itself)
+ schema.visit(QAPISchemaGenDepVisitor(env, qapidir))
+
+ vis = QAPISchemaGenRSTVisitor(self)
+ vis.visit_begin(schema)
+ for doc in schema.docs:
+ if doc.symbol:
+ vis.symbol(doc, schema.lookup_entity(doc.symbol))
+ else:
+ vis.freeform(doc)
+ return vis.get_document_nodes()
+ except QAPIError as err:
+ # Launder QAPI parse errors into Sphinx extension errors
+ # so they are displayed nicely to the user
+ raise ExtensionError(str(err))
+
+ def do_parse(self, rstlist, node):
+ """Parse rST source lines and add them to the specified node
+
+ Take the list of rST source lines rstlist, parse them as
+ rST, and add the resulting docutils nodes as children of node.
+ The nodes are parsed in a way that allows them to include
+ subheadings (titles) without confusing the rendering of
+ anything else.
+ """
+ # This is from kerneldoc.py -- it works around an API change in
+ # Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use
+ # sphinx.util.nodes.nested_parse_with_titles() rather than the
+ # plain self.state.nested_parse(), and so we can drop the saving
+ # of title_styles and section_level that kerneldoc.py does,
+ # because nested_parse_with_titles() does that for us.
+ if Use_SSI:
+ with switch_source_input(self.state, rstlist):
+ nested_parse_with_titles(self.state, rstlist, node)
+ else:
+ save = self.state.memo.reporter
+ self.state.memo.reporter = AutodocReporter(
+ rstlist, self.state.memo.reporter)
+ try:
+ nested_parse_with_titles(self.state, rstlist, node)
+ finally:
+ self.state.memo.reporter = save
+
+
+def setup(app):
+ """ Register qapi-doc directive with Sphinx"""
+ app.add_config_value('qapidoc_srctree', None, 'env')
+ app.add_directive('qapi-doc', QAPIDocDirective)
+
+ return dict(
+ version=__version__,
+ parallel_read_safe=True,
+ parallel_write_safe=True
+ )
diff --git a/MAINTAINERS b/MAINTAINERS
index 5eed1e692b4..dbddb0a7635 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3149,6 +3149,7 @@ M: Peter Maydell <peter.maydell@linaro.org>
S: Maintained
F: docs/conf.py
F: docs/*/conf.py
+F: docs/sphinx/
Miscellaneous
-------------
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (6 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 8:20 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref " Peter Maydell
` (17 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
Convert qemu-ga-ref to rST format. This includes dropping
the plain-text, pdf and info format outputs for this document;
as with all our other Sphinx-based documentation, we provide
HTML and manpage only.
The qemu-ga-ref.rst is somewhat more stripped down than
the .texi was, because we do not (currently) attempt to
generate indexes for the commands, events and data types
being documented.
As the GA ref is now part of the Sphinx 'interop' manual,
we can delete the direct link from index.html.in.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/index.html.in | 1 -
docs/interop/conf.py | 2 +
docs/interop/index.rst | 1 +
docs/interop/qemu-ga-ref.rst | 9 ++++
docs/interop/qemu-ga-ref.texi | 80 -----------------------------------
docs/meson.build | 1 +
meson.build | 3 --
MAINTAINERS | 2 +-
qga/meson.build | 3 +-
9 files changed, 15 insertions(+), 87 deletions(-)
create mode 100644 docs/interop/qemu-ga-ref.rst
delete mode 100644 docs/interop/qemu-ga-ref.texi
diff --git a/docs/index.html.in b/docs/index.html.in
index ca28047881e..5b0f4e2bc2c 100644
--- a/docs/index.html.in
+++ b/docs/index.html.in
@@ -13,7 +13,6 @@
<li><a href="interop/index.html">System Emulation Management and Interoperability Guide</a></li>
<li><a href="specs/index.html">System Emulation Guest Hardware Specifications</a></li>
<li><a href="interop/qemu-qmp-ref.html">QMP Reference Manual</a></li>
- <li><a href="interop/qemu-ga-ref.html">Guest Agent Protocol Reference</a></li>
</ul>
</body>
</html>
diff --git a/docs/interop/conf.py b/docs/interop/conf.py
index 42ce7e3d365..e83632e0108 100644
--- a/docs/interop/conf.py
+++ b/docs/interop/conf.py
@@ -19,4 +19,6 @@ html_theme_options['description'] = u'System Emulation Management and Interopera
man_pages = [
('qemu-ga', 'qemu-ga', u'QEMU Guest Agent',
['Michael Roth <mdroth@linux.vnet.ibm.com>'], 8),
+ ('qemu-ga-ref', 'qemu-ga-ref', u'QEMU Guest Agent Protocol Reference',
+ [], 7),
]
diff --git a/docs/interop/index.rst b/docs/interop/index.rst
index 006f9864208..738cdbe185e 100644
--- a/docs/interop/index.rst
+++ b/docs/interop/index.rst
@@ -18,6 +18,7 @@ Contents:
live-block-operations
pr-helper
qemu-ga
+ qemu-ga-ref
vhost-user
vhost-user-gpu
vhost-vdpa
diff --git a/docs/interop/qemu-ga-ref.rst b/docs/interop/qemu-ga-ref.rst
new file mode 100644
index 00000000000..669019de71d
--- /dev/null
+++ b/docs/interop/qemu-ga-ref.rst
@@ -0,0 +1,9 @@
+QEMU Guest Agent Protocol Reference
+===================================
+
+..
+ TODO: the old texinfo manual used to note that this manual
+ is GPL-v2-or-later. We should make that reader-visible
+ both here and in our Sphinx manuals more generally.
+
+.. qapi-doc:: qga/qapi-schema.json
diff --git a/docs/interop/qemu-ga-ref.texi b/docs/interop/qemu-ga-ref.texi
deleted file mode 100644
index a23cc2ed7f6..00000000000
--- a/docs/interop/qemu-ga-ref.texi
+++ /dev/null
@@ -1,80 +0,0 @@
-\input texinfo
-@setfilename qemu-ga-ref.info
-
-@include version.texi
-
-@exampleindent 0
-@paragraphindent 0
-
-@settitle QEMU Guest Agent Protocol Reference
-
-@iftex
-@center @image{docs/qemu_logo}
-@end iftex
-
-@copying
-This is the QEMU Guest Agent Protocol reference manual.
-
-Copyright @copyright{} 2016 The QEMU Project developers
-
-@quotation
-This manual is free documentation: you can redistribute it and/or
-modify it under the terms of the GNU General Public License as
-published by the Free Software Foundation, either version 2 of the
-License, or (at your option) any later version.
-
-This manual is distributed in the hope that it will be useful, but
-WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this manual. If not, see http://www.gnu.org/licenses/.
-@end quotation
-@end copying
-
-@dircategory QEMU
-@direntry
-* QEMU-GA-Ref: (qemu-ga-ref). QEMU Guest Agent Protocol Reference
-@end direntry
-
-@titlepage
-@title Guest Agent Protocol Reference Manual
-@subtitle QEMU version @value{VERSION}
-@page
-@vskip 0pt plus 1filll
-@insertcopying
-@end titlepage
-
-@contents
-
-@ifnottex
-@node Top
-@top QEMU Guest Agent protocol reference
-@end ifnottex
-
-@menu
-* API Reference::
-* Commands and Events Index::
-* Data Types Index::
-@end menu
-
-@node API Reference
-@chapter API Reference
-
-@c for texi2pod:
-@c man begin DESCRIPTION
-
-@include qga/qga-qapi-doc.texi
-
-@c man end
-
-@node Commands and Events Index
-@unnumbered Commands and Events Index
-@printindex fn
-
-@node Data Types Index
-@unnumbered Data Types Index
-@printindex tp
-
-@bye
diff --git a/docs/meson.build b/docs/meson.build
index 50f367349b8..2569dd52ad6 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -15,6 +15,7 @@ if build_docs
man_pages = {
'interop' : {
'qemu-ga.8': (have_tools ? 'man8' : ''),
+ 'qemu-ga-ref.7': 'man7',
},
'tools': {
'qemu-img.1': (have_tools ? 'man1' : ''),
diff --git a/meson.build b/meson.build
index f4d1ab10968..c01c6230b3f 100644
--- a/meson.build
+++ b/meson.build
@@ -1220,9 +1220,6 @@ if build_docs
texi = {
'qemu-qmp-ref': ['docs/interop/qemu-qmp-ref.texi', qapi_doc_texi, version_texi],
}
- if 'CONFIG_GUEST_AGENT' in config_host
- texi += {'qemu-ga-ref': ['docs/interop/qemu-ga-ref.texi', qga_qapi_doc_texi, version_texi]}
- endif
if makeinfo.found()
cmd = [
diff --git a/MAINTAINERS b/MAINTAINERS
index dbddb0a7635..4b2705fc143 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2418,9 +2418,9 @@ M: Michael Roth <mdroth@linux.vnet.ibm.com>
S: Maintained
F: qga/
F: docs/interop/qemu-ga.rst
+F: docs/interop/qemu-ga-ref.rst
F: scripts/qemu-guest-agent/
F: tests/test-qga.c
-F: docs/interop/qemu-ga-ref.texi
T: git https://github.com/mdroth/qemu.git qga
QOM
diff --git a/qga/meson.build b/qga/meson.build
index e5c5778a3e0..1c312b50cc9 100644
--- a/qga/meson.build
+++ b/qga/meson.build
@@ -16,7 +16,7 @@ qga_qapi_outputs = [
]
qga_qapi_files = custom_target('QGA QAPI files',
- output: qga_qapi_outputs + ['qga-qapi-doc.texi'],
+ output: qga_qapi_outputs,
input: 'qapi-schema.json',
command: [ qapi_gen, '-o', 'qga', '-p', 'qga-', '@INPUT0@' ],
depend_files: qapi_gen_depends)
@@ -27,7 +27,6 @@ foreach output: qga_qapi_outputs
qga_ss.add(qga_qapi_files[i])
i = i + 1
endforeach
-qga_qapi_doc_texi = qga_qapi_files[i]
qga_ss.add(files(
'commands.c',
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref to rST
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (7 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 8:27 ` Markus Armbruster
2020-09-29 8:42 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 10/21] qapi: Use rST markup for literal blocks Peter Maydell
` (16 subsequent siblings)
25 siblings, 2 replies; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
Convert qemu-qmp-ref to rST format. This includes dropping
the plain-text, pdf and info format outputs for this document;
as with all our other Sphinx-based documentation, we provide
HTML and manpage only.
The qemu-qmp-ref.rst is somewhat more stripped down than
the .texi was, because we do not (currently) attempt to
generate indexes for the commands, events and data types
being documented.
Again, we drop the direct link from index.html.in now that
the QMP ref is part of the interop manual.
This commit removes the code from the root meson.build file that
handled the various Texinfo-based outputs, because we no longer
generate any documentation except for the Sphinx HTML manuals and the
manpages, and the code can't handle having an empty list of files
to process.. We'll do further cleanup of the remainders of
Texinfo support in subsequent commits.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/index.html.in | 1 -
docs/interop/conf.py | 2 +
docs/interop/index.rst | 1 +
docs/interop/qemu-qmp-ref.rst | 9 ++++
docs/interop/qemu-qmp-ref.texi | 80 ---------------------------------
docs/meson.build | 1 +
meson.build | 82 ----------------------------------
qapi/meson.build | 4 +-
8 files changed, 14 insertions(+), 166 deletions(-)
create mode 100644 docs/interop/qemu-qmp-ref.rst
delete mode 100644 docs/interop/qemu-qmp-ref.texi
diff --git a/docs/index.html.in b/docs/index.html.in
index 5b0f4e2bc2c..33db4396ac8 100644
--- a/docs/index.html.in
+++ b/docs/index.html.in
@@ -12,7 +12,6 @@
<li><a href="tools/index.html">Tools Guide</a></li>
<li><a href="interop/index.html">System Emulation Management and Interoperability Guide</a></li>
<li><a href="specs/index.html">System Emulation Guest Hardware Specifications</a></li>
- <li><a href="interop/qemu-qmp-ref.html">QMP Reference Manual</a></li>
</ul>
</body>
</html>
diff --git a/docs/interop/conf.py b/docs/interop/conf.py
index e83632e0108..43de386d33d 100644
--- a/docs/interop/conf.py
+++ b/docs/interop/conf.py
@@ -21,4 +21,6 @@ man_pages = [
['Michael Roth <mdroth@linux.vnet.ibm.com>'], 8),
('qemu-ga-ref', 'qemu-ga-ref', u'QEMU Guest Agent Protocol Reference',
[], 7),
+ ('qemu-qmp-ref', 'qemu-qmp-ref', u'QEMU QMP Reference Manual',
+ [], 7),
]
diff --git a/docs/interop/index.rst b/docs/interop/index.rst
index 738cdbe185e..cd78d679d82 100644
--- a/docs/interop/index.rst
+++ b/docs/interop/index.rst
@@ -19,6 +19,7 @@ Contents:
pr-helper
qemu-ga
qemu-ga-ref
+ qemu-qmp-ref
vhost-user
vhost-user-gpu
vhost-vdpa
diff --git a/docs/interop/qemu-qmp-ref.rst b/docs/interop/qemu-qmp-ref.rst
new file mode 100644
index 00000000000..0c416fcaac2
--- /dev/null
+++ b/docs/interop/qemu-qmp-ref.rst
@@ -0,0 +1,9 @@
+QEMU QMP Reference Manual
+=========================
+
+..
+ TODO: the old texinfo manual used to note that this manual
+ is GPL-v2-or-later. We should make that reader-visible
+ both here and in our Sphinx manuals more generally.
+
+.. qapi-doc:: qapi/qapi-schema.json
diff --git a/docs/interop/qemu-qmp-ref.texi b/docs/interop/qemu-qmp-ref.texi
deleted file mode 100644
index ea1d7fe6c2d..00000000000
--- a/docs/interop/qemu-qmp-ref.texi
+++ /dev/null
@@ -1,80 +0,0 @@
-\input texinfo
-@setfilename qemu-qmp-ref.info
-
-@include version.texi
-
-@exampleindent 0
-@paragraphindent 0
-
-@settitle QEMU QMP Reference Manual
-
-@iftex
-@center @image{docs/qemu_logo}
-@end iftex
-
-@copying
-This is the QEMU QMP reference manual.
-
-Copyright @copyright{} 2016 The QEMU Project developers
-
-@quotation
-This manual is free documentation: you can redistribute it and/or
-modify it under the terms of the GNU General Public License as
-published by the Free Software Foundation, either version 2 of the
-License, or (at your option) any later version.
-
-This manual is distributed in the hope that it will be useful, but
-WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this manual. If not, see http://www.gnu.org/licenses/.
-@end quotation
-@end copying
-
-@dircategory QEMU
-@direntry
-* QEMU-QMP-Ref: (qemu-qmp-ref). QEMU QMP Reference Manual
-@end direntry
-
-@titlepage
-@title QMP Reference Manual
-@subtitle QEMU version @value{VERSION}
-@page
-@vskip 0pt plus 1filll
-@insertcopying
-@end titlepage
-
-@contents
-
-@ifnottex
-@node Top
-@top QEMU QMP reference
-@end ifnottex
-
-@menu
-* API Reference::
-* Commands and Events Index::
-* Data Types Index::
-@end menu
-
-@node API Reference
-@chapter API Reference
-
-@c for texi2pod:
-@c man begin DESCRIPTION
-
-@include qapi/qapi-doc.texi
-
-@c man end
-
-@node Commands and Events Index
-@unnumbered Commands and Events Index
-@printindex fn
-
-@node Data Types Index
-@unnumbered Data Types Index
-@printindex tp
-
-@bye
diff --git a/docs/meson.build b/docs/meson.build
index 2569dd52ad6..69097e2ca07 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -16,6 +16,7 @@ if build_docs
'interop' : {
'qemu-ga.8': (have_tools ? 'man8' : ''),
'qemu-ga-ref.7': 'man7',
+ 'qemu-qmp-ref.7': 'man7',
},
'tools': {
'qemu-img.1': (have_tools ? 'man1' : ''),
diff --git a/meson.build b/meson.build
index c01c6230b3f..73d675ca834 100644
--- a/meson.build
+++ b/meson.build
@@ -1203,88 +1203,6 @@ if 'CONFIG_GTK' in config_host
subdir('po')
endif
-if build_docs
- makeinfo = find_program('makeinfo', required: build_docs)
-
- docs_inc = [
- '-I', meson.current_source_dir(),
- '-I', meson.current_build_dir() / 'docs',
- '-I', '@OUTDIR@',
- ]
-
- version_texi = configure_file(output: 'version.texi',
- input: 'version.texi.in',
- configuration: {'VERSION': meson.project_version(),
- 'qemu_confdir': config_host['qemu_confdir']})
-
- texi = {
- 'qemu-qmp-ref': ['docs/interop/qemu-qmp-ref.texi', qapi_doc_texi, version_texi],
- }
-
- if makeinfo.found()
- cmd = [
- 'env', 'LC_ALL=C', makeinfo, '--no-split', '--number-sections', docs_inc,
- '@INPUT0@', '-o', '@OUTPUT@',
- ]
- foreach ext, args: {
- 'info': [],
- 'html': ['--no-headers', '--html'],
- 'txt': ['--no-headers', '--plaintext'],
- }
- t = []
- foreach doc, input: texi
- output = doc + '.' + ext
- t += custom_target(output,
- input: input,
- output: output,
- install: true,
- install_dir: qemu_docdir / 'interop',
- command: cmd + args)
- endforeach
- alias_target(ext, t)
- endforeach
- endif
-
- texi2pdf = find_program('texi2pdf', required: false)
-
- if texi2pdf.found()
- pdfs = []
- foreach doc, input: texi
- output = doc + '.pdf'
- pdfs += custom_target(output,
- input: input,
- output: output,
- command: [texi2pdf, '-q', docs_inc, '@INPUT0@', '-o', '@OUTPUT@'],
- build_by_default: false)
- endforeach
- alias_target('pdf', pdfs)
- endif
-
- texi2pod = find_program('scripts/texi2pod.pl')
- pod2man = find_program('pod2man', required: build_docs)
-
- if pod2man.found()
- foreach doc, input: texi
- man = doc + '.7'
- pod = custom_target(man + '.pod',
- input: input,
- output: man + '.pod',
- command: [texi2pod,
- '-DVERSION="' + meson.project_version() + '"',
- '-DCONFDIR="' + config_host['qemu_confdir'] + '"',
- '@INPUT0@', '@OUTPUT@'])
- man = custom_target(man,
- input: pod,
- output: man,
- capture: true,
- install: true,
- install_dir: get_option('mandir') / 'man7',
- command: [pod2man, '--utf8', '--section=7', '--center=" "',
- '--release=" "', '@INPUT@'])
- endforeach
- endif
-endif
-
if host_machine.system() == 'windows'
nsis_cmd = [
find_program('scripts/nsis.py'),
diff --git a/qapi/meson.build b/qapi/meson.build
index 2b2872a41d8..a287ca5d9d7 100644
--- a/qapi/meson.build
+++ b/qapi/meson.build
@@ -97,7 +97,7 @@ foreach module : qapi_all_modules
endforeach
qapi_files = custom_target('shared QAPI source files',
- output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs + ['qapi-doc.texi'],
+ output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs,
input: [ files('qapi-schema.json') ],
command: [ qapi_gen, '-o', 'qapi', '-b', '@INPUT0@' ],
depend_files: [ qapi_inputs, qapi_gen_depends ])
@@ -121,5 +121,3 @@ foreach output : qapi_specific_outputs + qapi_nonmodule_outputs
specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: qapi_files[i])
i = i + 1
endforeach
-
-qapi_doc_texi = qapi_files[i]
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 10/21] qapi: Use rST markup for literal blocks
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (8 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref " Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-25 16:23 ` [PATCH v6 11/21] qga/qapi-schema.json: Add some headings Peter Maydell
` (15 subsequent siblings)
25 siblings, 0 replies; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
There are exactly two places in our json doc comments where we
use the markup accepted by the texi doc generator where a '|' in
the first line of a doc comment means the line should be emitted
as a literal block (fixed-width font, whitespace preserved).
Since we use this syntax so rarely, instead of making the rST
generator support it, instead just convert the two uses to
rST-format literal blocks, which are indented and introduced
with '::'.
(The rST generator doesn't complain about the old style syntax,
it just emits it with the '|' and with the whitespace not
preserved, which looks odd, but means we can safely leave this
change until after we've stopped generating texinfo.)
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
qapi/block-core.json | 16 +++++++++-------
qapi/qapi-schema.json | 6 ++++--
2 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index dd77a91174c..86ed72ef9fa 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -570,13 +570,15 @@
# For the example above, @bins may be something like [3, 1, 5, 2],
# and corresponding histogram looks like:
#
-# | 5| *
-# | 4| *
-# | 3| * *
-# | 2| * * *
-# | 1| * * * *
-# | +------------------
-# | 10 50 100
+# ::
+#
+# 5| *
+# 4| *
+# 3| * *
+# 2| * * *
+# 1| * * * *
+# +------------------
+# 10 50 100
#
# Since: 4.0
##
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index f03ff91ceb5..6bafa81b727 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -21,8 +21,10 @@
#
# Example:
#
-# | -> data issued by the Client
-# | <- Server data response
+# ::
+#
+# -> data issued by the Client
+# <- Server data response
#
# Please, refer to the QMP specification (docs/interop/qmp-spec.txt) for
# detailed information on the Server command and response formats.
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 11/21] qga/qapi-schema.json: Add some headings
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (9 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 10/21] qapi: Use rST markup for literal blocks Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 8:30 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 12/21] tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis Peter Maydell
` (14 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
Add some section headings to the QGA json; this is purely so that we
have some H1 headings, as otherwise each command ends up being
visible in the interop/ manual's table of contents. In an ideal
world there might be a proper 'Introduction' section the way there is
in qapi/qapi-schema.json.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
qga/qapi-schema.json | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
index d2ea7446db6..cec98c7e065 100644
--- a/qga/qapi-schema.json
+++ b/qga/qapi-schema.json
@@ -2,14 +2,16 @@
# vim: filetype=python
##
-#
-# General note concerning the use of guest agent interfaces:
+# = General note concerning the use of guest agent interfaces
#
# "unsupported" is a higher-level error than the errors that individual
# commands might document. The caller should always be prepared to receive
# QERR_UNSUPPORTED, even if the given command doesn't specify it, or doesn't
# document any failure mode at all.
-#
+##
+
+##
+# = QEMU guest agent protocol commands and structs
##
{ 'pragma': { 'doc-required': true } }
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 12/21] tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (10 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 11/21] qga/qapi-schema.json: Add some headings Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 8:33 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 13/21] meson.build: Move SPHINX_ARGS to top level meson.build file Peter Maydell
` (13 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
doc-good.json currently uses the old *strong* and _emphasis_ markup.
As part of the conversion to rST this needs to switch to **strong**
and *emphasis*, because rST uses underscores as part of its markup
of hyperlinks and will otherwise warn about the syntax error.
In commit a660eed482063b we fixed up the in-tree uses of the
old markup:
1) _this_ was replaced with *this*
2) the only in-tree use of *this* was left alone (turning
a 'strong' into an 'emphasis')
(and so currently in-tree nothing is using either new-style
**strong** or old-style _emphasis_).
Update doc-good.json in a similar way:
1) replace _this_ with *this*
2) remove the usage of old-style *this*
(This slightly reduces the coverage for the old Texinfo generator,
which is about to go away, but is fine for the new rST generator
because that does not need to handle strong/emphasis itself because
it is simply passing the entire text as raw rST to Sphinx.)
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
tests/qapi-schema/doc-good.json | 2 +-
tests/qapi-schema/doc-good.out | 2 +-
tests/qapi-schema/doc-good.texi | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index c6822145c49..e9af0857db7 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -10,7 +10,7 @@
#
# == Subsection
#
-# *strong* _with emphasis_
+# *with emphasis*
# @var {in braces}
#
# * List item one
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index b7e3f4313da..419284dae29 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -73,7 +73,7 @@ doc freeform
body=
== Subsection
-*strong* _with emphasis_
+*with emphasis*
@var {in braces}
* List item one
diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
index 12808989ffb..29abef33e98 100644
--- a/tests/qapi-schema/doc-good.texi
+++ b/tests/qapi-schema/doc-good.texi
@@ -4,7 +4,7 @@
@subsection Subsection
-@strong{strong} @emph{with emphasis}
+@strong{with emphasis}
@code{var} @{in braces@}
@itemize @bullet
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 13/21] meson.build: Move SPHINX_ARGS to top level meson.build file
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (11 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 12/21] tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 8:45 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 14/21] meson.build: Make manuals depend on source to Sphinx extensions Peter Maydell
` (12 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
We're going to want to use SPHINX_ARGS in both docs/meson.build
and tests/qapi-schema/meson.build. Move the definition up to the
top level file so it is available to both subdirectories.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/meson.build | 8 --------
meson.build | 8 ++++++++
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/docs/meson.build b/docs/meson.build
index 69097e2ca07..99da609e813 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -1,11 +1,3 @@
-SPHINX_ARGS = [config_host['SPHINX_BUILD'],
- '-Dversion=' + meson.project_version(),
- '-Drelease=' + config_host['PKGVERSION']]
-
-if get_option('werror')
- SPHINX_ARGS += [ '-W' ]
-endif
-
if build_docs
configure_file(output: 'index.html',
input: files('index.html.in'),
diff --git a/meson.build b/meson.build
index 73d675ca834..6408ad442ea 100644
--- a/meson.build
+++ b/meson.build
@@ -671,6 +671,14 @@ foreach d : hx_headers
endforeach
genh += hxdep
+SPHINX_ARGS = [config_host['SPHINX_BUILD'],
+ '-Dversion=' + meson.project_version(),
+ '-Drelease=' + config_host['PKGVERSION']]
+
+if get_option('werror')
+ SPHINX_ARGS += [ '-W' ]
+endif
+
# Collect sourcesets.
util_ss = ss.source_set()
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 14/21] meson.build: Make manuals depend on source to Sphinx extensions
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (12 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 13/21] meson.build: Move SPHINX_ARGS to top level meson.build file Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 8:52 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn Peter Maydell
` (11 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
The automatic dependency handling for Sphinx manuals only makes the output
depend on the input documentation files. This means that if you edit
the Python source of an extension then the documentation won't be
rebuilt.
Create a list of all the source files for the extensions and add
it to the dependencies for the manuals. This is similar to how we
handle the qapi_gen_depends list.
Because we don't try to identify which manuals are using which
Sphinx extensions, a change to the source of one extension will
always rebuild the entire manual set, not merely the manuals
which have changed. This is acceptable because we don't change
the extensions all that often.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/meson.build | 1 +
meson.build | 8 ++++++++
2 files changed, 9 insertions(+)
diff --git a/docs/meson.build b/docs/meson.build
index 99da609e813..59fea873b10 100644
--- a/docs/meson.build
+++ b/docs/meson.build
@@ -36,6 +36,7 @@ if build_docs
output: [manual + '.stamp'],
input: [files('conf.py'), files(manual / 'conf.py')],
depfile: manual + '.d',
+ depend_files: sphinx_extn_depends,
command: [SPHINX_ARGS, '-Ddepfile=@DEPFILE@',
'-Ddepfile_stamp=@OUTPUT0@',
'-b', 'html', '-d', private_dir,
diff --git a/meson.build b/meson.build
index 6408ad442ea..3c07e75dbd5 100644
--- a/meson.build
+++ b/meson.build
@@ -679,6 +679,14 @@ if get_option('werror')
SPHINX_ARGS += [ '-W' ]
endif
+sphinx_extn_depends = [ meson.source_root() / 'docs/sphinx/depfile.py',
+ meson.source_root() / 'docs/sphinx/hxtool.py',
+ meson.source_root() / 'docs/sphinx/kerneldoc.py',
+ meson.source_root() / 'docs/sphinx/kernellog.py',
+ meson.source_root() / 'docs/sphinx/qapidoc.py',
+ meson.source_root() / 'docs/sphinx/qmp_lexer.py',
+ qapi_gen_depends ]
+
# Collect sourcesets.
util_ss = ss.source_set()
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (13 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 14/21] meson.build: Make manuals depend on source to Sphinx extensions Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 12:20 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 16/21] scripts/qapi: Remove texinfo generation support Peter Maydell
` (10 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
Add a test of the rST output from the QAPI doc-comment generator,
similar to what we currently have that tests the Texinfo output.
This is a bit more awkward with Sphinx, because the generated
output is not 100% under our control the way the QAPI-to-Texinfo
generator was. However, in practice Sphinx's plaintext output
generation has been identical between at least Sphinx 1.6 and
3.0, so we use that. (The HTML output has had changes across
versions). We use an exact-match comparison check, with the
understanding that perhaps changes in a future Sphinx version
might require us to implement something more clever to cope
with variation in the output.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
The meson.build changes are remarkably clunky, but this
appears to be unavoidable...
---
tests/qapi-schema/doc-good.rst | 5 +
tests/qapi-schema/doc-good.txt | 288 +++++++++++++++++++++++++++++++++
tests/qapi-schema/meson.build | 55 +++++++
3 files changed, 348 insertions(+)
create mode 100644 tests/qapi-schema/doc-good.rst
create mode 100644 tests/qapi-schema/doc-good.txt
diff --git a/tests/qapi-schema/doc-good.rst b/tests/qapi-schema/doc-good.rst
new file mode 100644
index 00000000000..1e4c23305a1
--- /dev/null
+++ b/tests/qapi-schema/doc-good.rst
@@ -0,0 +1,5 @@
+..
+ Test Sphinx manual that pulls in the test schema file. We will generate
+ a plain-text output file and compare it against a reference.
+
+.. qapi-doc:: tests/qapi-schema/doc-good.json
diff --git a/tests/qapi-schema/doc-good.txt b/tests/qapi-schema/doc-good.txt
new file mode 100644
index 00000000000..6ca03d49d0d
--- /dev/null
+++ b/tests/qapi-schema/doc-good.txt
@@ -0,0 +1,288 @@
+Section
+*******
+
+
+Subsection
+==========
+
+*with emphasis* "var" {in braces}
+
+* List item one
+
+* Two, multiple lines
+
+* Three Still in list
+
+Not in list
+
+* Second list Note: still in list
+
+Note: not in list
+
+1. Third list is numbered
+
+2. another item
+
+Returns: the King Since: the first age Notes:
+
+1. Lorem ipsum dolor sit amet
+
+2. Ut enim ad minim veniam
+
+Duis aute irure dolor
+
+Example:
+
+-> in <- out Examples: - *verbatim* - {braces}
+
+
+"Enum" (Enum)
+-------------
+
+
+Values
+~~~~~~
+
+"one" (**If: **"defined(IFONE)")
+ The _one_ {and only}
+
+"two"
+ Not documented
+
+
+Features
+~~~~~~~~
+
+"enum-feat"
+ Also _one_ {and only}
+
+"two" is undocumented
+
+
+If
+~~
+
+"defined(IFCOND)"
+
+
+"Base" (Object)
+---------------
+
+
+Members
+~~~~~~~
+
+"base1": "Enum"
+ the first member
+
+
+"Variant1" (Object)
+-------------------
+
+A paragraph
+
+Another paragraph (but no "var": line)
+
+
+Members
+~~~~~~~
+
+"var1": "string" (**If: **"defined(IFSTR)")
+ Not documented
+
+
+Features
+~~~~~~~~
+
+"variant1-feat"
+ a feature
+
+"member-feat"
+ a member feature
+
+
+"Variant2" (Object)
+-------------------
+
+
+"Object" (Object)
+-----------------
+
+
+Members
+~~~~~~~
+
+The members of "Base"
+The members of "Variant1" when "base1" is ""one""
+The members of "Variant2" when "base1" is ""two"" (**If: **"IFTWO")
+
+Features
+~~~~~~~~
+
+"union-feat1"
+ a feature
+
+
+"SugaredUnion" (Object)
+-----------------------
+
+
+Members
+~~~~~~~
+
+"type"
+ One of "one", "two"
+
+"data": "Variant1" when "type" is ""one""
+"data": "Variant2" when "type" is ""two"" (**If: **"IFTWO")
+
+Features
+~~~~~~~~
+
+"union-feat2"
+ a feature
+
+
+"Alternate" (Alternate)
+-----------------------
+
+
+Members
+~~~~~~~
+
+"i": "int"
+ an integer "b" is undocumented
+
+"b": "boolean"
+ Not documented
+
+
+Features
+~~~~~~~~
+
+"alt-feat"
+ a feature
+
+
+Another subsection
+==================
+
+
+"cmd" (Command)
+---------------
+
+
+Arguments
+~~~~~~~~~
+
+"arg1": "int"
+ the first argument
+
+"arg2": "string" (optional)
+ the second argument
+
+"arg3": "boolean"
+ Not documented
+
+
+Features
+~~~~~~~~
+
+"cmd-feat1"
+ a feature
+
+"cmd-feat2"
+ another feature
+
+
+Note
+~~~~
+
+"arg3" is undocumented
+
+
+Returns
+~~~~~~~
+
+"Object"
+
+
+TODO
+~~~~
+
+frobnicate
+
+
+Notes
+~~~~~
+
+* Lorem ipsum dolor sit amet
+
+* Ut enim ad minim veniam
+
+Duis aute irure dolor
+
+
+Example
+~~~~~~~
+
+ -> in
+ <- out
+
+
+Examples
+~~~~~~~~
+
+ - *verbatim*
+ - {braces}
+
+
+Since
+~~~~~
+
+2.10
+
+
+"cmd-boxed" (Command)
+---------------------
+
+If you're bored enough to read this, go see a video of boxed cats
+
+
+Arguments
+~~~~~~~~~
+
+The members of "Object"
+
+Features
+~~~~~~~~
+
+"cmd-feat1"
+ a feature
+
+"cmd-feat2"
+ another feature
+
+
+Example
+~~~~~~~
+
+ -> in
+
+ <- out
+
+
+"EVT-BOXED" (Event)
+-------------------
+
+
+Arguments
+~~~~~~~~~
+
+The members of "Object"
+
+Features
+~~~~~~~~
+
+"feat3"
+ a feature
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index 83a0a68389b..0c4a6a2936f 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -224,3 +224,58 @@ qapi_doc = custom_target('QAPI doc',
test('QAPI doc', diff, args: ['-b', '-u', files('doc-good.texi'), qapi_doc[0].full_path()],
depends: qapi_doc,
suite: ['qapi-schema', 'qapi-doc'])
+
+# Test the document-comment document generation code by running a test schema
+# file through Sphinx's plain-text builder and comparing the result against
+# a golden reference. This is in theory susceptible to failures if Sphinx
+# changes its output, but the text output has historically been very stable
+# (no changes between Sphinx 1.6 and 3.0), so it is a better bet than
+# texinfo or HTML generation, both of which have had changes. We might
+# need to add more sophisticated logic here in future for some sort of
+# fuzzy comparison if future Sphinx versions produce different text,
+# but for now the simple comparison suffices.
+qapi_doc_out = custom_target('QAPI rST doc',
+ output: ['doc-good.txt'],
+ input: files('doc-good.json', 'doc-good.rst'),
+ build_by_default: build_docs,
+ depend_files: sphinx_extn_depends,
+ # We use -E to suppress Sphinx's caching, because
+ # we want it to always really run the QAPI doc
+ # generation code. It also means we don't
+ # clutter up the build dir with the cache.
+ command: [SPHINX_ARGS,
+ '-b', 'text', '-E',
+ '-c', meson.source_root() / 'docs',
+ '-D', 'master_doc=doc-good',
+ meson.current_source_dir(),
+ meson.current_build_dir()])
+
+# Fix possible inconsistency in line endings in generated output and
+# in the golden reference (which could otherwise cause test failures
+# on Windows hosts). Unfortunately diff --strip-trailing-cr
+# is GNU-diff only. The odd-looking perl is because we must avoid
+# using an explicit '\' character in the command arguments to
+# a custom_target(), as Meson will unhelpfully replace it with a '/'
+# (https://github.com/mesonbuild/meson/issues/1564)
+qapi_doc_out_nocr = custom_target('QAPI rST doc newline-sanitized',
+ output: ['doc-good.txt.nocr'],
+ input: qapi_doc_out[0],
+ build_by_default: build_docs,
+ command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
+ capture: true)
+
+qapi_doc_ref_nocr = custom_target('QAPI rST doc reference newline-sanitized',
+ output: ['doc-good.ref.nocr'],
+ input: files('doc-good.txt'),
+ build_by_default: build_docs,
+ command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
+ capture: true)
+
+if build_docs
+ # "full_path()" needed here to work around
+ # https://github.com/mesonbuild/meson/issues/7585
+ test('QAPI rST doc', diff, args: ['-u', qapi_doc_ref_nocr[0].full_path(),
+ qapi_doc_out_nocr[0].full_path()],
+ depends: [qapi_doc_ref_nocr, qapi_doc_out_nocr],
+ suite: ['qapi-schema', 'qapi-doc'])
+endif
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 16/21] scripts/qapi: Remove texinfo generation support
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (14 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 12:22 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions Peter Maydell
` (9 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
We no longer use the generated texinfo format documentation,
so delete the code that generates it, and the test case for
the generation.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
meson.build | 2 -
scripts/qapi-gen.py | 2 -
scripts/qapi/doc.py | 302 ------------------------------
scripts/qapi/gen.py | 7 -
tests/qapi-schema/doc-good.texi | 313 --------------------------------
tests/qapi-schema/meson.build | 9 +-
6 files changed, 1 insertion(+), 634 deletions(-)
delete mode 100644 scripts/qapi/doc.py
delete mode 100644 tests/qapi-schema/doc-good.texi
diff --git a/meson.build b/meson.build
index 3c07e75dbd5..339149759c7 100644
--- a/meson.build
+++ b/meson.build
@@ -618,7 +618,6 @@ qapi_gen = find_program('scripts/qapi-gen.py')
qapi_gen_depends = [ meson.source_root() / 'scripts/qapi/__init__.py',
meson.source_root() / 'scripts/qapi/commands.py',
meson.source_root() / 'scripts/qapi/common.py',
- meson.source_root() / 'scripts/qapi/doc.py',
meson.source_root() / 'scripts/qapi/error.py',
meson.source_root() / 'scripts/qapi/events.py',
meson.source_root() / 'scripts/qapi/expr.py',
@@ -630,7 +629,6 @@ qapi_gen_depends = [ meson.source_root() / 'scripts/qapi/__init__.py',
meson.source_root() / 'scripts/qapi/types.py',
meson.source_root() / 'scripts/qapi/visit.py',
meson.source_root() / 'scripts/qapi/common.py',
- meson.source_root() / 'scripts/qapi/doc.py',
meson.source_root() / 'scripts/qapi-gen.py'
]
diff --git a/scripts/qapi-gen.py b/scripts/qapi-gen.py
index 4b03f7d53be..541e8c1f55d 100644
--- a/scripts/qapi-gen.py
+++ b/scripts/qapi-gen.py
@@ -10,7 +10,6 @@ import re
import sys
from qapi.commands import gen_commands
-from qapi.doc import gen_doc
from qapi.events import gen_events
from qapi.introspect import gen_introspect
from qapi.schema import QAPIError, QAPISchema
@@ -51,7 +50,6 @@ def main(argv):
gen_commands(schema, args.output_dir, args.prefix)
gen_events(schema, args.output_dir, args.prefix)
gen_introspect(schema, args.output_dir, args.prefix, args.unmask)
- gen_doc(schema, args.output_dir, args.prefix)
if __name__ == '__main__':
diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
deleted file mode 100644
index 7764de1e4bc..00000000000
--- a/scripts/qapi/doc.py
+++ /dev/null
@@ -1,302 +0,0 @@
-# QAPI texi generator
-#
-# This work is licensed under the terms of the GNU LGPL, version 2+.
-# See the COPYING file in the top-level directory.
-"""This script produces the documentation of a qapi schema in texinfo format"""
-
-import re
-from qapi.gen import QAPIGenDoc, QAPISchemaVisitor
-
-
-MSG_FMT = """
-@deftypefn {type} {{}} {name}
-
-{body}{members}{features}{sections}
-@end deftypefn
-
-""".format
-
-TYPE_FMT = """
-@deftp {{{type}}} {name}
-
-{body}{members}{features}{sections}
-@end deftp
-
-""".format
-
-EXAMPLE_FMT = """@example
-{code}
-@end example
-""".format
-
-
-def subst_strong(doc):
- """Replaces *foo* by @strong{foo}"""
- return re.sub(r'\*([^*\n]+)\*', r'@strong{\1}', doc)
-
-
-def subst_emph(doc):
- """Replaces _foo_ by @emph{foo}"""
- return re.sub(r'\b_([^_\n]+)_\b', r'@emph{\1}', doc)
-
-
-def subst_vars(doc):
- """Replaces @var by @code{var}"""
- return re.sub(r'@([\w-]+)', r'@code{\1}', doc)
-
-
-def subst_braces(doc):
- """Replaces {} with @{ @}"""
- return doc.replace('{', '@{').replace('}', '@}')
-
-
-def texi_example(doc):
- """Format @example"""
- # TODO: Neglects to escape @ characters.
- # We should probably escape them in subst_braces(), and rename the
- # function to subst_special() or subs_texi_special(). If we do that, we
- # need to delay it until after subst_vars() in texi_format().
- doc = subst_braces(doc).strip('\n')
- return EXAMPLE_FMT(code=doc)
-
-
-def texi_format(doc):
- """
- Format documentation
-
- Lines starting with:
- - |: generates an @example
- - =: generates @section
- - ==: generates @subsection
- - 1. or 1): generates an @enumerate @item
- - */-: generates an @itemize list
- """
- ret = ''
- doc = subst_braces(doc)
- doc = subst_vars(doc)
- doc = subst_emph(doc)
- doc = subst_strong(doc)
- inlist = ''
- lastempty = False
- for line in doc.split('\n'):
- line = line.strip()
- empty = line == ''
-
- # FIXME: Doing this in a single if / elif chain is
- # problematic. For instance, a line without markup terminates
- # a list if it follows a blank line (reaches the final elif),
- # but a line with some *other* markup, such as a = title
- # doesn't.
- #
- # Make sure to update section "Documentation markup" in
- # docs/devel/qapi-code-gen.txt when fixing this.
- if line.startswith('| '):
- line = EXAMPLE_FMT(code=line[2:])
- elif line.startswith('= '):
- line = '@section ' + line[2:]
- elif line.startswith('== '):
- line = '@subsection ' + line[3:]
- elif re.match(r'^([0-9]*\.) ', line):
- if not inlist:
- ret += '@enumerate\n'
- inlist = 'enumerate'
- ret += '@item\n'
- line = line[line.find(' ')+1:]
- elif re.match(r'^[*-] ', line):
- if not inlist:
- ret += '@itemize %s\n' % {'*': '@bullet',
- '-': '@minus'}[line[0]]
- inlist = 'itemize'
- ret += '@item\n'
- line = line[2:]
- elif lastempty and inlist:
- ret += '@end %s\n\n' % inlist
- inlist = ''
-
- lastempty = empty
- ret += line + '\n'
-
- if inlist:
- ret += '@end %s\n\n' % inlist
- return ret
-
-
-def texi_body(doc):
- """Format the main documentation body"""
- return texi_format(doc.body.text)
-
-
-def texi_if(ifcond, prefix='\n', suffix='\n'):
- """Format the #if condition"""
- if not ifcond:
- return ''
- return '%s@b{If:} @code{%s}%s' % (prefix, ', '.join(ifcond), suffix)
-
-
-def texi_enum_value(value, desc, suffix):
- """Format a table of members item for an enumeration value"""
- return '@item @code{%s}\n%s%s' % (
- value.name, desc, texi_if(value.ifcond, prefix='@*'))
-
-
-def texi_member(member, desc, suffix):
- """Format a table of members item for an object type member"""
- typ = member.type.doc_type()
- membertype = ': ' + typ if typ else ''
- return '@item @code{%s%s}%s%s\n%s%s' % (
- member.name, membertype,
- ' (optional)' if member.optional else '',
- suffix, desc, texi_if(member.ifcond, prefix='@*'))
-
-
-def texi_members(doc, what, base=None, variants=None,
- member_func=texi_member):
- """Format the table of members"""
- items = ''
- for section in doc.args.values():
- # TODO Drop fallbacks when undocumented members are outlawed
- if section.text:
- desc = texi_format(section.text)
- elif (variants and variants.tag_member == section.member
- and not section.member.type.doc_type()):
- values = section.member.type.member_names()
- members_text = ', '.join(['@t{"%s"}' % v for v in values])
- desc = 'One of ' + members_text + '\n'
- else:
- desc = 'Not documented\n'
- items += member_func(section.member, desc, suffix='')
- if base:
- items += '@item The members of @code{%s}\n' % base.doc_type()
- if variants:
- for v in variants.variants:
- when = ' when @code{%s} is @t{"%s"}%s' % (
- variants.tag_member.name, v.name, texi_if(v.ifcond, " (", ")"))
- if v.type.is_implicit():
- assert not v.type.base and not v.type.variants
- for m in v.type.local_members:
- items += member_func(m, desc='', suffix=when)
- else:
- items += '@item The members of @code{%s}%s\n' % (
- v.type.doc_type(), when)
- if not items:
- return ''
- return '\n@b{%s:}\n@table @asis\n%s@end table\n' % (what, items)
-
-
-def texi_arguments(doc, boxed_arg_type):
- if boxed_arg_type:
- assert not doc.args
- return ('\n@b{Arguments:} the members of @code{%s}\n'
- % boxed_arg_type.name)
- return texi_members(doc, 'Arguments')
-
-
-def texi_features(doc):
- """Format the table of features"""
- items = ''
- for section in doc.features.values():
- desc = texi_format(section.text)
- items += '@item @code{%s}\n%s' % (section.name, desc)
- if not items:
- return ''
- return '\n@b{Features:}\n@table @asis\n%s@end table\n' % (items)
-
-
-def texi_sections(doc, ifcond):
- """Format additional sections following arguments"""
- body = ''
- for section in doc.sections:
- if section.name:
- # prefer @b over @strong, so txt doesn't translate it to *Foo:*
- body += '\n@b{%s:}\n' % section.name
- if section.name and section.name.startswith('Example'):
- body += texi_example(section.text)
- else:
- body += texi_format(section.text)
- body += texi_if(ifcond, suffix='')
- return body
-
-
-def texi_type(typ, doc, ifcond, members):
- return TYPE_FMT(type=typ,
- name=doc.symbol,
- body=texi_body(doc),
- members=members,
- features=texi_features(doc),
- sections=texi_sections(doc, ifcond))
-
-
-def texi_msg(typ, doc, ifcond, members):
- return MSG_FMT(type=typ,
- name=doc.symbol,
- body=texi_body(doc),
- members=members,
- features=texi_features(doc),
- sections=texi_sections(doc, ifcond))
-
-
-class QAPISchemaGenDocVisitor(QAPISchemaVisitor):
- def __init__(self, prefix):
- self._prefix = prefix
- self._gen = QAPIGenDoc(self._prefix + 'qapi-doc.texi')
- self.cur_doc = None
-
- def write(self, output_dir):
- self._gen.write(output_dir)
-
- def visit_enum_type(self, name, info, ifcond, features, members, prefix):
- doc = self.cur_doc
- self._gen.add(texi_type('Enum', doc, ifcond,
- texi_members(doc, 'Values',
- member_func=texi_enum_value)))
-
- def visit_object_type(self, name, info, ifcond, features,
- base, members, variants):
- doc = self.cur_doc
- if base and base.is_implicit():
- base = None
- self._gen.add(texi_type('Object', doc, ifcond,
- texi_members(doc, 'Members', base, variants)))
-
- def visit_alternate_type(self, name, info, ifcond, features, variants):
- doc = self.cur_doc
- self._gen.add(texi_type('Alternate', doc, ifcond,
- texi_members(doc, 'Members')))
-
- def visit_command(self, name, info, ifcond, features,
- arg_type, ret_type, gen, success_response, boxed,
- allow_oob, allow_preconfig):
- doc = self.cur_doc
- self._gen.add(texi_msg('Command', doc, ifcond,
- texi_arguments(doc,
- arg_type if boxed else None)))
-
- def visit_event(self, name, info, ifcond, features, arg_type, boxed):
- doc = self.cur_doc
- self._gen.add(texi_msg('Event', doc, ifcond,
- texi_arguments(doc,
- arg_type if boxed else None)))
-
- def symbol(self, doc, entity):
- if self._gen._body:
- self._gen.add('\n')
- self.cur_doc = doc
- entity.visit(self)
- self.cur_doc = None
-
- def freeform(self, doc):
- assert not doc.args
- if self._gen._body:
- self._gen.add('\n')
- self._gen.add(texi_body(doc) + texi_sections(doc, None))
-
-
-def gen_doc(schema, output_dir, prefix):
- vis = QAPISchemaGenDocVisitor(prefix)
- vis.visit_begin(schema)
- for doc in schema.docs:
- if doc.symbol:
- vis.symbol(doc, schema.lookup_entity(doc.symbol))
- else:
- vis.freeform(doc)
- vis.write(output_dir)
diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
index bf5552a4e7f..ca66c82b5b8 100644
--- a/scripts/qapi/gen.py
+++ b/scripts/qapi/gen.py
@@ -178,13 +178,6 @@ def ifcontext(ifcond, *args):
arg.end_if()
-class QAPIGenDoc(QAPIGen):
-
- def _top(self):
- return (super()._top()
- + '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n\n')
-
-
class QAPISchemaMonolithicCVisitor(QAPISchemaVisitor):
def __init__(self, prefix, what, blurb, pydoc):
diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
deleted file mode 100644
index 29abef33e98..00000000000
--- a/tests/qapi-schema/doc-good.texi
+++ /dev/null
@@ -1,313 +0,0 @@
-@c AUTOMATICALLY GENERATED, DO NOT MODIFY
-
-@section Section
-
-@subsection Subsection
-
-@strong{with emphasis}
-@code{var} @{in braces@}
-
-@itemize @bullet
-@item
-List item one
-@item
-Two, multiple
-lines
-
-@item
-Three
-Still in list
-
-@end itemize
-
-Not in list
-
-@itemize @minus
-@item
-Second list
-Note: still in list
-
-@end itemize
-
-Note: not in list
-
-@enumerate
-@item
-Third list
-is numbered
-
-@item
-another item
-
-@end enumerate
-
-Returns: the King
-Since: the first age
-Notes:
-
-@enumerate
-@item
-Lorem ipsum dolor sit amet
-
-@item
-Ut enim ad minim veniam
-
-@end enumerate
-
-Duis aute irure dolor
-
-Example:
-
--> in
-<- out
-Examples:
-@itemize @minus
-@item
-@strong{verbatim}
-@item
-@{braces@}
-@end itemize
-
-
-
-@deftp {Enum} Enum
-
-
-
-@b{Values:}
-@table @asis
-@item @code{one}
-The @emph{one} @{and only@}
-@*@b{If:} @code{defined(IFONE)}
-@item @code{two}
-Not documented
-@end table
-
-@b{Features:}
-@table @asis
-@item @code{enum-feat}
-Also @emph{one} @{and only@}
-@end table
-@code{two} is undocumented
-
-@b{If:} @code{defined(IFCOND)}
-@end deftp
-
-
-
-@deftp {Object} Base
-
-
-
-@b{Members:}
-@table @asis
-@item @code{base1: Enum}
-the first member
-@end table
-
-@end deftp
-
-
-
-@deftp {Object} Variant1
-
-A paragraph
-
-Another paragraph (but no @code{var}: line)
-
-@b{Members:}
-@table @asis
-@item @code{var1: string}
-Not documented
-@*@b{If:} @code{defined(IFSTR)}
-@end table
-
-@b{Features:}
-@table @asis
-@item @code{variant1-feat}
-a feature
-@item @code{member-feat}
-a member feature
-@end table
-
-@end deftp
-
-
-
-@deftp {Object} Variant2
-
-
-
-@end deftp
-
-
-
-@deftp {Object} Object
-
-
-
-@b{Members:}
-@table @asis
-@item The members of @code{Base}
-@item The members of @code{Variant1} when @code{base1} is @t{"one"}
-@item The members of @code{Variant2} when @code{base1} is @t{"two"} (@b{If:} @code{IFTWO})
-@end table
-
-@b{Features:}
-@table @asis
-@item @code{union-feat1}
-a feature
-@end table
-
-@end deftp
-
-
-
-@deftp {Object} SugaredUnion
-
-
-
-@b{Members:}
-@table @asis
-@item @code{type}
-One of @t{"one"}, @t{"two"}
-@item @code{data: Variant1} when @code{type} is @t{"one"}
-@item @code{data: Variant2} when @code{type} is @t{"two"} (@b{If:} @code{IFTWO})
-@end table
-
-@b{Features:}
-@table @asis
-@item @code{union-feat2}
-a feature
-@end table
-
-@end deftp
-
-
-
-@deftp {Alternate} Alternate
-
-
-
-@b{Members:}
-@table @asis
-@item @code{i: int}
-an integer
-@code{b} is undocumented
-@item @code{b: boolean}
-Not documented
-@end table
-
-@b{Features:}
-@table @asis
-@item @code{alt-feat}
-a feature
-@end table
-
-@end deftp
-
-
-@subsection Another subsection
-
-
-@deftypefn Command {} cmd
-
-
-
-@b{Arguments:}
-@table @asis
-@item @code{arg1: int}
-the first argument
-@item @code{arg2: string} (optional)
-the second
-argument
-@item @code{arg3: boolean}
-Not documented
-@end table
-
-@b{Features:}
-@table @asis
-@item @code{cmd-feat1}
-a feature
-@item @code{cmd-feat2}
-another feature
-@end table
-
-@b{Note:}
-@code{arg3} is undocumented
-
-@b{Returns:}
-@code{Object}
-
-@b{TODO:}
-frobnicate
-
-@b{Notes:}
-@itemize @minus
-@item
-Lorem ipsum dolor sit amet
-@item
-Ut enim ad minim veniam
-
-@end itemize
-
-Duis aute irure dolor
-
-@b{Example:}
-@example
--> in
-<- out
-@end example
-
-@b{Examples:}
-@example
-- *verbatim*
-- @{braces@}
-@end example
-
-@b{Since:}
-2.10
-
-@end deftypefn
-
-
-
-@deftypefn Command {} cmd-boxed
-
-If you're bored enough to read this, go see a video of boxed cats
-
-@b{Arguments:} the members of @code{Object}
-
-@b{Features:}
-@table @asis
-@item @code{cmd-feat1}
-a feature
-@item @code{cmd-feat2}
-another feature
-@end table
-
-@b{Example:}
-@example
--> in
-
-<- out
-@end example
-
-@end deftypefn
-
-
-
-@deftypefn Event {} EVT-BOXED
-
-
-
-@b{Arguments:} the members of @code{Object}
-
-@b{Features:}
-@table @asis
-@item @code{feat3}
-a feature
-@end table
-
-@end deftypefn
-
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index 0c4a6a2936f..f08c902911e 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -206,8 +206,7 @@ test('QAPI schema regression tests', python, args: files('test-qapi.py', schemas
diff = find_program('diff')
qapi_doc = custom_target('QAPI doc',
- output: ['doc-good-qapi-doc.texi',
- 'doc-good-qapi-commands.c', 'doc-good-qapi-commands.h',
+ output: ['doc-good-qapi-commands.c', 'doc-good-qapi-commands.h',
'doc-good-qapi-emit-events.c', 'doc-good-qapi-emit-events.h',
'doc-good-qapi-events.c', 'doc-good-qapi-events.h',
'doc-good-qapi-init-commands.c', 'doc-good-qapi-init-commands.h',
@@ -219,12 +218,6 @@ qapi_doc = custom_target('QAPI doc',
'-p', 'doc-good-', '@INPUT0@' ],
depend_files: qapi_gen_depends)
-# "full_path()" needed here to work around
-# https://github.com/mesonbuild/meson/issues/7585
-test('QAPI doc', diff, args: ['-b', '-u', files('doc-good.texi'), qapi_doc[0].full_path()],
- depends: qapi_doc,
- suite: ['qapi-schema', 'qapi-doc'])
-
# Test the document-comment document generation code by running a test schema
# file through Sphinx's plain-text builder and comparing the result against
# a golden reference. This is in theory susceptible to failures if Sphinx
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (15 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 16/21] scripts/qapi: Remove texinfo generation support Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 12:35 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 18/21] scripts/texi2pod: Delete unused script Peter Maydell
` (8 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
Update the documentation of QAPI document comment syntax to match
the new rST backend requirements. The principal changes are:
* whitespace is now significant, and multiline definitions
must have their second and subsequent lines indented to
match the first line
* general rST format markup is permitted, not just the small
set of markup the old texinfo generator handled. For most
things (notably bulleted and itemized lists) the old format
is the same as rST was.
* Specific things that might trip people up:
- instead of *bold* and _italic_ rST has **bold** and *italic*
- lists need a preceding and following blank line
- a lone literal '*' will need to be backslash-escaped to
avoid a rST syntax error
* the old leading '|' for example (literal text) blocks is
replaced by the standard rST '::' literal block.
* we support arbitrary levels of sub- and sub-sub-heading, not
just a main and sub-heading like the old texinfo generator
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/devel/qapi-code-gen.txt | 83 +++++++++++++++++++++++-------------
1 file changed, 53 insertions(+), 30 deletions(-)
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 69eaffac376..7d2479dde82 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -824,23 +824,37 @@ See below for more on definition documentation.
Free-form documentation may be used to provide additional text and
structuring content.
+==== Headings and subheadings ====
+
+A free-form documentation comment containing a line which starts with
+some '=' symbols and then a space defines a section heading:
+
+ ##
+ # = This is a top level heading
+ #
+ # This is a free-form comment which will go under the
+ # top level heading.
+ ##
+
+ ##
+ # == This is a second level heading
+ ##
+
+A heading line must be the first line of the documentation
+comment block.
+
+Section headings must always be correctly nested, so you can only
+define a third-level heading inside a second-level heading, and so on.
==== Documentation markup ====
-Comment text starting with '=' is a section title:
+Documentation comments can use most rST markup. In particular,
+a '::' literal block can be used for examples:
- # = Section title
-
-Double the '=' for a subsection title:
-
- # == Subsection title
-
-Both are only permitted in free-form documentation.
-
-'|' denotes examples:
-
- # | Text of the example, may span
- # | multiple lines
+ # ::
+ #
+ # Text of the example, may span
+ # multiple lines
'*' starts an itemized list:
@@ -856,34 +870,33 @@ A decimal number followed by '.' starts a numbered list:
# multiple lines
# 2. Second item
-The actual number doesn't matter. You could even use '*' instead of
-'2.' for the second item.
+The actual number doesn't matter.
-Lists can't be nested. Blank lines are currently not supported within
-lists.
+Lists of either kind must be preceded and followed by a blank line.
+If a list item's text spans multiple lines, then the second and
+subsequent lines must be correctly indented to line up with the
+first character of the first line.
-Additional whitespace between the initial '#' and the comment text is
-permitted.
-
-*foo* and _foo_ are for strong and emphasis styles respectively (they
-do not work over multiple lines). @foo is used to reference a name in
-the schema.
+The usual '**strong**', '*emphasised*' and '``literal``' markup should
+be used. If you need a single literal '*' you will need to
+backslash-escape it. As an extension beyond the usual rST syntax, you
+can also use '@foo' to reference a name in the schema; this is
+rendered the same way as '``foo``'.
Example:
##
-# = Section
-# == Subsection
-#
-# Some text foo with *strong* and _emphasis_
+# Some text foo with **bold** and *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
-# | $ echo foo
-# | -> do this
-# | <- get that
#
+# ::
+#
+# $ echo foo
+# -> do this
+# <- get that
##
@@ -937,6 +950,16 @@ multiline argument descriptions.
A 'Since: x.y.z' tagged section lists the release that introduced the
definition.
+The text of a section can start on a new line, in
+which case it must not be indented at all. It can also start
+on the same line as the 'Note:', 'Returns:', etc tag. In this
+case if it spans multiple lines then second and subsequent
+lines must be indented to match the first.
+
+An 'Example' or 'Examples' section is automatically rendered
+entirely as literal fixed-width text. In other sections,
+the text is formatted, and rST markup can be used.
+
For example:
##
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 18/21] scripts/texi2pod: Delete unused script
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (16 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 12:36 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 19/21] Remove Texinfo related line from git.orderfile Peter Maydell
` (7 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
We no longer need the texi2pod script, so we can delete it, and
the special-casing it had in the checkpatch script.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
scripts/checkpatch.pl | 2 +-
scripts/texi2pod.pl | 536 ------------------------------------------
2 files changed, 1 insertion(+), 537 deletions(-)
delete mode 100755 scripts/texi2pod.pl
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 50910899f2e..1ba8a3810b6 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -1659,7 +1659,7 @@ sub process {
# tabs are only allowed in assembly source code, and in
# some scripts we imported from other projects.
next if ($realfile =~ /\.(s|S)$/);
- next if ($realfile =~ /(checkpatch|get_maintainer|texi2pod)\.pl$/);
+ next if ($realfile =~ /(checkpatch|get_maintainer)\.pl$/);
if ($rawline =~ /^\+.*\t/) {
my $herevet = "$here\n" . cat_vet($rawline) . "\n";
diff --git a/scripts/texi2pod.pl b/scripts/texi2pod.pl
deleted file mode 100755
index 8bfc6f6f4c4..00000000000
--- a/scripts/texi2pod.pl
+++ /dev/null
@@ -1,536 +0,0 @@
-#! /usr/bin/env perl
-
-# Copyright (C) 1999, 2000, 2001, 2003 Free Software Foundation, Inc.
-
-# This file is part of GCC.
-
-# GCC is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-
-# GCC is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-
-# You should have received a copy of the GNU General Public License
-# along with GCC; see the file COPYING. If not,
-# see <http://www.gnu.org/licenses/>.
-
-# This does trivial (and I mean _trivial_) conversion of Texinfo
-# markup to Perl POD format. It's intended to be used to extract
-# something suitable for a manpage from a Texinfo document.
-
-use warnings;
-
-$output = 0;
-$skipping = 0;
-%sects = ();
-$section = "";
-@icstack = ();
-@endwstack = ();
-@skstack = ();
-@instack = ();
-$shift = "";
-%defs = ();
-$fnno = 1;
-$inf = "";
-$ibase = "";
-@ipath = ();
-$encoding = undef;
-@args = ();
-
-while ($_ = shift) {
- if (/^-D(.*)$/) {
- if ($1 ne "") {
- $flag = $1;
- } else {
- $flag = shift;
- }
- $value = "";
- ($flag, $value) = ($flag =~ /^([^=]+)(?:=(.+))?/);
- die "no flag specified for -D\n"
- unless $flag ne "";
- die "flags may only contain letters, digits, hyphens, dashes and underscores\n"
- unless $flag =~ /^[a-zA-Z0-9_-]+$/;
- $defs{$flag} = $value;
- } elsif (/^-I(.*)$/) {
- if ($1 ne "") {
- $flag = $1;
- } else {
- $flag = shift;
- }
- push (@ipath, $flag);
- } elsif (/^-/) {
- usage();
- } else {
- $in = $_, next unless defined $in;
- $out = $_, next unless defined $out;
- usage();
- }
-}
-
-if (defined $in) {
- $inf = gensym();
- open($inf, "<$in") or die "opening \"$in\": $!\n";
- $ibase = $1 if $in =~ m|^(.+)/[^/]+$|;
-} else {
- $inf = \*STDIN;
-}
-
-if (defined $out) {
- open(STDOUT, ">$out") or die "opening \"$out\": $!\n";
-}
-
-while(defined $inf) {
-while(<$inf>) {
- # Certain commands are discarded without further processing.
- /^\@(?:
- [a-z]+index # @*index: useful only in complete manual
- |need # @need: useful only in printed manual
- |(?:end\s+)?group # @group .. @end group: ditto
- |page # @page: ditto
- |node # @node: useful only in .info file
- |(?:end\s+)?ifnottex # @ifnottex .. @end ifnottex: use contents
- )\b/x and next;
-
- chomp;
-
- # Look for filename and title markers.
- /^\@setfilename\s+([^.]+)/ and $fn = $1, next;
- /^\@settitle\s+([^.]+)/ and $tl = postprocess($1), next;
-
- # Look for document encoding
- /^\@documentencoding\s+([^.]+)/ and do {
- $encoding = $1 unless defined $encoding;
- next;
- };
-
- # Identify a man title but keep only the one we are interested in.
- /^\@c\s+man\s+title\s+([A-Za-z0-9-]+)\s+(.+)/ and do {
- if (exists $defs{$1}) {
- $fn = $1;
- $tl = postprocess($2);
- }
- next;
- };
-
- # Look for blocks surrounded by @c man begin SECTION ... @c man end.
- # This really oughta be @ifman ... @end ifman and the like, but such
- # would require rev'ing all other Texinfo translators.
- /^\@c\s+man\s+begin\s+([A-Z]+)\s+([A-Za-z0-9-]+)/ and do {
- $output = 1 if exists $defs{$2};
- $sect = $1;
- next;
- };
- /^\@c\s+man\s+begin\s+([A-Z]+)/ and $sect = $1, $output = 1, next;
- /^\@c\s+man\s+end/ and do {
- $sects{$sect} = "" unless exists $sects{$sect};
- $sects{$sect} .= postprocess($section);
- $section = "";
- $output = 0;
- next;
- };
-
- # handle variables
- /^\@set\s+([a-zA-Z0-9_-]+)\s*(.*)$/ and do {
- $defs{$1} = $2;
- next;
- };
- /^\@clear\s+([a-zA-Z0-9_-]+)/ and do {
- delete $defs{$1};
- next;
- };
-
- # Single line command handlers.
-
- /^\@include\s+(.+)$/ and do {
- push @instack, $inf;
- $inf = gensym();
- $file = postprocess($1);
-
- # Try cwd and $ibase, then explicit -I paths.
- $done = 0;
- foreach $path ("", $ibase, @ipath) {
- $mypath = $file;
- $mypath = $path . "/" . $mypath if ($path ne "");
- open($inf, "<" . $mypath) and ($done = 1, last);
- }
- die "cannot find $file" if !$done;
- next;
- };
-
- next unless $output;
-
- # Discard comments. (Can't do it above, because then we'd never see
- # @c man lines.)
- /^\@c\b/ and next;
-
- # End-block handler goes up here because it needs to operate even
- # if we are skipping.
- /^\@end\s+([a-z]+)/ and do {
- # Ignore @end foo, where foo is not an operation which may
- # cause us to skip, if we are presently skipping.
- my $ended = $1;
- next if $skipping && $ended !~ /^(?:ifset|ifclear|ignore|menu|iftex|copying)$/;
-
- die "\@end $ended without \@$ended at line $.\n" unless defined $endw;
- die "\@$endw ended by \@end $ended at line $.\n" unless $ended eq $endw;
-
- $endw = pop @endwstack;
-
- if ($ended =~ /^(?:ifset|ifclear|ignore|menu|iftex)$/) {
- $skipping = pop @skstack;
- next;
- } elsif ($ended =~ /^(?:example|smallexample|display
- |quotation|deftp|deftypefn)$/x) {
- $shift = "";
- $_ = ""; # need a paragraph break
- } elsif ($ended =~ /^(?:itemize|enumerate|[fv]?table)$/) {
- $_ = "\n=back\n";
- $ic = pop @icstack;
- } elsif ($ended eq "multitable") {
- $_ = "\n=back\n";
- } else {
- die "unknown command \@end $ended at line $.\n";
- }
- };
-
- # We must handle commands which can cause skipping even while we
- # are skipping, otherwise we will not process nested conditionals
- # correctly.
- /^\@ifset\s+([a-zA-Z0-9_-]+)/ and do {
- push @endwstack, $endw;
- push @skstack, $skipping;
- $endw = "ifset";
- $skipping = 1 unless exists $defs{$1};
- next;
- };
-
- /^\@ifclear\s+([a-zA-Z0-9_-]+)/ and do {
- push @endwstack, $endw;
- push @skstack, $skipping;
- $endw = "ifclear";
- $skipping = 1 if exists $defs{$1};
- next;
- };
-
- /^\@(ignore|menu|iftex|copying)\b/ and do {
- push @endwstack, $endw;
- push @skstack, $skipping;
- $endw = $1;
- $skipping = 1;
- next;
- };
-
- next if $skipping;
-
- # Character entities. First the ones that can be replaced by raw text
- # or discarded outright:
- s/\@copyright\{\}/(c)/g;
- s/\@dots\{\}/.../g;
- s/\@enddots\{\}/..../g;
- s/\@([.!? ])/$1/g;
- s/\@[:-]//g;
- s/\@bullet(?:\{\})?/*/g;
- s/\@TeX\{\}/TeX/g;
- s/\@pounds\{\}/\#/g;
- s/\@minus(?:\{\})?/-/g;
- s/\\,/,/g;
-
- # Now the ones that have to be replaced by special escapes
- # (which will be turned back into text by unmunge())
- s/&/&/g;
- s/\@\{/{/g;
- s/\@\}/}/g;
- s/\@\@/&at;/g;
-
- # Inside a verbatim block, handle @var specially.
- if ($shift ne "") {
- s/\@var\{([^\}]*)\}/<$1>/g;
- }
-
- # POD doesn't interpret E<> inside a verbatim block.
- if ($shift eq "") {
- s/</</g;
- s/>/>/g;
- } else {
- s/</</g;
- s/>/>/g;
- }
-
- /^\@(?:section|unnumbered|unnumberedsec|center)\s+(.+)$/
- and $_ = "\n=head2 $1\n";
- /^\@subsection\s+(.+)$/
- and $_ = "\n=head3 $1\n";
- /^\@subsubsection\s+(.+)$/
- and $_ = "\n=head4 $1\n";
-
- # Block command handlers:
- /^\@itemize(?:\s+(\@[a-z]+|\*|-))?/ and do {
- push @endwstack, $endw;
- push @icstack, $ic;
- if (defined $1) {
- $ic = $1;
- } else {
- $ic = '*';
- }
- $_ = "\n=over 4\n";
- $endw = "itemize";
- };
-
- /^\@enumerate(?:\s+([a-zA-Z0-9]+))?/ and do {
- push @endwstack, $endw;
- push @icstack, $ic;
- if (defined $1) {
- $ic = $1 . ".";
- } else {
- $ic = "1.";
- }
- $_ = "\n=over 4\n";
- $endw = "enumerate";
- };
-
- /^\@multitable\s.*/ and do {
- push @endwstack, $endw;
- $endw = "multitable";
- $_ = "\n=over 4\n";
- };
-
- /^\@([fv]?table)\s+(\@[a-z]+)/ and do {
- push @endwstack, $endw;
- push @icstack, $ic;
- $endw = $1;
- $ic = $2;
- $ic =~ s/\@(?:samp|strong|key|gcctabopt|option|env)/B/;
- $ic =~ s/\@(?:code|kbd)/C/;
- $ic =~ s/\@(?:dfn|var|emph|cite|i)/I/;
- $ic =~ s/\@(?:file)/F/;
- $ic =~ s/\@(?:asis)//;
- $_ = "\n=over 4\n";
- };
-
- /^\@((?:small)?example|display)/ and do {
- push @endwstack, $endw;
- $endw = $1;
- $shift = "\t";
- $_ = ""; # need a paragraph break
- };
-
- /^\@item\s+(.*\S)\s*$/ and $endw eq "multitable" and do {
- @columns = ();
- for $column (split (/\s*\@tab\s*/, $1)) {
- # @strong{...} is used a @headitem work-alike
- $column =~ s/^\@strong\{(.*)\}$/$1/;
- push @columns, $column;
- }
- $_ = "\n=item ".join (" : ", @columns)."\n";
- };
-
- /^\@(quotation)\s*(.+)?$/ and do {
- push @endwstack, $endw;
- $endw = $1;
- $_ = "\n$2:"
- };
-
- /^{(.*)}$|^(.*)$/ and $#args > 0 and do {
- $kind = $args[0];
- $arguments = $1 // "";
- if ($endw eq "deftypefn") {
- $ret = $args[1];
- $fname = "B<$args[2]>";
- $_ = $ret ? "$ret " : "";
- $_ .= "$fname $arguments ($kind)";
- } else {
- $_ = "B<$args[1]> ($kind)\n\n$arguments";
- }
- @args = ();
- };
-
- /^\@(deftp)\s*(.+)?$/ and do {
- push @endwstack, $endw;
- $endw = $1;
- $arg = $2;
- $arg =~ s/{([^}]*)}/$1/g;
- $arg =~ s/\@$//;
- @args = split (/ /, $arg);
- $_ = "";
- };
-
- /^\@(deftypefn)\s*(.+)?$/ and do {
- push @endwstack, $endw;
- $endw = $1;
- $arg = $2;
- $arg =~ s/{([^}]*)}/$1/g;
- $arg =~ s/\@$//;
- @args = split (/ /, $arg);
- $_ = "";
- };
-
- /^\@itemx?\s*(.+)?$/ and do {
- if (defined $1) {
- if ($ic eq "") {
- $_ = "\n=item $1\n";
- } else {
- # Entity escapes prevent munging by the <> processing below.
- $_ = "\n=item $ic\<$1\>\n";
- }
- } else {
- $_ = "\n=item $ic\n";
- $ic =~ y/A-Ya-y/B-Zb-z/;
- $ic =~ s/(\d+)/$1 + 1/eg;
- }
- };
-
- $section .= $shift.$_."\n";
-}
-# End of current file.
-close($inf);
-$inf = pop @instack;
-}
-
-die "No filename or title\n" unless defined $fn && defined $tl;
-
-print "=encoding $encoding\n\n" if defined $encoding;
-
-$sects{NAME} = "$fn \- $tl\n";
-$sects{FOOTNOTES} .= "=back\n" if exists $sects{FOOTNOTES};
-
-for $sect (qw(NAME SYNOPSIS DESCRIPTION OPTIONS ENVIRONMENT FILES
- BUGS NOTES FOOTNOTES EXAMPLES SEEALSO AUTHOR COPYRIGHT)) {
- if(exists $sects{$sect}) {
- $head = $sect;
- $head =~ s/SEEALSO/SEE ALSO/;
- print "=head1 $head\n\n";
- print scalar unmunge ($sects{$sect});
- print "\n";
- }
-}
-
-sub usage
-{
- die "usage: $0 [-D toggle...] [infile [outfile]]\n";
-}
-
-sub postprocess
-{
- local $_ = $_[0];
-
- # @value{foo} is replaced by whatever 'foo' is defined as.
- while (m/(\@value\{([a-zA-Z0-9_-]+)\})/g) {
- if (! exists $defs{$2}) {
- print STDERR "Option $2 not defined\n";
- s/\Q$1\E//;
- } else {
- $value = $defs{$2};
- s/\Q$1\E/$value/;
- }
- }
-
- # Formatting commands.
- # Temporary escape for @r.
- s/\@r\{([^\}]*)\}/R<$1>/g;
- s/\@(?:dfn|var|emph|cite|i)\{([^\}]*)\}/I<$1>/g;
- s/\@(?:code|kbd)\{([^\}]*)\}/C<$1>/g;
- s/\@(?:gccoptlist|samp|strong|key|option|env|command|b)\{([^\}]*)\}/B<$1>/g;
- s/\@sc\{([^\}]*)\}/\U$1/g;
- s/\@file\{([^\}]*)\}/F<$1>/g;
- s/\@w\{([^\}]*)\}/S<$1>/g;
- s/\@t\{([^\}]*)\}/$1/g;
- s/\@(?:dmn|math)\{([^\}]*)\}/$1/g;
-
- # keep references of the form @ref{...}, print them bold
- s/\@(?:ref)\{([^\}]*)\}/B<$1>/g;
-
- # Change double single quotes to double quotes.
- s/''/"/g;
- s/``/"/g;
-
- # Cross references are thrown away, as are @noindent and @refill.
- # (@noindent is impossible in .pod, and @refill is unnecessary.)
- # @* is also impossible in .pod; we discard it and any newline that
- # follows it. Similarly, our macro @gol must be discarded.
-
- s/\(?\@xref\{(?:[^\}]*)\}(?:[^.<]|(?:<[^<>]*>))*\.\)?//g;
- s/\s+\(\@pxref\{(?:[^\}]*)\}\)//g;
- s/;\s+\@pxref\{(?:[^\}]*)\}//g;
- s/\@noindent\s*//g;
- s/\@refill//g;
- s/\@gol//g;
- s/\@\*\s*\n?//g;
-
- # Anchors are thrown away
- s/\@anchor\{(?:[^\}]*)\}//g;
-
- # @uref can take one, two, or three arguments, with different
- # semantics each time. @url and @email are just like @uref with
- # one argument, for our purposes.
- s/\@(?:uref|url|email)\{([^\},]*)\}/<B<$1>>/g;
- s/\@uref\{([^\},]*),([^\},]*)\}/$2 (C<$1>)/g;
- s/\@uref\{([^\},]*),([^\},]*),([^\},]*)\}/$3/g;
-
- # Un-escape <> at this point.
- s/</</g;
- s/>/>/g;
-
- # Now un-nest all B<>, I<>, R<>. Theoretically we could have
- # indefinitely deep nesting; in practice, one level suffices.
- 1 while s/([BIR])<([^<>]*)([BIR])<([^<>]*)>/$1<$2>$3<$4>$1</g;
-
- # Replace R<...> with bare ...; eliminate empty markup, B<>;
- # shift white space at the ends of [BI]<...> expressions outside
- # the expression.
- s/R<([^<>]*)>/$1/g;
- s/[BI]<>//g;
- s/([BI])<(\s+)([^>]+)>/$2$1<$3>/g;
- s/([BI])<([^>]+?)(\s+)>/$1<$2>$3/g;
-
- # Extract footnotes. This has to be done after all other
- # processing because otherwise the regexp will choke on formatting
- # inside @footnote.
- while (/\@footnote/g) {
- s/\@footnote\{([^\}]+)\}/[$fnno]/;
- add_footnote($1, $fnno);
- $fnno++;
- }
-
- return $_;
-}
-
-sub unmunge
-{
- # Replace escaped symbols with their equivalents.
- local $_ = $_[0];
-
- s/</E<lt>/g;
- s/>/E<gt>/g;
- s/{/\{/g;
- s/}/\}/g;
- s/&at;/\@/g;
- s/&/&/g;
- return $_;
-}
-
-sub add_footnote
-{
- unless (exists $sects{FOOTNOTES}) {
- $sects{FOOTNOTES} = "\n=over 4\n\n";
- }
-
- $sects{FOOTNOTES} .= "=item $fnno.\n\n"; $fnno++;
- $sects{FOOTNOTES} .= $_[0];
- $sects{FOOTNOTES} .= "\n\n";
-}
-
-# stolen from Symbol.pm
-{
- my $genseq = 0;
- sub gensym
- {
- my $name = "GEN" . $genseq++;
- my $ref = \*{$name};
- delete $::{$name};
- return $ref;
- }
-}
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 19/21] Remove Texinfo related line from git.orderfile
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (17 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 18/21] scripts/texi2pod: Delete unused script Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 12:37 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 20/21] configure: Drop texinfo requirement Peter Maydell
` (6 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
We don't use Texinfo any more; we can remove the references to the
.texi source file from our git.orderfile.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
scripts/git.orderfile | 1 -
1 file changed, 1 deletion(-)
diff --git a/scripts/git.orderfile b/scripts/git.orderfile
index a021afc2d51..3736c1d6aff 100644
--- a/scripts/git.orderfile
+++ b/scripts/git.orderfile
@@ -12,7 +12,6 @@
# Documentation
docs/*
*.rst
-*.texi
# build system
configure
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 20/21] configure: Drop texinfo requirement
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (18 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 19/21] Remove Texinfo related line from git.orderfile Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 12:38 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 21/21] Remove texinfo dependency from docker and CI configs Peter Maydell
` (5 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
We don't need the texinfo and pod2man programs to build our documentation
any more, so remove them from configure's tests.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
configure | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
diff --git a/configure b/configure
index e8e8e984f24..190d88ffa37 100755
--- a/configure
+++ b/configure
@@ -4873,14 +4873,14 @@ if test "$docs" != "no" ; then
else
sphinx_ok=no
fi
- if has makeinfo && has pod2man && test "$sphinx_ok" = "yes"; then
+ if test "$sphinx_ok" = "yes"; then
docs=yes
else
if test "$docs" = "yes" ; then
if has $sphinx_build && test "$sphinx_ok" != "yes"; then
echo "Warning: $sphinx_build exists but it is either too old or uses too old a Python version" >&2
fi
- feature_not_found "docs" "Install texinfo, Perl/perl-podlators and a Python 3 version of python-sphinx"
+ feature_not_found "docs" "Install a Python 3 version of python-sphinx"
fi
docs=no
fi
@@ -6300,13 +6300,6 @@ if test "$solaris" = "no" && test "$tsan" = "no"; then
fi
fi
-# test if pod2man has --utf8 option
-if pod2man --help | grep -q utf8; then
- POD2MAN="pod2man --utf8"
-else
- POD2MAN="pod2man"
-fi
-
# Use ASLR, no-SEH and DEP if available
if test "$mingw32" = "yes" ; then
for flag in --dynamicbase --no-seh --nxcompat; do
@@ -7455,7 +7448,6 @@ echo "HOST_DSOSUF=$HOST_DSOSUF" >> $config_host_mak
echo "LIBS_QGA=$libs_qga" >> $config_host_mak
echo "TASN1_LIBS=$tasn1_libs" >> $config_host_mak
echo "TASN1_CFLAGS=$tasn1_cflags" >> $config_host_mak
-echo "POD2MAN=$POD2MAN" >> $config_host_mak
if test "$gcov" = "yes" ; then
echo "CONFIG_GCOV=y" >> $config_host_mak
fi
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH v6 21/21] Remove texinfo dependency from docker and CI configs
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (19 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 20/21] configure: Drop texinfo requirement Peter Maydell
@ 2020-09-25 16:23 ` Peter Maydell
2020-09-29 12:39 ` Markus Armbruster
2020-09-25 16:54 ` [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo John Snow
` (4 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 16:23 UTC (permalink / raw)
To: qemu-devel; +Cc: John Snow, Markus Armbruster
We don't need texinfo to build the docs any more, so we can
drop that dependency from our docker and other CI configs.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
.travis.yml | 1 -
scripts/coverity-scan/coverity-scan.docker | 1 -
tests/docker/dockerfiles/debian10.docker | 1 -
tests/docker/dockerfiles/fedora.docker | 1 -
tests/docker/dockerfiles/ubuntu.docker | 1 -
tests/docker/dockerfiles/ubuntu1804.docker | 1 -
tests/docker/dockerfiles/ubuntu2004.docker | 1 -
7 files changed, 7 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index c75221dca3e..bd9a6fc06cc 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -192,7 +192,6 @@ jobs:
apt:
packages:
- python3-sphinx
- - texinfo
- perl
diff --git a/scripts/coverity-scan/coverity-scan.docker b/scripts/coverity-scan/coverity-scan.docker
index 018c03de6d5..501ac672332 100644
--- a/scripts/coverity-scan/coverity-scan.docker
+++ b/scripts/coverity-scan/coverity-scan.docker
@@ -110,7 +110,6 @@ ENV PACKAGES \
systemd-devel \
systemtap-sdt-devel \
tar \
- texinfo \
usbredir-devel \
virglrenderer-devel \
vte291-devel \
diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
index e3c11a454ee..1e4188ba225 100644
--- a/tests/docker/dockerfiles/debian10.docker
+++ b/tests/docker/dockerfiles/debian10.docker
@@ -31,7 +31,6 @@ RUN apt update && \
python3 \
python3-setuptools \
python3-sphinx \
- texinfo \
$(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\ -f2)
ENV FEATURES docs
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 70b6186bd3e..71e4b569770 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -96,7 +96,6 @@ ENV PACKAGES \
tar \
tesseract \
tesseract-langpack-eng \
- texinfo \
usbredir-devel \
virglrenderer-devel \
vte291-devel \
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index 161806e6b8c..b556ed17d29 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -63,7 +63,6 @@ ENV PACKAGES \
python3-yaml \
python3-sphinx \
sparse \
- texinfo \
xfslibs-dev
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
index a10ea2850b6..a6a7617da67 100644
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -49,7 +49,6 @@ ENV PACKAGES \
python3-yaml \
python3-sphinx \
sparse \
- texinfo \
xfslibs-dev
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index 8d10934a2a7..cafe8443fbf 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -57,7 +57,6 @@ ENV PACKAGES flex bison \
sparse \
tesseract-ocr \
tesseract-ocr-eng \
- texinfo \
xfslibs-dev\
vim
RUN apt-get update && \
--
2.20.1
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (20 preceding siblings ...)
2020-09-25 16:23 ` [PATCH v6 21/21] Remove texinfo dependency from docker and CI configs Peter Maydell
@ 2020-09-25 16:54 ` John Snow
2020-09-25 17:02 ` Peter Maydell
2020-09-25 19:25 ` no-reply
` (3 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: John Snow @ 2020-09-25 16:54 UTC (permalink / raw)
To: Peter Maydell, qemu-devel; +Cc: Markus Armbruster
On 9/25/20 12:22 PM, Peter Maydell wrote:
> * although there are HTML anchors on all the command/object/etc
> headings, they are not stable but just serial-number based
> tags like '#qapidoc-35', so not suitable for trying to link
> to from other parts of the docs
I suppose this answers my question if qmp commands were going to be able
to be cross-referenced yet.
I'd love to get that working so I can make smart references in the
bitmaps doc.
We could even amend the QMP code parser to insert cross-references to
the function names, which would be *extremely* slick.
--js
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 16:54 ` [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo John Snow
@ 2020-09-25 17:02 ` Peter Maydell
2020-09-25 17:09 ` John Snow
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 17:02 UTC (permalink / raw)
To: John Snow; +Cc: QEMU Developers, Markus Armbruster
On Fri, 25 Sep 2020 at 17:54, John Snow <jsnow@redhat.com> wrote:
>
> On 9/25/20 12:22 PM, Peter Maydell wrote:
> > * although there are HTML anchors on all the command/object/etc
> > headings, they are not stable but just serial-number based
> > tags like '#qapidoc-35', so not suitable for trying to link
> > to from other parts of the docs
>
> I suppose this answers my question if qmp commands were going to be able
> to be cross-referenced yet.
>
> I'd love to get that working so I can make smart references in the
> bitmaps doc.
>
> We could even amend the QMP code parser to insert cross-references to
> the function names, which would be *extremely* slick.
If you'd like to have a closer look into Sphinx's indexing
support, be my guest :-)
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 17:02 ` Peter Maydell
@ 2020-09-25 17:09 ` John Snow
0 siblings, 0 replies; 69+ messages in thread
From: John Snow @ 2020-09-25 17:09 UTC (permalink / raw)
To: Peter Maydell; +Cc: QEMU Developers, Markus Armbruster
On 9/25/20 1:02 PM, Peter Maydell wrote:
> On Fri, 25 Sep 2020 at 17:54, John Snow <jsnow@redhat.com> wrote:
>>
>> On 9/25/20 12:22 PM, Peter Maydell wrote:
>>> * although there are HTML anchors on all the command/object/etc
>>> headings, they are not stable but just serial-number based
>>> tags like '#qapidoc-35', so not suitable for trying to link
>>> to from other parts of the docs
>>
>> I suppose this answers my question if qmp commands were going to be able
>> to be cross-referenced yet.
>>
>> I'd love to get that working so I can make smart references in the
>> bitmaps doc.
>>
>> We could even amend the QMP code parser to insert cross-references to
>> the function names, which would be *extremely* slick.
>
> If you'd like to have a closer look into Sphinx's indexing
> support, be my guest :-)
>
Yeah, I am *absolutely* not suggesting this for this series.
I did start taking a look at Sphinx's default_role. As an exercise for
the Python cleanup, I looked at what would happen if I set our default
role to "Any" to be able to cross-reference using `backticks`.
I used this to test my docstring consistency for the QAPI module
cleanup. It involved me converting a few usages of single backticks in
./docs proper to double-backticks, but it wasn't too bad.
(See the "DO-NOT-MERGE" commits on my part1 V3.)
I'll look into how we *add* reference targets; that part I still have no
clue on. but adding a qmp reference scope would be *delightful*.
> -- PMM
>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (21 preceding siblings ...)
2020-09-25 16:54 ` [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo John Snow
@ 2020-09-25 19:25 ` no-reply
2020-09-25 21:37 ` Peter Maydell
2020-09-25 19:25 ` no-reply
` (2 subsequent siblings)
25 siblings, 1 reply; 69+ messages in thread
From: no-reply @ 2020-09-25 19:25 UTC (permalink / raw)
To: peter.maydell; +Cc: jsnow, qemu-devel, armbru
Patchew URL: https://patchew.org/QEMU/20200925162316.21205-1-peter.maydell@linaro.org/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===
perl-Encode-devel x86_64 4:3.07-457.fc32 updates 41 k
perl-Env noarch 1.04-440.fc32 fedora 19 k
perl-Errno x86_64 1.30-456.fc32 updates 24 k
perl-Error noarch 1:0.17029-1.fc32 fedora 42 k
perl-Exporter noarch 5.74-2.fc32 fedora 32 k
perl-ExtUtils-CBuilder noarch 1:0.280234-2.fc32 fedora 47 k
perl-ExtUtils-Command noarch 2:7.46-1.fc32 updates 14 k
---
(648/845): perl-Digest-SHA-6.02-442.fc32.x86_64 647 kB/s | 64 kB 00:00
(649/845): perl-Encode-Locale-1.05-15.fc32.noar 196 kB/s | 19 kB 00:00
(650/845): perl-Env-1.04-440.fc32.noarch.rpm 166 kB/s | 19 kB 00:00
(651/845): perl-Error-0.17029-1.fc32.noarch.rpm 371 kB/s | 42 kB 00:00
(652/845): perl-Exporter-5.74-2.fc32.noarch.rpm 331 kB/s | 32 kB 00:00
(653/845): perl-ExtUtils-CBuilder-0.280234-2.fc 478 kB/s | 47 kB 00:00
(654/845): perl-ExtUtils-Manifest-1.72-440.fc32 358 kB/s | 34 kB 00:00
---
Installing : perl-DB_File-1.853-2.fc32.x86_64 302/869
Installing : perl-Devel-Size-0.83-5.fc32.x86_64 303/869
Installing : perl-Env-1.04-440.fc32.noarch 304/869
Installing : perl-Error-1:0.17029-1.fc32.noarch 305/869
Installing : perl-IPC-SysV-2.07-442.fc32.x86_64 306/869
Installing : perl-IPC-System-Simple-1.30-1.fc32.noarch 307/869
Installing : perl-autodie-2.32-2.fc32.noarch 308/869
---
Verifying : perl-Digest-SHA-1:6.02-442.fc32.x86_64 648/869
Verifying : perl-Encode-Locale-1.05-15.fc32.noarch 649/869
Verifying : perl-Env-1.04-440.fc32.noarch 650/869
Verifying : perl-Error-1:0.17029-1.fc32.noarch 651/869
Verifying : perl-Exporter-5.74-2.fc32.noarch 652/869
Verifying : perl-ExtUtils-CBuilder-1:0.280234-2.fc32.noarch 653/869
Verifying : perl-ExtUtils-Install-2.14-441.fc32.noarch 654/869
---
perl-Encode-devel-4:3.07-457.fc32.x86_64
perl-Env-1.04-440.fc32.noarch
perl-Errno-1.30-456.fc32.x86_64
perl-Error-1:0.17029-1.fc32.noarch
perl-Exporter-5.74-2.fc32.noarch
perl-ExtUtils-CBuilder-1:0.280234-2.fc32.noarch
perl-ExtUtils-Command-2:7.46-1.fc32.noarch
---
Host machine cpu: x86_64
Target machine cpu family: x86
Target machine cpu: x86_64
../src/meson.build:10: WARNING: Module unstable-keyval has no backwards or forwards compatibility and might not exist in future releases.
Program sh found: YES
Program python3 found: YES (/usr/bin/python3)
Configuring ninjatool using configuration
---
Generating trace-hw_alpha.h with a meson_exe.py custom command
In file included from ../src/qapi/qapi-schema.json:78:
../src/qapi/migration.json:1747:1: unexpected de-indent (expected at least 13 spaces)
make: *** [Makefile.ninja:39: CUSTOM_COMMAND@d09afa93bc2.stamp] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 709, in <module>
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--rm', '--label', 'com.qemu.instance.uuid=8e891e79f2d34224a1d3856f15796b12', '-u', '1001', '--security-opt', 'seccomp=unconfined', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-9zsv4zbd/src/docker-src.2020-09-25-15.22.55.20619:/var/tmp/qemu:z,ro', 'qemu/fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=8e891e79f2d34224a1d3856f15796b12
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-9zsv4zbd/src'
make: *** [docker-run-test-mingw@fedora] Error 2
real 2m43.820s
user 0m21.216s
The full log is available at
http://patchew.org/logs/20200925162316.21205-1-peter.maydell@linaro.org/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (22 preceding siblings ...)
2020-09-25 19:25 ` no-reply
@ 2020-09-25 19:25 ` no-reply
2020-09-29 13:31 ` Markus Armbruster
2020-09-29 20:17 ` Markus Armbruster
25 siblings, 0 replies; 69+ messages in thread
From: no-reply @ 2020-09-25 19:25 UTC (permalink / raw)
To: peter.maydell; +Cc: jsnow, qemu-devel, armbru
Patchew URL: https://patchew.org/QEMU/20200925162316.21205-1-peter.maydell@linaro.org/
Hi,
This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===
C linker for the host machine: cc ld.bfd 2.27-43
Host machine cpu family: x86_64
Host machine cpu: x86_64
../src/meson.build:10: WARNING: Module unstable-keyval has no backwards or forwards compatibility and might not exist in future releases.
Program sh found: YES
Program python3 found: YES (/usr/bin/python3)
Configuring ninjatool using configuration
---
Generating trace-scsi.h with a meson_exe.py custom command
In file included from ../src/qapi/qapi-schema.json:78:
../src/qapi/migration.json:1747:1: unexpected de-indent (expected at least 13 spaces)
make: *** [CUSTOM_COMMAND@d09afa93bc2.stamp] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 709, in <module>
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--rm', '--label', 'com.qemu.instance.uuid=a2209dd2d512478b8d26c94a5b889ce1', '-u', '1003', '--security-opt', 'seccomp=unconfined', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-tvzj2v5u/src/docker-src.2020-09-25-15.22.42.19873:/var/tmp/qemu:z,ro', 'qemu/centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=a2209dd2d512478b8d26c94a5b889ce1
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-tvzj2v5u/src'
make: *** [docker-run-test-quick@centos7] Error 2
real 3m9.538s
user 0m19.483s
The full log is available at
http://patchew.org/logs/20200925162316.21205-1-peter.maydell@linaro.org/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 19:25 ` no-reply
@ 2020-09-25 21:37 ` Peter Maydell
2020-09-28 13:04 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-25 21:37 UTC (permalink / raw)
To: QEMU Developers; +Cc: John Snow, Markus Armbruster
On Fri, 25 Sep 2020 at 20:25, <no-reply@patchew.org> wrote:
> In file included from ../src/qapi/qapi-schema.json:78:
> ../src/qapi/migration.json:1747:1: unexpected de-indent (expected at least 13 spaces)
This is yet another mis-indented line in a change to the QAPI
doc-comments (commit 4c437254b807). It hit master in the
latest migration pull after I'd sent out this patchseries
but before patchew got round to testing..
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 01/21] qapi: Fix doc comment indentation again
2020-09-25 16:22 ` [PATCH v6 01/21] qapi: Fix doc comment indentation again Peter Maydell
@ 2020-09-28 12:39 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-28 12:39 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> In commit 26ec4e53f2 and similar commits we fixed the indentation
> for doc comments in our qapi json files to follow a new stricter
> standard for indentation, which permits only:
> @arg: description line 1
> description line 2
>
> or:
> @arg:
> line 1
> line 2
>
> but because the script updates that enforce this are not yet in the
> tree we have had a steady trickle of subsequent changes which didn't
> follow the new rules.
>
> Fix the latest round of mis-indented doc comments.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> qapi/block-core.json | 4 +-
> qapi/migration.json | 102 +++++++++++++++++++++----------------------
> 2 files changed, 53 insertions(+), 53 deletions(-)
>
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 3c16f1e11d6..dd77a91174c 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -4316,8 +4316,8 @@
> # @data-file-raw: True if the external data file must stay valid as a
> # standalone (read-only) raw image without looking at qcow2
> # metadata (default: false; since: 4.0)
> -# @extended-l2 True to make the image have extended L2 entries
> -# (default: false; since 5.2)
> +# @extended-l2: True to make the image have extended L2 entries
> +# (default: false; since 5.2)
> # @size: Size of the virtual disk in bytes
> # @version: Compatibility level (default: v3)
> # @backing-file: File name of the backing file if a backing file
Fixes a missing colon in addition to indentation.
[...]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set
2020-09-25 16:22 ` [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set Peter Maydell
@ 2020-09-28 12:42 ` Markus Armbruster
2020-09-28 12:49 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-28 12:42 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> The block-latency-histogram-set command is the only one which uses
> the Example/Examples section with the first line of the documentation
> immediately following the ':'. Bring it into line with the rest.
>
> This allows us to avoid special-casing the indentation handling for
> "Examples" sections; instead for Examples as for any other section
> header these two indentations will be equivalent:
I figure you're talking about PATCH 5 here. Correct?
If yes, I'd like to tweak this to "This will allow us". Okay?
>
> Examples:
> Line one
> Line two
>
> Examples: Line one
> Line two
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set
2020-09-28 12:42 ` Markus Armbruster
@ 2020-09-28 12:49 ` Peter Maydell
2020-09-28 18:04 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-28 12:49 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Mon, 28 Sep 2020 at 13:42, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > The block-latency-histogram-set command is the only one which uses
> > the Example/Examples section with the first line of the documentation
> > immediately following the ':'. Bring it into line with the rest.
> >
> > This allows us to avoid special-casing the indentation handling for
> > "Examples" sections; instead for Examples as for any other section
> > header these two indentations will be equivalent:
>
> I figure you're talking about PATCH 5 here. Correct?
>
> If yes, I'd like to tweak this to "This will allow us". Okay?
Sure. (In my idiolect both those phrasings have pretty much
identical meaning in this context.)
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 21:37 ` Peter Maydell
@ 2020-09-28 13:04 ` Markus Armbruster
2020-09-28 13:05 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-28 13:04 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers, Markus Armbruster
Peter Maydell <peter.maydell@linaro.org> writes:
> On Fri, 25 Sep 2020 at 20:25, <no-reply@patchew.org> wrote:
>
>> In file included from ../src/qapi/qapi-schema.json:78:
>> ../src/qapi/migration.json:1747:1: unexpected de-indent (expected at least 13 spaces)
>
> This is yet another mis-indented line in a change to the QAPI
> doc-comments (commit 4c437254b807). It hit master in the
> latest migration pull after I'd sent out this patchseries
> but before patchew got round to testing..
Obvious fixup for your PATCH 01:
diff --git a/qapi/migration.json b/qapi/migration.json
index 7d9342c064..7f5e6fd681 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1744,9 +1744,9 @@
# Information about current dirty page rate of vm.
#
# @dirty-rate: @dirtyrate describing the dirty page rate of vm
-# in units of MB/s.
-# If this field returns '-1', it means querying has not
-# yet started or completed.
+# in units of MB/s.
+# If this field returns '-1', it means querying has not
+# yet started or completed.
#
# @status: status containing dirtyrate query status includes
# 'unstarted' or 'measuring' or 'measured'
Happy to fix it up in my tree.
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-28 13:04 ` Markus Armbruster
@ 2020-09-28 13:05 ` Peter Maydell
2020-09-29 15:26 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-28 13:05 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Mon, 28 Sep 2020 at 14:04, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > On Fri, 25 Sep 2020 at 20:25, <no-reply@patchew.org> wrote:
> >
> >> In file included from ../src/qapi/qapi-schema.json:78:
> >> ../src/qapi/migration.json:1747:1: unexpected de-indent (expected at least 13 spaces)
> >
> > This is yet another mis-indented line in a change to the QAPI
> > doc-comments (commit 4c437254b807). It hit master in the
> > latest migration pull after I'd sent out this patchseries
> > but before patchew got round to testing..
>
> Obvious fixup for your PATCH 01:
>
> diff --git a/qapi/migration.json b/qapi/migration.json
> index 7d9342c064..7f5e6fd681 100644
> --- a/qapi/migration.json
> +++ b/qapi/migration.json
> @@ -1744,9 +1744,9 @@
> # Information about current dirty page rate of vm.
> #
> # @dirty-rate: @dirtyrate describing the dirty page rate of vm
> -# in units of MB/s.
> -# If this field returns '-1', it means querying has not
> -# yet started or completed.
> +# in units of MB/s.
> +# If this field returns '-1', it means querying has not
> +# yet started or completed.
> #
> # @status: status containing dirtyrate query status includes
> # 'unstarted' or 'measuring' or 'measured'
>
> Happy to fix it up in my tree.
Yes, please.
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set
2020-09-28 12:49 ` Peter Maydell
@ 2020-09-28 18:04 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-28 18:04 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Mon, 28 Sep 2020 at 13:42, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>> > The block-latency-histogram-set command is the only one which uses
>> > the Example/Examples section with the first line of the documentation
>> > immediately following the ':'. Bring it into line with the rest.
>> >
>> > This allows us to avoid special-casing the indentation handling for
>> > "Examples" sections; instead for Examples as for any other section
>> > header these two indentations will be equivalent:
>>
>> I figure you're talking about PATCH 5 here. Correct?
>>
>> If yes, I'd like to tweak this to "This will allow us". Okay?
>
> Sure. (In my idiolect both those phrasings have pretty much
> identical meaning in this context.)
>
> thanks
> -- PMM
Neglected to say
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling
2020-09-25 16:23 ` [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling Peter Maydell
@ 2020-09-28 19:15 ` Markus Armbruster
2020-09-29 8:55 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-28 19:15 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel, Markus Armbruster
Peter Maydell <peter.maydell@linaro.org> writes:
> Make the handling of indentation in doc comments more sophisticated,
> so that when we see a section like:
>
> Notes: some text
> some more text
> indented line 3
>
> we save it for the doc-comment processing code as:
>
> some text
> some more text
> indented line 3
>
> and when we see a section with the heading on its own line:
>
> Notes:
>
> some text
> some more text
> indented text
>
> we also accept that and save it in the same form.
>
> If we detect that the comment document text is not indented as much
> as we expect it to be, we throw a parse error. (We don't complain
> about over-indented sections, because for rST this can be legitimate
> markup.)
>
> The golden reference for the doc comment text is updated to remove
> the two 'wrong' indents; these now form a test case that we correctly
> stripped leading whitespace from an indented multi-line argument
> definition.
>
> We update the documentation in docs/devel/qapi-code-gen.txt to
> describe the new indentation rules.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> docs/devel/qapi-code-gen.txt | 23 +++++++
> scripts/qapi/parser.py | 93 +++++++++++++++++++++------
> tests/qapi-schema/doc-bad-indent.err | 1 +
> tests/qapi-schema/doc-bad-indent.json | 8 +++
> tests/qapi-schema/doc-bad-indent.out | 0
> tests/qapi-schema/doc-good.out | 4 +-
> tests/qapi-schema/meson.build | 1 +
> 7 files changed, 109 insertions(+), 21 deletions(-)
> create mode 100644 tests/qapi-schema/doc-bad-indent.err
> create mode 100644 tests/qapi-schema/doc-bad-indent.json
> create mode 100644 tests/qapi-schema/doc-bad-indent.out
>
> diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
> index 9eede44350c..69eaffac376 100644
> --- a/docs/devel/qapi-code-gen.txt
> +++ b/docs/devel/qapi-code-gen.txt
> @@ -901,6 +901,22 @@ commands and events), member (for structs and unions), branch (for
> alternates), or value (for enums), and finally optional tagged
> sections.
>
> +Descriptions of arguments can span multiple lines. The description
> +text can start on the line following the '@argname:', in which case
> +it must not be indented at all. It can also start on the same line
> +as the '@argname:'. In this case if it spans multiple lines then
> +second and subsequent lines must be indented to line up with the
> +first character of the first line of the description:
Please put two spaces after sentence-ending punctuation, for local
consistency, and to keep Emacs sentence commands working.
Can touch this up in my tree, of course.
> +
> +# @argone:
> +# This is a two line description
> +# in the first style.
> +#
> +# @argtwo: This is a two line description
> +# in the second style.
> +
> +The number of spaces between the ':' and the text is not significant.
> +
> FIXME: the parser accepts these things in almost any order.
> FIXME: union branches should be described, too.
>
> @@ -911,6 +927,13 @@ A tagged section starts with one of the following words:
> "Note:"/"Notes:", "Since:", "Example"/"Examples", "Returns:", "TODO:".
> The section ends with the start of a new section.
>
> +The text of a section can start on a new line, in
> +which case it must not be indented at all. It can also start
> +on the same line as the 'Note:', 'Returns:', etc tag. In this
> +case if it spans multiple lines then second and subsequent
> +lines must be indented to match the first, in the same way as
> +multiline argument descriptions.
> +
> A 'Since: x.y.z' tagged section lists the release that introduced the
> definition.
>
> diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
> index 04bf10db378..6c3455b41f3 100644
> --- a/scripts/qapi/parser.py
> +++ b/scripts/qapi/parser.py
> @@ -319,17 +319,32 @@ class QAPIDoc:
> """
>
> class Section:
> - def __init__(self, name=None):
> + def __init__(self, parser, name=None, indent=0):
> + # parser, for error messages about indentation
> + self._parser = parser
> # optional section name (argument/member or section name)
> self.name = name
> self.text = ''
> + # the expected indent level of the text of this section
> + self._indent = indent
>
> def append(self, line):
> + # Strip leading spaces corresponding to the expected indent level
> + # Blank lines are always OK.
> + if line:
> + indent = re.match(r'\s*', line).end()
> + if indent < self._indent:
> + raise QAPIParseError(
> + self._parser,
> + "unexpected de-indent (expected at least %d spaces)" %
> + self._indent)
> + line = line[self._indent:]
> +
> self.text += line.rstrip() + '\n'
>
> class ArgSection(Section):
> - def __init__(self, name):
> - super().__init__(name)
> + def __init__(self, parser, name, indent=0):
> + super().__init__(parser, name, indent)
> self.member = None
>
> def connect(self, member):
> @@ -343,7 +358,7 @@ class QAPIDoc:
> self._parser = parser
> self.info = info
> self.symbol = None
> - self.body = QAPIDoc.Section()
> + self.body = QAPIDoc.Section(parser)
> # dict mapping parameter name to ArgSection
> self.args = OrderedDict()
> self.features = OrderedDict()
> @@ -447,8 +462,21 @@ class QAPIDoc:
> name = line.split(' ', 1)[0]
>
> if name.startswith('@') and name.endswith(':'):
> - line = line[len(name)+1:]
> - self._start_args_section(name[1:-1])
> + # If line is "@arg: first line of description", find the
> + # index of 'f', which is the indent we expect for any
> + # following lines. We then remove the leading "@arg:" from line
PEP 8: You should use two spaces after a sentence-ending period in
multi-sentence comments, except after the final sentence.
More of the same below. Can touch it up in my tree.
> + # and replace it with spaces so that 'f' has the same index
> + # as it did in the original line and can be handled the same
> + # way we will handle following lines.
> + indent = re.match(r'@\S*:\s*', line).end()
> + line = line[indent:]
> + if not line:
> + # Line was just the "@arg:" header; following lines
> + # are not indented
> + indent = 0
> + else:
> + line = ' ' * indent + line
> + self._start_args_section(name[1:-1], indent)
> elif self._is_section_tag(name):
> self._append_line = self._append_various_line
> self._append_various_line(line)
> @@ -469,8 +497,21 @@ class QAPIDoc:
> name = line.split(' ', 1)[0]
>
> if name.startswith('@') and name.endswith(':'):
> - line = line[len(name)+1:]
> - self._start_features_section(name[1:-1])
> + # If line is "@arg: first line of description", find the
> + # index of 'f', which is the indent we expect for any
> + # following lines. We then remove the leading "@arg:" from line
> + # and replace it with spaces so that 'f' has the same index
> + # as it did in the original line and can be handled the same
> + # way we will handle following lines.
> + indent = re.match(r'@\S*:\s*', line).end()
> + line = line[indent:]
> + if not line:
> + # Line was just the "@arg:" header; following lines
> + # are not indented
> + indent = 0
> + else:
> + line = ' ' * indent + line
> + self._start_features_section(name[1:-1], indent)
> elif self._is_section_tag(name):
> self._append_line = self._append_various_line
> self._append_various_line(line)
> @@ -502,12 +543,25 @@ class QAPIDoc:
> "'%s' can't follow '%s' section"
> % (name, self.sections[0].name))
> if self._is_section_tag(name):
> - line = line[len(name)+1:]
> - self._start_section(name[:-1])
> + # If line is "Section: first line of description", find the
> + # index of 'f', which is the indent we expect for any
> + # following lines. We then remove the leading "Section:" from line
> + # and replace it with spaces so that 'f' has the same index
> + # as it did in the original line and can be handled the same
> + # way we will handle following lines.
> + indent = re.match(r'\S*:\s*', line).end()
> + line = line[indent:]
> + if not line:
> + # Line was just the "Section:" header; following lines
> + # are not indented
> + indent = 0
> + else:
> + line = ' ' * indent + line
We may want to de-triplicate. Not today.
> + self._start_section(name[:-1], indent)
>
> self._append_freeform(line)
>
> - def _start_symbol_section(self, symbols_dict, name):
> + def _start_symbol_section(self, symbols_dict, name, indent):
> # FIXME invalid names other than the empty string aren't flagged
> if not name:
> raise QAPIParseError(self._parser, "invalid parameter name")
> @@ -516,21 +570,21 @@ class QAPIDoc:
> "'%s' parameter name duplicated" % name)
> assert not self.sections
> self._end_section()
> - self._section = QAPIDoc.ArgSection(name)
> + self._section = QAPIDoc.ArgSection(self._parser, name, indent)
> symbols_dict[name] = self._section
>
> - def _start_args_section(self, name):
> - self._start_symbol_section(self.args, name)
> + def _start_args_section(self, name, indent):
> + self._start_symbol_section(self.args, name, indent)
>
> - def _start_features_section(self, name):
> - self._start_symbol_section(self.features, name)
> + def _start_features_section(self, name, indent):
> + self._start_symbol_section(self.features, name, indent)
>
> - def _start_section(self, name=None):
> + def _start_section(self, name=None, indent=0):
> if name in ('Returns', 'Since') and self.has_section(name):
> raise QAPIParseError(self._parser,
> "duplicated '%s' section" % name)
> self._end_section()
> - self._section = QAPIDoc.Section(name)
> + self._section = QAPIDoc.Section(self._parser, name, indent)
> self.sections.append(self._section)
>
> def _end_section(self):
> @@ -553,7 +607,8 @@ class QAPIDoc:
> def connect_member(self, member):
> if member.name not in self.args:
> # Undocumented TODO outlaw
> - self.args[member.name] = QAPIDoc.ArgSection(member.name)
> + self.args[member.name] = QAPIDoc.ArgSection(self._parser,
> + member.name)
> self.args[member.name].connect(member)
>
> def connect_feature(self, feature):
> diff --git a/tests/qapi-schema/doc-bad-indent.err b/tests/qapi-schema/doc-bad-indent.err
> new file mode 100644
> index 00000000000..67844539bd2
> --- /dev/null
> +++ b/tests/qapi-schema/doc-bad-indent.err
> @@ -0,0 +1 @@
> +doc-bad-indent.json:6:1: unexpected de-indent (expected at least 4 spaces)
> diff --git a/tests/qapi-schema/doc-bad-indent.json b/tests/qapi-schema/doc-bad-indent.json
> new file mode 100644
> index 00000000000..edde8f21dc7
> --- /dev/null
> +++ b/tests/qapi-schema/doc-bad-indent.json
> @@ -0,0 +1,8 @@
> +# Multiline doc comments should have consistent indentation
> +
> +##
> +# @foo:
> +# @a: line one
> +# line two is wrongly indented
> +##
> +{ 'command': 'foo', 'data': { 'a': 'int' } }
> diff --git a/tests/qapi-schema/doc-bad-indent.out b/tests/qapi-schema/doc-bad-indent.out
> new file mode 100644
> index 00000000000..e69de29bb2d
> diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
> index 9993ffcd89d..b7e3f4313da 100644
> --- a/tests/qapi-schema/doc-good.out
> +++ b/tests/qapi-schema/doc-good.out
> @@ -159,7 +159,7 @@ doc symbol=Alternate
>
> arg=i
> an integer
> - @b is undocumented
> +@b is undocumented
> arg=b
>
> feature=alt-feat
> @@ -174,7 +174,7 @@ doc symbol=cmd
> the first argument
> arg=arg2
> the second
> - argument
> +argument
> arg=arg3
>
> feature=cmd-feat1
> diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
> index f1449298b07..83a0a68389b 100644
> --- a/tests/qapi-schema/meson.build
> +++ b/tests/qapi-schema/meson.build
> @@ -53,6 +53,7 @@ schemas = [
> 'doc-bad-enum-member.json',
> 'doc-bad-event-arg.json',
> 'doc-bad-feature.json',
> + 'doc-bad-indent.json',
> 'doc-bad-section.json',
> 'doc-bad-symbol.json',
> 'doc-bad-union-member.json',
With trivial whitespace touch-ups:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 06/21] qapi/machine.json: Escape a literal '*' in doc comment
2020-09-25 16:23 ` [PATCH v6 06/21] qapi/machine.json: Escape a literal '*' in doc comment Peter Maydell
@ 2020-09-29 4:57 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 4:57 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> For rST, '*' is a kind of inline markup (for emphasis), so
> "*-softmmu" is a syntax error because of the missing closing '*'.
> Escape the '*' with a '\'.
>
> The texinfo document generator will leave the '\' in the
> output, which is not ideal, but that generator is going to
> go away in a subsequent commit.
Yes, in PATCH 09. Tolerable.
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> qapi/machine.json | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 0ac1880e4a4..9c45b04363c 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -13,7 +13,7 @@
> #
> # The comprehensive enumeration of QEMU system emulation ("softmmu")
> # targets. Run "./configure --help" in the project root directory, and
> -# look for the *-softmmu targets near the "--target-list" option. The
> +# look for the \*-softmmu targets near the "--target-list" option. The
> # individual target constants are not documented here, for the time
> # being.
> #
A better markup might be
# The comprehensive enumeration of QEMU system emulation ("softmmu")
# targets. Run ``./configure --help`` in the project root directory, and
# look for the ``*-softmmu`` targets near the ``--target-list``
# option. The individual target constants are not documented here, for
# the time being.
But that should be done systematically, not just here, which makes it
worse than your patch at this point of the conversion.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension
2020-09-25 16:23 ` [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension Peter Maydell
@ 2020-09-29 6:54 ` Markus Armbruster
2020-09-29 9:05 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 6:54 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> Some of our documentation is auto-generated from documentation
> comments in the JSON schema.
>
> For Sphinx, rather than creating a file to include, the most natural
> way to handle this is to have a small custom Sphinx extension which
> processes the JSON file and inserts documentation into the rST
> file being processed.
>
> This is the same approach that kerneldoc and hxtool use.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Down to a few trivial things, which I can tidy up in my tree.
> ---
> docs/conf.py | 6 +-
> docs/sphinx/qapidoc.py | 550 +++++++++++++++++++++++++++++++++++++++++
> MAINTAINERS | 1 +
> 3 files changed, 556 insertions(+), 1 deletion(-)
> create mode 100644 docs/sphinx/qapidoc.py
>
> diff --git a/docs/conf.py b/docs/conf.py
> index 0dbd90dc112..606f623211d 100644
> --- a/docs/conf.py
> +++ b/docs/conf.py
> @@ -52,7 +52,10 @@ except NameError:
> # add these directories to sys.path here. If the directory is relative to the
> # documentation root, use an absolute path starting from qemu_docdir.
> #
> +# Our extensions are in docs/sphinx; the qapidoc extension requires
> +# the QAPI modules from scripts/.
> sys.path.insert(0, os.path.join(qemu_docdir, "sphinx"))
> +sys.path.insert(0, os.path.join(qemu_docdir, "../scripts"))
>
>
> # -- General configuration ------------------------------------------------
> @@ -67,7 +70,7 @@ needs_sphinx = '1.6'
> # Add any Sphinx extension module names here, as strings. They can be
> # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
> # ones.
> -extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile']
> +extensions = ['kerneldoc', 'qmp_lexer', 'hxtool', 'depfile', 'qapidoc']
>
> # Add any paths that contain templates here, relative to this directory.
> templates_path = ['_templates']
> @@ -241,3 +244,4 @@ texinfo_documents = [
> kerneldoc_bin = os.path.join(qemu_docdir, '../scripts/kernel-doc')
> kerneldoc_srctree = os.path.join(qemu_docdir, '..')
> hxtool_srctree = os.path.join(qemu_docdir, '..')
> +qapidoc_srctree = os.path.join(qemu_docdir, '..')
> diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
> new file mode 100644
> index 00000000000..afc4167f8b1
> --- /dev/null
> +++ b/docs/sphinx/qapidoc.py
> @@ -0,0 +1,550 @@
> +# coding=utf-8
> +#
> +# QEMU qapidoc QAPI file parsing extension
> +#
> +# Copyright (c) 2020 Linaro
> +#
> +# This work is licensed under the terms of the GNU GPLv2 or later.
> +# See the COPYING file in the top-level directory.
Blank line, please.
> +"""qapidoc is a Sphinx extension that implements the qapi-doc directive"""
> +
> +# The purpose of this extension is to read the documentation comments
> +# in QAPI schema files, and insert them all into the current document.
> +#
> +# It implements one new rST directive, "qapi-doc::".
> +# Each qapi-doc:: directive takes one argument, which is the
> +# pathname of the schema file to process, relative to the source tree.
> +#
> +# The docs/conf.py file must set the qapidoc_srctree config value to
> +# the root of the QEMU source tree.
> +#
> +# The Sphinx documentation on writing extensions is at:
> +# https://www.sphinx-doc.org/en/master/development/index.html
Should this go into the doc string?
Blank line, please.
> +import os
> +import re
> +
> +from docutils import nodes
> +from docutils.statemachine import ViewList
> +from docutils.parsers.rst import directives, Directive
> +from sphinx.errors import ExtensionError
> +from sphinx.util.nodes import nested_parse_with_titles
> +import sphinx
> +from qapi.gen import QAPISchemaVisitor
> +from qapi.schema import QAPIError, QAPISemError, QAPISchema
> +
> +# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
> +# use switch_source_input. Check borrowed from kerneldoc.py.
> +Use_SSI = sphinx.__version__[:3] >= '1.7'
> +if Use_SSI:
> + from sphinx.util.docutils import switch_source_input
> +else:
> + from sphinx.ext.autodoc import AutodocReporter
> +
> +
> +__version__ = '1.0'
> +
> +
> +# Function borrowed from pydash, which is under the MIT license
> +def intersperse(iterable, separator):
> + """Yield the members of *iterable* interspersed with *separator*."""
> + iterable = iter(iterable)
> + yield next(iterable)
> + for item in iterable:
> + yield separator
> + yield item
> +
> +
> +class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
> + """A QAPI schema visitor which generates docutils/Sphinx nodes
> +
> + This class builds up a tree of docutils/Sphinx nodes corresponding
> + to documentation for the various QAPI objects. To use it, first create
> + a QAPISchemaGenRSTVisitor object, and call its visit_begin() method.
> + Then you can call one of the two methods 'freeform' (to add documentation
> + for a freeform documentation chunk) or 'symbol' (to add documentation
> + for a QAPI symbol). These will cause the visitor to build up the
> + tree of document nodes. Once you've added all the documentation
> + via 'freeform' and 'symbol' method calls, you can call 'get_document_nodes'
> + to get the final list of document nodes (in a form suitable for returning
> + from a Sphinx directive's 'run' method).
> + """
PEP 8: For flowing long blocks of text with fewer structural
restrictions (docstrings or comments), the line length should be limited
to 72 characters.
John aims to get us to consistent, Sphinx-compatible doc string format
and markup. We could try to do the doc strings in this file right from
the start, but I figure it's easier not to complicate things now, and
let John massage the doc strings in due time.
> + def __init__(self, sphinx_directive):
> + self._cur_doc = None
> + self._sphinx_directive = sphinx_directive
> + self._top_node = nodes.section()
> + self._active_headings = [self._top_node]
> +
> + def _serror(self, msg):
> + """Raise an exception giving a user-friendly syntax error message"""
> + file = self._cur_doc.info.fname
> + line = self._cur_doc.info.line
> + raise ExtensionError(
> + '%s line %d: syntax error: %s' % (file, line, msg))
Unused, let's drop.
> +
> + def _make_dlitem(self, term, defn):
> + """Return a dlitem node with the specified term and definition.
> +
> + term should be a list of Text and literal nodes.
> + defn should be one of:
> + - a string, which will be handed to _parse_text_into_node
> + - a list of Text and literal nodes, which will be put into
> + a paragraph node
> + """
> + dlitem = nodes.definition_list_item()
> + dlterm = nodes.term('', '', *term)
> + dlitem += dlterm
> + if defn:
> + dldef = nodes.definition()
> + if isinstance(defn, list):
> + dldef += nodes.paragraph('', '', *defn)
> + else:
> + self._parse_text_into_node(defn, dldef)
> + dlitem += dldef
> + return dlitem
> +
> + def _make_section(self, title):
> + """Return a section node with optional title"""
> + section = nodes.section(ids=[self._sphinx_directive.new_serialno()])
> + if title:
> + section += nodes.title(title, title)
> + return section
> +
> + def _nodes_for_ifcond(self, ifcond, with_if=True):
> + """Return list of Text, literal nodes for the ifcond
> +
> + Return a list which gives text like ' (If: cond1, cond2, cond3)', where
> + the conditions are in literal-text and the commas are not.
> + If with_if is False, we don't return the "(If: " and ")".
> + """
> + condlist = intersperse([nodes.literal('', c) for c in ifcond],
> + nodes.Text(', '))
> + if not with_if:
> + return condlist
> +
> + nodelist = [nodes.Text(' ('), nodes.strong('', 'If: ')]
> + nodelist.extend(condlist)
> + nodelist.append(nodes.Text(')'))
> + return nodelist
> +
> + def _nodes_for_one_member(self, member):
> + """Return list of Text, literal nodes for this member
> +
> + Return a list of doctree nodes which give text like
> + 'name: type (optional) (If: ...)' suitable for use as the
> + 'term' part of a definition list item.
> + """
> + term = [nodes.literal('', member.name)]
> + if member.type.doc_type():
> + term.append(nodes.Text(': '))
> + term.append(nodes.literal('', member.type.doc_type()))
> + if member.optional:
> + term.append(nodes.Text(' (optional)'))
> + if member.ifcond:
> + term.extend(self._nodes_for_ifcond(member.ifcond))
> + return term
> +
> + def _nodes_for_variant_when(self, variants, variant):
> + """Return list of Text, literal nodes for variant 'when' clause
> +
> + Return a list of doctree nodes which give text like
> + 'when tagname is variant (If: ...)' suitable for use in
> + the 'variants' part of a definition list.
> + """
> + term = [nodes.Text(' when '),
> + nodes.literal('', variants.tag_member.name),
> + nodes.Text(' is '),
> + nodes.literal('', '"%s"' % variant.name)]
> + if variant.ifcond:
> + term.extend(self._nodes_for_ifcond(variant.ifcond))
> + return term
> +
> + def _nodes_for_members(self, doc, what, base=None, variants=None):
> + """Return list of doctree nodes for the table of members"""
> + dlnode = nodes.definition_list()
> + for section in doc.args.values():
> + term = self._nodes_for_one_member(section.member)
> + # TODO drop fallbacks when undocumented members are outlawed
> + if section.text:
> + defn = section.text
> + elif (variants and variants.tag_member == section.member
> + and not section.member.type.doc_type()):
> + values = section.member.type.member_names()
> + defn = [nodes.Text('One of ')]
> + defn.extend(intersperse([nodes.literal('', v) for v in values],
> + nodes.Text(', ')))
> + else:
> + defn = [nodes.Text('Not documented')]
> +
> + dlnode += self._make_dlitem(term, defn)
> +
> + if base:
> + dlnode += self._make_dlitem([nodes.Text('The members of '),
> + nodes.literal('', base.doc_type())],
> + None)
> +
> + if variants:
> + for v in variants.variants:
> + if v.type.is_implicit():
> + assert not v.type.base and not v.type.variants
> + for m in v.type.local_members:
> + term = self._nodes_for_one_member(m)
> + term.extend(self._nodes_for_variant_when(variants, v))
> + dlnode += self._make_dlitem(term, None)
> + else:
> + term = [nodes.Text('The members of '),
> + nodes.literal('', v.type.doc_type())]
> + term.extend(self._nodes_for_variant_when(variants, v))
> + dlnode += self._make_dlitem(term, None)
> +
> + if not dlnode.children:
> + return []
> +
> + section = self._make_section(what)
> + section += dlnode
> + return [section]
> +
> + def _nodes_for_enum_values(self, doc):
> + """Return list of doctree nodes for the table of enum values"""
> + seen_item = False
> + dlnode = nodes.definition_list()
> + for section in doc.args.values():
> + termtext = [nodes.literal('', section.member.name)]
> + if section.member.ifcond:
> + termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
> + # TODO drop fallbacks when undocumented members are outlawed
> + if section.text:
> + defn = section.text
> + else:
> + defn = [nodes.Text('Not documented')]
> +
> + dlnode += self._make_dlitem(termtext, defn)
> + seen_item = True
> +
> + if not seen_item:
> + return []
> +
> + section = self._make_section('Values')
> + section += dlnode
> + return [section]
> +
> + def _nodes_for_arguments(self, doc, boxed_arg_type):
> + """Return list of doctree nodes for the arguments section"""
> + if boxed_arg_type:
> + assert not doc.args
> + section = self._make_section('Arguments')
> + dlnode = nodes.definition_list()
> + dlnode += self._make_dlitem(
> + [nodes.Text('The members of '),
> + nodes.literal('', boxed_arg_type.name)],
> + None)
> + section += dlnode
> + return [section]
> +
> + return self._nodes_for_members(doc, 'Arguments')
> +
> + def _nodes_for_features(self, doc):
> + """Return list of doctree nodes for the table of features"""
> + seen_item = False
> + dlnode = nodes.definition_list()
> + for section in doc.features.values():
> + dlnode += self._make_dlitem([nodes.literal('', section.name)],
> + section.text)
> + seen_item = True
> +
> + if not seen_item:
> + return []
> +
> + section = self._make_section('Features')
> + section += dlnode
> + return [section]
> +
> + def _nodes_for_example(self, exampletext):
> + """Return list of doctree nodes for a code example snippet"""
> + return [nodes.literal_block(exampletext, exampletext)]
> +
> + def _nodes_for_sections(self, doc):
> + """Return list of doctree nodes for additional sections"""
> + nodelist = []
> + for section in doc.sections:
> + snode = self._make_section(section.name)
> + if section.name and section.name.startswith('Example'):
> + snode += self._nodes_for_example(section.text)
> + else:
> + self._parse_text_into_node(section.text, snode)
> + nodelist.append(snode)
> + return nodelist
> +
> + def _nodes_for_if_section(self, ifcond):
> + """Return list of doctree nodes for the "If" section"""
> + nodelist = []
> + if ifcond:
> + snode = self._make_section('If')
> + snode += self._nodes_for_ifcond(ifcond, with_if=False)
> + nodelist.append(snode)
> + return nodelist
> +
> + def _add_doc(self, typ, sections):
> + """Add documentation for a command/object/enum...
> +
> + We assume we're documenting the thing defined in self._cur_doc.
> + typ is the type of thing being added ("Command", "Object", etc)
> +
> + sections is a list of nodes for sections to add to the definition.
> + """
> +
> + doc = self._cur_doc
> + snode = nodes.section(ids=[self._sphinx_directive.new_serialno()])
> + snode += nodes.title('', '', *[nodes.literal(doc.symbol, doc.symbol),
> + nodes.Text(' (' + typ + ')')])
> + self._parse_text_into_node(doc.body.text, snode)
> + for s in sections:
> + if s is not None:
> + snode += s
> + self._add_node_to_current_heading(snode)
> +
> + def visit_enum_type(self, name, info, ifcond, features, members, prefix):
> + doc = self._cur_doc
> + self._add_doc('Enum',
> + self._nodes_for_enum_values(doc) +
> + self._nodes_for_features(doc) +
> + self._nodes_for_sections(doc) +
> + self._nodes_for_if_section(ifcond))
PEP 8: In Python code, it is permissible to break before or after a
binary operator, as long as the convention is consistent locally. For
new code Knuth's style is suggested.
Mind switching to Knuth style, i.e. break before the operator?
More of the same below.
> +
> + def visit_object_type(self, name, info, ifcond, features,
> + base, members, variants):
> + doc = self._cur_doc
> + if base and base.is_implicit():
> + base = None
> + self._add_doc('Object',
> + self._nodes_for_members(doc, 'Members', base, variants) +
> + self._nodes_for_features(doc) +
> + self._nodes_for_sections(doc) +
> + self._nodes_for_if_section(ifcond))
> +
> + def visit_alternate_type(self, name, info, ifcond, features, variants):
> + doc = self._cur_doc
> + self._add_doc('Alternate',
> + self._nodes_for_members(doc, 'Members') +
> + self._nodes_for_features(doc) +
> + self._nodes_for_sections(doc) +
> + self._nodes_for_if_section(ifcond))
> +
> + def visit_command(self, name, info, ifcond, features, arg_type,
> + ret_type, gen, success_response, boxed, allow_oob,
> + allow_preconfig):
> + doc = self._cur_doc
> + self._add_doc('Command',
> + self._nodes_for_arguments(doc,
> + arg_type if boxed else None) +
> + self._nodes_for_features(doc) +
> + self._nodes_for_sections(doc) +
> + self._nodes_for_if_section(ifcond))
> +
> + def visit_event(self, name, info, ifcond, features, arg_type, boxed):
> + doc = self._cur_doc
> + self._add_doc('Event',
> + self._nodes_for_arguments(doc,
> + arg_type if boxed else None) +
> + self._nodes_for_features(doc) +
> + self._nodes_for_sections(doc) +
> + self._nodes_for_if_section(ifcond))
> +
> + def symbol(self, doc, entity):
> + """Add documentation for one symbol to the document tree
> +
> + This is the main entry point which causes us to add documentation
> + nodes for a symbol (which could be a 'command', 'object', 'event',
> + etc). We do this by calling 'visit' on the schema entity, which
> + will then call back into one of our visit_* methods, depending
> + on what kind of thing this symbol is.
> + """
> + self._cur_doc = doc
> + entity.visit(self)
> + self._cur_doc = None
> +
> + def _start_new_heading(self, heading, level):
> + """Start a new heading at the specified heading level
> +
> + Create a new section whose title is 'heading' and which is placed
> + in the docutils node tree as a child of the most recent level-1
> + heading. Subsequent document sections (commands, freeform doc chunks,
> + etc) will be placed as children of this new heading section.
> + """
> + if len(self._active_headings) < level:
> + raise QAPISemError(self._cur_doc.info,
> + 'Level %d subheading found outside a '
> + 'level %d heading'
> + % (level, level - 1))
> + snode = self._make_section(heading)
> + self._active_headings[level - 1] += snode
> + self._active_headings = self._active_headings[:level]
> + self._active_headings.append(snode)
> +
> + def _add_node_to_current_heading(self, node):
> + """Add the node to whatever the current active heading is"""
> + self._active_headings[-1] += node
> +
> + def freeform(self, doc):
> + """Add a piece of 'freeform' documentation to the document tree
> +
> + A 'freeform' document chunk doesn't relate to any particular
> + symbol (for instance, it could be an introduction).
> +
> + If the freeform document starts with a line of the form
> + '= Heading text', this is a section or subsection heading, with
> + the heading level indicated by the number of '=' signs.
> + """
> +
> + # QAPIDoc documentation says free-form documentation blocks
> + # must have only a body section, nothing else.
> + assert not doc.sections
> + assert not doc.args
> + assert not doc.features
> + self._cur_doc = doc
> +
> + text = doc.body.text
> + if re.match(r'=+ ', text):
> + # Section/subsection heading (if present, will always be
> + # the first line of the block)
> + (heading, _, text) = text.partition('\n')
> + (leader, _, heading) = heading.partition(' ')
> + self._start_new_heading(heading, len(leader))
> + if text == '':
> + return
> +
> + node = self._make_section(None)
> + self._parse_text_into_node(text, node)
> + self._add_node_to_current_heading(node)
> + self._cur_doc = None
> +
> + def _parse_text_into_node(self, doctext, node):
> + """Parse a chunk of QAPI-doc-format text into the node
> +
> + The doc comment can contain most inline rST markup, including
> + bulleted and enumerated lists.
> + As an extra permitted piece of markup, @var will be turned
> + into ``var``.
> + """
> +
> + # Handle the "@var means ``var`` case
> + doctext = re.sub(r'@([\w-]+)', r'``\1``', doctext)
> +
> + rstlist = ViewList()
> + for line in doctext.splitlines():
> + # The reported line number will always be that of the start line
> + # of the doc comment, rather than the actual location of the error.
> + # Being more precise would require overhaul of the QAPIDoc class
> + # to track lines more exactly within all the sub-parts of the doc
> + # comment, as well as counting lines here.
> + rstlist.append(line, self._cur_doc.info.fname,
> + self._cur_doc.info.line)
> + # Append a blank line -- in some cases rST syntax errors get
> + # attributed to the line after one with actual text, and if there
> + # isn't anything in the ViewList corresponding to that then Sphinx
> + # 1.6's AutodocReporter will then misidentify the source/line location
> + # in the error message (usually attributing it to the top-level
> + # .rst file rather than the offending .json file). The extra blank
> + # line won't affect the rendered output.
> + rstlist.append("", self._cur_doc.info.fname, self._cur_doc.info.line)
> + self._sphinx_directive.do_parse(rstlist, node)
> +
> + def get_document_nodes(self):
> + """Return the list of docutils nodes which make up the document"""
> + return self._top_node.children
> +
> +
> +class QAPISchemaGenDepVisitor(QAPISchemaVisitor):
> + """A QAPI schema visitor which adds Sphinx dependencies each module
> +
> + This class calls the Sphinx note_dependency() function to tell Sphinx
> + that the generated documentation output depends on the input
> + schema file associated with each module in the QAPI input.
> + """
> + def __init__(self, env, qapidir):
> + self._env = env
> + self._qapidir = qapidir
> +
> + def visit_module(self, name):
> + if name is not None:
> + qapifile = self._qapidir + '/' + name
> + self._env.note_dependency(os.path.abspath(qapifile))
> + super().visit_module(name)
> +
> +
> +class QAPIDocDirective(Directive):
> + """Extract documentation from the specified QAPI .json file"""
> + required_argument = 1
> + optional_arguments = 1
> + option_spec = {
> + 'qapifile': directives.unchanged_required
> + }
> + has_content = False
> +
> + def new_serialno(self):
> + """Return a unique new ID string suitable for use as a node's ID"""
> + env = self.state.document.settings.env
> + return 'qapidoc-%d' % env.new_serialno('qapidoc')
> +
> + def run(self):
> + env = self.state.document.settings.env
> + qapifile = env.config.qapidoc_srctree + '/' + self.arguments[0]
> + qapidir = os.path.dirname(qapifile)
> +
> + try:
> + schema = QAPISchema(qapifile)
> +
> + # First tell Sphinx about all the schema files that the
> + # output documentation depends on (including 'qapifile' itself)
> + schema.visit(QAPISchemaGenDepVisitor(env, qapidir))
> +
> + vis = QAPISchemaGenRSTVisitor(self)
> + vis.visit_begin(schema)
> + for doc in schema.docs:
> + if doc.symbol:
> + vis.symbol(doc, schema.lookup_entity(doc.symbol))
> + else:
> + vis.freeform(doc)
> + return vis.get_document_nodes()
> + except QAPIError as err:
> + # Launder QAPI parse errors into Sphinx extension errors
> + # so they are displayed nicely to the user
> + raise ExtensionError(str(err))
> +
> + def do_parse(self, rstlist, node):
> + """Parse rST source lines and add them to the specified node
> +
> + Take the list of rST source lines rstlist, parse them as
> + rST, and add the resulting docutils nodes as children of node.
> + The nodes are parsed in a way that allows them to include
> + subheadings (titles) without confusing the rendering of
> + anything else.
> + """
> + # This is from kerneldoc.py -- it works around an API change in
> + # Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use
> + # sphinx.util.nodes.nested_parse_with_titles() rather than the
> + # plain self.state.nested_parse(), and so we can drop the saving
> + # of title_styles and section_level that kerneldoc.py does,
> + # because nested_parse_with_titles() does that for us.
> + if Use_SSI:
> + with switch_source_input(self.state, rstlist):
> + nested_parse_with_titles(self.state, rstlist, node)
> + else:
> + save = self.state.memo.reporter
> + self.state.memo.reporter = AutodocReporter(
> + rstlist, self.state.memo.reporter)
> + try:
> + nested_parse_with_titles(self.state, rstlist, node)
> + finally:
> + self.state.memo.reporter = save
> +
> +
> +def setup(app):
> + """ Register qapi-doc directive with Sphinx"""
> + app.add_config_value('qapidoc_srctree', None, 'env')
> + app.add_directive('qapi-doc', QAPIDocDirective)
> +
> + return dict(
> + version=__version__,
> + parallel_read_safe=True,
> + parallel_write_safe=True
> + )
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 5eed1e692b4..dbddb0a7635 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3149,6 +3149,7 @@ M: Peter Maydell <peter.maydell@linaro.org>
> S: Maintained
> F: docs/conf.py
> F: docs/*/conf.py
> +F: docs/sphinx/
>
> Miscellaneous
> -------------
Maintainers of scripts/qapi/ should help review patches to
docs/sphinx/qapidoc.py. Two options:
* Add docs/sphinx/qapidoc.py to section QAPI. The QAPI maintainers
become M: of this small part of "Sphinx documentation configuration
and build machinery". R: would be more accurate. The inaccuracy
feels harmless.
* Do nothing. scripts/get_maintainer.pl won't loop in the QAPI
maintainers. The Sphinx documentation maintainers may have to do that
manually.
What do you think?
My review is of uneven value: the QAPI-facing parts are good, the
Sphinx-related parts merely look good to ignorant me. I guess that's
good enough, since we also have "generated docs look sane".
Preferably tidied up a bit more:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST
2020-09-25 16:23 ` [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST Peter Maydell
@ 2020-09-29 8:20 ` Markus Armbruster
2020-09-29 9:26 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 8:20 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> Convert qemu-ga-ref to rST format. This includes dropping
> the plain-text, pdf and info format outputs for this document;
> as with all our other Sphinx-based documentation, we provide
> HTML and manpage only.
>
> The qemu-ga-ref.rst is somewhat more stripped down than
> the .texi was, because we do not (currently) attempt to
> generate indexes for the commands, events and data types
> being documented.
>
> As the GA ref is now part of the Sphinx 'interop' manual,
> we can delete the direct link from index.html.in.
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> docs/index.html.in | 1 -
> docs/interop/conf.py | 2 +
> docs/interop/index.rst | 1 +
> docs/interop/qemu-ga-ref.rst | 9 ++++
> docs/interop/qemu-ga-ref.texi | 80 -----------------------------------
> docs/meson.build | 1 +
> meson.build | 3 --
> MAINTAINERS | 2 +-
> qga/meson.build | 3 +-
> 9 files changed, 15 insertions(+), 87 deletions(-)
> create mode 100644 docs/interop/qemu-ga-ref.rst
> delete mode 100644 docs/interop/qemu-ga-ref.texi
>
> diff --git a/docs/index.html.in b/docs/index.html.in
> index ca28047881e..5b0f4e2bc2c 100644
> --- a/docs/index.html.in
> +++ b/docs/index.html.in
> @@ -13,7 +13,6 @@
> <li><a href="interop/index.html">System Emulation Management and Interoperability Guide</a></li>
> <li><a href="specs/index.html">System Emulation Guest Hardware Specifications</a></li>
> <li><a href="interop/qemu-qmp-ref.html">QMP Reference Manual</a></li>
> - <li><a href="interop/qemu-ga-ref.html">Guest Agent Protocol Reference</a></li>
> </ul>
> </body>
> </html>
> diff --git a/docs/interop/conf.py b/docs/interop/conf.py
> index 42ce7e3d365..e83632e0108 100644
> --- a/docs/interop/conf.py
> +++ b/docs/interop/conf.py
> @@ -19,4 +19,6 @@ html_theme_options['description'] = u'System Emulation Management and Interopera
> man_pages = [
> ('qemu-ga', 'qemu-ga', u'QEMU Guest Agent',
> ['Michael Roth <mdroth@linux.vnet.ibm.com>'], 8),
> + ('qemu-ga-ref', 'qemu-ga-ref', u'QEMU Guest Agent Protocol Reference',
> + [], 7),
> ]
Why do you make the description a unicode legacy literal? I see it
matches existing entries. I'd like to know regardless :)
For what it's worth, the kernel's Documentation/conf.py uses a plain
string there.
> diff --git a/docs/interop/index.rst b/docs/interop/index.rst
> index 006f9864208..738cdbe185e 100644
> --- a/docs/interop/index.rst
> +++ b/docs/interop/index.rst
> @@ -18,6 +18,7 @@ Contents:
> live-block-operations
> pr-helper
> qemu-ga
> + qemu-ga-ref
> vhost-user
> vhost-user-gpu
> vhost-vdpa
> diff --git a/docs/interop/qemu-ga-ref.rst b/docs/interop/qemu-ga-ref.rst
> new file mode 100644
> index 00000000000..669019de71d
> --- /dev/null
> +++ b/docs/interop/qemu-ga-ref.rst
> @@ -0,0 +1,9 @@
> +QEMU Guest Agent Protocol Reference
> +===================================
> +
> +..
> + TODO: the old texinfo manual used to note that this manual
> + is GPL-v2-or-later. We should make that reader-visible
> + both here and in our Sphinx manuals more generally.
> +
> +.. qapi-doc:: qga/qapi-schema.json
> diff --git a/docs/interop/qemu-ga-ref.texi b/docs/interop/qemu-ga-ref.texi
> deleted file mode 100644
> index a23cc2ed7f6..00000000000
> --- a/docs/interop/qemu-ga-ref.texi
> +++ /dev/null
> @@ -1,80 +0,0 @@
> -\input texinfo
> -@setfilename qemu-ga-ref.info
> -
> -@include version.texi
> -
> -@exampleindent 0
> -@paragraphindent 0
> -
> -@settitle QEMU Guest Agent Protocol Reference
> -
> -@iftex
> -@center @image{docs/qemu_logo}
> -@end iftex
> -
> -@copying
> -This is the QEMU Guest Agent Protocol reference manual.
> -
> -Copyright @copyright{} 2016 The QEMU Project developers
> -
> -@quotation
> -This manual is free documentation: you can redistribute it and/or
> -modify it under the terms of the GNU General Public License as
> -published by the Free Software Foundation, either version 2 of the
> -License, or (at your option) any later version.
> -
> -This manual is distributed in the hope that it will be useful, but
> -WITHOUT ANY WARRANTY; without even the implied warranty of
> -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> -General Public License for more details.
> -
> -You should have received a copy of the GNU General Public License
> -along with this manual. If not, see http://www.gnu.org/licenses/.
> -@end quotation
> -@end copying
> -
> -@dircategory QEMU
> -@direntry
> -* QEMU-GA-Ref: (qemu-ga-ref). QEMU Guest Agent Protocol Reference
> -@end direntry
> -
> -@titlepage
> -@title Guest Agent Protocol Reference Manual
> -@subtitle QEMU version @value{VERSION}
There is no obvious equivalent to @value{VERSION} in
docs/interop/qemu-ga-ref.rst.
The manual page generated from it has the version in the footer. Good.
I can't find it in the generated HTML. Not so good, but it wasn't there
before the patch, either.
The generated PDF had it on the title page.
Suggest to add a TODO comment like the one about the licensing
information.
> -@page
> -@vskip 0pt plus 1filll
> -@insertcopying
> -@end titlepage
> -
> -@contents
> -
> -@ifnottex
> -@node Top
> -@top QEMU Guest Agent protocol reference
> -@end ifnottex
> -
> -@menu
> -* API Reference::
> -* Commands and Events Index::
> -* Data Types Index::
> -@end menu
> -
> -@node API Reference
> -@chapter API Reference
> -
> -@c for texi2pod:
> -@c man begin DESCRIPTION
> -
> -@include qga/qga-qapi-doc.texi
> -
> -@c man end
> -
> -@node Commands and Events Index
> -@unnumbered Commands and Events Index
> -@printindex fn
> -
> -@node Data Types Index
> -@unnumbered Data Types Index
> -@printindex tp
> -
> -@bye
> diff --git a/docs/meson.build b/docs/meson.build
> index 50f367349b8..2569dd52ad6 100644
> --- a/docs/meson.build
> +++ b/docs/meson.build
> @@ -15,6 +15,7 @@ if build_docs
> man_pages = {
> 'interop' : {
> 'qemu-ga.8': (have_tools ? 'man8' : ''),
> + 'qemu-ga-ref.7': 'man7',
> },
> 'tools': {
> 'qemu-img.1': (have_tools ? 'man1' : ''),
> diff --git a/meson.build b/meson.build
> index f4d1ab10968..c01c6230b3f 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1220,9 +1220,6 @@ if build_docs
> texi = {
> 'qemu-qmp-ref': ['docs/interop/qemu-qmp-ref.texi', qapi_doc_texi, version_texi],
> }
> - if 'CONFIG_GUEST_AGENT' in config_host
> - texi += {'qemu-ga-ref': ['docs/interop/qemu-ga-ref.texi', qga_qapi_doc_texi, version_texi]}
> - endif
>
> if makeinfo.found()
> cmd = [
> diff --git a/MAINTAINERS b/MAINTAINERS
> index dbddb0a7635..4b2705fc143 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2418,9 +2418,9 @@ M: Michael Roth <mdroth@linux.vnet.ibm.com>
> S: Maintained
> F: qga/
> F: docs/interop/qemu-ga.rst
> +F: docs/interop/qemu-ga-ref.rst
> F: scripts/qemu-guest-agent/
> F: tests/test-qga.c
> -F: docs/interop/qemu-ga-ref.texi
> T: git https://github.com/mdroth/qemu.git qga
>
> QOM
> diff --git a/qga/meson.build b/qga/meson.build
> index e5c5778a3e0..1c312b50cc9 100644
> --- a/qga/meson.build
> +++ b/qga/meson.build
> @@ -16,7 +16,7 @@ qga_qapi_outputs = [
> ]
>
> qga_qapi_files = custom_target('QGA QAPI files',
> - output: qga_qapi_outputs + ['qga-qapi-doc.texi'],
> + output: qga_qapi_outputs,
> input: 'qapi-schema.json',
> command: [ qapi_gen, '-o', 'qga', '-p', 'qga-', '@INPUT0@' ],
> depend_files: qapi_gen_depends)
> @@ -27,7 +27,6 @@ foreach output: qga_qapi_outputs
> qga_ss.add(qga_qapi_files[i])
> i = i + 1
> endforeach
> -qga_qapi_doc_texi = qga_qapi_files[i]
>
> qga_ss.add(files(
> 'commands.c',
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref to rST
2020-09-25 16:23 ` [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref " Peter Maydell
@ 2020-09-29 8:27 ` Markus Armbruster
2020-09-29 9:41 ` Peter Maydell
2020-09-29 8:42 ` Markus Armbruster
1 sibling, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 8:27 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> Convert qemu-qmp-ref to rST format. This includes dropping
> the plain-text, pdf and info format outputs for this document;
> as with all our other Sphinx-based documentation, we provide
> HTML and manpage only.
>
> The qemu-qmp-ref.rst is somewhat more stripped down than
> the .texi was, because we do not (currently) attempt to
> generate indexes for the commands, events and data types
> being documented.
>
> Again, we drop the direct link from index.html.in now that
> the QMP ref is part of the interop manual.
>
> This commit removes the code from the root meson.build file that
> handled the various Texinfo-based outputs, because we no longer
> generate any documentation except for the Sphinx HTML manuals and the
> manpages, and the code can't handle having an empty list of files
> to process.. We'll do further cleanup of the remainders of
Extra period.
> Texinfo support in subsequent commits.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> docs/index.html.in | 1 -
> docs/interop/conf.py | 2 +
> docs/interop/index.rst | 1 +
> docs/interop/qemu-qmp-ref.rst | 9 ++++
> docs/interop/qemu-qmp-ref.texi | 80 ---------------------------------
> docs/meson.build | 1 +
> meson.build | 82 ----------------------------------
> qapi/meson.build | 4 +-
> 8 files changed, 14 insertions(+), 166 deletions(-)
> create mode 100644 docs/interop/qemu-qmp-ref.rst
> delete mode 100644 docs/interop/qemu-qmp-ref.texi
>
> diff --git a/docs/index.html.in b/docs/index.html.in
> index 5b0f4e2bc2c..33db4396ac8 100644
> --- a/docs/index.html.in
> +++ b/docs/index.html.in
> @@ -12,7 +12,6 @@
> <li><a href="tools/index.html">Tools Guide</a></li>
> <li><a href="interop/index.html">System Emulation Management and Interoperability Guide</a></li>
> <li><a href="specs/index.html">System Emulation Guest Hardware Specifications</a></li>
> - <li><a href="interop/qemu-qmp-ref.html">QMP Reference Manual</a></li>
> </ul>
> </body>
> </html>
> diff --git a/docs/interop/conf.py b/docs/interop/conf.py
> index e83632e0108..43de386d33d 100644
> --- a/docs/interop/conf.py
> +++ b/docs/interop/conf.py
> @@ -21,4 +21,6 @@ man_pages = [
> ['Michael Roth <mdroth@linux.vnet.ibm.com>'], 8),
> ('qemu-ga-ref', 'qemu-ga-ref', u'QEMU Guest Agent Protocol Reference',
> [], 7),
> + ('qemu-qmp-ref', 'qemu-qmp-ref', u'QEMU QMP Reference Manual',
> + [], 7),
> ]
> diff --git a/docs/interop/index.rst b/docs/interop/index.rst
> index 738cdbe185e..cd78d679d82 100644
> --- a/docs/interop/index.rst
> +++ b/docs/interop/index.rst
> @@ -19,6 +19,7 @@ Contents:
> pr-helper
> qemu-ga
> qemu-ga-ref
> + qemu-qmp-ref
> vhost-user
> vhost-user-gpu
> vhost-vdpa
> diff --git a/docs/interop/qemu-qmp-ref.rst b/docs/interop/qemu-qmp-ref.rst
> new file mode 100644
> index 00000000000..0c416fcaac2
> --- /dev/null
> +++ b/docs/interop/qemu-qmp-ref.rst
> @@ -0,0 +1,9 @@
> +QEMU QMP Reference Manual
> +=========================
> +
> +..
> + TODO: the old texinfo manual used to note that this manual
> + is GPL-v2-or-later. We should make that reader-visible
> + both here and in our Sphinx manuals more generally.
> +
> +.. qapi-doc:: qapi/qapi-schema.json
> diff --git a/docs/interop/qemu-qmp-ref.texi b/docs/interop/qemu-qmp-ref.texi
> deleted file mode 100644
> index ea1d7fe6c2d..00000000000
> --- a/docs/interop/qemu-qmp-ref.texi
> +++ /dev/null
> @@ -1,80 +0,0 @@
> -\input texinfo
> -@setfilename qemu-qmp-ref.info
> -
> -@include version.texi
> -
> -@exampleindent 0
> -@paragraphindent 0
> -
> -@settitle QEMU QMP Reference Manual
> -
> -@iftex
> -@center @image{docs/qemu_logo}
> -@end iftex
> -
> -@copying
> -This is the QEMU QMP reference manual.
> -
> -Copyright @copyright{} 2016 The QEMU Project developers
> -
> -@quotation
> -This manual is free documentation: you can redistribute it and/or
> -modify it under the terms of the GNU General Public License as
> -published by the Free Software Foundation, either version 2 of the
> -License, or (at your option) any later version.
> -
> -This manual is distributed in the hope that it will be useful, but
> -WITHOUT ANY WARRANTY; without even the implied warranty of
> -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> -General Public License for more details.
> -
> -You should have received a copy of the GNU General Public License
> -along with this manual. If not, see http://www.gnu.org/licenses/.
> -@end quotation
> -@end copying
> -
> -@dircategory QEMU
> -@direntry
> -* QEMU-QMP-Ref: (qemu-qmp-ref). QEMU QMP Reference Manual
> -@end direntry
> -
> -@titlepage
> -@title QMP Reference Manual
> -@subtitle QEMU version @value{VERSION}
> -@page
> -@vskip 0pt plus 1filll
> -@insertcopying
> -@end titlepage
> -
> -@contents
> -
> -@ifnottex
> -@node Top
> -@top QEMU QMP reference
> -@end ifnottex
> -
> -@menu
> -* API Reference::
> -* Commands and Events Index::
> -* Data Types Index::
> -@end menu
> -
> -@node API Reference
> -@chapter API Reference
> -
> -@c for texi2pod:
> -@c man begin DESCRIPTION
> -
> -@include qapi/qapi-doc.texi
> -
> -@c man end
> -
> -@node Commands and Events Index
> -@unnumbered Commands and Events Index
> -@printindex fn
> -
> -@node Data Types Index
> -@unnumbered Data Types Index
> -@printindex tp
> -
> -@bye
> diff --git a/docs/meson.build b/docs/meson.build
> index 2569dd52ad6..69097e2ca07 100644
> --- a/docs/meson.build
> +++ b/docs/meson.build
> @@ -16,6 +16,7 @@ if build_docs
> 'interop' : {
> 'qemu-ga.8': (have_tools ? 'man8' : ''),
> 'qemu-ga-ref.7': 'man7',
> + 'qemu-qmp-ref.7': 'man7',
> },
> 'tools': {
> 'qemu-img.1': (have_tools ? 'man1' : ''),
> diff --git a/meson.build b/meson.build
> index c01c6230b3f..73d675ca834 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1203,88 +1203,6 @@ if 'CONFIG_GTK' in config_host
> subdir('po')
> endif
>
> -if build_docs
> - makeinfo = find_program('makeinfo', required: build_docs)
> -
> - docs_inc = [
> - '-I', meson.current_source_dir(),
> - '-I', meson.current_build_dir() / 'docs',
> - '-I', '@OUTDIR@',
> - ]
> -
> - version_texi = configure_file(output: 'version.texi',
> - input: 'version.texi.in',
> - configuration: {'VERSION': meson.project_version(),
> - 'qemu_confdir': config_host['qemu_confdir']})
> -
> - texi = {
> - 'qemu-qmp-ref': ['docs/interop/qemu-qmp-ref.texi', qapi_doc_texi, version_texi],
> - }
> -
> - if makeinfo.found()
> - cmd = [
> - 'env', 'LC_ALL=C', makeinfo, '--no-split', '--number-sections', docs_inc,
> - '@INPUT0@', '-o', '@OUTPUT@',
> - ]
> - foreach ext, args: {
> - 'info': [],
> - 'html': ['--no-headers', '--html'],
> - 'txt': ['--no-headers', '--plaintext'],
> - }
> - t = []
> - foreach doc, input: texi
> - output = doc + '.' + ext
> - t += custom_target(output,
> - input: input,
> - output: output,
> - install: true,
> - install_dir: qemu_docdir / 'interop',
> - command: cmd + args)
> - endforeach
> - alias_target(ext, t)
> - endforeach
> - endif
> -
> - texi2pdf = find_program('texi2pdf', required: false)
> -
> - if texi2pdf.found()
> - pdfs = []
> - foreach doc, input: texi
> - output = doc + '.pdf'
> - pdfs += custom_target(output,
> - input: input,
> - output: output,
> - command: [texi2pdf, '-q', docs_inc, '@INPUT0@', '-o', '@OUTPUT@'],
> - build_by_default: false)
> - endforeach
> - alias_target('pdf', pdfs)
> - endif
> -
> - texi2pod = find_program('scripts/texi2pod.pl')
> - pod2man = find_program('pod2man', required: build_docs)
> -
> - if pod2man.found()
> - foreach doc, input: texi
> - man = doc + '.7'
> - pod = custom_target(man + '.pod',
> - input: input,
> - output: man + '.pod',
> - command: [texi2pod,
> - '-DVERSION="' + meson.project_version() + '"',
> - '-DCONFDIR="' + config_host['qemu_confdir'] + '"',
> - '@INPUT0@', '@OUTPUT@'])
> - man = custom_target(man,
> - input: pod,
> - output: man,
> - capture: true,
> - install: true,
> - install_dir: get_option('mandir') / 'man7',
> - command: [pod2man, '--utf8', '--section=7', '--center=" "',
> - '--release=" "', '@INPUT@'])
> - endforeach
> - endif
> -endif
> -
> if host_machine.system() == 'windows'
> nsis_cmd = [
> find_program('scripts/nsis.py'),
> diff --git a/qapi/meson.build b/qapi/meson.build
> index 2b2872a41d8..a287ca5d9d7 100644
> --- a/qapi/meson.build
> +++ b/qapi/meson.build
> @@ -97,7 +97,7 @@ foreach module : qapi_all_modules
> endforeach
>
> qapi_files = custom_target('shared QAPI source files',
> - output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs + ['qapi-doc.texi'],
> + output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs,
> input: [ files('qapi-schema.json') ],
> command: [ qapi_gen, '-o', 'qapi', '-b', '@INPUT0@' ],
> depend_files: [ qapi_inputs, qapi_gen_depends ])
> @@ -121,5 +121,3 @@ foreach output : qapi_specific_outputs + qapi_nonmodule_outputs
> specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: qapi_files[i])
> i = i + 1
> endforeach
> -
> -qapi_doc_texi = qapi_files[i]
Doesn't storage-daemon/qapi/meson.build need a similar update?
My comments on the previous patch apply.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 11/21] qga/qapi-schema.json: Add some headings
2020-09-25 16:23 ` [PATCH v6 11/21] qga/qapi-schema.json: Add some headings Peter Maydell
@ 2020-09-29 8:30 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 8:30 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> Add some section headings to the QGA json; this is purely so that we
> have some H1 headings, as otherwise each command ends up being
> visible in the interop/ manual's table of contents. In an ideal
> world there might be a proper 'Introduction' section the way there is
> in qapi/qapi-schema.json.
Definitely.
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> qga/qapi-schema.json | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
> index d2ea7446db6..cec98c7e065 100644
> --- a/qga/qapi-schema.json
> +++ b/qga/qapi-schema.json
> @@ -2,14 +2,16 @@
> # vim: filetype=python
>
> ##
> -#
> -# General note concerning the use of guest agent interfaces:
> +# = General note concerning the use of guest agent interfaces
> #
> # "unsupported" is a higher-level error than the errors that individual
> # commands might document. The caller should always be prepared to receive
> # QERR_UNSUPPORTED, even if the given command doesn't specify it, or doesn't
> # document any failure mode at all.
> -#
> +##
> +
> +##
> +# = QEMU guest agent protocol commands and structs
> ##
>
> { 'pragma': { 'doc-required': true } }
Good enough for now.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 12/21] tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis
2020-09-25 16:23 ` [PATCH v6 12/21] tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis Peter Maydell
@ 2020-09-29 8:33 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 8:33 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> doc-good.json currently uses the old *strong* and _emphasis_ markup.
> As part of the conversion to rST this needs to switch to **strong**
> and *emphasis*, because rST uses underscores as part of its markup
> of hyperlinks and will otherwise warn about the syntax error.
>
> In commit a660eed482063b we fixed up the in-tree uses of the
> old markup:
> 1) _this_ was replaced with *this*
> 2) the only in-tree use of *this* was left alone (turning
> a 'strong' into an 'emphasis')
> (and so currently in-tree nothing is using either new-style
> **strong** or old-style _emphasis_).
>
> Update doc-good.json in a similar way:
> 1) replace _this_ with *this*
> 2) remove the usage of old-style *this*
>
> (This slightly reduces the coverage for the old Texinfo generator,
> which is about to go away, but is fine for the new rST generator
> because that does not need to handle strong/emphasis itself because
> it is simply passing the entire text as raw rST to Sphinx.)
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref to rST
2020-09-25 16:23 ` [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref " Peter Maydell
2020-09-29 8:27 ` Markus Armbruster
@ 2020-09-29 8:42 ` Markus Armbruster
2020-09-29 9:46 ` Peter Maydell
1 sibling, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 8:42 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Appears to break documented make target html:
$ make -C bld-x86 html
make: Entering directory '/work/armbru/qemu/bld-x86'
make: *** No rule to make target 'html'. Stop.
make: Leaving directory '/work/armbru/qemu/bld-x86'
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 13/21] meson.build: Move SPHINX_ARGS to top level meson.build file
2020-09-25 16:23 ` [PATCH v6 13/21] meson.build: Move SPHINX_ARGS to top level meson.build file Peter Maydell
@ 2020-09-29 8:45 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 8:45 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> We're going to want to use SPHINX_ARGS in both docs/meson.build
> and tests/qapi-schema/meson.build. Move the definition up to the
> top level file so it is available to both subdirectories.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> docs/meson.build | 8 --------
> meson.build | 8 ++++++++
> 2 files changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/docs/meson.build b/docs/meson.build
> index 69097e2ca07..99da609e813 100644
> --- a/docs/meson.build
> +++ b/docs/meson.build
> @@ -1,11 +1,3 @@
> -SPHINX_ARGS = [config_host['SPHINX_BUILD'],
> - '-Dversion=' + meson.project_version(),
> - '-Drelease=' + config_host['PKGVERSION']]
> -
> -if get_option('werror')
> - SPHINX_ARGS += [ '-W' ]
> -endif
> -
> if build_docs
> configure_file(output: 'index.html',
> input: files('index.html.in'),
> diff --git a/meson.build b/meson.build
> index 73d675ca834..6408ad442ea 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -671,6 +671,14 @@ foreach d : hx_headers
> endforeach
> genh += hxdep
>
> +SPHINX_ARGS = [config_host['SPHINX_BUILD'],
> + '-Dversion=' + meson.project_version(),
> + '-Drelease=' + config_host['PKGVERSION']]
> +
> +if get_option('werror')
> + SPHINX_ARGS += [ '-W' ]
> +endif
> +
> # Collect sourcesets.
>
> util_ss = ss.source_set()
I double-checked we still pass SPHINX_ARGS to sphinx.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 14/21] meson.build: Make manuals depend on source to Sphinx extensions
2020-09-25 16:23 ` [PATCH v6 14/21] meson.build: Make manuals depend on source to Sphinx extensions Peter Maydell
@ 2020-09-29 8:52 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 8:52 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> The automatic dependency handling for Sphinx manuals only makes the output
> depend on the input documentation files. This means that if you edit
> the Python source of an extension then the documentation won't be
> rebuilt.
>
> Create a list of all the source files for the extensions and add
> it to the dependencies for the manuals. This is similar to how we
> handle the qapi_gen_depends list.
>
> Because we don't try to identify which manuals are using which
> Sphinx extensions, a change to the source of one extension will
> always rebuild the entire manual set, not merely the manuals
> which have changed. This is acceptable because we don't change
> the extensions all that often.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> docs/meson.build | 1 +
> meson.build | 8 ++++++++
> 2 files changed, 9 insertions(+)
>
> diff --git a/docs/meson.build b/docs/meson.build
> index 99da609e813..59fea873b10 100644
> --- a/docs/meson.build
> +++ b/docs/meson.build
> @@ -36,6 +36,7 @@ if build_docs
> output: [manual + '.stamp'],
> input: [files('conf.py'), files(manual / 'conf.py')],
> depfile: manual + '.d',
> + depend_files: sphinx_extn_depends,
> command: [SPHINX_ARGS, '-Ddepfile=@DEPFILE@',
> '-Ddepfile_stamp=@OUTPUT0@',
> '-b', 'html', '-d', private_dir,
> diff --git a/meson.build b/meson.build
> index 6408ad442ea..3c07e75dbd5 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -679,6 +679,14 @@ if get_option('werror')
> SPHINX_ARGS += [ '-W' ]
> endif
>
> +sphinx_extn_depends = [ meson.source_root() / 'docs/sphinx/depfile.py',
> + meson.source_root() / 'docs/sphinx/hxtool.py',
> + meson.source_root() / 'docs/sphinx/kerneldoc.py',
> + meson.source_root() / 'docs/sphinx/kernellog.py',
> + meson.source_root() / 'docs/sphinx/qapidoc.py',
> + meson.source_root() / 'docs/sphinx/qmp_lexer.py',
> + qapi_gen_depends ]
> +
> # Collect sourcesets.
>
> util_ss = ss.source_set()
I touched docs/sphinx/qapidoc.py and the manual got rebuilt. Good.
Same for scripts/qapi/gen.py. Good, except it makes me disbelieve your
claim "we don't change the extensions all that often."
Any ideas beyond amending the commit message with something like "except
for the QAPI guys, pity the fools"? ;)
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling
2020-09-28 19:15 ` Markus Armbruster
@ 2020-09-29 8:55 ` Peter Maydell
2020-09-29 13:03 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 8:55 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Mon, 28 Sep 2020 at 20:16, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
> > +Descriptions of arguments can span multiple lines. The description
> > +text can start on the line following the '@argname:', in which case
> > +it must not be indented at all. It can also start on the same line
> > +as the '@argname:'. In this case if it spans multiple lines then
> > +second and subsequent lines must be indented to line up with the
> > +first character of the first line of the description:
>
> Please put two spaces after sentence-ending punctuation, for local
> consistency, and to keep Emacs sentence commands working.
Is there a python lint program that can auto-check this?
Otherwise I am going to continue to put single-spaces at
least some of the time, because that's the way I write all
the other English text I write...
> Can touch this up in my tree, of course.
That would certainly be easier for me :-)
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension
2020-09-29 6:54 ` Markus Armbruster
@ 2020-09-29 9:05 ` Peter Maydell
0 siblings, 0 replies; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 9:05 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Tue, 29 Sep 2020 at 07:54, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > Some of our documentation is auto-generated from documentation
> > comments in the JSON schema.
> >
> > For Sphinx, rather than creating a file to include, the most natural
> > way to handle this is to have a small custom Sphinx extension which
> > processes the JSON file and inserts documentation into the rST
> > file being processed.
> >
> > This is the same approach that kerneldoc and hxtool use.
> >
> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>
> Down to a few trivial things, which I can tidy up in my tree.
Thanks, that would be great.
> > + def _serror(self, msg):
> > + """Raise an exception giving a user-friendly syntax error message"""
> > + file = self._cur_doc.info.fname
> > + line = self._cur_doc.info.line
> > + raise ExtensionError(
> > + '%s line %d: syntax error: %s' % (file, line, msg))
>
> Unused, let's drop.
Oops, yes. I refactored the error-handling code below and forgot
to delete this now-unused method.
> > + def visit_enum_type(self, name, info, ifcond, features, members, prefix):
> > + doc = self._cur_doc
> > + self._add_doc('Enum',
> > + self._nodes_for_enum_values(doc) +
> > + self._nodes_for_features(doc) +
> > + self._nodes_for_sections(doc) +
> > + self._nodes_for_if_section(ifcond))
>
> PEP 8: In Python code, it is permissible to break before or after a
> binary operator, as long as the convention is consistent locally. For
> new code Knuth's style is suggested.
>
> Mind switching to Knuth style, i.e. break before the operator?
Fine with me, I have no strong Python style preferences.
I'm unlikely to be able to reliably follow any particular
style rules in future code I write unless there's a linter
that complains about violations, though...
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 5eed1e692b4..dbddb0a7635 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -3149,6 +3149,7 @@ M: Peter Maydell <peter.maydell@linaro.org>
> > S: Maintained
> > F: docs/conf.py
> > F: docs/*/conf.py
> > +F: docs/sphinx/
> >
> > Miscellaneous
> > -------------
>
> Maintainers of scripts/qapi/ should help review patches to
> docs/sphinx/qapidoc.py. Two options:
>
> * Add docs/sphinx/qapidoc.py to section QAPI. The QAPI maintainers
> become M: of this small part of "Sphinx documentation configuration
> and build machinery". R: would be more accurate. The inaccuracy
> feels harmless.
>
> * Do nothing. scripts/get_maintainer.pl won't loop in the QAPI
> maintainers. The Sphinx documentation maintainers may have to do that
> manually.
>
> What do you think?
No strong preference. I guess putting qapidoc.py in the QAPI
section means that the right people get cc'd so that's the
better approach.
> My review is of uneven value: the QAPI-facing parts are good, the
> Sphinx-related parts merely look good to ignorant me. I guess that's
> good enough, since we also have "generated docs look sane".
>
> Preferably tidied up a bit more:
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST
2020-09-29 8:20 ` Markus Armbruster
@ 2020-09-29 9:26 ` Peter Maydell
2020-09-29 13:11 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 9:26 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Tue, 29 Sep 2020 at 09:20, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > Convert qemu-ga-ref to rST format. This includes dropping
> > the plain-text, pdf and info format outputs for this document;
> > as with all our other Sphinx-based documentation, we provide
> > HTML and manpage only.
> >
> > --- a/docs/interop/conf.py
> > +++ b/docs/interop/conf.py
> > @@ -19,4 +19,6 @@ html_theme_options['description'] = u'System Emulation Management and Interopera
> > man_pages = [
> > ('qemu-ga', 'qemu-ga', u'QEMU Guest Agent',
> > ['Michael Roth <mdroth@linux.vnet.ibm.com>'], 8),
> > + ('qemu-ga-ref', 'qemu-ga-ref', u'QEMU Guest Agent Protocol Reference',
> > + [], 7),
> > ]
>
> Why do you make the description a unicode legacy literal? I see it
> matches existing entries. I'd like to know regardless :)
I was probably just copying some other example of how to
write the man_pages[] definition. This also all used to have
to work with Python 2.7, which might or might not be relevant here.
> > -@titlepage
> > -@title Guest Agent Protocol Reference Manual
> > -@subtitle QEMU version @value{VERSION}
>
> There is no obvious equivalent to @value{VERSION} in
> docs/interop/qemu-ga-ref.rst.
>
> The manual page generated from it has the version in the footer. Good.
>
> I can't find it in the generated HTML. Not so good, but it wasn't there
> before the patch, either.
>
> The generated PDF had it on the title page.
>
> Suggest to add a TODO comment like the one about the licensing
> information.
So the version is in the manual page, as it was before the conversion,
and it's not in the HTML version, which it wasn't before the
conversion. That doesn't sound to me like there's anything here
to do... You can add a TODO if you want one, of course.
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref to rST
2020-09-29 8:27 ` Markus Armbruster
@ 2020-09-29 9:41 ` Peter Maydell
2020-09-29 13:12 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 9:41 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Tue, 29 Sep 2020 at 09:28, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
> > diff --git a/qapi/meson.build b/qapi/meson.build
> > index 2b2872a41d8..a287ca5d9d7 100644
> > --- a/qapi/meson.build
> > +++ b/qapi/meson.build
> > @@ -97,7 +97,7 @@ foreach module : qapi_all_modules
> > endforeach
> >
> > qapi_files = custom_target('shared QAPI source files',
> > - output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs + ['qapi-doc.texi'],
> > + output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs,
> > input: [ files('qapi-schema.json') ],
> > command: [ qapi_gen, '-o', 'qapi', '-b', '@INPUT0@' ],
> > depend_files: [ qapi_inputs, qapi_gen_depends ])
> > @@ -121,5 +121,3 @@ foreach output : qapi_specific_outputs + qapi_nonmodule_outputs
> > specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: qapi_files[i])
> > i = i + 1
> > endforeach
> > -
> > -qapi_doc_texi = qapi_files[i]
>
> Doesn't storage-daemon/qapi/meson.build need a similar update?
I was previously unaware of storage-daemon/qapi...
It looks like we don't actually do anything with the generated
qapi-doc.texi there, so I'm not sure why we were listing it as an output.
I think we should only need to remove the " + ['qapi-doc.texi']"
in storage-daemon/qapi/meson.build, and that should be a separate
change after this one and before we remove scripts/qapi/doc.py.
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref to rST
2020-09-29 8:42 ` Markus Armbruster
@ 2020-09-29 9:46 ` Peter Maydell
2020-09-29 13:13 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 9:46 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Tue, 29 Sep 2020 at 09:42, Markus Armbruster <armbru@redhat.com> wrote:
>
> Appears to break documented make target html:
>
> $ make -C bld-x86 html
> make: Entering directory '/work/armbru/qemu/bld-x86'
> make: *** No rule to make target 'html'. Stop.
> make: Leaving directory '/work/armbru/qemu/bld-x86'
Whoops. Should be fixable by adding
alias_target('html', sphinxdocs)
under the other two alias_target() calls at the bottom of
docs/meson.build, I think.
Looking at the code I think it also breaks the 'info',
'pdf' and 'txt' targets, which I propose that we fix
by removing them from the documentation, since not providing
info, pdf or txt output is an intentional change. I believe that
the only documentation we would need to update is the
$(call print-help,html info pdf txt man,Build documentation in
specified format)
line in Makefile.
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn
2020-09-25 16:23 ` [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn Peter Maydell
@ 2020-09-29 12:20 ` Markus Armbruster
2020-09-29 12:33 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 12:20 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
In subject, s/outputn/output/
Peter Maydell <peter.maydell@linaro.org> writes:
> Add a test of the rST output from the QAPI doc-comment generator,
> similar to what we currently have that tests the Texinfo output.
>
> This is a bit more awkward with Sphinx, because the generated
> output is not 100% under our control the way the QAPI-to-Texinfo
> generator was. However, in practice Sphinx's plaintext output
> generation has been identical between at least Sphinx 1.6 and
> 3.0, so we use that. (The HTML output has had changes across
> versions). We use an exact-match comparison check, with the
> understanding that perhaps changes in a future Sphinx version
> might require us to implement something more clever to cope
> with variation in the output.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
It's not just the potential Sphinx version dependence that makes this
awkward.
We can no longer check what our doc generator does (at least not without
substantial additional coding), we can only check what it does together
with Sphinx. We do so for one output format.
Our doc generator output could change in ways that are not visible in
the Sphinx output format we test, but are visible in some other output
format.
We choose to test plain text, because it has the lowest risk of unwanted
Sphinx version dependence, even though it probably has the highest risk
of "rendering stuff invisible".
Certainly better than nothing, and probably the best we can do now, but
let's capture the tradeoff in the commit message. Perhaps:
This is a bit more awkward with Sphinx, because the generated output
is not 100% under our control the way the QAPI-to-Texinfo generator
was. We can't observe the data we generate, only the Sphinx
output. Two issues.
One, the output can vary with the Sphinx version. In practice
Sphinx's plaintext output generation has been identical between at
least Sphinx 1.6 and 3.0, so we use that. (The HTML output has had
changes across versions). We use an exact-match comparison check, with
the understanding that perhaps changes in a future Sphinx version
might require us to implement something more clever to cope with
variation in the output.
Two, the test can only protect us from changes in the data we generate
that are visible in plain text.
What do you think?
> ---
> The meson.build changes are remarkably clunky, but this
> appears to be unavoidable...
> ---
> tests/qapi-schema/doc-good.rst | 5 +
> tests/qapi-schema/doc-good.txt | 288 +++++++++++++++++++++++++++++++++
> tests/qapi-schema/meson.build | 55 +++++++
> 3 files changed, 348 insertions(+)
> create mode 100644 tests/qapi-schema/doc-good.rst
> create mode 100644 tests/qapi-schema/doc-good.txt
>
> diff --git a/tests/qapi-schema/doc-good.rst b/tests/qapi-schema/doc-good.rst
> new file mode 100644
> index 00000000000..1e4c23305a1
> --- /dev/null
> +++ b/tests/qapi-schema/doc-good.rst
> @@ -0,0 +1,5 @@
> +..
> + Test Sphinx manual that pulls in the test schema file. We will generate
> + a plain-text output file and compare it against a reference.
> +
> +.. qapi-doc:: tests/qapi-schema/doc-good.json
> diff --git a/tests/qapi-schema/doc-good.txt b/tests/qapi-schema/doc-good.txt
> new file mode 100644
> index 00000000000..6ca03d49d0d
> --- /dev/null
> +++ b/tests/qapi-schema/doc-good.txt
> @@ -0,0 +1,288 @@
> +Section
> +*******
> +
> +
> +Subsection
> +==========
> +
> +*with emphasis* "var" {in braces}
> +
> +* List item one
> +
> +* Two, multiple lines
> +
> +* Three Still in list
> +
> +Not in list
> +
> +* Second list Note: still in list
> +
> +Note: not in list
> +
> +1. Third list is numbered
> +
> +2. another item
> +
> +Returns: the King Since: the first age Notes:
> +
> +1. Lorem ipsum dolor sit amet
> +
> +2. Ut enim ad minim veniam
> +
> +Duis aute irure dolor
> +
> +Example:
> +
> +-> in <- out Examples: - *verbatim* - {braces}
> +
> +
> +"Enum" (Enum)
> +-------------
> +
> +
> +Values
> +~~~~~~
> +
> +"one" (**If: **"defined(IFONE)")
> + The _one_ {and only}
> +
> +"two"
> + Not documented
> +
> +
> +Features
> +~~~~~~~~
> +
> +"enum-feat"
> + Also _one_ {and only}
> +
> +"two" is undocumented
> +
> +
> +If
> +~~
> +
> +"defined(IFCOND)"
> +
> +
> +"Base" (Object)
> +---------------
> +
> +
> +Members
> +~~~~~~~
> +
> +"base1": "Enum"
> + the first member
> +
> +
> +"Variant1" (Object)
> +-------------------
> +
> +A paragraph
> +
> +Another paragraph (but no "var": line)
> +
> +
> +Members
> +~~~~~~~
> +
> +"var1": "string" (**If: **"defined(IFSTR)")
> + Not documented
> +
> +
> +Features
> +~~~~~~~~
> +
> +"variant1-feat"
> + a feature
> +
> +"member-feat"
> + a member feature
> +
> +
> +"Variant2" (Object)
> +-------------------
> +
> +
> +"Object" (Object)
> +-----------------
> +
> +
> +Members
> +~~~~~~~
> +
> +The members of "Base"
> +The members of "Variant1" when "base1" is ""one""
> +The members of "Variant2" when "base1" is ""two"" (**If: **"IFTWO")
> +
> +Features
> +~~~~~~~~
> +
> +"union-feat1"
> + a feature
> +
> +
> +"SugaredUnion" (Object)
> +-----------------------
> +
> +
> +Members
> +~~~~~~~
> +
> +"type"
> + One of "one", "two"
> +
> +"data": "Variant1" when "type" is ""one""
> +"data": "Variant2" when "type" is ""two"" (**If: **"IFTWO")
> +
> +Features
> +~~~~~~~~
> +
> +"union-feat2"
> + a feature
> +
> +
> +"Alternate" (Alternate)
> +-----------------------
> +
> +
> +Members
> +~~~~~~~
> +
> +"i": "int"
> + an integer "b" is undocumented
> +
> +"b": "boolean"
> + Not documented
> +
> +
> +Features
> +~~~~~~~~
> +
> +"alt-feat"
> + a feature
> +
> +
> +Another subsection
> +==================
> +
> +
> +"cmd" (Command)
> +---------------
> +
> +
> +Arguments
> +~~~~~~~~~
> +
> +"arg1": "int"
> + the first argument
> +
> +"arg2": "string" (optional)
> + the second argument
> +
> +"arg3": "boolean"
> + Not documented
> +
> +
> +Features
> +~~~~~~~~
> +
> +"cmd-feat1"
> + a feature
> +
> +"cmd-feat2"
> + another feature
> +
> +
> +Note
> +~~~~
> +
> +"arg3" is undocumented
> +
> +
> +Returns
> +~~~~~~~
> +
> +"Object"
> +
> +
> +TODO
> +~~~~
> +
> +frobnicate
> +
> +
> +Notes
> +~~~~~
> +
> +* Lorem ipsum dolor sit amet
> +
> +* Ut enim ad minim veniam
> +
> +Duis aute irure dolor
> +
> +
> +Example
> +~~~~~~~
> +
> + -> in
> + <- out
> +
> +
> +Examples
> +~~~~~~~~
> +
> + - *verbatim*
> + - {braces}
> +
> +
> +Since
> +~~~~~
> +
> +2.10
> +
> +
> +"cmd-boxed" (Command)
> +---------------------
> +
> +If you're bored enough to read this, go see a video of boxed cats
> +
> +
> +Arguments
> +~~~~~~~~~
> +
> +The members of "Object"
> +
> +Features
> +~~~~~~~~
> +
> +"cmd-feat1"
> + a feature
> +
> +"cmd-feat2"
> + another feature
> +
> +
> +Example
> +~~~~~~~
> +
> + -> in
> +
> + <- out
> +
> +
> +"EVT-BOXED" (Event)
> +-------------------
> +
> +
> +Arguments
> +~~~~~~~~~
> +
> +The members of "Object"
> +
> +Features
> +~~~~~~~~
> +
> +"feat3"
> + a feature
> diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
> index 83a0a68389b..0c4a6a2936f 100644
> --- a/tests/qapi-schema/meson.build
> +++ b/tests/qapi-schema/meson.build
> @@ -224,3 +224,58 @@ qapi_doc = custom_target('QAPI doc',
> test('QAPI doc', diff, args: ['-b', '-u', files('doc-good.texi'), qapi_doc[0].full_path()],
> depends: qapi_doc,
> suite: ['qapi-schema', 'qapi-doc'])
> +
> +# Test the document-comment document generation code by running a test schema
> +# file through Sphinx's plain-text builder and comparing the result against
> +# a golden reference. This is in theory susceptible to failures if Sphinx
> +# changes its output, but the text output has historically been very stable
> +# (no changes between Sphinx 1.6 and 3.0), so it is a better bet than
> +# texinfo or HTML generation, both of which have had changes. We might
Texinfo
> +# need to add more sophisticated logic here in future for some sort of
> +# fuzzy comparison if future Sphinx versions produce different text,
> +# but for now the simple comparison suffices.
> +qapi_doc_out = custom_target('QAPI rST doc',
> + output: ['doc-good.txt'],
> + input: files('doc-good.json', 'doc-good.rst'),
Gawk at my Meson ignorance...
Looks like this builds doc-good.txt from doc.good.json and doc-good.rst.
doc-good.txt is also a source file. Works, because we use a separate
build tree. Might be confusing, though.
> + build_by_default: build_docs,
> + depend_files: sphinx_extn_depends,
> + # We use -E to suppress Sphinx's caching, because
> + # we want it to always really run the QAPI doc
> + # generation code. It also means we don't
> + # clutter up the build dir with the cache.
> + command: [SPHINX_ARGS,
> + '-b', 'text', '-E',
> + '-c', meson.source_root() / 'docs',
> + '-D', 'master_doc=doc-good',
> + meson.current_source_dir(),
> + meson.current_build_dir()])
> +
> +# Fix possible inconsistency in line endings in generated output and
> +# in the golden reference (which could otherwise cause test failures
> +# on Windows hosts). Unfortunately diff --strip-trailing-cr
> +# is GNU-diff only. The odd-looking perl is because we must avoid
> +# using an explicit '\' character in the command arguments to
> +# a custom_target(), as Meson will unhelpfully replace it with a '/'
> +# (https://github.com/mesonbuild/meson/issues/1564)
Rather disappointing.
> +qapi_doc_out_nocr = custom_target('QAPI rST doc newline-sanitized',
> + output: ['doc-good.txt.nocr'],
> + input: qapi_doc_out[0],
> + build_by_default: build_docs,
> + command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
> + capture: true)
I figure this strips \r from the build tree's doc-good.txt.
> +
> +qapi_doc_ref_nocr = custom_target('QAPI rST doc reference newline-sanitized',
> + output: ['doc-good.ref.nocr'],
> + input: files('doc-good.txt'),
> + build_by_default: build_docs,
> + command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
> + capture: true)
Uh, this strips it from the source tree's doc-good.txt, right? Why is
that necessary?
Correcting myself: it *is* confusing.
> +
> +if build_docs
> + # "full_path()" needed here to work around
> + # https://github.com/mesonbuild/meson/issues/7585
> + test('QAPI rST doc', diff, args: ['-u', qapi_doc_ref_nocr[0].full_path(),
> + qapi_doc_out_nocr[0].full_path()],
> + depends: [qapi_doc_ref_nocr, qapi_doc_out_nocr],
> + suite: ['qapi-schema', 'qapi-doc'])
> +endif
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 16/21] scripts/qapi: Remove texinfo generation support
2020-09-25 16:23 ` [PATCH v6 16/21] scripts/qapi: Remove texinfo generation support Peter Maydell
@ 2020-09-29 12:22 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 12:22 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> We no longer use the generated texinfo format documentation,
> so delete the code that generates it, and the test case for
> the generation.
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn
2020-09-29 12:20 ` Markus Armbruster
@ 2020-09-29 12:33 ` Peter Maydell
2020-09-29 13:18 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 12:33 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Tue, 29 Sep 2020 at 13:20, Markus Armbruster <armbru@redhat.com> wrote:
>
> In subject, s/outputn/output/
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > Add a test of the rST output from the QAPI doc-comment generator,
> > similar to what we currently have that tests the Texinfo output.
> >
> > This is a bit more awkward with Sphinx, because the generated
> > output is not 100% under our control the way the QAPI-to-Texinfo
> > generator was. However, in practice Sphinx's plaintext output
> > generation has been identical between at least Sphinx 1.6 and
> > 3.0, so we use that. (The HTML output has had changes across
> > versions). We use an exact-match comparison check, with the
> > understanding that perhaps changes in a future Sphinx version
> > might require us to implement something more clever to cope
> > with variation in the output.
> >
> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>
> It's not just the potential Sphinx version dependence that makes this
> awkward.
>
> We can no longer check what our doc generator does (at least not without
> substantial additional coding), we can only check what it does together
> with Sphinx. We do so for one output format.
>
> Our doc generator output could change in ways that are not visible in
> the Sphinx output format we test, but are visible in some other output
> format.
>
> We choose to test plain text, because it has the lowest risk of unwanted
> Sphinx version dependence, even though it probably has the highest risk
> of "rendering stuff invisible".
>
> Certainly better than nothing, and probably the best we can do now, but
> let's capture the tradeoff in the commit message. Perhaps:
>
> This is a bit more awkward with Sphinx, because the generated output
> is not 100% under our control the way the QAPI-to-Texinfo generator
> was. We can't observe the data we generate, only the Sphinx
> output. Two issues.
>
> One, the output can vary with the Sphinx version. In practice
> Sphinx's plaintext output generation has been identical between at
> least Sphinx 1.6 and 3.0, so we use that. (The HTML output has had
> changes across versions). We use an exact-match comparison check, with
> the understanding that perhaps changes in a future Sphinx version
> might require us to implement something more clever to cope with
> variation in the output.
>
> Two, the test can only protect us from changes in the data we generate
> that are visible in plain text.
>
> What do you think?
Yes, seems worth recording that in the commit message (especially
now you've written the text :-)).
> > +# Test the document-comment document generation code by running a test schema
> > +# file through Sphinx's plain-text builder and comparing the result against
> > +# a golden reference. This is in theory susceptible to failures if Sphinx
> > +# changes its output, but the text output has historically been very stable
> > +# (no changes between Sphinx 1.6 and 3.0), so it is a better bet than
> > +# texinfo or HTML generation, both of which have had changes. We might
>
> Texinfo
>
> > +# need to add more sophisticated logic here in future for some sort of
> > +# fuzzy comparison if future Sphinx versions produce different text,
> > +# but for now the simple comparison suffices.
> > +qapi_doc_out = custom_target('QAPI rST doc',
> > + output: ['doc-good.txt'],
> > + input: files('doc-good.json', 'doc-good.rst'),
>
> Gawk at my Meson ignorance...
>
> Looks like this builds doc-good.txt from doc.good.json and doc-good.rst.
>
> doc-good.txt is also a source file. Works, because we use a separate
> build tree. Might be confusing, though.
Yes. We could change the name of the reference source file that
we have checked into the git repo if you wanted. (The output file
written by Sphinx has to be the same name as the input .rst file AFAICT.)
> > + build_by_default: build_docs,
> > + depend_files: sphinx_extn_depends,
> > + # We use -E to suppress Sphinx's caching, because
> > + # we want it to always really run the QAPI doc
> > + # generation code. It also means we don't
> > + # clutter up the build dir with the cache.
> > + command: [SPHINX_ARGS,
> > + '-b', 'text', '-E',
> > + '-c', meson.source_root() / 'docs',
> > + '-D', 'master_doc=doc-good',
> > + meson.current_source_dir(),
> > + meson.current_build_dir()])
> > +
> > +# Fix possible inconsistency in line endings in generated output and
> > +# in the golden reference (which could otherwise cause test failures
> > +# on Windows hosts). Unfortunately diff --strip-trailing-cr
> > +# is GNU-diff only. The odd-looking perl is because we must avoid
> > +# using an explicit '\' character in the command arguments to
> > +# a custom_target(), as Meson will unhelpfully replace it with a '/'
> > +# (https://github.com/mesonbuild/meson/issues/1564)
>
> Rather disappointing.
>
> > +qapi_doc_out_nocr = custom_target('QAPI rST doc newline-sanitized',
> > + output: ['doc-good.txt.nocr'],
> > + input: qapi_doc_out[0],
> > + build_by_default: build_docs,
> > + command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
> > + capture: true)
>
> I figure this strips \r from the build tree's doc-good.txt.
Close; it turns either CRLF or LF into the host OS's
line-ending sequence (see below).
> > +qapi_doc_ref_nocr = custom_target('QAPI rST doc reference newline-sanitized',
> > + output: ['doc-good.ref.nocr'],
> > + input: files('doc-good.txt'),
> > + build_by_default: build_docs,
> > + command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
> > + capture: true)
>
> Uh, this strips it from the source tree's doc-good.txt, right? Why is
> that necessary?
This is in case the user has a setup that eg has git
doing line-ending conversion on checkout somehow. As a
non-Windows user I opted to be belt-and-braces about
converting both files to a known-consistent line ending.
It's also necessary because the perl rune isn't really
"delete \r"; it's "delete any \r and then output the
line with the OS line ending" because the files it processes
are being read and written in text mode. So the output
will be \r\n on Windows and \n on Unix; the test passes
in both cases because both files have the same
line endings after conversion.
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions
2020-09-25 16:23 ` [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions Peter Maydell
@ 2020-09-29 12:35 ` Markus Armbruster
2020-09-29 12:43 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 12:35 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel, Markus Armbruster
Peter Maydell <peter.maydell@linaro.org> writes:
> Update the documentation of QAPI document comment syntax to match
> the new rST backend requirements. The principal changes are:
> * whitespace is now significant,
Well, differently significant :) Anyway, close enough.
> and multiline definitions
> must have their second and subsequent lines indented to
> match the first line
> * general rST format markup is permitted, not just the small
> set of markup the old texinfo generator handled. For most
> things (notably bulleted and itemized lists) the old format
> is the same as rST was.
"was the same as rST is"?
> * Specific things that might trip people up:
> - instead of *bold* and _italic_ rST has **bold** and *italic*
> - lists need a preceding and following blank line
> - a lone literal '*' will need to be backslash-escaped to
> avoid a rST syntax error
> * the old leading '|' for example (literal text) blocks is
> replaced by the standard rST '::' literal block.
> * we support arbitrary levels of sub- and sub-sub-heading, not
> just a main and sub-heading like the old texinfo generator
Possibly noteworthy: lists can now be nested. Can add that in my tree.
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> docs/devel/qapi-code-gen.txt | 83 +++++++++++++++++++++++-------------
> 1 file changed, 53 insertions(+), 30 deletions(-)
>
> diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
> index 69eaffac376..7d2479dde82 100644
> --- a/docs/devel/qapi-code-gen.txt
> +++ b/docs/devel/qapi-code-gen.txt
> @@ -824,23 +824,37 @@ See below for more on definition documentation.
> Free-form documentation may be used to provide additional text and
> structuring content.
>
> +==== Headings and subheadings ====
> +
> +A free-form documentation comment containing a line which starts with
> +some '=' symbols and then a space defines a section heading:
> +
> + ##
> + # = This is a top level heading
> + #
> + # This is a free-form comment which will go under the
> + # top level heading.
> + ##
> +
> + ##
> + # == This is a second level heading
> + ##
> +
> +A heading line must be the first line of the documentation
> +comment block.
> +
> +Section headings must always be correctly nested, so you can only
> +define a third-level heading inside a second-level heading, and so on.
>
> ==== Documentation markup ====
>
> -Comment text starting with '=' is a section title:
> +Documentation comments can use most rST markup. In particular,
> +a '::' literal block can be used for examples:
>
> - # = Section title
> -
> -Double the '=' for a subsection title:
> -
> - # == Subsection title
> -
> -Both are only permitted in free-form documentation.
> -
> -'|' denotes examples:
> -
> - # | Text of the example, may span
> - # | multiple lines
> + # ::
> + #
> + # Text of the example, may span
> + # multiple lines
>
> '*' starts an itemized list:
>
> @@ -856,34 +870,33 @@ A decimal number followed by '.' starts a numbered list:
> # multiple lines
> # 2. Second item
>
> -The actual number doesn't matter. You could even use '*' instead of
> -'2.' for the second item.
> +The actual number doesn't matter.
>
> -Lists can't be nested. Blank lines are currently not supported within
> -lists.
> +Lists of either kind must be preceded and followed by a blank line.
> +If a list item's text spans multiple lines, then the second and
> +subsequent lines must be correctly indented to line up with the
> +first character of the first line.
>
> -Additional whitespace between the initial '#' and the comment text is
> -permitted.
> -
> -*foo* and _foo_ are for strong and emphasis styles respectively (they
> -do not work over multiple lines). @foo is used to reference a name in
> -the schema.
> +The usual '**strong**', '*emphasised*' and '``literal``' markup should
> +be used. If you need a single literal '*' you will need to
> +backslash-escape it. As an extension beyond the usual rST syntax, you
> +can also use '@foo' to reference a name in the schema; this is
> +rendered the same way as '``foo``'.
>
> Example:
>
> ##
> -# = Section
> -# == Subsection
> -#
> -# Some text foo with *strong* and _emphasis_
> +# Some text foo with **bold** and *emphasis*
> # 1. with a list
> # 2. like that
> #
> # And some code:
> -# | $ echo foo
> -# | -> do this
> -# | <- get that
> #
> +# ::
> +#
> +# $ echo foo
> +# -> do this
> +# <- get that
> ##
>
>
v5 had
@@ -899,6 +915,12 @@ commands and events), member (for structs and unions), branch (for
alternates), or value (for enums), and finally optional tagged
sections.
+Descriptions of arguments can span multiple lines; if they
+do then the second and subsequent lines must be indented
+to line up with the first character of the first line of the
+description. The parser will report a syntax error if there
+is insufficient indentation.
+
FIXME: the parser accepts these things in almost any order.
FIXME: union branches should be described, too.
I questioned the value of the last sentence. You dropped both.
Intentional?
> @@ -937,6 +950,16 @@ multiline argument descriptions.
> A 'Since: x.y.z' tagged section lists the release that introduced the
> definition.
>
> +The text of a section can start on a new line, in
> +which case it must not be indented at all. It can also start
> +on the same line as the 'Note:', 'Returns:', etc tag. In this
> +case if it spans multiple lines then second and subsequent
> +lines must be indented to match the first.
> +
> +An 'Example' or 'Examples' section is automatically rendered
> +entirely as literal fixed-width text. In other sections,
> +the text is formatted, and rST markup can be used.
> +
> For example:
>
> ##
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 18/21] scripts/texi2pod: Delete unused script
2020-09-25 16:23 ` [PATCH v6 18/21] scripts/texi2pod: Delete unused script Peter Maydell
@ 2020-09-29 12:36 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 12:36 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> We no longer need the texi2pod script, so we can delete it, and
> the special-casing it had in the checkpatch script.
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> scripts/checkpatch.pl | 2 +-
> scripts/texi2pod.pl | 536 ------------------------------------------
> 2 files changed, 1 insertion(+), 537 deletions(-)
> delete mode 100755 scripts/texi2pod.pl
Extra point for remembering checkpatch.pl :)
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 19/21] Remove Texinfo related line from git.orderfile
2020-09-25 16:23 ` [PATCH v6 19/21] Remove Texinfo related line from git.orderfile Peter Maydell
@ 2020-09-29 12:37 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 12:37 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel, Markus Armbruster
Peter Maydell <peter.maydell@linaro.org> writes:
> We don't use Texinfo any more; we can remove the references to the
> .texi source file from our git.orderfile.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 20/21] configure: Drop texinfo requirement
2020-09-25 16:23 ` [PATCH v6 20/21] configure: Drop texinfo requirement Peter Maydell
@ 2020-09-29 12:38 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 12:38 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel, Markus Armbruster
Peter Maydell <peter.maydell@linaro.org> writes:
> We don't need the texinfo and pod2man programs to build our documentation
> any more, so remove them from configure's tests.
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 21/21] Remove texinfo dependency from docker and CI configs
2020-09-25 16:23 ` [PATCH v6 21/21] Remove texinfo dependency from docker and CI configs Peter Maydell
@ 2020-09-29 12:39 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 12:39 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> We don't need texinfo to build the docs any more, so we can
> drop that dependency from our docker and other CI configs.
>
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions
2020-09-29 12:35 ` Markus Armbruster
@ 2020-09-29 12:43 ` Peter Maydell
2020-09-29 13:27 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 12:43 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Tue, 29 Sep 2020 at 13:35, Markus Armbruster <armbru@redhat.com> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > Update the documentation of QAPI document comment syntax to match
> > the new rST backend requirements. The principal changes are:
> > * whitespace is now significant,
>
> Well, differently significant :) Anyway, close enough.
>
> > and multiline definitions
> > must have their second and subsequent lines indented to
> > match the first line
> > * general rST format markup is permitted, not just the small
> > set of markup the old texinfo generator handled. For most
> > things (notably bulleted and itemized lists) the old format
> > is the same as rST was.
>
> "was the same as rST is"?
Yes :-)
> v5 had
>
> @@ -899,6 +915,12 @@ commands and events), member (for structs and unions), branch (for
> alternates), or value (for enums), and finally optional tagged
> sections.
>
> +Descriptions of arguments can span multiple lines; if they
> +do then the second and subsequent lines must be indented
> +to line up with the first character of the first line of the
> +description. The parser will report a syntax error if there
> +is insufficient indentation.
> +
> FIXME: the parser accepts these things in almost any order.
> FIXME: union branches should be described, too.
>
> I questioned the value of the last sentence. You dropped both.
> Intentional?
I moved the first sentence to patch 5 in v6 (ie to the patch
which updates parser.py to enforce those indentation restrictions),
so as to make patches 1..5 suitable for merging even if we needed
to respin the second half of the series.
> > @@ -937,6 +950,16 @@ multiline argument descriptions.
> > A 'Since: x.y.z' tagged section lists the release that introduced the
> > definition.
> >
> > +The text of a section can start on a new line, in
> > +which case it must not be indented at all. It can also start
> > +on the same line as the 'Note:', 'Returns:', etc tag. In this
> > +case if it spans multiple lines then second and subsequent
> > +lines must be indented to match the first.
I also moved this paragraph into patch 5 (where it appears just
above the "A 'Since:..." text you can see in the context here)
but forgot to delete the copy of it here, so at this point it is
duplicate text and should not be in this patch. Oops.
> > +
> > +An 'Example' or 'Examples' section is automatically rendered
> > +entirely as literal fixed-width text. In other sections,
> > +the text is formatted, and rST markup can be used.
(This patch is the right place for this paragraph.)
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling
2020-09-29 8:55 ` Peter Maydell
@ 2020-09-29 13:03 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 13:03 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Mon, 28 Sep 2020 at 20:16, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>> > +Descriptions of arguments can span multiple lines. The description
>> > +text can start on the line following the '@argname:', in which case
>> > +it must not be indented at all. It can also start on the same line
>> > +as the '@argname:'. In this case if it spans multiple lines then
>> > +second and subsequent lines must be indented to line up with the
>> > +first character of the first line of the description:
>>
>> Please put two spaces after sentence-ending punctuation, for local
>> consistency, and to keep Emacs sentence commands working.
>
> Is there a python lint program that can auto-check this?
> Otherwise I am going to continue to put single-spaces at
> least some of the time, because that's the way I write all
> the other English text I write...
John, any idea? The ones I use apparently don't, even though PEP 8 asks
for this style.
>> Can touch this up in my tree, of course.
>
> That would certainly be easier for me :-)
Happy to help :)
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST
2020-09-29 9:26 ` Peter Maydell
@ 2020-09-29 13:11 ` Markus Armbruster
2020-09-29 14:25 ` Markus Armbruster
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 13:11 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Tue, 29 Sep 2020 at 09:20, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>> > Convert qemu-ga-ref to rST format. This includes dropping
>> > the plain-text, pdf and info format outputs for this document;
>> > as with all our other Sphinx-based documentation, we provide
>> > HTML and manpage only.
>> >
>
>> > --- a/docs/interop/conf.py
>> > +++ b/docs/interop/conf.py
>> > @@ -19,4 +19,6 @@ html_theme_options['description'] = u'System Emulation Management and Interopera
>> > man_pages = [
>> > ('qemu-ga', 'qemu-ga', u'QEMU Guest Agent',
>> > ['Michael Roth <mdroth@linux.vnet.ibm.com>'], 8),
>> > + ('qemu-ga-ref', 'qemu-ga-ref', u'QEMU Guest Agent Protocol Reference',
>> > + [], 7),
>> > ]
>>
>> Why do you make the description a unicode legacy literal? I see it
>> matches existing entries. I'd like to know regardless :)
>
> I was probably just copying some other example of how to
> write the man_pages[] definition. This also all used to have
> to work with Python 2.7, which might or might not be relevant here.
Let's switch to plain string. Can do in my tree.
>> > -@titlepage
>> > -@title Guest Agent Protocol Reference Manual
>> > -@subtitle QEMU version @value{VERSION}
>>
>> There is no obvious equivalent to @value{VERSION} in
>> docs/interop/qemu-ga-ref.rst.
>>
>> The manual page generated from it has the version in the footer. Good.
>>
>> I can't find it in the generated HTML. Not so good, but it wasn't there
>> before the patch, either.
>>
>> The generated PDF had it on the title page.
>>
>> Suggest to add a TODO comment like the one about the licensing
>> information.
>
> So the version is in the manual page, as it was before the conversion,
> and it's not in the HTML version, which it wasn't before the
> conversion. That doesn't sound to me like there's anything here
> to do...
I think readers of the HTML version will appreciate the version
information.
Similar situation as for the licensing information: your patch doesn't
make things worse[*], but we found something to improve during review.
> You can add a TODO if you want one, of course.
Thanks!
[*] I guess it would for PDF, if we still supported PDF.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref to rST
2020-09-29 9:41 ` Peter Maydell
@ 2020-09-29 13:12 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 13:12 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Tue, 29 Sep 2020 at 09:28, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>> > diff --git a/qapi/meson.build b/qapi/meson.build
>> > index 2b2872a41d8..a287ca5d9d7 100644
>> > --- a/qapi/meson.build
>> > +++ b/qapi/meson.build
>> > @@ -97,7 +97,7 @@ foreach module : qapi_all_modules
>> > endforeach
>> >
>> > qapi_files = custom_target('shared QAPI source files',
>> > - output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs + ['qapi-doc.texi'],
>> > + output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs,
>> > input: [ files('qapi-schema.json') ],
>> > command: [ qapi_gen, '-o', 'qapi', '-b', '@INPUT0@' ],
>> > depend_files: [ qapi_inputs, qapi_gen_depends ])
>> > @@ -121,5 +121,3 @@ foreach output : qapi_specific_outputs + qapi_nonmodule_outputs
>> > specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: qapi_files[i])
>> > i = i + 1
>> > endforeach
>> > -
>> > -qapi_doc_texi = qapi_files[i]
>>
>> Doesn't storage-daemon/qapi/meson.build need a similar update?
>
> I was previously unaware of storage-daemon/qapi...
> It looks like we don't actually do anything with the generated
> qapi-doc.texi there, so I'm not sure why we were listing it as an output.
> I think we should only need to remove the " + ['qapi-doc.texi']"
> in storage-daemon/qapi/meson.build, and that should be a separate
> change after this one and before we remove scripts/qapi/doc.py.
I'll give it a try.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref to rST
2020-09-29 9:46 ` Peter Maydell
@ 2020-09-29 13:13 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 13:13 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, Markus Armbruster, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Tue, 29 Sep 2020 at 09:42, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Appears to break documented make target html:
>>
>> $ make -C bld-x86 html
>> make: Entering directory '/work/armbru/qemu/bld-x86'
>> make: *** No rule to make target 'html'. Stop.
>> make: Leaving directory '/work/armbru/qemu/bld-x86'
>
> Whoops. Should be fixable by adding
> alias_target('html', sphinxdocs)
>
> under the other two alias_target() calls at the bottom of
> docs/meson.build, I think.
>
> Looking at the code I think it also breaks the 'info',
> 'pdf' and 'txt' targets, which I propose that we fix
> by removing them from the documentation, since not providing
> info, pdf or txt output is an intentional change.
Yes.
> I believe that
> the only documentation we would need to update is the
> $(call print-help,html info pdf txt man,Build documentation in
> specified format)
> line in Makefile.
I'll give it a try.
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn
2020-09-29 12:33 ` Peter Maydell
@ 2020-09-29 13:18 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 13:18 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Tue, 29 Sep 2020 at 13:20, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> In subject, s/outputn/output/
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>> > Add a test of the rST output from the QAPI doc-comment generator,
>> > similar to what we currently have that tests the Texinfo output.
>> >
>> > This is a bit more awkward with Sphinx, because the generated
>> > output is not 100% under our control the way the QAPI-to-Texinfo
>> > generator was. However, in practice Sphinx's plaintext output
>> > generation has been identical between at least Sphinx 1.6 and
>> > 3.0, so we use that. (The HTML output has had changes across
>> > versions). We use an exact-match comparison check, with the
>> > understanding that perhaps changes in a future Sphinx version
>> > might require us to implement something more clever to cope
>> > with variation in the output.
>> >
>> > Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>>
>> It's not just the potential Sphinx version dependence that makes this
>> awkward.
>>
>> We can no longer check what our doc generator does (at least not without
>> substantial additional coding), we can only check what it does together
>> with Sphinx. We do so for one output format.
>>
>> Our doc generator output could change in ways that are not visible in
>> the Sphinx output format we test, but are visible in some other output
>> format.
>>
>> We choose to test plain text, because it has the lowest risk of unwanted
>> Sphinx version dependence, even though it probably has the highest risk
>> of "rendering stuff invisible".
>>
>> Certainly better than nothing, and probably the best we can do now, but
>> let's capture the tradeoff in the commit message. Perhaps:
>>
>> This is a bit more awkward with Sphinx, because the generated output
>> is not 100% under our control the way the QAPI-to-Texinfo generator
>> was. We can't observe the data we generate, only the Sphinx
>> output. Two issues.
>>
>> One, the output can vary with the Sphinx version. In practice
>> Sphinx's plaintext output generation has been identical between at
>> least Sphinx 1.6 and 3.0, so we use that. (The HTML output has had
>> changes across versions). We use an exact-match comparison check, with
>> the understanding that perhaps changes in a future Sphinx version
>> might require us to implement something more clever to cope with
>> variation in the output.
>>
>> Two, the test can only protect us from changes in the data we generate
>> that are visible in plain text.
>>
>> What do you think?
>
> Yes, seems worth recording that in the commit message (especially
> now you've written the text :-)).
:)
>> > +# Test the document-comment document generation code by running a test schema
>> > +# file through Sphinx's plain-text builder and comparing the result against
>> > +# a golden reference. This is in theory susceptible to failures if Sphinx
>> > +# changes its output, but the text output has historically been very stable
>> > +# (no changes between Sphinx 1.6 and 3.0), so it is a better bet than
>> > +# texinfo or HTML generation, both of which have had changes. We might
>>
>> Texinfo
>>
>> > +# need to add more sophisticated logic here in future for some sort of
>> > +# fuzzy comparison if future Sphinx versions produce different text,
>> > +# but for now the simple comparison suffices.
>> > +qapi_doc_out = custom_target('QAPI rST doc',
>> > + output: ['doc-good.txt'],
>> > + input: files('doc-good.json', 'doc-good.rst'),
>>
>> Gawk at my Meson ignorance...
>>
>> Looks like this builds doc-good.txt from doc.good.json and doc-good.rst.
>>
>> doc-good.txt is also a source file. Works, because we use a separate
>> build tree. Might be confusing, though.
>
> Yes. We could change the name of the reference source file that
> we have checked into the git repo if you wanted. (The output file
> written by Sphinx has to be the same name as the input .rst file AFAICT.)
I'll see what I can do (and thanks for the hint).
>> > + build_by_default: build_docs,
>> > + depend_files: sphinx_extn_depends,
>> > + # We use -E to suppress Sphinx's caching, because
>> > + # we want it to always really run the QAPI doc
>> > + # generation code. It also means we don't
>> > + # clutter up the build dir with the cache.
>> > + command: [SPHINX_ARGS,
>> > + '-b', 'text', '-E',
>> > + '-c', meson.source_root() / 'docs',
>> > + '-D', 'master_doc=doc-good',
>> > + meson.current_source_dir(),
>> > + meson.current_build_dir()])
>> > +
>> > +# Fix possible inconsistency in line endings in generated output and
>> > +# in the golden reference (which could otherwise cause test failures
>> > +# on Windows hosts). Unfortunately diff --strip-trailing-cr
>> > +# is GNU-diff only. The odd-looking perl is because we must avoid
>> > +# using an explicit '\' character in the command arguments to
>> > +# a custom_target(), as Meson will unhelpfully replace it with a '/'
>> > +# (https://github.com/mesonbuild/meson/issues/1564)
>>
>> Rather disappointing.
>>
>> > +qapi_doc_out_nocr = custom_target('QAPI rST doc newline-sanitized',
>> > + output: ['doc-good.txt.nocr'],
>> > + input: qapi_doc_out[0],
>> > + build_by_default: build_docs,
>> > + command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
>> > + capture: true)
>>
>> I figure this strips \r from the build tree's doc-good.txt.
>
> Close; it turns either CRLF or LF into the host OS's
> line-ending sequence (see below).
>
>> > +qapi_doc_ref_nocr = custom_target('QAPI rST doc reference newline-sanitized',
>> > + output: ['doc-good.ref.nocr'],
>> > + input: files('doc-good.txt'),
>> > + build_by_default: build_docs,
>> > + command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
>> > + capture: true)
>>
>> Uh, this strips it from the source tree's doc-good.txt, right? Why is
>> that necessary?
>
> This is in case the user has a setup that eg has git
> doing line-ending conversion on checkout somehow. As a
> non-Windows user I opted to be belt-and-braces about
> converting both files to a known-consistent line ending.
> It's also necessary because the perl rune isn't really
> "delete \r"; it's "delete any \r and then output the
> line with the OS line ending" because the files it processes
> are being read and written in text mode. So the output
> will be \r\n on Windows and \n on Unix; the test passes
> in both cases because both files have the same
> line endings after conversion.
Uff. Thanks!
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions
2020-09-29 12:43 ` Peter Maydell
@ 2020-09-29 13:27 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 13:27 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, Markus Armbruster, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Tue, 29 Sep 2020 at 13:35, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>> > Update the documentation of QAPI document comment syntax to match
>> > the new rST backend requirements. The principal changes are:
>> > * whitespace is now significant,
>>
>> Well, differently significant :) Anyway, close enough.
>>
>> > and multiline definitions
>> > must have their second and subsequent lines indented to
>> > match the first line
>> > * general rST format markup is permitted, not just the small
>> > set of markup the old texinfo generator handled. For most
>> > things (notably bulleted and itemized lists) the old format
>> > is the same as rST was.
>>
>> "was the same as rST is"?
>
> Yes :-)
Can fix in my tree.
>
>> v5 had
>>
>> @@ -899,6 +915,12 @@ commands and events), member (for structs and unions), branch (for
>> alternates), or value (for enums), and finally optional tagged
>> sections.
>>
>> +Descriptions of arguments can span multiple lines; if they
>> +do then the second and subsequent lines must be indented
>> +to line up with the first character of the first line of the
>> +description. The parser will report a syntax error if there
>> +is insufficient indentation.
>> +
>> FIXME: the parser accepts these things in almost any order.
>> FIXME: union branches should be described, too.
>>
>> I questioned the value of the last sentence. You dropped both.
>> Intentional?
>
> I moved the first sentence to patch 5 in v6 (ie to the patch
> which updates parser.py to enforce those indentation restrictions),
> so as to make patches 1..5 suitable for merging even if we needed
> to respin the second half of the series.
I see.
>> > @@ -937,6 +950,16 @@ multiline argument descriptions.
>> > A 'Since: x.y.z' tagged section lists the release that introduced the
>> > definition.
>> >
>> > +The text of a section can start on a new line, in
>> > +which case it must not be indented at all. It can also start
>> > +on the same line as the 'Note:', 'Returns:', etc tag. In this
>> > +case if it spans multiple lines then second and subsequent
>> > +lines must be indented to match the first.
>
> I also moved this paragraph into patch 5 (where it appears just
> above the "A 'Since:..." text you can see in the context here)
> but forgot to delete the copy of it here, so at this point it is
> duplicate text and should not be in this patch. Oops.
>
>> > +
>> > +An 'Example' or 'Examples' section is automatically rendered
>> > +entirely as literal fixed-width text. In other sections,
>> > +the text is formatted, and rST markup can be used.
>
> (This patch is the right place for this paragraph.)
Thanks!
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (23 preceding siblings ...)
2020-09-25 19:25 ` no-reply
@ 2020-09-29 13:31 ` Markus Armbruster
2020-09-29 20:17 ` Markus Armbruster
25 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 13:31 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> This series switches all our QAPI doc comments over from texinfo
> format to rST. It then removes all the texinfo machinery, because
> this was the last user of texinfo.
>
> I think I have now resolved all of Markus' issues raised in his
> review of v5, and IMHO this is ready to commit. If there are still
> things needing fixing, it would be nice if we were able to commit
> patches 1-5, which are the ones which add the new indent-sensitive
> code to the QAPI parser. That would put a stop to the steady trickle
> of doc-comment changes which break the new rules...
I found several small things to improve. I'll now try to address them
in my tree. If I fail, I'll take PATCH 01-05 now, and ask for a respin
of the rest.
[...]
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST
2020-09-29 13:11 ` Markus Armbruster
@ 2020-09-29 14:25 ` Markus Armbruster
0 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 14:25 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers
Neglected to say
With the unicode legacy string literal made plain:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-28 13:05 ` Peter Maydell
@ 2020-09-29 15:26 ` Markus Armbruster
2020-09-29 15:43 ` Peter Maydell
0 siblings, 1 reply; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 15:26 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, QEMU Developers
Peter Maydell <peter.maydell@linaro.org> writes:
> On Mon, 28 Sep 2020 at 14:04, Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>
>> > On Fri, 25 Sep 2020 at 20:25, <no-reply@patchew.org> wrote:
>> >
>> >> In file included from ../src/qapi/qapi-schema.json:78:
>> >> ../src/qapi/migration.json:1747:1: unexpected de-indent (expected at least 13 spaces)
>> >
>> > This is yet another mis-indented line in a change to the QAPI
>> > doc-comments (commit 4c437254b807). It hit master in the
>> > latest migration pull after I'd sent out this patchseries
>> > but before patchew got round to testing..
>>
>> Obvious fixup for your PATCH 01:
>>
>> diff --git a/qapi/migration.json b/qapi/migration.json
>> index 7d9342c064..7f5e6fd681 100644
>> --- a/qapi/migration.json
>> +++ b/qapi/migration.json
>> @@ -1744,9 +1744,9 @@
>> # Information about current dirty page rate of vm.
>> #
>> # @dirty-rate: @dirtyrate describing the dirty page rate of vm
>> -# in units of MB/s.
>> -# If this field returns '-1', it means querying has not
>> -# yet started or completed.
>> +# in units of MB/s.
>> +# If this field returns '-1', it means querying has not
>> +# yet started or completed.
>> #
>> # @status: status containing dirtyrate query status includes
>> # 'unstarted' or 'measuring' or 'measured'
>>
>> Happy to fix it up in my tree.
>
> Yes, please.
One more issue:
/work/armbru/qemu/docs/../qapi/machine.json:1000: WARNING: Unexpected indentation.
/work/armbru/qemu/docs/../qapi/machine.json:1000: WARNING: Block quote ends without a blank line; unexpected unindent.
Line 1000 is at the beginning of a comment block. Suboptimal.
After a bit of guessing, I arrived at this fix:
diff --git a/qapi/machine.json b/qapi/machine.json
index 7c9e69a9f5..756dacb06f 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1001,9 +1001,11 @@
#
# Request the balloon driver to change its balloon size.
#
-# @value: the target logical size of the VM in bytes
+# @value: the target logical size of the VM in bytes.
# We can deduce the size of the balloon using this formula:
+#
# logical_vm_size = vm_ram_size - balloon_size
+#
# From it we have: balloon_size = vm_ram_size - @value
#
# Returns: - Nothing on success
Looks good?
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-29 15:26 ` Markus Armbruster
@ 2020-09-29 15:43 ` Peter Maydell
0 siblings, 0 replies; 69+ messages in thread
From: Peter Maydell @ 2020-09-29 15:43 UTC (permalink / raw)
To: Markus Armbruster; +Cc: John Snow, QEMU Developers
On Tue, 29 Sep 2020 at 16:26, Markus Armbruster <armbru@redhat.com> wrote:
> One more issue:
>
> /work/armbru/qemu/docs/../qapi/machine.json:1000: WARNING: Unexpected indentation.
> /work/armbru/qemu/docs/../qapi/machine.json:1000: WARNING: Block quote ends without a blank line; unexpected unindent.
>
> Line 1000 is at the beginning of a comment block. Suboptimal.
Yes. There's a comment in qapidoc.py about that:
+ # The reported line number will always be that of the start line
+ # of the doc comment, rather than the actual location of the error.
+ # Being more precise would require overhaul of the QAPIDoc class
+ # to track lines more exactly within all the sub-parts of the doc
+ # comment, as well as counting lines here.
(That is, within qapidoc.py we can count the offset of each line within
this doc comment fragment from the start of the fragment (here, from
the "@value:" line), but only the scripts/qapi code is in a position to
know the offset from the start of the block comment on line 1000 to
the @value line, and it doesn't currently track that.)
> After a bit of guessing, I arrived at this fix:
>
> diff --git a/qapi/machine.json b/qapi/machine.json
> index 7c9e69a9f5..756dacb06f 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -1001,9 +1001,11 @@
> #
> # Request the balloon driver to change its balloon size.
> #
> -# @value: the target logical size of the VM in bytes
> +# @value: the target logical size of the VM in bytes.
> # We can deduce the size of the balloon using this formula:
> +#
> # logical_vm_size = vm_ram_size - balloon_size
> +#
> # From it we have: balloon_size = vm_ram_size - @value
> #
> # Returns: - Nothing on success
>
> Looks good?
That will work: it will render the indented text as a block quote.
If you want the formula to be rendered as a fixed-width code
example, you could additionally do s/formula:/formula::/ to
create a rST literal block. But then you'd probably want to
do similar with the equation on the following line, for consistency.
thanks
-- PMM
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
` (24 preceding siblings ...)
2020-09-29 13:31 ` Markus Armbruster
@ 2020-09-29 20:17 ` Markus Armbruster
25 siblings, 0 replies; 69+ messages in thread
From: Markus Armbruster @ 2020-09-29 20:17 UTC (permalink / raw)
To: Peter Maydell; +Cc: John Snow, qemu-devel
Peter Maydell <peter.maydell@linaro.org> writes:
> This series switches all our QAPI doc comments over from texinfo
> format to rST. It then removes all the texinfo machinery, because
> this was the last user of texinfo.
>
> I think I have now resolved all of Markus' issues raised in his
> review of v5, and IMHO this is ready to commit. If there are still
> things needing fixing, it would be nice if we were able to commit
> patches 1-5, which are the ones which add the new indent-sensitive
> code to the QAPI parser. That would put a stop to the steady trickle
> of doc-comment changes which break the new rules...
>
> Also available as a git branch at
> https://git.linaro.org/people/peter.maydell/qemu-arm.git sphinx-conversions-v6
>
> The basic approach is somewhat similar to how we deal with kerneldoc
> and hxtool: we have a custom Sphinx extension which is passed a
> filename which is the json file it should run the QAPI parser over and
> generate documentation for. Unlike 'kerneldoc' but somewhat like
> hxtool, I have chosen to generate documentation by generating a tree
> of docutils nodes, rather than by generating rST source that is then
> fed to the rST parser to generate docutils nodes. Individual lumps of
> doc comment go to the rST parser, but the structured parts we render
> directly. This makes it easier to get the structure and heading level
> nesting correct.
>
> Changes from v5:
> * rebased (in particular, updated to meson build system)
> * new patch 1 fixes indent issues that hit master since v5
> * new patch 2 makes block-latency-histogram-set's use of Example
> sections match everybody else's, instead of special casing it
> in the parser
> * the .gitignore got pruned after meson conversion so we only
> need to change git.orderfile now
> * slightly reordered patches to bring the parser.py indent change nearer
> the start of the series in the hopes of being able to get at least
> that much of the series into master
> * we now tell Sphinx about all the json files for dependency info,
> so editing a json file correctly rebuilds the docs
> * added a test case for the bad-de-indent parser error
> * Adopted the various Python scripting suggestions from Markus
> * We don't insist on section headings being the only thing in their
> doc comment block any more (the existing "must be first line"
> requirement is sufficient)
> * added a test case for doc-generation that does a compare of
> the sphinx plain-text builder output against a reference file
> * Added the Python source files for Sphinx extensions (including
> the QAPI source files) to the dependency lists for the manuals,
> so that changes made to them correctly trigger a docs rebuild
> * qemu-ga-ref.rst and qemu-qmp-ref.rst now have a TODO note about
> making the manual licensing more visible to readers
> * fixed bug in reported file/line info for some errors in rST
> in doc comments when using Sphinx 1.6
> * don't insist on section headers being in their own freeform doc
> comment block; they're (after commit dcdc07a97cbe) always the
> first line in the comment block, so just handle the possibility
> of having text after that.
>
> There are a few things I have left out of this initial series:
>
> * unlike the texinfo, there is no generation of index entries
> or an index in the HTML docs
> * although there are HTML anchors on all the command/object/etc
> headings, they are not stable but just serial-number based
> tags like '#qapidoc-35', so not suitable for trying to link
> to from other parts of the docs
>
> My view is that we can add niceties like this later; the series
> already seems big enough to me.
Queued, thanks!
^ permalink raw reply [flat|nested] 69+ messages in thread
end of thread, other threads:[~2020-09-29 20:18 UTC | newest]
Thread overview: 69+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
2020-09-25 16:22 ` [PATCH v6 01/21] qapi: Fix doc comment indentation again Peter Maydell
2020-09-28 12:39 ` Markus Armbruster
2020-09-25 16:22 ` [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set Peter Maydell
2020-09-28 12:42 ` Markus Armbruster
2020-09-28 12:49 ` Peter Maydell
2020-09-28 18:04 ` Markus Armbruster
2020-09-25 16:22 ` [PATCH v6 03/21] tests/qapi/doc-good.json: Prepare for qapi-doc Sphinx extension Peter Maydell
2020-09-25 16:22 ` [PATCH v6 04/21] scripts/qapi: Move doc-comment whitespace stripping to doc.py Peter Maydell
2020-09-25 16:23 ` [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling Peter Maydell
2020-09-28 19:15 ` Markus Armbruster
2020-09-29 8:55 ` Peter Maydell
2020-09-29 13:03 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 06/21] qapi/machine.json: Escape a literal '*' in doc comment Peter Maydell
2020-09-29 4:57 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension Peter Maydell
2020-09-29 6:54 ` Markus Armbruster
2020-09-29 9:05 ` Peter Maydell
2020-09-25 16:23 ` [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST Peter Maydell
2020-09-29 8:20 ` Markus Armbruster
2020-09-29 9:26 ` Peter Maydell
2020-09-29 13:11 ` Markus Armbruster
2020-09-29 14:25 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref " Peter Maydell
2020-09-29 8:27 ` Markus Armbruster
2020-09-29 9:41 ` Peter Maydell
2020-09-29 13:12 ` Markus Armbruster
2020-09-29 8:42 ` Markus Armbruster
2020-09-29 9:46 ` Peter Maydell
2020-09-29 13:13 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 10/21] qapi: Use rST markup for literal blocks Peter Maydell
2020-09-25 16:23 ` [PATCH v6 11/21] qga/qapi-schema.json: Add some headings Peter Maydell
2020-09-29 8:30 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 12/21] tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis Peter Maydell
2020-09-29 8:33 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 13/21] meson.build: Move SPHINX_ARGS to top level meson.build file Peter Maydell
2020-09-29 8:45 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 14/21] meson.build: Make manuals depend on source to Sphinx extensions Peter Maydell
2020-09-29 8:52 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn Peter Maydell
2020-09-29 12:20 ` Markus Armbruster
2020-09-29 12:33 ` Peter Maydell
2020-09-29 13:18 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 16/21] scripts/qapi: Remove texinfo generation support Peter Maydell
2020-09-29 12:22 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions Peter Maydell
2020-09-29 12:35 ` Markus Armbruster
2020-09-29 12:43 ` Peter Maydell
2020-09-29 13:27 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 18/21] scripts/texi2pod: Delete unused script Peter Maydell
2020-09-29 12:36 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 19/21] Remove Texinfo related line from git.orderfile Peter Maydell
2020-09-29 12:37 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 20/21] configure: Drop texinfo requirement Peter Maydell
2020-09-29 12:38 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 21/21] Remove texinfo dependency from docker and CI configs Peter Maydell
2020-09-29 12:39 ` Markus Armbruster
2020-09-25 16:54 ` [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo John Snow
2020-09-25 17:02 ` Peter Maydell
2020-09-25 17:09 ` John Snow
2020-09-25 19:25 ` no-reply
2020-09-25 21:37 ` Peter Maydell
2020-09-28 13:04 ` Markus Armbruster
2020-09-28 13:05 ` Peter Maydell
2020-09-29 15:26 ` Markus Armbruster
2020-09-29 15:43 ` Peter Maydell
2020-09-25 19:25 ` no-reply
2020-09-29 13:31 ` Markus Armbruster
2020-09-29 20:17 ` Markus Armbruster
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.