From: Peter Maydell <peter.maydell@linaro.org>
To: qemu-devel@nongnu.org
Cc: John Snow <jsnow@redhat.com>, Markus Armbruster <armbru@redhat.com>
Subject: [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling
Date: Fri, 25 Sep 2020 17:23:00 +0100 [thread overview]
Message-ID: <20200925162316.21205-6-peter.maydell@linaro.org> (raw)
In-Reply-To: <20200925162316.21205-1-peter.maydell@linaro.org>
Make the handling of indentation in doc comments more sophisticated,
so that when we see a section like:
Notes: some text
some more text
indented line 3
we save it for the doc-comment processing code as:
some text
some more text
indented line 3
and when we see a section with the heading on its own line:
Notes:
some text
some more text
indented text
we also accept that and save it in the same form.
If we detect that the comment document text is not indented as much
as we expect it to be, we throw a parse error. (We don't complain
about over-indented sections, because for rST this can be legitimate
markup.)
The golden reference for the doc comment text is updated to remove
the two 'wrong' indents; these now form a test case that we correctly
stripped leading whitespace from an indented multi-line argument
definition.
We update the documentation in docs/devel/qapi-code-gen.txt to
describe the new indentation rules.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
docs/devel/qapi-code-gen.txt | 23 +++++++
scripts/qapi/parser.py | 93 +++++++++++++++++++++------
tests/qapi-schema/doc-bad-indent.err | 1 +
tests/qapi-schema/doc-bad-indent.json | 8 +++
tests/qapi-schema/doc-bad-indent.out | 0
tests/qapi-schema/doc-good.out | 4 +-
tests/qapi-schema/meson.build | 1 +
7 files changed, 109 insertions(+), 21 deletions(-)
create mode 100644 tests/qapi-schema/doc-bad-indent.err
create mode 100644 tests/qapi-schema/doc-bad-indent.json
create mode 100644 tests/qapi-schema/doc-bad-indent.out
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 9eede44350c..69eaffac376 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -901,6 +901,22 @@ commands and events), member (for structs and unions), branch (for
alternates), or value (for enums), and finally optional tagged
sections.
+Descriptions of arguments can span multiple lines. The description
+text can start on the line following the '@argname:', in which case
+it must not be indented at all. It can also start on the same line
+as the '@argname:'. In this case if it spans multiple lines then
+second and subsequent lines must be indented to line up with the
+first character of the first line of the description:
+
+# @argone:
+# This is a two line description
+# in the first style.
+#
+# @argtwo: This is a two line description
+# in the second style.
+
+The number of spaces between the ':' and the text is not significant.
+
FIXME: the parser accepts these things in almost any order.
FIXME: union branches should be described, too.
@@ -911,6 +927,13 @@ A tagged section starts with one of the following words:
"Note:"/"Notes:", "Since:", "Example"/"Examples", "Returns:", "TODO:".
The section ends with the start of a new section.
+The text of a section can start on a new line, in
+which case it must not be indented at all. It can also start
+on the same line as the 'Note:', 'Returns:', etc tag. In this
+case if it spans multiple lines then second and subsequent
+lines must be indented to match the first, in the same way as
+multiline argument descriptions.
+
A 'Since: x.y.z' tagged section lists the release that introduced the
definition.
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 04bf10db378..6c3455b41f3 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -319,17 +319,32 @@ class QAPIDoc:
"""
class Section:
- def __init__(self, name=None):
+ def __init__(self, parser, name=None, indent=0):
+ # parser, for error messages about indentation
+ self._parser = parser
# optional section name (argument/member or section name)
self.name = name
self.text = ''
+ # the expected indent level of the text of this section
+ self._indent = indent
def append(self, line):
+ # Strip leading spaces corresponding to the expected indent level
+ # Blank lines are always OK.
+ if line:
+ indent = re.match(r'\s*', line).end()
+ if indent < self._indent:
+ raise QAPIParseError(
+ self._parser,
+ "unexpected de-indent (expected at least %d spaces)" %
+ self._indent)
+ line = line[self._indent:]
+
self.text += line.rstrip() + '\n'
class ArgSection(Section):
- def __init__(self, name):
- super().__init__(name)
+ def __init__(self, parser, name, indent=0):
+ super().__init__(parser, name, indent)
self.member = None
def connect(self, member):
@@ -343,7 +358,7 @@ class QAPIDoc:
self._parser = parser
self.info = info
self.symbol = None
- self.body = QAPIDoc.Section()
+ self.body = QAPIDoc.Section(parser)
# dict mapping parameter name to ArgSection
self.args = OrderedDict()
self.features = OrderedDict()
@@ -447,8 +462,21 @@ class QAPIDoc:
name = line.split(' ', 1)[0]
if name.startswith('@') and name.endswith(':'):
- line = line[len(name)+1:]
- self._start_args_section(name[1:-1])
+ # If line is "@arg: first line of description", find the
+ # index of 'f', which is the indent we expect for any
+ # following lines. We then remove the leading "@arg:" from line
+ # and replace it with spaces so that 'f' has the same index
+ # as it did in the original line and can be handled the same
+ # way we will handle following lines.
+ indent = re.match(r'@\S*:\s*', line).end()
+ line = line[indent:]
+ if not line:
+ # Line was just the "@arg:" header; following lines
+ # are not indented
+ indent = 0
+ else:
+ line = ' ' * indent + line
+ self._start_args_section(name[1:-1], indent)
elif self._is_section_tag(name):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -469,8 +497,21 @@ class QAPIDoc:
name = line.split(' ', 1)[0]
if name.startswith('@') and name.endswith(':'):
- line = line[len(name)+1:]
- self._start_features_section(name[1:-1])
+ # If line is "@arg: first line of description", find the
+ # index of 'f', which is the indent we expect for any
+ # following lines. We then remove the leading "@arg:" from line
+ # and replace it with spaces so that 'f' has the same index
+ # as it did in the original line and can be handled the same
+ # way we will handle following lines.
+ indent = re.match(r'@\S*:\s*', line).end()
+ line = line[indent:]
+ if not line:
+ # Line was just the "@arg:" header; following lines
+ # are not indented
+ indent = 0
+ else:
+ line = ' ' * indent + line
+ self._start_features_section(name[1:-1], indent)
elif self._is_section_tag(name):
self._append_line = self._append_various_line
self._append_various_line(line)
@@ -502,12 +543,25 @@ class QAPIDoc:
"'%s' can't follow '%s' section"
% (name, self.sections[0].name))
if self._is_section_tag(name):
- line = line[len(name)+1:]
- self._start_section(name[:-1])
+ # If line is "Section: first line of description", find the
+ # index of 'f', which is the indent we expect for any
+ # following lines. We then remove the leading "Section:" from line
+ # and replace it with spaces so that 'f' has the same index
+ # as it did in the original line and can be handled the same
+ # way we will handle following lines.
+ indent = re.match(r'\S*:\s*', line).end()
+ line = line[indent:]
+ if not line:
+ # Line was just the "Section:" header; following lines
+ # are not indented
+ indent = 0
+ else:
+ line = ' ' * indent + line
+ self._start_section(name[:-1], indent)
self._append_freeform(line)
- def _start_symbol_section(self, symbols_dict, name):
+ def _start_symbol_section(self, symbols_dict, name, indent):
# FIXME invalid names other than the empty string aren't flagged
if not name:
raise QAPIParseError(self._parser, "invalid parameter name")
@@ -516,21 +570,21 @@ class QAPIDoc:
"'%s' parameter name duplicated" % name)
assert not self.sections
self._end_section()
- self._section = QAPIDoc.ArgSection(name)
+ self._section = QAPIDoc.ArgSection(self._parser, name, indent)
symbols_dict[name] = self._section
- def _start_args_section(self, name):
- self._start_symbol_section(self.args, name)
+ def _start_args_section(self, name, indent):
+ self._start_symbol_section(self.args, name, indent)
- def _start_features_section(self, name):
- self._start_symbol_section(self.features, name)
+ def _start_features_section(self, name, indent):
+ self._start_symbol_section(self.features, name, indent)
- def _start_section(self, name=None):
+ def _start_section(self, name=None, indent=0):
if name in ('Returns', 'Since') and self.has_section(name):
raise QAPIParseError(self._parser,
"duplicated '%s' section" % name)
self._end_section()
- self._section = QAPIDoc.Section(name)
+ self._section = QAPIDoc.Section(self._parser, name, indent)
self.sections.append(self._section)
def _end_section(self):
@@ -553,7 +607,8 @@ class QAPIDoc:
def connect_member(self, member):
if member.name not in self.args:
# Undocumented TODO outlaw
- self.args[member.name] = QAPIDoc.ArgSection(member.name)
+ self.args[member.name] = QAPIDoc.ArgSection(self._parser,
+ member.name)
self.args[member.name].connect(member)
def connect_feature(self, feature):
diff --git a/tests/qapi-schema/doc-bad-indent.err b/tests/qapi-schema/doc-bad-indent.err
new file mode 100644
index 00000000000..67844539bd2
--- /dev/null
+++ b/tests/qapi-schema/doc-bad-indent.err
@@ -0,0 +1 @@
+doc-bad-indent.json:6:1: unexpected de-indent (expected at least 4 spaces)
diff --git a/tests/qapi-schema/doc-bad-indent.json b/tests/qapi-schema/doc-bad-indent.json
new file mode 100644
index 00000000000..edde8f21dc7
--- /dev/null
+++ b/tests/qapi-schema/doc-bad-indent.json
@@ -0,0 +1,8 @@
+# Multiline doc comments should have consistent indentation
+
+##
+# @foo:
+# @a: line one
+# line two is wrongly indented
+##
+{ 'command': 'foo', 'data': { 'a': 'int' } }
diff --git a/tests/qapi-schema/doc-bad-indent.out b/tests/qapi-schema/doc-bad-indent.out
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 9993ffcd89d..b7e3f4313da 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -159,7 +159,7 @@ doc symbol=Alternate
arg=i
an integer
- @b is undocumented
+@b is undocumented
arg=b
feature=alt-feat
@@ -174,7 +174,7 @@ doc symbol=cmd
the first argument
arg=arg2
the second
- argument
+argument
arg=arg3
feature=cmd-feat1
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index f1449298b07..83a0a68389b 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -53,6 +53,7 @@ schemas = [
'doc-bad-enum-member.json',
'doc-bad-event-arg.json',
'doc-bad-feature.json',
+ 'doc-bad-indent.json',
'doc-bad-section.json',
'doc-bad-symbol.json',
'doc-bad-union-member.json',
--
2.20.1
next prev parent reply other threads:[~2020-09-25 16:33 UTC|newest]
Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-25 16:22 [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
2020-09-25 16:22 ` [PATCH v6 01/21] qapi: Fix doc comment indentation again Peter Maydell
2020-09-28 12:39 ` Markus Armbruster
2020-09-25 16:22 ` [PATCH v6 02/21] qapi/block.json: Add newline after "Example:" for block-latency-histogram-set Peter Maydell
2020-09-28 12:42 ` Markus Armbruster
2020-09-28 12:49 ` Peter Maydell
2020-09-28 18:04 ` Markus Armbruster
2020-09-25 16:22 ` [PATCH v6 03/21] tests/qapi/doc-good.json: Prepare for qapi-doc Sphinx extension Peter Maydell
2020-09-25 16:22 ` [PATCH v6 04/21] scripts/qapi: Move doc-comment whitespace stripping to doc.py Peter Maydell
2020-09-25 16:23 ` Peter Maydell [this message]
2020-09-28 19:15 ` [PATCH v6 05/21] scripts/qapi/parser.py: improve doc comment indent handling Markus Armbruster
2020-09-29 8:55 ` Peter Maydell
2020-09-29 13:03 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 06/21] qapi/machine.json: Escape a literal '*' in doc comment Peter Maydell
2020-09-29 4:57 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 07/21] docs/sphinx: Add new qapi-doc Sphinx extension Peter Maydell
2020-09-29 6:54 ` Markus Armbruster
2020-09-29 9:05 ` Peter Maydell
2020-09-25 16:23 ` [PATCH v6 08/21] docs/interop: Convert qemu-ga-ref to rST Peter Maydell
2020-09-29 8:20 ` Markus Armbruster
2020-09-29 9:26 ` Peter Maydell
2020-09-29 13:11 ` Markus Armbruster
2020-09-29 14:25 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 09/21] docs/interop: Convert qemu-qmp-ref " Peter Maydell
2020-09-29 8:27 ` Markus Armbruster
2020-09-29 9:41 ` Peter Maydell
2020-09-29 13:12 ` Markus Armbruster
2020-09-29 8:42 ` Markus Armbruster
2020-09-29 9:46 ` Peter Maydell
2020-09-29 13:13 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 10/21] qapi: Use rST markup for literal blocks Peter Maydell
2020-09-25 16:23 ` [PATCH v6 11/21] qga/qapi-schema.json: Add some headings Peter Maydell
2020-09-29 8:30 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 12/21] tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis Peter Maydell
2020-09-29 8:33 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 13/21] meson.build: Move SPHINX_ARGS to top level meson.build file Peter Maydell
2020-09-29 8:45 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 14/21] meson.build: Make manuals depend on source to Sphinx extensions Peter Maydell
2020-09-29 8:52 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 15/21] tests/qapi-schema: Add test of the rST QAPI doc-comment outputn Peter Maydell
2020-09-29 12:20 ` Markus Armbruster
2020-09-29 12:33 ` Peter Maydell
2020-09-29 13:18 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 16/21] scripts/qapi: Remove texinfo generation support Peter Maydell
2020-09-29 12:22 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 17/21] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions Peter Maydell
2020-09-29 12:35 ` Markus Armbruster
2020-09-29 12:43 ` Peter Maydell
2020-09-29 13:27 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 18/21] scripts/texi2pod: Delete unused script Peter Maydell
2020-09-29 12:36 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 19/21] Remove Texinfo related line from git.orderfile Peter Maydell
2020-09-29 12:37 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 20/21] configure: Drop texinfo requirement Peter Maydell
2020-09-29 12:38 ` Markus Armbruster
2020-09-25 16:23 ` [PATCH v6 21/21] Remove texinfo dependency from docker and CI configs Peter Maydell
2020-09-29 12:39 ` Markus Armbruster
2020-09-25 16:54 ` [PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo John Snow
2020-09-25 17:02 ` Peter Maydell
2020-09-25 17:09 ` John Snow
2020-09-25 19:25 ` no-reply
2020-09-25 21:37 ` Peter Maydell
2020-09-28 13:04 ` Markus Armbruster
2020-09-28 13:05 ` Peter Maydell
2020-09-29 15:26 ` Markus Armbruster
2020-09-29 15:43 ` Peter Maydell
2020-09-25 19:25 ` no-reply
2020-09-29 13:31 ` Markus Armbruster
2020-09-29 20:17 ` Markus Armbruster
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200925162316.21205-6-peter.maydell@linaro.org \
--to=peter.maydell@linaro.org \
--cc=armbru@redhat.com \
--cc=jsnow@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.