QEMU-Devel Archive on lore.kernel.org
 help / color / Atom feed
From: Peter Maydell <peter.maydell@linaro.org>
To: qemu-devel@nongnu.org
Cc: "Daniel P. Berrangé" <berrange@redhat.com>,
	"Markus Armbruster" <armbru@redhat.com>,
	"Michael Roth" <mdroth@linux.vnet.ibm.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	"John Snow" <jsnow@redhat.com>
Subject: [PATCH v2 23/30] scripts/qapi/parser.py: improve doc comment indent handling
Date: Thu, 13 Feb 2020 17:56:40 +0000
Message-ID: <20200213175647.17628-24-peter.maydell@linaro.org> (raw)
In-Reply-To: <20200213175647.17628-1-peter.maydell@linaro.org>

Make the handling of indentation in doc comments more sophisticated,
so that when we see a section like:

Notes: some text
       some more text
          indented line 3

we save it for the doc-comment processing code as:

some text
some more text
   indented line 3

and when we see a section with the heading on its own line:

Notes:

some text
some more text
   indented text

we also accept that and save it in the same form.

The exception is that we always retain indentation as-is for Examples
sections, because these are literal text.

If we detect that the comment document text is not indented as much
as we expect it to be, we throw a parse error.  (We don't complain
about over-indented sections, because for rST this can be legitimate
markup.)

The golden reference for the doc comment text is updated to remove
the two 'wrong' indents; these now form a test case that we correctly
stripped leading whitespace from an indented multi-line argument
definition.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
v1->v2: Update doc-good.out as per final para.
---
 scripts/qapi/parser.py         | 82 +++++++++++++++++++++++++++-------
 tests/qapi-schema/doc-good.out |  4 +-
 2 files changed, 67 insertions(+), 19 deletions(-)

diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index 2196ec5de1e..66f802641c9 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -313,18 +313,32 @@ class QAPIDoc(object):
     """
 
     class Section(object):
-        def __init__(self, name=None):
+        def __init__(self, parser, name=None, indent=0):
+            # parser, for error messages about indentation
+            self._parser = parser
             # optional section name (argument/member or section name)
             self.name = name
             # the list of lines for this section
             self.text = ''
+            # the expected indent level of the text of this section
+            self._indent = indent
 
         def append(self, line):
+            # Strip leading spaces corresponding to the expected indent level
+            # Blank lines are always OK.
+            if line:
+                spacecount = len(line) - len(line.lstrip(" "))
+                if spacecount > self._indent:
+                    spacecount = self._indent
+                if spacecount < self._indent:
+                    raise QAPIParseError(self._parser, "unexpected de-indent")
+                line = line[spacecount:]
+
             self.text += line.rstrip() + '\n'
 
     class ArgSection(Section):
-        def __init__(self, name):
-            QAPIDoc.Section.__init__(self, name)
+        def __init__(self, parser, name, indent=0):
+            QAPIDoc.Section.__init__(self, parser, name, indent)
             self.member = None
 
         def connect(self, member):
@@ -338,7 +352,7 @@ class QAPIDoc(object):
         self._parser = parser
         self.info = info
         self.symbol = None
-        self.body = QAPIDoc.Section()
+        self.body = QAPIDoc.Section(parser)
         # dict mapping parameter name to ArgSection
         self.args = OrderedDict()
         self.features = OrderedDict()
@@ -443,7 +457,18 @@ class QAPIDoc(object):
 
         if name.startswith('@') and name.endswith(':'):
             line = line[len(name)+1:]
-            self._start_args_section(name[1:-1])
+            if not line or line.isspace():
+                # Line was just the "@arg:" header; following lines
+                # are not indented
+                indent = 0
+                line = ''
+            else:
+                # Line is "@arg: first line of description"; following
+                # lines should be indented by len(name) + 1, and we
+                # pad out this first line so it is handled the same way
+                indent = len(name) + 1
+                line = ' ' * indent + line
+            self._start_args_section(name[1:-1], indent)
         elif self._is_section_tag(name):
             self._append_line = self._append_various_line
             self._append_various_line(line)
@@ -465,7 +490,17 @@ class QAPIDoc(object):
 
         if name.startswith('@') and name.endswith(':'):
             line = line[len(name)+1:]
-            self._start_features_section(name[1:-1])
+            if not line or line.isspace():
+                # Line is just the "@name:" header, no ident for following lines
+                indent = 0
+                line = ''
+            else:
+                # Line is "@arg: first line of description"; following
+                # lines should be indented by len(name) + 3, and we
+                # pad out this first line so it is handled the same way
+                indent = len(name) + 1
+                line = ' ' * indent + line
+            self._start_features_section(name[1:-1], indent)
         elif self._is_section_tag(name):
             self._append_line = self._append_various_line
             self._append_various_line(line)
@@ -498,11 +533,23 @@ class QAPIDoc(object):
                                  % (name, self.sections[0].name))
         elif self._is_section_tag(name):
             line = line[len(name)+1:]
-            self._start_section(name[:-1])
+            if not line or line.isspace():
+                # Line is just "SectionName:", no indent for following lines
+                indent = 0
+                line = ''
+            elif name.startswith("Example"):
+                # The "Examples" section is literal-text, so preserve
+                # all the indentation as-is
+                indent = 0
+            else:
+                # Line is "SectionName: some text", indent required
+                indent = len(name) + 1
+                line = ' ' * indent + line
+            self._start_section(name[:-1], indent)
 
         self._append_freeform(line)
 
-    def _start_symbol_section(self, symbols_dict, name):
+    def _start_symbol_section(self, symbols_dict, name, indent):
         # FIXME invalid names other than the empty string aren't flagged
         if not name:
             raise QAPIParseError(self._parser, "invalid parameter name")
@@ -511,21 +558,21 @@ class QAPIDoc(object):
                                  "'%s' parameter name duplicated" % name)
         assert not self.sections
         self._end_section()
-        self._section = QAPIDoc.ArgSection(name)
+        self._section = QAPIDoc.ArgSection(self._parser, name, indent)
         symbols_dict[name] = self._section
 
-    def _start_args_section(self, name):
-        self._start_symbol_section(self.args, name)
+    def _start_args_section(self, name, indent):
+        self._start_symbol_section(self.args, name, indent)
 
-    def _start_features_section(self, name):
-        self._start_symbol_section(self.features, name)
+    def _start_features_section(self, name, indent):
+        self._start_symbol_section(self.features, name, indent)
 
-    def _start_section(self, name=None):
+    def _start_section(self, name=None, indent=0):
         if name in ('Returns', 'Since') and self.has_section(name):
             raise QAPIParseError(self._parser,
                                  "duplicated '%s' section" % name)
         self._end_section()
-        self._section = QAPIDoc.Section(name)
+        self._section = QAPIDoc.Section(self._parser, name, indent)
         self.sections.append(self._section)
 
     def _end_section(self):
@@ -548,7 +595,7 @@ class QAPIDoc(object):
     def connect_member(self, member):
         if member.name not in self.args:
             # Undocumented TODO outlaw
-            self.args[member.name] = QAPIDoc.ArgSection(member.name)
+            self.args[member.name] = QAPIDoc.ArgSection(self._parser, member.name)
         self.args[member.name].connect(member)
 
     def connect_feature(self, feature):
@@ -556,7 +603,8 @@ class QAPIDoc(object):
             raise QAPISemError(feature.info,
                                "feature '%s' lacks documentation"
                                % feature.name)
-            self.features[feature.name] = QAPIDoc.ArgSection(feature.name)
+            self.features[feature.name] = QAPIDoc.ArgSection(self._parser,
+                                                             feature.name)
         self.features[feature.name].connect(feature)
 
     def check_expr(self, expr):
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index a65bce639ff..0302ce0bde1 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -144,7 +144,7 @@ doc symbol=Alternate
 
     arg=i
 an integer
-    @b is undocumented
+@b is undocumented
     arg=b
 
 doc freeform
@@ -157,7 +157,7 @@ doc symbol=cmd
 the first argument
     arg=arg2
 the second
-       argument
+argument
     arg=arg3
 
     feature=cmd-feat1
-- 
2.20.1



  parent reply index

Thread overview: 69+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-13 17:56 [PATCH v2 00/30] Convert QAPI doc comments to generate rST instead of texinfo Peter Maydell
2020-02-13 17:56 ` [PATCH v2 01/30] configure: Allow user to specify sphinx-build binary Peter Maydell
2020-02-14  6:33   ` Markus Armbruster
2020-02-14  9:21     ` Peter Maydell
2020-02-14 12:20       ` Markus Armbruster
2020-02-14 12:39         ` Peter Maydell
2020-02-14 17:18           ` Markus Armbruster
2020-02-14 17:36             ` Peter Maydell
2020-02-15 10:37               ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 02/30] configure: Check that sphinx-build is using Python 3 Peter Maydell
2020-02-13 17:56 ` [PATCH v2 03/30] Makefile: Fix typo in dependency list for interop manpages Peter Maydell
2020-02-13 17:56 ` [PATCH v2 04/30] qga/qapi-schema.json: Fix missing '-' in GuestDiskBusType doc comment Peter Maydell
2020-02-13 17:56 ` [PATCH v2 05/30] qga/qapi-schema.json: Fix indent level on doc comments Peter Maydell
2020-02-14 12:36   ` Markus Armbruster
2020-02-14 12:40     ` Peter Maydell
2020-02-14 14:26       ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 06/30] qga/qapi-schema.json: minor format fixups for rST Peter Maydell
2020-02-14 12:46   ` Markus Armbruster
2020-02-14 14:33     ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 07/30] qapi/block-core.json: Use literal block for ascii art Peter Maydell
2020-02-13 22:59   ` Aleksandar Markovic
2020-02-15 20:56     ` Philippe Mathieu-Daudé
2020-02-15 21:01       ` Aleksandar Markovic
2020-02-17  0:44         ` Philippe Mathieu-Daudé
2020-02-17  5:53           ` Samuel Thibault
2020-02-14 12:53   ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 08/30] qapi: Use ':' after @argument in doc comments Peter Maydell
2020-02-14 13:02   ` Markus Armbruster
2020-02-14 13:28     ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 09/30] qapi: Fix indent level on doc comments in json files Peter Maydell
2020-02-14 13:45   ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 10/30] qapi: Remove hardcoded tabs Peter Maydell
2020-02-13 17:56 ` [PATCH v2 11/30] qapi/ui.json: Put input-send-event body text in the right place Peter Maydell
2020-02-13 17:56 ` [PATCH v2 12/30] qapi/ui.json: Avoid `...' texinfo style quoting Peter Maydell
2020-02-13 17:56 ` [PATCH v2 13/30] qapi/block-core.json: Use explicit bulleted lists Peter Maydell
2020-02-13 17:56 ` [PATCH v2 14/30] qapi/ui.json: " Peter Maydell
2020-02-14 14:20   ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 15/30] qapi/{block, misc, tmp, net}.json: " Peter Maydell
2020-02-14 14:23   ` Markus Armbruster
2020-02-14 14:28     ` Peter Maydell
2020-02-14 15:46       ` Markus Armbruster
2020-02-14 15:48         ` Peter Maydell
2020-02-13 17:56 ` [PATCH v2 16/30] qapi: Add blank lines before " Peter Maydell
2020-02-14 14:33   ` Markus Armbruster
2020-02-14 16:02     ` Markus Armbruster
2020-02-14 16:16       ` Peter Maydell
2020-02-14 17:11         ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 17/30] qapi/migration.json: Replace _this_ with *this* Peter Maydell
2020-02-13 19:02   ` Philippe Mathieu-Daudé
2020-02-14 14:35   ` Markus Armbruster
2020-02-13 17:56 ` [PATCH v2 18/30] qapi: Delete all the "foo: dropped in n.n" notes Peter Maydell
2020-02-14  6:55   ` Markus Armbruster
2020-02-14 15:13     ` Markus Armbruster
2020-02-14 15:20       ` Peter Maydell
2020-02-13 17:56 ` [PATCH v2 19/30] qapi/qapi-schema.json: Put headers in their own doc-comment blocks Peter Maydell
2020-02-13 17:56 ` [PATCH v2 20/30] qapi/machine.json: Escape a literal '*' in doc comment Peter Maydell
2020-02-13 19:01   ` Philippe Mathieu-Daudé
2020-02-13 17:56 ` [PATCH v2 21/30] tests/qapi/doc-good.json: Clean up markup Peter Maydell
2020-02-13 17:56 ` [PATCH v2 22/30] scripts/qapi: Move doc-comment whitespace stripping to doc.py Peter Maydell
2020-02-13 17:56 ` Peter Maydell [this message]
2020-02-13 17:56 ` [PATCH v2 24/30] docs/sphinx: Add new qapi-doc Sphinx extension Peter Maydell
2020-02-13 17:56 ` [PATCH v2 25/30] docs/interop: Convert qemu-ga-ref to rST Peter Maydell
2020-02-13 17:56 ` [PATCH v2 26/30] docs/interop: Convert qemu-qmp-ref " Peter Maydell
2020-02-17 15:10   ` Peter Maydell
2020-02-13 17:56 ` [PATCH v2 27/30] qapi: Use rST markup for literal blocks Peter Maydell
2020-02-13 17:56 ` [PATCH v2 28/30] qga/qapi-schema.json: Add some headings Peter Maydell
2020-02-13 17:56 ` [PATCH v2 29/30] scripts/qapi: Remove texinfo generation support Peter Maydell
2020-02-13 17:56 ` [PATCH v2 30/30] docs/devel/qapi-code-gen.txt: Update to new rST backend conventions Peter Maydell
2020-02-13 20:51 ` [PATCH v2 00/30] Convert QAPI doc comments to generate rST instead of texinfo no-reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200213175647.17628-24-peter.maydell@linaro.org \
    --to=peter.maydell@linaro.org \
    --cc=armbru@redhat.com \
    --cc=berrange@redhat.com \
    --cc=jsnow@redhat.com \
    --cc=mdroth@linux.vnet.ibm.com \
    --cc=qemu-devel@nongnu.org \
    --cc=stefanha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

QEMU-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/qemu-devel/0 qemu-devel/git/0.git
	git clone --mirror https://lore.kernel.org/qemu-devel/1 qemu-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 qemu-devel qemu-devel/ https://lore.kernel.org/qemu-devel \
		qemu-devel@nongnu.org
	public-inbox-index qemu-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.nongnu.qemu-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git