From: John Snow <jsnow@redhat.com>
To: qemu-devel@nongnu.org
Cc: Michael Roth <michael.roth@amd.com>,
Cleber Rosa <crosa@redhat.com>, John Snow <jsnow@redhat.com>,
Markus Armbruster <armbru@redhat.com>,
Eduardo Habkost <ehabkost@redhat.com>
Subject: [PATCH 16/22] qapi/parser: add docstrings
Date: Wed, 21 Apr 2021 23:07:14 -0400 [thread overview]
Message-ID: <20210422030720.3685766-17-jsnow@redhat.com> (raw)
In-Reply-To: <20210422030720.3685766-1-jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
---
My hubris is infinite.
OK, I only added a few -- to help me remember how the parser works at a glance.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qapi/parser.py | 66 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 66 insertions(+)
diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
index dbbd0fcbc2f..8fc77808ace 100644
--- a/scripts/qapi/parser.py
+++ b/scripts/qapi/parser.py
@@ -51,7 +51,24 @@ def __init__(self, parser: 'QAPISchemaParser', msg: str):
class QAPISchemaParser:
+ """
+ Performs parsing of a QAPI schema source file.
+ :param fname: Path to the source file.
+ :param previously_included:
+ The absolute paths of previously included source files.
+ Only used by recursive calls to avoid re-parsing files.
+ :param incl_info:
+ `QAPISourceInfo` for the parent document.
+ This may be None if this is the root schema document.
+
+ :ivar exprs: Resulting parsed expressions.
+ :ivar docs: Resulting parsed documentation blocks.
+
+ :raise OSError: For problems opening the root schema document.
+ :raise QAPIParseError: For JSON or QAPIDoc syntax problems.
+ :raise QAPISemError: For various semantic issues with the schema.
+ """
def __init__(self,
fname: str,
previously_included: Optional[Set[str]] = None,
@@ -77,6 +94,11 @@ def __init__(self,
self._parse()
def _parse(self) -> None:
+ """
+ Parse the QAPI schema document.
+
+ :return: None; results are stored in ``exprs`` and ``docs``.
+ """
cur_doc = None
with open(self._fname, 'r', encoding='utf-8') as fp:
@@ -197,6 +219,50 @@ def _check(name: str, value: object) -> List[str]:
raise QAPISemError(info, "unknown pragma '%s'" % name)
def accept(self, skip_comment: bool = True) -> None:
+ """
+ Read the next lexeme and process it into a token.
+
+ :Object state:
+ :tok: represents the token type. See below for values.
+ :pos: is the position of the first character in the lexeme.
+ :cursor: is the position of the next character.
+ :val: is the variable value of the token, if any.
+
+ Single-character tokens:
+
+ These include ``LBRACE``, ``RBRACE``, ``COLON``, ``COMMA``,
+ ``LSQB``, and ``RSQB``. ``tok`` holds the single character
+ lexeme. ``val`` is ``None``.
+
+ Multi-character tokens:
+
+ - ``COMMENT``:
+
+ - This token is not normally yielded by the lexer, but it
+ can be when ``skip_comment`` is False.
+ - ``tok`` is the value ``"#"``.
+ - ``val`` is a string including all chars until end-of-line.
+
+ - ``STRING``:
+
+ - ``tok`` is the ``"'"``, the single quote.
+ - ``value`` is the string, *excluding* the quotes.
+
+ - ``TRUE`` and ``FALSE``:
+
+ - ``tok`` is either ``"t"`` or ``"f"`` accordingly.
+ - ``val`` is either ``True`` or ``False`` accordingly.
+
+ - ``NEWLINE`` and ``SPACE``:
+
+ - These are consumed by the lexer directly. ``line_pos`` and
+ ``info`` are advanced when ``NEWLINE`` is encountered.
+ ``tok`` is set to ``None`` upon reaching EOF.
+
+ :param skip_comment:
+ When false, return ``COMMENT`` tokens.
+ This is used when reading documentation blocks.
+ """
while True:
self.tok = self.src[self.cursor]
self.pos = self.cursor
--
2.30.2
next prev parent reply other threads:[~2021-04-22 3:12 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-22 3:06 [PATCH 00/22] qapi: static typing conversion, pt5a John Snow
2021-04-22 3:06 ` [PATCH 01/22] qapi/parser: Don't try to handle file errors John Snow
2021-04-23 15:46 ` Markus Armbruster
2021-04-23 19:20 ` John Snow
2021-04-27 13:47 ` Markus Armbruster
2021-04-27 17:58 ` John Snow
2021-04-28 5:48 ` Markus Armbruster
2021-04-22 3:07 ` [PATCH 02/22] qapi/source: [RFC] add "with_column" contextmanager John Snow
2021-04-27 9:33 ` Markus Armbruster
2021-04-22 3:07 ` [PATCH 03/22] qapi/source: Remove line number from QAPISourceInfo initializer John Snow
2021-04-24 6:38 ` Markus Armbruster
2021-04-26 17:39 ` John Snow
2021-04-26 23:14 ` John Snow
2021-04-27 6:07 ` Markus Armbruster
2021-04-22 3:07 ` [PATCH 04/22] qapi/parser: factor parsing routine into method John Snow
2021-04-22 3:07 ` [PATCH 05/22] qapi/parser: Assert lexer value is a string John Snow
2021-04-24 8:33 ` Markus Armbruster
2021-04-26 17:43 ` John Snow
2021-04-27 12:30 ` Markus Armbruster
2021-04-27 13:58 ` John Snow
2021-04-22 3:07 ` [PATCH 06/22] qapi/parser: assert get_expr returns object in outer loop John Snow
2021-04-25 7:23 ` Markus Armbruster
2021-04-27 15:03 ` John Snow
2021-04-22 3:07 ` [PATCH 07/22] qapi/parser: assert object keys are strings John Snow
2021-04-25 7:27 ` Markus Armbruster
2021-04-26 17:46 ` John Snow
2021-04-27 6:13 ` Markus Armbruster
2021-04-27 14:15 ` John Snow
2021-04-22 3:07 ` [PATCH 08/22] qapi/parser: Use @staticmethod where appropriate John Snow
2021-04-22 3:07 ` [PATCH 09/22] qapi: add match_nofail helper John Snow
2021-04-25 7:54 ` Markus Armbruster
2021-04-26 17:48 ` John Snow
2021-04-22 3:07 ` [PATCH 10/22] qapi/parser: Fix typing of token membership tests John Snow
2021-04-25 7:59 ` Markus Armbruster
2021-04-26 17:51 ` John Snow
2021-04-27 7:00 ` Markus Armbruster
2021-05-04 1:01 ` John Snow
2021-05-05 6:29 ` Markus Armbruster
2021-04-22 3:07 ` [PATCH 11/22] qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard John Snow
2021-04-25 12:32 ` Markus Armbruster
2021-04-26 23:48 ` John Snow
2021-04-27 7:15 ` Markus Armbruster
2021-05-05 19:09 ` John Snow
2021-04-22 3:07 ` [PATCH 12/22] qapi/parser: add type hint annotations John Snow
2021-04-25 12:34 ` Markus Armbruster
2021-04-26 18:00 ` John Snow
2021-04-27 8:21 ` Markus Armbruster
2021-04-26 23:55 ` John Snow
2021-04-27 8:43 ` Markus Armbruster
2021-05-06 1:49 ` John Snow
2021-05-06 1:27 ` John Snow
2021-04-22 3:07 ` [PATCH 13/22] qapi/parser: [RFC] overload the return type of get_expr John Snow
2021-04-22 3:07 ` [PATCH 14/22] qapi/parser: Remove superfluous list constructor John Snow
2021-04-22 3:07 ` [PATCH 15/22] qapi/parser: allow 'ch' variable name John Snow
2021-04-22 3:07 ` John Snow [this message]
2021-04-25 13:27 ` [PATCH 16/22] qapi/parser: add docstrings Markus Armbruster
2021-04-26 18:26 ` John Snow
2021-04-27 9:03 ` Markus Armbruster
2021-05-06 2:08 ` John Snow
2021-05-07 1:34 ` John Snow
2021-05-07 8:25 ` Markus Armbruster
2021-04-22 3:07 ` [PATCH 17/22] CHECKPOINT John Snow
2021-04-22 3:07 ` [PATCH 18/22] qapi: [WIP] Rip QAPIDoc out of parser.py John Snow
2021-04-22 3:07 ` [PATCH 19/22] qapi: [WIP] Add type ignores for qapidoc.py John Snow
2021-04-22 3:07 ` [PATCH 20/22] qapi: [WIP] Import QAPIDoc from qapidoc Signed-off-by: John Snow <jsnow@redhat.com> John Snow
2021-04-22 3:07 ` [PATCH 21/22] qapi: [WIP] Add QAPIDocError John Snow
2021-04-22 3:07 ` [PATCH 22/22] qapi: [WIP] Enable linters on parser.py John Snow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210422030720.3685766-17-jsnow@redhat.com \
--to=jsnow@redhat.com \
--cc=armbru@redhat.com \
--cc=crosa@redhat.com \
--cc=ehabkost@redhat.com \
--cc=michael.roth@amd.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).