From: John Snow <jsnow@redhat.com>
To: Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <michael.roth@amd.com>,
Cleber Rosa <crosa@redhat.com>,
qemu-devel@nongnu.org, Eduardo Habkost <ehabkost@redhat.com>
Subject: Re: [PATCH v2 12/21] qapi/parser: add type hint annotations
Date: Tue, 18 May 2021 09:25:17 -0400 [thread overview]
Message-ID: <05bc229f-7200-04e3-ec9d-bf506b54d28b@redhat.com> (raw)
In-Reply-To: <87r1i445ez.fsf@dusky.pond.sub.org>
On 5/18/21 8:01 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
>
>> Annotations do not change runtime behavior.
>> This commit *only* adds annotations.
>>
>> (Annotations for QAPIDoc are in a forthcoming commit.)
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> scripts/qapi/parser.py | 58 +++++++++++++++++++++++++++---------------
>> 1 file changed, 38 insertions(+), 20 deletions(-)
>>
>> diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py
>> index 336959cbbb1..631863bac14 100644
>> --- a/scripts/qapi/parser.py
>> +++ b/scripts/qapi/parser.py
>> @@ -17,16 +17,26 @@
>> from collections import OrderedDict
>> import os
>> import re
>> -from typing import List
>> +from typing import (
>> + Dict,
>> + List,
>> + Optional,
>> + Set,
>> + Union,
>> +)
>>
>> from .common import must_match
>> from .error import QAPISemError, QAPISourceError
>> from .source import QAPISourceInfo
>>
>>
>> +# Return value alias for get_expr().
>> +_ExprValue = Union[List[object], Dict[str, object], str, bool]
>> +
>> +
>> class QAPIParseError(QAPISourceError):
>> """Error class for all QAPI schema parsing errors."""
>> - def __init__(self, parser, msg):
>> + def __init__(self, parser: 'QAPISchemaParser', msg: str):
>> col = 1
>> for ch in parser.src[parser.line_pos:parser.pos]:
>> if ch == '\t':
>> @@ -38,7 +48,10 @@ def __init__(self, parser, msg):
>>
>> class QAPISchemaParser:
>>
>> - def __init__(self, fname, previously_included=None, incl_info=None):
>> + def __init__(self,
>> + fname: str,
>> + previously_included: Optional[Set[str]] = None,
>
> We talked about the somewhat unnatural use of None for the empty set,
> and ways to avoid it. I agree with simply typing what we have.
>
Yup ... "later". We'll get to it.
>> + incl_info: Optional[QAPISourceInfo] = None):
>> self._fname = fname
>> self._included = previously_included or set()
>> self._included.add(os.path.abspath(self._fname))
>> @@ -46,20 +59,20 @@ def __init__(self, fname, previously_included=None, incl_info=None):
>>
>> # Lexer state (see `accept` for details):
>> self.info = QAPISourceInfo(self._fname, incl_info)
>> - self.tok = None
>> + self.tok: Union[None, str] = None
>> self.pos = 0
>> self.cursor = 0
>> - self.val = None
>> + self.val: Optional[Union[bool, str]] = None
>> self.line_pos = 0
>>
>> # Parser output:
>> - self.exprs = []
>> - self.docs = []
>> + self.exprs: List[Dict[str, object]] = []
>> + self.docs: List[QAPIDoc] = []
>>
>> # Showtime!
>> self._parse()
>>
>> - def _parse(self):
>> + def _parse(self) -> None:
>> cur_doc = None
>>
>> # May raise OSError; allow the caller to handle it.
>> @@ -125,7 +138,7 @@ def _parse(self):
>> self.reject_expr_doc(cur_doc)
>>
>> @staticmethod
>> - def reject_expr_doc(doc):
>> + def reject_expr_doc(doc: Optional['QAPIDoc']) -> None:
>> if doc and doc.symbol:
>> raise QAPISemError(
>> doc.info,
>> @@ -133,10 +146,14 @@ def reject_expr_doc(doc):
>> % doc.symbol)
>>
>> @staticmethod
>> - def _include(include, info, incl_fname, previously_included):
>> + def _include(include: str,
>> + info: QAPISourceInfo,
>> + incl_fname: str,
>> + previously_included: Set[str]
>> + ) -> Optional['QAPISchemaParser']:
>> incl_abs_fname = os.path.abspath(incl_fname)
>> # catch inclusion cycle
>> - inf = info
>> + inf: Optional[QAPISourceInfo] = info
>> while inf:
>> if incl_abs_fname == os.path.abspath(inf.fname):
>> raise QAPISemError(info, "inclusion loop for %s" % include)
>> @@ -155,9 +172,9 @@ def _include(include, info, incl_fname, previously_included):
>> ) from err
>>
>> @staticmethod
>> - def _pragma(name, value, info):
>> + def _pragma(name: str, value: object, info: QAPISourceInfo) -> None:
>>
>> - def check_list_str(name, value) -> List[str]:
>> + def check_list_str(name: str, value: object) -> List[str]:
>> if (not isinstance(value, list) or
>> any([not isinstance(elt, str) for elt in value])):
>> raise QAPISemError(
>> @@ -181,7 +198,7 @@ def check_list_str(name, value) -> List[str]:
>> else:
>> raise QAPISemError(info, "unknown pragma '%s'" % name)
>>
>> - def accept(self, skip_comment=True):
>> + def accept(self, skip_comment: bool = True) -> None:
>> while True:
>> self.tok = self.src[self.cursor]
>> self.pos = self.cursor
>> @@ -245,8 +262,8 @@ def accept(self, skip_comment=True):
>> self.src[self.cursor-1:])
>> raise QAPIParseError(self, "stray '%s'" % match.group(0))
>>
>> - def get_members(self):
>> - expr = OrderedDict()
>> + def get_members(self) -> Dict[str, object]:
>> + expr: Dict[str, object] = OrderedDict()
>
> I wish we didn't have to repeat the type in
>
> variable: type_of_thing = constructor_of_thing
>
> So clumsy. Using the constructor of a subtype doesn't exactly help. Oh
> well, that part should go away when we drop OrderedDict.
>
Yeah. Without an initial value it can't determine the types of the keys
and values.
>> if self.tok == '}':
>> self.accept()
>> return expr
>> @@ -272,8 +289,8 @@ def get_members(self):
>> if self.tok != "'":
>> raise QAPIParseError(self, "expected string")
>>
>> - def get_values(self):
>> - expr = []
>> + def get_values(self) -> List[object]:
>> + expr: List[object] = []
>> if self.tok == ']':
>> self.accept()
>> return expr
>> @@ -289,7 +306,8 @@ def get_values(self):
>> raise QAPIParseError(self, "expected ',' or ']'")
>> self.accept()
>>
>> - def get_expr(self):
>> + def get_expr(self) -> _ExprValue:
>> + expr: _ExprValue
>> if self.tok == '{':
>> self.accept()
>> expr = self.get_members()
>> @@ -305,7 +323,7 @@ def get_expr(self):
>> self, "expected '{', '[', string, or boolean")
>> return expr
>>
>> - def get_doc(self, info):
>> + def get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']:
>> if self.val != '##':
>> raise QAPIParseError(
>> self, "junk after '##' at start of documentation comment")
next prev parent reply other threads:[~2021-05-18 13:40 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-11 22:05 [PATCH v2 00/21] qapi: static typing conversion, pt5a John Snow
2021-05-11 22:05 ` [PATCH v2 01/21] qapi/parser: Don't try to handle file errors John Snow
2021-05-18 9:28 ` Markus Armbruster
2021-05-18 13:14 ` John Snow
2021-05-19 7:01 ` Markus Armbruster
2021-05-19 17:17 ` John Snow
2021-05-19 17:51 ` John Snow
2021-05-18 19:01 ` John Snow
2021-05-19 6:51 ` Markus Armbruster
2021-05-11 22:05 ` [PATCH v2 02/21] qapi: Add test for nonexistent schema file John Snow
2021-05-11 22:05 ` [PATCH v2 03/21] qapi/source: Remove line number from QAPISourceInfo initializer John Snow
2021-05-11 22:05 ` [PATCH v2 04/21] qapi/parser: factor parsing routine into method John Snow
2021-05-18 9:57 ` Markus Armbruster
2021-05-18 15:11 ` John Snow
2021-05-11 22:05 ` [PATCH v2 05/21] qapi/parser: Assert lexer value is a string John Snow
2021-05-18 10:02 ` Markus Armbruster
2021-05-18 15:19 ` John Snow
2021-05-11 22:05 ` [PATCH v2 06/21] qapi/parser: enforce all top-level expressions must be dict in _parse() John Snow
2021-05-11 22:05 ` [PATCH v2 07/21] qapi/parser: assert object keys are strings John Snow
2021-05-11 22:05 ` [PATCH v2 08/21] qapi/parser: Use @staticmethod where appropriate John Snow
2021-05-11 22:05 ` [PATCH v2 09/21] qapi: add must_match helper John Snow
2021-05-11 22:05 ` [PATCH v2 10/21] qapi/parser: Fix token membership tests when token can be None John Snow
2021-05-11 22:05 ` [PATCH v2 11/21] qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard John Snow
2021-05-11 22:05 ` [PATCH v2 12/21] qapi/parser: add type hint annotations John Snow
2021-05-18 12:01 ` Markus Armbruster
2021-05-18 13:25 ` John Snow [this message]
2021-05-11 22:05 ` [PATCH v2 13/21] qapi/parser: Remove superfluous list comprehension John Snow
2021-05-11 22:05 ` [PATCH v2 14/21] qapi/parser: allow 'ch' variable name John Snow
2021-05-11 22:05 ` [PATCH v2 15/21] qapi/parser: add docstrings John Snow
2021-05-19 6:41 ` Markus Armbruster
2021-05-19 18:21 ` John Snow
2021-05-11 22:05 ` [PATCH v2 16/21] CHECKPOINT John Snow
2021-05-11 22:05 ` [PATCH v2 17/21] qapi: [WIP] Rip QAPIDoc out of parser.py John Snow
2021-05-11 22:05 ` [PATCH v2 18/21] qapi: [WIP] Add type ignores for qapidoc.py John Snow
2021-05-11 22:05 ` [PATCH v2 19/21] qapi: [WIP] Import QAPIDoc from qapidoc Signed-off-by: John Snow <jsnow@redhat.com> John Snow
2021-05-11 22:06 ` [PATCH v2 20/21] qapi: [WIP] Add QAPIDocError John Snow
2021-05-11 22:06 ` [PATCH v2 21/21] qapi: [WIP] Enable linters on parser.py John Snow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=05bc229f-7200-04e3-ec9d-bf506b54d28b@redhat.com \
--to=jsnow@redhat.com \
--cc=armbru@redhat.com \
--cc=crosa@redhat.com \
--cc=ehabkost@redhat.com \
--cc=michael.roth@amd.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).