All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Snow <jsnow@redhat.com>
To: qemu-devel@nongnu.org, Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <michael.roth@amd.com>, John Snow <jsnow@redhat.com>,
	Eduardo Habkost <ehabkost@redhat.com>,
	Cleber Rosa <crosa@redhat.com>
Subject: [PATCH v4 12/14] qapi/introspect.py: add introspect.json dummy types
Date: Tue,  2 Feb 2021 12:46:49 -0500	[thread overview]
Message-ID: <20210202174651.2274166-13-jsnow@redhat.com> (raw)
In-Reply-To: <20210202174651.2274166-1-jsnow@redhat.com>

Add some aliases that declare intent for some of the "dictly-typed"
objects we pass around in introspect.py.

Signed-off-by: John Snow <jsnow@redhat.com>

---

This patch is optional, it can be dropped if desired. It is purely for
the sake of demonstrating what the _DObject was meant to convey: a
Python Dict that represents some JSON Object. It does not add any type
safety in and of itself, but does have some (light) annotational
benefit. In this case, it's usually a specific data structure from the
QAPI Schema we are referencing, but do not have "compile-time" access to
in Python.

These are loosely typed and don't bother reproducing the exact structure
of the real types. Python 3.6 does not have support for TypedDict
structures, so this is as good as we can do without involving a
third-party library (e.g. Pydantic), in which we might be able to say:


class SchemaMetaType(str, enum.Enum):
    BUILTIN = "builtin"
    ENUM = "enum"
    ARRAY = "array"
    OBJECT = "object"
    ALTERNATE = "alternate"
    COMMAND = "command"
    EVENT = "event"


class SchemaInfo(pydantic.BaseModel):
    name: str
    meta-type: SchemaMetaType
    features: Optional[List[str]]
    data: Union[SchemaInfoBuiltin, SchemaInfoEnum, SchemaInfoArray,
                SchemaInfoObject, SchemaInfoAlternate, SchemaInfoCommand,
                SchemaInfoEvent]


However, the cost of reproducing and keeping these structure definitions
in-sync between the user-defined portion of the schema and the code
generator is likely not worth doing any such thing. However, it does
illustrate an interesting dependency the generator has on the
user-defined schema itself in terms of types.

So, I settled on using some light types that suggest the form of the
object instead of enforcing the form.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 scripts/qapi/introspect.py | 49 ++++++++++++++++++++++++--------------
 1 file changed, 31 insertions(+), 18 deletions(-)

diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index b7f2a6cf260..eb5d34cdb42 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -66,8 +66,15 @@
 _value = Union[_scalar, _nonscalar]
 TreeValue = Union[_value, 'Annotated[_value]']
 
-# This is a (strict) alias for an arbitrary object non-scalar, as above:
-_DObject = Dict[str, object]
+# These types are based on structures defined in QEMU's schema, so we lack
+# precise types for them here. Python 3.6 does not offer TypedDict constructs,
+# so they are loosely typed here as simple python Dicts.
+SchemaInfo = Dict[str, object]
+SchemaInfoObject = Dict[str, object]
+SchemaInfoObjectVariant = Dict[str, object]
+SchemaInfoObjectMember = Dict[str, object]
+SchemaInfoCommand = Dict[str, object]
+
 
 _NodeT = TypeVar('_NodeT', bound=TreeValue)
 
@@ -160,7 +167,7 @@ def __init__(self, prefix: str, unmask: bool):
             ' * QAPI/QMP schema introspection', __doc__)
         self._unmask = unmask
         self._schema: Optional[QAPISchema] = None
-        self._trees: List[Annotated[_DObject]] = []
+        self._trees: List[Annotated[SchemaInfo]] = []
         self._used_types: List[QAPISchemaType] = []
         self._name_map: Dict[str, str] = {}
         self._genc.add(mcgen('''
@@ -232,9 +239,18 @@ def _gen_features(features: List[QAPISchemaFeature]
                       ) -> List[Annotated[str]]:
         return [Annotated(f.name, f.ifcond) for f in features]
 
-    def _gen_tree(self, name: str, mtype: str, obj: _DObject,
+    def _gen_tree(self, name: str, mtype: str, obj: Dict[str, object],
                   ifcond: List[str],
                   features: Optional[List[QAPISchemaFeature]]) -> None:
+        """
+        Build and append a SchemaInfo object to self._trees.
+
+        :param name: The entity's name.
+        :param mtype: The entity's meta-type.
+        :param obj: Additional entity fields, as appropriate for the meta-type.
+        :param ifcond: List of conditionals that apply to this entire entity.
+        :param features: Optional features field for SchemaInfo.
+        """
         comment: Optional[str] = None
         if mtype not in ('command', 'event', 'builtin', 'array'):
             if not self._unmask:
@@ -249,8 +265,8 @@ def _gen_tree(self, name: str, mtype: str, obj: _DObject,
         self._trees.append(Annotated(obj, ifcond, comment))
 
     def _gen_member(self, member: QAPISchemaObjectTypeMember
-                    ) -> Annotated[_DObject]:
-        obj: _DObject = {
+                    ) -> Annotated[SchemaInfoObjectMember]:
+        obj: SchemaInfoObjectMember = {
             'name': member.name,
             'type': self._use_type(member.type)
         }
@@ -260,13 +276,9 @@ def _gen_member(self, member: QAPISchemaObjectTypeMember
             obj['features'] = self._gen_features(member.features)
         return Annotated(obj, member.ifcond)
 
-    def _gen_variants(self, tag_name: str,
-                      variants: List[QAPISchemaVariant]) -> _DObject:
-        return {'tag': tag_name,
-                'variants': [self._gen_variant(v) for v in variants]}
-
-    def _gen_variant(self, variant: QAPISchemaVariant) -> Annotated[_DObject]:
-        obj: _DObject = {
+    def _gen_variant(self, variant: QAPISchemaVariant
+                     ) -> Annotated[SchemaInfoObjectVariant]:
+        obj: SchemaInfoObjectVariant = {
             'case': variant.name,
             'type': self._use_type(variant.type)
         }
@@ -298,11 +310,12 @@ def visit_object_type_flat(self, name: str, info: Optional[QAPISourceInfo],
                                features: List[QAPISchemaFeature],
                                members: List[QAPISchemaObjectTypeMember],
                                variants: Optional[QAPISchemaVariants]) -> None:
-        obj: _DObject = {'members': [self._gen_member(m) for m in members]}
+        obj: SchemaInfoObject = {
+            'members': [self._gen_member(m) for m in members]
+        }
         if variants:
-            obj.update(self._gen_variants(variants.tag_member.name,
-                                          variants.variants))
-
+            obj['tag'] = variants.tag_member.name
+            obj['variants'] = [self._gen_variant(v) for v in variants.variants]
         self._gen_tree(name, 'object', obj, ifcond, features)
 
     def visit_alternate_type(self, name: str, info: Optional[QAPISourceInfo],
@@ -327,7 +340,7 @@ def visit_command(self, name: str, info: Optional[QAPISourceInfo],
 
         arg_type = arg_type or self._schema.the_empty_object_type
         ret_type = ret_type or self._schema.the_empty_object_type
-        obj: _DObject = {
+        obj: SchemaInfoCommand = {
             'arg-type': self._use_type(arg_type),
             'ret-type': self._use_type(ret_type)
         }
-- 
2.29.2



  parent reply	other threads:[~2021-02-02 17:56 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-02 17:46 [PATCH v4 00/14] qapi: static typing conversion, pt2 John Snow
2021-02-02 17:46 ` [PATCH v4 01/14] qapi/introspect.py: assert schema is not None John Snow
2021-02-02 17:46 ` [PATCH v4 02/14] qapi/introspect.py: use _make_tree for features nodes John Snow
2021-02-03 13:49   ` Markus Armbruster
2021-02-02 17:46 ` [PATCH v4 03/14] qapi/introspect.py: add _gen_features helper John Snow
2021-02-02 17:46 ` [PATCH v4 04/14] qapi/introspect.py: guard against ifcond/comment misuse John Snow
2021-02-03 14:08   ` Markus Armbruster
2021-02-03 20:42     ` John Snow
2021-02-03 21:18       ` Eduardo Habkost
2021-02-04 15:06       ` Markus Armbruster
2021-02-02 17:46 ` [PATCH v4 05/14] qapi/introspect.py: Unify return type of _make_tree() John Snow
2021-02-02 17:46 ` [PATCH v4 06/14] qapi/introspect.py: replace 'extra' dict with 'comment' argument John Snow
2021-02-03 14:23   ` Markus Armbruster
2021-02-03 21:21     ` John Snow
2021-02-04  8:37       ` Markus Armbruster
2021-02-02 17:46 ` [PATCH v4 07/14] qapi/introspect.py: Introduce preliminary tree typing John Snow
2021-02-03 14:30   ` Markus Armbruster
2021-02-03 21:40     ` John Snow
2021-02-02 17:46 ` [PATCH v4 08/14] qapi/introspect.py: create a typed 'Annotated' data strutcure John Snow
2021-02-03 14:47   ` Markus Armbruster
2021-02-03 21:50     ` Eduardo Habkost
2021-02-04 15:37       ` Markus Armbruster
2021-02-04 16:20         ` John Snow
2021-02-04 16:28         ` Eduardo Habkost
2021-02-05  8:45           ` Markus Armbruster
2021-02-03 23:12     ` John Snow
2021-02-05  9:10       ` Markus Armbruster
2021-02-02 17:46 ` [PATCH v4 09/14] qapi/introspect.py: improve _tree_to_qlit error message John Snow
2021-02-02 17:46 ` [PATCH v4 10/14] qapi/introspect.py: improve readability of _tree_to_qlit John Snow
2021-02-02 17:46 ` [PATCH v4 11/14] qapi/introspect.py: add type hint annotations John Snow
2021-02-03 15:15   ` Markus Armbruster
2021-02-03 23:27     ` John Snow
2021-02-05 13:42       ` Markus Armbruster
2021-02-08 21:39         ` John Snow
2021-02-08 21:53           ` John Snow
2021-02-09  9:06           ` Markus Armbruster
2021-02-10 17:31             ` John Snow
2021-02-02 17:46 ` John Snow [this message]
2021-02-02 17:46 ` [PATCH v4 13/14] qapi/introspect.py: Add docstring to _tree_to_qlit John Snow
2021-02-02 17:46 ` [PATCH v4 14/14] qapi/introspect.py: Update copyright and authors list John Snow

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210202174651.2274166-13-jsnow@redhat.com \
    --to=jsnow@redhat.com \
    --cc=armbru@redhat.com \
    --cc=crosa@redhat.com \
    --cc=ehabkost@redhat.com \
    --cc=michael.roth@amd.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.