From: Markus Armbruster <armbru@redhat.com>
To: John Snow <jsnow@redhat.com>
Cc: qemu-devel@nongnu.org, Eduardo Habkost <ehabkost@redhat.com>,
Cleber Rosa <crosa@redhat.com>
Subject: Re: [PATCH v5 03/36] qapi-gen: Separate arg-parsing from generation
Date: Thu, 08 Oct 2020 07:56:44 +0200 [thread overview]
Message-ID: <87h7r545wj.fsf@dusky.pond.sub.org> (raw)
In-Reply-To: <65c2a120-e29e-b572-680a-ee6d141ff9c8@redhat.com> (John Snow's message of "Wed, 7 Oct 2020 10:52:54 -0400")
John Snow <jsnow@redhat.com> writes:
> On 10/7/20 3:54 AM, Markus Armbruster wrote:
>> John Snow <jsnow@redhat.com> writes:
>>
>>> On 10/6/20 7:51 AM, Markus Armbruster wrote:
>>>> John Snow <jsnow@redhat.com> writes:
>>>>
>>>>> This is a minor re-work of the entrypoint script. It isolates a
>>>>> generate() method from the actual command-line mechanism.
>>>>>
>>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>>> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
>>>>> Reviewed-by: Cleber Rosa <crosa@redhat.com>
>>>>> Tested-by: Cleber Rosa <crosa@redhat.com>
>>>>> ---
>>>>> scripts/qapi-gen.py | 85 +++++++++++++++++++++++++++++++++------------
>>>>> 1 file changed, 62 insertions(+), 23 deletions(-)
>>>>>
>>>>> diff --git a/scripts/qapi-gen.py b/scripts/qapi-gen.py
>>>>> index 541e8c1f55d..117b396a595 100644
>>>>> --- a/scripts/qapi-gen.py
>>>>> +++ b/scripts/qapi-gen.py
>>>>> @@ -1,30 +1,77 @@
>>>>> #!/usr/bin/env python3
>>>>> -# QAPI generator
>>>>> -#
>>>>> +
>>>>> # This work is licensed under the terms of the GNU GPL, version 2 or later.
>>>>> # See the COPYING file in the top-level directory.
>>>>> +"""
>>>>> +QAPI Generator
>>>>> +
>>>>> +This script is the main entry point for generating C code from the QAPI schema.
>>>>> +"""
>>>>> import argparse
>>>>> import re
>>>>> import sys
>>>>> from qapi.commands import gen_commands
>>>>> +from qapi.error import QAPIError
>>>>> from qapi.events import gen_events
>>>>> from qapi.introspect import gen_introspect
>>>>> -from qapi.schema import QAPIError, QAPISchema
>>>>> +from qapi.schema import QAPISchema
>>>>> from qapi.types import gen_types
>>>>> from qapi.visit import gen_visit
>>>> Unrelated cleanup. Okay.
>>>>
>>>>> -def main(argv):
>>>>> +DEFAULT_OUTPUT_DIR = ''
>>>>> +DEFAULT_PREFIX = ''
>>>>> +
>>>>> +
>>>>> +def generate(schema_file: str,
>>>>> + output_dir: str,
>>>>> + prefix: str,
>>>>> + unmask: bool = False,
>>>>> + builtins: bool = False) -> None:
>>>>> + """
>>>>> + generate uses a given schema to produce C code in the target directory.
>>>>> +
>>>>> + :param schema_file: The primary QAPI schema file.
>>>>> + :param output_dir: The output directory to store generated code.
>>>>> + :param prefix: Optional C-code prefix for symbol names.
>>>>> + :param unmask: Expose non-ABI names through introspection?
>>>>> + :param builtins: Generate code for built-in types?
>>>>> +
>>>>> + :raise QAPIError: On failures.
>>>>> + """
>>>>> + match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix)
>>>>> + if match.end() != len(prefix):
>>>>> + msg = "funny character '{:s}' in prefix '{:s}'".format(
>>>>> + prefix[match.end()], prefix)
>>>>> + raise QAPIError('', None, msg)
>>>> Uh...
>>>> $ python3 scripts/qapi-gen.py --prefix=@ x
>>>> scripts/qapi-gen.py: : funny character '@' in prefix '@'
>>>> Unwanted " :".
>>>> This is due to a hack: you pass '' for info (*quack*). Everything
>>>> else
>>>> passes QAPISourceInfo (I believe).
>>>>
>>>
>>> Quack indeed - why does our base error class require so much
>>> information from a specific part of the generation process?
>> Because it's not "a base error class", it's a base error class for
>> the
>> QAPI schema compiler frontend.
>>
>
> Well. It's the base for every error we /had/.
You asked why the class has the init it has, and I answered :)
>>> Ah, someone changes this in part 4 so that we have a more generic
>>> error class to use as a base when we are missing such information.
>> Evolving it to satisfy a need for a more widely usable error class
>> is
>> okay.
>>
>
> Yep. It's helpful to keep a very generic form on which we grow other
> errors from, so that things like the entry point can be written
> legibly.
If you have a non-trivial error message format convention, you have a
use for a function formatting error messages.
If you have a separation between diagnose and report of errors, you you
have a use for a transport from diagnose to report. In Python, that's
raise.
The existing error message in main() has neither.
The existing error class QAPIError caters for the existing users.
>>> You are witnessing some more future-bleed.
>>>> Is it really a good idea to do this in generate? It's not about
>>>> generating code, it's about validating a CLI option.
>>>>
>>>
>>> One might also ask: Is it a good idea to only validate this on a
>>> frontend, and not in the implementation?
>> Yes, because that's where you can emit the better error message more
>> easily.
>> $ python3 scripts/qapi-gen.py --prefix=@ x
>> scripts/qapi-gen.py: 'funny character '@' in argument of --prefix
>> is better than
>> $ python3 scripts/qapi-gen.py --prefix=@ x
>> scripts/qapi-gen.py: funny character '@' in prefix '@'
>> In generate(), the knowledge where the offending prefix value comes
>> from
>> is no longer available.
>> To emit this error message, you'd have to raise a sufficiently
>> distinct
>> error in generate, catch it in main(), then put the error message
>> together somehow. Bah.
>> Aside: there's a stray ' in the old error message.
>>
>>> The idea here was to create a function that could be used in a script
>>> (for tests, debugging interfaces, other python packages) to do all of
>>> the same things that the CLI tool did, just sans the actual CLI.
>> YAGNI.
>>
>
> It's useful for testing and debugging to be able to just call it
> outside of the CLI, though. Maybe you won't use it, but I will.
For testing and debugging, treating "prefix is sane" as a precondition
is fine. I wouldn't even bother checking it. A check would catch
accidents, and these accidents seem vanishingly unlikely to me.
Evidence: we did without *any* prefix checking for *years*. I added it
in commit 1cf47a15f18 just for completeness.
> I could always add the prefix check into a tiny function and give the
> good error message in main(), and just assert in generate() if you
> insist on the slightly more specific error message from the CLI script.
If you genuinely think a check is needed there, that's the way to go.
>>> Wouldn't make sense to allow garbage to flow in from one interface but
>>> not the other; so the check is here.
>> "@prefix is sane" is a precondition of generate().
>> When there's a real risk of preconditions getting violated, or
>> readers
>> getting confused about preconditions, check them with assert.
[...]
next prev parent reply other threads:[~2020-10-08 5:57 UTC|newest]
Thread overview: 125+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-05 19:51 [PATCH v5 00/36] qapi: static typing conversion, pt1 John Snow
2020-10-05 19:51 ` [PATCH v5 01/36] docs: repair broken references John Snow
2020-10-05 19:51 ` [PATCH v5 02/36] qapi: modify docstrings to be sphinx-compatible John Snow
2020-10-06 11:21 ` Markus Armbruster
2020-10-06 15:23 ` John Snow
2020-10-07 7:24 ` Markus Armbruster
2020-10-07 17:00 ` John Snow
2020-10-05 19:51 ` [PATCH v5 03/36] qapi-gen: Separate arg-parsing from generation John Snow
2020-10-06 11:51 ` Markus Armbruster
2020-10-06 15:59 ` John Snow
2020-10-07 7:54 ` Markus Armbruster
2020-10-07 14:52 ` John Snow
2020-10-08 5:56 ` Markus Armbruster [this message]
2020-10-08 17:33 ` John Snow
2020-10-06 16:46 ` John Snow
2020-10-07 8:07 ` Markus Armbruster
2020-10-07 14:36 ` John Snow
2020-10-08 6:51 ` Markus Armbruster
2020-10-08 16:37 ` John Snow
2020-10-08 16:50 ` John Snow
2020-10-09 7:12 ` Markus Armbruster
2020-10-07 8:12 ` Markus Armbruster
2020-10-07 14:41 ` John Snow
2020-10-08 7:15 ` Markus Armbruster
2020-10-08 17:14 ` John Snow
2020-10-09 7:19 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 04/36] qapi: move generator entrypoint into module John Snow
2020-10-05 19:51 ` [PATCH v5 05/36] qapi: Prefer explicit relative imports John Snow
2020-10-06 11:33 ` Philippe Mathieu-Daudé
2020-10-05 19:51 ` [PATCH v5 06/36] qapi: Remove wildcard includes John Snow
2020-10-06 11:34 ` Philippe Mathieu-Daudé
2020-10-05 19:51 ` [PATCH v5 07/36] qapi: enforce import order/styling with isort John Snow
2020-10-07 8:15 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 08/36] qapi: delint using flake8 John Snow
2020-10-07 8:19 ` Markus Armbruster
2020-10-07 14:54 ` John Snow
2020-10-05 19:51 ` [PATCH v5 09/36] qapi: add pylintrc John Snow
2020-10-05 19:51 ` [PATCH v5 10/36] qapi/common.py: Remove python compatibility workaround John Snow
2020-10-05 19:51 ` [PATCH v5 11/36] qapi/common.py: Add indent manager John Snow
2020-10-07 8:50 ` Markus Armbruster
2020-10-07 18:08 ` John Snow
2020-10-07 18:18 ` Eduardo Habkost
2020-10-08 7:23 ` Markus Armbruster
2020-10-08 17:45 ` John Snow
2020-10-05 19:51 ` [PATCH v5 12/36] qapi/common.py: delint with pylint John Snow
2020-10-05 19:51 ` [PATCH v5 13/36] qapi/common.py: Replace one-letter 'c' variable John Snow
2020-10-06 11:35 ` Philippe Mathieu-Daudé
2020-10-05 19:51 ` [PATCH v5 14/36] qapi/common.py: check with pylint John Snow
2020-10-05 19:51 ` [PATCH v5 15/36] qapi/common.py: add type hint annotations John Snow
2020-10-07 9:03 ` Markus Armbruster
2020-10-07 15:01 ` John Snow
2020-10-05 19:51 ` [PATCH v5 16/36] qapi/common.py: Convert comments into docstrings, and elaborate John Snow
2020-10-07 9:14 ` Markus Armbruster
2020-10-07 15:23 ` John Snow
2020-10-08 7:20 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 17/36] qapi/common.py: move build_params into gen.py John Snow
2020-10-07 9:21 ` Markus Armbruster
2020-10-07 15:26 ` John Snow
2020-10-07 18:10 ` Eduardo Habkost
2020-10-05 19:51 ` [PATCH v5 18/36] qapi: establish mypy type-checking baseline John Snow
2020-10-07 9:25 ` Markus Armbruster
2020-10-07 15:33 ` John Snow
2020-10-08 7:29 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 19/36] qapi/events.py: add type hint annotations John Snow
2020-10-07 11:32 ` Markus Armbruster
2020-10-07 11:49 ` Markus Armbruster
2020-10-07 15:46 ` John Snow
2020-10-08 9:16 ` Markus Armbruster
2020-10-08 16:19 ` John Snow
2020-10-09 7:21 ` Markus Armbruster
2020-10-07 15:39 ` John Snow
2020-10-08 7:41 ` Markus Armbruster
2020-10-08 15:35 ` John Snow
2020-10-05 19:51 ` [PATCH v5 20/36] qapi/events.py: Move comments into docstrings John Snow
2020-10-05 19:51 ` [PATCH v5 21/36] qapi/commands.py: Don't re-bind to variable of different type John Snow
2020-10-07 11:34 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 22/36] qapi/commands.py: add type hint annotations John Snow
2020-10-05 19:51 ` [PATCH v5 23/36] qapi/commands.py: enable checking with mypy John Snow
2020-10-07 11:37 ` Markus Armbruster
2020-10-07 15:49 ` John Snow
2020-10-08 7:52 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 24/36] qapi/source.py: add type hint annotations John Snow
2020-10-07 11:55 ` Markus Armbruster
2020-10-07 16:04 ` John Snow
2020-10-08 8:42 ` Markus Armbruster
2020-10-09 14:30 ` John Snow
2020-10-09 14:37 ` John Snow
2020-10-05 19:51 ` [PATCH v5 25/36] qapi/source.py: delint with pylint John Snow
2020-10-05 19:51 ` [PATCH v5 26/36] qapi/gen.py: Fix edge-case of _is_user_module John Snow
2020-10-06 11:44 ` Philippe Mathieu-Daudé
2020-10-07 12:02 ` Markus Armbruster
2020-10-07 16:09 ` John Snow
2020-10-05 19:51 ` [PATCH v5 27/36] qapi/gen.py: add type hint annotations John Snow
2020-10-07 12:21 ` Markus Armbruster
2020-10-07 16:21 ` John Snow
2020-10-07 13:20 ` Markus Armbruster
2020-10-07 16:50 ` John Snow
2020-10-08 8:44 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 28/36] qapi/gen.py: Enable checking with mypy John Snow
2020-10-05 19:51 ` [PATCH v5 29/36] qapi/gen.py: Remove unused parameter John Snow
2020-10-07 12:22 ` Markus Armbruster
2020-10-07 16:23 ` John Snow
2020-10-05 19:51 ` [PATCH v5 30/36] qapi/gen.py: update write() to be more idiomatic John Snow
2020-10-07 12:32 ` Markus Armbruster
2020-10-07 16:25 ` John Snow
2020-10-05 19:51 ` [PATCH v5 31/36] qapi/gen.py: delint with pylint John Snow
2020-10-05 19:51 ` [PATCH v5 32/36] qapi/types.py: add type hint annotations John Snow
2020-10-05 19:51 ` [PATCH v5 33/36] qapi/types.py: remove one-letter variables John Snow
2020-10-07 12:42 ` Markus Armbruster
2020-10-07 16:31 ` John Snow
2020-10-05 19:51 ` [PATCH v5 34/36] qapi/visit.py: assert tag_member contains a QAPISchemaEnumType John Snow
2020-10-07 12:43 ` Markus Armbruster
2020-10-07 16:40 ` John Snow
2020-10-08 9:06 ` Markus Armbruster
2020-10-08 15:49 ` John Snow
2020-10-09 7:24 ` Markus Armbruster
2020-10-05 19:51 ` [PATCH v5 35/36] qapi/visit.py: remove unused parameters from gen_visit_object John Snow
2020-10-06 11:43 ` Philippe Mathieu-Daudé
2020-10-05 19:51 ` [PATCH v5 36/36] qapi/visit.py: add type hint annotations John Snow
2020-10-07 13:00 ` Markus Armbruster
2020-10-07 16:43 ` John Snow
2020-10-05 23:05 ` [PATCH v5 00/36] qapi: static typing conversion, pt1 Cleber Rosa
2020-10-05 23:57 ` John Snow
2020-10-06 17:51 ` Cleber Rosa
2020-10-07 13:05 ` Markus Armbruster
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87h7r545wj.fsf@dusky.pond.sub.org \
--to=armbru@redhat.com \
--cc=crosa@redhat.com \
--cc=ehabkost@redhat.com \
--cc=jsnow@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).