All of lore.kernel.org
 help / color / mirror / Atom feed
From: Eric Blake <eblake@redhat.com>
To: Markus Armbruster <armbru@redhat.com>
Cc: kwolf@redhat.com, berto@igalia.com, qemu-devel@nongnu.org,
	mdroth@linux.vnet.ibm.com
Subject: Re: [Qemu-devel] [PATCH RFC v2 47/47] qapi-introspect: Hide type names
Date: Tue, 28 Jul 2015 15:26:39 -0600	[thread overview]
Message-ID: <55B7F38F.4080604@redhat.com> (raw)
In-Reply-To: <87mvygp1mo.fsf@blackfin.pond.sub.org>

[-- Attachment #1: Type: text/plain, Size: 3438 bytes --]

On 07/28/2015 12:39 PM, Markus Armbruster wrote:

> 
> Could do plain integer.  I guess I started down the base32 road to
> squeeze out a few more characters, then sabotaged myself by always using
> three base32 characters.
> 

> 
> In the introspection schema, every 'str' that's really a type name needs
> to be replaced by 'int'.

Sadly, our introspection union is based on 'name':'str',
'meta-type':'...' as the base members of the union; and that 'name'
would be one of the places where we'd want an integer; so we probably
have to stick to a string.  That said, using a string-ized integer may
be a bit more legible (for some reason, humans are better at reading
decimal numbers than base32 numbers), and since no valid command or
event starts with an integer, we could use the stringized integer
directly without having to use a colon (in fact, for the first 999 types
encountered, a stringized integer is shorter than the base32 compression
+ the leading colon for namespacing) :)

> 
> Should we later decide we don't want to hide type names after all, then
> backward compatibility will make it very hard to go back.

Indeed - changing 'str' to 'int' in the schema is a much stronger
commitment than leaving things 'str' but passing a stringized integer in
that string.

> 
> I wouldn't expect clients to find stuff with a linear search.  Use a
> dictionary.  Should be plenty fast enough for processing the schema.

Requires more memory in the client to create that hash table on the one
pass through the JSON - but as is often the case in computer science,
asking the client to trade space for time is not too onerous.
(Libvirt's current JSON parser does NOT create a hash table of the
dictionaries that it reads, so libvirt would actually have to do two
passes and/or update its use of libyajl into creating hash tables when
appropriate - but that should not be a driving factor in your design.)

> 
>>                     Or if type names are truly unimportant, then omit
>> names for type elements (by making name optional in the introspection
>> qapi description), and using ONLY offsets in the returned JSON array for
>> referring to types.  Of course, if we do this, life gets a lot trickier
>> for adding filtering down to a subset of the overall schema (unless you
>> don't mind populating lots of 'null' entries for parts that get filtered
>> out so that the parts that are displayed are always at the same array
>> offset, just with less overall output bulk due to the filtering).
> 
> Filtering is a headache I'd prefer to avoid.

Well, since I've had most of the ideas about how filtering could even be
done, I'm perfectly okay with you leaving the guts of filtering for me
(or someone else) to do as a followup series; and even then, I first
have to decide if it would help libvirt to have a filtered list.  What's
important for this series is that we haven't precluded the possibility
of adding filtering later, and I think we've succeeded at that.  And as
for using raw integers to represent offsets into the returned JSON
array, I think that is a bit too brittle; so I'm happy with forcing
clients to create their own dictionary/hash lookup of string type names,
even if the strings are munged to avoid leaking qapi types as non-ABI.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

  reply	other threads:[~2015-07-28 21:26 UTC|newest]

Thread overview: 199+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-07-01 20:21 [Qemu-devel] [PATCH RFC v2 00/47] qapi: QMP introspection Markus Armbruster
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 01/47] qapi: Clarify docs on including the same file multiple times Markus Armbruster
2015-07-20 15:17   ` Eric Blake
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 02/47] qapi: Clean up cgen() and mcgen() Markus Armbruster
2015-07-20 16:45   ` Eric Blake
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 03/47] qapi: Simplify guardname() Markus Armbruster
2015-07-20 17:32   ` Eric Blake
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 04/47] qapi-event: Clean up how name of enum QAPIEvent is made Markus Armbruster
2015-07-20 17:46   ` Eric Blake
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 05/47] qapi: Reject -p arguments that break qapi-event.py Markus Armbruster
2015-07-20 17:57   ` Eric Blake
2015-07-20 18:04     ` Eric Blake
2015-07-24 11:41     ` Markus Armbruster
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 06/47] qapi: Drop unused and useless parameters and variables Markus Armbruster
2015-07-20 21:14   ` Eric Blake
2015-07-24 11:44     ` Markus Armbruster
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 07/47] qapi: Generate a nicer struct for flat unions Markus Armbruster
2015-07-20 23:07   ` Eric Blake
2015-07-24 12:01     ` Markus Armbruster
2015-07-27 21:34       ` Eric Blake
2015-07-28  6:15         ` Markus Armbruster
2015-07-28 20:09   ` Eric Blake
2015-07-29  7:33     ` Markus Armbruster
2015-07-29 20:15       ` Eric Blake
2015-07-30  7:11         ` Markus Armbruster
2015-07-30 14:14           ` Eric Blake
2015-07-30 15:44             ` Markus Armbruster
2015-07-30 23:08               ` Eric Blake
2015-07-31 11:00         ` Markus Armbruster
2015-07-31  9:46       ` Markus Armbruster
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 08/47] qapi-visit: Fix generated code when schema has forward refs Markus Armbruster
2015-07-20 23:19   ` Eric Blake
2015-07-27  7:31     ` Markus Armbruster
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 09/47] qapi-visit: Replace list implicit_structs by set Markus Armbruster
2015-07-20 23:21   ` Eric Blake
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 10/47] qapi-visit: Fix two name arguments passed to visitors Markus Armbruster
2015-07-21  2:26   ` Eric Blake
2015-07-01 20:21 ` [Qemu-devel] [PATCH RFC v2 11/47] tests/qapi-schema: Document alternate's enum lacks visit function Markus Armbruster
2015-07-21  3:06   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 12/47] tests/qapi-schema: Document events with with base don't work Markus Armbruster
2015-07-21  3:08   ` Eric Blake
2015-07-30 22:33   ` [Qemu-devel] [RFC PATCH 12.5/47] qapi: Document that input visitor semantics are prone to leaks Eric Blake
2015-07-31  9:50     ` Markus Armbruster
2015-07-30 23:07   ` [Qemu-devel] [RFC PATCH 12.6/47] qapi: Document shortcoming with union 'data' branch Eric Blake
2015-07-31  9:50     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 13/47] tests/qapi-schema: Restore test case for flat union base bug Markus Armbruster
2015-07-21  3:19   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 14/47] qapi-tests: New tests for union, alternate command arguments Markus Armbruster
2015-07-21 12:43   ` Eric Blake
2015-07-23 14:59     ` Eric Blake
2015-07-27  7:50       ` Markus Armbruster
2015-07-27 13:06         ` Eric Blake
2015-07-31 13:15       ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 15/47] qapi: Fix to reject union " Markus Armbruster
2015-07-21 14:17   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 16/47] qapi-commands: Fix gen_err_check(e) for e and e != 'local_err' Markus Armbruster
2015-07-21 16:23   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 17/47] qapi-commands: Inline gen_marshal_output_call() Markus Armbruster
2015-07-21 16:41   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 18/47] qapi-commands: Don't feed output of mcgen() to mcgen() again Markus Armbruster
2015-07-21 17:20   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 19/47] qapi: Generated code cleanup Markus Armbruster
2015-07-21 17:43   ` Eric Blake
2015-07-27  8:07     ` Markus Armbruster
2015-08-04  9:08       ` Markus Armbruster
2015-08-04 12:31         ` Eric Blake
2015-08-04 14:35           ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 20/47] qapi: Rename class QAPISchema to QAPISchemaParser Markus Armbruster
2015-07-21 17:52   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 21/47] qapi: New QAPISchema intermediate reperesentation Markus Armbruster
2015-07-21 20:32   ` Eric Blake
2015-07-27  9:23     ` Markus Armbruster
2015-07-27 14:01       ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 22/47] qapi: QAPISchema code generation helper methods Markus Armbruster
2015-07-21 21:02   ` Eric Blake
2015-07-27  9:36     ` Markus Armbruster
2015-07-23 12:36   ` Eric Blake
2015-07-27  9:54     ` Markus Armbruster
2015-07-27 14:05       ` Eric Blake
2015-07-31 14:00         ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 23/47] qapi: New QAPISchemaVisitor Markus Armbruster
2015-07-21 21:59   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 24/47] tests/qapi-schema: Convert test harness to QAPISchemaVisitor Markus Armbruster
2015-07-21 22:23   ` Eric Blake
2015-07-27 14:03     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 25/47] qapi: Make generators work on sorted schema expressions Markus Armbruster
2015-07-21 22:50   ` Eric Blake
2015-07-27 14:19     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 26/47] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions Markus Armbruster
2015-07-22 17:34   ` Eric Blake
2015-07-22 20:07     ` Eric Blake
2015-07-27 15:59     ` Markus Armbruster
2015-07-22 21:21   ` Eric Blake
2015-07-22 22:56     ` Eric Blake
2015-07-27 16:09     ` Markus Armbruster
2015-07-27 16:25       ` Eric Blake
2015-07-28  6:16         ` Markus Armbruster
2015-07-29 23:11   ` Eric Blake
2015-07-30  6:42     ` Markus Armbruster
2015-07-30 12:46       ` Eric Blake
2015-07-30 15:53         ` Markus Armbruster
2015-07-30 16:36           ` Eric Blake
2015-07-30 21:51             ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 27/47] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs Markus Armbruster
2015-07-22 22:28   ` Eric Blake
2015-07-27 17:53     ` Markus Armbruster
2015-07-27 19:01       ` Eric Blake
2015-07-28  6:41         ` Markus Armbruster
2015-07-28 14:46           ` Eric Blake
2015-07-29  7:59             ` Markus Armbruster
2015-07-27 21:35   ` Eric Blake
2015-07-28  6:44     ` Markus Armbruster
2015-07-28 20:41       ` Eric Blake
2015-07-29  8:00         ` Markus Armbruster
2015-07-29 16:56           ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 28/47] qapi-commands: Convert to QAPISchemaVisitor Markus Armbruster
2015-07-22 23:05   ` Eric Blake
2015-07-27 18:08     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 29/47] qapi: Replace dirty is_c_ptr() by method c_null() Markus Armbruster
2015-07-22 23:22   ` Eric Blake
2015-07-28  7:34     ` Markus Armbruster
2015-07-28 14:53       ` Eric Blake
2015-07-29  8:32         ` Markus Armbruster
2015-07-29 15:41           ` Eric Blake
2015-07-29 17:22             ` Markus Armbruster
2015-07-30 14:19               ` Eric Blake
2015-07-30 15:57                 ` Markus Armbruster
2015-07-30 22:48                   ` Eric Blake
2015-07-31  7:43                     ` Markus Armbruster
2015-07-23 12:32   ` Eric Blake
2015-07-28  7:57     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 30/47] qapi: De-duplicate enum code generation Markus Armbruster
2015-07-23 12:46   ` Eric Blake
2015-07-28  8:13     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 31/47] qapi-event: Eliminate global variable event_enum_value Markus Armbruster
2015-07-23 14:31   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 32/47] qapi-event: Convert to QAPISchemaVisitor, fixing data with base Markus Armbruster
2015-07-23 15:14   ` Eric Blake
2015-07-28  8:32     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 33/47] qapi: Clean up after recent conversions to QAPISchemaVisitor Markus Armbruster
2015-07-23 16:48   ` Eric Blake
2015-07-28  9:18     ` Markus Armbruster
2015-07-28 21:13       ` Eric Blake
2015-07-28 21:37         ` Eric Blake
2015-07-29  8:33           ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 34/47] qapi-visit: Rearrange code a bit Markus Armbruster
2015-07-23 17:00   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 35/47] qapi-commands: Rearrange code Markus Armbruster
2015-07-23 17:41   ` Eric Blake
2015-07-28  9:18     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 36/47] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO() Markus Armbruster
2015-07-23 19:07   ` Eric Blake
2015-07-28  9:19     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 37/47] qapi: De-duplicate parameter list generation Markus Armbruster
2015-07-23 19:27   ` Eric Blake
2015-07-28 11:15     ` Markus Armbruster
2015-07-28 17:48       ` Eric Blake
2015-07-29  8:36         ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 38/47] qapi-commands: De-duplicate output marshaling functions Markus Armbruster
2015-07-23 19:47   ` Eric Blake
2015-07-28 11:20     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 39/47] qapi: Improve built-in type documentation Markus Armbruster
2015-07-23 21:29   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 40/47] qapi: Introduce a first class 'any' type Markus Armbruster
2015-07-23 22:04   ` Eric Blake
2015-07-28 11:31     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 41/47] qom: Don't use 'gen': false for qom-get, qom-set, object-add Markus Armbruster
2015-07-23 22:21   ` Eric Blake
2015-07-28 11:59     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 42/47] qapi-schema: Fix up misleading specification of netdev_add Markus Armbruster
2015-07-23 22:59   ` Eric Blake
2015-07-28 12:04     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 43/47] qmp: Improve netdev_add usage example in the manual Markus Armbruster
2015-07-23 23:01   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 44/47] qapi: Pseudo-type '**' is now unused, drop it Markus Armbruster
2015-07-23 23:20   ` Eric Blake
2015-07-28 12:24     ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 45/47] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
2015-07-24  3:29   ` Eric Blake
2015-07-28 14:33     ` Markus Armbruster
2015-07-28 19:11       ` Eric Blake
2015-07-29  9:19         ` Markus Armbruster
2015-07-29 15:56           ` Eric Blake
2015-07-29 17:26             ` Markus Armbruster
2015-08-03 15:15               ` Markus Armbruster
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 46/47] qapi-introspect: Map all integer types to 'int' Markus Armbruster
2015-07-24  3:33   ` Eric Blake
2015-07-01 20:22 ` [Qemu-devel] [PATCH RFC v2 47/47] qapi-introspect: Hide type names Markus Armbruster
2015-07-24  3:44   ` Eric Blake
2015-07-27 16:15     ` Eric Blake
2015-07-28 18:39       ` Markus Armbruster
2015-07-28 21:26         ` Eric Blake [this message]
2015-07-29  9:24           ` Markus Armbruster
2015-07-28 18:24     ` Markus Armbruster
2015-07-28 21:32       ` Eric Blake
2015-07-29  9:34         ` Markus Armbruster
2015-07-29 16:03           ` Eric Blake
2015-07-28 23:19   ` Eric Blake
2015-07-29  9:35     ` Markus Armbruster

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=55B7F38F.4080604@redhat.com \
    --to=eblake@redhat.com \
    --cc=armbru@redhat.com \
    --cc=berto@igalia.com \
    --cc=kwolf@redhat.com \
    --cc=mdroth@linux.vnet.ibm.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.