[Qemu-devel] [PATCH 0/5] qapi doc cleanups

[Qemu-devel] [PATCH 0/5] qapi doc cleanups

[Eric Blake]

Markus had some nice review comments on my earlier patch:
https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg05310.html

Depends on the full series "[PATCH v5 00/28] drop qapi nested structs",
although if I have to respin that, I'll probably just squash the
approved portions of this series directly into those patches rather
than respinning this series.  Broken into several patches, to try and
logically group the changes, and so that it is easier to pick and
choose whether to take all of the changes or drop some as overkill.

Eric Blake (5):
  qapi: Minor tweaks to qapi-code-gen.txt
  qapi: Clean up documentation of alternate mappings
  qapi: Make placeholders more obvious in doc usage statements
  qapi: Tweak doc references to QMP when QGA is also meant
  qapi: Fix QMP spec references to JSON

 docs/qapi-code-gen.txt | 226 +++++++++++++++++++++++++++----------------------
 docs/qmp/qmp-spec.txt  |  14 +--
 2 files changed, 135 insertions(+), 105 deletions(-)

-- 
2.1.0

[Qemu-devel] [PATCH 1/5] qapi: Minor tweaks to qapi-code-gen.txt

[Eric Blake]

Several minor things pointed out by Markus during review of my
sweeping changes.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 docs/qapi-code-gen.txt | 44 ++++++++++++++++++++++++++------------------
 1 file changed, 26 insertions(+), 18 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 8792b94..1f7d0ca 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -82,13 +82,14 @@ schema.  A type name should not be defined more than once.
 There are seven top-level expressions recognized by the parser:
 'include', 'command', 'type', 'enum', 'union', 'alternate', and
 'event'.  There are several built-in types, such as 'int' and 'str';
-additionally, the top-level expressions can define complex types,
-enumeration types, two flavors of union types, and an alternate type.
-The 'command' expression can refer to existing types by name, or list
-an anonymous type as a dictionary.  Listing a type name inside an
-array refers to a single-dimension array of that type; multi-dimension
-arrays are not directly supported (although an array of a complex
-struct that contains an array member is possible).
+additionally, the top-level expressions can define complex types
+(structs and two flavors of unions), as well as enumeration and
+alternate types.  The 'command' and 'event' expressions can refer to
+existing types by name, or list an anonymous type as a dictionary.
+Listing a type name inside an array refers to a single-dimension array
+of that type; multi-dimension arrays are not directly supported
+(although an array of a complex struct that contains an array member
+is possible).

 Types, commands, and events share a common namespace.  Therefore,
 generally speaking, type definitions should always use CamelCase for
@@ -117,6 +118,12 @@ underscore.  It is okay to reuse names that match C keywords; the
 generator will rename a field named "default" in the QAPI to
 "q_default" in the generated C code.

+In the rest of this document, usage lines are given for each
+expression type, written in the style of the QAPI syntax.  For
+example, a usage statement that includes '*base':'complex-type-name'
+means that an expression has an optional key 'base', which if present
+must have a value that forms a 'complex-type-name'.
+

 === Built-in Types ===

@@ -138,7 +145,7 @@ Usage: { 'include': 'str' }

 The QAPI schema definitions can be modularized using the 'include' directive:

- { 'include': 'path/to/file.json'}
+ { 'include': 'path/to/file.json' }

 The directive is evaluated recursively, and include paths are relative to the
 file using the directive. Multiple includes of the same file are
@@ -159,8 +166,8 @@ Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
 A complex type is a dictionary containing a single 'data' key whose
 value is a dictionary.  This corresponds to a struct in C or an Object
 in JSON. Each value of the 'data' dictionary must be the name of a
-complex, enum, union, alternate, or built-in type, or a one-element
-array containing a type name.  An example of a complex type is:
+type, or a one-element array containing a type name.  An example of a
+complex type is:

  { 'type': 'MyType',
    'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
@@ -280,8 +287,9 @@ implicit enum.
 A flat union definition specifies a complex type as its base, and
 avoids nesting on the wire.  In this case, the fields of the complex
 type are included as top-level fields of the union dictionary in the
-QMP wire format, and the 'discriminator' field must be the name of an
-enum-typed member of the base type. An example definition is:
+QMP wire format common to all branches of the union, and the
+'discriminator' field must be the name of an enum-typed member of the
+base type. An example definition is:

  { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
  { 'union': 'BlockdevOptions',
@@ -350,7 +358,7 @@ This example allows using both of the following example objects:
 === Commands ===

 Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
-         '*returns': 'type',
+         '*returns': 'dict-or-type-name',
          '*gen': false, '*success-response': false }

 Commands are defined by using a dictionary containing several members,
@@ -358,13 +366,13 @@ where three members are most common.  The 'command' member is a
 mandatory string, and determines the "execute" value passed in a QMP
 command exchange.

-The 'data' member is optional; if absent, the command accepts an
-optional empty dictionary.  If present, it must be the string name of
-a complex type, a one-element array containing the name of a complex
+The 'data' argument maps to the "arguments" dictionary passed in as
+part of a QMP command.  The 'data' member is optional and defaults to
+{} (an empty dictionary).  If present, it must be the string name of a
+complex type, a one-element array containing the name of a complex
 type, or a dictionary that declares an anonymous type with the same
 semantics as a 'type' expression, with one exception noted below when
-'gen' is used.  The 'data' argument maps to the "arguments" dictionary
-passed in as part of a QMP command.
+'gen' is used.

 The 'returns' member describes what will appear in the "return" field
 of a QMP reply on successful completion of a command.  The member is
-- 
2.1.0

[Qemu-devel] [PATCH 2/5] qapi: Clean up documentation of alternate mappings

[Eric Blake]

QObject is an internal coding concept and requires the reader to
reverse engineer the mapping; it is nicer to be explicit and call
out specific JSON types.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 docs/qapi-code-gen.txt | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 1f7d0ca..7fb0db7 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -331,17 +331,21 @@ Resulting in this JSON object:

 Usage: { 'alternate: 'str', 'data': 'dict' }

-An alternate type is one that allows a choice between two or more
-QObject data types (string, integer, number, or dictionary, but not
-array) on the wire.  The definition is similar to a simple union type,
-where each branch of the dictionary names a type, and where an
-implicit C enum NameKind is created for the alternate Name.  But
+An alternate type is one that allows a choice between two or more JSON
+data types on the wire.  The definition is similar to a simple union
+type, where each branch of the dictionary names a QAPI type, and where
+an implicit C enum NameKind is created for the alternate Name.  But
 unlike a union, the discriminator string is never passed on the wire
-for QMP, instead appearing only in the generated C code.  The type on
-the wire serves an implicit discriminator, which in turn means that an
-alternate can express a choice between a string and a single complex
-type (passed as a dictionary), but cannot distinguish between two
-different complex types.  For example:
+for QMP, instead appearing only in the generated C code.  Rather, the
+first byte of the value on the wire serves an implicit discriminator:
+if the branch is typed as the 'bool' built-in, it accepts true and
+false; if it is typed as any of the various numeric built-ins, it
+accepts a JSON number; if it is typed as a 'str' built-in or named
+enum types it accepts a JSON string, and if it is typed as a named
+struct or union type it accepts a JSON object.  Thus, an alternate can
+express a choice between a string and a single complex type (passed as
+a dictionary), but cannot distinguish between two different complex
+types or two different numeric built-in types.  For example:

  { 'alternate': 'BlockRef',
    'data': { 'definition': 'BlockdevOptions',
-- 
2.1.0

[Qemu-devel] [PATCH 3/5] qapi: Make placeholders more obvious in doc usage statements

[Eric Blake]

Using QAPI syntax to describe QAPI syntax is a bit meta; using
different typographic conventions makes it a bit easier to see
which strings are literal vs. placeholders.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 docs/qapi-code-gen.txt | 30 ++++++++++++++++--------------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 7fb0db7..c155f95 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -119,10 +119,12 @@ generator will rename a field named "default" in the QAPI to
 "q_default" in the generated C code.

 In the rest of this document, usage lines are given for each
-expression type, written in the style of the QAPI syntax.  For
-example, a usage statement that includes '*base':'complex-type-name'
-means that an expression has an optional key 'base', which if present
-must have a value that forms a 'complex-type-name'.
+expression type, with literal strings written in lower case and
+placeholders written in capitals.  If a literal string includes a
+prefix of '*', it can be omitted from that expression.  For example, a
+usage statement that includes '*base':COMPLEX-TYPE-NAME means that an
+expression has an optional key 'base', which if present must have a
+value that forms a complex type name.


 === Built-in Types ===
@@ -141,7 +143,7 @@ The following types are built-in to the parser:

 === Includes ===

-Usage: { 'include': 'str' }
+Usage: { 'include': STRING }

 The QAPI schema definitions can be modularized using the 'include' directive:

@@ -161,7 +163,7 @@ prevent incomplete include files.

 === Complex types ===

-Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
+Usage: { 'type': STRING, 'data': DICT, '*base': COMPLEX-TYPE-NAME }

 A complex type is a dictionary containing a single 'data' key whose
 value is a dictionary.  This corresponds to a struct in C or an Object
@@ -221,7 +223,7 @@ both fields like this:

 === Enumeration types ===

-Usage: { 'enum': 'str', 'data': [ 'str' ] }
+Usage: { 'enum': STRING, 'data': ARRAY-OF-STRING }

 An enumeration type is a dictionary containing a single 'data' key
 whose value is a list of strings.  An example enumeration is:
@@ -250,9 +252,9 @@ open-coding the field to be type 'str'.

 === Union types ===

-Usage: { 'union': 'str', 'data': 'dict' }
-or:    { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
-         'discriminator': 'enum-member-of-base' }
+Usage: { 'union': STRING, 'data': DICT }
+or:    { 'union': STRING, 'data': DICT, 'base': COMPLEX-TYPE-NAME,
+         'discriminator': ENUM-MEMBER-OF-BASE }

 Union types are used to let the user choose between several different
 data types.  There are two flavors: simple (no discriminator or base),
@@ -329,7 +331,7 @@ Resulting in this JSON object:

 === Alternate types ===

-Usage: { 'alternate: 'str', 'data': 'dict' }
+Usage: { 'alternate: STRING, 'data': DICT }

 An alternate type is one that allows a choice between two or more JSON
 data types on the wire.  The definition is similar to a simple union
@@ -361,8 +363,8 @@ This example allows using both of the following example objects:

 === Commands ===

-Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
-         '*returns': 'dict-or-type-name',
+Usage: { 'command': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT,
+         '*returns': TYPE-NAME-OR-DICT,
          '*gen': false, '*success-response': false }

 Commands are defined by using a dictionary containing several members,
@@ -439,7 +441,7 @@ qemu-guest-agent makes use of this field.

 === Events ===

-Usage: { 'event': 'str', '*data': 'dict-or-complex-type-name' }
+Usage: { 'event': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT }

 Events are defined with the keyword 'event'.  It is not allowed to
 name an event 'MAX', since the generator also produces a C enumeration
-- 
2.1.0

[Qemu-devel] [PATCH 4/5] qapi: Tweak doc references to QMP when QGA is also meant

[Eric Blake]

We have more than one qapi schema in use by more than one protocol.
Add a new term 'Client JSON Protocol' for use throughout the
document, to avoid confusion on whether something refers only to
QMP and not QGA.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 docs/qapi-code-gen.txt | 168 ++++++++++++++++++++++++++-----------------------
 1 file changed, 91 insertions(+), 77 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index c155f95..38cf1e9 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -5,7 +5,9 @@ This document is licensed under the GPLv2 (or later).
 QAPI is a native C API within QEMU which provides management-level
 functionality to internal and external users. For external
 users/processes, this interface is made available by a JSON-based
-QEMU Monitor protocol that is provided by the QMP server.
+wire format for the QEMU Monitor Protocol (QMP) for controlling
+qemu, as well as the QEMU Guest Agent (QGA) for communicating with the
+guest.

 To map QMP-defined interfaces to the native C QAPI implementations, a
 JSON-based schema is used to define types and function signatures, and
@@ -20,21 +22,28 @@ code are used.

 == QMP/Guest agent schema ==

-This file defines the types, commands, and events used by QMP.  It should
-fully describe the interface used by QMP.
+This file defines the types, commands, and events documented in a QAPI
+schema.  The QMP and QGA protocols should be fully described by their
+respective schema.  The remainder of this document uses "Client JSON
+Protocol" when referring to the wire contents of a QMP or QGA
+connection.

-A QAPI file is designed to be loosely based on JSON, which is then
-parsed by a python code generation program.  A valid QAPI schema
-consists of a list of top-level expressions, with no commas between
-them.  Where dictionaries are used, they are parsed as OrderedDicts so
-that ordering is preserved (for predictable layout of generated C
-structs and parameter lists).  Ordering doesn't matter for top-level
-expressions, but does matter within 'data' members of a single
-expression.  QAPI input is written using 'single quotes' instead of
-JSON's "double quotes" (in contrast, QMP is strict JSON and only uses
-"double quotes").  As in JSON, trailing commas are not permitted in
-arrays or dictionaries.  Input must be ASCII (although QMP supports
-full Unicode strings, the QAPI parser does not).
+A QAPI file is designed to be loosely based on JSON
+(http://www.ietf.org/rfc/rfc7159.txt), which is then parsed by a
+python code generation program.  A valid QAPI schema consists of a
+list of top-level expressions, all resembling JSON objects, with no
+commas between expressions.  Where dictionaries (JSON objects) are
+used, they are parsed as OrderedDicts so that ordering is preserved
+(for predictable layout of generated C structs and parameter lists).
+Ordering doesn't matter between top-level expressions or for the
+top-level keys within the expression, but does matter within 'data'
+members of a single expression.  QAPI input is written using 'single
+quotes' instead of JSON's "double quotes" (in contrast, client use of
+JSON Protocol is strict JSON and only uses "double quotes").  As in
+JSON, trailing commas are not permitted in arrays or dictionaries.
+Input must be ASCII (although Client JSON Protocol supports full
+Unicode strings, the QAPI parser does not).  At present, there is no
+place where a QAPI schema requires the use of JSON numbers or null.

 Comments are allowed; anything between an unquoted # and the following
 newline is ignored.  Although there is not yet a documentation
@@ -76,8 +85,11 @@ that will use those types.  Forward references are allowed: the parser
 scans in two passes, where the first pass learns all type names, and
 the second validates the schema and generates the code.  This allows
 the definition of complex structs that can have mutually recursive
-types, and allows for indefinite nesting of QMP that satisfies the
-schema.  A type name should not be defined more than once.
+types, and allows for indefinite nesting of Client JSON Protocol that
+satisfies the schema.  A type name should not be defined more than
+once.  It is permissible for the schema to contain additional types
+not used by any commands or events in the Client JSON Protocol, for
+the side effect of generated C code used internally.

 There are seven top-level expressions recognized by the parser:
 'include', 'command', 'type', 'enum', 'union', 'alternate', and
@@ -175,7 +187,7 @@ complex type is:
    'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }

 The use of '*' as a prefix to the name means the member is optional in
-the corresponding QMP usage.
+the corresponding JSON protocol usage.

 The default initialization value of an optional argument should not be changed
 between versions of QEMU unless the new default maintains backward
@@ -206,8 +218,8 @@ of use.

 A complex type definition can specify another complex type as its base.
 In this case, the fields of the base type are included as top-level fields
-of the new complex type's dictionary in the QMP wire format. An example
-definition is:
+of the new complex type's dictionary in the Client JSON Protocol wire
+format. An example definition is:

  { 'type': 'BlockdevOptionsGenericFormat', 'data': { 'file': 'str' } }
  { 'type': 'BlockdevOptionsGenericCOWFormat',
@@ -235,19 +247,19 @@ useful.  The list of strings should be lower case; if an enum name
 represents multiple words, use '-' between words.  The string 'max' is
 not allowed as an enum value, and values should not be repeated.

-The enumeration values are passed as strings over the QMP protocol,
-but are encoded as C enum integral values in generated code.  While
-the C code starts numbering at 0, it is better to use explicit
+The enumeration values are passed as strings over the Client JSON
+Protocol, but are encoded as C enum integral values in generated code.
+While the C code starts numbering at 0, it is better to use explicit
 comparisons to enum values than implicit comparisons to 0; the C code
 will also include a generated enum member ending in _MAX for tracking
 the size of the enum, useful when using common functions for
 converting between strings and enum values.  Since the wire format
 always passes by name, it is acceptable to reorder or add new
-enumeration members in any location without breaking QMP clients;
-however, removing enum values would break compatibility.  For any
-complex type that has a field that will only contain a finite set of
-string values, using an enum type for that field is better than
-open-coding the field to be type 'str'.
+enumeration members in any location without breaking clients of Client
+JSON Protocol; however, removing enum values would break
+compatibility.  For any complex type that has a field that will only
+contain a finite set of string values, using an enum type for that
+field is better than open-coding the field to be type 'str'.


 === Union types ===
@@ -273,7 +285,7 @@ values to data types like in this example:
    'data': { 'file': 'FileOptions',
              'qcow2': 'Qcow2Options' } }

-In the QMP wire format, a simple union is represented by a dictionary that
+In the Client JSON Protocol, a simple union is represented by a dictionary that
 contains the 'type' field as a discriminator, and a 'data' field that is of the
 specified data type corresponding to the discriminator value:

@@ -288,8 +300,8 @@ implicit enum.

 A flat union definition specifies a complex type as its base, and
 avoids nesting on the wire.  In this case, the fields of the complex
-type are included as top-level fields of the union dictionary in the
-QMP wire format common to all branches of the union, and the
+type are included as top-level fields of the union, common to all
+branches of the dictionary in the Client JSON Protocol, and the
 'discriminator' field must be the name of an enum-typed member of the
 base type. An example definition is:

@@ -337,17 +349,18 @@ An alternate type is one that allows a choice between two or more JSON
 data types on the wire.  The definition is similar to a simple union
 type, where each branch of the dictionary names a QAPI type, and where
 an implicit C enum NameKind is created for the alternate Name.  But
-unlike a union, the discriminator string is never passed on the wire
-for QMP, instead appearing only in the generated C code.  Rather, the
-first byte of the value on the wire serves an implicit discriminator:
-if the branch is typed as the 'bool' built-in, it accepts true and
-false; if it is typed as any of the various numeric built-ins, it
-accepts a JSON number; if it is typed as a 'str' built-in or named
-enum types it accepts a JSON string, and if it is typed as a named
-struct or union type it accepts a JSON object.  Thus, an alternate can
-express a choice between a string and a single complex type (passed as
-a dictionary), but cannot distinguish between two different complex
-types or two different numeric built-in types.  For example:
+unlike a union, the discriminator string is never passed in the Client
+JSON Protocol, instead appearing only in the generated C code.
+Rather, the first byte of the value on the wire serves an implicit
+discriminator: if the branch is typed as the 'bool' built-in, it
+accepts true and false; if it is typed as any of the various numeric
+built-ins, it accepts a JSON number; if it is typed as a 'str'
+built-in or named enum types it accepts a JSON string, and if it is
+typed as a named struct or union type it accepts a JSON object.  Thus,
+an alternate can express a choice between a string and a single
+complex type (passed as a dictionary), but cannot distinguish between
+two different complex types or two different numeric built-in types.
+For example:

  { 'alternate': 'BlockRef',
    'data': { 'definition': 'BlockdevOptions',
@@ -369,37 +382,37 @@ Usage: { 'command': STRING, '*data': COMPLEX-TYPE-NAME-OR-DICT,

 Commands are defined by using a dictionary containing several members,
 where three members are most common.  The 'command' member is a
-mandatory string, and determines the "execute" value passed in a QMP
-command exchange.
+mandatory string, and determines the "execute" value passed in a
+Client JSON Protocol command exchange.

 The 'data' argument maps to the "arguments" dictionary passed in as
-part of a QMP command.  The 'data' member is optional and defaults to
-{} (an empty dictionary).  If present, it must be the string name of a
-complex type, a one-element array containing the name of a complex
-type, or a dictionary that declares an anonymous type with the same
-semantics as a 'type' expression, with one exception noted below when
-'gen' is used.
+part of a Client JSON Protocol command.  The 'data' member is optional
+and defaults to {} (an empty dictionary).  If present, it must be the
+string name of a complex type, a one-element array containing the name
+of a complex type, or a dictionary that declares an anonymous type
+with the same semantics as a 'type' expression, with one exception
+noted below when 'gen' is used.

 The 'returns' member describes what will appear in the "return" field
-of a QMP reply on successful completion of a command.  The member is
-optional from the command declaration; if absent, the "return" field
-will be an empty dictionary.  If 'returns' is present, it must be the
-string name of a complex or built-in type, a one-element array
-containing the name of a complex or built-in type, or a dictionary
-that declares an anonymous type with the same semantics as a 'type'
-expression, with one exception noted below when 'gen' is used.
-Although it is permitted to have the 'returns' member name a built-in
-type or an array of built-in types, any command that does this cannot
-be extended to return additional information in the future; thus, new
-commands should strongly consider returning a dictionary-based type or
-an array of dictionaries, even if the dictionary only contains one
-field at the present.
+of a Client JSON Protocol reply on successful completion of a command.
+The member is optional from the command declaration; if absent, the
+"return" field will be an empty dictionary.  If 'returns' is present,
+it must be the string name of a complex or built-in type, a
+one-element array containing the name of a complex or built-in type,
+or a dictionary that declares an anonymous type with the same
+semantics as a 'type' expression, with one exception noted below when
+'gen' is used.  Although it is permitted to have the 'returns' member
+name a built-in type or an array of built-in types, any command that
+does this cannot be extended to return additional information in the
+future; thus, new commands should strongly consider returning a
+dictionary-based type or an array of dictionaries, even if the
+dictionary only contains one field at the present.

-All commands use a dictionary to report failure, with no way to
-specify that in QAPI.  Where the error return is different than the
-usual GenericError class in order to help the client react differently
-to certain error conditions, it is worth documenting this in the
-comments before the command declaration.
+All commands in Client JSON Protocol use a dictionary to report
+failure, with no way to specify that in QAPI.  Where the error return
+is different than the usual GenericError class in order to help the
+client react differently to certain error conditions, it is worth
+documenting this in the comments before the command declaration.

 Some example commands:

@@ -409,7 +422,7 @@ Some example commands:
  { 'command': 'my-second-command',
    'returns': [ 'MyType' ] }

-which would validate this QMP transaction:
+which would validate this Client JSON Protocol transaction:

  => { "execute": "my-first-command",
       "arguments": { "arg1": "hello" } }
@@ -418,12 +431,13 @@ which would validate this QMP transaction:
  <= { "return": [ { "value": "one" }, { } ] }

 In rare cases, QAPI cannot express a type-safe representation of a
-corresponding QMP command.  In these cases, if the command expression
-includes the key 'gen' with boolean value false, then the 'data' or
-'returns' member that intends to bypass generated type-safety and do
-its own manual validation should use '**' rather than a valid type
-name.  Please try to avoid adding new commands that rely on this, and
-instead use type-safe unions.  For an example of bypass usage:
+corresponding Client JSON Protocol command.  In these cases, if the
+command expression includes the key 'gen' with boolean value false,
+then the 'data' or 'returns' member that intends to bypass generated
+type-safety and do its own manual validation should use '**' rather
+than a valid type name.  Please try to avoid adding new commands that
+rely on this, and instead use type-safe unions.  For an example of
+bypass usage:

  { 'command': 'netdev_add',
    'data': {'type': 'str', 'id': 'str', '*props': '**'},
@@ -467,9 +481,9 @@ Resulting in this JSON object:

 Schemas are fed into 3 scripts to generate all the code/files that, paired
 with the core QAPI libraries, comprise everything required to take JSON
-commands read in by a QMP/guest agent server, unmarshal the arguments into
+commands read in by a Client JSON Protocol server, unmarshal the arguments into
 the underlying C types, call into the corresponding C function, and map the
-response back to a QMP/guest agent response to be returned to the user.
+response back to a Client JSON Protocol response to be returned to the user.

 As an example, we'll use the following schema, which describes a single
 complex user-defined type (which will produce a C struct, along with a list
-- 
2.1.0

[Qemu-devel] [PATCH 5/5] qapi: Fix QMP spec references to JSON

[Eric Blake]

I had assumed that 'id' of a command/return transaction could only
be an integer, and therefore that 'json-value' meant integer, so
I invented the term 'json-entity' to express a choice between
json-object, json-string, and json-array.  But in reality, the JSON
RFC defines json-value as the combination of all other types, and
the QMP parser currently accepts any JSON value that does not
include a literal JSON null.  So, clean up the docs to be consistent
with the RFC.  And while at it, use the newer version of the RFC.

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 docs/qmp/qmp-spec.txt | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/docs/qmp/qmp-spec.txt b/docs/qmp/qmp-spec.txt
index 1fb642b..f7d8c30 100644
--- a/docs/qmp/qmp-spec.txt
+++ b/docs/qmp/qmp-spec.txt
@@ -28,10 +28,10 @@ following format:

     json-DATA-STRUCTURE-NAME

-Where DATA-STRUCTURE-NAME is any valid JSON data structure, as defined by
-the JSON standard:
+Where DATA-STRUCTURE-NAME is any valid JSON data structure except the
+'null' literal, as defined by the JSON standard:

-http://www.ietf.org/rfc/rfc4627.txt
+http://www.ietf.org/rfc/rfc7159.txt

 For convenience, json-object members mentioned in this document will
 be in a certain order. However, in real protocol usage they can be in
@@ -92,7 +92,9 @@ The format for command execution is:
   execution of the command, it is optional when no arguments are required
 - The "id" member is a transaction identification associated with the
   command execution, it is optional and will be part of the response if
-  provided
+  provided. The "id" member can be any json-value that does not use a
+  json-null; however, most clients use a json-number incremented for
+  each successive command

 2.4 Commands Responses
 ----------------------
@@ -105,13 +107,13 @@ of a command execution: success or error.

 The format of a success response is:

-{ "return": json-entity, "id": json-value }
+{ "return": json-value, "id": json-value }

  Where,

 - The "return" member contains the data returned by the command, which
   is defined on a per-command basis (usually a json-object or
-  json-array of json-objects, but sometimes a json-value, json-string,
+  json-array of json-objects, but sometimes a json-number, json-string,
   or json-array of json-strings); it is an empty json-object if the
   command does not return data
 - The "id" member contains the transaction identification associated
-- 
2.1.0

Re: [Qemu-devel] [PATCH 2/5] qapi: Clean up documentation of alternate mappings

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> QObject is an internal coding concept and requires the reader to
> reverse engineer the mapping; it is nicer to be explicit and call
> out specific JSON types.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
>  docs/qapi-code-gen.txt | 24 ++++++++++++++----------
>  1 file changed, 14 insertions(+), 10 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 1f7d0ca..7fb0db7 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -331,17 +331,21 @@ Resulting in this JSON object:
>
>  Usage: { 'alternate: 'str', 'data': 'dict' }
>
> -An alternate type is one that allows a choice between two or more
> -QObject data types (string, integer, number, or dictionary, but not
> -array) on the wire.  The definition is similar to a simple union type,
> -where each branch of the dictionary names a type, and where an
> -implicit C enum NameKind is created for the alternate Name.  But
> +An alternate type is one that allows a choice between two or more JSON
> +data types on the wire.  The definition is similar to a simple union
> +type, where each branch of the dictionary names a QAPI type, and where
> +an implicit C enum NameKind is created for the alternate Name.  But
>  unlike a union, the discriminator string is never passed on the wire
> -for QMP, instead appearing only in the generated C code.  The type on
> -the wire serves an implicit discriminator, which in turn means that an
> -alternate can express a choice between a string and a single complex
> -type (passed as a dictionary), but cannot distinguish between two
> -different complex types.  For example:
> +for QMP, instead appearing only in the generated C code.  Rather, the
> +first byte of the value on the wire serves an implicit discriminator:

I think "first byte of" is implementation detail, and should be left
out.

> +if the branch is typed as the 'bool' built-in, it accepts true and
> +false; if it is typed as any of the various numeric built-ins, it
> +accepts a JSON number; if it is typed as a 'str' built-in or named
> +enum types it accepts a JSON string, and if it is typed as a named
> +struct or union type it accepts a JSON object.  Thus, an alternate can
> +express a choice between a string and a single complex type (passed as
> +a dictionary), but cannot distinguish between two different complex
> +types or two different numeric built-in types.  For example:
>
>   { 'alternate': 'BlockRef',
>     'data': { 'definition': 'BlockdevOptions',

Re: [Qemu-devel] [PATCH 0/5] qapi doc cleanups

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> Markus had some nice review comments on my earlier patch:
> https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg05310.html
>
> Depends on the full series "[PATCH v5 00/28] drop qapi nested structs",
> although if I have to respin that, I'll probably just squash the
> approved portions of this series directly into those patches rather
> than respinning this series.  Broken into several patches, to try and
> logically group the changes, and so that it is easier to pick and
> choose whether to take all of the changes or drop some as overkill.

With the small touch up to 2/5 I asked for:

Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH 2/5] qapi: Clean up documentation of alternate mappings

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 653 bytes --]

On 03/27/2015 05:02 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
> 
>> QObject is an internal coding concept and requires the reader to
>> reverse engineer the mapping; it is nicer to be explicit and call
>> out specific JSON types.
>>

>> +for QMP, instead appearing only in the generated C code.  Rather, the
>> +first byte of the value on the wire serves an implicit discriminator:
> 
> I think "first byte of" is implementation detail, and should be left
> out.

Sure; sounds better and shorter.


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 0/5] qapi doc cleanups

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> Markus had some nice review comments on my earlier patch:
> https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg05310.html
>
> Depends on the full series "[PATCH v5 00/28] drop qapi nested structs",
> although if I have to respin that, I'll probably just squash the
> approved portions of this series directly into those patches rather
> than respinning this series.  Broken into several patches, to try and
> logically group the changes, and so that it is easier to pick and
> choose whether to take all of the changes or drop some as overkill.

Anything left in here that hasn't been squashed into "[PATCH v8 00/40]
drop qapi nested structs"?

Re: [Qemu-devel] [PATCH 0/5] qapi doc cleanups

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

On 05/08/2015 08:29 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
> 
>> Markus had some nice review comments on my earlier patch:
>> https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg05310.html
>>
>> Depends on the full series "[PATCH v5 00/28] drop qapi nested structs",
>> although if I have to respin that, I'll probably just squash the
>> approved portions of this series directly into those patches rather
>> than respinning this series.  Broken into several patches, to try and
>> logically group the changes, and so that it is easier to pick and
>> choose whether to take all of the changes or drop some as overkill.
> 
> Anything left in here that hasn't been squashed into "[PATCH v8 00/40]
> drop qapi nested structs"?

Nope, this series has been subsumed.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]