[Qemu-devel] [PATCH v6 00/27] qapi: add #if pre-processor conditions to generated code (part 2)

[Qemu-devel] [PATCH v6 00/27] qapi: add #if pre-processor conditions to generated code (part 2)

[Marc-André Lureau]

Hi,

This is the second part of the "add #if pre-processor conditions to
generated code" series, adding schema member conditions (roughly
16-38/49).

Members can be exploded as dictionnary with 'type'/'if' keys:

{ 'struct': 'TestIfStruct', 'data':
  { 'foo': 'int',
    'bar': { 'type': 'int', 'if': 'defined(TEST_IF_STRUCT_BAR)'} } }

Enum values can be exploded as dictionnary with 'type'/'if' keys:

{ 'enum': 'TestIfEnum', 'data':
  [ 'foo',
    { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ] }

v6:
- subset of v5 series: add schema member conditions
- split "qapi: factor out check_known_keys()", error report improvements
- add a patch to "qapi: include osdep.h in type headers" to avoid
  potential build configuration change issues
- rebased, on top of Markus's qapi-next branch (first 4 patches,
  included for patchew testing)

Marc-André Lureau (26):
  qmp-shell: learn to send commands with quoted arguments
  json: remove useless return value from lexer/parser
  tests: change /0.15/* tests to /qmp/*
  qapi: include osdep.h in type headers
  qapi: do not define enumeration value explicitly
  qapi: rename QAPISchemaEnumType.values to .members
  qapi: change enum visitor and gen_enum* to take QAPISchemaMember
  tests: print enum type members more like object type members
  qapi: factor out checking for keys
  qapi: improve reporting of unknown or missing keys
  qapi: add a dictionnary form with 'name' key for enum members
  qapi: add 'if' to enum members
  qapi-event: add 'if' condition to implicit event enum
  qapi: rename allow_dict to allow_implicit
  qapi: add a dictionary form with 'type' key for members
  qapi: add 'if' to implicit struct members
  qapi: add an error in case a discriminator is conditionnal
  qapi: add 'if' on union members
  qapi: add 'if' to alternate members
  qapi: add #if conditions to generated code members
  docs: document schema configuration
  qapi: add 'If:' condition to enum values documentation
  qapi: add 'If:' condition to struct members documentation
  qapi: add condition to variants documentation
  qapi: add more conditions to SPICE
  qapi: add conditions to REPLICATION type/commands on the schema

Markus Armbruster (1):
  qapi: Fix some pycodestyle-3 complaints

 qapi/block-core.json                          |  13 +-
 qapi/char.json                                |  16 +-
 qapi/migration.json                           |  12 +-
 scripts/qapi/common.py                        | 256 ++++++++++++------
 scripts/qapi/doc.py                           |  16 +-
 scripts/qapi/events.py                        |   2 +-
 scripts/qapi/introspect.py                    |  16 +-
 scripts/qapi/types.py                         |  11 +-
 scripts/qapi/visit.py                         |  10 +-
 include/qapi/qmp/json-lexer.h                 |   4 +-
 include/qapi/qmp/json-streamer.h              |   4 +-
 migration/colo.c                              |  16 +-
 monitor.c                                     |   5 -
 qobject/json-lexer.c                          |  23 +-
 qobject/json-streamer.c                       |   8 +-
 tests/test-qmp-cmds.c                         |  10 +-
 docs/devel/qapi-code-gen.txt                  |  19 ++
 scripts/qmp/qmp-shell                         |   3 +-
 tests/Makefile.include                        |   8 +-
 tests/qapi-schema/alternate-base.err          |   1 +
 tests/qapi-schema/alternate-invalid-dict.err  |   1 +
 ...ember.exit => alternate-invalid-dict.exit} |   0
 tests/qapi-schema/alternate-invalid-dict.json |   4 +
 ...-member.out => alternate-invalid-dict.out} |   0
 tests/qapi-schema/comments.out                |  14 +-
 tests/qapi-schema/doc-bad-section.out         |  13 +-
 tests/qapi-schema/doc-good.json               |  11 +-
 tests/qapi-schema/doc-good.out                |  22 +-
 tests/qapi-schema/doc-good.texi               |   8 +-
 tests/qapi-schema/double-type.err             |   1 +
 tests/qapi-schema/empty.out                   |   9 +-
 tests/qapi-schema/enum-bad-member.err         |   1 +
 tests/qapi-schema/enum-bad-member.exit        |   1 +
 tests/qapi-schema/enum-bad-member.json        |   2 +
 tests/qapi-schema/enum-bad-member.out         |   0
 .../qapi-schema/enum-dict-member-unknown.err  |   2 +
 .../qapi-schema/enum-dict-member-unknown.exit |   1 +
 .../qapi-schema/enum-dict-member-unknown.json |   2 +
 .../qapi-schema/enum-dict-member-unknown.out  |   0
 tests/qapi-schema/enum-dict-member.err        |   1 -
 tests/qapi-schema/enum-dict-member.json       |   2 -
 tests/qapi-schema/enum-if-invalid.err         |   1 +
 tests/qapi-schema/enum-if-invalid.exit        |   1 +
 tests/qapi-schema/enum-if-invalid.json        |   3 +
 tests/qapi-schema/enum-if-invalid.out         |   0
 tests/qapi-schema/enum-missing-data.err       |   2 +-
 tests/qapi-schema/event-case.out              |   9 +-
 tests/qapi-schema/event-nest-struct.err       |   2 +-
 tests/qapi-schema/flat-union-inline.err       |   2 +-
 .../flat-union-invalid-if-discriminator.err   |   1 +
 .../flat-union-invalid-if-discriminator.exit  |   1 +
 .../flat-union-invalid-if-discriminator.json  |  17 ++
 .../flat-union-invalid-if-discriminator.out   |   0
 tests/qapi-schema/ident-with-escape.out       |   9 +-
 tests/qapi-schema/include-relpath.out         |  14 +-
 tests/qapi-schema/include-repetition.out      |  14 +-
 tests/qapi-schema/include-simple.out          |  14 +-
 tests/qapi-schema/indented-expr.out           |   9 +-
 tests/qapi-schema/nested-struct-data.err      |   2 +-
 tests/qapi-schema/qapi-schema-test.json       |  38 ++-
 tests/qapi-schema/qapi-schema-test.out        |  76 +++++-
 .../struct-member-invalid-dict.err            |   1 +
 .../struct-member-invalid-dict.exit           |   1 +
 .../struct-member-invalid-dict.json           |   3 +
 .../struct-member-invalid-dict.out            |   0
 tests/qapi-schema/test-qapi.py                |  22 +-
 .../qapi-schema/union-branch-invalid-dict.err |   1 +
 .../union-branch-invalid-dict.exit            |   1 +
 .../union-branch-invalid-dict.json            |   4 +
 .../qapi-schema/union-branch-invalid-dict.out |   0
 tests/qapi-schema/unknown-expr-key.err        |   3 +-
 tests/qapi-schema/unknown-expr-key.json       |   2 +-
 72 files changed, 574 insertions(+), 227 deletions(-)
 create mode 100644 tests/qapi-schema/alternate-invalid-dict.err
 rename tests/qapi-schema/{enum-dict-member.exit => alternate-invalid-dict.exit} (100%)
 create mode 100644 tests/qapi-schema/alternate-invalid-dict.json
 rename tests/qapi-schema/{enum-dict-member.out => alternate-invalid-dict.out} (100%)
 create mode 100644 tests/qapi-schema/enum-bad-member.err
 create mode 100644 tests/qapi-schema/enum-bad-member.exit
 create mode 100644 tests/qapi-schema/enum-bad-member.json
 create mode 100644 tests/qapi-schema/enum-bad-member.out
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.err
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.exit
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.json
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.out
 delete mode 100644 tests/qapi-schema/enum-dict-member.err
 delete mode 100644 tests/qapi-schema/enum-dict-member.json
 create mode 100644 tests/qapi-schema/enum-if-invalid.err
 create mode 100644 tests/qapi-schema/enum-if-invalid.exit
 create mode 100644 tests/qapi-schema/enum-if-invalid.json
 create mode 100644 tests/qapi-schema/enum-if-invalid.out
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.err
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.exit
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.json
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.out
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.err
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.exit
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.json
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.out
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.err
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.exit
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.json
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.out

-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 01/27] qmp-shell: learn to send commands with quoted arguments

[Marc-André Lureau]

Use shlex to split the CLI command, respecting quoted arguments, and
also comments. This allows to call for ex:

(QEMU) human-monitor-command command-line="screendump /dev/null"
{"execute": "human-monitor-command", "arguments": {"command-line": "screendump /dev/null"}}

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180326150916.9602-3-marcandre.lureau@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 scripts/qmp/qmp-shell | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index a42306dd89..770140772d 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -74,6 +74,7 @@ import sys
 import os
 import errno
 import atexit
+import shlex
 
 class QMPCompleter(list):
     def complete(self, text, state):
@@ -219,7 +220,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
 
             < command-name > [ arg-name1=arg1 ] ... [ arg-nameN=argN ]
         """
-        cmdargs = cmdline.split()
+        cmdargs = shlex.split(cmdline)
 
         # Transactional CLI entry/exit:
         if cmdargs[0] == 'transaction(':
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 02/27] json: remove useless return value from lexer/parser

[Marc-André Lureau]

The lexer always returns 0 when char feeding. Furthermore, none of the
caller care about the return value.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180326150916.9602-10-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 include/qapi/qmp/json-lexer.h    |  4 ++--
 include/qapi/qmp/json-streamer.h |  4 ++--
 qobject/json-lexer.c             | 23 ++++++++---------------
 qobject/json-streamer.c          |  8 ++++----
 4 files changed, 16 insertions(+), 23 deletions(-)

diff --git a/include/qapi/qmp/json-lexer.h b/include/qapi/qmp/json-lexer.h
index afee7828cd..66ccf0357c 100644
--- a/include/qapi/qmp/json-lexer.h
+++ b/include/qapi/qmp/json-lexer.h
@@ -47,9 +47,9 @@ struct JSONLexer
 
 void json_lexer_init(JSONLexer *lexer, JSONLexerEmitter func);
 
-int json_lexer_feed(JSONLexer *lexer, const char *buffer, size_t size);
+void json_lexer_feed(JSONLexer *lexer, const char *buffer, size_t size);
 
-int json_lexer_flush(JSONLexer *lexer);
+void json_lexer_flush(JSONLexer *lexer);
 
 void json_lexer_destroy(JSONLexer *lexer);
 
diff --git a/include/qapi/qmp/json-streamer.h b/include/qapi/qmp/json-streamer.h
index 00d8a23af8..cb808cf27d 100644
--- a/include/qapi/qmp/json-streamer.h
+++ b/include/qapi/qmp/json-streamer.h
@@ -36,10 +36,10 @@ typedef struct JSONMessageParser
 void json_message_parser_init(JSONMessageParser *parser,
                               void (*func)(JSONMessageParser *, GQueue *));
 
-int json_message_parser_feed(JSONMessageParser *parser,
+void json_message_parser_feed(JSONMessageParser *parser,
                              const char *buffer, size_t size);
 
-int json_message_parser_flush(JSONMessageParser *parser);
+void json_message_parser_flush(JSONMessageParser *parser);
 
 void json_message_parser_destroy(JSONMessageParser *parser);
 
diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c
index 980ba159d6..df9b674322 100644
--- a/qobject/json-lexer.c
+++ b/qobject/json-lexer.c
@@ -290,7 +290,7 @@ void json_lexer_init(JSONLexer *lexer, JSONLexerEmitter func)
     lexer->x = lexer->y = 0;
 }
 
-static int json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush)
+static void json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush)
 {
     int char_consumed, new_state;
 
@@ -344,7 +344,7 @@ static int json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush)
             g_string_truncate(lexer->token, 0);
             new_state = IN_START;
             lexer->state = new_state;
-            return 0;
+            return;
         default:
             break;
         }
@@ -359,29 +359,22 @@ static int json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush)
         g_string_truncate(lexer->token, 0);
         lexer->state = IN_START;
     }
-
-    return 0;
 }
 
-int json_lexer_feed(JSONLexer *lexer, const char *buffer, size_t size)
+void json_lexer_feed(JSONLexer *lexer, const char *buffer, size_t size)
 {
     size_t i;
 
     for (i = 0; i < size; i++) {
-        int err;
-
-        err = json_lexer_feed_char(lexer, buffer[i], false);
-        if (err < 0) {
-            return err;
-        }
+        json_lexer_feed_char(lexer, buffer[i], false);
     }
-
-    return 0;
 }
 
-int json_lexer_flush(JSONLexer *lexer)
+void json_lexer_flush(JSONLexer *lexer)
 {
-    return lexer->state == IN_START ? 0 : json_lexer_feed_char(lexer, 0, true);
+    if (lexer->state != IN_START) {
+        json_lexer_feed_char(lexer, 0, true);
+    }
 }
 
 void json_lexer_destroy(JSONLexer *lexer)
diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c
index c51c2021f9..78dfff2aa0 100644
--- a/qobject/json-streamer.c
+++ b/qobject/json-streamer.c
@@ -118,15 +118,15 @@ void json_message_parser_init(JSONMessageParser *parser,
     json_lexer_init(&parser->lexer, json_message_process_token);
 }
 
-int json_message_parser_feed(JSONMessageParser *parser,
+void json_message_parser_feed(JSONMessageParser *parser,
                              const char *buffer, size_t size)
 {
-    return json_lexer_feed(&parser->lexer, buffer, size);
+    json_lexer_feed(&parser->lexer, buffer, size);
 }
 
-int json_message_parser_flush(JSONMessageParser *parser)
+void json_message_parser_flush(JSONMessageParser *parser)
 {
-    return json_lexer_flush(&parser->lexer);
+    json_lexer_flush(&parser->lexer);
 }
 
 void json_message_parser_destroy(JSONMessageParser *parser)
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 03/27] tests: change /0.15/* tests to /qmp/*

[Marc-André Lureau]

Presumably 0.15 was the version it was first introduced, but
qmp keeps evolving. There is no point in having that version
as test prefix, 'qmp' makes more sense here.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180326150916.9602-12-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 tests/test-qmp-cmds.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c
index ba41a6161e..ab414fa0c9 100644
--- a/tests/test-qmp-cmds.c
+++ b/tests/test-qmp-cmds.c
@@ -286,11 +286,11 @@ int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
 
-    g_test_add_func("/0.15/dispatch_cmd", test_dispatch_cmd);
-    g_test_add_func("/0.15/dispatch_cmd_failure", test_dispatch_cmd_failure);
-    g_test_add_func("/0.15/dispatch_cmd_io", test_dispatch_cmd_io);
-    g_test_add_func("/0.15/dealloc_types", test_dealloc_types);
-    g_test_add_func("/0.15/dealloc_partial", test_dealloc_partial);
+    g_test_add_func("/qmp/dispatch_cmd", test_dispatch_cmd);
+    g_test_add_func("/qmp/dispatch_cmd_failure", test_dispatch_cmd_failure);
+    g_test_add_func("/qmp/dispatch_cmd_io", test_dispatch_cmd_io);
+    g_test_add_func("/qmp/dealloc_types", test_dealloc_types);
+    g_test_add_func("/qmp/dealloc_partial", test_dealloc_partial);
 
     test_qmp_init_marshal(&qmp_commands);
     g_test_run();
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 04/27] qapi: Fix some pycodestyle-3 complaints

[Marc-André Lureau]

From: Markus Armbruster <armbru@redhat.com>

Fix the following issues:

    common.py:873:13: E129 visually indented line with same indent as next logical line
    common.py:1766:5: E741 ambiguous variable name 'l'
    common.py:1784:1: E305 expected 2 blank lines after class or function definition, found 1
    common.py:1833:1: E305 expected 2 blank lines after class or function definition, found 1
    common.py:1843:1: E305 expected 2 blank lines after class or function definition, found 1
    visit.py:181:18: E127 continuation line over-indented for visual indent

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180621083551.775-1-armbru@redhat.com>
[Fixup squashed in:]
Message-ID: <871sd0nzw9.fsf@dusky.pond.sub.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
---
 scripts/qapi/common.py         | 15 +++++++++------
 scripts/qapi/visit.py          |  2 +-
 tests/qapi-schema/test-qapi.py | 13 +++++++------
 3 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 9230a2a3e8..02c5c6767a 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -884,12 +884,12 @@ def check_keys(expr_elem, meta, required, optional=[]):
         if key not in required and key not in optional:
             raise QAPISemError(info, "Unknown key '%s' in %s '%s'"
                                % (key, meta, name))
-        if (key == 'gen' or key == 'success-response') and value is not False:
+        if key in ['gen', 'success-response'] and value is not False:
             raise QAPISemError(info,
                                "'%s' of %s '%s' should only use false value"
                                % (key, meta, name))
-        if (key == 'boxed' or key == 'allow-oob' or
-            key == 'allow-preconfig') and value is not True:
+        if (key in ['boxed', 'allow-oob', 'allow-preconfig']
+                and value is not True):
             raise QAPISemError(info,
                                "'%s' of %s '%s' should only use true value"
                                % (key, meta, name))
@@ -1845,12 +1845,12 @@ def camel_to_upper(value):
         return c_fun_str
 
     new_name = ''
-    l = len(c_fun_str)
-    for i in range(l):
+    length = len(c_fun_str)
+    for i in range(length):
         c = c_fun_str[i]
         # When c is upper and no '_' appears before, do more checks
         if c.isupper() and (i > 0) and c_fun_str[i - 1] != '_':
-            if i < l - 1 and c_fun_str[i + 1].islower():
+            if i < length - 1 and c_fun_str[i + 1].islower():
                 new_name += '_'
             elif c_fun_str[i - 1].isdigit():
                 new_name += '_'
@@ -1863,6 +1863,7 @@ def c_enum_const(type_name, const_name, prefix=None):
         type_name = prefix
     return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
 
+
 if hasattr(str, 'maketrans'):
     c_name_trans = str.maketrans('.-', '__')
 else:
@@ -1912,6 +1913,7 @@ def c_name(name, protect=True):
         return 'q_' + name
     return name
 
+
 eatspace = '\033EATSPACE.'
 pointer_suffix = ' *' + eatspace
 
@@ -1922,6 +1924,7 @@ def genindent(count):
         ret += ' '
     return ret
 
+
 indent_level = 0
 
 
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index dd5034a66a..460cf12989 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -187,7 +187,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
     }
     switch ((*obj)->type) {
 ''',
-                 c_name=c_name(name))
+                c_name=c_name(name))
 
     for var in variants.variants:
         ret += mcgen('''
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index f514fe71e4..cea21c773a 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -34,8 +34,8 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
         if base:
             print('    base %s' % base.name)
         for m in members:
-            print('    member %s: %s optional=%s' % \
-                  (m.name, m.type.name, m.optional))
+            print('    member %s: %s optional=%s'
+                  % (m.name, m.type.name, m.optional))
         self._print_variants(variants)
         self._print_if(ifcond)
 
@@ -46,10 +46,11 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
 
     def visit_command(self, name, info, ifcond, arg_type, ret_type, gen,
                       success_response, boxed, allow_oob, allow_preconfig):
-        print('command %s %s -> %s' % \
-              (name, arg_type and arg_type.name, ret_type and ret_type.name))
-        print('   gen=%s success_response=%s boxed=%s oob=%s preconfig=%s' % \
-              (gen, success_response, boxed, allow_oob, allow_preconfig))
+        print('command %s %s -> %s'
+              % (name, arg_type and arg_type.name,
+                 ret_type and ret_type.name))
+        print('   gen=%s success_response=%s boxed=%s oob=%s preconfig=%s'
+              % (gen, success_response, boxed, allow_oob, allow_preconfig))
         self._print_if(ifcond)
 
     def visit_event(self, name, info, ifcond, arg_type, boxed):
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 05/27] qapi: include osdep.h in type headers

[Marc-André Lureau]

Now that the schema can be configured, it is crucial that all types
are configured the same. Make sure config-host.h is included, so
build-sys tracks the dependency and rebuilds the types, by including
osdep.h first.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/types.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
index fd7808103c..91f87d0b8f 100644
--- a/scripts/qapi/types.py
+++ b/scripts/qapi/types.py
@@ -201,6 +201,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
 ''',
                                       types=types, visit=visit))
         self._genh.preamble_add(mcgen('''
+#include "qemu/osdep.h"
 #include "qapi/qapi-builtin-types.h"
 '''))
 
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 06/27] qapi: do not define enumeration value explicitly

[Marc-André Lureau]

The C standard has the initial value at 0 and the subsequent values
incremented by 1. No need to set this explicitely.

This will prevent from artificial "gaps" when compiling out some enum
values and having unnecessarily large MAX values & enums arrays, or
simplifying iterating over valid enum values.

Whenever config-host.h is changed, all the enum/types are recompiled.

(a subsequent patch will split the schema. Target-specific poisoined
conditionals will be added. They cannot be mixed with the common
schema: it is not possible to end up with enums of different values in
common and target builds)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 02c5c6767a..6f9498566e 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -2045,14 +2045,11 @@ typedef enum %(c_name)s {
 ''',
                 c_name=c_name(name))
 
-    i = 0
     for value in enum_values:
         ret += mcgen('''
-    %(c_enum)s = %(i)d,
+    %(c_enum)s,
 ''',
-                     c_enum=c_enum_const(name, value, prefix),
-                     i=i)
-        i += 1
+                     c_enum=c_enum_const(name, value, prefix))
 
     ret += mcgen('''
 } %(c_name)s;
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 07/27] qapi: rename QAPISchemaEnumType.values to .members

[Marc-André Lureau]

Rename QAPISchemaEnumType.values and related variables to members.
Makes sense ever since commit 93bda4dd4 changed .values from list of
string to list of QAPISchemaMember. Obvious no-op.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 6f9498566e..7020b88abc 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -1161,22 +1161,22 @@ class QAPISchemaBuiltinType(QAPISchemaType):
 
 
 class QAPISchemaEnumType(QAPISchemaType):
-    def __init__(self, name, info, doc, ifcond, values, prefix):
+    def __init__(self, name, info, doc, ifcond, members, prefix):
         QAPISchemaType.__init__(self, name, info, doc, ifcond)
-        for v in values:
-            assert isinstance(v, QAPISchemaMember)
-            v.set_owner(name)
+        for m in members:
+            assert isinstance(m, QAPISchemaMember)
+            m.set_owner(name)
         assert prefix is None or isinstance(prefix, str)
-        self.values = values
+        self.members = members
         self.prefix = prefix
 
     def check(self, schema):
         QAPISchemaType.check(self, schema)
         seen = {}
-        for v in self.values:
-            v.check_clash(self.info, seen)
+        for m in self.members:
+            m.check_clash(self.info, seen)
             if self.doc:
-                self.doc.connect_member(v)
+                self.doc.connect_member(m)
 
     def is_implicit(self):
         # See QAPISchema._make_implicit_enum_type() and ._def_predefineds()
@@ -1186,7 +1186,7 @@ class QAPISchemaEnumType(QAPISchemaType):
         return c_name(self.name)
 
     def member_names(self):
-        return [v.name for v in self.values]
+        return [m.name for m in self.members]
 
     def json_type(self):
         return 'string'
@@ -1403,9 +1403,9 @@ class QAPISchemaObjectTypeVariants(object):
         if self._tag_name:    # flat union
             # branches that are not explicitly covered get an empty type
             cases = set([v.name for v in self.variants])
-            for val in self.tag_member.type.values:
-                if val.name not in cases:
-                    v = QAPISchemaObjectTypeVariant(val.name, 'q_empty')
+            for m in self.tag_member.type.members:
+                if m.name not in cases:
+                    v = QAPISchemaObjectTypeVariant(m.name, 'q_empty')
                     v.set_owner(self.tag_member.owner)
                     self.variants.append(v)
         for v in self.variants:
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 08/27] qapi: change enum visitor and gen_enum* to take QAPISchemaMember

[Marc-André Lureau]

This will allow to add and access more properties associated with enum
values/members, like the associated 'if' condition. We may want to
have a specialized type QAPISchemaEnumMember, for now this will do.

While at it, also modify gen_enum() and gen_enum_lookup() for the
same reason.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py         | 22 +++++++++++-----------
 scripts/qapi/doc.py            |  2 +-
 scripts/qapi/events.py         |  2 +-
 scripts/qapi/introspect.py     |  5 +++--
 scripts/qapi/types.py          |  6 +++---
 scripts/qapi/visit.py          |  2 +-
 tests/qapi-schema/test-qapi.py |  4 ++--
 7 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 7020b88abc..a353670079 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -1063,7 +1063,7 @@ class QAPISchemaVisitor(object):
     def visit_builtin_type(self, name, info, json_type):
         pass
 
-    def visit_enum_type(self, name, info, ifcond, values, prefix):
+    def visit_enum_type(self, name, info, ifcond, members, prefix):
         pass
 
     def visit_array_type(self, name, info, ifcond, element_type):
@@ -1193,7 +1193,7 @@ class QAPISchemaEnumType(QAPISchemaType):
 
     def visit(self, visitor):
         visitor.visit_enum_type(self.name, self.info, self.ifcond,
-                                self.member_names(), self.prefix)
+                                self.members, self.prefix)
 
 
 class QAPISchemaArrayType(QAPISchemaType):
@@ -2012,19 +2012,19 @@ def _wrap_ifcond(ifcond, before, after):
     return out
 
 
-def gen_enum_lookup(name, values, prefix=None):
+def gen_enum_lookup(name, members, prefix=None):
     ret = mcgen('''
 
 const QEnumLookup %(c_name)s_lookup = {
     .array = (const char *const[]) {
 ''',
                 c_name=c_name(name))
-    for value in values:
-        index = c_enum_const(name, value, prefix)
+    for m in members:
+        index = c_enum_const(name, m.name, prefix)
         ret += mcgen('''
-        [%(index)s] = "%(value)s",
+        [%(index)s] = "%(name)s",
 ''',
-                     index=index, value=value)
+                     index=index, name=m.name)
 
     ret += mcgen('''
     },
@@ -2035,9 +2035,9 @@ const QEnumLookup %(c_name)s_lookup = {
     return ret
 
 
-def gen_enum(name, values, prefix=None):
+def gen_enum(name, members, prefix=None):
     # append automatically generated _MAX value
-    enum_values = values + ['_MAX']
+    enum_members = members + [QAPISchemaMember('_MAX')]
 
     ret = mcgen('''
 
@@ -2045,11 +2045,11 @@ typedef enum %(c_name)s {
 ''',
                 c_name=c_name(name))
 
-    for value in enum_values:
+    for m in enum_members:
         ret += mcgen('''
     %(c_enum)s,
 ''',
-                     c_enum=c_enum_const(name, value, prefix))
+                     c_enum=c_enum_const(name, m.name, prefix))
 
     ret += mcgen('''
 } %(c_name)s;
diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 987fd3c943..76cb186ff9 100755
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -206,7 +206,7 @@ class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor):
     def write(self, output_dir):
         self._gen.write(output_dir, self._prefix + 'qapi-doc.texi')
 
-    def visit_enum_type(self, name, info, ifcond, values, prefix):
+    def visit_enum_type(self, name, info, ifcond, members, prefix):
         doc = self.cur_doc
         self._gen.add(TYPE_FMT(type='Enum',
                                name=doc.symbol,
diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index 764ef177ab..ea4dac6a05 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -189,7 +189,7 @@ class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
             self._genh.add(gen_event_send_decl(name, arg_type, boxed))
             self._genc.add(gen_event_send(name, arg_type, boxed,
                                           self._enum_name))
-        self._event_names.append(name)
+        self._event_names.append(QAPISchemaMember(name))
 
 
 def gen_events(schema, output_dir, prefix):
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 71d4a779ce..3f1ca99f6d 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -160,8 +160,9 @@ const QLitObject %(c_name)s = %(c_string)s;
     def visit_builtin_type(self, name, info, json_type):
         self._gen_qlit(name, 'builtin', {'json-type': json_type}, [])
 
-    def visit_enum_type(self, name, info, ifcond, values, prefix):
-        self._gen_qlit(name, 'enum', {'values': values}, ifcond)
+    def visit_enum_type(self, name, info, ifcond, members, prefix):
+        self._gen_qlit(name, 'enum',
+                       {'values': [m.name for m in members]}, ifcond)
 
     def visit_array_type(self, name, info, ifcond, element_type):
         element = self._use_type(element_type)
diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
index 91f87d0b8f..2d4a70f810 100644
--- a/scripts/qapi/types.py
+++ b/scripts/qapi/types.py
@@ -213,10 +213,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
         self._genh.add(gen_type_cleanup_decl(name))
         self._genc.add(gen_type_cleanup(name))
 
-    def visit_enum_type(self, name, info, ifcond, values, prefix):
+    def visit_enum_type(self, name, info, ifcond, members, prefix):
         with ifcontext(ifcond, self._genh, self._genc):
-            self._genh.preamble_add(gen_enum(name, values, prefix))
-            self._genc.add(gen_enum_lookup(name, values, prefix))
+            self._genh.preamble_add(gen_enum(name, members, prefix))
+            self._genc.add(gen_enum_lookup(name, members, prefix))
 
     def visit_array_type(self, name, info, ifcond, element_type):
         with ifcontext(ifcond, self._genh, self._genc):
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index 460cf12989..24f85a2e85 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -310,7 +310,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaModularCVisitor):
 ''',
                                       types=types))
 
-    def visit_enum_type(self, name, info, ifcond, values, prefix):
+    def visit_enum_type(self, name, info, ifcond, members, prefix):
         with ifcontext(ifcond, self._genh, self._genc):
             self._genh.add(gen_visit_decl(name, scalar=True))
             self._genc.add(gen_visit_enum(name))
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index cea21c773a..27f776693e 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -23,8 +23,8 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
     def visit_include(self, name, info):
         print('include %s' % name)
 
-    def visit_enum_type(self, name, info, ifcond, values, prefix):
-        print('enum %s %s' % (name, values))
+    def visit_enum_type(self, name, info, ifcond, members, prefix):
+        print('enum %s %s' % (name, [m.name for m in members]))
         if prefix:
             print('    prefix %s' % prefix)
         self._print_if(ifcond)
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 09/27] tests: print enum type members more like object type members

[Marc-André Lureau]

Commit 93bda4dd461 changed the internal representation of enum type
members from str to QAPISchemaMember, but we still print only a
string.  Has been good enough, as the name is the member's only
attribute of interest, but that's about to change.  To prepare, print
them more like object type members.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 tests/qapi-schema/comments.out           | 14 ++++++-
 tests/qapi-schema/doc-bad-section.out    | 13 +++++-
 tests/qapi-schema/doc-good.out           | 17 ++++++--
 tests/qapi-schema/empty.out              |  9 ++++-
 tests/qapi-schema/event-case.out         |  9 ++++-
 tests/qapi-schema/ident-with-escape.out  |  9 ++++-
 tests/qapi-schema/include-relpath.out    | 14 ++++++-
 tests/qapi-schema/include-repetition.out | 14 ++++++-
 tests/qapi-schema/include-simple.out     | 14 ++++++-
 tests/qapi-schema/indented-expr.out      |  9 ++++-
 tests/qapi-schema/qapi-schema-test.out   | 50 +++++++++++++++++++-----
 tests/qapi-schema/test-qapi.py           |  4 +-
 12 files changed, 149 insertions(+), 27 deletions(-)

diff --git a/tests/qapi-schema/comments.out b/tests/qapi-schema/comments.out
index 8d2f1ce8a2..d1abc4b5a1 100644
--- a/tests/qapi-schema/comments.out
+++ b/tests/qapi-schema/comments.out
@@ -1,5 +1,15 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module comments.json
-enum Status ['good', 'bad', 'ugly']
+enum Status
+    member good
+    member bad
+    member ugly
diff --git a/tests/qapi-schema/doc-bad-section.out b/tests/qapi-schema/doc-bad-section.out
index cd28721568..db8014eed0 100644
--- a/tests/qapi-schema/doc-bad-section.out
+++ b/tests/qapi-schema/doc-bad-section.out
@@ -1,8 +1,17 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module doc-bad-section.json
-enum Enum ['one', 'two']
+enum Enum
+    member one
+    member two
 doc symbol=Enum
     body=
 == Produces *invalid* texinfo
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index 35f3f1164c..c2fc5c774a 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -1,8 +1,17 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module doc-good.json
-enum Enum ['one', 'two']
+enum Enum
+    member one
+    member two
     if ['defined(IFCOND)']
 object Base
     member base1: Enum optional=False
@@ -18,7 +27,9 @@ object q_obj_Variant1-wrapper
     member data: Variant1 optional=False
 object q_obj_Variant2-wrapper
     member data: Variant2 optional=False
-enum SugaredUnionKind ['one', 'two']
+enum SugaredUnionKind
+    member one
+    member two
 object SugaredUnion
     member type: SugaredUnionKind optional=False
     tag type
diff --git a/tests/qapi-schema/empty.out b/tests/qapi-schema/empty.out
index 0ec234eec4..5483cb7bc6 100644
--- a/tests/qapi-schema/empty.out
+++ b/tests/qapi-schema/empty.out
@@ -1,3 +1,10 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
index 88c0964917..f69d4ffe4e 100644
--- a/tests/qapi-schema/event-case.out
+++ b/tests/qapi-schema/event-case.out
@@ -1,6 +1,13 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module event-case.json
 event oops None
    boxed=False
diff --git a/tests/qapi-schema/ident-with-escape.out b/tests/qapi-schema/ident-with-escape.out
index 24c976f473..7f891f7e90 100644
--- a/tests/qapi-schema/ident-with-escape.out
+++ b/tests/qapi-schema/ident-with-escape.out
@@ -1,6 +1,13 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module ident-with-escape.json
 object q_obj_fooA-arg
     member bar1: str optional=False
diff --git a/tests/qapi-schema/include-relpath.out b/tests/qapi-schema/include-relpath.out
index ebbabd7a18..783ccfc855 100644
--- a/tests/qapi-schema/include-relpath.out
+++ b/tests/qapi-schema/include-relpath.out
@@ -1,9 +1,19 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module include-relpath.json
 include include/relpath.json
 module include/relpath.json
 include include-relpath-sub.json
 module include-relpath-sub.json
-enum Status ['good', 'bad', 'ugly']
+enum Status
+    member good
+    member bad
+    member ugly
diff --git a/tests/qapi-schema/include-repetition.out b/tests/qapi-schema/include-repetition.out
index 7235e055bc..d45977ee56 100644
--- a/tests/qapi-schema/include-repetition.out
+++ b/tests/qapi-schema/include-repetition.out
@@ -1,10 +1,20 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module include-repetition.json
 include comments.json
 module comments.json
-enum Status ['good', 'bad', 'ugly']
+enum Status
+    member good
+    member bad
+    member ugly
 module include-repetition.json
 include include-repetition-sub.json
 module include-repetition-sub.json
diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
index 006f723eeb..1afe20802a 100644
--- a/tests/qapi-schema/include-simple.out
+++ b/tests/qapi-schema/include-simple.out
@@ -1,7 +1,17 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module include-simple.json
 include include-simple-sub.json
 module include-simple-sub.json
-enum Status ['good', 'bad', 'ugly']
+enum Status
+    member good
+    member bad
+    member ugly
diff --git a/tests/qapi-schema/indented-expr.out b/tests/qapi-schema/indented-expr.out
index bd8a48630e..c0cf3243f3 100644
--- a/tests/qapi-schema/indented-expr.out
+++ b/tests/qapi-schema/indented-expr.out
@@ -1,6 +1,13 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module indented-expr.json
 command eins None -> None
    gen=True success_response=True boxed=False oob=False preconfig=False
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 0da92455da..edd22bc306 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -1,6 +1,13 @@
 object q_empty
-enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+enum QType
     prefix QTYPE
+    member none
+    member qnull
+    member qnum
+    member qstring
+    member qdict
+    member qlist
+    member qbool
 module qapi-schema-test.json
 object TestStruct
     member integer: int optional=False
@@ -11,19 +18,25 @@ object NestedEnumsOne
     member enum2: EnumOne optional=True
     member enum3: EnumOne optional=False
     member enum4: EnumOne optional=True
-enum MyEnum []
+enum MyEnum
 object Empty1
 object Empty2
     base Empty1
 command user_def_cmd0 Empty2 -> Empty2
    gen=True success_response=True boxed=False oob=False preconfig=False
-enum QEnumTwo ['value1', 'value2']
+enum QEnumTwo
     prefix QENUM_TWO
+    member value1
+    member value2
 object UserDefOne
     base UserDefZero
     member string: str optional=False
     member enum1: EnumOne optional=True
-enum EnumOne ['value1', 'value2', 'value3', 'value4']
+enum EnumOne
+    member value1
+    member value2
+    member value3
+    member value4
 object UserDefZero
     member integer: int optional=False
 object UserDefTwoDictDict
@@ -127,7 +140,21 @@ object q_obj_sizeList-wrapper
     member data: sizeList optional=False
 object q_obj_anyList-wrapper
     member data: anyList optional=False
-enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes', 'any']
+enum UserDefNativeListUnionKind
+    member integer
+    member s8
+    member s16
+    member s32
+    member s64
+    member u8
+    member u16
+    member u32
+    member u64
+    member number
+    member boolean
+    member string
+    member sizes
+    member any
 object UserDefNativeListUnion
     member type: UserDefNativeListUnionKind optional=False
     tag type
@@ -202,7 +229,8 @@ event EVENT_E UserDefZero
    boxed=True
 event EVENT_F UserDefAlternate
    boxed=True
-enum __org.qemu_x-Enum ['__org.qemu_x-value']
+enum __org.qemu_x-Enum
+    member __org.qemu_x-value
 object __org.qemu_x-Base
     member __org.qemu_x-member1: __org.qemu_x-Enum optional=False
 object __org.qemu_x-Struct
@@ -211,7 +239,8 @@ object __org.qemu_x-Struct
     member wchar-t: int optional=True
 object q_obj_str-wrapper
     member data: str optional=False
-enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch']
+enum __org.qemu_x-Union1Kind
+    member __org.qemu_x-branch
 object __org.qemu_x-Union1
     member type: __org.qemu_x-Union1Kind optional=False
     tag type
@@ -238,11 +267,14 @@ command __org.qemu_x-command q_obj___org.qemu_x-command-arg -> __org.qemu_x-Unio
 object TestIfStruct
     member foo: int optional=False
     if ['defined(TEST_IF_STRUCT)']
-enum TestIfEnum ['foo', 'bar']
+enum TestIfEnum
+    member foo
+    member bar
     if ['defined(TEST_IF_ENUM)']
 object q_obj_TestStruct-wrapper
     member data: TestStruct optional=False
-enum TestIfUnionKind ['foo']
+enum TestIfUnionKind
+    member foo
     if ['defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)']
 object TestIfUnion
     member type: TestIfUnionKind optional=False
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 27f776693e..641a18f06d 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -24,9 +24,11 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
         print('include %s' % name)
 
     def visit_enum_type(self, name, info, ifcond, members, prefix):
-        print('enum %s %s' % (name, [m.name for m in members]))
+        print('enum %s' % name)
         if prefix:
             print('    prefix %s' % prefix)
+        for m in members:
+            print('    member %s' % m.name)
         self._print_if(ifcond)
 
     def visit_object_type(self, name, info, ifcond, base, members, variants):
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 10/27] qapi: factor out checking for keys

[Marc-André Lureau]

Introduce a new helper function to check if the given keys are known,
and if mandatory keys are present. The function will be reused in
other places in the following code changes.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index a353670079..8313c478c4 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -873,6 +873,17 @@ def check_struct(expr, info):
                allow_metas=['struct'])
 
 
+def check_known_keys(info, source, keys, required, optional):
+    for key in keys:
+        if key not in required and key not in optional:
+            raise QAPISemError(info, "Unknown key '%s' in %s" % (key, source))
+
+    for key in required:
+        if key not in keys:
+            raise QAPISemError(info, "Key '%s' is missing from %s"
+                               % (key, source))
+
+
 def check_keys(expr_elem, meta, required, optional=[]):
     expr = expr_elem['expr']
     info = expr_elem['info']
@@ -880,10 +891,9 @@ def check_keys(expr_elem, meta, required, optional=[]):
     if not isinstance(name, str):
         raise QAPISemError(info, "'%s' key must have a string value" % meta)
     required = required + [meta]
+    source = "%s '%s'" % (meta, name)
+    check_known_keys(info, source, list(expr.keys()), required, optional)
     for (key, value) in expr.items():
-        if key not in required and key not in optional:
-            raise QAPISemError(info, "Unknown key '%s' in %s '%s'"
-                               % (key, meta, name))
         if key in ['gen', 'success-response'] and value is not False:
             raise QAPISemError(info,
                                "'%s' of %s '%s' should only use false value"
@@ -895,10 +905,6 @@ def check_keys(expr_elem, meta, required, optional=[]):
                                % (key, meta, name))
         if key == 'if':
             check_if(expr, info)
-    for key in required:
-        if key not in expr:
-            raise QAPISemError(info, "Key '%s' is missing from %s '%s'"
-                               % (key, meta, name))
 
 
 def check_exprs(exprs):
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 11/27] qapi: improve reporting of unknown or missing keys

[Marc-André Lureau]

Report the set of missing or unknown keys. And give a hint about the
accepted keys.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                  | 23 +++++++++++++++--------
 tests/qapi-schema/alternate-base.err    |  1 +
 tests/qapi-schema/double-type.err       |  1 +
 tests/qapi-schema/unknown-expr-key.err  |  3 ++-
 tests/qapi-schema/unknown-expr-key.json |  2 +-
 5 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 8313c478c4..cf8dab2866 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -874,14 +874,21 @@ def check_struct(expr, info):
 
 
 def check_known_keys(info, source, keys, required, optional):
-    for key in keys:
-        if key not in required and key not in optional:
-            raise QAPISemError(info, "Unknown key '%s' in %s" % (key, source))
-
-    for key in required:
-        if key not in keys:
-            raise QAPISemError(info, "Key '%s' is missing from %s"
-                               % (key, source))
+
+    def pprint(elems):
+        return ', '.join("'" + e + "'" for e in sorted(elems))
+
+    missing = set(required) - set(keys)
+    if missing:
+        raise QAPISemError(info, "Key%s %s %s missing from %s"
+                           % ('s' if len(missing) > 1 else '', pprint(missing),
+                              'are' if len(missing) > 1 else 'is', source))
+    allowed = set(required + optional)
+    unknown = set(keys) - allowed
+    if unknown:
+        raise QAPISemError(info, "Unknown key%s %s in %s\nValid keys are %s."
+                           % ('s' if len(unknown) > 1 else '', pprint(unknown),
+                              source, pprint(allowed)))
 
 
 def check_keys(expr_elem, meta, required, optional=[]):
diff --git a/tests/qapi-schema/alternate-base.err b/tests/qapi-schema/alternate-base.err
index 30d8a34373..ebe05bc898 100644
--- a/tests/qapi-schema/alternate-base.err
+++ b/tests/qapi-schema/alternate-base.err
@@ -1 +1,2 @@
 tests/qapi-schema/alternate-base.json:4: Unknown key 'base' in alternate 'Alt'
+Valid keys are 'alternate', 'data', 'if'.
diff --git a/tests/qapi-schema/double-type.err b/tests/qapi-schema/double-type.err
index f9613c6d6b..799193dba1 100644
--- a/tests/qapi-schema/double-type.err
+++ b/tests/qapi-schema/double-type.err
@@ -1 +1,2 @@
 tests/qapi-schema/double-type.json:2: Unknown key 'command' in struct 'bar'
+Valid keys are 'base', 'data', 'if', 'struct'.
diff --git a/tests/qapi-schema/unknown-expr-key.err b/tests/qapi-schema/unknown-expr-key.err
index 12f5ed5b43..83b83121e0 100644
--- a/tests/qapi-schema/unknown-expr-key.err
+++ b/tests/qapi-schema/unknown-expr-key.err
@@ -1 +1,2 @@
-tests/qapi-schema/unknown-expr-key.json:2: Unknown key 'bogus' in struct 'bar'
+tests/qapi-schema/unknown-expr-key.json:2: Unknown keys 'bogus', 'foo' in struct 'bar'
+Valid keys are 'base', 'data', 'if', 'struct'.
diff --git a/tests/qapi-schema/unknown-expr-key.json b/tests/qapi-schema/unknown-expr-key.json
index 3b2be00cc4..5bcb8efd1d 100644
--- a/tests/qapi-schema/unknown-expr-key.json
+++ b/tests/qapi-schema/unknown-expr-key.json
@@ -1,2 +1,2 @@
 # we reject an expression with unknown top-level keys
-{ 'struct': 'bar', 'data': { 'string': 'str'}, 'bogus': { } }
+{ 'struct': 'bar', 'data': { 'string': 'str'}, 'bogus': { }, 'foo': { } }
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 12/27] qapi: add a dictionnary form with 'name' key for enum members

[Marc-André Lureau]

Desugar the enum NAME form to { 'name': NAME }. This will allow to add
new enum members, such as 'if' in the following patch.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                        | 47 ++++++++++++++++---
 tests/Makefile.include                        |  3 +-
 tests/qapi-schema/enum-bad-member.err         |  1 +
 ...-dict-member.exit => enum-bad-member.exit} |  0
 tests/qapi-schema/enum-bad-member.json        |  2 +
 ...um-dict-member.out => enum-bad-member.out} |  0
 .../qapi-schema/enum-dict-member-unknown.err  |  2 +
 .../qapi-schema/enum-dict-member-unknown.exit |  1 +
 .../qapi-schema/enum-dict-member-unknown.json |  2 +
 .../qapi-schema/enum-dict-member-unknown.out  |  0
 tests/qapi-schema/enum-dict-member.err        |  1 -
 tests/qapi-schema/enum-dict-member.json       |  2 -
 tests/qapi-schema/enum-missing-data.err       |  2 +-
 13 files changed, 51 insertions(+), 12 deletions(-)
 create mode 100644 tests/qapi-schema/enum-bad-member.err
 rename tests/qapi-schema/{enum-dict-member.exit => enum-bad-member.exit} (100%)
 create mode 100644 tests/qapi-schema/enum-bad-member.json
 rename tests/qapi-schema/{enum-dict-member.out => enum-bad-member.out} (100%)
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.err
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.exit
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.json
 create mode 100644 tests/qapi-schema/enum-dict-member-unknown.out
 delete mode 100644 tests/qapi-schema/enum-dict-member.err
 delete mode 100644 tests/qapi-schema/enum-dict-member.json

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index cf8dab2866..e9fb736d46 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -740,6 +740,10 @@ def check_event(expr, info):
                allow_metas=meta)
 
 
+def enum_get_names(expr):
+    return [e['name'] for e in expr['data']]
+
+
 def check_union(expr, info):
     name = expr['union']
     base = expr.get('base')
@@ -799,7 +803,7 @@ def check_union(expr, info):
         # If the discriminator names an enum type, then all members
         # of 'data' must also be members of the enum type.
         if enum_define:
-            if key not in enum_define['data']:
+            if key not in enum_get_names(enum_define):
                 raise QAPISemError(info,
                                    "Discriminator value '%s' is not found in "
                                    "enum '%s'"
@@ -831,10 +835,10 @@ def check_alternate(expr, info):
         if qtype == 'QTYPE_QSTRING':
             enum_expr = enum_types.get(value)
             if enum_expr:
-                for v in enum_expr['data']:
+                for v in enum_get_names(enum_expr):
                     if v in ['on', 'off']:
                         conflicting.add('QTYPE_QBOOL')
-                    if re.match(r'[-+0-9.]', v): # lazy, could be tightened
+                    if re.match(r'[-+0-9.]', v):  # lazy, could be tightened
                         conflicting.add('QTYPE_QNUM')
             else:
                 conflicting.add('QTYPE_QNUM')
@@ -847,19 +851,34 @@ def check_alternate(expr, info):
             types_seen[qt] = key
 
 
-def check_enum(expr, info):
+def normalize_enum(expr, info):
     name = expr['enum']
     members = expr.get('data')
-    prefix = expr.get('prefix')
 
     if not isinstance(members, list):
         raise QAPISemError(info,
                            "Enum '%s' requires an array for 'data'" % name)
+
+    # translate short member form to dict form
+    for i, member in enumerate(members):
+        if not isinstance(member, dict):
+            member = {'name': member}
+            members[i] = member
+
+
+def check_enum(expr, info):
+    name = expr['enum']
+    members = expr.get('data')
+    prefix = expr.get('prefix')
+
     if prefix is not None and not isinstance(prefix, str):
         raise QAPISemError(info,
                            "Enum '%s' requires a string for 'prefix'" % name)
+
     for member in members:
-        check_name(info, "Member of enum '%s'" % name, member,
+        source = "dictionary member of enum '%s'" % name
+        check_known_keys(info, source, member, ['name'], [])
+        check_name(info, "Member of enum '%s'" % name, member['name'],
                    enum_member=True)
 
 
@@ -921,6 +940,13 @@ def check_exprs(exprs):
     for builtin in builtin_types.keys():
         all_names[builtin] = 'built-in'
 
+    # Normalize exprs
+    for expr_elem in exprs:
+        expr = expr_elem['expr']
+        info = expr_elem['info']
+        if 'enum' in expr:
+            normalize_enum(expr, info)
+
     # Learn the types and check for valid expression keys
     for expr_elem in exprs:
         expr = expr_elem['expr']
@@ -1640,7 +1666,14 @@ class QAPISchema(object):
                                             qtype_values, 'QTYPE'))
 
     def _make_enum_members(self, values):
-        return [QAPISchemaMember(v) for v in values]
+        enum = []
+        for v in values:
+            if isinstance(v, dict):
+                name = v['name']
+            else:
+                name = v
+            enum.append(QAPISchemaMember(name))
+        return enum
 
     def _make_implicit_enum_type(self, name, info, ifcond, values):
         # See also QAPISchemaObjectTypeMember._pretty_owner()
diff --git a/tests/Makefile.include b/tests/Makefile.include
index a49282704e..8e1b122cf2 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -482,10 +482,11 @@ qapi-schema += double-data.json
 qapi-schema += double-type.json
 qapi-schema += duplicate-key.json
 qapi-schema += empty.json
+qapi-schema += enum-bad-member.json
 qapi-schema += enum-bad-name.json
 qapi-schema += enum-bad-prefix.json
 qapi-schema += enum-clash-member.json
-qapi-schema += enum-dict-member.json
+qapi-schema += enum-dict-member-unknown.json
 qapi-schema += enum-int-member.json
 qapi-schema += enum-member-case.json
 qapi-schema += enum-missing-data.json
diff --git a/tests/qapi-schema/enum-bad-member.err b/tests/qapi-schema/enum-bad-member.err
new file mode 100644
index 0000000000..211db9e6fc
--- /dev/null
+++ b/tests/qapi-schema/enum-bad-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-bad-member.json:2: Member of enum 'MyEnum' requires a string name
diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-bad-member.exit
similarity index 100%
rename from tests/qapi-schema/enum-dict-member.exit
rename to tests/qapi-schema/enum-bad-member.exit
diff --git a/tests/qapi-schema/enum-bad-member.json b/tests/qapi-schema/enum-bad-member.json
new file mode 100644
index 0000000000..98da6828b4
--- /dev/null
+++ b/tests/qapi-schema/enum-bad-member.json
@@ -0,0 +1,2 @@
+# we reject any enum member that is not a string
+{ 'enum': 'MyEnum', 'data': [ [ ] ] }
diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-bad-member.out
similarity index 100%
rename from tests/qapi-schema/enum-dict-member.out
rename to tests/qapi-schema/enum-bad-member.out
diff --git a/tests/qapi-schema/enum-dict-member-unknown.err b/tests/qapi-schema/enum-dict-member-unknown.err
new file mode 100644
index 0000000000..76bd0471db
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member-unknown.err
@@ -0,0 +1,2 @@
+tests/qapi-schema/enum-dict-member-unknown.json:2: Unknown key 'bad-key' in dictionary member of enum 'MyEnum'
+Valid keys are 'name'.
diff --git a/tests/qapi-schema/enum-dict-member-unknown.exit b/tests/qapi-schema/enum-dict-member-unknown.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member-unknown.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/enum-dict-member-unknown.json b/tests/qapi-schema/enum-dict-member-unknown.json
new file mode 100644
index 0000000000..6664c59201
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member-unknown.json
@@ -0,0 +1,2 @@
+# we reject any enum member that is not a string or a dict with 'name'
+{ 'enum': 'MyEnum', 'data': [ { 'name': 'foo', 'bad-key': 'str' } ] }
diff --git a/tests/qapi-schema/enum-dict-member-unknown.out b/tests/qapi-schema/enum-dict-member-unknown.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
deleted file mode 100644
index 8ca146ea59..0000000000
--- a/tests/qapi-schema/enum-dict-member.err
+++ /dev/null
@@ -1 +0,0 @@
-tests/qapi-schema/enum-dict-member.json:2: Member of enum 'MyEnum' requires a string name
diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
deleted file mode 100644
index 79672e0f09..0000000000
--- a/tests/qapi-schema/enum-dict-member.json
+++ /dev/null
@@ -1,2 +0,0 @@
-# we reject any enum member that is not a string
-{ 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
index ba4873ae69..b8ccae071b 100644
--- a/tests/qapi-schema/enum-missing-data.err
+++ b/tests/qapi-schema/enum-missing-data.err
@@ -1 +1 @@
-tests/qapi-schema/enum-missing-data.json:2: Key 'data' is missing from enum 'MyEnum'
+tests/qapi-schema/enum-missing-data.json:2: Enum 'MyEnum' requires an array for 'data'
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 13/27] qapi: add 'if' to enum members

[Marc-André Lureau]

QAPISchemaMember gains .ifcond for enum members: inherited classes,
such as QAPISchemaObjectTypeMember, will thus have an ifcond member
after this (those different types will also use the .ifcond to store
the condition and generate conditional code in the following patches).

Generated code is not changed by this patch, but with "qapi: add #if
conditions to generated code" patch.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                         | 10 +++++++---
 tests/Makefile.include                         |  1 +
 tests/qapi-schema/enum-dict-member-unknown.err |  2 +-
 tests/qapi-schema/enum-if-invalid.err          |  1 +
 tests/qapi-schema/enum-if-invalid.exit         |  1 +
 tests/qapi-schema/enum-if-invalid.json         |  3 +++
 tests/qapi-schema/enum-if-invalid.out          |  0
 tests/qapi-schema/qapi-schema-test.json        |  5 +++--
 tests/qapi-schema/qapi-schema-test.out         |  2 ++
 tests/qapi-schema/test-qapi.py                 |  1 +
 10 files changed, 20 insertions(+), 6 deletions(-)
 create mode 100644 tests/qapi-schema/enum-if-invalid.err
 create mode 100644 tests/qapi-schema/enum-if-invalid.exit
 create mode 100644 tests/qapi-schema/enum-if-invalid.json
 create mode 100644 tests/qapi-schema/enum-if-invalid.out

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index e9fb736d46..95b7cd74ee 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -877,7 +877,8 @@ def check_enum(expr, info):
 
     for member in members:
         source = "dictionary member of enum '%s'" % name
-        check_known_keys(info, source, member, ['name'], [])
+        check_known_keys(info, source, member, ['name'], ['if'])
+        check_if(member, info)
         check_name(info, "Member of enum '%s'" % name, member['name'],
                    enum_member=True)
 
@@ -1357,9 +1358,10 @@ class QAPISchemaObjectType(QAPISchemaType):
 class QAPISchemaMember(object):
     role = 'member'
 
-    def __init__(self, name):
+    def __init__(self, name, ifcond=None):
         assert isinstance(name, str)
         self.name = name
+        self.ifcond = listify_cond(ifcond)
         self.owner = None
 
     def set_owner(self, name):
@@ -1670,9 +1672,11 @@ class QAPISchema(object):
         for v in values:
             if isinstance(v, dict):
                 name = v['name']
+                ifcond = v.get('if')
             else:
                 name = v
-            enum.append(QAPISchemaMember(name))
+                ifcond = None
+            enum.append(QAPISchemaMember(name, ifcond))
         return enum
 
     def _make_implicit_enum_type(self, name, info, ifcond, values):
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8e1b122cf2..3ba9097892 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -487,6 +487,7 @@ qapi-schema += enum-bad-name.json
 qapi-schema += enum-bad-prefix.json
 qapi-schema += enum-clash-member.json
 qapi-schema += enum-dict-member-unknown.json
+qapi-schema += enum-if-invalid.json
 qapi-schema += enum-int-member.json
 qapi-schema += enum-member-case.json
 qapi-schema += enum-missing-data.json
diff --git a/tests/qapi-schema/enum-dict-member-unknown.err b/tests/qapi-schema/enum-dict-member-unknown.err
index 76bd0471db..2aae618be0 100644
--- a/tests/qapi-schema/enum-dict-member-unknown.err
+++ b/tests/qapi-schema/enum-dict-member-unknown.err
@@ -1,2 +1,2 @@
 tests/qapi-schema/enum-dict-member-unknown.json:2: Unknown key 'bad-key' in dictionary member of enum 'MyEnum'
-Valid keys are 'name'.
+Valid keys are 'if', 'name'.
diff --git a/tests/qapi-schema/enum-if-invalid.err b/tests/qapi-schema/enum-if-invalid.err
new file mode 100644
index 0000000000..54c3cf887b
--- /dev/null
+++ b/tests/qapi-schema/enum-if-invalid.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings
diff --git a/tests/qapi-schema/enum-if-invalid.exit b/tests/qapi-schema/enum-if-invalid.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/enum-if-invalid.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/enum-if-invalid.json b/tests/qapi-schema/enum-if-invalid.json
new file mode 100644
index 0000000000..60bd0ef1d7
--- /dev/null
+++ b/tests/qapi-schema/enum-if-invalid.json
@@ -0,0 +1,3 @@
+# check invalid 'if' type
+{ 'enum': 'TestIfEnum', 'data':
+  [ 'foo', { 'name' : 'bar', 'if': { 'val': 'foo' } } ] }
diff --git a/tests/qapi-schema/enum-if-invalid.out b/tests/qapi-schema/enum-if-invalid.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 11aa4c8f8d..35ca94d991 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -202,7 +202,8 @@
 { 'struct': 'TestIfStruct', 'data': { 'foo': 'int' },
   'if': 'defined(TEST_IF_STRUCT)' }
 
-{ 'enum': 'TestIfEnum', 'data': [ 'foo', 'bar' ],
+{ 'enum': 'TestIfEnum', 'data':
+  [ 'foo', { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ],
   'if': 'defined(TEST_IF_ENUM)' }
 
 { 'union': 'TestIfUnion', 'data': { 'foo': 'TestStruct' },
@@ -211,7 +212,7 @@
 { 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
   'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
 
-{ 'command': 'TestIfCmd', 'data': { 'foo': 'TestIfStruct' },
+{ 'command': 'TestIfCmd', 'data': { 'foo': 'TestIfStruct', 'bar': 'TestIfEnum' },
   'returns': 'UserDefThree',
   'if': ['defined(TEST_IF_CMD)', 'defined(TEST_IF_STRUCT)'] }
 
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index edd22bc306..1fb33f302c 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -270,6 +270,7 @@ object TestIfStruct
 enum TestIfEnum
     member foo
     member bar
+        if ['defined(TEST_IF_ENUM_BAR)']
     if ['defined(TEST_IF_ENUM)']
 object q_obj_TestStruct-wrapper
     member data: TestStruct optional=False
@@ -288,6 +289,7 @@ alternate TestIfAlternate
     if ['defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)']
 object q_obj_TestIfCmd-arg
     member foo: TestIfStruct optional=False
+    member bar: TestIfEnum optional=False
     if ['defined(TEST_IF_CMD)', 'defined(TEST_IF_STRUCT)']
 command TestIfCmd q_obj_TestIfCmd-arg -> UserDefThree
    gen=True success_response=True boxed=False oob=False preconfig=False
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 641a18f06d..b2f4ce6134 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -29,6 +29,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
             print('    prefix %s' % prefix)
         for m in members:
             print('    member %s' % m.name)
+            self._print_if(m.ifcond, 8)
         self._print_if(ifcond)
 
     def visit_object_type(self, name, info, ifcond, base, members, variants):
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 14/27] qapi-event: add 'if' condition to implicit event enum

[Marc-André Lureau]

Add condition to QAPIEvent enum members based on the event 'if'.

Generated code is not changed by this patch, but with "qapi: add #if
conditions to generated code" patch.

There is no coverage of this change in qapi-schema-test.out since the
event_names enum is an implicit type created by qapi-event.py.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/events.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
index ea4dac6a05..c9beef8b4b 100644
--- a/scripts/qapi/events.py
+++ b/scripts/qapi/events.py
@@ -189,7 +189,7 @@ class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
             self._genh.add(gen_event_send_decl(name, arg_type, boxed))
             self._genc.add(gen_event_send(name, arg_type, boxed,
                                           self._enum_name))
-        self._event_names.append(QAPISchemaMember(name))
+        self._event_names.append(QAPISchemaMember(name, ifcond))
 
 
 def gen_events(schema, output_dir, prefix):
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 15/27] qapi: rename allow_dict to allow_implicit

[Marc-André Lureau]

This makes it a bit clearer what is the intent of the dictionnary for
the check_type() function, since there was some confusion on a
previous iteration of this series.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 95b7cd74ee..d83fa1900e 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -661,7 +661,7 @@ def check_if(expr, info):
 
 
 def check_type(info, source, value, allow_array=False,
-               allow_dict=False, allow_optional=False,
+               allow_implicit=False, allow_optional=False,
                allow_metas=[]):
     global all_names
 
@@ -688,7 +688,7 @@ def check_type(info, source, value, allow_array=False,
                                (source, all_names[value], value))
         return
 
-    if not allow_dict:
+    if not allow_implicit:
         raise QAPISemError(info, "%s should be a type name" % source)
 
     if not isinstance(value, OrderedDict):
@@ -718,7 +718,7 @@ def check_command(expr, info):
     if boxed:
         args_meta += ['union', 'alternate']
     check_type(info, "'data' for command '%s'" % name,
-               expr.get('data'), allow_dict=not boxed, allow_optional=True,
+               expr.get('data'), allow_implicit=not boxed, allow_optional=True,
                allow_metas=args_meta)
     returns_meta = ['union', 'struct']
     if name in returns_whitelist:
@@ -736,7 +736,7 @@ def check_event(expr, info):
     if boxed:
         meta += ['union', 'alternate']
     check_type(info, "'data' for event '%s'" % name,
-               expr.get('data'), allow_dict=not boxed, allow_optional=True,
+               expr.get('data'), allow_implicit=not boxed, allow_optional=True,
                allow_metas=meta)
 
 
@@ -764,7 +764,7 @@ def check_union(expr, info):
     else:
         # The object must have a string or dictionary 'base'.
         check_type(info, "'base' for union '%s'" % name,
-                   base, allow_dict=True, allow_optional=True,
+                   base, allow_implicit=True, allow_optional=True,
                    allow_metas=['struct'])
         if not base:
             raise QAPISemError(info, "Flat union '%s' must have a base"
@@ -888,7 +888,7 @@ def check_struct(expr, info):
     members = expr['data']
 
     check_type(info, "'data' for struct '%s'" % name, members,
-               allow_dict=True, allow_optional=True)
+               allow_implicit=True, allow_optional=True)
     check_type(info, "'base' for struct '%s'" % name, expr.get('base'),
                allow_metas=['struct'])
 
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 16/27] qapi: add a dictionary form with 'type' key for members

[Marc-André Lureau]

Wherever a struct/union/alternate/command/event member with NAME: TYPE
form is accepted, desugar it to a NAME: { 'type': TYPE } form.

This will allow to add new member details, such as 'if' in the
following patch to introduce conditionals, or 'default' for default
values etc.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                        | 56 +++++++++++++------
 tests/Makefile.include                        |  3 +
 tests/qapi-schema/alternate-invalid-dict.err  |  1 +
 tests/qapi-schema/alternate-invalid-dict.exit |  1 +
 tests/qapi-schema/alternate-invalid-dict.json |  4 ++
 tests/qapi-schema/alternate-invalid-dict.out  |  0
 tests/qapi-schema/event-nest-struct.err       |  2 +-
 tests/qapi-schema/flat-union-inline.err       |  2 +-
 tests/qapi-schema/nested-struct-data.err      |  2 +-
 tests/qapi-schema/qapi-schema-test.json       | 10 ++--
 .../struct-member-invalid-dict.err            |  1 +
 .../struct-member-invalid-dict.exit           |  1 +
 .../struct-member-invalid-dict.json           |  3 +
 .../struct-member-invalid-dict.out            |  0
 .../qapi-schema/union-branch-invalid-dict.err |  1 +
 .../union-branch-invalid-dict.exit            |  1 +
 .../union-branch-invalid-dict.json            |  4 ++
 .../qapi-schema/union-branch-invalid-dict.out |  0
 18 files changed, 66 insertions(+), 26 deletions(-)
 create mode 100644 tests/qapi-schema/alternate-invalid-dict.err
 create mode 100644 tests/qapi-schema/alternate-invalid-dict.exit
 create mode 100644 tests/qapi-schema/alternate-invalid-dict.json
 create mode 100644 tests/qapi-schema/alternate-invalid-dict.out
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.err
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.exit
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.json
 create mode 100644 tests/qapi-schema/struct-member-invalid-dict.out
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.err
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.exit
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.json
 create mode 100644 tests/qapi-schema/union-branch-invalid-dict.out

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index d83fa1900e..fc68bb472e 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -588,11 +588,11 @@ def discriminator_find_enum_define(expr):
     if not base_members:
         return None
 
-    discriminator_type = base_members.get(discriminator)
-    if not discriminator_type:
+    discriminator_member = base_members.get(discriminator)
+    if not discriminator_member:
         return None
 
-    return enum_types.get(discriminator_type)
+    return enum_types.get(discriminator_member['type'])
 
 
 # Names must be letters, numbers, -, and _.  They must start with letter,
@@ -660,6 +660,15 @@ def check_if(expr, info):
         check_if_str(ifcond, info)
 
 
+def normalize_members(expr, field):
+    members = expr.get(field)
+    if isinstance(members, OrderedDict):
+        for key, arg in members.items():
+            if isinstance(arg, dict):
+                continue
+            members[key] = {'type': arg}
+
+
 def check_type(info, source, value, allow_array=False,
                allow_implicit=False, allow_optional=False,
                allow_metas=[]):
@@ -704,8 +713,10 @@ def check_type(info, source, value, allow_array=False,
                                % (source, key))
         # Todo: allow dictionaries to represent default values of
         # an optional argument.
-        check_type(info, "Member '%s' of %s" % (key, source), arg,
-                   allow_array=True,
+        check_known_keys(info, "member '%s' of %s" % (key, source),
+                         arg, ['type'], [])
+        check_type(info, "Member '%s' of %s" % (key, source),
+                   arg['type'], allow_array=True,
                    allow_metas=['built-in', 'union', 'alternate', 'struct',
                                 'enum'])
 
@@ -776,13 +787,13 @@ def check_union(expr, info):
         # member of the base struct.
         check_name(info, "Discriminator of flat union '%s'" % name,
                    discriminator)
-        discriminator_type = base_members.get(discriminator)
-        if not discriminator_type:
+        discriminator_member = base_members.get(discriminator)
+        if not discriminator_member:
             raise QAPISemError(info,
                                "Discriminator '%s' is not a member of base "
                                "struct '%s'"
                                % (discriminator, base))
-        enum_define = enum_types.get(discriminator_type)
+        enum_define = enum_types.get(discriminator_member['type'])
         allow_metas = ['struct']
         # Do not allow string discriminator
         if not enum_define:
@@ -795,10 +806,13 @@ def check_union(expr, info):
         raise QAPISemError(info, "Union '%s' cannot have empty 'data'" % name)
     for (key, value) in members.items():
         check_name(info, "Member of union '%s'" % name, key)
+        source = "member '%s' of union '%s'" % (key, name)
+        check_known_keys(info, source, value, ['type'], [])
+        typ = value['type']
 
         # Each value must name a known type
         check_type(info, "Member '%s' of union '%s'" % (key, name),
-                   value, allow_array=not base, allow_metas=allow_metas)
+                   typ, allow_array=not base, allow_metas=allow_metas)
 
         # If the discriminator names an enum type, then all members
         # of 'data' must also be members of the enum type.
@@ -822,18 +836,20 @@ def check_alternate(expr, info):
                            "in 'data'" % name)
     for (key, value) in members.items():
         check_name(info, "Member of alternate '%s'" % name, key)
+        source = "member '%s' of alternate '%s'" % (key, name)
+        check_known_keys(info, source, value, ['type'], [])
+        typ = value['type']
 
         # Ensure alternates have no type conflicts.
-        check_type(info, "Member '%s' of alternate '%s'" % (key, name),
-                   value,
+        check_type(info, "Member '%s' of alternate '%s'" % (key, name), typ,
                    allow_metas=['built-in', 'union', 'struct', 'enum'])
-        qtype = find_alternate_member_qtype(value)
+        qtype = find_alternate_member_qtype(typ)
         if not qtype:
             raise QAPISemError(info, "Alternate '%s' member '%s' cannot use "
-                               "type '%s'" % (name, key, value))
+                               "type '%s'" % (name, key, typ))
         conflicting = set([qtype])
         if qtype == 'QTYPE_QSTRING':
-            enum_expr = enum_types.get(value)
+            enum_expr = enum_types.get(typ)
             if enum_expr:
                 for v in enum_get_names(enum_expr):
                     if v in ['on', 'off']:
@@ -947,6 +963,10 @@ def check_exprs(exprs):
         info = expr_elem['info']
         if 'enum' in expr:
             normalize_enum(expr, info)
+        elif 'union' in expr:
+            normalize_members(expr, 'base')
+        if {'union', 'alternate', 'struct', 'command', 'event'} & set(expr):
+            normalize_members(expr, 'data')
 
     # Learn the types and check for valid expression keys
     for expr_elem in exprs:
@@ -1735,7 +1755,7 @@ class QAPISchema(object):
         return QAPISchemaObjectTypeMember(name, typ, optional)
 
     def _make_members(self, data, info):
-        return [self._make_member(key, value, info)
+        return [self._make_member(key, value['type'], info)
                 for (key, value) in data.items()]
 
     def _def_struct_type(self, expr, info, doc):
@@ -1771,11 +1791,11 @@ class QAPISchema(object):
                 name, info, doc, ifcond,
                 'base', self._make_members(base, info))
         if tag_name:
-            variants = [self._make_variant(key, value)
+            variants = [self._make_variant(key, value['type'])
                         for (key, value) in data.items()]
             members = []
         else:
-            variants = [self._make_simple_variant(key, value, info)
+            variants = [self._make_simple_variant(key, value['type'], info)
                         for (key, value) in data.items()]
             typ = self._make_implicit_enum_type(name, info, ifcond,
                                                 [v.name for v in variants])
@@ -1791,7 +1811,7 @@ class QAPISchema(object):
         name = expr['alternate']
         data = expr['data']
         ifcond = expr.get('if')
-        variants = [self._make_variant(key, value)
+        variants = [self._make_variant(key, value['type'])
                     for (key, value) in data.items()]
         tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
         self._def_entity(
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 3ba9097892..43e100a6cd 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -421,6 +421,7 @@ qapi-schema += alternate-conflict-string.json
 qapi-schema += alternate-conflict-bool-string.json
 qapi-schema += alternate-conflict-num-string.json
 qapi-schema += alternate-empty.json
+qapi-schema += alternate-invalid-dict.json
 qapi-schema += alternate-nested.json
 qapi-schema += alternate-unknown.json
 qapi-schema += args-alternate.json
@@ -563,6 +564,7 @@ qapi-schema += returns-whitelist.json
 qapi-schema += struct-base-clash-deep.json
 qapi-schema += struct-base-clash.json
 qapi-schema += struct-data-invalid.json
+qapi-schema += struct-member-invalid-dict.json
 qapi-schema += struct-member-invalid.json
 qapi-schema += trailing-comma-list.json
 qapi-schema += trailing-comma-object.json
@@ -574,6 +576,7 @@ qapi-schema += unicode-str.json
 qapi-schema += union-base-empty.json
 qapi-schema += union-base-no-discriminator.json
 qapi-schema += union-branch-case.json
+qapi-schema += union-branch-invalid-dict.json
 qapi-schema += union-clash-branches.json
 qapi-schema += union-empty.json
 qapi-schema += union-invalid-base.json
diff --git a/tests/qapi-schema/alternate-invalid-dict.err b/tests/qapi-schema/alternate-invalid-dict.err
new file mode 100644
index 0000000000..631d46628e
--- /dev/null
+++ b/tests/qapi-schema/alternate-invalid-dict.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-invalid-dict.json:2: Key 'type' is missing from member 'two' of alternate 'Alt'
diff --git a/tests/qapi-schema/alternate-invalid-dict.exit b/tests/qapi-schema/alternate-invalid-dict.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/alternate-invalid-dict.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/alternate-invalid-dict.json b/tests/qapi-schema/alternate-invalid-dict.json
new file mode 100644
index 0000000000..45f2c8ebef
--- /dev/null
+++ b/tests/qapi-schema/alternate-invalid-dict.json
@@ -0,0 +1,4 @@
+# invalid field dictionnary, missing type
+{ 'alternate': 'Alt',
+  'data': { 'one': 'str',
+            'two': { 'if': 'foo' } } }
diff --git a/tests/qapi-schema/alternate-invalid-dict.out b/tests/qapi-schema/alternate-invalid-dict.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err
index 5a42701b8f..ea1be65d89 100644
--- a/tests/qapi-schema/event-nest-struct.err
+++ b/tests/qapi-schema/event-nest-struct.err
@@ -1 +1 @@
-tests/qapi-schema/event-nest-struct.json:1: Member 'a' of 'data' for event 'EVENT_A' should be a type name
+tests/qapi-schema/event-nest-struct.json:1: Key 'type' is missing from member 'a' of 'data' for event 'EVENT_A'
diff --git a/tests/qapi-schema/flat-union-inline.err b/tests/qapi-schema/flat-union-inline.err
index 2333358d28..3d29433bfb 100644
--- a/tests/qapi-schema/flat-union-inline.err
+++ b/tests/qapi-schema/flat-union-inline.err
@@ -1 +1 @@
-tests/qapi-schema/flat-union-inline.json:7: Member 'value1' of union 'TestUnion' should be a type name
+tests/qapi-schema/flat-union-inline.json:7: Key 'type' is missing from member 'value1' of union 'TestUnion'
diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
index da767bade2..48355adcbe 100644
--- a/tests/qapi-schema/nested-struct-data.err
+++ b/tests/qapi-schema/nested-struct-data.err
@@ -1 +1 @@
-tests/qapi-schema/nested-struct-data.json:2: Member 'a' of 'data' for command 'foo' should be a type name
+tests/qapi-schema/nested-struct-data.json:2: Key 'type' is missing from member 'a' of 'data' for command 'foo'
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 35ca94d991..30f635d484 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -11,7 +11,7 @@
         'guest-sync' ] } }
 
 { 'struct': 'TestStruct',
-  'data': { 'integer': 'int', 'boolean': 'bool', 'string': 'str' } }
+  'data': { 'integer': {'type': 'int'}, 'boolean': 'bool', 'string': 'str' } }
 
 # for testing enums
 { 'struct': 'NestedEnumsOne',
@@ -77,7 +77,7 @@
 { 'union': 'UserDefFlatUnion',
   'base': 'UserDefUnionBase',   # intentional forward reference
   'discriminator': 'enum1',
-  'data': { 'value1' : 'UserDefA',
+  'data': { 'value1' : {'type': 'UserDefA'},
             'value2' : 'UserDefB',
             'value3' : 'UserDefB'
             # 'value4' defaults to empty
@@ -98,7 +98,7 @@
 { 'struct': 'WrapAlternate',
   'data': { 'alt': 'UserDefAlternate' } }
 { 'alternate': 'UserDefAlternate',
-  'data': { 'udfu': 'UserDefFlatUnion', 'e': 'EnumOne', 'i': 'int',
+  'data': { 'udfu': {'type': 'UserDefFlatUnion'}, 'e': 'EnumOne', 'i': 'int',
             'n': 'null' } }
 
 { 'struct': 'UserDefC',
@@ -134,7 +134,7 @@
 { 'command': 'user_def_cmd', 'data': {} }
 { 'command': 'user_def_cmd1', 'data': {'ud1a': 'UserDefOne'} }
 { 'command': 'user_def_cmd2',
-  'data': {'ud1a': 'UserDefOne', '*ud1b': 'UserDefOne'},
+  'data': {'ud1a': {'type': 'UserDefOne'}, '*ud1b': 'UserDefOne'},
   'returns': 'UserDefTwo' }
 
 # Returning a non-dictionary requires a name from the whitelist
@@ -164,7 +164,7 @@
 
 # testing event
 { 'struct': 'EventStructOne',
-  'data': { 'struct1': 'UserDefOne', 'string': 'str', '*enum2': 'EnumOne' } }
+  'data': { 'struct1': {'type': 'UserDefOne'}, 'string': 'str', '*enum2': 'EnumOne' } }
 
 { 'event': 'EVENT_A' }
 { 'event': 'EVENT_B',
diff --git a/tests/qapi-schema/struct-member-invalid-dict.err b/tests/qapi-schema/struct-member-invalid-dict.err
new file mode 100644
index 0000000000..6a765bc668
--- /dev/null
+++ b/tests/qapi-schema/struct-member-invalid-dict.err
@@ -0,0 +1 @@
+tests/qapi-schema/struct-member-invalid-dict.json:2: Key 'type' is missing from member '*a' of 'data' for struct 'foo'
diff --git a/tests/qapi-schema/struct-member-invalid-dict.exit b/tests/qapi-schema/struct-member-invalid-dict.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/struct-member-invalid-dict.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/struct-member-invalid-dict.json b/tests/qapi-schema/struct-member-invalid-dict.json
new file mode 100644
index 0000000000..ebd9733b49
--- /dev/null
+++ b/tests/qapi-schema/struct-member-invalid-dict.json
@@ -0,0 +1,3 @@
+# exploded member form must have a 'type'
+{ 'struct': 'foo',
+  'data': { '*a': { 'case': 'foo' } } }
diff --git a/tests/qapi-schema/struct-member-invalid-dict.out b/tests/qapi-schema/struct-member-invalid-dict.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/union-branch-invalid-dict.err b/tests/qapi-schema/union-branch-invalid-dict.err
new file mode 100644
index 0000000000..89f9b36791
--- /dev/null
+++ b/tests/qapi-schema/union-branch-invalid-dict.err
@@ -0,0 +1 @@
+tests/qapi-schema/union-branch-invalid-dict.json:2: Key 'type' is missing from member 'integer' of union 'UnionInvalidBranch'
diff --git a/tests/qapi-schema/union-branch-invalid-dict.exit b/tests/qapi-schema/union-branch-invalid-dict.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/union-branch-invalid-dict.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/union-branch-invalid-dict.json b/tests/qapi-schema/union-branch-invalid-dict.json
new file mode 100644
index 0000000000..19c5d9cacd
--- /dev/null
+++ b/tests/qapi-schema/union-branch-invalid-dict.json
@@ -0,0 +1,4 @@
+# exploded member form must have a 'type'
+{ 'union': 'UnionInvalidBranch',
+  'data': { 'integer': { 'if': 'foo'},
+            's8': 'int8' } }
diff --git a/tests/qapi-schema/union-branch-invalid-dict.out b/tests/qapi-schema/union-branch-invalid-dict.out
new file mode 100644
index 0000000000..e69de29bb2
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 17/27] qapi: add 'if' to implicit struct members

[Marc-André Lureau]

Generated code is not changed by this patch, but with "qapi: add #if
conditions to generated code" patch.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                  | 14 +++++++-------
 tests/qapi-schema/qapi-schema-test.json | 12 +++++++++---
 tests/qapi-schema/qapi-schema-test.out  |  5 +++++
 tests/qapi-schema/test-qapi.py          |  1 +
 4 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index fc68bb472e..9b95f8cfe9 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -714,7 +714,7 @@ def check_type(info, source, value, allow_array=False,
         # Todo: allow dictionaries to represent default values of
         # an optional argument.
         check_known_keys(info, "member '%s' of %s" % (key, source),
-                         arg, ['type'], [])
+                         arg, ['type'], ['if'])
         check_type(info, "Member '%s' of %s" % (key, source),
                    arg['type'], allow_array=True,
                    allow_metas=['built-in', 'union', 'alternate', 'struct',
@@ -1422,8 +1422,8 @@ class QAPISchemaMember(object):
 
 
 class QAPISchemaObjectTypeMember(QAPISchemaMember):
-    def __init__(self, name, typ, optional):
-        QAPISchemaMember.__init__(self, name)
+    def __init__(self, name, typ, optional, ifcond=None):
+        QAPISchemaMember.__init__(self, name, ifcond)
         assert isinstance(typ, str)
         assert isinstance(optional, bool)
         self._type_name = typ
@@ -1744,7 +1744,7 @@ class QAPISchema(object):
             name, info, doc, ifcond,
             self._make_enum_members(data), prefix))
 
-    def _make_member(self, name, typ, info):
+    def _make_member(self, name, typ, ifcond, info):
         optional = False
         if name.startswith('*'):
             name = name[1:]
@@ -1752,10 +1752,10 @@ class QAPISchema(object):
         if isinstance(typ, list):
             assert len(typ) == 1
             typ = self._make_array_type(typ[0], info)
-        return QAPISchemaObjectTypeMember(name, typ, optional)
+        return QAPISchemaObjectTypeMember(name, typ, optional, ifcond)
 
     def _make_members(self, data, info):
-        return [self._make_member(key, value['type'], info)
+        return [self._make_member(key, value['type'], value.get('if'), info)
                 for (key, value) in data.items()]
 
     def _def_struct_type(self, expr, info, doc):
@@ -1776,7 +1776,7 @@ class QAPISchema(object):
             typ = self._make_array_type(typ[0], info)
         typ = self._make_implicit_object_type(
             typ, info, None, self.lookup_type(typ),
-            'wrapper', [self._make_member('data', typ, info)])
+            'wrapper', [self._make_member('data', typ, None, info)])
         return QAPISchemaObjectTypeVariant(case, typ)
 
     def _def_union_type(self, expr, info, doc):
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 30f635d484..3bf440aab4 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -199,7 +199,9 @@
 
 # test 'if' condition handling
 
-{ 'struct': 'TestIfStruct', 'data': { 'foo': 'int' },
+{ 'struct': 'TestIfStruct', 'data':
+  { 'foo': 'int',
+    'bar': { 'type': 'int', 'if': 'defined(TEST_IF_STRUCT_BAR)'} },
   'if': 'defined(TEST_IF_STRUCT)' }
 
 { 'enum': 'TestIfEnum', 'data':
@@ -212,11 +214,15 @@
 { 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
   'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
 
-{ 'command': 'TestIfCmd', 'data': { 'foo': 'TestIfStruct', 'bar': 'TestIfEnum' },
+{ 'command': 'TestIfCmd', 'data':
+  { 'foo': 'TestIfStruct',
+    'bar': { 'type': 'TestIfEnum', 'if': 'defined(TEST_IF_CMD_BAR)' } },
   'returns': 'UserDefThree',
   'if': ['defined(TEST_IF_CMD)', 'defined(TEST_IF_STRUCT)'] }
 
 { 'command': 'TestCmdReturnDefThree', 'returns': 'UserDefThree' }
 
-{ 'event': 'TestIfEvent', 'data': { 'foo': 'TestIfStruct' },
+{ 'event': 'TestIfEvent', 'data':
+  { 'foo': 'TestIfStruct',
+    'bar': { 'type': 'TestIfEnum', 'if': 'defined(TEST_IF_EVT_BAR)' } },
   'if': 'defined(TEST_IF_EVT) && defined(TEST_IF_STRUCT)' }
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 1fb33f302c..71b84878c7 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -266,6 +266,8 @@ command __org.qemu_x-command q_obj___org.qemu_x-command-arg -> __org.qemu_x-Unio
    gen=True success_response=True boxed=False oob=False preconfig=False
 object TestIfStruct
     member foo: int optional=False
+    member bar: int optional=False
+        if ['defined(TEST_IF_STRUCT_BAR)']
     if ['defined(TEST_IF_STRUCT)']
 enum TestIfEnum
     member foo
@@ -290,6 +292,7 @@ alternate TestIfAlternate
 object q_obj_TestIfCmd-arg
     member foo: TestIfStruct optional=False
     member bar: TestIfEnum optional=False
+        if ['defined(TEST_IF_CMD_BAR)']
     if ['defined(TEST_IF_CMD)', 'defined(TEST_IF_STRUCT)']
 command TestIfCmd q_obj_TestIfCmd-arg -> UserDefThree
    gen=True success_response=True boxed=False oob=False preconfig=False
@@ -298,6 +301,8 @@ command TestCmdReturnDefThree None -> UserDefThree
    gen=True success_response=True boxed=False oob=False preconfig=False
 object q_obj_TestIfEvent-arg
     member foo: TestIfStruct optional=False
+    member bar: TestIfEnum optional=False
+        if ['defined(TEST_IF_EVT_BAR)']
     if ['defined(TEST_IF_EVT) && defined(TEST_IF_STRUCT)']
 event TestIfEvent q_obj_TestIfEvent-arg
    boxed=False
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index b2f4ce6134..69e40d87d2 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -39,6 +39,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
         for m in members:
             print('    member %s: %s optional=%s'
                   % (m.name, m.type.name, m.optional))
+            self._print_if(m.ifcond, 8)
         self._print_variants(variants)
         self._print_if(ifcond)
 
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 18/27] qapi: add an error in case a discriminator is conditionnal

[Marc-André Lureau]

Making a discriminator conditonal doesn't make much sense. Instead,
the union could be made conditional.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                          | 11 +++++++++--
 tests/Makefile.include                          |  1 +
 .../flat-union-invalid-if-discriminator.err     |  1 +
 .../flat-union-invalid-if-discriminator.exit    |  1 +
 .../flat-union-invalid-if-discriminator.json    | 17 +++++++++++++++++
 .../flat-union-invalid-if-discriminator.out     |  0
 6 files changed, 29 insertions(+), 2 deletions(-)
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.err
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.exit
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.json
 create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.out

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 9b95f8cfe9..13fbb28493 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -577,7 +577,8 @@ def find_alternate_member_qtype(qapi_type):
 
 # Return the discriminator enum define if discriminator is specified as an
 # enum type, otherwise return None.
-def discriminator_find_enum_define(expr):
+def discriminator_find_enum_define(expr, info):
+    name = expr['union']
     base = expr.get('base')
     discriminator = expr.get('discriminator')
 
@@ -592,6 +593,11 @@ def discriminator_find_enum_define(expr):
     if not discriminator_member:
         return None
 
+    if discriminator_member.get('if'):
+        raise QAPISemError(info, 'The discriminator %s.%s for union %s '
+                           'must not be conditional' %
+                           (base, discriminator, name))
+
     return enum_types.get(discriminator_member['type'])
 
 
@@ -1020,7 +1026,8 @@ def check_exprs(exprs):
 
         if 'include' in expr:
             continue
-        if 'union' in expr and not discriminator_find_enum_define(expr):
+        info = expr_elem['info']
+        if 'union' in expr and not discriminator_find_enum_define(expr, info):
             name = '%sKind' % expr['union']
         elif 'alternate' in expr:
             name = '%sKind' % expr['alternate']
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 43e100a6cd..abc3fdf764 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -510,6 +510,7 @@ qapi-schema += flat-union-inline.json
 qapi-schema += flat-union-int-branch.json
 qapi-schema += flat-union-invalid-branch-key.json
 qapi-schema += flat-union-invalid-discriminator.json
+qapi-schema += flat-union-invalid-if-discriminator.json
 qapi-schema += flat-union-no-base.json
 qapi-schema += flat-union-optional-discriminator.json
 qapi-schema += flat-union-string-discriminator.json
diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.err b/tests/qapi-schema/flat-union-invalid-if-discriminator.err
new file mode 100644
index 0000000000..0c94c9860d
--- /dev/null
+++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-invalid-if-discriminator.json:13: The discriminator TestBase.enum1 for union TestUnion must not be conditional
diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.exit b/tests/qapi-schema/flat-union-invalid-if-discriminator.exit
new file mode 100644
index 0000000000..d00491fd7e
--- /dev/null
+++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.json b/tests/qapi-schema/flat-union-invalid-if-discriminator.json
new file mode 100644
index 0000000000..618ec36396
--- /dev/null
+++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.json
@@ -0,0 +1,17 @@
+{ 'enum': 'TestEnum',
+  'data': [ 'value1', 'value2' ] }
+
+{ 'struct': 'TestBase',
+  'data': { 'enum1': { 'type': 'TestEnum', 'if': 'FOO' } } }
+
+{ 'struct': 'TestTypeA',
+  'data': { 'string': 'str' } }
+
+{ 'struct': 'TestTypeB',
+  'data': { 'integer': 'int' } }
+
+{ 'union': 'TestUnion',
+  'base': 'TestBase',
+  'discriminator': 'enum1',
+  'data': { 'value1': 'TestTypeA',
+            'value2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.out b/tests/qapi-schema/flat-union-invalid-if-discriminator.out
new file mode 100644
index 0000000000..e69de29bb2
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 19/27] qapi: add 'if' on union members

[Marc-André Lureau]

Add 'if' key to union members:

{ 'union': 'TestIfUnion', 'data':
    'mem': { 'type': 'str', 'if': 'COND'} }

Generated code is not changed by this patch but with "qapi: add #if
conditions to generated code".

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                  | 17 +++++++++--------
 tests/qapi-schema/qapi-schema-test.json |  7 ++++++-
 tests/qapi-schema/qapi-schema-test.out  | 10 ++++++++++
 tests/qapi-schema/test-qapi.py          |  1 +
 4 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 13fbb28493..e1bd9a22ba 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -813,7 +813,7 @@ def check_union(expr, info):
     for (key, value) in members.items():
         check_name(info, "Member of union '%s'" % name, key)
         source = "member '%s' of union '%s'" % (key, name)
-        check_known_keys(info, source, value, ['type'], [])
+        check_known_keys(info, source, value, ['type'], ['if'])
         typ = value['type']
 
         # Each value must name a known type
@@ -1496,8 +1496,8 @@ class QAPISchemaObjectTypeVariants(object):
 class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
     role = 'branch'
 
-    def __init__(self, name, typ):
-        QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
+    def __init__(self, name, typ, ifcond=None):
+        QAPISchemaObjectTypeMember.__init__(self, name, typ, False, ifcond)
 
 
 class QAPISchemaAlternateType(QAPISchemaType):
@@ -1777,14 +1777,14 @@ class QAPISchema(object):
     def _make_variant(self, case, typ):
         return QAPISchemaObjectTypeVariant(case, typ)
 
-    def _make_simple_variant(self, case, typ, info):
+    def _make_simple_variant(self, case, typ, ifcond, info):
         if isinstance(typ, list):
             assert len(typ) == 1
             typ = self._make_array_type(typ[0], info)
         typ = self._make_implicit_object_type(
             typ, info, None, self.lookup_type(typ),
             'wrapper', [self._make_member('data', typ, None, info)])
-        return QAPISchemaObjectTypeVariant(case, typ)
+        return QAPISchemaObjectTypeVariant(case, typ, ifcond)
 
     def _def_union_type(self, expr, info, doc):
         name = expr['union']
@@ -1802,10 +1802,11 @@ class QAPISchema(object):
                         for (key, value) in data.items()]
             members = []
         else:
-            variants = [self._make_simple_variant(key, value['type'], info)
+            variants = [self._make_simple_variant(key, value['type'],
+                                                  value.get('if'), info)
                         for (key, value) in data.items()]
-            typ = self._make_implicit_enum_type(name, info, ifcond,
-                                                [v.name for v in variants])
+            enum = [{'name': v.name, 'if': v.ifcond} for v in variants]
+            typ = self._make_implicit_enum_type(name, info, ifcond, enum)
             tag_member = QAPISchemaObjectTypeMember('type', typ, False)
             members = [tag_member]
         self._def_entity(
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 3bf440aab4..6d3c6c0b53 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -208,9 +208,14 @@
   [ 'foo', { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ],
   'if': 'defined(TEST_IF_ENUM)' }
 
-{ 'union': 'TestIfUnion', 'data': { 'foo': 'TestStruct' },
+{ 'union': 'TestIfUnion', 'data':
+  { 'foo': 'TestStruct',
+    'union_bar': { 'type': 'str', 'if': 'defined(TEST_IF_UNION_BAR)'} },
   'if': 'defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)' }
 
+{ 'command': 'TestIfUnionCmd', 'data': { 'union_cmd_arg': 'TestIfUnion' },
+  'if': 'defined(TEST_IF_UNION)' }
+
 { 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
   'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
 
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 71b84878c7..ac1069cf1f 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -278,12 +278,22 @@ object q_obj_TestStruct-wrapper
     member data: TestStruct optional=False
 enum TestIfUnionKind
     member foo
+    member union_bar
+        if ['defined(TEST_IF_UNION_BAR)']
     if ['defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)']
 object TestIfUnion
     member type: TestIfUnionKind optional=False
     tag type
     case foo: q_obj_TestStruct-wrapper
+    case union_bar: q_obj_str-wrapper
+        if ['defined(TEST_IF_UNION_BAR)']
     if ['defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)']
+object q_obj_TestIfUnionCmd-arg
+    member union_cmd_arg: TestIfUnion optional=False
+    if ['defined(TEST_IF_UNION)']
+command TestIfUnionCmd q_obj_TestIfUnionCmd-arg -> None
+   gen=True success_response=True boxed=False oob=False preconfig=False
+    if ['defined(TEST_IF_UNION)']
 alternate TestIfAlternate
     tag type
     case foo: int
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 69e40d87d2..d977e936c7 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -68,6 +68,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
             print('    tag %s' % variants.tag_member.name)
             for v in variants.variants:
                 print('    case %s: %s' % (v.name, v.type.name))
+                QAPISchemaTestVisitor._print_if(v.ifcond, 8)
 
     @staticmethod
     def _print_if(ifcond, indent=4):
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 20/27] qapi: add 'if' to alternate members

[Marc-André Lureau]

Add 'if' key to alternate members:

{ 'alternate': 'TestIfAlternate', 'data':
  { 'alt': { 'type': 'TestStruct', 'if': 'COND' } } }

Generated code is not changed by this patch but with "qapi: add #if
conditions to generated code".

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py                  | 10 +++++-----
 tests/qapi-schema/qapi-schema-test.json |  6 +++++-
 tests/qapi-schema/qapi-schema-test.out  |  9 ++++++++-
 3 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index e1bd9a22ba..b3b64a60bf 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -843,7 +843,7 @@ def check_alternate(expr, info):
     for (key, value) in members.items():
         check_name(info, "Member of alternate '%s'" % name, key)
         source = "member '%s' of alternate '%s'" % (key, name)
-        check_known_keys(info, source, value, ['type'], [])
+        check_known_keys(info, source, value, ['type'], ['if'])
         typ = value['type']
 
         # Ensure alternates have no type conflicts.
@@ -1774,8 +1774,8 @@ class QAPISchema(object):
                                               self._make_members(data, info),
                                               None))
 
-    def _make_variant(self, case, typ):
-        return QAPISchemaObjectTypeVariant(case, typ)
+    def _make_variant(self, case, typ, ifcond):
+        return QAPISchemaObjectTypeVariant(case, typ, ifcond)
 
     def _make_simple_variant(self, case, typ, ifcond, info):
         if isinstance(typ, list):
@@ -1798,7 +1798,7 @@ class QAPISchema(object):
                 name, info, doc, ifcond,
                 'base', self._make_members(base, info))
         if tag_name:
-            variants = [self._make_variant(key, value['type'])
+            variants = [self._make_variant(key, value['type'], value.get('if'))
                         for (key, value) in data.items()]
             members = []
         else:
@@ -1819,7 +1819,7 @@ class QAPISchema(object):
         name = expr['alternate']
         data = expr['data']
         ifcond = expr.get('if')
-        variants = [self._make_variant(key, value['type'])
+        variants = [self._make_variant(key, value['type'], value.get('if'))
                     for (key, value) in data.items()]
         tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
         self._def_entity(
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 6d3c6c0b53..df3edf9d89 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -216,9 +216,13 @@
 { 'command': 'TestIfUnionCmd', 'data': { 'union_cmd_arg': 'TestIfUnion' },
   'if': 'defined(TEST_IF_UNION)' }
 
-{ 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
+{ 'alternate': 'TestIfAlternate', 'data':
+  { 'foo': 'int', 'alt_bar': { 'type': 'TestStruct', 'if': 'defined(TEST_IF_ALT_BAR)'} },
   'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
 
+{ 'command': 'TestIfAlternateCmd', 'data': { 'alt_cmd_arg': 'TestIfAlternate' },
+  'if': 'defined(TEST_IF_ALT)' }
+
 { 'command': 'TestIfCmd', 'data':
   { 'foo': 'TestIfStruct',
     'bar': { 'type': 'TestIfEnum', 'if': 'defined(TEST_IF_CMD_BAR)' } },
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index ac1069cf1f..cdbd5b87cc 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -297,8 +297,15 @@ command TestIfUnionCmd q_obj_TestIfUnionCmd-arg -> None
 alternate TestIfAlternate
     tag type
     case foo: int
-    case bar: TestStruct
+    case alt_bar: TestStruct
+        if ['defined(TEST_IF_ALT_BAR)']
     if ['defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)']
+object q_obj_TestIfAlternateCmd-arg
+    member alt_cmd_arg: TestIfAlternate optional=False
+    if ['defined(TEST_IF_ALT)']
+command TestIfAlternateCmd q_obj_TestIfAlternateCmd-arg -> None
+   gen=True success_response=True boxed=False oob=False preconfig=False
+    if ['defined(TEST_IF_ALT)']
 object q_obj_TestIfCmd-arg
     member foo: TestIfStruct optional=False
     member bar: TestIfEnum optional=False
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 21/27] qapi: add #if conditions to generated code members

[Marc-André Lureau]

Wrap generated enum/struct members and code with #if/#endif, using the
.ifcond members added in the previous patches.

Some types generate both enum and struct members for example, so a
step-by-step is unnecessarily complicated to deal with (it would
easily generate invalid intermediary code).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/common.py     |  4 ++++
 scripts/qapi/introspect.py | 13 +++++++++----
 scripts/qapi/types.py      |  4 ++++
 scripts/qapi/visit.py      |  6 ++++++
 4 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index b3b64a60bf..a66c035b91 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -2098,11 +2098,13 @@ const QEnumLookup %(c_name)s_lookup = {
 ''',
                 c_name=c_name(name))
     for m in members:
+        ret += gen_if(m.ifcond)
         index = c_enum_const(name, m.name, prefix)
         ret += mcgen('''
         [%(index)s] = "%(name)s",
 ''',
                      index=index, name=m.name)
+        ret += gen_endif(m.ifcond)
 
     ret += mcgen('''
     },
@@ -2124,10 +2126,12 @@ typedef enum %(c_name)s {
                 c_name=c_name(name))
 
     for m in enum_members:
+        ret += gen_if(m.ifcond)
         ret += mcgen('''
     %(c_enum)s,
 ''',
                      c_enum=c_enum_const(name, m.name, prefix))
+        ret += gen_endif(m.ifcond)
 
     ret += mcgen('''
 } %(c_name)s;
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 3f1ca99f6d..bf5db51f4a 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -148,6 +148,8 @@ const QLitObject %(c_name)s = %(c_string)s;
         ret = {'name': member.name, 'type': self._use_type(member.type)}
         if member.optional:
             ret['default'] = None
+        if member.ifcond:
+            ret = (ret, member.ifcond)
         return ret
 
     def _gen_variants(self, tag_name, variants):
@@ -155,14 +157,16 @@ const QLitObject %(c_name)s = %(c_string)s;
                 'variants': [self._gen_variant(v) for v in variants]}
 
     def _gen_variant(self, variant):
-        return {'case': variant.name, 'type': self._use_type(variant.type)}
+        return ({'case': variant.name, 'type': self._use_type(variant.type)},
+                variant.ifcond)
 
     def visit_builtin_type(self, name, info, json_type):
         self._gen_qlit(name, 'builtin', {'json-type': json_type}, [])
 
     def visit_enum_type(self, name, info, ifcond, members, prefix):
         self._gen_qlit(name, 'enum',
-                       {'values': [m.name for m in members]}, ifcond)
+                       {'values': [(m.name, m.ifcond) for m in members]},
+                       ifcond)
 
     def visit_array_type(self, name, info, ifcond, element_type):
         element = self._use_type(element_type)
@@ -178,8 +182,9 @@ const QLitObject %(c_name)s = %(c_string)s;
 
     def visit_alternate_type(self, name, info, ifcond, variants):
         self._gen_qlit(name, 'alternate',
-                       {'members': [{'type': self._use_type(m.type)}
-                                    for m in variants.variants]}, ifcond)
+                       {'members': [
+                           ({'type': self._use_type(m.type)}, m.ifcond)
+                           for m in variants.variants]}, ifcond)
 
     def visit_command(self, name, info, ifcond, arg_type, ret_type, gen,
                       success_response, boxed, allow_oob, allow_preconfig):
diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
index 2d4a70f810..7d9eef6320 100644
--- a/scripts/qapi/types.py
+++ b/scripts/qapi/types.py
@@ -43,6 +43,7 @@ struct %(c_name)s {
 def gen_struct_members(members):
     ret = ''
     for memb in members:
+        ret += gen_if(memb.ifcond)
         if memb.optional:
             ret += mcgen('''
     bool has_%(c_name)s;
@@ -52,6 +53,7 @@ def gen_struct_members(members):
     %(c_type)s %(c_name)s;
 ''',
                      c_type=memb.type.c_type(), c_name=c_name(memb.name))
+        ret += gen_endif(memb.ifcond)
     return ret
 
 
@@ -131,11 +133,13 @@ def gen_variants(variants):
     for var in variants.variants:
         if var.type.name == 'q_empty':
             continue
+        ret += gen_if(var.ifcond)
         ret += mcgen('''
         %(c_type)s %(c_name)s;
 ''',
                      c_type=var.type.c_unboxed_type(),
                      c_name=c_name(var.name))
+        ret += gen_endif(var.ifcond)
 
     ret += mcgen('''
     } u;
diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
index 24f85a2e85..82eab72b21 100644
--- a/scripts/qapi/visit.py
+++ b/scripts/qapi/visit.py
@@ -54,6 +54,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
                      c_type=base.c_name())
 
     for memb in members:
+        ret += gen_if(memb.ifcond)
         if memb.optional:
             ret += mcgen('''
     if (visit_optional(v, "%(name)s", &obj->has_%(c_name)s)) {
@@ -73,6 +74,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
             ret += mcgen('''
     }
 ''')
+        ret += gen_endif(memb.ifcond)
 
     if variants:
         ret += mcgen('''
@@ -84,6 +86,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
             case_str = c_enum_const(variants.tag_member.type.name,
                                     var.name,
                                     variants.tag_member.type.prefix)
+            ret += gen_if(var.ifcond)
             if var.type.name == 'q_empty':
                 # valid variant and nothing to do
                 ret += mcgen('''
@@ -100,6 +103,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
                              case=case_str,
                              c_type=var.type.c_name(), c_name=c_name(var.name))
 
+            ret += gen_endif(var.ifcond)
         ret += mcgen('''
     default:
         abort();
@@ -190,6 +194,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
                 c_name=c_name(name))
 
     for var in variants.variants:
+        ret += gen_if(var.ifcond)
         ret += mcgen('''
     case %(case)s:
 ''',
@@ -217,6 +222,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
         ret += mcgen('''
         break;
 ''')
+        ret += gen_endif(var.ifcond)
 
     ret += mcgen('''
     case QTYPE_NONE:
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 22/27] docs: document schema configuration

[Marc-André Lureau]

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/devel/qapi-code-gen.txt | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index c2e11465f0..799aef7728 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -752,6 +752,25 @@ gets its generated code guarded like this:
  #endif /* defined(HAVE_BAR) */
  #endif /* defined(CONFIG_FOO) */
 
+Where a member can be defined with a single string value for its type,
+it is also possible to supply a dictionary instead with both 'type'
+and 'if' keys.
+
+Example: a conditional 'bar' member
+
+{ 'struct': 'IfStruct', 'data':
+  { 'foo': 'int',
+    'bar': { 'type': 'int', 'if': 'defined(IFCOND)'} } }
+
+An enum value can be replaced by a dictionary with a 'name' and a 'if'
+key.
+
+Example: a conditional 'bar' enum member.
+
+{ 'enum': 'IfEnum', 'data':
+  [ 'foo',
+    { 'name' : 'bar', 'if': 'defined(IFCOND)' } ] }
+
 Please note that you are responsible to ensure that the C code will
 compile with an arbitrary combination of conditions, since the
 generators are unable to check it at this point.
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 23/27] qapi: add 'If:' condition to enum values documentation

[Marc-André Lureau]

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/doc.py             | 5 ++++-
 tests/qapi-schema/doc-good.json | 4 +++-
 tests/qapi-schema/doc-good.out  | 1 +
 tests/qapi-schema/doc-good.texi | 2 ++
 4 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 76cb186ff9..196deea11d 100755
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -128,7 +128,10 @@ def texi_body(doc):
 
 def texi_enum_value(value):
     """Format a table of members item for an enumeration value"""
-    return '@item @code{%s}\n' % value.name
+    return '@item @code{%s}%s\n' % (
+        value.name,
+        '\n@b{If:} @code{%s}\n' %
+        ', '.join(value.ifcond) if value.ifcond else '')
 
 
 def texi_member(member, suffix=''):
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 984cd8ed06..c7fe08c530 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -55,7 +55,9 @@
 #
 # @two is undocumented
 ##
-{ 'enum': 'Enum', 'data': [ 'one', 'two' ], 'if': 'defined(IFCOND)' }
+{ 'enum': 'Enum', 'data':
+  [ { 'name': 'one', 'if': 'defined(IFENUM)' }, 'two' ],
+  'if': 'defined(IFCOND)' }
 
 ##
 # @Base:
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index c2fc5c774a..a05535b69b 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -11,6 +11,7 @@ enum QType
 module doc-good.json
 enum Enum
     member one
+        if ['defined(IFENUM)']
     member two
     if ['defined(IFCOND)']
 object Base
diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
index e42eace474..2446a1c702 100644
--- a/tests/qapi-schema/doc-good.texi
+++ b/tests/qapi-schema/doc-good.texi
@@ -83,6 +83,8 @@ Examples:
 @b{Values:}
 @table @asis
 @item @code{one}
+@b{If:} @code{defined(IFENUM)}
+
 The @emph{one} @{and only@}
 @item @code{two}
 Not documented
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 24/27] qapi: add 'If:' condition to struct members documentation

[Marc-André Lureau]

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/doc.py             | 4 +++-
 tests/qapi-schema/doc-good.json | 3 ++-
 tests/qapi-schema/doc-good.out  | 1 +
 tests/qapi-schema/doc-good.texi | 2 ++
 4 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 196deea11d..56dc98c068 100755
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -138,9 +138,11 @@ def texi_member(member, suffix=''):
     """Format a table of members item for an object type member"""
     typ = member.type.doc_type()
     membertype = ': ' + typ if typ else ''
-    return '@item @code{%s%s}%s%s\n' % (
+    return '@item @code{%s%s}%s%s%s\n' % (
         member.name, membertype,
         ' (optional)' if member.optional else '',
+        '\n@b{If:} @code{%s}\n' %
+        ', '.join(member.ifcond) if member.ifcond else '',
         suffix)
 
 
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index c7fe08c530..158443b1a3 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -72,7 +72,8 @@
 #
 # Another paragraph (but no @var: line)
 ##
-{ 'struct': 'Variant1', 'data': { 'var1': 'str' } }
+{ 'struct': 'Variant1',
+  'data': { 'var1': { 'type': 'str', 'if': 'defined(IFSTR)' } } }
 
 ##
 # @Variant2:
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index a05535b69b..c310b47be2 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -18,6 +18,7 @@ object Base
     member base1: Enum optional=False
 object Variant1
     member var1: str optional=False
+        if ['defined(IFSTR)']
 object Variant2
 object Object
     base Base
diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
index 2446a1c702..255eed9a30 100644
--- a/tests/qapi-schema/doc-good.texi
+++ b/tests/qapi-schema/doc-good.texi
@@ -120,6 +120,8 @@ Another paragraph (but no @code{var}: line)
 @b{Members:}
 @table @asis
 @item @code{var1: string}
+@b{If:} @code{defined(IFSTR)}
+
 Not documented
 @end table
 
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 25/27] qapi: add condition to variants documentation

[Marc-André Lureau]

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 scripts/qapi/doc.py             | 5 +++--
 tests/qapi-schema/doc-good.json | 4 ++--
 tests/qapi-schema/doc-good.out  | 3 +++
 tests/qapi-schema/doc-good.texi | 4 ++--
 4 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 56dc98c068..e9dfdc7d23 100755
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -165,8 +165,9 @@ def texi_members(doc, what, base, variants, member_func):
         items += '@item The members of @code{%s}\n' % base.doc_type()
     if variants:
         for v in variants.variants:
-            when = ' when @code{%s} is @t{"%s"}' % (
-                variants.tag_member.name, v.name)
+            when = ' when @code{%s} is @t{"%s"}%s' % (
+                variants.tag_member.name, v.name,
+                ' (@b{If:} @code{%s})' % v.ifcond if v.ifcond else '')
             if v.type.is_implicit():
                 assert not v.type.base and not v.type.variants
                 for m in v.type.local_members:
diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
index 158443b1a3..afe46d93f0 100644
--- a/tests/qapi-schema/doc-good.json
+++ b/tests/qapi-schema/doc-good.json
@@ -86,13 +86,13 @@
 { 'union': 'Object',
   'base': 'Base',
   'discriminator': 'base1',
-  'data': { 'one': 'Variant1', 'two': 'Variant2' } }
+  'data': { 'one': 'Variant1', 'two': { 'type': 'Variant2', 'if': 'IFTWO' } } }
 
 ##
 # @SugaredUnion:
 ##
 { 'union': 'SugaredUnion',
-  'data': { 'one': 'Variant1', 'two': 'Variant2' } }
+  'data': { 'one': 'Variant1', 'two': { 'type': 'Variant2', 'if': 'IFTWO' } } }
 
 ##
 # == Another subsection
diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
index c310b47be2..ed647b82be 100644
--- a/tests/qapi-schema/doc-good.out
+++ b/tests/qapi-schema/doc-good.out
@@ -25,6 +25,7 @@ object Object
     tag base1
     case one: Variant1
     case two: Variant2
+        if ['IFTWO']
 object q_obj_Variant1-wrapper
     member data: Variant1 optional=False
 object q_obj_Variant2-wrapper
@@ -32,11 +33,13 @@ object q_obj_Variant2-wrapper
 enum SugaredUnionKind
     member one
     member two
+        if ['IFTWO']
 object SugaredUnion
     member type: SugaredUnionKind optional=False
     tag type
     case one: q_obj_Variant1-wrapper
     case two: q_obj_Variant2-wrapper
+        if ['IFTWO']
 object q_obj_cmd-arg
     member arg1: int optional=False
     member arg2: str optional=True
diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
index 255eed9a30..6e226a6231 100644
--- a/tests/qapi-schema/doc-good.texi
+++ b/tests/qapi-schema/doc-good.texi
@@ -145,7 +145,7 @@ Not documented
 @table @asis
 @item The members of @code{Base}
 @item The members of @code{Variant1} when @code{base1} is @t{"one"}
-@item The members of @code{Variant2} when @code{base1} is @t{"two"}
+@item The members of @code{Variant2} when @code{base1} is @t{"two"} (@b{If:} @code{['IFTWO']})
 @end table
 
 @end deftp
@@ -161,7 +161,7 @@ Not documented
 @item @code{type}
 One of @t{"one"}, @t{"two"}
 @item @code{data: Variant1} when @code{type} is @t{"one"}
-@item @code{data: Variant2} when @code{type} is @t{"two"}
+@item @code{data: Variant2} when @code{type} is @t{"two"} (@b{If:} @code{['IFTWO']})
 @end table
 
 @end deftp
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 26/27] qapi: add more conditions to SPICE

[Marc-André Lureau]

Now that member can be made conditional, let's make SPICE chardev
conditional:

* spiceport, spicevmc

  Before and after the patch for !CONFIG_SPICE, the error is the
  same ('spiceport' is not a valid char driver name).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 qapi/char.json | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/qapi/char.json b/qapi/char.json
index b7b2a05766..54e3b970f6 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -319,8 +319,8 @@
 # Since: 1.5
 ##
 { 'struct': 'ChardevSpiceChannel', 'data': { 'type'  : 'str' },
-  'base': 'ChardevCommon' }
-# TODO: 'if': 'defined(CONFIG_SPICE)'
+  'base': 'ChardevCommon',
+  'if': 'defined(CONFIG_SPICE)' }
 
 ##
 # @ChardevSpicePort:
@@ -332,8 +332,8 @@
 # Since: 1.5
 ##
 { 'struct': 'ChardevSpicePort', 'data': { 'fqdn'  : 'str' },
-  'base': 'ChardevCommon' }
-# TODO: 'if': 'defined(CONFIG_SPICE)'
+  'base': 'ChardevCommon',
+  'if': 'defined(CONFIG_SPICE)' }
 
 ##
 # @ChardevVC:
@@ -387,10 +387,10 @@
                                        'testdev': 'ChardevCommon',
                                        'stdio'  : 'ChardevStdio',
                                        'console': 'ChardevCommon',
-                                       'spicevmc': 'ChardevSpiceChannel',
-# TODO: { 'type': 'ChardevSpiceChannel', 'if': 'defined(CONFIG_SPICE)' },
-                                       'spiceport': 'ChardevSpicePort',
-# TODO: { 'type': 'ChardevSpicePort', 'if': 'defined(CONFIG_SPICE)' },
+                                       'spicevmc': { 'type': 'ChardevSpiceChannel',
+                                                     'if': 'defined(CONFIG_SPICE)' },
+                                       'spiceport': { 'type': 'ChardevSpicePort',
+                                                      'if': 'defined(CONFIG_SPICE)' },
                                        'vc'     : 'ChardevVC',
                                        'ringbuf': 'ChardevRingbuf',
                                        # next one is just for compatibility
-- 
2.18.0.rc1

[Qemu-devel] [PATCH v6 27/27] qapi: add conditions to REPLICATION type/commands on the schema

[Marc-André Lureau]

Add #if defined(CONFIG_REPLICATION) in generated code, and adjust the
code accordingly.

Made conditional:

* xen-set-replication, query-xen-replication-status,
  xen-colo-do-checkpoint

  Before the patch, we first register the commands unconditionally in
  generated code (requires a stub), then conditionally unregister in
  qmp_unregister_commands_hack().

  Afterwards, we register only when CONFIG_REPLICATION.  The command
  fails exactly the same, with CommandNotFound.

  Improvement, because now query-qmp-schema is accurate, and we're one
  step closer to killing qmp_unregister_commands_hack().

* enum BlockdevDriver value "replication" in command blockdev-add

* BlockdevOptions variant @replication

And related structures.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 qapi/block-core.json | 13 +++++++++----
 qapi/migration.json  | 12 ++++++++----
 migration/colo.c     | 16 ++++------------
 monitor.c            |  5 -----
 4 files changed, 21 insertions(+), 25 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 38b31250f9..6baeac23a7 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2542,7 +2542,9 @@
             'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', 'gluster',
             'host_cdrom', 'host_device', 'http', 'https', 'iscsi', 'luks',
             'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qcow',
-            'qcow2', 'qed', 'quorum', 'raw', 'rbd', 'replication', 'sheepdog',
+            'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+            { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
+            'sheepdog',
             'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxhs' ] }
 
 ##
@@ -3292,7 +3294,8 @@
 #
 # Since: 2.9
 ##
-{ 'enum' : 'ReplicationMode', 'data' : [ 'primary', 'secondary' ] }
+{ 'enum' : 'ReplicationMode', 'data' : [ 'primary', 'secondary' ],
+  'if': 'defined(CONFIG_REPLICATION)' }
 
 ##
 # @BlockdevOptionsReplication:
@@ -3310,7 +3313,8 @@
 { 'struct': 'BlockdevOptionsReplication',
   'base': 'BlockdevOptionsGenericFormat',
   'data': { 'mode': 'ReplicationMode',
-            '*top-id': 'str' } }
+            '*top-id': 'str' },
+  'if': 'defined(CONFIG_REPLICATION)' }
 
 ##
 # @NFSTransport:
@@ -3616,7 +3620,8 @@
       'quorum':     'BlockdevOptionsQuorum',
       'raw':        'BlockdevOptionsRaw',
       'rbd':        'BlockdevOptionsRbd',
-      'replication':'BlockdevOptionsReplication',
+      'replication': { 'type': 'BlockdevOptionsReplication',
+                       'if': 'defined(CONFIG_REPLICATION)' },
       'sheepdog':   'BlockdevOptionsSheepdog',
       'ssh':        'BlockdevOptionsSsh',
       'throttle':   'BlockdevOptionsThrottle',
diff --git a/qapi/migration.json b/qapi/migration.json
index 186e8a7303..554231a2f4 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1163,7 +1163,8 @@
 # Since: 2.9
 ##
 { 'command': 'xen-set-replication',
-  'data': { 'enable': 'bool', 'primary': 'bool', '*failover' : 'bool' } }
+  'data': { 'enable': 'bool', 'primary': 'bool', '*failover' : 'bool' },
+  'if': 'defined(CONFIG_REPLICATION)' }
 
 ##
 # @ReplicationStatus:
@@ -1178,7 +1179,8 @@
 # Since: 2.9
 ##
 { 'struct': 'ReplicationStatus',
-  'data': { 'error': 'bool', '*desc': 'str' } }
+  'data': { 'error': 'bool', '*desc': 'str' },
+  'if': 'defined(CONFIG_REPLICATION)' }
 
 ##
 # @query-xen-replication-status:
@@ -1195,7 +1197,8 @@
 # Since: 2.9
 ##
 { 'command': 'query-xen-replication-status',
-  'returns': 'ReplicationStatus' }
+  'returns': 'ReplicationStatus',
+  'if': 'defined(CONFIG_REPLICATION)' }
 
 ##
 # @xen-colo-do-checkpoint:
@@ -1211,7 +1214,8 @@
 #
 # Since: 2.9
 ##
-{ 'command': 'xen-colo-do-checkpoint' }
+{ 'command': 'xen-colo-do-checkpoint',
+  'if': 'defined(CONFIG_REPLICATION)' }
 
 ##
 # @migrate-recover:
diff --git a/migration/colo.c b/migration/colo.c
index 4381067ed4..2329e78e7f 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -24,7 +24,9 @@
 #include "trace.h"
 #include "qemu/error-report.h"
 #include "migration/failover.h"
+#ifdef CONFIG_REPLICATION
 #include "replication.h"
+#endif
 
 static bool vmstate_loading;
 
@@ -148,11 +150,11 @@ void colo_do_failover(MigrationState *s)
     }
 }
 
+#ifdef CONFIG_REPLICATION
 void qmp_xen_set_replication(bool enable, bool primary,
                              bool has_failover, bool failover,
                              Error **errp)
 {
-#ifdef CONFIG_REPLICATION
     ReplicationMode mode = primary ?
                            REPLICATION_MODE_PRIMARY :
                            REPLICATION_MODE_SECONDARY;
@@ -171,14 +173,10 @@ void qmp_xen_set_replication(bool enable, bool primary,
         }
         replication_stop_all(failover, failover ? NULL : errp);
     }
-#else
-    abort();
-#endif
 }
 
 ReplicationStatus *qmp_query_xen_replication_status(Error **errp)
 {
-#ifdef CONFIG_REPLICATION
     Error *err = NULL;
     ReplicationStatus *s = g_new0(ReplicationStatus, 1);
 
@@ -193,19 +191,13 @@ ReplicationStatus *qmp_query_xen_replication_status(Error **errp)
 
     error_free(err);
     return s;
-#else
-    abort();
-#endif
 }
 
 void qmp_xen_colo_do_checkpoint(Error **errp)
 {
-#ifdef CONFIG_REPLICATION
     replication_do_checkpoint_all(errp);
-#else
-    abort();
-#endif
 }
+#endif
 
 static void colo_send_message(QEMUFile *f, COLOMessage msg,
                               Error **errp)
diff --git a/monitor.c b/monitor.c
index 3c9c97b73f..c3414377f9 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1194,11 +1194,6 @@ static void qmp_query_qmp_schema(QDict *qdict, QObject **ret_data,
  */
 static void qmp_unregister_commands_hack(void)
 {
-#ifndef CONFIG_REPLICATION
-    qmp_unregister_command(&qmp_commands, "xen-set-replication");
-    qmp_unregister_command(&qmp_commands, "query-xen-replication-status");
-    qmp_unregister_command(&qmp_commands, "xen-colo-do-checkpoint");
-#endif
 #ifndef TARGET_I386
     qmp_unregister_command(&qmp_commands, "rtc-reset-reinjection");
     qmp_unregister_command(&qmp_commands, "query-sev");
-- 
2.18.0.rc1

Re: [Qemu-devel] [PATCH v6 03/27] tests: change /0.15/* tests to /qmp/*

[Thomas Huth]

On 06.07.2018 12:57, Marc-André Lureau wrote:
> Presumably 0.15 was the version it was first introduced, but
> qmp keeps evolving. There is no point in having that version
> as test prefix, 'qmp' makes more sense here.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Message-Id: <20180326150916.9602-12-marcandre.lureau@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  tests/test-qmp-cmds.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c
> index ba41a6161e..ab414fa0c9 100644
> --- a/tests/test-qmp-cmds.c
> +++ b/tests/test-qmp-cmds.c
> @@ -286,11 +286,11 @@ int main(int argc, char **argv)
>  {
>      g_test_init(&argc, &argv, NULL);
>  
> -    g_test_add_func("/0.15/dispatch_cmd", test_dispatch_cmd);
> -    g_test_add_func("/0.15/dispatch_cmd_failure", test_dispatch_cmd_failure);
> -    g_test_add_func("/0.15/dispatch_cmd_io", test_dispatch_cmd_io);
> -    g_test_add_func("/0.15/dealloc_types", test_dealloc_types);
> -    g_test_add_func("/0.15/dealloc_partial", test_dealloc_partial);
> +    g_test_add_func("/qmp/dispatch_cmd", test_dispatch_cmd);
> +    g_test_add_func("/qmp/dispatch_cmd_failure", test_dispatch_cmd_failure);
> +    g_test_add_func("/qmp/dispatch_cmd_io", test_dispatch_cmd_io);
> +    g_test_add_func("/qmp/dealloc_types", test_dealloc_types);
> +    g_test_add_func("/qmp/dealloc_partial", test_dealloc_partial);
>  
>      test_qmp_init_marshal(&qmp_commands);
>      g_test_run();
> 

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [Qemu-devel] [PATCH v6 02/27] json: remove useless return value from lexer/parser

[Thomas Huth]

On 06.07.2018 12:57, Marc-André Lureau wrote:
> The lexer always returns 0 when char feeding. Furthermore, none of the
> caller care about the return value.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Message-Id: <20180326150916.9602-10-marcandre.lureau@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  include/qapi/qmp/json-lexer.h    |  4 ++--
>  include/qapi/qmp/json-streamer.h |  4 ++--
>  qobject/json-lexer.c             | 23 ++++++++---------------
>  qobject/json-streamer.c          |  8 ++++----
>  4 files changed, 16 insertions(+), 23 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [Qemu-devel] [PATCH v6 00/27] qapi: add #if pre-processor conditions to generated code (part 2)

[Marc-André Lureau]

Hi

On Fri, Jul 6, 2018 at 2:59 PM Marc-André Lureau
<marcandre.lureau@redhat.com> wrote:
>
> Hi,
>
> This is the second part of the "add #if pre-processor conditions to
> generated code" series, adding schema member conditions (roughly
> 16-38/49).
>
> Members can be exploded as dictionnary with 'type'/'if' keys:
>
> { 'struct': 'TestIfStruct', 'data':
>   { 'foo': 'int',
>     'bar': { 'type': 'int', 'if': 'defined(TEST_IF_STRUCT_BAR)'} } }
>
> Enum values can be exploded as dictionnary with 'type'/'if' keys:
>
> { 'enum': 'TestIfEnum', 'data':
>   [ 'foo',
>     { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ] }
>
> v6:
> - subset of v5 series: add schema member conditions
> - split "qapi: factor out check_known_keys()", error report improvements
> - add a patch to "qapi: include osdep.h in type headers" to avoid
>   potential build configuration change issues
> - rebased, on top of Markus's qapi-next branch (first 4 patches,
>   included for patchew testing)
>
> Marc-André Lureau (26):
>   qmp-shell: learn to send commands with quoted arguments
>   json: remove useless return value from lexer/parser
>   tests: change /0.15/* tests to /qmp/*
>   qapi: include osdep.h in type headers
>   qapi: do not define enumeration value explicitly
>   qapi: rename QAPISchemaEnumType.values to .members
>   qapi: change enum visitor and gen_enum* to take QAPISchemaMember
>   tests: print enum type members more like object type members
>   qapi: factor out checking for keys
>   qapi: improve reporting of unknown or missing keys
>   qapi: add a dictionnary form with 'name' key for enum members
>   qapi: add 'if' to enum members
>   qapi-event: add 'if' condition to implicit event enum
>   qapi: rename allow_dict to allow_implicit
>   qapi: add a dictionary form with 'type' key for members
>   qapi: add 'if' to implicit struct members
>   qapi: add an error in case a discriminator is conditionnal
>   qapi: add 'if' on union members
>   qapi: add 'if' to alternate members
>   qapi: add #if conditions to generated code members
>   docs: document schema configuration
>   qapi: add 'If:' condition to enum values documentation
>   qapi: add 'If:' condition to struct members documentation
>   qapi: add condition to variants documentation
>   qapi: add more conditions to SPICE
>   qapi: add conditions to REPLICATION type/commands on the schema
>
> Markus Armbruster (1):
>   qapi: Fix some pycodestyle-3 complaints

Rebased series updated on github branch:
https://github.com/elmarco/qemu/commits/qapi-if

(mostly unchanged, except patch "add #if conditions to generated code
members" which had to accommodate with Eric's introspection comment
patch)

thanks


>  qapi/block-core.json                          |  13 +-
>  qapi/char.json                                |  16 +-
>  qapi/migration.json                           |  12 +-
>  scripts/qapi/common.py                        | 256 ++++++++++++------
>  scripts/qapi/doc.py                           |  16 +-
>  scripts/qapi/events.py                        |   2 +-
>  scripts/qapi/introspect.py                    |  16 +-
>  scripts/qapi/types.py                         |  11 +-
>  scripts/qapi/visit.py                         |  10 +-
>  include/qapi/qmp/json-lexer.h                 |   4 +-
>  include/qapi/qmp/json-streamer.h              |   4 +-
>  migration/colo.c                              |  16 +-
>  monitor.c                                     |   5 -
>  qobject/json-lexer.c                          |  23 +-
>  qobject/json-streamer.c                       |   8 +-
>  tests/test-qmp-cmds.c                         |  10 +-
>  docs/devel/qapi-code-gen.txt                  |  19 ++
>  scripts/qmp/qmp-shell                         |   3 +-
>  tests/Makefile.include                        |   8 +-
>  tests/qapi-schema/alternate-base.err          |   1 +
>  tests/qapi-schema/alternate-invalid-dict.err  |   1 +
>  ...ember.exit => alternate-invalid-dict.exit} |   0
>  tests/qapi-schema/alternate-invalid-dict.json |   4 +
>  ...-member.out => alternate-invalid-dict.out} |   0
>  tests/qapi-schema/comments.out                |  14 +-
>  tests/qapi-schema/doc-bad-section.out         |  13 +-
>  tests/qapi-schema/doc-good.json               |  11 +-
>  tests/qapi-schema/doc-good.out                |  22 +-
>  tests/qapi-schema/doc-good.texi               |   8 +-
>  tests/qapi-schema/double-type.err             |   1 +
>  tests/qapi-schema/empty.out                   |   9 +-
>  tests/qapi-schema/enum-bad-member.err         |   1 +
>  tests/qapi-schema/enum-bad-member.exit        |   1 +
>  tests/qapi-schema/enum-bad-member.json        |   2 +
>  tests/qapi-schema/enum-bad-member.out         |   0
>  .../qapi-schema/enum-dict-member-unknown.err  |   2 +
>  .../qapi-schema/enum-dict-member-unknown.exit |   1 +
>  .../qapi-schema/enum-dict-member-unknown.json |   2 +
>  .../qapi-schema/enum-dict-member-unknown.out  |   0
>  tests/qapi-schema/enum-dict-member.err        |   1 -
>  tests/qapi-schema/enum-dict-member.json       |   2 -
>  tests/qapi-schema/enum-if-invalid.err         |   1 +
>  tests/qapi-schema/enum-if-invalid.exit        |   1 +
>  tests/qapi-schema/enum-if-invalid.json        |   3 +
>  tests/qapi-schema/enum-if-invalid.out         |   0
>  tests/qapi-schema/enum-missing-data.err       |   2 +-
>  tests/qapi-schema/event-case.out              |   9 +-
>  tests/qapi-schema/event-nest-struct.err       |   2 +-
>  tests/qapi-schema/flat-union-inline.err       |   2 +-
>  .../flat-union-invalid-if-discriminator.err   |   1 +
>  .../flat-union-invalid-if-discriminator.exit  |   1 +
>  .../flat-union-invalid-if-discriminator.json  |  17 ++
>  .../flat-union-invalid-if-discriminator.out   |   0
>  tests/qapi-schema/ident-with-escape.out       |   9 +-
>  tests/qapi-schema/include-relpath.out         |  14 +-
>  tests/qapi-schema/include-repetition.out      |  14 +-
>  tests/qapi-schema/include-simple.out          |  14 +-
>  tests/qapi-schema/indented-expr.out           |   9 +-
>  tests/qapi-schema/nested-struct-data.err      |   2 +-
>  tests/qapi-schema/qapi-schema-test.json       |  38 ++-
>  tests/qapi-schema/qapi-schema-test.out        |  76 +++++-
>  .../struct-member-invalid-dict.err            |   1 +
>  .../struct-member-invalid-dict.exit           |   1 +
>  .../struct-member-invalid-dict.json           |   3 +
>  .../struct-member-invalid-dict.out            |   0
>  tests/qapi-schema/test-qapi.py                |  22 +-
>  .../qapi-schema/union-branch-invalid-dict.err |   1 +
>  .../union-branch-invalid-dict.exit            |   1 +
>  .../union-branch-invalid-dict.json            |   4 +
>  .../qapi-schema/union-branch-invalid-dict.out |   0
>  tests/qapi-schema/unknown-expr-key.err        |   3 +-
>  tests/qapi-schema/unknown-expr-key.json       |   2 +-
>  72 files changed, 574 insertions(+), 227 deletions(-)
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.err
>  rename tests/qapi-schema/{enum-dict-member.exit => alternate-invalid-dict.exit} (100%)
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.json
>  rename tests/qapi-schema/{enum-dict-member.out => alternate-invalid-dict.out} (100%)
>  create mode 100644 tests/qapi-schema/enum-bad-member.err
>  create mode 100644 tests/qapi-schema/enum-bad-member.exit
>  create mode 100644 tests/qapi-schema/enum-bad-member.json
>  create mode 100644 tests/qapi-schema/enum-bad-member.out
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.err
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.exit
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.json
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.out
>  delete mode 100644 tests/qapi-schema/enum-dict-member.err
>  delete mode 100644 tests/qapi-schema/enum-dict-member.json
>  create mode 100644 tests/qapi-schema/enum-if-invalid.err
>  create mode 100644 tests/qapi-schema/enum-if-invalid.exit
>  create mode 100644 tests/qapi-schema/enum-if-invalid.json
>  create mode 100644 tests/qapi-schema/enum-if-invalid.out
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.err
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.exit
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.json
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.out
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.err
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.exit
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.json
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.out
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.err
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.exit
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.json
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.out
>
> --
> 2.18.0.rc1
>
>


-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v6 00/27] qapi: add #if pre-processor conditions to generated code (part 2)

[Marc-André Lureau]

Hi

On Fri, Jul 6, 2018 at 2:59 PM Marc-André Lureau
<marcandre.lureau@redhat.com> wrote:
>
> Hi,
>
> This is the second part of the "add #if pre-processor conditions to
> generated code" series, adding schema member conditions (roughly
> 16-38/49).
>
> Members can be exploded as dictionnary with 'type'/'if' keys:
>
> { 'struct': 'TestIfStruct', 'data':
>   { 'foo': 'int',
>     'bar': { 'type': 'int', 'if': 'defined(TEST_IF_STRUCT_BAR)'} } }
>
> Enum values can be exploded as dictionnary with 'type'/'if' keys:
>
> { 'enum': 'TestIfEnum', 'data':
>   [ 'foo',
>     { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ] }
>
> v6:
> - subset of v5 series: add schema member conditions
> - split "qapi: factor out check_known_keys()", error report improvements
> - add a patch to "qapi: include osdep.h in type headers" to avoid
>   potential build configuration change issues
> - rebased, on top of Markus's qapi-next branch (first 4 patches,
>   included for patchew testing)

Rebased series updated on github branch:
https://github.com/elmarco/qemu/commits/qapi-if

(minor conflicts and changes, thus not resending, still waiting for
review of v6)

thanks

>
> Marc-André Lureau (26):
>   qmp-shell: learn to send commands with quoted arguments
>   json: remove useless return value from lexer/parser
>   tests: change /0.15/* tests to /qmp/*
>   qapi: include osdep.h in type headers
>   qapi: do not define enumeration value explicitly
>   qapi: rename QAPISchemaEnumType.values to .members
>   qapi: change enum visitor and gen_enum* to take QAPISchemaMember
>   tests: print enum type members more like object type members
>   qapi: factor out checking for keys
>   qapi: improve reporting of unknown or missing keys
>   qapi: add a dictionnary form with 'name' key for enum members
>   qapi: add 'if' to enum members
>   qapi-event: add 'if' condition to implicit event enum
>   qapi: rename allow_dict to allow_implicit
>   qapi: add a dictionary form with 'type' key for members
>   qapi: add 'if' to implicit struct members
>   qapi: add an error in case a discriminator is conditionnal
>   qapi: add 'if' on union members
>   qapi: add 'if' to alternate members
>   qapi: add #if conditions to generated code members
>   docs: document schema configuration
>   qapi: add 'If:' condition to enum values documentation
>   qapi: add 'If:' condition to struct members documentation
>   qapi: add condition to variants documentation
>   qapi: add more conditions to SPICE
>   qapi: add conditions to REPLICATION type/commands on the schema
>
> Markus Armbruster (1):
>   qapi: Fix some pycodestyle-3 complaints
>
>  qapi/block-core.json                          |  13 +-
>  qapi/char.json                                |  16 +-
>  qapi/migration.json                           |  12 +-
>  scripts/qapi/common.py                        | 256 ++++++++++++------
>  scripts/qapi/doc.py                           |  16 +-
>  scripts/qapi/events.py                        |   2 +-
>  scripts/qapi/introspect.py                    |  16 +-
>  scripts/qapi/types.py                         |  11 +-
>  scripts/qapi/visit.py                         |  10 +-
>  include/qapi/qmp/json-lexer.h                 |   4 +-
>  include/qapi/qmp/json-streamer.h              |   4 +-
>  migration/colo.c                              |  16 +-
>  monitor.c                                     |   5 -
>  qobject/json-lexer.c                          |  23 +-
>  qobject/json-streamer.c                       |   8 +-
>  tests/test-qmp-cmds.c                         |  10 +-
>  docs/devel/qapi-code-gen.txt                  |  19 ++
>  scripts/qmp/qmp-shell                         |   3 +-
>  tests/Makefile.include                        |   8 +-
>  tests/qapi-schema/alternate-base.err          |   1 +
>  tests/qapi-schema/alternate-invalid-dict.err  |   1 +
>  ...ember.exit => alternate-invalid-dict.exit} |   0
>  tests/qapi-schema/alternate-invalid-dict.json |   4 +
>  ...-member.out => alternate-invalid-dict.out} |   0
>  tests/qapi-schema/comments.out                |  14 +-
>  tests/qapi-schema/doc-bad-section.out         |  13 +-
>  tests/qapi-schema/doc-good.json               |  11 +-
>  tests/qapi-schema/doc-good.out                |  22 +-
>  tests/qapi-schema/doc-good.texi               |   8 +-
>  tests/qapi-schema/double-type.err             |   1 +
>  tests/qapi-schema/empty.out                   |   9 +-
>  tests/qapi-schema/enum-bad-member.err         |   1 +
>  tests/qapi-schema/enum-bad-member.exit        |   1 +
>  tests/qapi-schema/enum-bad-member.json        |   2 +
>  tests/qapi-schema/enum-bad-member.out         |   0
>  .../qapi-schema/enum-dict-member-unknown.err  |   2 +
>  .../qapi-schema/enum-dict-member-unknown.exit |   1 +
>  .../qapi-schema/enum-dict-member-unknown.json |   2 +
>  .../qapi-schema/enum-dict-member-unknown.out  |   0
>  tests/qapi-schema/enum-dict-member.err        |   1 -
>  tests/qapi-schema/enum-dict-member.json       |   2 -
>  tests/qapi-schema/enum-if-invalid.err         |   1 +
>  tests/qapi-schema/enum-if-invalid.exit        |   1 +
>  tests/qapi-schema/enum-if-invalid.json        |   3 +
>  tests/qapi-schema/enum-if-invalid.out         |   0
>  tests/qapi-schema/enum-missing-data.err       |   2 +-
>  tests/qapi-schema/event-case.out              |   9 +-
>  tests/qapi-schema/event-nest-struct.err       |   2 +-
>  tests/qapi-schema/flat-union-inline.err       |   2 +-
>  .../flat-union-invalid-if-discriminator.err   |   1 +
>  .../flat-union-invalid-if-discriminator.exit  |   1 +
>  .../flat-union-invalid-if-discriminator.json  |  17 ++
>  .../flat-union-invalid-if-discriminator.out   |   0
>  tests/qapi-schema/ident-with-escape.out       |   9 +-
>  tests/qapi-schema/include-relpath.out         |  14 +-
>  tests/qapi-schema/include-repetition.out      |  14 +-
>  tests/qapi-schema/include-simple.out          |  14 +-
>  tests/qapi-schema/indented-expr.out           |   9 +-
>  tests/qapi-schema/nested-struct-data.err      |   2 +-
>  tests/qapi-schema/qapi-schema-test.json       |  38 ++-
>  tests/qapi-schema/qapi-schema-test.out        |  76 +++++-
>  .../struct-member-invalid-dict.err            |   1 +
>  .../struct-member-invalid-dict.exit           |   1 +
>  .../struct-member-invalid-dict.json           |   3 +
>  .../struct-member-invalid-dict.out            |   0
>  tests/qapi-schema/test-qapi.py                |  22 +-
>  .../qapi-schema/union-branch-invalid-dict.err |   1 +
>  .../union-branch-invalid-dict.exit            |   1 +
>  .../union-branch-invalid-dict.json            |   4 +
>  .../qapi-schema/union-branch-invalid-dict.out |   0
>  tests/qapi-schema/unknown-expr-key.err        |   3 +-
>  tests/qapi-schema/unknown-expr-key.json       |   2 +-
>  72 files changed, 574 insertions(+), 227 deletions(-)
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.err
>  rename tests/qapi-schema/{enum-dict-member.exit => alternate-invalid-dict.exit} (100%)
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.json
>  rename tests/qapi-schema/{enum-dict-member.out => alternate-invalid-dict.out} (100%)
>  create mode 100644 tests/qapi-schema/enum-bad-member.err
>  create mode 100644 tests/qapi-schema/enum-bad-member.exit
>  create mode 100644 tests/qapi-schema/enum-bad-member.json
>  create mode 100644 tests/qapi-schema/enum-bad-member.out
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.err
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.exit
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.json
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.out
>  delete mode 100644 tests/qapi-schema/enum-dict-member.err
>  delete mode 100644 tests/qapi-schema/enum-dict-member.json
>  create mode 100644 tests/qapi-schema/enum-if-invalid.err
>  create mode 100644 tests/qapi-schema/enum-if-invalid.exit
>  create mode 100644 tests/qapi-schema/enum-if-invalid.json
>  create mode 100644 tests/qapi-schema/enum-if-invalid.out
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.err
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.exit
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.json
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.out
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.err
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.exit
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.json
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.out
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.err
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.exit
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.json
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.out
>
> --
> 2.18.0.rc1
>
>


-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v6 05/27] qapi: include osdep.h in type headers

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Now that the schema can be configured, it is crucial that all types
> are configured the same. Make sure config-host.h is included, so
> build-sys tracks the dependency and rebuilds the types, by including
> osdep.h first.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/types.py | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
> index fd7808103c..91f87d0b8f 100644
> --- a/scripts/qapi/types.py
> +++ b/scripts/qapi/types.py
> @@ -201,6 +201,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
>  ''',
>                                        types=types, visit=visit))
>          self._genh.preamble_add(mcgen('''
> +#include "qemu/osdep.h"
>  #include "qapi/qapi-builtin-types.h"
>  '''))

No.  Every .c must include qemu/osdep.h first.  No .h may include it.
We clean this up periodically.  scripts/clean-includes can help with
that.  We currently have a few offenders in the tree.

Re: [Qemu-devel] [PATCH v6 05/27] qapi: include osdep.h in type headers

[Marc-André Lureau]

Hi

On Tue, Dec 4, 2018 at 7:23 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Now that the schema can be configured, it is crucial that all types
> > are configured the same. Make sure config-host.h is included, so
> > build-sys tracks the dependency and rebuilds the types, by including
> > osdep.h first.
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/types.py | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
> > index fd7808103c..91f87d0b8f 100644
> > --- a/scripts/qapi/types.py
> > +++ b/scripts/qapi/types.py
> > @@ -201,6 +201,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
> >  ''',
> >                                        types=types, visit=visit))
> >          self._genh.preamble_add(mcgen('''
> > +#include "qemu/osdep.h"
> >  #include "qapi/qapi-builtin-types.h"
> >  '''))
>
> No.  Every .c must include qemu/osdep.h first.  No .h may include it.
> We clean this up periodically.  scripts/clean-includes can help with
> that.  We currently have a few offenders in the tree.

Ok, I don't know the reason an internal header couldn't include osdep,
could you explain?

I think we can replace osdep.h by config-host.h for the same result,
I'd have to check.

Re: [Qemu-devel] [PATCH v6 05/27] qapi: include osdep.h in type headers

[Eric Blake]

On 12/4/18 9:32 AM, Marc-André Lureau wrote:

>>> +++ b/scripts/qapi/types.py
>>> @@ -201,6 +201,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
>>>   ''',
>>>                                         types=types, visit=visit))
>>>           self._genh.preamble_add(mcgen('''
>>> +#include "qemu/osdep.h"
>>>   #include "qapi/qapi-builtin-types.h"
>>>   '''))
>>
>> No.  Every .c must include qemu/osdep.h first.  No .h may include it.
>> We clean this up periodically.  scripts/clean-includes can help with
>> that.  We currently have a few offenders in the tree.
> 
> Ok, I don't know the reason an internal header couldn't include osdep,
> could you explain?

Per HACKING:

>> Do not include "qemu/osdep.h" from header files since the .c file will have
>> already included it.


That is, if we follow the rule that every .c file included osdep.h 
before anything else, then by the time any other internal .h is included 
(necessarily second or later) from a .c, then osdep.h is already active, 
so no .h needs to reproduce the effort that has already occurred.

> 
> I think we can replace osdep.h by config-host.h for the same result,
> I'd have to check.

osdep.h includes config-host.h, so you don't need that either.

What are you really trying to accomplish by including a header from 
another header?  And why isn't it already happening by our coding rule 
that all .c files include osdep.h first?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v6 05/27] qapi: include osdep.h in type headers

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Hi
>
> On Tue, Dec 4, 2018 at 7:23 PM Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>>
>> > Now that the schema can be configured, it is crucial that all types
>> > are configured the same. Make sure config-host.h is included, so
>> > build-sys tracks the dependency and rebuilds the types, by including
>> > osdep.h first.
>> >
>> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> > ---
>> >  scripts/qapi/types.py | 1 +
>> >  1 file changed, 1 insertion(+)
>> >
>> > diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
>> > index fd7808103c..91f87d0b8f 100644
>> > --- a/scripts/qapi/types.py
>> > +++ b/scripts/qapi/types.py
>> > @@ -201,6 +201,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
>> >  ''',
>> >                                        types=types, visit=visit))
>> >          self._genh.preamble_add(mcgen('''
>> > +#include "qemu/osdep.h"
>> >  #include "qapi/qapi-builtin-types.h"
>> >  '''))
>>
>> No.  Every .c must include qemu/osdep.h first.  No .h may include it.
>> We clean this up periodically.  scripts/clean-includes can help with
>> that.  We currently have a few offenders in the tree.
>
> Ok, I don't know the reason an internal header couldn't include osdep,
> could you explain?
>
> I think we can replace osdep.h by config-host.h for the same result,
> I'd have to check.

Quote ./HACKING:

1.2. Include directives

Order include directives as follows:

#include "qemu/osdep.h"  /* Always first... */
#include <...>           /* then system headers... */
#include "..."           /* and finally QEMU headers. */

The "qemu/osdep.h" header contains preprocessor macros that affect the behavior
of core system headers like <stdint.h>.  It must be the first include so that
core system headers included by external libraries get the preprocessor macros
that QEMU depends on.

Do not include "qemu/osdep.h" from header files since the .c file will have
already included it.

Re: [Qemu-devel] [PATCH v6 06/27] qapi: do not define enumeration value explicitly

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> The C standard has the initial value at 0 and the subsequent values
> incremented by 1. No need to set this explicitely.
>
> This will prevent from artificial "gaps" when compiling out some enum
> values and having unnecessarily large MAX values & enums arrays, or
> simplifying iterating over valid enum values.

Yes, gaps can be annoying, e.g. in lookup tables where gaps get confused
with sentinels.

Avoiding gaps scares me.  You have to religiously compile the enum with
the exact same configuration everywhere in the program, or else you end
up with bugs that are hard to spot.  Therefore:

> Whenever config-host.h is changed, all the enum/types are recompiled.
>
> (a subsequent patch will split the schema. Target-specific poisoined
> conditionals will be added. They cannot be mixed with the common
> schema: it is not possible to end up with enums of different values in
> common and target builds)

Should we mention config-target.h here?

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index 02c5c6767a..6f9498566e 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -2045,14 +2045,11 @@ typedef enum %(c_name)s {
>  ''',
>                  c_name=c_name(name))
>  
> -    i = 0
>      for value in enum_values:
>          ret += mcgen('''
> -    %(c_enum)s = %(i)d,
> +    %(c_enum)s,
>  ''',
> -                     c_enum=c_enum_const(name, value, prefix),
> -                     i=i)
> -        i += 1
> +                     c_enum=c_enum_const(name, value, prefix))
>  
>      ret += mcgen('''
>  } %(c_name)s;

I need to consider the whole series to decide whether avoiding gaps is a
good idea.  If it is, then this patch is fine.

Re: [Qemu-devel] [PATCH v6 07/27] qapi: rename QAPISchemaEnumType.values to .members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Rename QAPISchemaEnumType.values and related variables to members.
> Makes sense ever since commit 93bda4dd4 changed .values from list of
> string to list of QAPISchemaMember. Obvious no-op.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 08/27] qapi: change enum visitor and gen_enum* to take QAPISchemaMember

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> This will allow to add and access more properties associated with enum
> values/members, like the associated 'if' condition. We may want to
> have a specialized type QAPISchemaEnumMember, for now this will do.

Well, we can add all we want without this patch.  The patch is about
enabling access in visit_enum_type() and ...

> While at it, also modify gen_enum() and gen_enum_lookup() for the
> same reason.

... these two helpers.  Worthwhile if later patches need such access.
We'll see.

> Suggested-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py         | 22 +++++++++++-----------
>  scripts/qapi/doc.py            |  2 +-
>  scripts/qapi/events.py         |  2 +-
>  scripts/qapi/introspect.py     |  5 +++--
>  scripts/qapi/types.py          |  6 +++---
>  scripts/qapi/visit.py          |  2 +-
>  tests/qapi-schema/test-qapi.py |  4 ++--
>  7 files changed, 22 insertions(+), 21 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index 7020b88abc..a353670079 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -1063,7 +1063,7 @@ class QAPISchemaVisitor(object):
>      def visit_builtin_type(self, name, info, json_type):
>          pass
>  
> -    def visit_enum_type(self, name, info, ifcond, values, prefix):
> +    def visit_enum_type(self, name, info, ifcond, members, prefix):
>          pass
>  
>      def visit_array_type(self, name, info, ifcond, element_type):
> @@ -1193,7 +1193,7 @@ class QAPISchemaEnumType(QAPISchemaType):
>  
>      def visit(self, visitor):
>          visitor.visit_enum_type(self.name, self.info, self.ifcond,
> -                                self.member_names(), self.prefix)
> +                                self.members, self.prefix)
>  
>  

Just two uses of QAPISchemaType.member_names() left.  See also below.

>  class QAPISchemaArrayType(QAPISchemaType):
> @@ -2012,19 +2012,19 @@ def _wrap_ifcond(ifcond, before, after):
>      return out
>  
>  
> -def gen_enum_lookup(name, values, prefix=None):
> +def gen_enum_lookup(name, members, prefix=None):
>      ret = mcgen('''
>  
>  const QEnumLookup %(c_name)s_lookup = {
>      .array = (const char *const[]) {
>  ''',
>                  c_name=c_name(name))
> -    for value in values:
> -        index = c_enum_const(name, value, prefix)
> +    for m in members:
> +        index = c_enum_const(name, m.name, prefix)
>          ret += mcgen('''
> -        [%(index)s] = "%(value)s",
> +        [%(index)s] = "%(name)s",
>  ''',
> -                     index=index, value=value)
> +                     index=index, name=m.name)
>  
>      ret += mcgen('''
>      },
> @@ -2035,9 +2035,9 @@ const QEnumLookup %(c_name)s_lookup = {
>      return ret
>  
>  
> -def gen_enum(name, values, prefix=None):
> +def gen_enum(name, members, prefix=None):
>      # append automatically generated _MAX value
> -    enum_values = values + ['_MAX']
> +    enum_members = members + [QAPISchemaMember('_MAX')]
>  
>      ret = mcgen('''
>  
> @@ -2045,11 +2045,11 @@ typedef enum %(c_name)s {
>  ''',
>                  c_name=c_name(name))
>  
> -    for value in enum_values:
> +    for m in enum_members:
>          ret += mcgen('''
>      %(c_enum)s,
>  ''',
> -                     c_enum=c_enum_const(name, value, prefix))
> +                     c_enum=c_enum_const(name, m.name, prefix))
>  
>      ret += mcgen('''
>  } %(c_name)s;
> diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
> index 987fd3c943..76cb186ff9 100755
> --- a/scripts/qapi/doc.py
> +++ b/scripts/qapi/doc.py
> @@ -206,7 +206,7 @@ class QAPISchemaGenDocVisitor(qapi.common.QAPISchemaVisitor):
>      def write(self, output_dir):
>          self._gen.write(output_dir, self._prefix + 'qapi-doc.texi')
>  
> -    def visit_enum_type(self, name, info, ifcond, values, prefix):
> +    def visit_enum_type(self, name, info, ifcond, members, prefix):
>          doc = self.cur_doc
>          self._gen.add(TYPE_FMT(type='Enum',
>                                 name=doc.symbol,
> diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
> index 764ef177ab..ea4dac6a05 100644
> --- a/scripts/qapi/events.py
> +++ b/scripts/qapi/events.py
> @@ -189,7 +189,7 @@ class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
>              self._genh.add(gen_event_send_decl(name, arg_type, boxed))
>              self._genc.add(gen_event_send(name, arg_type, boxed,
>                                            self._enum_name))
> -        self._event_names.append(name)
> +        self._event_names.append(QAPISchemaMember(name))
>  
>  

Rename ._event_names to ._event_enum_members?

May want to rename .enum_name to ._event_enum_name then.

>  def gen_events(schema, output_dir, prefix):
> diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
> index 71d4a779ce..3f1ca99f6d 100644
> --- a/scripts/qapi/introspect.py
> +++ b/scripts/qapi/introspect.py
> @@ -160,8 +160,9 @@ const QLitObject %(c_name)s = %(c_string)s;
>      def visit_builtin_type(self, name, info, json_type):
>          self._gen_qlit(name, 'builtin', {'json-type': json_type}, [])
>  
> -    def visit_enum_type(self, name, info, ifcond, values, prefix):
> -        self._gen_qlit(name, 'enum', {'values': values}, ifcond)
> +    def visit_enum_type(self, name, info, ifcond, members, prefix):
> +        self._gen_qlit(name, 'enum',
> +                       {'values': [m.name for m in members]}, ifcond)

Can't use QAPISchemaEnumType.member_names() here, since we don't have
the QAPISchemaEnumType, only its .members.

>  
>      def visit_array_type(self, name, info, ifcond, element_type):
>          element = self._use_type(element_type)
> diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
> index 91f87d0b8f..2d4a70f810 100644
> --- a/scripts/qapi/types.py
> +++ b/scripts/qapi/types.py
> @@ -213,10 +213,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
>          self._genh.add(gen_type_cleanup_decl(name))
>          self._genc.add(gen_type_cleanup(name))
>  
> -    def visit_enum_type(self, name, info, ifcond, values, prefix):
> +    def visit_enum_type(self, name, info, ifcond, members, prefix):
>          with ifcontext(ifcond, self._genh, self._genc):
> -            self._genh.preamble_add(gen_enum(name, values, prefix))
> -            self._genc.add(gen_enum_lookup(name, values, prefix))
> +            self._genh.preamble_add(gen_enum(name, members, prefix))
> +            self._genc.add(gen_enum_lookup(name, members, prefix))
>  
>      def visit_array_type(self, name, info, ifcond, element_type):
>          with ifcontext(ifcond, self._genh, self._genc):
> diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
> index 460cf12989..24f85a2e85 100644
> --- a/scripts/qapi/visit.py
> +++ b/scripts/qapi/visit.py
> @@ -310,7 +310,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaModularCVisitor):
>  ''',
>                                        types=types))
>  
> -    def visit_enum_type(self, name, info, ifcond, values, prefix):
> +    def visit_enum_type(self, name, info, ifcond, members, prefix):
>          with ifcontext(ifcond, self._genh, self._genc):
>              self._genh.add(gen_visit_decl(name, scalar=True))
>              self._genc.add(gen_visit_enum(name))
> diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
> index cea21c773a..27f776693e 100644
> --- a/tests/qapi-schema/test-qapi.py
> +++ b/tests/qapi-schema/test-qapi.py
> @@ -23,8 +23,8 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
>      def visit_include(self, name, info):
>          print('include %s' % name)
>  
> -    def visit_enum_type(self, name, info, ifcond, values, prefix):
> -        print('enum %s %s' % (name, values))
> +    def visit_enum_type(self, name, info, ifcond, members, prefix):
> +        print('enum %s %s' % (name, [m.name for m in members]))
>          if prefix:
>              print('    prefix %s' % prefix)
>          self._print_if(ifcond)

Likewise.

Perhaps we can get rid of .member_names().  Not this patch's business,
of course.

Assuming later patches make use of this:
Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 09/27] tests: print enum type members more like object type members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Commit 93bda4dd461 changed the internal representation of enum type
> members from str to QAPISchemaMember, but we still print only a
> string.  Has been good enough, as the name is the member's only
> attribute of interest, but that's about to change.  To prepare, print
> them more like object type members.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 10/27] qapi: factor out checking for keys

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Introduce a new helper function to check if the given keys are known,
> and if mandatory keys are present. The function will be reused in
> other places in the following code changes.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py | 20 +++++++++++++-------
>  1 file changed, 13 insertions(+), 7 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index a353670079..8313c478c4 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -873,6 +873,17 @@ def check_struct(expr, info):
>                 allow_metas=['struct'])
>  
>  
> +def check_known_keys(info, source, keys, required, optional):
> +    for key in keys:
> +        if key not in required and key not in optional:
> +            raise QAPISemError(info, "Unknown key '%s' in %s" % (key, source))
> +
> +    for key in required:
> +        if key not in keys:
> +            raise QAPISemError(info, "Key '%s' is missing from %s"
> +                               % (key, source))
> +
> +
>  def check_keys(expr_elem, meta, required, optional=[]):
>      expr = expr_elem['expr']
>      info = expr_elem['info']
> @@ -880,10 +891,9 @@ def check_keys(expr_elem, meta, required, optional=[]):
>      if not isinstance(name, str):
>          raise QAPISemError(info, "'%s' key must have a string value" % meta)
>      required = required + [meta]
> +    source = "%s '%s'" % (meta, name)
> +    check_known_keys(info, source, list(expr.keys()), required, optional)

Sure you need list() here?

>      for (key, value) in expr.items():
> -        if key not in required and key not in optional:
> -            raise QAPISemError(info, "Unknown key '%s' in %s '%s'"
> -                               % (key, meta, name))
>          if key in ['gen', 'success-response'] and value is not False:
>              raise QAPISemError(info,
>                                 "'%s' of %s '%s' should only use false value"
> @@ -895,10 +905,6 @@ def check_keys(expr_elem, meta, required, optional=[]):
>                                 % (key, meta, name))
>          if key == 'if':
>              check_if(expr, info)
> -    for key in required:
> -        if key not in expr:
> -            raise QAPISemError(info, "Key '%s' is missing from %s '%s'"
> -                               % (key, meta, name))
>  
>  
>  def check_exprs(exprs):

Re: [Qemu-devel] [PATCH v6 11/27] qapi: improve reporting of unknown or missing keys

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Report the set of missing or unknown keys. And give a hint about the
> accepted keys.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py                  | 23 +++++++++++++++--------
>  tests/qapi-schema/alternate-base.err    |  1 +
>  tests/qapi-schema/double-type.err       |  1 +
>  tests/qapi-schema/unknown-expr-key.err  |  3 ++-
>  tests/qapi-schema/unknown-expr-key.json |  2 +-
>  5 files changed, 20 insertions(+), 10 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index 8313c478c4..cf8dab2866 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -874,14 +874,21 @@ def check_struct(expr, info):
>  
>  
>  def check_known_keys(info, source, keys, required, optional):
> -    for key in keys:
> -        if key not in required and key not in optional:
> -            raise QAPISemError(info, "Unknown key '%s' in %s" % (key, source))
> -
> -    for key in required:
> -        if key not in keys:
> -            raise QAPISemError(info, "Key '%s' is missing from %s"
> -                               % (key, source))
> +
> +    def pprint(elems):
> +        return ', '.join("'" + e + "'" for e in sorted(elems))
> +
> +    missing = set(required) - set(keys)
> +    if missing:
> +        raise QAPISemError(info, "Key%s %s %s missing from %s"
> +                           % ('s' if len(missing) > 1 else '', pprint(missing),
> +                              'are' if len(missing) > 1 else 'is', source))
> +    allowed = set(required + optional)
> +    unknown = set(keys) - allowed
> +    if unknown:
> +        raise QAPISemError(info, "Unknown key%s %s in %s\nValid keys are %s."
> +                           % ('s' if len(unknown) > 1 else '', pprint(unknown),
> +                              source, pprint(allowed)))
>  
>  
>  def check_keys(expr_elem, meta, required, optional=[]):

I wouldn't habe bothered, but since you already wrote it...

> diff --git a/tests/qapi-schema/alternate-base.err b/tests/qapi-schema/alternate-base.err
> index 30d8a34373..ebe05bc898 100644
> --- a/tests/qapi-schema/alternate-base.err
> +++ b/tests/qapi-schema/alternate-base.err
> @@ -1 +1,2 @@
>  tests/qapi-schema/alternate-base.json:4: Unknown key 'base' in alternate 'Alt'
> +Valid keys are 'alternate', 'data', 'if'.
> diff --git a/tests/qapi-schema/double-type.err b/tests/qapi-schema/double-type.err
> index f9613c6d6b..799193dba1 100644
> --- a/tests/qapi-schema/double-type.err
> +++ b/tests/qapi-schema/double-type.err
> @@ -1 +1,2 @@
>  tests/qapi-schema/double-type.json:2: Unknown key 'command' in struct 'bar'
> +Valid keys are 'base', 'data', 'if', 'struct'.

The error message is just as suboptimal as before in this case, and the
hint doesn't really help.  Can't say I care.

> diff --git a/tests/qapi-schema/unknown-expr-key.err b/tests/qapi-schema/unknown-expr-key.err
> index 12f5ed5b43..83b83121e0 100644
> --- a/tests/qapi-schema/unknown-expr-key.err
> +++ b/tests/qapi-schema/unknown-expr-key.err
> @@ -1 +1,2 @@
> -tests/qapi-schema/unknown-expr-key.json:2: Unknown key 'bogus' in struct 'bar'
> +tests/qapi-schema/unknown-expr-key.json:2: Unknown keys 'bogus', 'foo' in struct 'bar'
> +Valid keys are 'base', 'data', 'if', 'struct'.
> diff --git a/tests/qapi-schema/unknown-expr-key.json b/tests/qapi-schema/unknown-expr-key.json
> index 3b2be00cc4..5bcb8efd1d 100644
> --- a/tests/qapi-schema/unknown-expr-key.json
> +++ b/tests/qapi-schema/unknown-expr-key.json
> @@ -1,2 +1,2 @@
>  # we reject an expression with unknown top-level keys
> -{ 'struct': 'bar', 'data': { 'string': 'str'}, 'bogus': { } }
> +{ 'struct': 'bar', 'data': { 'string': 'str'}, 'bogus': { }, 'foo': { } }

Let's call the second unknown key 'phony'.

Re: [Qemu-devel] [PATCH v6 12/27] qapi: add a dictionnary form with 'name' key for enum members

[Markus Armbruster]

In subject, s/dictionnary/dictionary/

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Desugar the enum NAME form to { 'name': NAME }. This will allow to add
> new enum members, such as 'if' in the following patch.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py                        | 47 ++++++++++++++++---
>  tests/Makefile.include                        |  3 +-
>  tests/qapi-schema/enum-bad-member.err         |  1 +
>  ...-dict-member.exit => enum-bad-member.exit} |  0
>  tests/qapi-schema/enum-bad-member.json        |  2 +
>  ...um-dict-member.out => enum-bad-member.out} |  0
>  .../qapi-schema/enum-dict-member-unknown.err  |  2 +
>  .../qapi-schema/enum-dict-member-unknown.exit |  1 +
>  .../qapi-schema/enum-dict-member-unknown.json |  2 +
>  .../qapi-schema/enum-dict-member-unknown.out  |  0
>  tests/qapi-schema/enum-dict-member.err        |  1 -
>  tests/qapi-schema/enum-dict-member.json       |  2 -
>  tests/qapi-schema/enum-missing-data.err       |  2 +-
>  13 files changed, 51 insertions(+), 12 deletions(-)
>  create mode 100644 tests/qapi-schema/enum-bad-member.err
>  rename tests/qapi-schema/{enum-dict-member.exit => enum-bad-member.exit} (100%)
>  create mode 100644 tests/qapi-schema/enum-bad-member.json
>  rename tests/qapi-schema/{enum-dict-member.out => enum-bad-member.out} (100%)
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.err
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.exit
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.json
>  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.out
>  delete mode 100644 tests/qapi-schema/enum-dict-member.err
>  delete mode 100644 tests/qapi-schema/enum-dict-member.json
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index cf8dab2866..e9fb736d46 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -740,6 +740,10 @@ def check_event(expr, info):
>                 allow_metas=meta)
>  
>  
> +def enum_get_names(expr):
> +    return [e['name'] for e in expr['data']]
> +
> +
>  def check_union(expr, info):
>      name = expr['union']
>      base = expr.get('base')
> @@ -799,7 +803,7 @@ def check_union(expr, info):
>          # If the discriminator names an enum type, then all members
>          # of 'data' must also be members of the enum type.
>          if enum_define:
> -            if key not in enum_define['data']:
> +            if key not in enum_get_names(enum_define):
>                  raise QAPISemError(info,
>                                     "Discriminator value '%s' is not found in "
>                                     "enum '%s'"
> @@ -831,10 +835,10 @@ def check_alternate(expr, info):
>          if qtype == 'QTYPE_QSTRING':
>              enum_expr = enum_types.get(value)
>              if enum_expr:
> -                for v in enum_expr['data']:
> +                for v in enum_get_names(enum_expr):
>                      if v in ['on', 'off']:
>                          conflicting.add('QTYPE_QBOOL')
> -                    if re.match(r'[-+0-9.]', v): # lazy, could be tightened
> +                    if re.match(r'[-+0-9.]', v):  # lazy, could be tightened

Unrelated change.

>                          conflicting.add('QTYPE_QNUM')
>              else:
>                  conflicting.add('QTYPE_QNUM')

The helper enum_get_names() is used just twice.

The second instance

                for v in enum_get_names(enum_expr):

could just as well iterate over enum_expr

                for d in enum_expr['data']:
                    v = d['name']

The first instance could simply be inlined then.

> @@ -847,19 +851,34 @@ def check_alternate(expr, info):
>              types_seen[qt] = key
>  
>  
> -def check_enum(expr, info):
> +def normalize_enum(expr, info):
>      name = expr['enum']
>      members = expr.get('data')
> -    prefix = expr.get('prefix')
>  
>      if not isinstance(members, list):
>          raise QAPISemError(info,
>                             "Enum '%s' requires an array for 'data'" % name)
> +
> +    # translate short member form to dict form
> +    for i, member in enumerate(members):
> +        if not isinstance(member, dict):
> +            member = {'name': member}
> +            members[i] = member

Simpler:

       members = [m if isinstance(m, dict) else {'name': m} for m in members]

> +
> +
> +def check_enum(expr, info):
> +    name = expr['enum']
> +    members = expr.get('data')

Preexisting: this should be expr['data'], because we already checked
required keys.

> +    prefix = expr.get('prefix')
> +
>      if prefix is not None and not isinstance(prefix, str):
>          raise QAPISemError(info,
>                             "Enum '%s' requires a string for 'prefix'" % name)
> +
>      for member in members:
> -        check_name(info, "Member of enum '%s'" % name, member,
> +        source = "dictionary member of enum '%s'" % name
> +        check_known_keys(info, source, member, ['name'], [])
> +        check_name(info, "Member of enum '%s'" % name, member['name'],
>                     enum_member=True)
>  
>  
> @@ -921,6 +940,13 @@ def check_exprs(exprs):
>      for builtin in builtin_types.keys():
>          all_names[builtin] = 'built-in'
>  
> +    # Normalize exprs
> +    for expr_elem in exprs:
> +        expr = expr_elem['expr']
> +        info = expr_elem['info']
> +        if 'enum' in expr:
> +            normalize_enum(expr, info)
> +

As demonstrated by tests/qapi-schema/enum-missing-data.json below, the
error message for

    {'enum': 'MyEnum'}

changes from

    Key 'data' is missing from enum 'MyEnum'

to

    Enum 'MyEnum' requires an array for 'data'

because you normalize before checking keys.  I don't think that's a good
idea.

>      # Learn the types and check for valid expression keys
>      for expr_elem in exprs:
>          expr = expr_elem['expr']
           info = expr_elem['info']
           doc = expr_elem.get('doc')

           if 'include' in expr:
               continue

           if not doc and doc_required:
               raise QAPISemError(info,
                                  "Expression missing documentation comment")

           if 'enum' in expr:
               meta = 'enum'
               check_keys(expr_elem, 'enum', ['data'], ['if', 'prefix'])

This seems a better place to call normalize_enum(expr, info).

               enum_types[expr[meta]] = expr
           elif 'union' in expr:
> @@ -1640,7 +1666,14 @@ class QAPISchema(object):
>                                              qtype_values, 'QTYPE'))
>  
>      def _make_enum_members(self, values):
> -        return [QAPISchemaMember(v) for v in values]
> +        enum = []
> +        for v in values:
> +            if isinstance(v, dict):
> +                name = v['name']
> +            else:
> +                name = v
> +            enum.append(QAPISchemaMember(name))
> +        return enum

Despite normalization to the long form, we still deal with both the long
and the short form here.  Three callers:

* _def_predefineds(): passes the short form.

* _make_implicit_enum_type(): passes the short form until PATCH 19
  changes it to pass the long form.

* _def_enum_type(): passes the long form.

I believe always passing the long form would be simpler.

>  
>      def _make_implicit_enum_type(self, name, info, ifcond, values):
>          # See also QAPISchemaObjectTypeMember._pretty_owner()
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index a49282704e..8e1b122cf2 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -482,10 +482,11 @@ qapi-schema += double-data.json
>  qapi-schema += double-type.json
>  qapi-schema += duplicate-key.json
>  qapi-schema += empty.json
> +qapi-schema += enum-bad-member.json
>  qapi-schema += enum-bad-name.json
>  qapi-schema += enum-bad-prefix.json
>  qapi-schema += enum-clash-member.json
> -qapi-schema += enum-dict-member.json
> +qapi-schema += enum-dict-member-unknown.json
>  qapi-schema += enum-int-member.json
>  qapi-schema += enum-member-case.json
>  qapi-schema += enum-missing-data.json
> diff --git a/tests/qapi-schema/enum-bad-member.err b/tests/qapi-schema/enum-bad-member.err
> new file mode 100644
> index 0000000000..211db9e6fc
> --- /dev/null
> +++ b/tests/qapi-schema/enum-bad-member.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/enum-bad-member.json:2: Member of enum 'MyEnum' requires a string name
> diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-bad-member.exit
> similarity index 100%
> rename from tests/qapi-schema/enum-dict-member.exit
> rename to tests/qapi-schema/enum-bad-member.exit
> diff --git a/tests/qapi-schema/enum-bad-member.json b/tests/qapi-schema/enum-bad-member.json
> new file mode 100644
> index 0000000000..98da6828b4
> --- /dev/null
> +++ b/tests/qapi-schema/enum-bad-member.json
> @@ -0,0 +1,2 @@
> +# we reject any enum member that is not a string
> +{ 'enum': 'MyEnum', 'data': [ [ ] ] }
> diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-bad-member.out
> similarity index 100%
> rename from tests/qapi-schema/enum-dict-member.out
> rename to tests/qapi-schema/enum-bad-member.out

This replaces enum-dict-member.json.  Okay.  Took me a while to see this
%-/

> diff --git a/tests/qapi-schema/enum-dict-member-unknown.err b/tests/qapi-schema/enum-dict-member-unknown.err
> new file mode 100644
> index 0000000000..76bd0471db
> --- /dev/null
> +++ b/tests/qapi-schema/enum-dict-member-unknown.err
> @@ -0,0 +1,2 @@
> +tests/qapi-schema/enum-dict-member-unknown.json:2: Unknown key 'bad-key' in dictionary member of enum 'MyEnum'
> +Valid keys are 'name'.
> diff --git a/tests/qapi-schema/enum-dict-member-unknown.exit b/tests/qapi-schema/enum-dict-member-unknown.exit
> new file mode 100644
> index 0000000000..d00491fd7e
> --- /dev/null
> +++ b/tests/qapi-schema/enum-dict-member-unknown.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/enum-dict-member-unknown.json b/tests/qapi-schema/enum-dict-member-unknown.json
> new file mode 100644
> index 0000000000..6664c59201
> --- /dev/null
> +++ b/tests/qapi-schema/enum-dict-member-unknown.json
> @@ -0,0 +1,2 @@
> +# we reject any enum member that is not a string or a dict with 'name'
> +{ 'enum': 'MyEnum', 'data': [ { 'name': 'foo', 'bad-key': 'str' } ] }
> diff --git a/tests/qapi-schema/enum-dict-member-unknown.out b/tests/qapi-schema/enum-dict-member-unknown.out
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
> deleted file mode 100644
> index 8ca146ea59..0000000000
> --- a/tests/qapi-schema/enum-dict-member.err
> +++ /dev/null
> @@ -1 +0,0 @@
> -tests/qapi-schema/enum-dict-member.json:2: Member of enum 'MyEnum' requires a string name
> diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
> deleted file mode 100644
> index 79672e0f09..0000000000
> --- a/tests/qapi-schema/enum-dict-member.json
> +++ /dev/null
> @@ -1,2 +0,0 @@
> -# we reject any enum member that is not a string
> -{ 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
> diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
> index ba4873ae69..b8ccae071b 100644
> --- a/tests/qapi-schema/enum-missing-data.err
> +++ b/tests/qapi-schema/enum-missing-data.err
> @@ -1 +1 @@
> -tests/qapi-schema/enum-missing-data.json:2: Key 'data' is missing from enum 'MyEnum'
> +tests/qapi-schema/enum-missing-data.json:2: Enum 'MyEnum' requires an array for 'data'

Re: [Qemu-devel] [PATCH v6 13/27] qapi: add 'if' to enum members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> QAPISchemaMember gains .ifcond for enum members: inherited classes,
> such as QAPISchemaObjectTypeMember, will thus have an ifcond member
> after this (those different types will also use the .ifcond to store
> the condition and generate conditional code in the following patches).
>
> Generated code is not changed by this patch, but with "qapi: add #if
> conditions to generated code" patch.

It's actually "qapi: add #if conditions to generated code members".  I'd
sidestep this like you did in commit 967c885108f

    The generated code is for now *unconditional*. Later patches generate
    the conditionals.

except I'd say "The generated code remains unconditional for now."

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py                         | 10 +++++++---
>  tests/Makefile.include                         |  1 +
>  tests/qapi-schema/enum-dict-member-unknown.err |  2 +-
>  tests/qapi-schema/enum-if-invalid.err          |  1 +
>  tests/qapi-schema/enum-if-invalid.exit         |  1 +
>  tests/qapi-schema/enum-if-invalid.json         |  3 +++
>  tests/qapi-schema/enum-if-invalid.out          |  0
>  tests/qapi-schema/qapi-schema-test.json        |  5 +++--
>  tests/qapi-schema/qapi-schema-test.out         |  2 ++
>  tests/qapi-schema/test-qapi.py                 |  1 +
>  10 files changed, 20 insertions(+), 6 deletions(-)
>  create mode 100644 tests/qapi-schema/enum-if-invalid.err
>  create mode 100644 tests/qapi-schema/enum-if-invalid.exit
>  create mode 100644 tests/qapi-schema/enum-if-invalid.json
>  create mode 100644 tests/qapi-schema/enum-if-invalid.out
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index e9fb736d46..95b7cd74ee 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -877,7 +877,8 @@ def check_enum(expr, info):
>  
>      for member in members:
>          source = "dictionary member of enum '%s'" % name
> -        check_known_keys(info, source, member, ['name'], [])
> +        check_known_keys(info, source, member, ['name'], ['if'])
> +        check_if(member, info)
>          check_name(info, "Member of enum '%s'" % name, member['name'],
>                     enum_member=True)
>  
> @@ -1357,9 +1358,10 @@ class QAPISchemaObjectType(QAPISchemaType):
>  class QAPISchemaMember(object):
>      role = 'member'
>  
> -    def __init__(self, name):
> +    def __init__(self, name, ifcond=None):
>          assert isinstance(name, str)
>          self.name = name
> +        self.ifcond = listify_cond(ifcond)
>          self.owner = None
>  
>      def set_owner(self, name):
> @@ -1670,9 +1672,11 @@ class QAPISchema(object):
>          for v in values:
>              if isinstance(v, dict):
>                  name = v['name']
> +                ifcond = v.get('if')
>              else:
>                  name = v
> -            enum.append(QAPISchemaMember(name))
> +                ifcond = None
> +            enum.append(QAPISchemaMember(name, ifcond))
>          return enum
>  
>      def _make_implicit_enum_type(self, name, info, ifcond, values):
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 8e1b122cf2..3ba9097892 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -487,6 +487,7 @@ qapi-schema += enum-bad-name.json
>  qapi-schema += enum-bad-prefix.json
>  qapi-schema += enum-clash-member.json
>  qapi-schema += enum-dict-member-unknown.json
> +qapi-schema += enum-if-invalid.json
>  qapi-schema += enum-int-member.json
>  qapi-schema += enum-member-case.json
>  qapi-schema += enum-missing-data.json
> diff --git a/tests/qapi-schema/enum-dict-member-unknown.err b/tests/qapi-schema/enum-dict-member-unknown.err
> index 76bd0471db..2aae618be0 100644
> --- a/tests/qapi-schema/enum-dict-member-unknown.err
> +++ b/tests/qapi-schema/enum-dict-member-unknown.err
> @@ -1,2 +1,2 @@
>  tests/qapi-schema/enum-dict-member-unknown.json:2: Unknown key 'bad-key' in dictionary member of enum 'MyEnum'
> -Valid keys are 'name'.
> +Valid keys are 'if', 'name'.
> diff --git a/tests/qapi-schema/enum-if-invalid.err b/tests/qapi-schema/enum-if-invalid.err
> new file mode 100644
> index 0000000000..54c3cf887b
> --- /dev/null
> +++ b/tests/qapi-schema/enum-if-invalid.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings
> diff --git a/tests/qapi-schema/enum-if-invalid.exit b/tests/qapi-schema/enum-if-invalid.exit
> new file mode 100644
> index 0000000000..d00491fd7e
> --- /dev/null
> +++ b/tests/qapi-schema/enum-if-invalid.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/enum-if-invalid.json b/tests/qapi-schema/enum-if-invalid.json
> new file mode 100644
> index 0000000000..60bd0ef1d7
> --- /dev/null
> +++ b/tests/qapi-schema/enum-if-invalid.json
> @@ -0,0 +1,3 @@
> +# check invalid 'if' type
> +{ 'enum': 'TestIfEnum', 'data':
> +  [ 'foo', { 'name' : 'bar', 'if': { 'val': 'foo' } } ] }
> diff --git a/tests/qapi-schema/enum-if-invalid.out b/tests/qapi-schema/enum-if-invalid.out
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> index 11aa4c8f8d..35ca94d991 100644
> --- a/tests/qapi-schema/qapi-schema-test.json
> +++ b/tests/qapi-schema/qapi-schema-test.json
> @@ -202,7 +202,8 @@
>  { 'struct': 'TestIfStruct', 'data': { 'foo': 'int' },
>    'if': 'defined(TEST_IF_STRUCT)' }
>  
> -{ 'enum': 'TestIfEnum', 'data': [ 'foo', 'bar' ],
> +{ 'enum': 'TestIfEnum', 'data':
> +  [ 'foo', { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ],
>    'if': 'defined(TEST_IF_ENUM)' }
>  
>  { 'union': 'TestIfUnion', 'data': { 'foo': 'TestStruct' },
> @@ -211,7 +212,7 @@
>  { 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
>    'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
>  
> -{ 'command': 'TestIfCmd', 'data': { 'foo': 'TestIfStruct' },
> +{ 'command': 'TestIfCmd', 'data': { 'foo': 'TestIfStruct', 'bar': 'TestIfEnum' },
>    'returns': 'UserDefThree',
>    'if': ['defined(TEST_IF_CMD)', 'defined(TEST_IF_STRUCT)'] }
>  
> diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> index edd22bc306..1fb33f302c 100644
> --- a/tests/qapi-schema/qapi-schema-test.out
> +++ b/tests/qapi-schema/qapi-schema-test.out
> @@ -270,6 +270,7 @@ object TestIfStruct
>  enum TestIfEnum
>      member foo
>      member bar
> +        if ['defined(TEST_IF_ENUM_BAR)']
>      if ['defined(TEST_IF_ENUM)']
>  object q_obj_TestStruct-wrapper
>      member data: TestStruct optional=False
> @@ -288,6 +289,7 @@ alternate TestIfAlternate
>      if ['defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)']
>  object q_obj_TestIfCmd-arg
>      member foo: TestIfStruct optional=False
> +    member bar: TestIfEnum optional=False
>      if ['defined(TEST_IF_CMD)', 'defined(TEST_IF_STRUCT)']
>  command TestIfCmd q_obj_TestIfCmd-arg -> UserDefThree
>     gen=True success_response=True boxed=False oob=False preconfig=False
> diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
> index 641a18f06d..b2f4ce6134 100644
> --- a/tests/qapi-schema/test-qapi.py
> +++ b/tests/qapi-schema/test-qapi.py
> @@ -29,6 +29,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
>              print('    prefix %s' % prefix)
>          for m in members:
>              print('    member %s' % m.name)
> +            self._print_if(m.ifcond, 8)

Could write indent=8.  Your choice.

>          self._print_if(ifcond)
>  
>      def visit_object_type(self, name, info, ifcond, base, members, variants):

Preferably with a tweaked commit message:
Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 14/27] qapi-event: add 'if' condition to implicit event enum

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Add condition to QAPIEvent enum members based on the event 'if'.
>
> Generated code is not changed by this patch, but with "qapi: add #if
> conditions to generated code" patch.
>
> There is no coverage of this change in qapi-schema-test.out since the
> event_names enum is an implicit type created by qapi-event.py.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/events.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
> index ea4dac6a05..c9beef8b4b 100644
> --- a/scripts/qapi/events.py
> +++ b/scripts/qapi/events.py
> @@ -189,7 +189,7 @@ class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
>              self._genh.add(gen_event_send_decl(name, arg_type, boxed))
>              self._genc.add(gen_event_send(name, arg_type, boxed,
>                                            self._enum_name))
> -        self._event_names.append(QAPISchemaMember(name))
> +        self._event_names.append(QAPISchemaMember(name, ifcond))
>  
>  
>  def gen_events(schema, output_dir, prefix):

Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 15/27] qapi: rename allow_dict to allow_implicit

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> This makes it a bit clearer what is the intent of the dictionnary for

dictionary

> the check_type() function, since there was some confusion on a
> previous iteration of this series.

I don't remember... got a pointer to that confusion handy?

> Suggested-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Re: [Qemu-devel] [PATCH v6 16/27] qapi: add a dictionary form with 'type' key for members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Wherever a struct/union/alternate/command/event member with NAME: TYPE
> form is accepted, desugar it to a NAME: { 'type': TYPE } form.
>
> This will allow to add new member details, such as 'if' in the
> following patch to introduce conditionals, or 'default' for default
> values etc.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py                        | 56 +++++++++++++------
>  tests/Makefile.include                        |  3 +
>  tests/qapi-schema/alternate-invalid-dict.err  |  1 +
>  tests/qapi-schema/alternate-invalid-dict.exit |  1 +
>  tests/qapi-schema/alternate-invalid-dict.json |  4 ++
>  tests/qapi-schema/alternate-invalid-dict.out  |  0
>  tests/qapi-schema/event-nest-struct.err       |  2 +-
>  tests/qapi-schema/flat-union-inline.err       |  2 +-
>  tests/qapi-schema/nested-struct-data.err      |  2 +-
>  tests/qapi-schema/qapi-schema-test.json       | 10 ++--
>  .../struct-member-invalid-dict.err            |  1 +
>  .../struct-member-invalid-dict.exit           |  1 +
>  .../struct-member-invalid-dict.json           |  3 +
>  .../struct-member-invalid-dict.out            |  0
>  .../qapi-schema/union-branch-invalid-dict.err |  1 +
>  .../union-branch-invalid-dict.exit            |  1 +
>  .../union-branch-invalid-dict.json            |  4 ++
>  .../qapi-schema/union-branch-invalid-dict.out |  0
>  18 files changed, 66 insertions(+), 26 deletions(-)
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.err
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.exit
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.json
>  create mode 100644 tests/qapi-schema/alternate-invalid-dict.out
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.err
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.exit
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.json
>  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.out
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.err
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.exit
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.json
>  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.out
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index d83fa1900e..fc68bb472e 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -588,11 +588,11 @@ def discriminator_find_enum_define(expr):
>      if not base_members:
>          return None
>  
> -    discriminator_type = base_members.get(discriminator)
> -    if not discriminator_type:
> +    discriminator_member = base_members.get(discriminator)
> +    if not discriminator_member:
>          return None
>  
> -    return enum_types.get(discriminator_type)
> +    return enum_types.get(discriminator_member['type'])
>  
>  

The name discriminator_member is confusing.  It suggests its value is
the (desugared) member definition, i.e. something like

     {'enum1': {'type': 'EnumOne'}}

it's actually only the definition's value part, i.e. something like

     {'type': 'EnumOne'}

Naming is hard... discriminator_value?  discriminator_def_rhs?

Same in check_union() below.

>  # Names must be letters, numbers, -, and _.  They must start with letter,
> @@ -660,6 +660,15 @@ def check_if(expr, info):
>          check_if_str(ifcond, info)
>  
>  
> +def normalize_members(expr, field):
> +    members = expr.get(field)
> +    if isinstance(members, OrderedDict):
> +        for key, arg in members.items():
> +            if isinstance(arg, dict):
> +                continue
> +            members[key] = {'type': arg}
> +
> +

PATCH 12's normalize_enum() conflates normalization and checking.  This
one doesn't.  Better.

>  def check_type(info, source, value, allow_array=False,
>                 allow_implicit=False, allow_optional=False,
>                 allow_metas=[]):
> @@ -704,8 +713,10 @@ def check_type(info, source, value, allow_array=False,
>                                 % (source, key))
>          # Todo: allow dictionaries to represent default values of
>          # an optional argument.
> -        check_type(info, "Member '%s' of %s" % (key, source), arg,
> -                   allow_array=True,
> +        check_known_keys(info, "member '%s' of %s" % (key, source),
> +                         arg, ['type'], [])
> +        check_type(info, "Member '%s' of %s" % (key, source),
> +                   arg['type'], allow_array=True,
>                     allow_metas=['built-in', 'union', 'alternate', 'struct',
>                                  'enum'])
>  
> @@ -776,13 +787,13 @@ def check_union(expr, info):
>          # member of the base struct.
>          check_name(info, "Discriminator of flat union '%s'" % name,
>                     discriminator)
> -        discriminator_type = base_members.get(discriminator)
> -        if not discriminator_type:
> +        discriminator_member = base_members.get(discriminator)
> +        if not discriminator_member:
>              raise QAPISemError(info,
>                                 "Discriminator '%s' is not a member of base "
>                                 "struct '%s'"
>                                 % (discriminator, base))
> -        enum_define = enum_types.get(discriminator_type)
> +        enum_define = enum_types.get(discriminator_member['type'])
>          allow_metas = ['struct']
>          # Do not allow string discriminator
>          if not enum_define:
> @@ -795,10 +806,13 @@ def check_union(expr, info):
>          raise QAPISemError(info, "Union '%s' cannot have empty 'data'" % name)
>      for (key, value) in members.items():
>          check_name(info, "Member of union '%s'" % name, key)
> +        source = "member '%s' of union '%s'" % (key, name)
> +        check_known_keys(info, source, value, ['type'], [])

Elsewhere, you do it like

           check_known_keys(info, "member '%s' of union '%s'" % (key, name),
                            value, ['type'], [])

Let's stick to that.

> +        typ = value['type']
>  
>          # Each value must name a known type
>          check_type(info, "Member '%s' of union '%s'" % (key, name),
> -                   value, allow_array=not base, allow_metas=allow_metas)
> +                   typ, allow_array=not base, allow_metas=allow_metas)
>  
>          # If the discriminator names an enum type, then all members
>          # of 'data' must also be members of the enum type.
> @@ -822,18 +836,20 @@ def check_alternate(expr, info):
>                             "in 'data'" % name)
>      for (key, value) in members.items():
>          check_name(info, "Member of alternate '%s'" % name, key)
> +        source = "member '%s' of alternate '%s'" % (key, name)
> +        check_known_keys(info, source, value, ['type'], [])

Likewise.

> +        typ = value['type']
>  
>          # Ensure alternates have no type conflicts.
> -        check_type(info, "Member '%s' of alternate '%s'" % (key, name),
> -                   value,
> +        check_type(info, "Member '%s' of alternate '%s'" % (key, name), typ,
>                     allow_metas=['built-in', 'union', 'struct', 'enum'])
> -        qtype = find_alternate_member_qtype(value)
> +        qtype = find_alternate_member_qtype(typ)
>          if not qtype:
>              raise QAPISemError(info, "Alternate '%s' member '%s' cannot use "
> -                               "type '%s'" % (name, key, value))
> +                               "type '%s'" % (name, key, typ))
>          conflicting = set([qtype])
>          if qtype == 'QTYPE_QSTRING':
> -            enum_expr = enum_types.get(value)
> +            enum_expr = enum_types.get(typ)
>              if enum_expr:
>                  for v in enum_get_names(enum_expr):
>                      if v in ['on', 'off']:
> @@ -947,6 +963,10 @@ def check_exprs(exprs):
>          info = expr_elem['info']
>          if 'enum' in expr:
>              normalize_enum(expr, info)
> +        elif 'union' in expr:
> +            normalize_members(expr, 'base')
> +        if {'union', 'alternate', 'struct', 'command', 'event'} & set(expr):
> +            normalize_members(expr, 'data')
>  
>      # Learn the types and check for valid expression keys
>      for expr_elem in exprs:

PATCH 12 had an issue in this loop, and the obvious fix was to fold it
into the next loop.  I think this can be done here too.

> @@ -1735,7 +1755,7 @@ class QAPISchema(object):
>          return QAPISchemaObjectTypeMember(name, typ, optional)
>  
>      def _make_members(self, data, info):
> -        return [self._make_member(key, value, info)
> +        return [self._make_member(key, value['type'], info)
>                  for (key, value) in data.items()]
>  
>      def _def_struct_type(self, expr, info, doc):
> @@ -1771,11 +1791,11 @@ class QAPISchema(object):
>                  name, info, doc, ifcond,
>                  'base', self._make_members(base, info))
>          if tag_name:
> -            variants = [self._make_variant(key, value)
> +            variants = [self._make_variant(key, value['type'])
>                          for (key, value) in data.items()]
>              members = []
>          else:
> -            variants = [self._make_simple_variant(key, value, info)
> +            variants = [self._make_simple_variant(key, value['type'], info)
>                          for (key, value) in data.items()]
>              typ = self._make_implicit_enum_type(name, info, ifcond,
>                                                  [v.name for v in variants])
> @@ -1791,7 +1811,7 @@ class QAPISchema(object):
>          name = expr['alternate']
>          data = expr['data']
>          ifcond = expr.get('if')
> -        variants = [self._make_variant(key, value)
> +        variants = [self._make_variant(key, value['type'])
>                      for (key, value) in data.items()]
>          tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
>          self._def_entity(
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 3ba9097892..43e100a6cd 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -421,6 +421,7 @@ qapi-schema += alternate-conflict-string.json
>  qapi-schema += alternate-conflict-bool-string.json
>  qapi-schema += alternate-conflict-num-string.json
>  qapi-schema += alternate-empty.json
> +qapi-schema += alternate-invalid-dict.json
>  qapi-schema += alternate-nested.json
>  qapi-schema += alternate-unknown.json
>  qapi-schema += args-alternate.json
> @@ -563,6 +564,7 @@ qapi-schema += returns-whitelist.json
>  qapi-schema += struct-base-clash-deep.json
>  qapi-schema += struct-base-clash.json
>  qapi-schema += struct-data-invalid.json
> +qapi-schema += struct-member-invalid-dict.json
>  qapi-schema += struct-member-invalid.json
>  qapi-schema += trailing-comma-list.json
>  qapi-schema += trailing-comma-object.json
> @@ -574,6 +576,7 @@ qapi-schema += unicode-str.json
>  qapi-schema += union-base-empty.json
>  qapi-schema += union-base-no-discriminator.json
>  qapi-schema += union-branch-case.json
> +qapi-schema += union-branch-invalid-dict.json
>  qapi-schema += union-clash-branches.json
>  qapi-schema += union-empty.json
>  qapi-schema += union-invalid-base.json
> diff --git a/tests/qapi-schema/alternate-invalid-dict.err b/tests/qapi-schema/alternate-invalid-dict.err
> new file mode 100644
> index 0000000000..631d46628e
> --- /dev/null
> +++ b/tests/qapi-schema/alternate-invalid-dict.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/alternate-invalid-dict.json:2: Key 'type' is missing from member 'two' of alternate 'Alt'
> diff --git a/tests/qapi-schema/alternate-invalid-dict.exit b/tests/qapi-schema/alternate-invalid-dict.exit
> new file mode 100644
> index 0000000000..d00491fd7e
> --- /dev/null
> +++ b/tests/qapi-schema/alternate-invalid-dict.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/alternate-invalid-dict.json b/tests/qapi-schema/alternate-invalid-dict.json
> new file mode 100644
> index 0000000000..45f2c8ebef
> --- /dev/null
> +++ b/tests/qapi-schema/alternate-invalid-dict.json
> @@ -0,0 +1,4 @@
> +# invalid field dictionnary, missing type

dictionary

Suggest a grep over the whole series.

Even better, reuse struct-member-invalid-dict.json's comment here.

> +{ 'alternate': 'Alt',
> +  'data': { 'one': 'str',
> +            'two': { 'if': 'foo' } } }
> diff --git a/tests/qapi-schema/alternate-invalid-dict.out b/tests/qapi-schema/alternate-invalid-dict.out
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err
> index 5a42701b8f..ea1be65d89 100644
> --- a/tests/qapi-schema/event-nest-struct.err
> +++ b/tests/qapi-schema/event-nest-struct.err
> @@ -1 +1 @@
> -tests/qapi-schema/event-nest-struct.json:1: Member 'a' of 'data' for event 'EVENT_A' should be a type name
> +tests/qapi-schema/event-nest-struct.json:1: Key 'type' is missing from member 'a' of 'data' for event 'EVENT_A'

We lose the test of the "should be a type name" error.  Let's copy this
test to event-member-invalid-dict.json to cover "Key 'type' is missing",
and update the original to keep covering "should be a type name".


> diff --git a/tests/qapi-schema/flat-union-inline.err b/tests/qapi-schema/flat-union-inline.err
> index 2333358d28..3d29433bfb 100644
> --- a/tests/qapi-schema/flat-union-inline.err
> +++ b/tests/qapi-schema/flat-union-inline.err
> @@ -1 +1 @@
> -tests/qapi-schema/flat-union-inline.json:7: Member 'value1' of union 'TestUnion' should be a type name
> +tests/qapi-schema/flat-union-inline.json:7: Key 'type' is missing from member 'value1' of union 'TestUnion'

Likewise.

> diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
> index da767bade2..48355adcbe 100644
> --- a/tests/qapi-schema/nested-struct-data.err
> +++ b/tests/qapi-schema/nested-struct-data.err
> @@ -1 +1 @@
> -tests/qapi-schema/nested-struct-data.json:2: Member 'a' of 'data' for command 'foo' should be a type name
> +tests/qapi-schema/nested-struct-data.json:2: Key 'type' is missing from member 'a' of 'data' for command 'foo'

Likewise.

> diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> index 35ca94d991..30f635d484 100644
> --- a/tests/qapi-schema/qapi-schema-test.json
> +++ b/tests/qapi-schema/qapi-schema-test.json

Positive tests for ...

> @@ -11,7 +11,7 @@
>          'guest-sync' ] } }
>  
>  { 'struct': 'TestStruct',
> -  'data': { 'integer': 'int', 'boolean': 'bool', 'string': 'str' } }
> +  'data': { 'integer': {'type': 'int'}, 'boolean': 'bool', 'string': 'str' } }

... struct member, ...

>  
>  # for testing enums
>  { 'struct': 'NestedEnumsOne',
> @@ -77,7 +77,7 @@
>  { 'union': 'UserDefFlatUnion',
>    'base': 'UserDefUnionBase',   # intentional forward reference
>    'discriminator': 'enum1',
> -  'data': { 'value1' : 'UserDefA',
> +  'data': { 'value1' : {'type': 'UserDefA'},
>              'value2' : 'UserDefB',
>              'value3' : 'UserDefB'
>              # 'value4' defaults to empty

... flat union member, ...

> @@ -98,7 +98,7 @@
>  { 'struct': 'WrapAlternate',
>    'data': { 'alt': 'UserDefAlternate' } }
>  { 'alternate': 'UserDefAlternate',
> -  'data': { 'udfu': 'UserDefFlatUnion', 'e': 'EnumOne', 'i': 'int',
> +  'data': { 'udfu': {'type': 'UserDefFlatUnion'}, 'e': 'EnumOne', 'i': 'int',
>              'n': 'null' } }
>  

... alternate member, ...

>  { 'struct': 'UserDefC',
> @@ -134,7 +134,7 @@
>  { 'command': 'user_def_cmd', 'data': {} }
>  { 'command': 'user_def_cmd1', 'data': {'ud1a': 'UserDefOne'} }
>  { 'command': 'user_def_cmd2',
> -  'data': {'ud1a': 'UserDefOne', '*ud1b': 'UserDefOne'},
> +  'data': {'ud1a': {'type': 'UserDefOne'}, '*ud1b': 'UserDefOne'},
>    'returns': 'UserDefTwo' }
>  

... command argument, ...

>  # Returning a non-dictionary requires a name from the whitelist
> @@ -164,7 +164,7 @@
>  
>  # testing event
>  { 'struct': 'EventStructOne',
> -  'data': { 'struct1': 'UserDefOne', 'string': 'str', '*enum2': 'EnumOne' } }
> +  'data': { 'struct1': {'type': 'UserDefOne'}, 'string': 'str', '*enum2': 'EnumOne' } }
>  
>  { 'event': 'EVENT_A' }
>  { 'event': 'EVENT_B',

... and event argument.

Missing: simple union.  Okay because the desugaring is oblivious of the
difference between flat and simple unions.

> diff --git a/tests/qapi-schema/struct-member-invalid-dict.err b/tests/qapi-schema/struct-member-invalid-dict.err
> new file mode 100644
> index 0000000000..6a765bc668
> --- /dev/null
> +++ b/tests/qapi-schema/struct-member-invalid-dict.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/struct-member-invalid-dict.json:2: Key 'type' is missing from member '*a' of 'data' for struct 'foo'
> diff --git a/tests/qapi-schema/struct-member-invalid-dict.exit b/tests/qapi-schema/struct-member-invalid-dict.exit
> new file mode 100644
> index 0000000000..d00491fd7e
> --- /dev/null
> +++ b/tests/qapi-schema/struct-member-invalid-dict.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/struct-member-invalid-dict.json b/tests/qapi-schema/struct-member-invalid-dict.json
> new file mode 100644
> index 0000000000..ebd9733b49
> --- /dev/null
> +++ b/tests/qapi-schema/struct-member-invalid-dict.json
> @@ -0,0 +1,3 @@
> +# exploded member form must have a 'type'

Suggest

   # Long form of member must have a value member 'type'

or simply

   # Missing type

> +{ 'struct': 'foo',
> +  'data': { '*a': { 'case': 'foo' } } }
> diff --git a/tests/qapi-schema/struct-member-invalid-dict.out b/tests/qapi-schema/struct-member-invalid-dict.out
> new file mode 100644
> index 0000000000..e69de29bb2
> diff --git a/tests/qapi-schema/union-branch-invalid-dict.err b/tests/qapi-schema/union-branch-invalid-dict.err
> new file mode 100644
> index 0000000000..89f9b36791
> --- /dev/null
> +++ b/tests/qapi-schema/union-branch-invalid-dict.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/union-branch-invalid-dict.json:2: Key 'type' is missing from member 'integer' of union 'UnionInvalidBranch'
> diff --git a/tests/qapi-schema/union-branch-invalid-dict.exit b/tests/qapi-schema/union-branch-invalid-dict.exit
> new file mode 100644
> index 0000000000..d00491fd7e
> --- /dev/null
> +++ b/tests/qapi-schema/union-branch-invalid-dict.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/union-branch-invalid-dict.json b/tests/qapi-schema/union-branch-invalid-dict.json
> new file mode 100644
> index 0000000000..19c5d9cacd
> --- /dev/null
> +++ b/tests/qapi-schema/union-branch-invalid-dict.json
> @@ -0,0 +1,4 @@
> +# exploded member form must have a 'type'

Likewise.

> +{ 'union': 'UnionInvalidBranch',
> +  'data': { 'integer': { 'if': 'foo'},
> +            's8': 'int8' } }

I avoid simple unions like the plague.  That said, having one here is
tolerable; we'll just have to replace it when we abolish simple unions.

> diff --git a/tests/qapi-schema/union-branch-invalid-dict.out b/tests/qapi-schema/union-branch-invalid-dict.out
> new file mode 100644
> index 0000000000..e69de29bb2

Re: [Qemu-devel] [PATCH v6 14/27] qapi-event: add 'if' condition to implicit event enum

[Markus Armbruster]

One more thing...

Markus Armbruster <armbru@redhat.com> writes:

> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
>> Add condition to QAPIEvent enum members based on the event 'if'.
>>
>> Generated code is not changed by this patch, but with "qapi: add #if
>> conditions to generated code" patch.

My suggestion on PATCH 13's commit message applies.

>> There is no coverage of this change in qapi-schema-test.out since the
>> event_names enum is an implicit type created by qapi-event.py.
>>
>> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> ---
>>  scripts/qapi/events.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
>> index ea4dac6a05..c9beef8b4b 100644
>> --- a/scripts/qapi/events.py
>> +++ b/scripts/qapi/events.py
>> @@ -189,7 +189,7 @@ class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
>>              self._genh.add(gen_event_send_decl(name, arg_type, boxed))
>>              self._genc.add(gen_event_send(name, arg_type, boxed,
>>                                            self._enum_name))
>> -        self._event_names.append(QAPISchemaMember(name))
>> +        self._event_names.append(QAPISchemaMember(name, ifcond))
>>  
>>  
>>  def gen_events(schema, output_dir, prefix):
>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 17/27] qapi: add 'if' to implicit struct members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Generated code is not changed by this patch, but with "qapi: add #if
> conditions to generated code" patch.

My suggestion on PATCH 13's commit message applies.

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Other than that,
Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 18/27] qapi: add an error in case a discriminator is conditionnal

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Making a discriminator conditonal doesn't make much sense.

Good point (so easy to overlook!), but why first add the 'if' feature
broken that way in PATCH 16, then fix it up in PATCH 18?

>                                                            Instead,
> the union could be made conditional.

What do you mean by that?

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py                          | 11 +++++++++--
>  tests/Makefile.include                          |  1 +
>  .../flat-union-invalid-if-discriminator.err     |  1 +
>  .../flat-union-invalid-if-discriminator.exit    |  1 +
>  .../flat-union-invalid-if-discriminator.json    | 17 +++++++++++++++++
>  .../flat-union-invalid-if-discriminator.out     |  0
>  6 files changed, 29 insertions(+), 2 deletions(-)
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.err
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.exit
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.json
>  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.out
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index 9b95f8cfe9..13fbb28493 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -577,7 +577,8 @@ def find_alternate_member_qtype(qapi_type):
>  
>  # Return the discriminator enum define if discriminator is specified as an
>  # enum type, otherwise return None.
> -def discriminator_find_enum_define(expr):
> +def discriminator_find_enum_define(expr, info):
> +    name = expr['union']
>      base = expr.get('base')
>      discriminator = expr.get('discriminator')
>  
> @@ -592,6 +593,11 @@ def discriminator_find_enum_define(expr):
>      if not discriminator_member:
>          return None
>  
> +    if discriminator_member.get('if'):
> +        raise QAPISemError(info, 'The discriminator %s.%s for union %s '
> +                           'must not be conditional' %
> +                           (base, discriminator, name))
> +
>      return enum_types.get(discriminator_member['type'])
>  
>  

I'm afraid this isn't the proper place to check.  It's an accessor
function for @struct_types and @enum_types.  The check should go into
check_union(), like this:

diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 0cf39df404..c1bc9e9c29 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -799,6 +794,11 @@ def check_union(expr, info):
                                "Discriminator '%s' is not a member of base "
                                "struct '%s'"
                                % (discriminator, base))
+        if discriminator_member.get('if'):
+            raise QAPISemError(info, 'The discriminator %s.%s for union %s '
+                               'must not be conditional' %
+                               (base, discriminator, name))
+
         enum_define = enum_types.get(discriminator_member['type'])
         allow_metas = ['struct']
         # Do not allow string discriminator

> @@ -1020,7 +1026,8 @@ def check_exprs(exprs):
>  
>          if 'include' in expr:
>              continue
> -        if 'union' in expr and not discriminator_find_enum_define(expr):
> +        info = expr_elem['info']
> +        if 'union' in expr and not discriminator_find_enum_define(expr, info):
>              name = '%sKind' % expr['union']
>          elif 'alternate' in expr:
>              name = '%sKind' % expr['alternate']
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 43e100a6cd..abc3fdf764 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -510,6 +510,7 @@ qapi-schema += flat-union-inline.json
>  qapi-schema += flat-union-int-branch.json
>  qapi-schema += flat-union-invalid-branch-key.json
>  qapi-schema += flat-union-invalid-discriminator.json
> +qapi-schema += flat-union-invalid-if-discriminator.json
>  qapi-schema += flat-union-no-base.json
>  qapi-schema += flat-union-optional-discriminator.json
>  qapi-schema += flat-union-string-discriminator.json
> diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.err b/tests/qapi-schema/flat-union-invalid-if-discriminator.err
> new file mode 100644
> index 0000000000..0c94c9860d
> --- /dev/null
> +++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/flat-union-invalid-if-discriminator.json:13: The discriminator TestBase.enum1 for union TestUnion must not be conditional
> diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.exit b/tests/qapi-schema/flat-union-invalid-if-discriminator.exit
> new file mode 100644
> index 0000000000..d00491fd7e
> --- /dev/null
> +++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.json b/tests/qapi-schema/flat-union-invalid-if-discriminator.json
> new file mode 100644
> index 0000000000..618ec36396
> --- /dev/null
> +++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.json
> @@ -0,0 +1,17 @@
> +{ 'enum': 'TestEnum',
> +  'data': [ 'value1', 'value2' ] }
> +
> +{ 'struct': 'TestBase',
> +  'data': { 'enum1': { 'type': 'TestEnum', 'if': 'FOO' } } }
> +
> +{ 'struct': 'TestTypeA',
> +  'data': { 'string': 'str' } }
> +
> +{ 'struct': 'TestTypeB',
> +  'data': { 'integer': 'int' } }
> +
> +{ 'union': 'TestUnion',
> +  'base': 'TestBase',
> +  'discriminator': 'enum1',
> +  'data': { 'value1': 'TestTypeA',
> +            'value2': 'TestTypeB' } }
> diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.out b/tests/qapi-schema/flat-union-invalid-if-discriminator.out
> new file mode 100644
> index 0000000000..e69de29bb2

Re: [Qemu-devel] [PATCH v6 18/27] qapi: add an error in case a discriminator is conditionnal

[Markus Armbruster]

One more thing: in the subject, s/conditionnal/conditional/.

Re: [Qemu-devel] [PATCH v6 19/27] qapi: add 'if' on union members

[Markus Armbruster]

In the subject, s/ on / to /.

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Add 'if' key to union members:
>
> { 'union': 'TestIfUnion', 'data':
>     'mem': { 'type': 'str', 'if': 'COND'} }
>
> Generated code is not changed by this patch but with "qapi: add #if
> conditions to generated code".

My suggestion on PATCH 13's commit message applies.

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py                  | 17 +++++++++--------
>  tests/qapi-schema/qapi-schema-test.json |  7 ++++++-
>  tests/qapi-schema/qapi-schema-test.out  | 10 ++++++++++
>  tests/qapi-schema/test-qapi.py          |  1 +
>  4 files changed, 26 insertions(+), 9 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index 13fbb28493..e1bd9a22ba 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -813,7 +813,7 @@ def check_union(expr, info):
>      for (key, value) in members.items():
>          check_name(info, "Member of union '%s'" % name, key)
>          source = "member '%s' of union '%s'" % (key, name)
> -        check_known_keys(info, source, value, ['type'], [])
> +        check_known_keys(info, source, value, ['type'], ['if'])
>          typ = value['type']
>  
>          # Each value must name a known type
> @@ -1496,8 +1496,8 @@ class QAPISchemaObjectTypeVariants(object):
>  class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
>      role = 'branch'
>  
> -    def __init__(self, name, typ):
> -        QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
> +    def __init__(self, name, typ, ifcond=None):
> +        QAPISchemaObjectTypeMember.__init__(self, name, typ, False, ifcond)
>  
>  
>  class QAPISchemaAlternateType(QAPISchemaType):
> @@ -1777,14 +1777,14 @@ class QAPISchema(object):
>      def _make_variant(self, case, typ):
>          return QAPISchemaObjectTypeVariant(case, typ)
>  
> -    def _make_simple_variant(self, case, typ, info):
> +    def _make_simple_variant(self, case, typ, ifcond, info):
>          if isinstance(typ, list):
>              assert len(typ) == 1
>              typ = self._make_array_type(typ[0], info)
>          typ = self._make_implicit_object_type(
>              typ, info, None, self.lookup_type(typ),
>              'wrapper', [self._make_member('data', typ, None, info)])
> -        return QAPISchemaObjectTypeVariant(case, typ)
> +        return QAPISchemaObjectTypeVariant(case, typ, ifcond)
>  
>      def _def_union_type(self, expr, info, doc):
>          name = expr['union']
> @@ -1802,10 +1802,11 @@ class QAPISchema(object):
>                          for (key, value) in data.items()]
>              members = []
>          else:
> -            variants = [self._make_simple_variant(key, value['type'], info)
> +            variants = [self._make_simple_variant(key, value['type'],
> +                                                  value.get('if'), info)
>                          for (key, value) in data.items()]
> -            typ = self._make_implicit_enum_type(name, info, ifcond,
> -                                                [v.name for v in variants])
> +            enum = [{'name': v.name, 'if': v.ifcond} for v in variants]
> +            typ = self._make_implicit_enum_type(name, info, ifcond, enum)
>              tag_member = QAPISchemaObjectTypeMember('type', typ, False)
>              members = [tag_member]
>          self._def_entity(
> diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> index 3bf440aab4..6d3c6c0b53 100644
> --- a/tests/qapi-schema/qapi-schema-test.json
> +++ b/tests/qapi-schema/qapi-schema-test.json
> @@ -208,9 +208,14 @@
>    [ 'foo', { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ],
>    'if': 'defined(TEST_IF_ENUM)' }
>  
> -{ 'union': 'TestIfUnion', 'data': { 'foo': 'TestStruct' },
> +{ 'union': 'TestIfUnion', 'data':
> +  { 'foo': 'TestStruct',
> +    'union_bar': { 'type': 'str', 'if': 'defined(TEST_IF_UNION_BAR)'} },

"Union Bar" sounds like a cocktail lounge in northeast US :)

Back to serious: this is a simple union.  I'd prefer to test flat
unions.  Changing TestIfUnion accordingly could be done either before
this patch, or on top of this series, the latter not necessarily by
you.  Your choice.

>    'if': 'defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)' }
>  
> +{ 'command': 'TestIfUnionCmd', 'data': { 'union_cmd_arg': 'TestIfUnion' },
> +  'if': 'defined(TEST_IF_UNION)' }
> +

I'm feeling dense... why does this change belong to this patch?

>  { 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
>    'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
>  
> diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> index 71b84878c7..ac1069cf1f 100644
> --- a/tests/qapi-schema/qapi-schema-test.out
> +++ b/tests/qapi-schema/qapi-schema-test.out
> @@ -278,12 +278,22 @@ object q_obj_TestStruct-wrapper
>      member data: TestStruct optional=False
>  enum TestIfUnionKind
>      member foo
> +    member union_bar
> +        if ['defined(TEST_IF_UNION_BAR)']
>      if ['defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)']
>  object TestIfUnion
>      member type: TestIfUnionKind optional=False
>      tag type
>      case foo: q_obj_TestStruct-wrapper
> +    case union_bar: q_obj_str-wrapper
> +        if ['defined(TEST_IF_UNION_BAR)']
>      if ['defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)']
> +object q_obj_TestIfUnionCmd-arg
> +    member union_cmd_arg: TestIfUnion optional=False
> +    if ['defined(TEST_IF_UNION)']
> +command TestIfUnionCmd q_obj_TestIfUnionCmd-arg -> None
> +   gen=True success_response=True boxed=False oob=False preconfig=False
> +    if ['defined(TEST_IF_UNION)']
>  alternate TestIfAlternate
>      tag type
>      case foo: int
> diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
> index 69e40d87d2..d977e936c7 100644
> --- a/tests/qapi-schema/test-qapi.py
> +++ b/tests/qapi-schema/test-qapi.py
> @@ -68,6 +68,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
>              print('    tag %s' % variants.tag_member.name)
>              for v in variants.variants:
>                  print('    case %s: %s' % (v.name, v.type.name))
> +                QAPISchemaTestVisitor._print_if(v.ifcond, 8)
>  
>      @staticmethod
>      def _print_if(ifcond, indent=4):

Re: [Qemu-devel] [PATCH v6 20/27] qapi: add 'if' to alternate members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Add 'if' key to alternate members:
>
> { 'alternate': 'TestIfAlternate', 'data':
>   { 'alt': { 'type': 'TestStruct', 'if': 'COND' } } }
>
> Generated code is not changed by this patch but with "qapi: add #if
> conditions to generated code".
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py                  | 10 +++++-----
>  tests/qapi-schema/qapi-schema-test.json |  6 +++++-
>  tests/qapi-schema/qapi-schema-test.out  |  9 ++++++++-
>  3 files changed, 18 insertions(+), 7 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index e1bd9a22ba..b3b64a60bf 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -843,7 +843,7 @@ def check_alternate(expr, info):
>      for (key, value) in members.items():
>          check_name(info, "Member of alternate '%s'" % name, key)
>          source = "member '%s' of alternate '%s'" % (key, name)
> -        check_known_keys(info, source, value, ['type'], [])
> +        check_known_keys(info, source, value, ['type'], ['if'])
>          typ = value['type']
>  
>          # Ensure alternates have no type conflicts.
> @@ -1774,8 +1774,8 @@ class QAPISchema(object):
>                                                self._make_members(data, info),
>                                                None))
>  
> -    def _make_variant(self, case, typ):
> -        return QAPISchemaObjectTypeVariant(case, typ)
> +    def _make_variant(self, case, typ, ifcond):
> +        return QAPISchemaObjectTypeVariant(case, typ, ifcond)
>  
>      def _make_simple_variant(self, case, typ, ifcond, info):
>          if isinstance(typ, list):
> @@ -1798,7 +1798,7 @@ class QAPISchema(object):
>                  name, info, doc, ifcond,
>                  'base', self._make_members(base, info))
>          if tag_name:
> -            variants = [self._make_variant(key, value['type'])
> +            variants = [self._make_variant(key, value['type'], value.get('if'))
>                          for (key, value) in data.items()]
>              members = []
>          else:
> @@ -1819,7 +1819,7 @@ class QAPISchema(object):
>          name = expr['alternate']
>          data = expr['data']
>          ifcond = expr.get('if')
> -        variants = [self._make_variant(key, value['type'])
> +        variants = [self._make_variant(key, value['type'], value.get('if'))
>                      for (key, value) in data.items()]
>          tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
>          self._def_entity(
> diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> index 6d3c6c0b53..df3edf9d89 100644
> --- a/tests/qapi-schema/qapi-schema-test.json
> +++ b/tests/qapi-schema/qapi-schema-test.json
> @@ -216,9 +216,13 @@
>  { 'command': 'TestIfUnionCmd', 'data': { 'union_cmd_arg': 'TestIfUnion' },
>    'if': 'defined(TEST_IF_UNION)' }
>  
> -{ 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
> +{ 'alternate': 'TestIfAlternate', 'data':
> +  { 'foo': 'int', 'alt_bar': { 'type': 'TestStruct', 'if': 'defined(TEST_IF_ALT_BAR)'} },

Let's break the long line betwen the members.

Why rename member 'bar' to 'alt_bar'?

>    'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
>  
> +{ 'command': 'TestIfAlternateCmd', 'data': { 'alt_cmd_arg': 'TestIfAlternate' },

Another long line.  And I'm feeling dense again: why does this change
belong to this patch?

> +  'if': 'defined(TEST_IF_ALT)' }
> +
>  { 'command': 'TestIfCmd', 'data':
>    { 'foo': 'TestIfStruct',
>      'bar': { 'type': 'TestIfEnum', 'if': 'defined(TEST_IF_CMD_BAR)' } },
> diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> index ac1069cf1f..cdbd5b87cc 100644
> --- a/tests/qapi-schema/qapi-schema-test.out
> +++ b/tests/qapi-schema/qapi-schema-test.out
> @@ -297,8 +297,15 @@ command TestIfUnionCmd q_obj_TestIfUnionCmd-arg -> None
>  alternate TestIfAlternate
>      tag type
>      case foo: int
> -    case bar: TestStruct
> +    case alt_bar: TestStruct
> +        if ['defined(TEST_IF_ALT_BAR)']
>      if ['defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)']
> +object q_obj_TestIfAlternateCmd-arg
> +    member alt_cmd_arg: TestIfAlternate optional=False
> +    if ['defined(TEST_IF_ALT)']
> +command TestIfAlternateCmd q_obj_TestIfAlternateCmd-arg -> None
> +   gen=True success_response=True boxed=False oob=False preconfig=False
> +    if ['defined(TEST_IF_ALT)']
>  object q_obj_TestIfCmd-arg
>      member foo: TestIfStruct optional=False
>      member bar: TestIfEnum optional=False

Re: [Qemu-devel] [PATCH v6 21/27] qapi: add #if conditions to generated code members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Wrap generated enum/struct members and code with #if/#endif, using the

enum and struct members

> .ifcond members added in the previous patches.
>
> Some types generate both enum and struct members for example, so a
> step-by-step is unnecessarily complicated to deal with (it would
> easily generate invalid intermediary code).

Can you give an example of a schema definition that would lead to
complications?

>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/common.py     |  4 ++++
>  scripts/qapi/introspect.py | 13 +++++++++----
>  scripts/qapi/types.py      |  4 ++++
>  scripts/qapi/visit.py      |  6 ++++++
>  4 files changed, 23 insertions(+), 4 deletions(-)
>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index b3b64a60bf..a66c035b91 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -2098,11 +2098,13 @@ const QEnumLookup %(c_name)s_lookup = {
>  ''',
>                  c_name=c_name(name))
>      for m in members:
> +        ret += gen_if(m.ifcond)
>          index = c_enum_const(name, m.name, prefix)
>          ret += mcgen('''
>          [%(index)s] = "%(name)s",
>  ''',
>                       index=index, name=m.name)
> +        ret += gen_endif(m.ifcond)
>  
>      ret += mcgen('''
>      },
> @@ -2124,10 +2126,12 @@ typedef enum %(c_name)s {
>                  c_name=c_name(name))
>  
>      for m in enum_members:
> +        ret += gen_if(m.ifcond)
>          ret += mcgen('''
>      %(c_enum)s,
>  ''',
>                       c_enum=c_enum_const(name, m.name, prefix))
> +        ret += gen_endif(m.ifcond)
>  
>      ret += mcgen('''
>  } %(c_name)s;
> diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
> index 3f1ca99f6d..bf5db51f4a 100644
> --- a/scripts/qapi/introspect.py
> +++ b/scripts/qapi/introspect.py
> @@ -148,6 +148,8 @@ const QLitObject %(c_name)s = %(c_string)s;
>          ret = {'name': member.name, 'type': self._use_type(member.type)}
>          if member.optional:
>              ret['default'] = None
> +        if member.ifcond:
> +            ret = (ret, member.ifcond)
>          return ret
>  
>      def _gen_variants(self, tag_name, variants):
> @@ -155,14 +157,16 @@ const QLitObject %(c_name)s = %(c_string)s;
>                  'variants': [self._gen_variant(v) for v in variants]}
>  
>      def _gen_variant(self, variant):
> -        return {'case': variant.name, 'type': self._use_type(variant.type)}
> +        return ({'case': variant.name, 'type': self._use_type(variant.type)},
> +                variant.ifcond)

Looks different in your rebased version at
https://github.com/elmarco/qemu.git branch qapi-if.  I'm only skimming
this version.

Note to self: always creates the tuple form, even when there's no
if-condition.  Differs from _gen_qlit(), which creates a tuple only when
there's a if-condition.  Not wrong, just a bit inconsistent.

>  
>      def visit_builtin_type(self, name, info, json_type):
>          self._gen_qlit(name, 'builtin', {'json-type': json_type}, [])
>  
>      def visit_enum_type(self, name, info, ifcond, members, prefix):
>          self._gen_qlit(name, 'enum',
> -                       {'values': [m.name for m in members]}, ifcond)
> +                       {'values': [(m.name, m.ifcond) for m in members]},
> +                       ifcond)

Likewise.

>  
>      def visit_array_type(self, name, info, ifcond, element_type):
>          element = self._use_type(element_type)
> @@ -178,8 +182,9 @@ const QLitObject %(c_name)s = %(c_string)s;
>  
>      def visit_alternate_type(self, name, info, ifcond, variants):
>          self._gen_qlit(name, 'alternate',
> -                       {'members': [{'type': self._use_type(m.type)}
> -                                    for m in variants.variants]}, ifcond)
> +                       {'members': [
> +                           ({'type': self._use_type(m.type)}, m.ifcond)
> +                           for m in variants.variants]}, ifcond)

Likewise.

>  
>      def visit_command(self, name, info, ifcond, arg_type, ret_type, gen,
>                        success_response, boxed, allow_oob, allow_preconfig):
> diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
> index 2d4a70f810..7d9eef6320 100644
> --- a/scripts/qapi/types.py
> +++ b/scripts/qapi/types.py
> @@ -43,6 +43,7 @@ struct %(c_name)s {
>  def gen_struct_members(members):
>      ret = ''
>      for memb in members:
> +        ret += gen_if(memb.ifcond)
>          if memb.optional:
>              ret += mcgen('''
>      bool has_%(c_name)s;
> @@ -52,6 +53,7 @@ def gen_struct_members(members):
>      %(c_type)s %(c_name)s;
>  ''',
>                       c_type=memb.type.c_type(), c_name=c_name(memb.name))
> +        ret += gen_endif(memb.ifcond)
>      return ret
>  
>  
> @@ -131,11 +133,13 @@ def gen_variants(variants):
>      for var in variants.variants:
>          if var.type.name == 'q_empty':
>              continue
> +        ret += gen_if(var.ifcond)
>          ret += mcgen('''
>          %(c_type)s %(c_name)s;
>  ''',
>                       c_type=var.type.c_unboxed_type(),
>                       c_name=c_name(var.name))
> +        ret += gen_endif(var.ifcond)
>  
>      ret += mcgen('''
>      } u;
> diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
> index 24f85a2e85..82eab72b21 100644
> --- a/scripts/qapi/visit.py
> +++ b/scripts/qapi/visit.py
> @@ -54,6 +54,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
>                       c_type=base.c_name())
>  
>      for memb in members:
> +        ret += gen_if(memb.ifcond)
>          if memb.optional:
>              ret += mcgen('''
>      if (visit_optional(v, "%(name)s", &obj->has_%(c_name)s)) {
> @@ -73,6 +74,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
>              ret += mcgen('''
>      }
>  ''')
> +        ret += gen_endif(memb.ifcond)
>  
>      if variants:
>          ret += mcgen('''
> @@ -84,6 +86,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
>              case_str = c_enum_const(variants.tag_member.type.name,
>                                      var.name,
>                                      variants.tag_member.type.prefix)
> +            ret += gen_if(var.ifcond)
>              if var.type.name == 'q_empty':
>                  # valid variant and nothing to do
>                  ret += mcgen('''
> @@ -100,6 +103,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
>                               case=case_str,
>                               c_type=var.type.c_name(), c_name=c_name(var.name))
>  
> +            ret += gen_endif(var.ifcond)
>          ret += mcgen('''
>      default:
>          abort();
> @@ -190,6 +194,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
>                  c_name=c_name(name))
>  
>      for var in variants.variants:
> +        ret += gen_if(var.ifcond)
>          ret += mcgen('''
>      case %(case)s:
>  ''',
> @@ -217,6 +222,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
>          ret += mcgen('''
>          break;
>  ''')
> +        ret += gen_endif(var.ifcond)
>  
>      ret += mcgen('''
>      case QTYPE_NONE:

Re: [Qemu-devel] [PATCH v6 22/27] docs: document schema configuration

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  docs/devel/qapi-code-gen.txt | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
>
> diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
> index c2e11465f0..799aef7728 100644
> --- a/docs/devel/qapi-code-gen.txt
> +++ b/docs/devel/qapi-code-gen.txt
> @@ -752,6 +752,25 @@ gets its generated code guarded like this:
>   #endif /* defined(HAVE_BAR) */
>   #endif /* defined(CONFIG_FOO) */
>  
> +Where a member can be defined with a single string value for its type,
> +it is also possible to supply a dictionary instead with both 'type'
> +and 'if' keys.
> +
> +Example: a conditional 'bar' member
> +
> +{ 'struct': 'IfStruct', 'data':
> +  { 'foo': 'int',
> +    'bar': { 'type': 'int', 'if': 'defined(IFCOND)'} } }
> +
> +An enum value can be replaced by a dictionary with a 'name' and a 'if'
> +key.
> +
> +Example: a conditional 'bar' enum member.
> +
> +{ 'enum': 'IfEnum', 'data':
> +  [ 'foo',
> +    { 'name' : 'bar', 'if': 'defined(IFCOND)' } ] }
> +
>  Please note that you are responsible to ensure that the C code will
>  compile with an arbitrary combination of conditions, since the
>  generators are unable to check it at this point.

I'd prefer to update qapi-code-gen.txt right when we extend the schema
language, like you did in part 1 (commit 967c885108f qapi: add 'if' to
top-level expressions).  I understand it's a bit more churn, since you
need four steps (enum, struct, union, alternate members).  I think the
following would be the least work for you that still satisfies me:

* Add the part about enum values in PATCH 13.

* Add the rest in PATCH 17 (the first patch that implements a part of
  it), with a TODO implement <list of things that get done in later
  patches>.  Then just update the TODO as you go.

Okay?

Re: [Qemu-devel] [PATCH v6 23/27] qapi: add 'If:' condition to enum values documentation

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/doc.py             | 5 ++++-
>  tests/qapi-schema/doc-good.json | 4 +++-
>  tests/qapi-schema/doc-good.out  | 1 +
>  tests/qapi-schema/doc-good.texi | 2 ++
>  4 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
> index 76cb186ff9..196deea11d 100755
> --- a/scripts/qapi/doc.py
> +++ b/scripts/qapi/doc.py
> @@ -128,7 +128,10 @@ def texi_body(doc):
>  
>  def texi_enum_value(value):
>      """Format a table of members item for an enumeration value"""
> -    return '@item @code{%s}\n' % value.name
> +    return '@item @code{%s}%s\n' % (
> +        value.name,
> +        '\n@b{If:} @code{%s}\n' %
> +        ', '.join(value.ifcond) if value.ifcond else '')
>  
>  

This code is a puzzle.  I don't like code puzzles.

>  def texi_member(member, suffix=''):
> diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
> index 984cd8ed06..c7fe08c530 100644
> --- a/tests/qapi-schema/doc-good.json
> +++ b/tests/qapi-schema/doc-good.json
> @@ -55,7 +55,9 @@
>  #
>  # @two is undocumented
>  ##
> -{ 'enum': 'Enum', 'data': [ 'one', 'two' ], 'if': 'defined(IFCOND)' }
> +{ 'enum': 'Enum', 'data':
> +  [ { 'name': 'one', 'if': 'defined(IFENUM)' }, 'two' ],
> +  'if': 'defined(IFCOND)' }
>  
>  ##
>  # @Base:
> diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
> index c2fc5c774a..a05535b69b 100644
> --- a/tests/qapi-schema/doc-good.out
> +++ b/tests/qapi-schema/doc-good.out
> @@ -11,6 +11,7 @@ enum QType
>  module doc-good.json
>  enum Enum
>      member one
> +        if ['defined(IFENUM)']
>      member two
>      if ['defined(IFCOND)']
>  object Base
> diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
> index e42eace474..2446a1c702 100644
> --- a/tests/qapi-schema/doc-good.texi
> +++ b/tests/qapi-schema/doc-good.texi
> @@ -83,6 +83,8 @@ Examples:
>  @b{Values:}
>  @table @asis
>  @item @code{one}
> +@b{If:} @code{defined(IFENUM)}
> +
>  The @emph{one} @{and only@}
>  @item @code{two}
>  Not documented

Odd: the @item is followed by a blank line only if its conditional.
Formatted to plain text, it looks like this:

 -- Enum: Enum

     Values:
     'one'
          If: 'defined(IFENUM)'

          The _one_ {and only}
     'two'
          Not documented
     'two' is undocumented

     If: 'defined(IFCOND)'

Odd^2: the "If:" line comes first for members, last for top-level
expressions like types.

Re: [Qemu-devel] [PATCH v6 24/27] qapi: add 'If:' condition to struct members documentation

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  scripts/qapi/doc.py             | 4 +++-
>  tests/qapi-schema/doc-good.json | 3 ++-
>  tests/qapi-schema/doc-good.out  | 1 +
>  tests/qapi-schema/doc-good.texi | 2 ++
>  4 files changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
> index 196deea11d..56dc98c068 100755
> --- a/scripts/qapi/doc.py
> +++ b/scripts/qapi/doc.py
> @@ -138,9 +138,11 @@ def texi_member(member, suffix=''):
>      """Format a table of members item for an object type member"""
>      typ = member.type.doc_type()
>      membertype = ': ' + typ if typ else ''
> -    return '@item @code{%s%s}%s%s\n' % (
> +    return '@item @code{%s%s}%s%s%s\n' % (
>          member.name, membertype,
>          ' (optional)' if member.optional else '',
> +        '\n@b{If:} @code{%s}\n' %
> +        ', '.join(member.ifcond) if member.ifcond else '',
>          suffix)
>  
>  
> diff --git a/tests/qapi-schema/doc-good.json b/tests/qapi-schema/doc-good.json
> index c7fe08c530..158443b1a3 100644
> --- a/tests/qapi-schema/doc-good.json
> +++ b/tests/qapi-schema/doc-good.json
> @@ -72,7 +72,8 @@
>  #
>  # Another paragraph (but no @var: line)
>  ##
> -{ 'struct': 'Variant1', 'data': { 'var1': 'str' } }
> +{ 'struct': 'Variant1',
> +  'data': { 'var1': { 'type': 'str', 'if': 'defined(IFSTR)' } } }
>  
>  ##
>  # @Variant2:
> diff --git a/tests/qapi-schema/doc-good.out b/tests/qapi-schema/doc-good.out
> index a05535b69b..c310b47be2 100644
> --- a/tests/qapi-schema/doc-good.out
> +++ b/tests/qapi-schema/doc-good.out
> @@ -18,6 +18,7 @@ object Base
>      member base1: Enum optional=False
>  object Variant1
>      member var1: str optional=False
> +        if ['defined(IFSTR)']
>  object Variant2
>  object Object
>      base Base
> diff --git a/tests/qapi-schema/doc-good.texi b/tests/qapi-schema/doc-good.texi
> index 2446a1c702..255eed9a30 100644
> --- a/tests/qapi-schema/doc-good.texi
> +++ b/tests/qapi-schema/doc-good.texi
> @@ -120,6 +120,8 @@ Another paragraph (but no @code{var}: line)
>  @b{Members:}
>  @table @asis
>  @item @code{var1: string}
> +@b{If:} @code{defined(IFSTR)}
> +
>  Not documented
>  @end table

Comments on the previous patch apply.

Re: [Qemu-devel] [PATCH v6 26/27] qapi: add more conditions to SPICE

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Now that member can be made conditional, let's make SPICE chardev
> conditional:
>
> * spiceport, spicevmc
>
>   Before and after the patch for !CONFIG_SPICE, the error is the
>   same ('spiceport' is not a valid char driver name).
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  qapi/char.json | 16 ++++++++--------
>  1 file changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/qapi/char.json b/qapi/char.json
> index b7b2a05766..54e3b970f6 100644
> --- a/qapi/char.json
> +++ b/qapi/char.json
> @@ -319,8 +319,8 @@
>  # Since: 1.5
>  ##
>  { 'struct': 'ChardevSpiceChannel', 'data': { 'type'  : 'str' },
> -  'base': 'ChardevCommon' }
> -# TODO: 'if': 'defined(CONFIG_SPICE)'
> +  'base': 'ChardevCommon',
> +  'if': 'defined(CONFIG_SPICE)' }
>  
>  ##
>  # @ChardevSpicePort:
> @@ -332,8 +332,8 @@
>  # Since: 1.5
>  ##
>  { 'struct': 'ChardevSpicePort', 'data': { 'fqdn'  : 'str' },
> -  'base': 'ChardevCommon' }
> -# TODO: 'if': 'defined(CONFIG_SPICE)'
> +  'base': 'ChardevCommon',
> +  'if': 'defined(CONFIG_SPICE)' }
>  
>  ##
>  # @ChardevVC:
> @@ -387,10 +387,10 @@
>                                         'testdev': 'ChardevCommon',
>                                         'stdio'  : 'ChardevStdio',
>                                         'console': 'ChardevCommon',
> -                                       'spicevmc': 'ChardevSpiceChannel',
> -# TODO: { 'type': 'ChardevSpiceChannel', 'if': 'defined(CONFIG_SPICE)' },
> -                                       'spiceport': 'ChardevSpicePort',
> -# TODO: { 'type': 'ChardevSpicePort', 'if': 'defined(CONFIG_SPICE)' },
> +                                       'spicevmc': { 'type': 'ChardevSpiceChannel',
> +                                                     'if': 'defined(CONFIG_SPICE)' },
> +                                       'spiceport': { 'type': 'ChardevSpicePort',
> +                                                      'if': 'defined(CONFIG_SPICE)' },
>                                         'vc'     : 'ChardevVC',
>                                         'ringbuf': 'ChardevRingbuf',
>                                         # next one is just for compatibility

Long lines due to foolish indentation.

Let me grep for the pattern:

    $ grep ", 'data.*,$" qapi/*json
    qapi/block-core.json:  'data': {'start': 'int', 'length': 'int', 'data': 'bool',
    qapi/block-core.json:{ 'command': 'block_passwd', 'data': {'*device': 'str',
    qapi/block-core.json:{ 'command': 'block_resize', 'data': { '*device': 'str',
    qapi/block-core.json:{ 'enum' : 'ReplicationMode', 'data' : [ 'primary', 'secondary' ],
    qapi/char.json:{ 'struct': 'ChardevInfo', 'data': {'label': 'str',
    qapi/char.json:  'data': {'device': 'str', 'data': 'str',
    qapi/char.json:{ 'struct': 'ChardevCommon', 'data': { '*logfile': 'str',
    qapi/char.json:{ 'struct': 'ChardevFile', 'data': { '*in' : 'str',
    qapi/char.json:{ 'struct': 'ChardevHostdev', 'data': { 'device' : 'str' },
    qapi/char.json:{ 'struct': 'ChardevSocket', 'data': { 'addr'       : 'SocketAddressLegacy',
    qapi/char.json:{ 'struct': 'ChardevUdp', 'data': { 'remote' : 'SocketAddressLegacy',
    qapi/char.json:{ 'struct': 'ChardevMux', 'data': { 'chardev' : 'str' },
    qapi/char.json:{ 'struct': 'ChardevStdio', 'data': { '*signal' : 'bool' },
    qapi/char.json:{ 'struct': 'ChardevSpiceChannel', 'data': { 'type'  : 'str' },
    qapi/char.json:{ 'struct': 'ChardevSpicePort', 'data': { 'fqdn'  : 'str' },
    qapi/char.json:{ 'struct': 'ChardevVC', 'data': { '*width'  : 'int',
    qapi/char.json:{ 'struct': 'ChardevRingbuf', 'data': { '*size'  : 'int' },
    qapi/char.json:{ 'union': 'ChardevBackend', 'data': { 'file'   : 'ChardevFile',
    qapi/char.json:{ 'command': 'chardev-add', 'data': {'id'      : 'str',
    qapi/char.json:{ 'command': 'chardev-change', 'data': {'id'      : 'str',
    qapi/migration.json:{ 'command': 'migrate-recover', 'data': { 'uri': 'str' },
    qapi/misc.json:{ 'command': 'add-fd', 'data': {'*fdset-id': 'int', '*opaque': 'str'},
    qapi/misc.json:{'command': 'query-command-line-options', 'data': { '*option': 'str' },
    qapi/net.json:{ 'command': 'query-rx-filter', 'data': { '*name': 'str' },
    qapi/tpm.json:{ 'struct': 'TPMPassthroughOptions', 'data': { '*path' : 'str',
    qapi/ui.json:{ 'command': 'change-vnc-password', 'data': {'password': 'str'},
The first hit is a false positive.  For the rest, let's break the line
before 'data'.

Re: [Qemu-devel] [PATCH v6 27/27] qapi: add conditions to REPLICATION type/commands on the schema

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Add #if defined(CONFIG_REPLICATION) in generated code, and adjust the
> code accordingly.
>
> Made conditional:
>
> * xen-set-replication, query-xen-replication-status,
>   xen-colo-do-checkpoint
>
>   Before the patch, we first register the commands unconditionally in
>   generated code (requires a stub), then conditionally unregister in
>   qmp_unregister_commands_hack().
>
>   Afterwards, we register only when CONFIG_REPLICATION.  The command
>   fails exactly the same, with CommandNotFound.
>
>   Improvement, because now query-qmp-schema is accurate, and we're one
>   step closer to killing qmp_unregister_commands_hack().
>
> * enum BlockdevDriver value "replication" in command blockdev-add
>
> * BlockdevOptions variant @replication
>
> And related structures.

This sentence no verb :)

>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Reviewed-by: Markus Armbruster <armbru@redhat.com>

Re: [Qemu-devel] [PATCH v6 00/27] qapi: add #if pre-processor conditions to generated code (part 2)

[Markus Armbruster]

The patches are neat.  Most of my review comments are minor.  Looking
forward to v7.  Thanks!

Re: [Qemu-devel] [PATCH v6 10/27] qapi: factor out checking for keys

[Marc-André Lureau]

On Wed, Dec 5, 2018 at 8:26 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Introduce a new helper function to check if the given keys are known,
> > and if mandatory keys are present. The function will be reused in
> > other places in the following code changes.
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py | 20 +++++++++++++-------
> >  1 file changed, 13 insertions(+), 7 deletions(-)
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index a353670079..8313c478c4 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -873,6 +873,17 @@ def check_struct(expr, info):
> >                 allow_metas=['struct'])
> >
> >
> > +def check_known_keys(info, source, keys, required, optional):
> > +    for key in keys:
> > +        if key not in required and key not in optional:
> > +            raise QAPISemError(info, "Unknown key '%s' in %s" % (key, source))
> > +
> > +    for key in required:
> > +        if key not in keys:
> > +            raise QAPISemError(info, "Key '%s' is missing from %s"
> > +                               % (key, source))
> > +
> > +
> >  def check_keys(expr_elem, meta, required, optional=[]):
> >      expr = expr_elem['expr']
> >      info = expr_elem['info']
> > @@ -880,10 +891,9 @@ def check_keys(expr_elem, meta, required, optional=[]):
> >      if not isinstance(name, str):
> >          raise QAPISemError(info, "'%s' key must have a string value" % meta)
> >      required = required + [meta]
> > +    source = "%s '%s'" % (meta, name)
> > +    check_known_keys(info, source, list(expr.keys()), required, optional)
>
> Sure you need list() here?

I don't know, that's what python-modernize suggests.

Also the top answer from:
https://stackoverflow.com/questions/16819222/how-to-return-dictionary-keys-as-a-list-in-python#16819250

Do you mind if I leave the list() ? (mostly for python-modernize to be happy)


>
> >      for (key, value) in expr.items():
> > -        if key not in required and key not in optional:
> > -            raise QAPISemError(info, "Unknown key '%s' in %s '%s'"
> > -                               % (key, meta, name))
> >          if key in ['gen', 'success-response'] and value is not False:
> >              raise QAPISemError(info,
> >                                 "'%s' of %s '%s' should only use false value"
> > @@ -895,10 +905,6 @@ def check_keys(expr_elem, meta, required, optional=[]):
> >                                 % (key, meta, name))
> >          if key == 'if':
> >              check_if(expr, info)
> > -    for key in required:
> > -        if key not in expr:
> > -            raise QAPISemError(info, "Key '%s' is missing from %s '%s'"
> > -                               % (key, meta, name))
> >
> >
> >  def check_exprs(exprs):

Re: [Qemu-devel] [PATCH v6 21/27] qapi: add #if conditions to generated code members

[Marc-André Lureau]

Hi
On Thu, Dec 6, 2018 at 9:42 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Wrap generated enum/struct members and code with #if/#endif, using the
>
> enum and struct members

ok

>
> > .ifcond members added in the previous patches.
> >
> > Some types generate both enum and struct members for example, so a
> > step-by-step is unnecessarily complicated to deal with (it would
> > easily generate invalid intermediary code).
>
> Can you give an example of a schema definition that would lead to
> complications?
>

Honestly, I don't remember well (it's been a while I wrote that code).
It must be related to implicit enums, such as union kind... If there
is no strong need to split this patch, I would rather not do that
extra work.

> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py     |  4 ++++
> >  scripts/qapi/introspect.py | 13 +++++++++----
> >  scripts/qapi/types.py      |  4 ++++
> >  scripts/qapi/visit.py      |  6 ++++++
> >  4 files changed, 23 insertions(+), 4 deletions(-)
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index b3b64a60bf..a66c035b91 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -2098,11 +2098,13 @@ const QEnumLookup %(c_name)s_lookup = {
> >  ''',
> >                  c_name=c_name(name))
> >      for m in members:
> > +        ret += gen_if(m.ifcond)
> >          index = c_enum_const(name, m.name, prefix)
> >          ret += mcgen('''
> >          [%(index)s] = "%(name)s",
> >  ''',
> >                       index=index, name=m.name)
> > +        ret += gen_endif(m.ifcond)
> >
> >      ret += mcgen('''
> >      },
> > @@ -2124,10 +2126,12 @@ typedef enum %(c_name)s {
> >                  c_name=c_name(name))
> >
> >      for m in enum_members:
> > +        ret += gen_if(m.ifcond)
> >          ret += mcgen('''
> >      %(c_enum)s,
> >  ''',
> >                       c_enum=c_enum_const(name, m.name, prefix))
> > +        ret += gen_endif(m.ifcond)
> >
> >      ret += mcgen('''
> >  } %(c_name)s;
> > diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
> > index 3f1ca99f6d..bf5db51f4a 100644
> > --- a/scripts/qapi/introspect.py
> > +++ b/scripts/qapi/introspect.py
> > @@ -148,6 +148,8 @@ const QLitObject %(c_name)s = %(c_string)s;
> >          ret = {'name': member.name, 'type': self._use_type(member.type)}
> >          if member.optional:
> >              ret['default'] = None
> > +        if member.ifcond:
> > +            ret = (ret, member.ifcond)
> >          return ret
> >
> >      def _gen_variants(self, tag_name, variants):
> > @@ -155,14 +157,16 @@ const QLitObject %(c_name)s = %(c_string)s;
> >                  'variants': [self._gen_variant(v) for v in variants]}
> >
> >      def _gen_variant(self, variant):
> > -        return {'case': variant.name, 'type': self._use_type(variant.type)}
> > +        return ({'case': variant.name, 'type': self._use_type(variant.type)},
> > +                variant.ifcond)
>
> Looks different in your rebased version at
> https://github.com/elmarco/qemu.git branch qapi-if.  I'm only skimming
> this version.
>
> Note to self: always creates the tuple form, even when there's no
> if-condition.  Differs from _gen_qlit(), which creates a tuple only when
> there's a if-condition.  Not wrong, just a bit inconsistent.
>
> >
> >      def visit_builtin_type(self, name, info, json_type):
> >          self._gen_qlit(name, 'builtin', {'json-type': json_type}, [])
> >
> >      def visit_enum_type(self, name, info, ifcond, members, prefix):
> >          self._gen_qlit(name, 'enum',
> > -                       {'values': [m.name for m in members]}, ifcond)
> > +                       {'values': [(m.name, m.ifcond) for m in members]},
> > +                       ifcond)
>
> Likewise.
>
> >
> >      def visit_array_type(self, name, info, ifcond, element_type):
> >          element = self._use_type(element_type)
> > @@ -178,8 +182,9 @@ const QLitObject %(c_name)s = %(c_string)s;
> >
> >      def visit_alternate_type(self, name, info, ifcond, variants):
> >          self._gen_qlit(name, 'alternate',
> > -                       {'members': [{'type': self._use_type(m.type)}
> > -                                    for m in variants.variants]}, ifcond)
> > +                       {'members': [
> > +                           ({'type': self._use_type(m.type)}, m.ifcond)
> > +                           for m in variants.variants]}, ifcond)
>
> Likewise.
>
> >
> >      def visit_command(self, name, info, ifcond, arg_type, ret_type, gen,
> >                        success_response, boxed, allow_oob, allow_preconfig):
> > diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
> > index 2d4a70f810..7d9eef6320 100644
> > --- a/scripts/qapi/types.py
> > +++ b/scripts/qapi/types.py
> > @@ -43,6 +43,7 @@ struct %(c_name)s {
> >  def gen_struct_members(members):
> >      ret = ''
> >      for memb in members:
> > +        ret += gen_if(memb.ifcond)
> >          if memb.optional:
> >              ret += mcgen('''
> >      bool has_%(c_name)s;
> > @@ -52,6 +53,7 @@ def gen_struct_members(members):
> >      %(c_type)s %(c_name)s;
> >  ''',
> >                       c_type=memb.type.c_type(), c_name=c_name(memb.name))
> > +        ret += gen_endif(memb.ifcond)
> >      return ret
> >
> >
> > @@ -131,11 +133,13 @@ def gen_variants(variants):
> >      for var in variants.variants:
> >          if var.type.name == 'q_empty':
> >              continue
> > +        ret += gen_if(var.ifcond)
> >          ret += mcgen('''
> >          %(c_type)s %(c_name)s;
> >  ''',
> >                       c_type=var.type.c_unboxed_type(),
> >                       c_name=c_name(var.name))
> > +        ret += gen_endif(var.ifcond)
> >
> >      ret += mcgen('''
> >      } u;
> > diff --git a/scripts/qapi/visit.py b/scripts/qapi/visit.py
> > index 24f85a2e85..82eab72b21 100644
> > --- a/scripts/qapi/visit.py
> > +++ b/scripts/qapi/visit.py
> > @@ -54,6 +54,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
> >                       c_type=base.c_name())
> >
> >      for memb in members:
> > +        ret += gen_if(memb.ifcond)
> >          if memb.optional:
> >              ret += mcgen('''
> >      if (visit_optional(v, "%(name)s", &obj->has_%(c_name)s)) {
> > @@ -73,6 +74,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
> >              ret += mcgen('''
> >      }
> >  ''')
> > +        ret += gen_endif(memb.ifcond)
> >
> >      if variants:
> >          ret += mcgen('''
> > @@ -84,6 +86,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
> >              case_str = c_enum_const(variants.tag_member.type.name,
> >                                      var.name,
> >                                      variants.tag_member.type.prefix)
> > +            ret += gen_if(var.ifcond)
> >              if var.type.name == 'q_empty':
> >                  # valid variant and nothing to do
> >                  ret += mcgen('''
> > @@ -100,6 +103,7 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s *obj, Error **errp)
> >                               case=case_str,
> >                               c_type=var.type.c_name(), c_name=c_name(var.name))
> >
> > +            ret += gen_endif(var.ifcond)
> >          ret += mcgen('''
> >      default:
> >          abort();
> > @@ -190,6 +194,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
> >                  c_name=c_name(name))
> >
> >      for var in variants.variants:
> > +        ret += gen_if(var.ifcond)
> >          ret += mcgen('''
> >      case %(case)s:
> >  ''',
> > @@ -217,6 +222,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, Error
> >          ret += mcgen('''
> >          break;
> >  ''')
> > +        ret += gen_endif(var.ifcond)
> >
> >      ret += mcgen('''
> >      case QTYPE_NONE:

thanks

Re: [Qemu-devel] [PATCH v6 22/27] docs: document schema configuration

[Marc-André Lureau]

On Thu, Dec 6, 2018 at 10:14 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  docs/devel/qapi-code-gen.txt | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> >
> > diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
> > index c2e11465f0..799aef7728 100644
> > --- a/docs/devel/qapi-code-gen.txt
> > +++ b/docs/devel/qapi-code-gen.txt
> > @@ -752,6 +752,25 @@ gets its generated code guarded like this:
> >   #endif /* defined(HAVE_BAR) */
> >   #endif /* defined(CONFIG_FOO) */
> >
> > +Where a member can be defined with a single string value for its type,
> > +it is also possible to supply a dictionary instead with both 'type'
> > +and 'if' keys.
> > +
> > +Example: a conditional 'bar' member
> > +
> > +{ 'struct': 'IfStruct', 'data':
> > +  { 'foo': 'int',
> > +    'bar': { 'type': 'int', 'if': 'defined(IFCOND)'} } }
> > +
> > +An enum value can be replaced by a dictionary with a 'name' and a 'if'
> > +key.
> > +
> > +Example: a conditional 'bar' enum member.
> > +
> > +{ 'enum': 'IfEnum', 'data':
> > +  [ 'foo',
> > +    { 'name' : 'bar', 'if': 'defined(IFCOND)' } ] }
> > +
> >  Please note that you are responsible to ensure that the C code will
> >  compile with an arbitrary combination of conditions, since the
> >  generators are unable to check it at this point.
>
> I'd prefer to update qapi-code-gen.txt right when we extend the schema
> language, like you did in part 1 (commit 967c885108f qapi: add 'if' to
> top-level expressions).  I understand it's a bit more churn, since you
> need four steps (enum, struct, union, alternate members).  I think the
> following would be the least work for you that still satisfies me:
>
> * Add the part about enum values in PATCH 13.
>
> * Add the rest in PATCH 17 (the first patch that implements a part of
>   it), with a TODO implement <list of things that get done in later
>   patches>.  Then just update the TODO as you go.
>
> Okay?

Easy enough, done.

Re: [Qemu-devel] [PATCH v6 20/27] qapi: add 'if' to alternate members

[Marc-André Lureau]

Hi
On Thu, Dec 6, 2018 at 8:41 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Add 'if' key to alternate members:
> >
> > { 'alternate': 'TestIfAlternate', 'data':
> >   { 'alt': { 'type': 'TestStruct', 'if': 'COND' } } }
> >
> > Generated code is not changed by this patch but with "qapi: add #if
> > conditions to generated code".
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py                  | 10 +++++-----
> >  tests/qapi-schema/qapi-schema-test.json |  6 +++++-
> >  tests/qapi-schema/qapi-schema-test.out  |  9 ++++++++-
> >  3 files changed, 18 insertions(+), 7 deletions(-)
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index e1bd9a22ba..b3b64a60bf 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -843,7 +843,7 @@ def check_alternate(expr, info):
> >      for (key, value) in members.items():
> >          check_name(info, "Member of alternate '%s'" % name, key)
> >          source = "member '%s' of alternate '%s'" % (key, name)
> > -        check_known_keys(info, source, value, ['type'], [])
> > +        check_known_keys(info, source, value, ['type'], ['if'])
> >          typ = value['type']
> >
> >          # Ensure alternates have no type conflicts.
> > @@ -1774,8 +1774,8 @@ class QAPISchema(object):
> >                                                self._make_members(data, info),
> >                                                None))
> >
> > -    def _make_variant(self, case, typ):
> > -        return QAPISchemaObjectTypeVariant(case, typ)
> > +    def _make_variant(self, case, typ, ifcond):
> > +        return QAPISchemaObjectTypeVariant(case, typ, ifcond)
> >
> >      def _make_simple_variant(self, case, typ, ifcond, info):
> >          if isinstance(typ, list):
> > @@ -1798,7 +1798,7 @@ class QAPISchema(object):
> >                  name, info, doc, ifcond,
> >                  'base', self._make_members(base, info))
> >          if tag_name:
> > -            variants = [self._make_variant(key, value['type'])
> > +            variants = [self._make_variant(key, value['type'], value.get('if'))
> >                          for (key, value) in data.items()]
> >              members = []
> >          else:
> > @@ -1819,7 +1819,7 @@ class QAPISchema(object):
> >          name = expr['alternate']
> >          data = expr['data']
> >          ifcond = expr.get('if')
> > -        variants = [self._make_variant(key, value['type'])
> > +        variants = [self._make_variant(key, value['type'], value.get('if'))
> >                      for (key, value) in data.items()]
> >          tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
> >          self._def_entity(
> > diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> > index 6d3c6c0b53..df3edf9d89 100644
> > --- a/tests/qapi-schema/qapi-schema-test.json
> > +++ b/tests/qapi-schema/qapi-schema-test.json
> > @@ -216,9 +216,13 @@
> >  { 'command': 'TestIfUnionCmd', 'data': { 'union_cmd_arg': 'TestIfUnion' },
> >    'if': 'defined(TEST_IF_UNION)' }
> >
> > -{ 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
> > +{ 'alternate': 'TestIfAlternate', 'data':
> > +  { 'foo': 'int', 'alt_bar': { 'type': 'TestStruct', 'if': 'defined(TEST_IF_ALT_BAR)'} },
>
> Let's break the long line betwen the members.

ok

>
> Why rename member 'bar' to 'alt_bar'?

no reason I remember, dropped

>
> >    'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
> >
> > +{ 'command': 'TestIfAlternateCmd', 'data': { 'alt_cmd_arg': 'TestIfAlternate' },
>
> Another long line.  And I'm feeling dense again: why does this change
> belong to this patch?

No reason I remember, let's split it

>
> > +  'if': 'defined(TEST_IF_ALT)' }
> > +
> >  { 'command': 'TestIfCmd', 'data':
> >    { 'foo': 'TestIfStruct',
> >      'bar': { 'type': 'TestIfEnum', 'if': 'defined(TEST_IF_CMD_BAR)' } },
> > diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> > index ac1069cf1f..cdbd5b87cc 100644
> > --- a/tests/qapi-schema/qapi-schema-test.out
> > +++ b/tests/qapi-schema/qapi-schema-test.out
> > @@ -297,8 +297,15 @@ command TestIfUnionCmd q_obj_TestIfUnionCmd-arg -> None
> >  alternate TestIfAlternate
> >      tag type
> >      case foo: int
> > -    case bar: TestStruct
> > +    case alt_bar: TestStruct
> > +        if ['defined(TEST_IF_ALT_BAR)']
> >      if ['defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)']
> > +object q_obj_TestIfAlternateCmd-arg
> > +    member alt_cmd_arg: TestIfAlternate optional=False
> > +    if ['defined(TEST_IF_ALT)']
> > +command TestIfAlternateCmd q_obj_TestIfAlternateCmd-arg -> None
> > +   gen=True success_response=True boxed=False oob=False preconfig=False
> > +    if ['defined(TEST_IF_ALT)']
> >  object q_obj_TestIfCmd-arg
> >      member foo: TestIfStruct optional=False
> >      member bar: TestIfEnum optional=False

Re: [Qemu-devel] [PATCH v6 19/27] qapi: add 'if' on union members

[Marc-André Lureau]

Hi
On Thu, Dec 6, 2018 at 8:37 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> In the subject, s/ on / to /.
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Add 'if' key to union members:
> >
> > { 'union': 'TestIfUnion', 'data':
> >     'mem': { 'type': 'str', 'if': 'COND'} }
> >
> > Generated code is not changed by this patch but with "qapi: add #if
> > conditions to generated code".
>
> My suggestion on PATCH 13's commit message applies.

ok

>
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py                  | 17 +++++++++--------
> >  tests/qapi-schema/qapi-schema-test.json |  7 ++++++-
> >  tests/qapi-schema/qapi-schema-test.out  | 10 ++++++++++
> >  tests/qapi-schema/test-qapi.py          |  1 +
> >  4 files changed, 26 insertions(+), 9 deletions(-)
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index 13fbb28493..e1bd9a22ba 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -813,7 +813,7 @@ def check_union(expr, info):
> >      for (key, value) in members.items():
> >          check_name(info, "Member of union '%s'" % name, key)
> >          source = "member '%s' of union '%s'" % (key, name)
> > -        check_known_keys(info, source, value, ['type'], [])
> > +        check_known_keys(info, source, value, ['type'], ['if'])
> >          typ = value['type']
> >
> >          # Each value must name a known type
> > @@ -1496,8 +1496,8 @@ class QAPISchemaObjectTypeVariants(object):
> >  class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> >      role = 'branch'
> >
> > -    def __init__(self, name, typ):
> > -        QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
> > +    def __init__(self, name, typ, ifcond=None):
> > +        QAPISchemaObjectTypeMember.__init__(self, name, typ, False, ifcond)
> >
> >
> >  class QAPISchemaAlternateType(QAPISchemaType):
> > @@ -1777,14 +1777,14 @@ class QAPISchema(object):
> >      def _make_variant(self, case, typ):
> >          return QAPISchemaObjectTypeVariant(case, typ)
> >
> > -    def _make_simple_variant(self, case, typ, info):
> > +    def _make_simple_variant(self, case, typ, ifcond, info):
> >          if isinstance(typ, list):
> >              assert len(typ) == 1
> >              typ = self._make_array_type(typ[0], info)
> >          typ = self._make_implicit_object_type(
> >              typ, info, None, self.lookup_type(typ),
> >              'wrapper', [self._make_member('data', typ, None, info)])
> > -        return QAPISchemaObjectTypeVariant(case, typ)
> > +        return QAPISchemaObjectTypeVariant(case, typ, ifcond)
> >
> >      def _def_union_type(self, expr, info, doc):
> >          name = expr['union']
> > @@ -1802,10 +1802,11 @@ class QAPISchema(object):
> >                          for (key, value) in data.items()]
> >              members = []
> >          else:
> > -            variants = [self._make_simple_variant(key, value['type'], info)
> > +            variants = [self._make_simple_variant(key, value['type'],
> > +                                                  value.get('if'), info)
> >                          for (key, value) in data.items()]
> > -            typ = self._make_implicit_enum_type(name, info, ifcond,
> > -                                                [v.name for v in variants])
> > +            enum = [{'name': v.name, 'if': v.ifcond} for v in variants]
> > +            typ = self._make_implicit_enum_type(name, info, ifcond, enum)
> >              tag_member = QAPISchemaObjectTypeMember('type', typ, False)
> >              members = [tag_member]
> >          self._def_entity(
> > diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> > index 3bf440aab4..6d3c6c0b53 100644
> > --- a/tests/qapi-schema/qapi-schema-test.json
> > +++ b/tests/qapi-schema/qapi-schema-test.json
> > @@ -208,9 +208,14 @@
> >    [ 'foo', { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ],
> >    'if': 'defined(TEST_IF_ENUM)' }
> >
> > -{ 'union': 'TestIfUnion', 'data': { 'foo': 'TestStruct' },
> > +{ 'union': 'TestIfUnion', 'data':
> > +  { 'foo': 'TestStruct',
> > +    'union_bar': { 'type': 'str', 'if': 'defined(TEST_IF_UNION_BAR)'} },
>
> "Union Bar" sounds like a cocktail lounge in northeast US :)
>
> Back to serious: this is a simple union.  I'd prefer to test flat
> unions.  Changing TestIfUnion accordingly could be done either before
> this patch, or on top of this series, the latter not necessarily by
> you.  Your choice.

I prefer to defer

>
> >    'if': 'defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)' }
> >
> > +{ 'command': 'TestIfUnionCmd', 'data': { 'union_cmd_arg': 'TestIfUnion' },
> > +  'if': 'defined(TEST_IF_UNION)' }
> > +
>
> I'm feeling dense... why does this change belong to this patch?

Hmm, no strong reason, I'll make it a separate commit.
>
> >  { 'alternate': 'TestIfAlternate', 'data': { 'foo': 'int', 'bar': 'TestStruct' },
> >    'if': 'defined(TEST_IF_ALT) && defined(TEST_IF_STRUCT)' }
> >
> > diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> > index 71b84878c7..ac1069cf1f 100644
> > --- a/tests/qapi-schema/qapi-schema-test.out
> > +++ b/tests/qapi-schema/qapi-schema-test.out
> > @@ -278,12 +278,22 @@ object q_obj_TestStruct-wrapper
> >      member data: TestStruct optional=False
> >  enum TestIfUnionKind
> >      member foo
> > +    member union_bar
> > +        if ['defined(TEST_IF_UNION_BAR)']
> >      if ['defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)']
> >  object TestIfUnion
> >      member type: TestIfUnionKind optional=False
> >      tag type
> >      case foo: q_obj_TestStruct-wrapper
> > +    case union_bar: q_obj_str-wrapper
> > +        if ['defined(TEST_IF_UNION_BAR)']
> >      if ['defined(TEST_IF_UNION) && defined(TEST_IF_STRUCT)']
> > +object q_obj_TestIfUnionCmd-arg
> > +    member union_cmd_arg: TestIfUnion optional=False
> > +    if ['defined(TEST_IF_UNION)']
> > +command TestIfUnionCmd q_obj_TestIfUnionCmd-arg -> None
> > +   gen=True success_response=True boxed=False oob=False preconfig=False
> > +    if ['defined(TEST_IF_UNION)']
> >  alternate TestIfAlternate
> >      tag type
> >      case foo: int
> > diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
> > index 69e40d87d2..d977e936c7 100644
> > --- a/tests/qapi-schema/test-qapi.py
> > +++ b/tests/qapi-schema/test-qapi.py
> > @@ -68,6 +68,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
> >              print('    tag %s' % variants.tag_member.name)
> >              for v in variants.variants:
> >                  print('    case %s: %s' % (v.name, v.type.name))
> > +                QAPISchemaTestVisitor._print_if(v.ifcond, 8)
> >
> >      @staticmethod
> >      def _print_if(ifcond, indent=4):

Re: [Qemu-devel] [PATCH v6 16/27] qapi: add a dictionary form with 'type' key for members

[Marc-André Lureau]

Hi

On Thu, Dec 6, 2018 at 7:56 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Wherever a struct/union/alternate/command/event member with NAME: TYPE
> > form is accepted, desugar it to a NAME: { 'type': TYPE } form.
> >
> > This will allow to add new member details, such as 'if' in the
> > following patch to introduce conditionals, or 'default' for default
> > values etc.
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py                        | 56 +++++++++++++------
> >  tests/Makefile.include                        |  3 +
> >  tests/qapi-schema/alternate-invalid-dict.err  |  1 +
> >  tests/qapi-schema/alternate-invalid-dict.exit |  1 +
> >  tests/qapi-schema/alternate-invalid-dict.json |  4 ++
> >  tests/qapi-schema/alternate-invalid-dict.out  |  0
> >  tests/qapi-schema/event-nest-struct.err       |  2 +-
> >  tests/qapi-schema/flat-union-inline.err       |  2 +-
> >  tests/qapi-schema/nested-struct-data.err      |  2 +-
> >  tests/qapi-schema/qapi-schema-test.json       | 10 ++--
> >  .../struct-member-invalid-dict.err            |  1 +
> >  .../struct-member-invalid-dict.exit           |  1 +
> >  .../struct-member-invalid-dict.json           |  3 +
> >  .../struct-member-invalid-dict.out            |  0
> >  .../qapi-schema/union-branch-invalid-dict.err |  1 +
> >  .../union-branch-invalid-dict.exit            |  1 +
> >  .../union-branch-invalid-dict.json            |  4 ++
> >  .../qapi-schema/union-branch-invalid-dict.out |  0
> >  18 files changed, 66 insertions(+), 26 deletions(-)
> >  create mode 100644 tests/qapi-schema/alternate-invalid-dict.err
> >  create mode 100644 tests/qapi-schema/alternate-invalid-dict.exit
> >  create mode 100644 tests/qapi-schema/alternate-invalid-dict.json
> >  create mode 100644 tests/qapi-schema/alternate-invalid-dict.out
> >  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.err
> >  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.exit
> >  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.json
> >  create mode 100644 tests/qapi-schema/struct-member-invalid-dict.out
> >  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.err
> >  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.exit
> >  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.json
> >  create mode 100644 tests/qapi-schema/union-branch-invalid-dict.out
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index d83fa1900e..fc68bb472e 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -588,11 +588,11 @@ def discriminator_find_enum_define(expr):
> >      if not base_members:
> >          return None
> >
> > -    discriminator_type = base_members.get(discriminator)
> > -    if not discriminator_type:
> > +    discriminator_member = base_members.get(discriminator)
> > +    if not discriminator_member:
> >          return None
> >
> > -    return enum_types.get(discriminator_type)
> > +    return enum_types.get(discriminator_member['type'])
> >
> >
>
> The name discriminator_member is confusing.  It suggests its value is
> the (desugared) member definition, i.e. something like
>
>      {'enum1': {'type': 'EnumOne'}}
>
> it's actually only the definition's value part, i.e. something like
>
>      {'type': 'EnumOne'}
>
> Naming is hard... discriminator_value?  discriminator_def_rhs?
>
> Same in check_union() below.
>

discriminator_value, ok

> >  # Names must be letters, numbers, -, and _.  They must start with letter,
> > @@ -660,6 +660,15 @@ def check_if(expr, info):
> >          check_if_str(ifcond, info)
> >
> >
> > +def normalize_members(expr, field):
> > +    members = expr.get(field)
> > +    if isinstance(members, OrderedDict):
> > +        for key, arg in members.items():
> > +            if isinstance(arg, dict):
> > +                continue
> > +            members[key] = {'type': arg}
> > +
> > +
>
> PATCH 12's normalize_enum() conflates normalization and checking.  This
> one doesn't.  Better.

I fixed patch 12

>
> >  def check_type(info, source, value, allow_array=False,
> >                 allow_implicit=False, allow_optional=False,
> >                 allow_metas=[]):
> > @@ -704,8 +713,10 @@ def check_type(info, source, value, allow_array=False,
> >                                 % (source, key))
> >          # Todo: allow dictionaries to represent default values of
> >          # an optional argument.
> > -        check_type(info, "Member '%s' of %s" % (key, source), arg,
> > -                   allow_array=True,
> > +        check_known_keys(info, "member '%s' of %s" % (key, source),
> > +                         arg, ['type'], [])
> > +        check_type(info, "Member '%s' of %s" % (key, source),
> > +                   arg['type'], allow_array=True,
> >                     allow_metas=['built-in', 'union', 'alternate', 'struct',
> >                                  'enum'])
> >
> > @@ -776,13 +787,13 @@ def check_union(expr, info):
> >          # member of the base struct.
> >          check_name(info, "Discriminator of flat union '%s'" % name,
> >                     discriminator)
> > -        discriminator_type = base_members.get(discriminator)
> > -        if not discriminator_type:
> > +        discriminator_member = base_members.get(discriminator)
> > +        if not discriminator_member:
> >              raise QAPISemError(info,
> >                                 "Discriminator '%s' is not a member of base "
> >                                 "struct '%s'"
> >                                 % (discriminator, base))
> > -        enum_define = enum_types.get(discriminator_type)
> > +        enum_define = enum_types.get(discriminator_member['type'])
> >          allow_metas = ['struct']
> >          # Do not allow string discriminator
> >          if not enum_define:
> > @@ -795,10 +806,13 @@ def check_union(expr, info):
> >          raise QAPISemError(info, "Union '%s' cannot have empty 'data'" % name)
> >      for (key, value) in members.items():
> >          check_name(info, "Member of union '%s'" % name, key)
> > +        source = "member '%s' of union '%s'" % (key, name)
> > +        check_known_keys(info, source, value, ['type'], [])
>
> Elsewhere, you do it like
>
>            check_known_keys(info, "member '%s' of union '%s'" % (key, name),
>                             value, ['type'], [])
>
> Let's stick to that.

ok

>
> > +        typ = value['type']
> >
> >          # Each value must name a known type
> >          check_type(info, "Member '%s' of union '%s'" % (key, name),
> > -                   value, allow_array=not base, allow_metas=allow_metas)
> > +                   typ, allow_array=not base, allow_metas=allow_metas)
> >
> >          # If the discriminator names an enum type, then all members
> >          # of 'data' must also be members of the enum type.
> > @@ -822,18 +836,20 @@ def check_alternate(expr, info):
> >                             "in 'data'" % name)
> >      for (key, value) in members.items():
> >          check_name(info, "Member of alternate '%s'" % name, key)
> > +        source = "member '%s' of alternate '%s'" % (key, name)
> > +        check_known_keys(info, source, value, ['type'], [])
>
> Likewise.
>
> > +        typ = value['type']
> >
> >          # Ensure alternates have no type conflicts.
> > -        check_type(info, "Member '%s' of alternate '%s'" % (key, name),
> > -                   value,
> > +        check_type(info, "Member '%s' of alternate '%s'" % (key, name), typ,
> >                     allow_metas=['built-in', 'union', 'struct', 'enum'])
> > -        qtype = find_alternate_member_qtype(value)
> > +        qtype = find_alternate_member_qtype(typ)
> >          if not qtype:
> >              raise QAPISemError(info, "Alternate '%s' member '%s' cannot use "
> > -                               "type '%s'" % (name, key, value))
> > +                               "type '%s'" % (name, key, typ))
> >          conflicting = set([qtype])
> >          if qtype == 'QTYPE_QSTRING':
> > -            enum_expr = enum_types.get(value)
> > +            enum_expr = enum_types.get(typ)
> >              if enum_expr:
> >                  for v in enum_get_names(enum_expr):
> >                      if v in ['on', 'off']:
> > @@ -947,6 +963,10 @@ def check_exprs(exprs):
> >          info = expr_elem['info']
> >          if 'enum' in expr:
> >              normalize_enum(expr, info)
> > +        elif 'union' in expr:
> > +            normalize_members(expr, 'base')
> > +        if {'union', 'alternate', 'struct', 'command', 'event'} & set(expr):
> > +            normalize_members(expr, 'data')
> >
> >      # Learn the types and check for valid expression keys
> >      for expr_elem in exprs:
>
> PATCH 12 had an issue in this loop, and the obvious fix was to fold it
> into the next loop.  I think this can be done here too.

moved to end-of-check

>
> > @@ -1735,7 +1755,7 @@ class QAPISchema(object):
> >          return QAPISchemaObjectTypeMember(name, typ, optional)
> >
> >      def _make_members(self, data, info):
> > -        return [self._make_member(key, value, info)
> > +        return [self._make_member(key, value['type'], info)
> >                  for (key, value) in data.items()]
> >
> >      def _def_struct_type(self, expr, info, doc):
> > @@ -1771,11 +1791,11 @@ class QAPISchema(object):
> >                  name, info, doc, ifcond,
> >                  'base', self._make_members(base, info))
> >          if tag_name:
> > -            variants = [self._make_variant(key, value)
> > +            variants = [self._make_variant(key, value['type'])
> >                          for (key, value) in data.items()]
> >              members = []
> >          else:
> > -            variants = [self._make_simple_variant(key, value, info)
> > +            variants = [self._make_simple_variant(key, value['type'], info)
> >                          for (key, value) in data.items()]
> >              typ = self._make_implicit_enum_type(name, info, ifcond,
> >                                                  [v.name for v in variants])
> > @@ -1791,7 +1811,7 @@ class QAPISchema(object):
> >          name = expr['alternate']
> >          data = expr['data']
> >          ifcond = expr.get('if')
> > -        variants = [self._make_variant(key, value)
> > +        variants = [self._make_variant(key, value['type'])
> >                      for (key, value) in data.items()]
> >          tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
> >          self._def_entity(
> > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > index 3ba9097892..43e100a6cd 100644
> > --- a/tests/Makefile.include
> > +++ b/tests/Makefile.include
> > @@ -421,6 +421,7 @@ qapi-schema += alternate-conflict-string.json
> >  qapi-schema += alternate-conflict-bool-string.json
> >  qapi-schema += alternate-conflict-num-string.json
> >  qapi-schema += alternate-empty.json
> > +qapi-schema += alternate-invalid-dict.json
> >  qapi-schema += alternate-nested.json
> >  qapi-schema += alternate-unknown.json
> >  qapi-schema += args-alternate.json
> > @@ -563,6 +564,7 @@ qapi-schema += returns-whitelist.json
> >  qapi-schema += struct-base-clash-deep.json
> >  qapi-schema += struct-base-clash.json
> >  qapi-schema += struct-data-invalid.json
> > +qapi-schema += struct-member-invalid-dict.json
> >  qapi-schema += struct-member-invalid.json
> >  qapi-schema += trailing-comma-list.json
> >  qapi-schema += trailing-comma-object.json
> > @@ -574,6 +576,7 @@ qapi-schema += unicode-str.json
> >  qapi-schema += union-base-empty.json
> >  qapi-schema += union-base-no-discriminator.json
> >  qapi-schema += union-branch-case.json
> > +qapi-schema += union-branch-invalid-dict.json
> >  qapi-schema += union-clash-branches.json
> >  qapi-schema += union-empty.json
> >  qapi-schema += union-invalid-base.json
> > diff --git a/tests/qapi-schema/alternate-invalid-dict.err b/tests/qapi-schema/alternate-invalid-dict.err
> > new file mode 100644
> > index 0000000000..631d46628e
> > --- /dev/null
> > +++ b/tests/qapi-schema/alternate-invalid-dict.err
> > @@ -0,0 +1 @@
> > +tests/qapi-schema/alternate-invalid-dict.json:2: Key 'type' is missing from member 'two' of alternate 'Alt'
> > diff --git a/tests/qapi-schema/alternate-invalid-dict.exit b/tests/qapi-schema/alternate-invalid-dict.exit
> > new file mode 100644
> > index 0000000000..d00491fd7e
> > --- /dev/null
> > +++ b/tests/qapi-schema/alternate-invalid-dict.exit
> > @@ -0,0 +1 @@
> > +1
> > diff --git a/tests/qapi-schema/alternate-invalid-dict.json b/tests/qapi-schema/alternate-invalid-dict.json
> > new file mode 100644
> > index 0000000000..45f2c8ebef
> > --- /dev/null
> > +++ b/tests/qapi-schema/alternate-invalid-dict.json
> > @@ -0,0 +1,4 @@
> > +# invalid field dictionnary, missing type
>
> dictionary

argh..

>
> Suggest a grep over the whole series.
>
> Even better, reuse struct-member-invalid-dict.json's comment here.

ok

>
> > +{ 'alternate': 'Alt',
> > +  'data': { 'one': 'str',
> > +            'two': { 'if': 'foo' } } }
> > diff --git a/tests/qapi-schema/alternate-invalid-dict.out b/tests/qapi-schema/alternate-invalid-dict.out
> > new file mode 100644
> > index 0000000000..e69de29bb2
> > diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err
> > index 5a42701b8f..ea1be65d89 100644
> > --- a/tests/qapi-schema/event-nest-struct.err
> > +++ b/tests/qapi-schema/event-nest-struct.err
> > @@ -1 +1 @@
> > -tests/qapi-schema/event-nest-struct.json:1: Member 'a' of 'data' for event 'EVENT_A' should be a type name
> > +tests/qapi-schema/event-nest-struct.json:1: Key 'type' is missing from member 'a' of 'data' for event 'EVENT_A'
>
> We lose the test of the "should be a type name" error.  Let's copy this
> test to event-member-invalid-dict.json to cover "Key 'type' is missing",
> and update the original to keep covering "should be a type name".
>

ok

>
> > diff --git a/tests/qapi-schema/flat-union-inline.err b/tests/qapi-schema/flat-union-inline.err
> > index 2333358d28..3d29433bfb 100644
> > --- a/tests/qapi-schema/flat-union-inline.err
> > +++ b/tests/qapi-schema/flat-union-inline.err
> > @@ -1 +1 @@
> > -tests/qapi-schema/flat-union-inline.json:7: Member 'value1' of union 'TestUnion' should be a type name
> > +tests/qapi-schema/flat-union-inline.json:7: Key 'type' is missing from member 'value1' of union 'TestUnion'
>
> Likewise.
>
> > diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
> > index da767bade2..48355adcbe 100644
> > --- a/tests/qapi-schema/nested-struct-data.err
> > +++ b/tests/qapi-schema/nested-struct-data.err
> > @@ -1 +1 @@
> > -tests/qapi-schema/nested-struct-data.json:2: Member 'a' of 'data' for command 'foo' should be a type name
> > +tests/qapi-schema/nested-struct-data.json:2: Key 'type' is missing from member 'a' of 'data' for command 'foo'
>
> Likewise.
>
> > diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> > index 35ca94d991..30f635d484 100644
> > --- a/tests/qapi-schema/qapi-schema-test.json
> > +++ b/tests/qapi-schema/qapi-schema-test.json
>
> Positive tests for ...
>
> > @@ -11,7 +11,7 @@
> >          'guest-sync' ] } }
> >
> >  { 'struct': 'TestStruct',
> > -  'data': { 'integer': 'int', 'boolean': 'bool', 'string': 'str' } }
> > +  'data': { 'integer': {'type': 'int'}, 'boolean': 'bool', 'string': 'str' } }
>
> ... struct member, ...
>
> >
> >  # for testing enums
> >  { 'struct': 'NestedEnumsOne',
> > @@ -77,7 +77,7 @@
> >  { 'union': 'UserDefFlatUnion',
> >    'base': 'UserDefUnionBase',   # intentional forward reference
> >    'discriminator': 'enum1',
> > -  'data': { 'value1' : 'UserDefA',
> > +  'data': { 'value1' : {'type': 'UserDefA'},
> >              'value2' : 'UserDefB',
> >              'value3' : 'UserDefB'
> >              # 'value4' defaults to empty
>
> ... flat union member, ...
>
> > @@ -98,7 +98,7 @@
> >  { 'struct': 'WrapAlternate',
> >    'data': { 'alt': 'UserDefAlternate' } }
> >  { 'alternate': 'UserDefAlternate',
> > -  'data': { 'udfu': 'UserDefFlatUnion', 'e': 'EnumOne', 'i': 'int',
> > +  'data': { 'udfu': {'type': 'UserDefFlatUnion'}, 'e': 'EnumOne', 'i': 'int',
> >              'n': 'null' } }
> >
>
> ... alternate member, ...
>
> >  { 'struct': 'UserDefC',
> > @@ -134,7 +134,7 @@
> >  { 'command': 'user_def_cmd', 'data': {} }
> >  { 'command': 'user_def_cmd1', 'data': {'ud1a': 'UserDefOne'} }
> >  { 'command': 'user_def_cmd2',
> > -  'data': {'ud1a': 'UserDefOne', '*ud1b': 'UserDefOne'},
> > +  'data': {'ud1a': {'type': 'UserDefOne'}, '*ud1b': 'UserDefOne'},
> >    'returns': 'UserDefTwo' }
> >
>
> ... command argument, ...
>
> >  # Returning a non-dictionary requires a name from the whitelist
> > @@ -164,7 +164,7 @@
> >
> >  # testing event
> >  { 'struct': 'EventStructOne',
> > -  'data': { 'struct1': 'UserDefOne', 'string': 'str', '*enum2': 'EnumOne' } }
> > +  'data': { 'struct1': {'type': 'UserDefOne'}, 'string': 'str', '*enum2': 'EnumOne' } }
> >
> >  { 'event': 'EVENT_A' }
> >  { 'event': 'EVENT_B',
>
> ... and event argument.
>
> Missing: simple union.  Okay because the desugaring is oblivious of the
> difference between flat and simple unions.
>
> > diff --git a/tests/qapi-schema/struct-member-invalid-dict.err b/tests/qapi-schema/struct-member-invalid-dict.err
> > new file mode 100644
> > index 0000000000..6a765bc668
> > --- /dev/null
> > +++ b/tests/qapi-schema/struct-member-invalid-dict.err
> > @@ -0,0 +1 @@
> > +tests/qapi-schema/struct-member-invalid-dict.json:2: Key 'type' is missing from member '*a' of 'data' for struct 'foo'
> > diff --git a/tests/qapi-schema/struct-member-invalid-dict.exit b/tests/qapi-schema/struct-member-invalid-dict.exit
> > new file mode 100644
> > index 0000000000..d00491fd7e
> > --- /dev/null
> > +++ b/tests/qapi-schema/struct-member-invalid-dict.exit
> > @@ -0,0 +1 @@
> > +1
> > diff --git a/tests/qapi-schema/struct-member-invalid-dict.json b/tests/qapi-schema/struct-member-invalid-dict.json
> > new file mode 100644
> > index 0000000000..ebd9733b49
> > --- /dev/null
> > +++ b/tests/qapi-schema/struct-member-invalid-dict.json
> > @@ -0,0 +1,3 @@
> > +# exploded member form must have a 'type'
>
> Suggest
>
>    # Long form of member must have a value member 'type'
>

ok

> or simply
>
>    # Missing type
>
> > +{ 'struct': 'foo',
> > +  'data': { '*a': { 'case': 'foo' } } }
> > diff --git a/tests/qapi-schema/struct-member-invalid-dict.out b/tests/qapi-schema/struct-member-invalid-dict.out
> > new file mode 100644
> > index 0000000000..e69de29bb2
> > diff --git a/tests/qapi-schema/union-branch-invalid-dict.err b/tests/qapi-schema/union-branch-invalid-dict.err
> > new file mode 100644
> > index 0000000000..89f9b36791
> > --- /dev/null
> > +++ b/tests/qapi-schema/union-branch-invalid-dict.err
> > @@ -0,0 +1 @@
> > +tests/qapi-schema/union-branch-invalid-dict.json:2: Key 'type' is missing from member 'integer' of union 'UnionInvalidBranch'
> > diff --git a/tests/qapi-schema/union-branch-invalid-dict.exit b/tests/qapi-schema/union-branch-invalid-dict.exit
> > new file mode 100644
> > index 0000000000..d00491fd7e
> > --- /dev/null
> > +++ b/tests/qapi-schema/union-branch-invalid-dict.exit
> > @@ -0,0 +1 @@
> > +1
> > diff --git a/tests/qapi-schema/union-branch-invalid-dict.json b/tests/qapi-schema/union-branch-invalid-dict.json
> > new file mode 100644
> > index 0000000000..19c5d9cacd
> > --- /dev/null
> > +++ b/tests/qapi-schema/union-branch-invalid-dict.json
> > @@ -0,0 +1,4 @@
> > +# exploded member form must have a 'type'
>
> Likewise.
>
> > +{ 'union': 'UnionInvalidBranch',
> > +  'data': { 'integer': { 'if': 'foo'},
> > +            's8': 'int8' } }
>
> I avoid simple unions like the plague.  That said, having one here is
> tolerable; we'll just have to replace it when we abolish simple unions.
>
> > diff --git a/tests/qapi-schema/union-branch-invalid-dict.out b/tests/qapi-schema/union-branch-invalid-dict.out
> > new file mode 100644
> > index 0000000000..e69de29bb2

Re: [Qemu-devel] [PATCH v6 15/27] qapi: rename allow_dict to allow_implicit

[Marc-André Lureau]

On Wed, Dec 5, 2018 at 10:41 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > This makes it a bit clearer what is the intent of the dictionnary for
>
> dictionary

sigh, this must be a very common misspell (dictionnaire in french)

>
> > the check_type() function, since there was some confusion on a
> > previous iteration of this series.
>
> I don't remember... got a pointer to that confusion handy?

https://lists.nongnu.org/archive/html/qemu-devel/2017-12/msg01584.html

>
> > Suggested-by: Markus Armbruster <armbru@redhat.com>
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Re: [Qemu-devel] [PATCH v6 12/27] qapi: add a dictionnary form with 'name' key for enum members

[Marc-André Lureau]

Hi
On Wed, Dec 5, 2018 at 10:02 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> In subject, s/dictionnary/dictionary/
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Desugar the enum NAME form to { 'name': NAME }. This will allow to add
> > new enum members, such as 'if' in the following patch.
> >
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py                        | 47 ++++++++++++++++---
> >  tests/Makefile.include                        |  3 +-
> >  tests/qapi-schema/enum-bad-member.err         |  1 +
> >  ...-dict-member.exit => enum-bad-member.exit} |  0
> >  tests/qapi-schema/enum-bad-member.json        |  2 +
> >  ...um-dict-member.out => enum-bad-member.out} |  0
> >  .../qapi-schema/enum-dict-member-unknown.err  |  2 +
> >  .../qapi-schema/enum-dict-member-unknown.exit |  1 +
> >  .../qapi-schema/enum-dict-member-unknown.json |  2 +
> >  .../qapi-schema/enum-dict-member-unknown.out  |  0
> >  tests/qapi-schema/enum-dict-member.err        |  1 -
> >  tests/qapi-schema/enum-dict-member.json       |  2 -
> >  tests/qapi-schema/enum-missing-data.err       |  2 +-
> >  13 files changed, 51 insertions(+), 12 deletions(-)
> >  create mode 100644 tests/qapi-schema/enum-bad-member.err
> >  rename tests/qapi-schema/{enum-dict-member.exit => enum-bad-member.exit} (100%)
> >  create mode 100644 tests/qapi-schema/enum-bad-member.json
> >  rename tests/qapi-schema/{enum-dict-member.out => enum-bad-member.out} (100%)
> >  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.err
> >  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.exit
> >  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.json
> >  create mode 100644 tests/qapi-schema/enum-dict-member-unknown.out
> >  delete mode 100644 tests/qapi-schema/enum-dict-member.err
> >  delete mode 100644 tests/qapi-schema/enum-dict-member.json
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index cf8dab2866..e9fb736d46 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -740,6 +740,10 @@ def check_event(expr, info):
> >                 allow_metas=meta)
> >
> >
> > +def enum_get_names(expr):
> > +    return [e['name'] for e in expr['data']]
> > +
> > +
> >  def check_union(expr, info):
> >      name = expr['union']
> >      base = expr.get('base')
> > @@ -799,7 +803,7 @@ def check_union(expr, info):
> >          # If the discriminator names an enum type, then all members
> >          # of 'data' must also be members of the enum type.
> >          if enum_define:
> > -            if key not in enum_define['data']:
> > +            if key not in enum_get_names(enum_define):
> >                  raise QAPISemError(info,
> >                                     "Discriminator value '%s' is not found in "
> >                                     "enum '%s'"
> > @@ -831,10 +835,10 @@ def check_alternate(expr, info):
> >          if qtype == 'QTYPE_QSTRING':
> >              enum_expr = enum_types.get(value)
> >              if enum_expr:
> > -                for v in enum_expr['data']:
> > +                for v in enum_get_names(enum_expr):
> >                      if v in ['on', 'off']:
> >                          conflicting.add('QTYPE_QBOOL')
> > -                    if re.match(r'[-+0-9.]', v): # lazy, could be tightened
> > +                    if re.match(r'[-+0-9.]', v):  # lazy, could be tightened
>
> Unrelated change.
>

ok

> >                          conflicting.add('QTYPE_QNUM')
> >              else:
> >                  conflicting.add('QTYPE_QNUM')
>
> The helper enum_get_names() is used just twice.
>
> The second instance
>
>                 for v in enum_get_names(enum_expr):
>
> could just as well iterate over enum_expr
>
>                 for d in enum_expr['data']:
>                     v = d['name']
>
> The first instance could simply be inlined then.

well, with unnormalized form (your request below), this becomes even
less readable. I'll keep it, even if it's a one-liner, I much prefer
an explicit function name and common code.

>
> > @@ -847,19 +851,34 @@ def check_alternate(expr, info):
> >              types_seen[qt] = key
> >
> >
> > -def check_enum(expr, info):
> > +def normalize_enum(expr, info):
> >      name = expr['enum']
> >      members = expr.get('data')
> > -    prefix = expr.get('prefix')
> >
> >      if not isinstance(members, list):
> >          raise QAPISemError(info,
> >                             "Enum '%s' requires an array for 'data'" % name)
> > +
> > +    # translate short member form to dict form
> > +    for i, member in enumerate(members):
> > +        if not isinstance(member, dict):
> > +            member = {'name': member}
> > +            members[i] = member
>
> Simpler:
>
>        members = [m if isinstance(m, dict) else {'name': m} for m in members]
>

Yes, except that this doesn't change the dict in place. No pb, updated.

> > +
> > +
> > +def check_enum(expr, info):
> > +    name = expr['enum']
> > +    members = expr.get('data')
>
> Preexisting: this should be expr['data'], because we already checked
> required keys.
>

ok

> > +    prefix = expr.get('prefix')
> > +
> >      if prefix is not None and not isinstance(prefix, str):
> >          raise QAPISemError(info,
> >                             "Enum '%s' requires a string for 'prefix'" % name)
> > +
> >      for member in members:
> > -        check_name(info, "Member of enum '%s'" % name, member,
> > +        source = "dictionary member of enum '%s'" % name
> > +        check_known_keys(info, source, member, ['name'], [])
> > +        check_name(info, "Member of enum '%s'" % name, member['name'],
> >                     enum_member=True)
> >
> >
> > @@ -921,6 +940,13 @@ def check_exprs(exprs):
> >      for builtin in builtin_types.keys():
> >          all_names[builtin] = 'built-in'
> >
> > +    # Normalize exprs
> > +    for expr_elem in exprs:
> > +        expr = expr_elem['expr']
> > +        info = expr_elem['info']
> > +        if 'enum' in expr:
> > +            normalize_enum(expr, info)
> > +
>
> As demonstrated by tests/qapi-schema/enum-missing-data.json below, the
> error message for
>
>     {'enum': 'MyEnum'}
>
> changes from
>
>     Key 'data' is missing from enum 'MyEnum'
>
> to
>
>     Enum 'MyEnum' requires an array for 'data'
>
> because you normalize before checking keys.  I don't think that's a good
> idea.

The alternative is to check and normalize after, sure it's a bit more
user-friendly, ok.

>
> >      # Learn the types and check for valid expression keys
> >      for expr_elem in exprs:
> >          expr = expr_elem['expr']
>            info = expr_elem['info']
>            doc = expr_elem.get('doc')
>
>            if 'include' in expr:
>                continue
>
>            if not doc and doc_required:
>                raise QAPISemError(info,
>                                   "Expression missing documentation comment")
>
>            if 'enum' in expr:
>                meta = 'enum'
>                check_keys(expr_elem, 'enum', ['data'], ['if', 'prefix'])
>
> This seems a better place to call normalize_enum(expr, info).
>
>                enum_types[expr[meta]] = expr
>            elif 'union' in expr:
> > @@ -1640,7 +1666,14 @@ class QAPISchema(object):
> >                                              qtype_values, 'QTYPE'))
> >
> >      def _make_enum_members(self, values):
> > -        return [QAPISchemaMember(v) for v in values]
> > +        enum = []
> > +        for v in values:
> > +            if isinstance(v, dict):
> > +                name = v['name']
> > +            else:
> > +                name = v
> > +            enum.append(QAPISchemaMember(name))
> > +        return enum
>
> Despite normalization to the long form, we still deal with both the long
> and the short form here.  Three callers:
>
> * _def_predefineds(): passes the short form.
>
> * _make_implicit_enum_type(): passes the short form until PATCH 19
>   changes it to pass the long form.
>
> * _def_enum_type(): passes the long form.
>
> I believe always passing the long form would be simpler.

ok, I'll add a patch for def_predefineds()

>
> >
> >      def _make_implicit_enum_type(self, name, info, ifcond, values):
> >          # See also QAPISchemaObjectTypeMember._pretty_owner()
> > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > index a49282704e..8e1b122cf2 100644
> > --- a/tests/Makefile.include
> > +++ b/tests/Makefile.include
> > @@ -482,10 +482,11 @@ qapi-schema += double-data.json
> >  qapi-schema += double-type.json
> >  qapi-schema += duplicate-key.json
> >  qapi-schema += empty.json
> > +qapi-schema += enum-bad-member.json
> >  qapi-schema += enum-bad-name.json
> >  qapi-schema += enum-bad-prefix.json
> >  qapi-schema += enum-clash-member.json
> > -qapi-schema += enum-dict-member.json
> > +qapi-schema += enum-dict-member-unknown.json
> >  qapi-schema += enum-int-member.json
> >  qapi-schema += enum-member-case.json
> >  qapi-schema += enum-missing-data.json
> > diff --git a/tests/qapi-schema/enum-bad-member.err b/tests/qapi-schema/enum-bad-member.err
> > new file mode 100644
> > index 0000000000..211db9e6fc
> > --- /dev/null
> > +++ b/tests/qapi-schema/enum-bad-member.err
> > @@ -0,0 +1 @@
> > +tests/qapi-schema/enum-bad-member.json:2: Member of enum 'MyEnum' requires a string name
> > diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-bad-member.exit
> > similarity index 100%
> > rename from tests/qapi-schema/enum-dict-member.exit
> > rename to tests/qapi-schema/enum-bad-member.exit
> > diff --git a/tests/qapi-schema/enum-bad-member.json b/tests/qapi-schema/enum-bad-member.json
> > new file mode 100644
> > index 0000000000..98da6828b4
> > --- /dev/null
> > +++ b/tests/qapi-schema/enum-bad-member.json
> > @@ -0,0 +1,2 @@
> > +# we reject any enum member that is not a string
> > +{ 'enum': 'MyEnum', 'data': [ [ ] ] }
> > diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-bad-member.out
> > similarity index 100%
> > rename from tests/qapi-schema/enum-dict-member.out
> > rename to tests/qapi-schema/enum-bad-member.out
>
> This replaces enum-dict-member.json.  Okay.  Took me a while to see this
> %-/
>
> > diff --git a/tests/qapi-schema/enum-dict-member-unknown.err b/tests/qapi-schema/enum-dict-member-unknown.err
> > new file mode 100644
> > index 0000000000..76bd0471db
> > --- /dev/null
> > +++ b/tests/qapi-schema/enum-dict-member-unknown.err
> > @@ -0,0 +1,2 @@
> > +tests/qapi-schema/enum-dict-member-unknown.json:2: Unknown key 'bad-key' in dictionary member of enum 'MyEnum'
> > +Valid keys are 'name'.
> > diff --git a/tests/qapi-schema/enum-dict-member-unknown.exit b/tests/qapi-schema/enum-dict-member-unknown.exit
> > new file mode 100644
> > index 0000000000..d00491fd7e
> > --- /dev/null
> > +++ b/tests/qapi-schema/enum-dict-member-unknown.exit
> > @@ -0,0 +1 @@
> > +1
> > diff --git a/tests/qapi-schema/enum-dict-member-unknown.json b/tests/qapi-schema/enum-dict-member-unknown.json
> > new file mode 100644
> > index 0000000000..6664c59201
> > --- /dev/null
> > +++ b/tests/qapi-schema/enum-dict-member-unknown.json
> > @@ -0,0 +1,2 @@
> > +# we reject any enum member that is not a string or a dict with 'name'
> > +{ 'enum': 'MyEnum', 'data': [ { 'name': 'foo', 'bad-key': 'str' } ] }
> > diff --git a/tests/qapi-schema/enum-dict-member-unknown.out b/tests/qapi-schema/enum-dict-member-unknown.out
> > new file mode 100644
> > index 0000000000..e69de29bb2
> > diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
> > deleted file mode 100644
> > index 8ca146ea59..0000000000
> > --- a/tests/qapi-schema/enum-dict-member.err
> > +++ /dev/null
> > @@ -1 +0,0 @@
> > -tests/qapi-schema/enum-dict-member.json:2: Member of enum 'MyEnum' requires a string name
> > diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
> > deleted file mode 100644
> > index 79672e0f09..0000000000
> > --- a/tests/qapi-schema/enum-dict-member.json
> > +++ /dev/null
> > @@ -1,2 +0,0 @@
> > -# we reject any enum member that is not a string
> > -{ 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
> > diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
> > index ba4873ae69..b8ccae071b 100644
> > --- a/tests/qapi-schema/enum-missing-data.err
> > +++ b/tests/qapi-schema/enum-missing-data.err
> > @@ -1 +1 @@
> > -tests/qapi-schema/enum-missing-data.json:2: Key 'data' is missing from enum 'MyEnum'
> > +tests/qapi-schema/enum-missing-data.json:2: Enum 'MyEnum' requires an array for 'data'

Re: [Qemu-devel] [PATCH v6 06/27] qapi: do not define enumeration value explicitly

[Marc-André Lureau]

Hi
On Wed, Dec 5, 2018 at 5:19 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > The C standard has the initial value at 0 and the subsequent values
> > incremented by 1. No need to set this explicitely.
> >
> > This will prevent from artificial "gaps" when compiling out some enum
> > values and having unnecessarily large MAX values & enums arrays, or
> > simplifying iterating over valid enum values.
>
> Yes, gaps can be annoying, e.g. in lookup tables where gaps get confused
> with sentinels.
>
> Avoiding gaps scares me.  You have to religiously compile the enum with
> the exact same configuration everywhere in the program, or else you end
> up with bugs that are hard to spot.  Therefore:
>
> > Whenever config-host.h is changed, all the enum/types are recompiled.
> >
> > (a subsequent patch will split the schema. Target-specific poisoined
> > conditionals will be added. They cannot be mixed with the common
> > schema: it is not possible to end up with enums of different values in
> > common and target builds)
>
> Should we mention config-target.h here?
>

My memory isn't clear. I'll drop that comment from here, we will
discuss the poisoned conditionals again with the patch splitting the
schema.

> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py | 7 ++-----
> >  1 file changed, 2 insertions(+), 5 deletions(-)
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index 02c5c6767a..6f9498566e 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -2045,14 +2045,11 @@ typedef enum %(c_name)s {
> >  ''',
> >                  c_name=c_name(name))
> >
> > -    i = 0
> >      for value in enum_values:
> >          ret += mcgen('''
> > -    %(c_enum)s = %(i)d,
> > +    %(c_enum)s,
> >  ''',
> > -                     c_enum=c_enum_const(name, value, prefix),
> > -                     i=i)
> > -        i += 1
> > +                     c_enum=c_enum_const(name, value, prefix))
> >
> >      ret += mcgen('''
> >  } %(c_name)s;
>
> I need to consider the whole series to decide whether avoiding gaps is a
> good idea.  If it is, then this patch is fine.

Re: [Qemu-devel] [PATCH v6 21/27] qapi: add #if conditions to generated code members

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> Hi
> On Thu, Dec 6, 2018 at 9:42 PM Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>>
>> > Wrap generated enum/struct members and code with #if/#endif, using the
>>
>> enum and struct members
>
> ok
>
>>
>> > .ifcond members added in the previous patches.
>> >
>> > Some types generate both enum and struct members for example, so a
>> > step-by-step is unnecessarily complicated to deal with (it would
>> > easily generate invalid intermediary code).
>>
>> Can you give an example of a schema definition that would lead to
>> complications?
>>
>
> Honestly, I don't remember well (it's been a while I wrote that code).

I know...

> It must be related to implicit enums, such as union kind... If there
> is no strong need to split this patch, I would rather not do that
> extra work.

I'm not looking for reasons to split this patch, I'm looking for
stronger reasons to keep it just like it is :)

Your hunch that complications would arise for simple unions plausible:
there the same conditional needs to be applied both to the C enum's
member and the C union member.

For the generated C code to compile, each union tag enum member
conditional must imply the corresponding variant conditional.

For flat unions, the two are separate.  The QAPI generator makes no
effort to check the enum member's if condition implies the union
variant's if condition; if you mess them up in the schema, you get to
deal with the C compilation errors.

For simple unions, the two are one.

If we separate the generator updates for enums and for union members,
and do enum members first, then unions with conditional tag members
can't compile.  Corrollary: simple unions with conditional variants
can't compile.

What if we do union members first?

Again, I'm not asking for patch splitting here, I'm just trying to
arrive at a clearer understanding to avoid making insufficiently
supported claims in the commit message.  The combined patch looks small
and clean enough to keep it combined.

[...]

Re: [Qemu-devel] [PATCH v6 15/27] qapi: rename allow_dict to allow_implicit

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@redhat.com> writes:

> On Wed, Dec 5, 2018 at 10:41 PM Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>>
>> > This makes it a bit clearer what is the intent of the dictionnary for
>>
>> dictionary
>
> sigh, this must be a very common misspell (dictionnaire in french)

Muscle memory...

>> > the check_type() function, since there was some confusion on a
>> > previous iteration of this series.
>>
>> I don't remember... got a pointer to that confusion handy?
>
> https://lists.nongnu.org/archive/html/qemu-devel/2017-12/msg01584.html

Thanks!

My review comment was

    > diff --git a/scripts/qapi.py b/scripts/qapi.py
    > index df2a304e8f..15711f96fa 100644
    > --- a/scripts/qapi.py
    > +++ b/scripts/qapi.py
    > @@ -696,7 +696,15 @@ def check_type(info, source, value, allow_array=False,
    >          return
    >  
    >      if not allow_dict:
    > -        raise QAPISemError(info, "%s should be a type name" % source)
    > +        if isinstance(value, dict) and 'type' in value:
    > +            check_type(info, source, value['type'], allow_array,
    > +                       allow_dict, allow_optional, allow_metas)
    > +            if 'if' in value:
    > +                check_if(value, info)
    > +            check_unknown_keys(info, value, {'type', 'if'})
    > +            return
    > +        else:
    > +            raise QAPISemError(info, "%s should be a type name" % source)

    @allow_dict becomes a misnomer: dictionaries are now always allowed, but
    they have different meaning (implicit type vs. named type with
    additional attributes).

    Rename to @allow_implicit?

As far as I can tell, it's not an issue in the current version of your
patches anymore:

    def check_type(info, source, value, allow_array=False,
                   allow_implicit=False, allow_optional=False,
                   allow_metas=[]):
        global all_names

        if value is None:
            return

        # Check if array type for value is okay
        if isinstance(value, list):
            if not allow_array:
                raise QAPISemError(info, "%s cannot be an array" % source)
            if len(value) != 1 or not isinstance(value[0], str):
                raise QAPISemError(info,
                                   "%s: array type must contain single type name" %
                                   source)
            value = value[0]

        # Check if type name for value is okay
        if isinstance(value, str):
            if value not in all_names:
                raise QAPISemError(info, "%s uses unknown type '%s'"
                                   % (source, value))
            if not all_names[value] in allow_metas:
                raise QAPISemError(info, "%s cannot use %s type '%s'" %
                                   (source, all_names[value], value))
            return

        if not allow_implicit:
            raise QAPISemError(info, "%s should be a type name" % source)

        if not isinstance(value, OrderedDict):
            raise QAPISemError(info,
                               "%s should be a dictionary or type name" % source)

        # value is a dictionary, check that each member is okay
        for (key, arg) in value.items():
            check_name(info, "Member of %s" % source, key,
                       allow_optional=allow_optional)
            if c_name(key, False) == 'u' or c_name(key, False).startswith('has_'):
                raise QAPISemError(info, "Member of %s uses reserved name '%s'"
                                   % (source, key))
            # Todo: allow dictionaries to represent default values of
            # an optional argument.
            if isinstance(arg, dict):
                check_known_keys(info, "member '%s' of %s" % (key, source),
                                 arg, ['type'], ['if'])
                typ = arg['type']
            else:
                typ = arg
            check_type(info, "Member '%s' of %s" % (key, source),
                       typ, allow_array=True,
                       allow_metas=['built-in', 'union', 'alternate', 'struct',
                                    'enum'])


>> > Suggested-by: Markus Armbruster <armbru@redhat.com>
>> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

Re: [Qemu-devel] [PATCH v6 21/27] qapi: add #if conditions to generated code members

[Markus Armbruster]

Markus Armbruster <armbru@redhat.com> writes:

> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
>> Hi
>> On Thu, Dec 6, 2018 at 9:42 PM Markus Armbruster <armbru@redhat.com> wrote:
>>>
>>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>>>
>>> > Wrap generated enum/struct members and code with #if/#endif, using the
>>>
>>> enum and struct members
>>
>> ok
>>
>>>
>>> > .ifcond members added in the previous patches.
>>> >
>>> > Some types generate both enum and struct members for example, so a
>>> > step-by-step is unnecessarily complicated to deal with (it would
>>> > easily generate invalid intermediary code).
>>>
>>> Can you give an example of a schema definition that would lead to
>>> complications?
>>>
>>
>> Honestly, I don't remember well (it's been a while I wrote that code).
>
> I know...
>
>> It must be related to implicit enums, such as union kind... If there
>> is no strong need to split this patch, I would rather not do that
>> extra work.
>
> I'm not looking for reasons to split this patch, I'm looking for
> stronger reasons to keep it just like it is :)
>
> Your hunch that complications would arise for simple unions plausible:
> there the same conditional needs to be applied both to the C enum's
> member and the C union member.
>
> For the generated C code to compile, each union tag enum member
> conditional must imply the corresponding variant conditional.
>
> For flat unions, the two are separate.  The QAPI generator makes no
> effort to check the enum member's if condition implies the union
> variant's if condition; if you mess them up in the schema, you get to
> deal with the C compilation errors.
>
> For simple unions, the two are one.
>
> If we separate the generator updates for enums and for union members,
> and do enum members first, then unions with conditional tag members
> can't compile.  Corrollary: simple unions with conditional variants
> can't compile.
>
> What if we do union members first?
>
> Again, I'm not asking for patch splitting here, I'm just trying to
> arrive at a clearer understanding to avoid making insufficiently
> supported claims in the commit message.  The combined patch looks small
> and clean enough to keep it combined.
>
> [...]

What about this commit message:

    qapi: Add #if conditions to generated code members

    Wrap generated enum and struct members and their supporting code with
    #if/#endif, using the .ifcond members added in the previous patches.

    We do enum and struct in a single patch because union tag enum and the
    associated variants tie them together, and dealing with that to split
    the patch doesn't seem worthwhile.

Re: [Qemu-devel] [PATCH v6 15/27] qapi: rename allow_dict to allow_implicit

[Marc-André Lureau]

Hi
On Mon, Dec 10, 2018 at 12:51 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > On Wed, Dec 5, 2018 at 10:41 PM Markus Armbruster <armbru@redhat.com> wrote:
> >>
> >> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
> >>
> >> > This makes it a bit clearer what is the intent of the dictionnary for
> >>
> >> dictionary
> >
> > sigh, this must be a very common misspell (dictionnaire in french)
>
> Muscle memory...
>
> >> > the check_type() function, since there was some confusion on a
> >> > previous iteration of this series.
> >>
> >> I don't remember... got a pointer to that confusion handy?
> >
> > https://lists.nongnu.org/archive/html/qemu-devel/2017-12/msg01584.html
>
> Thanks!
>
> My review comment was
>
>     > diff --git a/scripts/qapi.py b/scripts/qapi.py
>     > index df2a304e8f..15711f96fa 100644
>     > --- a/scripts/qapi.py
>     > +++ b/scripts/qapi.py
>     > @@ -696,7 +696,15 @@ def check_type(info, source, value, allow_array=False,
>     >          return
>     >
>     >      if not allow_dict:
>     > -        raise QAPISemError(info, "%s should be a type name" % source)
>     > +        if isinstance(value, dict) and 'type' in value:
>     > +            check_type(info, source, value['type'], allow_array,
>     > +                       allow_dict, allow_optional, allow_metas)
>     > +            if 'if' in value:
>     > +                check_if(value, info)
>     > +            check_unknown_keys(info, value, {'type', 'if'})
>     > +            return
>     > +        else:
>     > +            raise QAPISemError(info, "%s should be a type name" % source)
>
>     @allow_dict becomes a misnomer: dictionaries are now always allowed, but
>     they have different meaning (implicit type vs. named type with
>     additional attributes).
>
>     Rename to @allow_implicit?
>
> As far as I can tell, it's not an issue in the current version of your
> patches anymore:
>
>     def check_type(info, source, value, allow_array=False,
>                    allow_implicit=False, allow_optional=False,
>                    allow_metas=[]):
>         global all_names
>
>         if value is None:
>             return
>
>         # Check if array type for value is okay
>         if isinstance(value, list):
>             if not allow_array:
>                 raise QAPISemError(info, "%s cannot be an array" % source)
>             if len(value) != 1 or not isinstance(value[0], str):
>                 raise QAPISemError(info,
>                                    "%s: array type must contain single type name" %
>                                    source)
>             value = value[0]
>
>         # Check if type name for value is okay
>         if isinstance(value, str):
>             if value not in all_names:
>                 raise QAPISemError(info, "%s uses unknown type '%s'"
>                                    % (source, value))
>             if not all_names[value] in allow_metas:
>                 raise QAPISemError(info, "%s cannot use %s type '%s'" %
>                                    (source, all_names[value], value))
>             return
>
>         if not allow_implicit:
>             raise QAPISemError(info, "%s should be a type name" % source)
>
>         if not isinstance(value, OrderedDict):
>             raise QAPISemError(info,
>                                "%s should be a dictionary or type name" % source)
>
>         # value is a dictionary, check that each member is okay
>         for (key, arg) in value.items():
>             check_name(info, "Member of %s" % source, key,
>                        allow_optional=allow_optional)
>             if c_name(key, False) == 'u' or c_name(key, False).startswith('has_'):
>                 raise QAPISemError(info, "Member of %s uses reserved name '%s'"
>                                    % (source, key))
>             # Todo: allow dictionaries to represent default values of
>             # an optional argument.
>             if isinstance(arg, dict):
>                 check_known_keys(info, "member '%s' of %s" % (key, source),
>                                  arg, ['type'], ['if'])
>                 typ = arg['type']
>             else:
>                 typ = arg
>             check_type(info, "Member '%s' of %s" % (key, source),
>                        typ, allow_array=True,
>                        allow_metas=['built-in', 'union', 'alternate', 'struct',
>                                     'enum'])
>
>
> >> > Suggested-by: Markus Armbruster <armbru@redhat.com>
> >> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>

Feel free to drop it (allow_implicit seems a bit more clear to me though).

-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v6 21/27] qapi: add #if conditions to generated code members

[Marc-André Lureau]

On Mon, Dec 10, 2018 at 2:11 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Markus Armbruster <armbru@redhat.com> writes:
>
> > Marc-André Lureau <marcandre.lureau@redhat.com> writes:
> >
> >> Hi
> >> On Thu, Dec 6, 2018 at 9:42 PM Markus Armbruster <armbru@redhat.com> wrote:
> >>>
> >>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
> >>>
> >>> > Wrap generated enum/struct members and code with #if/#endif, using the
> >>>
> >>> enum and struct members
> >>
> >> ok
> >>
> >>>
> >>> > .ifcond members added in the previous patches.
> >>> >
> >>> > Some types generate both enum and struct members for example, so a
> >>> > step-by-step is unnecessarily complicated to deal with (it would
> >>> > easily generate invalid intermediary code).
> >>>
> >>> Can you give an example of a schema definition that would lead to
> >>> complications?
> >>>
> >>
> >> Honestly, I don't remember well (it's been a while I wrote that code).
> >
> > I know...
> >
> >> It must be related to implicit enums, such as union kind... If there
> >> is no strong need to split this patch, I would rather not do that
> >> extra work.
> >
> > I'm not looking for reasons to split this patch, I'm looking for
> > stronger reasons to keep it just like it is :)
> >
> > Your hunch that complications would arise for simple unions plausible:
> > there the same conditional needs to be applied both to the C enum's
> > member and the C union member.
> >
> > For the generated C code to compile, each union tag enum member
> > conditional must imply the corresponding variant conditional.
> >
> > For flat unions, the two are separate.  The QAPI generator makes no
> > effort to check the enum member's if condition implies the union
> > variant's if condition; if you mess them up in the schema, you get to
> > deal with the C compilation errors.
> >
> > For simple unions, the two are one.
> >
> > If we separate the generator updates for enums and for union members,
> > and do enum members first, then unions with conditional tag members
> > can't compile.  Corrollary: simple unions with conditional variants
> > can't compile.
> >
> > What if we do union members first?
> >
> > Again, I'm not asking for patch splitting here, I'm just trying to
> > arrive at a clearer understanding to avoid making insufficiently
> > supported claims in the commit message.  The combined patch looks small
> > and clean enough to keep it combined.
> >
> > [...]
>
> What about this commit message:
>
>     qapi: Add #if conditions to generated code members
>
>     Wrap generated enum and struct members and their supporting code with
>     #if/#endif, using the .ifcond members added in the previous patches.
>
>     We do enum and struct in a single patch because union tag enum and the
>     associated variants tie them together, and dealing with that to split
>     the patch doesn't seem worthwhile.
>


ack, thanks

-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v6 18/27] qapi: add an error in case a discriminator is conditionnal

[Marc-André Lureau]

Hi

On Thu, Dec 6, 2018 at 8:25 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>
> > Making a discriminator conditonal doesn't make much sense.
>
> Good point (so easy to overlook!), but why first add the 'if' feature
> broken that way in PATCH 16, then fix it up in PATCH 18?

Not sure, I guess I found out after. Feel free to squash

>
> >                                                            Instead,
> > the union could be made conditional.
>
> What do you mean by that?

That the alternative is probably to make the whole union conditional

>
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >  scripts/qapi/common.py                          | 11 +++++++++--
> >  tests/Makefile.include                          |  1 +
> >  .../flat-union-invalid-if-discriminator.err     |  1 +
> >  .../flat-union-invalid-if-discriminator.exit    |  1 +
> >  .../flat-union-invalid-if-discriminator.json    | 17 +++++++++++++++++
> >  .../flat-union-invalid-if-discriminator.out     |  0
> >  6 files changed, 29 insertions(+), 2 deletions(-)
> >  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.err
> >  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.exit
> >  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.json
> >  create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.out
> >
> > diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> > index 9b95f8cfe9..13fbb28493 100644
> > --- a/scripts/qapi/common.py
> > +++ b/scripts/qapi/common.py
> > @@ -577,7 +577,8 @@ def find_alternate_member_qtype(qapi_type):
> >
> >  # Return the discriminator enum define if discriminator is specified as an
> >  # enum type, otherwise return None.
> > -def discriminator_find_enum_define(expr):
> > +def discriminator_find_enum_define(expr, info):
> > +    name = expr['union']
> >      base = expr.get('base')
> >      discriminator = expr.get('discriminator')
> >
> > @@ -592,6 +593,11 @@ def discriminator_find_enum_define(expr):
> >      if not discriminator_member:
> >          return None
> >
> > +    if discriminator_member.get('if'):
> > +        raise QAPISemError(info, 'The discriminator %s.%s for union %s '
> > +                           'must not be conditional' %
> > +                           (base, discriminator, name))
> > +
> >      return enum_types.get(discriminator_member['type'])
> >
> >
>
> I'm afraid this isn't the proper place to check.  It's an accessor
> function for @struct_types and @enum_types.  The check should go into
> check_union(), like this:

indeed, thanks for the hint

>
> diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
> index 0cf39df404..c1bc9e9c29 100644
> --- a/scripts/qapi/common.py
> +++ b/scripts/qapi/common.py
> @@ -799,6 +794,11 @@ def check_union(expr, info):
>                                 "Discriminator '%s' is not a member of base "
>                                 "struct '%s'"
>                                 % (discriminator, base))
> +        if discriminator_member.get('if'):
> +            raise QAPISemError(info, 'The discriminator %s.%s for union %s '
> +                               'must not be conditional' %
> +                               (base, discriminator, name))
> +
>          enum_define = enum_types.get(discriminator_member['type'])
>          allow_metas = ['struct']
>          # Do not allow string discriminator
>
> > @@ -1020,7 +1026,8 @@ def check_exprs(exprs):
> >
> >          if 'include' in expr:
> >              continue
> > -        if 'union' in expr and not discriminator_find_enum_define(expr):
> > +        info = expr_elem['info']
> > +        if 'union' in expr and not discriminator_find_enum_define(expr, info):
> >              name = '%sKind' % expr['union']
> >          elif 'alternate' in expr:
> >              name = '%sKind' % expr['alternate']
> > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > index 43e100a6cd..abc3fdf764 100644
> > --- a/tests/Makefile.include
> > +++ b/tests/Makefile.include
> > @@ -510,6 +510,7 @@ qapi-schema += flat-union-inline.json
> >  qapi-schema += flat-union-int-branch.json
> >  qapi-schema += flat-union-invalid-branch-key.json
> >  qapi-schema += flat-union-invalid-discriminator.json
> > +qapi-schema += flat-union-invalid-if-discriminator.json
> >  qapi-schema += flat-union-no-base.json
> >  qapi-schema += flat-union-optional-discriminator.json
> >  qapi-schema += flat-union-string-discriminator.json
> > diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.err b/tests/qapi-schema/flat-union-invalid-if-discriminator.err
> > new file mode 100644
> > index 0000000000..0c94c9860d
> > --- /dev/null
> > +++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.err
> > @@ -0,0 +1 @@
> > +tests/qapi-schema/flat-union-invalid-if-discriminator.json:13: The discriminator TestBase.enum1 for union TestUnion must not be conditional
> > diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.exit b/tests/qapi-schema/flat-union-invalid-if-discriminator.exit
> > new file mode 100644
> > index 0000000000..d00491fd7e
> > --- /dev/null
> > +++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.exit
> > @@ -0,0 +1 @@
> > +1
> > diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.json b/tests/qapi-schema/flat-union-invalid-if-discriminator.json
> > new file mode 100644
> > index 0000000000..618ec36396
> > --- /dev/null
> > +++ b/tests/qapi-schema/flat-union-invalid-if-discriminator.json
> > @@ -0,0 +1,17 @@
> > +{ 'enum': 'TestEnum',
> > +  'data': [ 'value1', 'value2' ] }
> > +
> > +{ 'struct': 'TestBase',
> > +  'data': { 'enum1': { 'type': 'TestEnum', 'if': 'FOO' } } }
> > +
> > +{ 'struct': 'TestTypeA',
> > +  'data': { 'string': 'str' } }
> > +
> > +{ 'struct': 'TestTypeB',
> > +  'data': { 'integer': 'int' } }
> > +
> > +{ 'union': 'TestUnion',
> > +  'base': 'TestBase',
> > +  'discriminator': 'enum1',
> > +  'data': { 'value1': 'TestTypeA',
> > +            'value2': 'TestTypeB' } }
> > diff --git a/tests/qapi-schema/flat-union-invalid-if-discriminator.out b/tests/qapi-schema/flat-union-invalid-if-discriminator.out
> > new file mode 100644
> > index 0000000000..e69de29bb2
>


--
Marc-André Lureau

Re: [Qemu-devel] [PATCH v6 21/27] qapi: add #if conditions to generated code members

[Marc-André Lureau]

On Mon, Dec 10, 2018 at 2:11 PM Markus Armbruster <armbru@redhat.com> wrote:
>
> Markus Armbruster <armbru@redhat.com> writes:
>
> > Marc-André Lureau <marcandre.lureau@redhat.com> writes:
> >
> >> Hi
> >> On Thu, Dec 6, 2018 at 9:42 PM Markus Armbruster <armbru@redhat.com> wrote:
> >>>
> >>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
> >>>
> >>> > Wrap generated enum/struct members and code with #if/#endif, using the
> >>>
> >>> enum and struct members
> >>
> >> ok
> >>
> >>>
> >>> > .ifcond members added in the previous patches.
> >>> >
> >>> > Some types generate both enum and struct members for example, so a
> >>> > step-by-step is unnecessarily complicated to deal with (it would
> >>> > easily generate invalid intermediary code).
> >>>
> >>> Can you give an example of a schema definition that would lead to
> >>> complications?
> >>>
> >>
> >> Honestly, I don't remember well (it's been a while I wrote that code).
> >
> > I know...
> >
> >> It must be related to implicit enums, such as union kind... If there
> >> is no strong need to split this patch, I would rather not do that
> >> extra work.
> >
> > I'm not looking for reasons to split this patch, I'm looking for
> > stronger reasons to keep it just like it is :)
> >
> > Your hunch that complications would arise for simple unions plausible:
> > there the same conditional needs to be applied both to the C enum's
> > member and the C union member.
> >
> > For the generated C code to compile, each union tag enum member
> > conditional must imply the corresponding variant conditional.
> >
> > For flat unions, the two are separate.  The QAPI generator makes no
> > effort to check the enum member's if condition implies the union
> > variant's if condition; if you mess them up in the schema, you get to
> > deal with the C compilation errors.
> >
> > For simple unions, the two are one.
> >
> > If we separate the generator updates for enums and for union members,
> > and do enum members first, then unions with conditional tag members
> > can't compile.  Corrollary: simple unions with conditional variants
> > can't compile.
> >
> > What if we do union members first?
> >
> > Again, I'm not asking for patch splitting here, I'm just trying to
> > arrive at a clearer understanding to avoid making insufficiently
> > supported claims in the commit message.  The combined patch looks small
> > and clean enough to keep it combined.
> >
> > [...]
>
> What about this commit message:
>
>     qapi: Add #if conditions to generated code members
>
>     Wrap generated enum and struct members and their supporting code with
>     #if/#endif, using the .ifcond members added in the previous patches.
>
>     We do enum and struct in a single patch because union tag enum and the
>     associated variants tie them together, and dealing with that to split
>     the patch doesn't seem worthwhile.

ack, thanks