* [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API
@ 2016-06-02 16:46 Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
` (11 more replies)
0 siblings, 12 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
This is a followup of previously posted work in 2.6 cycle:
v1: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg04618.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01454.html
v3: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02498.html
v4: https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01661.html
Many years ago I was responsible for adding the 'qemu_acl' type
and associated HMP commands. Looking back at it now, it is quite
a poor facility with a couple of bad limitations. First, the
responsibility for creating the ACLs was left with the QEMU network
service (VNC server was only thing ever doing it). This meant you
could not share ACLs across multiple services. Second, there was
no way to populate ACLs on the command line, you had no choice but
to use the HMP commands. Third, the API was hardcoded around the
idea of an in-QEMU implementation, leaving no scope for plugging
in alternative implementations backed by, for example, LDAP or PAM.
This series introduces a much better authorization API design
to QEMU that addresses all these problems, and maintains back
compatibility. It of course is based on the QOM framework, so
that immediately gives us ability to create objects via the
CLI, HMP or QMP. There is an abstract base clss "QAuthZ" which
defines the basic API for QEMU network services to use, and a
specific implementation "QAuthZ" simple which replicates the
functionality of 'qemu_acl'. It is thus possible to add other
impls, without changing any other part of QEMU in the future.
Finally, the user is responsible for creating the ACL objects,
so they can have one ACL associated with all their TLS enabled
network services.
There was only one small problem with this, specifically the
-object CLI arg and HMP 'object_add' command had no way to let
the user specify non-scalar properties for objects. eg if an
object had a property which is a list of structs, you are out
of luck if you want to create it without using QMP.
Thus the first three patches do some work around QAPI / QOM
to make it possible to specify non-scalar properties with
the -object CLI arg and HMP 'object_add' command. See the
respective patches for illustration of the syntax used. Some
of Max's recent block patches also depend on the qdict_crumple
method in patch 1.
The patches 4 and 5 introduce the new base class and specific
implementation.
Patch 6 kills the old qemu_acl code, updating any existing
callers of it to use the QAuthZSimple QOM class instead.
Patches 7-11 add support for associating ACLs with the
network services supporting TLS encryption (NBD, migration,
chardev and VNC).
Changed in v5:
- Resolved conflicts with Eric's visitor refactoring which
made it stricter about struct begin/end calls
- Added support for ACLs to migration code now its TLS
support is merged.
- Fixed typos in example in commit message
Changed in v4:
- Ensure examples use shell escaping for '*' (Eric)
- Add more tests for crumple impl (Eric)
- Raise error if sasl-acl/tls-acl are requested but
sasl/tls auth are not enabled (Eric)
- Document return codes for auth check more clearly (Eric)
- Don't silently turn a glob match into a strcmp
if fnmatch is not present (Eric)
- Other misc small typos/fixes (Eric)
Changed in v3:
- Created separate qdict_list_size method (Max)
- Added unit tests for case of empty dict (Max)
- Fix variable names to use underscore separator (Max)
- Fix potential free of uninitialized variables (Max)
- Use QObject APIs for casts, instead of C type casts (Max)
Changed in v2:
- Adapt to changes in qapi visitor APIs
- Add a 'bool recursive' flag to qdict_crumple (Max)
- Fix memory leaks in qdict_crumple (Max)
- Split out key splitting code from qdict_crumple (Max)
- Use saner variable names in qdict_crumple (Max)
- Added some tests for bad inputs to qdict_crumple
Daniel P. Berrange (11):
qdict: implement a qdict_crumple method for un-flattening a dict
qapi: allow QmpInputVisitor to auto-cast types
qom: support arbitrary non-scalar properties with -object
util: add QAuthZ object as an authorization base class
util: add QAuthZSimple object type for a simple access control list
acl: delete existing ACL implementation
qemu-nbd: add support for ACLs for TLS clients
nbd: allow an ACL to be set with nbd-server-start QMP command
migration: add support for a "tls-acl" migration parameter
chardev: add support for ACLs for TLS clients
vnc: allow specifying a custom ACL object name
MAINTAINERS | 7 +
Makefile | 9 +-
Makefile.objs | 2 +
Makefile.target | 2 +
blockdev-nbd.c | 10 +-
crypto/tlssession.c | 28 +++-
docs/qapi-code-gen.txt | 2 +-
hmp.c | 28 ++--
include/qapi/qmp-input-visitor.h | 6 +-
include/qapi/qmp/qdict.h | 1 +
include/qemu/acl.h | 74 ---------
include/qemu/authz-simple.h | 115 ++++++++++++++
include/qemu/authz.h | 89 +++++++++++
include/qom/object_interfaces.h | 10 +-
migration/migration.c | 7 +
migration/tls.c | 2 +-
monitor.c | 181 ++++++++++++++-------
qapi-schema.json | 28 +++-
qapi/block.json | 4 +-
qapi/opts-visitor.c | 1 +
qapi/qmp-input-visitor.c | 89 +++++++++--
qapi/util.json | 47 ++++++
qemu-char.c | 11 +-
qemu-nbd.c | 13 +-
qemu-nbd.texi | 4 +
qmp-commands.hx | 2 +-
qmp.c | 4 +-
qobject/qdict.c | 282 +++++++++++++++++++++++++++++++++
qom/object_interfaces.c | 49 ++++--
qom/qom-qobject.c | 2 +-
replay/replay-input.c | 2 +-
scripts/qapi-commands.py | 2 +-
tests/.gitignore | 1 +
tests/Makefile | 5 +-
tests/check-qdict.c | 229 +++++++++++++++++++++++++++
tests/check-qnull.c | 2 +-
tests/check-qom-proplist.c | 317 ++++++++++++++++++++++++++++++++++++-
tests/test-authz-simple.c | 183 +++++++++++++++++++++
tests/test-crypto-tlssession.c | 13 +-
tests/test-io-channel-tls.c | 14 +-
tests/test-qmp-commands.c | 2 +-
tests/test-qmp-input-strict.c | 2 +-
tests/test-qmp-input-visitor.c | 115 +++++++++++++-
tests/test-visitor-serialization.c | 2 +-
ui/vnc-auth-sasl.c | 2 +-
ui/vnc-auth-sasl.h | 4 +-
ui/vnc.c | 80 ++++++++--
util/Makefile.objs | 4 +-
util/acl.c | 188 ----------------------
util/authz-simple.c | 314 ++++++++++++++++++++++++++++++++++++
util/authz.c | 46 ++++++
util/qemu-sockets.c | 2 +-
52 files changed, 2206 insertions(+), 432 deletions(-)
delete mode 100644 include/qemu/acl.h
create mode 100644 include/qemu/authz-simple.h
create mode 100644 include/qemu/authz.h
create mode 100644 qapi/util.json
create mode 100644 tests/test-authz-simple.c
delete mode 100644 util/acl.c
create mode 100644 util/authz-simple.c
create mode 100644 util/authz.c
--
2.5.5
^ permalink raw reply [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-09 13:20 ` Markus Armbruster
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types Daniel P. Berrange
` (10 subsequent siblings)
11 siblings, 1 reply; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
The qdict_flatten() method will take a dict whose elements are
further nested dicts/lists and flatten them by concatenating
keys.
The qdict_crumple() method aims to do the reverse, taking a flat
qdict, and turning it into a set of nested dicts/lists. It will
apply nesting based on the key name, with a '.' indicating a
new level in the hierarchy. If the keys in the nested structure
are all numeric, it will create a list, otherwise it will create
a dict.
If the keys are a mixture of numeric and non-numeric, or the
numeric keys are not in strictly ascending order, an error will
be reported.
As an example, a flat dict containing
{
'foo.0.bar': 'one',
'foo.0.wizz': '1',
'foo.1.bar': 'two',
'foo.1.wizz': '2'
}
will get turned into a dict with one element 'foo' whose
value is a list. The list elements will each in turn be
dicts.
{
'foo': [
{ 'bar': 'one', 'wizz': '1' },
{ 'bar': 'two', 'wizz': '2' }
],
}
If the key is intended to contain a literal '.', then it must
be escaped as '..'. ie a flat dict
{
'foo..bar': 'wizz',
'bar.foo..bar': 'eek',
'bar.hello': 'world'
}
Will end up as
{
'foo.bar': 'wizz',
'bar': {
'foo.bar': 'eek',
'hello': 'world'
}
}
The intent of this function is that it allows a set of QemuOpts
to be turned into a nested data structure that mirrors the nesting
used when the same object is defined over QMP.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
include/qapi/qmp/qdict.h | 1 +
qobject/qdict.c | 282 +++++++++++++++++++++++++++++++++++++++++++++++
tests/check-qdict.c | 229 ++++++++++++++++++++++++++++++++++++++
3 files changed, 512 insertions(+)
diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index 71b8eb0..8a3ac13 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
void qdict_array_split(QDict *src, QList **dst);
int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
void qdict_join(QDict *dest, QDict *src, bool overwrite);
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 60f158c..ad6a563 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -17,6 +17,7 @@
#include "qapi/qmp/qbool.h"
#include "qapi/qmp/qstring.h"
#include "qapi/qmp/qobject.h"
+#include "qapi/error.h"
#include "qemu/queue.h"
#include "qemu-common.h"
#include "qemu/cutils.h"
@@ -683,6 +684,287 @@ void qdict_array_split(QDict *src, QList **dst)
}
}
+
+/**
+ * qdict_split_flat_key:
+ * @key: the key string to split
+ * @prefix: non-NULL pointer to hold extracted prefix
+ * @suffix: non-NULL pointer to hold extracted suffix
+ *
+ * Given a flattened key such as 'foo.0.bar', split it
+ * into two parts at the first '.' separator. Allows
+ * double dot ('..') to escape the normal separator.
+ *
+ * eg
+ * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned
+ * 'prefix' string. The 'suffix' string will be left
+ * in escaped format, so it can be fed back into the
+ * qdict_split_flat_key() key as the input later.
+ *
+ * The caller is responsible for freeing the strings
+ * returned in @prefix and @suffix using g_free().
+ */
+static void qdict_split_flat_key(const char *key, char **prefix, char **suffix)
+{
+ const char *separator;
+ size_t i, j;
+
+ /* Find first '.' separator, but if there is a pair '..'
+ * that acts as an escape, so skip over '..' */
+ separator = NULL;
+ do {
+ if (separator) {
+ separator += 2;
+ } else {
+ separator = key;
+ }
+ separator = strchr(separator, '.');
+ } while (separator && separator[1] == '.');
+
+ if (separator) {
+ *prefix = g_strndup(key,
+ separator - key);
+ *suffix = g_strdup(separator + 1);
+ } else {
+ *prefix = g_strdup(key);
+ *suffix = NULL;
+ }
+
+ /* Unescape the '..' sequence into '.' */
+ for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+ if ((*prefix)[i] == '.') {
+ assert((*prefix)[i + 1] == '.');
+ i++;
+ }
+ (*prefix)[j] = (*prefix)[i];
+ }
+ (*prefix)[j] = '\0';
+}
+
+
+/**
+ * qdict_list_size:
+ * @maybe_list: dict that may be only list elements
+ *
+ * Determine whether all keys in @maybe_list are
+ * valid list elements. If they are all valid,
+ * then this returns the number of elements. If
+ * they all look like non-numeric keys, then returns
+ * zero. If there is a mix of numeric and non-numeric
+ * keys, then an error is set as it is both a list
+ * and a dict at once.
+ *
+ * Returns: number of list elements, 0 if a dict, -1 on error
+ */
+static ssize_t qdict_list_size(QDict *maybe_list, Error **errp)
+{
+ const QDictEntry *entry, *next;
+ ssize_t len = 0;
+ ssize_t max = -1;
+ int is_list = -1;
+ int64_t val;
+
+ entry = qdict_first(maybe_list);
+ while (entry != NULL) {
+ next = qdict_next(maybe_list, entry);
+
+ if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
+ if (is_list == -1) {
+ is_list = 1;
+ } else if (!is_list) {
+ error_setg(errp,
+ "Cannot crumple a dictionary with a mix of list "
+ "and non-list keys");
+ return -1;
+ }
+ len++;
+ if (val > max) {
+ max = val;
+ }
+ } else {
+ if (is_list == -1) {
+ is_list = 0;
+ } else if (is_list) {
+ error_setg(errp,
+ "Cannot crumple a dictionary with a mix of list "
+ "and non-list keys");
+ return -1;
+ }
+ }
+
+ entry = next;
+ }
+
+ if (is_list == -1) {
+ is_list = 0;
+ }
+
+ if (len != (max + 1)) {
+ error_setg(errp, "List indexes are not continuous, "
+ "saw %zd elements but %zd largest index",
+ len, max);
+ return -1;
+ }
+
+ return is_list ? len : 0;
+}
+
+/**
+ * qdict_crumple:
+ * @src: the original flat dictionary to crumple
+ * @recursive: true to recursively crumple nested dictionaries
+ *
+ * Takes a flat dictionary whose keys use '.' separator to
+ * indicate nesting, and values are scalars, crumplings it
+ * into a nested structure. If the @recursive parameter is
+ * false, then only the first level of structure implied
+ * by the keys will be crumpled. If @recursive is true,
+ * then the input will be recursively crumpled to expand
+ * all levels of structure in the keys.
+ *
+ * To include a literal '.' in a key name, it must be escaped
+ * as '..'
+ *
+ * For example, an input of:
+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * will result in any output of:
+ *
+ * {
+ * 'foo': [
+ * { 'bar': 'one', 'wizz': '1' },
+ * { 'bar': 'two', 'wizz': '2' }
+ * ],
+ * }
+ *
+ * Returns: either a QDict or QList for the nested data structure
+ */
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp)
+{
+ const QDictEntry *entry, *next;
+ QDict *two_level, *multi_level = NULL;
+ QObject *dst = NULL, *child;
+ ssize_t list_len;
+ size_t i;
+ char *prefix = NULL, *suffix = NULL;
+
+ two_level = qdict_new();
+ entry = qdict_first(src);
+
+ /* Step 1: split our totally flat dict into a two level dict */
+ while (entry != NULL) {
+ next = qdict_next(src, entry);
+
+ if (qobject_type(entry->value) == QTYPE_QDICT ||
+ qobject_type(entry->value) == QTYPE_QLIST) {
+ error_setg(errp, "Value %s is not a scalar",
+ entry->key);
+ goto error;
+ }
+
+ qdict_split_flat_key(entry->key, &prefix, &suffix);
+
+ child = qdict_get(two_level, prefix);
+ if (suffix) {
+ if (child) {
+ if (qobject_type(child) != QTYPE_QDICT) {
+ error_setg(errp, "Key %s prefix is already set as a scalar",
+ prefix);
+ goto error;
+ }
+ } else {
+ child = QOBJECT(qdict_new());
+ qdict_put_obj(two_level, prefix, child);
+ }
+ qobject_incref(entry->value);
+ qdict_put_obj(qobject_to_qdict(child), suffix, entry->value);
+ } else {
+ if (child) {
+ error_setg(errp, "Key %s prefix is already set as a dict",
+ prefix);
+ goto error;
+ }
+ qobject_incref(entry->value);
+ qdict_put_obj(two_level, prefix, entry->value);
+ }
+
+ g_free(suffix);
+ g_free(prefix);
+ suffix = prefix = NULL;
+
+ entry = next;
+ }
+
+ /* Step 2: optionally process the two level dict recursively
+ * into a multi-level dict */
+ if (recursive) {
+ multi_level = qdict_new();
+ entry = qdict_first(two_level);
+ while (entry != NULL) {
+ next = qdict_next(two_level, entry);
+
+ if (qobject_type(entry->value) == QTYPE_QDICT) {
+ child = qdict_crumple(qobject_to_qdict(entry->value),
+ recursive, errp);
+ if (!child) {
+ goto error;
+ }
+
+ qdict_put_obj(multi_level, entry->key, child);
+ } else {
+ qobject_incref(entry->value);
+ qdict_put_obj(multi_level, entry->key, entry->value);
+ }
+
+ entry = next;
+ }
+ QDECREF(two_level);
+ } else {
+ multi_level = two_level;
+ }
+ two_level = NULL;
+
+ /* Step 3: detect if we need to turn our dict into list */
+ list_len = qdict_list_size(multi_level, errp);
+ if (list_len < 0) {
+ goto error;
+ }
+
+ if (list_len) {
+ dst = QOBJECT(qlist_new());
+
+ for (i = 0; i < list_len; i++) {
+ char *key = g_strdup_printf("%zu", i);
+
+ child = qdict_get(multi_level, key);
+ g_free(key);
+ assert(child);
+
+ qobject_incref(child);
+ qlist_append_obj(qobject_to_qlist(dst), child);
+ }
+ QDECREF(multi_level);
+ } else {
+ dst = QOBJECT(multi_level);
+ }
+
+ return dst;
+
+ error:
+ g_free(suffix);
+ g_free(prefix);
+ QDECREF(multi_level);
+ QDECREF(two_level);
+ qobject_decref(dst);
+ return NULL;
+}
+
+
/**
* qdict_array_entries(): Returns the number of direct array entries if the
* sub-QDict of src specified by the prefix in subqdict (or src itself for
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index a43056c..0d12f40 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -15,6 +15,7 @@
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qstring.h"
+#include "qapi/error.h"
#include "qemu-common.h"
/*
@@ -596,6 +597,223 @@ static void qdict_join_test(void)
QDECREF(dict2);
}
+
+static void qdict_crumple_test_nonrecursive(void)
+{
+ QDict *src, *dst, *rules, *vnc;
+ QObject *child, *res;
+
+ src = qdict_new();
+ qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
+ qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
+ qdict_put(src, "rule.0.match", qstring_from_str("fred"));
+ qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+ qdict_put(src, "rule.1.match", qstring_from_str("bob"));
+ qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
+
+ res = qdict_crumple(src, false, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
+
+ dst = qobject_to_qdict(res);
+
+ g_assert_cmpint(qdict_size(dst), ==, 2);
+
+ child = qdict_get(dst, "vnc");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ vnc = qdict_get_qdict(dst, "vnc");
+
+ g_assert_cmpint(qdict_size(vnc), ==, 2);
+
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(vnc, "listen.addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(vnc, "listen.port"));
+
+ child = qdict_get(dst, "rule");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ rules = qdict_get_qdict(dst, "rule");
+
+ g_assert_cmpint(qdict_size(rules), ==, 4);
+
+ g_assert_cmpstr("fred", ==, qdict_get_str(rules, "0.match"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rules, "0.policy"));
+ g_assert_cmpstr("bob", ==, qdict_get_str(rules, "1.match"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rules, "1.policy"));
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_listtop(void)
+{
+ QDict *src, *rule;
+ QList *rules;
+ QObject *res;
+
+ src = qdict_new();
+ qdict_put(src, "0.match.name", qstring_from_str("Fred"));
+ qdict_put(src, "0.match.email", qstring_from_str("fred@example.com"));
+ qdict_put(src, "0.policy", qstring_from_str("allow"));
+ qdict_put(src, "1.match.name", qstring_from_str("Bob"));
+ qdict_put(src, "1.match.email", qstring_from_str("bob@example.com"));
+ qdict_put(src, "1.policy", qstring_from_str("deny"));
+
+ res = qdict_crumple(src, false, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QLIST);
+
+ rules = qobject_to_qlist(res);
+
+ g_assert_cmpint(qlist_size(rules), ==, 2);
+
+ g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 3);
+
+ g_assert_cmpstr("Fred", ==, qdict_get_str(rule, "match.name"));
+ g_assert_cmpstr("fred@example.com", ==, qdict_get_str(rule, "match.email"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 3);
+
+ g_assert_cmpstr("Bob", ==, qdict_get_str(rule, "match.name"));
+ g_assert_cmpstr("bob@example.com", ==, qdict_get_str(rule, "match.email"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ QDECREF(src);
+ qobject_decref(res);
+}
+
+
+static void qdict_crumple_test_recursive(void)
+{
+ QDict *src, *dst, *rule, *vnc, *acl, *listen;
+ QObject *child, *res;
+ QList *rules;
+
+ src = qdict_new();
+ qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
+ qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
+ qdict_put(src, "vnc.acl.rules.0.match", qstring_from_str("fred"));
+ qdict_put(src, "vnc.acl.rules.0.policy", qstring_from_str("allow"));
+ qdict_put(src, "vnc.acl.rules.1.match", qstring_from_str("bob"));
+ qdict_put(src, "vnc.acl.rules.1.policy", qstring_from_str("deny"));
+ qdict_put(src, "vnc.acl.default", qstring_from_str("deny"));
+
+ res = qdict_crumple(src, true, &error_abort);
+
+ g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
+
+ dst = qobject_to_qdict(res);
+
+ g_assert_cmpint(qdict_size(dst), ==, 1);
+
+ child = qdict_get(dst, "vnc");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ vnc = qobject_to_qdict(child);
+
+ child = qdict_get(vnc, "listen");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ listen = qobject_to_qdict(child);
+ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
+ g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
+
+ child = qdict_get(vnc, "acl");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+ acl = qobject_to_qdict(child);
+
+ child = qdict_get(acl, "rules");
+ g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
+ rules = qobject_to_qlist(child);
+ g_assert_cmpint(qlist_size(rules), ==, 2);
+
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ rule = qobject_to_qdict(qlist_pop(rules));
+ g_assert_cmpint(qdict_size(rule), ==, 2);
+ g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+ QDECREF(rule);
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_empty(void)
+{
+ QDict *src, *dst;
+
+ src = qdict_new();
+
+ dst = (QDict *)qdict_crumple(src, true, &error_abort);
+
+ g_assert_cmpint(qdict_size(dst), ==, 0);
+
+ QDECREF(src);
+ QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_bad_inputs(void)
+{
+ QDict *src;
+ Error *error = NULL;
+
+ src = qdict_new();
+ /* rule.0 can't be both a string and a dict */
+ qdict_put(src, "rule.0", qstring_from_str("fred"));
+ qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+
+ src = qdict_new();
+ /* rule can't be both a list and a dict */
+ qdict_put(src, "rule.0", qstring_from_str("fred"));
+ qdict_put(src, "rule.a", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+
+ src = qdict_new();
+ /* The input should be flat, ie no dicts or lists */
+ qdict_put(src, "rule.a", qdict_new());
+ qdict_put(src, "rule.b", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+
+
+ src = qdict_new();
+ /* List indexes must not have gaps */
+ qdict_put(src, "rule.0", qdict_new());
+ qdict_put(src, "rule.3", qstring_from_str("allow"));
+
+ g_assert(qdict_crumple(src, true, &error) == NULL);
+ g_assert(error != NULL);
+ error_free(error);
+ error = NULL;
+ QDECREF(src);
+}
+
/*
* Errors test-cases
*/
@@ -743,6 +961,17 @@ int main(int argc, char **argv)
g_test_add_func("/errors/put_exists", qdict_put_exists_test);
g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
+ g_test_add_func("/public/crumple/nonrecursive",
+ qdict_crumple_test_nonrecursive);
+ g_test_add_func("/public/crumple/recursive",
+ qdict_crumple_test_recursive);
+ g_test_add_func("/public/crumple/listtop",
+ qdict_crumple_test_listtop);
+ g_test_add_func("/public/crumple/empty",
+ qdict_crumple_test_empty);
+ g_test_add_func("/public/crumple/bad_inputs",
+ qdict_crumple_test_bad_inputs);
+
/* The Big one */
if (g_test_slow()) {
g_test_add_func("/stress/test", qdict_stress_test);
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-08 12:01 ` Paolo Bonzini
2016-06-09 14:03 ` Markus Armbruster
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object Daniel P. Berrange
` (9 subsequent siblings)
11 siblings, 2 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
Currently the QmpInputVisitor assumes that all scalar
values are directly represented as their final types.
ie it assumes an 'int' is using QInt, and a 'bool' is
using QBool.
This extends it so that QString is optionally permitted
for any of the non-string scalar types. This behaviour
is turned on by requesting the 'autocast' flag in the
constructor.
This makes it possible to use QmpInputVisitor with a
QDict produced from QemuOpts, where everything is in
string format.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
docs/qapi-code-gen.txt | 2 +-
include/qapi/qmp-input-visitor.h | 6 +-
qapi/opts-visitor.c | 1 +
qapi/qmp-input-visitor.c | 89 ++++++++++++++++++++++------
qmp.c | 2 +-
qom/qom-qobject.c | 2 +-
replay/replay-input.c | 2 +-
scripts/qapi-commands.py | 2 +-
tests/check-qnull.c | 2 +-
tests/test-qmp-commands.c | 2 +-
tests/test-qmp-input-strict.c | 2 +-
tests/test-qmp-input-visitor.c | 115 ++++++++++++++++++++++++++++++++++++-
tests/test-visitor-serialization.c | 2 +-
util/qemu-sockets.c | 2 +-
14 files changed, 201 insertions(+), 30 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index d7d6987..e21773e 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -1008,7 +1008,7 @@ Example:
{
Error *err = NULL;
UserDefOne *retval;
- QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true);
+ QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true, false);
QapiDeallocVisitor *qdv;
Visitor *v;
UserDefOneList *arg1 = NULL;
diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qmp-input-visitor.h
index b0624d8..d35a053 100644
--- a/include/qapi/qmp-input-visitor.h
+++ b/include/qapi/qmp-input-visitor.h
@@ -24,8 +24,12 @@ typedef struct QmpInputVisitor QmpInputVisitor;
*
* Set @strict to reject a parse that doesn't consume all keys of a
* dictionary; otherwise excess input is ignored.
+ * Set @autocast to automatically convert string values into more
+ * specific types (numbers, bools, etc)
*/
-QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict);
+QmpInputVisitor *qmp_input_visitor_new(QObject *obj,
+ bool strict,
+ bool autocast);
void qmp_input_visitor_cleanup(QmpInputVisitor *v);
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index 4cf1cf8..00e4b1a 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -347,6 +347,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
}
if (opt->str) {
+ /* Keep these values in sync with same code in qmp-input-visitor.c */
if (strcmp(opt->str, "on") == 0 ||
strcmp(opt->str, "yes") == 0 ||
strcmp(opt->str, "y") == 0) {
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index aea90a1..92023b1 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -20,6 +20,7 @@
#include "qemu-common.h"
#include "qapi/qmp/types.h"
#include "qapi/qmp/qerror.h"
+#include "qemu/cutils.h"
#define QIV_STACK_SIZE 1024
@@ -45,6 +46,7 @@ struct QmpInputVisitor
/* True to reject parse in visit_end_struct() if unvisited keys remain. */
bool strict;
+ bool autocast;
};
static QmpInputVisitor *to_qiv(Visitor *v)
@@ -254,15 +256,25 @@ static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
Error **errp)
{
QmpInputVisitor *qiv = to_qiv(v);
- QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+ QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QInt *qint;
+ QString *qstr;
- if (!qint) {
- error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
- "integer");
+ qint = qobject_to_qint(qobj);
+ if (qint) {
+ *obj = qint_get_int(qint);
return;
}
- *obj = qint_get_int(qint);
+ qstr = qobject_to_qstring(qobj);
+ if (qstr && qstr->string && qiv->autocast) {
+ if (qemu_strtoll(qstr->string, NULL, 10, obj) == 0) {
+ return;
+ }
+ }
+
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "integer");
}
static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
@@ -270,30 +282,61 @@ static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
{
/* FIXME: qobject_to_qint mishandles values over INT64_MAX */
QmpInputVisitor *qiv = to_qiv(v);
- QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+ QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QInt *qint;
+ QString *qstr;
- if (!qint) {
- error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
- "integer");
+ qint = qobject_to_qint(qobj);
+ if (qint) {
+ *obj = qint_get_int(qint);
return;
}
- *obj = qint_get_int(qint);
+ qstr = qobject_to_qstring(qobj);
+ if (qstr && qstr->string && qiv->autocast) {
+ if (qemu_strtoull(qstr->string, NULL, 10, obj) == 0) {
+ return;
+ }
+ }
+
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "integer");
}
static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
Error **errp)
{
QmpInputVisitor *qiv = to_qiv(v);
- QBool *qbool = qobject_to_qbool(qmp_input_get_object(qiv, name, true));
+ QObject *qobj = qmp_input_get_object(qiv, name, true);
+ QBool *qbool;
+ QString *qstr;
- if (!qbool) {
- error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
- "boolean");
+ qbool = qobject_to_qbool(qobj);
+ if (qbool) {
+ *obj = qbool_get_bool(qbool);
return;
}
- *obj = qbool_get_bool(qbool);
+
+ qstr = qobject_to_qstring(qobj);
+ if (qstr && qstr->string && qiv->autocast) {
+ /* Keep these values in sync with same code in opts-visitor.c */
+ if (!strcasecmp(qstr->string, "on") ||
+ !strcasecmp(qstr->string, "yes") ||
+ !strcasecmp(qstr->string, "true")) {
+ *obj = true;
+ return;
+ }
+ if (!strcasecmp(qstr->string, "off") ||
+ !strcasecmp(qstr->string, "no") ||
+ !strcasecmp(qstr->string, "false")) {
+ *obj = false;
+ return;
+ }
+ }
+
+ error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "boolean");
}
static void qmp_input_type_str(Visitor *v, const char *name, char **obj,
@@ -319,6 +362,8 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
QObject *qobj = qmp_input_get_object(qiv, name, true);
QInt *qint;
QFloat *qfloat;
+ QString *qstr;
+ char *endp;
qint = qobject_to_qint(qobj);
if (qint) {
@@ -332,6 +377,15 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
return;
}
+ qstr = qobject_to_qstring(qobj);
+ if (qstr && qstr->string && qiv->autocast) {
+ errno = 0;
+ *obj = strtod(qstr->string, &endp);
+ if (errno == 0 && endp != qstr->string && *endp == '\0') {
+ return;
+ }
+ }
+
error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
"number");
}
@@ -381,7 +435,9 @@ void qmp_input_visitor_cleanup(QmpInputVisitor *v)
g_free(v);
}
-QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict)
+QmpInputVisitor *qmp_input_visitor_new(QObject *obj,
+ bool strict,
+ bool autocast)
{
QmpInputVisitor *v;
@@ -404,6 +460,7 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict)
v->visitor.type_null = qmp_input_type_null;
v->visitor.optional = qmp_input_optional;
v->strict = strict;
+ v->autocast = autocast;
v->root = obj;
qobject_incref(obj);
diff --git a/qmp.c b/qmp.c
index 3165f87..dcd0953 100644
--- a/qmp.c
+++ b/qmp.c
@@ -665,7 +665,7 @@ void qmp_object_add(const char *type, const char *id,
}
}
- qiv = qmp_input_visitor_new(props, true);
+ qiv = qmp_input_visitor_new(props, true, false);
obj = user_creatable_add_type(type, id, pdict,
qmp_input_get_visitor(qiv), errp);
qmp_input_visitor_cleanup(qiv);
diff --git a/qom/qom-qobject.c b/qom/qom-qobject.c
index b66088d..99666ce 100644
--- a/qom/qom-qobject.c
+++ b/qom/qom-qobject.c
@@ -23,7 +23,7 @@ void object_property_set_qobject(Object *obj, QObject *value,
{
QmpInputVisitor *qiv;
/* TODO: Should we reject, rather than ignore, excess input? */
- qiv = qmp_input_visitor_new(value, false);
+ qiv = qmp_input_visitor_new(value, false, false);
object_property_set(obj, qmp_input_get_visitor(qiv), name, errp);
qmp_input_visitor_cleanup(qiv);
diff --git a/replay/replay-input.c b/replay/replay-input.c
index 03e99d5..de82a59 100644
--- a/replay/replay-input.c
+++ b/replay/replay-input.c
@@ -37,7 +37,7 @@ static InputEvent *qapi_clone_InputEvent(InputEvent *src)
return NULL;
}
- qiv = qmp_input_visitor_new(obj, true);
+ qiv = qmp_input_visitor_new(obj, true, false);
iv = qmp_input_get_visitor(qiv);
visit_type_InputEvent(iv, NULL, &dst, &error_abort);
qmp_input_visitor_cleanup(qiv);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 8c6acb3..e48995d 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -115,7 +115,7 @@ def gen_marshal(name, arg_type, ret_type):
if arg_type and arg_type.members:
ret += mcgen('''
- QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true);
+ QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true, false);
QapiDeallocVisitor *qdv;
Visitor *v;
%(c_name)s arg = {0};
diff --git a/tests/check-qnull.c b/tests/check-qnull.c
index fd9c68f..4c11755 100644
--- a/tests/check-qnull.c
+++ b/tests/check-qnull.c
@@ -49,7 +49,7 @@ static void qnull_visit_test(void)
g_assert(qnull_.refcnt == 1);
obj = qnull();
- qiv = qmp_input_visitor_new(obj, true);
+ qiv = qmp_input_visitor_new(obj, true, false);
qobject_decref(obj);
visit_type_null(qmp_input_get_visitor(qiv), NULL, &error_abort);
qmp_input_visitor_cleanup(qiv);
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 5c3edd7..c86b282 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -222,7 +222,7 @@ static void test_dealloc_partial(void)
ud2_dict = qdict_new();
qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
- qiv = qmp_input_visitor_new(QOBJECT(ud2_dict), true);
+ qiv = qmp_input_visitor_new(QOBJECT(ud2_dict), true, false);
visit_type_UserDefTwo(qmp_input_get_visitor(qiv), NULL, &ud2, &err);
qmp_input_visitor_cleanup(qiv);
QDECREF(ud2_dict);
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 4602529..f7f1f00 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -55,7 +55,7 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
data->obj = qobject_from_jsonv(json_string, ap);
g_assert(data->obj);
- data->qiv = qmp_input_visitor_new(data->obj, true);
+ data->qiv = qmp_input_visitor_new(data->obj, true, false);
g_assert(data->qiv);
v = qmp_input_get_visitor(data->qiv);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index cee07ce..5691dc3 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -41,6 +41,7 @@ static void visitor_input_teardown(TestInputVisitorData *data,
function so that the JSON string used by the tests are kept in the test
functions (and not in main()). */
static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
+ bool strict, bool autocast,
const char *json_string,
va_list *ap)
{
@@ -51,7 +52,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
data->obj = qobject_from_jsonv(json_string, ap);
g_assert(data->obj);
- data->qiv = qmp_input_visitor_new(data->obj, false);
+ data->qiv = qmp_input_visitor_new(data->obj, strict, autocast);
g_assert(data->qiv);
v = qmp_input_get_visitor(data->qiv);
@@ -60,6 +61,21 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
return v;
}
+static GCC_FMT_ATTR(4, 5)
+Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
+ bool strict, bool autocast,
+ const char *json_string, ...)
+{
+ Visitor *v;
+ va_list ap;
+
+ va_start(ap, json_string);
+ v = visitor_input_test_init_internal(data, strict, autocast,
+ json_string, &ap);
+ va_end(ap);
+ return v;
+}
+
static GCC_FMT_ATTR(2, 3)
Visitor *visitor_input_test_init(TestInputVisitorData *data,
const char *json_string, ...)
@@ -68,7 +84,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
va_list ap;
va_start(ap, json_string);
- v = visitor_input_test_init_internal(data, json_string, &ap);
+ v = visitor_input_test_init_internal(data, false, false,
+ json_string, &ap);
va_end(ap);
return v;
}
@@ -83,7 +100,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
const char *json_string)
{
- return visitor_input_test_init_internal(data, json_string, NULL);
+ return visitor_input_test_init_internal(data, false, false,
+ json_string, NULL);
}
static void test_visitor_in_int(TestInputVisitorData *data,
@@ -115,6 +133,33 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
error_free_or_abort(&err);
}
+static void test_visitor_in_int_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ int64_t res = 0, value = -42;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, false, true,
+ "\"-42\"");
+
+ visit_type_int(v, NULL, &res, &error_abort);
+ g_assert_cmpint(res, ==, value);
+}
+
+static void test_visitor_in_int_noautocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ int64_t res = 0;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init(data, "\"-42\"");
+
+ visit_type_int(v, NULL, &res, &err);
+ g_assert(err != NULL);
+ error_free(err);
+}
+
static void test_visitor_in_bool(TestInputVisitorData *data,
const void *unused)
{
@@ -127,6 +172,32 @@ static void test_visitor_in_bool(TestInputVisitorData *data,
g_assert_cmpint(res, ==, true);
}
+static void test_visitor_in_bool_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ bool res = false;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, false, true, "\"true\"");
+
+ visit_type_bool(v, NULL, &res, &error_abort);
+ g_assert_cmpint(res, ==, true);
+}
+
+static void test_visitor_in_bool_noautocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ bool res = false;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init(data, "\"true\"");
+
+ visit_type_bool(v, NULL, &res, &err);
+ g_assert(err != NULL);
+ error_free(err);
+}
+
static void test_visitor_in_number(TestInputVisitorData *data,
const void *unused)
{
@@ -139,6 +210,32 @@ static void test_visitor_in_number(TestInputVisitorData *data,
g_assert_cmpfloat(res, ==, value);
}
+static void test_visitor_in_number_autocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ double res = 0, value = 3.14;
+ Visitor *v;
+
+ v = visitor_input_test_init_full(data, false, true, "\"3.14\"");
+
+ visit_type_number(v, NULL, &res, &error_abort);
+ g_assert_cmpfloat(res, ==, value);
+}
+
+static void test_visitor_in_number_noautocast(TestInputVisitorData *data,
+ const void *unused)
+{
+ double res = 0;
+ Visitor *v;
+ Error *err = NULL;
+
+ v = visitor_input_test_init(data, "\"3.14\"");
+
+ visit_type_number(v, NULL, &res, &err);
+ g_assert(err != NULL);
+ error_free(err);
+}
+
static void test_visitor_in_string(TestInputVisitorData *data,
const void *unused)
{
@@ -835,10 +932,22 @@ int main(int argc, char **argv)
&in_visitor_data, test_visitor_in_int);
input_visitor_test_add("/visitor/input/int_overflow",
&in_visitor_data, test_visitor_in_int_overflow);
+ input_visitor_test_add("/visitor/input/int_autocast",
+ &in_visitor_data, test_visitor_in_int_autocast);
+ input_visitor_test_add("/visitor/input/int_noautocast",
+ &in_visitor_data, test_visitor_in_int_noautocast);
input_visitor_test_add("/visitor/input/bool",
&in_visitor_data, test_visitor_in_bool);
+ input_visitor_test_add("/visitor/input/bool_autocast",
+ &in_visitor_data, test_visitor_in_bool_autocast);
+ input_visitor_test_add("/visitor/input/bool_noautocast",
+ &in_visitor_data, test_visitor_in_bool_noautocast);
input_visitor_test_add("/visitor/input/number",
&in_visitor_data, test_visitor_in_number);
+ input_visitor_test_add("/visitor/input/number_autocast",
+ &in_visitor_data, test_visitor_in_number_autocast);
+ input_visitor_test_add("/visitor/input/number_noautocast",
+ &in_visitor_data, test_visitor_in_number_noautocast);
input_visitor_test_add("/visitor/input/string",
&in_visitor_data, test_visitor_in_string);
input_visitor_test_add("/visitor/input/enum",
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 7b14b5a..db618d8 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -1038,7 +1038,7 @@ static void qmp_deserialize(void **native_out, void *datap,
obj = qobject_from_json(qstring_get_str(output_json));
QDECREF(output_json);
- d->qiv = qmp_input_visitor_new(obj, true);
+ d->qiv = qmp_input_visitor_new(obj, true, false);
qobject_decref(obj_orig);
qobject_decref(obj);
visit(qmp_input_get_visitor(d->qiv), native_out, errp);
diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index 0d6cd1f..51c6a8e 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -1145,7 +1145,7 @@ void qapi_copy_SocketAddress(SocketAddress **p_dest,
return;
}
- qiv = qmp_input_visitor_new(obj, true);
+ qiv = qmp_input_visitor_new(obj, true, false);
iv = qmp_input_get_visitor(qiv);
visit_type_SocketAddress(iv, NULL, p_dest, &error_abort);
qmp_input_visitor_cleanup(qiv);
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-09 14:43 ` Markus Armbruster
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 04/11] util: add QAuthZ object as an authorization base class Daniel P. Berrange
` (8 subsequent siblings)
11 siblings, 1 reply; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
The current -object command line syntax only allows for
creation of objects with scalar properties, or a list
with a fixed scalar element type. Objects which have
properties that are represented as structs in the QAPI
schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor
is written. It simply iterates over the QemuOpts values
as a flat list. The support for lists is enabled by
allowing the same key to be repeated in the opts string.
It is not practical to extend the OptsVisitor to support
more complex data structures while also maintaining
the existing list handling behaviour that is relied upon
by other areas of QEMU.
Fortunately there is no existing object that implements
the UserCreatable interface that relies on the list
handling behaviour, so it is possible to swap out the
OptsVisitor for a different visitor implementation, so
-object supports non-scalar properties, thus leaving
other users of OptsVisitor unaffected.
The previously added qdict_crumple() method is able to
take a qdict containing a flat set of properties and
turn that into a arbitrarily nested set of dicts and
lists. By combining qemu_opts_to_qdict and qdict_crumple()
together, we can turn the opt string into a data structure
that is practically identical to that passed over QMP
when defining an object. The only difference is that all
the scalar values are represented as strings, rather than
strings, ints and bools. This is sufficient to let us
replace the OptsVisitor with the QMPInputVisitor for
use with -object.
Thus -object can now support non-scalar properties,
for example the QMP object
{
"execute": "object-add",
"arguments": {
"qom-type": "demo",
"id": "demo0",
"parameters": {
"foo": [
{ "bar": "one", "wizz": "1" },
{ "bar": "two", "wizz": "2" }
]
}
}
}
Would be creatable via the CLI now using
$QEMU \
-object demo,id=demo0,\
foo.0.bar=one,foo.0.wizz=1,\
foo.1.bar=two,foo.1.wizz=2
Notice that this syntax is intentionally compatible
with that currently used by block drivers.
This is also wired up to work for the 'object_add' command
in the HMP monitor with the same syntax.
(hmp) object_add demo,id=demo0,\
foo.0.bar=one,foo.0.wizz=1,\
foo.1.bar=two,foo.1.wizz=2
NB indentation should not be used with HMP commands, this
is just for convenient formatting in this commit message.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
hmp.c | 18 +--
include/qom/object_interfaces.h | 10 +-
qmp.c | 2 +-
qom/object_interfaces.c | 49 +++++--
tests/check-qom-proplist.c | 317 +++++++++++++++++++++++++++++++++++++++-
5 files changed, 363 insertions(+), 33 deletions(-)
diff --git a/hmp.c b/hmp.c
index a4b1d3d..1972bef 100644
--- a/hmp.c
+++ b/hmp.c
@@ -25,7 +25,7 @@
#include "qemu/sockets.h"
#include "monitor/monitor.h"
#include "monitor/qdev.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
#include "qapi/qmp/qerror.h"
#include "qapi/string-output-visitor.h"
#include "qapi/util.h"
@@ -1717,20 +1717,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
void hmp_object_add(Monitor *mon, const QDict *qdict)
{
Error *err = NULL;
- QemuOpts *opts;
- OptsVisitor *ov;
+ QmpInputVisitor *qiv;
Object *obj = NULL;
- opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
- if (err) {
- hmp_handle_error(mon, &err);
- return;
- }
-
- ov = opts_visitor_new(opts);
- obj = user_creatable_add(qdict, opts_get_visitor(ov), &err);
- opts_visitor_cleanup(ov);
- qemu_opts_del(opts);
+ qiv = qmp_input_visitor_new((QObject *)qdict, true, true);
+ obj = user_creatable_add(qdict, qmp_input_get_visitor(qiv), &err);
+ qmp_input_visitor_cleanup(qiv);
if (err) {
hmp_handle_error(mon, &err);
diff --git a/include/qom/object_interfaces.h b/include/qom/object_interfaces.h
index 8b17f4d..997563a 100644
--- a/include/qom/object_interfaces.h
+++ b/include/qom/object_interfaces.h
@@ -96,18 +96,24 @@ Object *user_creatable_add(const QDict *qdict,
* user_creatable_add_type:
* @type: the object type name
* @id: the unique ID for the object
+ * @nested: whether to recurse into the visitor for properties
* @qdict: the object properties
* @v: the visitor
* @errp: if an error occurs, a pointer to an area to store the error
*
* Create an instance of the user creatable object @type, placing
* it in the object composition tree with name @id, initializing
- * it with properties from @qdict
+ * it with properties from @qdict.
+ *
+ * If the visitor is already positioned to read the properties
+ * in @qdict, @nested should be false. Conversely, if it is
+ * neccessary to open/close a struct to read the properties in
+ * @qdict, @nested should be true.
*
* Returns: the newly created object or NULL on error
*/
Object *user_creatable_add_type(const char *type, const char *id,
- const QDict *qdict,
+ bool nested, const QDict *qdict,
Visitor *v, Error **errp);
/**
diff --git a/qmp.c b/qmp.c
index dcd0953..dc5fff0 100644
--- a/qmp.c
+++ b/qmp.c
@@ -666,7 +666,7 @@ void qmp_object_add(const char *type, const char *id,
}
qiv = qmp_input_visitor_new(props, true, false);
- obj = user_creatable_add_type(type, id, pdict,
+ obj = user_creatable_add_type(type, id, true, pdict,
qmp_input_get_visitor(qiv), errp);
qmp_input_visitor_cleanup(qiv);
if (obj) {
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index 51e62e2..0883c91 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -3,6 +3,7 @@
#include "qom/object_interfaces.h"
#include "qemu/module.h"
#include "qapi-visit.h"
+#include "qapi/qmp-input-visitor.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi/opts-visitor.h"
@@ -63,12 +64,16 @@ Object *user_creatable_add(const QDict *qdict,
if (local_err) {
goto out_visit;
}
- visit_check_struct(v, &local_err);
+
+ obj = user_creatable_add_type(type, id, false, pdict, v, &local_err);
if (local_err) {
goto out_visit;
}
- obj = user_creatable_add_type(type, id, pdict, v, &local_err);
+ visit_check_struct(v, &local_err);
+ if (local_err) {
+ goto out_visit;
+ }
out_visit:
visit_end_struct(v);
@@ -87,7 +92,7 @@ out:
Object *user_creatable_add_type(const char *type, const char *id,
- const QDict *qdict,
+ bool nested, const QDict *qdict,
Visitor *v, Error **errp)
{
Object *obj;
@@ -114,9 +119,11 @@ Object *user_creatable_add_type(const char *type, const char *id,
assert(qdict);
obj = object_new(type);
- visit_start_struct(v, NULL, NULL, 0, &local_err);
- if (local_err) {
- goto out;
+ if (nested) {
+ visit_start_struct(v, NULL, NULL, 0, &local_err);
+ if (local_err) {
+ goto out;
+ }
}
for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
object_property_set(obj, v, e->key, &local_err);
@@ -124,12 +131,14 @@ Object *user_creatable_add_type(const char *type, const char *id,
break;
}
}
- if (!local_err) {
- visit_check_struct(v, &local_err);
- }
- visit_end_struct(v);
- if (local_err) {
- goto out;
+ if (nested) {
+ if (!local_err) {
+ visit_check_struct(v, &local_err);
+ }
+ visit_end_struct(v);
+ if (local_err) {
+ goto out;
+ }
}
object_property_add_child(object_get_objects_root(),
@@ -156,15 +165,23 @@ out:
Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
{
- OptsVisitor *ov;
+ QmpInputVisitor *qiv;
QDict *pdict;
+ QObject *pobj;
Object *obj = NULL;
- ov = opts_visitor_new(opts);
pdict = qemu_opts_to_qdict(opts, NULL);
- obj = user_creatable_add(pdict, opts_get_visitor(ov), errp);
- opts_visitor_cleanup(ov);
+ pobj = qdict_crumple(pdict, true, errp);
+ if (!pobj) {
+ goto cleanup;
+ }
+ qiv = qmp_input_visitor_new(pobj, true, true);
+
+ obj = user_creatable_add((QDict *)pobj, qmp_input_get_visitor(qiv), errp);
+ qmp_input_visitor_cleanup(qiv);
+ qobject_decref(pobj);
+ cleanup:
QDECREF(pdict);
return obj;
}
diff --git a/tests/check-qom-proplist.c b/tests/check-qom-proplist.c
index ffffd87..de37d3a 100644
--- a/tests/check-qom-proplist.c
+++ b/tests/check-qom-proplist.c
@@ -24,6 +24,14 @@
#include "qapi/error.h"
#include "qom/object.h"
#include "qemu/module.h"
+#include "qapi/visitor.h"
+#include "qom/object_interfaces.h"
+#include "qemu/option.h"
+#include "qemu/config-file.h"
+#include "qapi/qmp/qjson.h"
+#include "qapi/qmp-input-visitor.h"
+#include "qapi-visit.h"
+#include "qapi/dealloc-visitor.h"
#define TYPE_DUMMY "qemu-dummy"
@@ -31,6 +39,11 @@
typedef struct DummyObject DummyObject;
typedef struct DummyObjectClass DummyObjectClass;
+typedef struct DummyPerson DummyPerson;
+typedef struct DummyAddr DummyAddr;
+typedef struct DummyAddrList DummyAddrList;
+typedef struct DummySizeList DummySizeList;
+
#define DUMMY_OBJECT(obj) \
OBJECT_CHECK(DummyObject, (obj), TYPE_DUMMY)
@@ -51,12 +64,35 @@ static const char *const dummy_animal_map[DUMMY_LAST + 1] = {
[DUMMY_LAST] = NULL,
};
+
+struct DummyAddr {
+ char *ip;
+ int64_t prefix;
+ bool ipv6only;
+};
+
+struct DummyAddrList {
+ DummyAddrList *next;
+ struct DummyAddr *value;
+};
+
+struct DummyPerson {
+ char *name;
+ int64_t age;
+};
+
struct DummyObject {
Object parent_obj;
bool bv;
DummyAnimal av;
char *sv;
+
+ intList *sizes;
+
+ DummyPerson *person;
+
+ DummyAddrList *addrs;
};
struct DummyObjectClass {
@@ -118,6 +154,157 @@ static char *dummy_get_sv(Object *obj,
return g_strdup(dobj->sv);
}
+static void visit_type_DummyPerson_fields(Visitor *v, DummyPerson **obj,
+ Error **errp)
+{
+ Error *err = NULL;
+
+ visit_type_str(v, "name", &(*obj)->name, &err);
+ if (err) {
+ goto out;
+ }
+ visit_type_int(v, "age", &(*obj)->age, &err);
+ if (err) {
+ goto out;
+ }
+
+out:
+ error_propagate(errp, err);
+}
+
+static void visit_type_DummyPerson(Visitor *v, const char *name,
+ DummyPerson **obj, Error **errp)
+{
+ Error *err = NULL;
+
+ visit_start_struct(v, name, (void **)obj, sizeof(DummyPerson), &err);
+ if (err) {
+ goto out;
+ }
+ if (!*obj) {
+ goto out_obj;
+ }
+ visit_type_DummyPerson_fields(v, obj, &err);
+ error_propagate(errp, err);
+ err = NULL;
+out_obj:
+ visit_end_struct(v);
+out:
+ error_propagate(errp, err);
+}
+
+static void visit_type_DummyAddr_members(Visitor *v, DummyAddr **obj,
+ Error **errp)
+{
+ Error *err = NULL;
+
+ visit_type_str(v, "ip", &(*obj)->ip, &err);
+ if (err) {
+ goto out;
+ }
+ visit_type_int(v, "prefix", &(*obj)->prefix, &err);
+ if (err) {
+ goto out;
+ }
+ visit_type_bool(v, "ipv6only", &(*obj)->ipv6only, &err);
+ if (err) {
+ goto out;
+ }
+
+out:
+ error_propagate(errp, err);
+}
+
+static void visit_type_DummyAddr(Visitor *v, const char *name,
+ DummyAddr **obj, Error **errp)
+{
+ Error *err = NULL;
+
+ visit_start_struct(v, name, (void **)obj, sizeof(DummyAddr), &err);
+ if (err) {
+ goto out;
+ }
+ if (!*obj) {
+ goto out_obj;
+ }
+ visit_type_DummyAddr_members(v, obj, &err);
+ error_propagate(errp, err);
+ err = NULL;
+out_obj:
+ visit_end_struct(v);
+out:
+ error_propagate(errp, err);
+}
+
+static void qapi_free_DummyAddrList(DummyAddrList *obj);
+
+static void visit_type_DummyAddrList(Visitor *v, const char *name,
+ DummyAddrList **obj, Error **errp)
+{
+ Error *err = NULL;
+ DummyAddrList *tail;
+ size_t size = sizeof(**obj);
+
+ visit_start_list(v, name, (GenericList **)obj, size, &err);
+ if (err) {
+ goto out;
+ }
+
+ for (tail = *obj; tail;
+ tail = (DummyAddrList *)visit_next_list(v, (GenericList *)tail, size)) {
+ visit_type_DummyAddr(v, NULL, &tail->value, &err);
+ if (err) {
+ break;
+ }
+ }
+
+ visit_end_list(v);
+ if (err && visit_is_input(v)) {
+ qapi_free_DummyAddrList(*obj);
+ *obj = NULL;
+ }
+out:
+ error_propagate(errp, err);
+}
+
+static void qapi_free_DummyAddrList(DummyAddrList *obj)
+{
+ QapiDeallocVisitor *qdv;
+ Visitor *v;
+
+ if (!obj) {
+ return;
+ }
+
+ qdv = qapi_dealloc_visitor_new();
+ v = qapi_dealloc_get_visitor(qdv);
+ visit_type_DummyAddrList(v, NULL, &obj, NULL);
+ qapi_dealloc_visitor_cleanup(qdv);
+}
+
+static void dummy_set_sizes(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ DummyObject *dobj = DUMMY_OBJECT(obj);
+
+ visit_type_intList(v, name, &dobj->sizes, errp);
+}
+
+static void dummy_set_person(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ DummyObject *dobj = DUMMY_OBJECT(obj);
+
+ visit_type_DummyPerson(v, name, &dobj->person, errp);
+}
+
+static void dummy_set_addrs(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ DummyObject *dobj = DUMMY_OBJECT(obj);
+
+ visit_type_DummyAddrList(v, name, &dobj->addrs, errp);
+}
static void dummy_init(Object *obj)
{
@@ -127,9 +314,16 @@ static void dummy_init(Object *obj)
NULL);
}
+static void
+dummy_complete(UserCreatable *uc, Error **errp)
+{
+}
static void dummy_class_init(ObjectClass *cls, void *data)
{
+ UserCreatableClass *ucc = USER_CREATABLE_CLASS(cls);
+ ucc->complete = dummy_complete;
+
object_class_property_add_bool(cls, "bv",
dummy_get_bv,
dummy_set_bv,
@@ -144,12 +338,36 @@ static void dummy_class_init(ObjectClass *cls, void *data)
dummy_get_av,
dummy_set_av,
NULL);
+ object_class_property_add(cls, "sizes",
+ "int[]",
+ NULL,
+ dummy_set_sizes,
+ NULL, NULL, NULL);
+ object_class_property_add(cls, "person",
+ "DummyPerson",
+ NULL,
+ dummy_set_person,
+ NULL, NULL, NULL);
+ object_class_property_add(cls, "addrs",
+ "DummyAddrList",
+ NULL,
+ dummy_set_addrs,
+ NULL, NULL, NULL);
}
static void dummy_finalize(Object *obj)
{
DummyObject *dobj = DUMMY_OBJECT(obj);
+ QapiDeallocVisitor *qdv;
+ Visitor *v;
+
+ qdv = qapi_dealloc_visitor_new();
+ v = qapi_dealloc_get_visitor(qdv);
+ visit_type_intList(v, NULL, &dobj->sizes, NULL);
+ visit_type_DummyAddrList(v, NULL, &dobj->addrs, NULL);
+ visit_type_DummyPerson(v, NULL, &dobj->person, NULL);
+ qapi_dealloc_visitor_cleanup(qdv);
g_free(dobj->sv);
}
@@ -163,6 +381,10 @@ static const TypeInfo dummy_info = {
.instance_finalize = dummy_finalize,
.class_size = sizeof(DummyObjectClass),
.class_init = dummy_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_USER_CREATABLE },
+ { }
+ }
};
@@ -458,7 +680,9 @@ static void test_dummy_iterator(void)
ObjectProperty *prop;
ObjectPropertyIterator iter;
- bool seenbv = false, seensv = false, seenav = false, seentype;
+ bool seenbv = false, seensv = false, seenav = false,
+ seentype = false, seenaddrs = false, seenperson = false,
+ seensizes = false;
object_property_iter_init(&iter, OBJECT(dobj));
while ((prop = object_property_iter_next(&iter))) {
@@ -471,6 +695,12 @@ static void test_dummy_iterator(void)
} else if (g_str_equal(prop->name, "type")) {
/* This prop comes from the base Object class */
seentype = true;
+ } else if (g_str_equal(prop->name, "addrs")) {
+ seenaddrs = true;
+ } else if (g_str_equal(prop->name, "person")) {
+ seenperson = true;
+ } else if (g_str_equal(prop->name, "sizes")) {
+ seensizes = true;
} else {
g_printerr("Found prop '%s'\n", prop->name);
g_assert_not_reached();
@@ -480,6 +710,9 @@ static void test_dummy_iterator(void)
g_assert(seenav);
g_assert(seensv);
g_assert(seentype);
+ g_assert(seenaddrs);
+ g_assert(seenperson);
+ g_assert(seensizes);
object_unparent(OBJECT(dobj));
}
@@ -498,11 +731,91 @@ static void test_dummy_delchild(void)
object_unparent(OBJECT(dev));
}
+
+static QemuOptsList dummy_opts = {
+ .name = "object",
+ .implied_opt_name = "qom-type",
+ .head = QTAILQ_HEAD_INITIALIZER(dummy_opts.head),
+ .desc = {
+ { }
+ },
+};
+
+static void test_dummy_create_complex(DummyObject *dummy)
+{
+ g_assert(dummy->person != NULL);
+ g_assert_cmpstr(dummy->person->name, ==, "fred");
+ g_assert_cmpint(dummy->person->age, ==, 52);
+
+ g_assert(dummy->sizes != NULL);
+ g_assert_cmpint(dummy->sizes->value, ==, 12);
+ g_assert_cmpint(dummy->sizes->next->value, ==, 65);
+ g_assert_cmpint(dummy->sizes->next->next->value, ==, 8139);
+
+ g_assert(dummy->addrs != NULL);
+ g_assert_cmpstr(dummy->addrs->value->ip, ==, "127.0.0.1");
+ g_assert_cmpint(dummy->addrs->value->prefix, ==, 24);
+ g_assert(dummy->addrs->value->ipv6only);
+ g_assert_cmpstr(dummy->addrs->next->value->ip, ==, "0.0.0.0");
+ g_assert_cmpint(dummy->addrs->next->value->prefix, ==, 16);
+ g_assert(!dummy->addrs->next->value->ipv6only);
+}
+
+
+static void test_dummy_createopts(void)
+{
+ const char *optstr = "qemu-dummy,id=dummy0,bv=yes,av=alligator,sv=hiss,"
+ "person.name=fred,person.age=52,sizes.0=12,sizes.1=65,sizes.2=8139,"
+ "addrs.0.ip=127.0.0.1,addrs.0.prefix=24,addrs.0.ipv6only=yes,"
+ "addrs.1.ip=0.0.0.0,addrs.1.prefix=16,addrs.1.ipv6only=no";
+ QemuOpts *opts;
+ DummyObject *dummy;
+
+ opts = qemu_opts_parse_noisily(&dummy_opts,
+ optstr, true);
+ g_assert(opts != NULL);
+
+ dummy = DUMMY_OBJECT(user_creatable_add_opts(opts, &error_abort));
+
+ test_dummy_create_complex(dummy);
+
+ object_unparent(OBJECT(dummy));
+ object_unref(OBJECT(dummy));
+}
+
+
+static void test_dummy_createqmp(void)
+{
+ const char *jsonstr =
+ "{ 'qom-type': 'qemu-dummy', 'id': 'dummy0', "
+ " 'bv': true, 'av': 'alligator', 'sv': 'hiss', "
+ " 'person': { 'name': 'fred', 'age': 52 }, "
+ " 'sizes': [12, 65, 8139], "
+ " 'addrs': [ { 'ip': '127.0.0.1', 'prefix': 24, 'ipv6only': true }, "
+ " { 'ip': '0.0.0.0', 'prefix': 16, 'ipv6only': false } ] }";
+ QObject *obj = qobject_from_json(jsonstr);
+ QmpInputVisitor *qiv = qmp_input_visitor_new(obj, true, false);
+ DummyObject *dummy;
+ g_assert(obj);
+ dummy = DUMMY_OBJECT(user_creatable_add(qobject_to_qdict(obj),
+ qmp_input_get_visitor(qiv),
+ &error_abort));
+
+ test_dummy_create_complex(dummy);
+ qmp_input_visitor_cleanup(qiv);
+ object_unparent(OBJECT(dummy));
+ object_unref(OBJECT(dummy));
+ qobject_decref(obj);
+}
+
int main(int argc, char **argv)
{
g_test_init(&argc, &argv, NULL);
module_call_init(MODULE_INIT_QOM);
+
+ qemu_add_opts(&dummy_opts);
+
type_register_static(&dummy_info);
type_register_static(&dummy_dev_info);
type_register_static(&dummy_bus_info);
@@ -514,6 +827,8 @@ int main(int argc, char **argv)
g_test_add_func("/qom/proplist/getenum", test_dummy_getenum);
g_test_add_func("/qom/proplist/iterator", test_dummy_iterator);
g_test_add_func("/qom/proplist/delchild", test_dummy_delchild);
+ g_test_add_func("/qom/proplist/createopts", test_dummy_createopts);
+ g_test_add_func("/qom/proplist/createqmp", test_dummy_createqmp);
return g_test_run();
}
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 04/11] util: add QAuthZ object as an authorization base class
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (2 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 05/11] util: add QAuthZSimple object type for a simple access control list Daniel P. Berrange
` (7 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
The current qemu_acl module provides a simple access control
list facility inside QEMU, which is used via a set of monitor
commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.
Note there is no ability to create ACLs - the network services
(eg VNC server) were expected to create ACLs that they want to
check.
There is also no way to define ACLs on the command line, nor
potentially integrate with external authorization systems like
polkit, pam, ldap lookup, etc.
The QAuthZ object defines a minimal abstract QOM class that can
be subclassed for creating different authorization providers.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
MAINTAINERS | 7 +++++
Makefile | 1 +
Makefile.objs | 2 ++
Makefile.target | 2 ++
include/qemu/authz.h | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++
util/Makefile.objs | 2 ++
util/authz.c | 46 +++++++++++++++++++++++++++
7 files changed, 149 insertions(+)
create mode 100644 include/qemu/authz.h
create mode 100644 util/authz.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 3c949d5..6ae3676 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1306,6 +1306,13 @@ F: include/qemu/throttle.h
F: util/throttle.c
L: qemu-block@nongnu.org
+Authorization
+M: Daniel P. Berrange <berrange@redhat.com>
+S: Maintained
+F: util/authz*
+F: include/qemu/authz*
+F: tests/test-authz-*
+
Usermode Emulation
------------------
Overall
diff --git a/Makefile b/Makefile
index 3a3c5dc..de83cfe 100644
--- a/Makefile
+++ b/Makefile
@@ -147,6 +147,7 @@ endif
dummy := $(call unnest-vars,, \
stub-obj-y \
util-obj-y \
+ util-qom-obj-y \
qga-obj-y \
ivshmem-client-obj-y \
ivshmem-server-obj-y \
diff --git a/Makefile.objs b/Makefile.objs
index da49b71..24db7f7 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -4,6 +4,8 @@ stub-obj-y = stubs/ crypto/
util-obj-y = util/ qobject/ qapi/
util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
+util-qom-obj-y += util/
+
#######################################################################
# block-obj-y is code used by both qemu system emulation and qemu-img
diff --git a/Makefile.target b/Makefile.target
index 5b80dd7..329796a 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -176,6 +176,7 @@ include $(SRC_PATH)/Makefile.objs
dummy := $(call unnest-vars,,target-obj-y)
target-obj-y-save := $(target-obj-y)
dummy := $(call unnest-vars,.., \
+ util-qom-obj-y \
block-obj-y \
block-obj-m \
crypto-obj-y \
@@ -188,6 +189,7 @@ target-obj-y := $(target-obj-y-save)
all-obj-y += $(common-obj-y)
all-obj-y += $(target-obj-y)
all-obj-y += $(qom-obj-y)
+all-obj-y += $(util-qom-obj-y)
all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
diff --git a/include/qemu/authz.h b/include/qemu/authz.h
new file mode 100644
index 0000000..6a73063
--- /dev/null
+++ b/include/qemu/authz.h
@@ -0,0 +1,89 @@
+/*
+ * QEMU authorization framework
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QAUTHZ_H__
+#define QAUTHZ_H__
+
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+
+
+#define TYPE_QAUTHZ "authz"
+
+#define QAUTHZ_CLASS(klass) \
+ OBJECT_CLASS_CHECK(QAuthZClass, (klass), \
+ TYPE_QAUTHZ)
+#define QAUTHZ_GET_CLASS(obj) \
+ OBJECT_GET_CLASS(QAuthZClass, (obj), \
+ TYPE_QAUTHZ)
+#define QAUTHZ(obj) \
+ INTERFACE_CHECK(QAuthZ, (obj), \
+ TYPE_QAUTHZ)
+
+typedef struct QAuthZ QAuthZ;
+typedef struct QAuthZClass QAuthZClass;
+
+/**
+ * QAuthZ:
+ *
+ * The QAuthZ class defines an API contract to be used
+ * for providing an authorization driver for services
+ * with user identities.
+ */
+
+struct QAuthZ {
+ Object parent_obj;
+};
+
+
+struct QAuthZClass {
+ ObjectClass parent_class;
+
+ bool (*is_allowed)(QAuthZ *authz,
+ const char *identity,
+ Error **errp);
+};
+
+
+/**
+ * qauthz_is_allowed:
+ * @authz: the authorization object
+ * @identity: the user identity to authorize
+ * @errp: pointer to a NULL initialized error object
+ *
+ * Check if a user @identity is authorized. If an error
+ * occurrs this method will return false to indicate
+ * denial, as well as setting @errp to contain the details.
+ * Callers are recommended to treat the denial and error
+ * scenarios identically. Specifically the error info in
+ * @errp should never be fed back to the user being
+ * authorized, it is merely for benefit of administrator
+ * debugging.
+ *
+ * Returns: true if @identity is authorized, false if denied or if
+ * an error occurred.
+ */
+bool qauthz_is_allowed(QAuthZ *authz,
+ const char *identity,
+ Error **errp);
+
+#endif /* QAUTHZ_H__ */
+
diff --git a/util/Makefile.objs b/util/Makefile.objs
index a8a777e..b29fb45 100644
--- a/util/Makefile.objs
+++ b/util/Makefile.objs
@@ -32,3 +32,5 @@ util-obj-y += buffer.o
util-obj-y += timed-average.o
util-obj-y += base64.o
util-obj-y += log.o
+
+util-qom-obj-y += authz.o
diff --git a/util/authz.c b/util/authz.c
new file mode 100644
index 0000000..fd9f84e
--- /dev/null
+++ b/util/authz.c
@@ -0,0 +1,46 @@
+/*
+ * QEMU authorization framework
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/authz.h"
+
+bool qauthz_is_allowed(QAuthZ *authz,
+ const char *identity,
+ Error **errp)
+{
+ QAuthZClass *cls = QAUTHZ_GET_CLASS(authz);
+
+ return cls->is_allowed(authz, identity, errp);
+}
+
+static const TypeInfo authz_info = {
+ .parent = TYPE_OBJECT,
+ .name = TYPE_QAUTHZ,
+ .instance_size = sizeof(QAuthZ),
+ .class_size = sizeof(QAuthZClass),
+};
+
+static void qauthz_register_types(void)
+{
+ type_register_static(&authz_info);
+}
+
+type_init(qauthz_register_types)
+
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 05/11] util: add QAuthZSimple object type for a simple access control list
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (3 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 04/11] util: add QAuthZ object as an authorization base class Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 06/11] acl: delete existing ACL implementation Daniel P. Berrange
` (6 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
Add a QAuthZSimple object type that implements the QAuthZ
interface. This simple built-in implementation maintains
a trivial access control list with a sequence of match
rules and a final default policy. This replicates the
functionality currently provided by the qemu_acl module.
To create an instance of this object via the QMP monitor,
the syntax used would be
{
"execute": "object-add",
"arguments": {
"qom-type": "authz-simple",
"id": "auth0",
"parameters": {
"rules": [
{ "match": "fred", "policy": "allow", "format": "exact" },
{ "match": "bob", "policy": "allow", "format": "exact" },
{ "match": "danb", "policy": "deny", "format": "glob" },
{ "match": "dan*", "policy": "allow", "format": "exact" },
],
"policy": "deny"
}
}
}
Or via the -object command line
$QEMU \
-object authz-simple,id=acl0,policy=deny,\
rules.0.match=fred,rules.0.policy=allow,rules.0.format=exact,\
rules.1.match=bob,rules.1.policy=allow,rules.1.format=exact,\
rules.2.match=danb,rules.2.policy=deny,rules.2.format=glob,\
rules.3.match=dan\*,rules.3.policy=allow,rules.3.format=exact
This sets up an authorization rule that allows 'fred',
'bob' and anyone whose name starts with 'dan', except
for 'danb'. Everyone unmatched is denied.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
Makefile | 2 +-
include/qemu/authz-simple.h | 115 ++++++++++++++++
qapi-schema.json | 6 +-
qapi/util.json | 47 +++++++
tests/.gitignore | 1 +
tests/Makefile | 3 +
tests/test-authz-simple.c | 183 ++++++++++++++++++++++++++
util/Makefile.objs | 1 +
util/authz-simple.c | 314 ++++++++++++++++++++++++++++++++++++++++++++
9 files changed, 670 insertions(+), 2 deletions(-)
create mode 100644 include/qemu/authz-simple.h
create mode 100644 qapi/util.json
create mode 100644 tests/test-authz-simple.c
create mode 100644 util/authz-simple.c
diff --git a/Makefile b/Makefile
index de83cfe..ae3ffac 100644
--- a/Makefile
+++ b/Makefile
@@ -271,7 +271,7 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
$(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
$(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
$(SRC_PATH)/qapi/crypto.json $(SRC_PATH)/qapi/rocker.json \
- $(SRC_PATH)/qapi/trace.json
+ $(SRC_PATH)/qapi/trace.json $(SRC_PATH)/qapi/util.json
qapi-types.c qapi-types.h :\
$(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
diff --git a/include/qemu/authz-simple.h b/include/qemu/authz-simple.h
new file mode 100644
index 0000000..30e78bd
--- /dev/null
+++ b/include/qemu/authz-simple.h
@@ -0,0 +1,115 @@
+/*
+ * QEMU simple authorization driver
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QAUTHZ_SIMPLE_H__
+#define QAUTHZ_SIMPLE_H__
+
+#include "qemu/authz.h"
+
+
+#define TYPE_QAUTHZ_SIMPLE "authz-simple"
+
+#define QAUTHZ_SIMPLE_CLASS(klass) \
+ OBJECT_CLASS_CHECK(QAuthZSimpleClass, (klass), \
+ TYPE_QAUTHZ_SIMPLE)
+#define QAUTHZ_SIMPLE_GET_CLASS(obj) \
+ OBJECT_GET_CLASS(QAuthZSimpleClass, (obj), \
+ TYPE_QAUTHZ_SIMPLE)
+#define QAUTHZ_SIMPLE(obj) \
+ INTERFACE_CHECK(QAuthZSimple, (obj), \
+ TYPE_QAUTHZ_SIMPLE)
+
+typedef struct QAuthZSimple QAuthZSimple;
+typedef struct QAuthZSimpleClass QAuthZSimpleClass;
+
+
+/**
+ * QAuthZSimple:
+ *
+ * This authorization driver provides a simple mechanism
+ * for granting access by matching user names against a
+ * list of globs. Each match rule has an associated policy
+ * and a catch all policy applies if no rule matches
+ *
+ * To create an instance of this class via QMP:
+ *
+ * {
+ * "execute": "object-add",
+ * "arguments": {
+ * "qom-type": "authz-simple",
+ * "id": "auth0",
+ * "parameters": {
+ * "rules": [
+ * { "match": "fred", "policy": "allow", "format": "exact" },
+ * { "match": "bob", "policy": "allow", "format": "exact" },
+ * { "match": "danb", "policy": "deny", "format": "exact" },
+ * { "match": "dan*", "policy": "allow", "format": "glob" }
+ * ],
+ * "policy": "deny"
+ * }
+ * }
+ * }
+ *
+ * Or via the CLI:
+ *
+ * $QEMU \
+ * -object authz-simple,id=acl0,policy=deny, \
+ * match.0.name=fred,match.0.policy=allow,format=exact, \
+ * match.1.name=bob,match.1.policy=allow,format=exact, \
+ * match.2.name=danb,match.2.policy=deny,format=exact, \
+ * match.3.name=dan\*,match.3.policy=allow,format=exact
+ *
+ */
+struct QAuthZSimple {
+ QAuthZ parent_obj;
+
+ QAuthZSimplePolicy policy;
+ QAuthZSimpleRuleList *rules;
+};
+
+
+struct QAuthZSimpleClass {
+ QAuthZClass parent_class;
+};
+
+
+QAuthZSimple *qauthz_simple_new(const char *id,
+ QAuthZSimplePolicy policy,
+ Error **errp);
+
+ssize_t qauthz_simple_append_rule(QAuthZSimple *auth,
+ const char *match,
+ QAuthZSimplePolicy policy,
+ QAuthZSimpleFormat format,
+ Error **errp);
+
+ssize_t qauthz_simple_insert_rule(QAuthZSimple *auth,
+ const char *match,
+ QAuthZSimplePolicy policy,
+ QAuthZSimpleFormat format,
+ size_t index,
+ Error **errp);
+
+ssize_t qauthz_simple_delete_rule(QAuthZSimple *auth,
+ const char *match);
+
+
+#endif /* QAUTHZ_SIMPLE_H__ */
+
diff --git a/qapi-schema.json b/qapi-schema.json
index 8483bdf..337a6ce 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -5,6 +5,9 @@
# QAPI common definitions
{ 'include': 'qapi/common.json' }
+# QAPI util definitions
+{ 'include': 'qapi/util.json' }
+
# QAPI crypto definitions
{ 'include': 'qapi/crypto.json' }
@@ -3729,7 +3732,8 @@
# Since 2.5
##
{ 'struct': 'DummyForceArrays',
- 'data': { 'unused': ['X86CPUFeatureWordInfo'] } }
+ 'data': { 'unused1': ['X86CPUFeatureWordInfo'],
+ 'unused2': ['QAuthZSimpleRule'] } }
##
diff --git a/qapi/util.json b/qapi/util.json
new file mode 100644
index 0000000..fec43a8
--- /dev/null
+++ b/qapi/util.json
@@ -0,0 +1,47 @@
+# -*- Mode: Python -*-
+#
+# QAPI util definitions
+
+##
+# QAuthZSimplePolicy:
+#
+# The authorization policy result
+#
+# @deny: deny access
+# @allow: allow access
+#
+# Since: 2.6
+##
+{ 'enum': 'QAuthZSimplePolicy',
+ 'prefix': 'QAUTHZ_SIMPLE_POLICY',
+ 'data': ['deny', 'allow']}
+
+##
+# QAuthZSimpleFormat:
+#
+# The authorization policy result
+#
+# @exact: an exact string match
+# @glob: string with ? and * shell wildcard support
+#
+# Since: 2.6
+##
+{ 'enum': 'QAuthZSimpleFormat',
+ 'prefix': 'QAUTHZ_SIMPLE_FORMAT',
+ 'data': ['exact', 'glob']}
+
+##
+# QAuthZSimpleRule:
+#
+# A single authorization rule.
+#
+# @match: a glob to match against a user identity
+# @policy: the result to return if @match evaluates to true
+# @format: (optional) the format ofthe @match rule (default 'exact')
+#
+# Since: 2.6
+##
+{ 'struct': 'QAuthZSimpleRule',
+ 'data': {'match': 'str',
+ 'policy': 'QAuthZSimplePolicy',
+ '*format': 'QAuthZSimpleFormat'}}
diff --git a/tests/.gitignore b/tests/.gitignore
index a06a8ba..2e204a0 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -9,6 +9,7 @@ check-qom-interface
check-qom-proplist
rcutorture
test-aio
+test-authz-simple
test-base64
test-bitops
test-blockjob-txn
diff --git a/tests/Makefile b/tests/Makefile
index 4bc041c..c4c5fc5 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -103,6 +103,7 @@ check-unit-y += tests/test-crypto-xts$(EXESUF)
check-unit-y += tests/test-crypto-block$(EXESUF)
gcov-files-test-logging-y = tests/test-logging.c
check-unit-y += tests/test-logging$(EXESUF)
+check-unit-y += tests/test-authz-simple$(EXESUF)
check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
@@ -446,6 +447,8 @@ tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
$(test-util-obj-y)
tests/test-base64$(EXESUF): tests/test-base64.o \
libqemuutil.a libqemustub.a
+tests/test-authz-simple$(EXESUF): tests/test-authz-simple.o \
+ libqemuutil.a libqemustub.a $(util-qom-obj-y) $(qom-obj-y)
tests/test-logging$(EXESUF): tests/test-logging.o $(test-util-obj-y)
diff --git a/tests/test-authz-simple.c b/tests/test-authz-simple.c
new file mode 100644
index 0000000..d72cf97
--- /dev/null
+++ b/tests/test-authz-simple.c
@@ -0,0 +1,183 @@
+/*
+ * QEMU simple authorization object
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include <glib.h>
+
+#include "qemu/authz-simple.h"
+
+static void test_authz_default_deny(void)
+{
+ QAuthZSimple *auth = qauthz_simple_new("auth0",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort);
+
+ g_assert(!qauthz_is_allowed(QAUTHZ(auth), "fred", &error_abort));
+
+ object_unparent(OBJECT(auth));
+}
+
+static void test_authz_default_allow(void)
+{
+ QAuthZSimple *auth = qauthz_simple_new("auth0",
+ QAUTHZ_SIMPLE_POLICY_ALLOW,
+ &error_abort);
+
+ g_assert(qauthz_is_allowed(QAUTHZ(auth), "fred", &error_abort));
+
+ object_unparent(OBJECT(auth));
+}
+
+static void test_authz_explicit_deny(void)
+{
+ QAuthZSimple *auth = qauthz_simple_new("auth0",
+ QAUTHZ_SIMPLE_POLICY_ALLOW,
+ &error_abort);
+
+ qauthz_simple_append_rule(auth, "fred", QAUTHZ_SIMPLE_POLICY_DENY,
+ QAUTHZ_SIMPLE_FORMAT_EXACT, &error_abort);
+
+ g_assert(!qauthz_is_allowed(QAUTHZ(auth), "fred", &error_abort));
+
+ object_unparent(OBJECT(auth));
+}
+
+static void test_authz_explicit_allow(void)
+{
+ QAuthZSimple *auth = qauthz_simple_new("auth0",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort);
+
+ qauthz_simple_append_rule(auth, "fred", QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_EXACT, &error_abort);
+
+ g_assert(qauthz_is_allowed(QAUTHZ(auth), "fred", &error_abort));
+
+ object_unparent(OBJECT(auth));
+}
+
+
+static void test_authz_complex(void)
+{
+#ifndef CONFIG_FNMATCH
+ Error *local_err = NULL;
+#endif
+ QAuthZSimple *auth = qauthz_simple_new("auth0",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort);
+
+ qauthz_simple_append_rule(auth, "fred", QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_EXACT, &error_abort);
+ qauthz_simple_append_rule(auth, "bob", QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_EXACT, &error_abort);
+ qauthz_simple_append_rule(auth, "dan", QAUTHZ_SIMPLE_POLICY_DENY,
+ QAUTHZ_SIMPLE_FORMAT_EXACT, &error_abort);
+#ifdef CONFIG_FNMATCH
+ qauthz_simple_append_rule(auth, "dan*", QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_GLOB, &error_abort);
+
+ g_assert(qauthz_is_allowed(QAUTHZ(auth), "fred", &error_abort));
+ g_assert(qauthz_is_allowed(QAUTHZ(auth), "bob", &error_abort));
+ g_assert(!qauthz_is_allowed(QAUTHZ(auth), "dan", &error_abort));
+ g_assert(qauthz_is_allowed(QAUTHZ(auth), "danb", &error_abort));
+#else
+ g_assert(qauthz_simple_append_rule(auth, "dan*", QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_GLOB, &local_err) < 0);
+ g_assert(local_err != NULL);
+ error_free(local_err);
+#endif
+
+ object_unparent(OBJECT(auth));
+}
+
+static void test_authz_add_remove(void)
+{
+ QAuthZSimple *auth = qauthz_simple_new("auth0",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort);
+
+ g_assert_cmpint(qauthz_simple_append_rule(auth, "fred",
+ QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_EXACT,
+ &error_abort),
+ ==, 0);
+ g_assert_cmpint(qauthz_simple_append_rule(auth, "bob",
+ QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_EXACT,
+ &error_abort),
+ ==, 1);
+ g_assert_cmpint(qauthz_simple_append_rule(auth, "dan",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ QAUTHZ_SIMPLE_FORMAT_EXACT,
+ &error_abort),
+ ==, 2);
+#ifdef CONFIG_FNMATCH
+ g_assert_cmpint(qauthz_simple_append_rule(auth, "dan*",
+ QAUTHZ_SIMPLE_POLICY_ALLOW,
+ QAUTHZ_SIMPLE_FORMAT_GLOB,
+ &error_abort),
+ ==, 3);
+#endif
+
+ g_assert(!qauthz_is_allowed(QAUTHZ(auth), "dan", &error_abort));
+
+ g_assert_cmpint(qauthz_simple_delete_rule(auth, "dan"),
+ ==, 2);
+
+ g_assert(qauthz_is_allowed(QAUTHZ(auth), "dan", &error_abort));
+
+ g_assert_cmpint(qauthz_simple_append_rule(auth, "dan",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ QAUTHZ_SIMPLE_FORMAT_EXACT,
+ &error_abort),
+ ==, 3);
+
+ g_assert(qauthz_is_allowed(QAUTHZ(auth), "dan", &error_abort));
+
+ g_assert_cmpint(qauthz_simple_delete_rule(auth, "dan"),
+ ==, 3);
+
+ g_assert_cmpint(qauthz_simple_insert_rule(auth, "dan",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ QAUTHZ_SIMPLE_FORMAT_EXACT,
+ 2,
+ &error_abort),
+ ==, 2);
+
+ g_assert(!qauthz_is_allowed(QAUTHZ(auth), "dan", &error_abort));
+
+ object_unparent(OBJECT(auth));
+}
+
+int main(int argc, char **argv)
+{
+ g_test_init(&argc, &argv, NULL);
+
+ module_call_init(MODULE_INIT_QOM);
+
+ g_test_add_func("/auth/simple/default/deny", test_authz_default_deny);
+ g_test_add_func("/auth/simple/default/allow", test_authz_default_allow);
+ g_test_add_func("/auth/simple/explicit/deny", test_authz_explicit_deny);
+ g_test_add_func("/auth/simple/explicit/allow", test_authz_explicit_allow);
+ g_test_add_func("/auth/simple/complex", test_authz_complex);
+ g_test_add_func("/auth/simple/add-remove", test_authz_add_remove);
+
+ return g_test_run();
+}
diff --git a/util/Makefile.objs b/util/Makefile.objs
index b29fb45..4870905 100644
--- a/util/Makefile.objs
+++ b/util/Makefile.objs
@@ -34,3 +34,4 @@ util-obj-y += base64.o
util-obj-y += log.o
util-qom-obj-y += authz.o
+util-qom-obj-y += authz-simple.o
diff --git a/util/authz-simple.c b/util/authz-simple.c
new file mode 100644
index 0000000..a26177f
--- /dev/null
+++ b/util/authz-simple.c
@@ -0,0 +1,314 @@
+/*
+ * QEMU simple authorization driver
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/authz-simple.h"
+#include "qom/object_interfaces.h"
+#include "qapi-visit.h"
+
+#ifdef CONFIG_FNMATCH
+#include <fnmatch.h>
+#endif
+
+static bool qauthz_simple_is_allowed(QAuthZ *authz,
+ const char *identity,
+ Error **errp)
+{
+ QAuthZSimple *sauthz = QAUTHZ_SIMPLE(authz);
+ QAuthZSimpleRuleList *rules = sauthz->rules;
+
+ while (rules) {
+ QAuthZSimpleRule *rule = rules->value;
+ QAuthZSimpleFormat format = rule->has_format ? rule->format :
+ QAUTHZ_SIMPLE_FORMAT_EXACT;
+
+ switch (format) {
+ case QAUTHZ_SIMPLE_FORMAT_EXACT:
+ if (strcmp(rule->match, identity) == 0) {
+ return rule->policy == QAUTHZ_SIMPLE_POLICY_ALLOW;
+ }
+ break;
+#ifdef CONFIG_FNMATCH
+ case QAUTHZ_SIMPLE_FORMAT_GLOB:
+ if (fnmatch(rule->match, identity, 0) == 0) {
+ return rule->policy == QAUTHZ_SIMPLE_POLICY_ALLOW;
+ }
+ break;
+#else
+ return false;
+#endif
+ default:
+ return false;
+ }
+ rules = rules->next;
+ }
+
+ return sauthz->policy == QAUTHZ_SIMPLE_POLICY_ALLOW;
+}
+
+
+static void
+qauthz_simple_prop_set_policy(Object *obj,
+ int value,
+ Error **errp G_GNUC_UNUSED)
+{
+ QAuthZSimple *sauthz = QAUTHZ_SIMPLE(obj);
+
+ sauthz->policy = value;
+}
+
+
+static int
+qauthz_simple_prop_get_policy(Object *obj,
+ Error **errp G_GNUC_UNUSED)
+{
+ QAuthZSimple *sauthz = QAUTHZ_SIMPLE(obj);
+
+ return sauthz->policy;
+}
+
+
+static void
+qauthz_simple_prop_get_rules(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ QAuthZSimple *sauthz = QAUTHZ_SIMPLE(obj);
+
+ visit_type_QAuthZSimpleRuleList(v, name, &sauthz->rules, errp);
+}
+
+static void
+qauthz_simple_prop_set_rules(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ QAuthZSimple *sauthz = QAUTHZ_SIMPLE(obj);
+ QAuthZSimpleRuleList *oldrules;
+#ifndef CONFIG_FNMATCH
+ QAuthZSimpleRuleList *rules;
+#endif
+
+ oldrules = sauthz->rules;
+ visit_type_QAuthZSimpleRuleList(v, name, &sauthz->rules, errp);
+
+#ifndef CONFIG_FNMATCH
+ rules = sauthz->rules;
+ while (rules) {
+ QAuthZSimpleRule *rule = rules->value;
+ if (rule->has_format &&
+ rule->format == QAUTHZ_SIMPLE_FORMAT_GLOB) {
+ error_setg(errp, "Glob format not supported on this platform");
+ qapi_free_QAuthZSimpleRuleList(sauthz->rules);
+ sauthz->rules = oldrules;
+ return;
+ }
+ }
+#endif
+
+ qapi_free_QAuthZSimpleRuleList(oldrules);
+}
+
+
+static void
+qauthz_simple_complete(UserCreatable *uc, Error **errp)
+{
+}
+
+
+static void
+qauthz_simple_finalize(Object *obj)
+{
+ QAuthZSimple *sauthz = QAUTHZ_SIMPLE(obj);
+
+ qapi_free_QAuthZSimpleRuleList(sauthz->rules);
+}
+
+
+static void
+qauthz_simple_class_init(ObjectClass *oc, void *data)
+{
+ UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
+ QAuthZClass *authz = QAUTHZ_CLASS(oc);
+
+ ucc->complete = qauthz_simple_complete;
+ authz->is_allowed = qauthz_simple_is_allowed;
+
+ object_class_property_add_enum(oc, "policy",
+ "QAuthZSimplePolicy",
+ QAuthZSimplePolicy_lookup,
+ qauthz_simple_prop_get_policy,
+ qauthz_simple_prop_set_policy,
+ NULL);
+
+ object_class_property_add(oc, "rules", "QAuthZSimpleRule",
+ qauthz_simple_prop_get_rules,
+ qauthz_simple_prop_set_rules,
+ NULL, NULL, NULL);
+}
+
+
+QAuthZSimple *qauthz_simple_new(const char *id,
+ QAuthZSimplePolicy policy,
+ Error **errp)
+{
+ return QAUTHZ_SIMPLE(
+ object_new_with_props(TYPE_QAUTHZ_SIMPLE,
+ object_get_objects_root(),
+ id, errp,
+ "policy", QAuthZSimplePolicy_lookup[policy],
+ NULL));
+}
+
+
+ssize_t qauthz_simple_append_rule(QAuthZSimple *auth,
+ const char *match,
+ QAuthZSimplePolicy policy,
+ QAuthZSimpleFormat format,
+ Error **errp)
+{
+ QAuthZSimpleRule *rule;
+ QAuthZSimpleRuleList *rules, *tmp;
+ size_t i = 0;
+
+#ifndef CONFIG_FNMATCH
+ if (format == QAUTHZ_SIMPLE_FORMAT_GLOB) {
+ error_setg(errp, "Glob format not supported on this platform");
+ return -1;
+ }
+#endif
+
+ rule = g_new0(QAuthZSimpleRule, 1);
+ rule->policy = policy;
+ rule->match = g_strdup(match);
+ rule->format = format;
+ rule->has_format = true;
+
+ tmp = g_new0(QAuthZSimpleRuleList, 1);
+ tmp->value = rule;
+
+ rules = auth->rules;
+ if (rules) {
+ while (rules->next) {
+ i++;
+ rules = rules->next;
+ }
+ rules->next = tmp;
+ return i + 1;
+ } else {
+ auth->rules = tmp;
+ return 0;
+ }
+}
+
+
+ssize_t qauthz_simple_insert_rule(QAuthZSimple *auth,
+ const char *match,
+ QAuthZSimplePolicy policy,
+ QAuthZSimpleFormat format,
+ size_t index,
+ Error **errp)
+{
+ QAuthZSimpleRule *rule;
+ QAuthZSimpleRuleList *rules, *tmp;
+ size_t i = 0;
+
+#ifndef CONFIG_FNMATCH
+ if (format == QAUTHZ_SIMPLE_FORMAT_GLOB) {
+ error_setg(errp, "Glob format not supported on this platform");
+ return -1;
+ }
+#endif
+
+ rule = g_new0(QAuthZSimpleRule, 1);
+ rule->policy = policy;
+ rule->match = g_strdup(match);
+ rule->format = format;
+ rule->has_format = true;
+
+ tmp = g_new0(QAuthZSimpleRuleList, 1);
+ tmp->value = rule;
+
+ rules = auth->rules;
+ if (rules && index > 0) {
+ while (rules->next && i < (index - 1)) {
+ i++;
+ rules = rules->next;
+ }
+ tmp->next = rules->next;
+ rules->next = tmp;
+ return i + 1;
+ } else {
+ tmp->next = auth->rules;
+ auth->rules = tmp;
+ return 0;
+ }
+}
+
+
+ssize_t qauthz_simple_delete_rule(QAuthZSimple *auth, const char *match)
+{
+ QAuthZSimpleRule *rule;
+ QAuthZSimpleRuleList *rules, *prev;
+ size_t i = 0;
+
+ prev = NULL;
+ rules = auth->rules;
+ while (rules) {
+ rule = rules->value;
+ if (g_str_equal(rule->match, match)) {
+ if (prev) {
+ prev->next = rules->next;
+ } else {
+ auth->rules = rules->next;
+ }
+ rules->next = NULL;
+ qapi_free_QAuthZSimpleRuleList(rules);
+ return i;
+ }
+ prev = rules;
+ rules = rules->next;
+ i++;
+ }
+
+ return -1;
+}
+
+
+static const TypeInfo qauthz_simple_info = {
+ .parent = TYPE_QAUTHZ,
+ .name = TYPE_QAUTHZ_SIMPLE,
+ .instance_size = sizeof(QAuthZSimple),
+ .instance_finalize = qauthz_simple_finalize,
+ .class_size = sizeof(QAuthZSimpleClass),
+ .class_init = qauthz_simple_class_init,
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_USER_CREATABLE },
+ { }
+ }
+};
+
+
+static void
+qauthz_simple_register_types(void)
+{
+ type_register_static(&qauthz_simple_info);
+}
+
+
+type_init(qauthz_simple_register_types);
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 06/11] acl: delete existing ACL implementation
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (4 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 05/11] util: add QAuthZSimple object type for a simple access control list Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 07/11] qemu-nbd: add support for ACLs for TLS clients Daniel P. Berrange
` (5 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
The 'qemu_acl' type was a previous non-QOM based attempt to
provide an authorization facility in QEMU. Because it is
non-QOM based it cannot be created via the command line and
requires special monitor commands to manipulate it.
The new QAuthZ and QAuthZSimple QOM classes provide a superset
of the functionality in qemu_acl, so the latter can now be
deleted. The HMP 'acl_*' monitor commands are converted to
use the new QAuthZSimple data type instead in order to provide
backwards compatibility, but their use is discouraged.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
Makefile | 6 +-
crypto/tlssession.c | 28 ++++--
include/qemu/acl.h | 74 ----------------
monitor.c | 181 ++++++++++++++++++++++++++-------------
tests/Makefile | 2 +-
tests/test-crypto-tlssession.c | 13 +--
tests/test-io-channel-tls.c | 14 +--
ui/vnc-auth-sasl.c | 2 +-
ui/vnc-auth-sasl.h | 4 +-
ui/vnc.c | 11 ++-
util/Makefile.objs | 1 -
util/acl.c | 188 -----------------------------------------
12 files changed, 176 insertions(+), 348 deletions(-)
delete mode 100644 include/qemu/acl.h
delete mode 100644 util/acl.c
diff --git a/Makefile b/Makefile
index ae3ffac..97e13da 100644
--- a/Makefile
+++ b/Makefile
@@ -232,9 +232,9 @@ util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)'
qemu-img.o: qemu-img-cmds.h
-qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
+qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o libqemuutil.a libqemustub.a
diff --git a/crypto/tlssession.c b/crypto/tlssession.c
index a543e5a..78bb0f5 100644
--- a/crypto/tlssession.c
+++ b/crypto/tlssession.c
@@ -23,7 +23,7 @@
#include "crypto/tlscredsanon.h"
#include "crypto/tlscredsx509.h"
#include "qapi/error.h"
-#include "qemu/acl.h"
+#include "qemu/authz.h"
#include "trace.h"
#ifdef CONFIG_GNUTLS
@@ -208,6 +208,7 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
unsigned int nCerts, i;
time_t now;
gnutls_x509_crt_t cert = NULL;
+ Error *err = NULL;
now = time(NULL);
if (now == ((time_t)-1)) {
@@ -296,16 +297,33 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
goto error;
}
if (session->aclname) {
- qemu_acl *acl = qemu_acl_find(session->aclname);
- int allow;
- if (!acl) {
+ QAuthZ *acl;
+ Object *obj;
+ Object *container;
+ bool allow;
+
+ container = object_get_objects_root();
+ obj = object_resolve_path_component(container,
+ session->aclname);
+ if (!obj) {
error_setg(errp, "Cannot find ACL %s",
session->aclname);
goto error;
}
- allow = qemu_acl_party_is_allowed(acl, session->peername);
+ if (!object_dynamic_cast(obj, TYPE_QAUTHZ)) {
+ error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+ session->aclname);
+ goto error;
+ }
+ acl = QAUTHZ(obj);
+
+ allow = qauthz_is_allowed(acl, session->peername, &err);
+ if (err) {
+ error_propagate(errp, err);
+ goto error;
+ }
if (!allow) {
error_setg(errp, "TLS x509 ACL check for %s is denied",
session->peername);
diff --git a/include/qemu/acl.h b/include/qemu/acl.h
deleted file mode 100644
index 116487e..0000000
--- a/include/qemu/acl.h
+++ /dev/null
@@ -1,74 +0,0 @@
-/*
- * QEMU access control list management
- *
- * Copyright (C) 2009 Red Hat, Inc
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-
-#ifndef __QEMU_ACL_H__
-#define __QEMU_ACL_H__
-
-#include "qemu/queue.h"
-
-typedef struct qemu_acl_entry qemu_acl_entry;
-typedef struct qemu_acl qemu_acl;
-
-struct qemu_acl_entry {
- char *match;
- int deny;
-
- QTAILQ_ENTRY(qemu_acl_entry) next;
-};
-
-struct qemu_acl {
- char *aclname;
- unsigned int nentries;
- QTAILQ_HEAD(,qemu_acl_entry) entries;
- int defaultDeny;
-};
-
-qemu_acl *qemu_acl_init(const char *aclname);
-
-qemu_acl *qemu_acl_find(const char *aclname);
-
-int qemu_acl_party_is_allowed(qemu_acl *acl,
- const char *party);
-
-void qemu_acl_reset(qemu_acl *acl);
-
-int qemu_acl_append(qemu_acl *acl,
- int deny,
- const char *match);
-int qemu_acl_insert(qemu_acl *acl,
- int deny,
- const char *match,
- int index);
-int qemu_acl_remove(qemu_acl *acl,
- const char *match);
-
-#endif /* __QEMU_ACL_H__ */
-
-/*
- * Local variables:
- * c-indent-level: 4
- * c-basic-offset: 4
- * tab-width: 8
- * End:
- */
diff --git a/monitor.c b/monitor.c
index 404d594..83eb4ac 100644
--- a/monitor.c
+++ b/monitor.c
@@ -51,7 +51,7 @@
#include "qemu/timer.h"
#include "migration/migration.h"
#include "sysemu/kvm.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
#include "sysemu/tpm.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/qint.h"
@@ -62,6 +62,7 @@
#include "qapi/qmp/qjson.h"
#include "qapi/qmp/json-streamer.h"
#include "qapi/qmp/json-parser.h"
+#include "qapi/util.h"
#include <qom/object_interfaces.h>
#include "cpu.h"
#include "trace.h"
@@ -1596,61 +1597,88 @@ static void hmp_wavcapture(Monitor *mon, const QDict *qdict)
QLIST_INSERT_HEAD (&capture_head, s, entries);
}
-static qemu_acl *find_acl(Monitor *mon, const char *name)
+static QAuthZSimple *find_auth(Monitor *mon, const char *name)
{
- qemu_acl *acl = qemu_acl_find(name);
+ Object *obj;
+ Object *container;
- if (!acl) {
+ container = object_get_objects_root();
+ obj = object_resolve_path_component(container, name);
+ if (!obj) {
monitor_printf(mon, "acl: unknown list '%s'\n", name);
+ return NULL;
}
- return acl;
+
+ return QAUTHZ_SIMPLE(obj);
}
static void hmp_acl_show(Monitor *mon, const QDict *qdict)
{
const char *aclname = qdict_get_str(qdict, "aclname");
- qemu_acl *acl = find_acl(mon, aclname);
- qemu_acl_entry *entry;
- int i = 0;
-
- if (acl) {
- monitor_printf(mon, "policy: %s\n",
- acl->defaultDeny ? "deny" : "allow");
- QTAILQ_FOREACH(entry, &acl->entries, next) {
- i++;
- monitor_printf(mon, "%d: %s %s\n", i,
- entry->deny ? "deny" : "allow", entry->match);
- }
+ QAuthZSimple *auth = find_auth(mon, aclname);
+ QAuthZSimpleRuleList *rules;
+ size_t i = 0;
+
+ if (!auth) {
+ return;
+ }
+
+ monitor_printf(mon, "policy: %s\n",
+ QAuthZSimplePolicy_lookup[auth->policy]);
+
+ rules = auth->rules;
+ while (rules) {
+ QAuthZSimpleRule *rule = rules->value;
+ i++;
+ monitor_printf(mon, "%zu: %s %s\n", i,
+ QAuthZSimplePolicy_lookup[rule->policy],
+ rule->match);
+ rules = rules->next;
}
}
static void hmp_acl_reset(Monitor *mon, const QDict *qdict)
{
const char *aclname = qdict_get_str(qdict, "aclname");
- qemu_acl *acl = find_acl(mon, aclname);
+ QAuthZSimple *auth = find_auth(mon, aclname);
- if (acl) {
- qemu_acl_reset(acl);
- monitor_printf(mon, "acl: removed all rules\n");
+ if (!auth) {
+ return;
}
+
+ auth->policy = QAUTHZ_SIMPLE_POLICY_DENY;
+ qapi_free_QAuthZSimpleRuleList(auth->rules);
+ auth->rules = NULL;
+ monitor_printf(mon, "acl: removed all rules\n");
}
static void hmp_acl_policy(Monitor *mon, const QDict *qdict)
{
const char *aclname = qdict_get_str(qdict, "aclname");
const char *policy = qdict_get_str(qdict, "policy");
- qemu_acl *acl = find_acl(mon, aclname);
+ QAuthZSimple *auth = find_auth(mon, aclname);
+ int val;
+ Error *err = NULL;
- if (acl) {
- if (strcmp(policy, "allow") == 0) {
- acl->defaultDeny = 0;
+ if (!auth) {
+ return;
+ }
+
+ val = qapi_enum_parse(QAuthZSimplePolicy_lookup,
+ policy,
+ QAUTHZ_SIMPLE_POLICY__MAX,
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &err);
+ if (err) {
+ error_free(err);
+ monitor_printf(mon, "acl: unknown policy '%s', "
+ "expected 'deny' or 'allow'\n", policy);
+ } else {
+ auth->policy = val;
+ if (auth->policy == QAUTHZ_SIMPLE_POLICY_ALLOW) {
monitor_printf(mon, "acl: policy set to 'allow'\n");
- } else if (strcmp(policy, "deny") == 0) {
- acl->defaultDeny = 1;
- monitor_printf(mon, "acl: policy set to 'deny'\n");
} else {
- monitor_printf(mon, "acl: unknown policy '%s', "
- "expected 'deny' or 'allow'\n", policy);
+ monitor_printf(mon, "acl: policy set to 'deny'\n");
}
}
}
@@ -1659,30 +1687,60 @@ static void hmp_acl_add(Monitor *mon, const QDict *qdict)
{
const char *aclname = qdict_get_str(qdict, "aclname");
const char *match = qdict_get_str(qdict, "match");
- const char *policy = qdict_get_str(qdict, "policy");
+ const char *policystr = qdict_get_str(qdict, "policy");
int has_index = qdict_haskey(qdict, "index");
int index = qdict_get_try_int(qdict, "index", -1);
- qemu_acl *acl = find_acl(mon, aclname);
- int deny, ret;
-
- if (acl) {
- if (strcmp(policy, "allow") == 0) {
- deny = 0;
- } else if (strcmp(policy, "deny") == 0) {
- deny = 1;
- } else {
- monitor_printf(mon, "acl: unknown policy '%s', "
- "expected 'deny' or 'allow'\n", policy);
- return;
- }
- if (has_index)
- ret = qemu_acl_insert(acl, deny, match, index);
- else
- ret = qemu_acl_append(acl, deny, match);
- if (ret < 0)
- monitor_printf(mon, "acl: unable to add acl entry\n");
- else
- monitor_printf(mon, "acl: added rule at position %d\n", ret);
+ QAuthZSimple *auth = find_auth(mon, aclname);
+ Error *err = NULL;
+ QAuthZSimplePolicy policy;
+ QAuthZSimpleFormat format;
+ size_t i = 0;
+
+ if (!auth) {
+ return;
+ }
+
+ policy = qapi_enum_parse(QAuthZSimplePolicy_lookup,
+ policystr,
+ QAUTHZ_SIMPLE_POLICY__MAX,
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &err);
+ if (err) {
+ error_free(err);
+ monitor_printf(mon, "acl: unknown policy '%s', "
+ "expected 'deny' or 'allow'\n", policystr);
+ return;
+ }
+
+#ifdef CONFIG_FNMATCH
+ if (strchr(match, '*')) {
+ format = QAUTHZ_SIMPLE_FORMAT_GLOB;
+ } else {
+ format = QAUTHZ_SIMPLE_FORMAT_EXACT;
+ }
+#else
+ /* Historically we silently degraded to plain strcmp
+ * when fnmatch() was missing */
+ format = QAUTHZ_SIMPLE_FORMAT_EXACT;
+#endif
+
+ if (has_index && index == 0) {
+ monitor_printf(mon, "acl: unable to add acl entry\n");
+ return;
+ }
+
+ if (has_index) {
+ i = qauthz_simple_insert_rule(auth, match, policy,
+ format, index - 1, &err);
+ } else {
+ i = qauthz_simple_append_rule(auth, match, policy,
+ format, &err);
+ }
+ if (err) {
+ monitor_printf(mon, "acl: unable to add rule: %s", error_get_pretty(err));
+ error_free(err);
+ } else {
+ monitor_printf(mon, "acl: added rule at position %zu\n", i + 1);
}
}
@@ -1690,15 +1748,18 @@ static void hmp_acl_remove(Monitor *mon, const QDict *qdict)
{
const char *aclname = qdict_get_str(qdict, "aclname");
const char *match = qdict_get_str(qdict, "match");
- qemu_acl *acl = find_acl(mon, aclname);
- int ret;
+ QAuthZSimple *auth = find_auth(mon, aclname);
+ ssize_t i = 0;
- if (acl) {
- ret = qemu_acl_remove(acl, match);
- if (ret < 0)
- monitor_printf(mon, "acl: no matching acl entry\n");
- else
- monitor_printf(mon, "acl: removed rule at position %d\n", ret);
+ if (!auth) {
+ return;
+ }
+
+ i = qauthz_simple_delete_rule(auth, match);
+ if (i >= 0) {
+ monitor_printf(mon, "acl: removed rule at position %zu\n", i + 1);
+ } else {
+ monitor_printf(mon, "acl: no matching acl entry\n");
}
}
diff --git a/tests/Makefile b/tests/Makefile
index c4c5fc5..3d62b3f 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -405,7 +405,7 @@ test-qom-obj-y = $(qom-obj-y) $(test-util-obj-y)
test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
tests/test-qapi-event.o tests/test-qmp-introspect.o \
$(test-qom-obj-y)
-test-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
+test-crypto-obj-y = $(crypto-obj-y) $(util-qom-obj-y) $(test-qom-obj-y)
test-io-obj-y = $(io-obj-y) $(test-crypto-obj-y)
test-block-obj-y = $(block-obj-y) $(test-io-obj-y)
diff --git a/tests/test-crypto-tlssession.c b/tests/test-crypto-tlssession.c
index 1a4a066..d3a30e9 100644
--- a/tests/test-crypto-tlssession.c
+++ b/tests/test-crypto-tlssession.c
@@ -26,7 +26,7 @@
#include "qom/object_interfaces.h"
#include "qapi/error.h"
#include "qemu/sockets.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
#ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
@@ -111,7 +111,7 @@ static void test_crypto_tls_session(const void *opaque)
QCryptoTLSCreds *serverCreds;
QCryptoTLSSession *clientSess = NULL;
QCryptoTLSSession *serverSess = NULL;
- qemu_acl *acl;
+ QAuthZSimple *auth;
const char * const *wildcards;
int channel[2];
bool clientShake = false;
@@ -170,11 +170,13 @@ static void test_crypto_tls_session(const void *opaque)
&err);
g_assert(serverCreds != NULL);
- acl = qemu_acl_init("tlssessionacl");
- qemu_acl_reset(acl);
+ auth = qauthz_simple_new("tlssessionacl",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort);
wildcards = data->wildcards;
while (wildcards && *wildcards) {
- qemu_acl_append(acl, 0, *wildcards);
+ qauthz_simple_append_rule(auth, *wildcards,
+ QAUTHZ_SIMPLE_POLICY_ALLOW);
wildcards++;
}
@@ -264,6 +266,7 @@ static void test_crypto_tls_session(const void *opaque)
object_unparent(OBJECT(serverCreds));
object_unparent(OBJECT(clientCreds));
+ object_unparent(OBJECT(auth));
qcrypto_tls_session_free(serverSess);
qcrypto_tls_session_free(clientSess);
diff --git a/tests/test-io-channel-tls.c b/tests/test-io-channel-tls.c
index 3c361a7..390d3e9 100644
--- a/tests/test-io-channel-tls.c
+++ b/tests/test-io-channel-tls.c
@@ -28,7 +28,7 @@
#include "io/channel-socket.h"
#include "io-channel-helpers.h"
#include "crypto/tlscredsx509.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
#include "qom/object_interfaces.h"
#ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
@@ -115,7 +115,7 @@ static void test_io_channel_tls(const void *opaque)
QIOChannelTLS *serverChanTLS;
QIOChannelSocket *clientChanSock;
QIOChannelSocket *serverChanSock;
- qemu_acl *acl;
+ QAuthZSimple *auth;
const char * const *wildcards;
int channel[2];
struct QIOChannelTLSHandshakeData clientHandshake = { false, false };
@@ -166,11 +166,13 @@ static void test_io_channel_tls(const void *opaque)
&err);
g_assert(serverCreds != NULL);
- acl = qemu_acl_init("channeltlsacl");
- qemu_acl_reset(acl);
+ auth = qauthz_simple_new("channeltlsacl",
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort);
wildcards = data->wildcards;
while (wildcards && *wildcards) {
- qemu_acl_append(acl, 0, *wildcards);
+ qauthz_simple_append_rule(auth, *wildcards,
+ QAUTHZ_SIMPLE_POLICY_ALLOW);
wildcards++;
}
@@ -256,6 +258,8 @@ static void test_io_channel_tls(const void *opaque)
object_unref(OBJECT(serverChanSock));
object_unref(OBJECT(clientChanSock));
+ object_unparent(OBJECT(auth));
+
close(channel[0]);
close(channel[1]);
}
diff --git a/ui/vnc-auth-sasl.c b/ui/vnc-auth-sasl.c
index 5ae29c1..b1064a7 100644
--- a/ui/vnc-auth-sasl.c
+++ b/ui/vnc-auth-sasl.c
@@ -150,7 +150,7 @@ static int vnc_auth_sasl_check_access(VncState *vs)
return 0;
}
- allow = qemu_acl_party_is_allowed(vs->vd->sasl.acl, vs->sasl.username);
+ allow = qauthz_is_allowed(vs->vd->sasl.acl, vs->sasl.username, NULL);
VNC_DEBUG("SASL client %s %s by ACL\n", vs->sasl.username,
allow ? "allowed" : "denied");
diff --git a/ui/vnc-auth-sasl.h b/ui/vnc-auth-sasl.h
index 3f59da6..a6ffb7e 100644
--- a/ui/vnc-auth-sasl.h
+++ b/ui/vnc-auth-sasl.h
@@ -32,7 +32,7 @@
typedef struct VncStateSASL VncStateSASL;
typedef struct VncDisplaySASL VncDisplaySASL;
-#include "qemu/acl.h"
+#include "qemu/authz.h"
#include "qemu/main-loop.h"
struct VncStateSASL {
@@ -61,7 +61,7 @@ struct VncStateSASL {
};
struct VncDisplaySASL {
- qemu_acl *acl;
+ QAuthZ *acl;
};
void vnc_sasl_client_cleanup(VncState *vs);
diff --git a/ui/vnc.c b/ui/vnc.c
index d2ebf1f..ba07715 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -33,7 +33,7 @@
#include "qemu/error-report.h"
#include "qemu/sockets.h"
#include "qemu/timer.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
#include "qemu/config-file.h"
#include "qapi/qmp/qerror.h"
#include "qapi/qmp/types.h"
@@ -3707,7 +3707,9 @@ void vnc_display_open(const char *id, Error **errp)
} else {
vs->tlsaclname = g_strdup_printf("vnc.%s.x509dname", vs->id);
}
- qemu_acl_init(vs->tlsaclname);
+ qauthz_simple_new(vs->tlsaclname,
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort);
}
#ifdef CONFIG_VNC_SASL
if (acl && sasl) {
@@ -3718,7 +3720,10 @@ void vnc_display_open(const char *id, Error **errp)
} else {
aclname = g_strdup_printf("vnc.%s.username", vs->id);
}
- vs->sasl.acl = qemu_acl_init(aclname);
+ vs->sasl.acl =
+ QAUTHZ(qauthz_simple_new(aclname,
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort));
g_free(aclname);
}
#endif
diff --git a/util/Makefile.objs b/util/Makefile.objs
index 4870905..44078c1 100644
--- a/util/Makefile.objs
+++ b/util/Makefile.objs
@@ -13,7 +13,6 @@ util-obj-y += envlist.o path.o module.o
util-obj-$(call lnot,$(CONFIG_INT128)) += host-utils.o
util-obj-y += bitmap.o bitops.o hbitmap.o
util-obj-y += fifo8.o
-util-obj-y += acl.o
util-obj-y += error.o qemu-error.o
util-obj-y += id.o
util-obj-y += iov.o qemu-config.o qemu-sockets.o uri.o notify.o
diff --git a/util/acl.c b/util/acl.c
deleted file mode 100644
index 723b6a8..0000000
--- a/util/acl.c
+++ /dev/null
@@ -1,188 +0,0 @@
-/*
- * QEMU access control list management
- *
- * Copyright (C) 2009 Red Hat, Inc
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-
-
-#include "qemu/osdep.h"
-#include "qemu-common.h"
-#include "qemu/acl.h"
-
-#ifdef CONFIG_FNMATCH
-#include <fnmatch.h>
-#endif
-
-
-static unsigned int nacls = 0;
-static qemu_acl **acls = NULL;
-
-
-
-qemu_acl *qemu_acl_find(const char *aclname)
-{
- int i;
- for (i = 0 ; i < nacls ; i++) {
- if (strcmp(acls[i]->aclname, aclname) == 0)
- return acls[i];
- }
-
- return NULL;
-}
-
-qemu_acl *qemu_acl_init(const char *aclname)
-{
- qemu_acl *acl;
-
- acl = qemu_acl_find(aclname);
- if (acl)
- return acl;
-
- acl = g_malloc(sizeof(*acl));
- acl->aclname = g_strdup(aclname);
- /* Deny by default, so there is no window of "open
- * access" between QEMU starting, and the user setting
- * up ACLs in the monitor */
- acl->defaultDeny = 1;
-
- acl->nentries = 0;
- QTAILQ_INIT(&acl->entries);
-
- acls = g_realloc(acls, sizeof(*acls) * (nacls +1));
- acls[nacls] = acl;
- nacls++;
-
- return acl;
-}
-
-int qemu_acl_party_is_allowed(qemu_acl *acl,
- const char *party)
-{
- qemu_acl_entry *entry;
-
- QTAILQ_FOREACH(entry, &acl->entries, next) {
-#ifdef CONFIG_FNMATCH
- if (fnmatch(entry->match, party, 0) == 0)
- return entry->deny ? 0 : 1;
-#else
- /* No fnmatch, so fallback to exact string matching
- * instead of allowing wildcards */
- if (strcmp(entry->match, party) == 0)
- return entry->deny ? 0 : 1;
-#endif
- }
-
- return acl->defaultDeny ? 0 : 1;
-}
-
-
-void qemu_acl_reset(qemu_acl *acl)
-{
- qemu_acl_entry *entry, *next_entry;
-
- /* Put back to deny by default, so there is no window
- * of "open access" while the user re-initializes the
- * access control list */
- acl->defaultDeny = 1;
- QTAILQ_FOREACH_SAFE(entry, &acl->entries, next, next_entry) {
- QTAILQ_REMOVE(&acl->entries, entry, next);
- g_free(entry->match);
- g_free(entry);
- }
- acl->nentries = 0;
-}
-
-
-int qemu_acl_append(qemu_acl *acl,
- int deny,
- const char *match)
-{
- qemu_acl_entry *entry;
-
- entry = g_malloc(sizeof(*entry));
- entry->match = g_strdup(match);
- entry->deny = deny;
-
- QTAILQ_INSERT_TAIL(&acl->entries, entry, next);
- acl->nentries++;
-
- return acl->nentries;
-}
-
-
-int qemu_acl_insert(qemu_acl *acl,
- int deny,
- const char *match,
- int index)
-{
- qemu_acl_entry *tmp;
- int i = 0;
-
- if (index <= 0)
- return -1;
- if (index > acl->nentries) {
- return qemu_acl_append(acl, deny, match);
- }
-
- QTAILQ_FOREACH(tmp, &acl->entries, next) {
- i++;
- if (i == index) {
- qemu_acl_entry *entry;
- entry = g_malloc(sizeof(*entry));
- entry->match = g_strdup(match);
- entry->deny = deny;
-
- QTAILQ_INSERT_BEFORE(tmp, entry, next);
- acl->nentries++;
- break;
- }
- }
-
- return i;
-}
-
-int qemu_acl_remove(qemu_acl *acl,
- const char *match)
-{
- qemu_acl_entry *entry;
- int i = 0;
-
- QTAILQ_FOREACH(entry, &acl->entries, next) {
- i++;
- if (strcmp(entry->match, match) == 0) {
- QTAILQ_REMOVE(&acl->entries, entry, next);
- acl->nentries--;
- g_free(entry->match);
- g_free(entry);
- return i;
- }
- }
- return -1;
-}
-
-
-/*
- * Local variables:
- * c-indent-level: 4
- * c-basic-offset: 4
- * tab-width: 8
- * End:
- */
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 07/11] qemu-nbd: add support for ACLs for TLS clients
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (5 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 06/11] acl: delete existing ACL implementation Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 08/11] nbd: allow an ACL to be set with nbd-server-start QMP command Daniel P. Berrange
` (4 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
Currently any client which can complete the TLS handshake
is able to use the NBD server. The server admin can turn
on the 'verify-peer' option for the x509 creds to require
the client to provide a x509 certificate. This means the
client will have to acquire a certificate from the CA before
they are permitted to use the NBD server. This is still a
fairly weak bar.
This adds a '--tls-acl ACL-ID' option to the qemu-nbd command
which takes the ID of a previously added 'QAuthZ' object
instance. This ACL will be used to validate the client's
x509 distinguished name. Clients failing the ACL will not be
permitted to use the NBD server.
For example to setup an ACL that only allows connection from
a client whose x509 certificate distinguished name contains
'CN=fred', you would use:
qemu-nbd -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
endpoint=server,verify-peer=yes \
-object authz-simple,id=acl0,policy=deny,\
rules.0.match=*CN=fred,rules.0.policy=allow \
-tls-creds tls0 \
-tls-acl acl0
....other qemu-nbd args...
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
qemu-nbd.c | 13 ++++++++++++-
qemu-nbd.texi | 4 ++++
2 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 6554f0a..2450214 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -46,6 +46,7 @@
#define QEMU_NBD_OPT_OBJECT 260
#define QEMU_NBD_OPT_TLSCREDS 261
#define QEMU_NBD_OPT_IMAGE_OPTS 262
+#define QEMU_NBD_OPT_TLSACL 263
#define MBR_SIZE 512
@@ -61,6 +62,7 @@ static int nb_fds;
static QIOChannelSocket *server_ioc;
static int server_watch = -1;
static QCryptoTLSCreds *tlscreds;
+static const char *tlsacl;
static void usage(const char *name)
{
@@ -354,7 +356,7 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition cond, gpointer opaque)
nb_fds++;
nbd_update_server_watch();
nbd_client_new(newproto ? NULL : exp, cioc,
- tlscreds, NULL, nbd_client_closed);
+ tlscreds, tlsacl, nbd_client_closed);
object_unref(OBJECT(cioc));
return TRUE;
@@ -498,6 +500,7 @@ int main(int argc, char **argv)
{ "export-name", required_argument, NULL, 'x' },
{ "tls-creds", required_argument, NULL, QEMU_NBD_OPT_TLSCREDS },
{ "image-opts", no_argument, NULL, QEMU_NBD_OPT_IMAGE_OPTS },
+ { "tls-acl", no_argument, NULL, QEMU_NBD_OPT_TLSACL },
{ NULL, 0, NULL, 0 }
};
int ch;
@@ -703,6 +706,9 @@ int main(int argc, char **argv)
case QEMU_NBD_OPT_IMAGE_OPTS:
imageOpts = true;
break;
+ case QEMU_NBD_OPT_TLSACL:
+ tlsacl = optarg;
+ break;
}
}
@@ -738,6 +744,11 @@ int main(int argc, char **argv)
error_get_pretty(local_err));
exit(EXIT_FAILURE);
}
+ } else {
+ if (tlsacl) {
+ error_report("--tls-acl is not permitted without --tls-creds");
+ exit(EXIT_FAILURE);
+ }
}
if (disconnect) {
diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 9f23343..69f32cb 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -86,6 +86,10 @@ the new style NBD protocol negotiation
Enable mandatory TLS encryption for the server by setting the ID
of the TLS credentials object previously created with the --object
option.
+@item --tls-acl=ID
+Specify the ID of a qauthz object previously created with the
+--object option. This will be used to authorize users who
+connect against their x509 distinguish name.
@item -v, --verbose
Display extra debugging information
@item -h, --help
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 08/11] nbd: allow an ACL to be set with nbd-server-start QMP command
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (6 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 07/11] qemu-nbd: add support for ACLs for TLS clients Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 09/11] migration: add support for a "tls-acl" migration parameter Daniel P. Berrange
` (3 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
As with the previous patch to qemu-nbd, the nbd-server-start
QMP command also needs to be able to specify an ACL when
enabling TLS encryption.
First the client must create a QAuthZ object instance using
the 'object-add' command:
{
'execute': 'object-add',
'arguments': {
'qom-type': 'authz-simple',
'id': 'tls0',
'parameters': {
'policy': 'deny',
'rules': [
{
'match': '*CN=fred',
'policy': 'allow'
}
]
}
}
}
They can then reference this in the new 'tls-acl' parameter
when executing the 'nbd-server-start' command.
{
'execute': 'nbd-server-start',
'arguments': {
'addr': {
'type': 'inet',
'host': '127.0.0.1',
'port': '9000'
},
'tls-creds': 'tls0',
'tls-acl': 'tlsacl0'
}
}
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
blockdev-nbd.c | 10 +++++++++-
hmp.c | 2 +-
qapi/block.json | 4 +++-
qmp-commands.hx | 2 +-
4 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 12cae0e..ae5335e 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -24,6 +24,7 @@ typedef struct NBDServerData {
QIOChannelSocket *listen_ioc;
int watch;
QCryptoTLSCreds *tlscreds;
+ char *tlsacl;
} NBDServerData;
static NBDServerData *nbd_server;
@@ -45,7 +46,8 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition condition,
}
nbd_client_new(NULL, cioc,
- nbd_server->tlscreds, NULL,
+ nbd_server->tlscreds,
+ nbd_server->tlsacl,
nbd_client_put);
object_unref(OBJECT(cioc));
return TRUE;
@@ -65,6 +67,7 @@ static void nbd_server_free(NBDServerData *server)
if (server->tlscreds) {
object_unref(OBJECT(server->tlscreds));
}
+ g_free(server->tlsacl);
g_free(server);
}
@@ -101,6 +104,7 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
void qmp_nbd_server_start(SocketAddress *addr,
bool has_tls_creds, const char *tls_creds,
+ bool has_tls_acl, const char *tls_acl,
Error **errp)
{
if (nbd_server) {
@@ -128,6 +132,10 @@ void qmp_nbd_server_start(SocketAddress *addr,
}
}
+ if (has_tls_acl) {
+ nbd_server->tlsacl = g_strdup(tls_acl);
+ }
+
nbd_server->watch = qio_channel_add_watch(
QIO_CHANNEL(nbd_server->listen_ioc),
G_IO_IN,
diff --git a/hmp.c b/hmp.c
index 1972bef..e0d0d8c 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1846,7 +1846,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
goto exit;
}
- qmp_nbd_server_start(addr, false, NULL, &local_err);
+ qmp_nbd_server_start(addr, false, NULL, false, NULL, &local_err);
qapi_free_SocketAddress(addr);
if (local_err != NULL) {
goto exit;
diff --git a/qapi/block.json b/qapi/block.json
index 937337d..a674865 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -147,6 +147,7 @@
#
# @addr: Address on which to listen.
# @tls-creds: (optional) ID of the TLS credentials object. Since 2.6
+# @tls-acl: (optional) ID of the QAuthZ authorization object. Since 2.6
#
# Returns: error if the server is already running.
#
@@ -154,7 +155,8 @@
##
{ 'command': 'nbd-server-start',
'data': { 'addr': 'SocketAddress',
- '*tls-creds': 'str'} }
+ '*tls-creds': 'str',
+ '*tls-acl': 'str'} }
##
# @nbd-server-add:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 28801a2..ad10b1e 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3859,7 +3859,7 @@ EQMP
{
.name = "nbd-server-start",
- .args_type = "addr:q,tls-creds:s?",
+ .args_type = "addr:q,tls-creds:s?,tls-acl:s?",
.mhandler.cmd_new = qmp_marshal_nbd_server_start,
},
{
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 09/11] migration: add support for a "tls-acl" migration parameter
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (7 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 08/11] nbd: allow an ACL to be set with nbd-server-start QMP command Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 10/11] chardev: add support for ACLs for TLS clients Daniel P. Berrange
` (2 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
The QEMU instance that runs as the server for the migration data
transport (ie the target QEMU) needs to be able to configure access
control so it can prevent unauthorized clients initiating an incoming
migration. This adds a new 'tls-acl' migration parameter that is used
to provide the QOM ID of a QAuthZ subclass instance that provides the
access control check. This ACL is checked against the x509 certificate
obtained during the TLS handshake.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
hmp.c | 8 ++++++++
migration/migration.c | 7 +++++++
migration/tls.c | 2 +-
qapi-schema.json | 20 +++++++++++++++++---
4 files changed, 33 insertions(+), 4 deletions(-)
diff --git a/hmp.c b/hmp.c
index e0d0d8c..4cbf37c 100644
--- a/hmp.c
+++ b/hmp.c
@@ -300,6 +300,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict)
monitor_printf(mon, " %s: '%s'",
MigrationParameter_lookup[MIGRATION_PARAMETER_TLS_HOSTNAME],
params->tls_hostname ? : "");
+ monitor_printf(mon, " %s: '%s'",
+ MigrationParameter_lookup[MIGRATION_PARAMETER_TLS_ACL],
+ params->tls_acl ? : "");
monitor_printf(mon, "\n");
}
@@ -1259,6 +1262,7 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
bool has_cpu_throttle_increment = false;
bool has_tls_creds = false;
bool has_tls_hostname = false;
+ bool has_tls_acl = false;
bool use_int_value = false;
int i;
@@ -1290,6 +1294,9 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
case MIGRATION_PARAMETER_TLS_HOSTNAME:
has_tls_hostname = true;
break;
+ case MIGRATION_PARAMETER_TLS_ACL:
+ has_tls_acl = true;
+ break;
}
if (use_int_value) {
@@ -1307,6 +1314,7 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
has_cpu_throttle_increment, valueint,
has_tls_creds, valuestr,
has_tls_hostname, valuestr,
+ has_tls_acl, valuestr,
&err);
break;
}
diff --git a/migration/migration.c b/migration/migration.c
index 7ecbade..b5e8e2f 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -566,6 +566,7 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp)
params->cpu_throttle_increment = s->parameters.cpu_throttle_increment;
params->tls_creds = g_strdup(s->parameters.tls_creds);
params->tls_hostname = g_strdup(s->parameters.tls_hostname);
+ params->tls_acl = g_strdup(s->parameters.tls_acl);
return params;
}
@@ -771,6 +772,8 @@ void qmp_migrate_set_parameters(bool has_compress_level,
const char *tls_creds,
bool has_tls_hostname,
const char *tls_hostname,
+ bool has_tls_acl,
+ const char *tls_acl,
Error **errp)
{
MigrationState *s = migrate_get_current();
@@ -830,6 +833,10 @@ void qmp_migrate_set_parameters(bool has_compress_level,
g_free(s->parameters.tls_hostname);
s->parameters.tls_hostname = g_strdup(tls_hostname);
}
+ if (has_tls_acl) {
+ g_free(s->parameters.tls_acl);
+ s->parameters.tls_acl = g_strdup(tls_acl);
+ }
}
diff --git a/migration/tls.c b/migration/tls.c
index 75f959f..968fe16 100644
--- a/migration/tls.c
+++ b/migration/tls.c
@@ -92,7 +92,7 @@ void migration_tls_set_incoming_channel(MigrationState *s,
tioc = qio_channel_tls_new_server(
ioc, creds,
- NULL, /* XXX pass ACL name */
+ s->parameters.tls_acl,
errp);
if (!tioc) {
return;
diff --git a/qapi-schema.json b/qapi-schema.json
index 337a6ce..e7ec2a1 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -636,12 +636,16 @@
# hostname must be provided so that the server's x509
# certificate identity canbe validated. (Since 2.7)
#
+# @tls-acl: ID of the 'authz' object subclass that provides access control
+# checking of the TLS x509 certificate distinguished name. (Since
+# 2.7)
+#
# Since: 2.4
##
{ 'enum': 'MigrationParameter',
'data': ['compress-level', 'compress-threads', 'decompress-threads',
'cpu-throttle-initial', 'cpu-throttle-increment',
- 'tls-creds', 'tls-hostname'] }
+ 'tls-creds', 'tls-hostname', 'tls-acl'] }
#
# @migrate-set-parameters
@@ -677,6 +681,10 @@
# hostname must be provided so that the server's x509
# certificate identity canbe validated. (Since 2.7)
#
+# @tls-acl: ID of the 'authz' object subclass that provides access control
+# checking of the TLS x509 certificate distinguished name. (Since
+# 2.7)
+#
# Since: 2.4
##
{ 'command': 'migrate-set-parameters',
@@ -686,7 +694,8 @@
'*cpu-throttle-initial': 'int',
'*cpu-throttle-increment': 'int',
'*tls-creds': 'str',
- '*tls-hostname': 'str'} }
+ '*tls-hostname': 'str',
+ '*tls-acl': 'str'} }
#
# @MigrationParameters
@@ -720,6 +729,10 @@
# hostname must be provided so that the server's x509
# certificate identity canbe validated. (Since 2.6)
#
+# @tls-acl: ID of the 'authz' object subclass that provides access control
+# checking of the TLS x509 certificate distinguished name. (Since
+# 2.7)
+#
# Since: 2.4
##
{ 'struct': 'MigrationParameters',
@@ -729,7 +742,8 @@
'cpu-throttle-initial': 'int',
'cpu-throttle-increment': 'int',
'tls-creds': 'str',
- 'tls-hostname': 'str'} }
+ 'tls-hostname': 'str',
+ 'tls-acl': 'str'} }
##
# @query-migrate-parameters
#
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 10/11] chardev: add support for ACLs for TLS clients
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (8 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 09/11] migration: add support for a "tls-acl" migration parameter Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 11/11] vnc: allow specifying a custom ACL object name Daniel P. Berrange
2016-06-08 11:53 ` [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
Currently any client which can complete the TLS handshake
is able to use a chardev server. The server admin can turn
on the 'verify-peer' option for the x509 creds to require
the client to provide a x509 certificate. This means the
client will have to acquire a certificate from the CA before
they are permitted to use the chardev server. This is still
a fairly weak bar.
This adds a 'tls-acl=ACL-ID' option to the socket chardev
backend which takes the ID of a previously added 'QAuthZ'
object instance. This ACL will be used to validate the client's
x509 distinguished name. Clients failing the ACL will not be
permitted to use the chardev server.
For example to setup an ACL that only allows connection from
a client whose x509 certificate distinguished name contains
'CN=fred', you would use:
$QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
endpoint=server,verify-peer=yes \
-object authz-simple,id=acl0,policy=deny,\
rules.0.match=\*CN=fred,rules.0.policy=allow \
-chardev socket,host=127.0.0.1,port=9000,server,\
tls-creds=tls0,tls-acl=acl0 \
...other qemud args...
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
qapi-schema.json | 2 ++
qemu-char.c | 11 ++++++++++-
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/qapi-schema.json b/qapi-schema.json
index e7ec2a1..b231e2f 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3293,6 +3293,7 @@
# @addr: socket address to listen on (server=true)
# or connect to (server=false)
# @tls-creds: #optional the ID of the TLS credentials object (since 2.6)
+# @tls-acl: #optional the ID of the QAuthZ authorization object (since 2.6)
# @server: #optional create server socket (default: true)
# @wait: #optional wait for incoming connection on server
# sockets (default: false).
@@ -3308,6 +3309,7 @@
##
{ 'struct': 'ChardevSocket', 'data': { 'addr' : 'SocketAddress',
'*tls-creds' : 'str',
+ '*tls-acl' : 'str',
'*server' : 'bool',
'*wait' : 'bool',
'*nodelay' : 'bool',
diff --git a/qemu-char.c b/qemu-char.c
index b597ee1..e3ced21 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -2604,6 +2604,7 @@ typedef struct {
QIOChannelSocket *listen_ioc;
guint listen_tag;
QCryptoTLSCreds *tls_creds;
+ char *tls_acl;
int connected;
int max_size;
int do_telnetopt;
@@ -3047,7 +3048,7 @@ static void tcp_chr_tls_init(CharDriverState *chr)
if (s->is_listen) {
tioc = qio_channel_tls_new_server(
s->ioc, s->tls_creds,
- NULL, /* XXX Use an ACL */
+ s->tls_acl,
&err);
} else {
tioc = qio_channel_tls_new_client(
@@ -3169,6 +3170,7 @@ static void tcp_chr_close(CharDriverState *chr)
if (s->tls_creds) {
object_unref(OBJECT(s->tls_creds));
}
+ g_free(s->tls_acl);
if (s->write_msgfds_num) {
g_free(s->write_msgfds);
}
@@ -3668,6 +3670,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
const char *host = qemu_opt_get(opts, "host");
const char *port = qemu_opt_get(opts, "port");
const char *tls_creds = qemu_opt_get(opts, "tls-creds");
+ const char *tls_acl = qemu_opt_get(opts, "tls-acl");
SocketAddress *addr;
ChardevSocket *sock;
@@ -3701,6 +3704,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
sock->has_reconnect = true;
sock->reconnect = reconnect;
sock->tls_creds = g_strdup(tls_creds);
+ sock->tls_acl = g_strdup(tls_acl);
addr = g_new0(SocketAddress, 1);
if (path) {
@@ -4155,6 +4159,9 @@ QemuOptsList qemu_chardev_opts = {
.name = "tls-creds",
.type = QEMU_OPT_STRING,
},{
+ .name = "tls-acl",
+ .type = QEMU_OPT_STRING,
+ },{
.name = "width",
.type = QEMU_OPT_NUMBER,
},{
@@ -4398,6 +4405,7 @@ static CharDriverState *qmp_chardev_open_socket(const char *id,
}
}
}
+ s->tls_acl = g_strdup(sock->tls_acl);
qapi_copy_SocketAddress(&s->addr, sock->addr);
@@ -4461,6 +4469,7 @@ static CharDriverState *qmp_chardev_open_socket(const char *id,
if (s->tls_creds) {
object_unref(OBJECT(s->tls_creds));
}
+ g_free(s->tls_acl);
g_free(s);
qemu_chr_free_common(chr);
return NULL;
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH v5 11/11] vnc: allow specifying a custom ACL object name
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (9 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 10/11] chardev: add support for ACLs for TLS clients Daniel P. Berrange
@ 2016-06-02 16:46 ` Daniel P. Berrange
2016-06-08 11:53 ` [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
11 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-02 16:46 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block, Daniel P. Berrange
The VNC server has historically had support for ACLs to check
both the SASL username and the TLS x509 distinguished name.
The VNC server was responsible for creating the initial ACL,
and the client app was then responsible for populating it with
rules using the HMP 'acl_add' command.
This is not satisfactory for a variety of reasons. There is
no way to populate the ACLs from the command line, users are
forced to use the HMP. With multiple network services all
supporting TLS and ACLs now, it is desirable to be able to
define a single ACL that is referenced by all services.
To address these limitations, two new options are added to the
VNC server CLI. The 'tls-acl' option takes the ID of a QAuthZ
object to use for checking TLS x509 distinguished names, and
the 'sasl-acl' option takes the ID of another object to use for
checking SASL usernames.
In this example, we setup two ACLs. The first allows any client
with a certificate issued by the 'RedHat' organization in the
'London' locality. The second ACL allows clients with either
the 'joe@REDHAT.COM' or 'fred@REDHAT.COM' kerberos usernames.
Both ACLs must pass for the user to be allowed.
$QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
endpoint=server,verify-peer=yes \
-object authz-simple,id=tlsacl0,policy=deny,\
rules.0.match=O=RedHat,,L=London,rules.0.policy=allow \
-object authz-simple,id=saslacl0,policy=deny,\
rules.0.match=fred@REDHAT.COM,rules.0.policy=allow \
rules.0.match=joe@REDHAT.COM,rules.0.policy=allow \
-vnc 0.0.0.0:1,tls-creds=tls0,tls-acl=tlsacl0,
sasl,sasl-acl=saslacl0 \
...other QEMU args...
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
---
ui/vnc.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 64 insertions(+), 13 deletions(-)
diff --git a/ui/vnc.c b/ui/vnc.c
index ba07715..939c3d3 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -3263,6 +3263,12 @@ static QemuOptsList qemu_vnc_opts = {
.name = "acl",
.type = QEMU_OPT_BOOL,
},{
+ .name = "tls-acl",
+ .type = QEMU_OPT_STRING,
+ },{
+ .name = "sasl-acl",
+ .type = QEMU_OPT_STRING,
+ },{
.name = "lossy",
.type = QEMU_OPT_BOOL,
},{
@@ -3485,6 +3491,8 @@ void vnc_display_open(const char *id, Error **errp)
int saslErr;
#endif
int acl = 0;
+ const char *tlsacl;
+ const char *saslacl;
int lock_key_sync = 1;
if (!vs) {
@@ -3672,6 +3680,27 @@ void vnc_display_open(const char *id, Error **errp)
}
}
acl = qemu_opt_get_bool(opts, "acl", false);
+ tlsacl = qemu_opt_get(opts, "tls-acl");
+ if (acl && tlsacl) {
+ error_setg(errp, "'acl' option is mutually exclusive with the "
+ "'tls-acl' options");
+ goto fail;
+ }
+ if (tlsacl && !vs->tlscreds) {
+ error_setg(errp, "'tls-acl' provided but TLS is not enabled");
+ goto fail;
+ }
+
+ saslacl = qemu_opt_get(opts, "sasl-acl");
+ if (acl && saslacl) {
+ error_setg(errp, "'acl' option is mutually exclusive with the "
+ "'sasl-acl' options");
+ goto fail;
+ }
+ if (saslacl && !sasl) {
+ error_setg(errp, "'sasl-acl' provided but SASL auth is not enabled");
+ goto fail;
+ }
share = qemu_opt_get(opts, "share");
if (share) {
@@ -3701,7 +3730,9 @@ void vnc_display_open(const char *id, Error **errp)
vs->non_adaptive = true;
}
- if (acl) {
+ if (tlsacl) {
+ vs->tlsaclname = g_strdup(tlsacl);
+ } else if (acl) {
if (strcmp(vs->id, "default") == 0) {
vs->tlsaclname = g_strdup("vnc.x509dname");
} else {
@@ -3712,19 +3743,39 @@ void vnc_display_open(const char *id, Error **errp)
&error_abort);
}
#ifdef CONFIG_VNC_SASL
- if (acl && sasl) {
- char *aclname;
+ if (sasl) {
+ if (saslacl) {
+ Object *container, *acl;
+ container = object_get_objects_root();
+ acl = object_resolve_path_component(container, saslacl);
+ if (!acl) {
+ error_setg(errp, "Cannot find ACL %s", saslacl);
+ goto fail;
+ }
- if (strcmp(vs->id, "default") == 0) {
- aclname = g_strdup("vnc.username");
- } else {
- aclname = g_strdup_printf("vnc.%s.username", vs->id);
- }
- vs->sasl.acl =
- QAUTHZ(qauthz_simple_new(aclname,
- QAUTHZ_SIMPLE_POLICY_DENY,
- &error_abort));
- g_free(aclname);
+ if (!object_dynamic_cast(acl, TYPE_QAUTHZ)) {
+ error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+ saslacl);
+ goto fail;
+ }
+ vs->sasl.acl = QAUTHZ(acl);
+ } else if (acl) {
+ char *aclname;
+
+ if (strcmp(vs->id, "default") == 0) {
+ aclname = g_strdup("vnc.username");
+ } else {
+ aclname = g_strdup_printf("vnc.%s.username", vs->id);
+ }
+ vs->sasl.acl =
+ QAUTHZ(qauthz_simple_new(aclname,
+ QAUTHZ_SIMPLE_POLICY_DENY,
+ &error_abort));
+ g_free(aclname);
+ }
+ } else if (saslacl) {
+ error_setg(errp, "SASL ACL provided when SASL is disabled");
+ goto fail;
}
#endif
--
2.5.5
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
` (10 preceding siblings ...)
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 11/11] vnc: allow specifying a custom ACL object name Daniel P. Berrange
@ 2016-06-08 11:53 ` Daniel P. Berrange
2016-06-08 14:26 ` Markus Armbruster
11 siblings, 1 reply; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-08 11:53 UTC (permalink / raw)
To: qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
Paolo Bonzini, qemu-block
On Thu, Jun 02, 2016 at 05:46:16PM +0100, Daniel P. Berrange wrote:
> This is a followup of previously posted work in 2.6 cycle:
>
> v1: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg04618.html
> v2: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01454.html
> v3: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02498.html
> v4: https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01661.html
>
> Many years ago I was responsible for adding the 'qemu_acl' type
> and associated HMP commands. Looking back at it now, it is quite
> a poor facility with a couple of bad limitations. First, the
> responsibility for creating the ACLs was left with the QEMU network
> service (VNC server was only thing ever doing it). This meant you
> could not share ACLs across multiple services. Second, there was
> no way to populate ACLs on the command line, you had no choice but
> to use the HMP commands. Third, the API was hardcoded around the
> idea of an in-QEMU implementation, leaving no scope for plugging
> in alternative implementations backed by, for example, LDAP or PAM.
>
> This series introduces a much better authorization API design
> to QEMU that addresses all these problems, and maintains back
> compatibility. It of course is based on the QOM framework, so
> that immediately gives us ability to create objects via the
> CLI, HMP or QMP. There is an abstract base clss "QAuthZ" which
> defines the basic API for QEMU network services to use, and a
> specific implementation "QAuthZ" simple which replicates the
> functionality of 'qemu_acl'. It is thus possible to add other
> impls, without changing any other part of QEMU in the future.
> Finally, the user is responsible for creating the ACL objects,
> so they can have one ACL associated with all their TLS enabled
> network services.
>
> There was only one small problem with this, specifically the
> -object CLI arg and HMP 'object_add' command had no way to let
> the user specify non-scalar properties for objects. eg if an
> object had a property which is a list of structs, you are out
> of luck if you want to create it without using QMP.
>
> Thus the first three patches do some work around QAPI / QOM
> to make it possible to specify non-scalar properties with
> the -object CLI arg and HMP 'object_add' command. See the
> respective patches for illustration of the syntax used. Some
> of Max's recent block patches also depend on the qdict_crumple
> method in patch 1.
Ping, does anyone have any feedback on these first 3 patches
in particular. If we get agreement on those and get them merged,
then we can feed the rest of this series to the individual
subsystem maintainers that are affected, and also unblock
Max's patch series.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types Daniel P. Berrange
@ 2016-06-08 12:01 ` Paolo Bonzini
2016-06-14 14:10 ` Daniel P. Berrange
2016-06-09 14:03 ` Markus Armbruster
1 sibling, 1 reply; 25+ messages in thread
From: Paolo Bonzini @ 2016-06-08 12:01 UTC (permalink / raw)
To: Daniel P. Berrange, qemu-devel
Cc: Eric Blake, Max Reitz, Markus Armbruster, Andreas Färber,
qemu-block
On 02/06/2016 18:46, Daniel P. Berrange wrote:
> Currently the QmpInputVisitor assumes that all scalar
> values are directly represented as their final types.
> ie it assumes an 'int' is using QInt, and a 'bool' is
> using QBool.
>
> This extends it so that QString is optionally permitted
> for any of the non-string scalar types. This behaviour
> is turned on by requesting the 'autocast' flag in the
> constructor.
>
> This makes it possible to use QmpInputVisitor with a
> QDict produced from QemuOpts, where everything is in
> string format.
Perhaps this should instead be a separate QmpStringInputVisitor visitor
that _only_ accepts strings? You can reuse most of the QmpInputVisitor
by putting it in the same file, because the struct and list visitors are
compatible.
I wonder if OptsVisitor could also be replaced by qemu_opts_to_qdict, an
optional crumple step and the QmpStringInputVisitor (self-answer:
probably not because of the magic integer range parsing).
Paolo
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> docs/qapi-code-gen.txt | 2 +-
> include/qapi/qmp-input-visitor.h | 6 +-
> qapi/opts-visitor.c | 1 +
> qapi/qmp-input-visitor.c | 89 ++++++++++++++++++++++------
> qmp.c | 2 +-
> qom/qom-qobject.c | 2 +-
> replay/replay-input.c | 2 +-
> scripts/qapi-commands.py | 2 +-
> tests/check-qnull.c | 2 +-
> tests/test-qmp-commands.c | 2 +-
> tests/test-qmp-input-strict.c | 2 +-
> tests/test-qmp-input-visitor.c | 115 ++++++++++++++++++++++++++++++++++++-
> tests/test-visitor-serialization.c | 2 +-
> util/qemu-sockets.c | 2 +-
> 14 files changed, 201 insertions(+), 30 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index d7d6987..e21773e 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -1008,7 +1008,7 @@ Example:
> {
> Error *err = NULL;
> UserDefOne *retval;
> - QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true);
> + QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true, false);
> QapiDeallocVisitor *qdv;
> Visitor *v;
> UserDefOneList *arg1 = NULL;
> diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qmp-input-visitor.h
> index b0624d8..d35a053 100644
> --- a/include/qapi/qmp-input-visitor.h
> +++ b/include/qapi/qmp-input-visitor.h
> @@ -24,8 +24,12 @@ typedef struct QmpInputVisitor QmpInputVisitor;
> *
> * Set @strict to reject a parse that doesn't consume all keys of a
> * dictionary; otherwise excess input is ignored.
> + * Set @autocast to automatically convert string values into more
> + * specific types (numbers, bools, etc)
> */
> -QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict);
> +QmpInputVisitor *qmp_input_visitor_new(QObject *obj,
> + bool strict,
> + bool autocast);
>
> void qmp_input_visitor_cleanup(QmpInputVisitor *v);
>
> diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
> index 4cf1cf8..00e4b1a 100644
> --- a/qapi/opts-visitor.c
> +++ b/qapi/opts-visitor.c
> @@ -347,6 +347,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
> }
>
> if (opt->str) {
> + /* Keep these values in sync with same code in qmp-input-visitor.c */
> if (strcmp(opt->str, "on") == 0 ||
> strcmp(opt->str, "yes") == 0 ||
> strcmp(opt->str, "y") == 0) {
> diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
> index aea90a1..92023b1 100644
> --- a/qapi/qmp-input-visitor.c
> +++ b/qapi/qmp-input-visitor.c
> @@ -20,6 +20,7 @@
> #include "qemu-common.h"
> #include "qapi/qmp/types.h"
> #include "qapi/qmp/qerror.h"
> +#include "qemu/cutils.h"
>
> #define QIV_STACK_SIZE 1024
>
> @@ -45,6 +46,7 @@ struct QmpInputVisitor
>
> /* True to reject parse in visit_end_struct() if unvisited keys remain. */
> bool strict;
> + bool autocast;
> };
>
> static QmpInputVisitor *to_qiv(Visitor *v)
> @@ -254,15 +256,25 @@ static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
> Error **errp)
> {
> QmpInputVisitor *qiv = to_qiv(v);
> - QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
> + QObject *qobj = qmp_input_get_object(qiv, name, true);
> + QInt *qint;
> + QString *qstr;
>
> - if (!qint) {
> - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> - "integer");
> + qint = qobject_to_qint(qobj);
> + if (qint) {
> + *obj = qint_get_int(qint);
> return;
> }
>
> - *obj = qint_get_int(qint);
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + if (qemu_strtoll(qstr->string, NULL, 10, obj) == 0) {
> + return;
> + }
> + }
> +
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "integer");
> }
>
> static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
> @@ -270,30 +282,61 @@ static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
> {
> /* FIXME: qobject_to_qint mishandles values over INT64_MAX */
> QmpInputVisitor *qiv = to_qiv(v);
> - QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
> + QObject *qobj = qmp_input_get_object(qiv, name, true);
> + QInt *qint;
> + QString *qstr;
>
> - if (!qint) {
> - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> - "integer");
> + qint = qobject_to_qint(qobj);
> + if (qint) {
> + *obj = qint_get_int(qint);
> return;
> }
>
> - *obj = qint_get_int(qint);
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + if (qemu_strtoull(qstr->string, NULL, 10, obj) == 0) {
> + return;
> + }
> + }
> +
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "integer");
> }
>
> static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
> Error **errp)
> {
> QmpInputVisitor *qiv = to_qiv(v);
> - QBool *qbool = qobject_to_qbool(qmp_input_get_object(qiv, name, true));
> + QObject *qobj = qmp_input_get_object(qiv, name, true);
> + QBool *qbool;
> + QString *qstr;
>
> - if (!qbool) {
> - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> - "boolean");
> + qbool = qobject_to_qbool(qobj);
> + if (qbool) {
> + *obj = qbool_get_bool(qbool);
> return;
> }
>
> - *obj = qbool_get_bool(qbool);
> +
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + /* Keep these values in sync with same code in opts-visitor.c */
> + if (!strcasecmp(qstr->string, "on") ||
> + !strcasecmp(qstr->string, "yes") ||
> + !strcasecmp(qstr->string, "true")) {
> + *obj = true;
> + return;
> + }
> + if (!strcasecmp(qstr->string, "off") ||
> + !strcasecmp(qstr->string, "no") ||
> + !strcasecmp(qstr->string, "false")) {
> + *obj = false;
> + return;
> + }
> + }
> +
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "boolean");
> }
>
> static void qmp_input_type_str(Visitor *v, const char *name, char **obj,
> @@ -319,6 +362,8 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
> QObject *qobj = qmp_input_get_object(qiv, name, true);
> QInt *qint;
> QFloat *qfloat;
> + QString *qstr;
> + char *endp;
>
> qint = qobject_to_qint(qobj);
> if (qint) {
> @@ -332,6 +377,15 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
> return;
> }
>
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + errno = 0;
> + *obj = strtod(qstr->string, &endp);
> + if (errno == 0 && endp != qstr->string && *endp == '\0') {
> + return;
> + }
> + }
> +
> error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> "number");
> }
> @@ -381,7 +435,9 @@ void qmp_input_visitor_cleanup(QmpInputVisitor *v)
> g_free(v);
> }
>
> -QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict)
> +QmpInputVisitor *qmp_input_visitor_new(QObject *obj,
> + bool strict,
> + bool autocast)
> {
> QmpInputVisitor *v;
>
> @@ -404,6 +460,7 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict)
> v->visitor.type_null = qmp_input_type_null;
> v->visitor.optional = qmp_input_optional;
> v->strict = strict;
> + v->autocast = autocast;
>
> v->root = obj;
> qobject_incref(obj);
> diff --git a/qmp.c b/qmp.c
> index 3165f87..dcd0953 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -665,7 +665,7 @@ void qmp_object_add(const char *type, const char *id,
> }
> }
>
> - qiv = qmp_input_visitor_new(props, true);
> + qiv = qmp_input_visitor_new(props, true, false);
> obj = user_creatable_add_type(type, id, pdict,
> qmp_input_get_visitor(qiv), errp);
> qmp_input_visitor_cleanup(qiv);
> diff --git a/qom/qom-qobject.c b/qom/qom-qobject.c
> index b66088d..99666ce 100644
> --- a/qom/qom-qobject.c
> +++ b/qom/qom-qobject.c
> @@ -23,7 +23,7 @@ void object_property_set_qobject(Object *obj, QObject *value,
> {
> QmpInputVisitor *qiv;
> /* TODO: Should we reject, rather than ignore, excess input? */
> - qiv = qmp_input_visitor_new(value, false);
> + qiv = qmp_input_visitor_new(value, false, false);
> object_property_set(obj, qmp_input_get_visitor(qiv), name, errp);
>
> qmp_input_visitor_cleanup(qiv);
> diff --git a/replay/replay-input.c b/replay/replay-input.c
> index 03e99d5..de82a59 100644
> --- a/replay/replay-input.c
> +++ b/replay/replay-input.c
> @@ -37,7 +37,7 @@ static InputEvent *qapi_clone_InputEvent(InputEvent *src)
> return NULL;
> }
>
> - qiv = qmp_input_visitor_new(obj, true);
> + qiv = qmp_input_visitor_new(obj, true, false);
> iv = qmp_input_get_visitor(qiv);
> visit_type_InputEvent(iv, NULL, &dst, &error_abort);
> qmp_input_visitor_cleanup(qiv);
> diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
> index 8c6acb3..e48995d 100644
> --- a/scripts/qapi-commands.py
> +++ b/scripts/qapi-commands.py
> @@ -115,7 +115,7 @@ def gen_marshal(name, arg_type, ret_type):
>
> if arg_type and arg_type.members:
> ret += mcgen('''
> - QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true);
> + QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true, false);
> QapiDeallocVisitor *qdv;
> Visitor *v;
> %(c_name)s arg = {0};
> diff --git a/tests/check-qnull.c b/tests/check-qnull.c
> index fd9c68f..4c11755 100644
> --- a/tests/check-qnull.c
> +++ b/tests/check-qnull.c
> @@ -49,7 +49,7 @@ static void qnull_visit_test(void)
>
> g_assert(qnull_.refcnt == 1);
> obj = qnull();
> - qiv = qmp_input_visitor_new(obj, true);
> + qiv = qmp_input_visitor_new(obj, true, false);
> qobject_decref(obj);
> visit_type_null(qmp_input_get_visitor(qiv), NULL, &error_abort);
> qmp_input_visitor_cleanup(qiv);
> diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
> index 5c3edd7..c86b282 100644
> --- a/tests/test-qmp-commands.c
> +++ b/tests/test-qmp-commands.c
> @@ -222,7 +222,7 @@ static void test_dealloc_partial(void)
> ud2_dict = qdict_new();
> qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
>
> - qiv = qmp_input_visitor_new(QOBJECT(ud2_dict), true);
> + qiv = qmp_input_visitor_new(QOBJECT(ud2_dict), true, false);
> visit_type_UserDefTwo(qmp_input_get_visitor(qiv), NULL, &ud2, &err);
> qmp_input_visitor_cleanup(qiv);
> QDECREF(ud2_dict);
> diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
> index 4602529..f7f1f00 100644
> --- a/tests/test-qmp-input-strict.c
> +++ b/tests/test-qmp-input-strict.c
> @@ -55,7 +55,7 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
> data->obj = qobject_from_jsonv(json_string, ap);
> g_assert(data->obj);
>
> - data->qiv = qmp_input_visitor_new(data->obj, true);
> + data->qiv = qmp_input_visitor_new(data->obj, true, false);
> g_assert(data->qiv);
>
> v = qmp_input_get_visitor(data->qiv);
> diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
> index cee07ce..5691dc3 100644
> --- a/tests/test-qmp-input-visitor.c
> +++ b/tests/test-qmp-input-visitor.c
> @@ -41,6 +41,7 @@ static void visitor_input_teardown(TestInputVisitorData *data,
> function so that the JSON string used by the tests are kept in the test
> functions (and not in main()). */
> static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> + bool strict, bool autocast,
> const char *json_string,
> va_list *ap)
> {
> @@ -51,7 +52,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> data->obj = qobject_from_jsonv(json_string, ap);
> g_assert(data->obj);
>
> - data->qiv = qmp_input_visitor_new(data->obj, false);
> + data->qiv = qmp_input_visitor_new(data->obj, strict, autocast);
> g_assert(data->qiv);
>
> v = qmp_input_get_visitor(data->qiv);
> @@ -60,6 +61,21 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> return v;
> }
>
> +static GCC_FMT_ATTR(4, 5)
> +Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
> + bool strict, bool autocast,
> + const char *json_string, ...)
> +{
> + Visitor *v;
> + va_list ap;
> +
> + va_start(ap, json_string);
> + v = visitor_input_test_init_internal(data, strict, autocast,
> + json_string, &ap);
> + va_end(ap);
> + return v;
> +}
> +
> static GCC_FMT_ATTR(2, 3)
> Visitor *visitor_input_test_init(TestInputVisitorData *data,
> const char *json_string, ...)
> @@ -68,7 +84,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> va_list ap;
>
> va_start(ap, json_string);
> - v = visitor_input_test_init_internal(data, json_string, &ap);
> + v = visitor_input_test_init_internal(data, false, false,
> + json_string, &ap);
> va_end(ap);
> return v;
> }
> @@ -83,7 +100,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
> const char *json_string)
> {
> - return visitor_input_test_init_internal(data, json_string, NULL);
> + return visitor_input_test_init_internal(data, false, false,
> + json_string, NULL);
> }
>
> static void test_visitor_in_int(TestInputVisitorData *data,
> @@ -115,6 +133,33 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
> error_free_or_abort(&err);
> }
>
> +static void test_visitor_in_int_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + int64_t res = 0, value = -42;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, false, true,
> + "\"-42\"");
> +
> + visit_type_int(v, NULL, &res, &error_abort);
> + g_assert_cmpint(res, ==, value);
> +}
> +
> +static void test_visitor_in_int_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + int64_t res = 0;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"-42\"");
> +
> + visit_type_int(v, NULL, &res, &err);
> + g_assert(err != NULL);
> + error_free(err);
> +}
> +
> static void test_visitor_in_bool(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -127,6 +172,32 @@ static void test_visitor_in_bool(TestInputVisitorData *data,
> g_assert_cmpint(res, ==, true);
> }
>
> +static void test_visitor_in_bool_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + bool res = false;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, false, true, "\"true\"");
> +
> + visit_type_bool(v, NULL, &res, &error_abort);
> + g_assert_cmpint(res, ==, true);
> +}
> +
> +static void test_visitor_in_bool_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + bool res = false;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"true\"");
> +
> + visit_type_bool(v, NULL, &res, &err);
> + g_assert(err != NULL);
> + error_free(err);
> +}
> +
> static void test_visitor_in_number(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -139,6 +210,32 @@ static void test_visitor_in_number(TestInputVisitorData *data,
> g_assert_cmpfloat(res, ==, value);
> }
>
> +static void test_visitor_in_number_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + double res = 0, value = 3.14;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, false, true, "\"3.14\"");
> +
> + visit_type_number(v, NULL, &res, &error_abort);
> + g_assert_cmpfloat(res, ==, value);
> +}
> +
> +static void test_visitor_in_number_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + double res = 0;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"3.14\"");
> +
> + visit_type_number(v, NULL, &res, &err);
> + g_assert(err != NULL);
> + error_free(err);
> +}
> +
> static void test_visitor_in_string(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -835,10 +932,22 @@ int main(int argc, char **argv)
> &in_visitor_data, test_visitor_in_int);
> input_visitor_test_add("/visitor/input/int_overflow",
> &in_visitor_data, test_visitor_in_int_overflow);
> + input_visitor_test_add("/visitor/input/int_autocast",
> + &in_visitor_data, test_visitor_in_int_autocast);
> + input_visitor_test_add("/visitor/input/int_noautocast",
> + &in_visitor_data, test_visitor_in_int_noautocast);
> input_visitor_test_add("/visitor/input/bool",
> &in_visitor_data, test_visitor_in_bool);
> + input_visitor_test_add("/visitor/input/bool_autocast",
> + &in_visitor_data, test_visitor_in_bool_autocast);
> + input_visitor_test_add("/visitor/input/bool_noautocast",
> + &in_visitor_data, test_visitor_in_bool_noautocast);
> input_visitor_test_add("/visitor/input/number",
> &in_visitor_data, test_visitor_in_number);
> + input_visitor_test_add("/visitor/input/number_autocast",
> + &in_visitor_data, test_visitor_in_number_autocast);
> + input_visitor_test_add("/visitor/input/number_noautocast",
> + &in_visitor_data, test_visitor_in_number_noautocast);
> input_visitor_test_add("/visitor/input/string",
> &in_visitor_data, test_visitor_in_string);
> input_visitor_test_add("/visitor/input/enum",
> diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
> index 7b14b5a..db618d8 100644
> --- a/tests/test-visitor-serialization.c
> +++ b/tests/test-visitor-serialization.c
> @@ -1038,7 +1038,7 @@ static void qmp_deserialize(void **native_out, void *datap,
> obj = qobject_from_json(qstring_get_str(output_json));
>
> QDECREF(output_json);
> - d->qiv = qmp_input_visitor_new(obj, true);
> + d->qiv = qmp_input_visitor_new(obj, true, false);
> qobject_decref(obj_orig);
> qobject_decref(obj);
> visit(qmp_input_get_visitor(d->qiv), native_out, errp);
> diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
> index 0d6cd1f..51c6a8e 100644
> --- a/util/qemu-sockets.c
> +++ b/util/qemu-sockets.c
> @@ -1145,7 +1145,7 @@ void qapi_copy_SocketAddress(SocketAddress **p_dest,
> return;
> }
>
> - qiv = qmp_input_visitor_new(obj, true);
> + qiv = qmp_input_visitor_new(obj, true, false);
> iv = qmp_input_get_visitor(qiv);
> visit_type_SocketAddress(iv, NULL, p_dest, &error_abort);
> qmp_input_visitor_cleanup(qiv);
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API
2016-06-08 11:53 ` [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
@ 2016-06-08 14:26 ` Markus Armbruster
0 siblings, 0 replies; 25+ messages in thread
From: Markus Armbruster @ 2016-06-08 14:26 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Ping, does anyone have any feedback on these first 3 patches
> in particular. If we get agreement on those and get them merged,
> then we can feed the rest of this series to the individual
> subsystem maintainers that are affected, and also unblock
> Max's patch series.
I'll try to get to them this week.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
@ 2016-06-09 13:20 ` Markus Armbruster
2016-06-09 13:28 ` Daniel P. Berrange
2016-06-14 11:39 ` Daniel P. Berrange
0 siblings, 2 replies; 25+ messages in thread
From: Markus Armbruster @ 2016-06-09 13:20 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
I apologize for the lateness of this review.
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The qdict_flatten() method will take a dict whose elements are
> further nested dicts/lists and flatten them by concatenating
> keys.
>
> The qdict_crumple() method aims to do the reverse, taking a flat
> qdict, and turning it into a set of nested dicts/lists. It will
> apply nesting based on the key name, with a '.' indicating a
> new level in the hierarchy. If the keys in the nested structure
> are all numeric, it will create a list, otherwise it will create
> a dict.
>
> If the keys are a mixture of numeric and non-numeric, or the
> numeric keys are not in strictly ascending order, an error will
> be reported.
>
> As an example, a flat dict containing
>
> {
> 'foo.0.bar': 'one',
> 'foo.0.wizz': '1',
> 'foo.1.bar': 'two',
> 'foo.1.wizz': '2'
> }
>
> will get turned into a dict with one element 'foo' whose
> value is a list. The list elements will each in turn be
> dicts.
>
> {
> 'foo': [
> { 'bar': 'one', 'wizz': '1' },
> { 'bar': 'two', 'wizz': '2' }
> ],
> }
>
> If the key is intended to contain a literal '.', then it must
> be escaped as '..'. ie a flat dict
>
> {
> 'foo..bar': 'wizz',
> 'bar.foo..bar': 'eek',
> 'bar.hello': 'world'
> }
>
> Will end up as
>
> {
> 'foo.bar': 'wizz',
> 'bar': {
> 'foo.bar': 'eek',
> 'hello': 'world'
> }
> }
>
> The intent of this function is that it allows a set of QemuOpts
> to be turned into a nested data structure that mirrors the nesting
> used when the same object is defined over QMP.
>
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Not an objection to your patch, just an observation: we're digging
ourselves ever deeper into the QemuOpts / QDict hole. We've pushed
QemuOpts far beyond its original purpose "parse key=value,... option
arguments". In my opinion, we'd better parse straight to QAPI-generated
structs.
> ---
> include/qapi/qmp/qdict.h | 1 +
> qobject/qdict.c | 282 +++++++++++++++++++++++++++++++++++++++++++++++
> tests/check-qdict.c | 229 ++++++++++++++++++++++++++++++++++++++
> 3 files changed, 512 insertions(+)
>
> diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
> index 71b8eb0..8a3ac13 100644
> --- a/include/qapi/qmp/qdict.h
> +++ b/include/qapi/qmp/qdict.h
> @@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
> void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
> void qdict_array_split(QDict *src, QList **dst);
> int qdict_array_entries(QDict *src, const char *subqdict);
> +QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
>
> void qdict_join(QDict *dest, QDict *src, bool overwrite);
>
> diff --git a/qobject/qdict.c b/qobject/qdict.c
> index 60f158c..ad6a563 100644
> --- a/qobject/qdict.c
> +++ b/qobject/qdict.c
> @@ -17,6 +17,7 @@
> #include "qapi/qmp/qbool.h"
> #include "qapi/qmp/qstring.h"
> #include "qapi/qmp/qobject.h"
> +#include "qapi/error.h"
> #include "qemu/queue.h"
> #include "qemu-common.h"
> #include "qemu/cutils.h"
> @@ -683,6 +684,287 @@ void qdict_array_split(QDict *src, QList **dst)
> }
> }
>
> +
> +/**
> + * qdict_split_flat_key:
> + * @key: the key string to split
> + * @prefix: non-NULL pointer to hold extracted prefix
> + * @suffix: non-NULL pointer to hold extracted suffix
> + *
> + * Given a flattened key such as 'foo.0.bar', split it
> + * into two parts at the first '.' separator. Allows
> + * double dot ('..') to escape the normal separator.
> + *
> + * eg
> + * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
> + * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
> + *
> + * The '..' sequence will be unescaped in the returned
> + * 'prefix' string. The 'suffix' string will be left
> + * in escaped format, so it can be fed back into the
> + * qdict_split_flat_key() key as the input later.
Why is the suffix strdup'ed then?
> + *
> + * The caller is responsible for freeing the strings
> + * returned in @prefix and @suffix using g_free().
Wrap comment lines around column 70, please.
> + */
> +static void qdict_split_flat_key(const char *key, char **prefix, char **suffix)
> +{
> + const char *separator;
> + size_t i, j;
> +
> + /* Find first '.' separator, but if there is a pair '..'
> + * that acts as an escape, so skip over '..' */
> + separator = NULL;
> + do {
> + if (separator) {
> + separator += 2;
> + } else {
> + separator = key;
> + }
> + separator = strchr(separator, '.');
> + } while (separator && separator[1] == '.');
> +
> + if (separator) {
> + *prefix = g_strndup(key,
> + separator - key);
> + *suffix = g_strdup(separator + 1);
> + } else {
> + *prefix = g_strdup(key);
> + *suffix = NULL;
> + }
> +
> + /* Unescape the '..' sequence into '.' */
> + for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
> + if ((*prefix)[i] == '.') {
> + assert((*prefix)[i + 1] == '.');
> + i++;
> + }
> + (*prefix)[j] = (*prefix)[i];
> + }
> + (*prefix)[j] = '\0';
Runs through the key twice, once in g_strndup(), and once here. Would
be easy enough to avoid, but since this function isn't performance
critical, it's okay as is.
> +}
> +
> +
> +/**
> + * qdict_list_size:
> + * @maybe_list: dict that may be only list elements
Huh? How can a dictionary "be only list elements"?
Do you mean "the dictionary to test?"
> + *
> + * Determine whether all keys in @maybe_list are
> + * valid list elements. If they are all valid,
> + * then this returns the number of elements. If
> + * they all look like non-numeric keys, then returns
> + * zero. If there is a mix of numeric and non-numeric
> + * keys, then an error is set as it is both a list
> + * and a dict at once.
This is well-defined only if empty @maybe_list is considered to have
dict nature, not list nature. Else, return value zero could be the
length of the empty list or the special "has dict nature" value.
Please spell out behavior for empty @maybe_list.
> + *
> + * Returns: number of list elements, 0 if a dict, -1 on error
Awkward function name. qdict_list_size_if_list() would be clear.
But I'd simply turn this into a predicate qdict_is_list(), and have the
caller use qdict_size() to get the number of elements.
> + */
> +static ssize_t qdict_list_size(QDict *maybe_list, Error **errp)
> +{
> + const QDictEntry *entry, *next;
> + ssize_t len = 0;
> + ssize_t max = -1;
> + int is_list = -1;
> + int64_t val;
> +
> + entry = qdict_first(maybe_list);
> + while (entry != NULL) {
> + next = qdict_next(maybe_list, entry);
Please keep the loop control in one place:
for (entry = qdict_first(maybe_list); entry; entry = qdict_next(entry)) {
I'd rename some variables for less verbiage:
for (ent = qdict_first(dict); ent; ent = qdict_next(ent)) {
Your loop control also works when the loop body deletes @entry from
@maybe_list. Seeing such loop control in a function that isn't supposed
to change the its argument makes the reviewer go "WTF?!?" :)
> +
> + if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
> + if (is_list == -1) {
> + is_list = 1;
> + } else if (!is_list) {
> + error_setg(errp,
> + "Cannot crumple a dictionary with a mix of list "
> + "and non-list keys");
> + return -1;
> + }
> + len++;
> + if (val > max) {
> + max = val;
> + }
> + } else {
> + if (is_list == -1) {
> + is_list = 0;
> + } else if (is_list) {
> + error_setg(errp,
> + "Cannot crumple a dictionary with a mix of list "
> + "and non-list keys");
> + return -1;
> + }
> + }
> +
> + entry = next;
> + }
> +
> + if (is_list == -1) {
> + is_list = 0;
This can happen only when @maybe_list is empty. Okay, but perhaps you'd
like to assert(!qdict_size(maybe_list)).
> + }
> +
> + if (len != (max + 1)) {
> + error_setg(errp, "List indexes are not continuous, "
> + "saw %zd elements but %zd largest index",
> + len, max);
> + return -1;
contiguous?
What if we saw indexes 0, 2, 2?
Hmm, see [*] below.
> + }
> +
> + return is_list ? len : 0;
> +}
> +
> +/**
> + * qdict_crumple:
> + * @src: the original flat dictionary to crumple
"Flat" means all values are scalar. Should we spell that out?
> + * @recursive: true to recursively crumple nested dictionaries
> + *
> + * Takes a flat dictionary whose keys use '.' separator to
> + * indicate nesting, and values are scalars, crumplings it
s/, crumplings/, and crumples/
> + * into a nested structure. If the @recursive parameter is
> + * false, then only the first level of structure implied
> + * by the keys will be crumpled. If @recursive is true,
> + * then the input will be recursively crumpled to expand
> + * all levels of structure in the keys.
> + *
> + * To include a literal '.' in a key name, it must be escaped
> + * as '..'
> + *
> + * For example, an input of:
> + *
> + * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
> + * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
> + *
> + * will result in any output of:
> + *
> + * {
> + * 'foo': [
> + * { 'bar': 'one', 'wizz': '1' },
> + * { 'bar': 'two', 'wizz': '2' }
> + * ],
> + * }
> + *
> + * Returns: either a QDict or QList for the nested data structure
I think you should discuss how this can fail.
> + */
> +QObject *qdict_crumple(QDict *src, bool recursive, Error **errp)
> +{
> + const QDictEntry *entry, *next;
> + QDict *two_level, *multi_level = NULL;
> + QObject *dst = NULL, *child;
> + ssize_t list_len;
> + size_t i;
> + char *prefix = NULL, *suffix = NULL;
> +
> + two_level = qdict_new();
> + entry = qdict_first(src);
> +
> + /* Step 1: split our totally flat dict into a two level dict */
> + while (entry != NULL) {
> + next = qdict_next(src, entry);
Again, keep the loop control in one place.
> +
> + if (qobject_type(entry->value) == QTYPE_QDICT ||
> + qobject_type(entry->value) == QTYPE_QLIST) {
> + error_setg(errp, "Value %s is not a scalar",
> + entry->key);
> + goto error;
> + }
> +
> + qdict_split_flat_key(entry->key, &prefix, &suffix);
> +
> + child = qdict_get(two_level, prefix);
> + if (suffix) {
> + if (child) {
> + if (qobject_type(child) != QTYPE_QDICT) {
> + error_setg(errp, "Key %s prefix is already set as a scalar",
> + prefix);
> + goto error;
> + }
> + } else {
> + child = QOBJECT(qdict_new());
> + qdict_put_obj(two_level, prefix, child);
> + }
> + qobject_incref(entry->value);
> + qdict_put_obj(qobject_to_qdict(child), suffix, entry->value);
> + } else {
> + if (child) {
> + error_setg(errp, "Key %s prefix is already set as a dict",
> + prefix);
> + goto error;
> + }
> + qobject_incref(entry->value);
> + qdict_put_obj(two_level, prefix, entry->value);
> + }
Works, because we put only QDicts we've created ourselves (first
qdict_put_obj() above) and values we got from @src (second
qdict_put_obj()), and we fail when such a value isn't scalar.
> +
> + g_free(suffix);
As I suspected, qdict_split_flat_key() strdup'ing the suffix is useless.
> + g_free(prefix);
> + suffix = prefix = NULL;
Dead stores.
> +
> + entry = next;
> + }
> +
> + /* Step 2: optionally process the two level dict recursively
> + * into a multi-level dict */
> + if (recursive) {
> + multi_level = qdict_new();
> + entry = qdict_first(two_level);
> + while (entry != NULL) {
> + next = qdict_next(two_level, entry);
Again, keep the loop control in one place.
> +
> + if (qobject_type(entry->value) == QTYPE_QDICT) {
> + child = qdict_crumple(qobject_to_qdict(entry->value),
> + recursive, errp);
> + if (!child) {
> + goto error;
> + }
> +
> + qdict_put_obj(multi_level, entry->key, child);
> + } else {
> + qobject_incref(entry->value);
> + qdict_put_obj(multi_level, entry->key, entry->value);
> + }
> +
> + entry = next;
> + }
> + QDECREF(two_level);
> + } else {
> + multi_level = two_level;
> + }
> + two_level = NULL;
> +
> + /* Step 3: detect if we need to turn our dict into list */
> + list_len = qdict_list_size(multi_level, errp);
> + if (list_len < 0) {
> + goto error;
> + }
> +
> + if (list_len) {
> + dst = QOBJECT(qlist_new());
> +
> + for (i = 0; i < list_len; i++) {
> + char *key = g_strdup_printf("%zu", i);
> +
> + child = qdict_get(multi_level, key);
> + g_free(key);
> + assert(child);
qdict_list_size() accepts as list index any (string) key qemu_strtoll()
accepts. If %zu formats it back into the same string, we'll find it
here. Else we die. Please spell this out in the function contract.
[*] I'm afraid we also die if qdict_list_size()'s "List indexes are not
continuous" check gets fooled. Suggest to drop that check, and replace
this assertion by error_setg(errp, "Malformed list indexes").
Admittedly not the nicest error message; perhaps you can come up with a
better one.
> +
> + qobject_incref(child);
> + qlist_append_obj(qobject_to_qlist(dst), child);
> + }
> + QDECREF(multi_level);
Do we need
multi_level = NULL;
here?
> + } else {
> + dst = QOBJECT(multi_level);
> + }
> +
> + return dst;
> +
> + error:
> + g_free(suffix);
> + g_free(prefix);
> + QDECREF(multi_level);
> + QDECREF(two_level);
> + qobject_decref(dst);
> + return NULL;
> +}
> +
> +
> /**
> * qdict_array_entries(): Returns the number of direct array entries if the
> * sub-QDict of src specified by the prefix in subqdict (or src itself for
> diff --git a/tests/check-qdict.c b/tests/check-qdict.c
> index a43056c..0d12f40 100644
> --- a/tests/check-qdict.c
> +++ b/tests/check-qdict.c
> @@ -15,6 +15,7 @@
> #include "qapi/qmp/qint.h"
> #include "qapi/qmp/qdict.h"
> #include "qapi/qmp/qstring.h"
> +#include "qapi/error.h"
> #include "qemu-common.h"
>
> /*
> @@ -596,6 +597,223 @@ static void qdict_join_test(void)
> QDECREF(dict2);
> }
>
> +
> +static void qdict_crumple_test_nonrecursive(void)
> +{
> + QDict *src, *dst, *rules, *vnc;
> + QObject *child, *res;
> +
> + src = qdict_new();
> + qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
> + qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
> + qdict_put(src, "rule.0.match", qstring_from_str("fred"));
> + qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
> + qdict_put(src, "rule.1.match", qstring_from_str("bob"));
> + qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
> +
> + res = qdict_crumple(src, false, &error_abort);
> +
> + g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
> +
> + dst = qobject_to_qdict(res);
> +
> + g_assert_cmpint(qdict_size(dst), ==, 2);
> +
> + child = qdict_get(dst, "vnc");
> + g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
> + vnc = qdict_get_qdict(dst, "vnc");
> +
> + g_assert_cmpint(qdict_size(vnc), ==, 2);
> +
> + g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(vnc, "listen.addr"));
> + g_assert_cmpstr("5901", ==, qdict_get_str(vnc, "listen.port"));
> +
> + child = qdict_get(dst, "rule");
> + g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
> + rules = qdict_get_qdict(dst, "rule");
> +
> + g_assert_cmpint(qdict_size(rules), ==, 4);
> +
> + g_assert_cmpstr("fred", ==, qdict_get_str(rules, "0.match"));
> + g_assert_cmpstr("allow", ==, qdict_get_str(rules, "0.policy"));
> + g_assert_cmpstr("bob", ==, qdict_get_str(rules, "1.match"));
> + g_assert_cmpstr("deny", ==, qdict_get_str(rules, "1.policy"));
> +
> + QDECREF(src);
> + QDECREF(dst);
> +}
> +
> +
> +static void qdict_crumple_test_listtop(void)
> +{
> + QDict *src, *rule;
> + QList *rules;
> + QObject *res;
> +
> + src = qdict_new();
> + qdict_put(src, "0.match.name", qstring_from_str("Fred"));
> + qdict_put(src, "0.match.email", qstring_from_str("fred@example.com"));
> + qdict_put(src, "0.policy", qstring_from_str("allow"));
> + qdict_put(src, "1.match.name", qstring_from_str("Bob"));
> + qdict_put(src, "1.match.email", qstring_from_str("bob@example.com"));
> + qdict_put(src, "1.policy", qstring_from_str("deny"));
> +
> + res = qdict_crumple(src, false, &error_abort);
> +
> + g_assert_cmpint(qobject_type(res), ==, QTYPE_QLIST);
> +
> + rules = qobject_to_qlist(res);
> +
> + g_assert_cmpint(qlist_size(rules), ==, 2);
> +
> + g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
> + rule = qobject_to_qdict(qlist_pop(rules));
> + g_assert_cmpint(qdict_size(rule), ==, 3);
> +
> + g_assert_cmpstr("Fred", ==, qdict_get_str(rule, "match.name"));
> + g_assert_cmpstr("fred@example.com", ==, qdict_get_str(rule, "match.email"));
> + g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
> + QDECREF(rule);
> +
> + g_assert_cmpint(qobject_type(qlist_peek(rules)), ==, QTYPE_QDICT);
> + rule = qobject_to_qdict(qlist_pop(rules));
> + g_assert_cmpint(qdict_size(rule), ==, 3);
> +
> + g_assert_cmpstr("Bob", ==, qdict_get_str(rule, "match.name"));
> + g_assert_cmpstr("bob@example.com", ==, qdict_get_str(rule, "match.email"));
> + g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
> + QDECREF(rule);
> +
> + QDECREF(src);
> + qobject_decref(res);
> +}
> +
> +
> +static void qdict_crumple_test_recursive(void)
> +{
> + QDict *src, *dst, *rule, *vnc, *acl, *listen;
> + QObject *child, *res;
> + QList *rules;
> +
> + src = qdict_new();
> + qdict_put(src, "vnc.listen.addr", qstring_from_str("127.0.0.1"));
> + qdict_put(src, "vnc.listen.port", qstring_from_str("5901"));
> + qdict_put(src, "vnc.acl.rules.0.match", qstring_from_str("fred"));
> + qdict_put(src, "vnc.acl.rules.0.policy", qstring_from_str("allow"));
> + qdict_put(src, "vnc.acl.rules.1.match", qstring_from_str("bob"));
> + qdict_put(src, "vnc.acl.rules.1.policy", qstring_from_str("deny"));
> + qdict_put(src, "vnc.acl.default", qstring_from_str("deny"));
> +
> + res = qdict_crumple(src, true, &error_abort);
> +
> + g_assert_cmpint(qobject_type(res), ==, QTYPE_QDICT);
> +
> + dst = qobject_to_qdict(res);
> +
> + g_assert_cmpint(qdict_size(dst), ==, 1);
> +
> + child = qdict_get(dst, "vnc");
> + g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
> + vnc = qobject_to_qdict(child);
> +
> + child = qdict_get(vnc, "listen");
> + g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
> + listen = qobject_to_qdict(child);
> + g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr"));
> + g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port"));
> +
> + child = qdict_get(vnc, "acl");
> + g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
> + acl = qobject_to_qdict(child);
> +
> + child = qdict_get(acl, "rules");
> + g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
> + rules = qobject_to_qlist(child);
> + g_assert_cmpint(qlist_size(rules), ==, 2);
> +
> + rule = qobject_to_qdict(qlist_pop(rules));
> + g_assert_cmpint(qdict_size(rule), ==, 2);
> + g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
> + g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
> + QDECREF(rule);
> +
> + rule = qobject_to_qdict(qlist_pop(rules));
> + g_assert_cmpint(qdict_size(rule), ==, 2);
> + g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
> + g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
> + QDECREF(rule);
> +
> + QDECREF(src);
> + QDECREF(dst);
> +}
> +
> +
> +static void qdict_crumple_test_empty(void)
> +{
> + QDict *src, *dst;
> +
> + src = qdict_new();
> +
> + dst = (QDict *)qdict_crumple(src, true, &error_abort);
> +
> + g_assert_cmpint(qdict_size(dst), ==, 0);
> +
> + QDECREF(src);
> + QDECREF(dst);
> +}
> +
> +
> +static void qdict_crumple_test_bad_inputs(void)
> +{
> + QDict *src;
> + Error *error = NULL;
> +
> + src = qdict_new();
> + /* rule.0 can't be both a string and a dict */
> + qdict_put(src, "rule.0", qstring_from_str("fred"));
> + qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
> +
> + g_assert(qdict_crumple(src, true, &error) == NULL);
> + g_assert(error != NULL);
> + error_free(error);
> + error = NULL;
> + QDECREF(src);
> +
> + src = qdict_new();
> + /* rule can't be both a list and a dict */
> + qdict_put(src, "rule.0", qstring_from_str("fred"));
> + qdict_put(src, "rule.a", qstring_from_str("allow"));
> +
> + g_assert(qdict_crumple(src, true, &error) == NULL);
> + g_assert(error != NULL);
> + error_free(error);
> + error = NULL;
> + QDECREF(src);
> +
> + src = qdict_new();
> + /* The input should be flat, ie no dicts or lists */
> + qdict_put(src, "rule.a", qdict_new());
> + qdict_put(src, "rule.b", qstring_from_str("allow"));
> +
> + g_assert(qdict_crumple(src, true, &error) == NULL);
> + g_assert(error != NULL);
> + error_free(error);
> + error = NULL;
> + QDECREF(src);
> +
> +
> + src = qdict_new();
> + /* List indexes must not have gaps */
> + qdict_put(src, "rule.0", qdict_new());
> + qdict_put(src, "rule.3", qstring_from_str("allow"));
> +
> + g_assert(qdict_crumple(src, true, &error) == NULL);
> + g_assert(error != NULL);
> + error_free(error);
> + error = NULL;
> + QDECREF(src);
Suggest to add test case
/* List indexes must not have gaps (more creative version) */
qdict_put(src, "rule.0", ...);
qdict_put(src, "rule.2", ...);
qdict_put(src, "rule.2", ...);
and
/* List indexes must be in %zu format */
qdict_put(src, "rule.+0", ...);
> +}
> +
> /*
> * Errors test-cases
> */
> @@ -743,6 +961,17 @@ int main(int argc, char **argv)
> g_test_add_func("/errors/put_exists", qdict_put_exists_test);
> g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
>
> + g_test_add_func("/public/crumple/nonrecursive",
> + qdict_crumple_test_nonrecursive);
> + g_test_add_func("/public/crumple/recursive",
> + qdict_crumple_test_recursive);
> + g_test_add_func("/public/crumple/listtop",
> + qdict_crumple_test_listtop);
> + g_test_add_func("/public/crumple/empty",
> + qdict_crumple_test_empty);
> + g_test_add_func("/public/crumple/bad_inputs",
> + qdict_crumple_test_bad_inputs);
> +
> /* The Big one */
> if (g_test_slow()) {
> g_test_add_func("/stress/test", qdict_stress_test);
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict
2016-06-09 13:20 ` Markus Armbruster
@ 2016-06-09 13:28 ` Daniel P. Berrange
2016-06-14 11:39 ` Daniel P. Berrange
1 sibling, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-09 13:28 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Thu, Jun 09, 2016 at 03:20:47PM +0200, Markus Armbruster wrote:
> I apologize for the lateness of this review.
>
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > The qdict_flatten() method will take a dict whose elements are
> > further nested dicts/lists and flatten them by concatenating
> > keys.
> >
> > The qdict_crumple() method aims to do the reverse, taking a flat
> > qdict, and turning it into a set of nested dicts/lists. It will
> > apply nesting based on the key name, with a '.' indicating a
> > new level in the hierarchy. If the keys in the nested structure
> > are all numeric, it will create a list, otherwise it will create
> > a dict.
> >
> > If the keys are a mixture of numeric and non-numeric, or the
> > numeric keys are not in strictly ascending order, an error will
> > be reported.
> >
> > As an example, a flat dict containing
> >
> > {
> > 'foo.0.bar': 'one',
> > 'foo.0.wizz': '1',
> > 'foo.1.bar': 'two',
> > 'foo.1.wizz': '2'
> > }
> >
> > will get turned into a dict with one element 'foo' whose
> > value is a list. The list elements will each in turn be
> > dicts.
> >
> > {
> > 'foo': [
> > { 'bar': 'one', 'wizz': '1' },
> > { 'bar': 'two', 'wizz': '2' }
> > ],
> > }
> >
> > If the key is intended to contain a literal '.', then it must
> > be escaped as '..'. ie a flat dict
> >
> > {
> > 'foo..bar': 'wizz',
> > 'bar.foo..bar': 'eek',
> > 'bar.hello': 'world'
> > }
> >
> > Will end up as
> >
> > {
> > 'foo.bar': 'wizz',
> > 'bar': {
> > 'foo.bar': 'eek',
> > 'hello': 'world'
> > }
> > }
> >
> > The intent of this function is that it allows a set of QemuOpts
> > to be turned into a nested data structure that mirrors the nesting
> > used when the same object is defined over QMP.
> >
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>
> Not an objection to your patch, just an observation: we're digging
> ourselves ever deeper into the QemuOpts / QDict hole. We've pushed
> QemuOpts far beyond its original purpose "parse key=value,... option
> arguments". In my opinion, we'd better parse straight to QAPI-generated
> structs.
NB we're not actually digging the hole any deeper here. The BlockDev
code already fully dug the hole. This code is just generalizing
the code for dealing with the hole (intentionally designed to
100% mirror the syntax that BlockDev already defined), so that we
get consistent handling across the codebase, rather than having
many separate subtley different holes :-) I have patches that
I've not posted yet, which start to convert some of the blockdev
code over to use this method.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types Daniel P. Berrange
2016-06-08 12:01 ` Paolo Bonzini
@ 2016-06-09 14:03 ` Markus Armbruster
2016-06-14 13:25 ` Daniel P. Berrange
1 sibling, 1 reply; 25+ messages in thread
From: Markus Armbruster @ 2016-06-09 14:03 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> Currently the QmpInputVisitor assumes that all scalar
> values are directly represented as their final types.
> ie it assumes an 'int' is using QInt, and a 'bool' is
> using QBool.
>
> This extends it so that QString is optionally permitted
> for any of the non-string scalar types. This behaviour
> is turned on by requesting the 'autocast' flag in the
> constructor.
>
> This makes it possible to use QmpInputVisitor with a
> QDict produced from QemuOpts, where everything is in
> string format.
We're still digging.
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> docs/qapi-code-gen.txt | 2 +-
> include/qapi/qmp-input-visitor.h | 6 +-
> qapi/opts-visitor.c | 1 +
> qapi/qmp-input-visitor.c | 89 ++++++++++++++++++++++------
> qmp.c | 2 +-
> qom/qom-qobject.c | 2 +-
> replay/replay-input.c | 2 +-
> scripts/qapi-commands.py | 2 +-
> tests/check-qnull.c | 2 +-
> tests/test-qmp-commands.c | 2 +-
> tests/test-qmp-input-strict.c | 2 +-
> tests/test-qmp-input-visitor.c | 115 ++++++++++++++++++++++++++++++++++++-
> tests/test-visitor-serialization.c | 2 +-
> util/qemu-sockets.c | 2 +-
> 14 files changed, 201 insertions(+), 30 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index d7d6987..e21773e 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -1008,7 +1008,7 @@ Example:
> {
> Error *err = NULL;
> UserDefOne *retval;
> - QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true);
> + QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true, false);
> QapiDeallocVisitor *qdv;
> Visitor *v;
> UserDefOneList *arg1 = NULL;
> diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qmp-input-visitor.h
> index b0624d8..d35a053 100644
> --- a/include/qapi/qmp-input-visitor.h
> +++ b/include/qapi/qmp-input-visitor.h
> @@ -24,8 +24,12 @@ typedef struct QmpInputVisitor QmpInputVisitor;
> *
> * Set @strict to reject a parse that doesn't consume all keys of a
> * dictionary; otherwise excess input is ignored.
> + * Set @autocast to automatically convert string values into more
> + * specific types (numbers, bools, etc)
> */
> -QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict);
> +QmpInputVisitor *qmp_input_visitor_new(QObject *obj,
> + bool strict,
> + bool autocast);
>
> void qmp_input_visitor_cleanup(QmpInputVisitor *v);
>
> diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
> index 4cf1cf8..00e4b1a 100644
> --- a/qapi/opts-visitor.c
> +++ b/qapi/opts-visitor.c
> @@ -347,6 +347,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
> }
>
> if (opt->str) {
> + /* Keep these values in sync with same code in qmp-input-visitor.c */
Also with parse_option_bool(). That's the canonical place, in fact.
> if (strcmp(opt->str, "on") == 0 ||
> strcmp(opt->str, "yes") == 0 ||
> strcmp(opt->str, "y") == 0) {
> diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
> index aea90a1..92023b1 100644
> --- a/qapi/qmp-input-visitor.c
> +++ b/qapi/qmp-input-visitor.c
> @@ -20,6 +20,7 @@
> #include "qemu-common.h"
> #include "qapi/qmp/types.h"
> #include "qapi/qmp/qerror.h"
> +#include "qemu/cutils.h"
>
> #define QIV_STACK_SIZE 1024
>
> @@ -45,6 +46,7 @@ struct QmpInputVisitor
>
> /* True to reject parse in visit_end_struct() if unvisited keys remain. */
> bool strict;
> + bool autocast;
> };
>
> static QmpInputVisitor *to_qiv(Visitor *v)
> @@ -254,15 +256,25 @@ static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
> Error **errp)
> {
> QmpInputVisitor *qiv = to_qiv(v);
> - QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
> + QObject *qobj = qmp_input_get_object(qiv, name, true);
> + QInt *qint;
> + QString *qstr;
>
> - if (!qint) {
> - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> - "integer");
> + qint = qobject_to_qint(qobj);
> + if (qint) {
> + *obj = qint_get_int(qint);
> return;
> }
>
> - *obj = qint_get_int(qint);
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + if (qemu_strtoll(qstr->string, NULL, 10, obj) == 0) {
In the commit message, you mentioned intended use: "with a QDict
produced from QemuOpts, where everything is in string format." I figure
you mean opts with empty opts->list->desc[]. For those, we accept any
key and parse all values as strings.
The idea with such QemuOpts is to *delay* parsing until we know how to
parse. The delay should be transparent to the user. In particular,
values should be parsed the same whether the parsing is delayed or not.
The canonical QemuOpts value parser is qemu_opt_parse(). It parses
integers with parse_option_number(). That function parses like
stroull(qstr->string, ..., 0). Two differences:
* stroll() vs. strtoull(): they differ in ERANGE behavior. This might
be tolerable, but I haven't thought it through.
* Base 0 vs 10: different syntax and semantics. Oops.
> + return;
> + }
> + }
> +
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "integer");
> }
>
> static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
> @@ -270,30 +282,61 @@ static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
> {
> /* FIXME: qobject_to_qint mishandles values over INT64_MAX */
> QmpInputVisitor *qiv = to_qiv(v);
> - QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
> + QObject *qobj = qmp_input_get_object(qiv, name, true);
> + QInt *qint;
> + QString *qstr;
>
> - if (!qint) {
> - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> - "integer");
> + qint = qobject_to_qint(qobj);
> + if (qint) {
> + *obj = qint_get_int(qint);
> return;
> }
>
> - *obj = qint_get_int(qint);
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + if (qemu_strtoull(qstr->string, NULL, 10, obj) == 0) {
Same issue with base 0 vs. 10.
> + return;
> + }
> + }
> +
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "integer");
> }
>
> static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
> Error **errp)
> {
> QmpInputVisitor *qiv = to_qiv(v);
> - QBool *qbool = qobject_to_qbool(qmp_input_get_object(qiv, name, true));
> + QObject *qobj = qmp_input_get_object(qiv, name, true);
> + QBool *qbool;
> + QString *qstr;
>
> - if (!qbool) {
> - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> - "boolean");
> + qbool = qobject_to_qbool(qobj);
> + if (qbool) {
> + *obj = qbool_get_bool(qbool);
> return;
> }
>
> - *obj = qbool_get_bool(qbool);
> +
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + /* Keep these values in sync with same code in opts-visitor.c */
Also with parse_option_bool().
> + if (!strcasecmp(qstr->string, "on") ||
> + !strcasecmp(qstr->string, "yes") ||
> + !strcasecmp(qstr->string, "true")) {
> + *obj = true;
> + return;
> + }
> + if (!strcasecmp(qstr->string, "off") ||
> + !strcasecmp(qstr->string, "no") ||
> + !strcasecmp(qstr->string, "false")) {
> + *obj = false;
> + return;
> + }
You follow opts-visitor.c. It accepts more than parse_option_bool().
Glorious.
I guess the only way out is to add another onion dome to qemu-option.c
and accept yes, no, true, false there as well.
> + }
> +
> + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "boolean");
> }
>
> static void qmp_input_type_str(Visitor *v, const char *name, char **obj,
> @@ -319,6 +362,8 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
> QObject *qobj = qmp_input_get_object(qiv, name, true);
> QInt *qint;
> QFloat *qfloat;
> + QString *qstr;
> + char *endp;
>
> qint = qobject_to_qint(qobj);
> if (qint) {
> @@ -332,6 +377,15 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
> return;
> }
>
> + qstr = qobject_to_qstring(qobj);
> + if (qstr && qstr->string && qiv->autocast) {
> + errno = 0;
> + *obj = strtod(qstr->string, &endp);
> + if (errno == 0 && endp != qstr->string && *endp == '\0') {
> + return;
> + }
> + }
> +
> error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> "number");
> }
QemuOpts do not support floating-point numbers. Doesn't make accepting
them here wrong.
> @@ -381,7 +435,9 @@ void qmp_input_visitor_cleanup(QmpInputVisitor *v)
> g_free(v);
> }
>
> -QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict)
> +QmpInputVisitor *qmp_input_visitor_new(QObject *obj,
> + bool strict,
> + bool autocast)
> {
> QmpInputVisitor *v;
>
> @@ -404,6 +460,7 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj, bool strict)
> v->visitor.type_null = qmp_input_type_null;
> v->visitor.optional = qmp_input_optional;
> v->strict = strict;
> + v->autocast = autocast;
>
> v->root = obj;
> qobject_incref(obj);
> diff --git a/qmp.c b/qmp.c
> index 3165f87..dcd0953 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -665,7 +665,7 @@ void qmp_object_add(const char *type, const char *id,
> }
> }
>
> - qiv = qmp_input_visitor_new(props, true);
> + qiv = qmp_input_visitor_new(props, true, false);
> obj = user_creatable_add_type(type, id, pdict,
> qmp_input_get_visitor(qiv), errp);
> qmp_input_visitor_cleanup(qiv);
> diff --git a/qom/qom-qobject.c b/qom/qom-qobject.c
> index b66088d..99666ce 100644
> --- a/qom/qom-qobject.c
> +++ b/qom/qom-qobject.c
> @@ -23,7 +23,7 @@ void object_property_set_qobject(Object *obj, QObject *value,
> {
> QmpInputVisitor *qiv;
> /* TODO: Should we reject, rather than ignore, excess input? */
> - qiv = qmp_input_visitor_new(value, false);
> + qiv = qmp_input_visitor_new(value, false, false);
> object_property_set(obj, qmp_input_get_visitor(qiv), name, errp);
>
> qmp_input_visitor_cleanup(qiv);
> diff --git a/replay/replay-input.c b/replay/replay-input.c
> index 03e99d5..de82a59 100644
> --- a/replay/replay-input.c
> +++ b/replay/replay-input.c
> @@ -37,7 +37,7 @@ static InputEvent *qapi_clone_InputEvent(InputEvent *src)
> return NULL;
> }
>
> - qiv = qmp_input_visitor_new(obj, true);
> + qiv = qmp_input_visitor_new(obj, true, false);
> iv = qmp_input_get_visitor(qiv);
> visit_type_InputEvent(iv, NULL, &dst, &error_abort);
> qmp_input_visitor_cleanup(qiv);
> diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
> index 8c6acb3..e48995d 100644
> --- a/scripts/qapi-commands.py
> +++ b/scripts/qapi-commands.py
> @@ -115,7 +115,7 @@ def gen_marshal(name, arg_type, ret_type):
>
> if arg_type and arg_type.members:
> ret += mcgen('''
> - QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true);
> + QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true, false);
> QapiDeallocVisitor *qdv;
> Visitor *v;
> %(c_name)s arg = {0};
> diff --git a/tests/check-qnull.c b/tests/check-qnull.c
> index fd9c68f..4c11755 100644
> --- a/tests/check-qnull.c
> +++ b/tests/check-qnull.c
> @@ -49,7 +49,7 @@ static void qnull_visit_test(void)
>
> g_assert(qnull_.refcnt == 1);
> obj = qnull();
> - qiv = qmp_input_visitor_new(obj, true);
> + qiv = qmp_input_visitor_new(obj, true, false);
> qobject_decref(obj);
> visit_type_null(qmp_input_get_visitor(qiv), NULL, &error_abort);
> qmp_input_visitor_cleanup(qiv);
> diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
> index 5c3edd7..c86b282 100644
> --- a/tests/test-qmp-commands.c
> +++ b/tests/test-qmp-commands.c
> @@ -222,7 +222,7 @@ static void test_dealloc_partial(void)
> ud2_dict = qdict_new();
> qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
>
> - qiv = qmp_input_visitor_new(QOBJECT(ud2_dict), true);
> + qiv = qmp_input_visitor_new(QOBJECT(ud2_dict), true, false);
> visit_type_UserDefTwo(qmp_input_get_visitor(qiv), NULL, &ud2, &err);
> qmp_input_visitor_cleanup(qiv);
> QDECREF(ud2_dict);
> diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
> index 4602529..f7f1f00 100644
> --- a/tests/test-qmp-input-strict.c
> +++ b/tests/test-qmp-input-strict.c
> @@ -55,7 +55,7 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
> data->obj = qobject_from_jsonv(json_string, ap);
> g_assert(data->obj);
>
> - data->qiv = qmp_input_visitor_new(data->obj, true);
> + data->qiv = qmp_input_visitor_new(data->obj, true, false);
> g_assert(data->qiv);
>
> v = qmp_input_get_visitor(data->qiv);
> diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
> index cee07ce..5691dc3 100644
> --- a/tests/test-qmp-input-visitor.c
> +++ b/tests/test-qmp-input-visitor.c
> @@ -41,6 +41,7 @@ static void visitor_input_teardown(TestInputVisitorData *data,
> function so that the JSON string used by the tests are kept in the test
> functions (and not in main()). */
> static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> + bool strict, bool autocast,
> const char *json_string,
> va_list *ap)
> {
> @@ -51,7 +52,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> data->obj = qobject_from_jsonv(json_string, ap);
> g_assert(data->obj);
>
> - data->qiv = qmp_input_visitor_new(data->obj, false);
> + data->qiv = qmp_input_visitor_new(data->obj, strict, autocast);
> g_assert(data->qiv);
>
> v = qmp_input_get_visitor(data->qiv);
> @@ -60,6 +61,21 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> return v;
> }
>
> +static GCC_FMT_ATTR(4, 5)
> +Visitor *visitor_input_test_init_full(TestInputVisitorData *data,
> + bool strict, bool autocast,
> + const char *json_string, ...)
> +{
> + Visitor *v;
> + va_list ap;
> +
> + va_start(ap, json_string);
> + v = visitor_input_test_init_internal(data, strict, autocast,
> + json_string, &ap);
> + va_end(ap);
> + return v;
> +}
> +
> static GCC_FMT_ATTR(2, 3)
> Visitor *visitor_input_test_init(TestInputVisitorData *data,
> const char *json_string, ...)
> @@ -68,7 +84,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> va_list ap;
>
> va_start(ap, json_string);
> - v = visitor_input_test_init_internal(data, json_string, &ap);
> + v = visitor_input_test_init_internal(data, false, false,
> + json_string, &ap);
> va_end(ap);
> return v;
> }
> @@ -83,7 +100,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
> const char *json_string)
> {
> - return visitor_input_test_init_internal(data, json_string, NULL);
> + return visitor_input_test_init_internal(data, false, false,
> + json_string, NULL);
> }
>
> static void test_visitor_in_int(TestInputVisitorData *data,
> @@ -115,6 +133,33 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
> error_free_or_abort(&err);
> }
>
> +static void test_visitor_in_int_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + int64_t res = 0, value = -42;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, false, true,
> + "\"-42\"");
> +
> + visit_type_int(v, NULL, &res, &error_abort);
> + g_assert_cmpint(res, ==, value);
> +}
> +
> +static void test_visitor_in_int_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + int64_t res = 0;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"-42\"");
> +
> + visit_type_int(v, NULL, &res, &err);
> + g_assert(err != NULL);
> + error_free(err);
> +}
> +
> static void test_visitor_in_bool(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -127,6 +172,32 @@ static void test_visitor_in_bool(TestInputVisitorData *data,
> g_assert_cmpint(res, ==, true);
> }
>
> +static void test_visitor_in_bool_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + bool res = false;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, false, true, "\"true\"");
> +
> + visit_type_bool(v, NULL, &res, &error_abort);
> + g_assert_cmpint(res, ==, true);
> +}
> +
> +static void test_visitor_in_bool_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + bool res = false;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"true\"");
> +
> + visit_type_bool(v, NULL, &res, &err);
> + g_assert(err != NULL);
> + error_free(err);
> +}
> +
> static void test_visitor_in_number(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -139,6 +210,32 @@ static void test_visitor_in_number(TestInputVisitorData *data,
> g_assert_cmpfloat(res, ==, value);
> }
>
> +static void test_visitor_in_number_autocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + double res = 0, value = 3.14;
> + Visitor *v;
> +
> + v = visitor_input_test_init_full(data, false, true, "\"3.14\"");
> +
> + visit_type_number(v, NULL, &res, &error_abort);
> + g_assert_cmpfloat(res, ==, value);
> +}
> +
> +static void test_visitor_in_number_noautocast(TestInputVisitorData *data,
> + const void *unused)
> +{
> + double res = 0;
> + Visitor *v;
> + Error *err = NULL;
> +
> + v = visitor_input_test_init(data, "\"3.14\"");
> +
> + visit_type_number(v, NULL, &res, &err);
> + g_assert(err != NULL);
> + error_free(err);
> +}
> +
> static void test_visitor_in_string(TestInputVisitorData *data,
> const void *unused)
> {
> @@ -835,10 +932,22 @@ int main(int argc, char **argv)
> &in_visitor_data, test_visitor_in_int);
> input_visitor_test_add("/visitor/input/int_overflow",
> &in_visitor_data, test_visitor_in_int_overflow);
> + input_visitor_test_add("/visitor/input/int_autocast",
> + &in_visitor_data, test_visitor_in_int_autocast);
> + input_visitor_test_add("/visitor/input/int_noautocast",
> + &in_visitor_data, test_visitor_in_int_noautocast);
> input_visitor_test_add("/visitor/input/bool",
> &in_visitor_data, test_visitor_in_bool);
> + input_visitor_test_add("/visitor/input/bool_autocast",
> + &in_visitor_data, test_visitor_in_bool_autocast);
> + input_visitor_test_add("/visitor/input/bool_noautocast",
> + &in_visitor_data, test_visitor_in_bool_noautocast);
> input_visitor_test_add("/visitor/input/number",
> &in_visitor_data, test_visitor_in_number);
> + input_visitor_test_add("/visitor/input/number_autocast",
> + &in_visitor_data, test_visitor_in_number_autocast);
> + input_visitor_test_add("/visitor/input/number_noautocast",
> + &in_visitor_data, test_visitor_in_number_noautocast);
> input_visitor_test_add("/visitor/input/string",
> &in_visitor_data, test_visitor_in_string);
> input_visitor_test_add("/visitor/input/enum",
> diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
> index 7b14b5a..db618d8 100644
> --- a/tests/test-visitor-serialization.c
> +++ b/tests/test-visitor-serialization.c
> @@ -1038,7 +1038,7 @@ static void qmp_deserialize(void **native_out, void *datap,
> obj = qobject_from_json(qstring_get_str(output_json));
>
> QDECREF(output_json);
> - d->qiv = qmp_input_visitor_new(obj, true);
> + d->qiv = qmp_input_visitor_new(obj, true, false);
> qobject_decref(obj_orig);
> qobject_decref(obj);
> visit(qmp_input_get_visitor(d->qiv), native_out, errp);
> diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
> index 0d6cd1f..51c6a8e 100644
> --- a/util/qemu-sockets.c
> +++ b/util/qemu-sockets.c
> @@ -1145,7 +1145,7 @@ void qapi_copy_SocketAddress(SocketAddress **p_dest,
> return;
> }
>
> - qiv = qmp_input_visitor_new(obj, true);
> + qiv = qmp_input_visitor_new(obj, true, false);
> iv = qmp_input_get_visitor(qiv);
> visit_type_SocketAddress(iv, NULL, p_dest, &error_abort);
> qmp_input_visitor_cleanup(qiv);
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object Daniel P. Berrange
@ 2016-06-09 14:43 ` Markus Armbruster
2016-06-14 14:16 ` Daniel P. Berrange
0 siblings, 1 reply; 25+ messages in thread
From: Markus Armbruster @ 2016-06-09 14:43 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
"Daniel P. Berrange" <berrange@redhat.com> writes:
> The current -object command line syntax only allows for
> creation of objects with scalar properties, or a list
> with a fixed scalar element type. Objects which have
> properties that are represented as structs in the QAPI
> schema cannot be created using -object.
>
> This is a design limitation of the way the OptsVisitor
> is written. It simply iterates over the QemuOpts values
> as a flat list. The support for lists is enabled by
> allowing the same key to be repeated in the opts string.
>
> It is not practical to extend the OptsVisitor to support
> more complex data structures while also maintaining
> the existing list handling behaviour that is relied upon
> by other areas of QEMU.
It should be practical to create a new option input visitor that parses
command line syntax straight into a QAPI visit, without the list magic,
and with fewer or no restrictions. Then we could start defining complex
command line arguments as QAPI types, parse them straight into generated
QAPI types, and dispense with the QDict wrangling.
But I can't blame you for solving a simpler problem instead. In
particular since there's ample precedence for QDict wrangling.
> Fortunately there is no existing object that implements
> the UserCreatable interface that relies on the list
> handling behaviour, so it is possible to swap out the
> OptsVisitor for a different visitor implementation, so
> -object supports non-scalar properties, thus leaving
> other users of OptsVisitor unaffected.
>
> The previously added qdict_crumple() method is able to
> take a qdict containing a flat set of properties and
> turn that into a arbitrarily nested set of dicts and
> lists. By combining qemu_opts_to_qdict and qdict_crumple()
> together, we can turn the opt string into a data structure
> that is practically identical to that passed over QMP
> when defining an object. The only difference is that all
> the scalar values are represented as strings, rather than
> strings, ints and bools. This is sufficient to let us
> replace the OptsVisitor with the QMPInputVisitor for
> use with -object.
>
> Thus -object can now support non-scalar properties,
> for example the QMP object
>
> {
> "execute": "object-add",
> "arguments": {
> "qom-type": "demo",
> "id": "demo0",
> "parameters": {
> "foo": [
> { "bar": "one", "wizz": "1" },
> { "bar": "two", "wizz": "2" }
> ]
> }
> }
> }
>
> Would be creatable via the CLI now using
>
> $QEMU \
> -object demo,id=demo0,\
> foo.0.bar=one,foo.0.wizz=1,\
> foo.1.bar=two,foo.1.wizz=2
>
> Notice that this syntax is intentionally compatible
> with that currently used by block drivers.
>
> This is also wired up to work for the 'object_add' command
> in the HMP monitor with the same syntax.
>
> (hmp) object_add demo,id=demo0,\
> foo.0.bar=one,foo.0.wizz=1,\
> foo.1.bar=two,foo.1.wizz=2
>
> NB indentation should not be used with HMP commands, this
> is just for convenient formatting in this commit message.
Cannot be used, actually (and neither can line continuation), but I get
what you mean.
> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> ---
> hmp.c | 18 +--
> include/qom/object_interfaces.h | 10 +-
> qmp.c | 2 +-
> qom/object_interfaces.c | 49 +++++--
> tests/check-qom-proplist.c | 317 +++++++++++++++++++++++++++++++++++++++-
> 5 files changed, 363 insertions(+), 33 deletions(-)
>
> diff --git a/hmp.c b/hmp.c
> index a4b1d3d..1972bef 100644
> --- a/hmp.c
> +++ b/hmp.c
> @@ -25,7 +25,7 @@
> #include "qemu/sockets.h"
> #include "monitor/monitor.h"
> #include "monitor/qdev.h"
> -#include "qapi/opts-visitor.h"
> +#include "qapi/qmp-input-visitor.h"
> #include "qapi/qmp/qerror.h"
> #include "qapi/string-output-visitor.h"
> #include "qapi/util.h"
> @@ -1717,20 +1717,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
> void hmp_object_add(Monitor *mon, const QDict *qdict)
> {
> Error *err = NULL;
> - QemuOpts *opts;
> - OptsVisitor *ov;
> + QmpInputVisitor *qiv;
> Object *obj = NULL;
>
> - opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
> - if (err) {
> - hmp_handle_error(mon, &err);
> - return;
> - }
> -
> - ov = opts_visitor_new(opts);
> - obj = user_creatable_add(qdict, opts_get_visitor(ov), &err);
> - opts_visitor_cleanup(ov);
> - qemu_opts_del(opts);
> + qiv = qmp_input_visitor_new((QObject *)qdict, true, true);
> + obj = user_creatable_add(qdict, qmp_input_get_visitor(qiv), &err);
> + qmp_input_visitor_cleanup(qiv);
Any patch that gets rid of a qemu_opts_from_qdict() deserves support.
>
> if (err) {
> hmp_handle_error(mon, &err);
> diff --git a/include/qom/object_interfaces.h b/include/qom/object_interfaces.h
> index 8b17f4d..997563a 100644
> --- a/include/qom/object_interfaces.h
> +++ b/include/qom/object_interfaces.h
> @@ -96,18 +96,24 @@ Object *user_creatable_add(const QDict *qdict,
> * user_creatable_add_type:
> * @type: the object type name
> * @id: the unique ID for the object
> + * @nested: whether to recurse into the visitor for properties
> * @qdict: the object properties
> * @v: the visitor
> * @errp: if an error occurs, a pointer to an area to store the error
> *
> * Create an instance of the user creatable object @type, placing
> * it in the object composition tree with name @id, initializing
> - * it with properties from @qdict
> + * it with properties from @qdict.
> + *
> + * If the visitor is already positioned to read the properties
> + * in @qdict, @nested should be false. Conversely, if it is
> + * neccessary to open/close a struct to read the properties in
necessary
> + * @qdict, @nested should be true.
I don't like this interface much, but I'm not the maintainer here. Have
you considered creating a function that does nested=false and can be
used by the nested=true function?
> *
> * Returns: the newly created object or NULL on error
> */
> Object *user_creatable_add_type(const char *type, const char *id,
> - const QDict *qdict,
> + bool nested, const QDict *qdict,
> Visitor *v, Error **errp);
>
> /**
> diff --git a/qmp.c b/qmp.c
> index dcd0953..dc5fff0 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -666,7 +666,7 @@ void qmp_object_add(const char *type, const char *id,
> }
>
> qiv = qmp_input_visitor_new(props, true, false);
> - obj = user_creatable_add_type(type, id, pdict,
> + obj = user_creatable_add_type(type, id, true, pdict,
> qmp_input_get_visitor(qiv), errp);
> qmp_input_visitor_cleanup(qiv);
> if (obj) {
> diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
> index 51e62e2..0883c91 100644
> --- a/qom/object_interfaces.c
> +++ b/qom/object_interfaces.c
> @@ -3,6 +3,7 @@
> #include "qom/object_interfaces.h"
> #include "qemu/module.h"
> #include "qapi-visit.h"
> +#include "qapi/qmp-input-visitor.h"
> #include "qapi/qmp-output-visitor.h"
> #include "qapi/opts-visitor.h"
>
> @@ -63,12 +64,16 @@ Object *user_creatable_add(const QDict *qdict,
> if (local_err) {
> goto out_visit;
> }
> - visit_check_struct(v, &local_err);
> +
> + obj = user_creatable_add_type(type, id, false, pdict, v, &local_err);
> if (local_err) {
> goto out_visit;
> }
>
> - obj = user_creatable_add_type(type, id, pdict, v, &local_err);
> + visit_check_struct(v, &local_err);
> + if (local_err) {
> + goto out_visit;
> + }
>
> out_visit:
> visit_end_struct(v);
> @@ -87,7 +92,7 @@ out:
>
>
> Object *user_creatable_add_type(const char *type, const char *id,
> - const QDict *qdict,
> + bool nested, const QDict *qdict,
> Visitor *v, Error **errp)
> {
> Object *obj;
> @@ -114,9 +119,11 @@ Object *user_creatable_add_type(const char *type, const char *id,
>
> assert(qdict);
> obj = object_new(type);
> - visit_start_struct(v, NULL, NULL, 0, &local_err);
> - if (local_err) {
> - goto out;
> + if (nested) {
> + visit_start_struct(v, NULL, NULL, 0, &local_err);
> + if (local_err) {
> + goto out;
> + }
> }
> for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
> object_property_set(obj, v, e->key, &local_err);
> @@ -124,12 +131,14 @@ Object *user_creatable_add_type(const char *type, const char *id,
> break;
> }
> }
> - if (!local_err) {
> - visit_check_struct(v, &local_err);
> - }
> - visit_end_struct(v);
> - if (local_err) {
> - goto out;
> + if (nested) {
> + if (!local_err) {
> + visit_check_struct(v, &local_err);
> + }
> + visit_end_struct(v);
> + if (local_err) {
> + goto out;
> + }
> }
>
> object_property_add_child(object_get_objects_root(),
> @@ -156,15 +165,23 @@ out:
>
> Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
> {
> - OptsVisitor *ov;
> + QmpInputVisitor *qiv;
> QDict *pdict;
> + QObject *pobj;
> Object *obj = NULL;
>
> - ov = opts_visitor_new(opts);
> pdict = qemu_opts_to_qdict(opts, NULL);
>
> - obj = user_creatable_add(pdict, opts_get_visitor(ov), errp);
> - opts_visitor_cleanup(ov);
> + pobj = qdict_crumple(pdict, true, errp);
> + if (!pobj) {
> + goto cleanup;
> + }
> + qiv = qmp_input_visitor_new(pobj, true, true);
> +
> + obj = user_creatable_add((QDict *)pobj, qmp_input_get_visitor(qiv), errp);
> + qmp_input_visitor_cleanup(qiv);
> + qobject_decref(pobj);
> + cleanup:
> QDECREF(pdict);
> return obj;
> }
[...]
I'm not familiar enough with this code to provide a real review with
reasonable effort. It looks sane to ignorant me.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict
2016-06-09 13:20 ` Markus Armbruster
2016-06-09 13:28 ` Daniel P. Berrange
@ 2016-06-14 11:39 ` Daniel P. Berrange
2016-06-16 9:16 ` Markus Armbruster
1 sibling, 1 reply; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-14 11:39 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Thu, Jun 09, 2016 at 03:20:47PM +0200, Markus Armbruster wrote:
> I apologize for the lateness of this review.
>
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > The qdict_flatten() method will take a dict whose elements are
> > further nested dicts/lists and flatten them by concatenating
> > keys.
> >
> > The qdict_crumple() method aims to do the reverse, taking a flat
> > qdict, and turning it into a set of nested dicts/lists. It will
> > apply nesting based on the key name, with a '.' indicating a
> > new level in the hierarchy. If the keys in the nested structure
> > are all numeric, it will create a list, otherwise it will create
> > a dict.
> >
> > If the keys are a mixture of numeric and non-numeric, or the
> > numeric keys are not in strictly ascending order, an error will
> > be reported.
> >
> > As an example, a flat dict containing
> >
> > {
> > 'foo.0.bar': 'one',
> > 'foo.0.wizz': '1',
> > 'foo.1.bar': 'two',
> > 'foo.1.wizz': '2'
> > }
> >
> > will get turned into a dict with one element 'foo' whose
> > value is a list. The list elements will each in turn be
> > dicts.
> >
> > {
> > 'foo': [
> > { 'bar': 'one', 'wizz': '1' },
> > { 'bar': 'two', 'wizz': '2' }
> > ],
> > }
> >
> > If the key is intended to contain a literal '.', then it must
> > be escaped as '..'. ie a flat dict
> >
> > {
> > 'foo..bar': 'wizz',
> > 'bar.foo..bar': 'eek',
> > 'bar.hello': 'world'
> > }
> >
> > Will end up as
> >
> > {
> > 'foo.bar': 'wizz',
> > 'bar': {
> > 'foo.bar': 'eek',
> > 'hello': 'world'
> > }
> > }
> >
> > The intent of this function is that it allows a set of QemuOpts
> > to be turned into a nested data structure that mirrors the nesting
> > used when the same object is defined over QMP.
> >
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>
> > +/**
> > + * qdict_split_flat_key:
> > + * @key: the key string to split
> > + * @prefix: non-NULL pointer to hold extracted prefix
> > + * @suffix: non-NULL pointer to hold extracted suffix
> > + *
> > + * Given a flattened key such as 'foo.0.bar', split it
> > + * into two parts at the first '.' separator. Allows
> > + * double dot ('..') to escape the normal separator.
> > + *
> > + * eg
> > + * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
> > + * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
> > + *
> > + * The '..' sequence will be unescaped in the returned
> > + * 'prefix' string. The 'suffix' string will be left
> > + * in escaped format, so it can be fed back into the
> > + * qdict_split_flat_key() key as the input later.
>
> Why is the suffix strdup'ed then?
It doesn't need to be - i'll make it const.
> > +}
> > +
> > +
> > +/**
> > + * qdict_list_size:
> > + * @maybe_list: dict that may be only list elements
>
> Huh? How can a dictionary "be only list elements"?
>
> Do you mean "the dictionary to test?"
I'll say "dict to search for keys representing list elements."
> > + *
> > + * Determine whether all keys in @maybe_list are
> > + * valid list elements. If they are all valid,
> > + * then this returns the number of elements. If
> > + * they all look like non-numeric keys, then returns
> > + * zero. If there is a mix of numeric and non-numeric
> > + * keys, then an error is set as it is both a list
> > + * and a dict at once.
>
> This is well-defined only if empty @maybe_list is considered to have
> dict nature, not list nature. Else, return value zero could be the
> length of the empty list or the special "has dict nature" value.
>
> Please spell out behavior for empty @maybe_list.
Yep, empty list will imply qdict nature and so return zero.
> > + *
> > + * Returns: number of list elements, 0 if a dict, -1 on error
>
> Awkward function name. qdict_list_size_if_list() would be clear.
>
> But I'd simply turn this into a predicate qdict_is_list(), and have the
> caller use qdict_size() to get the number of elements.
I thought that qdict_size() would cause another iteration, but
I see now it just returns a cached size. So I'll switch to
qidct_is_list().
> > +static ssize_t qdict_list_size(QDict *maybe_list, Error **errp)
> > +{
> > + const QDictEntry *entry, *next;
> > + ssize_t len = 0;
> > + ssize_t max = -1;
> > + int is_list = -1;
> > + int64_t val;
> > +
> > + entry = qdict_first(maybe_list);
> > + while (entry != NULL) {
> > + next = qdict_next(maybe_list, entry);
>
> Please keep the loop control in one place:
>
> for (entry = qdict_first(maybe_list); entry; entry = qdict_next(entry)) {
>
> I'd rename some variables for less verbiage:
>
> for (ent = qdict_first(dict); ent; ent = qdict_next(ent)) {
>
> Your loop control also works when the loop body deletes @entry from
> @maybe_list. Seeing such loop control in a function that isn't supposed
> to change the its argument makes the reviewer go "WTF?!?" :)
This pattern was left from an earlier version where all the code was in
one method. I'll change it to a for() loop now.
>
> > +
> > + if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
> > + if (is_list == -1) {
> > + is_list = 1;
> > + } else if (!is_list) {
> > + error_setg(errp,
> > + "Cannot crumple a dictionary with a mix of list "
> > + "and non-list keys");
> > + return -1;
> > + }
> > + len++;
> > + if (val > max) {
> > + max = val;
> > + }
> > + } else {
> > + if (is_list == -1) {
> > + is_list = 0;
> > + } else if (is_list) {
> > + error_setg(errp,
> > + "Cannot crumple a dictionary with a mix of list "
> > + "and non-list keys");
> > + return -1;
> > + }
> > + }
> > +
> > + entry = next;
> > + }
> > +
> > + if (is_list == -1) {
> > + is_list = 0;
>
> This can happen only when @maybe_list is empty. Okay, but perhaps you'd
> like to assert(!qdict_size(maybe_list)).
Ok
>
> > + }
> > +
> > + if (len != (max + 1)) {
> > + error_setg(errp, "List indexes are not continuous, "
> > + "saw %zd elements but %zd largest index",
> > + len, max);
> > + return -1;
>
> contiguous?
>
> What if we saw indexes 0, 2, 2?
That would imply that the dict had two entries with the same
key, which by definition is impossible with a dict data
structure.
> > + }
> > +
> > + return is_list ? len : 0;
> > +}
> > +
> > +/**
> > + * qdict_crumple:
> > + * @src: the original flat dictionary to crumple
>
> "Flat" means all values are scalar. Should we spell that out?
Yep, ok.
> > + * @recursive: true to recursively crumple nested dictionaries
> > + *
> > + * Takes a flat dictionary whose keys use '.' separator to
> > + * indicate nesting, and values are scalars, crumplings it
>
> s/, crumplings/, and crumples/
>
> > + * into a nested structure. If the @recursive parameter is
> > + * false, then only the first level of structure implied
> > + * by the keys will be crumpled. If @recursive is true,
> > + * then the input will be recursively crumpled to expand
> > + * all levels of structure in the keys.
> > + *
> > + * To include a literal '.' in a key name, it must be escaped
> > + * as '..'
> > + *
> > + * For example, an input of:
> > + *
> > + * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
> > + * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
> > + *
> > + * will result in any output of:
> > + *
> > + * {
> > + * 'foo': [
> > + * { 'bar': 'one', 'wizz': '1' },
> > + * { 'bar': 'two', 'wizz': '2' }
> > + * ],
> > + * }
> > + *
> > + * Returns: either a QDict or QList for the nested data structure
>
> I think you should discuss how this can fail.
Will do.
> > +QObject *qdict_crumple(QDict *src, bool recursive, Error **errp)
> > +{
> > + const QDictEntry *entry, *next;
> > + QDict *two_level, *multi_level = NULL;
> > + QObject *dst = NULL, *child;
> > + ssize_t list_len;
> > + size_t i;
> > + char *prefix = NULL, *suffix = NULL;
> > +
> > + two_level = qdict_new();
> > + entry = qdict_first(src);
> > +
> > + /* Step 1: split our totally flat dict into a two level dict */
> > + while (entry != NULL) {
> > + next = qdict_next(src, entry);
>
> Again, keep the loop control in one place.
>
> > +
> > + if (qobject_type(entry->value) == QTYPE_QDICT ||
> > + qobject_type(entry->value) == QTYPE_QLIST) {
> > + error_setg(errp, "Value %s is not a scalar",
> > + entry->key);
> > + goto error;
> > + }
> > +
> > + qdict_split_flat_key(entry->key, &prefix, &suffix);
> > +
> > + child = qdict_get(two_level, prefix);
> > + if (suffix) {
> > + if (child) {
> > + if (qobject_type(child) != QTYPE_QDICT) {
> > + error_setg(errp, "Key %s prefix is already set as a scalar",
> > + prefix);
> > + goto error;
> > + }
> > + } else {
> > + child = QOBJECT(qdict_new());
> > + qdict_put_obj(two_level, prefix, child);
> > + }
> > + qobject_incref(entry->value);
> > + qdict_put_obj(qobject_to_qdict(child), suffix, entry->value);
> > + } else {
> > + if (child) {
> > + error_setg(errp, "Key %s prefix is already set as a dict",
> > + prefix);
> > + goto error;
> > + }
> > + qobject_incref(entry->value);
> > + qdict_put_obj(two_level, prefix, entry->value);
> > + }
>
> Works, because we put only QDicts we've created ourselves (first
> qdict_put_obj() above) and values we got from @src (second
> qdict_put_obj()), and we fail when such a value isn't scalar.
>
> > +
> > + g_free(suffix);
>
> As I suspected, qdict_split_flat_key() strdup'ing the suffix is useless.
>
> > + g_free(prefix);
> > + suffix = prefix = NULL;
>
> Dead stores.
No, they're not dead. The end of this method has a 'g_free(prefix)' so
we must be sure to clear this out to prevent a double-free if a later
codebase jumps to the error label.
> > + entry = next;
> > + }
> > +
> > + /* Step 2: optionally process the two level dict recursively
> > + * into a multi-level dict */
> > + if (recursive) {
> > + multi_level = qdict_new();
> > + entry = qdict_first(two_level);
> > + while (entry != NULL) {
> > + next = qdict_next(two_level, entry);
>
> Again, keep the loop control in one place.
OK
>
> > +
> > + if (qobject_type(entry->value) == QTYPE_QDICT) {
> > + child = qdict_crumple(qobject_to_qdict(entry->value),
> > + recursive, errp);
> > + if (!child) {
> > + goto error;
> > + }
> > +
> > + qdict_put_obj(multi_level, entry->key, child);
> > + } else {
> > + qobject_incref(entry->value);
> > + qdict_put_obj(multi_level, entry->key, entry->value);
> > + }
> > +
> > + entry = next;
> > + }
> > + QDECREF(two_level);
> > + } else {
> > + multi_level = two_level;
> > + }
> > + two_level = NULL;
> > +
> > + /* Step 3: detect if we need to turn our dict into list */
> > + list_len = qdict_list_size(multi_level, errp);
> > + if (list_len < 0) {
> > + goto error;
> > + }
> > +
> > + if (list_len) {
> > + dst = QOBJECT(qlist_new());
> > +
> > + for (i = 0; i < list_len; i++) {
> > + char *key = g_strdup_printf("%zu", i);
> > +
> > + child = qdict_get(multi_level, key);
> > + g_free(key);
> > + assert(child);
>
> qdict_list_size() accepts as list index any (string) key qemu_strtoll()
> accepts. If %zu formats it back into the same string, we'll find it
> here. Else we die. Please spell this out in the function contract.
OK.
> [*] I'm afraid we also die if qdict_list_size()'s "List indexes are not
> continuous" check gets fooled. Suggest to drop that check, and replace
> this assertion by error_setg(errp, "Malformed list indexes").
> Admittedly not the nicest error message; perhaps you can come up with a
> better one.
We can't be fooled per my note earlier
>
> > +
> > + qobject_incref(child);
> > + qlist_append_obj(qobject_to_qlist(dst), child);
> > + }
> > + QDECREF(multi_level);
>
> Do we need
>
> multi_level = NULL;
>
> here?
It isn't needed right now, since we're at the end of the method now
and nothing will touch this var again. Setting it to NULL is a
worthwhile safety net for future refactoring though.
>
> > + } else {
> > + dst = QOBJECT(multi_level);
> > + }
> > +
> > + return dst;
> > +
> > + error:
> > + g_free(suffix);
> > + g_free(prefix);
> > + QDECREF(multi_level);
> > + QDECREF(two_level);
> > + qobject_decref(dst);
> > + return NULL;
> > +}
> > +
> > +
> > /**
> > * qdict_array_entries(): Returns the number of direct array entries if the
> > * sub-QDict of src specified by the prefix in subqdict (or src itself for
> > diff --git a/tests/check-qdict.c b/tests/check-qdict.c
> > index a43056c..0d12f40 100644
> > --- a/tests/check-qdict.c
> > +++ b/tests/check-qdict.c
> > @@ -15,6 +15,7 @@
> > #include "qapi/qmp/qint.h"
> > #include "qapi/qmp/qdict.h"
> > #include "qapi/qmp/qstring.h"
> > +#include "qapi/error.h"
> > #include "qemu-common.h"
> > +static void qdict_crumple_test_bad_inputs(void)
> > +{
> > + QDict *src;
> > + Error *error = NULL;
> > +
> > + src = qdict_new();
> > + /* rule.0 can't be both a string and a dict */
> > + qdict_put(src, "rule.0", qstring_from_str("fred"));
> > + qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
> > +
> > + g_assert(qdict_crumple(src, true, &error) == NULL);
> > + g_assert(error != NULL);
> > + error_free(error);
> > + error = NULL;
> > + QDECREF(src);
> > +
> > + src = qdict_new();
> > + /* rule can't be both a list and a dict */
> > + qdict_put(src, "rule.0", qstring_from_str("fred"));
> > + qdict_put(src, "rule.a", qstring_from_str("allow"));
> > +
> > + g_assert(qdict_crumple(src, true, &error) == NULL);
> > + g_assert(error != NULL);
> > + error_free(error);
> > + error = NULL;
> > + QDECREF(src);
> > +
> > + src = qdict_new();
> > + /* The input should be flat, ie no dicts or lists */
> > + qdict_put(src, "rule.a", qdict_new());
> > + qdict_put(src, "rule.b", qstring_from_str("allow"));
> > +
> > + g_assert(qdict_crumple(src, true, &error) == NULL);
> > + g_assert(error != NULL);
> > + error_free(error);
> > + error = NULL;
> > + QDECREF(src);
> > +
> > +
> > + src = qdict_new();
> > + /* List indexes must not have gaps */
> > + qdict_put(src, "rule.0", qdict_new());
> > + qdict_put(src, "rule.3", qstring_from_str("allow"));
> > +
> > + g_assert(qdict_crumple(src, true, &error) == NULL);
> > + g_assert(error != NULL);
> > + error_free(error);
> > + error = NULL;
> > + QDECREF(src);
>
> Suggest to add test case
>
> /* List indexes must not have gaps (more creative version) */
> qdict_put(src, "rule.0", ...);
> qdict_put(src, "rule.2", ...);
> qdict_put(src, "rule.2", ...);
That's surely impossible, as dict keys have to be unique.
> and
>
> /* List indexes must be in %zu format */
> qdict_put(src, "rule.+0", ...);
Ok.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types
2016-06-09 14:03 ` Markus Armbruster
@ 2016-06-14 13:25 ` Daniel P. Berrange
2016-06-16 9:23 ` Markus Armbruster
0 siblings, 1 reply; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-14 13:25 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Thu, Jun 09, 2016 at 04:03:50PM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > Currently the QmpInputVisitor assumes that all scalar
> > values are directly represented as their final types.
> > ie it assumes an 'int' is using QInt, and a 'bool' is
> > using QBool.
> >
> > This extends it so that QString is optionally permitted
> > for any of the non-string scalar types. This behaviour
> > is turned on by requesting the 'autocast' flag in the
> > constructor.
> >
> > This makes it possible to use QmpInputVisitor with a
> > QDict produced from QemuOpts, where everything is in
> > string format.
>
> We're still digging.
>
> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
> > ---
> > docs/qapi-code-gen.txt | 2 +-
> > include/qapi/qmp-input-visitor.h | 6 +-
> > qapi/opts-visitor.c | 1 +
> > qapi/qmp-input-visitor.c | 89 ++++++++++++++++++++++------
> > qmp.c | 2 +-
> > qom/qom-qobject.c | 2 +-
> > replay/replay-input.c | 2 +-
> > scripts/qapi-commands.py | 2 +-
> > tests/check-qnull.c | 2 +-
> > tests/test-qmp-commands.c | 2 +-
> > tests/test-qmp-input-strict.c | 2 +-
> > tests/test-qmp-input-visitor.c | 115 ++++++++++++++++++++++++++++++++++++-
> > tests/test-visitor-serialization.c | 2 +-
> > util/qemu-sockets.c | 2 +-
> > 14 files changed, 201 insertions(+), 30 deletions(-)
> > diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
> > index 4cf1cf8..00e4b1a 100644
> > --- a/qapi/opts-visitor.c
> > +++ b/qapi/opts-visitor.c
> > @@ -347,6 +347,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
> > }
> >
> > if (opt->str) {
> > + /* Keep these values in sync with same code in qmp-input-visitor.c */
>
> Also with parse_option_bool(). That's the canonical place, in fact.
....except parse_option_bool() doesn't allow "yes", "no", "y", "n" as valid
values - it only permits "on" and "off" :-( I guess I could make the
parse_option_bool() method non-static, make it accept these missing values,
and then call it from all places so we have guaranteed consistency.
>
> > if (strcmp(opt->str, "on") == 0 ||
> > strcmp(opt->str, "yes") == 0 ||
> > strcmp(opt->str, "y") == 0) {
> > diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
> > index aea90a1..92023b1 100644
> > --- a/qapi/qmp-input-visitor.c
> > +++ b/qapi/qmp-input-visitor.c
> > @@ -20,6 +20,7 @@
> > #include "qemu-common.h"
> > #include "qapi/qmp/types.h"
> > #include "qapi/qmp/qerror.h"
> > +#include "qemu/cutils.h"
> >
> > #define QIV_STACK_SIZE 1024
> >
> > @@ -45,6 +46,7 @@ struct QmpInputVisitor
> >
> > /* True to reject parse in visit_end_struct() if unvisited keys remain. */
> > bool strict;
> > + bool autocast;
> > };
> >
> > static QmpInputVisitor *to_qiv(Visitor *v)
> > @@ -254,15 +256,25 @@ static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
> > Error **errp)
> > {
> > QmpInputVisitor *qiv = to_qiv(v);
> > - QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
> > + QObject *qobj = qmp_input_get_object(qiv, name, true);
> > + QInt *qint;
> > + QString *qstr;
> >
> > - if (!qint) {
> > - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> > - "integer");
> > + qint = qobject_to_qint(qobj);
> > + if (qint) {
> > + *obj = qint_get_int(qint);
> > return;
> > }
> >
> > - *obj = qint_get_int(qint);
> > + qstr = qobject_to_qstring(qobj);
> > + if (qstr && qstr->string && qiv->autocast) {
> > + if (qemu_strtoll(qstr->string, NULL, 10, obj) == 0) {
>
> In the commit message, you mentioned intended use: "with a QDict
> produced from QemuOpts, where everything is in string format." I figure
> you mean opts with empty opts->list->desc[]. For those, we accept any
> key and parse all values as strings.
>
> The idea with such QemuOpts is to *delay* parsing until we know how to
> parse. The delay should be transparent to the user. In particular,
> values should be parsed the same whether the parsing is delayed or not.
>
> The canonical QemuOpts value parser is qemu_opt_parse(). It parses
> integers with parse_option_number(). That function parses like
> stroull(qstr->string, ..., 0). Two differences:
>
> * stroll() vs. strtoull(): they differ in ERANGE behavior. This might
> be tolerable, but I haven't thought it through.
>
> * Base 0 vs 10: different syntax and semantics. Oops.
Sigh, I originally had my code using strtoull() directly as the
parse_option_number() method does, but had to change it to make
checkpatch.pl stfu thus creating incosistency. This is a great
example of why I hate the fact that we only validate our style
rules against new patches and not our existing codebase :-(
Anyway, it seems to be something where we should refactor code to
use the same parsing method from both places. eg by making the
parse_option_number method non-static and just calling it from
here.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types
2016-06-08 12:01 ` Paolo Bonzini
@ 2016-06-14 14:10 ` Daniel P. Berrange
0 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-14 14:10 UTC (permalink / raw)
To: Paolo Bonzini
Cc: qemu-devel, Eric Blake, Max Reitz, Markus Armbruster,
Andreas Färber, qemu-block
On Wed, Jun 08, 2016 at 02:01:23PM +0200, Paolo Bonzini wrote:
>
>
> On 02/06/2016 18:46, Daniel P. Berrange wrote:
> > Currently the QmpInputVisitor assumes that all scalar
> > values are directly represented as their final types.
> > ie it assumes an 'int' is using QInt, and a 'bool' is
> > using QBool.
> >
> > This extends it so that QString is optionally permitted
> > for any of the non-string scalar types. This behaviour
> > is turned on by requesting the 'autocast' flag in the
> > constructor.
> >
> > This makes it possible to use QmpInputVisitor with a
> > QDict produced from QemuOpts, where everything is in
> > string format.
>
> Perhaps this should instead be a separate QmpStringInputVisitor visitor
> that _only_ accepts strings? You can reuse most of the QmpInputVisitor
> by putting it in the same file, because the struct and list visitors are
> compatible.
Yes, that actually works out quite nicely indeed, and in fact
showed up a bug in my unit tests too :-)
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object
2016-06-09 14:43 ` Markus Armbruster
@ 2016-06-14 14:16 ` Daniel P. Berrange
0 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrange @ 2016-06-14 14:16 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Max Reitz, Paolo Bonzini, Andreas Färber
On Thu, Jun 09, 2016 at 04:43:35PM +0200, Markus Armbruster wrote:
> "Daniel P. Berrange" <berrange@redhat.com> writes:
>
> > The current -object command line syntax only allows for
> > creation of objects with scalar properties, or a list
> > with a fixed scalar element type. Objects which have
> > properties that are represented as structs in the QAPI
> > schema cannot be created using -object.
> >
> > This is a design limitation of the way the OptsVisitor
> > is written. It simply iterates over the QemuOpts values
> > as a flat list. The support for lists is enabled by
> > allowing the same key to be repeated in the opts string.
> >
> > It is not practical to extend the OptsVisitor to support
> > more complex data structures while also maintaining
> > the existing list handling behaviour that is relied upon
> > by other areas of QEMU.
>
> It should be practical to create a new option input visitor that parses
> command line syntax straight into a QAPI visit, without the list magic,
> and with fewer or no restrictions. Then we could start defining complex
> command line arguments as QAPI types, parse them straight into generated
> QAPI types, and dispense with the QDict wrangling.
Funnily enough I did actually start out trying to implement pretty
much exactly that. I found that in my new opts visitor, I was essentially
parsing the QemuOpts into QDict/QLists to handle the recursion from the
visitor. At this point I was starting to duplicate much code from QDict
and QmpInputVisitor, so it abandoned that approach and went for the more
general QemuOpts -> QDict crumping instead, since it the goal to be
achieved with simpler code overall.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict
2016-06-14 11:39 ` Daniel P. Berrange
@ 2016-06-16 9:16 ` Markus Armbruster
0 siblings, 0 replies; 25+ messages in thread
From: Markus Armbruster @ 2016-06-16 9:16 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Thu, Jun 09, 2016 at 03:20:47PM +0200, Markus Armbruster wrote:
>> I apologize for the lateness of this review.
>>
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > The qdict_flatten() method will take a dict whose elements are
>> > further nested dicts/lists and flatten them by concatenating
>> > keys.
>> >
>> > The qdict_crumple() method aims to do the reverse, taking a flat
>> > qdict, and turning it into a set of nested dicts/lists. It will
>> > apply nesting based on the key name, with a '.' indicating a
>> > new level in the hierarchy. If the keys in the nested structure
>> > are all numeric, it will create a list, otherwise it will create
>> > a dict.
>> >
>> > If the keys are a mixture of numeric and non-numeric, or the
>> > numeric keys are not in strictly ascending order, an error will
>> > be reported.
>> >
>> > As an example, a flat dict containing
>> >
>> > {
>> > 'foo.0.bar': 'one',
>> > 'foo.0.wizz': '1',
>> > 'foo.1.bar': 'two',
>> > 'foo.1.wizz': '2'
>> > }
>> >
>> > will get turned into a dict with one element 'foo' whose
>> > value is a list. The list elements will each in turn be
>> > dicts.
>> >
>> > {
>> > 'foo': [
>> > { 'bar': 'one', 'wizz': '1' },
>> > { 'bar': 'two', 'wizz': '2' }
>> > ],
>> > }
>> >
>> > If the key is intended to contain a literal '.', then it must
>> > be escaped as '..'. ie a flat dict
>> >
>> > {
>> > 'foo..bar': 'wizz',
>> > 'bar.foo..bar': 'eek',
>> > 'bar.hello': 'world'
>> > }
>> >
>> > Will end up as
>> >
>> > {
>> > 'foo.bar': 'wizz',
>> > 'bar': {
>> > 'foo.bar': 'eek',
>> > 'hello': 'world'
>> > }
>> > }
>> >
>> > The intent of this function is that it allows a set of QemuOpts
>> > to be turned into a nested data structure that mirrors the nesting
>> > used when the same object is defined over QMP.
>> >
>> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>>
>
>> > +/**
>> > + * qdict_split_flat_key:
>> > + * @key: the key string to split
>> > + * @prefix: non-NULL pointer to hold extracted prefix
>> > + * @suffix: non-NULL pointer to hold extracted suffix
>> > + *
>> > + * Given a flattened key such as 'foo.0.bar', split it
>> > + * into two parts at the first '.' separator. Allows
>> > + * double dot ('..') to escape the normal separator.
>> > + *
>> > + * eg
>> > + * 'foo.0.bar' -> prefix='foo' and suffix='0.bar'
>> > + * 'foo..0.bar' -> prefix='foo.0' and suffix='bar'
>> > + *
>> > + * The '..' sequence will be unescaped in the returned
>> > + * 'prefix' string. The 'suffix' string will be left
>> > + * in escaped format, so it can be fed back into the
>> > + * qdict_split_flat_key() key as the input later.
>>
>> Why is the suffix strdup'ed then?
>
> It doesn't need to be - i'll make it const.
>
>
>
>> > +}
>> > +
>> > +
>> > +/**
>> > + * qdict_list_size:
>> > + * @maybe_list: dict that may be only list elements
>>
>> Huh? How can a dictionary "be only list elements"?
>>
>> Do you mean "the dictionary to test?"
>
> I'll say "dict to search for keys representing list elements."
Yes, that's better.
>> > + *
>> > + * Determine whether all keys in @maybe_list are
>> > + * valid list elements. If they are all valid,
>> > + * then this returns the number of elements. If
>> > + * they all look like non-numeric keys, then returns
>> > + * zero. If there is a mix of numeric and non-numeric
>> > + * keys, then an error is set as it is both a list
>> > + * and a dict at once.
>>
>> This is well-defined only if empty @maybe_list is considered to have
>> dict nature, not list nature. Else, return value zero could be the
>> length of the empty list or the special "has dict nature" value.
>>
>> Please spell out behavior for empty @maybe_list.
>
> Yep, empty list will imply qdict nature and so return zero.
>
>> > + *
>> > + * Returns: number of list elements, 0 if a dict, -1 on error
>>
>> Awkward function name. qdict_list_size_if_list() would be clear.
>>
>> But I'd simply turn this into a predicate qdict_is_list(), and have the
>> caller use qdict_size() to get the number of elements.
>
> I thought that qdict_size() would cause another iteration, but
> I see now it just returns a cached size. So I'll switch to
> qidct_is_list().
>
>> > +static ssize_t qdict_list_size(QDict *maybe_list, Error **errp)
>> > +{
>> > + const QDictEntry *entry, *next;
>> > + ssize_t len = 0;
>> > + ssize_t max = -1;
>> > + int is_list = -1;
>> > + int64_t val;
>> > +
>> > + entry = qdict_first(maybe_list);
>> > + while (entry != NULL) {
>> > + next = qdict_next(maybe_list, entry);
>>
>> Please keep the loop control in one place:
>>
>> for (entry = qdict_first(maybe_list); entry; entry = qdict_next(entry)) {
>>
>> I'd rename some variables for less verbiage:
>>
>> for (ent = qdict_first(dict); ent; ent = qdict_next(ent)) {
>>
>> Your loop control also works when the loop body deletes @entry from
>> @maybe_list. Seeing such loop control in a function that isn't supposed
>> to change the its argument makes the reviewer go "WTF?!?" :)
>
> This pattern was left from an earlier version where all the code was in
> one method. I'll change it to a for() loop now.
>
>>
>> > +
>> > + if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
>> > + if (is_list == -1) {
>> > + is_list = 1;
>> > + } else if (!is_list) {
>> > + error_setg(errp,
>> > + "Cannot crumple a dictionary with a mix of list "
>> > + "and non-list keys");
>> > + return -1;
>> > + }
>> > + len++;
>> > + if (val > max) {
>> > + max = val;
>> > + }
>> > + } else {
>> > + if (is_list == -1) {
>> > + is_list = 0;
>> > + } else if (is_list) {
>> > + error_setg(errp,
>> > + "Cannot crumple a dictionary with a mix of list "
>> > + "and non-list keys");
>> > + return -1;
>> > + }
>> > + }
>> > +
>> > + entry = next;
>> > + }
>> > +
>> > + if (is_list == -1) {
>> > + is_list = 0;
>>
>> This can happen only when @maybe_list is empty. Okay, but perhaps you'd
>> like to assert(!qdict_size(maybe_list)).
>
> Ok
>
>>
>> > + }
>> > +
>> > + if (len != (max + 1)) {
>> > + error_setg(errp, "List indexes are not continuous, "
>> > + "saw %zd elements but %zd largest index",
>> > + len, max);
>> > + return -1;
>>
>> contiguous?
>>
>> What if we saw indexes 0, 2, 2?
>
> That would imply that the dict had two entries with the same
> key, which by definition is impossible with a dict data
> structure.
I got confused :)
>> > + }
>> > +
>> > + return is_list ? len : 0;
>> > +}
>> > +
>> > +/**
>> > + * qdict_crumple:
>> > + * @src: the original flat dictionary to crumple
>>
>> "Flat" means all values are scalar. Should we spell that out?
>
> Yep, ok.
>
>> > + * @recursive: true to recursively crumple nested dictionaries
>> > + *
>> > + * Takes a flat dictionary whose keys use '.' separator to
>> > + * indicate nesting, and values are scalars, crumplings it
>>
>> s/, crumplings/, and crumples/
>>
>> > + * into a nested structure. If the @recursive parameter is
>> > + * false, then only the first level of structure implied
>> > + * by the keys will be crumpled. If @recursive is true,
>> > + * then the input will be recursively crumpled to expand
>> > + * all levels of structure in the keys.
>> > + *
>> > + * To include a literal '.' in a key name, it must be escaped
>> > + * as '..'
>> > + *
>> > + * For example, an input of:
>> > + *
>> > + * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
>> > + * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
>> > + *
>> > + * will result in any output of:
>> > + *
>> > + * {
>> > + * 'foo': [
>> > + * { 'bar': 'one', 'wizz': '1' },
>> > + * { 'bar': 'two', 'wizz': '2' }
>> > + * ],
>> > + * }
>> > + *
>> > + * Returns: either a QDict or QList for the nested data structure
>>
>> I think you should discuss how this can fail.
>
> Will do.
>
>> > +QObject *qdict_crumple(QDict *src, bool recursive, Error **errp)
>> > +{
>> > + const QDictEntry *entry, *next;
>> > + QDict *two_level, *multi_level = NULL;
>> > + QObject *dst = NULL, *child;
>> > + ssize_t list_len;
>> > + size_t i;
>> > + char *prefix = NULL, *suffix = NULL;
>> > +
>> > + two_level = qdict_new();
>> > + entry = qdict_first(src);
>> > +
>> > + /* Step 1: split our totally flat dict into a two level dict */
>> > + while (entry != NULL) {
>> > + next = qdict_next(src, entry);
>>
>> Again, keep the loop control in one place.
>>
>> > +
>> > + if (qobject_type(entry->value) == QTYPE_QDICT ||
>> > + qobject_type(entry->value) == QTYPE_QLIST) {
>> > + error_setg(errp, "Value %s is not a scalar",
>> > + entry->key);
>> > + goto error;
>> > + }
>> > +
>> > + qdict_split_flat_key(entry->key, &prefix, &suffix);
>> > +
>> > + child = qdict_get(two_level, prefix);
>> > + if (suffix) {
>> > + if (child) {
>> > + if (qobject_type(child) != QTYPE_QDICT) {
>> > + error_setg(errp, "Key %s prefix is already set as a scalar",
>> > + prefix);
>> > + goto error;
>> > + }
>> > + } else {
>> > + child = QOBJECT(qdict_new());
>> > + qdict_put_obj(two_level, prefix, child);
>> > + }
>> > + qobject_incref(entry->value);
>> > + qdict_put_obj(qobject_to_qdict(child), suffix, entry->value);
>> > + } else {
>> > + if (child) {
>> > + error_setg(errp, "Key %s prefix is already set as a dict",
>> > + prefix);
>> > + goto error;
>> > + }
>> > + qobject_incref(entry->value);
>> > + qdict_put_obj(two_level, prefix, entry->value);
>> > + }
>>
>> Works, because we put only QDicts we've created ourselves (first
>> qdict_put_obj() above) and values we got from @src (second
>> qdict_put_obj()), and we fail when such a value isn't scalar.
>>
>> > +
>> > + g_free(suffix);
>>
>> As I suspected, qdict_split_flat_key() strdup'ing the suffix is useless.
>>
>> > + g_free(prefix);
>> > + suffix = prefix = NULL;
>>
>> Dead stores.
>
> No, they're not dead. The end of this method has a 'g_free(prefix)' so
> we must be sure to clear this out to prevent a double-free if a later
> codebase jumps to the error label.
>
>> > + entry = next;
>> > + }
>> > +
>> > + /* Step 2: optionally process the two level dict recursively
>> > + * into a multi-level dict */
>> > + if (recursive) {
>> > + multi_level = qdict_new();
>> > + entry = qdict_first(two_level);
>> > + while (entry != NULL) {
>> > + next = qdict_next(two_level, entry);
>>
>> Again, keep the loop control in one place.
>
> OK
>
>>
>> > +
>> > + if (qobject_type(entry->value) == QTYPE_QDICT) {
>> > + child = qdict_crumple(qobject_to_qdict(entry->value),
>> > + recursive, errp);
>> > + if (!child) {
>> > + goto error;
>> > + }
>> > +
>> > + qdict_put_obj(multi_level, entry->key, child);
>> > + } else {
>> > + qobject_incref(entry->value);
>> > + qdict_put_obj(multi_level, entry->key, entry->value);
>> > + }
>> > +
>> > + entry = next;
>> > + }
>> > + QDECREF(two_level);
>> > + } else {
>> > + multi_level = two_level;
>> > + }
>> > + two_level = NULL;
>> > +
>> > + /* Step 3: detect if we need to turn our dict into list */
>> > + list_len = qdict_list_size(multi_level, errp);
>> > + if (list_len < 0) {
>> > + goto error;
>> > + }
>> > +
>> > + if (list_len) {
>> > + dst = QOBJECT(qlist_new());
>> > +
>> > + for (i = 0; i < list_len; i++) {
>> > + char *key = g_strdup_printf("%zu", i);
>> > +
>> > + child = qdict_get(multi_level, key);
>> > + g_free(key);
>> > + assert(child);
>>
>> qdict_list_size() accepts as list index any (string) key qemu_strtoll()
>> accepts. If %zu formats it back into the same string, we'll find it
>> here. Else we die. Please spell this out in the function contract.
>
> OK.
>
>> [*] I'm afraid we also die if qdict_list_size()'s "List indexes are not
>> continuous" check gets fooled. Suggest to drop that check, and replace
>> this assertion by error_setg(errp, "Malformed list indexes").
>> Admittedly not the nicest error message; perhaps you can come up with a
>> better one.
>
> We can't be fooled per my note earlier
>
>>
>> > +
>> > + qobject_incref(child);
>> > + qlist_append_obj(qobject_to_qlist(dst), child);
>> > + }
>> > + QDECREF(multi_level);
>>
>> Do we need
>>
>> multi_level = NULL;
>>
>> here?
>
> It isn't needed right now, since we're at the end of the method now
> and nothing will touch this var again. Setting it to NULL is a
> worthwhile safety net for future refactoring though.
I generally don't bother to zap pointers "just in case". I saw the
QDECREF() after the error label, and missed the fact that we can't reach
it from here. Your patch is fine as is.
>> > + } else {
>> > + dst = QOBJECT(multi_level);
>> > + }
>> > +
>> > + return dst;
>> > +
>> > + error:
>> > + g_free(suffix);
>> > + g_free(prefix);
>> > + QDECREF(multi_level);
>> > + QDECREF(two_level);
>> > + qobject_decref(dst);
>> > + return NULL;
>> > +}
>> > +
>> > +
>> > /**
>> > * qdict_array_entries(): Returns the number of direct array entries if the
>> > * sub-QDict of src specified by the prefix in subqdict (or src itself for
>> > diff --git a/tests/check-qdict.c b/tests/check-qdict.c
>> > index a43056c..0d12f40 100644
>> > --- a/tests/check-qdict.c
>> > +++ b/tests/check-qdict.c
>> > @@ -15,6 +15,7 @@
>> > #include "qapi/qmp/qint.h"
>> > #include "qapi/qmp/qdict.h"
>> > #include "qapi/qmp/qstring.h"
>> > +#include "qapi/error.h"
>> > #include "qemu-common.h"
>
>> > +static void qdict_crumple_test_bad_inputs(void)
>> > +{
>> > + QDict *src;
>> > + Error *error = NULL;
>> > +
>> > + src = qdict_new();
>> > + /* rule.0 can't be both a string and a dict */
>> > + qdict_put(src, "rule.0", qstring_from_str("fred"));
>> > + qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
>> > +
>> > + g_assert(qdict_crumple(src, true, &error) == NULL);
>> > + g_assert(error != NULL);
>> > + error_free(error);
>> > + error = NULL;
>> > + QDECREF(src);
>> > +
>> > + src = qdict_new();
>> > + /* rule can't be both a list and a dict */
>> > + qdict_put(src, "rule.0", qstring_from_str("fred"));
>> > + qdict_put(src, "rule.a", qstring_from_str("allow"));
>> > +
>> > + g_assert(qdict_crumple(src, true, &error) == NULL);
>> > + g_assert(error != NULL);
>> > + error_free(error);
>> > + error = NULL;
>> > + QDECREF(src);
>> > +
>> > + src = qdict_new();
>> > + /* The input should be flat, ie no dicts or lists */
>> > + qdict_put(src, "rule.a", qdict_new());
>> > + qdict_put(src, "rule.b", qstring_from_str("allow"));
>> > +
>> > + g_assert(qdict_crumple(src, true, &error) == NULL);
>> > + g_assert(error != NULL);
>> > + error_free(error);
>> > + error = NULL;
>> > + QDECREF(src);
>> > +
>> > +
>> > + src = qdict_new();
>> > + /* List indexes must not have gaps */
>> > + qdict_put(src, "rule.0", qdict_new());
>> > + qdict_put(src, "rule.3", qstring_from_str("allow"));
>> > +
>> > + g_assert(qdict_crumple(src, true, &error) == NULL);
>> > + g_assert(error != NULL);
>> > + error_free(error);
>> > + error = NULL;
>> > + QDECREF(src);
>>
>> Suggest to add test case
>>
>> /* List indexes must not have gaps (more creative version) */
>> qdict_put(src, "rule.0", ...);
>> qdict_put(src, "rule.2", ...);
>> qdict_put(src, "rule.2", ...);
>
> That's surely impossible, as dict keys have to be unique.
It's surely possible that patch review melts my brain :)
>> and
>>
>> /* List indexes must be in %zu format */
>> qdict_put(src, "rule.+0", ...);
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types
2016-06-14 13:25 ` Daniel P. Berrange
@ 2016-06-16 9:23 ` Markus Armbruster
0 siblings, 0 replies; 25+ messages in thread
From: Markus Armbruster @ 2016-06-16 9:23 UTC (permalink / raw)
To: Daniel P. Berrange
Cc: Paolo Bonzini, Andreas Färber, qemu-devel, qemu-block, Max Reitz
"Daniel P. Berrange" <berrange@redhat.com> writes:
> On Thu, Jun 09, 2016 at 04:03:50PM +0200, Markus Armbruster wrote:
>> "Daniel P. Berrange" <berrange@redhat.com> writes:
>>
>> > Currently the QmpInputVisitor assumes that all scalar
>> > values are directly represented as their final types.
>> > ie it assumes an 'int' is using QInt, and a 'bool' is
>> > using QBool.
>> >
>> > This extends it so that QString is optionally permitted
>> > for any of the non-string scalar types. This behaviour
>> > is turned on by requesting the 'autocast' flag in the
>> > constructor.
>> >
>> > This makes it possible to use QmpInputVisitor with a
>> > QDict produced from QemuOpts, where everything is in
>> > string format.
>>
>> We're still digging.
>>
>> > Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
>> > ---
>> > docs/qapi-code-gen.txt | 2 +-
>> > include/qapi/qmp-input-visitor.h | 6 +-
>> > qapi/opts-visitor.c | 1 +
>> > qapi/qmp-input-visitor.c | 89 ++++++++++++++++++++++------
>> > qmp.c | 2 +-
>> > qom/qom-qobject.c | 2 +-
>> > replay/replay-input.c | 2 +-
>> > scripts/qapi-commands.py | 2 +-
>> > tests/check-qnull.c | 2 +-
>> > tests/test-qmp-commands.c | 2 +-
>> > tests/test-qmp-input-strict.c | 2 +-
>> > tests/test-qmp-input-visitor.c | 115 ++++++++++++++++++++++++++++++++++++-
>> > tests/test-visitor-serialization.c | 2 +-
>> > util/qemu-sockets.c | 2 +-
>> > 14 files changed, 201 insertions(+), 30 deletions(-)
>
>
>> > diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
>> > index 4cf1cf8..00e4b1a 100644
>> > --- a/qapi/opts-visitor.c
>> > +++ b/qapi/opts-visitor.c
>> > @@ -347,6 +347,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
>> > }
>> >
>> > if (opt->str) {
>> > + /* Keep these values in sync with same code in qmp-input-visitor.c */
>>
>> Also with parse_option_bool(). That's the canonical place, in fact.
>
> ....except parse_option_bool() doesn't allow "yes", "no", "y", "n" as valid
> values - it only permits "on" and "off" :-( I guess I could make the
> parse_option_bool() method non-static, make it accept these missing values,
> and then call it from all places so we have guaranteed consistency.
Or factor out its parsing guts and put them into cutils.c. Similar to
how qemu_strtol() & friends are (or should be) the common number
parsers.
But that's optional. All I'm asking for this patch is an updated
comment.
>>
>> > if (strcmp(opt->str, "on") == 0 ||
>> > strcmp(opt->str, "yes") == 0 ||
>> > strcmp(opt->str, "y") == 0) {
>> > diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
>> > index aea90a1..92023b1 100644
>> > --- a/qapi/qmp-input-visitor.c
>> > +++ b/qapi/qmp-input-visitor.c
>> > @@ -20,6 +20,7 @@
>> > #include "qemu-common.h"
>> > #include "qapi/qmp/types.h"
>> > #include "qapi/qmp/qerror.h"
>> > +#include "qemu/cutils.h"
>> >
>> > #define QIV_STACK_SIZE 1024
>> >
>> > @@ -45,6 +46,7 @@ struct QmpInputVisitor
>> >
>> > /* True to reject parse in visit_end_struct() if unvisited keys remain. */
>> > bool strict;
>> > + bool autocast;
>> > };
>> >
>> > static QmpInputVisitor *to_qiv(Visitor *v)
>> > @@ -254,15 +256,25 @@ static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
>> > Error **errp)
>> > {
>> > QmpInputVisitor *qiv = to_qiv(v);
>> > - QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
>> > + QObject *qobj = qmp_input_get_object(qiv, name, true);
>> > + QInt *qint;
>> > + QString *qstr;
>> >
>> > - if (!qint) {
>> > - error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>> > - "integer");
>> > + qint = qobject_to_qint(qobj);
>> > + if (qint) {
>> > + *obj = qint_get_int(qint);
>> > return;
>> > }
>> >
>> > - *obj = qint_get_int(qint);
>> > + qstr = qobject_to_qstring(qobj);
>> > + if (qstr && qstr->string && qiv->autocast) {
>> > + if (qemu_strtoll(qstr->string, NULL, 10, obj) == 0) {
>>
>> In the commit message, you mentioned intended use: "with a QDict
>> produced from QemuOpts, where everything is in string format." I figure
>> you mean opts with empty opts->list->desc[]. For those, we accept any
>> key and parse all values as strings.
>>
>> The idea with such QemuOpts is to *delay* parsing until we know how to
>> parse. The delay should be transparent to the user. In particular,
>> values should be parsed the same whether the parsing is delayed or not.
>>
>> The canonical QemuOpts value parser is qemu_opt_parse(). It parses
>> integers with parse_option_number(). That function parses like
>> stroull(qstr->string, ..., 0). Two differences:
>>
>> * stroll() vs. strtoull(): they differ in ERANGE behavior. This might
>> be tolerable, but I haven't thought it through.
>>
>> * Base 0 vs 10: different syntax and semantics. Oops.
>
> Sigh, I originally had my code using strtoull() directly as the
> parse_option_number() method does, but had to change it to make
> checkpatch.pl stfu thus creating incosistency. This is a great
> example of why I hate the fact that we only validate our style
> rules against new patches and not our existing codebase :-(
>
> Anyway, it seems to be something where we should refactor code to
> use the same parsing method from both places. eg by making the
> parse_option_number method non-static and just calling it from
> here.
To get this patch accepted, you need to fix the inconsistency.
Refactoring to improve our chances the inconsistency stays fixed is
welcome, but optional.
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2016-06-16 9:24 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-02 16:46 [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 01/11] qdict: implement a qdict_crumple method for un-flattening a dict Daniel P. Berrange
2016-06-09 13:20 ` Markus Armbruster
2016-06-09 13:28 ` Daniel P. Berrange
2016-06-14 11:39 ` Daniel P. Berrange
2016-06-16 9:16 ` Markus Armbruster
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 02/11] qapi: allow QmpInputVisitor to auto-cast types Daniel P. Berrange
2016-06-08 12:01 ` Paolo Bonzini
2016-06-14 14:10 ` Daniel P. Berrange
2016-06-09 14:03 ` Markus Armbruster
2016-06-14 13:25 ` Daniel P. Berrange
2016-06-16 9:23 ` Markus Armbruster
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object Daniel P. Berrange
2016-06-09 14:43 ` Markus Armbruster
2016-06-14 14:16 ` Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 04/11] util: add QAuthZ object as an authorization base class Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 05/11] util: add QAuthZSimple object type for a simple access control list Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 06/11] acl: delete existing ACL implementation Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 07/11] qemu-nbd: add support for ACLs for TLS clients Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 08/11] nbd: allow an ACL to be set with nbd-server-start QMP command Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 09/11] migration: add support for a "tls-acl" migration parameter Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 10/11] chardev: add support for ACLs for TLS clients Daniel P. Berrange
2016-06-02 16:46 ` [Qemu-devel] [PATCH v5 11/11] vnc: allow specifying a custom ACL object name Daniel P. Berrange
2016-06-08 11:53 ` [Qemu-devel] [PATCH v5 00/11] Provide a QOM-based authorization API Daniel P. Berrange
2016-06-08 14:26 ` Markus Armbruster
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.