* [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate @ 2019-08-08 15:03 Marc-André Lureau 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 1/2] qemu-file: move qemu_{get, put}_counted_string() declarations Marc-André Lureau ` (2 more replies) 0 siblings, 3 replies; 28+ messages in thread From: Marc-André Lureau @ 2019-08-08 15:03 UTC (permalink / raw) To: qemu-devel Cc: Laurent Vivier, Thomas Huth, Juan Quintela, Dr. David Alan Gilbert, Paolo Bonzini, Marc-André Lureau Hi, With external processes or helpers participating to the VM support, it becomes necessary to handle their migration. Various options exist to transfer their state: 1) as the VM memory, RAM or devices (we could say that's how vhost-user devices can be handled today, they are expected to restore from ring state) 2) other "vmstate" (as with TPM emulator state blobs) 3) left to be handled by management layer 1) is not practical, since an external processes may legitimatelly need arbitrary state date to back a device or a service, or may not even have an associated device. 2) needs ad-hoc code for each helper, but is simple and working 3) is complicated for management layer, QEMU has the migration timing The proposed "dbus-vmstate" object will connect to a given D-Bus peer address, and save/load from org.qemu.VMState1 interface. This way, helpers can have their state migrated with QEMU, without implementing another ad-hoc support (such as done for TPM emulation) I chose D-Bus as it is ubiquitous on Linux (it is systemd IPC), and can be made to work on various other OSes. There are several implementations and good bindings for various languages. (the tests/dbus-vmstate-test.c is a good example of how simple the implementation of services can be, even in C) v2: - D-Bus is most common and practical through a bus, but it requires a daemon to be running. I argue that the benefits outweight the cost of running an extra daemon in v1 in the context of multi-process qemu, but it is also possible to connect in p2p mode as done in this new version. dbus-vmstate is put into use by the libvirt series "[PATCH v2 00/23] Use a slirp helper process". Marc-André Lureau (2): qemu-file: move qemu_{get,put}_counted_string() declarations Add dbus-vmstate object MAINTAINERS | 6 + backends/Makefile.objs | 4 + backends/dbus-vmstate.c | 332 +++++++++++++++++++++++++ configure | 7 + docs/interop/dbus-vmstate.rst | 63 +++++ docs/interop/index.rst | 1 + include/migration/qemu-file-types.h | 4 + migration/qemu-file.h | 4 - tests/Makefile.include | 18 +- tests/dbus-vmstate-test.c | 371 ++++++++++++++++++++++++++++ tests/dbus-vmstate1.xml | 12 + 11 files changed, 817 insertions(+), 5 deletions(-) create mode 100644 backends/dbus-vmstate.c create mode 100644 docs/interop/dbus-vmstate.rst create mode 100644 tests/dbus-vmstate-test.c create mode 100644 tests/dbus-vmstate1.xml -- 2.23.0.rc1 ^ permalink raw reply [flat|nested] 28+ messages in thread
* [Qemu-devel] [PATCH v2 1/2] qemu-file: move qemu_{get, put}_counted_string() declarations 2019-08-08 15:03 [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Marc-André Lureau @ 2019-08-08 15:03 ` Marc-André Lureau 2019-08-09 18:32 ` Dr. David Alan Gilbert 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object Marc-André Lureau 2019-08-23 11:20 ` [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Daniel P. Berrangé 2 siblings, 1 reply; 28+ messages in thread From: Marc-André Lureau @ 2019-08-08 15:03 UTC (permalink / raw) To: qemu-devel Cc: Laurent Vivier, Thomas Huth, Juan Quintela, Dr. David Alan Gilbert, Paolo Bonzini, Marc-André Lureau Move migration helpers for strings under include/, so they can be used outside of migration/ Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> --- include/migration/qemu-file-types.h | 4 ++++ migration/qemu-file.h | 4 ---- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/migration/qemu-file-types.h b/include/migration/qemu-file-types.h index c0a1988155..2867e3da84 100644 --- a/include/migration/qemu-file-types.h +++ b/include/migration/qemu-file-types.h @@ -161,6 +161,10 @@ static inline void qemu_get_sbe64s(QEMUFile *f, int64_t *pv) qemu_get_be64s(f, (uint64_t *)pv); } +size_t qemu_get_counted_string(QEMUFile *f, char buf[256]); + +void qemu_put_counted_string(QEMUFile *f, const char *name); + int qemu_file_rate_limit(QEMUFile *f); #endif diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 13baf896bd..185d3de505 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -155,8 +155,6 @@ QEMUFile *qemu_file_get_return_path(QEMUFile *f); void qemu_fflush(QEMUFile *f); void qemu_file_set_blocking(QEMUFile *f, bool block); -size_t qemu_get_counted_string(QEMUFile *f, char buf[256]); - void ram_control_before_iterate(QEMUFile *f, uint64_t flags); void ram_control_after_iterate(QEMUFile *f, uint64_t flags); void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data); @@ -175,6 +173,4 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size, uint64_t *bytes_sent); -void qemu_put_counted_string(QEMUFile *f, const char *name); - #endif -- 2.23.0.rc1 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 1/2] qemu-file: move qemu_{get, put}_counted_string() declarations 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 1/2] qemu-file: move qemu_{get, put}_counted_string() declarations Marc-André Lureau @ 2019-08-09 18:32 ` Dr. David Alan Gilbert 0 siblings, 0 replies; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-09 18:32 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > Move migration helpers for strings under include/, so they can be used > outside of migration/ > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > Reviewed-by: Juan Quintela <quintela@redhat.com> I've queued this one (just 1/2) for the migration pull I'll do as soon as 4.2 opens. > --- > include/migration/qemu-file-types.h | 4 ++++ > migration/qemu-file.h | 4 ---- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/include/migration/qemu-file-types.h b/include/migration/qemu-file-types.h > index c0a1988155..2867e3da84 100644 > --- a/include/migration/qemu-file-types.h > +++ b/include/migration/qemu-file-types.h > @@ -161,6 +161,10 @@ static inline void qemu_get_sbe64s(QEMUFile *f, int64_t *pv) > qemu_get_be64s(f, (uint64_t *)pv); > } > > +size_t qemu_get_counted_string(QEMUFile *f, char buf[256]); > + > +void qemu_put_counted_string(QEMUFile *f, const char *name); > + > int qemu_file_rate_limit(QEMUFile *f); > > #endif > diff --git a/migration/qemu-file.h b/migration/qemu-file.h > index 13baf896bd..185d3de505 100644 > --- a/migration/qemu-file.h > +++ b/migration/qemu-file.h > @@ -155,8 +155,6 @@ QEMUFile *qemu_file_get_return_path(QEMUFile *f); > void qemu_fflush(QEMUFile *f); > void qemu_file_set_blocking(QEMUFile *f, bool block); > > -size_t qemu_get_counted_string(QEMUFile *f, char buf[256]); > - > void ram_control_before_iterate(QEMUFile *f, uint64_t flags); > void ram_control_after_iterate(QEMUFile *f, uint64_t flags); > void ram_control_load_hook(QEMUFile *f, uint64_t flags, void *data); > @@ -175,6 +173,4 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset, > ram_addr_t offset, size_t size, > uint64_t *bytes_sent); > > -void qemu_put_counted_string(QEMUFile *f, const char *name); > - > #endif > -- > 2.23.0.rc1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-08 15:03 [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Marc-André Lureau 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 1/2] qemu-file: move qemu_{get, put}_counted_string() declarations Marc-André Lureau @ 2019-08-08 15:03 ` Marc-André Lureau 2019-08-08 15:07 ` Marc-André Lureau 2019-08-22 10:55 ` Dr. David Alan Gilbert 2019-08-23 11:20 ` [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Daniel P. Berrangé 2 siblings, 2 replies; 28+ messages in thread From: Marc-André Lureau @ 2019-08-08 15:03 UTC (permalink / raw) To: qemu-devel Cc: Laurent Vivier, Thomas Huth, Juan Quintela, Dr. David Alan Gilbert, Paolo Bonzini, Marc-André Lureau When instanciated, this object will connect to the given D-Bus bus. During migration, it will take the data from org.qemu.VMState1 instances. See documentation for further details. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> --- MAINTAINERS | 6 + backends/Makefile.objs | 4 + backends/dbus-vmstate.c | 332 ++++++++++++++++++++++++++++++ configure | 7 + docs/interop/dbus-vmstate.rst | 63 ++++++ docs/interop/index.rst | 1 + tests/Makefile.include | 18 +- tests/dbus-vmstate-test.c | 371 ++++++++++++++++++++++++++++++++++ tests/dbus-vmstate1.xml | 12 ++ 9 files changed, 813 insertions(+), 1 deletion(-) create mode 100644 backends/dbus-vmstate.c create mode 100644 docs/interop/dbus-vmstate.rst create mode 100644 tests/dbus-vmstate-test.c create mode 100644 tests/dbus-vmstate1.xml diff --git a/MAINTAINERS b/MAINTAINERS index d6de200453..d136bf4f4b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2143,6 +2143,12 @@ F: tests/migration-test.c F: docs/devel/migration.rst F: qapi/migration.json +DBus VMState +M: Marc-André Lureau <marcandre.lureau@redhat.com> +S: Maintained +F: backends/dbus-vmstate.c +F: tests/dbus-vmstate* + Seccomp M: Eduardo Otubo <otubo@redhat.com> S: Supported diff --git a/backends/Makefile.objs b/backends/Makefile.objs index 981e8e122f..dbbe12b225 100644 --- a/backends/Makefile.objs +++ b/backends/Makefile.objs @@ -17,3 +17,7 @@ endif common-obj-$(call land,$(CONFIG_VHOST_USER),$(CONFIG_VIRTIO)) += vhost-user.o common-obj-$(CONFIG_LINUX) += hostmem-memfd.o + +common-obj-$(CONFIG_GIO) += dbus-vmstate.o +dbus-vmstate.o-cflags = $(GIO_CFLAGS) +dbus-vmstate.o-libs = $(GIO_LIBS) diff --git a/backends/dbus-vmstate.c b/backends/dbus-vmstate.c new file mode 100644 index 0000000000..a9d9cac388 --- /dev/null +++ b/backends/dbus-vmstate.c @@ -0,0 +1,332 @@ +/* + * QEMU dbus-vmstate + * + * Copyright (C) 2019 Red Hat Inc + * + * Authors: + * Marc-André Lureau <marcandre.lureau@redhat.com> + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/error-report.h" +#include "qapi/error.h" +#include "qom/object_interfaces.h" +#include "qapi/qmp/qerror.h" +#include "migration/register.h" +#include "migration/qemu-file-types.h" +#include <gio/gio.h> + +typedef struct DBusVMState DBusVMState; +typedef struct DBusVMStateClass DBusVMStateClass; + +#define TYPE_DBUS_VMSTATE "dbus-vmstate" +#define DBUS_VMSTATE(obj) \ + OBJECT_CHECK(DBusVMState, (obj), TYPE_DBUS_VMSTATE) +#define DBUS_VMSTATE_GET_CLASS(obj) \ + OBJECT_GET_CLASS(DBusVMStateClass, (obj), TYPE_DBUS_VMSTATE) +#define DBUS_VMSTATE_CLASS(klass) \ + OBJECT_CLASS_CHECK(DBusVMStateClass, (klass), TYPE_DBUS_VMSTATE) + +struct DBusVMStateClass { + ObjectClass parent_class; +}; + +struct DBusVMState { + Object parent; + + GDBusConnection *bus; + GDBusProxy *proxy; + char *id; + char *dbus_addr; +}; + +static const GDBusPropertyInfo vmstate_property_info[] = { + { -1, (char *) "Id", (char *) "s", + G_DBUS_PROPERTY_INFO_FLAGS_READABLE, NULL }, +}; + +static const GDBusPropertyInfo * const vmstate_property_info_pointers[] = { + &vmstate_property_info[0], + NULL +}; + +static const GDBusInterfaceInfo vmstate1_interface_info = { + -1, + (char *) "org.qemu.VMState1", + (GDBusMethodInfo **) NULL, + (GDBusSignalInfo **) NULL, + (GDBusPropertyInfo **) &vmstate_property_info_pointers, + NULL, +}; + +#define DBUS_VMSTATE_SIZE_LIMIT (1 << 20) /* 1mb */ + + +static char * +get_idstr(DBusVMState *self) +{ + return g_strdup_printf("%s-%s", TYPE_DBUS_VMSTATE, self->id); +} + +static char * +dbus_proxy_get_id(GDBusProxy *proxy, GError **err) +{ + char *id = NULL; + GVariant *result = NULL; + size_t size; + + result = g_dbus_proxy_get_cached_property(proxy, "Id"); + if (!result) { + g_set_error_literal(err, G_IO_ERROR, G_IO_ERROR_FAILED, + "Failed to get Id property"); + return NULL; + } + + id = g_variant_dup_string(result, &size); + g_clear_pointer(&result, g_variant_unref); + + if (size == 0 || size >= 256) { + g_set_error_literal(err, G_IO_ERROR, G_IO_ERROR_FAILED, + "Invalid Id property"); + g_free(id); + return NULL; + } + + return id; +} + +static int +dbus_load_state_proxy(GDBusProxy *proxy, const uint8_t *data, size_t size) +{ + GError *err = NULL; + GVariant *value, *result; + int ret = -1; + + value = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, + data, size, sizeof(char)); + result = g_dbus_proxy_call_sync(proxy, "Load", + g_variant_new("(@ay)", value), + G_DBUS_CALL_FLAGS_NO_AUTO_START, + -1, NULL, &err); + if (!result) { + error_report("Failed to Load: %s", err->message); + goto end; + } + + ret = 0; + +end: + g_clear_pointer(&result, g_variant_unref); + g_clear_error(&err); + return ret; +} + +static int +dbus_load_state(QEMUFile *f, void *opaque, int version_id) +{ + DBusVMState *self = DBUS_VMSTATE(opaque); + uint8_t *data = NULL; + int ret = -1; + char id[256]; + unsigned int size; + + if (qemu_get_counted_string(f, id) == 0) { + error_report("Invalid vmstate Id"); + goto end; + } + + if (g_strcmp0(id, self->id)) { + error_report("Invalid vmstate Id: %s != %s", id, self->id); + goto end; + } + + size = qemu_get_be32(f); + if (size > DBUS_VMSTATE_SIZE_LIMIT) { + error_report("Invalid vmstate size: %u", size); + goto end; + } + + data = g_malloc(size); + if (qemu_get_buffer(f, data, size) != size) { + error_report("Failed to read %u bytes", size); + goto end; + } + + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { + error_report("Failed to restore Id '%s'", id); + goto end; + } + + ret = 0; + +end: + g_clear_pointer(&data, g_free); + return ret; +} + +static void +dbus_save_state(QEMUFile *f, void *opaque) +{ + DBusVMState *self = DBUS_VMSTATE(opaque); + GVariant *result = NULL, *child = NULL; + const uint8_t *data; + size_t size; + GError *err = NULL; + + result = g_dbus_proxy_call_sync(self->proxy, "Save", + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, + -1, NULL, &err); + if (!result) { + error_report("Failed to Save: %s", err->message); + g_clear_error(&err); + goto end; + } + + child = g_variant_get_child_value(result, 0); + data = g_variant_get_fixed_array(child, &size, sizeof(char)); + if (!data) { + error_report("Failed to Save: not a byte array"); + goto end; + } + if (size > DBUS_VMSTATE_SIZE_LIMIT) { + error_report("Too much vmstate data to save: %zu", size); + goto end; + } + + qemu_put_counted_string(f, self->id); + qemu_put_be32(f, size); + qemu_put_buffer(f, data, size); + +end: + g_clear_pointer(&child, g_variant_unref); + g_clear_pointer(&result, g_variant_unref); +} + +static const SaveVMHandlers savevm_handlers = { + .save_state = dbus_save_state, + .load_state = dbus_load_state, +}; + +static void +dbus_vmstate_complete(UserCreatable *uc, Error **errp) +{ + DBusVMState *self = DBUS_VMSTATE(uc); + GError *err = NULL; + GDBusConnection *bus = NULL; + GDBusProxy *proxy = NULL; + char *idstr = NULL; + + if (!self->dbus_addr) { + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); + return; + } + + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, + NULL, NULL, &err); + if (err) { + error_setg(errp, "failed to connect to DBus: '%s'", err->message); + g_clear_error(&err); + return; + } + + self->bus = bus; + + proxy = g_dbus_proxy_new_sync(bus, + G_DBUS_PROXY_FLAGS_NONE, + (GDBusInterfaceInfo *) &vmstate1_interface_info, + NULL, + "/org/qemu/VMState1", + "org.qemu.VMState1", + NULL, &err); + + if (err) { + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); + g_clear_error(&err); + return; + } + + self->proxy = proxy; + + self->id = dbus_proxy_get_id(proxy, &err); + if (!self->id) { + error_setg(errp, "failed to get DBus Id: '%s'", err->message); + g_clear_error(&err); + return; + } + + idstr = get_idstr(self); + if (register_savevm_live(NULL, idstr, 0, 0, + &savevm_handlers, self) < 0) { + error_setg(errp, "Failed to register savevm handler"); + } + g_free(idstr); +} + +static void +dbus_vmstate_finalize(Object *o) +{ + DBusVMState *self = DBUS_VMSTATE(o); + char *idstr = get_idstr(self); + + unregister_savevm(NULL, idstr, self); + + g_clear_object(&self->bus); + g_clear_object(&self->proxy); + g_free(self->dbus_addr); + g_free(self->id); + g_free(idstr); +} + +static char * +get_dbus_addr(Object *o, Error **errp) +{ + DBusVMState *self = DBUS_VMSTATE(o); + + return g_strdup(self->dbus_addr); +} + +static void +set_dbus_addr(Object *o, const char *str, Error **errp) +{ + DBusVMState *self = DBUS_VMSTATE(o); + + g_free(self->dbus_addr); + self->dbus_addr = g_strdup(str); +} + +static void +dbus_vmstate_class_init(ObjectClass *oc, void *data) +{ + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); + + ucc->complete = dbus_vmstate_complete; + + object_class_property_add_str(oc, "addr", + get_dbus_addr, set_dbus_addr, + &error_abort); +} + +static const TypeInfo dbus_vmstate_info = { + .name = TYPE_DBUS_VMSTATE, + .parent = TYPE_OBJECT, + .instance_size = sizeof(DBusVMState), + .instance_finalize = dbus_vmstate_finalize, + .class_size = sizeof(DBusVMStateClass), + .class_init = dbus_vmstate_class_init, + .interfaces = (InterfaceInfo[]) { + { TYPE_USER_CREATABLE }, + { } + } +}; + +static void +register_types(void) +{ + type_register_static(&dbus_vmstate_info); +} + +type_init(register_types); diff --git a/configure b/configure index 714e7fb6a1..e5b34f5ca7 100755 --- a/configure +++ b/configure @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then gio=yes gio_cflags=$($pkg_config --cflags gio-2.0) gio_libs=$($pkg_config --libs gio-2.0) + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) else gio=no fi +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" +fi + # Sanity check that the current size_t matches the # size that glib thinks it should be. This catches # problems on multi-arch where people try to build @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then echo "CONFIG_GIO=y" >> $config_host_mak echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak echo "GIO_LIBS=$gio_libs" >> $config_host_mak + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak fi echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak if test "$gnutls" = "yes" ; then diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst new file mode 100644 index 0000000000..4a32a183fb --- /dev/null +++ b/docs/interop/dbus-vmstate.rst @@ -0,0 +1,63 @@ +============ +DBus VMState +============ + +Introduction +============ + +Helper processes may have their state migrated with the help of +QEMU "dbus-vmstate" objects. + +At this point, the connection to the helper is done in DBus +peer-to-peer mode (no initial Hello, and no bus name for +communication). The helper must be listening to the given address. + +Helper may save arbitrary data to be transferred in the migration +stream and restored/loaded on destination. + +The data amount to be transferred is limited to 1Mb. The state must be +saved quickly (a few seconds maximum). (DBus imposes a time limit on +reply anyway, and migration would fail if the data isn't given quickly +enough) + +Interface +========= + +On /org/qemu/VMState1 object path: + +.. code:: xml + + <interface name="org.qemu.VMState1"> + <property name="Id" type="s" access="read"/> + <method name="Load"> + <arg type="ay" name="data" direction="in"/> + </method> + <method name="Save"> + <arg type="ay" name="data" direction="out"/> + </method> + </interface> + +"Id" property +------------- + +A utf8 encoded string that identifies the helper uniquely. +Must be <256 bytes. + +Load(in u8[] bytes) method +-------------------------- + +The method called on destination with the state to restore. + +The helper may be initially started in a waiting state (with +an --incoming argument for example), and it may resume on load +success. + +An error may be returned to the caller. + +Save(out u8[] bytes) method +--------------------------- + +The method called on the source to get the current state to be +migrated. The helper should continue to run normally. + +An error may be returned to the caller. diff --git a/docs/interop/index.rst b/docs/interop/index.rst index b4bfcab417..6bb173cfa6 100644 --- a/docs/interop/index.rst +++ b/docs/interop/index.rst @@ -13,6 +13,7 @@ Contents: :maxdepth: 2 bitmaps + dbus-vmstate live-block-operations pr-helper vhost-user diff --git a/tests/Makefile.include b/tests/Makefile.include index fd7fdb8658..2c610086a7 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) - +ifneq ($(GDBUS_CODEGEN),) +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) +endif check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) check-qtest-i386-y += tests/fdc-test$(EXESUF) check-qtest-i386-y += tests/ide-test$(EXESUF) @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml + $(call quiet-command,$(GDBUS_CODEGEN) $< \ + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ + "GEN","$(@:%-timestamp=%)") + @>$@ + +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) + +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h + tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o @@ -1172,6 +1187,7 @@ check-clean: rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) rm -f tests/test-qapi-gen-timestamp + rm -f tests/dbus-vmstate1-gen-timestamp rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) clean: check-clean diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c new file mode 100644 index 0000000000..45d44916d2 --- /dev/null +++ b/tests/dbus-vmstate-test.c @@ -0,0 +1,371 @@ +#include "qemu/osdep.h" +#include <glib/gstdio.h> +#include <gio/gio.h> +#include "libqtest.h" +#include "qemu-common.h" +#include "dbus-vmstate1.h" + +static char *workdir; + +typedef struct TestServerId { + const char *name; + const char *data; + size_t size; +} TestServerId; + +static const TestServerId idA = { + "idA", "I'am\0idA!", sizeof("I'am\0idA!") +}; + +static const TestServerId idB = { + "idB", "I'am\0idB!", sizeof("I'am\0idB!") +}; + +typedef struct TestServer { + const TestServerId *id; + bool save_called; + bool load_called; + GDBusObjectManagerServer *om; + GDBusServer *server; +} TestServer; + +typedef struct Test { + bool migrate_fail; + TestServer srcA; + TestServer dstA; + TestServer srcB; + TestServer dstB; + GMainLoop *loop, *dbus_loop; + QTestState *src_qemu; +} Test; + +GMutex mutex; +GCond cond; + +static gboolean +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, + const gchar *arg_data, gpointer user_data) +{ + TestServer *h = user_data; + GVariant *args, *var; + const uint8_t *data; + size_t size; + + args = g_dbus_method_invocation_get_parameters(invocation); + var = g_variant_get_child_value(args, 0); + data = g_variant_get_fixed_array(var, &size, sizeof(char)); + g_assert_cmpuint(size, ==, h->id->size); + g_assert(!memcmp(data, h->id->data, h->id->size)); + h->load_called = true; + g_variant_unref(var); + + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); + return TRUE; +} + +static gboolean +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, + gpointer user_data) +{ + TestServer *h = user_data; + GVariant *var; + + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, + h->id->data, h->id->size, sizeof(char)); + g_dbus_method_invocation_return_value(invocation, + g_variant_new("(@ay)", var)); + h->save_called = true; + + return TRUE; +} + +static void +connection_closed(GDBusConnection *connection, + gboolean remote_peer_vanished, + GError *Error, + gpointer user_data) +{ + TestServer *h = user_data; + + g_clear_object(&h->om); + g_clear_object(&connection); +} + +static GDBusObjectManagerServer * +get_omserver(GDBusConnection *conn, gpointer user_data) +{ + TestServer *h = user_data; + GDBusObjectManagerServer *om; + GDBusObjectSkeleton *sk; + VMState1 *v; + + om = g_dbus_object_manager_server_new("/org/qemu"); + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); + + v = vmstate1_skeleton_new(); + g_object_set(v, "id", h->id->name, NULL); + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); + + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); + g_dbus_object_manager_server_export(om, sk); + g_dbus_object_manager_server_set_connection(om, conn); + + g_clear_object(&v); + g_clear_object(&sk); + + return om; +} + +static gboolean +on_new_connection(GDBusServer *server, + GDBusConnection *connection, + gpointer user_data) +{ + TestServer *h = user_data; + + g_object_ref(connection); + g_signal_connect(connection, "closed", + G_CALLBACK(connection_closed), user_data); + h->om = get_omserver(connection, user_data); + + return TRUE; +} + +static gboolean +allow_mechanism_cb(GDBusAuthObserver *observer, + const gchar *mechanism, + gpointer user_data) +{ + return g_strcmp0(mechanism, "EXTERNAL") == 0; +} + +static gboolean +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, + GIOStream *stream, + GCredentials *credentials, + gpointer user_data) +{ + gboolean authorized = FALSE; + + if (credentials != NULL) { + GCredentials *own_credentials = g_credentials_new(); + + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { + authorized = TRUE; + } + + g_clear_object(&own_credentials); + } + + return authorized; +} + +static GDBusServer * +server_start(TestServer *h, const char *p, const TestServerId *id) +{ + GDBusAuthObserver *observer = NULL; + GDBusServer *server = NULL; + gchar *guid = NULL; + GError *error = NULL; + char *addr = NULL; + + h->id = id; + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); + guid = g_dbus_generate_guid(); + observer = g_dbus_auth_observer_new(); + g_signal_connect(observer, "allow-mechanism", + G_CALLBACK(allow_mechanism_cb), h); + g_signal_connect(observer, "authorize-authenticated-peer", + G_CALLBACK(authorize_authenticated_peer_cb), h); + + server = g_dbus_server_new_sync(addr, + G_DBUS_SERVER_FLAGS_NONE, + guid, + observer, + NULL, /* GCancellable */ + &error); + g_dbus_server_start(server); + g_clear_object(&observer); + g_free(guid); + + if (server == NULL) { + g_printerr("Error creating server at address %s: %s\n", + addr, error->message); + g_error_free(error); + return NULL; + } + + g_signal_connect(server, "new-connection", + G_CALLBACK(on_new_connection), h); + + g_free(addr); + return server; +} + + +static gpointer +dbus_thread(gpointer p) +{ + Test *test = p; + GMainContext *context = g_main_context_new(); + GMainLoop *loop = g_main_loop_new(context, FALSE); + + g_main_context_push_thread_default(context); + + g_mutex_lock(&mutex); + test->srcA.server = server_start(&test->srcA, "src", &idA); + test->srcB.server = server_start(&test->srcB, "src", &idB); + test->dstA.server = server_start(&test->dstA, "dst", &idA); + test->dstB.server = server_start(&test->dstB, "dst", &idB); + test->dbus_loop = loop; + g_cond_signal(&cond); + g_mutex_unlock(&mutex); + + g_main_loop_run(loop); + + g_main_loop_unref(loop); + g_main_context_unref(context); + + g_mutex_lock(&mutex); + g_clear_object(&test->srcA.server); + g_clear_object(&test->srcB.server); + g_clear_object(&test->dstA.server); + g_clear_object(&test->dstB.server); + g_mutex_unlock(&mutex); + + return NULL; +} + +static gboolean +wait_for_migration_complete(gpointer user_data) +{ + Test *test = user_data; + QDict *rsp_return; + bool stop = false; + const char *status; + + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); + status = qdict_get_str(rsp_return, "status"); + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { + stop = true; + g_assert_cmpstr(status, ==, + test->migrate_fail ? "failed" : "completed"); + } + qobject_unref(rsp_return); + + if (stop) { + g_main_loop_quit(test->loop); + } + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; +} + +static void migrate(QTestState *who, const char *uri) +{ + QDict *args, *rsp; + + args = qdict_new(); + qdict_put_str(args, "uri", uri); + + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); + + g_assert(qdict_haskey(rsp, "return")); + qobject_unref(rsp); +} + +static void +test_dbus_vmstate(Test *test) +{ + QTestState *src_qemu = NULL, *dst_qemu = NULL; + char *src_qemu_args = NULL, *dst_qemu_args = NULL; + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); + GThread *t = g_thread_new("dbus", dbus_thread, test); + + g_mutex_lock(&mutex); + while (!test->dbus_loop) { + g_cond_wait(&cond, &mutex); + } + + src_qemu_args = + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " + "-object dbus-vmstate,id=dvB,addr=%s", + g_dbus_server_get_client_address(test->srcA.server), + g_dbus_server_get_client_address(test->srcB.server)); + + + dst_qemu_args = + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " + "-object dbus-vmstate,id=dvB,addr=%s " + "-incoming %s", + g_dbus_server_get_client_address(test->dstA.server), + g_dbus_server_get_client_address(test->dstB.server), + uri); + + src_qemu = qtest_init(src_qemu_args); + dst_qemu = qtest_init(dst_qemu_args); + + test->loop = g_main_loop_new(NULL, TRUE); + + migrate(src_qemu, uri); + test->src_qemu = src_qemu; + g_timeout_add_seconds(1, wait_for_migration_complete, test); + + g_main_loop_run(test->loop); + g_main_loop_unref(test->loop); + + g_free(uri); + qtest_quit(dst_qemu); + qtest_quit(src_qemu); + g_free(dst_qemu_args); + g_free(src_qemu_args); + + g_main_loop_quit(test->dbus_loop); + g_mutex_unlock(&mutex); + + g_thread_join(t); +} + +static void +check_migrated(TestServer *s, TestServer *d) +{ + assert(s->save_called); + assert(!s->load_called); + assert(!d->save_called); + assert(d->load_called); +} + +static void +test_dbus_vmstate_migrate(void) +{ + Test test = { }; + + test_dbus_vmstate(&test); + + check_migrated(&test.srcA, &test.dstA); + check_migrated(&test.srcB, &test.dstB); +} + +int +main(int argc, char **argv) +{ + GError *err = NULL; + int ret; + + g_test_init(&argc, &argv, NULL); + + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); + if (!workdir) { + g_error("Unable to create temporary dir: %s\n", err->message); + } + + qtest_add_func("/dbus-vmstate/migrate", + test_dbus_vmstate_migrate); + + ret = g_test_run(); + + rmdir(workdir); + g_free(workdir); + + return ret; +} diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml new file mode 100644 index 0000000000..cc8563be4c --- /dev/null +++ b/tests/dbus-vmstate1.xml @@ -0,0 +1,12 @@ +<?xml version="1.0"?> +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> + <interface name="org.qemu.VMState1"> + <property name="Id" type="s" access="read"/> + <method name="Load"> + <arg type="ay" name="data" direction="in"/> + </method> + <method name="Save"> + <arg type="ay" name="data" direction="out"/> + </method> + </interface> +</node> -- 2.23.0.rc1 ^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object Marc-André Lureau @ 2019-08-08 15:07 ` Marc-André Lureau 2019-08-22 10:55 ` Dr. David Alan Gilbert 1 sibling, 0 replies; 28+ messages in thread From: Marc-André Lureau @ 2019-08-08 15:07 UTC (permalink / raw) To: qemu-devel Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, Dr. David Alan Gilbert, Juan Quintela Hi On Thu, Aug 8, 2019 at 7:03 PM Marc-André Lureau <marcandre.lureau@redhat.com> wrote: > > When instanciated, this object will connect to the given D-Bus > bus. During migration, it will take the data from org.qemu.VMState1 > instances. I forgot to update the commit message. When instantiated, this object will connect to the given D-Bus peer. During migration, it will load/save the data from the org.qemu.VMState1 interface. See documentation for further details. > > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > --- > MAINTAINERS | 6 + > backends/Makefile.objs | 4 + > backends/dbus-vmstate.c | 332 ++++++++++++++++++++++++++++++ > configure | 7 + > docs/interop/dbus-vmstate.rst | 63 ++++++ > docs/interop/index.rst | 1 + > tests/Makefile.include | 18 +- > tests/dbus-vmstate-test.c | 371 ++++++++++++++++++++++++++++++++++ > tests/dbus-vmstate1.xml | 12 ++ > 9 files changed, 813 insertions(+), 1 deletion(-) > create mode 100644 backends/dbus-vmstate.c > create mode 100644 docs/interop/dbus-vmstate.rst > create mode 100644 tests/dbus-vmstate-test.c > create mode 100644 tests/dbus-vmstate1.xml > > diff --git a/MAINTAINERS b/MAINTAINERS > index d6de200453..d136bf4f4b 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -2143,6 +2143,12 @@ F: tests/migration-test.c > F: docs/devel/migration.rst > F: qapi/migration.json > > +DBus VMState > +M: Marc-André Lureau <marcandre.lureau@redhat.com> > +S: Maintained > +F: backends/dbus-vmstate.c > +F: tests/dbus-vmstate* > + > Seccomp > M: Eduardo Otubo <otubo@redhat.com> > S: Supported > diff --git a/backends/Makefile.objs b/backends/Makefile.objs > index 981e8e122f..dbbe12b225 100644 > --- a/backends/Makefile.objs > +++ b/backends/Makefile.objs > @@ -17,3 +17,7 @@ endif > common-obj-$(call land,$(CONFIG_VHOST_USER),$(CONFIG_VIRTIO)) += vhost-user.o > > common-obj-$(CONFIG_LINUX) += hostmem-memfd.o > + > +common-obj-$(CONFIG_GIO) += dbus-vmstate.o > +dbus-vmstate.o-cflags = $(GIO_CFLAGS) > +dbus-vmstate.o-libs = $(GIO_LIBS) > diff --git a/backends/dbus-vmstate.c b/backends/dbus-vmstate.c > new file mode 100644 > index 0000000000..a9d9cac388 > --- /dev/null > +++ b/backends/dbus-vmstate.c > @@ -0,0 +1,332 @@ > +/* > + * QEMU dbus-vmstate > + * > + * Copyright (C) 2019 Red Hat Inc > + * > + * Authors: > + * Marc-André Lureau <marcandre.lureau@redhat.com> > + * > + * This work is licensed under the terms of the GNU GPL, version 2 or later. > + * See the COPYING file in the top-level directory. > + */ > + > +#include "qemu/osdep.h" > +#include "qemu/error-report.h" > +#include "qapi/error.h" > +#include "qom/object_interfaces.h" > +#include "qapi/qmp/qerror.h" > +#include "migration/register.h" > +#include "migration/qemu-file-types.h" > +#include <gio/gio.h> > + > +typedef struct DBusVMState DBusVMState; > +typedef struct DBusVMStateClass DBusVMStateClass; > + > +#define TYPE_DBUS_VMSTATE "dbus-vmstate" > +#define DBUS_VMSTATE(obj) \ > + OBJECT_CHECK(DBusVMState, (obj), TYPE_DBUS_VMSTATE) > +#define DBUS_VMSTATE_GET_CLASS(obj) \ > + OBJECT_GET_CLASS(DBusVMStateClass, (obj), TYPE_DBUS_VMSTATE) > +#define DBUS_VMSTATE_CLASS(klass) \ > + OBJECT_CLASS_CHECK(DBusVMStateClass, (klass), TYPE_DBUS_VMSTATE) > + > +struct DBusVMStateClass { > + ObjectClass parent_class; > +}; > + > +struct DBusVMState { > + Object parent; > + > + GDBusConnection *bus; > + GDBusProxy *proxy; > + char *id; > + char *dbus_addr; > +}; > + > +static const GDBusPropertyInfo vmstate_property_info[] = { > + { -1, (char *) "Id", (char *) "s", > + G_DBUS_PROPERTY_INFO_FLAGS_READABLE, NULL }, > +}; > + > +static const GDBusPropertyInfo * const vmstate_property_info_pointers[] = { > + &vmstate_property_info[0], > + NULL > +}; > + > +static const GDBusInterfaceInfo vmstate1_interface_info = { > + -1, > + (char *) "org.qemu.VMState1", > + (GDBusMethodInfo **) NULL, > + (GDBusSignalInfo **) NULL, > + (GDBusPropertyInfo **) &vmstate_property_info_pointers, > + NULL, > +}; > + > +#define DBUS_VMSTATE_SIZE_LIMIT (1 << 20) /* 1mb */ > + > + > +static char * > +get_idstr(DBusVMState *self) > +{ > + return g_strdup_printf("%s-%s", TYPE_DBUS_VMSTATE, self->id); > +} > + > +static char * > +dbus_proxy_get_id(GDBusProxy *proxy, GError **err) > +{ > + char *id = NULL; > + GVariant *result = NULL; > + size_t size; > + > + result = g_dbus_proxy_get_cached_property(proxy, "Id"); > + if (!result) { > + g_set_error_literal(err, G_IO_ERROR, G_IO_ERROR_FAILED, > + "Failed to get Id property"); > + return NULL; > + } > + > + id = g_variant_dup_string(result, &size); > + g_clear_pointer(&result, g_variant_unref); > + > + if (size == 0 || size >= 256) { > + g_set_error_literal(err, G_IO_ERROR, G_IO_ERROR_FAILED, > + "Invalid Id property"); > + g_free(id); > + return NULL; > + } > + > + return id; > +} > + > +static int > +dbus_load_state_proxy(GDBusProxy *proxy, const uint8_t *data, size_t size) > +{ > + GError *err = NULL; > + GVariant *value, *result; > + int ret = -1; > + > + value = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > + data, size, sizeof(char)); > + result = g_dbus_proxy_call_sync(proxy, "Load", > + g_variant_new("(@ay)", value), > + G_DBUS_CALL_FLAGS_NO_AUTO_START, > + -1, NULL, &err); > + if (!result) { > + error_report("Failed to Load: %s", err->message); > + goto end; > + } > + > + ret = 0; > + > +end: > + g_clear_pointer(&result, g_variant_unref); > + g_clear_error(&err); > + return ret; > +} > + > +static int > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > +{ > + DBusVMState *self = DBUS_VMSTATE(opaque); > + uint8_t *data = NULL; > + int ret = -1; > + char id[256]; > + unsigned int size; > + > + if (qemu_get_counted_string(f, id) == 0) { > + error_report("Invalid vmstate Id"); > + goto end; > + } > + > + if (g_strcmp0(id, self->id)) { > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > + goto end; > + } > + > + size = qemu_get_be32(f); > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > + error_report("Invalid vmstate size: %u", size); > + goto end; > + } > + > + data = g_malloc(size); > + if (qemu_get_buffer(f, data, size) != size) { > + error_report("Failed to read %u bytes", size); > + goto end; > + } > + > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > + error_report("Failed to restore Id '%s'", id); > + goto end; > + } > + > + ret = 0; > + > +end: > + g_clear_pointer(&data, g_free); > + return ret; > +} > + > +static void > +dbus_save_state(QEMUFile *f, void *opaque) > +{ > + DBusVMState *self = DBUS_VMSTATE(opaque); > + GVariant *result = NULL, *child = NULL; > + const uint8_t *data; > + size_t size; > + GError *err = NULL; > + > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > + -1, NULL, &err); > + if (!result) { > + error_report("Failed to Save: %s", err->message); > + g_clear_error(&err); > + goto end; > + } > + > + child = g_variant_get_child_value(result, 0); > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > + if (!data) { > + error_report("Failed to Save: not a byte array"); > + goto end; > + } > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > + error_report("Too much vmstate data to save: %zu", size); > + goto end; > + } > + > + qemu_put_counted_string(f, self->id); > + qemu_put_be32(f, size); > + qemu_put_buffer(f, data, size); > + > +end: > + g_clear_pointer(&child, g_variant_unref); > + g_clear_pointer(&result, g_variant_unref); > +} > + > +static const SaveVMHandlers savevm_handlers = { > + .save_state = dbus_save_state, > + .load_state = dbus_load_state, > +}; > + > +static void > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > +{ > + DBusVMState *self = DBUS_VMSTATE(uc); > + GError *err = NULL; > + GDBusConnection *bus = NULL; > + GDBusProxy *proxy = NULL; > + char *idstr = NULL; > + > + if (!self->dbus_addr) { > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > + return; > + } > + > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > + NULL, NULL, &err); > + if (err) { > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > + g_clear_error(&err); > + return; > + } > + > + self->bus = bus; > + > + proxy = g_dbus_proxy_new_sync(bus, > + G_DBUS_PROXY_FLAGS_NONE, > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > + NULL, > + "/org/qemu/VMState1", > + "org.qemu.VMState1", > + NULL, &err); > + > + if (err) { > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > + g_clear_error(&err); > + return; > + } > + > + self->proxy = proxy; > + > + self->id = dbus_proxy_get_id(proxy, &err); > + if (!self->id) { > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > + g_clear_error(&err); > + return; > + } > + > + idstr = get_idstr(self); > + if (register_savevm_live(NULL, idstr, 0, 0, > + &savevm_handlers, self) < 0) { > + error_setg(errp, "Failed to register savevm handler"); > + } > + g_free(idstr); > +} > + > +static void > +dbus_vmstate_finalize(Object *o) > +{ > + DBusVMState *self = DBUS_VMSTATE(o); > + char *idstr = get_idstr(self); > + > + unregister_savevm(NULL, idstr, self); > + > + g_clear_object(&self->bus); > + g_clear_object(&self->proxy); > + g_free(self->dbus_addr); > + g_free(self->id); > + g_free(idstr); > +} > + > +static char * > +get_dbus_addr(Object *o, Error **errp) > +{ > + DBusVMState *self = DBUS_VMSTATE(o); > + > + return g_strdup(self->dbus_addr); > +} > + > +static void > +set_dbus_addr(Object *o, const char *str, Error **errp) > +{ > + DBusVMState *self = DBUS_VMSTATE(o); > + > + g_free(self->dbus_addr); > + self->dbus_addr = g_strdup(str); > +} > + > +static void > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > +{ > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > + > + ucc->complete = dbus_vmstate_complete; > + > + object_class_property_add_str(oc, "addr", > + get_dbus_addr, set_dbus_addr, > + &error_abort); > +} > + > +static const TypeInfo dbus_vmstate_info = { > + .name = TYPE_DBUS_VMSTATE, > + .parent = TYPE_OBJECT, > + .instance_size = sizeof(DBusVMState), > + .instance_finalize = dbus_vmstate_finalize, > + .class_size = sizeof(DBusVMStateClass), > + .class_init = dbus_vmstate_class_init, > + .interfaces = (InterfaceInfo[]) { > + { TYPE_USER_CREATABLE }, > + { } > + } > +}; > + > +static void > +register_types(void) > +{ > + type_register_static(&dbus_vmstate_info); > +} > + > +type_init(register_types); > diff --git a/configure b/configure > index 714e7fb6a1..e5b34f5ca7 100755 > --- a/configure > +++ b/configure > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > gio=yes > gio_cflags=$($pkg_config --cflags gio-2.0) > gio_libs=$($pkg_config --libs gio-2.0) > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > else > gio=no > fi > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > +fi > + > # Sanity check that the current size_t matches the > # size that glib thinks it should be. This catches > # problems on multi-arch where people try to build > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > echo "CONFIG_GIO=y" >> $config_host_mak > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > fi > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > if test "$gnutls" = "yes" ; then > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > new file mode 100644 > index 0000000000..4a32a183fb > --- /dev/null > +++ b/docs/interop/dbus-vmstate.rst > @@ -0,0 +1,63 @@ > +============ > +DBus VMState > +============ > + > +Introduction > +============ > + > +Helper processes may have their state migrated with the help of > +QEMU "dbus-vmstate" objects. > + > +At this point, the connection to the helper is done in DBus > +peer-to-peer mode (no initial Hello, and no bus name for > +communication). The helper must be listening to the given address. > + > +Helper may save arbitrary data to be transferred in the migration > +stream and restored/loaded on destination. > + > +The data amount to be transferred is limited to 1Mb. The state must be > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > +reply anyway, and migration would fail if the data isn't given quickly > +enough) > + > +Interface > +========= > + > +On /org/qemu/VMState1 object path: > + > +.. code:: xml > + > + <interface name="org.qemu.VMState1"> > + <property name="Id" type="s" access="read"/> > + <method name="Load"> > + <arg type="ay" name="data" direction="in"/> > + </method> > + <method name="Save"> > + <arg type="ay" name="data" direction="out"/> > + </method> > + </interface> > + > +"Id" property > +------------- > + > +A utf8 encoded string that identifies the helper uniquely. > +Must be <256 bytes. > + > +Load(in u8[] bytes) method > +-------------------------- > + > +The method called on destination with the state to restore. > + > +The helper may be initially started in a waiting state (with > +an --incoming argument for example), and it may resume on load > +success. > + > +An error may be returned to the caller. > + > +Save(out u8[] bytes) method > +--------------------------- > + > +The method called on the source to get the current state to be > +migrated. The helper should continue to run normally. > + > +An error may be returned to the caller. > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > index b4bfcab417..6bb173cfa6 100644 > --- a/docs/interop/index.rst > +++ b/docs/interop/index.rst > @@ -13,6 +13,7 @@ Contents: > :maxdepth: 2 > > bitmaps > + dbus-vmstate > live-block-operations > pr-helper > vhost-user > diff --git a/tests/Makefile.include b/tests/Makefile.include > index fd7fdb8658..2c610086a7 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > - > +ifneq ($(GDBUS_CODEGEN),) > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > +endif > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > check-qtest-i386-y += tests/fdc-test$(EXESUF) > check-qtest-i386-y += tests/ide-test$(EXESUF) > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > + "GEN","$(@:%-timestamp=%)") > + @>$@ > + > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > + > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > + > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > @@ -1172,6 +1187,7 @@ check-clean: > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > rm -f tests/test-qapi-gen-timestamp > + rm -f tests/dbus-vmstate1-gen-timestamp > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > clean: check-clean > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > new file mode 100644 > index 0000000000..45d44916d2 > --- /dev/null > +++ b/tests/dbus-vmstate-test.c > @@ -0,0 +1,371 @@ > +#include "qemu/osdep.h" > +#include <glib/gstdio.h> > +#include <gio/gio.h> > +#include "libqtest.h" > +#include "qemu-common.h" > +#include "dbus-vmstate1.h" > + > +static char *workdir; > + > +typedef struct TestServerId { > + const char *name; > + const char *data; > + size_t size; > +} TestServerId; > + > +static const TestServerId idA = { > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > +}; > + > +static const TestServerId idB = { > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > +}; > + > +typedef struct TestServer { > + const TestServerId *id; > + bool save_called; > + bool load_called; > + GDBusObjectManagerServer *om; > + GDBusServer *server; > +} TestServer; > + > +typedef struct Test { > + bool migrate_fail; > + TestServer srcA; > + TestServer dstA; > + TestServer srcB; > + TestServer dstB; > + GMainLoop *loop, *dbus_loop; > + QTestState *src_qemu; > +} Test; > + > +GMutex mutex; > +GCond cond; > + > +static gboolean > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > + const gchar *arg_data, gpointer user_data) > +{ > + TestServer *h = user_data; > + GVariant *args, *var; > + const uint8_t *data; > + size_t size; > + > + args = g_dbus_method_invocation_get_parameters(invocation); > + var = g_variant_get_child_value(args, 0); > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > + g_assert_cmpuint(size, ==, h->id->size); > + g_assert(!memcmp(data, h->id->data, h->id->size)); > + h->load_called = true; > + g_variant_unref(var); > + > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > + return TRUE; > +} > + > +static gboolean > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > + gpointer user_data) > +{ > + TestServer *h = user_data; > + GVariant *var; > + > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > + h->id->data, h->id->size, sizeof(char)); > + g_dbus_method_invocation_return_value(invocation, > + g_variant_new("(@ay)", var)); > + h->save_called = true; > + > + return TRUE; > +} > + > +static void > +connection_closed(GDBusConnection *connection, > + gboolean remote_peer_vanished, > + GError *Error, > + gpointer user_data) > +{ > + TestServer *h = user_data; > + > + g_clear_object(&h->om); > + g_clear_object(&connection); > +} > + > +static GDBusObjectManagerServer * > +get_omserver(GDBusConnection *conn, gpointer user_data) > +{ > + TestServer *h = user_data; > + GDBusObjectManagerServer *om; > + GDBusObjectSkeleton *sk; > + VMState1 *v; > + > + om = g_dbus_object_manager_server_new("/org/qemu"); > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > + > + v = vmstate1_skeleton_new(); > + g_object_set(v, "id", h->id->name, NULL); > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > + > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > + g_dbus_object_manager_server_export(om, sk); > + g_dbus_object_manager_server_set_connection(om, conn); > + > + g_clear_object(&v); > + g_clear_object(&sk); > + > + return om; > +} > + > +static gboolean > +on_new_connection(GDBusServer *server, > + GDBusConnection *connection, > + gpointer user_data) > +{ > + TestServer *h = user_data; > + > + g_object_ref(connection); > + g_signal_connect(connection, "closed", > + G_CALLBACK(connection_closed), user_data); > + h->om = get_omserver(connection, user_data); > + > + return TRUE; > +} > + > +static gboolean > +allow_mechanism_cb(GDBusAuthObserver *observer, > + const gchar *mechanism, > + gpointer user_data) > +{ > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > +} > + > +static gboolean > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > + GIOStream *stream, > + GCredentials *credentials, > + gpointer user_data) > +{ > + gboolean authorized = FALSE; > + > + if (credentials != NULL) { > + GCredentials *own_credentials = g_credentials_new(); > + > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > + authorized = TRUE; > + } > + > + g_clear_object(&own_credentials); > + } > + > + return authorized; > +} > + > +static GDBusServer * > +server_start(TestServer *h, const char *p, const TestServerId *id) > +{ > + GDBusAuthObserver *observer = NULL; > + GDBusServer *server = NULL; > + gchar *guid = NULL; > + GError *error = NULL; > + char *addr = NULL; > + > + h->id = id; > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > + guid = g_dbus_generate_guid(); > + observer = g_dbus_auth_observer_new(); > + g_signal_connect(observer, "allow-mechanism", > + G_CALLBACK(allow_mechanism_cb), h); > + g_signal_connect(observer, "authorize-authenticated-peer", > + G_CALLBACK(authorize_authenticated_peer_cb), h); > + > + server = g_dbus_server_new_sync(addr, > + G_DBUS_SERVER_FLAGS_NONE, > + guid, > + observer, > + NULL, /* GCancellable */ > + &error); > + g_dbus_server_start(server); > + g_clear_object(&observer); > + g_free(guid); > + > + if (server == NULL) { > + g_printerr("Error creating server at address %s: %s\n", > + addr, error->message); > + g_error_free(error); > + return NULL; > + } > + > + g_signal_connect(server, "new-connection", > + G_CALLBACK(on_new_connection), h); > + > + g_free(addr); > + return server; > +} > + > + > +static gpointer > +dbus_thread(gpointer p) > +{ > + Test *test = p; > + GMainContext *context = g_main_context_new(); > + GMainLoop *loop = g_main_loop_new(context, FALSE); > + > + g_main_context_push_thread_default(context); > + > + g_mutex_lock(&mutex); > + test->srcA.server = server_start(&test->srcA, "src", &idA); > + test->srcB.server = server_start(&test->srcB, "src", &idB); > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > + test->dbus_loop = loop; > + g_cond_signal(&cond); > + g_mutex_unlock(&mutex); > + > + g_main_loop_run(loop); > + > + g_main_loop_unref(loop); > + g_main_context_unref(context); > + > + g_mutex_lock(&mutex); > + g_clear_object(&test->srcA.server); > + g_clear_object(&test->srcB.server); > + g_clear_object(&test->dstA.server); > + g_clear_object(&test->dstB.server); > + g_mutex_unlock(&mutex); > + > + return NULL; > +} > + > +static gboolean > +wait_for_migration_complete(gpointer user_data) > +{ > + Test *test = user_data; > + QDict *rsp_return; > + bool stop = false; > + const char *status; > + > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > + status = qdict_get_str(rsp_return, "status"); > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > + stop = true; > + g_assert_cmpstr(status, ==, > + test->migrate_fail ? "failed" : "completed"); > + } > + qobject_unref(rsp_return); > + > + if (stop) { > + g_main_loop_quit(test->loop); > + } > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > +} > + > +static void migrate(QTestState *who, const char *uri) > +{ > + QDict *args, *rsp; > + > + args = qdict_new(); > + qdict_put_str(args, "uri", uri); > + > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > + > + g_assert(qdict_haskey(rsp, "return")); > + qobject_unref(rsp); > +} > + > +static void > +test_dbus_vmstate(Test *test) > +{ > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > + GThread *t = g_thread_new("dbus", dbus_thread, test); > + > + g_mutex_lock(&mutex); > + while (!test->dbus_loop) { > + g_cond_wait(&cond, &mutex); > + } > + > + src_qemu_args = > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > + "-object dbus-vmstate,id=dvB,addr=%s", > + g_dbus_server_get_client_address(test->srcA.server), > + g_dbus_server_get_client_address(test->srcB.server)); > + > + > + dst_qemu_args = > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > + "-object dbus-vmstate,id=dvB,addr=%s " > + "-incoming %s", > + g_dbus_server_get_client_address(test->dstA.server), > + g_dbus_server_get_client_address(test->dstB.server), > + uri); > + > + src_qemu = qtest_init(src_qemu_args); > + dst_qemu = qtest_init(dst_qemu_args); > + > + test->loop = g_main_loop_new(NULL, TRUE); > + > + migrate(src_qemu, uri); > + test->src_qemu = src_qemu; > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > + > + g_main_loop_run(test->loop); > + g_main_loop_unref(test->loop); > + > + g_free(uri); > + qtest_quit(dst_qemu); > + qtest_quit(src_qemu); > + g_free(dst_qemu_args); > + g_free(src_qemu_args); > + > + g_main_loop_quit(test->dbus_loop); > + g_mutex_unlock(&mutex); > + > + g_thread_join(t); > +} > + > +static void > +check_migrated(TestServer *s, TestServer *d) > +{ > + assert(s->save_called); > + assert(!s->load_called); > + assert(!d->save_called); > + assert(d->load_called); > +} > + > +static void > +test_dbus_vmstate_migrate(void) > +{ > + Test test = { }; > + > + test_dbus_vmstate(&test); > + > + check_migrated(&test.srcA, &test.dstA); > + check_migrated(&test.srcB, &test.dstB); > +} > + > +int > +main(int argc, char **argv) > +{ > + GError *err = NULL; > + int ret; > + > + g_test_init(&argc, &argv, NULL); > + > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > + if (!workdir) { > + g_error("Unable to create temporary dir: %s\n", err->message); > + } > + > + qtest_add_func("/dbus-vmstate/migrate", > + test_dbus_vmstate_migrate); > + > + ret = g_test_run(); > + > + rmdir(workdir); > + g_free(workdir); > + > + return ret; > +} > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > new file mode 100644 > index 0000000000..cc8563be4c > --- /dev/null > +++ b/tests/dbus-vmstate1.xml > @@ -0,0 +1,12 @@ > +<?xml version="1.0"?> > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > + <interface name="org.qemu.VMState1"> > + <property name="Id" type="s" access="read"/> > + <method name="Load"> > + <arg type="ay" name="data" direction="in"/> > + </method> > + <method name="Save"> > + <arg type="ay" name="data" direction="out"/> > + </method> > + </interface> > +</node> > -- > 2.23.0.rc1 > ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object Marc-André Lureau 2019-08-08 15:07 ` Marc-André Lureau @ 2019-08-22 10:55 ` Dr. David Alan Gilbert 2019-08-22 11:35 ` Marc-André Lureau 1 sibling, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-22 10:55 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > When instanciated, this object will connect to the given D-Bus > bus. During migration, it will take the data from org.qemu.VMState1 > instances. > > See documentation for further details. I'll leave the main review to someone who understands the details of DBUS, however from the migration side: > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> <snip> > + > +static int > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > +{ > + DBusVMState *self = DBUS_VMSTATE(opaque); > + uint8_t *data = NULL; > + int ret = -1; > + char id[256]; > + unsigned int size; Higher level question; do you actually need a separate ID, or does the ID that's stored by the migration code in the section name give you enough uniqueness? > + if (qemu_get_counted_string(f, id) == 0) { > + error_report("Invalid vmstate Id"); > + goto end; I generally prefer to include something telling me where the error has come from, just to make it easier to track down if I see the error in a log; e.g. use __func__ > + } > + > + if (g_strcmp0(id, self->id)) { > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > + goto end; > + } > + > + size = qemu_get_be32(f); > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > + error_report("Invalid vmstate size: %u", size); > + goto end; > + } > + > + data = g_malloc(size); > + if (qemu_get_buffer(f, data, size) != size) { > + error_report("Failed to read %u bytes", size); > + goto end; > + } > + > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > + error_report("Failed to restore Id '%s'", id); > + goto end; > + } > + > + ret = 0; > + > +end: > + g_clear_pointer(&data, g_free); > + return ret; > +} > + > +static void > +dbus_save_state(QEMUFile *f, void *opaque) > +{ > + DBusVMState *self = DBUS_VMSTATE(opaque); > + GVariant *result = NULL, *child = NULL; > + const uint8_t *data; > + size_t size; > + GError *err = NULL; > + > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > + -1, NULL, &err); > + if (!result) { > + error_report("Failed to Save: %s", err->message); > + g_clear_error(&err); > + goto end; > + } > + > + child = g_variant_get_child_value(result, 0); > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > + if (!data) { > + error_report("Failed to Save: not a byte array"); > + goto end; > + } > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > + error_report("Too much vmstate data to save: %zu", size); > + goto end; > + } > + > + qemu_put_counted_string(f, self->id); > + qemu_put_be32(f, size); > + qemu_put_buffer(f, data, size); > + > +end: > + g_clear_pointer(&child, g_variant_unref); > + g_clear_pointer(&result, g_variant_unref); > +} > + > +static const SaveVMHandlers savevm_handlers = { > + .save_state = dbus_save_state, > + .load_state = dbus_load_state, > +}; > + > +static void > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > +{ > + DBusVMState *self = DBUS_VMSTATE(uc); > + GError *err = NULL; > + GDBusConnection *bus = NULL; > + GDBusProxy *proxy = NULL; > + char *idstr = NULL; > + > + if (!self->dbus_addr) { > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > + return; > + } > + > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > + NULL, NULL, &err); > + if (err) { > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > + g_clear_error(&err); > + return; > + } > + > + self->bus = bus; > + > + proxy = g_dbus_proxy_new_sync(bus, > + G_DBUS_PROXY_FLAGS_NONE, > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > + NULL, > + "/org/qemu/VMState1", > + "org.qemu.VMState1", > + NULL, &err); > + > + if (err) { > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > + g_clear_error(&err); > + return; > + } > + > + self->proxy = proxy; > + > + self->id = dbus_proxy_get_id(proxy, &err); > + if (!self->id) { > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > + g_clear_error(&err); > + return; > + } > + > + idstr = get_idstr(self); > + if (register_savevm_live(NULL, idstr, 0, 0, > + &savevm_handlers, self) < 0) { > + error_setg(errp, "Failed to register savevm handler"); > + } Can you try and avoid register_savevm_live if possible and just wire a vmsd onto the class like most other devices? We don't have many register_savevm_live calls, and I'd like to get them down to just the iterative devices. Dave > + g_free(idstr); > +} > + > +static void > +dbus_vmstate_finalize(Object *o) > +{ > + DBusVMState *self = DBUS_VMSTATE(o); > + char *idstr = get_idstr(self); > + > + unregister_savevm(NULL, idstr, self); > + > + g_clear_object(&self->bus); > + g_clear_object(&self->proxy); > + g_free(self->dbus_addr); > + g_free(self->id); > + g_free(idstr); > +} > + > +static char * > +get_dbus_addr(Object *o, Error **errp) > +{ > + DBusVMState *self = DBUS_VMSTATE(o); > + > + return g_strdup(self->dbus_addr); > +} > + > +static void > +set_dbus_addr(Object *o, const char *str, Error **errp) > +{ > + DBusVMState *self = DBUS_VMSTATE(o); > + > + g_free(self->dbus_addr); > + self->dbus_addr = g_strdup(str); > +} > + > +static void > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > +{ > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > + > + ucc->complete = dbus_vmstate_complete; > + > + object_class_property_add_str(oc, "addr", > + get_dbus_addr, set_dbus_addr, > + &error_abort); > +} > + > +static const TypeInfo dbus_vmstate_info = { > + .name = TYPE_DBUS_VMSTATE, > + .parent = TYPE_OBJECT, > + .instance_size = sizeof(DBusVMState), > + .instance_finalize = dbus_vmstate_finalize, > + .class_size = sizeof(DBusVMStateClass), > + .class_init = dbus_vmstate_class_init, > + .interfaces = (InterfaceInfo[]) { > + { TYPE_USER_CREATABLE }, > + { } > + } > +}; > + > +static void > +register_types(void) > +{ > + type_register_static(&dbus_vmstate_info); > +} > + > +type_init(register_types); > diff --git a/configure b/configure > index 714e7fb6a1..e5b34f5ca7 100755 > --- a/configure > +++ b/configure > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > gio=yes > gio_cflags=$($pkg_config --cflags gio-2.0) > gio_libs=$($pkg_config --libs gio-2.0) > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > else > gio=no > fi > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > +fi > + > # Sanity check that the current size_t matches the > # size that glib thinks it should be. This catches > # problems on multi-arch where people try to build > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > echo "CONFIG_GIO=y" >> $config_host_mak > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > fi > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > if test "$gnutls" = "yes" ; then > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > new file mode 100644 > index 0000000000..4a32a183fb > --- /dev/null > +++ b/docs/interop/dbus-vmstate.rst > @@ -0,0 +1,63 @@ > +============ > +DBus VMState > +============ > + > +Introduction > +============ > + > +Helper processes may have their state migrated with the help of > +QEMU "dbus-vmstate" objects. > + > +At this point, the connection to the helper is done in DBus > +peer-to-peer mode (no initial Hello, and no bus name for > +communication). The helper must be listening to the given address. > + > +Helper may save arbitrary data to be transferred in the migration > +stream and restored/loaded on destination. > + > +The data amount to be transferred is limited to 1Mb. The state must be > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > +reply anyway, and migration would fail if the data isn't given quickly > +enough) > + > +Interface > +========= > + > +On /org/qemu/VMState1 object path: > + > +.. code:: xml > + > + <interface name="org.qemu.VMState1"> > + <property name="Id" type="s" access="read"/> > + <method name="Load"> > + <arg type="ay" name="data" direction="in"/> > + </method> > + <method name="Save"> > + <arg type="ay" name="data" direction="out"/> > + </method> > + </interface> > + > +"Id" property > +------------- > + > +A utf8 encoded string that identifies the helper uniquely. > +Must be <256 bytes. > + > +Load(in u8[] bytes) method > +-------------------------- > + > +The method called on destination with the state to restore. > + > +The helper may be initially started in a waiting state (with > +an --incoming argument for example), and it may resume on load > +success. > + > +An error may be returned to the caller. > + > +Save(out u8[] bytes) method > +--------------------------- > + > +The method called on the source to get the current state to be > +migrated. The helper should continue to run normally. > + > +An error may be returned to the caller. > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > index b4bfcab417..6bb173cfa6 100644 > --- a/docs/interop/index.rst > +++ b/docs/interop/index.rst > @@ -13,6 +13,7 @@ Contents: > :maxdepth: 2 > > bitmaps > + dbus-vmstate > live-block-operations > pr-helper > vhost-user > diff --git a/tests/Makefile.include b/tests/Makefile.include > index fd7fdb8658..2c610086a7 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > - > +ifneq ($(GDBUS_CODEGEN),) > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > +endif > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > check-qtest-i386-y += tests/fdc-test$(EXESUF) > check-qtest-i386-y += tests/ide-test$(EXESUF) > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > + "GEN","$(@:%-timestamp=%)") > + @>$@ > + > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > + > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > + > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > @@ -1172,6 +1187,7 @@ check-clean: > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > rm -f tests/test-qapi-gen-timestamp > + rm -f tests/dbus-vmstate1-gen-timestamp > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > clean: check-clean > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > new file mode 100644 > index 0000000000..45d44916d2 > --- /dev/null > +++ b/tests/dbus-vmstate-test.c > @@ -0,0 +1,371 @@ > +#include "qemu/osdep.h" > +#include <glib/gstdio.h> > +#include <gio/gio.h> > +#include "libqtest.h" > +#include "qemu-common.h" > +#include "dbus-vmstate1.h" > + > +static char *workdir; > + > +typedef struct TestServerId { > + const char *name; > + const char *data; > + size_t size; > +} TestServerId; > + > +static const TestServerId idA = { > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > +}; > + > +static const TestServerId idB = { > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > +}; > + > +typedef struct TestServer { > + const TestServerId *id; > + bool save_called; > + bool load_called; > + GDBusObjectManagerServer *om; > + GDBusServer *server; > +} TestServer; > + > +typedef struct Test { > + bool migrate_fail; > + TestServer srcA; > + TestServer dstA; > + TestServer srcB; > + TestServer dstB; > + GMainLoop *loop, *dbus_loop; > + QTestState *src_qemu; > +} Test; > + > +GMutex mutex; > +GCond cond; > + > +static gboolean > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > + const gchar *arg_data, gpointer user_data) > +{ > + TestServer *h = user_data; > + GVariant *args, *var; > + const uint8_t *data; > + size_t size; > + > + args = g_dbus_method_invocation_get_parameters(invocation); > + var = g_variant_get_child_value(args, 0); > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > + g_assert_cmpuint(size, ==, h->id->size); > + g_assert(!memcmp(data, h->id->data, h->id->size)); > + h->load_called = true; > + g_variant_unref(var); > + > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > + return TRUE; > +} > + > +static gboolean > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > + gpointer user_data) > +{ > + TestServer *h = user_data; > + GVariant *var; > + > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > + h->id->data, h->id->size, sizeof(char)); > + g_dbus_method_invocation_return_value(invocation, > + g_variant_new("(@ay)", var)); > + h->save_called = true; > + > + return TRUE; > +} > + > +static void > +connection_closed(GDBusConnection *connection, > + gboolean remote_peer_vanished, > + GError *Error, > + gpointer user_data) > +{ > + TestServer *h = user_data; > + > + g_clear_object(&h->om); > + g_clear_object(&connection); > +} > + > +static GDBusObjectManagerServer * > +get_omserver(GDBusConnection *conn, gpointer user_data) > +{ > + TestServer *h = user_data; > + GDBusObjectManagerServer *om; > + GDBusObjectSkeleton *sk; > + VMState1 *v; > + > + om = g_dbus_object_manager_server_new("/org/qemu"); > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > + > + v = vmstate1_skeleton_new(); > + g_object_set(v, "id", h->id->name, NULL); > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > + > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > + g_dbus_object_manager_server_export(om, sk); > + g_dbus_object_manager_server_set_connection(om, conn); > + > + g_clear_object(&v); > + g_clear_object(&sk); > + > + return om; > +} > + > +static gboolean > +on_new_connection(GDBusServer *server, > + GDBusConnection *connection, > + gpointer user_data) > +{ > + TestServer *h = user_data; > + > + g_object_ref(connection); > + g_signal_connect(connection, "closed", > + G_CALLBACK(connection_closed), user_data); > + h->om = get_omserver(connection, user_data); > + > + return TRUE; > +} > + > +static gboolean > +allow_mechanism_cb(GDBusAuthObserver *observer, > + const gchar *mechanism, > + gpointer user_data) > +{ > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > +} > + > +static gboolean > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > + GIOStream *stream, > + GCredentials *credentials, > + gpointer user_data) > +{ > + gboolean authorized = FALSE; > + > + if (credentials != NULL) { > + GCredentials *own_credentials = g_credentials_new(); > + > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > + authorized = TRUE; > + } > + > + g_clear_object(&own_credentials); > + } > + > + return authorized; > +} > + > +static GDBusServer * > +server_start(TestServer *h, const char *p, const TestServerId *id) > +{ > + GDBusAuthObserver *observer = NULL; > + GDBusServer *server = NULL; > + gchar *guid = NULL; > + GError *error = NULL; > + char *addr = NULL; > + > + h->id = id; > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > + guid = g_dbus_generate_guid(); > + observer = g_dbus_auth_observer_new(); > + g_signal_connect(observer, "allow-mechanism", > + G_CALLBACK(allow_mechanism_cb), h); > + g_signal_connect(observer, "authorize-authenticated-peer", > + G_CALLBACK(authorize_authenticated_peer_cb), h); > + > + server = g_dbus_server_new_sync(addr, > + G_DBUS_SERVER_FLAGS_NONE, > + guid, > + observer, > + NULL, /* GCancellable */ > + &error); > + g_dbus_server_start(server); > + g_clear_object(&observer); > + g_free(guid); > + > + if (server == NULL) { > + g_printerr("Error creating server at address %s: %s\n", > + addr, error->message); > + g_error_free(error); > + return NULL; > + } > + > + g_signal_connect(server, "new-connection", > + G_CALLBACK(on_new_connection), h); > + > + g_free(addr); > + return server; > +} > + > + > +static gpointer > +dbus_thread(gpointer p) > +{ > + Test *test = p; > + GMainContext *context = g_main_context_new(); > + GMainLoop *loop = g_main_loop_new(context, FALSE); > + > + g_main_context_push_thread_default(context); > + > + g_mutex_lock(&mutex); > + test->srcA.server = server_start(&test->srcA, "src", &idA); > + test->srcB.server = server_start(&test->srcB, "src", &idB); > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > + test->dbus_loop = loop; > + g_cond_signal(&cond); > + g_mutex_unlock(&mutex); > + > + g_main_loop_run(loop); > + > + g_main_loop_unref(loop); > + g_main_context_unref(context); > + > + g_mutex_lock(&mutex); > + g_clear_object(&test->srcA.server); > + g_clear_object(&test->srcB.server); > + g_clear_object(&test->dstA.server); > + g_clear_object(&test->dstB.server); > + g_mutex_unlock(&mutex); > + > + return NULL; > +} > + > +static gboolean > +wait_for_migration_complete(gpointer user_data) > +{ > + Test *test = user_data; > + QDict *rsp_return; > + bool stop = false; > + const char *status; > + > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > + status = qdict_get_str(rsp_return, "status"); > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > + stop = true; > + g_assert_cmpstr(status, ==, > + test->migrate_fail ? "failed" : "completed"); > + } > + qobject_unref(rsp_return); > + > + if (stop) { > + g_main_loop_quit(test->loop); > + } > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > +} > + > +static void migrate(QTestState *who, const char *uri) > +{ > + QDict *args, *rsp; > + > + args = qdict_new(); > + qdict_put_str(args, "uri", uri); > + > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > + > + g_assert(qdict_haskey(rsp, "return")); > + qobject_unref(rsp); > +} > + > +static void > +test_dbus_vmstate(Test *test) > +{ > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > + GThread *t = g_thread_new("dbus", dbus_thread, test); > + > + g_mutex_lock(&mutex); > + while (!test->dbus_loop) { > + g_cond_wait(&cond, &mutex); > + } > + > + src_qemu_args = > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > + "-object dbus-vmstate,id=dvB,addr=%s", > + g_dbus_server_get_client_address(test->srcA.server), > + g_dbus_server_get_client_address(test->srcB.server)); > + > + > + dst_qemu_args = > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > + "-object dbus-vmstate,id=dvB,addr=%s " > + "-incoming %s", > + g_dbus_server_get_client_address(test->dstA.server), > + g_dbus_server_get_client_address(test->dstB.server), > + uri); > + > + src_qemu = qtest_init(src_qemu_args); > + dst_qemu = qtest_init(dst_qemu_args); > + > + test->loop = g_main_loop_new(NULL, TRUE); > + > + migrate(src_qemu, uri); > + test->src_qemu = src_qemu; > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > + > + g_main_loop_run(test->loop); > + g_main_loop_unref(test->loop); > + > + g_free(uri); > + qtest_quit(dst_qemu); > + qtest_quit(src_qemu); > + g_free(dst_qemu_args); > + g_free(src_qemu_args); > + > + g_main_loop_quit(test->dbus_loop); > + g_mutex_unlock(&mutex); > + > + g_thread_join(t); > +} > + > +static void > +check_migrated(TestServer *s, TestServer *d) > +{ > + assert(s->save_called); > + assert(!s->load_called); > + assert(!d->save_called); > + assert(d->load_called); > +} > + > +static void > +test_dbus_vmstate_migrate(void) > +{ > + Test test = { }; > + > + test_dbus_vmstate(&test); > + > + check_migrated(&test.srcA, &test.dstA); > + check_migrated(&test.srcB, &test.dstB); > +} > + > +int > +main(int argc, char **argv) > +{ > + GError *err = NULL; > + int ret; > + > + g_test_init(&argc, &argv, NULL); > + > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > + if (!workdir) { > + g_error("Unable to create temporary dir: %s\n", err->message); > + } > + > + qtest_add_func("/dbus-vmstate/migrate", > + test_dbus_vmstate_migrate); > + > + ret = g_test_run(); > + > + rmdir(workdir); > + g_free(workdir); > + > + return ret; > +} > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > new file mode 100644 > index 0000000000..cc8563be4c > --- /dev/null > +++ b/tests/dbus-vmstate1.xml > @@ -0,0 +1,12 @@ > +<?xml version="1.0"?> > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > + <interface name="org.qemu.VMState1"> > + <property name="Id" type="s" access="read"/> > + <method name="Load"> > + <arg type="ay" name="data" direction="in"/> > + </method> > + <method name="Save"> > + <arg type="ay" name="data" direction="out"/> > + </method> > + </interface> > +</node> > -- > 2.23.0.rc1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-22 10:55 ` Dr. David Alan Gilbert @ 2019-08-22 11:35 ` Marc-André Lureau 2019-08-22 11:41 ` Dr. David Alan Gilbert 0 siblings, 1 reply; 28+ messages in thread From: Marc-André Lureau @ 2019-08-22 11:35 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela Hi On Thu, Aug 22, 2019 at 2:56 PM Dr. David Alan Gilbert <dgilbert@redhat.com> wrote: > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > When instanciated, this object will connect to the given D-Bus > > bus. During migration, it will take the data from org.qemu.VMState1 > > instances. > > > > See documentation for further details. > > I'll leave the main review to someone who understands the details of > DBUS, however from the migration side: > > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > > <snip> > > > + > > +static int > > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > > +{ > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > + uint8_t *data = NULL; > > + int ret = -1; > > + char id[256]; > > + unsigned int size; > > Higher level question; do you actually need a separate ID, or does > the ID that's stored by the migration code in the section name > give you enough uniqueness? The ID stored in migration section name is based on the handler ID: TYPE_DBUS_VMSTATE "-" handler_id By design, handler_id must be unique (and documented as such). Do you see a problem with that approach? > > > + if (qemu_get_counted_string(f, id) == 0) { > > + error_report("Invalid vmstate Id"); > > + goto end; > > I generally prefer to include something telling me where the error has > come from, just to make it easier to track down if I see the error in > a log; e.g. use __func__ ok > > > + } > > + > > + if (g_strcmp0(id, self->id)) { > > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > > + goto end; > > + } > > + > > + size = qemu_get_be32(f); > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > + error_report("Invalid vmstate size: %u", size); > > + goto end; > > + } > > + > > + data = g_malloc(size); > > + if (qemu_get_buffer(f, data, size) != size) { > > + error_report("Failed to read %u bytes", size); > > + goto end; > > + } > > + > > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > > + error_report("Failed to restore Id '%s'", id); > > + goto end; > > + } > > + > > + ret = 0; > > + > > +end: > > + g_clear_pointer(&data, g_free); > > + return ret; > > +} > > + > > +static void > > +dbus_save_state(QEMUFile *f, void *opaque) > > +{ > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > + GVariant *result = NULL, *child = NULL; > > + const uint8_t *data; > > + size_t size; > > + GError *err = NULL; > > + > > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > > + -1, NULL, &err); > > + if (!result) { > > + error_report("Failed to Save: %s", err->message); > > + g_clear_error(&err); > > + goto end; > > + } > > + > > + child = g_variant_get_child_value(result, 0); > > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > > + if (!data) { > > + error_report("Failed to Save: not a byte array"); > > + goto end; > > + } > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > + error_report("Too much vmstate data to save: %zu", size); > > + goto end; > > + } > > + > > + qemu_put_counted_string(f, self->id); > > + qemu_put_be32(f, size); > > + qemu_put_buffer(f, data, size); > > + > > +end: > > + g_clear_pointer(&child, g_variant_unref); > > + g_clear_pointer(&result, g_variant_unref); > > +} > > + > > +static const SaveVMHandlers savevm_handlers = { > > + .save_state = dbus_save_state, > > + .load_state = dbus_load_state, > > +}; > > + > > +static void > > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > > +{ > > + DBusVMState *self = DBUS_VMSTATE(uc); > > + GError *err = NULL; > > + GDBusConnection *bus = NULL; > > + GDBusProxy *proxy = NULL; > > + char *idstr = NULL; > > + > > + if (!self->dbus_addr) { > > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > > + return; > > + } > > + > > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > > + NULL, NULL, &err); > > + if (err) { > > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > > + g_clear_error(&err); > > + return; > > + } > > + > > + self->bus = bus; > > + > > + proxy = g_dbus_proxy_new_sync(bus, > > + G_DBUS_PROXY_FLAGS_NONE, > > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > > + NULL, > > + "/org/qemu/VMState1", > > + "org.qemu.VMState1", > > + NULL, &err); > > + > > + if (err) { > > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > > + g_clear_error(&err); > > + return; > > + } > > + > > + self->proxy = proxy; > > + > > + self->id = dbus_proxy_get_id(proxy, &err); > > + if (!self->id) { > > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > > + g_clear_error(&err); > > + return; > > + } > > + > > + idstr = get_idstr(self); > > + if (register_savevm_live(NULL, idstr, 0, 0, > > + &savevm_handlers, self) < 0) { > > + error_setg(errp, "Failed to register savevm handler"); > > + } > > Can you try and avoid register_savevm_live if possible and just wire a > vmsd onto the class like most other devices? We don't have many > register_savevm_live calls, and I'd like to get them down to just the > iterative devices. Sure if I could, but it's not a device. Perhaps we could have a qom interface for vmsd users? I can try that eventually. thanks > Dave > > > > + g_free(idstr); > > +} > > + > > +static void > > +dbus_vmstate_finalize(Object *o) > > +{ > > + DBusVMState *self = DBUS_VMSTATE(o); > > + char *idstr = get_idstr(self); > > + > > + unregister_savevm(NULL, idstr, self); > > + > > + g_clear_object(&self->bus); > > + g_clear_object(&self->proxy); > > + g_free(self->dbus_addr); > > + g_free(self->id); > > + g_free(idstr); > > +} > > + > > +static char * > > +get_dbus_addr(Object *o, Error **errp) > > +{ > > + DBusVMState *self = DBUS_VMSTATE(o); > > + > > + return g_strdup(self->dbus_addr); > > +} > > + > > +static void > > +set_dbus_addr(Object *o, const char *str, Error **errp) > > +{ > > + DBusVMState *self = DBUS_VMSTATE(o); > > + > > + g_free(self->dbus_addr); > > + self->dbus_addr = g_strdup(str); > > +} > > + > > +static void > > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > > +{ > > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > > + > > + ucc->complete = dbus_vmstate_complete; > > + > > + object_class_property_add_str(oc, "addr", > > + get_dbus_addr, set_dbus_addr, > > + &error_abort); > > +} > > + > > +static const TypeInfo dbus_vmstate_info = { > > + .name = TYPE_DBUS_VMSTATE, > > + .parent = TYPE_OBJECT, > > + .instance_size = sizeof(DBusVMState), > > + .instance_finalize = dbus_vmstate_finalize, > > + .class_size = sizeof(DBusVMStateClass), > > + .class_init = dbus_vmstate_class_init, > > + .interfaces = (InterfaceInfo[]) { > > + { TYPE_USER_CREATABLE }, > > + { } > > + } > > +}; > > + > > +static void > > +register_types(void) > > +{ > > + type_register_static(&dbus_vmstate_info); > > +} > > + > > +type_init(register_types); > > diff --git a/configure b/configure > > index 714e7fb6a1..e5b34f5ca7 100755 > > --- a/configure > > +++ b/configure > > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > > gio=yes > > gio_cflags=$($pkg_config --cflags gio-2.0) > > gio_libs=$($pkg_config --libs gio-2.0) > > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > > else > > gio=no > > fi > > > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > > +fi > > + > > # Sanity check that the current size_t matches the > > # size that glib thinks it should be. This catches > > # problems on multi-arch where people try to build > > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > > echo "CONFIG_GIO=y" >> $config_host_mak > > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > > fi > > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > > if test "$gnutls" = "yes" ; then > > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > > new file mode 100644 > > index 0000000000..4a32a183fb > > --- /dev/null > > +++ b/docs/interop/dbus-vmstate.rst > > @@ -0,0 +1,63 @@ > > +============ > > +DBus VMState > > +============ > > + > > +Introduction > > +============ > > + > > +Helper processes may have their state migrated with the help of > > +QEMU "dbus-vmstate" objects. > > + > > +At this point, the connection to the helper is done in DBus > > +peer-to-peer mode (no initial Hello, and no bus name for > > +communication). The helper must be listening to the given address. > > + > > +Helper may save arbitrary data to be transferred in the migration > > +stream and restored/loaded on destination. > > + > > +The data amount to be transferred is limited to 1Mb. The state must be > > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > > +reply anyway, and migration would fail if the data isn't given quickly > > +enough) > > + > > +Interface > > +========= > > + > > +On /org/qemu/VMState1 object path: > > + > > +.. code:: xml > > + > > + <interface name="org.qemu.VMState1"> > > + <property name="Id" type="s" access="read"/> > > + <method name="Load"> > > + <arg type="ay" name="data" direction="in"/> > > + </method> > > + <method name="Save"> > > + <arg type="ay" name="data" direction="out"/> > > + </method> > > + </interface> > > + > > +"Id" property > > +------------- > > + > > +A utf8 encoded string that identifies the helper uniquely. > > +Must be <256 bytes. > > + > > +Load(in u8[] bytes) method > > +-------------------------- > > + > > +The method called on destination with the state to restore. > > + > > +The helper may be initially started in a waiting state (with > > +an --incoming argument for example), and it may resume on load > > +success. > > + > > +An error may be returned to the caller. > > + > > +Save(out u8[] bytes) method > > +--------------------------- > > + > > +The method called on the source to get the current state to be > > +migrated. The helper should continue to run normally. > > + > > +An error may be returned to the caller. > > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > > index b4bfcab417..6bb173cfa6 100644 > > --- a/docs/interop/index.rst > > +++ b/docs/interop/index.rst > > @@ -13,6 +13,7 @@ Contents: > > :maxdepth: 2 > > > > bitmaps > > + dbus-vmstate > > live-block-operations > > pr-helper > > vhost-user > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > index fd7fdb8658..2c610086a7 100644 > > --- a/tests/Makefile.include > > +++ b/tests/Makefile.include > > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > > - > > +ifneq ($(GDBUS_CODEGEN),) > > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > > +endif > > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > > check-qtest-i386-y += tests/fdc-test$(EXESUF) > > check-qtest-i386-y += tests/ide-test$(EXESUF) > > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > > + "GEN","$(@:%-timestamp=%)") > > + @>$@ > > + > > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > > + > > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > > + > > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > > @@ -1172,6 +1187,7 @@ check-clean: > > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > > rm -f tests/test-qapi-gen-timestamp > > + rm -f tests/dbus-vmstate1-gen-timestamp > > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > > > clean: check-clean > > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > > new file mode 100644 > > index 0000000000..45d44916d2 > > --- /dev/null > > +++ b/tests/dbus-vmstate-test.c > > @@ -0,0 +1,371 @@ > > +#include "qemu/osdep.h" > > +#include <glib/gstdio.h> > > +#include <gio/gio.h> > > +#include "libqtest.h" > > +#include "qemu-common.h" > > +#include "dbus-vmstate1.h" > > + > > +static char *workdir; > > + > > +typedef struct TestServerId { > > + const char *name; > > + const char *data; > > + size_t size; > > +} TestServerId; > > + > > +static const TestServerId idA = { > > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > > +}; > > + > > +static const TestServerId idB = { > > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > > +}; > > + > > +typedef struct TestServer { > > + const TestServerId *id; > > + bool save_called; > > + bool load_called; > > + GDBusObjectManagerServer *om; > > + GDBusServer *server; > > +} TestServer; > > + > > +typedef struct Test { > > + bool migrate_fail; > > + TestServer srcA; > > + TestServer dstA; > > + TestServer srcB; > > + TestServer dstB; > > + GMainLoop *loop, *dbus_loop; > > + QTestState *src_qemu; > > +} Test; > > + > > +GMutex mutex; > > +GCond cond; > > + > > +static gboolean > > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > > + const gchar *arg_data, gpointer user_data) > > +{ > > + TestServer *h = user_data; > > + GVariant *args, *var; > > + const uint8_t *data; > > + size_t size; > > + > > + args = g_dbus_method_invocation_get_parameters(invocation); > > + var = g_variant_get_child_value(args, 0); > > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > > + g_assert_cmpuint(size, ==, h->id->size); > > + g_assert(!memcmp(data, h->id->data, h->id->size)); > > + h->load_called = true; > > + g_variant_unref(var); > > + > > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > > + return TRUE; > > +} > > + > > +static gboolean > > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > > + gpointer user_data) > > +{ > > + TestServer *h = user_data; > > + GVariant *var; > > + > > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > > + h->id->data, h->id->size, sizeof(char)); > > + g_dbus_method_invocation_return_value(invocation, > > + g_variant_new("(@ay)", var)); > > + h->save_called = true; > > + > > + return TRUE; > > +} > > + > > +static void > > +connection_closed(GDBusConnection *connection, > > + gboolean remote_peer_vanished, > > + GError *Error, > > + gpointer user_data) > > +{ > > + TestServer *h = user_data; > > + > > + g_clear_object(&h->om); > > + g_clear_object(&connection); > > +} > > + > > +static GDBusObjectManagerServer * > > +get_omserver(GDBusConnection *conn, gpointer user_data) > > +{ > > + TestServer *h = user_data; > > + GDBusObjectManagerServer *om; > > + GDBusObjectSkeleton *sk; > > + VMState1 *v; > > + > > + om = g_dbus_object_manager_server_new("/org/qemu"); > > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > > + > > + v = vmstate1_skeleton_new(); > > + g_object_set(v, "id", h->id->name, NULL); > > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > > + > > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > > + g_dbus_object_manager_server_export(om, sk); > > + g_dbus_object_manager_server_set_connection(om, conn); > > + > > + g_clear_object(&v); > > + g_clear_object(&sk); > > + > > + return om; > > +} > > + > > +static gboolean > > +on_new_connection(GDBusServer *server, > > + GDBusConnection *connection, > > + gpointer user_data) > > +{ > > + TestServer *h = user_data; > > + > > + g_object_ref(connection); > > + g_signal_connect(connection, "closed", > > + G_CALLBACK(connection_closed), user_data); > > + h->om = get_omserver(connection, user_data); > > + > > + return TRUE; > > +} > > + > > +static gboolean > > +allow_mechanism_cb(GDBusAuthObserver *observer, > > + const gchar *mechanism, > > + gpointer user_data) > > +{ > > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > > +} > > + > > +static gboolean > > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > > + GIOStream *stream, > > + GCredentials *credentials, > > + gpointer user_data) > > +{ > > + gboolean authorized = FALSE; > > + > > + if (credentials != NULL) { > > + GCredentials *own_credentials = g_credentials_new(); > > + > > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > > + authorized = TRUE; > > + } > > + > > + g_clear_object(&own_credentials); > > + } > > + > > + return authorized; > > +} > > + > > +static GDBusServer * > > +server_start(TestServer *h, const char *p, const TestServerId *id) > > +{ > > + GDBusAuthObserver *observer = NULL; > > + GDBusServer *server = NULL; > > + gchar *guid = NULL; > > + GError *error = NULL; > > + char *addr = NULL; > > + > > + h->id = id; > > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > > + guid = g_dbus_generate_guid(); > > + observer = g_dbus_auth_observer_new(); > > + g_signal_connect(observer, "allow-mechanism", > > + G_CALLBACK(allow_mechanism_cb), h); > > + g_signal_connect(observer, "authorize-authenticated-peer", > > + G_CALLBACK(authorize_authenticated_peer_cb), h); > > + > > + server = g_dbus_server_new_sync(addr, > > + G_DBUS_SERVER_FLAGS_NONE, > > + guid, > > + observer, > > + NULL, /* GCancellable */ > > + &error); > > + g_dbus_server_start(server); > > + g_clear_object(&observer); > > + g_free(guid); > > + > > + if (server == NULL) { > > + g_printerr("Error creating server at address %s: %s\n", > > + addr, error->message); > > + g_error_free(error); > > + return NULL; > > + } > > + > > + g_signal_connect(server, "new-connection", > > + G_CALLBACK(on_new_connection), h); > > + > > + g_free(addr); > > + return server; > > +} > > + > > + > > +static gpointer > > +dbus_thread(gpointer p) > > +{ > > + Test *test = p; > > + GMainContext *context = g_main_context_new(); > > + GMainLoop *loop = g_main_loop_new(context, FALSE); > > + > > + g_main_context_push_thread_default(context); > > + > > + g_mutex_lock(&mutex); > > + test->srcA.server = server_start(&test->srcA, "src", &idA); > > + test->srcB.server = server_start(&test->srcB, "src", &idB); > > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > > + test->dbus_loop = loop; > > + g_cond_signal(&cond); > > + g_mutex_unlock(&mutex); > > + > > + g_main_loop_run(loop); > > + > > + g_main_loop_unref(loop); > > + g_main_context_unref(context); > > + > > + g_mutex_lock(&mutex); > > + g_clear_object(&test->srcA.server); > > + g_clear_object(&test->srcB.server); > > + g_clear_object(&test->dstA.server); > > + g_clear_object(&test->dstB.server); > > + g_mutex_unlock(&mutex); > > + > > + return NULL; > > +} > > + > > +static gboolean > > +wait_for_migration_complete(gpointer user_data) > > +{ > > + Test *test = user_data; > > + QDict *rsp_return; > > + bool stop = false; > > + const char *status; > > + > > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > > + status = qdict_get_str(rsp_return, "status"); > > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > > + stop = true; > > + g_assert_cmpstr(status, ==, > > + test->migrate_fail ? "failed" : "completed"); > > + } > > + qobject_unref(rsp_return); > > + > > + if (stop) { > > + g_main_loop_quit(test->loop); > > + } > > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > > +} > > + > > +static void migrate(QTestState *who, const char *uri) > > +{ > > + QDict *args, *rsp; > > + > > + args = qdict_new(); > > + qdict_put_str(args, "uri", uri); > > + > > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > > + > > + g_assert(qdict_haskey(rsp, "return")); > > + qobject_unref(rsp); > > +} > > + > > +static void > > +test_dbus_vmstate(Test *test) > > +{ > > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > > + GThread *t = g_thread_new("dbus", dbus_thread, test); > > + > > + g_mutex_lock(&mutex); > > + while (!test->dbus_loop) { > > + g_cond_wait(&cond, &mutex); > > + } > > + > > + src_qemu_args = > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > + "-object dbus-vmstate,id=dvB,addr=%s", > > + g_dbus_server_get_client_address(test->srcA.server), > > + g_dbus_server_get_client_address(test->srcB.server)); > > + > > + > > + dst_qemu_args = > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > + "-object dbus-vmstate,id=dvB,addr=%s " > > + "-incoming %s", > > + g_dbus_server_get_client_address(test->dstA.server), > > + g_dbus_server_get_client_address(test->dstB.server), > > + uri); > > + > > + src_qemu = qtest_init(src_qemu_args); > > + dst_qemu = qtest_init(dst_qemu_args); > > + > > + test->loop = g_main_loop_new(NULL, TRUE); > > + > > + migrate(src_qemu, uri); > > + test->src_qemu = src_qemu; > > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > > + > > + g_main_loop_run(test->loop); > > + g_main_loop_unref(test->loop); > > + > > + g_free(uri); > > + qtest_quit(dst_qemu); > > + qtest_quit(src_qemu); > > + g_free(dst_qemu_args); > > + g_free(src_qemu_args); > > + > > + g_main_loop_quit(test->dbus_loop); > > + g_mutex_unlock(&mutex); > > + > > + g_thread_join(t); > > +} > > + > > +static void > > +check_migrated(TestServer *s, TestServer *d) > > +{ > > + assert(s->save_called); > > + assert(!s->load_called); > > + assert(!d->save_called); > > + assert(d->load_called); > > +} > > + > > +static void > > +test_dbus_vmstate_migrate(void) > > +{ > > + Test test = { }; > > + > > + test_dbus_vmstate(&test); > > + > > + check_migrated(&test.srcA, &test.dstA); > > + check_migrated(&test.srcB, &test.dstB); > > +} > > + > > +int > > +main(int argc, char **argv) > > +{ > > + GError *err = NULL; > > + int ret; > > + > > + g_test_init(&argc, &argv, NULL); > > + > > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > > + if (!workdir) { > > + g_error("Unable to create temporary dir: %s\n", err->message); > > + } > > + > > + qtest_add_func("/dbus-vmstate/migrate", > > + test_dbus_vmstate_migrate); > > + > > + ret = g_test_run(); > > + > > + rmdir(workdir); > > + g_free(workdir); > > + > > + return ret; > > +} > > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > > new file mode 100644 > > index 0000000000..cc8563be4c > > --- /dev/null > > +++ b/tests/dbus-vmstate1.xml > > @@ -0,0 +1,12 @@ > > +<?xml version="1.0"?> > > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > > + <interface name="org.qemu.VMState1"> > > + <property name="Id" type="s" access="read"/> > > + <method name="Load"> > > + <arg type="ay" name="data" direction="in"/> > > + </method> > > + <method name="Save"> > > + <arg type="ay" name="data" direction="out"/> > > + </method> > > + </interface> > > +</node> > > -- > > 2.23.0.rc1 > > > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-22 11:35 ` Marc-André Lureau @ 2019-08-22 11:41 ` Dr. David Alan Gilbert 2019-08-22 11:57 ` Marc-André Lureau 0 siblings, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-22 11:41 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > Hi > > On Thu, Aug 22, 2019 at 2:56 PM Dr. David Alan Gilbert > <dgilbert@redhat.com> wrote: > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > When instanciated, this object will connect to the given D-Bus > > > bus. During migration, it will take the data from org.qemu.VMState1 > > > instances. > > > > > > See documentation for further details. > > > > I'll leave the main review to someone who understands the details of > > DBUS, however from the migration side: > > > > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > <snip> > > > > > + > > > +static int > > > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > > > +{ > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > + uint8_t *data = NULL; > > > + int ret = -1; > > > + char id[256]; > > > + unsigned int size; > > > > Higher level question; do you actually need a separate ID, or does > > the ID that's stored by the migration code in the section name > > give you enough uniqueness? > > > The ID stored in migration section name is based on the handler ID: > TYPE_DBUS_VMSTATE "-" handler_id > > By design, handler_id must be unique (and documented as such). > > Do you see a problem with that approach? OK, so if that's unique, why do you need an 'id' stored here? > > > > > + if (qemu_get_counted_string(f, id) == 0) { > > > + error_report("Invalid vmstate Id"); > > > + goto end; > > > > I generally prefer to include something telling me where the error has > > come from, just to make it easier to track down if I see the error in > > a log; e.g. use __func__ > > ok > > > > > > + } > > > + > > > + if (g_strcmp0(id, self->id)) { > > > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > > > + goto end; > > > + } > > > + > > > + size = qemu_get_be32(f); > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > + error_report("Invalid vmstate size: %u", size); > > > + goto end; > > > + } > > > + > > > + data = g_malloc(size); > > > + if (qemu_get_buffer(f, data, size) != size) { > > > + error_report("Failed to read %u bytes", size); > > > + goto end; > > > + } > > > + > > > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > > > + error_report("Failed to restore Id '%s'", id); > > > + goto end; > > > + } > > > + > > > + ret = 0; > > > + > > > +end: > > > + g_clear_pointer(&data, g_free); > > > + return ret; > > > +} > > > + > > > +static void > > > +dbus_save_state(QEMUFile *f, void *opaque) > > > +{ > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > + GVariant *result = NULL, *child = NULL; > > > + const uint8_t *data; > > > + size_t size; > > > + GError *err = NULL; > > > + > > > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > > > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > > > + -1, NULL, &err); > > > + if (!result) { > > > + error_report("Failed to Save: %s", err->message); > > > + g_clear_error(&err); > > > + goto end; > > > + } > > > + > > > + child = g_variant_get_child_value(result, 0); > > > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > > > + if (!data) { > > > + error_report("Failed to Save: not a byte array"); > > > + goto end; > > > + } > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > + error_report("Too much vmstate data to save: %zu", size); > > > + goto end; > > > + } > > > + > > > + qemu_put_counted_string(f, self->id); > > > + qemu_put_be32(f, size); > > > + qemu_put_buffer(f, data, size); > > > + > > > +end: > > > + g_clear_pointer(&child, g_variant_unref); > > > + g_clear_pointer(&result, g_variant_unref); > > > +} > > > + > > > +static const SaveVMHandlers savevm_handlers = { > > > + .save_state = dbus_save_state, > > > + .load_state = dbus_load_state, > > > +}; > > > + > > > +static void > > > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > > > +{ > > > + DBusVMState *self = DBUS_VMSTATE(uc); > > > + GError *err = NULL; > > > + GDBusConnection *bus = NULL; > > > + GDBusProxy *proxy = NULL; > > > + char *idstr = NULL; > > > + > > > + if (!self->dbus_addr) { > > > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > > > + return; > > > + } > > > + > > > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > > > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > > > + NULL, NULL, &err); > > > + if (err) { > > > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > > > + g_clear_error(&err); > > > + return; > > > + } > > > + > > > + self->bus = bus; > > > + > > > + proxy = g_dbus_proxy_new_sync(bus, > > > + G_DBUS_PROXY_FLAGS_NONE, > > > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > > > + NULL, > > > + "/org/qemu/VMState1", > > > + "org.qemu.VMState1", > > > + NULL, &err); > > > + > > > + if (err) { > > > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > > > + g_clear_error(&err); > > > + return; > > > + } > > > + > > > + self->proxy = proxy; > > > + > > > + self->id = dbus_proxy_get_id(proxy, &err); > > > + if (!self->id) { > > > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > > > + g_clear_error(&err); > > > + return; > > > + } > > > + > > > + idstr = get_idstr(self); > > > + if (register_savevm_live(NULL, idstr, 0, 0, > > > + &savevm_handlers, self) < 0) { > > > + error_setg(errp, "Failed to register savevm handler"); > > > + } > > > > Can you try and avoid register_savevm_live if possible and just wire a > > vmsd onto the class like most other devices? We don't have many > > register_savevm_live calls, and I'd like to get them down to just the > > iterative devices. > > Sure if I could, but it's not a device. > > Perhaps we could have a qom interface for vmsd users? I can try that eventually. Hmm, why isn't it a device? Dave > thanks > > > Dave > > > > > > > + g_free(idstr); > > > +} > > > + > > > +static void > > > +dbus_vmstate_finalize(Object *o) > > > +{ > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > + char *idstr = get_idstr(self); > > > + > > > + unregister_savevm(NULL, idstr, self); > > > + > > > + g_clear_object(&self->bus); > > > + g_clear_object(&self->proxy); > > > + g_free(self->dbus_addr); > > > + g_free(self->id); > > > + g_free(idstr); > > > +} > > > + > > > +static char * > > > +get_dbus_addr(Object *o, Error **errp) > > > +{ > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > + > > > + return g_strdup(self->dbus_addr); > > > +} > > > + > > > +static void > > > +set_dbus_addr(Object *o, const char *str, Error **errp) > > > +{ > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > + > > > + g_free(self->dbus_addr); > > > + self->dbus_addr = g_strdup(str); > > > +} > > > + > > > +static void > > > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > > > +{ > > > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > > > + > > > + ucc->complete = dbus_vmstate_complete; > > > + > > > + object_class_property_add_str(oc, "addr", > > > + get_dbus_addr, set_dbus_addr, > > > + &error_abort); > > > +} > > > + > > > +static const TypeInfo dbus_vmstate_info = { > > > + .name = TYPE_DBUS_VMSTATE, > > > + .parent = TYPE_OBJECT, > > > + .instance_size = sizeof(DBusVMState), > > > + .instance_finalize = dbus_vmstate_finalize, > > > + .class_size = sizeof(DBusVMStateClass), > > > + .class_init = dbus_vmstate_class_init, > > > + .interfaces = (InterfaceInfo[]) { > > > + { TYPE_USER_CREATABLE }, > > > + { } > > > + } > > > +}; > > > + > > > +static void > > > +register_types(void) > > > +{ > > > + type_register_static(&dbus_vmstate_info); > > > +} > > > + > > > +type_init(register_types); > > > diff --git a/configure b/configure > > > index 714e7fb6a1..e5b34f5ca7 100755 > > > --- a/configure > > > +++ b/configure > > > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > > > gio=yes > > > gio_cflags=$($pkg_config --cflags gio-2.0) > > > gio_libs=$($pkg_config --libs gio-2.0) > > > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > > > else > > > gio=no > > > fi > > > > > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > > > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > > > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > > > +fi > > > + > > > # Sanity check that the current size_t matches the > > > # size that glib thinks it should be. This catches > > > # problems on multi-arch where people try to build > > > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > > > echo "CONFIG_GIO=y" >> $config_host_mak > > > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > > > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > > > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > > > fi > > > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > > > if test "$gnutls" = "yes" ; then > > > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > > > new file mode 100644 > > > index 0000000000..4a32a183fb > > > --- /dev/null > > > +++ b/docs/interop/dbus-vmstate.rst > > > @@ -0,0 +1,63 @@ > > > +============ > > > +DBus VMState > > > +============ > > > + > > > +Introduction > > > +============ > > > + > > > +Helper processes may have their state migrated with the help of > > > +QEMU "dbus-vmstate" objects. > > > + > > > +At this point, the connection to the helper is done in DBus > > > +peer-to-peer mode (no initial Hello, and no bus name for > > > +communication). The helper must be listening to the given address. > > > + > > > +Helper may save arbitrary data to be transferred in the migration > > > +stream and restored/loaded on destination. > > > + > > > +The data amount to be transferred is limited to 1Mb. The state must be > > > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > > > +reply anyway, and migration would fail if the data isn't given quickly > > > +enough) > > > + > > > +Interface > > > +========= > > > + > > > +On /org/qemu/VMState1 object path: > > > + > > > +.. code:: xml > > > + > > > + <interface name="org.qemu.VMState1"> > > > + <property name="Id" type="s" access="read"/> > > > + <method name="Load"> > > > + <arg type="ay" name="data" direction="in"/> > > > + </method> > > > + <method name="Save"> > > > + <arg type="ay" name="data" direction="out"/> > > > + </method> > > > + </interface> > > > + > > > +"Id" property > > > +------------- > > > + > > > +A utf8 encoded string that identifies the helper uniquely. > > > +Must be <256 bytes. > > > + > > > +Load(in u8[] bytes) method > > > +-------------------------- > > > + > > > +The method called on destination with the state to restore. > > > + > > > +The helper may be initially started in a waiting state (with > > > +an --incoming argument for example), and it may resume on load > > > +success. > > > + > > > +An error may be returned to the caller. > > > + > > > +Save(out u8[] bytes) method > > > +--------------------------- > > > + > > > +The method called on the source to get the current state to be > > > +migrated. The helper should continue to run normally. > > > + > > > +An error may be returned to the caller. > > > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > > > index b4bfcab417..6bb173cfa6 100644 > > > --- a/docs/interop/index.rst > > > +++ b/docs/interop/index.rst > > > @@ -13,6 +13,7 @@ Contents: > > > :maxdepth: 2 > > > > > > bitmaps > > > + dbus-vmstate > > > live-block-operations > > > pr-helper > > > vhost-user > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > index fd7fdb8658..2c610086a7 100644 > > > --- a/tests/Makefile.include > > > +++ b/tests/Makefile.include > > > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > > > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > > > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > > > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > > > - > > > +ifneq ($(GDBUS_CODEGEN),) > > > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > > > +endif > > > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > > > check-qtest-i386-y += tests/fdc-test$(EXESUF) > > > check-qtest-i386-y += tests/ide-test$(EXESUF) > > > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > > > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > > > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > > > > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > > > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > > > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > > > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > > > + "GEN","$(@:%-timestamp=%)") > > > + @>$@ > > > + > > > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > > > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > > > + > > > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > > > + > > > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > > > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > > > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > > > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > > > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > > > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > > > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > > > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > > > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > > > @@ -1172,6 +1187,7 @@ check-clean: > > > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > > > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > > > rm -f tests/test-qapi-gen-timestamp > > > + rm -f tests/dbus-vmstate1-gen-timestamp > > > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > > > > > clean: check-clean > > > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > > > new file mode 100644 > > > index 0000000000..45d44916d2 > > > --- /dev/null > > > +++ b/tests/dbus-vmstate-test.c > > > @@ -0,0 +1,371 @@ > > > +#include "qemu/osdep.h" > > > +#include <glib/gstdio.h> > > > +#include <gio/gio.h> > > > +#include "libqtest.h" > > > +#include "qemu-common.h" > > > +#include "dbus-vmstate1.h" > > > + > > > +static char *workdir; > > > + > > > +typedef struct TestServerId { > > > + const char *name; > > > + const char *data; > > > + size_t size; > > > +} TestServerId; > > > + > > > +static const TestServerId idA = { > > > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > > > +}; > > > + > > > +static const TestServerId idB = { > > > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > > > +}; > > > + > > > +typedef struct TestServer { > > > + const TestServerId *id; > > > + bool save_called; > > > + bool load_called; > > > + GDBusObjectManagerServer *om; > > > + GDBusServer *server; > > > +} TestServer; > > > + > > > +typedef struct Test { > > > + bool migrate_fail; > > > + TestServer srcA; > > > + TestServer dstA; > > > + TestServer srcB; > > > + TestServer dstB; > > > + GMainLoop *loop, *dbus_loop; > > > + QTestState *src_qemu; > > > +} Test; > > > + > > > +GMutex mutex; > > > +GCond cond; > > > + > > > +static gboolean > > > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > > > + const gchar *arg_data, gpointer user_data) > > > +{ > > > + TestServer *h = user_data; > > > + GVariant *args, *var; > > > + const uint8_t *data; > > > + size_t size; > > > + > > > + args = g_dbus_method_invocation_get_parameters(invocation); > > > + var = g_variant_get_child_value(args, 0); > > > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > > > + g_assert_cmpuint(size, ==, h->id->size); > > > + g_assert(!memcmp(data, h->id->data, h->id->size)); > > > + h->load_called = true; > > > + g_variant_unref(var); > > > + > > > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > > > + return TRUE; > > > +} > > > + > > > +static gboolean > > > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > > > + gpointer user_data) > > > +{ > > > + TestServer *h = user_data; > > > + GVariant *var; > > > + > > > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > > > + h->id->data, h->id->size, sizeof(char)); > > > + g_dbus_method_invocation_return_value(invocation, > > > + g_variant_new("(@ay)", var)); > > > + h->save_called = true; > > > + > > > + return TRUE; > > > +} > > > + > > > +static void > > > +connection_closed(GDBusConnection *connection, > > > + gboolean remote_peer_vanished, > > > + GError *Error, > > > + gpointer user_data) > > > +{ > > > + TestServer *h = user_data; > > > + > > > + g_clear_object(&h->om); > > > + g_clear_object(&connection); > > > +} > > > + > > > +static GDBusObjectManagerServer * > > > +get_omserver(GDBusConnection *conn, gpointer user_data) > > > +{ > > > + TestServer *h = user_data; > > > + GDBusObjectManagerServer *om; > > > + GDBusObjectSkeleton *sk; > > > + VMState1 *v; > > > + > > > + om = g_dbus_object_manager_server_new("/org/qemu"); > > > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > > > + > > > + v = vmstate1_skeleton_new(); > > > + g_object_set(v, "id", h->id->name, NULL); > > > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > > > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > > > + > > > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > > > + g_dbus_object_manager_server_export(om, sk); > > > + g_dbus_object_manager_server_set_connection(om, conn); > > > + > > > + g_clear_object(&v); > > > + g_clear_object(&sk); > > > + > > > + return om; > > > +} > > > + > > > +static gboolean > > > +on_new_connection(GDBusServer *server, > > > + GDBusConnection *connection, > > > + gpointer user_data) > > > +{ > > > + TestServer *h = user_data; > > > + > > > + g_object_ref(connection); > > > + g_signal_connect(connection, "closed", > > > + G_CALLBACK(connection_closed), user_data); > > > + h->om = get_omserver(connection, user_data); > > > + > > > + return TRUE; > > > +} > > > + > > > +static gboolean > > > +allow_mechanism_cb(GDBusAuthObserver *observer, > > > + const gchar *mechanism, > > > + gpointer user_data) > > > +{ > > > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > > > +} > > > + > > > +static gboolean > > > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > > > + GIOStream *stream, > > > + GCredentials *credentials, > > > + gpointer user_data) > > > +{ > > > + gboolean authorized = FALSE; > > > + > > > + if (credentials != NULL) { > > > + GCredentials *own_credentials = g_credentials_new(); > > > + > > > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > > > + authorized = TRUE; > > > + } > > > + > > > + g_clear_object(&own_credentials); > > > + } > > > + > > > + return authorized; > > > +} > > > + > > > +static GDBusServer * > > > +server_start(TestServer *h, const char *p, const TestServerId *id) > > > +{ > > > + GDBusAuthObserver *observer = NULL; > > > + GDBusServer *server = NULL; > > > + gchar *guid = NULL; > > > + GError *error = NULL; > > > + char *addr = NULL; > > > + > > > + h->id = id; > > > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > > > + guid = g_dbus_generate_guid(); > > > + observer = g_dbus_auth_observer_new(); > > > + g_signal_connect(observer, "allow-mechanism", > > > + G_CALLBACK(allow_mechanism_cb), h); > > > + g_signal_connect(observer, "authorize-authenticated-peer", > > > + G_CALLBACK(authorize_authenticated_peer_cb), h); > > > + > > > + server = g_dbus_server_new_sync(addr, > > > + G_DBUS_SERVER_FLAGS_NONE, > > > + guid, > > > + observer, > > > + NULL, /* GCancellable */ > > > + &error); > > > + g_dbus_server_start(server); > > > + g_clear_object(&observer); > > > + g_free(guid); > > > + > > > + if (server == NULL) { > > > + g_printerr("Error creating server at address %s: %s\n", > > > + addr, error->message); > > > + g_error_free(error); > > > + return NULL; > > > + } > > > + > > > + g_signal_connect(server, "new-connection", > > > + G_CALLBACK(on_new_connection), h); > > > + > > > + g_free(addr); > > > + return server; > > > +} > > > + > > > + > > > +static gpointer > > > +dbus_thread(gpointer p) > > > +{ > > > + Test *test = p; > > > + GMainContext *context = g_main_context_new(); > > > + GMainLoop *loop = g_main_loop_new(context, FALSE); > > > + > > > + g_main_context_push_thread_default(context); > > > + > > > + g_mutex_lock(&mutex); > > > + test->srcA.server = server_start(&test->srcA, "src", &idA); > > > + test->srcB.server = server_start(&test->srcB, "src", &idB); > > > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > > > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > > > + test->dbus_loop = loop; > > > + g_cond_signal(&cond); > > > + g_mutex_unlock(&mutex); > > > + > > > + g_main_loop_run(loop); > > > + > > > + g_main_loop_unref(loop); > > > + g_main_context_unref(context); > > > + > > > + g_mutex_lock(&mutex); > > > + g_clear_object(&test->srcA.server); > > > + g_clear_object(&test->srcB.server); > > > + g_clear_object(&test->dstA.server); > > > + g_clear_object(&test->dstB.server); > > > + g_mutex_unlock(&mutex); > > > + > > > + return NULL; > > > +} > > > + > > > +static gboolean > > > +wait_for_migration_complete(gpointer user_data) > > > +{ > > > + Test *test = user_data; > > > + QDict *rsp_return; > > > + bool stop = false; > > > + const char *status; > > > + > > > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > > > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > > > + status = qdict_get_str(rsp_return, "status"); > > > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > > > + stop = true; > > > + g_assert_cmpstr(status, ==, > > > + test->migrate_fail ? "failed" : "completed"); > > > + } > > > + qobject_unref(rsp_return); > > > + > > > + if (stop) { > > > + g_main_loop_quit(test->loop); > > > + } > > > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > > > +} > > > + > > > +static void migrate(QTestState *who, const char *uri) > > > +{ > > > + QDict *args, *rsp; > > > + > > > + args = qdict_new(); > > > + qdict_put_str(args, "uri", uri); > > > + > > > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > > > + > > > + g_assert(qdict_haskey(rsp, "return")); > > > + qobject_unref(rsp); > > > +} > > > + > > > +static void > > > +test_dbus_vmstate(Test *test) > > > +{ > > > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > > > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > > > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > > > + GThread *t = g_thread_new("dbus", dbus_thread, test); > > > + > > > + g_mutex_lock(&mutex); > > > + while (!test->dbus_loop) { > > > + g_cond_wait(&cond, &mutex); > > > + } > > > + > > > + src_qemu_args = > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > + "-object dbus-vmstate,id=dvB,addr=%s", > > > + g_dbus_server_get_client_address(test->srcA.server), > > > + g_dbus_server_get_client_address(test->srcB.server)); > > > + > > > + > > > + dst_qemu_args = > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > + "-object dbus-vmstate,id=dvB,addr=%s " > > > + "-incoming %s", > > > + g_dbus_server_get_client_address(test->dstA.server), > > > + g_dbus_server_get_client_address(test->dstB.server), > > > + uri); > > > + > > > + src_qemu = qtest_init(src_qemu_args); > > > + dst_qemu = qtest_init(dst_qemu_args); > > > + > > > + test->loop = g_main_loop_new(NULL, TRUE); > > > + > > > + migrate(src_qemu, uri); > > > + test->src_qemu = src_qemu; > > > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > > > + > > > + g_main_loop_run(test->loop); > > > + g_main_loop_unref(test->loop); > > > + > > > + g_free(uri); > > > + qtest_quit(dst_qemu); > > > + qtest_quit(src_qemu); > > > + g_free(dst_qemu_args); > > > + g_free(src_qemu_args); > > > + > > > + g_main_loop_quit(test->dbus_loop); > > > + g_mutex_unlock(&mutex); > > > + > > > + g_thread_join(t); > > > +} > > > + > > > +static void > > > +check_migrated(TestServer *s, TestServer *d) > > > +{ > > > + assert(s->save_called); > > > + assert(!s->load_called); > > > + assert(!d->save_called); > > > + assert(d->load_called); > > > +} > > > + > > > +static void > > > +test_dbus_vmstate_migrate(void) > > > +{ > > > + Test test = { }; > > > + > > > + test_dbus_vmstate(&test); > > > + > > > + check_migrated(&test.srcA, &test.dstA); > > > + check_migrated(&test.srcB, &test.dstB); > > > +} > > > + > > > +int > > > +main(int argc, char **argv) > > > +{ > > > + GError *err = NULL; > > > + int ret; > > > + > > > + g_test_init(&argc, &argv, NULL); > > > + > > > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > > > + if (!workdir) { > > > + g_error("Unable to create temporary dir: %s\n", err->message); > > > + } > > > + > > > + qtest_add_func("/dbus-vmstate/migrate", > > > + test_dbus_vmstate_migrate); > > > + > > > + ret = g_test_run(); > > > + > > > + rmdir(workdir); > > > + g_free(workdir); > > > + > > > + return ret; > > > +} > > > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > > > new file mode 100644 > > > index 0000000000..cc8563be4c > > > --- /dev/null > > > +++ b/tests/dbus-vmstate1.xml > > > @@ -0,0 +1,12 @@ > > > +<?xml version="1.0"?> > > > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > > > + <interface name="org.qemu.VMState1"> > > > + <property name="Id" type="s" access="read"/> > > > + <method name="Load"> > > > + <arg type="ay" name="data" direction="in"/> > > > + </method> > > > + <method name="Save"> > > > + <arg type="ay" name="data" direction="out"/> > > > + </method> > > > + </interface> > > > +</node> > > > -- > > > 2.23.0.rc1 > > > > > -- > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-22 11:41 ` Dr. David Alan Gilbert @ 2019-08-22 11:57 ` Marc-André Lureau 2019-08-22 12:19 ` Dr. David Alan Gilbert 0 siblings, 1 reply; 28+ messages in thread From: Marc-André Lureau @ 2019-08-22 11:57 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela Hi On Thu, Aug 22, 2019 at 3:41 PM Dr. David Alan Gilbert <dgilbert@redhat.com> wrote: > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > Hi > > > > On Thu, Aug 22, 2019 at 2:56 PM Dr. David Alan Gilbert > > <dgilbert@redhat.com> wrote: > > > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > > When instanciated, this object will connect to the given D-Bus > > > > bus. During migration, it will take the data from org.qemu.VMState1 > > > > instances. > > > > > > > > See documentation for further details. > > > > > > I'll leave the main review to someone who understands the details of > > > DBUS, however from the migration side: > > > > > > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > > > <snip> > > > > > > > + > > > > +static int > > > > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > > > > +{ > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > + uint8_t *data = NULL; > > > > + int ret = -1; > > > > + char id[256]; > > > > + unsigned int size; > > > > > > Higher level question; do you actually need a separate ID, or does > > > the ID that's stored by the migration code in the section name > > > give you enough uniqueness? > > > > > > The ID stored in migration section name is based on the handler ID: > > TYPE_DBUS_VMSTATE "-" handler_id > > > > By design, handler_id must be unique (and documented as such). > > > > Do you see a problem with that approach? > > OK, so if that's unique, why do you need an 'id' stored here? Oh I see, that's mostly a relic of v1, where multiple ID "sub-sections" would exist for each helper. However, if we want to allow having several "sub-sections" again, that might come handy to keep compatibility. It may be better to have a simple version field for that instead. > > > > > > > > + if (qemu_get_counted_string(f, id) == 0) { > > > > + error_report("Invalid vmstate Id"); > > > > + goto end; > > > > > > I generally prefer to include something telling me where the error has > > > come from, just to make it easier to track down if I see the error in > > > a log; e.g. use __func__ > > > > ok > > > > > > > > > + } > > > > + > > > > + if (g_strcmp0(id, self->id)) { > > > > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > > > > + goto end; > > > > + } > > > > + > > > > + size = qemu_get_be32(f); > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > + error_report("Invalid vmstate size: %u", size); > > > > + goto end; > > > > + } > > > > + > > > > + data = g_malloc(size); > > > > + if (qemu_get_buffer(f, data, size) != size) { > > > > + error_report("Failed to read %u bytes", size); > > > > + goto end; > > > > + } > > > > + > > > > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > > > > + error_report("Failed to restore Id '%s'", id); > > > > + goto end; > > > > + } > > > > + > > > > + ret = 0; > > > > + > > > > +end: > > > > + g_clear_pointer(&data, g_free); > > > > + return ret; > > > > +} > > > > + > > > > +static void > > > > +dbus_save_state(QEMUFile *f, void *opaque) > > > > +{ > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > + GVariant *result = NULL, *child = NULL; > > > > + const uint8_t *data; > > > > + size_t size; > > > > + GError *err = NULL; > > > > + > > > > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > > > > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > > > > + -1, NULL, &err); > > > > + if (!result) { > > > > + error_report("Failed to Save: %s", err->message); > > > > + g_clear_error(&err); > > > > + goto end; > > > > + } > > > > + > > > > + child = g_variant_get_child_value(result, 0); > > > > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > > > > + if (!data) { > > > > + error_report("Failed to Save: not a byte array"); > > > > + goto end; > > > > + } > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > + error_report("Too much vmstate data to save: %zu", size); > > > > + goto end; > > > > + } > > > > + > > > > + qemu_put_counted_string(f, self->id); > > > > + qemu_put_be32(f, size); > > > > + qemu_put_buffer(f, data, size); > > > > + > > > > +end: > > > > + g_clear_pointer(&child, g_variant_unref); > > > > + g_clear_pointer(&result, g_variant_unref); > > > > +} > > > > + > > > > +static const SaveVMHandlers savevm_handlers = { > > > > + .save_state = dbus_save_state, > > > > + .load_state = dbus_load_state, > > > > +}; > > > > + > > > > +static void > > > > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > > > > +{ > > > > + DBusVMState *self = DBUS_VMSTATE(uc); > > > > + GError *err = NULL; > > > > + GDBusConnection *bus = NULL; > > > > + GDBusProxy *proxy = NULL; > > > > + char *idstr = NULL; > > > > + > > > > + if (!self->dbus_addr) { > > > > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > > > > + return; > > > > + } > > > > + > > > > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > > > > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > > > > + NULL, NULL, &err); > > > > + if (err) { > > > > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > > > > + g_clear_error(&err); > > > > + return; > > > > + } > > > > + > > > > + self->bus = bus; > > > > + > > > > + proxy = g_dbus_proxy_new_sync(bus, > > > > + G_DBUS_PROXY_FLAGS_NONE, > > > > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > > > > + NULL, > > > > + "/org/qemu/VMState1", > > > > + "org.qemu.VMState1", > > > > + NULL, &err); > > > > + > > > > + if (err) { > > > > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > > > > + g_clear_error(&err); > > > > + return; > > > > + } > > > > + > > > > + self->proxy = proxy; > > > > + > > > > + self->id = dbus_proxy_get_id(proxy, &err); > > > > + if (!self->id) { > > > > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > > > > + g_clear_error(&err); > > > > + return; > > > > + } > > > > + > > > > + idstr = get_idstr(self); > > > > + if (register_savevm_live(NULL, idstr, 0, 0, > > > > + &savevm_handlers, self) < 0) { > > > > + error_setg(errp, "Failed to register savevm handler"); > > > > + } > > > > > > Can you try and avoid register_savevm_live if possible and just wire a > > > vmsd onto the class like most other devices? We don't have many > > > register_savevm_live calls, and I'd like to get them down to just the > > > iterative devices. > > > > Sure if I could, but it's not a device. > > > > Perhaps we could have a qom interface for vmsd users? I can try that eventually. > > Hmm, why isn't it a device? It doesn't need anything from Device (which is heavyweight), it is a simple user-creatable object that holds a dbus connection and reacts on migration events. If you take slirp helper process as an example, it's a "service" for a VM. Not a device. You could have the same helper for various devices etc, or more helpers for the same device. > > > Dave > > > thanks > > > > > Dave > > > > > > > > > > + g_free(idstr); > > > > +} > > > > + > > > > +static void > > > > +dbus_vmstate_finalize(Object *o) > > > > +{ > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > + char *idstr = get_idstr(self); > > > > + > > > > + unregister_savevm(NULL, idstr, self); > > > > + > > > > + g_clear_object(&self->bus); > > > > + g_clear_object(&self->proxy); > > > > + g_free(self->dbus_addr); > > > > + g_free(self->id); > > > > + g_free(idstr); > > > > +} > > > > + > > > > +static char * > > > > +get_dbus_addr(Object *o, Error **errp) > > > > +{ > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > + > > > > + return g_strdup(self->dbus_addr); > > > > +} > > > > + > > > > +static void > > > > +set_dbus_addr(Object *o, const char *str, Error **errp) > > > > +{ > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > + > > > > + g_free(self->dbus_addr); > > > > + self->dbus_addr = g_strdup(str); > > > > +} > > > > + > > > > +static void > > > > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > > > > +{ > > > > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > > > > + > > > > + ucc->complete = dbus_vmstate_complete; > > > > + > > > > + object_class_property_add_str(oc, "addr", > > > > + get_dbus_addr, set_dbus_addr, > > > > + &error_abort); > > > > +} > > > > + > > > > +static const TypeInfo dbus_vmstate_info = { > > > > + .name = TYPE_DBUS_VMSTATE, > > > > + .parent = TYPE_OBJECT, > > > > + .instance_size = sizeof(DBusVMState), > > > > + .instance_finalize = dbus_vmstate_finalize, > > > > + .class_size = sizeof(DBusVMStateClass), > > > > + .class_init = dbus_vmstate_class_init, > > > > + .interfaces = (InterfaceInfo[]) { > > > > + { TYPE_USER_CREATABLE }, > > > > + { } > > > > + } > > > > +}; > > > > + > > > > +static void > > > > +register_types(void) > > > > +{ > > > > + type_register_static(&dbus_vmstate_info); > > > > +} > > > > + > > > > +type_init(register_types); > > > > diff --git a/configure b/configure > > > > index 714e7fb6a1..e5b34f5ca7 100755 > > > > --- a/configure > > > > +++ b/configure > > > > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > > > > gio=yes > > > > gio_cflags=$($pkg_config --cflags gio-2.0) > > > > gio_libs=$($pkg_config --libs gio-2.0) > > > > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > > > > else > > > > gio=no > > > > fi > > > > > > > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > > > > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > > > > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > > > > +fi > > > > + > > > > # Sanity check that the current size_t matches the > > > > # size that glib thinks it should be. This catches > > > > # problems on multi-arch where people try to build > > > > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > > > > echo "CONFIG_GIO=y" >> $config_host_mak > > > > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > > > > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > > > > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > > > > fi > > > > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > > > > if test "$gnutls" = "yes" ; then > > > > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > > > > new file mode 100644 > > > > index 0000000000..4a32a183fb > > > > --- /dev/null > > > > +++ b/docs/interop/dbus-vmstate.rst > > > > @@ -0,0 +1,63 @@ > > > > +============ > > > > +DBus VMState > > > > +============ > > > > + > > > > +Introduction > > > > +============ > > > > + > > > > +Helper processes may have their state migrated with the help of > > > > +QEMU "dbus-vmstate" objects. > > > > + > > > > +At this point, the connection to the helper is done in DBus > > > > +peer-to-peer mode (no initial Hello, and no bus name for > > > > +communication). The helper must be listening to the given address. > > > > + > > > > +Helper may save arbitrary data to be transferred in the migration > > > > +stream and restored/loaded on destination. > > > > + > > > > +The data amount to be transferred is limited to 1Mb. The state must be > > > > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > > > > +reply anyway, and migration would fail if the data isn't given quickly > > > > +enough) > > > > + > > > > +Interface > > > > +========= > > > > + > > > > +On /org/qemu/VMState1 object path: > > > > + > > > > +.. code:: xml > > > > + > > > > + <interface name="org.qemu.VMState1"> > > > > + <property name="Id" type="s" access="read"/> > > > > + <method name="Load"> > > > > + <arg type="ay" name="data" direction="in"/> > > > > + </method> > > > > + <method name="Save"> > > > > + <arg type="ay" name="data" direction="out"/> > > > > + </method> > > > > + </interface> > > > > + > > > > +"Id" property > > > > +------------- > > > > + > > > > +A utf8 encoded string that identifies the helper uniquely. > > > > +Must be <256 bytes. > > > > + > > > > +Load(in u8[] bytes) method > > > > +-------------------------- > > > > + > > > > +The method called on destination with the state to restore. > > > > + > > > > +The helper may be initially started in a waiting state (with > > > > +an --incoming argument for example), and it may resume on load > > > > +success. > > > > + > > > > +An error may be returned to the caller. > > > > + > > > > +Save(out u8[] bytes) method > > > > +--------------------------- > > > > + > > > > +The method called on the source to get the current state to be > > > > +migrated. The helper should continue to run normally. > > > > + > > > > +An error may be returned to the caller. > > > > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > > > > index b4bfcab417..6bb173cfa6 100644 > > > > --- a/docs/interop/index.rst > > > > +++ b/docs/interop/index.rst > > > > @@ -13,6 +13,7 @@ Contents: > > > > :maxdepth: 2 > > > > > > > > bitmaps > > > > + dbus-vmstate > > > > live-block-operations > > > > pr-helper > > > > vhost-user > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > > index fd7fdb8658..2c610086a7 100644 > > > > --- a/tests/Makefile.include > > > > +++ b/tests/Makefile.include > > > > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > > > > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > > > > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > > > > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > > > > - > > > > +ifneq ($(GDBUS_CODEGEN),) > > > > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > > > > +endif > > > > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > > > > check-qtest-i386-y += tests/fdc-test$(EXESUF) > > > > check-qtest-i386-y += tests/ide-test$(EXESUF) > > > > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > > > > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > > > > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > > > > > > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > > > > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > > > > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > > > > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > > > > + "GEN","$(@:%-timestamp=%)") > > > > + @>$@ > > > > + > > > > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > > > > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > > > > + > > > > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > > > > + > > > > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > > > > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > > > > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > > > > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > > > > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > > > > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > > > > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > > > > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > > > > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > > > > @@ -1172,6 +1187,7 @@ check-clean: > > > > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > > > > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > > > > rm -f tests/test-qapi-gen-timestamp > > > > + rm -f tests/dbus-vmstate1-gen-timestamp > > > > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > > > > > > > clean: check-clean > > > > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > > > > new file mode 100644 > > > > index 0000000000..45d44916d2 > > > > --- /dev/null > > > > +++ b/tests/dbus-vmstate-test.c > > > > @@ -0,0 +1,371 @@ > > > > +#include "qemu/osdep.h" > > > > +#include <glib/gstdio.h> > > > > +#include <gio/gio.h> > > > > +#include "libqtest.h" > > > > +#include "qemu-common.h" > > > > +#include "dbus-vmstate1.h" > > > > + > > > > +static char *workdir; > > > > + > > > > +typedef struct TestServerId { > > > > + const char *name; > > > > + const char *data; > > > > + size_t size; > > > > +} TestServerId; > > > > + > > > > +static const TestServerId idA = { > > > > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > > > > +}; > > > > + > > > > +static const TestServerId idB = { > > > > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > > > > +}; > > > > + > > > > +typedef struct TestServer { > > > > + const TestServerId *id; > > > > + bool save_called; > > > > + bool load_called; > > > > + GDBusObjectManagerServer *om; > > > > + GDBusServer *server; > > > > +} TestServer; > > > > + > > > > +typedef struct Test { > > > > + bool migrate_fail; > > > > + TestServer srcA; > > > > + TestServer dstA; > > > > + TestServer srcB; > > > > + TestServer dstB; > > > > + GMainLoop *loop, *dbus_loop; > > > > + QTestState *src_qemu; > > > > +} Test; > > > > + > > > > +GMutex mutex; > > > > +GCond cond; > > > > + > > > > +static gboolean > > > > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > > > > + const gchar *arg_data, gpointer user_data) > > > > +{ > > > > + TestServer *h = user_data; > > > > + GVariant *args, *var; > > > > + const uint8_t *data; > > > > + size_t size; > > > > + > > > > + args = g_dbus_method_invocation_get_parameters(invocation); > > > > + var = g_variant_get_child_value(args, 0); > > > > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > > > > + g_assert_cmpuint(size, ==, h->id->size); > > > > + g_assert(!memcmp(data, h->id->data, h->id->size)); > > > > + h->load_called = true; > > > > + g_variant_unref(var); > > > > + > > > > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > > > > + return TRUE; > > > > +} > > > > + > > > > +static gboolean > > > > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > > > > + gpointer user_data) > > > > +{ > > > > + TestServer *h = user_data; > > > > + GVariant *var; > > > > + > > > > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > > > > + h->id->data, h->id->size, sizeof(char)); > > > > + g_dbus_method_invocation_return_value(invocation, > > > > + g_variant_new("(@ay)", var)); > > > > + h->save_called = true; > > > > + > > > > + return TRUE; > > > > +} > > > > + > > > > +static void > > > > +connection_closed(GDBusConnection *connection, > > > > + gboolean remote_peer_vanished, > > > > + GError *Error, > > > > + gpointer user_data) > > > > +{ > > > > + TestServer *h = user_data; > > > > + > > > > + g_clear_object(&h->om); > > > > + g_clear_object(&connection); > > > > +} > > > > + > > > > +static GDBusObjectManagerServer * > > > > +get_omserver(GDBusConnection *conn, gpointer user_data) > > > > +{ > > > > + TestServer *h = user_data; > > > > + GDBusObjectManagerServer *om; > > > > + GDBusObjectSkeleton *sk; > > > > + VMState1 *v; > > > > + > > > > + om = g_dbus_object_manager_server_new("/org/qemu"); > > > > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > > > > + > > > > + v = vmstate1_skeleton_new(); > > > > + g_object_set(v, "id", h->id->name, NULL); > > > > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > > > > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > > > > + > > > > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > > > > + g_dbus_object_manager_server_export(om, sk); > > > > + g_dbus_object_manager_server_set_connection(om, conn); > > > > + > > > > + g_clear_object(&v); > > > > + g_clear_object(&sk); > > > > + > > > > + return om; > > > > +} > > > > + > > > > +static gboolean > > > > +on_new_connection(GDBusServer *server, > > > > + GDBusConnection *connection, > > > > + gpointer user_data) > > > > +{ > > > > + TestServer *h = user_data; > > > > + > > > > + g_object_ref(connection); > > > > + g_signal_connect(connection, "closed", > > > > + G_CALLBACK(connection_closed), user_data); > > > > + h->om = get_omserver(connection, user_data); > > > > + > > > > + return TRUE; > > > > +} > > > > + > > > > +static gboolean > > > > +allow_mechanism_cb(GDBusAuthObserver *observer, > > > > + const gchar *mechanism, > > > > + gpointer user_data) > > > > +{ > > > > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > > > > +} > > > > + > > > > +static gboolean > > > > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > > > > + GIOStream *stream, > > > > + GCredentials *credentials, > > > > + gpointer user_data) > > > > +{ > > > > + gboolean authorized = FALSE; > > > > + > > > > + if (credentials != NULL) { > > > > + GCredentials *own_credentials = g_credentials_new(); > > > > + > > > > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > > > > + authorized = TRUE; > > > > + } > > > > + > > > > + g_clear_object(&own_credentials); > > > > + } > > > > + > > > > + return authorized; > > > > +} > > > > + > > > > +static GDBusServer * > > > > +server_start(TestServer *h, const char *p, const TestServerId *id) > > > > +{ > > > > + GDBusAuthObserver *observer = NULL; > > > > + GDBusServer *server = NULL; > > > > + gchar *guid = NULL; > > > > + GError *error = NULL; > > > > + char *addr = NULL; > > > > + > > > > + h->id = id; > > > > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > > > > + guid = g_dbus_generate_guid(); > > > > + observer = g_dbus_auth_observer_new(); > > > > + g_signal_connect(observer, "allow-mechanism", > > > > + G_CALLBACK(allow_mechanism_cb), h); > > > > + g_signal_connect(observer, "authorize-authenticated-peer", > > > > + G_CALLBACK(authorize_authenticated_peer_cb), h); > > > > + > > > > + server = g_dbus_server_new_sync(addr, > > > > + G_DBUS_SERVER_FLAGS_NONE, > > > > + guid, > > > > + observer, > > > > + NULL, /* GCancellable */ > > > > + &error); > > > > + g_dbus_server_start(server); > > > > + g_clear_object(&observer); > > > > + g_free(guid); > > > > + > > > > + if (server == NULL) { > > > > + g_printerr("Error creating server at address %s: %s\n", > > > > + addr, error->message); > > > > + g_error_free(error); > > > > + return NULL; > > > > + } > > > > + > > > > + g_signal_connect(server, "new-connection", > > > > + G_CALLBACK(on_new_connection), h); > > > > + > > > > + g_free(addr); > > > > + return server; > > > > +} > > > > + > > > > + > > > > +static gpointer > > > > +dbus_thread(gpointer p) > > > > +{ > > > > + Test *test = p; > > > > + GMainContext *context = g_main_context_new(); > > > > + GMainLoop *loop = g_main_loop_new(context, FALSE); > > > > + > > > > + g_main_context_push_thread_default(context); > > > > + > > > > + g_mutex_lock(&mutex); > > > > + test->srcA.server = server_start(&test->srcA, "src", &idA); > > > > + test->srcB.server = server_start(&test->srcB, "src", &idB); > > > > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > > > > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > > > > + test->dbus_loop = loop; > > > > + g_cond_signal(&cond); > > > > + g_mutex_unlock(&mutex); > > > > + > > > > + g_main_loop_run(loop); > > > > + > > > > + g_main_loop_unref(loop); > > > > + g_main_context_unref(context); > > > > + > > > > + g_mutex_lock(&mutex); > > > > + g_clear_object(&test->srcA.server); > > > > + g_clear_object(&test->srcB.server); > > > > + g_clear_object(&test->dstA.server); > > > > + g_clear_object(&test->dstB.server); > > > > + g_mutex_unlock(&mutex); > > > > + > > > > + return NULL; > > > > +} > > > > + > > > > +static gboolean > > > > +wait_for_migration_complete(gpointer user_data) > > > > +{ > > > > + Test *test = user_data; > > > > + QDict *rsp_return; > > > > + bool stop = false; > > > > + const char *status; > > > > + > > > > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > > > > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > > > > + status = qdict_get_str(rsp_return, "status"); > > > > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > > > > + stop = true; > > > > + g_assert_cmpstr(status, ==, > > > > + test->migrate_fail ? "failed" : "completed"); > > > > + } > > > > + qobject_unref(rsp_return); > > > > + > > > > + if (stop) { > > > > + g_main_loop_quit(test->loop); > > > > + } > > > > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > > > > +} > > > > + > > > > +static void migrate(QTestState *who, const char *uri) > > > > +{ > > > > + QDict *args, *rsp; > > > > + > > > > + args = qdict_new(); > > > > + qdict_put_str(args, "uri", uri); > > > > + > > > > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > > > > + > > > > + g_assert(qdict_haskey(rsp, "return")); > > > > + qobject_unref(rsp); > > > > +} > > > > + > > > > +static void > > > > +test_dbus_vmstate(Test *test) > > > > +{ > > > > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > > > > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > > > > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > > > > + GThread *t = g_thread_new("dbus", dbus_thread, test); > > > > + > > > > + g_mutex_lock(&mutex); > > > > + while (!test->dbus_loop) { > > > > + g_cond_wait(&cond, &mutex); > > > > + } > > > > + > > > > + src_qemu_args = > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > + "-object dbus-vmstate,id=dvB,addr=%s", > > > > + g_dbus_server_get_client_address(test->srcA.server), > > > > + g_dbus_server_get_client_address(test->srcB.server)); > > > > + > > > > + > > > > + dst_qemu_args = > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > + "-object dbus-vmstate,id=dvB,addr=%s " > > > > + "-incoming %s", > > > > + g_dbus_server_get_client_address(test->dstA.server), > > > > + g_dbus_server_get_client_address(test->dstB.server), > > > > + uri); > > > > + > > > > + src_qemu = qtest_init(src_qemu_args); > > > > + dst_qemu = qtest_init(dst_qemu_args); > > > > + > > > > + test->loop = g_main_loop_new(NULL, TRUE); > > > > + > > > > + migrate(src_qemu, uri); > > > > + test->src_qemu = src_qemu; > > > > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > > > > + > > > > + g_main_loop_run(test->loop); > > > > + g_main_loop_unref(test->loop); > > > > + > > > > + g_free(uri); > > > > + qtest_quit(dst_qemu); > > > > + qtest_quit(src_qemu); > > > > + g_free(dst_qemu_args); > > > > + g_free(src_qemu_args); > > > > + > > > > + g_main_loop_quit(test->dbus_loop); > > > > + g_mutex_unlock(&mutex); > > > > + > > > > + g_thread_join(t); > > > > +} > > > > + > > > > +static void > > > > +check_migrated(TestServer *s, TestServer *d) > > > > +{ > > > > + assert(s->save_called); > > > > + assert(!s->load_called); > > > > + assert(!d->save_called); > > > > + assert(d->load_called); > > > > +} > > > > + > > > > +static void > > > > +test_dbus_vmstate_migrate(void) > > > > +{ > > > > + Test test = { }; > > > > + > > > > + test_dbus_vmstate(&test); > > > > + > > > > + check_migrated(&test.srcA, &test.dstA); > > > > + check_migrated(&test.srcB, &test.dstB); > > > > +} > > > > + > > > > +int > > > > +main(int argc, char **argv) > > > > +{ > > > > + GError *err = NULL; > > > > + int ret; > > > > + > > > > + g_test_init(&argc, &argv, NULL); > > > > + > > > > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > > > > + if (!workdir) { > > > > + g_error("Unable to create temporary dir: %s\n", err->message); > > > > + } > > > > + > > > > + qtest_add_func("/dbus-vmstate/migrate", > > > > + test_dbus_vmstate_migrate); > > > > + > > > > + ret = g_test_run(); > > > > + > > > > + rmdir(workdir); > > > > + g_free(workdir); > > > > + > > > > + return ret; > > > > +} > > > > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > > > > new file mode 100644 > > > > index 0000000000..cc8563be4c > > > > --- /dev/null > > > > +++ b/tests/dbus-vmstate1.xml > > > > @@ -0,0 +1,12 @@ > > > > +<?xml version="1.0"?> > > > > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > > > > + <interface name="org.qemu.VMState1"> > > > > + <property name="Id" type="s" access="read"/> > > > > + <method name="Load"> > > > > + <arg type="ay" name="data" direction="in"/> > > > > + </method> > > > > + <method name="Save"> > > > > + <arg type="ay" name="data" direction="out"/> > > > > + </method> > > > > + </interface> > > > > +</node> > > > > -- > > > > 2.23.0.rc1 > > > > > > > -- > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-22 11:57 ` Marc-André Lureau @ 2019-08-22 12:19 ` Dr. David Alan Gilbert 2019-08-22 12:38 ` Marc-André Lureau 0 siblings, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-22 12:19 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > Hi > > On Thu, Aug 22, 2019 at 3:41 PM Dr. David Alan Gilbert > <dgilbert@redhat.com> wrote: > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > Hi > > > > > > On Thu, Aug 22, 2019 at 2:56 PM Dr. David Alan Gilbert > > > <dgilbert@redhat.com> wrote: > > > > > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > > > When instanciated, this object will connect to the given D-Bus > > > > > bus. During migration, it will take the data from org.qemu.VMState1 > > > > > instances. > > > > > > > > > > See documentation for further details. > > > > > > > > I'll leave the main review to someone who understands the details of > > > > DBUS, however from the migration side: > > > > > > > > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > > > > > <snip> > > > > > > > > > + > > > > > +static int > > > > > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > > > > > +{ > > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > > + uint8_t *data = NULL; > > > > > + int ret = -1; > > > > > + char id[256]; > > > > > + unsigned int size; > > > > > > > > Higher level question; do you actually need a separate ID, or does > > > > the ID that's stored by the migration code in the section name > > > > give you enough uniqueness? > > > > > > > > > The ID stored in migration section name is based on the handler ID: > > > TYPE_DBUS_VMSTATE "-" handler_id > > > > > > By design, handler_id must be unique (and documented as such). > > > > > > Do you see a problem with that approach? > > > > OK, so if that's unique, why do you need an 'id' stored here? > > Oh I see, that's mostly a relic of v1, where multiple ID > "sub-sections" would exist for each helper. > > However, if we want to allow having several "sub-sections" again, that > might come handy to keep compatibility. It may be better to have a > simple version field for that instead. OK, that's fine > > > > > > > > > > > + if (qemu_get_counted_string(f, id) == 0) { > > > > > + error_report("Invalid vmstate Id"); > > > > > + goto end; > > > > > > > > I generally prefer to include something telling me where the error has > > > > come from, just to make it easier to track down if I see the error in > > > > a log; e.g. use __func__ > > > > > > ok > > > > > > > > > > > > + } > > > > > + > > > > > + if (g_strcmp0(id, self->id)) { > > > > > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > > > > > + goto end; > > > > > + } > > > > > + > > > > > + size = qemu_get_be32(f); > > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > > + error_report("Invalid vmstate size: %u", size); > > > > > + goto end; > > > > > + } > > > > > + > > > > > + data = g_malloc(size); > > > > > + if (qemu_get_buffer(f, data, size) != size) { > > > > > + error_report("Failed to read %u bytes", size); > > > > > + goto end; > > > > > + } > > > > > + > > > > > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > > > > > + error_report("Failed to restore Id '%s'", id); > > > > > + goto end; > > > > > + } > > > > > + > > > > > + ret = 0; > > > > > + > > > > > +end: > > > > > + g_clear_pointer(&data, g_free); > > > > > + return ret; > > > > > +} > > > > > + > > > > > +static void > > > > > +dbus_save_state(QEMUFile *f, void *opaque) > > > > > +{ > > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > > + GVariant *result = NULL, *child = NULL; > > > > > + const uint8_t *data; > > > > > + size_t size; > > > > > + GError *err = NULL; > > > > > + > > > > > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > > > > > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > > > > > + -1, NULL, &err); > > > > > + if (!result) { > > > > > + error_report("Failed to Save: %s", err->message); > > > > > + g_clear_error(&err); > > > > > + goto end; > > > > > + } > > > > > + > > > > > + child = g_variant_get_child_value(result, 0); > > > > > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > > > > > + if (!data) { > > > > > + error_report("Failed to Save: not a byte array"); > > > > > + goto end; > > > > > + } > > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > > + error_report("Too much vmstate data to save: %zu", size); > > > > > + goto end; > > > > > + } > > > > > + > > > > > + qemu_put_counted_string(f, self->id); > > > > > + qemu_put_be32(f, size); > > > > > + qemu_put_buffer(f, data, size); > > > > > + > > > > > +end: > > > > > + g_clear_pointer(&child, g_variant_unref); > > > > > + g_clear_pointer(&result, g_variant_unref); > > > > > +} > > > > > + > > > > > +static const SaveVMHandlers savevm_handlers = { > > > > > + .save_state = dbus_save_state, > > > > > + .load_state = dbus_load_state, > > > > > +}; > > > > > + > > > > > +static void > > > > > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > > > > > +{ > > > > > + DBusVMState *self = DBUS_VMSTATE(uc); > > > > > + GError *err = NULL; > > > > > + GDBusConnection *bus = NULL; > > > > > + GDBusProxy *proxy = NULL; > > > > > + char *idstr = NULL; > > > > > + > > > > > + if (!self->dbus_addr) { > > > > > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > > > > > + return; > > > > > + } > > > > > + > > > > > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > > > > > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > > > > > + NULL, NULL, &err); > > > > > + if (err) { > > > > > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > > > > > + g_clear_error(&err); > > > > > + return; > > > > > + } > > > > > + > > > > > + self->bus = bus; > > > > > + > > > > > + proxy = g_dbus_proxy_new_sync(bus, > > > > > + G_DBUS_PROXY_FLAGS_NONE, > > > > > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > > > > > + NULL, > > > > > + "/org/qemu/VMState1", > > > > > + "org.qemu.VMState1", > > > > > + NULL, &err); > > > > > + > > > > > + if (err) { > > > > > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > > > > > + g_clear_error(&err); > > > > > + return; > > > > > + } > > > > > + > > > > > + self->proxy = proxy; > > > > > + > > > > > + self->id = dbus_proxy_get_id(proxy, &err); > > > > > + if (!self->id) { > > > > > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > > > > > + g_clear_error(&err); > > > > > + return; > > > > > + } > > > > > + > > > > > + idstr = get_idstr(self); > > > > > + if (register_savevm_live(NULL, idstr, 0, 0, > > > > > + &savevm_handlers, self) < 0) { > > > > > + error_setg(errp, "Failed to register savevm handler"); > > > > > + } > > > > > > > > Can you try and avoid register_savevm_live if possible and just wire a > > > > vmsd onto the class like most other devices? We don't have many > > > > register_savevm_live calls, and I'd like to get them down to just the > > > > iterative devices. > > > > > > Sure if I could, but it's not a device. > > > > > > Perhaps we could have a qom interface for vmsd users? I can try that eventually. > > > > Hmm, why isn't it a device? > > It doesn't need anything from Device (which is heavyweight), it is a > simple user-creatable object that holds a dbus connection and reacts > on migration events. > > If you take slirp helper process as an example, it's a "service" for a > VM. Not a device. You could have the same helper for various devices > etc, or more helpers for the same device. I think you're better calling this a device; the slirp world doesn't fit nicely either and it would be better if it was a device as well. Dave > > > > > > Dave > > > > > thanks > > > > > > > Dave > > > > > > > > > > > > > + g_free(idstr); > > > > > +} > > > > > + > > > > > +static void > > > > > +dbus_vmstate_finalize(Object *o) > > > > > +{ > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > + char *idstr = get_idstr(self); > > > > > + > > > > > + unregister_savevm(NULL, idstr, self); > > > > > + > > > > > + g_clear_object(&self->bus); > > > > > + g_clear_object(&self->proxy); > > > > > + g_free(self->dbus_addr); > > > > > + g_free(self->id); > > > > > + g_free(idstr); > > > > > +} > > > > > + > > > > > +static char * > > > > > +get_dbus_addr(Object *o, Error **errp) > > > > > +{ > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > + > > > > > + return g_strdup(self->dbus_addr); > > > > > +} > > > > > + > > > > > +static void > > > > > +set_dbus_addr(Object *o, const char *str, Error **errp) > > > > > +{ > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > + > > > > > + g_free(self->dbus_addr); > > > > > + self->dbus_addr = g_strdup(str); > > > > > +} > > > > > + > > > > > +static void > > > > > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > > > > > +{ > > > > > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > > > > > + > > > > > + ucc->complete = dbus_vmstate_complete; > > > > > + > > > > > + object_class_property_add_str(oc, "addr", > > > > > + get_dbus_addr, set_dbus_addr, > > > > > + &error_abort); > > > > > +} > > > > > + > > > > > +static const TypeInfo dbus_vmstate_info = { > > > > > + .name = TYPE_DBUS_VMSTATE, > > > > > + .parent = TYPE_OBJECT, > > > > > + .instance_size = sizeof(DBusVMState), > > > > > + .instance_finalize = dbus_vmstate_finalize, > > > > > + .class_size = sizeof(DBusVMStateClass), > > > > > + .class_init = dbus_vmstate_class_init, > > > > > + .interfaces = (InterfaceInfo[]) { > > > > > + { TYPE_USER_CREATABLE }, > > > > > + { } > > > > > + } > > > > > +}; > > > > > + > > > > > +static void > > > > > +register_types(void) > > > > > +{ > > > > > + type_register_static(&dbus_vmstate_info); > > > > > +} > > > > > + > > > > > +type_init(register_types); > > > > > diff --git a/configure b/configure > > > > > index 714e7fb6a1..e5b34f5ca7 100755 > > > > > --- a/configure > > > > > +++ b/configure > > > > > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > > > > > gio=yes > > > > > gio_cflags=$($pkg_config --cflags gio-2.0) > > > > > gio_libs=$($pkg_config --libs gio-2.0) > > > > > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > > > > > else > > > > > gio=no > > > > > fi > > > > > > > > > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > > > > > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > > > > > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > > > > > +fi > > > > > + > > > > > # Sanity check that the current size_t matches the > > > > > # size that glib thinks it should be. This catches > > > > > # problems on multi-arch where people try to build > > > > > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > > > > > echo "CONFIG_GIO=y" >> $config_host_mak > > > > > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > > > > > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > > > > > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > > > > > fi > > > > > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > > > > > if test "$gnutls" = "yes" ; then > > > > > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > > > > > new file mode 100644 > > > > > index 0000000000..4a32a183fb > > > > > --- /dev/null > > > > > +++ b/docs/interop/dbus-vmstate.rst > > > > > @@ -0,0 +1,63 @@ > > > > > +============ > > > > > +DBus VMState > > > > > +============ > > > > > + > > > > > +Introduction > > > > > +============ > > > > > + > > > > > +Helper processes may have their state migrated with the help of > > > > > +QEMU "dbus-vmstate" objects. > > > > > + > > > > > +At this point, the connection to the helper is done in DBus > > > > > +peer-to-peer mode (no initial Hello, and no bus name for > > > > > +communication). The helper must be listening to the given address. > > > > > + > > > > > +Helper may save arbitrary data to be transferred in the migration > > > > > +stream and restored/loaded on destination. > > > > > + > > > > > +The data amount to be transferred is limited to 1Mb. The state must be > > > > > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > > > > > +reply anyway, and migration would fail if the data isn't given quickly > > > > > +enough) > > > > > + > > > > > +Interface > > > > > +========= > > > > > + > > > > > +On /org/qemu/VMState1 object path: > > > > > + > > > > > +.. code:: xml > > > > > + > > > > > + <interface name="org.qemu.VMState1"> > > > > > + <property name="Id" type="s" access="read"/> > > > > > + <method name="Load"> > > > > > + <arg type="ay" name="data" direction="in"/> > > > > > + </method> > > > > > + <method name="Save"> > > > > > + <arg type="ay" name="data" direction="out"/> > > > > > + </method> > > > > > + </interface> > > > > > + > > > > > +"Id" property > > > > > +------------- > > > > > + > > > > > +A utf8 encoded string that identifies the helper uniquely. > > > > > +Must be <256 bytes. > > > > > + > > > > > +Load(in u8[] bytes) method > > > > > +-------------------------- > > > > > + > > > > > +The method called on destination with the state to restore. > > > > > + > > > > > +The helper may be initially started in a waiting state (with > > > > > +an --incoming argument for example), and it may resume on load > > > > > +success. > > > > > + > > > > > +An error may be returned to the caller. > > > > > + > > > > > +Save(out u8[] bytes) method > > > > > +--------------------------- > > > > > + > > > > > +The method called on the source to get the current state to be > > > > > +migrated. The helper should continue to run normally. > > > > > + > > > > > +An error may be returned to the caller. > > > > > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > > > > > index b4bfcab417..6bb173cfa6 100644 > > > > > --- a/docs/interop/index.rst > > > > > +++ b/docs/interop/index.rst > > > > > @@ -13,6 +13,7 @@ Contents: > > > > > :maxdepth: 2 > > > > > > > > > > bitmaps > > > > > + dbus-vmstate > > > > > live-block-operations > > > > > pr-helper > > > > > vhost-user > > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > > > index fd7fdb8658..2c610086a7 100644 > > > > > --- a/tests/Makefile.include > > > > > +++ b/tests/Makefile.include > > > > > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > > > > > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > > > > > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > > > > > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > > > > > - > > > > > +ifneq ($(GDBUS_CODEGEN),) > > > > > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > > > > > +endif > > > > > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > > > > > check-qtest-i386-y += tests/fdc-test$(EXESUF) > > > > > check-qtest-i386-y += tests/ide-test$(EXESUF) > > > > > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > > > > > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > > > > > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > > > > > > > > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > > > > > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > > > > > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > > > > > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > > > > > + "GEN","$(@:%-timestamp=%)") > > > > > + @>$@ > > > > > + > > > > > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > > > > > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > > > > > + > > > > > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > > > > > + > > > > > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > > > > > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > > > > > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > > > > > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > > > > > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > > > > > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > > > > > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > > > > > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > > > > > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > > > > > @@ -1172,6 +1187,7 @@ check-clean: > > > > > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > > > > > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > > > > > rm -f tests/test-qapi-gen-timestamp > > > > > + rm -f tests/dbus-vmstate1-gen-timestamp > > > > > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > > > > > > > > > clean: check-clean > > > > > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > > > > > new file mode 100644 > > > > > index 0000000000..45d44916d2 > > > > > --- /dev/null > > > > > +++ b/tests/dbus-vmstate-test.c > > > > > @@ -0,0 +1,371 @@ > > > > > +#include "qemu/osdep.h" > > > > > +#include <glib/gstdio.h> > > > > > +#include <gio/gio.h> > > > > > +#include "libqtest.h" > > > > > +#include "qemu-common.h" > > > > > +#include "dbus-vmstate1.h" > > > > > + > > > > > +static char *workdir; > > > > > + > > > > > +typedef struct TestServerId { > > > > > + const char *name; > > > > > + const char *data; > > > > > + size_t size; > > > > > +} TestServerId; > > > > > + > > > > > +static const TestServerId idA = { > > > > > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > > > > > +}; > > > > > + > > > > > +static const TestServerId idB = { > > > > > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > > > > > +}; > > > > > + > > > > > +typedef struct TestServer { > > > > > + const TestServerId *id; > > > > > + bool save_called; > > > > > + bool load_called; > > > > > + GDBusObjectManagerServer *om; > > > > > + GDBusServer *server; > > > > > +} TestServer; > > > > > + > > > > > +typedef struct Test { > > > > > + bool migrate_fail; > > > > > + TestServer srcA; > > > > > + TestServer dstA; > > > > > + TestServer srcB; > > > > > + TestServer dstB; > > > > > + GMainLoop *loop, *dbus_loop; > > > > > + QTestState *src_qemu; > > > > > +} Test; > > > > > + > > > > > +GMutex mutex; > > > > > +GCond cond; > > > > > + > > > > > +static gboolean > > > > > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > > > > > + const gchar *arg_data, gpointer user_data) > > > > > +{ > > > > > + TestServer *h = user_data; > > > > > + GVariant *args, *var; > > > > > + const uint8_t *data; > > > > > + size_t size; > > > > > + > > > > > + args = g_dbus_method_invocation_get_parameters(invocation); > > > > > + var = g_variant_get_child_value(args, 0); > > > > > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > > > > > + g_assert_cmpuint(size, ==, h->id->size); > > > > > + g_assert(!memcmp(data, h->id->data, h->id->size)); > > > > > + h->load_called = true; > > > > > + g_variant_unref(var); > > > > > + > > > > > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > > > > > + return TRUE; > > > > > +} > > > > > + > > > > > +static gboolean > > > > > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > > > > > + gpointer user_data) > > > > > +{ > > > > > + TestServer *h = user_data; > > > > > + GVariant *var; > > > > > + > > > > > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > > > > > + h->id->data, h->id->size, sizeof(char)); > > > > > + g_dbus_method_invocation_return_value(invocation, > > > > > + g_variant_new("(@ay)", var)); > > > > > + h->save_called = true; > > > > > + > > > > > + return TRUE; > > > > > +} > > > > > + > > > > > +static void > > > > > +connection_closed(GDBusConnection *connection, > > > > > + gboolean remote_peer_vanished, > > > > > + GError *Error, > > > > > + gpointer user_data) > > > > > +{ > > > > > + TestServer *h = user_data; > > > > > + > > > > > + g_clear_object(&h->om); > > > > > + g_clear_object(&connection); > > > > > +} > > > > > + > > > > > +static GDBusObjectManagerServer * > > > > > +get_omserver(GDBusConnection *conn, gpointer user_data) > > > > > +{ > > > > > + TestServer *h = user_data; > > > > > + GDBusObjectManagerServer *om; > > > > > + GDBusObjectSkeleton *sk; > > > > > + VMState1 *v; > > > > > + > > > > > + om = g_dbus_object_manager_server_new("/org/qemu"); > > > > > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > > > > > + > > > > > + v = vmstate1_skeleton_new(); > > > > > + g_object_set(v, "id", h->id->name, NULL); > > > > > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > > > > > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > > > > > + > > > > > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > > > > > + g_dbus_object_manager_server_export(om, sk); > > > > > + g_dbus_object_manager_server_set_connection(om, conn); > > > > > + > > > > > + g_clear_object(&v); > > > > > + g_clear_object(&sk); > > > > > + > > > > > + return om; > > > > > +} > > > > > + > > > > > +static gboolean > > > > > +on_new_connection(GDBusServer *server, > > > > > + GDBusConnection *connection, > > > > > + gpointer user_data) > > > > > +{ > > > > > + TestServer *h = user_data; > > > > > + > > > > > + g_object_ref(connection); > > > > > + g_signal_connect(connection, "closed", > > > > > + G_CALLBACK(connection_closed), user_data); > > > > > + h->om = get_omserver(connection, user_data); > > > > > + > > > > > + return TRUE; > > > > > +} > > > > > + > > > > > +static gboolean > > > > > +allow_mechanism_cb(GDBusAuthObserver *observer, > > > > > + const gchar *mechanism, > > > > > + gpointer user_data) > > > > > +{ > > > > > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > > > > > +} > > > > > + > > > > > +static gboolean > > > > > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > > > > > + GIOStream *stream, > > > > > + GCredentials *credentials, > > > > > + gpointer user_data) > > > > > +{ > > > > > + gboolean authorized = FALSE; > > > > > + > > > > > + if (credentials != NULL) { > > > > > + GCredentials *own_credentials = g_credentials_new(); > > > > > + > > > > > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > > > > > + authorized = TRUE; > > > > > + } > > > > > + > > > > > + g_clear_object(&own_credentials); > > > > > + } > > > > > + > > > > > + return authorized; > > > > > +} > > > > > + > > > > > +static GDBusServer * > > > > > +server_start(TestServer *h, const char *p, const TestServerId *id) > > > > > +{ > > > > > + GDBusAuthObserver *observer = NULL; > > > > > + GDBusServer *server = NULL; > > > > > + gchar *guid = NULL; > > > > > + GError *error = NULL; > > > > > + char *addr = NULL; > > > > > + > > > > > + h->id = id; > > > > > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > > > > > + guid = g_dbus_generate_guid(); > > > > > + observer = g_dbus_auth_observer_new(); > > > > > + g_signal_connect(observer, "allow-mechanism", > > > > > + G_CALLBACK(allow_mechanism_cb), h); > > > > > + g_signal_connect(observer, "authorize-authenticated-peer", > > > > > + G_CALLBACK(authorize_authenticated_peer_cb), h); > > > > > + > > > > > + server = g_dbus_server_new_sync(addr, > > > > > + G_DBUS_SERVER_FLAGS_NONE, > > > > > + guid, > > > > > + observer, > > > > > + NULL, /* GCancellable */ > > > > > + &error); > > > > > + g_dbus_server_start(server); > > > > > + g_clear_object(&observer); > > > > > + g_free(guid); > > > > > + > > > > > + if (server == NULL) { > > > > > + g_printerr("Error creating server at address %s: %s\n", > > > > > + addr, error->message); > > > > > + g_error_free(error); > > > > > + return NULL; > > > > > + } > > > > > + > > > > > + g_signal_connect(server, "new-connection", > > > > > + G_CALLBACK(on_new_connection), h); > > > > > + > > > > > + g_free(addr); > > > > > + return server; > > > > > +} > > > > > + > > > > > + > > > > > +static gpointer > > > > > +dbus_thread(gpointer p) > > > > > +{ > > > > > + Test *test = p; > > > > > + GMainContext *context = g_main_context_new(); > > > > > + GMainLoop *loop = g_main_loop_new(context, FALSE); > > > > > + > > > > > + g_main_context_push_thread_default(context); > > > > > + > > > > > + g_mutex_lock(&mutex); > > > > > + test->srcA.server = server_start(&test->srcA, "src", &idA); > > > > > + test->srcB.server = server_start(&test->srcB, "src", &idB); > > > > > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > > > > > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > > > > > + test->dbus_loop = loop; > > > > > + g_cond_signal(&cond); > > > > > + g_mutex_unlock(&mutex); > > > > > + > > > > > + g_main_loop_run(loop); > > > > > + > > > > > + g_main_loop_unref(loop); > > > > > + g_main_context_unref(context); > > > > > + > > > > > + g_mutex_lock(&mutex); > > > > > + g_clear_object(&test->srcA.server); > > > > > + g_clear_object(&test->srcB.server); > > > > > + g_clear_object(&test->dstA.server); > > > > > + g_clear_object(&test->dstB.server); > > > > > + g_mutex_unlock(&mutex); > > > > > + > > > > > + return NULL; > > > > > +} > > > > > + > > > > > +static gboolean > > > > > +wait_for_migration_complete(gpointer user_data) > > > > > +{ > > > > > + Test *test = user_data; > > > > > + QDict *rsp_return; > > > > > + bool stop = false; > > > > > + const char *status; > > > > > + > > > > > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > > > > > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > > > > > + status = qdict_get_str(rsp_return, "status"); > > > > > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > > > > > + stop = true; > > > > > + g_assert_cmpstr(status, ==, > > > > > + test->migrate_fail ? "failed" : "completed"); > > > > > + } > > > > > + qobject_unref(rsp_return); > > > > > + > > > > > + if (stop) { > > > > > + g_main_loop_quit(test->loop); > > > > > + } > > > > > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > > > > > +} > > > > > + > > > > > +static void migrate(QTestState *who, const char *uri) > > > > > +{ > > > > > + QDict *args, *rsp; > > > > > + > > > > > + args = qdict_new(); > > > > > + qdict_put_str(args, "uri", uri); > > > > > + > > > > > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > > > > > + > > > > > + g_assert(qdict_haskey(rsp, "return")); > > > > > + qobject_unref(rsp); > > > > > +} > > > > > + > > > > > +static void > > > > > +test_dbus_vmstate(Test *test) > > > > > +{ > > > > > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > > > > > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > > > > > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > > > > > + GThread *t = g_thread_new("dbus", dbus_thread, test); > > > > > + > > > > > + g_mutex_lock(&mutex); > > > > > + while (!test->dbus_loop) { > > > > > + g_cond_wait(&cond, &mutex); > > > > > + } > > > > > + > > > > > + src_qemu_args = > > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > > + "-object dbus-vmstate,id=dvB,addr=%s", > > > > > + g_dbus_server_get_client_address(test->srcA.server), > > > > > + g_dbus_server_get_client_address(test->srcB.server)); > > > > > + > > > > > + > > > > > + dst_qemu_args = > > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > > + "-object dbus-vmstate,id=dvB,addr=%s " > > > > > + "-incoming %s", > > > > > + g_dbus_server_get_client_address(test->dstA.server), > > > > > + g_dbus_server_get_client_address(test->dstB.server), > > > > > + uri); > > > > > + > > > > > + src_qemu = qtest_init(src_qemu_args); > > > > > + dst_qemu = qtest_init(dst_qemu_args); > > > > > + > > > > > + test->loop = g_main_loop_new(NULL, TRUE); > > > > > + > > > > > + migrate(src_qemu, uri); > > > > > + test->src_qemu = src_qemu; > > > > > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > > > > > + > > > > > + g_main_loop_run(test->loop); > > > > > + g_main_loop_unref(test->loop); > > > > > + > > > > > + g_free(uri); > > > > > + qtest_quit(dst_qemu); > > > > > + qtest_quit(src_qemu); > > > > > + g_free(dst_qemu_args); > > > > > + g_free(src_qemu_args); > > > > > + > > > > > + g_main_loop_quit(test->dbus_loop); > > > > > + g_mutex_unlock(&mutex); > > > > > + > > > > > + g_thread_join(t); > > > > > +} > > > > > + > > > > > +static void > > > > > +check_migrated(TestServer *s, TestServer *d) > > > > > +{ > > > > > + assert(s->save_called); > > > > > + assert(!s->load_called); > > > > > + assert(!d->save_called); > > > > > + assert(d->load_called); > > > > > +} > > > > > + > > > > > +static void > > > > > +test_dbus_vmstate_migrate(void) > > > > > +{ > > > > > + Test test = { }; > > > > > + > > > > > + test_dbus_vmstate(&test); > > > > > + > > > > > + check_migrated(&test.srcA, &test.dstA); > > > > > + check_migrated(&test.srcB, &test.dstB); > > > > > +} > > > > > + > > > > > +int > > > > > +main(int argc, char **argv) > > > > > +{ > > > > > + GError *err = NULL; > > > > > + int ret; > > > > > + > > > > > + g_test_init(&argc, &argv, NULL); > > > > > + > > > > > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > > > > > + if (!workdir) { > > > > > + g_error("Unable to create temporary dir: %s\n", err->message); > > > > > + } > > > > > + > > > > > + qtest_add_func("/dbus-vmstate/migrate", > > > > > + test_dbus_vmstate_migrate); > > > > > + > > > > > + ret = g_test_run(); > > > > > + > > > > > + rmdir(workdir); > > > > > + g_free(workdir); > > > > > + > > > > > + return ret; > > > > > +} > > > > > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > > > > > new file mode 100644 > > > > > index 0000000000..cc8563be4c > > > > > --- /dev/null > > > > > +++ b/tests/dbus-vmstate1.xml > > > > > @@ -0,0 +1,12 @@ > > > > > +<?xml version="1.0"?> > > > > > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > > > > > + <interface name="org.qemu.VMState1"> > > > > > + <property name="Id" type="s" access="read"/> > > > > > + <method name="Load"> > > > > > + <arg type="ay" name="data" direction="in"/> > > > > > + </method> > > > > > + <method name="Save"> > > > > > + <arg type="ay" name="data" direction="out"/> > > > > > + </method> > > > > > + </interface> > > > > > +</node> > > > > > -- > > > > > 2.23.0.rc1 > > > > > > > > > -- > > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > -- > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-22 12:19 ` Dr. David Alan Gilbert @ 2019-08-22 12:38 ` Marc-André Lureau 2019-08-22 12:51 ` Dr. David Alan Gilbert 0 siblings, 1 reply; 28+ messages in thread From: Marc-André Lureau @ 2019-08-22 12:38 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela Hi On Thu, Aug 22, 2019 at 4:20 PM Dr. David Alan Gilbert <dgilbert@redhat.com> wrote: > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > Hi > > > > On Thu, Aug 22, 2019 at 3:41 PM Dr. David Alan Gilbert > > <dgilbert@redhat.com> wrote: > > > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > > Hi > > > > > > > > On Thu, Aug 22, 2019 at 2:56 PM Dr. David Alan Gilbert > > > > <dgilbert@redhat.com> wrote: > > > > > > > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > > > > When instanciated, this object will connect to the given D-Bus > > > > > > bus. During migration, it will take the data from org.qemu.VMState1 > > > > > > instances. > > > > > > > > > > > > See documentation for further details. > > > > > > > > > > I'll leave the main review to someone who understands the details of > > > > > DBUS, however from the migration side: > > > > > > > > > > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > > > > > > > <snip> > > > > > > > > > > > + > > > > > > +static int > > > > > > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > > > > > > +{ > > > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > > > + uint8_t *data = NULL; > > > > > > + int ret = -1; > > > > > > + char id[256]; > > > > > > + unsigned int size; > > > > > > > > > > Higher level question; do you actually need a separate ID, or does > > > > > the ID that's stored by the migration code in the section name > > > > > give you enough uniqueness? > > > > > > > > > > > > The ID stored in migration section name is based on the handler ID: > > > > TYPE_DBUS_VMSTATE "-" handler_id > > > > > > > > By design, handler_id must be unique (and documented as such). > > > > > > > > Do you see a problem with that approach? > > > > > > OK, so if that's unique, why do you need an 'id' stored here? > > > > Oh I see, that's mostly a relic of v1, where multiple ID > > "sub-sections" would exist for each helper. > > > > However, if we want to allow having several "sub-sections" again, that > > might come handy to keep compatibility. It may be better to have a > > simple version field for that instead. > > OK, that's fine > > > > > > > > > > > > > > > + if (qemu_get_counted_string(f, id) == 0) { > > > > > > + error_report("Invalid vmstate Id"); > > > > > > + goto end; > > > > > > > > > > I generally prefer to include something telling me where the error has > > > > > come from, just to make it easier to track down if I see the error in > > > > > a log; e.g. use __func__ > > > > > > > > ok > > > > > > > > > > > > > > > + } > > > > > > + > > > > > > + if (g_strcmp0(id, self->id)) { > > > > > > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > > > > > > + goto end; > > > > > > + } > > > > > > + > > > > > > + size = qemu_get_be32(f); > > > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > > > + error_report("Invalid vmstate size: %u", size); > > > > > > + goto end; > > > > > > + } > > > > > > + > > > > > > + data = g_malloc(size); > > > > > > + if (qemu_get_buffer(f, data, size) != size) { > > > > > > + error_report("Failed to read %u bytes", size); > > > > > > + goto end; > > > > > > + } > > > > > > + > > > > > > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > > > > > > + error_report("Failed to restore Id '%s'", id); > > > > > > + goto end; > > > > > > + } > > > > > > + > > > > > > + ret = 0; > > > > > > + > > > > > > +end: > > > > > > + g_clear_pointer(&data, g_free); > > > > > > + return ret; > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +dbus_save_state(QEMUFile *f, void *opaque) > > > > > > +{ > > > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > > > + GVariant *result = NULL, *child = NULL; > > > > > > + const uint8_t *data; > > > > > > + size_t size; > > > > > > + GError *err = NULL; > > > > > > + > > > > > > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > > > > > > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > > > > > > + -1, NULL, &err); > > > > > > + if (!result) { > > > > > > + error_report("Failed to Save: %s", err->message); > > > > > > + g_clear_error(&err); > > > > > > + goto end; > > > > > > + } > > > > > > + > > > > > > + child = g_variant_get_child_value(result, 0); > > > > > > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > > > > > > + if (!data) { > > > > > > + error_report("Failed to Save: not a byte array"); > > > > > > + goto end; > > > > > > + } > > > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > > > + error_report("Too much vmstate data to save: %zu", size); > > > > > > + goto end; > > > > > > + } > > > > > > + > > > > > > + qemu_put_counted_string(f, self->id); > > > > > > + qemu_put_be32(f, size); > > > > > > + qemu_put_buffer(f, data, size); > > > > > > + > > > > > > +end: > > > > > > + g_clear_pointer(&child, g_variant_unref); > > > > > > + g_clear_pointer(&result, g_variant_unref); > > > > > > +} > > > > > > + > > > > > > +static const SaveVMHandlers savevm_handlers = { > > > > > > + .save_state = dbus_save_state, > > > > > > + .load_state = dbus_load_state, > > > > > > +}; > > > > > > + > > > > > > +static void > > > > > > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > > > > > > +{ > > > > > > + DBusVMState *self = DBUS_VMSTATE(uc); > > > > > > + GError *err = NULL; > > > > > > + GDBusConnection *bus = NULL; > > > > > > + GDBusProxy *proxy = NULL; > > > > > > + char *idstr = NULL; > > > > > > + > > > > > > + if (!self->dbus_addr) { > > > > > > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > > > > > > + return; > > > > > > + } > > > > > > + > > > > > > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > > > > > > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > > > > > > + NULL, NULL, &err); > > > > > > + if (err) { > > > > > > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > > > > > > + g_clear_error(&err); > > > > > > + return; > > > > > > + } > > > > > > + > > > > > > + self->bus = bus; > > > > > > + > > > > > > + proxy = g_dbus_proxy_new_sync(bus, > > > > > > + G_DBUS_PROXY_FLAGS_NONE, > > > > > > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > > > > > > + NULL, > > > > > > + "/org/qemu/VMState1", > > > > > > + "org.qemu.VMState1", > > > > > > + NULL, &err); > > > > > > + > > > > > > + if (err) { > > > > > > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > > > > > > + g_clear_error(&err); > > > > > > + return; > > > > > > + } > > > > > > + > > > > > > + self->proxy = proxy; > > > > > > + > > > > > > + self->id = dbus_proxy_get_id(proxy, &err); > > > > > > + if (!self->id) { > > > > > > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > > > > > > + g_clear_error(&err); > > > > > > + return; > > > > > > + } > > > > > > + > > > > > > + idstr = get_idstr(self); > > > > > > + if (register_savevm_live(NULL, idstr, 0, 0, > > > > > > + &savevm_handlers, self) < 0) { > > > > > > + error_setg(errp, "Failed to register savevm handler"); > > > > > > + } > > > > > > > > > > Can you try and avoid register_savevm_live if possible and just wire a > > > > > vmsd onto the class like most other devices? We don't have many > > > > > register_savevm_live calls, and I'd like to get them down to just the > > > > > iterative devices. > > > > > > > > Sure if I could, but it's not a device. > > > > > > > > Perhaps we could have a qom interface for vmsd users? I can try that eventually. > > > > > > Hmm, why isn't it a device? > > > > It doesn't need anything from Device (which is heavyweight), it is a > > simple user-creatable object that holds a dbus connection and reacts > > on migration events. > > > > If you take slirp helper process as an example, it's a "service" for a > > VM. Not a device. You could have the same helper for various devices > > etc, or more helpers for the same device. > > I think you're better calling this a device; the slirp world doesn't fit > nicely either and it would be better if it was a device as well. > The goal here is to migrate helper processes along with qemu. Helper processes may be things like slirp, network services, emulators (such as swtpm, which isn't a device either), or long-running jobs/process done on behalf of qemu, for example a AV compression, spice server... > Dave > > > > > > > > > > Dave > > > > > > > thanks > > > > > > > > > Dave > > > > > > > > > > > > > > > > + g_free(idstr); > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +dbus_vmstate_finalize(Object *o) > > > > > > +{ > > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > > + char *idstr = get_idstr(self); > > > > > > + > > > > > > + unregister_savevm(NULL, idstr, self); > > > > > > + > > > > > > + g_clear_object(&self->bus); > > > > > > + g_clear_object(&self->proxy); > > > > > > + g_free(self->dbus_addr); > > > > > > + g_free(self->id); > > > > > > + g_free(idstr); > > > > > > +} > > > > > > + > > > > > > +static char * > > > > > > +get_dbus_addr(Object *o, Error **errp) > > > > > > +{ > > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > > + > > > > > > + return g_strdup(self->dbus_addr); > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +set_dbus_addr(Object *o, const char *str, Error **errp) > > > > > > +{ > > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > > + > > > > > > + g_free(self->dbus_addr); > > > > > > + self->dbus_addr = g_strdup(str); > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > > > > > > +{ > > > > > > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > > > > > > + > > > > > > + ucc->complete = dbus_vmstate_complete; > > > > > > + > > > > > > + object_class_property_add_str(oc, "addr", > > > > > > + get_dbus_addr, set_dbus_addr, > > > > > > + &error_abort); > > > > > > +} > > > > > > + > > > > > > +static const TypeInfo dbus_vmstate_info = { > > > > > > + .name = TYPE_DBUS_VMSTATE, > > > > > > + .parent = TYPE_OBJECT, > > > > > > + .instance_size = sizeof(DBusVMState), > > > > > > + .instance_finalize = dbus_vmstate_finalize, > > > > > > + .class_size = sizeof(DBusVMStateClass), > > > > > > + .class_init = dbus_vmstate_class_init, > > > > > > + .interfaces = (InterfaceInfo[]) { > > > > > > + { TYPE_USER_CREATABLE }, > > > > > > + { } > > > > > > + } > > > > > > +}; > > > > > > + > > > > > > +static void > > > > > > +register_types(void) > > > > > > +{ > > > > > > + type_register_static(&dbus_vmstate_info); > > > > > > +} > > > > > > + > > > > > > +type_init(register_types); > > > > > > diff --git a/configure b/configure > > > > > > index 714e7fb6a1..e5b34f5ca7 100755 > > > > > > --- a/configure > > > > > > +++ b/configure > > > > > > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > > > > > > gio=yes > > > > > > gio_cflags=$($pkg_config --cflags gio-2.0) > > > > > > gio_libs=$($pkg_config --libs gio-2.0) > > > > > > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > > > > > > else > > > > > > gio=no > > > > > > fi > > > > > > > > > > > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > > > > > > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > > > > > > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > > > > > > +fi > > > > > > + > > > > > > # Sanity check that the current size_t matches the > > > > > > # size that glib thinks it should be. This catches > > > > > > # problems on multi-arch where people try to build > > > > > > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > > > > > > echo "CONFIG_GIO=y" >> $config_host_mak > > > > > > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > > > > > > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > > > > > > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > > > > > > fi > > > > > > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > > > > > > if test "$gnutls" = "yes" ; then > > > > > > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > > > > > > new file mode 100644 > > > > > > index 0000000000..4a32a183fb > > > > > > --- /dev/null > > > > > > +++ b/docs/interop/dbus-vmstate.rst > > > > > > @@ -0,0 +1,63 @@ > > > > > > +============ > > > > > > +DBus VMState > > > > > > +============ > > > > > > + > > > > > > +Introduction > > > > > > +============ > > > > > > + > > > > > > +Helper processes may have their state migrated with the help of > > > > > > +QEMU "dbus-vmstate" objects. > > > > > > + > > > > > > +At this point, the connection to the helper is done in DBus > > > > > > +peer-to-peer mode (no initial Hello, and no bus name for > > > > > > +communication). The helper must be listening to the given address. > > > > > > + > > > > > > +Helper may save arbitrary data to be transferred in the migration > > > > > > +stream and restored/loaded on destination. > > > > > > + > > > > > > +The data amount to be transferred is limited to 1Mb. The state must be > > > > > > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > > > > > > +reply anyway, and migration would fail if the data isn't given quickly > > > > > > +enough) > > > > > > + > > > > > > +Interface > > > > > > +========= > > > > > > + > > > > > > +On /org/qemu/VMState1 object path: > > > > > > + > > > > > > +.. code:: xml > > > > > > + > > > > > > + <interface name="org.qemu.VMState1"> > > > > > > + <property name="Id" type="s" access="read"/> > > > > > > + <method name="Load"> > > > > > > + <arg type="ay" name="data" direction="in"/> > > > > > > + </method> > > > > > > + <method name="Save"> > > > > > > + <arg type="ay" name="data" direction="out"/> > > > > > > + </method> > > > > > > + </interface> > > > > > > + > > > > > > +"Id" property > > > > > > +------------- > > > > > > + > > > > > > +A utf8 encoded string that identifies the helper uniquely. > > > > > > +Must be <256 bytes. > > > > > > + > > > > > > +Load(in u8[] bytes) method > > > > > > +-------------------------- > > > > > > + > > > > > > +The method called on destination with the state to restore. > > > > > > + > > > > > > +The helper may be initially started in a waiting state (with > > > > > > +an --incoming argument for example), and it may resume on load > > > > > > +success. > > > > > > + > > > > > > +An error may be returned to the caller. > > > > > > + > > > > > > +Save(out u8[] bytes) method > > > > > > +--------------------------- > > > > > > + > > > > > > +The method called on the source to get the current state to be > > > > > > +migrated. The helper should continue to run normally. > > > > > > + > > > > > > +An error may be returned to the caller. > > > > > > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > > > > > > index b4bfcab417..6bb173cfa6 100644 > > > > > > --- a/docs/interop/index.rst > > > > > > +++ b/docs/interop/index.rst > > > > > > @@ -13,6 +13,7 @@ Contents: > > > > > > :maxdepth: 2 > > > > > > > > > > > > bitmaps > > > > > > + dbus-vmstate > > > > > > live-block-operations > > > > > > pr-helper > > > > > > vhost-user > > > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > > > > index fd7fdb8658..2c610086a7 100644 > > > > > > --- a/tests/Makefile.include > > > > > > +++ b/tests/Makefile.include > > > > > > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > > > > > > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > > > > > > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > > > > > > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > > > > > > - > > > > > > +ifneq ($(GDBUS_CODEGEN),) > > > > > > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > > > > > > +endif > > > > > > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > > > > > > check-qtest-i386-y += tests/fdc-test$(EXESUF) > > > > > > check-qtest-i386-y += tests/ide-test$(EXESUF) > > > > > > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > > > > > > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > > > > > > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > > > > > > > > > > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > > > > > > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > > > > > > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > > > > > > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > > > > > > + "GEN","$(@:%-timestamp=%)") > > > > > > + @>$@ > > > > > > + > > > > > > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > > > > > > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > > > > > > + > > > > > > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > > > > > > + > > > > > > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > > > > > > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > > > > > > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > > > > > > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > > > > > > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > > > > > > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > > > > > > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > > > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > > > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > > > > > > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > > > > > > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > > > > > > @@ -1172,6 +1187,7 @@ check-clean: > > > > > > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > > > > > > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > > > > > > rm -f tests/test-qapi-gen-timestamp > > > > > > + rm -f tests/dbus-vmstate1-gen-timestamp > > > > > > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > > > > > > > > > > > clean: check-clean > > > > > > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > > > > > > new file mode 100644 > > > > > > index 0000000000..45d44916d2 > > > > > > --- /dev/null > > > > > > +++ b/tests/dbus-vmstate-test.c > > > > > > @@ -0,0 +1,371 @@ > > > > > > +#include "qemu/osdep.h" > > > > > > +#include <glib/gstdio.h> > > > > > > +#include <gio/gio.h> > > > > > > +#include "libqtest.h" > > > > > > +#include "qemu-common.h" > > > > > > +#include "dbus-vmstate1.h" > > > > > > + > > > > > > +static char *workdir; > > > > > > + > > > > > > +typedef struct TestServerId { > > > > > > + const char *name; > > > > > > + const char *data; > > > > > > + size_t size; > > > > > > +} TestServerId; > > > > > > + > > > > > > +static const TestServerId idA = { > > > > > > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > > > > > > +}; > > > > > > + > > > > > > +static const TestServerId idB = { > > > > > > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > > > > > > +}; > > > > > > + > > > > > > +typedef struct TestServer { > > > > > > + const TestServerId *id; > > > > > > + bool save_called; > > > > > > + bool load_called; > > > > > > + GDBusObjectManagerServer *om; > > > > > > + GDBusServer *server; > > > > > > +} TestServer; > > > > > > + > > > > > > +typedef struct Test { > > > > > > + bool migrate_fail; > > > > > > + TestServer srcA; > > > > > > + TestServer dstA; > > > > > > + TestServer srcB; > > > > > > + TestServer dstB; > > > > > > + GMainLoop *loop, *dbus_loop; > > > > > > + QTestState *src_qemu; > > > > > > +} Test; > > > > > > + > > > > > > +GMutex mutex; > > > > > > +GCond cond; > > > > > > + > > > > > > +static gboolean > > > > > > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > > > > > > + const gchar *arg_data, gpointer user_data) > > > > > > +{ > > > > > > + TestServer *h = user_data; > > > > > > + GVariant *args, *var; > > > > > > + const uint8_t *data; > > > > > > + size_t size; > > > > > > + > > > > > > + args = g_dbus_method_invocation_get_parameters(invocation); > > > > > > + var = g_variant_get_child_value(args, 0); > > > > > > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > > > > > > + g_assert_cmpuint(size, ==, h->id->size); > > > > > > + g_assert(!memcmp(data, h->id->data, h->id->size)); > > > > > > + h->load_called = true; > > > > > > + g_variant_unref(var); > > > > > > + > > > > > > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > > > > > > + return TRUE; > > > > > > +} > > > > > > + > > > > > > +static gboolean > > > > > > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > > > > > > + gpointer user_data) > > > > > > +{ > > > > > > + TestServer *h = user_data; > > > > > > + GVariant *var; > > > > > > + > > > > > > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > > > > > > + h->id->data, h->id->size, sizeof(char)); > > > > > > + g_dbus_method_invocation_return_value(invocation, > > > > > > + g_variant_new("(@ay)", var)); > > > > > > + h->save_called = true; > > > > > > + > > > > > > + return TRUE; > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +connection_closed(GDBusConnection *connection, > > > > > > + gboolean remote_peer_vanished, > > > > > > + GError *Error, > > > > > > + gpointer user_data) > > > > > > +{ > > > > > > + TestServer *h = user_data; > > > > > > + > > > > > > + g_clear_object(&h->om); > > > > > > + g_clear_object(&connection); > > > > > > +} > > > > > > + > > > > > > +static GDBusObjectManagerServer * > > > > > > +get_omserver(GDBusConnection *conn, gpointer user_data) > > > > > > +{ > > > > > > + TestServer *h = user_data; > > > > > > + GDBusObjectManagerServer *om; > > > > > > + GDBusObjectSkeleton *sk; > > > > > > + VMState1 *v; > > > > > > + > > > > > > + om = g_dbus_object_manager_server_new("/org/qemu"); > > > > > > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > > > > > > + > > > > > > + v = vmstate1_skeleton_new(); > > > > > > + g_object_set(v, "id", h->id->name, NULL); > > > > > > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > > > > > > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > > > > > > + > > > > > > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > > > > > > + g_dbus_object_manager_server_export(om, sk); > > > > > > + g_dbus_object_manager_server_set_connection(om, conn); > > > > > > + > > > > > > + g_clear_object(&v); > > > > > > + g_clear_object(&sk); > > > > > > + > > > > > > + return om; > > > > > > +} > > > > > > + > > > > > > +static gboolean > > > > > > +on_new_connection(GDBusServer *server, > > > > > > + GDBusConnection *connection, > > > > > > + gpointer user_data) > > > > > > +{ > > > > > > + TestServer *h = user_data; > > > > > > + > > > > > > + g_object_ref(connection); > > > > > > + g_signal_connect(connection, "closed", > > > > > > + G_CALLBACK(connection_closed), user_data); > > > > > > + h->om = get_omserver(connection, user_data); > > > > > > + > > > > > > + return TRUE; > > > > > > +} > > > > > > + > > > > > > +static gboolean > > > > > > +allow_mechanism_cb(GDBusAuthObserver *observer, > > > > > > + const gchar *mechanism, > > > > > > + gpointer user_data) > > > > > > +{ > > > > > > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > > > > > > +} > > > > > > + > > > > > > +static gboolean > > > > > > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > > > > > > + GIOStream *stream, > > > > > > + GCredentials *credentials, > > > > > > + gpointer user_data) > > > > > > +{ > > > > > > + gboolean authorized = FALSE; > > > > > > + > > > > > > + if (credentials != NULL) { > > > > > > + GCredentials *own_credentials = g_credentials_new(); > > > > > > + > > > > > > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > > > > > > + authorized = TRUE; > > > > > > + } > > > > > > + > > > > > > + g_clear_object(&own_credentials); > > > > > > + } > > > > > > + > > > > > > + return authorized; > > > > > > +} > > > > > > + > > > > > > +static GDBusServer * > > > > > > +server_start(TestServer *h, const char *p, const TestServerId *id) > > > > > > +{ > > > > > > + GDBusAuthObserver *observer = NULL; > > > > > > + GDBusServer *server = NULL; > > > > > > + gchar *guid = NULL; > > > > > > + GError *error = NULL; > > > > > > + char *addr = NULL; > > > > > > + > > > > > > + h->id = id; > > > > > > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > > > > > > + guid = g_dbus_generate_guid(); > > > > > > + observer = g_dbus_auth_observer_new(); > > > > > > + g_signal_connect(observer, "allow-mechanism", > > > > > > + G_CALLBACK(allow_mechanism_cb), h); > > > > > > + g_signal_connect(observer, "authorize-authenticated-peer", > > > > > > + G_CALLBACK(authorize_authenticated_peer_cb), h); > > > > > > + > > > > > > + server = g_dbus_server_new_sync(addr, > > > > > > + G_DBUS_SERVER_FLAGS_NONE, > > > > > > + guid, > > > > > > + observer, > > > > > > + NULL, /* GCancellable */ > > > > > > + &error); > > > > > > + g_dbus_server_start(server); > > > > > > + g_clear_object(&observer); > > > > > > + g_free(guid); > > > > > > + > > > > > > + if (server == NULL) { > > > > > > + g_printerr("Error creating server at address %s: %s\n", > > > > > > + addr, error->message); > > > > > > + g_error_free(error); > > > > > > + return NULL; > > > > > > + } > > > > > > + > > > > > > + g_signal_connect(server, "new-connection", > > > > > > + G_CALLBACK(on_new_connection), h); > > > > > > + > > > > > > + g_free(addr); > > > > > > + return server; > > > > > > +} > > > > > > + > > > > > > + > > > > > > +static gpointer > > > > > > +dbus_thread(gpointer p) > > > > > > +{ > > > > > > + Test *test = p; > > > > > > + GMainContext *context = g_main_context_new(); > > > > > > + GMainLoop *loop = g_main_loop_new(context, FALSE); > > > > > > + > > > > > > + g_main_context_push_thread_default(context); > > > > > > + > > > > > > + g_mutex_lock(&mutex); > > > > > > + test->srcA.server = server_start(&test->srcA, "src", &idA); > > > > > > + test->srcB.server = server_start(&test->srcB, "src", &idB); > > > > > > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > > > > > > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > > > > > > + test->dbus_loop = loop; > > > > > > + g_cond_signal(&cond); > > > > > > + g_mutex_unlock(&mutex); > > > > > > + > > > > > > + g_main_loop_run(loop); > > > > > > + > > > > > > + g_main_loop_unref(loop); > > > > > > + g_main_context_unref(context); > > > > > > + > > > > > > + g_mutex_lock(&mutex); > > > > > > + g_clear_object(&test->srcA.server); > > > > > > + g_clear_object(&test->srcB.server); > > > > > > + g_clear_object(&test->dstA.server); > > > > > > + g_clear_object(&test->dstB.server); > > > > > > + g_mutex_unlock(&mutex); > > > > > > + > > > > > > + return NULL; > > > > > > +} > > > > > > + > > > > > > +static gboolean > > > > > > +wait_for_migration_complete(gpointer user_data) > > > > > > +{ > > > > > > + Test *test = user_data; > > > > > > + QDict *rsp_return; > > > > > > + bool stop = false; > > > > > > + const char *status; > > > > > > + > > > > > > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > > > > > > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > > > > > > + status = qdict_get_str(rsp_return, "status"); > > > > > > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > > > > > > + stop = true; > > > > > > + g_assert_cmpstr(status, ==, > > > > > > + test->migrate_fail ? "failed" : "completed"); > > > > > > + } > > > > > > + qobject_unref(rsp_return); > > > > > > + > > > > > > + if (stop) { > > > > > > + g_main_loop_quit(test->loop); > > > > > > + } > > > > > > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > > > > > > +} > > > > > > + > > > > > > +static void migrate(QTestState *who, const char *uri) > > > > > > +{ > > > > > > + QDict *args, *rsp; > > > > > > + > > > > > > + args = qdict_new(); > > > > > > + qdict_put_str(args, "uri", uri); > > > > > > + > > > > > > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > > > > > > + > > > > > > + g_assert(qdict_haskey(rsp, "return")); > > > > > > + qobject_unref(rsp); > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +test_dbus_vmstate(Test *test) > > > > > > +{ > > > > > > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > > > > > > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > > > > > > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > > > > > > + GThread *t = g_thread_new("dbus", dbus_thread, test); > > > > > > + > > > > > > + g_mutex_lock(&mutex); > > > > > > + while (!test->dbus_loop) { > > > > > > + g_cond_wait(&cond, &mutex); > > > > > > + } > > > > > > + > > > > > > + src_qemu_args = > > > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > > > + "-object dbus-vmstate,id=dvB,addr=%s", > > > > > > + g_dbus_server_get_client_address(test->srcA.server), > > > > > > + g_dbus_server_get_client_address(test->srcB.server)); > > > > > > + > > > > > > + > > > > > > + dst_qemu_args = > > > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > > > + "-object dbus-vmstate,id=dvB,addr=%s " > > > > > > + "-incoming %s", > > > > > > + g_dbus_server_get_client_address(test->dstA.server), > > > > > > + g_dbus_server_get_client_address(test->dstB.server), > > > > > > + uri); > > > > > > + > > > > > > + src_qemu = qtest_init(src_qemu_args); > > > > > > + dst_qemu = qtest_init(dst_qemu_args); > > > > > > + > > > > > > + test->loop = g_main_loop_new(NULL, TRUE); > > > > > > + > > > > > > + migrate(src_qemu, uri); > > > > > > + test->src_qemu = src_qemu; > > > > > > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > > > > > > + > > > > > > + g_main_loop_run(test->loop); > > > > > > + g_main_loop_unref(test->loop); > > > > > > + > > > > > > + g_free(uri); > > > > > > + qtest_quit(dst_qemu); > > > > > > + qtest_quit(src_qemu); > > > > > > + g_free(dst_qemu_args); > > > > > > + g_free(src_qemu_args); > > > > > > + > > > > > > + g_main_loop_quit(test->dbus_loop); > > > > > > + g_mutex_unlock(&mutex); > > > > > > + > > > > > > + g_thread_join(t); > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +check_migrated(TestServer *s, TestServer *d) > > > > > > +{ > > > > > > + assert(s->save_called); > > > > > > + assert(!s->load_called); > > > > > > + assert(!d->save_called); > > > > > > + assert(d->load_called); > > > > > > +} > > > > > > + > > > > > > +static void > > > > > > +test_dbus_vmstate_migrate(void) > > > > > > +{ > > > > > > + Test test = { }; > > > > > > + > > > > > > + test_dbus_vmstate(&test); > > > > > > + > > > > > > + check_migrated(&test.srcA, &test.dstA); > > > > > > + check_migrated(&test.srcB, &test.dstB); > > > > > > +} > > > > > > + > > > > > > +int > > > > > > +main(int argc, char **argv) > > > > > > +{ > > > > > > + GError *err = NULL; > > > > > > + int ret; > > > > > > + > > > > > > + g_test_init(&argc, &argv, NULL); > > > > > > + > > > > > > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > > > > > > + if (!workdir) { > > > > > > + g_error("Unable to create temporary dir: %s\n", err->message); > > > > > > + } > > > > > > + > > > > > > + qtest_add_func("/dbus-vmstate/migrate", > > > > > > + test_dbus_vmstate_migrate); > > > > > > + > > > > > > + ret = g_test_run(); > > > > > > + > > > > > > + rmdir(workdir); > > > > > > + g_free(workdir); > > > > > > + > > > > > > + return ret; > > > > > > +} > > > > > > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > > > > > > new file mode 100644 > > > > > > index 0000000000..cc8563be4c > > > > > > --- /dev/null > > > > > > +++ b/tests/dbus-vmstate1.xml > > > > > > @@ -0,0 +1,12 @@ > > > > > > +<?xml version="1.0"?> > > > > > > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > > > > > > + <interface name="org.qemu.VMState1"> > > > > > > + <property name="Id" type="s" access="read"/> > > > > > > + <method name="Load"> > > > > > > + <arg type="ay" name="data" direction="in"/> > > > > > > + </method> > > > > > > + <method name="Save"> > > > > > > + <arg type="ay" name="data" direction="out"/> > > > > > > + </method> > > > > > > + </interface> > > > > > > +</node> > > > > > > -- > > > > > > 2.23.0.rc1 > > > > > > > > > > > -- > > > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > -- > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > -- Marc-André Lureau ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object 2019-08-22 12:38 ` Marc-André Lureau @ 2019-08-22 12:51 ` Dr. David Alan Gilbert 0 siblings, 0 replies; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-22 12:51 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Paolo Bonzini, Thomas Huth, qemu-devel, Juan Quintela * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > Hi > > On Thu, Aug 22, 2019 at 4:20 PM Dr. David Alan Gilbert > <dgilbert@redhat.com> wrote: > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > Hi > > > > > > On Thu, Aug 22, 2019 at 3:41 PM Dr. David Alan Gilbert > > > <dgilbert@redhat.com> wrote: > > > > > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > > > Hi > > > > > > > > > > On Thu, Aug 22, 2019 at 2:56 PM Dr. David Alan Gilbert > > > > > <dgilbert@redhat.com> wrote: > > > > > > > > > > > > * Marc-André Lureau (marcandre.lureau@redhat.com) wrote: > > > > > > > When instanciated, this object will connect to the given D-Bus > > > > > > > bus. During migration, it will take the data from org.qemu.VMState1 > > > > > > > instances. > > > > > > > > > > > > > > See documentation for further details. > > > > > > > > > > > > I'll leave the main review to someone who understands the details of > > > > > > DBUS, however from the migration side: > > > > > > > > > > > > > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> > > > > > > > > > > > > <snip> > > > > > > > > > > > > > + > > > > > > > +static int > > > > > > > +dbus_load_state(QEMUFile *f, void *opaque, int version_id) > > > > > > > +{ > > > > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > > > > + uint8_t *data = NULL; > > > > > > > + int ret = -1; > > > > > > > + char id[256]; > > > > > > > + unsigned int size; > > > > > > > > > > > > Higher level question; do you actually need a separate ID, or does > > > > > > the ID that's stored by the migration code in the section name > > > > > > give you enough uniqueness? > > > > > > > > > > > > > > > The ID stored in migration section name is based on the handler ID: > > > > > TYPE_DBUS_VMSTATE "-" handler_id > > > > > > > > > > By design, handler_id must be unique (and documented as such). > > > > > > > > > > Do you see a problem with that approach? > > > > > > > > OK, so if that's unique, why do you need an 'id' stored here? > > > > > > Oh I see, that's mostly a relic of v1, where multiple ID > > > "sub-sections" would exist for each helper. > > > > > > However, if we want to allow having several "sub-sections" again, that > > > might come handy to keep compatibility. It may be better to have a > > > simple version field for that instead. > > > > OK, that's fine > > > > > > > > > > > > > > > > > > > + if (qemu_get_counted_string(f, id) == 0) { > > > > > > > + error_report("Invalid vmstate Id"); > > > > > > > + goto end; > > > > > > > > > > > > I generally prefer to include something telling me where the error has > > > > > > come from, just to make it easier to track down if I see the error in > > > > > > a log; e.g. use __func__ > > > > > > > > > > ok > > > > > > > > > > > > > > > > > > + } > > > > > > > + > > > > > > > + if (g_strcmp0(id, self->id)) { > > > > > > > + error_report("Invalid vmstate Id: %s != %s", id, self->id); > > > > > > > + goto end; > > > > > > > + } > > > > > > > + > > > > > > > + size = qemu_get_be32(f); > > > > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > > > > + error_report("Invalid vmstate size: %u", size); > > > > > > > + goto end; > > > > > > > + } > > > > > > > + > > > > > > > + data = g_malloc(size); > > > > > > > + if (qemu_get_buffer(f, data, size) != size) { > > > > > > > + error_report("Failed to read %u bytes", size); > > > > > > > + goto end; > > > > > > > + } > > > > > > > + > > > > > > > + if (dbus_load_state_proxy(self->proxy, data, size) < 0) { > > > > > > > + error_report("Failed to restore Id '%s'", id); > > > > > > > + goto end; > > > > > > > + } > > > > > > > + > > > > > > > + ret = 0; > > > > > > > + > > > > > > > +end: > > > > > > > + g_clear_pointer(&data, g_free); > > > > > > > + return ret; > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +dbus_save_state(QEMUFile *f, void *opaque) > > > > > > > +{ > > > > > > > + DBusVMState *self = DBUS_VMSTATE(opaque); > > > > > > > + GVariant *result = NULL, *child = NULL; > > > > > > > + const uint8_t *data; > > > > > > > + size_t size; > > > > > > > + GError *err = NULL; > > > > > > > + > > > > > > > + result = g_dbus_proxy_call_sync(self->proxy, "Save", > > > > > > > + NULL, G_DBUS_CALL_FLAGS_NO_AUTO_START, > > > > > > > + -1, NULL, &err); > > > > > > > + if (!result) { > > > > > > > + error_report("Failed to Save: %s", err->message); > > > > > > > + g_clear_error(&err); > > > > > > > + goto end; > > > > > > > + } > > > > > > > + > > > > > > > + child = g_variant_get_child_value(result, 0); > > > > > > > + data = g_variant_get_fixed_array(child, &size, sizeof(char)); > > > > > > > + if (!data) { > > > > > > > + error_report("Failed to Save: not a byte array"); > > > > > > > + goto end; > > > > > > > + } > > > > > > > + if (size > DBUS_VMSTATE_SIZE_LIMIT) { > > > > > > > + error_report("Too much vmstate data to save: %zu", size); > > > > > > > + goto end; > > > > > > > + } > > > > > > > + > > > > > > > + qemu_put_counted_string(f, self->id); > > > > > > > + qemu_put_be32(f, size); > > > > > > > + qemu_put_buffer(f, data, size); > > > > > > > + > > > > > > > +end: > > > > > > > + g_clear_pointer(&child, g_variant_unref); > > > > > > > + g_clear_pointer(&result, g_variant_unref); > > > > > > > +} > > > > > > > + > > > > > > > +static const SaveVMHandlers savevm_handlers = { > > > > > > > + .save_state = dbus_save_state, > > > > > > > + .load_state = dbus_load_state, > > > > > > > +}; > > > > > > > + > > > > > > > +static void > > > > > > > +dbus_vmstate_complete(UserCreatable *uc, Error **errp) > > > > > > > +{ > > > > > > > + DBusVMState *self = DBUS_VMSTATE(uc); > > > > > > > + GError *err = NULL; > > > > > > > + GDBusConnection *bus = NULL; > > > > > > > + GDBusProxy *proxy = NULL; > > > > > > > + char *idstr = NULL; > > > > > > > + > > > > > > > + if (!self->dbus_addr) { > > > > > > > + error_setg(errp, QERR_MISSING_PARAMETER, "addr"); > > > > > > > + return; > > > > > > > + } > > > > > > > + > > > > > > > + bus = g_dbus_connection_new_for_address_sync(self->dbus_addr, > > > > > > > + G_DBUS_CONNECTION_FLAGS_AUTHENTICATION_CLIENT, > > > > > > > + NULL, NULL, &err); > > > > > > > + if (err) { > > > > > > > + error_setg(errp, "failed to connect to DBus: '%s'", err->message); > > > > > > > + g_clear_error(&err); > > > > > > > + return; > > > > > > > + } > > > > > > > + > > > > > > > + self->bus = bus; > > > > > > > + > > > > > > > + proxy = g_dbus_proxy_new_sync(bus, > > > > > > > + G_DBUS_PROXY_FLAGS_NONE, > > > > > > > + (GDBusInterfaceInfo *) &vmstate1_interface_info, > > > > > > > + NULL, > > > > > > > + "/org/qemu/VMState1", > > > > > > > + "org.qemu.VMState1", > > > > > > > + NULL, &err); > > > > > > > + > > > > > > > + if (err) { > > > > > > > + error_setg(errp, "failed to create DBus proxy: '%s'", err->message); > > > > > > > + g_clear_error(&err); > > > > > > > + return; > > > > > > > + } > > > > > > > + > > > > > > > + self->proxy = proxy; > > > > > > > + > > > > > > > + self->id = dbus_proxy_get_id(proxy, &err); > > > > > > > + if (!self->id) { > > > > > > > + error_setg(errp, "failed to get DBus Id: '%s'", err->message); > > > > > > > + g_clear_error(&err); > > > > > > > + return; > > > > > > > + } > > > > > > > + > > > > > > > + idstr = get_idstr(self); > > > > > > > + if (register_savevm_live(NULL, idstr, 0, 0, > > > > > > > + &savevm_handlers, self) < 0) { > > > > > > > + error_setg(errp, "Failed to register savevm handler"); > > > > > > > + } > > > > > > > > > > > > Can you try and avoid register_savevm_live if possible and just wire a > > > > > > vmsd onto the class like most other devices? We don't have many > > > > > > register_savevm_live calls, and I'd like to get them down to just the > > > > > > iterative devices. > > > > > > > > > > Sure if I could, but it's not a device. > > > > > > > > > > Perhaps we could have a qom interface for vmsd users? I can try that eventually. > > > > > > > > Hmm, why isn't it a device? > > > > > > It doesn't need anything from Device (which is heavyweight), it is a > > > simple user-creatable object that holds a dbus connection and reacts > > > on migration events. > > > > > > If you take slirp helper process as an example, it's a "service" for a > > > VM. Not a device. You could have the same helper for various devices > > > etc, or more helpers for the same device. > > > > I think you're better calling this a device; the slirp world doesn't fit > > nicely either and it would be better if it was a device as well. > > > > The goal here is to migrate helper processes along with qemu. > > Helper processes may be things like slirp, network services, emulators > (such as swtpm, which isn't a device either), or long-running > jobs/process done on behalf of qemu, for example a AV compression, > spice server... So I think either fix qom to wire vmstate up, or just wear some special glasses and see that they're really devices. Dave > > Dave > > > > > > > > > > > > > > Dave > > > > > > > > > thanks > > > > > > > > > > > Dave > > > > > > > > > > > > > > > > > > > + g_free(idstr); > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +dbus_vmstate_finalize(Object *o) > > > > > > > +{ > > > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > > > + char *idstr = get_idstr(self); > > > > > > > + > > > > > > > + unregister_savevm(NULL, idstr, self); > > > > > > > + > > > > > > > + g_clear_object(&self->bus); > > > > > > > + g_clear_object(&self->proxy); > > > > > > > + g_free(self->dbus_addr); > > > > > > > + g_free(self->id); > > > > > > > + g_free(idstr); > > > > > > > +} > > > > > > > + > > > > > > > +static char * > > > > > > > +get_dbus_addr(Object *o, Error **errp) > > > > > > > +{ > > > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > > > + > > > > > > > + return g_strdup(self->dbus_addr); > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +set_dbus_addr(Object *o, const char *str, Error **errp) > > > > > > > +{ > > > > > > > + DBusVMState *self = DBUS_VMSTATE(o); > > > > > > > + > > > > > > > + g_free(self->dbus_addr); > > > > > > > + self->dbus_addr = g_strdup(str); > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +dbus_vmstate_class_init(ObjectClass *oc, void *data) > > > > > > > +{ > > > > > > > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > > > > > > > + > > > > > > > + ucc->complete = dbus_vmstate_complete; > > > > > > > + > > > > > > > + object_class_property_add_str(oc, "addr", > > > > > > > + get_dbus_addr, set_dbus_addr, > > > > > > > + &error_abort); > > > > > > > +} > > > > > > > + > > > > > > > +static const TypeInfo dbus_vmstate_info = { > > > > > > > + .name = TYPE_DBUS_VMSTATE, > > > > > > > + .parent = TYPE_OBJECT, > > > > > > > + .instance_size = sizeof(DBusVMState), > > > > > > > + .instance_finalize = dbus_vmstate_finalize, > > > > > > > + .class_size = sizeof(DBusVMStateClass), > > > > > > > + .class_init = dbus_vmstate_class_init, > > > > > > > + .interfaces = (InterfaceInfo[]) { > > > > > > > + { TYPE_USER_CREATABLE }, > > > > > > > + { } > > > > > > > + } > > > > > > > +}; > > > > > > > + > > > > > > > +static void > > > > > > > +register_types(void) > > > > > > > +{ > > > > > > > + type_register_static(&dbus_vmstate_info); > > > > > > > +} > > > > > > > + > > > > > > > +type_init(register_types); > > > > > > > diff --git a/configure b/configure > > > > > > > index 714e7fb6a1..e5b34f5ca7 100755 > > > > > > > --- a/configure > > > > > > > +++ b/configure > > > > > > > @@ -3665,10 +3665,16 @@ if $pkg_config --atleast-version=$glib_req_ver gio-2.0; then > > > > > > > gio=yes > > > > > > > gio_cflags=$($pkg_config --cflags gio-2.0) > > > > > > > gio_libs=$($pkg_config --libs gio-2.0) > > > > > > > + gdbus_codegen=$($pkg_config --variable=gdbus_codegen gio-2.0) > > > > > > > else > > > > > > > gio=no > > > > > > > fi > > > > > > > > > > > > > > +if $pkg_config --atleast-version=$glib_req_ver gio-unix-2.0; then > > > > > > > + gio_cflags="$gio_cflags $($pkg_config --cflags gio-unix-2.0)" > > > > > > > + gio_libs="$gio_libs $($pkg_config --libs gio-unix-2.0)" > > > > > > > +fi > > > > > > > + > > > > > > > # Sanity check that the current size_t matches the > > > > > > > # size that glib thinks it should be. This catches > > > > > > > # problems on multi-arch where people try to build > > > > > > > @@ -6830,6 +6836,7 @@ if test "$gio" = "yes" ; then > > > > > > > echo "CONFIG_GIO=y" >> $config_host_mak > > > > > > > echo "GIO_CFLAGS=$gio_cflags" >> $config_host_mak > > > > > > > echo "GIO_LIBS=$gio_libs" >> $config_host_mak > > > > > > > + echo "GDBUS_CODEGEN=$gdbus_codegen" >> $config_host_mak > > > > > > > fi > > > > > > > echo "CONFIG_TLS_PRIORITY=\"$tls_priority\"" >> $config_host_mak > > > > > > > if test "$gnutls" = "yes" ; then > > > > > > > diff --git a/docs/interop/dbus-vmstate.rst b/docs/interop/dbus-vmstate.rst > > > > > > > new file mode 100644 > > > > > > > index 0000000000..4a32a183fb > > > > > > > --- /dev/null > > > > > > > +++ b/docs/interop/dbus-vmstate.rst > > > > > > > @@ -0,0 +1,63 @@ > > > > > > > +============ > > > > > > > +DBus VMState > > > > > > > +============ > > > > > > > + > > > > > > > +Introduction > > > > > > > +============ > > > > > > > + > > > > > > > +Helper processes may have their state migrated with the help of > > > > > > > +QEMU "dbus-vmstate" objects. > > > > > > > + > > > > > > > +At this point, the connection to the helper is done in DBus > > > > > > > +peer-to-peer mode (no initial Hello, and no bus name for > > > > > > > +communication). The helper must be listening to the given address. > > > > > > > + > > > > > > > +Helper may save arbitrary data to be transferred in the migration > > > > > > > +stream and restored/loaded on destination. > > > > > > > + > > > > > > > +The data amount to be transferred is limited to 1Mb. The state must be > > > > > > > +saved quickly (a few seconds maximum). (DBus imposes a time limit on > > > > > > > +reply anyway, and migration would fail if the data isn't given quickly > > > > > > > +enough) > > > > > > > + > > > > > > > +Interface > > > > > > > +========= > > > > > > > + > > > > > > > +On /org/qemu/VMState1 object path: > > > > > > > + > > > > > > > +.. code:: xml > > > > > > > + > > > > > > > + <interface name="org.qemu.VMState1"> > > > > > > > + <property name="Id" type="s" access="read"/> > > > > > > > + <method name="Load"> > > > > > > > + <arg type="ay" name="data" direction="in"/> > > > > > > > + </method> > > > > > > > + <method name="Save"> > > > > > > > + <arg type="ay" name="data" direction="out"/> > > > > > > > + </method> > > > > > > > + </interface> > > > > > > > + > > > > > > > +"Id" property > > > > > > > +------------- > > > > > > > + > > > > > > > +A utf8 encoded string that identifies the helper uniquely. > > > > > > > +Must be <256 bytes. > > > > > > > + > > > > > > > +Load(in u8[] bytes) method > > > > > > > +-------------------------- > > > > > > > + > > > > > > > +The method called on destination with the state to restore. > > > > > > > + > > > > > > > +The helper may be initially started in a waiting state (with > > > > > > > +an --incoming argument for example), and it may resume on load > > > > > > > +success. > > > > > > > + > > > > > > > +An error may be returned to the caller. > > > > > > > + > > > > > > > +Save(out u8[] bytes) method > > > > > > > +--------------------------- > > > > > > > + > > > > > > > +The method called on the source to get the current state to be > > > > > > > +migrated. The helper should continue to run normally. > > > > > > > + > > > > > > > +An error may be returned to the caller. > > > > > > > diff --git a/docs/interop/index.rst b/docs/interop/index.rst > > > > > > > index b4bfcab417..6bb173cfa6 100644 > > > > > > > --- a/docs/interop/index.rst > > > > > > > +++ b/docs/interop/index.rst > > > > > > > @@ -13,6 +13,7 @@ Contents: > > > > > > > :maxdepth: 2 > > > > > > > > > > > > > > bitmaps > > > > > > > + dbus-vmstate > > > > > > > live-block-operations > > > > > > > pr-helper > > > > > > > vhost-user > > > > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > > > > > > index fd7fdb8658..2c610086a7 100644 > > > > > > > --- a/tests/Makefile.include > > > > > > > +++ b/tests/Makefile.include > > > > > > > @@ -157,7 +157,9 @@ check-qtest-pci-$(CONFIG_RTL8139_PCI) += tests/rtl8139-test$(EXESUF) > > > > > > > check-qtest-pci-$(CONFIG_VGA) += tests/display-vga-test$(EXESUF) > > > > > > > check-qtest-pci-$(CONFIG_HDA) += tests/intel-hda-test$(EXESUF) > > > > > > > check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) > > > > > > > - > > > > > > > +ifneq ($(GDBUS_CODEGEN),) > > > > > > > +check-qtest-pci-$(CONFIG_GIO) += tests/dbus-vmstate-test$(EXESUF) > > > > > > > +endif > > > > > > > check-qtest-i386-$(CONFIG_ISA_TESTDEV) = tests/endianness-test$(EXESUF) > > > > > > > check-qtest-i386-y += tests/fdc-test$(EXESUF) > > > > > > > check-qtest-i386-y += tests/ide-test$(EXESUF) > > > > > > > @@ -618,6 +620,18 @@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.jso > > > > > > > @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ > > > > > > > @rm -f tests/qapi-schema/doc-good-qapi-*.[ch] tests/qapi-schema/doc-good-qmp-*.[ch] > > > > > > > > > > > > > > +tests/dbus-vmstate1.h tests/dbus-vmstate1.c: tests/dbus-vmstate1-gen-timestamp ; > > > > > > > +tests/dbus-vmstate1-gen-timestamp: $(SRC_PATH)/tests/dbus-vmstate1.xml > > > > > > > + $(call quiet-command,$(GDBUS_CODEGEN) $< \ > > > > > > > + --interface-prefix org.qemu --generate-c-code tests/dbus-vmstate1, \ > > > > > > > + "GEN","$(@:%-timestamp=%)") > > > > > > > + @>$@ > > > > > > > + > > > > > > > +tests/dbus-vmstate1.o-cflags := $(GIO_CFLAGS) > > > > > > > +tests/dbus-vmstate1.o-libs := $(GIO_LIBS) > > > > > > > + > > > > > > > +tests/dbus-vmstate-test.o: tests/dbus-vmstate1.h > > > > > > > + > > > > > > > tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) > > > > > > > tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) > > > > > > > tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qapi-events.o > > > > > > > @@ -820,6 +834,7 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) > > > > > > > tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) > > > > > > > tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) > > > > > > > tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > > > > +tests/dbus-vmstate-test$(EXESUF): tests/dbus-vmstate-test.o tests/dbus-vmstate1.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) > > > > > > > tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a > > > > > > > tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) > > > > > > > tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o > > > > > > > @@ -1172,6 +1187,7 @@ check-clean: > > > > > > > rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) > > > > > > > rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) > > > > > > > rm -f tests/test-qapi-gen-timestamp > > > > > > > + rm -f tests/dbus-vmstate1-gen-timestamp > > > > > > > rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) > > > > > > > > > > > > > > clean: check-clean > > > > > > > diff --git a/tests/dbus-vmstate-test.c b/tests/dbus-vmstate-test.c > > > > > > > new file mode 100644 > > > > > > > index 0000000000..45d44916d2 > > > > > > > --- /dev/null > > > > > > > +++ b/tests/dbus-vmstate-test.c > > > > > > > @@ -0,0 +1,371 @@ > > > > > > > +#include "qemu/osdep.h" > > > > > > > +#include <glib/gstdio.h> > > > > > > > +#include <gio/gio.h> > > > > > > > +#include "libqtest.h" > > > > > > > +#include "qemu-common.h" > > > > > > > +#include "dbus-vmstate1.h" > > > > > > > + > > > > > > > +static char *workdir; > > > > > > > + > > > > > > > +typedef struct TestServerId { > > > > > > > + const char *name; > > > > > > > + const char *data; > > > > > > > + size_t size; > > > > > > > +} TestServerId; > > > > > > > + > > > > > > > +static const TestServerId idA = { > > > > > > > + "idA", "I'am\0idA!", sizeof("I'am\0idA!") > > > > > > > +}; > > > > > > > + > > > > > > > +static const TestServerId idB = { > > > > > > > + "idB", "I'am\0idB!", sizeof("I'am\0idB!") > > > > > > > +}; > > > > > > > + > > > > > > > +typedef struct TestServer { > > > > > > > + const TestServerId *id; > > > > > > > + bool save_called; > > > > > > > + bool load_called; > > > > > > > + GDBusObjectManagerServer *om; > > > > > > > + GDBusServer *server; > > > > > > > +} TestServer; > > > > > > > + > > > > > > > +typedef struct Test { > > > > > > > + bool migrate_fail; > > > > > > > + TestServer srcA; > > > > > > > + TestServer dstA; > > > > > > > + TestServer srcB; > > > > > > > + TestServer dstB; > > > > > > > + GMainLoop *loop, *dbus_loop; > > > > > > > + QTestState *src_qemu; > > > > > > > +} Test; > > > > > > > + > > > > > > > +GMutex mutex; > > > > > > > +GCond cond; > > > > > > > + > > > > > > > +static gboolean > > > > > > > +vmstate_load(VMState1 *object, GDBusMethodInvocation *invocation, > > > > > > > + const gchar *arg_data, gpointer user_data) > > > > > > > +{ > > > > > > > + TestServer *h = user_data; > > > > > > > + GVariant *args, *var; > > > > > > > + const uint8_t *data; > > > > > > > + size_t size; > > > > > > > + > > > > > > > + args = g_dbus_method_invocation_get_parameters(invocation); > > > > > > > + var = g_variant_get_child_value(args, 0); > > > > > > > + data = g_variant_get_fixed_array(var, &size, sizeof(char)); > > > > > > > + g_assert_cmpuint(size, ==, h->id->size); > > > > > > > + g_assert(!memcmp(data, h->id->data, h->id->size)); > > > > > > > + h->load_called = true; > > > > > > > + g_variant_unref(var); > > > > > > > + > > > > > > > + g_dbus_method_invocation_return_value(invocation, g_variant_new("()")); > > > > > > > + return TRUE; > > > > > > > +} > > > > > > > + > > > > > > > +static gboolean > > > > > > > +vmstate_save(VMState1 *object, GDBusMethodInvocation *invocation, > > > > > > > + gpointer user_data) > > > > > > > +{ > > > > > > > + TestServer *h = user_data; > > > > > > > + GVariant *var; > > > > > > > + > > > > > > > + var = g_variant_new_fixed_array(G_VARIANT_TYPE_BYTE, > > > > > > > + h->id->data, h->id->size, sizeof(char)); > > > > > > > + g_dbus_method_invocation_return_value(invocation, > > > > > > > + g_variant_new("(@ay)", var)); > > > > > > > + h->save_called = true; > > > > > > > + > > > > > > > + return TRUE; > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +connection_closed(GDBusConnection *connection, > > > > > > > + gboolean remote_peer_vanished, > > > > > > > + GError *Error, > > > > > > > + gpointer user_data) > > > > > > > +{ > > > > > > > + TestServer *h = user_data; > > > > > > > + > > > > > > > + g_clear_object(&h->om); > > > > > > > + g_clear_object(&connection); > > > > > > > +} > > > > > > > + > > > > > > > +static GDBusObjectManagerServer * > > > > > > > +get_omserver(GDBusConnection *conn, gpointer user_data) > > > > > > > +{ > > > > > > > + TestServer *h = user_data; > > > > > > > + GDBusObjectManagerServer *om; > > > > > > > + GDBusObjectSkeleton *sk; > > > > > > > + VMState1 *v; > > > > > > > + > > > > > > > + om = g_dbus_object_manager_server_new("/org/qemu"); > > > > > > > + sk = g_dbus_object_skeleton_new("/org/qemu/VMState1"); > > > > > > > + > > > > > > > + v = vmstate1_skeleton_new(); > > > > > > > + g_object_set(v, "id", h->id->name, NULL); > > > > > > > + g_signal_connect(v, "handle-load", G_CALLBACK(vmstate_load), user_data); > > > > > > > + g_signal_connect(v, "handle-save", G_CALLBACK(vmstate_save), user_data); > > > > > > > + > > > > > > > + g_dbus_object_skeleton_add_interface(sk, G_DBUS_INTERFACE_SKELETON(v)); > > > > > > > + g_dbus_object_manager_server_export(om, sk); > > > > > > > + g_dbus_object_manager_server_set_connection(om, conn); > > > > > > > + > > > > > > > + g_clear_object(&v); > > > > > > > + g_clear_object(&sk); > > > > > > > + > > > > > > > + return om; > > > > > > > +} > > > > > > > + > > > > > > > +static gboolean > > > > > > > +on_new_connection(GDBusServer *server, > > > > > > > + GDBusConnection *connection, > > > > > > > + gpointer user_data) > > > > > > > +{ > > > > > > > + TestServer *h = user_data; > > > > > > > + > > > > > > > + g_object_ref(connection); > > > > > > > + g_signal_connect(connection, "closed", > > > > > > > + G_CALLBACK(connection_closed), user_data); > > > > > > > + h->om = get_omserver(connection, user_data); > > > > > > > + > > > > > > > + return TRUE; > > > > > > > +} > > > > > > > + > > > > > > > +static gboolean > > > > > > > +allow_mechanism_cb(GDBusAuthObserver *observer, > > > > > > > + const gchar *mechanism, > > > > > > > + gpointer user_data) > > > > > > > +{ > > > > > > > + return g_strcmp0(mechanism, "EXTERNAL") == 0; > > > > > > > +} > > > > > > > + > > > > > > > +static gboolean > > > > > > > +authorize_authenticated_peer_cb(GDBusAuthObserver *observer, > > > > > > > + GIOStream *stream, > > > > > > > + GCredentials *credentials, > > > > > > > + gpointer user_data) > > > > > > > +{ > > > > > > > + gboolean authorized = FALSE; > > > > > > > + > > > > > > > + if (credentials != NULL) { > > > > > > > + GCredentials *own_credentials = g_credentials_new(); > > > > > > > + > > > > > > > + if (g_credentials_is_same_user(credentials, own_credentials, NULL)) { > > > > > > > + authorized = TRUE; > > > > > > > + } > > > > > > > + > > > > > > > + g_clear_object(&own_credentials); > > > > > > > + } > > > > > > > + > > > > > > > + return authorized; > > > > > > > +} > > > > > > > + > > > > > > > +static GDBusServer * > > > > > > > +server_start(TestServer *h, const char *p, const TestServerId *id) > > > > > > > +{ > > > > > > > + GDBusAuthObserver *observer = NULL; > > > > > > > + GDBusServer *server = NULL; > > > > > > > + gchar *guid = NULL; > > > > > > > + GError *error = NULL; > > > > > > > + char *addr = NULL; > > > > > > > + > > > > > > > + h->id = id; > > > > > > > + addr = g_strdup_printf("unix:path=%s/dbus-%s%s", workdir, p, h->id->name); > > > > > > > + guid = g_dbus_generate_guid(); > > > > > > > + observer = g_dbus_auth_observer_new(); > > > > > > > + g_signal_connect(observer, "allow-mechanism", > > > > > > > + G_CALLBACK(allow_mechanism_cb), h); > > > > > > > + g_signal_connect(observer, "authorize-authenticated-peer", > > > > > > > + G_CALLBACK(authorize_authenticated_peer_cb), h); > > > > > > > + > > > > > > > + server = g_dbus_server_new_sync(addr, > > > > > > > + G_DBUS_SERVER_FLAGS_NONE, > > > > > > > + guid, > > > > > > > + observer, > > > > > > > + NULL, /* GCancellable */ > > > > > > > + &error); > > > > > > > + g_dbus_server_start(server); > > > > > > > + g_clear_object(&observer); > > > > > > > + g_free(guid); > > > > > > > + > > > > > > > + if (server == NULL) { > > > > > > > + g_printerr("Error creating server at address %s: %s\n", > > > > > > > + addr, error->message); > > > > > > > + g_error_free(error); > > > > > > > + return NULL; > > > > > > > + } > > > > > > > + > > > > > > > + g_signal_connect(server, "new-connection", > > > > > > > + G_CALLBACK(on_new_connection), h); > > > > > > > + > > > > > > > + g_free(addr); > > > > > > > + return server; > > > > > > > +} > > > > > > > + > > > > > > > + > > > > > > > +static gpointer > > > > > > > +dbus_thread(gpointer p) > > > > > > > +{ > > > > > > > + Test *test = p; > > > > > > > + GMainContext *context = g_main_context_new(); > > > > > > > + GMainLoop *loop = g_main_loop_new(context, FALSE); > > > > > > > + > > > > > > > + g_main_context_push_thread_default(context); > > > > > > > + > > > > > > > + g_mutex_lock(&mutex); > > > > > > > + test->srcA.server = server_start(&test->srcA, "src", &idA); > > > > > > > + test->srcB.server = server_start(&test->srcB, "src", &idB); > > > > > > > + test->dstA.server = server_start(&test->dstA, "dst", &idA); > > > > > > > + test->dstB.server = server_start(&test->dstB, "dst", &idB); > > > > > > > + test->dbus_loop = loop; > > > > > > > + g_cond_signal(&cond); > > > > > > > + g_mutex_unlock(&mutex); > > > > > > > + > > > > > > > + g_main_loop_run(loop); > > > > > > > + > > > > > > > + g_main_loop_unref(loop); > > > > > > > + g_main_context_unref(context); > > > > > > > + > > > > > > > + g_mutex_lock(&mutex); > > > > > > > + g_clear_object(&test->srcA.server); > > > > > > > + g_clear_object(&test->srcB.server); > > > > > > > + g_clear_object(&test->dstA.server); > > > > > > > + g_clear_object(&test->dstB.server); > > > > > > > + g_mutex_unlock(&mutex); > > > > > > > + > > > > > > > + return NULL; > > > > > > > +} > > > > > > > + > > > > > > > +static gboolean > > > > > > > +wait_for_migration_complete(gpointer user_data) > > > > > > > +{ > > > > > > > + Test *test = user_data; > > > > > > > + QDict *rsp_return; > > > > > > > + bool stop = false; > > > > > > > + const char *status; > > > > > > > + > > > > > > > + qtest_qmp_send(test->src_qemu, "{ 'execute': 'query-migrate' }"); > > > > > > > + rsp_return = qtest_qmp_receive_success(test->src_qemu, NULL, NULL); > > > > > > > + status = qdict_get_str(rsp_return, "status"); > > > > > > > + if (g_str_equal(status, "completed") || g_str_equal(status, "failed")) { > > > > > > > + stop = true; > > > > > > > + g_assert_cmpstr(status, ==, > > > > > > > + test->migrate_fail ? "failed" : "completed"); > > > > > > > + } > > > > > > > + qobject_unref(rsp_return); > > > > > > > + > > > > > > > + if (stop) { > > > > > > > + g_main_loop_quit(test->loop); > > > > > > > + } > > > > > > > + return stop ? G_SOURCE_REMOVE : G_SOURCE_CONTINUE; > > > > > > > +} > > > > > > > + > > > > > > > +static void migrate(QTestState *who, const char *uri) > > > > > > > +{ > > > > > > > + QDict *args, *rsp; > > > > > > > + > > > > > > > + args = qdict_new(); > > > > > > > + qdict_put_str(args, "uri", uri); > > > > > > > + > > > > > > > + rsp = qtest_qmp(who, "{ 'execute': 'migrate', 'arguments': %p }", args); > > > > > > > + > > > > > > > + g_assert(qdict_haskey(rsp, "return")); > > > > > > > + qobject_unref(rsp); > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +test_dbus_vmstate(Test *test) > > > > > > > +{ > > > > > > > + QTestState *src_qemu = NULL, *dst_qemu = NULL; > > > > > > > + char *src_qemu_args = NULL, *dst_qemu_args = NULL; > > > > > > > + char *uri = g_strdup_printf("unix:%s/migsocket", workdir); > > > > > > > + GThread *t = g_thread_new("dbus", dbus_thread, test); > > > > > > > + > > > > > > > + g_mutex_lock(&mutex); > > > > > > > + while (!test->dbus_loop) { > > > > > > > + g_cond_wait(&cond, &mutex); > > > > > > > + } > > > > > > > + > > > > > > > + src_qemu_args = > > > > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > > > > + "-object dbus-vmstate,id=dvB,addr=%s", > > > > > > > + g_dbus_server_get_client_address(test->srcA.server), > > > > > > > + g_dbus_server_get_client_address(test->srcB.server)); > > > > > > > + > > > > > > > + > > > > > > > + dst_qemu_args = > > > > > > > + g_strdup_printf("-object dbus-vmstate,id=dvA,addr=%s " > > > > > > > + "-object dbus-vmstate,id=dvB,addr=%s " > > > > > > > + "-incoming %s", > > > > > > > + g_dbus_server_get_client_address(test->dstA.server), > > > > > > > + g_dbus_server_get_client_address(test->dstB.server), > > > > > > > + uri); > > > > > > > + > > > > > > > + src_qemu = qtest_init(src_qemu_args); > > > > > > > + dst_qemu = qtest_init(dst_qemu_args); > > > > > > > + > > > > > > > + test->loop = g_main_loop_new(NULL, TRUE); > > > > > > > + > > > > > > > + migrate(src_qemu, uri); > > > > > > > + test->src_qemu = src_qemu; > > > > > > > + g_timeout_add_seconds(1, wait_for_migration_complete, test); > > > > > > > + > > > > > > > + g_main_loop_run(test->loop); > > > > > > > + g_main_loop_unref(test->loop); > > > > > > > + > > > > > > > + g_free(uri); > > > > > > > + qtest_quit(dst_qemu); > > > > > > > + qtest_quit(src_qemu); > > > > > > > + g_free(dst_qemu_args); > > > > > > > + g_free(src_qemu_args); > > > > > > > + > > > > > > > + g_main_loop_quit(test->dbus_loop); > > > > > > > + g_mutex_unlock(&mutex); > > > > > > > + > > > > > > > + g_thread_join(t); > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +check_migrated(TestServer *s, TestServer *d) > > > > > > > +{ > > > > > > > + assert(s->save_called); > > > > > > > + assert(!s->load_called); > > > > > > > + assert(!d->save_called); > > > > > > > + assert(d->load_called); > > > > > > > +} > > > > > > > + > > > > > > > +static void > > > > > > > +test_dbus_vmstate_migrate(void) > > > > > > > +{ > > > > > > > + Test test = { }; > > > > > > > + > > > > > > > + test_dbus_vmstate(&test); > > > > > > > + > > > > > > > + check_migrated(&test.srcA, &test.dstA); > > > > > > > + check_migrated(&test.srcB, &test.dstB); > > > > > > > +} > > > > > > > + > > > > > > > +int > > > > > > > +main(int argc, char **argv) > > > > > > > +{ > > > > > > > + GError *err = NULL; > > > > > > > + int ret; > > > > > > > + > > > > > > > + g_test_init(&argc, &argv, NULL); > > > > > > > + > > > > > > > + workdir = g_dir_make_tmp("dbus-vmstate-test-XXXXXX", &err); > > > > > > > + if (!workdir) { > > > > > > > + g_error("Unable to create temporary dir: %s\n", err->message); > > > > > > > + } > > > > > > > + > > > > > > > + qtest_add_func("/dbus-vmstate/migrate", > > > > > > > + test_dbus_vmstate_migrate); > > > > > > > + > > > > > > > + ret = g_test_run(); > > > > > > > + > > > > > > > + rmdir(workdir); > > > > > > > + g_free(workdir); > > > > > > > + > > > > > > > + return ret; > > > > > > > +} > > > > > > > diff --git a/tests/dbus-vmstate1.xml b/tests/dbus-vmstate1.xml > > > > > > > new file mode 100644 > > > > > > > index 0000000000..cc8563be4c > > > > > > > --- /dev/null > > > > > > > +++ b/tests/dbus-vmstate1.xml > > > > > > > @@ -0,0 +1,12 @@ > > > > > > > +<?xml version="1.0"?> > > > > > > > +<node name="/" xmlns:doc="http://www.freedesktop.org/dbus/1.0/doc.dtd"> > > > > > > > + <interface name="org.qemu.VMState1"> > > > > > > > + <property name="Id" type="s" access="read"/> > > > > > > > + <method name="Load"> > > > > > > > + <arg type="ay" name="data" direction="in"/> > > > > > > > + </method> > > > > > > > + <method name="Save"> > > > > > > > + <arg type="ay" name="data" direction="out"/> > > > > > > > + </method> > > > > > > > + </interface> > > > > > > > +</node> > > > > > > > -- > > > > > > > 2.23.0.rc1 > > > > > > > > > > > > > -- > > > > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > > -- > > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > -- > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > > > -- > Marc-André Lureau -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-08 15:03 [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Marc-André Lureau 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 1/2] qemu-file: move qemu_{get, put}_counted_string() declarations Marc-André Lureau 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object Marc-André Lureau @ 2019-08-23 11:20 ` Daniel P. Berrangé 2019-08-23 11:31 ` Marc-André Lureau 2 siblings, 1 reply; 28+ messages in thread From: Daniel P. Berrangé @ 2019-08-23 11:20 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Dr. David Alan Gilbert, Stefan Hajnoczi, Paolo Bonzini On Thu, Aug 08, 2019 at 07:03:23PM +0400, Marc-André Lureau wrote: > Hi, > > With external processes or helpers participating to the VM support, it > becomes necessary to handle their migration. Various options exist to > transfer their state: > 1) as the VM memory, RAM or devices (we could say that's how > vhost-user devices can be handled today, they are expected to > restore from ring state) > 2) other "vmstate" (as with TPM emulator state blobs) > 3) left to be handled by management layer > > 1) is not practical, since an external processes may legitimatelly > need arbitrary state date to back a device or a service, or may not > even have an associated device. > > 2) needs ad-hoc code for each helper, but is simple and working > > 3) is complicated for management layer, QEMU has the migration timing > > The proposed "dbus-vmstate" object will connect to a given D-Bus > peer address, and save/load from org.qemu.VMState1 interface. > > This way, helpers can have their state migrated with QEMU, without > implementing another ad-hoc support (such as done for TPM emulation) > > I chose D-Bus as it is ubiquitous on Linux (it is systemd IPC), and > can be made to work on various other OSes. There are several > implementations and good bindings for various languages. > (the tests/dbus-vmstate-test.c is a good example of how simple > the implementation of services can be, even in C) > > v2: > - D-Bus is most common and practical through a bus, but it requires a > daemon to be running. I argue that the benefits outweight the cost > of running an extra daemon in v1 in the context of multi-process > qemu, but it is also possible to connect in p2p mode as done in this > new version. So yesterday Stefanha brought up need for "mgmt apis" on the virtiofsd helper process & the conclusion is that dbus makes most sense for this purpose: https://www.redhat.com/archives/virtio-fs/2019-August/msg00339.html This use case is a slightly different from vmstate though. For vmstate we have two parties - virtiofsd and QEMU talking For the "mgmt apis" in virtiofsd, we have arbitrary parties involved - virtiofsd *and* an admin client tool, and/or maybe libvirt. I think this different scenario means that we do in fact need to have a bus present, as the p2p model doesn't scale well to many clients. Even if we have 1 dbus-daemon per QEMU instance, we need to cope with multiple instances of the same helper needing to connect. So we need to come up with some for identifying services. Normally DBus only allows 1 peer to own a given well known service name at any time. So we can't simply talk to a well-known 'org.qemu.virtiofsd' service name. Each service would need to to just rely on exporting objects under its unique service id (they look like :1.NNNN for some uniq NNN) QEMU still needs to known which connections on the bus are actually providing vhost-user services, and which are other things (like libvirt or random mgmt tools) So perhaps QEMU should expose a service 'org.qemu.VhostUserManager' with an object /org/qemu/VhostUSerManager Each helper supporting vmstate could register its existance by invoking a method org.qemu.VhostUserManager.Register(":1.NNNN") For things we want every vhost user daemon to implement, we can define well known objects paths & intefaces to expose at the path. eg If the helper supports vmstate dump, it should export an object at the well known path "/org/qemu/VMState" with methods x, y & z If the helper supports debug logging it should export an object at the well known path "/org/qemu/Logger" with method a, b & c etc Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 11:20 ` [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Daniel P. Berrangé @ 2019-08-23 11:31 ` Marc-André Lureau 2019-08-23 11:41 ` Daniel P. Berrangé 0 siblings, 1 reply; 28+ messages in thread From: Marc-André Lureau @ 2019-08-23 11:31 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Dr. David Alan Gilbert, Stefan Hajnoczi, Paolo Bonzini Hi On Fri, Aug 23, 2019 at 3:21 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Thu, Aug 08, 2019 at 07:03:23PM +0400, Marc-André Lureau wrote: > > Hi, > > > > With external processes or helpers participating to the VM support, it > > becomes necessary to handle their migration. Various options exist to > > transfer their state: > > 1) as the VM memory, RAM or devices (we could say that's how > > vhost-user devices can be handled today, they are expected to > > restore from ring state) > > 2) other "vmstate" (as with TPM emulator state blobs) > > 3) left to be handled by management layer > > > > 1) is not practical, since an external processes may legitimatelly > > need arbitrary state date to back a device or a service, or may not > > even have an associated device. > > > > 2) needs ad-hoc code for each helper, but is simple and working > > > > 3) is complicated for management layer, QEMU has the migration timing > > > > The proposed "dbus-vmstate" object will connect to a given D-Bus > > peer address, and save/load from org.qemu.VMState1 interface. > > > > This way, helpers can have their state migrated with QEMU, without > > implementing another ad-hoc support (such as done for TPM emulation) > > > > I chose D-Bus as it is ubiquitous on Linux (it is systemd IPC), and > > can be made to work on various other OSes. There are several > > implementations and good bindings for various languages. > > (the tests/dbus-vmstate-test.c is a good example of how simple > > the implementation of services can be, even in C) > > > > v2: > > - D-Bus is most common and practical through a bus, but it requires a > > daemon to be running. I argue that the benefits outweight the cost > > of running an extra daemon in v1 in the context of multi-process > > qemu, but it is also possible to connect in p2p mode as done in this > > new version. > > So yesterday Stefanha brought up need for "mgmt apis" on the > virtiofsd helper process & the conclusion is that dbus makes > most sense for this purpose: > > https://www.redhat.com/archives/virtio-fs/2019-August/msg00339.html > > This use case is a slightly different from vmstate though. > > For vmstate we have two parties - virtiofsd and QEMU talking > > For the "mgmt apis" in virtiofsd, we have arbitrary parties > involved - virtiofsd *and* an admin client tool, and/or > maybe libvirt. > > I think this different scenario means that we do in fact need > to have a bus present, as the p2p model doesn't scale well > to many clients. > > Even if we have 1 dbus-daemon per QEMU instance, we need to cope > with multiple instances of the same helper needing to connect. > So we need to come up with some for identifying services. Normally > DBus only allows 1 peer to own a given well known service name at > any time. So we can't simply talk to a well-known 'org.qemu.virtiofsd' > service name. > > Each service would need to to just rely on exporting objects under > its unique service id (they look like :1.NNNN for some uniq NNN) > > QEMU still needs to known which connections on the bus are actually > providing vhost-user services, and which are other things (like > libvirt or random mgmt tools) > > So perhaps QEMU should expose a service 'org.qemu.VhostUserManager' > with an object /org/qemu/VhostUSerManager > > Each helper supporting vmstate could register its existance > by invoking a method > > org.qemu.VhostUserManager.Register(":1.NNNN") There is no need for extra registration if the services are queued. You can then query the queue of org.qemu.VhostUser instances. This is the approach I took in v1 of this series with org.qemu.VMState1 service name. See https://patchew.org/QEMU/20190708072437.3339-1-marcandre.lureau@redhat.com/20190708072437.3339-4-marcandre.lureau@redhat.com/ as well as the libvirt series https://patchew.org/Libvirt/20190708070747.1962-1-marcandre.lureau@redhat.com/ Other approaches are common prefix (ex: org.mpris.MediaPlayer2.FooBar), which also allows to identify a particular implementation in a simple way. > > For things we want every vhost user daemon to implement, we can > define well known objects paths & intefaces to expose at the path. > > eg If the helper supports vmstate dump, it should export an object > at the well known path "/org/qemu/VMState" with methods x, y & z > > If the helper supports debug logging it should export an object at > the well known path "/org/qemu/Logger" with method a, b & c > > etc > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 11:31 ` Marc-André Lureau @ 2019-08-23 11:41 ` Daniel P. Berrangé 2019-08-23 11:47 ` Marc-André Lureau 2019-08-23 13:00 ` Dr. David Alan Gilbert 0 siblings, 2 replies; 28+ messages in thread From: Daniel P. Berrangé @ 2019-08-23 11:41 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Dr. David Alan Gilbert, Stefan Hajnoczi, Paolo Bonzini On Fri, Aug 23, 2019 at 03:31:16PM +0400, Marc-André Lureau wrote: > Hi > > On Fri, Aug 23, 2019 at 3:21 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > On Thu, Aug 08, 2019 at 07:03:23PM +0400, Marc-André Lureau wrote: > > > Hi, > > > > > > With external processes or helpers participating to the VM support, it > > > becomes necessary to handle their migration. Various options exist to > > > transfer their state: > > > 1) as the VM memory, RAM or devices (we could say that's how > > > vhost-user devices can be handled today, they are expected to > > > restore from ring state) > > > 2) other "vmstate" (as with TPM emulator state blobs) > > > 3) left to be handled by management layer > > > > > > 1) is not practical, since an external processes may legitimatelly > > > need arbitrary state date to back a device or a service, or may not > > > even have an associated device. > > > > > > 2) needs ad-hoc code for each helper, but is simple and working > > > > > > 3) is complicated for management layer, QEMU has the migration timing > > > > > > The proposed "dbus-vmstate" object will connect to a given D-Bus > > > peer address, and save/load from org.qemu.VMState1 interface. > > > > > > This way, helpers can have their state migrated with QEMU, without > > > implementing another ad-hoc support (such as done for TPM emulation) > > > > > > I chose D-Bus as it is ubiquitous on Linux (it is systemd IPC), and > > > can be made to work on various other OSes. There are several > > > implementations and good bindings for various languages. > > > (the tests/dbus-vmstate-test.c is a good example of how simple > > > the implementation of services can be, even in C) > > > > > > v2: > > > - D-Bus is most common and practical through a bus, but it requires a > > > daemon to be running. I argue that the benefits outweight the cost > > > of running an extra daemon in v1 in the context of multi-process > > > qemu, but it is also possible to connect in p2p mode as done in this > > > new version. > > > > So yesterday Stefanha brought up need for "mgmt apis" on the > > virtiofsd helper process & the conclusion is that dbus makes > > most sense for this purpose: > > > > https://www.redhat.com/archives/virtio-fs/2019-August/msg00339.html > > > > This use case is a slightly different from vmstate though. > > > > For vmstate we have two parties - virtiofsd and QEMU talking > > > > For the "mgmt apis" in virtiofsd, we have arbitrary parties > > involved - virtiofsd *and* an admin client tool, and/or > > maybe libvirt. > > > > I think this different scenario means that we do in fact need > > to have a bus present, as the p2p model doesn't scale well > > to many clients. > > > > Even if we have 1 dbus-daemon per QEMU instance, we need to cope > > with multiple instances of the same helper needing to connect. > > So we need to come up with some for identifying services. Normally > > DBus only allows 1 peer to own a given well known service name at > > any time. So we can't simply talk to a well-known 'org.qemu.virtiofsd' > > service name. > > > > Each service would need to to just rely on exporting objects under > > its unique service id (they look like :1.NNNN for some uniq NNN) > > > > QEMU still needs to known which connections on the bus are actually > > providing vhost-user services, and which are other things (like > > libvirt or random mgmt tools) > > > > So perhaps QEMU should expose a service 'org.qemu.VhostUserManager' > > with an object /org/qemu/VhostUSerManager > > > > Each helper supporting vmstate could register its existance > > by invoking a method > > > > org.qemu.VhostUserManager.Register(":1.NNNN") > > > There is no need for extra registration if the services are queued. > You can then query the queue of org.qemu.VhostUser instances. > > This is the approach I took in v1 of this series with > org.qemu.VMState1 service name. > > See https://patchew.org/QEMU/20190708072437.3339-1-marcandre.lureau@redhat.com/20190708072437.3339-4-marcandre.lureau@redhat.com/ I think that's a pretty gross hack tha is abusing the unique service concept, as we clearly don't have unique services anymore. > Other approaches are common prefix (ex: > org.mpris.MediaPlayer2.FooBar), which also allows to identify a > particular implementation in a simple way. This means QEMU still has to iterate over every single client on the bus to identify them. If you're doing that, there's no point in owning a well known service at all. Just iterate over the unique bus names and look for the exported object path /org/qemu/VMState Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 11:41 ` Daniel P. Berrangé @ 2019-08-23 11:47 ` Marc-André Lureau 2019-08-23 13:00 ` Dr. David Alan Gilbert 1 sibling, 0 replies; 28+ messages in thread From: Marc-André Lureau @ 2019-08-23 11:47 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Dr. David Alan Gilbert, Stefan Hajnoczi, Paolo Bonzini Hi On Fri, Aug 23, 2019 at 3:42 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Fri, Aug 23, 2019 at 03:31:16PM +0400, Marc-André Lureau wrote: > > Hi > > > > On Fri, Aug 23, 2019 at 3:21 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > > > > > > On Thu, Aug 08, 2019 at 07:03:23PM +0400, Marc-André Lureau wrote: > > > > Hi, > > > > > > > > With external processes or helpers participating to the VM support, it > > > > becomes necessary to handle their migration. Various options exist to > > > > transfer their state: > > > > 1) as the VM memory, RAM or devices (we could say that's how > > > > vhost-user devices can be handled today, they are expected to > > > > restore from ring state) > > > > 2) other "vmstate" (as with TPM emulator state blobs) > > > > 3) left to be handled by management layer > > > > > > > > 1) is not practical, since an external processes may legitimatelly > > > > need arbitrary state date to back a device or a service, or may not > > > > even have an associated device. > > > > > > > > 2) needs ad-hoc code for each helper, but is simple and working > > > > > > > > 3) is complicated for management layer, QEMU has the migration timing > > > > > > > > The proposed "dbus-vmstate" object will connect to a given D-Bus > > > > peer address, and save/load from org.qemu.VMState1 interface. > > > > > > > > This way, helpers can have their state migrated with QEMU, without > > > > implementing another ad-hoc support (such as done for TPM emulation) > > > > > > > > I chose D-Bus as it is ubiquitous on Linux (it is systemd IPC), and > > > > can be made to work on various other OSes. There are several > > > > implementations and good bindings for various languages. > > > > (the tests/dbus-vmstate-test.c is a good example of how simple > > > > the implementation of services can be, even in C) > > > > > > > > v2: > > > > - D-Bus is most common and practical through a bus, but it requires a > > > > daemon to be running. I argue that the benefits outweight the cost > > > > of running an extra daemon in v1 in the context of multi-process > > > > qemu, but it is also possible to connect in p2p mode as done in this > > > > new version. > > > > > > So yesterday Stefanha brought up need for "mgmt apis" on the > > > virtiofsd helper process & the conclusion is that dbus makes > > > most sense for this purpose: > > > > > > https://www.redhat.com/archives/virtio-fs/2019-August/msg00339.html > > > > > > This use case is a slightly different from vmstate though. > > > > > > For vmstate we have two parties - virtiofsd and QEMU talking > > > > > > For the "mgmt apis" in virtiofsd, we have arbitrary parties > > > involved - virtiofsd *and* an admin client tool, and/or > > > maybe libvirt. > > > > > > I think this different scenario means that we do in fact need > > > to have a bus present, as the p2p model doesn't scale well > > > to many clients. > > > > > > Even if we have 1 dbus-daemon per QEMU instance, we need to cope > > > with multiple instances of the same helper needing to connect. > > > So we need to come up with some for identifying services. Normally > > > DBus only allows 1 peer to own a given well known service name at > > > any time. So we can't simply talk to a well-known 'org.qemu.virtiofsd' > > > service name. > > > > > > Each service would need to to just rely on exporting objects under > > > its unique service id (they look like :1.NNNN for some uniq NNN) > > > > > > QEMU still needs to known which connections on the bus are actually > > > providing vhost-user services, and which are other things (like > > > libvirt or random mgmt tools) > > > > > > So perhaps QEMU should expose a service 'org.qemu.VhostUserManager' > > > with an object /org/qemu/VhostUSerManager > > > > > > Each helper supporting vmstate could register its existance > > > by invoking a method > > > > > > org.qemu.VhostUserManager.Register(":1.NNNN") > > > > > > There is no need for extra registration if the services are queued. > > You can then query the queue of org.qemu.VhostUser instances. > > > > This is the approach I took in v1 of this series with > > org.qemu.VMState1 service name. > > > > See https://patchew.org/QEMU/20190708072437.3339-1-marcandre.lureau@redhat.com/20190708072437.3339-4-marcandre.lureau@redhat.com/ > > I think that's a pretty gross hack tha is abusing the unique service > concept, as we clearly don't have unique services anymore. "well-known" names are not "unique". I think you are restricting what "well-known" names are and how to use them. The queued owner concept has always been there. > > > Other approaches are common prefix (ex: > > org.mpris.MediaPlayer2.FooBar), which also allows to identify a > > particular implementation in a simple way. > > This means QEMU still has to iterate over every single client > on the bus to identify them. If you're doing that, there's > no point in owning a well known service at all. Just iterate > over the unique bus names and look for the exported object > path /org/qemu/VMState Not exactly, since it wouldn't have to query each connection, but only the bus. > > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 11:41 ` Daniel P. Berrangé 2019-08-23 11:47 ` Marc-André Lureau @ 2019-08-23 13:00 ` Dr. David Alan Gilbert 2019-08-23 13:48 ` Marc-André Lureau 1 sibling, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-23 13:00 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Stefan Hajnoczi, Paolo Bonzini, Marc-André Lureau * Daniel P. Berrangé (berrange@redhat.com) wrote: <snip> > This means QEMU still has to iterate over every single client > on the bus to identify them. If you're doing that, there's > no point in owning a well known service at all. Just iterate > over the unique bus names and look for the exported object > path /org/qemu/VMState > Not knowing anything about DBus security, I want to ask how do we handle security here? I want to know that the external device that's giving me migration data is the device I think I'm speaking to, not one of the other devices; I also dont want different devices chatting to each other over dbus unless we're very careful. Dave > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 13:00 ` Dr. David Alan Gilbert @ 2019-08-23 13:48 ` Marc-André Lureau 2019-08-23 14:09 ` Daniel P. Berrangé 2019-08-23 14:09 ` Dr. David Alan Gilbert 0 siblings, 2 replies; 28+ messages in thread From: Marc-André Lureau @ 2019-08-23 13:48 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Thomas Huth, Daniel P. Berrangé, Juan Quintela, qemu-devel, Stefan Hajnoczi, Paolo Bonzini Hi On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert <dgilbert@redhat.com> wrote: > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > <snip> > > > This means QEMU still has to iterate over every single client > > on the bus to identify them. If you're doing that, there's > > no point in owning a well known service at all. Just iterate > > over the unique bus names and look for the exported object > > path /org/qemu/VMState > > > > Not knowing anything about DBus security, I want to ask how do > we handle security here? First of all, we are talking about cooperative processes, and having a specific bus for each qemu instance. So some amount of security/trust is already assumed. But if necessary, dbus can enforce policies on who is allowed to own a name, or to send/receive message from. As far as I know, this is mostly user/group policies. But there is also SELinux checks to send_msg and acquire_svc (see dbus-daemon(1)) > > I want to know that the external device that's giving me migration data > is the device I think I'm speaking to, not one of the other devices; DBus is not the problem nor the solution here. But what defines that device-service strong relationship? Can you generalize it? I don't think so. What DBus can guarantee is that the unique-id you are talking to is always the same connection (thus the same process). > I also dont want different devices chatting to each other over dbus > unless we're very careful. That's a bus policy job. > > Dave > > > Regards, > > Daniel > > -- > > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > > |: https://libvirt.org -o- https://fstop138.berrange.com :| > > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > -- Marc-André Lureau ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 13:48 ` Marc-André Lureau @ 2019-08-23 14:09 ` Daniel P. Berrangé 2019-08-23 14:09 ` Dr. David Alan Gilbert 1 sibling, 0 replies; 28+ messages in thread From: Daniel P. Berrangé @ 2019-08-23 14:09 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Dr. David Alan Gilbert, Stefan Hajnoczi, Paolo Bonzini On Fri, Aug 23, 2019 at 05:48:37PM +0400, Marc-André Lureau wrote: > Hi > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > <dgilbert@redhat.com> wrote: > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > <snip> > > > > > This means QEMU still has to iterate over every single client > > > on the bus to identify them. If you're doing that, there's > > > no point in owning a well known service at all. Just iterate > > > over the unique bus names and look for the exported object > > > path /org/qemu/VMState > > > > > > > Not knowing anything about DBus security, I want to ask how do > > we handle security here? > > First of all, we are talking about cooperative processes, and having a > specific bus for each qemu instance. So some amount of security/trust > is already assumed. > > But if necessary, dbus can enforce policies on who is allowed to own a > name, or to send/receive message from. As far as I know, this is > mostly user/group policies. Yep, the bulk of dbus policy restrictions are tied to user/groups. After all, if two processes are under the same user ID, in general any restrictions dbus tried to place on them are worthless as they can simply attack each other outside dbus. Running helpers as different UID from QEMU itself makes sense. Running every single helper as a distinct UID is likely unmanagably complex - especially if the helpers need certain privileges to do their job. > But there is also SELinux checks to send_msg and acquire_svc (see > dbus-daemon(1)) SELinux gives us more fine grained control. For example, consider each vhost user helper process "virtiofs_t" and QEMU is "svirt_t" You can write a rule which says svirt_t can send messages to virtiofs_t, which allows QEMU to talk to the helpers. Two helper processes both running "virtiofs_t" will, however, not be able to talk to each other. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 13:48 ` Marc-André Lureau 2019-08-23 14:09 ` Daniel P. Berrangé @ 2019-08-23 14:09 ` Dr. David Alan Gilbert 2019-08-23 14:20 ` Daniel P. Berrangé 1 sibling, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-23 14:09 UTC (permalink / raw) To: Marc-André Lureau Cc: Laurent Vivier, Thomas Huth, Daniel P. Berrangé, Juan Quintela, qemu-devel, Stefan Hajnoczi, Paolo Bonzini * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > Hi > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > <dgilbert@redhat.com> wrote: > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > <snip> > > > > > This means QEMU still has to iterate over every single client > > > on the bus to identify them. If you're doing that, there's > > > no point in owning a well known service at all. Just iterate > > > over the unique bus names and look for the exported object > > > path /org/qemu/VMState > > > > > > > Not knowing anything about DBus security, I want to ask how do > > we handle security here? > > First of all, we are talking about cooperative processes, and having a > specific bus for each qemu instance. So some amount of security/trust > is already assumed. Some but we need to keep it as limited as possible; for example two reasons for having separate processes both come down to security: a) vtpm - however screwy the qemu is, you can never get to the keys in the vtpm b) virtio-gpu, loads of complex GPU code that can't break the main qemu process. > But if necessary, dbus can enforce policies on who is allowed to own a > name, or to send/receive message from. As far as I know, this is > mostly user/group policies. > > But there is also SELinux checks to send_msg and acquire_svc (see > dbus-daemon(1)) But how does something like SELinux interact with a private dbus rather than the system dbus? > > > > I want to know that the external device that's giving me migration data > > is the device I think I'm speaking to, not one of the other devices; > > DBus is not the problem nor the solution here. Well, if the migration data was squirting down the existing vhost-user channel then there would be no risk here; so the use of dbus is creating the problem. > But what defines that device-service strong relationship? Can you > generalize it? I don't think so. > > What DBus can guarantee is that the unique-id you are talking to is > always the same connection (thus the same process). > > > I also dont want different devices chatting to each other over dbus > > unless we're very careful. > > That's a bus policy job. OK, as long as you somehow set it up. Dave > > > > Dave > > > > > Regards, > > > Daniel > > > -- > > > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > > > |: https://libvirt.org -o- https://fstop138.berrange.com :| > > > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > > -- > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > > > -- > Marc-André Lureau -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 14:09 ` Dr. David Alan Gilbert @ 2019-08-23 14:20 ` Daniel P. Berrangé 2019-08-23 14:26 ` Dr. David Alan Gilbert 0 siblings, 1 reply; 28+ messages in thread From: Daniel P. Berrangé @ 2019-08-23 14:20 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini On Fri, Aug 23, 2019 at 03:09:48PM +0100, Dr. David Alan Gilbert wrote: > * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > > Hi > > > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > > <dgilbert@redhat.com> wrote: > > > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > > <snip> > > > > > > > This means QEMU still has to iterate over every single client > > > > on the bus to identify them. If you're doing that, there's > > > > no point in owning a well known service at all. Just iterate > > > > over the unique bus names and look for the exported object > > > > path /org/qemu/VMState > > > > > > > > > > Not knowing anything about DBus security, I want to ask how do > > > we handle security here? > > > > First of all, we are talking about cooperative processes, and having a > > specific bus for each qemu instance. So some amount of security/trust > > is already assumed. > > Some but we need to keep it as limited as possible; for example two > reasons for having separate processes both come down to security: > > a) vtpm - however screwy the qemu is, you can never get to the keys in > the vtpm Processes connected to dbus can only call the DBus APIs that vtpm actually exports. The vtpm should simply *not* export a DBus API that allows anything to fetch the keys. If it did want to export APIs for fetching keys, then we would have to ensure suitable dbus /selinux policy was created to prevent unwarranted access. > b) virtio-gpu, loads of complex GPU code that can't break the main > qemu process. That's no problem - virtio-gpu crashes, it disappears from the dbus bus, but everything else keeps running. > > But if necessary, dbus can enforce policies on who is allowed to own a > > name, or to send/receive message from. As far as I know, this is > > mostly user/group policies. > > > > But there is also SELinux checks to send_msg and acquire_svc (see > > dbus-daemon(1)) > > But how does something like SELinux interact with a private dbus > rather than the system dbus? There's already two dbus-daemon's on each host - the system one and the session one, and they get different selinux contexts, system_dbus_t and unconfined_dbus_t. Since libvirt would be responsible for launching these private dbus daemons it would be easy to make it run svirt_dbus_t for example. Actually it would be svirt_dbus_t:s0:cNNN,cMMM to get uniqueness per VM. Will of course require us to talk to the SELinux maintainers to get some sensible policy rules created. > > > I want to know that the external device that's giving me migration data > > > is the device I think I'm speaking to, not one of the other devices; > > > > DBus is not the problem nor the solution here. > > Well, if the migration data was squirting down the existing vhost-user > channel then there would be no risk here; so the use of dbus is creating > the problem. > > > But what defines that device-service strong relationship? Can you > > generalize it? I don't think so. > > > > What DBus can guarantee is that the unique-id you are talking to is > > always the same connection (thus the same process). > > > > > I also dont want different devices chatting to each other over dbus > > > unless we're very careful. > > > > That's a bus policy job. > > OK, as long as you somehow set it up. > > Dave > > > > > > > Dave > > > > > > > Regards, > > > > Daniel > > > > -- > > > > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > > > > |: https://libvirt.org -o- https://fstop138.berrange.com :| > > > > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > > > -- > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > > > > > > > -- > > Marc-André Lureau > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 14:20 ` Daniel P. Berrangé @ 2019-08-23 14:26 ` Dr. David Alan Gilbert 2019-08-23 14:40 ` Daniel P. Berrangé 0 siblings, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-23 14:26 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini * Daniel P. Berrangé (berrange@redhat.com) wrote: > On Fri, Aug 23, 2019 at 03:09:48PM +0100, Dr. David Alan Gilbert wrote: > > * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > > > Hi > > > > > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > > > <dgilbert@redhat.com> wrote: > > > > > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > > > > <snip> > > > > > > > > > This means QEMU still has to iterate over every single client > > > > > on the bus to identify them. If you're doing that, there's > > > > > no point in owning a well known service at all. Just iterate > > > > > over the unique bus names and look for the exported object > > > > > path /org/qemu/VMState > > > > > > > > > > > > > Not knowing anything about DBus security, I want to ask how do > > > > we handle security here? > > > > > > First of all, we are talking about cooperative processes, and having a > > > specific bus for each qemu instance. So some amount of security/trust > > > is already assumed. > > > > Some but we need to keep it as limited as possible; for example two > > reasons for having separate processes both come down to security: > > > > a) vtpm - however screwy the qemu is, you can never get to the keys in > > the vtpm > > Processes connected to dbus can only call the DBus APIs that vtpm > actually exports. The vtpm should simply *not* export a DBus > API that allows anything to fetch the keys. > > If it did want to export APIs for fetching keys, then we would > have to ensure suitable dbus /selinux policy was created to > prevent unwarranted access. This was really just one example of where the security/trust isn't assumed; however a more concrete case is migration of a vtpm, and even though it's probably encrypted blob you still don't want some other device to grab the migration data - or to say reinitialise the vtpm. > > b) virtio-gpu, loads of complex GPU code that can't break the main > > qemu process. > > That's no problem - virtio-gpu crashes, it disappears from the dbus > bus, but everything else keeps running. Crashing is the easy case; assume it's malicious and you don't want it getting to say a storage device provided by another vhost-user device. > > > But if necessary, dbus can enforce policies on who is allowed to own a > > > name, or to send/receive message from. As far as I know, this is > > > mostly user/group policies. > > > > > > But there is also SELinux checks to send_msg and acquire_svc (see > > > dbus-daemon(1)) > > > > But how does something like SELinux interact with a private dbus > > rather than the system dbus? > > There's already two dbus-daemon's on each host - the system one and > the session one, and they get different selinux contexts, > system_dbus_t and unconfined_dbus_t. > > Since libvirt would be responsible for launching these private dbus > daemons it would be easy to make it run svirt_dbus_t for example. > Actually it would be svirt_dbus_t:s0:cNNN,cMMM to get uniqueness > per VM. > > Will of course require us to talk to the SELinux maintainers to > get some sensible policy rules created. This all relies on SELinux and running privileged qemu/vhost-user pairs; needing to do that purely to enforce security seems wrong. Dave > > > > I want to know that the external device that's giving me migration data > > > > is the device I think I'm speaking to, not one of the other devices; > > > > > > DBus is not the problem nor the solution here. > > > > Well, if the migration data was squirting down the existing vhost-user > > channel then there would be no risk here; so the use of dbus is creating > > the problem. > > > > > But what defines that device-service strong relationship? Can you > > > generalize it? I don't think so. > > > > > > What DBus can guarantee is that the unique-id you are talking to is > > > always the same connection (thus the same process). > > > > > > > I also dont want different devices chatting to each other over dbus > > > > unless we're very careful. > > > > > > That's a bus policy job. > > > > OK, as long as you somehow set it up. > > > > Dave > > > > > > > > > > Dave > > > > > > > > > Regards, > > > > > Daniel > > > > > -- > > > > > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > > > > > |: https://libvirt.org -o- https://fstop138.berrange.com :| > > > > > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > > > > -- > > > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > > > > > > > > > > > -- > > > Marc-André Lureau > > -- > > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 14:26 ` Dr. David Alan Gilbert @ 2019-08-23 14:40 ` Daniel P. Berrangé 2019-08-23 14:56 ` Dr. David Alan Gilbert 0 siblings, 1 reply; 28+ messages in thread From: Daniel P. Berrangé @ 2019-08-23 14:40 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini On Fri, Aug 23, 2019 at 03:26:02PM +0100, Dr. David Alan Gilbert wrote: > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > On Fri, Aug 23, 2019 at 03:09:48PM +0100, Dr. David Alan Gilbert wrote: > > > * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > > > > Hi > > > > > > > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > > > > <dgilbert@redhat.com> wrote: > > > > > > > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > > > > > > <snip> > > > > > > > > > > > This means QEMU still has to iterate over every single client > > > > > > on the bus to identify them. If you're doing that, there's > > > > > > no point in owning a well known service at all. Just iterate > > > > > > over the unique bus names and look for the exported object > > > > > > path /org/qemu/VMState > > > > > > > > > > > > > > > > Not knowing anything about DBus security, I want to ask how do > > > > > we handle security here? > > > > > > > > First of all, we are talking about cooperative processes, and having a > > > > specific bus for each qemu instance. So some amount of security/trust > > > > is already assumed. > > > > > > Some but we need to keep it as limited as possible; for example two > > > reasons for having separate processes both come down to security: > > > > > > a) vtpm - however screwy the qemu is, you can never get to the keys in > > > the vtpm > > > > Processes connected to dbus can only call the DBus APIs that vtpm > > actually exports. The vtpm should simply *not* export a DBus > > API that allows anything to fetch the keys. > > > > If it did want to export APIs for fetching keys, then we would > > have to ensure suitable dbus /selinux policy was created to > > prevent unwarranted access. > > This was really just one example of where the security/trust isn't > assumed; however a more concrete case is migration of a vtpm, and even > though it's probably encrypted blob you still don't want some other > device to grab the migration data - or to say reinitialise the vtpm. That can be dealt with by the dbus security policies, provided you either run the vtpm as a different user ID from the other untrustworthy helpers, or use a different selinux context for vtpm. You can then express that only the user that QEMU is running under can talk to vtpm over dbus. Where I think you could have problems is if you needed finer grainer control with selinux. eg if vstpm exports 2 different services, you can't allow access to one service, but forbid access to the other service. > > > b) virtio-gpu, loads of complex GPU code that can't break the main > > > qemu process. > > > > That's no problem - virtio-gpu crashes, it disappears from the dbus > > bus, but everything else keeps running. > > Crashing is the easy case; assume it's malicious and you don't want it > getting to say a storage device provided by another vhost-user device. If we assume that the 2 processes can't commnuicate / access each other outside DBus, then the attack avenues added by use of dbus are most likely either: - invoking some DBus method that should not be allowed due to incomplete dbus security policy. - finding a crash in a dbus client library that you can somehow exploit to get remote code execution in the separate process I won't claim this is impossible, but I think it helps to be using a standard, widely used battle tested RPC impl, rather than a home grown RPC protocol. > > > > But if necessary, dbus can enforce policies on who is allowed to own a > > > > name, or to send/receive message from. As far as I know, this is > > > > mostly user/group policies. > > > > > > > > But there is also SELinux checks to send_msg and acquire_svc (see > > > > dbus-daemon(1)) > > > > > > But how does something like SELinux interact with a private dbus > > > rather than the system dbus? > > > > There's already two dbus-daemon's on each host - the system one and > > the session one, and they get different selinux contexts, > > system_dbus_t and unconfined_dbus_t. > > > > Since libvirt would be responsible for launching these private dbus > > daemons it would be easy to make it run svirt_dbus_t for example. > > Actually it would be svirt_dbus_t:s0:cNNN,cMMM to get uniqueness > > per VM. > > > > Will of course require us to talk to the SELinux maintainers to > > get some sensible policy rules created. > > This all relies on SELinux and running privileged qemu/vhost-user pairs; > needing to do that purely to enforce security seems wrong. Compare to an alternative bus-less solution where each helper has a direct UNIX socket connection to QEMU. If two helpers are running as the same user ID, then can still directly attack each other via things like ptrace or /proc/$PID/mem, unless you've used SELinux to isolate them, or run each as a distinct user ID. If you do the latter, then we can still easily isolate them using dbus. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 14:40 ` Daniel P. Berrangé @ 2019-08-23 14:56 ` Dr. David Alan Gilbert 2019-08-23 15:05 ` Daniel P. Berrangé 0 siblings, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-23 14:56 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini * Daniel P. Berrangé (berrange@redhat.com) wrote: > On Fri, Aug 23, 2019 at 03:26:02PM +0100, Dr. David Alan Gilbert wrote: > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > On Fri, Aug 23, 2019 at 03:09:48PM +0100, Dr. David Alan Gilbert wrote: > > > > * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > > > > > Hi > > > > > > > > > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > > > > > <dgilbert@redhat.com> wrote: > > > > > > > > > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > > > > > > > > <snip> > > > > > > > > > > > > > This means QEMU still has to iterate over every single client > > > > > > > on the bus to identify them. If you're doing that, there's > > > > > > > no point in owning a well known service at all. Just iterate > > > > > > > over the unique bus names and look for the exported object > > > > > > > path /org/qemu/VMState > > > > > > > > > > > > > > > > > > > Not knowing anything about DBus security, I want to ask how do > > > > > > we handle security here? > > > > > > > > > > First of all, we are talking about cooperative processes, and having a > > > > > specific bus for each qemu instance. So some amount of security/trust > > > > > is already assumed. > > > > > > > > Some but we need to keep it as limited as possible; for example two > > > > reasons for having separate processes both come down to security: > > > > > > > > a) vtpm - however screwy the qemu is, you can never get to the keys in > > > > the vtpm > > > > > > Processes connected to dbus can only call the DBus APIs that vtpm > > > actually exports. The vtpm should simply *not* export a DBus > > > API that allows anything to fetch the keys. > > > > > > If it did want to export APIs for fetching keys, then we would > > > have to ensure suitable dbus /selinux policy was created to > > > prevent unwarranted access. > > > > This was really just one example of where the security/trust isn't > > assumed; however a more concrete case is migration of a vtpm, and even > > though it's probably encrypted blob you still don't want some other > > device to grab the migration data - or to say reinitialise the vtpm. > > That can be dealt with by the dbus security policies, provided > you either run the vtpm as a different user ID from the other > untrustworthy helpers, or use a different selinux context for > vtpm. You can then express that only the user that QEMU is > running under can talk to vtpm over dbus. The need for the extra user ID or selinux context is a pain; but probably warranted for the vTPM; in general though some of this exists because of the choice of DBus and wouldn't be a problem for something that had a point-to-point socket it sent everything over. > Where I think you could have problems is if you needed finer > grainer control with selinux. eg if vstpm exports 2 different > services, you can't allow access to one service, but forbid > access to the other service. > > > > > b) virtio-gpu, loads of complex GPU code that can't break the main > > > > qemu process. > > > > > > That's no problem - virtio-gpu crashes, it disappears from the dbus > > > bus, but everything else keeps running. > > > > Crashing is the easy case; assume it's malicious and you don't want it > > getting to say a storage device provided by another vhost-user device. > > If we assume that the 2 processes can't commnuicate / access each > other outside DBus, then the attack avenues added by use of dbus > are most likely either: > > - invoking some DBus method that should not be allowed due > to incomplete dbus security policy. > > - finding a crash in a dbus client library that you can somehow > exploit to get remote code execution in the separate process > > I won't claim this is impossible, but I think it helps to be > using a standard, widely used battle tested RPC impl, rather > than a home grown RPC protocol. It's only the policy case I worry about; and my point here is if we decide to use dbus then we have to think properly about security and defined stuff. > > > > > > > But if necessary, dbus can enforce policies on who is allowed to own a > > > > > name, or to send/receive message from. As far as I know, this is > > > > > mostly user/group policies. > > > > > > > > > > But there is also SELinux checks to send_msg and acquire_svc (see > > > > > dbus-daemon(1)) > > > > > > > > But how does something like SELinux interact with a private dbus > > > > rather than the system dbus? > > > > > > There's already two dbus-daemon's on each host - the system one and > > > the session one, and they get different selinux contexts, > > > system_dbus_t and unconfined_dbus_t. > > > > > > Since libvirt would be responsible for launching these private dbus > > > daemons it would be easy to make it run svirt_dbus_t for example. > > > Actually it would be svirt_dbus_t:s0:cNNN,cMMM to get uniqueness > > > per VM. > > > > > > Will of course require us to talk to the SELinux maintainers to > > > get some sensible policy rules created. > > > > This all relies on SELinux and running privileged qemu/vhost-user pairs; > > needing to do that purely to enforce security seems wrong. > > Compare to an alternative bus-less solution where each helper has > a direct UNIX socket connection to QEMU. > > If two helpers are running as the same user ID, then can still > directly attack each other via things like ptrace or /proc/$PID/mem, > unless you've used SELinux to isolate them, or run each as a distinct > user ID. If you do the latter, then we can still easily isolate > them using dbus. You can lock those down pretty easily though. Dave > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 14:56 ` Dr. David Alan Gilbert @ 2019-08-23 15:05 ` Daniel P. Berrangé 2019-08-23 15:14 ` Dr. David Alan Gilbert 0 siblings, 1 reply; 28+ messages in thread From: Daniel P. Berrangé @ 2019-08-23 15:05 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini On Fri, Aug 23, 2019 at 03:56:34PM +0100, Dr. David Alan Gilbert wrote: > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > On Fri, Aug 23, 2019 at 03:26:02PM +0100, Dr. David Alan Gilbert wrote: > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > On Fri, Aug 23, 2019 at 03:09:48PM +0100, Dr. David Alan Gilbert wrote: > > > > > * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > > > > > > Hi > > > > > > > > > > > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > > > > > > <dgilbert@redhat.com> wrote: > > > > > > > > > > > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > > > > > > > > > > <snip> > > > > > > > > > > > > > > > This means QEMU still has to iterate over every single client > > > > > > > > on the bus to identify them. If you're doing that, there's > > > > > > > > no point in owning a well known service at all. Just iterate > > > > > > > > over the unique bus names and look for the exported object > > > > > > > > path /org/qemu/VMState > > > > > > > > > > > > > > > > > > > > > > Not knowing anything about DBus security, I want to ask how do > > > > > > > we handle security here? > > > > > > > > > > > > First of all, we are talking about cooperative processes, and having a > > > > > > specific bus for each qemu instance. So some amount of security/trust > > > > > > is already assumed. > > > > > > > > > > Some but we need to keep it as limited as possible; for example two > > > > > reasons for having separate processes both come down to security: > > > > > > > > > > a) vtpm - however screwy the qemu is, you can never get to the keys in > > > > > the vtpm > > > > > > > > Processes connected to dbus can only call the DBus APIs that vtpm > > > > actually exports. The vtpm should simply *not* export a DBus > > > > API that allows anything to fetch the keys. > > > > > > > > If it did want to export APIs for fetching keys, then we would > > > > have to ensure suitable dbus /selinux policy was created to > > > > prevent unwarranted access. > > > > > > This was really just one example of where the security/trust isn't > > > assumed; however a more concrete case is migration of a vtpm, and even > > > though it's probably encrypted blob you still don't want some other > > > device to grab the migration data - or to say reinitialise the vtpm. > > > > That can be dealt with by the dbus security policies, provided > > you either run the vtpm as a different user ID from the other > > untrustworthy helpers, or use a different selinux context for > > vtpm. You can then express that only the user that QEMU is > > running under can talk to vtpm over dbus. > > The need for the extra user ID or selinux context is a pain; > but probably warranted for the vTPM; in general though some of this > exists because of the choice of DBus and wouldn't be a problem for > something that had a point-to-point socket it sent everything over. NB be careful to use s/DBus/DBus bus/ DBus the protocol is fine to be used in a point-to-point socket scenario - the use of the bus is strictly optional. If all communication we expect is exclusively Helper <-> QEMU, then I'd argue in favour of dbus in point-to-point mode. The use cases Stefan brought up for virtiofsd though is what I think brings the idea of using the bus relevant. It is the desire to allow online control/mgmt of the helper, which introduces a 3rd party which isn't QEMU. Instead either libvirt or a standalone admin/debugging tool. With multiple parties involved I think the bus becomes relevant With p2p mode you could have 2 dbus socket for Helper <-> QEMU and another dbus socket for Helper <-> libvirt/debugging, but this isn't an obvious security win over using the bus, as you now need different access rules for each of the p2p sockets to say who can connect to which socket. > > Where I think you could have problems is if you needed finer > > grainer control with selinux. eg if vstpm exports 2 different > > services, you can't allow access to one service, but forbid > > access to the other service. > > > > > > > b) virtio-gpu, loads of complex GPU code that can't break the main > > > > > qemu process. > > > > > > > > That's no problem - virtio-gpu crashes, it disappears from the dbus > > > > bus, but everything else keeps running. > > > > > > Crashing is the easy case; assume it's malicious and you don't want it > > > getting to say a storage device provided by another vhost-user device. > > > > If we assume that the 2 processes can't commnuicate / access each > > other outside DBus, then the attack avenues added by use of dbus > > are most likely either: > > > > - invoking some DBus method that should not be allowed due > > to incomplete dbus security policy. > > > > - finding a crash in a dbus client library that you can somehow > > exploit to get remote code execution in the separate process > > > > I won't claim this is impossible, but I think it helps to be > > using a standard, widely used battle tested RPC impl, rather > > than a home grown RPC protocol. > > It's only the policy case I worry about; and my point here is if we > decide to use dbus then we have to think properly about security and > defined stuff. > > > > > > > > > > > But if necessary, dbus can enforce policies on who is allowed to own a > > > > > > name, or to send/receive message from. As far as I know, this is > > > > > > mostly user/group policies. > > > > > > > > > > > > But there is also SELinux checks to send_msg and acquire_svc (see > > > > > > dbus-daemon(1)) > > > > > > > > > > But how does something like SELinux interact with a private dbus > > > > > rather than the system dbus? > > > > > > > > There's already two dbus-daemon's on each host - the system one and > > > > the session one, and they get different selinux contexts, > > > > system_dbus_t and unconfined_dbus_t. > > > > > > > > Since libvirt would be responsible for launching these private dbus > > > > daemons it would be easy to make it run svirt_dbus_t for example. > > > > Actually it would be svirt_dbus_t:s0:cNNN,cMMM to get uniqueness > > > > per VM. > > > > > > > > Will of course require us to talk to the SELinux maintainers to > > > > get some sensible policy rules created. > > > > > > This all relies on SELinux and running privileged qemu/vhost-user pairs; > > > needing to do that purely to enforce security seems wrong. > > > > Compare to an alternative bus-less solution where each helper has > > a direct UNIX socket connection to QEMU. > > > > If two helpers are running as the same user ID, then can still > > directly attack each other via things like ptrace or /proc/$PID/mem, > > unless you've used SELinux to isolate them, or run each as a distinct > > user ID. If you do the latter, then we can still easily isolate > > them using dbus. > > You can lock those down pretty easily though. How were you thinking ? If you're not using SELinux or separate user IDs, then AFAICT you've got a choice of using seccomp or containers. seccomp is really hard to get a useful policy out of with QEMU, and using containers for each helper process adds a level of complexity worse than selinux or separate user IDs, so isn't an obvious win over using dbus. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 15:05 ` Daniel P. Berrangé @ 2019-08-23 15:14 ` Dr. David Alan Gilbert 2019-08-23 15:21 ` Daniel P. Berrangé 0 siblings, 1 reply; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-23 15:14 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini * Daniel P. Berrangé (berrange@redhat.com) wrote: > On Fri, Aug 23, 2019 at 03:56:34PM +0100, Dr. David Alan Gilbert wrote: > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > On Fri, Aug 23, 2019 at 03:26:02PM +0100, Dr. David Alan Gilbert wrote: > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > On Fri, Aug 23, 2019 at 03:09:48PM +0100, Dr. David Alan Gilbert wrote: > > > > > > * Marc-André Lureau (marcandre.lureau@gmail.com) wrote: > > > > > > > Hi > > > > > > > > > > > > > > On Fri, Aug 23, 2019 at 5:00 PM Dr. David Alan Gilbert > > > > > > > <dgilbert@redhat.com> wrote: > > > > > > > > > > > > > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > > > > > > > > > > > > <snip> > > > > > > > > > > > > > > > > > This means QEMU still has to iterate over every single client > > > > > > > > > on the bus to identify them. If you're doing that, there's > > > > > > > > > no point in owning a well known service at all. Just iterate > > > > > > > > > over the unique bus names and look for the exported object > > > > > > > > > path /org/qemu/VMState > > > > > > > > > > > > > > > > > > > > > > > > > Not knowing anything about DBus security, I want to ask how do > > > > > > > > we handle security here? > > > > > > > > > > > > > > First of all, we are talking about cooperative processes, and having a > > > > > > > specific bus for each qemu instance. So some amount of security/trust > > > > > > > is already assumed. > > > > > > > > > > > > Some but we need to keep it as limited as possible; for example two > > > > > > reasons for having separate processes both come down to security: > > > > > > > > > > > > a) vtpm - however screwy the qemu is, you can never get to the keys in > > > > > > the vtpm > > > > > > > > > > Processes connected to dbus can only call the DBus APIs that vtpm > > > > > actually exports. The vtpm should simply *not* export a DBus > > > > > API that allows anything to fetch the keys. > > > > > > > > > > If it did want to export APIs for fetching keys, then we would > > > > > have to ensure suitable dbus /selinux policy was created to > > > > > prevent unwarranted access. > > > > > > > > This was really just one example of where the security/trust isn't > > > > assumed; however a more concrete case is migration of a vtpm, and even > > > > though it's probably encrypted blob you still don't want some other > > > > device to grab the migration data - or to say reinitialise the vtpm. > > > > > > That can be dealt with by the dbus security policies, provided > > > you either run the vtpm as a different user ID from the other > > > untrustworthy helpers, or use a different selinux context for > > > vtpm. You can then express that only the user that QEMU is > > > running under can talk to vtpm over dbus. > > > > The need for the extra user ID or selinux context is a pain; > > but probably warranted for the vTPM; in general though some of this > > exists because of the choice of DBus and wouldn't be a problem for > > something that had a point-to-point socket it sent everything over. > > NB be careful to use s/DBus/DBus bus/ > > DBus the protocol is fine to be used in a point-to-point socket > scenario - the use of the bus is strictly optional. > > If all communication we expect is exclusively Helper <-> QEMU, > then I'd argue in favour of dbus in point-to-point mode. > > The use cases Stefan brought up for virtiofsd though is what > I think brings the idea of using the bus relevant. It is the > desire to allow online control/mgmt of the helper, which > introduces a 3rd party which isn't QEMU. Instead either libvirt > or a standalone admin/debugging tool. With multiple parties > involved I think the bus becomes relevant > > With p2p mode you could have 2 dbus socket for Helper <-> QEMU > and another dbus socket for Helper <-> libvirt/debugging, but > this isn't an obvious security win over using the bus, as you > now need different access rules for each of the p2p sockets > to say who can connect to which socket. Right; point-2-point doesn't worry me much as long as we're careful; it's now we're suddenly proposing something much more general that I think we need to start being really careful. > > > Where I think you could have problems is if you needed finer > > > grainer control with selinux. eg if vstpm exports 2 different > > > services, you can't allow access to one service, but forbid > > > access to the other service. > > > > > > > > > b) virtio-gpu, loads of complex GPU code that can't break the main > > > > > > qemu process. > > > > > > > > > > That's no problem - virtio-gpu crashes, it disappears from the dbus > > > > > bus, but everything else keeps running. > > > > > > > > Crashing is the easy case; assume it's malicious and you don't want it > > > > getting to say a storage device provided by another vhost-user device. > > > > > > If we assume that the 2 processes can't commnuicate / access each > > > other outside DBus, then the attack avenues added by use of dbus > > > are most likely either: > > > > > > - invoking some DBus method that should not be allowed due > > > to incomplete dbus security policy. > > > > > > - finding a crash in a dbus client library that you can somehow > > > exploit to get remote code execution in the separate process > > > > > > I won't claim this is impossible, but I think it helps to be > > > using a standard, widely used battle tested RPC impl, rather > > > than a home grown RPC protocol. > > > > It's only the policy case I worry about; and my point here is if we > > decide to use dbus then we have to think properly about security and > > defined stuff. > > > > > > > > > > > > > > > But if necessary, dbus can enforce policies on who is allowed to own a > > > > > > > name, or to send/receive message from. As far as I know, this is > > > > > > > mostly user/group policies. > > > > > > > > > > > > > > But there is also SELinux checks to send_msg and acquire_svc (see > > > > > > > dbus-daemon(1)) > > > > > > > > > > > > But how does something like SELinux interact with a private dbus > > > > > > rather than the system dbus? > > > > > > > > > > There's already two dbus-daemon's on each host - the system one and > > > > > the session one, and they get different selinux contexts, > > > > > system_dbus_t and unconfined_dbus_t. > > > > > > > > > > Since libvirt would be responsible for launching these private dbus > > > > > daemons it would be easy to make it run svirt_dbus_t for example. > > > > > Actually it would be svirt_dbus_t:s0:cNNN,cMMM to get uniqueness > > > > > per VM. > > > > > > > > > > Will of course require us to talk to the SELinux maintainers to > > > > > get some sensible policy rules created. > > > > > > > > This all relies on SELinux and running privileged qemu/vhost-user pairs; > > > > needing to do that purely to enforce security seems wrong. > > > > > > Compare to an alternative bus-less solution where each helper has > > > a direct UNIX socket connection to QEMU. > > > > > > If two helpers are running as the same user ID, then can still > > > directly attack each other via things like ptrace or /proc/$PID/mem, > > > unless you've used SELinux to isolate them, or run each as a distinct > > > user ID. If you do the latter, then we can still easily isolate > > > them using dbus. > > > > You can lock those down pretty easily though. > > How were you thinking ? > > If you're not using SELinux or separate user IDs, then AFAICT you've > got a choice of using seccomp or containers. seccomp is really hard > to get a useful policy out of with QEMU, and using containers for > each helper process adds a level of complexity worse than selinux > or separate user IDs, so isn't an obvious win over using dbus. You can just drop the CAP_SYS_PTRACE on the whole lot for that; I thought there was something for /proc/.../mem as well. Dave > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 15:14 ` Dr. David Alan Gilbert @ 2019-08-23 15:21 ` Daniel P. Berrangé 2019-08-23 15:24 ` Dr. David Alan Gilbert 0 siblings, 1 reply; 28+ messages in thread From: Daniel P. Berrangé @ 2019-08-23 15:21 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini On Fri, Aug 23, 2019 at 04:14:48PM +0100, Dr. David Alan Gilbert wrote: > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > On Fri, Aug 23, 2019 at 03:56:34PM +0100, Dr. David Alan Gilbert wrote: > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > If two helpers are running as the same user ID, then can still > > > > directly attack each other via things like ptrace or /proc/$PID/mem, > > > > unless you've used SELinux to isolate them, or run each as a distinct > > > > user ID. If you do the latter, then we can still easily isolate > > > > them using dbus. > > > > > > You can lock those down pretty easily though. > > > > How were you thinking ? > > > > If you're not using SELinux or separate user IDs, then AFAICT you've > > got a choice of using seccomp or containers. seccomp is really hard > > to get a useful policy out of with QEMU, and using containers for > > each helper process adds a level of complexity worse than selinux > > or separate user IDs, so isn't an obvious win over using dbus. > > You can just drop the CAP_SYS_PTRACE on the whole lot for that; > I thought there was something for /proc/.../mem as well. If they're running the same user ID & not SELinux constrained, I don't think that trying to block PRACTE / /proc/$PID/mem offers a reassuring level of security separation, as there's still plenty of other files that will be readable & writable to both vhostuser helper daemons which can be leveraged as indirect attack vectors - auditing both helpers and every library they link to to ensure nothing can be exploited is very hard. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate 2019-08-23 15:21 ` Daniel P. Berrangé @ 2019-08-23 15:24 ` Dr. David Alan Gilbert 0 siblings, 0 replies; 28+ messages in thread From: Dr. David Alan Gilbert @ 2019-08-23 15:24 UTC (permalink / raw) To: Daniel P. Berrangé Cc: Laurent Vivier, Thomas Huth, Juan Quintela, qemu-devel, Marc-André Lureau, Stefan Hajnoczi, Paolo Bonzini * Daniel P. Berrangé (berrange@redhat.com) wrote: > On Fri, Aug 23, 2019 at 04:14:48PM +0100, Dr. David Alan Gilbert wrote: > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > On Fri, Aug 23, 2019 at 03:56:34PM +0100, Dr. David Alan Gilbert wrote: > > > > * Daniel P. Berrangé (berrange@redhat.com) wrote: > > > > > If two helpers are running as the same user ID, then can still > > > > > directly attack each other via things like ptrace or /proc/$PID/mem, > > > > > unless you've used SELinux to isolate them, or run each as a distinct > > > > > user ID. If you do the latter, then we can still easily isolate > > > > > them using dbus. > > > > > > > > You can lock those down pretty easily though. > > > > > > How were you thinking ? > > > > > > If you're not using SELinux or separate user IDs, then AFAICT you've > > > got a choice of using seccomp or containers. seccomp is really hard > > > to get a useful policy out of with QEMU, and using containers for > > > each helper process adds a level of complexity worse than selinux > > > or separate user IDs, so isn't an obvious win over using dbus. > > > > You can just drop the CAP_SYS_PTRACE on the whole lot for that; > > I thought there was something for /proc/.../mem as well. > > If they're running the same user ID & not SELinux constrained, I don't > think that trying to block PRACTE / /proc/$PID/mem offers a reassuring > level of security separation, as there's still plenty of other files > that will be readable & writable to both vhostuser helper daemons which > can be leveraged as indirect attack vectors - auditing both helpers and > every library they link to to ensure nothing can be exploited is very > hard. Still, it doesn't mean we shouldn't be careful about anything new we add. Dave > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2019-08-23 15:30 UTC | newest] Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-08-08 15:03 [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Marc-André Lureau 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 1/2] qemu-file: move qemu_{get, put}_counted_string() declarations Marc-André Lureau 2019-08-09 18:32 ` Dr. David Alan Gilbert 2019-08-08 15:03 ` [Qemu-devel] [PATCH v2 2/2] Add dbus-vmstate object Marc-André Lureau 2019-08-08 15:07 ` Marc-André Lureau 2019-08-22 10:55 ` Dr. David Alan Gilbert 2019-08-22 11:35 ` Marc-André Lureau 2019-08-22 11:41 ` Dr. David Alan Gilbert 2019-08-22 11:57 ` Marc-André Lureau 2019-08-22 12:19 ` Dr. David Alan Gilbert 2019-08-22 12:38 ` Marc-André Lureau 2019-08-22 12:51 ` Dr. David Alan Gilbert 2019-08-23 11:20 ` [Qemu-devel] [PATCH v2 0/2] Add dbus-vmstate Daniel P. Berrangé 2019-08-23 11:31 ` Marc-André Lureau 2019-08-23 11:41 ` Daniel P. Berrangé 2019-08-23 11:47 ` Marc-André Lureau 2019-08-23 13:00 ` Dr. David Alan Gilbert 2019-08-23 13:48 ` Marc-André Lureau 2019-08-23 14:09 ` Daniel P. Berrangé 2019-08-23 14:09 ` Dr. David Alan Gilbert 2019-08-23 14:20 ` Daniel P. Berrangé 2019-08-23 14:26 ` Dr. David Alan Gilbert 2019-08-23 14:40 ` Daniel P. Berrangé 2019-08-23 14:56 ` Dr. David Alan Gilbert 2019-08-23 15:05 ` Daniel P. Berrangé 2019-08-23 15:14 ` Dr. David Alan Gilbert 2019-08-23 15:21 ` Daniel P. Berrangé 2019-08-23 15:24 ` Dr. David Alan Gilbert
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).