From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: Steve Sistare <steven.sistare@oracle.com>
Cc: "Daniel P. Berrange" <berrange@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Juan Quintela" <quintela@redhat.com>,
qemu-devel@nongnu.org, "Markus Armbruster" <armbru@redhat.com>,
"Alex Williamson" <alex.williamson@redhat.com>,
"Stefan Hajnoczi" <stefanha@redhat.com>,
"Marc-André Lureau" <marcandre.lureau@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Philippe Mathieu-Daudé" <philmd@redhat.com>
Subject: Re: [PATCH V1 03/32] savevm: QMP command for cprsave
Date: Fri, 11 Sep 2020 17:43:38 +0100 [thread overview]
Message-ID: <20200911164338.GK3310@work-vm> (raw)
In-Reply-To: <1596122076-341293-4-git-send-email-steven.sistare@oracle.com>
* Steve Sistare (steven.sistare@oracle.com) wrote:
> To enable live reboot, provide the cprsave QMP command and the VMS_REBOOT
> vmstate-saving operation, which saves the state of the virtual machine in a
> simple file.
>
> Syntax:
> {'command':'cprsave', 'data':{'file':'str', 'mode':'str'}}
>
> The mode argument must be 'reboot'. Additional modes will be defined in
> the future.
>
> Unlike the savevm command, cprsave supports any type of guest image and
> block device. cprsave stops the VM so that guest ram and block devices are
> not modified after state is saved. Guest ram must be mapped to a persistent
> memory file such as /dev/dax0.0. The ram object vmstate handler and block
> device handler do not apply to VMS_REBOOT, so restrict them to VMS_MIGRATE
> or VMS_SNAPSHOT. After cprsave completes successfully, qemu exits.
>
> After issuing cprsave, the caller may update qemu, update the host kernel,
> reboot, start qemu using the same arguments as the original process, and
> issue the cprload command to restore the guest. cprload is added by
> subsequent patches.
>
> If the caller suspends the guest instead of stopping the VM, such as by
> issuing guest-suspend-ram to the qemu guest agent, then cprsave and cprload
> support guests with vfio devices. The guest drivers suspend methods flush
> outstanding requests and re-initialize the devices, and thus there is no
> device state to save and restore.
>
> Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
> Signed-off-by: Maran Wilson <maran.wilson@oracle.com>
Going back a step; could you.....
> ---
> include/migration/vmstate.h | 1 +
> include/sysemu/sysemu.h | 2 ++
> migration/block.c | 1 +
> migration/ram.c | 1 +
> migration/savevm.c | 59 +++++++++++++++++++++++++++++++++++++++++++++
> monitor/qmp-cmds.c | 6 +++++
> qapi/migration.json | 14 +++++++++++
> 7 files changed, 84 insertions(+)
>
> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> index fa575f9..c58551a 100644
> --- a/include/migration/vmstate.h
> +++ b/include/migration/vmstate.h
> @@ -161,6 +161,7 @@ typedef enum {
> typedef enum {
> VMS_MIGRATE = (1U << 1),
> VMS_SNAPSHOT = (1U << 2),
> + VMS_REBOOT = (1U << 3),
> VMS_MODE_ALL = ~0U
> } VMStateMode;
>
> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> index 4b6a5c4..6fe86e6 100644
> --- a/include/sysemu/sysemu.h
> +++ b/include/sysemu/sysemu.h
> @@ -24,6 +24,8 @@ extern bool machine_init_done;
> void qemu_add_machine_init_done_notifier(Notifier *notify);
> void qemu_remove_machine_init_done_notifier(Notifier *notify);
>
> +void save_cpr_snapshot(const char *file, const char *mode, Error **errp);
> +
> extern int autostart;
>
> typedef enum {
> diff --git a/migration/block.c b/migration/block.c
> index 737b649..a69accb 100644
> --- a/migration/block.c
> +++ b/migration/block.c
> @@ -1023,6 +1023,7 @@ static SaveVMHandlers savevm_block_handlers = {
> .load_state = block_load,
> .save_cleanup = block_migration_cleanup,
> .is_active = block_is_active,
> + .mode_mask = VMS_MIGRATE | VMS_SNAPSHOT,
> };
>
> void blk_mig_init(void)
> diff --git a/migration/ram.c b/migration/ram.c
> index 76d4fee..f0d5d9f 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -3795,6 +3795,7 @@ static SaveVMHandlers savevm_ram_handlers = {
> .load_setup = ram_load_setup,
> .load_cleanup = ram_load_cleanup,
> .resume_prepare = ram_resume_prepare,
> + .mode_mask = VMS_MIGRATE | VMS_SNAPSHOT,
> };
>
> void ram_mig_init(void)
> diff --git a/migration/savevm.c b/migration/savevm.c
> index ce02b6b..ff1a46e 100644
> --- a/migration/savevm.c
> +++ b/migration/savevm.c
> @@ -2680,6 +2680,65 @@ int qemu_load_device_state(QEMUFile *f)
> return 0;
> }
>
> +static QEMUFile *qf_file_open(const char *filename, int flags, int mode,
> + Error **errp)
> +{
> + QIOChannel *ioc;
> + int fd = qemu_open(filename, flags, mode);
> +
> + if (fd < 0) {
> + error_setg_errno(errp, errno, "%s(%s)", __func__, filename);
> + return NULL;
> + }
> +
> + ioc = QIO_CHANNEL(qio_channel_file_new_fd(fd));
> +
> + if (flags & O_WRONLY) {
> + return qemu_fopen_channel_output(ioc);
> + }
> +
> + return qemu_fopen_channel_input(ioc);
> +}
> +
> +void save_cpr_snapshot(const char *file, const char *mode, Error **errp)
> +{
> + int ret = 0;
> + QEMUFile *f;
> + VMStateMode op;
> +
> + if (!strcmp(mode, "reboot")) {
> + op = VMS_REBOOT;
> + } else {
> + error_setg(errp, "cprsave: bad mode %s", mode);
> + return;
> + }
> +
> + f = qf_file_open(file, O_CREAT | O_WRONLY | O_TRUNC, 0600, errp);
> + if (!f) {
> + return;
> + }
> +
> + ret = global_state_store();
> + if (ret) {
> + error_setg(errp, "Error saving global state");
> + qemu_fclose(f);
> + return;
> + }
> +
> + vm_stop(RUN_STATE_SAVE_VM);
> +
> + ret = qemu_savevm_state(f, op, errp);
> + if ((ret < 0) && !*errp) {
> + error_setg(errp, "qemu_savevm_state failed");
> + }
just call qemu_save_device_state(f) there rather than introducing the
modes?
What you're doing is VERY similar to qmp_xen_save_devices_state and also
COLO's device state saving.
(and also very similar to migration with the x-ignore-shared flag set).
Dave
> + qemu_fclose(f);
> +
> + if (op == VMS_REBOOT) {
> + no_shutdown = 0;
> + qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
> + }
> +}
> +
> int save_snapshot(const char *name, Error **errp)
> {
> BlockDriverState *bs, *bs1;
> diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
> index 864cbfa..9ec7b88 100644
> --- a/monitor/qmp-cmds.c
> +++ b/monitor/qmp-cmds.c
> @@ -35,6 +35,7 @@
> #include "qapi/qapi-commands-machine.h"
> #include "qapi/qapi-commands-misc.h"
> #include "qapi/qapi-commands-ui.h"
> +#include "qapi/qapi-commands-migration.h"
> #include "qapi/qmp/qerror.h"
> #include "hw/mem/memory-device.h"
> #include "hw/acpi/acpi_dev_interface.h"
> @@ -161,6 +162,11 @@ void qmp_cont(Error **errp)
> }
> }
>
> +void qmp_cprsave(const char *file, const char *mode, Error **errp)
> +{
> + save_cpr_snapshot(file, mode, errp);
> +}
> +
> void qmp_system_wakeup(Error **errp)
> {
> if (!qemu_wakeup_suspend_enabled()) {
> diff --git a/qapi/migration.json b/qapi/migration.json
> index d500055..b61df1d 100644
> --- a/qapi/migration.json
> +++ b/qapi/migration.json
> @@ -1621,3 +1621,17 @@
> ##
> { 'event': 'UNPLUG_PRIMARY',
> 'data': { 'device-id': 'str' } }
> +
> +##
> +# @cprsave:
> +#
> +# Create a checkpoint of the virtual machine device state in @file.
> +# Guest RAM and guest block device blocks are not saved.
> +#
> +# @file: name of checkpoint file
> +# @mode: 'reboot' : checkpoint can be cprload'ed after a host kexec reboot.
> +#
> +# Since 5.0
> +##
> +{ 'command': 'cprsave', 'data': { 'file': 'str', 'mode': 'str' } }
> +
> --
> 1.8.3.1
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
next prev parent reply other threads:[~2020-09-11 16:44 UTC|newest]
Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-30 15:14 [PATCH V1 00/32] Live Update Steve Sistare
2020-07-30 15:14 ` [PATCH V1 01/32] savevm: add vmstate handler iterators Steve Sistare
2020-09-11 16:24 ` Dr. David Alan Gilbert
2020-09-24 21:43 ` Steven Sistare
2020-09-25 9:07 ` Dr. David Alan Gilbert
2020-07-30 15:14 ` [PATCH V1 02/32] savevm: VM handlers mode mask Steve Sistare
2020-07-30 15:14 ` [PATCH V1 03/32] savevm: QMP command for cprsave Steve Sistare
2020-07-30 16:12 ` Eric Blake
2020-07-30 17:52 ` Steven Sistare
2020-09-11 16:43 ` Dr. David Alan Gilbert [this message]
2020-09-25 18:43 ` Steven Sistare
2020-09-25 22:22 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 04/32] savevm: HMP Command " Steve Sistare
2020-09-11 16:57 ` Dr. David Alan Gilbert
2020-09-24 21:44 ` Steven Sistare
2020-09-25 9:26 ` Dr. David Alan Gilbert
2020-07-30 15:14 ` [PATCH V1 05/32] savevm: QMP command for cprload Steve Sistare
2020-07-30 16:14 ` Eric Blake
2020-07-30 18:00 ` Steven Sistare
2020-09-11 17:18 ` Dr. David Alan Gilbert
2020-09-24 21:49 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 06/32] savevm: HMP Command " Steve Sistare
2020-07-30 15:14 ` [PATCH V1 07/32] savevm: QMP command for cprinfo Steve Sistare
2020-07-30 16:17 ` Eric Blake
2020-07-30 18:02 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 08/32] savevm: HMP " Steve Sistare
2020-09-11 17:27 ` Dr. David Alan Gilbert
2020-09-24 21:50 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 09/32] savevm: prevent cprsave if memory is volatile Steve Sistare
2020-09-11 17:35 ` Dr. David Alan Gilbert
2020-09-24 21:51 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 10/32] kvmclock: restore paused KVM clock Steve Sistare
2020-09-11 17:50 ` Dr. David Alan Gilbert
2020-09-25 18:07 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 11/32] cpu: disable ticks when suspended Steve Sistare
2020-09-11 17:53 ` Dr. David Alan Gilbert
2020-09-24 20:42 ` Steven Sistare
2020-09-25 9:03 ` Dr. David Alan Gilbert
2020-07-30 15:14 ` [PATCH V1 12/32] vl: pause option Steve Sistare
2020-07-30 16:20 ` Eric Blake
2020-07-30 18:11 ` Steven Sistare
2020-07-31 10:07 ` Daniel P. Berrangé
2020-07-31 15:18 ` Steven Sistare
2020-07-30 17:03 ` Alex Bennée
2020-07-30 18:14 ` Steven Sistare
2020-07-31 9:44 ` Alex Bennée
2020-09-11 17:59 ` Dr. David Alan Gilbert
2020-09-24 21:51 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 13/32] gdbstub: gdb support for suspended state Steve Sistare
2020-09-11 18:41 ` Dr. David Alan Gilbert
2020-09-24 21:51 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 14/32] savevm: VMS_RESTART and cprsave restart Steve Sistare
2020-07-30 16:22 ` Eric Blake
2020-07-30 18:14 ` Steven Sistare
2020-09-11 18:44 ` Dr. David Alan Gilbert
2020-09-24 21:44 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 15/32] vl: QEMU_START_FREEZE env var Steve Sistare
2020-09-11 18:49 ` Dr. David Alan Gilbert
2020-09-24 21:47 ` Steven Sistare
2020-09-25 15:52 ` Dr. David Alan Gilbert
2020-07-30 15:14 ` [PATCH V1 16/32] oslib: add qemu_clr_cloexec Steve Sistare
2020-09-11 18:52 ` Dr. David Alan Gilbert
2020-07-30 15:14 ` [PATCH V1 17/32] util: env var helpers Steve Sistare
2020-09-11 19:00 ` Dr. David Alan Gilbert
2020-09-24 21:52 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 18/32] osdep: import MADV_DOEXEC Steve Sistare
2020-08-17 18:30 ` Steven Sistare
2020-08-17 20:48 ` Alex Williamson
2020-08-17 21:20 ` Steven Sistare
2020-08-17 21:44 ` Alex Williamson
2020-08-18 2:42 ` Alex Williamson
2020-08-19 21:52 ` Steven Sistare
2020-08-24 22:30 ` Alex Williamson
2020-10-08 16:32 ` Steven Sistare
2020-10-15 20:36 ` Alex Williamson
2020-10-19 16:33 ` Steven Sistare
2020-10-26 18:28 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 19/32] memory: ram_block_add cosmetic changes Steve Sistare
2020-07-30 15:14 ` [PATCH V1 20/32] vl: add helper to request re-exec Steve Sistare
2020-07-30 15:14 ` [PATCH V1 21/32] exec, memory: exec(3) to restart Steve Sistare
2020-07-30 15:14 ` [PATCH V1 22/32] char: qio_channel_socket_accept reuse fd Steve Sistare
2020-09-15 17:33 ` Dr. David Alan Gilbert
2020-09-15 17:53 ` Daniel P. Berrangé
2020-09-24 21:54 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 23/32] char: save/restore chardev socket fds Steve Sistare
2020-07-30 15:14 ` [PATCH V1 24/32] ui: save/restore vnc " Steve Sistare
2020-07-31 9:06 ` Daniel P. Berrangé
2020-07-31 16:51 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 25/32] char: save/restore chardev pty fds Steve Sistare
2020-07-30 15:14 ` [PATCH V1 26/32] monitor: save/restore QMP negotiation status Steve Sistare
2020-07-30 15:14 ` [PATCH V1 27/32] vhost: reset vhost devices upon cprsave Steve Sistare
2020-07-30 15:14 ` [PATCH V1 28/32] char: restore terminal on restart Steve Sistare
2020-07-30 15:14 ` [PATCH V1 29/32] pci: export pci_update_mappings Steve Sistare
2020-07-30 15:14 ` [PATCH V1 30/32] vfio-pci: save and restore Steve Sistare
2020-08-06 10:22 ` Jason Zeng
2020-08-07 20:38 ` Steven Sistare
2020-08-10 3:50 ` Jason Zeng
2020-08-19 21:15 ` Steven Sistare
2020-08-20 10:33 ` Jason Zeng
2020-10-07 21:25 ` Steven Sistare
2020-07-30 15:14 ` [PATCH V1 31/32] vfio-pci: trace pci config Steve Sistare
2020-07-30 15:14 ` [PATCH V1 32/32] vfio-pci: improved tracing Steve Sistare
2020-09-15 18:49 ` Dr. David Alan Gilbert
2020-09-24 21:52 ` Steven Sistare
2020-07-30 16:52 ` [PATCH V1 00/32] Live Update Daniel P. Berrangé
2020-07-30 18:48 ` Steven Sistare
2020-07-31 8:53 ` Daniel P. Berrangé
2020-07-31 15:27 ` Steven Sistare
2020-07-31 15:52 ` Daniel P. Berrangé
2020-07-31 17:20 ` Steven Sistare
2020-08-11 19:08 ` Dr. David Alan Gilbert
2020-07-30 17:15 ` Paolo Bonzini
2020-07-30 19:09 ` Steven Sistare
2020-07-30 21:39 ` Paolo Bonzini
2020-07-31 19:22 ` Steven Sistare
2020-07-30 17:49 ` Dr. David Alan Gilbert
2020-07-30 19:31 ` Steven Sistare
2020-08-04 18:18 ` Steven Sistare
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200911164338.GK3310@work-vm \
--to=dgilbert@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=alex.williamson@redhat.com \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=marcandre.lureau@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=stefanha@redhat.com \
--cc=steven.sistare@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).