From: "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com>
To: qemu-devel@nongnu.org
Cc: aarcange@redhat.com, yamahata@private.email.ne.jp,
quintela@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com,
david@gibson.dropbear.id.au, yayanghy@cn.fujitsu.com
Subject: [Qemu-devel] [PATCH v6 00/47] Postcopy implementation
Date: Tue, 14 Apr 2015 18:03:26 +0100 [thread overview]
Message-ID: <1429031053-4454-1-git-send-email-dgilbert@redhat.com> (raw)
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
This is the 6th cut of my version of postcopy; it is designed for use with
the Linux kernel additions posted by Andrea Arcangeli here:
git clone --reference linux -b userfault18
git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git
(Note this is a different API from the last version)
This qemu series can be found at:
https://github.com/orbitfp7/qemu.git
on the wp3-postcopy-v6 tag.
It addresses some but not yet all of the previous review comments;
however there are a couple of large simplifications, so it seems
worth posting to meet the new kernel API and to stop people
reviewing deadcode.
Note: That the userfaultfd.h header is no longer included in this
tree:
- if you're building with the appropriate kernel headers it should find it
- if you're building on a host that doesn't have the kernel headers
installed in the right place then:
configure with: --extra-cflags="-D__NR_userfaultfd=323"
cp include/uapi/linux/userfaultfd.h into somewhere in the include
path, e.g. /usr/local/include/linux
v6
Removed the PMI bitmaps
- Andrea updated the kernel API so that userspace doesn't
need to do wakeups, and thus QEMU doesn't need to keep
track of which pages it's received; there is a price - which
is we end up sending more dupes to the source, but it simplifies
stuff a lot and makes the normal paths a lot quicker.
(10s of line change in kernel, 10%-ish simplification in this code!)
Changed discard message format to a simpler start/end address scheme
and rework discard and chunking code to work in long's to match bitmap
'qemu_get_buffer_less_copy' for postcopy pages
- avoids a userspace copy since the kernel now does it
- the new qemufile interface might also be useful for other places that
don't need a copy (maybe xbzrle?)
Changed the blockingness of the incoming fd
it was incorrectly blocking during the precopy phase after a postcopy was
enabled, causing the HMP to be unavailable. It's now blocking only once
the postcopy thread starts up, since it's not a coroutine it can't deal
with the yields in qemu_file.
An error on the return-path now marks the migration as failed
Fixups from Dave Gibson's comments
Removed can_postcopy, renamed save_complete to save_complete_precopy
added save_complete_postcopy
Simplified loadvm loop exits
discard message format changes above
and many more smaller changes.
small fixups for RCU
This work has been partially funded by the EU Orbit project:
see http://www.orbitproject.eu/about/
TODO:
The major work is to rework the page send/receive loops so that supporting
larger host pages doesn't make it quite as messy.
Dr. David Alan Gilbert (47):
Start documenting how postcopy works.
Split header writing out of qemu_savevm_state_begin
qemu_ram_foreach_block: pass up error value, and down the ramblock
name
Add qemu_get_counted_string to read a string prefixed by a count byte
Create MigrationIncomingState
Provide runtime Target page information
Move copy out of qemu_peek_buffer
Add qemu_get_buffer_less_copy to avoid copies some of the time
Add wrapper for setting blocking status on a QEMUFile
Rename save_live_complete to save_live_complete_precopy
Return path: Open a return path on QEMUFile for sockets
Return path: socket_writev_buffer: Block even on non-blocking fd's
Migration commands
Return path: Control commands
Return path: Send responses from destination to source
Return path: Source handling of return path
ram_debug_dump_bitmap: Dump a migration bitmap as text
Move loadvm_handlers into MigrationIncomingState
Rework loadvm path for subloops
Add migration-capability boolean for postcopy-ram.
Add wrappers and handlers for sending/receiving the postcopy-ram
migration messages.
MIG_CMD_PACKAGED: Send a packaged chunk of migration stream
migrate_init: Call from savevm
Modify save_live_pending for postcopy
postcopy: OS support test
migrate_start_postcopy: Command to trigger transition to postcopy
MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state
Add qemu_savevm_state_complete_postcopy
Postcopy: Maintain sentmap and calculate discard
postcopy: Incoming initialisation
postcopy: ram_enable_notify to switch on userfault
Postcopy: Postcopy startup in migration thread
Postcopy end in migration_thread
Page request: Add MIG_RP_MSG_REQ_PAGES reverse command
Page request: Process incoming page request
Page request: Consume pages off the post-copy queue
postcopy_ram.c: place_page and helpers
Postcopy: Use helpers to map pages during migration
qemu_ram_block_from_host
Don't sync dirty bitmaps in postcopy
Host page!=target page: Cleanup bitmaps
Postcopy; Handle userfault requests
Start up a postcopy/listener thread ready for incoming page data
postcopy: Wire up loadvm_postcopy_handle_ commands
End of migration for postcopy
Disable mlock around incoming postcopy
Inhibit ballooning during postcopy
arch_init.c | 868 ++++++++++++++++++++++++++++++++++++---
balloon.c | 11 +
docs/migration.txt | 167 ++++++++
exec.c | 74 +++-
hmp-commands.hx | 15 +
hmp.c | 7 +
hmp.h | 1 +
hw/ppc/spapr.c | 2 +-
hw/virtio/virtio-balloon.c | 4 +-
include/exec/cpu-all.h | 2 -
include/exec/cpu-common.h | 7 +-
include/migration/migration.h | 126 +++++-
include/migration/postcopy-ram.h | 88 ++++
include/migration/qemu-file.h | 15 +-
include/migration/vmstate.h | 10 +-
include/qemu/typedefs.h | 5 +
include/sysemu/balloon.h | 2 +
include/sysemu/sysemu.h | 45 +-
migration/Makefile.objs | 2 +-
migration/block.c | 9 +-
migration/migration.c | 743 +++++++++++++++++++++++++++++++--
migration/postcopy-ram.c | 715 ++++++++++++++++++++++++++++++++
migration/qemu-file-unix.c | 106 ++++-
migration/qemu-file.c | 100 ++++-
migration/rdma.c | 4 +-
migration/vmstate.c | 5 +-
qapi-schema.json | 19 +-
qmp-commands.hx | 19 +
savevm.c | 809 ++++++++++++++++++++++++++++++++----
trace-events | 77 +++-
30 files changed, 3832 insertions(+), 225 deletions(-)
create mode 100644 include/migration/postcopy-ram.h
create mode 100644 migration/postcopy-ram.c
--
2.1.0
next reply other threads:[~2015-04-14 17:04 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-14 17:03 Dr. David Alan Gilbert (git) [this message]
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 01/47] Start documenting how postcopy works Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 02/47] Split header writing out of qemu_savevm_state_begin Dr. David Alan Gilbert (git)
2015-05-11 11:16 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 03/47] qemu_ram_foreach_block: pass up error value, and down the ramblock name Dr. David Alan Gilbert (git)
2015-05-15 10:38 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 04/47] Add qemu_get_counted_string to read a string prefixed by a count byte Dr. David Alan Gilbert (git)
2015-05-15 13:50 ` Amit Shah
2015-05-15 14:06 ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 05/47] Create MigrationIncomingState Dr. David Alan Gilbert (git)
2015-05-18 6:58 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 06/47] Provide runtime Target page information Dr. David Alan Gilbert (git)
2015-05-18 7:06 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 07/47] Move copy out of qemu_peek_buffer Dr. David Alan Gilbert (git)
2015-05-21 6:47 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 08/47] Add qemu_get_buffer_less_copy to avoid copies some of the time Dr. David Alan Gilbert (git)
2015-05-21 7:09 ` Amit Shah
2015-05-21 8:45 ` Dr. David Alan Gilbert
2015-05-21 8:58 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 09/47] Add wrapper for setting blocking status on a QEMUFile Dr. David Alan Gilbert (git)
2015-05-18 7:35 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 10/47] Rename save_live_complete to save_live_complete_precopy Dr. David Alan Gilbert (git)
2015-05-18 7:35 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 11/47] Return path: Open a return path on QEMUFile for sockets Dr. David Alan Gilbert (git)
2015-06-10 9:00 ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 12/47] Return path: socket_writev_buffer: Block even on non-blocking fd's Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 13/47] Migration commands Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 14/47] Return path: Control commands Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 15/47] Return path: Send responses from destination to source Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 16/47] Return path: Source handling of return path Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 17/47] ram_debug_dump_bitmap: Dump a migration bitmap as text Dr. David Alan Gilbert (git)
2015-05-21 9:21 ` Amit Shah
2015-05-21 10:10 ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 18/47] Move loadvm_handlers into MigrationIncomingState Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 19/47] Rework loadvm path for subloops Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 20/47] Add migration-capability boolean for postcopy-ram Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 21/47] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 22/47] MIG_CMD_PACKAGED: Send a packaged chunk of migration stream Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 23/47] migrate_init: Call from savevm Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 24/47] Modify save_live_pending for postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 25/47] postcopy: OS support test Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 26/47] migrate_start_postcopy: Command to trigger transition to postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:38 ` Eric Blake
2015-04-14 17:40 ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 27/47] MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state Dr. David Alan Gilbert (git)
2015-04-14 17:40 ` Eric Blake
2015-04-14 18:00 ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 28/47] Add qemu_savevm_state_complete_postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 29/47] Postcopy: Maintain sentmap and calculate discard Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 30/47] postcopy: Incoming initialisation Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 31/47] postcopy: ram_enable_notify to switch on userfault Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 32/47] Postcopy: Postcopy startup in migration thread Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 33/47] Postcopy end in migration_thread Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 34/47] Page request: Add MIG_RP_MSG_REQ_PAGES reverse command Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 35/47] Page request: Process incoming page request Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 36/47] Page request: Consume pages off the post-copy queue Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 37/47] postcopy_ram.c: place_page and helpers Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 38/47] Postcopy: Use helpers to map pages during migration Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 39/47] qemu_ram_block_from_host Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 40/47] Don't sync dirty bitmaps in postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 41/47] Host page!=target page: Cleanup bitmaps Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 42/47] Postcopy; Handle userfault requests Dr. David Alan Gilbert (git)
2015-05-25 9:18 ` zhanghailiang
2015-05-26 9:50 ` Dr. David Alan Gilbert
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 43/47] Start up a postcopy/listener thread ready for incoming page data Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 44/47] postcopy: Wire up loadvm_postcopy_handle_ commands Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 45/47] End of migration for postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 46/47] Disable mlock around incoming postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 47/47] Inhibit ballooning during postcopy Dr. David Alan Gilbert (git)
2015-04-27 8:04 ` [Qemu-devel] [PATCH v6 00/47] Postcopy implementation Li, Liang Z
2015-04-29 17:23 ` Dr. David Alan Gilbert
2015-04-30 1:09 ` Li, Liang Z
[not found] ` <20150505150112.GM2126@work-vm>
[not found] ` <F2CBF3009FA73547804AE4C663CAB28E50F0E1@shsmsx102.ccr.corp.intel.com>
[not found] ` <20150506083056.GB2204@work-vm>
2015-05-07 1:21 ` Li, Liang Z
2015-05-07 8:01 ` Dr. David Alan Gilbert
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1429031053-4454-1-git-send-email-dgilbert@redhat.com \
--to=dgilbert@redhat.com \
--cc=aarcange@redhat.com \
--cc=amit.shah@redhat.com \
--cc=david@gibson.dropbear.id.au \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=yamahata@private.email.ne.jp \
--cc=yayanghy@cn.fujitsu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.