* [Qemu-devel] [PATCH 0/4] postcopy (& 1 test) patch for 2.7
@ 2016-04-27 19:08 Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 1/4] Postcopy: Avoid 0 length discards Dr. David Alan Gilbert (git)
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2016-04-27 19:08 UTC (permalink / raw)
To: qemu-devel, quintela, amit.shah; +Cc: aarcange, den, marcel.a
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Hi,
This is a small set of postcopy changes, the largest of which
is an x86 test for postcopy.
Andrea's libqtest change came about from running my test under very heavy
load.
The test includes a self contained migration workload that rapidly changes
RAM in a predictable fashion allowing us to end up in postcopy mode and
also to be able to check the contents of RAM. Note this sometimes fails
on Linux kernels 4.5 (and current 4.6) which have a KVM+THP bug.
Dave
Andrea Arcangeli (1):
tests: fix libqtest socket timeouts
Dr. David Alan Gilbert (3):
Postcopy: Avoid 0 length discards
Postcopy: Add stats on page requests
test: Postcopy
hmp.c | 4 +
include/migration/migration.h | 2 +
migration/migration.c | 59 +++---
migration/ram.c | 5 +-
qapi-schema.json | 6 +-
tests/Makefile | 2 +
tests/libqtest.c | 2 +-
tests/postcopy-test.c | 426 ++++++++++++++++++++++++++++++++++++++++++
8 files changed, 468 insertions(+), 38 deletions(-)
create mode 100644 tests/postcopy-test.c
--
2.5.5
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Qemu-devel] [PATCH 1/4] Postcopy: Avoid 0 length discards
2016-04-27 19:08 [Qemu-devel] [PATCH 0/4] postcopy (& 1 test) patch for 2.7 Dr. David Alan Gilbert (git)
@ 2016-04-27 19:08 ` Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests Dr. David Alan Gilbert (git)
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2016-04-27 19:08 UTC (permalink / raw)
To: qemu-devel, quintela, amit.shah; +Cc: aarcange, den, marcel.a
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
The discard code in migration/ram.c would send request for
zero length discards in the case where no discards were needed.
It doesn't appear to have had any bad effect.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/migration/ram.c b/migration/ram.c
index 3f05738..e96c2af 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1557,7 +1557,9 @@ static int postcopy_send_discard_bm_ram(MigrationState *ms,
} else {
discard_length = zero - one;
}
- postcopy_discard_send_range(ms, pds, one, discard_length);
+ if (discard_length) {
+ postcopy_discard_send_range(ms, pds, one, discard_length);
+ }
current = one + discard_length;
} else {
current = one;
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests
2016-04-27 19:08 [Qemu-devel] [PATCH 0/4] postcopy (& 1 test) patch for 2.7 Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 1/4] Postcopy: Avoid 0 length discards Dr. David Alan Gilbert (git)
@ 2016-04-27 19:08 ` Dr. David Alan Gilbert (git)
2016-04-27 19:18 ` Eric Blake
2016-04-27 19:08 ` [Qemu-devel] [PATCH 3/4] test: Postcopy Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 4/4] tests: fix libqtest socket timeouts Dr. David Alan Gilbert (git)
3 siblings, 1 reply; 7+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2016-04-27 19:08 UTC (permalink / raw)
To: qemu-devel, quintela, amit.shah; +Cc: aarcange, den, marcel.a
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
On the source, add a count of page requests received from the
destination.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
hmp.c | 4 +++
include/migration/migration.h | 2 ++
migration/migration.c | 59 ++++++++++++++++++-------------------------
migration/ram.c | 1 +
qapi-schema.json | 6 ++++-
5 files changed, 36 insertions(+), 36 deletions(-)
diff --git a/hmp.c b/hmp.c
index d510236..cd5fae3 100644
--- a/hmp.c
+++ b/hmp.c
@@ -209,6 +209,10 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
monitor_printf(mon, "dirty pages rate: %" PRIu64 " pages\n",
info->ram->dirty_pages_rate);
}
+ if (info->ram->postcopy_requests) {
+ monitor_printf(mon, "postcopy request count: %" PRIu64 "\n",
+ info->ram->postcopy_requests);
+ }
}
if (info->has_disk) {
diff --git a/include/migration/migration.h b/include/migration/migration.h
index ac2c12c..78fa59b 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -157,6 +157,8 @@ struct MigrationState
int64_t xbzrle_cache_size;
int64_t setup_time;
int64_t dirty_sync_count;
+ /* Count of requests incoming from destination */
+ int64_t postcopy_requests;
/* Flag set once the migration has been asked to enter postcopy */
bool start_postcopy;
diff --git a/migration/migration.c b/migration/migration.c
index 991313a..9d41618 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -561,6 +561,26 @@ static void get_xbzrle_cache_stats(MigrationInfo *info)
}
}
+static void populate_ram_info(MigrationInfo *info, MigrationState *s)
+{
+ info->has_ram = true;
+ info->ram = g_malloc0(sizeof(*info->ram));
+ info->ram->transferred = ram_bytes_transferred();
+ info->ram->total = ram_bytes_total();
+ info->ram->duplicate = dup_mig_pages_transferred();
+ info->ram->skipped = skipped_mig_pages_transferred();
+ info->ram->normal = norm_mig_pages_transferred();
+ info->ram->normal_bytes = norm_mig_bytes_transferred();
+ info->ram->mbps = s->mbps;
+ info->ram->dirty_sync_count = s->dirty_sync_count;
+ info->ram->postcopy_requests = s->postcopy_requests;
+
+ if (s->state != MIGRATION_STATUS_COMPLETED) {
+ info->ram->remaining = ram_bytes_remaining();
+ info->ram->dirty_pages_rate = s->dirty_pages_rate;
+ }
+}
+
MigrationInfo *qmp_query_migrate(Error **errp)
{
MigrationInfo *info = g_malloc0(sizeof(*info));
@@ -585,18 +605,7 @@ MigrationInfo *qmp_query_migrate(Error **errp)
info->has_setup_time = true;
info->setup_time = s->setup_time;
- info->has_ram = true;
- info->ram = g_malloc0(sizeof(*info->ram));
- info->ram->transferred = ram_bytes_transferred();
- info->ram->remaining = ram_bytes_remaining();
- info->ram->total = ram_bytes_total();
- info->ram->duplicate = dup_mig_pages_transferred();
- info->ram->skipped = skipped_mig_pages_transferred();
- info->ram->normal = norm_mig_pages_transferred();
- info->ram->normal_bytes = norm_mig_bytes_transferred();
- info->ram->dirty_pages_rate = s->dirty_pages_rate;
- info->ram->mbps = s->mbps;
- info->ram->dirty_sync_count = s->dirty_sync_count;
+ populate_ram_info(info, s);
if (blk_mig_active()) {
info->has_disk = true;
@@ -624,18 +633,7 @@ MigrationInfo *qmp_query_migrate(Error **errp)
info->has_setup_time = true;
info->setup_time = s->setup_time;
- info->has_ram = true;
- info->ram = g_malloc0(sizeof(*info->ram));
- info->ram->transferred = ram_bytes_transferred();
- info->ram->remaining = ram_bytes_remaining();
- info->ram->total = ram_bytes_total();
- info->ram->duplicate = dup_mig_pages_transferred();
- info->ram->skipped = skipped_mig_pages_transferred();
- info->ram->normal = norm_mig_pages_transferred();
- info->ram->normal_bytes = norm_mig_bytes_transferred();
- info->ram->dirty_pages_rate = s->dirty_pages_rate;
- info->ram->mbps = s->mbps;
- info->ram->dirty_sync_count = s->dirty_sync_count;
+ populate_ram_info(info, s);
if (blk_mig_active()) {
info->has_disk = true;
@@ -658,17 +656,7 @@ MigrationInfo *qmp_query_migrate(Error **errp)
info->has_setup_time = true;
info->setup_time = s->setup_time;
- info->has_ram = true;
- info->ram = g_malloc0(sizeof(*info->ram));
- info->ram->transferred = ram_bytes_transferred();
- info->ram->remaining = 0;
- info->ram->total = ram_bytes_total();
- info->ram->duplicate = dup_mig_pages_transferred();
- info->ram->skipped = skipped_mig_pages_transferred();
- info->ram->normal = norm_mig_pages_transferred();
- info->ram->normal_bytes = norm_mig_bytes_transferred();
- info->ram->mbps = s->mbps;
- info->ram->dirty_sync_count = s->dirty_sync_count;
+ populate_ram_info(info, s);
break;
case MIGRATION_STATUS_FAILED:
info->has_status = true;
@@ -946,6 +934,7 @@ MigrationState *migrate_init(const MigrationParams *params)
s->dirty_sync_count = 0;
s->start_postcopy = false;
s->postcopy_after_devices = false;
+ s->postcopy_requests = 0;
s->migration_thread_running = false;
s->last_req_rb = NULL;
diff --git a/migration/ram.c b/migration/ram.c
index e96c2af..eeb1902 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1169,6 +1169,7 @@ int ram_save_queue_pages(MigrationState *ms, const char *rbname,
{
RAMBlock *ramblock;
+ ms->postcopy_requests++;
rcu_read_lock();
if (!rbname) {
/* Reuse last RAMBlock */
diff --git a/qapi-schema.json b/qapi-schema.json
index 54634c4..8bc581e 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -382,13 +382,17 @@
#
# @dirty-sync-count: number of times that dirty ram was synchronized (since 2.1)
#
+# @postcopy-requests: The number of page requests received from the destination
+# (since 2.7)
+#
# Since: 0.14.0
##
{ 'struct': 'MigrationStats',
'data': {'transferred': 'int', 'remaining': 'int', 'total': 'int' ,
'duplicate': 'int', 'skipped': 'int', 'normal': 'int',
'normal-bytes': 'int', 'dirty-pages-rate' : 'int',
- 'mbps' : 'number', 'dirty-sync-count' : 'int' } }
+ 'mbps' : 'number', 'dirty-sync-count' : 'int',
+ 'postcopy-requests' : 'int' } }
##
# @XBZRLECacheStats
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [Qemu-devel] [PATCH 3/4] test: Postcopy
2016-04-27 19:08 [Qemu-devel] [PATCH 0/4] postcopy (& 1 test) patch for 2.7 Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 1/4] Postcopy: Avoid 0 length discards Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests Dr. David Alan Gilbert (git)
@ 2016-04-27 19:08 ` Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 4/4] tests: fix libqtest socket timeouts Dr. David Alan Gilbert (git)
3 siblings, 0 replies; 7+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2016-04-27 19:08 UTC (permalink / raw)
To: qemu-devel, quintela, amit.shah; +Cc: aarcange, den, marcel.a
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
This is a postcopy test (x86 only) that actually runs the guest
and checks the memory contents.
The test runs from an x86 boot block with the hex embedded in the test;
the source for this is:
...........
.code16
.org 0x7c00
.file "fill.s"
.text
.globl start
.type start, @function
start: # at 0x7c00 ?
cli
lgdt gdtdesc
mov $1,%eax
mov %eax,%cr0 # Protected mode enable
data32 ljmp $8,$0x7c20
.org 0x7c20
.code32
# A20 enable - not sure I actually need this
inb $0x92,%al
or $2,%al
outb %al, $0x92
# set up DS for the whole of RAM (needed on KVM)
mov $16,%eax
mov %eax,%ds
mov $65,%ax
mov $0x3f8,%dx
outb %al,%dx
# bl keeps a counter so we limit the output speed
mov $0, %bl
mainloop:
# Start from 1MB
mov $(1024*1024),%eax
innerloop:
incb (%eax)
add $4096,%eax
cmp $(100*1024*1024),%eax
jl innerloop
inc %bl
jnz mainloop
mov $66,%ax
mov $0x3f8,%dx
outb %al,%dx
jmp mainloop
# GDT magic from old (GPLv2) Grub startup.S
.p2align 2 /* force 4-byte alignment */
gdt:
.word 0, 0
.byte 0, 0, 0, 0
/* -- code segment --
* base = 0x00000000, limit = 0xFFFFF (4 KiB Granularity), present
* type = 32bit code execute/read, DPL = 0
*/
.word 0xFFFF, 0
.byte 0, 0x9A, 0xCF, 0
/* -- data segment --
* base = 0x00000000, limit 0xFFFFF (4 KiB Granularity), present
* type = 32 bit data read/write, DPL = 0
*/
.word 0xFFFF, 0
.byte 0, 0x92, 0xCF, 0
gdtdesc:
.word 0x27 /* limit */
.long gdt /* addr */
/* I'm a bootable disk */
.org 0x7dfe
.byte 0x55
.byte 0xAA
...........
and that can be assembled by the following magic:
as --32 -march=i486 fill.s -o fill.o
objcopy -O binary fill.o fill.boot
dd if=fill.boot of=bootsect bs=256 count=2 skip=124
xxd -i bootsect
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
tests/Makefile | 2 +
tests/postcopy-test.c | 426 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 428 insertions(+)
create mode 100644 tests/postcopy-test.c
diff --git a/tests/Makefile b/tests/Makefile
index 9194f18..f356f4a 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -224,6 +224,7 @@ endif
check-qtest-i386-y += tests/test-netfilter$(EXESUF)
check-qtest-i386-y += tests/test-filter-mirror$(EXESUF)
check-qtest-i386-y += tests/test-filter-redirector$(EXESUF)
+check-qtest-i386-y += tests/postcopy-test$(EXESUF)
check-qtest-x86_64-y = $(check-qtest-i386-y)
gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c
gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y))
@@ -579,6 +580,7 @@ tests/usb-hcd-uhci-test$(EXESUF): tests/usb-hcd-uhci-test.o $(libqos-usb-obj-y)
tests/usb-hcd-ehci-test$(EXESUF): tests/usb-hcd-ehci-test.o $(libqos-usb-obj-y)
tests/usb-hcd-xhci-test$(EXESUF): tests/usb-hcd-xhci-test.o $(libqos-usb-obj-y)
tests/pc-cpu-test$(EXESUF): tests/pc-cpu-test.o
+tests/postcopy-test$(EXESUF): tests/postcopy-test.o
tests/vhost-user-test$(EXESUF): tests/vhost-user-test.o qemu-char.o qemu-timer.o $(qtest-obj-y) $(test-io-obj-y)
tests/qemu-iotests/socket_scm_helper$(EXESUF): tests/qemu-iotests/socket_scm_helper.o
tests/test-qemu-opts$(EXESUF): tests/test-qemu-opts.o $(test-util-obj-y)
diff --git a/tests/postcopy-test.c b/tests/postcopy-test.c
new file mode 100644
index 0000000..c5343ff
--- /dev/null
+++ b/tests/postcopy-test.c
@@ -0,0 +1,426 @@
+/*
+ * QTest testcase for postcopy
+ *
+ * Copyright (c) 2016 Red Hat, Inc. and/or its affiliates
+ * based on the vhost-user-test.c that is:
+ * Copyright (c) 2014 Virtual Open Systems Sarl.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include <glib.h>
+
+#include "libqtest.h"
+#include "qemu/option.h"
+#include "qemu/range.h"
+#include "sysemu/char.h"
+#include "sysemu/sysemu.h"
+
+#include <sys/mman.h>
+#include <sys/vfs.h>
+#include <qemu/sockets.h>
+
+#if defined(__linux__)
+#include <sys/syscall.h>
+#endif
+
+#if defined(__linux__) && defined(__NR_userfaultfd) && defined(CONFIG_EVENTFD)
+#include <sys/eventfd.h>
+#include <sys/ioctl.h>
+#include <linux/userfaultfd.h>
+
+const unsigned start_address = 1024 * 1024;
+const unsigned end_address = 100 * 1024 * 1024;
+static bool ufd_version_check(void)
+{
+ struct uffdio_api api_struct;
+ uint64_t ioctl_mask;
+
+ int ufd = ufd = syscall(__NR_userfaultfd, O_CLOEXEC);
+
+ if (ufd == -1) {
+ g_test_message("Skipping test: userfaultfd not available");
+ return false;
+ }
+
+ api_struct.api = UFFD_API;
+ api_struct.features = 0;
+ if (ioctl(ufd, UFFDIO_API, &api_struct)) {
+ g_test_message("Skipping test: UFFDIO_API failed");
+ return false;
+ }
+
+ ioctl_mask = (__u64)1 << _UFFDIO_REGISTER |
+ (__u64)1 << _UFFDIO_UNREGISTER;
+ if ((api_struct.ioctls & ioctl_mask) != ioctl_mask) {
+ g_test_message("Skipping test: Missing userfault feature");
+ return false;
+ }
+
+ return true;
+}
+
+#else
+static bool ufd_version_check(void)
+{
+ g_test_message("Skipping test: Userfault not available (builtdtime)");
+ return false;
+}
+
+#endif
+
+static const char *tmpfs;
+
+/* A simple PC boot sector that modifies memory (1-100MB) quickly
+ * outputing a 'B' every so often if it's still running.
+ */
+unsigned char bootsect[] = {
+ 0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
+ 0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
+ 0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
+ 0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
+ 0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
+ 0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
+ 0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
+ 0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
+};
+
+/*
+ * Wait for some output in the serial output file,
+ * we get an 'A' followed by an endless string of 'B's
+ * but on the destination we won't have the A.
+ */
+static void wait_for_serial(const char *side)
+{
+ char *serialpath = g_strdup_printf("%s/%s", tmpfs, side);
+ FILE *serialfile = fopen(serialpath, "r");
+
+ do {
+ int readvalue = fgetc(serialfile);
+
+ switch (readvalue) {
+ case 'A':
+ /* Fine */
+ break;
+
+ case 'B':
+ /* It's alive! */
+ fclose(serialfile);
+ g_free(serialpath);
+ return;
+
+ case EOF:
+ fseek(serialfile, 0, SEEK_SET);
+ usleep(1000);
+ break;
+
+ default:
+ fprintf(stderr, "Unexpected %d on %s serial\n", readvalue, side);
+ assert(0);
+ }
+ } while (true);
+}
+
+/*
+ * It's tricky to use qemu's migration event capability with qtest,
+ * events suddenly appearing confuse the qmp()/hmp() responses.
+ * so wait for a couple of passes to have happened before
+ * going postcopy.
+ */
+
+static uint64_t get_migration_pass(void)
+{
+ QDict *rsp, *rsp_return, *rsp_ram;
+ uint64_t result;
+
+ rsp = qmp("{ 'execute': 'query-migrate' }");
+ if (!qdict_haskey(rsp, "return")) {
+ return 0;
+ }
+ rsp_return = qdict_get_qdict(rsp, "return");
+ if (!qdict_haskey(rsp_return, "ram")) {
+ /* Still in setup */
+ result = 0;
+ } else {
+ rsp_ram = qdict_get_qdict(rsp_return, "ram");
+ result = qdict_get_try_int(rsp_ram, "dirty-sync-count", 0);
+ QDECREF(rsp);
+ }
+ return result;
+}
+
+static void wait_for_migration_complete(void)
+{
+ QDict *rsp, *rsp_return;
+ bool completed;
+
+ do {
+ const char *status;
+
+ rsp = qmp("{ 'execute': 'query-migrate' }");
+ if (qdict_haskey(rsp, "return")) {
+ rsp_return = qdict_get_qdict(rsp, "return");
+ status = qdict_get_str(rsp_return, "status");
+ completed = strcmp(status, "completed") == 0;
+ assert(strcmp(status, "failed"));
+ }
+ QDECREF(rsp);
+ usleep(1000 * 100);
+ } while (!completed);
+}
+
+static void wait_for_migration_pass(void)
+{
+ uint64_t initial_pass = get_migration_pass();
+ uint64_t pass;
+
+ do {
+ usleep(1000 * 100);
+ pass = get_migration_pass();
+ } while (pass == initial_pass);
+}
+
+static void check_guests_ram(void)
+{
+ /* Our ASM test will have been incrementing one byte from each page from
+ * 1MB to <100MB in order.
+ * This gives us a constraint that any page's byte should be equal or less
+ * than the previous pages byte (mod 256); and they should all be equal
+ * except for one transition at the point where we meet the incrementer.
+ * (We're running this with the guest stopped).
+ */
+ unsigned address;
+ uint8_t first_byte;
+ uint8_t last_byte;
+ bool hit_edge = false;
+ bool bad = false;
+
+ qtest_memread(global_qtest, start_address, &first_byte, 1);
+ last_byte = first_byte;
+
+ for (address = start_address + 4096; address < end_address; address += 4096)
+ {
+ uint8_t b;
+ qtest_memread(global_qtest, address, &b, 1);
+ if (b != last_byte) {
+ if (((b + 1) % 256) == last_byte && !hit_edge) {
+ /* This is OK, the guest stopped at the point of
+ * incrementing the previous page but didn't get
+ * to us yet.
+ */
+ hit_edge = true;
+ } else {
+ fprintf(stderr, "Memory content inconsistency at %x"
+ " first_byte = %x last_byte = %x current = %x"
+ " hit_edge = %x\n",
+ address, first_byte, last_byte, b, hit_edge);
+ bad = true;
+ }
+ }
+ last_byte = b;
+ }
+ assert(!bad);
+}
+
+static void cleanup(const char *filename)
+{
+ char *path = g_strdup_printf("%s/%s", tmpfs, filename);
+
+ unlink(path);
+}
+
+static void test_migrate(void)
+{
+ char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+ QTestState *global = global_qtest, *from, *to;
+ unsigned char dest_byte_a, dest_byte_b, dest_byte_c, dest_byte_d;
+ gchar *cmd;
+ QDict *rsp;
+
+ char *bootpath = g_strdup_printf("%s/bootsect", tmpfs);
+ FILE *bootfile = fopen(bootpath, "wb");
+
+ assert(fwrite(bootsect, 512, 1, bootfile) == 1);
+ fclose(bootfile);
+
+ cmd = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
+ " -name pcsource,debug-threads=on"
+ " -serial file:%s/src_serial"
+ " -drive file=%s,format=raw",
+ tmpfs, bootpath);
+ from = qtest_start(cmd);
+ g_free(cmd);
+
+ cmd = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
+ " -name pcdest,debug-threads=on"
+ " -serial file:%s/dest_serial"
+ " -drive file=%s,format=raw"
+ " -incoming %s",
+ tmpfs, bootpath, uri);
+ to = qtest_init(cmd);
+ g_free(cmd);
+
+ global_qtest = from;
+ rsp = qmp("{ 'execute': 'migrate-set-capabilities',"
+ "'arguments': { "
+ "'capabilities': [ {"
+ "'capability': 'postcopy-ram',"
+ "'state': true } ] } }");
+ g_assert(qdict_haskey(rsp, "return"));
+ QDECREF(rsp);
+
+ global_qtest = to;
+ rsp = qmp("{ 'execute': 'migrate-set-capabilities',"
+ "'arguments': { "
+ "'capabilities': [ {"
+ "'capability': 'postcopy-ram',"
+ "'state': true } ] } }");
+ g_assert(qdict_haskey(rsp, "return"));
+ QDECREF(rsp);
+
+ /* We want to pick a speed slow enough that the test completes
+ * quickly, but that it doesn't complete precopy even on a slow
+ * machine, so also set the downtime.
+ */
+ global_qtest = from;
+ rsp = qmp("{ 'execute': 'migrate_set_speed',"
+ "'arguments': { 'value': 100000000 } }");
+ g_assert(qdict_haskey(rsp, "return"));
+ QDECREF(rsp);
+
+ /* 1ms downtime - it should never finish precopy */
+ rsp = qmp("{ 'execute': 'migrate_set_downtime',"
+ "'arguments': { 'value': 0.001 } }");
+ g_assert(qdict_haskey(rsp, "return"));
+ QDECREF(rsp);
+
+
+ /* Wait for the first serial output from the source */
+ wait_for_serial("src_serial");
+
+ cmd = g_strdup_printf("{ 'execute': 'migrate',"
+ "'arguments': { 'uri': '%s' } }",
+ uri);
+ rsp = qmp(cmd);
+ g_free(cmd);
+ g_assert(qdict_haskey(rsp, "return"));
+ QDECREF(rsp);
+
+ wait_for_migration_pass();
+
+ rsp = qmp("{ 'execute': 'migrate-start-postcopy' }");
+ g_assert(qdict_haskey(rsp, "return"));
+ QDECREF(rsp);
+
+ qmp_eventwait("STOP");
+
+ global_qtest = to;
+ qmp_eventwait("RESUME");
+
+ wait_for_serial("dest_serial");
+ global_qtest = from;
+ wait_for_migration_complete();
+
+ qtest_quit(from);
+
+ global_qtest = to;
+
+ qtest_memread(to, start_address, &dest_byte_a, 1);
+
+ /* Destination still running, wait for a byte to change */
+ do {
+ qtest_memread(to, start_address, &dest_byte_b, 1);
+ usleep(10 * 1000);
+ } while (dest_byte_a == dest_byte_b);
+
+ qmp("{ 'execute' : 'stop'}");
+ /* With it stopped, check nothing changes */
+ qtest_memread(to, start_address, &dest_byte_c, 1);
+ sleep(1);
+ qtest_memread(to, start_address, &dest_byte_d, 1);
+ assert(dest_byte_c == dest_byte_d);
+
+ check_guests_ram();
+
+ qtest_quit(to);
+ g_free(uri);
+
+ global_qtest = global;
+
+ cleanup("bootsect");
+ cleanup("migsocket");
+ cleanup("src_serial");
+ cleanup("dest_serial");
+}
+
+int main(int argc, char **argv)
+{
+ char template[] = "/tmp/postcopy-test-XXXXXX";
+ int ret;
+
+ g_test_init(&argc, &argv, NULL);
+
+ if (!ufd_version_check()) {
+ return 0;
+ }
+
+ tmpfs = mkdtemp(template);
+ if (!tmpfs) {
+ g_test_message("mkdtemp on path (%s): %s\n", template, strerror(errno));
+ }
+ g_assert(tmpfs);
+
+ module_call_init(MODULE_INIT_QOM);
+
+ qtest_add_func("/postcopy", test_migrate);
+
+ ret = g_test_run();
+
+ g_assert_cmpint(ret, ==, 0);
+
+ ret = rmdir(tmpfs);
+ if (ret != 0) {
+ g_test_message("unable to rmdir: path (%s): %s\n",
+ tmpfs, strerror(errno));
+ }
+
+ return ret;
+}
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [Qemu-devel] [PATCH 4/4] tests: fix libqtest socket timeouts
2016-04-27 19:08 [Qemu-devel] [PATCH 0/4] postcopy (& 1 test) patch for 2.7 Dr. David Alan Gilbert (git)
` (2 preceding siblings ...)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 3/4] test: Postcopy Dr. David Alan Gilbert (git)
@ 2016-04-27 19:08 ` Dr. David Alan Gilbert (git)
3 siblings, 0 replies; 7+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2016-04-27 19:08 UTC (permalink / raw)
To: qemu-devel, quintela, amit.shah; +Cc: aarcange, den, marcel.a
From: Andrea Arcangeli <aarcange@redhat.com>
I kept getting timeouts and unix socket accept failures under high
load, the patch fixes it.
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
tests/libqtest.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/libqtest.c b/tests/libqtest.c
index b12a9e4..57ce292 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -27,7 +27,7 @@
#include "qapi/qmp/qjson.h"
#define MAX_IRQ 256
-#define SOCKET_TIMEOUT 5
+#define SOCKET_TIMEOUT 50
QTestState *global_qtest;
--
2.5.5
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests
2016-04-27 19:08 ` [Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests Dr. David Alan Gilbert (git)
@ 2016-04-27 19:18 ` Eric Blake
2016-04-29 11:42 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 7+ messages in thread
From: Eric Blake @ 2016-04-27 19:18 UTC (permalink / raw)
To: Dr. David Alan Gilbert (git), qemu-devel, quintela, amit.shah
Cc: aarcange, den, marcel.a
[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]
On 04/27/2016 01:08 PM, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> On the source, add a count of page requests received from the
> destination.
>
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> hmp.c | 4 +++
> include/migration/migration.h | 2 ++
> migration/migration.c | 59 ++++++++++++++++++-------------------------
> migration/ram.c | 1 +
> qapi-schema.json | 6 ++++-
> 5 files changed, 36 insertions(+), 36 deletions(-)
>
> +++ b/migration/migration.c
> @@ -561,6 +561,26 @@ static void get_xbzrle_cache_stats(MigrationInfo *info)
> }
> }
>
> +static void populate_ram_info(MigrationInfo *info, MigrationState *s)
> +{
> + info->has_ram = true;
> + info->ram = g_malloc0(sizeof(*info->ram));
> @@ -585,18 +605,7 @@ MigrationInfo *qmp_query_migrate(Error **errp)
> info->has_setup_time = true;
> info->setup_time = s->setup_time;
>
> - info->has_ram = true;
> - info->ram = g_malloc0(sizeof(*info->ram));
If you respin, please split the refactoring into one patch, and the
addition of postcopy stats in another, so that the addition is not lost
in the noise.
Otherwise looks fine.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests
2016-04-27 19:18 ` Eric Blake
@ 2016-04-29 11:42 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 7+ messages in thread
From: Dr. David Alan Gilbert @ 2016-04-29 11:42 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, quintela, amit.shah, aarcange, den, marcel.a
* Eric Blake (eblake@redhat.com) wrote:
> On 04/27/2016 01:08 PM, Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> > On the source, add a count of page requests received from the
> > destination.
> >
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > ---
> > hmp.c | 4 +++
> > include/migration/migration.h | 2 ++
> > migration/migration.c | 59 ++++++++++++++++++-------------------------
> > migration/ram.c | 1 +
> > qapi-schema.json | 6 ++++-
> > 5 files changed, 36 insertions(+), 36 deletions(-)
> >
>
> > +++ b/migration/migration.c
> > @@ -561,6 +561,26 @@ static void get_xbzrle_cache_stats(MigrationInfo *info)
> > }
> > }
> >
> > +static void populate_ram_info(MigrationInfo *info, MigrationState *s)
> > +{
> > + info->has_ram = true;
> > + info->ram = g_malloc0(sizeof(*info->ram));
>
> > @@ -585,18 +605,7 @@ MigrationInfo *qmp_query_migrate(Error **errp)
> > info->has_setup_time = true;
> > info->setup_time = s->setup_time;
> >
> > - info->has_ram = true;
> > - info->ram = g_malloc0(sizeof(*info->ram));
>
> If you respin, please split the refactoring into one patch, and the
> addition of postcopy stats in another, so that the addition is not lost
> in the noise.
Done.
> Otherwise looks fine.
Thanks.
Dave
>
> --
> Eric Blake eblake redhat com +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-04-29 11:42 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-27 19:08 [Qemu-devel] [PATCH 0/4] postcopy (& 1 test) patch for 2.7 Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 1/4] Postcopy: Avoid 0 length discards Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 2/4] Postcopy: Add stats on page requests Dr. David Alan Gilbert (git)
2016-04-27 19:18 ` Eric Blake
2016-04-29 11:42 ` Dr. David Alan Gilbert
2016-04-27 19:08 ` [Qemu-devel] [PATCH 3/4] test: Postcopy Dr. David Alan Gilbert (git)
2016-04-27 19:08 ` [Qemu-devel] [PATCH 4/4] tests: fix libqtest socket timeouts Dr. David Alan Gilbert (git)
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.