[Qemu-devel] [PATCH 0/6] Add make check tests for Migration

[Qemu-devel] [PATCH 0/6] Add make check tests for Migration

[Juan Quintela]

His this series:
- add test for precopy for unix/tcp
  exec and fd to came, don't know how to test rdma without hardware
- add tests using deprecated interfaces
- add test for xbzrle
  Note to myself, there is no way to set the cache size with migraton_set_parameters
- Add test for compress threads
  disabled on the series, right now it appears that compression is not working at all
- Move postcopy to use new results
  Idea is to move it on top of migration-test.c, but first I want some reviews on basic idea

Please, review.

Juan.


Juan Quintela (6):
  tests: Add basic migration precopy test
  tests: Add basic migration precopy tcp test
  tests: Add precopy test using deprecated commands
  tests: Add migration xbzrle test
  tests: Add migration compress threads tests
  tests: Move postcopy migration test to migrate-set-parameters

 tests/Makefile.include |   3 +
 tests/migration-test.c | 707 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/postcopy-test.c  |   8 +-
 3 files changed, 714 insertions(+), 4 deletions(-)
 create mode 100644 tests/migration-test.c

-- 
2.13.5

[Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/Makefile.include |   3 +
 tests/migration-test.c | 497 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 500 insertions(+)
 create mode 100644 tests/migration-test.c

diff --git a/tests/Makefile.include b/tests/Makefile.include
index abc6707ef2..d23ca1ed1c 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -287,6 +287,7 @@ endif
 check-qtest-i386-$(CONFIG_SLIRP) += tests/test-netfilter$(EXESUF)
 check-qtest-i386-$(CONFIG_POSIX) += tests/test-filter-mirror$(EXESUF)
 check-qtest-i386-$(CONFIG_POSIX) += tests/test-filter-redirector$(EXESUF)
+check-qtest-i386-y += tests/migration-test$(EXESUF)
 check-qtest-i386-y += tests/postcopy-test$(EXESUF)
 check-qtest-i386-y += tests/test-x86-cpuid-compat$(EXESUF)
 check-qtest-i386-y += tests/numa-test$(EXESUF)
@@ -316,6 +317,7 @@ check-qtest-ppc64-y += tests/prom-env-test$(EXESUF)
 check-qtest-ppc64-y += tests/pnv-xscom-test$(EXESUF)
 check-qtest-ppc64-y += tests/drive_del-test$(EXESUF)
 check-qtest-ppc64-y += tests/postcopy-test$(EXESUF)
+check-qtest-ppc64-y += tests/migration-test$(EXESUF)
 check-qtest-ppc64-y += tests/boot-serial-test$(EXESUF)
 check-qtest-ppc64-y += tests/rtas-test$(EXESUF)
 check-qtest-ppc64-$(CONFIG_SLIRP) += tests/pxe-test$(EXESUF)
@@ -782,6 +784,7 @@ tests/usb-hcd-ehci-test$(EXESUF): tests/usb-hcd-ehci-test.o $(libqos-usb-obj-y)
 tests/usb-hcd-xhci-test$(EXESUF): tests/usb-hcd-xhci-test.o $(libqos-usb-obj-y)
 tests/pc-cpu-test$(EXESUF): tests/pc-cpu-test.o
 tests/postcopy-test$(EXESUF): tests/postcopy-test.o
+tests/migration-test$(EXESUF): tests/migration-test.o
 tests/vhost-user-test$(EXESUF): tests/vhost-user-test.o $(test-util-obj-y) \
 	$(qtest-obj-y) $(test-io-obj-y) $(libqos-virtio-obj-y) $(libqos-pc-obj-y) \
 	$(chardev-obj-y)
diff --git a/tests/migration-test.c b/tests/migration-test.c
new file mode 100644
index 0000000000..cd954caee4
--- /dev/null
+++ b/tests/migration-test.c
@@ -0,0 +1,497 @@
+/*
+ * QTest testcases for migration
+ *
+ * Copyright (c) 2017 Red Hat, Inc. and/or its affiliates
+ *   based on the postcopy-test.c which is based
+ *   based on the vhost-user-test.c that is:
+ *      Copyright (c) 2014 Virtual Open Systems Sarl.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+
+#include "libqtest.h"
+#include "qemu/option.h"
+#include "qemu/range.h"
+#include "qemu/sockets.h"
+#include "chardev/char.h"
+#include "sysemu/sysemu.h"
+#include "hw/nvram/chrp_nvram.h"
+
+#define MIN_NVRAM_SIZE 8192 /* from spapr_nvram.c */
+
+const unsigned start_address = 1024 * 1024;
+const unsigned end_address = 100 * 1024 * 1024;
+bool got_stop;
+
+#if defined(__linux__)
+#include <sys/syscall.h>
+#include <sys/vfs.h>
+#endif
+
+static const char *tmpfs;
+
+/* A simple PC boot sector that modifies memory (1-100MB) quickly
+ * outputing a 'B' every so often if it's still running.
+ */
+unsigned char bootsect[] = {
+  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
+  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
+  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
+  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
+  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
+  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
+  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
+  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
+};
+
+static void init_bootfile_x86(const char *bootpath)
+{
+    FILE *bootfile = fopen(bootpath, "wb");
+
+    g_assert_cmpint(fwrite(bootsect, 512, 1, bootfile), ==, 1);
+    fclose(bootfile);
+}
+
+static void init_bootfile_ppc(const char *bootpath)
+{
+    FILE *bootfile;
+    char buf[MIN_NVRAM_SIZE];
+    ChrpNvramPartHdr *header = (ChrpNvramPartHdr *)buf;
+
+    memset(buf, 0, MIN_NVRAM_SIZE);
+
+    /* Create a "common" partition in nvram to store boot-command property */
+
+    header->signature = CHRP_NVPART_SYSTEM;
+    memcpy(header->name, "common", 6);
+    chrp_nvram_finish_partition(header, MIN_NVRAM_SIZE);
+
+    /* FW_MAX_SIZE is 4MB, but slof.bin is only 900KB,
+     * so let's modify memory between 1MB and 100MB
+     * to do like PC bootsector
+     */
+
+    sprintf(buf + 16,
+            "boot-command=hex .\" _\" begin %x %x do i c@ 1 + i c! 1000 +loop "
+            ".\" B\" 0 until", end_address, start_address);
+
+    /* Write partition to the NVRAM file */
+
+    bootfile = fopen(bootpath, "wb");
+    g_assert_cmpint(fwrite(buf, MIN_NVRAM_SIZE, 1, bootfile), ==, 1);
+    fclose(bootfile);
+}
+
+/*
+ * Wait for some output in the serial output file,
+ * we get an 'A' followed by an endless string of 'B's
+ * but on the destination we won't have the A.
+ */
+static void wait_for_serial(const char *side)
+{
+    char *serialpath = g_strdup_printf("%s/%s", tmpfs, side);
+    FILE *serialfile = fopen(serialpath, "r");
+    const char *arch = qtest_get_arch();
+    int started = (strcmp(side, "src_serial") == 0 &&
+                   strcmp(arch, "ppc64") == 0) ? 0 : 1;
+
+    g_free(serialpath);
+    do {
+        int readvalue = fgetc(serialfile);
+
+        if (!started) {
+            /* SLOF prints its banner before starting test,
+             * to ignore it, mark the start of the test with '_',
+             * ignore all characters until this marker
+             */
+            switch (readvalue) {
+            case '_':
+                started = 1;
+                break;
+            case EOF:
+                fseek(serialfile, 0, SEEK_SET);
+                usleep(1000);
+                break;
+            }
+            continue;
+        }
+        switch (readvalue) {
+        case 'A':
+            /* Fine */
+            break;
+
+        case 'B':
+            /* It's alive! */
+            fclose(serialfile);
+            return;
+
+        case EOF:
+            started = (strcmp(side, "src_serial") == 0 &&
+                       strcmp(arch, "ppc64") == 0) ? 0 : 1;
+            fseek(serialfile, 0, SEEK_SET);
+            usleep(1000);
+            break;
+
+        default:
+            fprintf(stderr, "Unexpected %d on %s serial\n", readvalue, side);
+            g_assert_not_reached();
+        }
+    } while (true);
+}
+
+/*
+ * Events can get in the way of responses we are actually waiting for.
+ */
+static QDict *return_or_event(QDict *response, QTestState *who)
+{
+    const char *event_string;
+    if (!qdict_haskey(response, "event")) {
+        return response;
+    }
+
+    /* OK, it was an event */
+    event_string = qdict_get_str(response, "event");
+    if (!strcmp(event_string, "STOP")) {
+        got_stop = true;
+    }
+    QDECREF(response);
+    return return_or_event(qtest_qmp_receive(who), who);
+}
+
+
+/*
+ * It's tricky to use qemu's migration event capability with qtest,
+ * events suddenly appearing confuse the qmp()/hmp() responses.
+ * so wait for a couple of passes to have happened before
+ * going postcopy.
+ */
+
+static uint64_t get_migration_pass(QTestState *who)
+{
+    QDict *rsp, *rsp_return, *rsp_ram;
+    uint64_t result;
+
+    rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
+    rsp_return = qdict_get_qdict(rsp, "return");
+    if (!qdict_haskey(rsp_return, "ram")) {
+        /* Still in setup */
+        result = 0;
+    } else {
+        rsp_ram = qdict_get_qdict(rsp_return, "ram");
+        result = qdict_get_try_int(rsp_ram, "dirty-sync-count", 0);
+    }
+    QDECREF(rsp);
+    return result;
+}
+
+static void wait_for_migration_complete(QTestState *who)
+{
+    bool completed;
+
+    do {
+        QDict *rsp, *rsp_return;
+        const char *status;
+
+        rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
+        rsp_return = qdict_get_qdict(rsp, "return");
+        status = qdict_get_str(rsp_return, "status");
+        completed = strcmp(status, "completed") == 0;
+        g_assert_cmpstr(status, !=,  "failed");
+        QDECREF(rsp);
+        usleep(1000 * 100);
+    } while (!completed);
+}
+
+static void wait_for_migration_pass(QTestState *who)
+{
+    uint64_t initial_pass = get_migration_pass(who);
+    uint64_t pass;
+
+    /* Wait for the 1st sync */
+    do {
+        initial_pass = get_migration_pass(who);
+        if (got_stop || initial_pass) {
+            break;
+        }
+        usleep(1000 * 100);
+    } while (true);
+
+    do {
+        usleep(1000 * 100);
+        pass = get_migration_pass(who);
+    } while (pass == initial_pass && !got_stop);
+}
+
+static void check_guests_ram(QTestState *who)
+{
+    /* Our ASM test will have been incrementing one byte from each page from
+     * 1MB to <100MB in order.
+     * This gives us a constraint that any page's byte should be equal or less
+     * than the previous pages byte (mod 256); and they should all be equal
+     * except for one transition at the point where we meet the incrementer.
+     * (We're running this with the guest stopped).
+     */
+    unsigned address;
+    uint8_t first_byte;
+    uint8_t last_byte;
+    bool hit_edge = false;
+    bool bad = false;
+
+    qtest_memread(who, start_address, &first_byte, 1);
+    last_byte = first_byte;
+
+    for (address = start_address + 4096; address < end_address; address += 4096)
+    {
+        uint8_t b;
+        qtest_memread(who, address, &b, 1);
+        if (b != last_byte) {
+            if (((b + 1) % 256) == last_byte && !hit_edge) {
+                /* This is OK, the guest stopped at the point of
+                 * incrementing the previous page but didn't get
+                 * to us yet.
+                 */
+                hit_edge = true;
+            } else {
+                fprintf(stderr, "Memory content inconsistency at %x"
+                                " first_byte = %x last_byte = %x current = %x"
+                                " hit_edge = %x\n",
+                                address, first_byte, last_byte, b, hit_edge);
+                bad = true;
+            }
+        }
+        last_byte = b;
+    }
+    g_assert_false(bad);
+}
+
+static void cleanup(const char *filename)
+{
+    char *path = g_strdup_printf("%s/%s", tmpfs, filename);
+
+    unlink(path);
+    g_free(path);
+}
+
+static void test_migrate_start(QTestState **from, QTestState **to,
+                               const char *uri)
+{
+    gchar *cmd_src, *cmd_dst;
+    char *bootpath = g_strdup_printf("%s/bootsect", tmpfs);
+    const char *arch = qtest_get_arch();
+
+    got_stop = false;
+
+    if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) {
+        init_bootfile_x86(bootpath);
+        cmd_src = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
+                                  " -name pcsource,debug-threads=on"
+                                  " -serial file:%s/src_serial"
+                                  " -drive file=%s,format=raw",
+                                  tmpfs, bootpath);
+        cmd_dst = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
+                                  " -name pcdest,debug-threads=on"
+                                  " -serial file:%s/dest_serial"
+                                  " -drive file=%s,format=raw"
+                                  " -incoming %s",
+                                  tmpfs, bootpath, uri);
+    } else if (strcmp(arch, "ppc64") == 0) {
+        const char *accel;
+
+        /* On ppc64, the test only works with kvm-hv, but not with kvm-pr */
+        accel = access("/sys/module/kvm_hv", F_OK) ? "tcg" : "kvm:tcg";
+        init_bootfile_ppc(bootpath);
+        cmd_src = g_strdup_printf("-machine accel=%s -m 256M"
+                                  " -name pcsource,debug-threads=on"
+                                  " -serial file:%s/src_serial"
+                                  " -drive file=%s,if=pflash,format=raw",
+                                  accel, tmpfs, bootpath);
+        cmd_dst = g_strdup_printf("-machine accel=%s -m 256M"
+                                  " -name pcdest,debug-threads=on"
+                                  " -serial file:%s/dest_serial"
+                                  " -incoming %s",
+                                  accel, tmpfs, uri);
+    } else {
+        g_assert_not_reached();
+    }
+
+    g_free(bootpath);
+
+    *from = qtest_start(cmd_src);
+    g_free(cmd_src);
+
+    *to = qtest_init(cmd_dst);
+    g_free(cmd_dst);
+}
+
+static void test_migrate_end(QTestState *from, QTestState *to)
+{
+    unsigned char dest_byte_a, dest_byte_b, dest_byte_c, dest_byte_d;
+
+    qtest_quit(from);
+
+    qtest_memread(to, start_address, &dest_byte_a, 1);
+
+    /* Destination still running, wait for a byte to change */
+    do {
+        qtest_memread(to, start_address, &dest_byte_b, 1);
+        usleep(10 * 1000);
+    } while (dest_byte_a == dest_byte_b);
+
+    qtest_qmp_discard_response(to, "{ 'execute' : 'stop'}");
+    /* With it stopped, check nothing changes */
+    qtest_memread(to, start_address, &dest_byte_c, 1);
+    sleep(1);
+    qtest_memread(to, start_address, &dest_byte_d, 1);
+    g_assert_cmpint(dest_byte_c, ==, dest_byte_d);
+
+    check_guests_ram(to);
+
+    qtest_quit(to);
+
+    cleanup("bootsect");
+    cleanup("migsocket");
+    cleanup("src_serial");
+    cleanup("dest_serial");
+}
+
+static void migrate(QTestState *who, const char *uri)
+{
+    QDict *rsp;
+    gchar *cmd;
+
+    cmd = g_strdup_printf("{ 'execute': 'migrate',"
+                          "'arguments': { 'uri': '%s' } }",
+                          uri);
+    rsp = qtest_qmp(who, cmd);
+    g_free(cmd);
+    g_assert(qdict_haskey(rsp, "return"));
+    QDECREF(rsp);
+}
+
+static void migrate_set_parameter(QTestState *who, const char *parameter,
+                                  const char *value)
+{
+    QDict *rsp;
+    gchar *cmd;
+
+    cmd = g_strdup_printf("{ 'execute': 'migrate-set-parameters',"
+                          "'arguments': { '%s': %s } }",
+                          parameter, value);
+    rsp = qtest_qmp(who, cmd);
+    g_free(cmd);
+    g_assert(qdict_haskey(rsp, "return"));
+    QDECREF(rsp);
+}
+
+static void test_precopy(const char *uri)
+{
+    QTestState *from, *to;
+
+    test_migrate_start(&from, &to, uri);
+
+    /* We want to pick a speed slow enough that the test completes
+     * quickly, but that it doesn't complete precopy even on a slow
+     * machine, so also set the downtime.
+     */
+    /* 100 ms */
+    migrate_set_parameter(from, "downtime-limit", "100");
+    /* 1MB/s slow*/
+    migrate_set_parameter(from, "max-bandwidth", "100000000");
+
+    /* Wait for the first serial output from the source */
+    wait_for_serial("src_serial");
+
+    migrate(from, uri);
+
+    wait_for_migration_pass(from);
+
+    /* 1GB/s now it should converge */
+    migrate_set_parameter(from, "max-bandwidth", "1000000000");
+
+    if (!got_stop) {
+        qtest_qmp_eventwait(from, "STOP");
+    }
+    qtest_qmp_eventwait(to, "RESUME");
+
+    wait_for_serial("dest_serial");
+    wait_for_migration_complete(from);
+
+    test_migrate_end(from, to);
+}
+
+static void test_precopy_unix(void)
+{
+    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+
+    test_precopy(uri);
+    g_free(uri);
+}
+
+int main(int argc, char **argv)
+{
+    char template[] = "/tmp/migration-test-XXXXXX";
+    int ret;
+
+    g_test_init(&argc, &argv, NULL);
+
+    tmpfs = mkdtemp(template);
+    if (!tmpfs) {
+        g_test_message("mkdtemp on path (%s): %s\n", template, strerror(errno));
+    }
+    g_assert(tmpfs);
+
+    module_call_init(MODULE_INIT_QOM);
+
+    qtest_add_func("/migration/precopy/unix", test_precopy_unix);
+
+    ret = g_test_run();
+
+    g_assert_cmpint(ret, ==, 0);
+
+    ret = rmdir(tmpfs);
+    if (ret != 0) {
+        g_test_message("unable to rmdir: path (%s): %s\n",
+                       tmpfs, strerror(errno));
+    }
+
+    return ret;
+}
-- 
2.13.5

[Qemu-devel] [PATCH 2/6] tests: Add basic migration precopy tcp test

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/migration-test.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index cd954caee4..8ef2b72459 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -466,6 +466,13 @@ static void test_precopy_unix(void)
     g_free(uri);
 }
 
+static void test_precopy_tcp(void)
+{
+    const char *uri = "tcp:0:44444";
+
+    test_precopy(uri);
+}
+
 int main(int argc, char **argv)
 {
     char template[] = "/tmp/migration-test-XXXXXX";
@@ -482,6 +489,7 @@ int main(int argc, char **argv)
     module_call_init(MODULE_INIT_QOM);
 
     qtest_add_func("/migration/precopy/unix", test_precopy_unix);
+    qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
 
     ret = g_test_run();
 
-- 
2.13.5

[Qemu-devel] [PATCH 3/6] tests: Add precopy test using deprecated commands

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/migration-test.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index 8ef2b72459..8549e4c2c6 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -422,6 +422,76 @@ static void migrate_set_parameter(QTestState *who, const char *parameter,
     QDECREF(rsp);
 }
 
+static void migrate_set_downtime(QTestState *who, const char *value)
+{
+    QDict *rsp;
+    gchar *cmd;
+
+    cmd = g_strdup_printf("{ 'execute': 'migrate_set_downtime',"
+                          "'arguments': { 'value': %s } }", value);
+    rsp = qtest_qmp(who, cmd);
+    g_free(cmd);
+    g_assert(qdict_haskey(rsp, "return"));
+    QDECREF(rsp);
+}
+
+static void migrate_set_speed(QTestState *who, const char *value)
+{
+    QDict *rsp;
+    gchar *cmd;
+
+    cmd = g_strdup_printf("{ 'execute': 'migrate_set_speed',"
+                          "'arguments': { 'value': %s } }", value);
+    rsp = qtest_qmp(who, cmd);
+    g_free(cmd);
+    g_assert(qdict_haskey(rsp, "return"));
+    QDECREF(rsp);
+}
+
+static void test_precopy_deprecated(const char *uri)
+{
+    QTestState *from, *to;
+
+    test_migrate_start(&from, &to, uri);
+
+    /* We want to pick a speed slow enough that the test completes
+     * quickly, but that it doesn't complete precopy even on a slow
+     * machine, so also set the downtime.
+     */
+    /* 100 ms */
+    migrate_set_downtime(from, "100");
+    /* 1MB/s slow*/
+    migrate_set_speed(from, "100000000");
+
+    /* Wait for the first serial output from the source */
+    wait_for_serial("src_serial");
+
+    migrate(from, uri);
+
+    wait_for_migration_pass(from);
+
+    /* 1GB/s now it should converge */
+    migrate_set_speed(from, "1000000000");
+
+    if (!got_stop) {
+        qtest_qmp_eventwait(from, "STOP");
+    }
+    qtest_qmp_eventwait(to, "RESUME");
+
+    wait_for_serial("dest_serial");
+    wait_for_migration_complete(from);
+
+    test_migrate_end(from, to);
+}
+
+static void test_deprecated_unix(void)
+{
+    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+
+    test_precopy_deprecated(uri);
+    g_free(uri);
+}
+
 static void test_precopy(const char *uri)
 {
     QTestState *from, *to;
@@ -490,6 +560,7 @@ int main(int argc, char **argv)
 
     qtest_add_func("/migration/precopy/unix", test_precopy_unix);
     qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
+    qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
 
     ret = g_test_run();
 
-- 
2.13.5

[Qemu-devel] [PATCH 4/6] tests: Add migration xbzrle test

[Juan Quintela]

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/migration-test.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index 8549e4c2c6..f0734e4ba0 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -422,6 +422,23 @@ static void migrate_set_parameter(QTestState *who, const char *parameter,
     QDECREF(rsp);
 }
 
+static void migrate_set_capability(QTestState *who, const char *capability,
+                                   const char *value)
+{
+    QDict *rsp;
+    gchar *cmd;
+
+    cmd = g_strdup_printf("{ 'execute': 'migrate-set-capabilities',"
+                          "'arguments': { "
+                            "'capabilities': [ { "
+                              "'capability': '%s', 'state': %s } ] } }",
+                          capability, value);
+    rsp = qtest_qmp(who, cmd);
+    g_free(cmd);
+    g_assert(qdict_haskey(rsp, "return"));
+    QDECREF(rsp);
+}
+
 static void migrate_set_downtime(QTestState *who, const char *value)
 {
     QDict *rsp;
@@ -448,6 +465,19 @@ static void migrate_set_speed(QTestState *who, const char *value)
     QDECREF(rsp);
 }
 
+static void migrate_set_cache_size(QTestState *who, const char *value)
+{
+    QDict *rsp;
+    gchar *cmd;
+
+    cmd = g_strdup_printf("{ 'execute': 'migrate-set-cache-size',"
+                          "'arguments': { 'value': %s } }", value);
+    rsp = qtest_qmp(who, cmd);
+    g_free(cmd);
+    g_assert(qdict_haskey(rsp, "return"));
+    QDECREF(rsp);
+}
+
 static void test_precopy_deprecated(const char *uri)
 {
     QTestState *from, *to;
@@ -543,6 +573,55 @@ static void test_precopy_tcp(void)
     test_precopy(uri);
 }
 
+static void test_xbzrle(const char *uri)
+{
+    QTestState *from, *to;
+
+    test_migrate_start(&from, &to, uri);
+
+    /* We want to pick a speed slow enough that the test completes
+     * quickly, but that it doesn't complete precopy even on a slow
+     * machine, so also set the downtime.
+     */
+    /* 100 ms */
+    migrate_set_parameter(from, "downtime-limit", "100");
+    /* 1MB/s slow*/
+    migrate_set_parameter(from, "max-bandwidth", "100000000");
+
+    migrate_set_cache_size(from, "50000000");
+
+    migrate_set_capability(from, "xbzrle", "true");
+    migrate_set_capability(to, "xbzrle", "true");
+    /* Wait for the first serial output from the source */
+    wait_for_serial("src_serial");
+
+    migrate(from, uri);
+
+    wait_for_migration_pass(from);
+
+    /* 1GB/s now it should converge */
+    migrate_set_parameter(from, "max-bandwidth", "1000000000");
+
+    if (!got_stop) {
+        qtest_qmp_eventwait(from, "STOP");
+    }
+    qtest_qmp_eventwait(to, "RESUME");
+
+    wait_for_serial("dest_serial");
+    wait_for_migration_complete(from);
+
+    test_migrate_end(from, to);
+}
+
+static void test_xbzrle_unix(void)
+{
+    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+
+    test_xbzrle(uri);
+    g_free(uri);
+}
+
+
 int main(int argc, char **argv)
 {
     char template[] = "/tmp/migration-test-XXXXXX";
@@ -561,6 +640,7 @@ int main(int argc, char **argv)
     qtest_add_func("/migration/precopy/unix", test_precopy_unix);
     qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
     qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
+    qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
 
     ret = g_test_run();
 
-- 
2.13.5

[Qemu-devel] [PATCH 5/6] tests: Add migration compress threads tests

[Juan Quintela]

For some reason, compression is not working at the moment, test is
disabled until I found why.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/migration-test.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index f0734e4ba0..a8abc3d007 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -621,6 +621,54 @@ static void test_xbzrle_unix(void)
     g_free(uri);
 }
 
+static void test_compress(const char *uri)
+{
+    QTestState *from, *to;
+
+    test_migrate_start(&from, &to, uri);
+
+    /* We want to pick a speed slow enough that the test completes
+     * quickly, but that it doesn't complete precopy even on a slow
+     * machine, so also set the downtime.
+     */
+    /* 100 ms */
+    migrate_set_parameter(from, "downtime-limit", "100");
+    /* 1MB/s slow*/
+    migrate_set_parameter(from, "max-bandwidth", "100000000");
+
+    migrate_set_parameter(from, "compress-threads", "4");
+    migrate_set_parameter(from, "decompress-threads", "3"); 
+
+    migrate_set_capability(from, "compress", "true");
+    migrate_set_capability(to, "compress", "true");
+    /* Wait for the first serial output from the source */
+    wait_for_serial("src_serial");
+
+    migrate(from, uri);
+
+    wait_for_migration_pass(from);
+
+    /* 1GB/s now it should converge */
+    migrate_set_parameter(from, "max-bandwidth", "1000000000");
+
+    if (!got_stop) {
+        qtest_qmp_eventwait(from, "STOP");
+    }
+    qtest_qmp_eventwait(to, "RESUME");
+
+    wait_for_serial("dest_serial");
+    wait_for_migration_complete(from);
+
+    test_migrate_end(from, to);
+}
+
+static void test_compress_unix(void)
+{
+    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
+
+    test_compress(uri);
+    g_free(uri);
+}
 
 int main(int argc, char **argv)
 {
@@ -641,6 +689,9 @@ int main(int argc, char **argv)
     qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
     qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
     qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
+    if (0) {
+        qtest_add_func("/migration/compress/unix", test_compress_unix);
+    }
 
     ret = g_test_run();
 
-- 
2.13.5

[Qemu-devel] [PATCH 6/6] tests: Move postcopy migration test to migrate-set-parameters

[Juan Quintela]

migrate_set_speed and migrate_set_dowtime are deprecated since long ago.

Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/postcopy-test.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/postcopy-test.c b/tests/postcopy-test.c
index 8142f2ab90..991b954a83 100644
--- a/tests/postcopy-test.c
+++ b/tests/postcopy-test.c
@@ -430,14 +430,14 @@ static void test_migrate(void)
      * machine, so also set the downtime.
      */
     global_qtest = from;
-    rsp = qmp("{ 'execute': 'migrate_set_speed',"
-              "'arguments': { 'value': 100000000 } }");
+    rsp = qmp("{ 'execute': 'migrate-set-parameters',"
+              "'arguments': { 'max-bandwidth': 100000000 } }");
     g_assert(qdict_haskey(rsp, "return"));
     QDECREF(rsp);
 
     /* 1ms downtime - it should never finish precopy */
-    rsp = qmp("{ 'execute': 'migrate_set_downtime',"
-              "'arguments': { 'value': 0.001 } }");
+    rsp = qmp("{ 'execute': 'migrate-set-parameters',"
+              "'arguments': { 'downtime-limit': 1 } }");
     g_assert(qdict_haskey(rsp, "return"));
     QDECREF(rsp);
 
-- 
2.13.5

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Thomas Huth]

On 04.10.2017 12:39, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  tests/Makefile.include |   3 +
>  tests/migration-test.c | 497 +++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 500 insertions(+)
>  create mode 100644 tests/migration-test.c
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index abc6707ef2..d23ca1ed1c 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -287,6 +287,7 @@ endif
>  check-qtest-i386-$(CONFIG_SLIRP) += tests/test-netfilter$(EXESUF)
>  check-qtest-i386-$(CONFIG_POSIX) += tests/test-filter-mirror$(EXESUF)
>  check-qtest-i386-$(CONFIG_POSIX) += tests/test-filter-redirector$(EXESUF)
> +check-qtest-i386-y += tests/migration-test$(EXESUF)
>  check-qtest-i386-y += tests/postcopy-test$(EXESUF)
>  check-qtest-i386-y += tests/test-x86-cpuid-compat$(EXESUF)
>  check-qtest-i386-y += tests/numa-test$(EXESUF)
> @@ -316,6 +317,7 @@ check-qtest-ppc64-y += tests/prom-env-test$(EXESUF)
>  check-qtest-ppc64-y += tests/pnv-xscom-test$(EXESUF)
>  check-qtest-ppc64-y += tests/drive_del-test$(EXESUF)
>  check-qtest-ppc64-y += tests/postcopy-test$(EXESUF)
> +check-qtest-ppc64-y += tests/migration-test$(EXESUF)
>  check-qtest-ppc64-y += tests/boot-serial-test$(EXESUF)
>  check-qtest-ppc64-y += tests/rtas-test$(EXESUF)
>  check-qtest-ppc64-$(CONFIG_SLIRP) += tests/pxe-test$(EXESUF)
> @@ -782,6 +784,7 @@ tests/usb-hcd-ehci-test$(EXESUF): tests/usb-hcd-ehci-test.o $(libqos-usb-obj-y)
>  tests/usb-hcd-xhci-test$(EXESUF): tests/usb-hcd-xhci-test.o $(libqos-usb-obj-y)
>  tests/pc-cpu-test$(EXESUF): tests/pc-cpu-test.o
>  tests/postcopy-test$(EXESUF): tests/postcopy-test.o
> +tests/migration-test$(EXESUF): tests/migration-test.o
>  tests/vhost-user-test$(EXESUF): tests/vhost-user-test.o $(test-util-obj-y) \
>  	$(qtest-obj-y) $(test-io-obj-y) $(libqos-virtio-obj-y) $(libqos-pc-obj-y) \
>  	$(chardev-obj-y)
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> new file mode 100644
> index 0000000000..cd954caee4
> --- /dev/null
> +++ b/tests/migration-test.c
> @@ -0,0 +1,497 @@
> +/*
> + * QTest testcases for migration
> + *
> + * Copyright (c) 2017 Red Hat, Inc. and/or its affiliates
> + *   based on the postcopy-test.c which is based
> + *   based on the vhost-user-test.c that is:
> + *      Copyright (c) 2014 Virtual Open Systems Sarl.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +
> +#include "libqtest.h"
> +#include "qemu/option.h"
> +#include "qemu/range.h"
> +#include "qemu/sockets.h"
> +#include "chardev/char.h"
> +#include "sysemu/sysemu.h"
> +#include "hw/nvram/chrp_nvram.h"
> +
> +#define MIN_NVRAM_SIZE 8192 /* from spapr_nvram.c */
> +
> +const unsigned start_address = 1024 * 1024;
> +const unsigned end_address = 100 * 1024 * 1024;
> +bool got_stop;
> +
> +#if defined(__linux__)
> +#include <sys/syscall.h>
> +#include <sys/vfs.h>
> +#endif
> +
> +static const char *tmpfs;
> +
> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> + * outputing a 'B' every so often if it's still running.
> + */
> +unsigned char bootsect[] = {
> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> +};
> +
> +static void init_bootfile_x86(const char *bootpath)
> +{
> +    FILE *bootfile = fopen(bootpath, "wb");
> +
> +    g_assert_cmpint(fwrite(bootsect, 512, 1, bootfile), ==, 1);
> +    fclose(bootfile);
> +}
> +
> +static void init_bootfile_ppc(const char *bootpath)
> +{
> +    FILE *bootfile;
> +    char buf[MIN_NVRAM_SIZE];
> +    ChrpNvramPartHdr *header = (ChrpNvramPartHdr *)buf;
> +
> +    memset(buf, 0, MIN_NVRAM_SIZE);
> +
> +    /* Create a "common" partition in nvram to store boot-command property */
> +
> +    header->signature = CHRP_NVPART_SYSTEM;
> +    memcpy(header->name, "common", 6);
> +    chrp_nvram_finish_partition(header, MIN_NVRAM_SIZE);
> +
> +    /* FW_MAX_SIZE is 4MB, but slof.bin is only 900KB,
> +     * so let's modify memory between 1MB and 100MB
> +     * to do like PC bootsector
> +     */
> +
> +    sprintf(buf + 16,
> +            "boot-command=hex .\" _\" begin %x %x do i c@ 1 + i c! 1000 +loop "
> +            ".\" B\" 0 until", end_address, start_address);
> +
> +    /* Write partition to the NVRAM file */
> +
> +    bootfile = fopen(bootpath, "wb");
> +    g_assert_cmpint(fwrite(buf, MIN_NVRAM_SIZE, 1, bootfile), ==, 1);
> +    fclose(bootfile);
> +}
> +
> +/*
> + * Wait for some output in the serial output file,
> + * we get an 'A' followed by an endless string of 'B's
> + * but on the destination we won't have the A.
> + */
> +static void wait_for_serial(const char *side)
> +{
> +    char *serialpath = g_strdup_printf("%s/%s", tmpfs, side);
> +    FILE *serialfile = fopen(serialpath, "r");
> +    const char *arch = qtest_get_arch();
> +    int started = (strcmp(side, "src_serial") == 0 &&
> +                   strcmp(arch, "ppc64") == 0) ? 0 : 1;
> +
> +    g_free(serialpath);
> +    do {
> +        int readvalue = fgetc(serialfile);
> +
> +        if (!started) {
> +            /* SLOF prints its banner before starting test,
> +             * to ignore it, mark the start of the test with '_',
> +             * ignore all characters until this marker
> +             */
> +            switch (readvalue) {
> +            case '_':
> +                started = 1;
> +                break;
> +            case EOF:
> +                fseek(serialfile, 0, SEEK_SET);
> +                usleep(1000);
> +                break;
> +            }
> +            continue;
> +        }
> +        switch (readvalue) {
> +        case 'A':
> +            /* Fine */
> +            break;
> +
> +        case 'B':
> +            /* It's alive! */
> +            fclose(serialfile);
> +            return;
> +
> +        case EOF:
> +            started = (strcmp(side, "src_serial") == 0 &&
> +                       strcmp(arch, "ppc64") == 0) ? 0 : 1;
> +            fseek(serialfile, 0, SEEK_SET);
> +            usleep(1000);
> +            break;
> +
> +        default:
> +            fprintf(stderr, "Unexpected %d on %s serial\n", readvalue, side);
> +            g_assert_not_reached();
> +        }
> +    } while (true);
> +}
> +
> +/*
> + * Events can get in the way of responses we are actually waiting for.
> + */
> +static QDict *return_or_event(QDict *response, QTestState *who)
> +{
> +    const char *event_string;
> +    if (!qdict_haskey(response, "event")) {
> +        return response;
> +    }
> +
> +    /* OK, it was an event */
> +    event_string = qdict_get_str(response, "event");
> +    if (!strcmp(event_string, "STOP")) {
> +        got_stop = true;
> +    }
> +    QDECREF(response);
> +    return return_or_event(qtest_qmp_receive(who), who);
> +}
> +
> +
> +/*
> + * It's tricky to use qemu's migration event capability with qtest,
> + * events suddenly appearing confuse the qmp()/hmp() responses.
> + * so wait for a couple of passes to have happened before
> + * going postcopy.
> + */
> +
> +static uint64_t get_migration_pass(QTestState *who)
> +{
> +    QDict *rsp, *rsp_return, *rsp_ram;
> +    uint64_t result;
> +
> +    rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
> +    rsp_return = qdict_get_qdict(rsp, "return");
> +    if (!qdict_haskey(rsp_return, "ram")) {
> +        /* Still in setup */
> +        result = 0;
> +    } else {
> +        rsp_ram = qdict_get_qdict(rsp_return, "ram");
> +        result = qdict_get_try_int(rsp_ram, "dirty-sync-count", 0);
> +    }
> +    QDECREF(rsp);
> +    return result;
> +}
> +
> +static void wait_for_migration_complete(QTestState *who)
> +{
> +    bool completed;
> +
> +    do {
> +        QDict *rsp, *rsp_return;
> +        const char *status;
> +
> +        rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
> +        rsp_return = qdict_get_qdict(rsp, "return");
> +        status = qdict_get_str(rsp_return, "status");
> +        completed = strcmp(status, "completed") == 0;
> +        g_assert_cmpstr(status, !=,  "failed");
> +        QDECREF(rsp);
> +        usleep(1000 * 100);
> +    } while (!completed);
> +}
> +
> +static void wait_for_migration_pass(QTestState *who)
> +{
> +    uint64_t initial_pass = get_migration_pass(who);
> +    uint64_t pass;
> +
> +    /* Wait for the 1st sync */
> +    do {
> +        initial_pass = get_migration_pass(who);
> +        if (got_stop || initial_pass) {
> +            break;
> +        }
> +        usleep(1000 * 100);
> +    } while (true);
> +
> +    do {
> +        usleep(1000 * 100);
> +        pass = get_migration_pass(who);
> +    } while (pass == initial_pass && !got_stop);
> +}
> +
> +static void check_guests_ram(QTestState *who)
> +{
> +    /* Our ASM test will have been incrementing one byte from each page from
> +     * 1MB to <100MB in order.
> +     * This gives us a constraint that any page's byte should be equal or less
> +     * than the previous pages byte (mod 256); and they should all be equal
> +     * except for one transition at the point where we meet the incrementer.
> +     * (We're running this with the guest stopped).
> +     */
> +    unsigned address;
> +    uint8_t first_byte;
> +    uint8_t last_byte;
> +    bool hit_edge = false;
> +    bool bad = false;
> +
> +    qtest_memread(who, start_address, &first_byte, 1);
> +    last_byte = first_byte;
> +
> +    for (address = start_address + 4096; address < end_address; address += 4096)
> +    {
> +        uint8_t b;
> +        qtest_memread(who, address, &b, 1);
> +        if (b != last_byte) {
> +            if (((b + 1) % 256) == last_byte && !hit_edge) {
> +                /* This is OK, the guest stopped at the point of
> +                 * incrementing the previous page but didn't get
> +                 * to us yet.
> +                 */
> +                hit_edge = true;
> +            } else {
> +                fprintf(stderr, "Memory content inconsistency at %x"
> +                                " first_byte = %x last_byte = %x current = %x"
> +                                " hit_edge = %x\n",
> +                                address, first_byte, last_byte, b, hit_edge);
> +                bad = true;
> +            }
> +        }
> +        last_byte = b;
> +    }
> +    g_assert_false(bad);
> +}
> +
> +static void cleanup(const char *filename)
> +{
> +    char *path = g_strdup_printf("%s/%s", tmpfs, filename);
> +
> +    unlink(path);
> +    g_free(path);
> +}

Looks like a lot of this code is the same or very similar to the code in
postcopy-test.c ... would it make sense to factor-out that code there
into a separate file that could be used by both tests, so that we do not
have to maintain the code twice?

 Thomas

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Juan Quintela]

Thomas Huth <thuth@redhat.com> wrote:
> On 04.10.2017 12:39, Juan Quintela wrote:
>> Signed-off-by: Juan Quintela <quintela@redhat.com>

>
> Looks like a lot of this code is the same or very similar to the code in
> postcopy-test.c ... would it make sense to factor-out that code there
> into a separate file that could be used by both tests, so that we do not
> have to maintain the code twice?


I think I put that on the cover letter,  my idea is to add the postcopy
test here.  This is a streamlined version of the postcopy-test, just a
barebones, and insntead of doing things by hand, I added helpers like:

migrate_set_parameter(...);

You can see that on the following patches I added support for:
- tcp
- precopy with deprecated commands
- xbzrle
- compression threads (not working at the momment, have to investigate
  why).

Idea is if people like the approach to add on top of that:
- exec: migration (needs changes because command line option on source
  and destination are different)
- fd: Dunno how difficult is, I have really never passed fd's around.
- postocpy: It needs additional commandsn to be added.

- Probably move to use events, instead of "waiting" for 1st stage to
  finish.

Right now, the three tests on my laptop require around 14 seconds (not
running in parallel).  I want to "optimize" the tests to take a bit less
time.

Later, Juan.

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Thomas Huth]

On 04.10.2017 14:46, Juan Quintela wrote:
> Thomas Huth <thuth@redhat.com> wrote:
>> On 04.10.2017 12:39, Juan Quintela wrote:
>>> Signed-off-by: Juan Quintela <quintela@redhat.com>
> 
>>
>> Looks like a lot of this code is the same or very similar to the code in
>> postcopy-test.c ... would it make sense to factor-out that code there
>> into a separate file that could be used by both tests, so that we do not
>> have to maintain the code twice?
> 
> 
> I think I put that on the cover letter,  my idea is to add the postcopy
> test here.
OK, then I think you should remove the duplicated functions from
postcopy-test.c in the same patch here, so that it is more obvious that
this is a code movement.

 Thomas

PS: Please also add an entry to MAINTAINERS for the new files if feasible.

Re: [Qemu-devel] [PATCH 0/6] Add make check tests for Migration

[no-reply]

Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20171004103933.7898-1-quintela@redhat.com
Subject: [Qemu-devel] [PATCH 0/6] Add make check tests for Migration

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
4aeb6111b4 tests: Move postcopy migration test to migrate-set-parameters
1c9b2bdb92 tests: Add migration compress threads tests
56c99c4814 tests: Add migration xbzrle test
576f38004b tests: Add precopy test using deprecated commands
af499b81e9 tests: Add basic migration precopy tcp test
54217d4480 tests: Add basic migration precopy test

=== OUTPUT BEGIN ===
Checking PATCH 1/6: tests: Add basic migration precopy test...
WARNING: architecture specific defines should be avoided
#72: FILE: tests/migration-test.c:30:
+#if defined(__linux__)

total: 0 errors, 1 warnings, 518 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 2/6: tests: Add basic migration precopy tcp test...
Checking PATCH 3/6: tests: Add precopy test using deprecated commands...
Checking PATCH 4/6: tests: Add migration xbzrle test...
Checking PATCH 5/6: tests: Add migration compress threads tests...
ERROR: trailing whitespace
#36: FILE: tests/migration-test.c:640:
+    migrate_set_parameter(from, "decompress-threads", "3"); $

total: 1 errors, 0 warnings, 63 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 6/6: tests: Move postcopy migration test to migrate-set-parameters...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

On 10/04/2017 05:39 AM, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  tests/Makefile.include |   3 +
>  tests/migration-test.c | 497 +++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 500 insertions(+)
>  create mode 100644 tests/migration-test.c

New file; should it be mentioned in MAINTAINERS?

At least you picked a name that matches .gitignore's existing '*.test'
exclusion when the binary is built in-tree.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Peter Xu]

On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:

[...]

(having some unit tests for migration is really good...)

> +static void test_precopy(const char *uri)
> +{
> +    QTestState *from, *to;
> +
> +    test_migrate_start(&from, &to, uri);
> +
> +    /* We want to pick a speed slow enough that the test completes
> +     * quickly, but that it doesn't complete precopy even on a slow
> +     * machine, so also set the downtime.
> +     */
> +    /* 100 ms */
> +    migrate_set_parameter(from, "downtime-limit", "100");
> +    /* 1MB/s slow*/

This is 100MB/s?

> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
> +
> +    /* Wait for the first serial output from the source */
> +    wait_for_serial("src_serial");
> +
> +    migrate(from, uri);
> +
> +    wait_for_migration_pass(from);
> +
> +    /* 1GB/s now it should converge */
> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");

Curious: would it possible that 1GB/s won't suffice in some cases?
Is there any reason behind this 1GB/s value?

(I agree with Thomas that it would be good to work upon the existing
 postcopy-test.c file, at least easier to review :)

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 5/6] tests: Add migration compress threads tests

[Peter Xu]

On Wed, Oct 04, 2017 at 12:39:32PM +0200, Juan Quintela wrote:

[...]

>  int main(int argc, char **argv)
>  {
> @@ -641,6 +689,9 @@ int main(int argc, char **argv)
>      qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
>      qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
>      qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);

Nits: Not sure whether we can have better naming for the tests?  Say:

  /migration/precopy/unix
  /migration/precopy/tcp
  /migration/deprecated-cmds
  /migration/xbzrle
  /migration/compression

I'm fine with existing naming as well.

For the deprecated commands test, not sure whether we can just send
those commands and query using "query-migrate-parameters" to make sure
they are setup and valid.  I assume it can be faster than real
migrations.

> +    if (0) {

Is this intended? :)

> +        qtest_add_func("/migration/compress/unix", test_compress_unix);
> +    }
>  
>      ret = g_test_run();
>  
> -- 
> 2.13.5
> 

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>

1) I think I agree with the bits about needing to share this code with
postcopy; lets see how it goes.
2) I do have a bug I can reproduce in the postcopy test where it fails
under heavy load (e.g. ~10 TCG'd guests on one CPU) - not investigated
in detail yet, but watch out.

> +static void test_precopy(const char *uri)
> +{
> +    QTestState *from, *to;
> +
> +    test_migrate_start(&from, &to, uri);
> +
> +    /* We want to pick a speed slow enough that the test completes
> +     * quickly, but that it doesn't complete precopy even on a slow
> +     * machine, so also set the downtime.
> +     */
> +    /* 100 ms */
> +    migrate_set_parameter(from, "downtime-limit", "100");
> +    /* 1MB/s slow*/
> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
> +
> +    /* Wait for the first serial output from the source */
> +    wait_for_serial("src_serial");
> +
> +    migrate(from, uri);
> +
> +    wait_for_migration_pass(from);
> +
> +    /* 1GB/s now it should converge */
> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");

why is 1GB/s enough for it to converge? I'd add another 0 and
make the downtime larger as well.

> +    if (!got_stop) {
> +        qtest_qmp_eventwait(from, "STOP");
> +    }
> +    qtest_qmp_eventwait(to, "RESUME");
> +
> +    wait_for_serial("dest_serial");
> +    wait_for_migration_complete(from);
> +
> +    test_migrate_end(from, to);
> +}
> +
> +static void test_precopy_unix(void)
> +{
> +    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
> +
> +    test_precopy(uri);
> +    g_free(uri);
> +}
> +
> +int main(int argc, char **argv)
> +{
> +    char template[] = "/tmp/migration-test-XXXXXX";
> +    int ret;
> +
> +    g_test_init(&argc, &argv, NULL);
> +
> +    tmpfs = mkdtemp(template);
> +    if (!tmpfs) {
> +        g_test_message("mkdtemp on path (%s): %s\n", template, strerror(errno));
> +    }
> +    g_assert(tmpfs);

We've started using g_dir_make_tmp recently; it's certianly simpler.

Dave

> +    module_call_init(MODULE_INIT_QOM);
> +
> +    qtest_add_func("/migration/precopy/unix", test_precopy_unix);
> +
> +    ret = g_test_run();
> +
> +    g_assert_cmpint(ret, ==, 0);
> +
> +    ret = rmdir(tmpfs);
> +    if (ret != 0) {
> +        g_test_message("unable to rmdir: path (%s): %s\n",
> +                       tmpfs, strerror(errno));
> +    }
> +
> +    return ret;
> +}
> -- 
> 2.13.5
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 2/6] tests: Add basic migration precopy tcp test

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  tests/migration-test.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> index cd954caee4..8ef2b72459 100644
> --- a/tests/migration-test.c
> +++ b/tests/migration-test.c
> @@ -466,6 +466,13 @@ static void test_precopy_unix(void)
>      g_free(uri);
>  }
>  
> +static void test_precopy_tcp(void)
> +{
> +    const char *uri = "tcp:0:44444";
> +
> +    test_precopy(uri);
> +}

The problem is that this will fail if you make check -j  and two
tests use 4444 at the same time.
That's pretty common especially when you've built multiple architectures
as targets and all the same tests run at the same time.

Dave

>  int main(int argc, char **argv)
>  {
>      char template[] = "/tmp/migration-test-XXXXXX";
> @@ -482,6 +489,7 @@ int main(int argc, char **argv)
>      module_call_init(MODULE_INIT_QOM);
>  
>      qtest_add_func("/migration/precopy/unix", test_precopy_unix);
> +    qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
>  
>      ret = g_test_run();
>  
> -- 
> 2.13.5
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 3/6] tests: Add precopy test using deprecated commands

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  tests/migration-test.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 71 insertions(+)
> 
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> index 8ef2b72459..8549e4c2c6 100644
> --- a/tests/migration-test.c
> +++ b/tests/migration-test.c
> @@ -422,6 +422,76 @@ static void migrate_set_parameter(QTestState *who, const char *parameter,
>      QDECREF(rsp);
>  }
>  
> +static void migrate_set_downtime(QTestState *who, const char *value)
> +{
> +    QDict *rsp;
> +    gchar *cmd;
> +
> +    cmd = g_strdup_printf("{ 'execute': 'migrate_set_downtime',"
> +                          "'arguments': { 'value': %s } }", value);
> +    rsp = qtest_qmp(who, cmd);
> +    g_free(cmd);
> +    g_assert(qdict_haskey(rsp, "return"));
> +    QDECREF(rsp);
> +}
> +
> +static void migrate_set_speed(QTestState *who, const char *value)
> +{
> +    QDict *rsp;
> +    gchar *cmd;
> +
> +    cmd = g_strdup_printf("{ 'execute': 'migrate_set_speed',"
> +                          "'arguments': { 'value': %s } }", value);
> +    rsp = qtest_qmp(who, cmd);
> +    g_free(cmd);
> +    g_assert(qdict_haskey(rsp, "return"));
> +    QDECREF(rsp);
> +}
> +
> +static void test_precopy_deprecated(const char *uri)
> +{
> +    QTestState *from, *to;
> +
> +    test_migrate_start(&from, &to, uri);
> +
> +    /* We want to pick a speed slow enough that the test completes
> +     * quickly, but that it doesn't complete precopy even on a slow
> +     * machine, so also set the downtime.
> +     */
> +    /* 100 ms */
> +    migrate_set_downtime(from, "100");
> +    /* 1MB/s slow*/
> +    migrate_set_speed(from, "100000000");

Can't we share more/most of this function?
For example, how about we have a migrate_set_speed(who,value, old)
tht is :
  if (old) {
      migrate_set_speed...
  } else {
      migrate_set_parameter
  }

then test_precopy(const char *uri, bool old)

and we've duped most of it?

Dave

> +    /* Wait for the first serial output from the source */
> +    wait_for_serial("src_serial");
> +
> +    migrate(from, uri);
> +
> +    wait_for_migration_pass(from);
> +
> +    /* 1GB/s now it should converge */
> +    migrate_set_speed(from, "1000000000");
> +
> +    if (!got_stop) {
> +        qtest_qmp_eventwait(from, "STOP");
> +    }
> +    qtest_qmp_eventwait(to, "RESUME");
> +
> +    wait_for_serial("dest_serial");
> +    wait_for_migration_complete(from);
> +
> +    test_migrate_end(from, to);
> +}
> +
> +static void test_deprecated_unix(void)
> +{
> +    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
> +
> +    test_precopy_deprecated(uri);
> +    g_free(uri);
> +}
> +
>  static void test_precopy(const char *uri)
>  {
>      QTestState *from, *to;
> @@ -490,6 +560,7 @@ int main(int argc, char **argv)
>  
>      qtest_add_func("/migration/precopy/unix", test_precopy_unix);
>      qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
> +    qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
>  
>      ret = g_test_run();
>  
> -- 
> 2.13.5
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 4/6] tests: Add migration xbzrle test

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  tests/migration-test.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 80 insertions(+)
> 
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> index 8549e4c2c6..f0734e4ba0 100644
> --- a/tests/migration-test.c
> +++ b/tests/migration-test.c
> @@ -422,6 +422,23 @@ static void migrate_set_parameter(QTestState *who, const char *parameter,
>      QDECREF(rsp);
>  }
>  
> +static void migrate_set_capability(QTestState *who, const char *capability,
> +                                   const char *value)
> +{
> +    QDict *rsp;
> +    gchar *cmd;
> +
> +    cmd = g_strdup_printf("{ 'execute': 'migrate-set-capabilities',"
> +                          "'arguments': { "
> +                            "'capabilities': [ { "
> +                              "'capability': '%s', 'state': %s } ] } }",
> +                          capability, value);
> +    rsp = qtest_qmp(who, cmd);
> +    g_free(cmd);
> +    g_assert(qdict_haskey(rsp, "return"));
> +    QDECREF(rsp);
> +}
> +
>  static void migrate_set_downtime(QTestState *who, const char *value)
>  {
>      QDict *rsp;
> @@ -448,6 +465,19 @@ static void migrate_set_speed(QTestState *who, const char *value)
>      QDECREF(rsp);
>  }
>  
> +static void migrate_set_cache_size(QTestState *who, const char *value)
> +{
> +    QDict *rsp;
> +    gchar *cmd;
> +
> +    cmd = g_strdup_printf("{ 'execute': 'migrate-set-cache-size',"
> +                          "'arguments': { 'value': %s } }", value);
> +    rsp = qtest_qmp(who, cmd);
> +    g_free(cmd);
> +    g_assert(qdict_haskey(rsp, "return"));
> +    QDECREF(rsp);
> +}
> +
>  static void test_precopy_deprecated(const char *uri)
>  {
>      QTestState *from, *to;
> @@ -543,6 +573,55 @@ static void test_precopy_tcp(void)
>      test_precopy(uri);
>  }
>  
> +static void test_xbzrle(const char *uri)
> +{
> +    QTestState *from, *to;
> +
> +    test_migrate_start(&from, &to, uri);
> +
> +    /* We want to pick a speed slow enough that the test completes
> +     * quickly, but that it doesn't complete precopy even on a slow
> +     * machine, so also set the downtime.
> +     */
> +    /* 100 ms */
> +    migrate_set_parameter(from, "downtime-limit", "100");
> +    /* 1MB/s slow*/
> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
> +
> +    migrate_set_cache_size(from, "50000000");

Why 50MB?  I guess this is an interesting test; the program
dirties 100MB of RAM repeatedly, one byte/page.  So 50MB is probably
not much use becuase it'll thrash.

> +    migrate_set_capability(from, "xbzrle", "true");
> +    migrate_set_capability(to, "xbzrle", "true");
> +    /* Wait for the first serial output from the source */
> +    wait_for_serial("src_serial");
> +
> +    migrate(from, uri);
> +
> +    wait_for_migration_pass(from);
> +
> +    /* 1GB/s now it should converge */
> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");

I wonder about doing a set_cache_size 150MB here, with a delay before
the max-bandwidht?  If we're lucky at 150MB it'll start actually using
the XBZRLE compression.

> +    if (!got_stop) {
> +        qtest_qmp_eventwait(from, "STOP");
> +    }
> +    qtest_qmp_eventwait(to, "RESUME");
> +
> +    wait_for_serial("dest_serial");
> +    wait_for_migration_complete(from);
> +
> +    test_migrate_end(from, to);

Extract xbzrle stats?

Dave

> +}
> +
> +static void test_xbzrle_unix(void)
> +{
> +    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
> +
> +    test_xbzrle(uri);
> +    g_free(uri);
> +}
> +
> +
>  int main(int argc, char **argv)
>  {
>      char template[] = "/tmp/migration-test-XXXXXX";
> @@ -561,6 +640,7 @@ int main(int argc, char **argv)
>      qtest_add_func("/migration/precopy/unix", test_precopy_unix);
>      qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
>      qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
> +    qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
>  
>      ret = g_test_run();
>  
> -- 
> 2.13.5
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 5/6] tests: Add migration compress threads tests

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> For some reason, compression is not working at the moment, test is
> disabled until I found why.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  tests/migration-test.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 51 insertions(+)
> 
> diff --git a/tests/migration-test.c b/tests/migration-test.c
> index f0734e4ba0..a8abc3d007 100644
> --- a/tests/migration-test.c
> +++ b/tests/migration-test.c
> @@ -621,6 +621,54 @@ static void test_xbzrle_unix(void)
>      g_free(uri);
>  }
>  
> +static void test_compress(const char *uri)
> +{
> +    QTestState *from, *to;
> +
> +    test_migrate_start(&from, &to, uri);
> +
> +    /* We want to pick a speed slow enough that the test completes
> +     * quickly, but that it doesn't complete precopy even on a slow
> +     * machine, so also set the downtime.
> +     */
> +    /* 100 ms */
> +    migrate_set_parameter(from, "downtime-limit", "100");
> +    /* 1MB/s slow*/
> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
> +
> +    migrate_set_parameter(from, "compress-threads", "4");
> +    migrate_set_parameter(from, "decompress-threads", "3"); 

Should that be 'to' ?

I worry about having this many threads in a loaded test environment.

Dave

> +    migrate_set_capability(from, "compress", "true");
> +    migrate_set_capability(to, "compress", "true");
> +    /* Wait for the first serial output from the source */
> +    wait_for_serial("src_serial");
> +
> +    migrate(from, uri);
> +
> +    wait_for_migration_pass(from);
> +
> +    /* 1GB/s now it should converge */
> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");
> +
> +    if (!got_stop) {
> +        qtest_qmp_eventwait(from, "STOP");
> +    }
> +    qtest_qmp_eventwait(to, "RESUME");
> +
> +    wait_for_serial("dest_serial");
> +    wait_for_migration_complete(from);
> +
> +    test_migrate_end(from, to);
> +}
> +
> +static void test_compress_unix(void)
> +{
> +    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
> +
> +    test_compress(uri);
> +    g_free(uri);
> +}
>  
>  int main(int argc, char **argv)
>  {
> @@ -641,6 +689,9 @@ int main(int argc, char **argv)
>      qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
>      qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
>      qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
> +    if (0) {
> +        qtest_add_func("/migration/compress/unix", test_compress_unix);
> +    }
>  
>      ret = g_test_run();
>  
> -- 
> 2.13.5
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 6/6] tests: Move postcopy migration test to migrate-set-parameters

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> migrate_set_speed and migrate_set_dowtime are deprecated since long ago.
> 
> Signed-off-by: Juan Quintela <quintela@redhat.com>

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  tests/postcopy-test.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/postcopy-test.c b/tests/postcopy-test.c
> index 8142f2ab90..991b954a83 100644
> --- a/tests/postcopy-test.c
> +++ b/tests/postcopy-test.c
> @@ -430,14 +430,14 @@ static void test_migrate(void)
>       * machine, so also set the downtime.
>       */
>      global_qtest = from;
> -    rsp = qmp("{ 'execute': 'migrate_set_speed',"
> -              "'arguments': { 'value': 100000000 } }");
> +    rsp = qmp("{ 'execute': 'migrate-set-parameters',"
> +              "'arguments': { 'max-bandwidth': 100000000 } }");
>      g_assert(qdict_haskey(rsp, "return"));
>      QDECREF(rsp);
>  
>      /* 1ms downtime - it should never finish precopy */
> -    rsp = qmp("{ 'execute': 'migrate_set_downtime',"
> -              "'arguments': { 'value': 0.001 } }");
> +    rsp = qmp("{ 'execute': 'migrate-set-parameters',"
> +              "'arguments': { 'downtime-limit': 1 } }");
>      g_assert(qdict_haskey(rsp, "return"));
>      QDECREF(rsp);
>  
> -- 
> 2.13.5
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Peter Xu]

On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:

[...]

> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> + * outputing a 'B' every so often if it's still running.
> + */
> +unsigned char bootsect[] = {
> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> +};

Not sure whether it would be nicer to put this section as binary into
QEMU's tree as shared, then other test code can use it too directly,
and it would be easier if someone wants to boot a VM running this
bootstrap code.  Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Juan Quintela]

Peter Xu <peterx@redhat.com> wrote:
> On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
>
> [...]
>
>> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
>> + * outputing a 'B' every so often if it's still running.
>> + */
>> +unsigned char bootsect[] = {
>> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
>> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
>> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
>> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
>> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
>> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
>> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
>> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
>> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
>> +};
>
> Not sure whether it would be nicer to put this section as binary into
> QEMU's tree as shared, then other test code can use it too directly,
> and it would be easier if someone wants to boot a VM running this
> bootstrap code.  Thanks,

I am sharing this code now on a single place.  About getting this
outside ... I will wait for someone that understand how this works, I
have no clue neither how it works nor how to share/use/setup/....

Later, Juan.

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Juan Quintela]

Peter Xu <peterx@redhat.com> wrote:
> On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
>
> [...]
>
> (having some unit tests for migration is really good...)
>
>> +static void test_precopy(const char *uri)
>> +{
>> +    QTestState *from, *to;
>> +
>> +    test_migrate_start(&from, &to, uri);
>> +
>> +    /* We want to pick a speed slow enough that the test completes
>> +     * quickly, but that it doesn't complete precopy even on a slow
>> +     * machine, so also set the downtime.
>> +     */
>> +    /* 100 ms */
>> +    migrate_set_parameter(from, "downtime-limit", "100");
>> +    /* 1MB/s slow*/
>
> This is 100MB/s?
>
>> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
>> +
>> +    /* Wait for the first serial output from the source */
>> +    wait_for_serial("src_serial");
>> +
>> +    migrate(from, uri);
>> +
>> +    wait_for_migration_pass(from);
>> +
>> +    /* 1GB/s now it should converge */
>> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");
>
> Curious: would it possible that 1GB/s won't suffice in some cases?
> Is there any reason behind this 1GB/s value?

No.

My new test setup just uses 1GB/s

and plays with the downtime (1ms) while I want to wait for one full
cycle
and then 300ms.

Notice that the guest is around 100MB of size, so ....


>
> (I agree with Thomas that it would be good to work upon the existing
>  postcopy-test.c file, at least easier to review :)

Done.

did the other way around:

rename postcopy-test.c to migrate-test.c

and then went from there.

Later, Juan.

Re: [Qemu-devel] [PATCH 2/6] tests: Add basic migration precopy tcp test

[Juan Quintela]

"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  tests/migration-test.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>> 
>> diff --git a/tests/migration-test.c b/tests/migration-test.c
>> index cd954caee4..8ef2b72459 100644
>> --- a/tests/migration-test.c
>> +++ b/tests/migration-test.c
>> @@ -466,6 +466,13 @@ static void test_precopy_unix(void)
>>      g_free(uri);
>>  }
>>  
>> +static void test_precopy_tcp(void)
>> +{
>> +    const char *uri = "tcp:0:44444";
>> +
>> +    test_precopy(uri);
>> +}
>
> The problem is that this will fail if you make check -j  and two
> tests use 4444 at the same time.
> That's pretty common especially when you've built multiple architectures
> as targets and all the same tests run at the same time.

Options:

- Use different start value by architecture.
- as we have a directory use O_CREAT|O_EXCL  and do create the file with
- the port number that we want to use

Anything else that you want to propose?

Later, Juan.


>
> Dave
>
>>  int main(int argc, char **argv)
>>  {
>>      char template[] = "/tmp/migration-test-XXXXXX";
>> @@ -482,6 +489,7 @@ int main(int argc, char **argv)
>>      module_call_init(MODULE_INIT_QOM);
>>  
>>      qtest_add_func("/migration/precopy/unix", test_precopy_unix);
>> +    qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
>>  
>>      ret = g_test_run();
>>  
>> -- 
>> 2.13.5
>> 
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 3/6] tests: Add precopy test using deprecated commands

[Juan Quintela]

"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  tests/migration-test.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 71 insertions(+)
>> 
>> diff --git a/tests/migration-test.c b/tests/migration-test.c
>> index 8ef2b72459..8549e4c2c6 100644
>> --- a/tests/migration-test.c
>> +++ b/tests/migration-test.c
>> @@ -422,6 +422,76 @@ static void migrate_set_parameter(QTestState *who, const char *parameter,
>>      QDECREF(rsp);
>>  }
>>  
>> +static void migrate_set_downtime(QTestState *who, const char *value)
>> +{
>> +    QDict *rsp;
>> +    gchar *cmd;
>> +
>> +    cmd = g_strdup_printf("{ 'execute': 'migrate_set_downtime',"
>> +                          "'arguments': { 'value': %s } }", value);
>> +    rsp = qtest_qmp(who, cmd);
>> +    g_free(cmd);
>> +    g_assert(qdict_haskey(rsp, "return"));
>> +    QDECREF(rsp);
>> +}
>> +
>> +static void migrate_set_speed(QTestState *who, const char *value)
>> +{
>> +    QDict *rsp;
>> +    gchar *cmd;
>> +
>> +    cmd = g_strdup_printf("{ 'execute': 'migrate_set_speed',"
>> +                          "'arguments': { 'value': %s } }", value);
>> +    rsp = qtest_qmp(who, cmd);
>> +    g_free(cmd);
>> +    g_assert(qdict_haskey(rsp, "return"));
>> +    QDECREF(rsp);
>> +}
>> +
>> +static void test_precopy_deprecated(const char *uri)
>> +{
>> +    QTestState *from, *to;
>> +
>> +    test_migrate_start(&from, &to, uri);
>> +
>> +    /* We want to pick a speed slow enough that the test completes
>> +     * quickly, but that it doesn't complete precopy even on a slow
>> +     * machine, so also set the downtime.
>> +     */
>> +    /* 100 ms */
>> +    migrate_set_downtime(from, "100");
>> +    /* 1MB/s slow*/
>> +    migrate_set_speed(from, "100000000");
>
> Can't we share more/most of this function?
> For example, how about we have a migrate_set_speed(who,value, old)
> tht is :
>   if (old) {
>       migrate_set_speed...
>   } else {
>       migrate_set_parameter
>   }
>
> then test_precopy(const char *uri, bool old)
>
> and we've duped most of it?

Moved to different setup, we end having:

- migrate_set_parameter

and

- migrate_set_speed
- migrate_set_downtime
- migrate_set_cache_size

My plan would be to test that if you don't use deprecated interfaces,
you don't need to deal with them.

Later, Juan.

Re: [Qemu-devel] [PATCH 4/6] tests: Add migration xbzrle test

[Juan Quintela]

"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> Signed-off-by: Juan Quintela <quintela@redhat.com>

> +static void test_xbzrle(const char *uri)
>> +{
>> +    QTestState *from, *to;
>> +
>> +    test_migrate_start(&from, &to, uri);
>> +
>> +    /* We want to pick a speed slow enough that the test completes
>> +     * quickly, but that it doesn't complete precopy even on a slow
>> +     * machine, so also set the downtime.
>> +     */
>> +    /* 100 ms */
>> +    migrate_set_parameter(from, "downtime-limit", "100");
>> +    /* 1MB/s slow*/
>> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
>> +
>> +    migrate_set_cache_size(from, "50000000");
>
> Why 50MB?  I guess this is an interesting test; the program
> dirties 100MB of RAM repeatedly, one byte/page.  So 50MB is probably
> not much use becuase it'll thrash.

No good reason.
What I want (tm):
- if you have fast hardware, test is as fast as it can
- if you don't have fast hardware (or very overloaded), it don't fail

My current idea is:
- using 1GB/s for speed
- using downtime = 1ms when I want to wait for migration not converge
- using downtime = 300ms when I want it to finish

I am open to other numbers.

>> +    migrate_set_capability(from, "xbzrle", "true");
>> +    migrate_set_capability(to, "xbzrle", "true");
>> +    /* Wait for the first serial output from the source */
>> +    wait_for_serial("src_serial");
>> +
>> +    migrate(from, uri);
>> +
>> +    wait_for_migration_pass(from);
>> +
>> +    /* 1GB/s now it should converge */
>> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");
>
> I wonder about doing a set_cache_size 150MB here, with a delay before
> the max-bandwidht?  If we're lucky at 150MB it'll start actually using
> the XBZRLE compression.
>
>> +    if (!got_stop) {
>> +        qtest_qmp_eventwait(from, "STOP");
>> +    }
>> +    qtest_qmp_eventwait(to, "RESUME");
>> +
>> +    wait_for_serial("dest_serial");
>> +    wait_for_migration_complete(from);
>> +
>> +    test_migrate_end(from, to);
>
> Extract xbzrle stats?

Thinking about that.

I actually would want to test that when I set something, read it back
that it is what I setted.  I gues I would use a similar trick to what
you do to get the pass number.

Later, Juan.

Re: [Qemu-devel] [PATCH 5/6] tests: Add migration compress threads tests

[Juan Quintela]

"Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> * Juan Quintela (quintela@redhat.com) wrote:
>> For some reason, compression is not working at the moment, test is
>> disabled until I found why.
>> 
>> Signed-off-by: Juan Quintela <quintela@redhat.com>
>> ---
>>  tests/migration-test.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 51 insertions(+)
>> 
>> diff --git a/tests/migration-test.c b/tests/migration-test.c
>> index f0734e4ba0..a8abc3d007 100644
>> --- a/tests/migration-test.c
>> +++ b/tests/migration-test.c
>> @@ -621,6 +621,54 @@ static void test_xbzrle_unix(void)
>>      g_free(uri);
>>  }
>>  
>> +static void test_compress(const char *uri)
>> +{
>> +    QTestState *from, *to;
>> +
>> +    test_migrate_start(&from, &to, uri);
>> +
>> +    /* We want to pick a speed slow enough that the test completes
>> +     * quickly, but that it doesn't complete precopy even on a slow
>> +     * machine, so also set the downtime.
>> +     */
>> +    /* 100 ms */
>> +    migrate_set_parameter(from, "downtime-limit", "100");
>> +    /* 1MB/s slow*/
>> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
>> +
>> +    migrate_set_parameter(from, "compress-threads", "4");
>> +    migrate_set_parameter(from, "decompress-threads", "3"); 
>
> Should that be 'to' ?

Yes.

> I worry about having this many threads in a loaded test environment.

Suggestion is?

I am not getting they working reliabely without having it loaded.

Later, Juan.

Re: [Qemu-devel] [PATCH 5/6] tests: Add migration compress threads tests

[Juan Quintela]

Peter Xu <peterx@redhat.com> wrote:
> On Wed, Oct 04, 2017 at 12:39:32PM +0200, Juan Quintela wrote:
>
> [...]
>
>>  int main(int argc, char **argv)
>>  {
>> @@ -641,6 +689,9 @@ int main(int argc, char **argv)
>>      qtest_add_func("/migration/precopy/tcp", test_precopy_tcp);
>>      qtest_add_func("/migration/deprecated/unix", test_deprecated_unix);
>>      qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
>
> Nits: Not sure whether we can have better naming for the tests?  Say:
>
>   /migration/precopy/unix
>   /migration/precopy/tcp
>   /migration/deprecated-cmds
>   /migration/xbzrle
>   /migration/compression
>
> I'm fine with existing naming as well.
>
> For the deprecated commands test, not sure whether we can just send
> those commands and query using "query-migrate-parameters" to make sure
> they are setup and valid.  I assume it can be faster than real
> migrations.

Yeap, having done that because I wanted people to say what they thought
before I invested too much time.

>> +    if (0) {
>
> Is this intended? :)

I think I put that on the cover letter.  It is not working reliabely to
me, and as this was more an RFC than anything else ....

Thanks, Juan.

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Daniel P. Berrange]

On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> Signed-off-by: Juan Quintela <quintela@redhat.com>
> ---
>  tests/Makefile.include |   3 +
>  tests/migration-test.c | 497 +++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 500 insertions(+)
>  create mode 100644 tests/migration-test.c
> 


> +
> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> + * outputing a 'B' every so often if it's still running.
> + */


Can you add a comment about what it is actually doing to dirty the memory.

eg is it writing a different random value at each byte of RAM, or picking
a random value at the stasrt of each iteration and writing it to all RAM,
or it is just xoring current memory contents, or something else ?

Understanding this would be helpful in understanding how well the tests
for xbzrle / compression are working, because the pattern of dirtying
memory will affect whether the compression does anything useful or not.

> +unsigned char bootsect[] = {
> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> +};
> +
> +static void init_bootfile_x86(const char *bootpath)
> +{
> +    FILE *bootfile = fopen(bootpath, "wb");
> +
> +    g_assert_cmpint(fwrite(bootsect, 512, 1, bootfile), ==, 1);
> +    fclose(bootfile);
> +}
> +
> +static void init_bootfile_ppc(const char *bootpath)
> +{
> +    FILE *bootfile;
> +    char buf[MIN_NVRAM_SIZE];
> +    ChrpNvramPartHdr *header = (ChrpNvramPartHdr *)buf;
> +
> +    memset(buf, 0, MIN_NVRAM_SIZE);
> +
> +    /* Create a "common" partition in nvram to store boot-command property */
> +
> +    header->signature = CHRP_NVPART_SYSTEM;
> +    memcpy(header->name, "common", 6);
> +    chrp_nvram_finish_partition(header, MIN_NVRAM_SIZE);
> +
> +    /* FW_MAX_SIZE is 4MB, but slof.bin is only 900KB,
> +     * so let's modify memory between 1MB and 100MB
> +     * to do like PC bootsector
> +     */
> +
> +    sprintf(buf + 16,
> +            "boot-command=hex .\" _\" begin %x %x do i c@ 1 + i c! 1000 +loop "
> +            ".\" B\" 0 until", end_address, start_address);
> +
> +    /* Write partition to the NVRAM file */
> +
> +    bootfile = fopen(bootpath, "wb");
> +    g_assert_cmpint(fwrite(buf, MIN_NVRAM_SIZE, 1, bootfile), ==, 1);
> +    fclose(bootfile);
> +}
> +
> +/*
> + * Wait for some output in the serial output file,
> + * we get an 'A' followed by an endless string of 'B's
> + * but on the destination we won't have the A.
> + */
> +static void wait_for_serial(const char *side)
> +{
> +    char *serialpath = g_strdup_printf("%s/%s", tmpfs, side);
> +    FILE *serialfile = fopen(serialpath, "r");
> +    const char *arch = qtest_get_arch();
> +    int started = (strcmp(side, "src_serial") == 0 &&
> +                   strcmp(arch, "ppc64") == 0) ? 0 : 1;
> +
> +    g_free(serialpath);
> +    do {
> +        int readvalue = fgetc(serialfile);
> +
> +        if (!started) {
> +            /* SLOF prints its banner before starting test,
> +             * to ignore it, mark the start of the test with '_',
> +             * ignore all characters until this marker
> +             */
> +            switch (readvalue) {
> +            case '_':
> +                started = 1;
> +                break;
> +            case EOF:
> +                fseek(serialfile, 0, SEEK_SET);
> +                usleep(1000);
> +                break;
> +            }
> +            continue;
> +        }
> +        switch (readvalue) {
> +        case 'A':
> +            /* Fine */
> +            break;
> +
> +        case 'B':
> +            /* It's alive! */
> +            fclose(serialfile);
> +            return;
> +
> +        case EOF:
> +            started = (strcmp(side, "src_serial") == 0 &&
> +                       strcmp(arch, "ppc64") == 0) ? 0 : 1;
> +            fseek(serialfile, 0, SEEK_SET);
> +            usleep(1000);
> +            break;
> +
> +        default:
> +            fprintf(stderr, "Unexpected %d on %s serial\n", readvalue, side);
> +            g_assert_not_reached();
> +        }
> +    } while (true);
> +}
> +
> +/*
> + * Events can get in the way of responses we are actually waiting for.
> + */
> +static QDict *return_or_event(QDict *response, QTestState *who)
> +{
> +    const char *event_string;
> +    if (!qdict_haskey(response, "event")) {
> +        return response;
> +    }
> +
> +    /* OK, it was an event */
> +    event_string = qdict_get_str(response, "event");
> +    if (!strcmp(event_string, "STOP")) {
> +        got_stop = true;
> +    }
> +    QDECREF(response);
> +    return return_or_event(qtest_qmp_receive(who), who);
> +}
> +
> +
> +/*
> + * It's tricky to use qemu's migration event capability with qtest,
> + * events suddenly appearing confuse the qmp()/hmp() responses.
> + * so wait for a couple of passes to have happened before
> + * going postcopy.
> + */
> +
> +static uint64_t get_migration_pass(QTestState *who)
> +{
> +    QDict *rsp, *rsp_return, *rsp_ram;
> +    uint64_t result;
> +
> +    rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
> +    rsp_return = qdict_get_qdict(rsp, "return");
> +    if (!qdict_haskey(rsp_return, "ram")) {
> +        /* Still in setup */
> +        result = 0;
> +    } else {
> +        rsp_ram = qdict_get_qdict(rsp_return, "ram");
> +        result = qdict_get_try_int(rsp_ram, "dirty-sync-count", 0);
> +    }
> +    QDECREF(rsp);
> +    return result;
> +}
> +
> +static void wait_for_migration_complete(QTestState *who)
> +{
> +    bool completed;
> +
> +    do {
> +        QDict *rsp, *rsp_return;
> +        const char *status;
> +
> +        rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
> +        rsp_return = qdict_get_qdict(rsp, "return");
> +        status = qdict_get_str(rsp_return, "status");
> +        completed = strcmp(status, "completed") == 0;
> +        g_assert_cmpstr(status, !=,  "failed");
> +        QDECREF(rsp);
> +        usleep(1000 * 100);
> +    } while (!completed);
> +}
> +
> +static void wait_for_migration_pass(QTestState *who)
> +{
> +    uint64_t initial_pass = get_migration_pass(who);
> +    uint64_t pass;
> +
> +    /* Wait for the 1st sync */
> +    do {
> +        initial_pass = get_migration_pass(who);
> +        if (got_stop || initial_pass) {
> +            break;
> +        }
> +        usleep(1000 * 100);
> +    } while (true);
> +
> +    do {
> +        usleep(1000 * 100);
> +        pass = get_migration_pass(who);
> +    } while (pass == initial_pass && !got_stop);
> +}
> +
> +static void check_guests_ram(QTestState *who)
> +{
> +    /* Our ASM test will have been incrementing one byte from each page from
> +     * 1MB to <100MB in order.
> +     * This gives us a constraint that any page's byte should be equal or less
> +     * than the previous pages byte (mod 256); and they should all be equal
> +     * except for one transition at the point where we meet the incrementer.
> +     * (We're running this with the guest stopped).
> +     */
> +    unsigned address;
> +    uint8_t first_byte;
> +    uint8_t last_byte;
> +    bool hit_edge = false;
> +    bool bad = false;
> +
> +    qtest_memread(who, start_address, &first_byte, 1);
> +    last_byte = first_byte;
> +
> +    for (address = start_address + 4096; address < end_address; address += 4096)
> +    {
> +        uint8_t b;
> +        qtest_memread(who, address, &b, 1);
> +        if (b != last_byte) {
> +            if (((b + 1) % 256) == last_byte && !hit_edge) {
> +                /* This is OK, the guest stopped at the point of
> +                 * incrementing the previous page but didn't get
> +                 * to us yet.
> +                 */
> +                hit_edge = true;
> +            } else {
> +                fprintf(stderr, "Memory content inconsistency at %x"
> +                                " first_byte = %x last_byte = %x current = %x"
> +                                " hit_edge = %x\n",
> +                                address, first_byte, last_byte, b, hit_edge);
> +                bad = true;
> +            }
> +        }
> +        last_byte = b;
> +    }
> +    g_assert_false(bad);
> +}
> +
> +static void cleanup(const char *filename)
> +{
> +    char *path = g_strdup_printf("%s/%s", tmpfs, filename);
> +
> +    unlink(path);
> +    g_free(path);
> +}
> +
> +static void test_migrate_start(QTestState **from, QTestState **to,
> +                               const char *uri)
> +{
> +    gchar *cmd_src, *cmd_dst;
> +    char *bootpath = g_strdup_printf("%s/bootsect", tmpfs);
> +    const char *arch = qtest_get_arch();
> +
> +    got_stop = false;
> +
> +    if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) {
> +        init_bootfile_x86(bootpath);
> +        cmd_src = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
> +                                  " -name pcsource,debug-threads=on"
> +                                  " -serial file:%s/src_serial"
> +                                  " -drive file=%s,format=raw",
> +                                  tmpfs, bootpath);
> +        cmd_dst = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
> +                                  " -name pcdest,debug-threads=on"
> +                                  " -serial file:%s/dest_serial"
> +                                  " -drive file=%s,format=raw"
> +                                  " -incoming %s",
> +                                  tmpfs, bootpath, uri);
> +    } else if (strcmp(arch, "ppc64") == 0) {
> +        const char *accel;
> +
> +        /* On ppc64, the test only works with kvm-hv, but not with kvm-pr */
> +        accel = access("/sys/module/kvm_hv", F_OK) ? "tcg" : "kvm:tcg";
> +        init_bootfile_ppc(bootpath);
> +        cmd_src = g_strdup_printf("-machine accel=%s -m 256M"
> +                                  " -name pcsource,debug-threads=on"
> +                                  " -serial file:%s/src_serial"
> +                                  " -drive file=%s,if=pflash,format=raw",
> +                                  accel, tmpfs, bootpath);
> +        cmd_dst = g_strdup_printf("-machine accel=%s -m 256M"
> +                                  " -name pcdest,debug-threads=on"
> +                                  " -serial file:%s/dest_serial"
> +                                  " -incoming %s",
> +                                  accel, tmpfs, uri);
> +    } else {
> +        g_assert_not_reached();
> +    }
> +
> +    g_free(bootpath);
> +
> +    *from = qtest_start(cmd_src);
> +    g_free(cmd_src);
> +
> +    *to = qtest_init(cmd_dst);
> +    g_free(cmd_dst);
> +}
> +
> +static void test_migrate_end(QTestState *from, QTestState *to)
> +{
> +    unsigned char dest_byte_a, dest_byte_b, dest_byte_c, dest_byte_d;
> +
> +    qtest_quit(from);
> +
> +    qtest_memread(to, start_address, &dest_byte_a, 1);
> +
> +    /* Destination still running, wait for a byte to change */
> +    do {
> +        qtest_memread(to, start_address, &dest_byte_b, 1);
> +        usleep(10 * 1000);
> +    } while (dest_byte_a == dest_byte_b);
> +
> +    qtest_qmp_discard_response(to, "{ 'execute' : 'stop'}");
> +    /* With it stopped, check nothing changes */
> +    qtest_memread(to, start_address, &dest_byte_c, 1);
> +    sleep(1);
> +    qtest_memread(to, start_address, &dest_byte_d, 1);
> +    g_assert_cmpint(dest_byte_c, ==, dest_byte_d);
> +
> +    check_guests_ram(to);
> +
> +    qtest_quit(to);
> +
> +    cleanup("bootsect");
> +    cleanup("migsocket");
> +    cleanup("src_serial");
> +    cleanup("dest_serial");
> +}
> +
> +static void migrate(QTestState *who, const char *uri)
> +{
> +    QDict *rsp;
> +    gchar *cmd;
> +
> +    cmd = g_strdup_printf("{ 'execute': 'migrate',"
> +                          "'arguments': { 'uri': '%s' } }",
> +                          uri);
> +    rsp = qtest_qmp(who, cmd);
> +    g_free(cmd);
> +    g_assert(qdict_haskey(rsp, "return"));
> +    QDECREF(rsp);
> +}
> +
> +static void migrate_set_parameter(QTestState *who, const char *parameter,
> +                                  const char *value)
> +{
> +    QDict *rsp;
> +    gchar *cmd;
> +
> +    cmd = g_strdup_printf("{ 'execute': 'migrate-set-parameters',"
> +                          "'arguments': { '%s': %s } }",
> +                          parameter, value);
> +    rsp = qtest_qmp(who, cmd);
> +    g_free(cmd);
> +    g_assert(qdict_haskey(rsp, "return"));
> +    QDECREF(rsp);
> +}
> +
> +static void test_precopy(const char *uri)
> +{
> +    QTestState *from, *to;
> +
> +    test_migrate_start(&from, &to, uri);
> +
> +    /* We want to pick a speed slow enough that the test completes
> +     * quickly, but that it doesn't complete precopy even on a slow
> +     * machine, so also set the downtime.
> +     */
> +    /* 100 ms */
> +    migrate_set_parameter(from, "downtime-limit", "100");
> +    /* 1MB/s slow*/
> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
> +
> +    /* Wait for the first serial output from the source */
> +    wait_for_serial("src_serial");
> +
> +    migrate(from, uri);
> +
> +    wait_for_migration_pass(from);
> +
> +    /* 1GB/s now it should converge */
> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");
> +
> +    if (!got_stop) {
> +        qtest_qmp_eventwait(from, "STOP");
> +    }
> +    qtest_qmp_eventwait(to, "RESUME");
> +
> +    wait_for_serial("dest_serial");
> +    wait_for_migration_complete(from);
> +
> +    test_migrate_end(from, to);
> +}
> +
> +static void test_precopy_unix(void)
> +{
> +    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
> +
> +    test_precopy(uri);
> +    g_free(uri);
> +}
> +
> +int main(int argc, char **argv)
> +{
> +    char template[] = "/tmp/migration-test-XXXXXX";
> +    int ret;
> +
> +    g_test_init(&argc, &argv, NULL);
> +
> +    tmpfs = mkdtemp(template);
> +    if (!tmpfs) {
> +        g_test_message("mkdtemp on path (%s): %s\n", template, strerror(errno));
> +    }
> +    g_assert(tmpfs);
> +
> +    module_call_init(MODULE_INIT_QOM);
> +
> +    qtest_add_func("/migration/precopy/unix", test_precopy_unix);
> +
> +    ret = g_test_run();
> +
> +    g_assert_cmpint(ret, ==, 0);
> +
> +    ret = rmdir(tmpfs);
> +    if (ret != 0) {
> +        g_test_message("unable to rmdir: path (%s): %s\n",
> +                       tmpfs, strerror(errno));
> +    }
> +
> +    return ret;
> +}
> -- 
> 2.13.5
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Daniel P. Berrange]

On Wed, Oct 18, 2017 at 01:36:56PM +0200, Juan Quintela wrote:
> Peter Xu <peterx@redhat.com> wrote:
> > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> >
> > [...]
> >
> >> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> >> + * outputing a 'B' every so often if it's still running.
> >> + */
> >> +unsigned char bootsect[] = {
> >> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> >> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> >> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> >> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> >> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> >> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> >> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> >> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> >> +};
> >
> > Not sure whether it would be nicer to put this section as binary into
> > QEMU's tree as shared, then other test code can use it too directly,
> > and it would be easier if someone wants to boot a VM running this
> > bootstrap code.  Thanks,
> 
> I am sharing this code now on a single place.  About getting this
> outside ... I will wait for someone that understand how this works, I
> have no clue neither how it works nor how to share/use/setup/....

FWIW, we already have a reusuable program that can be used to generate
load stress for testing live migration see tests/migration/stress.c
This gets compiled into a static binary, put into an initrd, and then
run by booting the host OS kernel. So the caveat is that this approach
only lets you test a QEMU emulator whose target arch matches the host.
That isn't much different from this boot sector though which only lets
you test the x86 system emulator.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Dr. David Alan Gilbert]

* Daniel P. Berrange (berrange@redhat.com) wrote:
> On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > Signed-off-by: Juan Quintela <quintela@redhat.com>
> > ---
> >  tests/Makefile.include |   3 +
> >  tests/migration-test.c | 497 +++++++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 500 insertions(+)
> >  create mode 100644 tests/migration-test.c
> > 
> 
> 
> > +
> > +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > + * outputing a 'B' every so often if it's still running.
> > + */
> 
> 
> Can you add a comment about what it is actually doing to dirty the memory.

Yes, so this is the code from the postcopy tests, the commit message
ea0c6d62 has the actual source for it.

> eg is it writing a different random value at each byte of RAM, or picking
> a random value at the stasrt of each iteration and writing it to all RAM,
> or it is just xoring current memory contents, or something else ?
> 
> Understanding this would be helpful in understanding how well the tests
> for xbzrle / compression are working, because the pattern of dirtying
> memory will affect whether the compression does anything useful or not.

It rolls through 100MB of RAM, incrementing the 1st byte of each page in
sequence and then goes back and does it again.  So it should be
excellent for xbzrle.

Dave

> > +unsigned char bootsect[] = {
> > +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > +};
> > +
> > +static void init_bootfile_x86(const char *bootpath)
> > +{
> > +    FILE *bootfile = fopen(bootpath, "wb");
> > +
> > +    g_assert_cmpint(fwrite(bootsect, 512, 1, bootfile), ==, 1);
> > +    fclose(bootfile);
> > +}
> > +
> > +static void init_bootfile_ppc(const char *bootpath)
> > +{
> > +    FILE *bootfile;
> > +    char buf[MIN_NVRAM_SIZE];
> > +    ChrpNvramPartHdr *header = (ChrpNvramPartHdr *)buf;
> > +
> > +    memset(buf, 0, MIN_NVRAM_SIZE);
> > +
> > +    /* Create a "common" partition in nvram to store boot-command property */
> > +
> > +    header->signature = CHRP_NVPART_SYSTEM;
> > +    memcpy(header->name, "common", 6);
> > +    chrp_nvram_finish_partition(header, MIN_NVRAM_SIZE);
> > +
> > +    /* FW_MAX_SIZE is 4MB, but slof.bin is only 900KB,
> > +     * so let's modify memory between 1MB and 100MB
> > +     * to do like PC bootsector
> > +     */
> > +
> > +    sprintf(buf + 16,
> > +            "boot-command=hex .\" _\" begin %x %x do i c@ 1 + i c! 1000 +loop "
> > +            ".\" B\" 0 until", end_address, start_address);
> > +
> > +    /* Write partition to the NVRAM file */
> > +
> > +    bootfile = fopen(bootpath, "wb");
> > +    g_assert_cmpint(fwrite(buf, MIN_NVRAM_SIZE, 1, bootfile), ==, 1);
> > +    fclose(bootfile);
> > +}
> > +
> > +/*
> > + * Wait for some output in the serial output file,
> > + * we get an 'A' followed by an endless string of 'B's
> > + * but on the destination we won't have the A.
> > + */
> > +static void wait_for_serial(const char *side)
> > +{
> > +    char *serialpath = g_strdup_printf("%s/%s", tmpfs, side);
> > +    FILE *serialfile = fopen(serialpath, "r");
> > +    const char *arch = qtest_get_arch();
> > +    int started = (strcmp(side, "src_serial") == 0 &&
> > +                   strcmp(arch, "ppc64") == 0) ? 0 : 1;
> > +
> > +    g_free(serialpath);
> > +    do {
> > +        int readvalue = fgetc(serialfile);
> > +
> > +        if (!started) {
> > +            /* SLOF prints its banner before starting test,
> > +             * to ignore it, mark the start of the test with '_',
> > +             * ignore all characters until this marker
> > +             */
> > +            switch (readvalue) {
> > +            case '_':
> > +                started = 1;
> > +                break;
> > +            case EOF:
> > +                fseek(serialfile, 0, SEEK_SET);
> > +                usleep(1000);
> > +                break;
> > +            }
> > +            continue;
> > +        }
> > +        switch (readvalue) {
> > +        case 'A':
> > +            /* Fine */
> > +            break;
> > +
> > +        case 'B':
> > +            /* It's alive! */
> > +            fclose(serialfile);
> > +            return;
> > +
> > +        case EOF:
> > +            started = (strcmp(side, "src_serial") == 0 &&
> > +                       strcmp(arch, "ppc64") == 0) ? 0 : 1;
> > +            fseek(serialfile, 0, SEEK_SET);
> > +            usleep(1000);
> > +            break;
> > +
> > +        default:
> > +            fprintf(stderr, "Unexpected %d on %s serial\n", readvalue, side);
> > +            g_assert_not_reached();
> > +        }
> > +    } while (true);
> > +}
> > +
> > +/*
> > + * Events can get in the way of responses we are actually waiting for.
> > + */
> > +static QDict *return_or_event(QDict *response, QTestState *who)
> > +{
> > +    const char *event_string;
> > +    if (!qdict_haskey(response, "event")) {
> > +        return response;
> > +    }
> > +
> > +    /* OK, it was an event */
> > +    event_string = qdict_get_str(response, "event");
> > +    if (!strcmp(event_string, "STOP")) {
> > +        got_stop = true;
> > +    }
> > +    QDECREF(response);
> > +    return return_or_event(qtest_qmp_receive(who), who);
> > +}
> > +
> > +
> > +/*
> > + * It's tricky to use qemu's migration event capability with qtest,
> > + * events suddenly appearing confuse the qmp()/hmp() responses.
> > + * so wait for a couple of passes to have happened before
> > + * going postcopy.
> > + */
> > +
> > +static uint64_t get_migration_pass(QTestState *who)
> > +{
> > +    QDict *rsp, *rsp_return, *rsp_ram;
> > +    uint64_t result;
> > +
> > +    rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
> > +    rsp_return = qdict_get_qdict(rsp, "return");
> > +    if (!qdict_haskey(rsp_return, "ram")) {
> > +        /* Still in setup */
> > +        result = 0;
> > +    } else {
> > +        rsp_ram = qdict_get_qdict(rsp_return, "ram");
> > +        result = qdict_get_try_int(rsp_ram, "dirty-sync-count", 0);
> > +    }
> > +    QDECREF(rsp);
> > +    return result;
> > +}
> > +
> > +static void wait_for_migration_complete(QTestState *who)
> > +{
> > +    bool completed;
> > +
> > +    do {
> > +        QDict *rsp, *rsp_return;
> > +        const char *status;
> > +
> > +        rsp = return_or_event(qmp("{ 'execute': 'query-migrate' }"), who);
> > +        rsp_return = qdict_get_qdict(rsp, "return");
> > +        status = qdict_get_str(rsp_return, "status");
> > +        completed = strcmp(status, "completed") == 0;
> > +        g_assert_cmpstr(status, !=,  "failed");
> > +        QDECREF(rsp);
> > +        usleep(1000 * 100);
> > +    } while (!completed);
> > +}
> > +
> > +static void wait_for_migration_pass(QTestState *who)
> > +{
> > +    uint64_t initial_pass = get_migration_pass(who);
> > +    uint64_t pass;
> > +
> > +    /* Wait for the 1st sync */
> > +    do {
> > +        initial_pass = get_migration_pass(who);
> > +        if (got_stop || initial_pass) {
> > +            break;
> > +        }
> > +        usleep(1000 * 100);
> > +    } while (true);
> > +
> > +    do {
> > +        usleep(1000 * 100);
> > +        pass = get_migration_pass(who);
> > +    } while (pass == initial_pass && !got_stop);
> > +}
> > +
> > +static void check_guests_ram(QTestState *who)
> > +{
> > +    /* Our ASM test will have been incrementing one byte from each page from
> > +     * 1MB to <100MB in order.
> > +     * This gives us a constraint that any page's byte should be equal or less
> > +     * than the previous pages byte (mod 256); and they should all be equal
> > +     * except for one transition at the point where we meet the incrementer.
> > +     * (We're running this with the guest stopped).
> > +     */
> > +    unsigned address;
> > +    uint8_t first_byte;
> > +    uint8_t last_byte;
> > +    bool hit_edge = false;
> > +    bool bad = false;
> > +
> > +    qtest_memread(who, start_address, &first_byte, 1);
> > +    last_byte = first_byte;
> > +
> > +    for (address = start_address + 4096; address < end_address; address += 4096)
> > +    {
> > +        uint8_t b;
> > +        qtest_memread(who, address, &b, 1);
> > +        if (b != last_byte) {
> > +            if (((b + 1) % 256) == last_byte && !hit_edge) {
> > +                /* This is OK, the guest stopped at the point of
> > +                 * incrementing the previous page but didn't get
> > +                 * to us yet.
> > +                 */
> > +                hit_edge = true;
> > +            } else {
> > +                fprintf(stderr, "Memory content inconsistency at %x"
> > +                                " first_byte = %x last_byte = %x current = %x"
> > +                                " hit_edge = %x\n",
> > +                                address, first_byte, last_byte, b, hit_edge);
> > +                bad = true;
> > +            }
> > +        }
> > +        last_byte = b;
> > +    }
> > +    g_assert_false(bad);
> > +}
> > +
> > +static void cleanup(const char *filename)
> > +{
> > +    char *path = g_strdup_printf("%s/%s", tmpfs, filename);
> > +
> > +    unlink(path);
> > +    g_free(path);
> > +}
> > +
> > +static void test_migrate_start(QTestState **from, QTestState **to,
> > +                               const char *uri)
> > +{
> > +    gchar *cmd_src, *cmd_dst;
> > +    char *bootpath = g_strdup_printf("%s/bootsect", tmpfs);
> > +    const char *arch = qtest_get_arch();
> > +
> > +    got_stop = false;
> > +
> > +    if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) {
> > +        init_bootfile_x86(bootpath);
> > +        cmd_src = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
> > +                                  " -name pcsource,debug-threads=on"
> > +                                  " -serial file:%s/src_serial"
> > +                                  " -drive file=%s,format=raw",
> > +                                  tmpfs, bootpath);
> > +        cmd_dst = g_strdup_printf("-machine accel=kvm:tcg -m 150M"
> > +                                  " -name pcdest,debug-threads=on"
> > +                                  " -serial file:%s/dest_serial"
> > +                                  " -drive file=%s,format=raw"
> > +                                  " -incoming %s",
> > +                                  tmpfs, bootpath, uri);
> > +    } else if (strcmp(arch, "ppc64") == 0) {
> > +        const char *accel;
> > +
> > +        /* On ppc64, the test only works with kvm-hv, but not with kvm-pr */
> > +        accel = access("/sys/module/kvm_hv", F_OK) ? "tcg" : "kvm:tcg";
> > +        init_bootfile_ppc(bootpath);
> > +        cmd_src = g_strdup_printf("-machine accel=%s -m 256M"
> > +                                  " -name pcsource,debug-threads=on"
> > +                                  " -serial file:%s/src_serial"
> > +                                  " -drive file=%s,if=pflash,format=raw",
> > +                                  accel, tmpfs, bootpath);
> > +        cmd_dst = g_strdup_printf("-machine accel=%s -m 256M"
> > +                                  " -name pcdest,debug-threads=on"
> > +                                  " -serial file:%s/dest_serial"
> > +                                  " -incoming %s",
> > +                                  accel, tmpfs, uri);
> > +    } else {
> > +        g_assert_not_reached();
> > +    }
> > +
> > +    g_free(bootpath);
> > +
> > +    *from = qtest_start(cmd_src);
> > +    g_free(cmd_src);
> > +
> > +    *to = qtest_init(cmd_dst);
> > +    g_free(cmd_dst);
> > +}
> > +
> > +static void test_migrate_end(QTestState *from, QTestState *to)
> > +{
> > +    unsigned char dest_byte_a, dest_byte_b, dest_byte_c, dest_byte_d;
> > +
> > +    qtest_quit(from);
> > +
> > +    qtest_memread(to, start_address, &dest_byte_a, 1);
> > +
> > +    /* Destination still running, wait for a byte to change */
> > +    do {
> > +        qtest_memread(to, start_address, &dest_byte_b, 1);
> > +        usleep(10 * 1000);
> > +    } while (dest_byte_a == dest_byte_b);
> > +
> > +    qtest_qmp_discard_response(to, "{ 'execute' : 'stop'}");
> > +    /* With it stopped, check nothing changes */
> > +    qtest_memread(to, start_address, &dest_byte_c, 1);
> > +    sleep(1);
> > +    qtest_memread(to, start_address, &dest_byte_d, 1);
> > +    g_assert_cmpint(dest_byte_c, ==, dest_byte_d);
> > +
> > +    check_guests_ram(to);
> > +
> > +    qtest_quit(to);
> > +
> > +    cleanup("bootsect");
> > +    cleanup("migsocket");
> > +    cleanup("src_serial");
> > +    cleanup("dest_serial");
> > +}
> > +
> > +static void migrate(QTestState *who, const char *uri)
> > +{
> > +    QDict *rsp;
> > +    gchar *cmd;
> > +
> > +    cmd = g_strdup_printf("{ 'execute': 'migrate',"
> > +                          "'arguments': { 'uri': '%s' } }",
> > +                          uri);
> > +    rsp = qtest_qmp(who, cmd);
> > +    g_free(cmd);
> > +    g_assert(qdict_haskey(rsp, "return"));
> > +    QDECREF(rsp);
> > +}
> > +
> > +static void migrate_set_parameter(QTestState *who, const char *parameter,
> > +                                  const char *value)
> > +{
> > +    QDict *rsp;
> > +    gchar *cmd;
> > +
> > +    cmd = g_strdup_printf("{ 'execute': 'migrate-set-parameters',"
> > +                          "'arguments': { '%s': %s } }",
> > +                          parameter, value);
> > +    rsp = qtest_qmp(who, cmd);
> > +    g_free(cmd);
> > +    g_assert(qdict_haskey(rsp, "return"));
> > +    QDECREF(rsp);
> > +}
> > +
> > +static void test_precopy(const char *uri)
> > +{
> > +    QTestState *from, *to;
> > +
> > +    test_migrate_start(&from, &to, uri);
> > +
> > +    /* We want to pick a speed slow enough that the test completes
> > +     * quickly, but that it doesn't complete precopy even on a slow
> > +     * machine, so also set the downtime.
> > +     */
> > +    /* 100 ms */
> > +    migrate_set_parameter(from, "downtime-limit", "100");
> > +    /* 1MB/s slow*/
> > +    migrate_set_parameter(from, "max-bandwidth", "100000000");
> > +
> > +    /* Wait for the first serial output from the source */
> > +    wait_for_serial("src_serial");
> > +
> > +    migrate(from, uri);
> > +
> > +    wait_for_migration_pass(from);
> > +
> > +    /* 1GB/s now it should converge */
> > +    migrate_set_parameter(from, "max-bandwidth", "1000000000");
> > +
> > +    if (!got_stop) {
> > +        qtest_qmp_eventwait(from, "STOP");
> > +    }
> > +    qtest_qmp_eventwait(to, "RESUME");
> > +
> > +    wait_for_serial("dest_serial");
> > +    wait_for_migration_complete(from);
> > +
> > +    test_migrate_end(from, to);
> > +}
> > +
> > +static void test_precopy_unix(void)
> > +{
> > +    char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
> > +
> > +    test_precopy(uri);
> > +    g_free(uri);
> > +}
> > +
> > +int main(int argc, char **argv)
> > +{
> > +    char template[] = "/tmp/migration-test-XXXXXX";
> > +    int ret;
> > +
> > +    g_test_init(&argc, &argv, NULL);
> > +
> > +    tmpfs = mkdtemp(template);
> > +    if (!tmpfs) {
> > +        g_test_message("mkdtemp on path (%s): %s\n", template, strerror(errno));
> > +    }
> > +    g_assert(tmpfs);
> > +
> > +    module_call_init(MODULE_INIT_QOM);
> > +
> > +    qtest_add_func("/migration/precopy/unix", test_precopy_unix);
> > +
> > +    ret = g_test_run();
> > +
> > +    g_assert_cmpint(ret, ==, 0);
> > +
> > +    ret = rmdir(tmpfs);
> > +    if (ret != 0) {
> > +        g_test_message("unable to rmdir: path (%s): %s\n",
> > +                       tmpfs, strerror(errno));
> > +    }
> > +
> > +    return ret;
> > +}
> > -- 
> > 2.13.5
> > 
> > 
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Daniel P. Berrange]

On Wed, Oct 18, 2017 at 01:49:42PM +0100, Dr. David Alan Gilbert wrote:
> * Daniel P. Berrange (berrange@redhat.com) wrote:
> > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > > Signed-off-by: Juan Quintela <quintela@redhat.com>
> > > ---
> > >  tests/Makefile.include |   3 +
> > >  tests/migration-test.c | 497 +++++++++++++++++++++++++++++++++++++++++++++++++
> > >  2 files changed, 500 insertions(+)
> > >  create mode 100644 tests/migration-test.c
> > > 
> > 
> > 
> > > +
> > > +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > > + * outputing a 'B' every so often if it's still running.
> > > + */
> > 
> > 
> > Can you add a comment about what it is actually doing to dirty the memory.
> 
> Yes, so this is the code from the postcopy tests, the commit message
> ea0c6d62 has the actual source for it.

Ah ha, thanks.

> > eg is it writing a different random value at each byte of RAM, or picking
> > a random value at the stasrt of each iteration and writing it to all RAM,
> > or it is just xoring current memory contents, or something else ?
> > 
> > Understanding this would be helpful in understanding how well the tests
> > for xbzrle / compression are working, because the pattern of dirtying
> > memory will affect whether the compression does anything useful or not.
> 
> It rolls through 100MB of RAM, incrementing the 1st byte of each page in
> sequence and then goes back and does it again.  So it should be
> excellent for xbzrle.

Yes

> 
> Dave
> 
> > > +unsigned char bootsect[] = {
> > > +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > > +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > > +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > > +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > > +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > > +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > > +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > > +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > > +};

So lets delete this here. Take the bootsect variable in postcopy-test.c
and put it into a migration-bootsector.c file and link it to both tests.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Peter Xu]

On Wed, Oct 18, 2017 at 01:10:38PM +0100, Daniel P. Berrange wrote:
> On Wed, Oct 18, 2017 at 01:36:56PM +0200, Juan Quintela wrote:
> > Peter Xu <peterx@redhat.com> wrote:
> > > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > >
> > > [...]
> > >
> > >> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > >> + * outputing a 'B' every so often if it's still running.
> > >> + */
> > >> +unsigned char bootsect[] = {
> > >> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > >> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > >> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > >> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > >> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > >> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > >> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > >> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > >> +};
> > >
> > > Not sure whether it would be nicer to put this section as binary into
> > > QEMU's tree as shared, then other test code can use it too directly,
> > > and it would be easier if someone wants to boot a VM running this
> > > bootstrap code.  Thanks,
> > 
> > I am sharing this code now on a single place.  About getting this
> > outside ... I will wait for someone that understand how this works, I
> > have no clue neither how it works nor how to share/use/setup/....
> 
> FWIW, we already have a reusuable program that can be used to generate
> load stress for testing live migration see tests/migration/stress.c
> This gets compiled into a static binary, put into an initrd, and then
> run by booting the host OS kernel. So the caveat is that this approach
> only lets you test a QEMU emulator whose target arch matches the host.
> That isn't much different from this boot sector though which only lets
> you test the x86 system emulator.

Yes, if we can have some C program for all architecture then it sounds
nicer, though IIUC this program is somehow special since it should be
writting some "A"/"B" chars to console, and the test program is using
those chars during the process.

Maybe it would be nice to rewrite a C program to replace current
binary sector when someone wants to support postcopy migration test on
a 3rd platform besides x86 and ppc.

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 4/6] tests: Add migration xbzrle test

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> wrote:
> > * Juan Quintela (quintela@redhat.com) wrote:
> >> Signed-off-by: Juan Quintela <quintela@redhat.com>
> 
> > +static void test_xbzrle(const char *uri)
> >> +{
> >> +    QTestState *from, *to;
> >> +
> >> +    test_migrate_start(&from, &to, uri);
> >> +
> >> +    /* We want to pick a speed slow enough that the test completes
> >> +     * quickly, but that it doesn't complete precopy even on a slow
> >> +     * machine, so also set the downtime.
> >> +     */
> >> +    /* 100 ms */
> >> +    migrate_set_parameter(from, "downtime-limit", "100");
> >> +    /* 1MB/s slow*/
> >> +    migrate_set_parameter(from, "max-bandwidth", "100000000");
> >> +
> >> +    migrate_set_cache_size(from, "50000000");
> >
> > Why 50MB?  I guess this is an interesting test; the program
> > dirties 100MB of RAM repeatedly, one byte/page.  So 50MB is probably
> > not much use becuase it'll thrash.
> 
> No good reason.
> What I want (tm):
> - if you have fast hardware, test is as fast as it can
> - if you don't have fast hardware (or very overloaded), it don't fail
> 
> My current idea is:
> - using 1GB/s for speed
> - using downtime = 1ms when I want to wait for migration not converge
> - using downtime = 300ms when I want it to finish
> 
> I am open to other numbers.

Well I was thinking if we could get it to not-converge unless
the xbzrle cache was large enough it would actually make sure we test
xbzrle;  with a 50MB cache you wont test much of xbzrle because I think
it'll be rare to get a cache hit.

Dave
> >> +    migrate_set_capability(from, "xbzrle", "true");
> >> +    migrate_set_capability(to, "xbzrle", "true");
> >> +    /* Wait for the first serial output from the source */
> >> +    wait_for_serial("src_serial");
> >> +
> >> +    migrate(from, uri);
> >> +
> >> +    wait_for_migration_pass(from);
> >> +
> >> +    /* 1GB/s now it should converge */
> >> +    migrate_set_parameter(from, "max-bandwidth", "1000000000");
> >
> > I wonder about doing a set_cache_size 150MB here, with a delay before
> > the max-bandwidht?  If we're lucky at 150MB it'll start actually using
> > the XBZRLE compression.
> >
> >> +    if (!got_stop) {
> >> +        qtest_qmp_eventwait(from, "STOP");
> >> +    }
> >> +    qtest_qmp_eventwait(to, "RESUME");
> >> +
> >> +    wait_for_serial("dest_serial");
> >> +    wait_for_migration_complete(from);
> >> +
> >> +    test_migrate_end(from, to);
> >
> > Extract xbzrle stats?
> 
> Thinking about that.
> 
> I actually would want to test that when I set something, read it back
> that it is what I setted.  I gues I would use a similar trick to what
> you do to get the pass number.
> 
> Later, Juan.
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Daniel P. Berrange]

On Thu, Oct 19, 2017 at 11:18:33AM +0800, Peter Xu wrote:
> On Wed, Oct 18, 2017 at 01:10:38PM +0100, Daniel P. Berrange wrote:
> > On Wed, Oct 18, 2017 at 01:36:56PM +0200, Juan Quintela wrote:
> > > Peter Xu <peterx@redhat.com> wrote:
> > > > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > > >
> > > > [...]
> > > >
> > > >> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > > >> + * outputing a 'B' every so often if it's still running.
> > > >> + */
> > > >> +unsigned char bootsect[] = {
> > > >> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > > >> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > > >> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > > >> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > > >> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > > >> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > > >> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > > >> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > > >> +};
> > > >
> > > > Not sure whether it would be nicer to put this section as binary into
> > > > QEMU's tree as shared, then other test code can use it too directly,
> > > > and it would be easier if someone wants to boot a VM running this
> > > > bootstrap code.  Thanks,
> > > 
> > > I am sharing this code now on a single place.  About getting this
> > > outside ... I will wait for someone that understand how this works, I
> > > have no clue neither how it works nor how to share/use/setup/....
> > 
> > FWIW, we already have a reusuable program that can be used to generate
> > load stress for testing live migration see tests/migration/stress.c
> > This gets compiled into a static binary, put into an initrd, and then
> > run by booting the host OS kernel. So the caveat is that this approach
> > only lets you test a QEMU emulator whose target arch matches the host.
> > That isn't much different from this boot sector though which only lets
> > you test the x86 system emulator.
> 
> Yes, if we can have some C program for all architecture then it sounds
> nicer, though IIUC this program is somehow special since it should be
> writting some "A"/"B" chars to console, and the test program is using
> those chars during the process.

That would be easy enough to add to the current program I expect.

Oh the other difference with the program I wrote is that it is
multi-threaded, so it spawns a writer thread for each vCPU that
is given to the guest. This lets you generate even more extreme
memory dirtying rates if desired :-)

> Maybe it would be nice to rewrite a C program to replace current
> binary sector when someone wants to support postcopy migration test on
> a 3rd platform besides x86 and ppc.

Booting a real Linux kernel and then running the C program as 'init'
is easier to make portable since we don't need to know about special
ways each arch deals with initial boot up.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Dr. David Alan Gilbert]

* Peter Xu (peterx@redhat.com) wrote:
> On Wed, Oct 18, 2017 at 01:10:38PM +0100, Daniel P. Berrange wrote:
> > On Wed, Oct 18, 2017 at 01:36:56PM +0200, Juan Quintela wrote:
> > > Peter Xu <peterx@redhat.com> wrote:
> > > > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > > >
> > > > [...]
> > > >
> > > >> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > > >> + * outputing a 'B' every so often if it's still running.
> > > >> + */
> > > >> +unsigned char bootsect[] = {
> > > >> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > > >> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > > >> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > > >> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > > >> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > > >> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > > >> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > > >> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > > >> +};
> > > >
> > > > Not sure whether it would be nicer to put this section as binary into
> > > > QEMU's tree as shared, then other test code can use it too directly,
> > > > and it would be easier if someone wants to boot a VM running this
> > > > bootstrap code.  Thanks,
> > > 
> > > I am sharing this code now on a single place.  About getting this
> > > outside ... I will wait for someone that understand how this works, I
> > > have no clue neither how it works nor how to share/use/setup/....
> > 
> > FWIW, we already have a reusuable program that can be used to generate
> > load stress for testing live migration see tests/migration/stress.c
> > This gets compiled into a static binary, put into an initrd, and then
> > run by booting the host OS kernel. So the caveat is that this approach
> > only lets you test a QEMU emulator whose target arch matches the host.
> > That isn't much different from this boot sector though which only lets
> > you test the x86 system emulator.
> 
> Yes, if we can have some C program for all architecture then it sounds
> nicer, though IIUC this program is somehow special since it should be
> writting some "A"/"B" chars to console, and the test program is using
> those chars during the process.
> 
> Maybe it would be nice to rewrite a C program to replace current
> binary sector when someone wants to support postcopy migration test on
> a 3rd platform besides x86 and ppc.

The problem is you then rely on having Linux or booting a full guest;
and then you can't get the memory mapping reliably to allow the test to
examine the physical memory contents.

This little boot sector works quite nicely for x86, and Laurent's line
of Forth works nicely on Power.

Similar trivial tests for other architectures are probably easy.

Dave


> Thanks,
> 
> -- 
> Peter Xu
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Dr. David Alan Gilbert]

* Daniel P. Berrange (berrange@redhat.com) wrote:
> On Thu, Oct 19, 2017 at 11:18:33AM +0800, Peter Xu wrote:
> > On Wed, Oct 18, 2017 at 01:10:38PM +0100, Daniel P. Berrange wrote:
> > > On Wed, Oct 18, 2017 at 01:36:56PM +0200, Juan Quintela wrote:
> > > > Peter Xu <peterx@redhat.com> wrote:
> > > > > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > > > >
> > > > > [...]
> > > > >
> > > > >> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > > > >> + * outputing a 'B' every so often if it's still running.
> > > > >> + */
> > > > >> +unsigned char bootsect[] = {
> > > > >> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > > > >> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > > > >> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > > > >> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > > > >> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > > > >> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > > > >> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > > > >> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > > > >> +};
> > > > >
> > > > > Not sure whether it would be nicer to put this section as binary into
> > > > > QEMU's tree as shared, then other test code can use it too directly,
> > > > > and it would be easier if someone wants to boot a VM running this
> > > > > bootstrap code.  Thanks,
> > > > 
> > > > I am sharing this code now on a single place.  About getting this
> > > > outside ... I will wait for someone that understand how this works, I
> > > > have no clue neither how it works nor how to share/use/setup/....
> > > 
> > > FWIW, we already have a reusuable program that can be used to generate
> > > load stress for testing live migration see tests/migration/stress.c
> > > This gets compiled into a static binary, put into an initrd, and then
> > > run by booting the host OS kernel. So the caveat is that this approach
> > > only lets you test a QEMU emulator whose target arch matches the host.
> > > That isn't much different from this boot sector though which only lets
> > > you test the x86 system emulator.
> > 
> > Yes, if we can have some C program for all architecture then it sounds
> > nicer, though IIUC this program is somehow special since it should be
> > writting some "A"/"B" chars to console, and the test program is using
> > those chars during the process.
> 
> That would be easy enough to add to the current program I expect.
> 
> Oh the other difference with the program I wrote is that it is
> multi-threaded, so it spawns a writer thread for each vCPU that
> is given to the guest. This lets you generate even more extreme
> memory dirtying rates if desired :-)
> 
> > Maybe it would be nice to rewrite a C program to replace current
> > binary sector when someone wants to support postcopy migration test on
> > a 3rd platform besides x86 and ppc.
> 
> Booting a real Linux kernel and then running the C program as 'init'
> is easier to make portable since we don't need to know about special
> ways each arch deals with initial boot up.

No, it doesn't work.
This test reads a well defined area of physical memory; you can't
replace that test booting a full kernel and userland program.

Dave

> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Daniel P. Berrange]

On Thu, Oct 19, 2017 at 10:16:23AM +0100, Dr. David Alan Gilbert wrote:
> * Daniel P. Berrange (berrange@redhat.com) wrote:
> > On Thu, Oct 19, 2017 at 11:18:33AM +0800, Peter Xu wrote:
> > > On Wed, Oct 18, 2017 at 01:10:38PM +0100, Daniel P. Berrange wrote:
> > > > On Wed, Oct 18, 2017 at 01:36:56PM +0200, Juan Quintela wrote:
> > > > > Peter Xu <peterx@redhat.com> wrote:
> > > > > > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > > > > >
> > > > > > [...]
> > > > > >
> > > > > >> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > > > > >> + * outputing a 'B' every so often if it's still running.
> > > > > >> + */
> > > > > >> +unsigned char bootsect[] = {
> > > > > >> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > > > > >> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > > > > >> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > > > > >> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > > > > >> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > > > > >> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > > > > >> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > > > > >> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > > > > >> +};
> > > > > >
> > > > > > Not sure whether it would be nicer to put this section as binary into
> > > > > > QEMU's tree as shared, then other test code can use it too directly,
> > > > > > and it would be easier if someone wants to boot a VM running this
> > > > > > bootstrap code.  Thanks,
> > > > > 
> > > > > I am sharing this code now on a single place.  About getting this
> > > > > outside ... I will wait for someone that understand how this works, I
> > > > > have no clue neither how it works nor how to share/use/setup/....
> > > > 
> > > > FWIW, we already have a reusuable program that can be used to generate
> > > > load stress for testing live migration see tests/migration/stress.c
> > > > This gets compiled into a static binary, put into an initrd, and then
> > > > run by booting the host OS kernel. So the caveat is that this approach
> > > > only lets you test a QEMU emulator whose target arch matches the host.
> > > > That isn't much different from this boot sector though which only lets
> > > > you test the x86 system emulator.
> > > 
> > > Yes, if we can have some C program for all architecture then it sounds
> > > nicer, though IIUC this program is somehow special since it should be
> > > writting some "A"/"B" chars to console, and the test program is using
> > > those chars during the process.
> > 
> > That would be easy enough to add to the current program I expect.
> > 
> > Oh the other difference with the program I wrote is that it is
> > multi-threaded, so it spawns a writer thread for each vCPU that
> > is given to the guest. This lets you generate even more extreme
> > memory dirtying rates if desired :-)
> > 
> > > Maybe it would be nice to rewrite a C program to replace current
> > > binary sector when someone wants to support postcopy migration test on
> > > a 3rd platform besides x86 and ppc.
> > 
> > Booting a real Linux kernel and then running the C program as 'init'
> > is easier to make portable since we don't need to know about special
> > ways each arch deals with initial boot up.
> 
> No, it doesn't work.
> This test reads a well defined area of physical memory; you can't
> replace that test booting a full kernel and userland program.

Ah ok, I didn't realize this test was actually looking into guest RAM, I
thought it just treated the guest as opaque merely needing RAM dirtying
load.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH 1/6] tests: Add basic migration precopy test

[Dr. David Alan Gilbert]

* Daniel P. Berrange (berrange@redhat.com) wrote:
> On Thu, Oct 19, 2017 at 10:16:23AM +0100, Dr. David Alan Gilbert wrote:
> > * Daniel P. Berrange (berrange@redhat.com) wrote:
> > > On Thu, Oct 19, 2017 at 11:18:33AM +0800, Peter Xu wrote:
> > > > On Wed, Oct 18, 2017 at 01:10:38PM +0100, Daniel P. Berrange wrote:
> > > > > On Wed, Oct 18, 2017 at 01:36:56PM +0200, Juan Quintela wrote:
> > > > > > Peter Xu <peterx@redhat.com> wrote:
> > > > > > > On Wed, Oct 04, 2017 at 12:39:28PM +0200, Juan Quintela wrote:
> > > > > > >
> > > > > > > [...]
> > > > > > >
> > > > > > >> +/* A simple PC boot sector that modifies memory (1-100MB) quickly
> > > > > > >> + * outputing a 'B' every so often if it's still running.
> > > > > > >> + */
> > > > > > >> +unsigned char bootsect[] = {
> > > > > > >> +  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
> > > > > > >> +  0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
> > > > > > >> +  0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
> > > > > > >> +  0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
> > > > > > >> +  0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
> > > > > > >> +  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
> > > > > > >> +  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
> > > > > > >> +  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
> > > > > > >> +  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x55, 0xaa
> > > > > > >> +};
> > > > > > >
> > > > > > > Not sure whether it would be nicer to put this section as binary into
> > > > > > > QEMU's tree as shared, then other test code can use it too directly,
> > > > > > > and it would be easier if someone wants to boot a VM running this
> > > > > > > bootstrap code.  Thanks,
> > > > > > 
> > > > > > I am sharing this code now on a single place.  About getting this
> > > > > > outside ... I will wait for someone that understand how this works, I
> > > > > > have no clue neither how it works nor how to share/use/setup/....
> > > > > 
> > > > > FWIW, we already have a reusuable program that can be used to generate
> > > > > load stress for testing live migration see tests/migration/stress.c
> > > > > This gets compiled into a static binary, put into an initrd, and then
> > > > > run by booting the host OS kernel. So the caveat is that this approach
> > > > > only lets you test a QEMU emulator whose target arch matches the host.
> > > > > That isn't much different from this boot sector though which only lets
> > > > > you test the x86 system emulator.
> > > > 
> > > > Yes, if we can have some C program for all architecture then it sounds
> > > > nicer, though IIUC this program is somehow special since it should be
> > > > writting some "A"/"B" chars to console, and the test program is using
> > > > those chars during the process.
> > > 
> > > That would be easy enough to add to the current program I expect.
> > > 
> > > Oh the other difference with the program I wrote is that it is
> > > multi-threaded, so it spawns a writer thread for each vCPU that
> > > is given to the guest. This lets you generate even more extreme
> > > memory dirtying rates if desired :-)
> > > 
> > > > Maybe it would be nice to rewrite a C program to replace current
> > > > binary sector when someone wants to support postcopy migration test on
> > > > a 3rd platform besides x86 and ppc.
> > > 
> > > Booting a real Linux kernel and then running the C program as 'init'
> > > is easier to make portable since we don't need to know about special
> > > ways each arch deals with initial boot up.
> > 
> > No, it doesn't work.
> > This test reads a well defined area of physical memory; you can't
> > replace that test booting a full kernel and userland program.
> 
> Ah ok, I didn't realize this test was actually looking into guest RAM, I
> thought it just treated the guest as opaque merely needing RAM dirtying
> load.

No, it's verifying that the contents after migration are valid.
For which that's not a fixed value but matches a set of constraints;
i.e. that since the increment happens in order, that there's a single
point in the sequence where the value is lower than than the previous
page, since the increment hasn't got to that page yet.

Dave

> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK