qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PULL 0/8] Migration patches
@ 2019-06-12 10:48 Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 1/8] migration: fix multifd_recv event typo Juan Quintela
                   ` (8 more replies)
  0 siblings, 9 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Paolo Bonzini

The following changes since commit 219dca61ebf41625831d4f96a720852baf44b762:

  Merge remote-tracking branch 'remotes/ehabkost/tags/x86-next-pull-request' into staging (2019-06-11 16:02:07 +0100)

are available in the Git repository at:

  https://github.com/juanquintela/qemu.git tags/migration-pull-request

for you to fetch changes up to 87fa344f0684d7c62b0859632de4b6c28e32f2c7:

  migratioin/ram.c: reset complete_round when we gets a queued page (2019-06-12 12:38:12 +0200)

----------------------------------------------------------------
Migration pull request

* misc fixes from wei
* Fixes for the multifd compression tree (juan)

Please apply

----------------------------------------------------------------

Juan Quintela (3):
  migration: fix multifd_recv event typo
  migration-test: rename parameter to parameter_int
  migration-test: Add migration multifd test

Wei Yang (5):
  migration/multifd: call multifd_send_sync_main when sending
    RAM_SAVE_FLAG_EOS
  migration/xbzrle: update cache and current_data in one place
  cutils: remove one unnecessary pointer operation
  migration/multifd: sync packet_num after all thread are done
  migratioin/ram.c: reset complete_round when we gets a queued page

 migration/ram.c        |  41 ++++++++++------
 migration/trace-events |   2 +-
 tests/migration-test.c | 103 ++++++++++++++++++++++++++++++-----------
 util/cutils.c          |   8 ++--
 4 files changed, 107 insertions(+), 47 deletions(-)

-- 
2.21.0



^ permalink raw reply	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 1/8] migration: fix multifd_recv event typo
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 2/8] migration-test: rename parameter to parameter_int Juan Quintela
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

It uses num in multifd_send().  Make it coherent.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Wei Yang <richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/trace-events | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/migration/trace-events b/migration/trace-events
index de2e136e57..cd50a1e659 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -80,7 +80,7 @@ get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned
 migration_bitmap_sync_start(void) ""
 migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
 migration_throttle(void) ""
-multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet number %" PRIu64 " pages %d flags 0x%x next packet size %d"
+multifd_recv(uint8_t id, uint64_t packet_num, uint32_t used, uint32_t flags, uint32_t next_packet_size) "channel %d packet_num %" PRIu64 " pages %d flags 0x%x next packet size %d"
 multifd_recv_sync_main(long packet_num) "packet num %ld"
 multifd_recv_sync_main_signal(uint8_t id) "channel %d"
 multifd_recv_sync_main_wait(uint8_t id) "channel %d"
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 2/8] migration-test: rename parameter to parameter_int
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 1/8] migration: fix multifd_recv event typo Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 3/8] migration-test: Add migration multifd test Juan Quintela
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

We would need _str ones on the next patch.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Wei Yang <richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/migration-test.c | 55 +++++++++++++++++++++---------------------
 1 file changed, 28 insertions(+), 27 deletions(-)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index e0407576cb..ebabca8820 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -407,7 +407,8 @@ static char *migrate_get_socket_address(QTestState *who, const char *parameter)
     return result;
 }
 
-static long long migrate_get_parameter(QTestState *who, const char *parameter)
+static long long migrate_get_parameter_int(QTestState *who,
+                                           const char *parameter)
 {
     QDict *rsp;
     long long result;
@@ -418,17 +419,17 @@ static long long migrate_get_parameter(QTestState *who, const char *parameter)
     return result;
 }
 
-static void migrate_check_parameter(QTestState *who, const char *parameter,
-                                    long long value)
+static void migrate_check_parameter_int(QTestState *who, const char *parameter,
+                                        long long value)
 {
     long long result;
 
-    result = migrate_get_parameter(who, parameter);
+    result = migrate_get_parameter_int(who, parameter);
     g_assert_cmpint(result, ==, value);
 }
 
-static void migrate_set_parameter(QTestState *who, const char *parameter,
-                                  long long value)
+static void migrate_set_parameter_int(QTestState *who, const char *parameter,
+                                      long long value)
 {
     QDict *rsp;
 
@@ -438,7 +439,7 @@ static void migrate_set_parameter(QTestState *who, const char *parameter,
                     parameter, value);
     g_assert(qdict_haskey(rsp, "return"));
     qobject_unref(rsp);
-    migrate_check_parameter(who, parameter, value);
+    migrate_check_parameter_int(who, parameter, value);
 }
 
 static void migrate_pause(QTestState *who)
@@ -688,7 +689,7 @@ static void deprecated_set_downtime(QTestState *who, const double value)
                     " 'arguments': { 'value': %f } }", value);
     g_assert(qdict_haskey(rsp, "return"));
     qobject_unref(rsp);
-    migrate_check_parameter(who, "downtime-limit", value * 1000);
+    migrate_check_parameter_int(who, "downtime-limit", value * 1000);
 }
 
 static void deprecated_set_speed(QTestState *who, long long value)
@@ -699,7 +700,7 @@ static void deprecated_set_speed(QTestState *who, long long value)
                           "'arguments': { 'value': %lld } }", value);
     g_assert(qdict_haskey(rsp, "return"));
     qobject_unref(rsp);
-    migrate_check_parameter(who, "max-bandwidth", value);
+    migrate_check_parameter_int(who, "max-bandwidth", value);
 }
 
 static void deprecated_set_cache_size(QTestState *who, long long value)
@@ -710,7 +711,7 @@ static void deprecated_set_cache_size(QTestState *who, long long value)
                          "'arguments': { 'value': %lld } }", value);
     g_assert(qdict_haskey(rsp, "return"));
     qobject_unref(rsp);
-    migrate_check_parameter(who, "xbzrle-cache-size", value);
+    migrate_check_parameter_int(who, "xbzrle-cache-size", value);
 }
 
 static void test_deprecated(void)
@@ -745,8 +746,8 @@ static int migrate_postcopy_prepare(QTestState **from_ptr,
      * quickly, but that it doesn't complete precopy even on a slow
      * machine, so also set the downtime.
      */
-    migrate_set_parameter(from, "max-bandwidth", 100000000);
-    migrate_set_parameter(from, "downtime-limit", 1);
+    migrate_set_parameter_int(from, "max-bandwidth", 100000000);
+    migrate_set_parameter_int(from, "downtime-limit", 1);
 
     /* Wait for the first serial output from the source */
     wait_for_serial("src_serial");
@@ -797,7 +798,7 @@ static void test_postcopy_recovery(void)
     }
 
     /* Turn postcopy speed down, 4K/s is slow enough on any machines */
-    migrate_set_parameter(from, "max-postcopy-bandwidth", 4096);
+    migrate_set_parameter_int(from, "max-postcopy-bandwidth", 4096);
 
     /* Now we start the postcopy */
     migrate_postcopy_start(from, to);
@@ -838,7 +839,7 @@ static void test_postcopy_recovery(void)
     g_free(uri);
 
     /* Restore the postcopy bandwidth to unlimited */
-    migrate_set_parameter(from, "max-postcopy-bandwidth", 0);
+    migrate_set_parameter_int(from, "max-postcopy-bandwidth", 0);
 
     migrate_postcopy_complete(from, to);
 }
@@ -884,9 +885,9 @@ static void test_precopy_unix(void)
      * machine, so also set the downtime.
      */
     /* 1 ms should make it not converge*/
-    migrate_set_parameter(from, "downtime-limit", 1);
+    migrate_set_parameter_int(from, "downtime-limit", 1);
     /* 1GB/s */
-    migrate_set_parameter(from, "max-bandwidth", 1000000000);
+    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
 
     /* Wait for the first serial output from the source */
     wait_for_serial("src_serial");
@@ -896,7 +897,7 @@ static void test_precopy_unix(void)
     wait_for_migration_pass(from);
 
     /* 300 ms should converge */
-    migrate_set_parameter(from, "downtime-limit", 300);
+    migrate_set_parameter_int(from, "downtime-limit", 300);
 
     if (!got_stop) {
         qtest_qmp_eventwait(from, "STOP");
@@ -963,11 +964,11 @@ static void test_xbzrle(const char *uri)
      * machine, so also set the downtime.
      */
     /* 1 ms should make it not converge*/
-    migrate_set_parameter(from, "downtime-limit", 1);
+    migrate_set_parameter_int(from, "downtime-limit", 1);
     /* 1GB/s */
-    migrate_set_parameter(from, "max-bandwidth", 1000000000);
+    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
 
-    migrate_set_parameter(from, "xbzrle-cache-size", 33554432);
+    migrate_set_parameter_int(from, "xbzrle-cache-size", 33554432);
 
     migrate_set_capability(from, "xbzrle", "true");
     migrate_set_capability(to, "xbzrle", "true");
@@ -979,7 +980,7 @@ static void test_xbzrle(const char *uri)
     wait_for_migration_pass(from);
 
     /* 300ms should converge */
-    migrate_set_parameter(from, "downtime-limit", 300);
+    migrate_set_parameter_int(from, "downtime-limit", 300);
 
     if (!got_stop) {
         qtest_qmp_eventwait(from, "STOP");
@@ -1015,9 +1016,9 @@ static void test_precopy_tcp(void)
      * machine, so also set the downtime.
      */
     /* 1 ms should make it not converge*/
-    migrate_set_parameter(from, "downtime-limit", 1);
+    migrate_set_parameter_int(from, "downtime-limit", 1);
     /* 1GB/s */
-    migrate_set_parameter(from, "max-bandwidth", 1000000000);
+    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
 
     /* Wait for the first serial output from the source */
     wait_for_serial("src_serial");
@@ -1029,7 +1030,7 @@ static void test_precopy_tcp(void)
     wait_for_migration_pass(from);
 
     /* 300ms should converge */
-    migrate_set_parameter(from, "downtime-limit", 300);
+    migrate_set_parameter_int(from, "downtime-limit", 300);
 
     if (!got_stop) {
         qtest_qmp_eventwait(from, "STOP");
@@ -1061,9 +1062,9 @@ static void test_migrate_fd_proto(void)
      * machine, so also set the downtime.
      */
     /* 1 ms should make it not converge */
-    migrate_set_parameter(from, "downtime-limit", 1);
+    migrate_set_parameter_int(from, "downtime-limit", 1);
     /* 1GB/s */
-    migrate_set_parameter(from, "max-bandwidth", 1000000000);
+    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
 
     /* Wait for the first serial output from the source */
     wait_for_serial("src_serial");
@@ -1097,7 +1098,7 @@ static void test_migrate_fd_proto(void)
     wait_for_migration_pass(from);
 
     /* 300ms should converge */
-    migrate_set_parameter(from, "downtime-limit", 300);
+    migrate_set_parameter_int(from, "downtime-limit", 300);
 
     if (!got_stop) {
         qtest_qmp_eventwait(from, "STOP");
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 3/8] migration-test: Add migration multifd test
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 1/8] migration: fix multifd_recv event typo Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 2/8] migration-test: rename parameter to parameter_int Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 4/8] migration/multifd: call multifd_send_sync_main when sending RAM_SAVE_FLAG_EOS Juan Quintela
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

We set multifd-channels.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Wei Yang <richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 tests/migration-test.c | 48 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index ebabca8820..36d4910192 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -1128,6 +1128,53 @@ static void test_migrate_fd_proto(void)
     test_migrate_end(from, to, true);
 }
 
+static void test_multifd_tcp(void)
+{
+    char *uri;
+    QTestState *from, *to;
+
+    if (test_migrate_start(&from, &to, "tcp:127.0.0.1:0", false, false)) {
+        return;
+    }
+
+    /*
+     * We want to pick a speed slow enough that the test completes
+     * quickly, but that it doesn't complete precopy even on a slow
+     * machine, so also set the downtime.
+     */
+    /* 1 ms should make it not converge*/
+    migrate_set_parameter_int(from, "downtime-limit", 1);
+    /* 1GB/s */
+    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
+
+    migrate_set_parameter_int(from, "multifd-channels", 2);
+    migrate_set_parameter_int(to, "multifd-channels", 2);
+
+    migrate_set_capability(from, "multifd", "true");
+    migrate_set_capability(to, "multifd", "true");
+    /* Wait for the first serial output from the source */
+    wait_for_serial("src_serial");
+
+    uri = migrate_get_socket_address(to, "socket-address");
+
+    migrate(from, uri, "{}");
+
+    wait_for_migration_pass(from);
+
+    /* 300ms it should converge */
+    migrate_set_parameter_int(from, "downtime-limit", 600);
+
+    if (!got_stop) {
+        qtest_qmp_eventwait(from, "STOP");
+    }
+    qtest_qmp_eventwait(to, "RESUME");
+
+    wait_for_serial("dest_serial");
+    wait_for_migration_complete(from);
+    test_migrate_end(from, to, true);
+    free(uri);
+}
+
 int main(int argc, char **argv)
 {
     char template[] = "/tmp/migration-test-XXXXXX";
@@ -1183,6 +1230,7 @@ int main(int argc, char **argv)
     /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */
     qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
     qtest_add_func("/migration/fd_proto", test_migrate_fd_proto);
+    qtest_add_func("/migration/multifd/tcp", test_multifd_tcp);
 
     ret = g_test_run();
 
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 4/8] migration/multifd: call multifd_send_sync_main when sending RAM_SAVE_FLAG_EOS
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
                   ` (2 preceding siblings ...)
  2019-06-12 10:48 ` [Qemu-devel] [PULL 3/8] migration-test: Add migration multifd test Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 5/8] migration/xbzrle: update cache and current_data in one place Juan Quintela
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

From: Wei Yang <richardw.yang@linux.intel.com>

On receiving RAM_SAVE_FLAG_EOS, multifd_recv_sync_main() is called to
synchronize receive threads. Current synchronization mechanism is to wait
for each channel's sem_sync semaphore. This semaphore is triggered by a
packet with MULTIFD_FLAG_SYNC flag. While in current implementation, we
don't do multifd_send_sync_main() to send such packet when
blk_mig_bulk_active() is true.

This will leads to the receive threads won't notify
multifd_recv_sync_main() by sem_sync. And multifd_recv_sync_main() will
always wait there.

[Note]: normal migration test works, while didn't test the
blk_mig_bulk_active() case. Since not sure how to produce this
situation.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20190612014337.11255-1-richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/migration/ram.c b/migration/ram.c
index 908517fc2b..74c9306c78 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -3466,8 +3466,8 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
      */
     ram_control_after_iterate(f, RAM_CONTROL_ROUND);
 
-    multifd_send_sync_main();
 out:
+    multifd_send_sync_main();
     qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
     qemu_fflush(f);
     ram_counters.transferred += 8;
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 5/8] migration/xbzrle: update cache and current_data in one place
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
                   ` (3 preceding siblings ...)
  2019-06-12 10:48 ` [Qemu-devel] [PULL 4/8] migration/multifd: call multifd_send_sync_main when sending RAM_SAVE_FLAG_EOS Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 6/8] cutils: remove one unnecessary pointer operation Juan Quintela
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

From: Wei Yang <richardw.yang@linux.intel.com>

When we are not in the last_stage, we need to update the cache if page
is not the same.

Currently this procedure is scattered in two places and mixed with
encoding status check.

This patch extract this general step out to make the code a little bit
easy to read.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Message-Id: <20190610004159.20966-1-richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 74c9306c78..d3d72b6f4f 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1585,25 +1585,30 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **current_data,
     encoded_len = xbzrle_encode_buffer(prev_cached_page, XBZRLE.current_buf,
                                        TARGET_PAGE_SIZE, XBZRLE.encoded_buf,
                                        TARGET_PAGE_SIZE);
+
+    /*
+     * Update the cache contents, so that it corresponds to the data
+     * sent, in all cases except where we skip the page.
+     */
+    if (!last_stage && encoded_len != 0) {
+        memcpy(prev_cached_page, XBZRLE.current_buf, TARGET_PAGE_SIZE);
+        /*
+         * In the case where we couldn't compress, ensure that the caller
+         * sends the data from the cache, since the guest might have
+         * changed the RAM since we copied it.
+         */
+        *current_data = prev_cached_page;
+    }
+
     if (encoded_len == 0) {
         trace_save_xbzrle_page_skipping();
         return 0;
     } else if (encoded_len == -1) {
         trace_save_xbzrle_page_overflow();
         xbzrle_counters.overflow++;
-        /* update data in the cache */
-        if (!last_stage) {
-            memcpy(prev_cached_page, *current_data, TARGET_PAGE_SIZE);
-            *current_data = prev_cached_page;
-        }
         return -1;
     }
 
-    /* we need to update the data in the cache, in order to get the same data */
-    if (!last_stage) {
-        memcpy(prev_cached_page, XBZRLE.current_buf, TARGET_PAGE_SIZE);
-    }
-
     /* Send XBZRLE based compressed page */
     bytes_xbzrle = save_page_header(rs, rs->f, block,
                                     offset | RAM_SAVE_FLAG_XBZRLE);
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 6/8] cutils: remove one unnecessary pointer operation
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
                   ` (4 preceding siblings ...)
  2019-06-12 10:48 ` [Qemu-devel] [PULL 5/8] migration/xbzrle: update cache and current_data in one place Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 7/8] migration/multifd: sync packet_num after all thread are done Juan Quintela
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

From: Wei Yang <richardw.yang@linux.intel.com>

Since we will not operate on the next address pointed by out, it is not
necessary to do addition on it.

After removing the operation, the function size reduced 16/18 bytes.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Message-Id: <20190610030852.16039-2-richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 util/cutils.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/util/cutils.c b/util/cutils.c
index 9aacc422ca..1933a68da5 100644
--- a/util/cutils.c
+++ b/util/cutils.c
@@ -754,11 +754,11 @@ int uleb128_encode_small(uint8_t *out, uint32_t n)
 {
     g_assert(n <= 0x3fff);
     if (n < 0x80) {
-        *out++ = n;
+        *out = n;
         return 1;
     } else {
         *out++ = (n & 0x7f) | 0x80;
-        *out++ = n >> 7;
+        *out = n >> 7;
         return 2;
     }
 }
@@ -766,7 +766,7 @@ int uleb128_encode_small(uint8_t *out, uint32_t n)
 int uleb128_decode_small(const uint8_t *in, uint32_t *n)
 {
     if (!(*in & 0x80)) {
-        *n = *in++;
+        *n = *in;
         return 1;
     } else {
         *n = *in++ & 0x7f;
@@ -774,7 +774,7 @@ int uleb128_decode_small(const uint8_t *in, uint32_t *n)
         if (*in & 0x80) {
             return -1;
         }
-        *n |= *in++ << 7;
+        *n |= *in << 7;
         return 2;
     }
 }
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 7/8] migration/multifd: sync packet_num after all thread are done
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
                   ` (5 preceding siblings ...)
  2019-06-12 10:48 ` [Qemu-devel] [PULL 6/8] cutils: remove one unnecessary pointer operation Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-12 10:48 ` [Qemu-devel] [PULL 8/8] migratioin/ram.c: reset complete_round when we gets a queued page Juan Quintela
  2019-06-13 10:23 ` [Qemu-devel] [PULL 0/8] Migration patches Peter Maydell
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Peter Xu, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

From: Wei Yang <richardw.yang@linux.intel.com>

Notification from recv thread is not ordered, which means we may be
notified by one MultiFDRecvParams but adjust packet_num for another.

Move the adjustment after we are sure each recv thread are sync-ed.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20190604023540.26532-1-richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index d3d72b6f4f..96c84f770a 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1291,15 +1291,15 @@ static void multifd_recv_sync_main(void)
 
         trace_multifd_recv_sync_main_wait(p->id);
         qemu_sem_wait(&multifd_recv_state->sem_sync);
+    }
+    for (i = 0; i < migrate_multifd_channels(); i++) {
+        MultiFDRecvParams *p = &multifd_recv_state->params[i];
+
         qemu_mutex_lock(&p->mutex);
         if (multifd_recv_state->packet_num < p->packet_num) {
             multifd_recv_state->packet_num = p->packet_num;
         }
         qemu_mutex_unlock(&p->mutex);
-    }
-    for (i = 0; i < migrate_multifd_channels(); i++) {
-        MultiFDRecvParams *p = &multifd_recv_state->params[i];
-
         trace_multifd_recv_sync_main_signal(p->id);
         qemu_sem_post(&p->sem_sync);
     }
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [Qemu-devel] [PULL 8/8] migratioin/ram.c: reset complete_round when we gets a queued page
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
                   ` (6 preceding siblings ...)
  2019-06-12 10:48 ` [Qemu-devel] [PULL 7/8] migration/multifd: sync packet_num after all thread are done Juan Quintela
@ 2019-06-12 10:48 ` Juan Quintela
  2019-06-13 10:23 ` [Qemu-devel] [PULL 0/8] Migration patches Peter Maydell
  8 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-12 10:48 UTC (permalink / raw)
  To: qemu-devel
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Juan Quintela,
	Markus Armbruster, Dr. David Alan Gilbert, Wei Yang,
	Paolo Bonzini

From: Wei Yang <richardw.yang@linux.intel.com>

In case we gets a queued page, the order of block is interrupted. We may
not rely on the complete_round flag to say we have already searched the
whole blocks on the list.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20190605010828.6969-1-richardw.yang@linux.intel.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
 migration/ram.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/migration/ram.c b/migration/ram.c
index 96c84f770a..89eec7ee9d 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2286,6 +2286,12 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss)
          */
         pss->block = block;
         pss->page = offset >> TARGET_PAGE_BITS;
+
+        /*
+         * This unqueued page would break the "one round" check, even is
+         * really rare.
+         */
+        pss->complete_round = false;
     }
 
     return !!block;
-- 
2.21.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PULL 0/8] Migration patches
  2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
                   ` (7 preceding siblings ...)
  2019-06-12 10:48 ` [Qemu-devel] [PULL 8/8] migratioin/ram.c: reset complete_round when we gets a queued page Juan Quintela
@ 2019-06-13 10:23 ` Peter Maydell
  2019-06-13 11:15   ` Juan Quintela
  8 siblings, 1 reply; 11+ messages in thread
From: Peter Maydell @ 2019-06-13 10:23 UTC (permalink / raw)
  To: Juan Quintela
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Markus Armbruster,
	QEMU Developers, Paolo Bonzini, Dr. David Alan Gilbert

On Wed, 12 Jun 2019 at 11:54, Juan Quintela <quintela@redhat.com> wrote:
>
> The following changes since commit 219dca61ebf41625831d4f96a720852baf44b762:
>
>   Merge remote-tracking branch 'remotes/ehabkost/tags/x86-next-pull-request' into staging (2019-06-11 16:02:07 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/juanquintela/qemu.git tags/migration-pull-request
>
> for you to fetch changes up to 87fa344f0684d7c62b0859632de4b6c28e32f2c7:
>
>   migratioin/ram.c: reset complete_round when we gets a queued page (2019-06-12 12:38:12 +0200)
>
> ----------------------------------------------------------------
> Migration pull request
>
> * misc fixes from wei
> * Fixes for the multifd compression tree (juan)
>
> Please apply

Hi. This fails 'make check' on aarch32 (possibly on all
32-bit guests, aarch32 is my only 32-bit host):

MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_BINARY=aarch64-softmmu/qemu-system-aarch64
QTEST_QEMU_IMG=qemu-img tests/migration-test -m=quick -k --tap <
/dev/null | ./scripts/tap-driver.pl --test-name="migration-test"
PASS 1 migration-test /aarch64/migration/deprecated
PASS 2 migration-test /aarch64/migration/bad_dest
PASS 3 migration-test /aarch64/migration/fd_proto
PASS 4 migration-test /aarch64/migration/postcopy/unix
PASS 5 migration-test /aarch64/migration/postcopy/recovery
PASS 6 migration-test /aarch64/migration/precopy/unix
PASS 7 migration-test /aarch64/migration/precopy/tcp
PASS 8 migration-test /aarch64/migration/xbzrle/unix
malloc_consolidate(): invalid chunk size
Broken pipe
qemu-system-aarch64: check_section_footer: Read section footer failed: -5
qemu-system-aarch64: load of migration failed: Invalid argument
/home/peter.maydell/qemu/tests/libqtest.c:137: kill_qemu() tried to
terminate QEMU process but encountered exit status 1
Aborted
ERROR - too few tests run (expected 9, got 8)
/home/peter.maydell/qemu/tests/Makefile.include:888: recipe for target
'check-qtest-aarch64' failed

thanks
-- PMM


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [Qemu-devel] [PULL 0/8] Migration patches
  2019-06-13 10:23 ` [Qemu-devel] [PULL 0/8] Migration patches Peter Maydell
@ 2019-06-13 11:15   ` Juan Quintela
  0 siblings, 0 replies; 11+ messages in thread
From: Juan Quintela @ 2019-06-13 11:15 UTC (permalink / raw)
  To: Peter Maydell
  Cc: Laurent Vivier, Thomas Huth, zhanghailiang, Markus Armbruster,
	QEMU Developers, Paolo Bonzini, Dr. David Alan Gilbert

Peter Maydell <peter.maydell@linaro.org> wrote:
> On Wed, 12 Jun 2019 at 11:54, Juan Quintela <quintela@redhat.com> wrote:
>>
>> The following changes since commit 219dca61ebf41625831d4f96a720852baf44b762:
>>
>>   Merge remote-tracking branch
>> 'remotes/ehabkost/tags/x86-next-pull-request' into staging
>> (2019-06-11 16:02:07 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/juanquintela/qemu.git tags/migration-pull-request
>>
>> for you to fetch changes up to 87fa344f0684d7c62b0859632de4b6c28e32f2c7:
>>
>>   migratioin/ram.c: reset complete_round when we gets a queued page
>> (2019-06-12 12:38:12 +0200)
>>
>> ----------------------------------------------------------------
>> Migration pull request
>>
>> * misc fixes from wei
>> * Fixes for the multifd compression tree (juan)
>>
>> Please apply
>
> Hi. This fails 'make check' on aarch32 (possibly on all
> 32-bit guests, aarch32 is my only 32-bit host):
>
> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_BINARY=aarch64-softmmu/qemu-system-aarch64
> QTEST_QEMU_IMG=qemu-img tests/migration-test -m=quick -k --tap <
> /dev/null | ./scripts/tap-driver.pl --test-name="migration-test"
> PASS 1 migration-test /aarch64/migration/deprecated
> PASS 2 migration-test /aarch64/migration/bad_dest
> PASS 3 migration-test /aarch64/migration/fd_proto
> PASS 4 migration-test /aarch64/migration/postcopy/unix
> PASS 5 migration-test /aarch64/migration/postcopy/recovery
> PASS 6 migration-test /aarch64/migration/precopy/unix
> PASS 7 migration-test /aarch64/migration/precopy/tcp
> PASS 8 migration-test /aarch64/migration/xbzrle/unix
> malloc_consolidate(): invalid chunk size
> Broken pipe
> qemu-system-aarch64: check_section_footer: Read section footer failed: -5
> qemu-system-aarch64: load of migration failed: Invalid argument
> /home/peter.maydell/qemu/tests/libqtest.c:137: kill_qemu() tried to
> terminate QEMU process but encountered exit status 1
> Aborted
> ERROR - too few tests run (expected 9, got 8)
> /home/peter.maydell/qemu/tests/Makefile.include:888: recipe for target
> 'check-qtest-aarch64' failed

Thanks, looking into it.

Later, Juan.


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2019-06-13 11:18 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-12 10:48 [Qemu-devel] [PULL 0/8] Migration patches Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 1/8] migration: fix multifd_recv event typo Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 2/8] migration-test: rename parameter to parameter_int Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 3/8] migration-test: Add migration multifd test Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 4/8] migration/multifd: call multifd_send_sync_main when sending RAM_SAVE_FLAG_EOS Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 5/8] migration/xbzrle: update cache and current_data in one place Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 6/8] cutils: remove one unnecessary pointer operation Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 7/8] migration/multifd: sync packet_num after all thread are done Juan Quintela
2019-06-12 10:48 ` [Qemu-devel] [PULL 8/8] migratioin/ram.c: reset complete_round when we gets a queued page Juan Quintela
2019-06-13 10:23 ` [Qemu-devel] [PULL 0/8] Migration patches Peter Maydell
2019-06-13 11:15   ` Juan Quintela

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).