* [Qemu-devel] [PULL 1/6] migration: fix minor finalize leak
2018-03-09 17:54 [Qemu-devel] [PULL 0/6] migration queue Dr. David Alan Gilbert (git)
@ 2018-03-09 17:54 ` Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 2/6] migration: do not transfer ram during bulk storage migration Dr. David Alan Gilbert (git)
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2018-03-09 17:54 UTC (permalink / raw)
To: qemu-devel; +Cc: marcandre.lureau, pl, peterx, quintela
From: Marc-André Lureau <marcandre.lureau@redhat.com>
Spotted thanks to ASAN:
QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 tests/migration-test -p /x86_64/migration/bad_dest
==30302==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 48 byte(s) in 1 object(s) allocated from:
#0 0x7f60efba1a38 in __interceptor_calloc (/lib64/libasan.so.4+0xdea38)
#1 0x7f60eef3cf75 in g_malloc0 ../glib/gmem.c:124
#2 0x55ca9094702c in error_copy /home/elmarco/src/qemu/util/error.c:203
#3 0x55ca9037a30f in migrate_set_error /home/elmarco/src/qemu/migration/migration.c:1139
#4 0x55ca9037a462 in migrate_fd_error /home/elmarco/src/qemu/migration/migration.c:1150
#5 0x55ca9038162b in migrate_fd_connect /home/elmarco/src/qemu/migration/migration.c:2411
#6 0x55ca90386e41 in migration_channel_connect /home/elmarco/src/qemu/migration/channel.c:81
#7 0x55ca9038335e in socket_outgoing_migration /home/elmarco/src/qemu/migration/socket.c:85
#8 0x55ca9083dd3a in qio_task_complete /home/elmarco/src/qemu/io/task.c:142
#9 0x55ca9083d6cc in gio_task_thread_result /home/elmarco/src/qemu/io/task.c:88
#10 0x7f60eef37317 in g_idle_dispatch ../glib/gmain.c:5552
#11 0x7f60eef3490b in g_main_dispatch ../glib/gmain.c:3182
#12 0x7f60eef357ac in g_main_context_dispatch ../glib/gmain.c:3847
#13 0x55ca90927231 in glib_pollfds_poll /home/elmarco/src/qemu/util/main-loop.c:214
#14 0x55ca90927420 in os_host_main_loop_wait /home/elmarco/src/qemu/util/main-loop.c:261
#15 0x55ca909275fa in main_loop_wait /home/elmarco/src/qemu/util/main-loop.c:515
#16 0x55ca8fc1c2a4 in main_loop /home/elmarco/src/qemu/vl.c:1942
#17 0x55ca8fc2eb3a in main /home/elmarco/src/qemu/vl.c:4724
#18 0x7f60e4082009 in __libc_start_main (/lib64/libc.so.6+0x21009)
Indirect leak of 45 byte(s) in 1 object(s) allocated from:
#0 0x7f60efba1850 in malloc (/lib64/libasan.so.4+0xde850)
#1 0x7f60eef3cf0c in g_malloc ../glib/gmem.c:94
#2 0x7f60eef3d1cf in g_malloc_n ../glib/gmem.c:331
#3 0x7f60eef596eb in g_strdup ../glib/gstrfuncs.c:363
#4 0x55ca90947085 in error_copy /home/elmarco/src/qemu/util/error.c:204
#5 0x55ca9037a30f in migrate_set_error /home/elmarco/src/qemu/migration/migration.c:1139
#6 0x55ca9037a462 in migrate_fd_error /home/elmarco/src/qemu/migration/migration.c:1150
#7 0x55ca9038162b in migrate_fd_connect /home/elmarco/src/qemu/migration/migration.c:2411
#8 0x55ca90386e41 in migration_channel_connect /home/elmarco/src/qemu/migration/channel.c:81
#9 0x55ca9038335e in socket_outgoing_migration /home/elmarco/src/qemu/migration/socket.c:85
#10 0x55ca9083dd3a in qio_task_complete /home/elmarco/src/qemu/io/task.c:142
#11 0x55ca9083d6cc in gio_task_thread_result /home/elmarco/src/qemu/io/task.c:88
#12 0x7f60eef37317 in g_idle_dispatch ../glib/gmain.c:5552
#13 0x7f60eef3490b in g_main_dispatch ../glib/gmain.c:3182
#14 0x7f60eef357ac in g_main_context_dispatch ../glib/gmain.c:3847
#15 0x55ca90927231 in glib_pollfds_poll /home/elmarco/src/qemu/util/main-loop.c:214
#16 0x55ca90927420 in os_host_main_loop_wait /home/elmarco/src/qemu/util/main-loop.c:261
#17 0x55ca909275fa in main_loop_wait /home/elmarco/src/qemu/util/main-loop.c:515
#18 0x55ca8fc1c2a4 in main_loop /home/elmarco/src/qemu/vl.c:1942
#19 0x55ca8fc2eb3a in main /home/elmarco/src/qemu/vl.c:4724
#20 0x7f60e4082009 in __libc_start_main (/lib64/libc.so.6+0x21009)
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180306170959.3921-1-marcandre.lureau@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/migration.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/migration/migration.c b/migration/migration.c
index e345d0cc7e..62c243d2d4 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2541,6 +2541,7 @@ static void migration_instance_finalize(Object *obj)
g_free(params->tls_hostname);
g_free(params->tls_creds);
qemu_sem_destroy(&ms->pause_sem);
+ error_free(ms->error);
}
static void migration_instance_init(Object *obj)
--
2.14.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PULL 2/6] migration: do not transfer ram during bulk storage migration
2018-03-09 17:54 [Qemu-devel] [PULL 0/6] migration queue Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 1/6] migration: fix minor finalize leak Dr. David Alan Gilbert (git)
@ 2018-03-09 17:54 ` Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 3/6] migration/block: reset dirty bitmap before read in bulk phase Dr. David Alan Gilbert (git)
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2018-03-09 17:54 UTC (permalink / raw)
To: qemu-devel; +Cc: marcandre.lureau, pl, peterx, quintela
From: Peter Lieven <pl@kamp.de>
this patch makes the bulk phase of a block migration to take
place before we start transferring ram. As the bulk block migration
can take a long time its pointless to transfer ram during that phase.
Signed-off-by: Peter Lieven <pl@kamp.de>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <1520507908-16743-2-git-send-email-pl@kamp.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/migration/ram.c b/migration/ram.c
index 3b6c077964..7266351fd0 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2258,6 +2258,13 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
int64_t t0;
int done = 0;
+ if (blk_mig_bulk_active()) {
+ /* Avoid transferring ram during bulk phase of block migration as
+ * the bulk phase will usually take a long time and transferring
+ * ram updates during that time is pointless. */
+ goto out;
+ }
+
rcu_read_lock();
if (ram_list.version != rs->last_version) {
ram_state_reset(rs);
@@ -2304,6 +2311,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
*/
ram_control_after_iterate(f, RAM_CONTROL_ROUND);
+out:
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
ram_counters.transferred += 8;
--
2.14.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PULL 3/6] migration/block: reset dirty bitmap before read in bulk phase
2018-03-09 17:54 [Qemu-devel] [PULL 0/6] migration queue Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 1/6] migration: fix minor finalize leak Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 2/6] migration: do not transfer ram during bulk storage migration Dr. David Alan Gilbert (git)
@ 2018-03-09 17:54 ` Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 4/6] migration/block: rename MAX_INFLIGHT_IO to MAX_IO_BUFFERS Dr. David Alan Gilbert (git)
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2018-03-09 17:54 UTC (permalink / raw)
To: qemu-devel; +Cc: marcandre.lureau, pl, peterx, quintela
From: Peter Lieven <pl@kamp.de>
Reset the dirty bitmap before reading to make sure we don't miss
any new data.
Cc: qemu-stable@nongnu.org
Signed-off-by: Peter Lieven <pl@kamp.de>
Message-Id: <1520507908-16743-3-git-send-email-pl@kamp.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/block.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/migration/block.c b/migration/block.c
index 1f03946797..87bb35ce63 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -331,11 +331,10 @@ static int mig_save_device_bulk(QEMUFile *f, BlkMigDevState *bmds)
*/
qemu_mutex_lock_iothread();
aio_context_acquire(blk_get_aio_context(bmds->blk));
- blk->aiocb = blk_aio_preadv(bb, cur_sector * BDRV_SECTOR_SIZE, &blk->qiov,
- 0, blk_mig_read_cb, blk);
-
bdrv_reset_dirty_bitmap(bmds->dirty_bitmap, cur_sector * BDRV_SECTOR_SIZE,
nr_sectors * BDRV_SECTOR_SIZE);
+ blk->aiocb = blk_aio_preadv(bb, cur_sector * BDRV_SECTOR_SIZE, &blk->qiov,
+ 0, blk_mig_read_cb, blk);
aio_context_release(blk_get_aio_context(bmds->blk));
qemu_mutex_unlock_iothread();
--
2.14.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PULL 4/6] migration/block: rename MAX_INFLIGHT_IO to MAX_IO_BUFFERS
2018-03-09 17:54 [Qemu-devel] [PULL 0/6] migration queue Dr. David Alan Gilbert (git)
` (2 preceding siblings ...)
2018-03-09 17:54 ` [Qemu-devel] [PULL 3/6] migration/block: reset dirty bitmap before read in bulk phase Dr. David Alan Gilbert (git)
@ 2018-03-09 17:54 ` Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 5/6] migration: fix applying wrong capabilities Dr. David Alan Gilbert (git)
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2018-03-09 17:54 UTC (permalink / raw)
To: qemu-devel; +Cc: marcandre.lureau, pl, peterx, quintela
From: Peter Lieven <pl@kamp.de>
this actually limits (as the original commit mesage suggests) the
number of I/O buffers that can be allocated and not the number
of parallel (inflight) I/O requests.
Signed-off-by: Peter Lieven <pl@kamp.de>
Message-Id: <1520507908-16743-4-git-send-email-pl@kamp.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/block.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/migration/block.c b/migration/block.c
index 87bb35ce63..41b95d1dd8 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -36,7 +36,7 @@
#define MAX_IS_ALLOCATED_SEARCH (65536 * BDRV_SECTOR_SIZE)
-#define MAX_INFLIGHT_IO 512
+#define MAX_IO_BUFFERS 512
//#define DEBUG_BLK_MIGRATION
@@ -775,9 +775,8 @@ static int block_save_iterate(QEMUFile *f, void *opaque)
while ((block_mig_state.submitted +
block_mig_state.read_done) * BLOCK_SIZE <
qemu_file_get_rate_limit(f) &&
- (block_mig_state.submitted +
- block_mig_state.read_done) <
- MAX_INFLIGHT_IO) {
+ (block_mig_state.submitted + block_mig_state.read_done) <
+ MAX_IO_BUFFERS) {
blk_mig_unlock();
if (block_mig_state.bulk_completed == 0) {
/* first finish the bulk phase */
--
2.14.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PULL 5/6] migration: fix applying wrong capabilities
2018-03-09 17:54 [Qemu-devel] [PULL 0/6] migration queue Dr. David Alan Gilbert (git)
` (3 preceding siblings ...)
2018-03-09 17:54 ` [Qemu-devel] [PULL 4/6] migration/block: rename MAX_INFLIGHT_IO to MAX_IO_BUFFERS Dr. David Alan Gilbert (git)
@ 2018-03-09 17:54 ` Dr. David Alan Gilbert (git)
2018-03-09 17:54 ` [Qemu-devel] [PULL 6/6] tests: Silence migration-test 'bad' test Dr. David Alan Gilbert (git)
2018-03-12 14:06 ` [Qemu-devel] [PULL 0/6] migration queue Peter Maydell
6 siblings, 0 replies; 8+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2018-03-09 17:54 UTC (permalink / raw)
To: qemu-devel; +Cc: marcandre.lureau, pl, peterx, quintela
From: Peter Xu <peterx@redhat.com>
When setting migration capabilities via QMP/HMP, we'll apply them even
if the capability check failed. Fix it.
Fixes: 4a84214ebe ("migration: provide migrate_caps_check()", 2017-07-18)
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180305094938.31374-1-peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/migration.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/migration/migration.c b/migration/migration.c
index 62c243d2d4..6a4780ef6f 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -747,13 +747,15 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params,
{
MigrationState *s = migrate_get_current();
MigrationCapabilityStatusList *cap;
+ bool cap_list[MIGRATION_CAPABILITY__MAX];
if (migration_is_setup_or_active(s->state)) {
error_setg(errp, QERR_MIGRATION_ACTIVE);
return;
}
- if (!migrate_caps_check(s->enabled_capabilities, params, errp)) {
+ memcpy(cap_list, s->enabled_capabilities, sizeof(cap_list));
+ if (!migrate_caps_check(cap_list, params, errp)) {
return;
}
--
2.14.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PULL 6/6] tests: Silence migration-test 'bad' test
2018-03-09 17:54 [Qemu-devel] [PULL 0/6] migration queue Dr. David Alan Gilbert (git)
` (4 preceding siblings ...)
2018-03-09 17:54 ` [Qemu-devel] [PULL 5/6] migration: fix applying wrong capabilities Dr. David Alan Gilbert (git)
@ 2018-03-09 17:54 ` Dr. David Alan Gilbert (git)
2018-03-12 14:06 ` [Qemu-devel] [PULL 0/6] migration queue Peter Maydell
6 siblings, 0 replies; 8+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2018-03-09 17:54 UTC (permalink / raw)
To: qemu-devel; +Cc: marcandre.lureau, pl, peterx, quintela
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
In 2c9bb29703c I added a migration test that purposely fails;
unfortunately it prints a copy of the failure message to stderr
which makes the output a bit messy.
Hide stderr for that test.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20180306173042.24572-1-dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Tested-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
tests/migration-test.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/tests/migration-test.c b/tests/migration-test.c
index 74f9361bdd..422bf1afdf 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -382,7 +382,7 @@ static void migrate_start_postcopy(QTestState *who)
}
static void test_migrate_start(QTestState **from, QTestState **to,
- const char *uri)
+ const char *uri, bool hide_stderr)
{
gchar *cmd_src, *cmd_dst;
char *bootpath = g_strdup_printf("%s/bootsect", tmpfs);
@@ -427,6 +427,17 @@ static void test_migrate_start(QTestState **from, QTestState **to,
g_free(bootpath);
+ if (hide_stderr) {
+ gchar *tmp;
+ tmp = g_strdup_printf("%s 2>/dev/null", cmd_src);
+ g_free(cmd_src);
+ cmd_src = tmp;
+
+ tmp = g_strdup_printf("%s 2>/dev/null", cmd_dst);
+ g_free(cmd_dst);
+ cmd_dst = tmp;
+ }
+
*from = qtest_start(cmd_src);
g_free(cmd_src);
@@ -518,7 +529,7 @@ static void test_migrate(void)
char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
QTestState *from, *to;
- test_migrate_start(&from, &to, uri);
+ test_migrate_start(&from, &to, uri, false);
migrate_set_capability(from, "postcopy-ram", "true");
migrate_set_capability(to, "postcopy-ram", "true");
@@ -560,7 +571,7 @@ static void test_baddest(void)
const char *status;
bool failed;
- test_migrate_start(&from, &to, "tcp:0:0");
+ test_migrate_start(&from, &to, "tcp:0:0", true);
migrate(from, "tcp:0:0");
do {
rsp = wait_command(from, "{ 'execute': 'query-migrate' }");
--
2.14.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PULL 0/6] migration queue
2018-03-09 17:54 [Qemu-devel] [PULL 0/6] migration queue Dr. David Alan Gilbert (git)
` (5 preceding siblings ...)
2018-03-09 17:54 ` [Qemu-devel] [PULL 6/6] tests: Silence migration-test 'bad' test Dr. David Alan Gilbert (git)
@ 2018-03-12 14:06 ` Peter Maydell
6 siblings, 0 replies; 8+ messages in thread
From: Peter Maydell @ 2018-03-12 14:06 UTC (permalink / raw)
To: Dr. David Alan Gilbert (git)
Cc: QEMU Developers, Marc-André Lureau, Peter Lieven, Peter Xu,
Juan Quintela
On 9 March 2018 at 17:54, Dr. David Alan Gilbert (git)
<dgilbert@redhat.com> wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The following changes since commit b39b61e410022f96ceb53d4381d25cba5126ac44:
>
> memory: fix flatview_access_valid RCU read lock/unlock imbalance (2018-03-09 15:55:20 +0000)
>
> are available in the Git repository at:
>
> git://github.com/dagrh/qemu.git tags/pull-migration-20180309a
>
> for you to fetch changes up to f96d6651e4b7cb8a8e91774b9330d82c333171d6:
>
> tests: Silence migration-test 'bad' test (2018-03-09 17:39:25 +0000)
>
> ----------------------------------------------------------------
> Migration pull 2018-03-09
>
> ----------------------------------------------------------------
> Dr. David Alan Gilbert (1):
> tests: Silence migration-test 'bad' test
>
> Marc-André Lureau (1):
> migration: fix minor finalize leak
>
> Peter Lieven (3):
> migration: do not transfer ram during bulk storage migration
> migration/block: reset dirty bitmap before read in bulk phase
> migration/block: rename MAX_INFLIGHT_IO to MAX_IO_BUFFERS
>
> Peter Xu (1):
> migration: fix applying wrong capabilities
>
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 8+ messages in thread