* [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration
@ 2018-08-03 9:13 Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 01/12] migration: disable RDMA WRITE after postcopy started Lidong Chen
` (13 more replies)
0 siblings, 14 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert; +Cc: qemu-devel, Lidong Chen
The RDMA QIOChannel does not support bi-directional communication, so when RDMA
live migration with postcopy enabled, the source qemu return path get qemu file
error.
These patches implement bi-directional communication for RDMA QIOChannel and
disable the RDMA WRITE during the postcopy phase.
This patch just make postcopy works, and will improve performance later.
[v6]
- rebase
- add the check whether release rdma resource has finished(David)
- remove unnecessary RDMA_CONTROL_ERROR when cleanup(David)
- poll the cm event for destination qemu
[v5]
- rebase
- fix bug for create a dedicated thread to release rdma resource(David)
- fix bug for poll the cm event while wait RDMA work request completion(David,Gal)
[v4]
- not wait RDMA_CM_EVENT_DISCONNECTED event after rdma_disconnect
- implement io_set_aio_fd_handler function for RDMA QIOChannel (Juan Quintela)
- invoke qio_channel_yield only when qemu_in_coroutine() (Juan Quintela)
- create a dedicated thread to release rdma resource
- poll the cm event while wait RDMA work request completion
- implement the shutdown function for RDMA QIOChannel
[v3]
- add a mutex in QEMUFile struct to avoid concurrent channel close (Daniel)
- destroy the mutex before free QEMUFile (David)
- use rdmain and rmdaout instead of rdma->return_path (Daniel)
[v2]
- does not update bytes_xfer when disable RDMA WRITE (David)
- implement bi-directional communication for RDMA QIOChannel (Daniel)
Lidong Chen (12):
migration: disable RDMA WRITE after postcopy started
migration: create a dedicated connection for rdma return path
migration: avoid concurrent invoke channel_close by different threads
migration: implement bi-directional RDMA QIOChannel
migration: Stop rdma yielding during incoming postcopy
migration: implement io_set_aio_fd_handler function for RDMA
QIOChannel
migration: invoke qio_channel_yield only when qemu_in_coroutine()
migration: poll the cm event while wait RDMA work request completion
migration: implement the shutdown for RDMA QIOChannel
migration: poll the cm event for destination qemu
migration: remove the unnecessary RDMA_CONTROL_ERROR message
migration: create a dedicated thread to release rdma resource
migration/colo.c | 2 +
migration/migration.c | 13 +-
migration/migration.h | 10 +
migration/postcopy-ram.c | 2 +
migration/qemu-file-channel.c | 12 +-
migration/qemu-file.c | 14 +-
migration/ram.c | 4 +
migration/rdma.c | 448 ++++++++++++++++++++++++++++++++++++++----
migration/savevm.c | 3 +
9 files changed, 458 insertions(+), 50 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 01/12] migration: disable RDMA WRITE after postcopy started
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 02/12] migration: create a dedicated connection for rdma return path Lidong Chen
` (12 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
RDMA WRITE operations are performed with no notification to the destination
qemu, then the destination qemu can not wakeup. This patch disable RDMA WRITE
after postcopy started.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/qemu-file.c | 8 ++++++--
migration/rdma.c | 12 ++++++++++++
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 0463f4c..977b9ae 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -253,8 +253,12 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset,
if (f->hooks && f->hooks->save_page) {
int ret = f->hooks->save_page(f, f->opaque, block_offset,
offset, size, bytes_sent);
- f->bytes_xfer += size;
- if (ret != RAM_SAVE_CONTROL_DELAYED) {
+ if (ret != RAM_SAVE_CONTROL_NOT_SUPP) {
+ f->bytes_xfer += size;
+ }
+
+ if (ret != RAM_SAVE_CONTROL_DELAYED &&
+ ret != RAM_SAVE_CONTROL_NOT_SUPP) {
if (bytes_sent && *bytes_sent > 0) {
qemu_update_position(f, *bytes_sent);
} else if (ret < 0) {
diff --git a/migration/rdma.c b/migration/rdma.c
index 8bd7159..76424a5 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2921,6 +2921,10 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void *opaque,
CHECK_ERROR_STATE();
+ if (migrate_get_current()->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+ return RAM_SAVE_CONTROL_NOT_SUPP;
+ }
+
qemu_fflush(f);
if (size > 0) {
@@ -3480,6 +3484,10 @@ static int qemu_rdma_registration_start(QEMUFile *f, void *opaque,
CHECK_ERROR_STATE();
+ if (migrate_get_current()->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+ return 0;
+ }
+
trace_qemu_rdma_registration_start(flags);
qemu_put_be64(f, RAM_SAVE_FLAG_HOOK);
qemu_fflush(f);
@@ -3502,6 +3510,10 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque,
CHECK_ERROR_STATE();
+ if (migrate_get_current()->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+ return 0;
+ }
+
qemu_fflush(f);
ret = qemu_rdma_drain_cq(f, rdma);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 02/12] migration: create a dedicated connection for rdma return path
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 01/12] migration: disable RDMA WRITE after postcopy started Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 03/12] migration: avoid concurrent invoke channel_close by different threads Lidong Chen
` (11 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
If start a RDMA migration with postcopy enabled, the source qemu
establish a dedicated connection for return path.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/rdma.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 91 insertions(+), 3 deletions(-)
diff --git a/migration/rdma.c b/migration/rdma.c
index 76424a5..57af5ed 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -387,6 +387,10 @@ typedef struct RDMAContext {
uint64_t unregistrations[RDMA_SIGNALED_SEND_MAX];
GHashTable *blockmap;
+
+ /* the RDMAContext for return path */
+ struct RDMAContext *return_path;
+ bool is_return_path;
} RDMAContext;
#define TYPE_QIO_CHANNEL_RDMA "qio-channel-rdma"
@@ -2323,10 +2327,22 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
rdma_destroy_id(rdma->cm_id);
rdma->cm_id = NULL;
}
+
+ /* the destination side, listen_id and channel is shared */
if (rdma->listen_id) {
- rdma_destroy_id(rdma->listen_id);
+ if (!rdma->is_return_path) {
+ rdma_destroy_id(rdma->listen_id);
+ }
rdma->listen_id = NULL;
+
+ if (rdma->channel) {
+ if (!rdma->is_return_path) {
+ rdma_destroy_event_channel(rdma->channel);
+ }
+ rdma->channel = NULL;
+ }
}
+
if (rdma->channel) {
rdma_destroy_event_channel(rdma->channel);
rdma->channel = NULL;
@@ -2555,6 +2571,25 @@ err_dest_init_create_listen_id:
}
+static void qemu_rdma_return_path_dest_init(RDMAContext *rdma_return_path,
+ RDMAContext *rdma)
+{
+ int idx;
+
+ for (idx = 0; idx < RDMA_WRID_MAX; idx++) {
+ rdma_return_path->wr_data[idx].control_len = 0;
+ rdma_return_path->wr_data[idx].control_curr = NULL;
+ }
+
+ /*the CM channel and CM id is shared*/
+ rdma_return_path->channel = rdma->channel;
+ rdma_return_path->listen_id = rdma->listen_id;
+
+ rdma->return_path = rdma_return_path;
+ rdma_return_path->return_path = rdma;
+ rdma_return_path->is_return_path = true;
+}
+
static void *qemu_rdma_data_init(const char *host_port, Error **errp)
{
RDMAContext *rdma = NULL;
@@ -3012,6 +3047,8 @@ err:
return ret;
}
+static void rdma_accept_incoming_migration(void *opaque);
+
static int qemu_rdma_accept(RDMAContext *rdma)
{
RDMACapabilities cap;
@@ -3106,7 +3143,14 @@ static int qemu_rdma_accept(RDMAContext *rdma)
}
}
- qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
+ /* Accept the second connection request for return path */
+ if (migrate_postcopy() && !rdma->is_return_path) {
+ qemu_set_fd_handler(rdma->channel->fd, rdma_accept_incoming_migration,
+ NULL,
+ (void *)(intptr_t)rdma->return_path);
+ } else {
+ qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
+ }
ret = rdma_accept(rdma->cm_id, &conn_param);
if (ret) {
@@ -3691,6 +3735,10 @@ static void rdma_accept_incoming_migration(void *opaque)
trace_qemu_rdma_accept_incoming_migration_accepted();
+ if (rdma->is_return_path) {
+ return;
+ }
+
f = qemu_fopen_rdma(rdma, "rb");
if (f == NULL) {
ERROR(errp, "could not qemu_fopen_rdma!");
@@ -3705,7 +3753,7 @@ static void rdma_accept_incoming_migration(void *opaque)
void rdma_start_incoming_migration(const char *host_port, Error **errp)
{
int ret;
- RDMAContext *rdma;
+ RDMAContext *rdma, *rdma_return_path;
Error *local_err = NULL;
trace_rdma_start_incoming_migration();
@@ -3732,12 +3780,24 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp)
trace_rdma_start_incoming_migration_after_rdma_listen();
+ /* initialize the RDMAContext for return path */
+ if (migrate_postcopy()) {
+ rdma_return_path = qemu_rdma_data_init(host_port, &local_err);
+
+ if (rdma_return_path == NULL) {
+ goto err;
+ }
+
+ qemu_rdma_return_path_dest_init(rdma_return_path, rdma);
+ }
+
qemu_set_fd_handler(rdma->channel->fd, rdma_accept_incoming_migration,
NULL, (void *)(intptr_t)rdma);
return;
err:
error_propagate(errp, local_err);
g_free(rdma);
+ g_free(rdma_return_path);
}
void rdma_start_outgoing_migration(void *opaque,
@@ -3745,6 +3805,7 @@ void rdma_start_outgoing_migration(void *opaque,
{
MigrationState *s = opaque;
RDMAContext *rdma = qemu_rdma_data_init(host_port, errp);
+ RDMAContext *rdma_return_path = NULL;
int ret = 0;
if (rdma == NULL) {
@@ -3765,6 +3826,32 @@ void rdma_start_outgoing_migration(void *opaque,
goto err;
}
+ /* RDMA postcopy need a seprate queue pair for return path */
+ if (migrate_postcopy()) {
+ rdma_return_path = qemu_rdma_data_init(host_port, errp);
+
+ if (rdma_return_path == NULL) {
+ goto err;
+ }
+
+ ret = qemu_rdma_source_init(rdma_return_path,
+ s->enabled_capabilities[MIGRATION_CAPABILITY_RDMA_PIN_ALL], errp);
+
+ if (ret) {
+ goto err;
+ }
+
+ ret = qemu_rdma_connect(rdma_return_path, errp);
+
+ if (ret) {
+ goto err;
+ }
+
+ rdma->return_path = rdma_return_path;
+ rdma_return_path->return_path = rdma;
+ rdma_return_path->is_return_path = true;
+ }
+
trace_rdma_start_outgoing_migration_after_rdma_connect();
s->to_dst_file = qemu_fopen_rdma(rdma, "wb");
@@ -3772,4 +3859,5 @@ void rdma_start_outgoing_migration(void *opaque,
return;
err:
g_free(rdma);
+ g_free(rdma_return_path);
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 03/12] migration: avoid concurrent invoke channel_close by different threads
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 01/12] migration: disable RDMA WRITE after postcopy started Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 02/12] migration: create a dedicated connection for rdma return path Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-06 9:07 ` 858585 jemmy
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 04/12] migration: implement bi-directional RDMA QIOChannel Lidong Chen
` (10 subsequent siblings)
13 siblings, 1 reply; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
The channel_close maybe invoked by different threads. For example, source
qemu invokes qemu_fclose in main thread, migration thread and return path
thread. Destination qemu invokes qemu_fclose in main thread, listen thread
and COLO incoming thread.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
---
migration/migration.c | 2 ++
migration/migration.h | 7 +++++++
migration/qemu-file.c | 6 ++++--
3 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/migration/migration.c b/migration/migration.c
index b7d9854..a3a0756 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -3200,6 +3200,7 @@ static void migration_instance_finalize(Object *obj)
qemu_sem_destroy(&ms->postcopy_pause_sem);
qemu_sem_destroy(&ms->postcopy_pause_rp_sem);
qemu_sem_destroy(&ms->rp_state.rp_sem);
+ qemu_mutex_destroy(&ms->qemu_file_close_lock);
error_free(ms->error);
}
@@ -3236,6 +3237,7 @@ static void migration_instance_init(Object *obj)
qemu_sem_init(&ms->rp_state.rp_sem, 0);
qemu_sem_init(&ms->rate_limit_sem, 0);
qemu_mutex_init(&ms->qemu_file_lock);
+ qemu_mutex_init(&ms->qemu_file_close_lock);
}
/*
diff --git a/migration/migration.h b/migration/migration.h
index 64a7b33..a50c2de 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -122,6 +122,13 @@ struct MigrationState
QemuMutex qemu_file_lock;
/*
+ * The to_src_file and from_dst_file point to one QIOChannelRDMA,
+ * And qemu_fclose maybe invoked by different threads. use this lock
+ * to avoid concurrent invoke channel_close by different threads.
+ */
+ QemuMutex qemu_file_close_lock;
+
+ /*
* Used to allow urgent requests to override rate limiting.
*/
QemuSemaphore rate_limit_sem;
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 977b9ae..74c48e0 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -323,12 +323,14 @@ void qemu_update_position(QEMUFile *f, size_t size)
*/
int qemu_fclose(QEMUFile *f)
{
- int ret;
+ int ret, ret2;
qemu_fflush(f);
ret = qemu_file_get_error(f);
if (f->ops->close) {
- int ret2 = f->ops->close(f->opaque);
+ qemu_mutex_lock(&migrate_get_current()->qemu_file_close_lock);
+ ret2 = f->ops->close(f->opaque);
+ qemu_mutex_unlock(&migrate_get_current()->qemu_file_close_lock);
if (ret >= 0) {
ret = ret2;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 04/12] migration: implement bi-directional RDMA QIOChannel
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (2 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 03/12] migration: avoid concurrent invoke channel_close by different threads Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 05/12] migration: Stop rdma yielding during incoming postcopy Lidong Chen
` (9 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
This patch implements bi-directional RDMA QIOChannel. Because different
threads may access RDMAQIOChannel currently, this patch use RCU to protect it.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/colo.c | 2 +
migration/migration.c | 2 +
migration/postcopy-ram.c | 2 +
migration/ram.c | 4 +
migration/rdma.c | 196 ++++++++++++++++++++++++++++++++++++++++-------
migration/savevm.c | 3 +
6 files changed, 183 insertions(+), 26 deletions(-)
diff --git a/migration/colo.c b/migration/colo.c
index 4381067..88936f5 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -534,6 +534,7 @@ void *colo_process_incoming_thread(void *opaque)
uint64_t value;
Error *local_err = NULL;
+ rcu_register_thread();
qemu_sem_init(&mis->colo_incoming_sem, 0);
migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE,
@@ -666,5 +667,6 @@ out:
}
migration_incoming_exit_colo();
+ rcu_unregister_thread();
return NULL;
}
diff --git a/migration/migration.c b/migration/migration.c
index a3a0756..f190964 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2104,6 +2104,7 @@ static void *source_return_path_thread(void *opaque)
int res;
trace_source_return_path_thread_entry();
+ rcu_register_thread();
retry:
while (!ms->rp_state.error && !qemu_file_get_error(rp) &&
@@ -2243,6 +2244,7 @@ out:
trace_source_return_path_thread_end();
ms->rp_state.from_dst_file = NULL;
qemu_fclose(rp);
+ rcu_unregister_thread();
return NULL;
}
diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 932f188..3952d78 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -853,6 +853,7 @@ static void *postcopy_ram_fault_thread(void *opaque)
RAMBlock *rb = NULL;
trace_postcopy_ram_fault_thread_entry();
+ rcu_register_thread();
mis->last_rb = NULL; /* last RAMBlock we sent part of */
qemu_sem_post(&mis->fault_thread_sem);
@@ -1059,6 +1060,7 @@ retry:
}
}
}
+ rcu_unregister_thread();
trace_postcopy_ram_fault_thread_exit();
g_free(pfd);
return NULL;
diff --git a/migration/ram.c b/migration/ram.c
index 24dea27..4da0930 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -988,6 +988,7 @@ static void *multifd_send_thread(void *opaque)
int ret;
trace_multifd_send_thread_start(p->id);
+ rcu_register_thread();
if (multifd_send_initial_packet(p, &local_err) < 0) {
goto out;
@@ -1050,6 +1051,7 @@ out:
p->running = false;
qemu_mutex_unlock(&p->mutex);
+ rcu_unregister_thread();
trace_multifd_send_thread_end(p->id, p->num_packets, p->num_pages);
return NULL;
@@ -1219,6 +1221,7 @@ static void *multifd_recv_thread(void *opaque)
int ret;
trace_multifd_recv_thread_start(p->id);
+ rcu_register_thread();
while (true) {
uint32_t used;
@@ -1265,6 +1268,7 @@ static void *multifd_recv_thread(void *opaque)
p->running = false;
qemu_mutex_unlock(&p->mutex);
+ rcu_unregister_thread();
trace_multifd_recv_thread_end(p->id, p->num_packets, p->num_pages);
return NULL;
diff --git a/migration/rdma.c b/migration/rdma.c
index 57af5ed..a5535fb 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -86,6 +86,7 @@ static uint32_t known_capabilities = RDMA_CAPABILITY_PIN_ALL;
" to abort!"); \
rdma->error_reported = 1; \
} \
+ rcu_read_unlock(); \
return rdma->error_state; \
} \
} while (0)
@@ -402,7 +403,8 @@ typedef struct QIOChannelRDMA QIOChannelRDMA;
struct QIOChannelRDMA {
QIOChannel parent;
- RDMAContext *rdma;
+ RDMAContext *rdmain;
+ RDMAContext *rdmaout;
QEMUFile *file;
bool blocking; /* XXX we don't actually honour this yet */
};
@@ -2630,12 +2632,20 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *ioc,
{
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
QEMUFile *f = rioc->file;
- RDMAContext *rdma = rioc->rdma;
+ RDMAContext *rdma;
int ret;
ssize_t done = 0;
size_t i;
size_t len = 0;
+ rcu_read_lock();
+ rdma = atomic_rcu_read(&rioc->rdmaout);
+
+ if (!rdma) {
+ rcu_read_unlock();
+ return -EIO;
+ }
+
CHECK_ERROR_STATE();
/*
@@ -2645,6 +2655,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *ioc,
ret = qemu_rdma_write_flush(f, rdma);
if (ret < 0) {
rdma->error_state = ret;
+ rcu_read_unlock();
return ret;
}
@@ -2664,6 +2675,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *ioc,
if (ret < 0) {
rdma->error_state = ret;
+ rcu_read_unlock();
return ret;
}
@@ -2672,6 +2684,7 @@ static ssize_t qio_channel_rdma_writev(QIOChannel *ioc,
}
}
+ rcu_read_unlock();
return done;
}
@@ -2705,12 +2718,20 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc,
Error **errp)
{
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
- RDMAContext *rdma = rioc->rdma;
+ RDMAContext *rdma;
RDMAControlHeader head;
int ret = 0;
ssize_t i;
size_t done = 0;
+ rcu_read_lock();
+ rdma = atomic_rcu_read(&rioc->rdmain);
+
+ if (!rdma) {
+ rcu_read_unlock();
+ return -EIO;
+ }
+
CHECK_ERROR_STATE();
for (i = 0; i < niov; i++) {
@@ -2722,7 +2743,7 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc,
* were given and dish out the bytes until we run
* out of bytes.
*/
- ret = qemu_rdma_fill(rioc->rdma, data, want, 0);
+ ret = qemu_rdma_fill(rdma, data, want, 0);
done += ret;
want -= ret;
/* Got what we needed, so go to next iovec */
@@ -2744,25 +2765,28 @@ static ssize_t qio_channel_rdma_readv(QIOChannel *ioc,
if (ret < 0) {
rdma->error_state = ret;
+ rcu_read_unlock();
return ret;
}
/*
* SEND was received with new bytes, now try again.
*/
- ret = qemu_rdma_fill(rioc->rdma, data, want, 0);
+ ret = qemu_rdma_fill(rdma, data, want, 0);
done += ret;
want -= ret;
/* Still didn't get enough, so lets just return */
if (want) {
if (done == 0) {
+ rcu_read_unlock();
return QIO_CHANNEL_ERR_BLOCK;
} else {
break;
}
}
}
+ rcu_read_unlock();
return done;
}
@@ -2814,15 +2838,29 @@ qio_channel_rdma_source_prepare(GSource *source,
gint *timeout)
{
QIOChannelRDMASource *rsource = (QIOChannelRDMASource *)source;
- RDMAContext *rdma = rsource->rioc->rdma;
+ RDMAContext *rdma;
GIOCondition cond = 0;
*timeout = -1;
+ rcu_read_lock();
+ if (rsource->condition == G_IO_IN) {
+ rdma = atomic_rcu_read(&rsource->rioc->rdmain);
+ } else {
+ rdma = atomic_rcu_read(&rsource->rioc->rdmaout);
+ }
+
+ if (!rdma) {
+ error_report("RDMAContext is NULL when prepare Gsource");
+ rcu_read_unlock();
+ return FALSE;
+ }
+
if (rdma->wr_data[0].control_len) {
cond |= G_IO_IN;
}
cond |= G_IO_OUT;
+ rcu_read_unlock();
return cond & rsource->condition;
}
@@ -2830,14 +2868,28 @@ static gboolean
qio_channel_rdma_source_check(GSource *source)
{
QIOChannelRDMASource *rsource = (QIOChannelRDMASource *)source;
- RDMAContext *rdma = rsource->rioc->rdma;
+ RDMAContext *rdma;
GIOCondition cond = 0;
+ rcu_read_lock();
+ if (rsource->condition == G_IO_IN) {
+ rdma = atomic_rcu_read(&rsource->rioc->rdmain);
+ } else {
+ rdma = atomic_rcu_read(&rsource->rioc->rdmaout);
+ }
+
+ if (!rdma) {
+ error_report("RDMAContext is NULL when check Gsource");
+ rcu_read_unlock();
+ return FALSE;
+ }
+
if (rdma->wr_data[0].control_len) {
cond |= G_IO_IN;
}
cond |= G_IO_OUT;
+ rcu_read_unlock();
return cond & rsource->condition;
}
@@ -2848,14 +2900,28 @@ qio_channel_rdma_source_dispatch(GSource *source,
{
QIOChannelFunc func = (QIOChannelFunc)callback;
QIOChannelRDMASource *rsource = (QIOChannelRDMASource *)source;
- RDMAContext *rdma = rsource->rioc->rdma;
+ RDMAContext *rdma;
GIOCondition cond = 0;
+ rcu_read_lock();
+ if (rsource->condition == G_IO_IN) {
+ rdma = atomic_rcu_read(&rsource->rioc->rdmain);
+ } else {
+ rdma = atomic_rcu_read(&rsource->rioc->rdmaout);
+ }
+
+ if (!rdma) {
+ error_report("RDMAContext is NULL when dispatch Gsource");
+ rcu_read_unlock();
+ return FALSE;
+ }
+
if (rdma->wr_data[0].control_len) {
cond |= G_IO_IN;
}
cond |= G_IO_OUT;
+ rcu_read_unlock();
return (*func)(QIO_CHANNEL(rsource->rioc),
(cond & rsource->condition),
user_data);
@@ -2900,15 +2966,32 @@ static int qio_channel_rdma_close(QIOChannel *ioc,
Error **errp)
{
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
+ RDMAContext *rdmain, *rdmaout;
trace_qemu_rdma_close();
- if (rioc->rdma) {
- if (!rioc->rdma->error_state) {
- rioc->rdma->error_state = qemu_file_get_error(rioc->file);
- }
- qemu_rdma_cleanup(rioc->rdma);
- g_free(rioc->rdma);
- rioc->rdma = NULL;
+
+ rdmain = rioc->rdmain;
+ if (rdmain) {
+ atomic_rcu_set(&rioc->rdmain, NULL);
+ }
+
+ rdmaout = rioc->rdmaout;
+ if (rdmaout) {
+ atomic_rcu_set(&rioc->rdmaout, NULL);
}
+
+ synchronize_rcu();
+
+ if (rdmain) {
+ qemu_rdma_cleanup(rdmain);
+ }
+
+ if (rdmaout) {
+ qemu_rdma_cleanup(rdmaout);
+ }
+
+ g_free(rdmain);
+ g_free(rdmaout);
+
return 0;
}
@@ -2951,12 +3034,21 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void *opaque,
size_t size, uint64_t *bytes_sent)
{
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(opaque);
- RDMAContext *rdma = rioc->rdma;
+ RDMAContext *rdma;
int ret;
+ rcu_read_lock();
+ rdma = atomic_rcu_read(&rioc->rdmaout);
+
+ if (!rdma) {
+ rcu_read_unlock();
+ return -EIO;
+ }
+
CHECK_ERROR_STATE();
if (migrate_get_current()->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+ rcu_read_unlock();
return RAM_SAVE_CONTROL_NOT_SUPP;
}
@@ -3041,9 +3133,11 @@ static size_t qemu_rdma_save_page(QEMUFile *f, void *opaque,
}
}
+ rcu_read_unlock();
return RAM_SAVE_CONTROL_DELAYED;
err:
rdma->error_state = ret;
+ rcu_read_unlock();
return ret;
}
@@ -3219,8 +3313,8 @@ static int qemu_rdma_registration_handle(QEMUFile *f, void *opaque)
RDMAControlHeader blocks = { .type = RDMA_CONTROL_RAM_BLOCKS_RESULT,
.repeat = 1 };
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(opaque);
- RDMAContext *rdma = rioc->rdma;
- RDMALocalBlocks *local = &rdma->local_ram_blocks;
+ RDMAContext *rdma;
+ RDMALocalBlocks *local;
RDMAControlHeader head;
RDMARegister *reg, *registers;
RDMACompress *comp;
@@ -3233,8 +3327,17 @@ static int qemu_rdma_registration_handle(QEMUFile *f, void *opaque)
int count = 0;
int i = 0;
+ rcu_read_lock();
+ rdma = atomic_rcu_read(&rioc->rdmain);
+
+ if (!rdma) {
+ rcu_read_unlock();
+ return -EIO;
+ }
+
CHECK_ERROR_STATE();
+ local = &rdma->local_ram_blocks;
do {
trace_qemu_rdma_registration_handle_wait();
@@ -3468,6 +3571,7 @@ out:
if (ret < 0) {
rdma->error_state = ret;
}
+ rcu_read_unlock();
return ret;
}
@@ -3481,10 +3585,18 @@ out:
static int
rdma_block_notification_handle(QIOChannelRDMA *rioc, const char *name)
{
- RDMAContext *rdma = rioc->rdma;
+ RDMAContext *rdma;
int curr;
int found = -1;
+ rcu_read_lock();
+ rdma = atomic_rcu_read(&rioc->rdmain);
+
+ if (!rdma) {
+ rcu_read_unlock();
+ return -EIO;
+ }
+
/* Find the matching RAMBlock in our local list */
for (curr = 0; curr < rdma->local_ram_blocks.nb_blocks; curr++) {
if (!strcmp(rdma->local_ram_blocks.block[curr].block_name, name)) {
@@ -3495,6 +3607,7 @@ rdma_block_notification_handle(QIOChannelRDMA *rioc, const char *name)
if (found == -1) {
error_report("RAMBlock '%s' not found on destination", name);
+ rcu_read_unlock();
return -ENOENT;
}
@@ -3502,6 +3615,7 @@ rdma_block_notification_handle(QIOChannelRDMA *rioc, const char *name)
trace_rdma_block_notification_handle(name, rdma->next_src_index);
rdma->next_src_index++;
+ rcu_read_unlock();
return 0;
}
@@ -3524,11 +3638,19 @@ static int qemu_rdma_registration_start(QEMUFile *f, void *opaque,
uint64_t flags, void *data)
{
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(opaque);
- RDMAContext *rdma = rioc->rdma;
+ RDMAContext *rdma;
+
+ rcu_read_lock();
+ rdma = atomic_rcu_read(&rioc->rdmaout);
+ if (!rdma) {
+ rcu_read_unlock();
+ return -EIO;
+ }
CHECK_ERROR_STATE();
if (migrate_get_current()->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+ rcu_read_unlock();
return 0;
}
@@ -3536,6 +3658,7 @@ static int qemu_rdma_registration_start(QEMUFile *f, void *opaque,
qemu_put_be64(f, RAM_SAVE_FLAG_HOOK);
qemu_fflush(f);
+ rcu_read_unlock();
return 0;
}
@@ -3548,13 +3671,21 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque,
{
Error *local_err = NULL, **errp = &local_err;
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(opaque);
- RDMAContext *rdma = rioc->rdma;
+ RDMAContext *rdma;
RDMAControlHeader head = { .len = 0, .repeat = 1 };
int ret = 0;
+ rcu_read_lock();
+ rdma = atomic_rcu_read(&rioc->rdmaout);
+ if (!rdma) {
+ rcu_read_unlock();
+ return -EIO;
+ }
+
CHECK_ERROR_STATE();
if (migrate_get_current()->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+ rcu_read_unlock();
return 0;
}
@@ -3586,6 +3717,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque,
qemu_rdma_reg_whole_ram_blocks : NULL);
if (ret < 0) {
ERROR(errp, "receiving remote info!");
+ rcu_read_unlock();
return ret;
}
@@ -3609,6 +3741,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque,
"not identical on both the source and destination.",
local->nb_blocks, nb_dest_blocks);
rdma->error_state = -EINVAL;
+ rcu_read_unlock();
return -EINVAL;
}
@@ -3625,6 +3758,7 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque,
local->block[i].length,
rdma->dest_blocks[i].length);
rdma->error_state = -EINVAL;
+ rcu_read_unlock();
return -EINVAL;
}
local->block[i].remote_host_addr =
@@ -3642,9 +3776,11 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque,
goto err;
}
+ rcu_read_unlock();
return 0;
err:
rdma->error_state = ret;
+ rcu_read_unlock();
return ret;
}
@@ -3662,10 +3798,15 @@ static const QEMUFileHooks rdma_write_hooks = {
static void qio_channel_rdma_finalize(Object *obj)
{
QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(obj);
- if (rioc->rdma) {
- qemu_rdma_cleanup(rioc->rdma);
- g_free(rioc->rdma);
- rioc->rdma = NULL;
+ if (rioc->rdmain) {
+ qemu_rdma_cleanup(rioc->rdmain);
+ g_free(rioc->rdmain);
+ rioc->rdmain = NULL;
+ }
+ if (rioc->rdmaout) {
+ qemu_rdma_cleanup(rioc->rdmaout);
+ g_free(rioc->rdmaout);
+ rioc->rdmaout = NULL;
}
}
@@ -3705,13 +3846,16 @@ static QEMUFile *qemu_fopen_rdma(RDMAContext *rdma, const char *mode)
}
rioc = QIO_CHANNEL_RDMA(object_new(TYPE_QIO_CHANNEL_RDMA));
- rioc->rdma = rdma;
if (mode[0] == 'w') {
rioc->file = qemu_fopen_channel_output(QIO_CHANNEL(rioc));
+ rioc->rdmaout = rdma;
+ rioc->rdmain = rdma->return_path;
qemu_file_set_hooks(rioc->file, &rdma_write_hooks);
} else {
rioc->file = qemu_fopen_channel_input(QIO_CHANNEL(rioc));
+ rioc->rdmain = rdma;
+ rioc->rdmaout = rdma->return_path;
qemu_file_set_hooks(rioc->file, &rdma_read_hooks);
}
diff --git a/migration/savevm.c b/migration/savevm.c
index 7f92567..13e51f0 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1622,6 +1622,7 @@ static void *postcopy_ram_listen_thread(void *opaque)
qemu_sem_post(&mis->listen_thread_sem);
trace_postcopy_ram_listen_thread_start();
+ rcu_register_thread();
/*
* Because we're a thread and not a coroutine we can't yield
* in qemu_file, and thus we must be blocking now.
@@ -1662,6 +1663,7 @@ static void *postcopy_ram_listen_thread(void *opaque)
* to leave the guest running and fire MCEs for pages that never
* arrived as a desperate recovery step.
*/
+ rcu_unregister_thread();
exit(EXIT_FAILURE);
}
@@ -1676,6 +1678,7 @@ static void *postcopy_ram_listen_thread(void *opaque)
migration_incoming_state_destroy();
qemu_loadvm_state_cleanup();
+ rcu_unregister_thread();
return NULL;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 05/12] migration: Stop rdma yielding during incoming postcopy
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (3 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 04/12] migration: implement bi-directional RDMA QIOChannel Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 06/12] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel Lidong Chen
` (8 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
During incoming postcopy, the destination qemu will invoke
qemu_rdma_wait_comp_channel in a seprate thread. So does not use rdma
yield, and poll the completion channel fd instead.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/rdma.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/migration/rdma.c b/migration/rdma.c
index a5535fb..cfb0671 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -1493,11 +1493,13 @@ static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)
* Coroutine doesn't start until migration_fd_process_incoming()
* so don't yield unless we know we're running inside of a coroutine.
*/
- if (rdma->migration_started_on_destination) {
+ if (rdma->migration_started_on_destination &&
+ migration_incoming_get_current()->state == MIGRATION_STATUS_ACTIVE) {
yield_until_fd_readable(rdma->comp_channel->fd);
} else {
/* This is the source side, we're in a separate thread
* or destination prior to migration_fd_process_incoming()
+ * after postcopy, the destination also in a seprate thread.
* we can't yield; so we have to poll the fd.
* But we need to be able to handle 'cancel' or an error
* without hanging forever.
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 06/12] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (4 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 05/12] migration: Stop rdma yielding during incoming postcopy Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 07/12] migration: invoke qio_channel_yield only when qemu_in_coroutine() Lidong Chen
` (7 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
if qio_channel_rdma_readv return QIO_CHANNEL_ERR_BLOCK, the destination qemu
crash.
The backtrace is:
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00000000008db50e in qio_channel_set_aio_fd_handler (ioc=0x38111e0, ctx=0x3726080,
io_read=0x8db841 <qio_channel_restart_read>, io_write=0x0, opaque=0x38111e0) at io/channel.c:
#2 0x00000000008db952 in qio_channel_set_aio_fd_handlers (ioc=0x38111e0) at io/channel.c:438
#3 0x00000000008dbab4 in qio_channel_yield (ioc=0x38111e0, condition=G_IO_IN) at io/channel.c:47
#4 0x00000000007a870b in channel_get_buffer (opaque=0x38111e0, buf=0x440c038 "", pos=0, size=327
at migration/qemu-file-channel.c:83
#5 0x00000000007a70f6 in qemu_fill_buffer (f=0x440c000) at migration/qemu-file.c:299
#6 0x00000000007a79d0 in qemu_peek_byte (f=0x440c000, offset=0) at migration/qemu-file.c:562
#7 0x00000000007a7a22 in qemu_get_byte (f=0x440c000) at migration/qemu-file.c:575
#8 0x00000000007a7c78 in qemu_get_be32 (f=0x440c000) at migration/qemu-file.c:655
#9 0x00000000007a0508 in qemu_loadvm_state (f=0x440c000) at migration/savevm.c:2126
#10 0x0000000000794141 in process_incoming_migration_co (opaque=0x0) at migration/migration.c:366
#11 0x000000000095c598 in coroutine_trampoline (i0=84033984, i1=0) at util/coroutine-ucontext.c:1
#12 0x00007f9c0db56d40 in ?? () from /lib64/libc.so.6
#13 0x00007f96fe858760 in ?? ()
#14 0x0000000000000000 in ?? ()
RDMA QIOChannel not implement io_set_aio_fd_handler. so
qio_channel_set_aio_fd_handler will access NULL pointer.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
---
migration/rdma.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/migration/rdma.c b/migration/rdma.c
index cfb0671..d6bbf28 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2963,6 +2963,21 @@ static GSource *qio_channel_rdma_create_watch(QIOChannel *ioc,
return source;
}
+static void qio_channel_rdma_set_aio_fd_handler(QIOChannel *ioc,
+ AioContext *ctx,
+ IOHandler *io_read,
+ IOHandler *io_write,
+ void *opaque)
+{
+ QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
+ if (io_read) {
+ aio_set_fd_handler(ctx, rioc->rdmain->comp_channel->fd,
+ false, io_read, io_write, NULL, opaque);
+ } else {
+ aio_set_fd_handler(ctx, rioc->rdmaout->comp_channel->fd,
+ false, io_read, io_write, NULL, opaque);
+ }
+}
static int qio_channel_rdma_close(QIOChannel *ioc,
Error **errp)
@@ -3822,6 +3837,7 @@ static void qio_channel_rdma_class_init(ObjectClass *klass,
ioc_klass->io_set_blocking = qio_channel_rdma_set_blocking;
ioc_klass->io_close = qio_channel_rdma_close;
ioc_klass->io_create_watch = qio_channel_rdma_create_watch;
+ ioc_klass->io_set_aio_fd_handler = qio_channel_rdma_set_aio_fd_handler;
}
static const TypeInfo qio_channel_rdma_info = {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 07/12] migration: invoke qio_channel_yield only when qemu_in_coroutine()
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (5 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 06/12] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 08/12] migration: poll the cm event while wait RDMA work request completion Lidong Chen
` (6 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
when qio_channel_read return QIO_CHANNEL_ERR_BLOCK, the source qemu crash.
The backtrace is:
(gdb) bt
#0 0x00007fb20aba91d7 in raise () from /lib64/libc.so.6
#1 0x00007fb20abaa8c8 in abort () from /lib64/libc.so.6
#2 0x00007fb20aba2146 in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007fb20aba21f2 in __assert_fail () from /lib64/libc.so.6
#4 0x00000000008dba2d in qio_channel_yield (ioc=0x22f9e20, condition=G_IO_IN) at io/channel.c:460
#5 0x00000000007a870b in channel_get_buffer (opaque=0x22f9e20, buf=0x3d54038 "", pos=0, size=32768)
at migration/qemu-file-channel.c:83
#6 0x00000000007a70f6 in qemu_fill_buffer (f=0x3d54000) at migration/qemu-file.c:299
#7 0x00000000007a79d0 in qemu_peek_byte (f=0x3d54000, offset=0) at migration/qemu-file.c:562
#8 0x00000000007a7a22 in qemu_get_byte (f=0x3d54000) at migration/qemu-file.c:575
#9 0x00000000007a7c46 in qemu_get_be16 (f=0x3d54000) at migration/qemu-file.c:647
#10 0x0000000000796db7 in source_return_path_thread (opaque=0x2242280) at migration/migration.c:1794
#11 0x00000000009428fa in qemu_thread_start (args=0x3e58420) at util/qemu-thread-posix.c:504
#12 0x00007fb20af3ddc5 in start_thread () from /lib64/libpthread.so.0
#13 0x00007fb20ac6b74d in clone () from /lib64/libc.so.6
This patch fixed by invoke qio_channel_yield only when qemu_in_coroutine().
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
---
migration/qemu-file-channel.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/migration/qemu-file-channel.c b/migration/qemu-file-channel.c
index e202d73..8e639eb 100644
--- a/migration/qemu-file-channel.c
+++ b/migration/qemu-file-channel.c
@@ -49,7 +49,11 @@ static ssize_t channel_writev_buffer(void *opaque,
ssize_t len;
len = qio_channel_writev(ioc, local_iov, nlocal_iov, NULL);
if (len == QIO_CHANNEL_ERR_BLOCK) {
- qio_channel_wait(ioc, G_IO_OUT);
+ if (qemu_in_coroutine()) {
+ qio_channel_yield(ioc, G_IO_OUT);
+ } else {
+ qio_channel_wait(ioc, G_IO_OUT);
+ }
continue;
}
if (len < 0) {
@@ -80,7 +84,11 @@ static ssize_t channel_get_buffer(void *opaque,
ret = qio_channel_read(ioc, (char *)buf, size, NULL);
if (ret < 0) {
if (ret == QIO_CHANNEL_ERR_BLOCK) {
- qio_channel_yield(ioc, G_IO_IN);
+ if (qemu_in_coroutine()) {
+ qio_channel_yield(ioc, G_IO_IN);
+ } else {
+ qio_channel_wait(ioc, G_IO_IN);
+ }
} else {
/* XXX handle Error * object */
return -EIO;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 08/12] migration: poll the cm event while wait RDMA work request completion
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (6 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 07/12] migration: invoke qio_channel_yield only when qemu_in_coroutine() Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 09/12] migration: implement the shutdown for RDMA QIOChannel Lidong Chen
` (5 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen, Gal Shachaf, Aviad Yehezkel
From: Lidong Chen <jemmy858585@gmail.com>
If the peer qemu is crashed, the qemu_rdma_wait_comp_channel function
maybe loop forever. so we should also poll the cm event fd, and when
receive RDMA_CM_EVENT_DISCONNECTED and RDMA_CM_EVENT_DEVICE_REMOVAL,
we consider some error happened.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Signed-off-by: Gal Shachaf <galsha@mellanox.com>
Signed-off-by: Aviad Yehezkel <aviadye@mellanox.com>
---
migration/rdma.c | 33 ++++++++++++++++++++++++++++++---
1 file changed, 30 insertions(+), 3 deletions(-)
diff --git a/migration/rdma.c b/migration/rdma.c
index d6bbf28..673f126 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -1489,6 +1489,9 @@ static uint64_t qemu_rdma_poll(RDMAContext *rdma, uint64_t *wr_id_out,
*/
static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)
{
+ struct rdma_cm_event *cm_event;
+ int ret = -1;
+
/*
* Coroutine doesn't start until migration_fd_process_incoming()
* so don't yield unless we know we're running inside of a coroutine.
@@ -1505,13 +1508,37 @@ static int qemu_rdma_wait_comp_channel(RDMAContext *rdma)
* without hanging forever.
*/
while (!rdma->error_state && !rdma->received_error) {
- GPollFD pfds[1];
+ GPollFD pfds[2];
pfds[0].fd = rdma->comp_channel->fd;
pfds[0].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
+ pfds[0].revents = 0;
+
+ pfds[1].fd = rdma->channel->fd;
+ pfds[1].events = G_IO_IN | G_IO_HUP | G_IO_ERR;
+ pfds[1].revents = 0;
+
/* 0.1s timeout, should be fine for a 'cancel' */
- switch (qemu_poll_ns(pfds, 1, 100 * 1000 * 1000)) {
+ switch (qemu_poll_ns(pfds, 2, 100 * 1000 * 1000)) {
+ case 2:
case 1: /* fd active */
- return 0;
+ if (pfds[0].revents) {
+ return 0;
+ }
+
+ if (pfds[1].revents) {
+ ret = rdma_get_cm_event(rdma->channel, &cm_event);
+ if (!ret) {
+ rdma_ack_cm_event(cm_event);
+ }
+
+ error_report("receive cm event while wait comp channel,"
+ "cm event is %d", cm_event->event);
+ if (cm_event->event == RDMA_CM_EVENT_DISCONNECTED ||
+ cm_event->event == RDMA_CM_EVENT_DEVICE_REMOVAL) {
+ return -EPIPE;
+ }
+ }
+ break;
case 0: /* Timeout, go around again */
break;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 09/12] migration: implement the shutdown for RDMA QIOChannel
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (7 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 08/12] migration: poll the cm event while wait RDMA work request completion Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 10/12] migration: poll the cm event for destination qemu Lidong Chen
` (4 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert
Cc: qemu-devel, Lidong Chen, Lidong Chen
From: Lidong Chen <jemmy858585@gmail.com>
Because RDMA QIOChannel not implement shutdown function,
If the to_dst_file was set error, the return path thread
will wait forever. and the migration thread will wait
return path thread exit.
the backtrace of return path thread is:
(gdb) bt
#0 0x00007f372a76bb0f in ppoll () from /lib64/libc.so.6
#1 0x000000000071dc24 in qemu_poll_ns (fds=0x7ef7091d0580, nfds=2, timeout=100000000)
at qemu-timer.c:325
#2 0x00000000006b2fba in qemu_rdma_wait_comp_channel (rdma=0xd424000)
at migration/rdma.c:1501
#3 0x00000000006b3191 in qemu_rdma_block_for_wrid (rdma=0xd424000, wrid_requested=4000,
byte_len=0x7ef7091d0640) at migration/rdma.c:1580
#4 0x00000000006b3638 in qemu_rdma_exchange_get_response (rdma=0xd424000,
head=0x7ef7091d0720, expecting=3, idx=0) at migration/rdma.c:1726
#5 0x00000000006b3ad6 in qemu_rdma_exchange_recv (rdma=0xd424000, head=0x7ef7091d0720,
expecting=3) at migration/rdma.c:1903
#6 0x00000000006b5d03 in qemu_rdma_get_buffer (opaque=0x6a57dc0, buf=0x5c80030 "", pos=8,
size=32768) at migration/rdma.c:2714
#7 0x00000000006a9635 in qemu_fill_buffer (f=0x5c80000) at migration/qemu-file.c:232
#8 0x00000000006a9ecd in qemu_peek_byte (f=0x5c80000, offset=0)
at migration/qemu-file.c:502
#9 0x00000000006a9f1f in qemu_get_byte (f=0x5c80000) at migration/qemu-file.c:515
#10 0x00000000006aa162 in qemu_get_be16 (f=0x5c80000) at migration/qemu-file.c:591
#11 0x00000000006a46d3 in source_return_path_thread (
opaque=0xd826a0 <current_migration.37100>) at migration/migration.c:1331
#12 0x00007f372aa49e25 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f372a77635d in clone () from /lib64/libc.so.6
the backtrace of migration thread is:
(gdb) bt
#0 0x00007f372aa4af57 in pthread_join () from /lib64/libpthread.so.0
#1 0x00000000007d5711 in qemu_thread_join (thread=0xd826f8 <current_migration.37100+88>)
at util/qemu-thread-posix.c:504
#2 0x00000000006a4bc5 in await_return_path_close_on_source (
ms=0xd826a0 <current_migration.37100>) at migration/migration.c:1460
#3 0x00000000006a53e4 in migration_completion (s=0xd826a0 <current_migration.37100>,
current_active_state=4, old_vm_running=0x7ef7089cf976, start_time=0x7ef7089cf980)
at migration/migration.c:1695
#4 0x00000000006a5c54 in migration_thread (opaque=0xd826a0 <current_migration.37100>)
at migration/migration.c:1837
#5 0x00007f372aa49e25 in start_thread () from /lib64/libpthread.so.0
#6 0x00007f372a77635d in clone () from /lib64/libc.so.6
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/rdma.c | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/migration/rdma.c b/migration/rdma.c
index 673f126..1affc46 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3039,6 +3039,45 @@ static int qio_channel_rdma_close(QIOChannel *ioc,
return 0;
}
+static int
+qio_channel_rdma_shutdown(QIOChannel *ioc,
+ QIOChannelShutdown how,
+ Error **errp)
+{
+ QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
+ RDMAContext *rdmain, *rdmaout;
+
+ rcu_read_lock();
+
+ rdmain = atomic_rcu_read(&rioc->rdmain);
+ rdmaout = atomic_rcu_read(&rioc->rdmain);
+
+ switch (how) {
+ case QIO_CHANNEL_SHUTDOWN_READ:
+ if (rdmain) {
+ rdmain->error_state = -1;
+ }
+ break;
+ case QIO_CHANNEL_SHUTDOWN_WRITE:
+ if (rdmaout) {
+ rdmaout->error_state = -1;
+ }
+ break;
+ case QIO_CHANNEL_SHUTDOWN_BOTH:
+ default:
+ if (rdmain) {
+ rdmain->error_state = -1;
+ }
+ if (rdmaout) {
+ rdmaout->error_state = -1;
+ }
+ break;
+ }
+
+ rcu_read_unlock();
+ return 0;
+}
+
/*
* Parameters:
* @offset == 0 :
@@ -3865,6 +3904,7 @@ static void qio_channel_rdma_class_init(ObjectClass *klass,
ioc_klass->io_close = qio_channel_rdma_close;
ioc_klass->io_create_watch = qio_channel_rdma_create_watch;
ioc_klass->io_set_aio_fd_handler = qio_channel_rdma_set_aio_fd_handler;
+ ioc_klass->io_shutdown = qio_channel_rdma_shutdown;
}
static const TypeInfo qio_channel_rdma_info = {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 10/12] migration: poll the cm event for destination qemu
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (8 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 09/12] migration: implement the shutdown for RDMA QIOChannel Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 11/12] migration: remove the unnecessary RDMA_CONTROL_ERROR message Lidong Chen
` (3 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert; +Cc: qemu-devel, Lidong Chen
The destination qemu only poll the comp_channel->fd in
qemu_rdma_wait_comp_channel. But when source qemu disconnnect
the rdma connection, the destination qemu should be notified.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
---
migration/migration.c | 3 ++-
migration/rdma.c | 31 ++++++++++++++++++++++++++++++-
2 files changed, 32 insertions(+), 2 deletions(-)
diff --git a/migration/migration.c b/migration/migration.c
index f190964..360ee94 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -389,6 +389,7 @@ static void process_incoming_migration_co(void *opaque)
int ret;
assert(mis->from_src_file);
+ mis->migration_incoming_co = qemu_coroutine_self();
mis->largest_page_size = qemu_ram_pagesize_largest();
postcopy_state_set(POSTCOPY_INCOMING_NONE);
migrate_set_state(&mis->state, MIGRATION_STATUS_NONE,
@@ -418,7 +419,6 @@ static void process_incoming_migration_co(void *opaque)
/* we get COLO info, and know if we are in COLO mode */
if (!ret && migration_incoming_enable_colo()) {
- mis->migration_incoming_co = qemu_coroutine_self();
qemu_thread_create(&mis->colo_incoming_thread, "COLO incoming",
colo_process_incoming_thread, mis, QEMU_THREAD_JOINABLE);
mis->have_colo_incoming_thread = true;
@@ -442,6 +442,7 @@ static void process_incoming_migration_co(void *opaque)
}
mis->bh = qemu_bh_new(process_incoming_migration_bh, mis);
qemu_bh_schedule(mis->bh);
+ mis->migration_incoming_co = NULL;
}
static void migration_incoming_setup(QEMUFile *f)
diff --git a/migration/rdma.c b/migration/rdma.c
index 1affc46..62de2ec 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3226,6 +3226,34 @@ err:
static void rdma_accept_incoming_migration(void *opaque);
+static void rdma_cm_poll_handler(void *opaque)
+{
+ RDMAContext *rdma = opaque;
+ int ret;
+ struct rdma_cm_event *cm_event;
+ MigrationIncomingState *mis = migration_incoming_get_current();
+
+ ret = rdma_get_cm_event(rdma->channel, &cm_event);
+ if (ret) {
+ return;
+ }
+
+ if (cm_event->event == RDMA_CM_EVENT_DISCONNECTED ||
+ cm_event->event == RDMA_CM_EVENT_DEVICE_REMOVAL) {
+ rdma_ack_cm_event(cm_event);
+ error_report("receive cm event, cm event is %d", cm_event->event);
+ rdma->error_state = -EPIPE;
+ if (rdma->return_path) {
+ rdma->return_path->error_state = -EPIPE;
+ }
+
+ if (mis->migration_incoming_co) {
+ qemu_coroutine_enter(mis->migration_incoming_co);
+ }
+ return;
+ }
+}
+
static int qemu_rdma_accept(RDMAContext *rdma)
{
RDMACapabilities cap;
@@ -3326,7 +3354,8 @@ static int qemu_rdma_accept(RDMAContext *rdma)
NULL,
(void *)(intptr_t)rdma->return_path);
} else {
- qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
+ qemu_set_fd_handler(rdma->channel->fd, rdma_cm_poll_handler,
+ NULL, rdma);
}
ret = rdma_accept(rdma->cm_id, &conn_param);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 11/12] migration: remove the unnecessary RDMA_CONTROL_ERROR message
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (9 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 10/12] migration: poll the cm event for destination qemu Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 12/12] migration: create a dedicated thread to release rdma resource Lidong Chen
` (2 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert; +Cc: qemu-devel, Lidong Chen
It's not necessary to send RDMA_CONTROL_ERROR when clean up rdma resource.
If rdma->error_state is ture, the message may not send successfully.
and the cm event can also notify the peer qemu.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
---
migration/rdma.c | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/migration/rdma.c b/migration/rdma.c
index 62de2ec..14cdf82 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2305,17 +2305,6 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
int idx;
if (rdma->cm_id && rdma->connected) {
- if ((rdma->error_state ||
- migrate_get_current()->state == MIGRATION_STATUS_CANCELLING) &&
- !rdma->received_error) {
- RDMAControlHeader head = { .len = 0,
- .type = RDMA_CONTROL_ERROR,
- .repeat = 1,
- };
- error_report("Early error. Sending error.");
- qemu_rdma_post_send_control(rdma, NULL, &head);
- }
-
rdma_disconnect(rdma->cm_id);
trace_qemu_rdma_cleanup_disconnect();
rdma->connected = false;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v6 12/12] migration: create a dedicated thread to release rdma resource
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (10 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 11/12] migration: remove the unnecessary RDMA_CONTROL_ERROR message Lidong Chen
@ 2018-08-03 9:13 ` Lidong Chen
2018-08-03 9:42 ` [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration no-reply
2018-08-06 6:09 ` 858585 jemmy
13 siblings, 0 replies; 16+ messages in thread
From: Lidong Chen @ 2018-08-03 9:13 UTC (permalink / raw)
To: zhang.zhanghailiang, quintela, dgilbert; +Cc: qemu-devel, Lidong Chen
ibv_dereg_mr wait for a long time for big memory size virtual server.
The test result is:
10GB 326ms
20GB 699ms
30GB 1021ms
40GB 1387ms
50GB 1712ms
60GB 2034ms
70GB 2457ms
80GB 2807ms
90GB 3107ms
100GB 3474ms
110GB 3735ms
120GB 4064ms
130GB 4567ms
140GB 4886ms
this will cause the guest os hang for a while when migration finished.
So create a dedicated thread to release rdma resource.
Signed-off-by: Lidong Chen <lidongchen@tencent.com>
---
migration/migration.c | 6 ++++++
migration/migration.h | 3 +++
migration/rdma.c | 47 +++++++++++++++++++++++++++++++----------------
3 files changed, 40 insertions(+), 16 deletions(-)
diff --git a/migration/migration.c b/migration/migration.c
index 360ee94..f58fe45 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1499,6 +1499,7 @@ void migrate_init(MigrationState *s)
s->vm_was_running = false;
s->iteration_initial_bytes = 0;
s->threshold_size = 0;
+ s->rdma_cleanup_thread_quit = true;
}
static GSList *migration_blockers;
@@ -1660,6 +1661,10 @@ static bool migrate_prepare(MigrationState *s, bool blk, bool blk_inc,
return false;
}
+ if (s->rdma_cleanup_thread_quit != true) {
+ return false;
+ }
+
if (runstate_check(RUN_STATE_INMIGRATE)) {
error_setg(errp, "Guest is waiting for an incoming migration");
return false;
@@ -3214,6 +3219,7 @@ static void migration_instance_init(Object *obj)
ms->state = MIGRATION_STATUS_NONE;
ms->mbps = -1;
+ ms->rdma_cleanup_thread_quit = true;
qemu_sem_init(&ms->pause_sem, 0);
qemu_mutex_init(&ms->error_mutex);
diff --git a/migration/migration.h b/migration/migration.h
index a50c2de..4d1be08 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -231,6 +231,9 @@ struct MigrationState
* do not trigger spurious decompression errors.
*/
bool decompress_error_check;
+
+ /* Set this when rdma resource have released */
+ bool rdma_cleanup_thread_quit;
};
void migrate_set_state(int *state, int old_state, int new_state);
diff --git a/migration/rdma.c b/migration/rdma.c
index 14cdf82..3d1a4ad 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2995,35 +2995,50 @@ static void qio_channel_rdma_set_aio_fd_handler(QIOChannel *ioc,
}
}
-static int qio_channel_rdma_close(QIOChannel *ioc,
- Error **errp)
+static void *qio_channel_rdma_close_thread(void *arg)
{
- QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
- RDMAContext *rdmain, *rdmaout;
- trace_qemu_rdma_close();
-
- rdmain = rioc->rdmain;
- if (rdmain) {
- atomic_rcu_set(&rioc->rdmain, NULL);
- }
+ RDMAContext **rdma = arg;
+ RDMAContext *rdmain = rdma[0];
+ RDMAContext *rdmaout = rdma[1];
+ MigrationState *s = migrate_get_current();
- rdmaout = rioc->rdmaout;
- if (rdmaout) {
- atomic_rcu_set(&rioc->rdmaout, NULL);
- }
+ rcu_register_thread();
synchronize_rcu();
-
if (rdmain) {
qemu_rdma_cleanup(rdmain);
}
-
if (rdmaout) {
qemu_rdma_cleanup(rdmaout);
}
g_free(rdmain);
g_free(rdmaout);
+ g_free(rdma);
+
+ rcu_unregister_thread();
+ s->rdma_cleanup_thread_quit = true;
+ return NULL;
+}
+
+static int qio_channel_rdma_close(QIOChannel *ioc,
+ Error **errp)
+{
+ QemuThread t;
+ QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
+ RDMAContext **rdma = g_new0(RDMAContext*, 2);
+ MigrationState *s = migrate_get_current();
+
+ trace_qemu_rdma_close();
+ if (rioc->rdmain || rioc->rdmaout) {
+ rdma[0] = rioc->rdmain;
+ rdma[1] = rioc->rdmaout;
+ atomic_rcu_set(&rioc->rdmain, NULL);
+ atomic_rcu_set(&rioc->rdmaout, NULL);
+ s->rdma_cleanup_thread_quit = false;
+ qemu_thread_create(&t, "rdma cleanup", qio_channel_rdma_close_thread,
+ rdma, QEMU_THREAD_DETACHED);
+ }
return 0;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (11 preceding siblings ...)
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 12/12] migration: create a dedicated thread to release rdma resource Lidong Chen
@ 2018-08-03 9:42 ` no-reply
2018-08-06 6:09 ` 858585 jemmy
13 siblings, 0 replies; 16+ messages in thread
From: no-reply @ 2018-08-03 9:42 UTC (permalink / raw)
To: jemmy858585
Cc: famz, zhang.zhanghailiang, quintela, dgilbert, qemu-devel, lidongchen
Hi,
This series failed docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 1533287630-4221-1-git-send-email-lidongchen@tencent.com
Subject: [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration
=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-quick@centos7 SHOW_ENV=1 J=8
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
1eac4b0550 migration: create a dedicated thread to release rdma resource
0574cc5827 migration: remove the unnecessary RDMA_CONTROL_ERROR message
789cec1658 migration: poll the cm event for destination qemu
9e9de9d958 migration: implement the shutdown for RDMA QIOChannel
cb67fd3251 migration: poll the cm event while wait RDMA work request completion
74ce3d3434 migration: invoke qio_channel_yield only when qemu_in_coroutine()
0e22153ef5 migration: implement io_set_aio_fd_handler function for RDMA QIOChannel
cece3fcc4a migration: Stop rdma yielding during incoming postcopy
4bba0c6abd migration: implement bi-directional RDMA QIOChannel
f2535777ff migration: avoid concurrent invoke channel_close by different threads
ad1ab57f58 migration: create a dedicated connection for rdma return path
1e6823b350 migration: disable RDMA WRITE after postcopy started
=== OUTPUT BEGIN ===
BUILD centos7
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-jlamec93/src'
GEN /var/tmp/patchew-tester-tmp-jlamec93/src/docker-src.2018-08-03-05.40.22.11421/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-jlamec93/src/docker-src.2018-08-03-05.40.22.11421/qemu.tar.vroot'...
done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-jlamec93/src/docker-src.2018-08-03-05.40.22.11421/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-jlamec93/src/docker-src.2018-08-03-05.40.22.11421/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
COPY RUNNER
RUN test-quick in qemu:centos7
Packages installed:
SDL-devel-1.2.15-14.el7.x86_64
bison-3.0.4-1.el7.x86_64
bzip2-devel-1.0.6-13.el7.x86_64
ccache-3.3.4-1.el7.x86_64
csnappy-devel-0-6.20150729gitd7bc683.el7.x86_64
flex-2.5.37-3.el7.x86_64
gcc-4.8.5-16.el7_4.2.x86_64
gettext-0.19.8.1-2.el7.x86_64
git-1.8.3.1-12.el7_4.x86_64
glib2-devel-2.50.3-3.el7.x86_64
libepoxy-devel-1.3.1-1.el7.x86_64
libfdt-devel-1.4.6-1.el7.x86_64
lzo-devel-2.06-8.el7.x86_64
make-3.82-23.el7.x86_64
mesa-libEGL-devel-17.0.1-6.20170307.el7.x86_64
mesa-libgbm-devel-17.0.1-6.20170307.el7.x86_64
package g++ is not installed
package librdmacm-devel is not installed
pixman-devel-0.34.0-1.el7.x86_64
spice-glib-devel-0.33-6.el7_4.1.x86_64
spice-server-devel-0.12.8-2.el7.1.x86_64
tar-1.26-32.el7.x86_64
vte-devel-0.28.2-10.el7.x86_64
xen-devel-4.6.6-10.el7.x86_64
zlib-devel-1.2.7-17.el7.x86_64
Environment variables:
PACKAGES=bison bzip2-devel ccache csnappy-devel flex g++ gcc gettext git glib2-devel libepoxy-devel libfdt-devel librdmacm-devel lzo-devel make mesa-libEGL-devel mesa-libgbm-devel pixman-devel SDL-devel spice-glib-devel spice-server-devel tar vte-devel xen-devel zlib-devel
HOSTNAME=279a98be3851
MAKEFLAGS= -j8
J=8
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
TARGET_LIST=
SHLVL=1
HOME=/
TEST_DIR=/tmp/qemu-test
FEATURES= dtc
DEBUG=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install
No C++ compiler available; disabling C++ specific optional code
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install/share/qemu
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install/bin
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib/qemu
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install/etc
local state directory /tmp/qemu-test/install/var
Manual directory /tmp/qemu-test/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler cc
Host C compiler cc
C++ compiler
Objective-C compiler cc
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/include/pixman-1 -Werror -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -Wno-missing-braces -I/usr/include/libpng15 -I/usr/include/spice-server -I/usr/include/cacard -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/nss3 -I/usr/include/nspr4 -I/usr/include/spice-1
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
QEMU_LDFLAGS
make make
install install
python python -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.15)
GTK support yes (2.24.31)
GTK GL support no
VTE support yes (0.28.2)
TLS priority NORMAL
GNUTLS support no
GNUTLS rnd no
libgcrypt no
libgcrypt kdf no
nettle no
nettle kdf no
libtasn1 no
curses support yes
virgl support no
curl support no
mingw32 support no
Audio drivers oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support yes
xen support yes
xen ctrl version 40600
pv dom build no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support yes
Install blobs yes
KVM support yes
HAX support no
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support yes
RDMA support yes
fdt support system
membarrier no
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
posix_memalign yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends log
spice support yes (0.12.12/0.12.8)
rbd support no
xfsctl support no
smartcard support yes
libusb no
usb net redir no
OpenGL support yes
OpenGL dmabufs yes
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
mutex debugging no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough yes
TPM emulator yes
QOM debugging yes
Live block migration yes
lzo support yes
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
docker no
WARNING: Use of GTK 2.0 is deprecated and will be removed in
WARNING: future releases. Please switch to using GTK 3.0
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
NOTE: cross-compilers enabled: 'cc'
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN qemu-options.def
GEN qapi-gen
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.c
GEN trace/generated-helpers.h
GEN module_block.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN ui/input-keymap-osx-to-qcode.c
GEN trace-root.h
GEN tests/test-qapi-gen
GEN accel/kvm/trace.h
GEN accel/tcg/trace.h
GEN audio/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN crypto/trace.h
GEN hw/9pfs/trace.h
GEN hw/acpi/trace.h
GEN hw/alpha/trace.h
GEN hw/arm/trace.h
GEN hw/audio/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/display/trace.h
GEN hw/dma/trace.h
GEN hw/hppa/trace.h
GEN hw/i2c/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/ide/trace.h
GEN hw/input/trace.h
GEN hw/intc/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/net/trace.h
GEN hw/nvram/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/ppc/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/s390x/trace.h
GEN hw/scsi/trace.h
GEN hw/sd/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/timer/trace.h
GEN hw/tpm/trace.h
GEN hw/usb/trace.h
GEN hw/vfio/trace.h
GEN hw/virtio/trace.h
GEN hw/xen/trace.h
GEN io/trace.h
GEN linux-user/trace.h
GEN migration/trace.h
GEN nbd/trace.h
GEN net/trace.h
GEN qapi/trace.h
GEN qom/trace.h
GEN scsi/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/ppc/trace.h
GEN target/s390x/trace.h
GEN target/sparc/trace.h
GEN ui/trace.h
GEN util/trace.h
GEN trace-root.c
GEN accel/kvm/trace.c
GEN accel/tcg/trace.c
GEN audio/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN crypto/trace.c
GEN hw/9pfs/trace.c
GEN hw/acpi/trace.c
GEN hw/alpha/trace.c
GEN hw/arm/trace.c
GEN hw/audio/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/display/trace.c
GEN hw/dma/trace.c
GEN hw/hppa/trace.c
GEN hw/i2c/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/ide/trace.c
GEN hw/input/trace.c
GEN hw/intc/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/net/trace.c
GEN hw/nvram/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/ppc/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/s390x/trace.c
GEN hw/scsi/trace.c
GEN hw/sd/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/timer/trace.c
GEN hw/tpm/trace.c
GEN hw/usb/trace.c
GEN hw/vfio/trace.c
GEN hw/virtio/trace.c
GEN hw/xen/trace.c
GEN io/trace.c
GEN linux-user/trace.c
GEN migration/trace.c
GEN nbd/trace.c
GEN net/trace.c
GEN qapi/trace.c
GEN qom/trace.c
GEN scsi/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/ppc/trace.c
GEN target/s390x/trace.c
GEN target/sparc/trace.c
GEN ui/trace.c
GEN util/trace.c
GEN config-all-devices.mak
CC tests/qemu-iotests/socket_scm_helper.o
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-types.o
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types-block-core.o
CC qapi/qapi-types-block.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-job.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-job.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-job.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-introspect.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qbool.o
CC qobject/qlit.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC qobject/block-qdict.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/lockcnt.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/qemu-timer.o
CC util/main-loop.o
CC util/iohandler.o
CC util/aio-posix.o
CC util/compatfd.o
CC util/event_notifier-posix.o
CC util/mmap-alloc.o
CC util/oslib-posix.o
CC util/qemu-openpty.o
CC util/qemu-thread-posix.o
CC util/memfd.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/acl.o
CC util/fifo8.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-ucontext.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/qdist.o
CC util/pagesize.o
CC util/qht.o
CC util/range.o
CC util/stats64.o
CC util/systemd.o
CC util/iova-tree.o
CC util/vfio-helpers.o
CC trace-root.o
CC accel/kvm/trace.o
CC accel/tcg/trace.o
CC audio/trace.o
CC block/trace.o
CC chardev/trace.o
CC crypto/trace.o
CC hw/9pfs/trace.o
CC hw/acpi/trace.o
CC hw/alpha/trace.o
CC hw/arm/trace.o
CC hw/audio/trace.o
CC hw/block/trace.o
CC hw/block/dataplane/trace.o
CC hw/char/trace.o
CC hw/display/trace.o
CC hw/dma/trace.o
CC hw/hppa/trace.o
CC hw/i2c/trace.o
CC hw/i386/trace.o
CC hw/i386/xen/trace.o
CC hw/ide/trace.o
CC hw/input/trace.o
CC hw/intc/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/net/trace.o
CC hw/nvram/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/ppc/trace.o
CC hw/rdma/trace.o
CC hw/rdma/vmw/trace.o
CC hw/s390x/trace.o
CC hw/scsi/trace.o
CC hw/sd/trace.o
CC hw/sparc/trace.o
CC hw/sparc64/trace.o
CC hw/timer/trace.o
CC hw/tpm/trace.o
CC hw/usb/trace.o
CC hw/vfio/trace.o
CC hw/virtio/trace.o
CC hw/xen/trace.o
CC io/trace.o
CC linux-user/trace.o
CC migration/trace.o
CC nbd/trace.o
CC net/trace.o
CC qapi/trace.o
CC qom/trace.o
CC scsi/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/mips/trace.o
CC target/ppc/trace.o
CC target/s390x/trace.o
CC target/sparc/trace.o
CC ui/trace.o
CC util/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/is-daemonized.o
CC stubs/iothread-lock.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/runstate-check.o
CC stubs/replay.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/vm-stop.o
CC stubs/uuid.o
CC stubs/vmstate.o
CC stubs/qmp_memory_device.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vmgenid.o
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
CC contrib/ivshmem-client/ivshmem-client.o
CC contrib/ivshmem-client/main.o
CC contrib/ivshmem-server/ivshmem-server.o
CC contrib/ivshmem-server/main.o
CC qemu-nbd.o
CC block.o
CC blockjob.o
CC job.o
CC qemu-io-cmds.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/quorum.o
CC block/vhdx-log.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/blklogwrites.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-posix.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/create.o
CC block/throttle-groups.o
CC block/nvme.o
CC block/nbd.o
CC block/nbd-client.o
CC block/sheepdog.o
CC block/accounting.o
CC block/dirty-bitmap.o
CC block/write-threshold.o
CC block/backup.o
CC block/replication.o
CC block/throttle.o
CC block/copy-on-read.o
CC block/crypto.o
CC nbd/server.o
CC nbd/client.o
CC nbd/common.o
CC scsi/utils.o
CC scsi/pr-manager.o
CC scsi/pr-manager-helper.o
CC block/dmg-bz2.o
CC crypto/init.o
CC crypto/hash.o
CC crypto/hash-glib.o
CC crypto/hmac.o
CC crypto/hmac-glib.o
CC crypto/aes.o
CC crypto/desrfb.o
CC crypto/cipher.o
CC crypto/tlscreds.o
CC crypto/tlscredsanon.o
CC crypto/tlscredspsk.o
CC crypto/tlscredsx509.o
CC crypto/tlssession.o
CC crypto/secret.o
CC crypto/random-platform.o
CC crypto/pbkdf.o
CC crypto/ivgen.o
CC crypto/ivgen-essiv.o
CC crypto/ivgen-plain.o
CC crypto/ivgen-plain64.o
CC crypto/afsplit.o
CC crypto/xts.o
CC crypto/block.o
CC crypto/block-qcow.o
CC crypto/block-luks.o
CC io/channel.o
CC io/channel-command.o
CC io/channel-buffer.o
CC io/channel-file.o
CC io/channel-socket.o
CC io/channel-tls.o
CC io/channel-watch.o
CC io/channel-websock.o
CC io/channel-util.o
CC io/dns-resolver.o
CC io/net-listener.o
CC io/task.o
CC qom/object.o
CC qom/container.o
CC qom/qom-qobject.o
GEN qemu-img-cmds.h
CC qom/object_interfaces.o
CC qemu-io.o
CC scsi/qemu-pr-helper.o
CC qemu-bridge-helper.o
CC blockdev.o
CC blockdev-nbd.o
CC bootdevice.o
CC iothread.o
CC job-qmp.o
CC qdev-monitor.o
CC device-hotplug.o
CC os-posix.o
CC bt-host.o
CC bt-vhci.o
CC dma-helpers.o
CC vl.o
CC tpm.o
CC device_tree.o
CC qapi/qapi-commands.o
CC qapi/qapi-commands-block-core.o
CC qapi/qapi-commands-block.o
CC qapi/qapi-commands-char.o
CC qapi/qapi-commands-common.o
CC qapi/qapi-commands-crypto.o
CC qapi/qapi-commands-introspect.o
CC qapi/qapi-commands-job.o
CC qapi/qapi-commands-migration.o
CC qapi/qapi-commands-misc.o
CC qapi/qapi-commands-net.o
CC qapi/qapi-commands-rocker.o
CC qapi/qapi-commands-run-state.o
CC qapi/qapi-commands-sockets.o
CC qapi/qapi-commands-tpm.o
CC qapi/qapi-commands-trace.o
CC qapi/qapi-commands-transaction.o
CC qapi/qapi-commands-ui.o
CC qmp.o
CC hmp.o
CC cpus-common.o
CC audio/audio.o
CC audio/noaudio.o
CC audio/wavaudio.o
CC audio/mixeng.o
CC audio/spiceaudio.o
CC audio/wavcapture.o
CC backends/rng.o
CC backends/rng-egd.o
CC backends/rng-random.o
CC backends/tpm.o
CC backends/hostmem.o
CC backends/hostmem-ram.o
CC backends/hostmem-file.o
CC backends/cryptodev.o
CC backends/cryptodev-builtin.o
CC backends/cryptodev-vhost.o
CC backends/cryptodev-vhost-user.o
CC backends/hostmem-memfd.o
CC block/stream.o
CC chardev/msmouse.o
CC chardev/wctablet.o
CC chardev/testdev.o
CC chardev/spice.o
CC disas/arm.o
CC disas/i386.o
CC fsdev/qemu-fsdev-dummy.o
CC fsdev/qemu-fsdev-opts.o
CC fsdev/qemu-fsdev-throttle.o
CC hw/acpi/core.o
CC hw/acpi/piix4.o
CC hw/acpi/ich9.o
CC hw/acpi/pcihp.o
CC hw/acpi/cpu_hotplug.o
CC hw/acpi/tco.o
CC hw/acpi/memory_hotplug.o
CC hw/acpi/cpu.o
CC hw/acpi/nvdimm.o
CC hw/acpi/vmgenid.o
CC hw/acpi/acpi_interface.o
CC hw/acpi/bios-linker-loader.o
CC hw/acpi/aml-build.o
CC hw/acpi/ipmi.o
CC hw/acpi/acpi-stub.o
CC hw/acpi/ipmi-stub.o
CC hw/audio/sb16.o
CC hw/audio/es1370.o
CC hw/audio/ac97.o
CC hw/audio/fmopl.o
CC hw/audio/adlib.o
CC hw/audio/gus.o
CC hw/audio/gusemu_hal.o
CC hw/audio/gusemu_mixer.o
CC hw/audio/cs4231a.o
CC hw/audio/intel-hda.o
CC hw/audio/hda-codec.o
CC hw/audio/pcspk.o
CC hw/audio/pl041.o
CC hw/audio/wm8750.o
CC hw/audio/lm4549.o
CC hw/audio/marvell_88w8618.o
CC hw/audio/soundhw.o
CC hw/block/block.o
CC hw/block/cdrom.o
CC hw/block/hd-geometry.o
CC hw/block/fdc.o
CC hw/block/m25p80.o
CC hw/block/nand.o
CC hw/block/pflash_cfi01.o
CC hw/block/pflash_cfi02.o
CC hw/block/xen_disk.o
CC hw/block/ecc.o
CC hw/block/onenand.o
CC hw/block/nvme.o
CC hw/bt/core.o
CC hw/bt/l2cap.o
CC hw/bt/sdp.o
CC hw/bt/hci.o
CC hw/bt/hid.o
CC hw/bt/hci-csr.o
CC hw/char/ipoctal232.o
CC hw/char/parallel.o
CC hw/char/parallel-isa.o
CC hw/char/pl011.o
CC hw/char/serial.o
CC hw/char/serial-isa.o
CC hw/char/serial-pci.o
CC hw/char/virtio-console.o
CC hw/char/xen_console.o
CC hw/char/cadence_uart.o
CC hw/char/cmsdk-apb-uart.o
CC hw/char/debugcon.o
CC hw/char/imx_serial.o
CC hw/core/qdev.o
CC hw/core/qdev-properties.o
CC hw/core/bus.o
CC hw/core/reset.o
CC hw/core/qdev-fw.o
CC hw/core/fw-path-provider.o
CC hw/core/irq.o
CC hw/core/hotplug.o
CC hw/core/nmi.o
CC hw/core/stream.o
CC hw/core/ptimer.o
CC hw/core/sysbus.o
CC hw/core/loader.o
CC hw/core/machine.o
CC hw/core/qdev-properties-system.o
CC hw/core/register.o
CC hw/core/or-irq.o
CC hw/core/split-irq.o
CC hw/core/platform-bus.o
CC hw/cpu/core.o
CC hw/display/ramfb.o
CC hw/display/ramfb-standalone.o
CC hw/display/ads7846.o
CC hw/display/cirrus_vga.o
CC hw/display/pl110.o
CC hw/display/sii9022.o
CC hw/display/ssd0303.o
CC hw/display/ssd0323.o
CC hw/display/xenfb.o
CC hw/display/vga-pci.o
CC hw/display/bochs-display.o
CC hw/display/vga-isa.o
CC hw/display/vmware_vga.o
CC hw/display/blizzard.o
CC hw/display/exynos4210_fimd.o
CC hw/display/framebuffer.o
CC hw/display/tc6393xb.o
CC hw/display/qxl.o
CC hw/display/qxl-logger.o
CC hw/display/qxl-render.o
CC hw/dma/pl080.o
CC hw/dma/pl330.o
CC hw/dma/i8257.o
CC hw/dma/xilinx_axidma.o
CC hw/dma/xlnx-zynq-devcfg.o
CC hw/dma/xlnx-zdma.o
CC hw/gpio/max7310.o
CC hw/gpio/pl061.o
CC hw/gpio/gpio_key.o
CC hw/gpio/zaurus.o
CC hw/i2c/core.o
CC hw/i2c/smbus.o
CC hw/i2c/smbus_eeprom.o
CC hw/i2c/i2c-ddc.o
CC hw/i2c/smbus_ich9.o
CC hw/i2c/versatile_i2c.o
CC hw/i2c/pm_smbus.o
CC hw/i2c/bitbang_i2c.o
CC hw/i2c/exynos4210_i2c.o
CC hw/i2c/imx_i2c.o
CC hw/i2c/aspeed_i2c.o
CC hw/ide/core.o
CC hw/ide/atapi.o
CC hw/ide/qdev.o
CC hw/ide/pci.o
CC hw/ide/isa.o
CC hw/ide/piix.o
CC hw/ide/microdrive.o
CC hw/ide/ahci.o
CC hw/ide/ich.o
CC hw/ide/ahci-allwinner.o
CC hw/input/hid.o
CC hw/input/pckbd.o
CC hw/input/lm832x.o
CC hw/input/pl050.o
CC hw/input/ps2.o
CC hw/input/stellaris_input.o
CC hw/input/tsc2005.o
CC hw/input/virtio-input.o
CC hw/input/virtio-input-hid.o
CC hw/input/virtio-input-host.o
CC hw/intc/i8259_common.o
CC hw/intc/i8259.o
CC hw/intc/pl190.o
CC hw/intc/xlnx-pmu-iomod-intc.o
CC hw/intc/xlnx-zynqmp-ipi.o
CC hw/intc/imx_avic.o
CC hw/intc/imx_gpcv2.o
CC hw/intc/realview_gic.o
CC hw/intc/ioapic_common.o
CC hw/intc/arm_gic_common.o
CC hw/intc/arm_gic.o
CC hw/intc/arm_gicv2m.o
CC hw/intc/arm_gicv3_common.o
CC hw/intc/arm_gicv3_dist.o
CC hw/intc/arm_gicv3.o
CC hw/intc/arm_gicv3_its_common.o
CC hw/intc/arm_gicv3_redist.o
CC hw/intc/intc.o
CC hw/ipack/ipack.o
CC hw/ipack/tpci200.o
CC hw/ipmi/ipmi.o
CC hw/ipmi/ipmi_bmc_sim.o
CC hw/ipmi/ipmi_bmc_extern.o
CC hw/ipmi/isa_ipmi_kcs.o
CC hw/ipmi/isa_ipmi_bt.o
CC hw/isa/isa-bus.o
CC hw/isa/isa-superio.o
CC hw/isa/smc37c669-superio.o
CC hw/isa/apm.o
CC hw/mem/pc-dimm.o
CC hw/mem/memory-device.o
CC hw/mem/nvdimm.o
CC hw/misc/applesmc.o
CC hw/misc/max111x.o
CC hw/misc/tmp105.o
CC hw/misc/tmp421.o
CC hw/misc/debugexit.o
CC hw/misc/sga.o
CC hw/misc/pc-testdev.o
CC hw/misc/pci-testdev.o
CC hw/misc/edu.o
CC hw/misc/pca9552.o
CC hw/misc/unimp.o
CC hw/misc/vmcoreinfo.o
CC hw/misc/arm_l2x0.o
CC hw/misc/arm_integrator_debug.o
CC hw/misc/a9scu.o
CC hw/misc/arm11scu.o
CC hw/net/xen_nic.o
CC hw/net/ne2000.o
CC hw/net/eepro100.o
CC hw/net/pcnet-pci.o
CC hw/net/pcnet.o
CC hw/net/e1000.o
CC hw/net/e1000x_common.o
CC hw/net/net_tx_pkt.o
CC hw/net/net_rx_pkt.o
CC hw/net/e1000e.o
CC hw/net/e1000e_core.o
CC hw/net/rtl8139.o
CC hw/net/vmxnet3.o
CC hw/net/smc91c111.o
CC hw/net/lan9118.o
CC hw/net/ne2000-isa.o
CC hw/net/xgmac.o
CC hw/net/xilinx_axienet.o
CC hw/net/allwinner_emac.o
CC hw/net/imx_fec.o
CC hw/net/cadence_gem.o
CC hw/net/stellaris_enet.o
CC hw/net/ftgmac100.o
CC hw/net/rocker/rocker.o
CC hw/net/rocker/rocker_fp.o
CC hw/net/rocker/rocker_desc.o
CC hw/net/rocker/rocker_world.o
CC hw/net/rocker/rocker_of_dpa.o
CC hw/net/can/can_sja1000.o
CC hw/net/can/can_kvaser_pci.o
CC hw/net/can/can_pcm3680_pci.o
CC hw/net/can/can_mioe3680_pci.o
CC hw/nvram/eeprom93xx.o
CC hw/nvram/eeprom_at24c.o
CC hw/nvram/fw_cfg.o
CC hw/nvram/chrp_nvram.o
CC hw/pci-bridge/pci_bridge_dev.o
CC hw/pci-bridge/pcie_root_port.o
CC hw/pci-bridge/gen_pcie_root_port.o
CC hw/pci-bridge/pcie_pci_bridge.o
CC hw/pci-bridge/pci_expander_bridge.o
CC hw/pci-bridge/xio3130_upstream.o
CC hw/pci-bridge/xio3130_downstream.o
CC hw/pci-bridge/ioh3420.o
CC hw/pci-bridge/i82801b11.o
CC hw/pci-host/pam.o
CC hw/pci-host/versatile.o
CC hw/pci-host/piix.o
CC hw/pci-host/q35.o
CC hw/pci-host/gpex.o
CC hw/pci-host/designware.o
CC hw/pci/pci.o
CC hw/pci/pci_bridge.o
CC hw/pci/msix.o
CC hw/pci/msi.o
CC hw/pci/shpc.o
CC hw/pci/slotid_cap.o
CC hw/pci/pci_host.o
CC hw/pci/pcie_host.o
CC hw/pci/pcie.o
CC hw/pci/pcie_aer.o
CC hw/pci/pcie_port.o
CC hw/pci/pci-stub.o
CC hw/pcmcia/pcmcia.o
CC hw/scsi/scsi-disk.o
CC hw/scsi/scsi-generic.o
CC hw/scsi/scsi-bus.o
CC hw/scsi/lsi53c895a.o
CC hw/scsi/mptsas.o
CC hw/scsi/mptconfig.o
CC hw/scsi/mptendian.o
CC hw/scsi/megasas.o
CC hw/scsi/vmw_pvscsi.o
CC hw/scsi/esp.o
CC hw/scsi/esp-pci.o
CC hw/sd/pl181.o
CC hw/sd/ssi-sd.o
CC hw/sd/sd.o
CC hw/sd/core.o
CC hw/sd/sdmmc-internal.o
CC hw/sd/sdhci.o
CC hw/smbios/smbios.o
CC hw/smbios/smbios_type_38.o
CC hw/smbios/smbios-stub.o
CC hw/smbios/smbios_type_38-stub.o
CC hw/ssi/pl022.o
CC hw/ssi/ssi.o
CC hw/ssi/xilinx_spips.o
CC hw/ssi/aspeed_smc.o
CC hw/ssi/stm32f2xx_spi.o
CC hw/ssi/mss-spi.o
CC hw/timer/arm_timer.o
CC hw/timer/arm_mptimer.o
CC hw/timer/armv7m_systick.o
CC hw/timer/a9gtimer.o
CC hw/timer/cadence_ttc.o
CC hw/timer/ds1338.o
CC hw/timer/hpet.o
CC hw/timer/i8254_common.o
CC hw/timer/i8254.o
CC hw/timer/pl031.o
CC hw/timer/twl92230.o
CC hw/timer/imx_epit.o
CC hw/timer/imx_gpt.o
CC hw/timer/xlnx-zynqmp-rtc.o
CC hw/timer/stm32f2xx_timer.o
CC hw/timer/aspeed_timer.o
CC hw/timer/cmsdk-apb-timer.o
CC hw/timer/mss-timer.o
CC hw/tpm/tpm_util.o
CC hw/tpm/tpm_tis.o
CC hw/tpm/tpm_crb.o
CC hw/tpm/tpm_passthrough.o
CC hw/tpm/tpm_emulator.o
CC hw/usb/core.o
CC hw/usb/combined-packet.o
CC hw/usb/bus.o
CC hw/usb/libhw.o
CC hw/usb/desc.o
CC hw/usb/desc-msos.o
CC hw/usb/hcd-uhci.o
CC hw/usb/hcd-ohci.o
CC hw/usb/hcd-ehci.o
CC hw/usb/hcd-ehci-pci.o
CC hw/usb/hcd-ehci-sysbus.o
CC hw/usb/hcd-xhci.o
CC hw/usb/hcd-xhci-nec.o
CC hw/usb/hcd-musb.o
CC hw/usb/dev-hub.o
CC hw/usb/dev-hid.o
CC hw/usb/dev-wacom.o
CC hw/usb/dev-storage.o
CC hw/usb/dev-uas.o
CC hw/usb/dev-audio.o
CC hw/usb/dev-serial.o
CC hw/usb/dev-network.o
CC hw/usb/dev-bluetooth.o
CC hw/usb/dev-smartcard-reader.o
CC hw/usb/ccid-card-passthru.o
CC hw/usb/ccid-card-emulated.o
CC hw/usb/dev-mtp.o
CC hw/usb/host-stub.o
CC hw/virtio/virtio-bus.o
CC hw/virtio/virtio-rng.o
CC hw/virtio/virtio-pci.o
CC hw/virtio/virtio-mmio.o
CC hw/virtio/vhost-stub.o
CC hw/watchdog/watchdog.o
CC hw/watchdog/wdt_i6300esb.o
CC hw/watchdog/wdt_ib700.o
CC hw/watchdog/wdt_aspeed.o
CC hw/xen/xen_backend.o
CC hw/xen/xen_devconfig.o
CC hw/xen/xen_pvdev.o
CC hw/xen/xen-common.o
CC migration/migration.o
CC migration/socket.o
CC migration/fd.o
CC migration/exec.o
CC migration/tls.o
CC migration/channel.o
CC migration/savevm.o
CC migration/colo-comm.o
CC migration/colo.o
CC migration/colo-failover.o
CC migration/vmstate.o
CC migration/vmstate-types.o
CC migration/page_cache.o
CC migration/qemu-file.o
CC migration/global_state.o
CC migration/qemu-file-channel.o
CC migration/xbzrle.o
CC migration/postcopy-ram.o
CC migration/qjson.o
CC migration/block-dirty-bitmap.o
CC migration/rdma.o
CC migration/block.o
CC net/net.o
CC net/queue.o
CC net/checksum.o
CC net/util.o
CC net/hub.o
CC net/socket.o
CC net/dump.o
CC net/eth.o
CC net/l2tpv3.o
CC net/vhost-user.o
CC net/slirp.o
CC net/filter.o
CC net/filter-buffer.o
CC net/filter-mirror.o
CC net/colo-compare.o
CC net/colo.o
CC net/filter-rewriter.o
CC net/filter-replay.o
CC net/tap.o
CC net/tap-linux.o
CC net/can/can_core.o
CC net/can/can_host.o
CC net/can/can_socketcan.o
CC qom/cpu.o
CC replay/replay.o
CC replay/replay-events.o
CC replay/replay-internal.o
CC replay/replay-time.o
CC replay/replay-input.o
CC replay/replay-char.o
CC replay/replay-snapshot.o
CC replay/replay-net.o
CC replay/replay-audio.o
CC slirp/cksum.o
CC slirp/if.o
CC slirp/ip_icmp.o
CC slirp/ip6_output.o
CC slirp/ip6_icmp.o
CC slirp/ip6_input.o
CC slirp/ip_input.o
CC slirp/ip_output.o
CC slirp/dnssearch.o
CC slirp/dhcpv6.o
CC slirp/slirp.o
CC slirp/mbuf.o
CC slirp/sbuf.o
CC slirp/misc.o
CC slirp/socket.o
CC slirp/tcp_input.o
CC slirp/tcp_output.o
CC slirp/tcp_subr.o
CC slirp/tcp_timer.o
CC slirp/udp.o
CC slirp/udp6.o
CC slirp/bootp.o
CC slirp/tftp.o
CC slirp/arp_table.o
CC slirp/ndp_table.o
CC slirp/ncsi.o
CC ui/keymaps.o
CC ui/console.o
CC ui/cursor.o
CC ui/qemu-pixman.o
CC ui/input.o
CC ui/input-keymap.o
CC ui/input-legacy.o
CC ui/input-linux.o
CC ui/spice-core.o
CC ui/spice-input.o
CC ui/spice-display.o
CC ui/vnc.o
CC ui/vnc-enc-zlib.o
CC ui/vnc-enc-hextile.o
CC ui/vnc-enc-tight.o
CC ui/vnc-palette.o
CC ui/vnc-enc-zrle.o
CC ui/vnc-auth-vencrypt.o
CC ui/vnc-ws.o
CC ui/vnc-jobs.o
VERT ui/shader/texture-blit-vert.h
VERT ui/shader/texture-blit-flip-vert.h
FRAG ui/shader/texture-blit-frag.h
CC ui/console-gl.o
CC ui/egl-helpers.o
CC ui/egl-context.o
CC ui/egl-headless.o
CC audio/ossaudio.o
CC ui/sdl.o
CC ui/sdl_zoom.o
CC ui/x_keymap.o
CC ui/gtk.o
CC ui/gtk-egl.o
CC ui/curses.o
CC chardev/char.o
CC chardev/char-fd.o
CC chardev/char-fe.o
CC chardev/char-file.o
CC chardev/char-io.o
CC chardev/char-mux.o
CC chardev/char-null.o
CC chardev/char-parallel.o
CC chardev/char-pipe.o
CC chardev/char-pty.o
CC chardev/char-ringbuf.o
CC chardev/char-serial.o
CC chardev/char-socket.o
CC chardev/char-stdio.o
CC chardev/char-udp.o
LINK tests/qemu-iotests/socket_scm_helper
CC qga/commands.o
CC qga/guest-agent-command-state.o
CC qga/main.o
CC qga/channel-posix.o
CC qga/commands-posix.o
CC qga/qapi-generated/qga-qapi-types.o
AS optionrom/multiboot.o
AS optionrom/linuxboot.o
CC optionrom/linuxboot_dma.o
CC qga/qapi-generated/qga-qapi-visit.o
CC qga/qapi-generated/qga-qapi-commands.o
AR libqemuutil.a
CC qemu-img.o
AS optionrom/kvmvapic.o
BUILD optionrom/multiboot.img
BUILD optionrom/linuxboot.img
CC ui/shader.o
BUILD optionrom/linuxboot_dma.img
BUILD optionrom/kvmvapic.img
BUILD optionrom/linuxboot.raw
BUILD optionrom/multiboot.raw
BUILD optionrom/linuxboot_dma.raw
BUILD optionrom/kvmvapic.raw
SIGN optionrom/multiboot.bin
SIGN optionrom/linuxboot.bin
SIGN optionrom/linuxboot_dma.bin
SIGN optionrom/kvmvapic.bin
LINK qemu-ga
LINK ivshmem-client
LINK ivshmem-server
LINK qemu-nbd
LINK qemu-img
LINK qemu-io
LINK scsi/qemu-pr-helper
LINK qemu-bridge-helper
GEN x86_64-softmmu/hmp-commands.h
GEN x86_64-softmmu/hmp-commands-info.h
GEN x86_64-softmmu/config-target.h
GEN aarch64-softmmu/hmp-commands.h
GEN aarch64-softmmu/hmp-commands-info.h
GEN aarch64-softmmu/config-target.h
CC x86_64-softmmu/exec.o
CC x86_64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/tcg/tcg.o
CC x86_64-softmmu/tcg/tcg-op-vec.o
CC x86_64-softmmu/tcg/tcg-op-gvec.o
CC x86_64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/tcg/optimize.o
CC aarch64-softmmu/exec.o
CC x86_64-softmmu/fpu/softfloat.o
CC x86_64-softmmu/disas.o
GEN x86_64-softmmu/gdbstub-xml.c
CC x86_64-softmmu/arch_init.o
CC x86_64-softmmu/cpus.o
CC x86_64-softmmu/monitor.o
CC x86_64-softmmu/gdbstub.o
CC aarch64-softmmu/tcg/tcg.o
CC x86_64-softmmu/balloon.o
CC x86_64-softmmu/ioport.o
CC x86_64-softmmu/numa.o
CC x86_64-softmmu/qtest.o
CC x86_64-softmmu/memory.o
CC aarch64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/memory_mapping.o
CC x86_64-softmmu/dump.o
CC x86_64-softmmu/win_dump.o
CC aarch64-softmmu/tcg/tcg-op-vec.o
CC x86_64-softmmu/migration/ram.o
CC x86_64-softmmu/accel/accel.o
CC aarch64-softmmu/tcg/tcg-op-gvec.o
CC x86_64-softmmu/accel/kvm/kvm-all.o
CC aarch64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/accel/stubs/hax-stub.o
CC x86_64-softmmu/accel/stubs/hvf-stub.o
CC aarch64-softmmu/tcg/optimize.o
CC aarch64-softmmu/fpu/softfloat.o
CC x86_64-softmmu/accel/stubs/whpx-stub.o
CC x86_64-softmmu/accel/tcg/tcg-all.o
CC aarch64-softmmu/disas.o
CC x86_64-softmmu/accel/tcg/cputlb.o
CC x86_64-softmmu/accel/tcg/tcg-runtime.o
CC x86_64-softmmu/accel/tcg/tcg-runtime-gvec.o
CC x86_64-softmmu/accel/tcg/cpu-exec.o
GEN aarch64-softmmu/gdbstub-xml.c
CC x86_64-softmmu/accel/tcg/cpu-exec-common.o
CC x86_64-softmmu/accel/tcg/translate-all.o
CC aarch64-softmmu/arch_init.o
CC x86_64-softmmu/accel/tcg/translator.o
CC x86_64-softmmu/hw/block/virtio-blk.o
CC aarch64-softmmu/cpus.o
CC x86_64-softmmu/hw/block/vhost-user-blk.o
CC aarch64-softmmu/monitor.o
CC x86_64-softmmu/hw/block/dataplane/virtio-blk.o
CC x86_64-softmmu/hw/char/virtio-serial-bus.o
CC aarch64-softmmu/gdbstub.o
CC aarch64-softmmu/balloon.o
CC aarch64-softmmu/ioport.o
CC aarch64-softmmu/numa.o
CC x86_64-softmmu/hw/core/generic-loader.o
CC aarch64-softmmu/qtest.o
CC aarch64-softmmu/memory.o
CC x86_64-softmmu/hw/core/null-machine.o
CC x86_64-softmmu/hw/display/vga.o
CC aarch64-softmmu/memory_mapping.o
CC aarch64-softmmu/dump.o
CC aarch64-softmmu/migration/ram.o
CC x86_64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/accel/accel.o
CC aarch64-softmmu/accel/stubs/hax-stub.o
CC aarch64-softmmu/accel/stubs/hvf-stub.o
CC x86_64-softmmu/hw/display/virtio-gpu-3d.o
CC x86_64-softmmu/hw/display/virtio-gpu-pci.o
CC x86_64-softmmu/hw/display/virtio-vga.o
CC x86_64-softmmu/hw/intc/apic.o
CC aarch64-softmmu/accel/stubs/whpx-stub.o
CC x86_64-softmmu/hw/intc/apic_common.o
CC aarch64-softmmu/accel/stubs/kvm-stub.o
CC aarch64-softmmu/accel/tcg/tcg-all.o
CC x86_64-softmmu/hw/intc/ioapic.o
CC x86_64-softmmu/hw/isa/lpc_ich9.o
CC x86_64-softmmu/hw/misc/ivshmem.o
CC aarch64-softmmu/accel/tcg/cputlb.o
CC aarch64-softmmu/accel/tcg/tcg-runtime.o
CC x86_64-softmmu/hw/misc/pvpanic.o
CC x86_64-softmmu/hw/misc/hyperv_testdev.o
CC x86_64-softmmu/hw/misc/mmio_interface.o
CC x86_64-softmmu/hw/net/virtio-net.o
CC x86_64-softmmu/hw/net/vhost_net.o
CC x86_64-softmmu/hw/rdma/rdma_utils.o
CC x86_64-softmmu/hw/rdma/rdma_backend.o
CC x86_64-softmmu/hw/rdma/rdma_rm.o
CC x86_64-softmmu/hw/rdma/vmw/pvrdma_dev_ring.o
CC aarch64-softmmu/accel/tcg/tcg-runtime-gvec.o
CC x86_64-softmmu/hw/rdma/vmw/pvrdma_cmd.o
CC x86_64-softmmu/hw/rdma/vmw/pvrdma_qp_ops.o
CC x86_64-softmmu/hw/rdma/vmw/pvrdma_main.o
CC x86_64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/accel/tcg/cpu-exec.o
CC aarch64-softmmu/accel/tcg/cpu-exec-common.o
CC x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC x86_64-softmmu/hw/scsi/vhost-scsi-common.o
CC x86_64-softmmu/hw/scsi/vhost-scsi.o
CC aarch64-softmmu/accel/tcg/translate-all.o
CC aarch64-softmmu/accel/tcg/translator.o
CC x86_64-softmmu/hw/scsi/vhost-user-scsi.o
CC x86_64-softmmu/hw/timer/mc146818rtc.o
CC x86_64-softmmu/hw/vfio/common.o
CC aarch64-softmmu/hw/adc/stm32f2xx_adc.o
CC aarch64-softmmu/hw/block/virtio-blk.o
CC x86_64-softmmu/hw/vfio/pci.o
CC aarch64-softmmu/hw/block/vhost-user-blk.o
CC x86_64-softmmu/hw/vfio/pci-quirks.o
CC aarch64-softmmu/hw/block/dataplane/virtio-blk.o
CC x86_64-softmmu/hw/vfio/display.o
CC x86_64-softmmu/hw/vfio/platform.o
CC aarch64-softmmu/hw/char/exynos4210_uart.o
CC aarch64-softmmu/hw/char/omap_uart.o
CC x86_64-softmmu/hw/vfio/spapr.o
CC x86_64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/char/digic-uart.o
CC x86_64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/char/stm32f2xx_usart.o
CC aarch64-softmmu/hw/char/bcm2835_aux.o
CC x86_64-softmmu/hw/virtio/virtio-crypto.o
CC aarch64-softmmu/hw/char/virtio-serial-bus.o
CC aarch64-softmmu/hw/core/generic-loader.o
CC x86_64-softmmu/hw/virtio/virtio-crypto-pci.o
CC x86_64-softmmu/hw/virtio/vhost.o
CC aarch64-softmmu/hw/core/null-machine.o
CC aarch64-softmmu/hw/cpu/arm11mpcore.o
CC x86_64-softmmu/hw/virtio/vhost-backend.o
CC x86_64-softmmu/hw/virtio/vhost-user.o
CC x86_64-softmmu/hw/virtio/vhost-vsock.o
CC x86_64-softmmu/hw/xen/xen-host-pci-device.o
CC x86_64-softmmu/hw/xen/xen_pt.o
CC aarch64-softmmu/hw/cpu/realview_mpcore.o
CC x86_64-softmmu/hw/xen/xen_pt_config_init.o
CC x86_64-softmmu/hw/xen/xen_pt_graphics.o
CC aarch64-softmmu/hw/cpu/a9mpcore.o
CC aarch64-softmmu/hw/cpu/a15mpcore.o
CC x86_64-softmmu/hw/xen/xen_pt_msi.o
CC x86_64-softmmu/hw/xen/xen_pt_load_rom.o
CC x86_64-softmmu/hw/i386/multiboot.o
CC x86_64-softmmu/hw/i386/pc.o
CC x86_64-softmmu/hw/i386/pc_piix.o
CC x86_64-softmmu/hw/i386/pc_q35.o
CC x86_64-softmmu/hw/i386/pc_sysfw.o
CC x86_64-softmmu/hw/i386/x86-iommu.o
CC x86_64-softmmu/hw/i386/intel_iommu.o
CC x86_64-softmmu/hw/i386/amd_iommu.o
CC x86_64-softmmu/hw/i386/vmport.o
CC x86_64-softmmu/hw/i386/vmmouse.o
CC x86_64-softmmu/hw/i386/kvmvapic.o
CC x86_64-softmmu/hw/i386/acpi-build.o
CC x86_64-softmmu/hw/i386/../xenpv/xen_machine_pv.o
CC x86_64-softmmu/hw/i386/kvm/clock.o
CC x86_64-softmmu/hw/i386/kvm/apic.o
CC x86_64-softmmu/hw/i386/kvm/i8259.o
CC x86_64-softmmu/hw/i386/kvm/ioapic.o
CC aarch64-softmmu/hw/display/omap_dss.o
CC x86_64-softmmu/hw/i386/kvm/i8254.o
CC x86_64-softmmu/hw/i386/xen/xen_platform.o
CC x86_64-softmmu/hw/i386/xen/xen_apic.o
CC x86_64-softmmu/hw/i386/xen/xen_pvdevice.o
CC x86_64-softmmu/hw/i386/xen/xen-hvm.o
CC x86_64-softmmu/hw/i386/xen/xen-mapcache.o
CC x86_64-softmmu/target/i386/helper.o
CC x86_64-softmmu/target/i386/cpu.o
CC x86_64-softmmu/target/i386/gdbstub.o
CC x86_64-softmmu/target/i386/xsave_helper.o
CC x86_64-softmmu/target/i386/translate.o
CC x86_64-softmmu/target/i386/cc_helper.o
CC x86_64-softmmu/target/i386/bpt_helper.o
CC x86_64-softmmu/target/i386/excp_helper.o
CC x86_64-softmmu/target/i386/fpu_helper.o
CC x86_64-softmmu/target/i386/int_helper.o
CC x86_64-softmmu/target/i386/mem_helper.o
CC x86_64-softmmu/target/i386/misc_helper.o
CC x86_64-softmmu/target/i386/mpx_helper.o
CC x86_64-softmmu/target/i386/seg_helper.o
CC x86_64-softmmu/target/i386/smm_helper.o
CC x86_64-softmmu/target/i386/svm_helper.o
CC x86_64-softmmu/target/i386/machine.o
CC x86_64-softmmu/target/i386/arch_memory_mapping.o
CC x86_64-softmmu/target/i386/arch_dump.o
CC x86_64-softmmu/target/i386/monitor.o
CC x86_64-softmmu/target/i386/kvm.o
CC aarch64-softmmu/hw/display/omap_lcdc.o
CC x86_64-softmmu/target/i386/hyperv.o
CC x86_64-softmmu/target/i386/sev.o
CC aarch64-softmmu/hw/display/pxa2xx_lcd.o
GEN trace/generated-helpers.c
CC aarch64-softmmu/hw/display/bcm2835_fb.o
CC x86_64-softmmu/trace/control-target.o
CC aarch64-softmmu/hw/display/vga.o
CC x86_64-softmmu/gdbstub-xml.o
CC aarch64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/hw/display/virtio-gpu-3d.o
CC aarch64-softmmu/hw/display/virtio-gpu-pci.o
CC aarch64-softmmu/hw/display/dpcd.o
CC aarch64-softmmu/hw/display/xlnx_dp.o
CC aarch64-softmmu/hw/dma/xlnx_dpdma.o
CC aarch64-softmmu/hw/dma/omap_dma.o
CC aarch64-softmmu/hw/dma/soc_dma.o
CC x86_64-softmmu/trace/generated-helpers.o
CC aarch64-softmmu/hw/dma/pxa2xx_dma.o
CC aarch64-softmmu/hw/dma/bcm2835_dma.o
CC aarch64-softmmu/hw/gpio/omap_gpio.o
CC aarch64-softmmu/hw/gpio/imx_gpio.o
CC aarch64-softmmu/hw/gpio/bcm2835_gpio.o
CC aarch64-softmmu/hw/i2c/omap_i2c.o
CC aarch64-softmmu/hw/input/pxa2xx_keypad.o
CC aarch64-softmmu/hw/input/tsc210x.o
CC aarch64-softmmu/hw/intc/armv7m_nvic.o
CC aarch64-softmmu/hw/intc/exynos4210_gic.o
CC aarch64-softmmu/hw/intc/exynos4210_combiner.o
CC aarch64-softmmu/hw/intc/omap_intc.o
CC aarch64-softmmu/hw/intc/bcm2835_ic.o
LINK x86_64-softmmu/qemu-system-x86_64
CC aarch64-softmmu/hw/intc/bcm2836_control.o
CC aarch64-softmmu/hw/intc/allwinner-a10-pic.o
CC aarch64-softmmu/hw/intc/aspeed_vic.o
CC aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
CC aarch64-softmmu/hw/misc/ivshmem.o
CC aarch64-softmmu/hw/misc/arm_sysctl.o
CC aarch64-softmmu/hw/misc/cbus.o
CC aarch64-softmmu/hw/misc/exynos4210_pmu.o
CC aarch64-softmmu/hw/misc/exynos4210_clk.o
CC aarch64-softmmu/hw/misc/exynos4210_rng.o
CC aarch64-softmmu/hw/misc/imx_ccm.o
CC aarch64-softmmu/hw/misc/imx31_ccm.o
CC aarch64-softmmu/hw/misc/imx25_ccm.o
CC aarch64-softmmu/hw/misc/imx6_ccm.o
CC aarch64-softmmu/hw/misc/imx6_src.o
CC aarch64-softmmu/hw/misc/imx7_ccm.o
CC aarch64-softmmu/hw/misc/imx2_wdt.o
CC aarch64-softmmu/hw/misc/imx7_snvs.o
CC aarch64-softmmu/hw/misc/imx7_gpr.o
CC aarch64-softmmu/hw/misc/omap_clk.o
CC aarch64-softmmu/hw/misc/omap_gpmc.o
CC aarch64-softmmu/hw/misc/mst_fpga.o
CC aarch64-softmmu/hw/misc/omap_l4.o
CC aarch64-softmmu/hw/misc/omap_sdrc.o
CC aarch64-softmmu/hw/misc/omap_tap.o
CC aarch64-softmmu/hw/misc/bcm2835_mbox.o
CC aarch64-softmmu/hw/misc/bcm2835_property.o
CC aarch64-softmmu/hw/misc/bcm2835_rng.o
CC aarch64-softmmu/hw/misc/zynq_slcr.o
CC aarch64-softmmu/hw/misc/zynq-xadc.o
CC aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
CC aarch64-softmmu/hw/misc/mps2-fpgaio.o
CC aarch64-softmmu/hw/misc/mps2-scc.o
CC aarch64-softmmu/hw/misc/tz-mpc.o
CC aarch64-softmmu/hw/misc/tz-ppc.o
CC aarch64-softmmu/hw/misc/iotkit-secctl.o
CC aarch64-softmmu/hw/misc/auxbus.o
CC aarch64-softmmu/hw/misc/aspeed_scu.o
CC aarch64-softmmu/hw/misc/aspeed_sdmc.o
CC aarch64-softmmu/hw/misc/mmio_interface.o
CC aarch64-softmmu/hw/misc/msf2-sysreg.o
CC aarch64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/hw/net/vhost_net.o
CC aarch64-softmmu/hw/pcmcia/pxa2xx.o
CC aarch64-softmmu/hw/rdma/rdma_utils.o
CC aarch64-softmmu/hw/rdma/rdma_backend.o
CC aarch64-softmmu/hw/rdma/rdma_rm.o
CC aarch64-softmmu/hw/rdma/vmw/pvrdma_dev_ring.o
CC aarch64-softmmu/hw/rdma/vmw/pvrdma_cmd.o
CC aarch64-softmmu/hw/rdma/vmw/pvrdma_qp_ops.o
CC aarch64-softmmu/hw/rdma/vmw/pvrdma_main.o
CC aarch64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC aarch64-softmmu/hw/scsi/vhost-scsi-common.o
CC aarch64-softmmu/hw/scsi/vhost-scsi.o
CC aarch64-softmmu/hw/scsi/vhost-user-scsi.o
CC aarch64-softmmu/hw/sd/omap_mmc.o
CC aarch64-softmmu/hw/sd/pxa2xx_mmci.o
CC aarch64-softmmu/hw/sd/bcm2835_sdhost.o
CC aarch64-softmmu/hw/ssi/omap_spi.o
CC aarch64-softmmu/hw/ssi/imx_spi.o
CC aarch64-softmmu/hw/timer/exynos4210_pwm.o
CC aarch64-softmmu/hw/timer/exynos4210_mct.o
CC aarch64-softmmu/hw/timer/exynos4210_rtc.o
CC aarch64-softmmu/hw/timer/omap_gptimer.o
CC aarch64-softmmu/hw/timer/omap_synctimer.o
CC aarch64-softmmu/hw/timer/pxa2xx_timer.o
CC aarch64-softmmu/hw/timer/digic-timer.o
CC aarch64-softmmu/hw/timer/allwinner-a10-pit.o
CC aarch64-softmmu/hw/usb/tusb6010.o
CC aarch64-softmmu/hw/usb/chipidea.o
CC aarch64-softmmu/hw/vfio/common.o
CC aarch64-softmmu/hw/vfio/pci.o
CC aarch64-softmmu/hw/vfio/pci-quirks.o
CC aarch64-softmmu/hw/vfio/display.o
CC aarch64-softmmu/hw/vfio/platform.o
CC aarch64-softmmu/hw/vfio/calxeda-xgmac.o
CC aarch64-softmmu/hw/vfio/amd-xgbe.o
CC aarch64-softmmu/hw/vfio/spapr.o
CC aarch64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/virtio/virtio-crypto.o
CC aarch64-softmmu/hw/virtio/virtio-crypto-pci.o
CC aarch64-softmmu/hw/virtio/vhost.o
CC aarch64-softmmu/hw/virtio/vhost-backend.o
CC aarch64-softmmu/hw/virtio/vhost-user.o
CC aarch64-softmmu/hw/virtio/vhost-vsock.o
CC aarch64-softmmu/hw/arm/boot.o
CC aarch64-softmmu/hw/arm/virt.o
CC aarch64-softmmu/hw/arm/sysbus-fdt.o
CC aarch64-softmmu/hw/arm/virt-acpi-build.o
CC aarch64-softmmu/hw/arm/digic_boards.o
CC aarch64-softmmu/hw/arm/exynos4_boards.o
CC aarch64-softmmu/hw/arm/highbank.o
CC aarch64-softmmu/hw/arm/integratorcp.o
CC aarch64-softmmu/hw/arm/mainstone.o
CC aarch64-softmmu/hw/arm/musicpal.o
CC aarch64-softmmu/hw/arm/netduino2.o
CC aarch64-softmmu/hw/arm/nseries.o
CC aarch64-softmmu/hw/arm/omap_sx1.o
CC aarch64-softmmu/hw/arm/palm.o
CC aarch64-softmmu/hw/arm/gumstix.o
CC aarch64-softmmu/hw/arm/spitz.o
CC aarch64-softmmu/hw/arm/tosa.o
CC aarch64-softmmu/hw/arm/z2.o
CC aarch64-softmmu/hw/arm/realview.o
CC aarch64-softmmu/hw/arm/stellaris.o
CC aarch64-softmmu/hw/arm/collie.o
CC aarch64-softmmu/hw/arm/vexpress.o
CC aarch64-softmmu/hw/arm/versatilepb.o
CC aarch64-softmmu/hw/arm/xilinx_zynq.o
CC aarch64-softmmu/hw/arm/armv7m.o
CC aarch64-softmmu/hw/arm/exynos4210.o
CC aarch64-softmmu/hw/arm/pxa2xx.o
CC aarch64-softmmu/hw/arm/pxa2xx_gpio.o
CC aarch64-softmmu/hw/arm/pxa2xx_pic.o
CC aarch64-softmmu/hw/arm/digic.o
CC aarch64-softmmu/hw/arm/omap1.o
CC aarch64-softmmu/hw/arm/omap2.o
CC aarch64-softmmu/hw/arm/strongarm.o
CC aarch64-softmmu/hw/arm/allwinner-a10.o
CC aarch64-softmmu/hw/arm/cubieboard.o
CC aarch64-softmmu/hw/arm/bcm2835_peripherals.o
CC aarch64-softmmu/hw/arm/bcm2836.o
CC aarch64-softmmu/hw/arm/raspi.o
CC aarch64-softmmu/hw/arm/stm32f205_soc.o
CC aarch64-softmmu/hw/arm/xlnx-zynqmp.o
CC aarch64-softmmu/hw/arm/xlnx-zcu102.o
CC aarch64-softmmu/hw/arm/fsl-imx25.o
CC aarch64-softmmu/hw/arm/imx25_pdk.o
CC aarch64-softmmu/hw/arm/fsl-imx31.o
CC aarch64-softmmu/hw/arm/kzm.o
CC aarch64-softmmu/hw/arm/fsl-imx6.o
CC aarch64-softmmu/hw/arm/sabrelite.o
CC aarch64-softmmu/hw/arm/aspeed_soc.o
CC aarch64-softmmu/hw/arm/aspeed.o
CC aarch64-softmmu/hw/arm/mps2.o
CC aarch64-softmmu/hw/arm/mps2-tz.o
CC aarch64-softmmu/hw/arm/msf2-soc.o
CC aarch64-softmmu/hw/arm/msf2-som.o
CC aarch64-softmmu/hw/arm/iotkit.o
CC aarch64-softmmu/hw/arm/fsl-imx7.o
CC aarch64-softmmu/hw/arm/mcimx7d-sabre.o
CC aarch64-softmmu/hw/arm/smmu-common.o
CC aarch64-softmmu/hw/arm/smmuv3.o
CC aarch64-softmmu/target/arm/arm-semi.o
CC aarch64-softmmu/target/arm/machine.o
CC aarch64-softmmu/target/arm/psci.o
CC aarch64-softmmu/target/arm/arch_dump.o
CC aarch64-softmmu/target/arm/monitor.o
CC aarch64-softmmu/target/arm/kvm-stub.o
CC aarch64-softmmu/target/arm/translate.o
CC aarch64-softmmu/target/arm/op_helper.o
CC aarch64-softmmu/target/arm/helper.o
CC aarch64-softmmu/target/arm/cpu.o
CC aarch64-softmmu/target/arm/neon_helper.o
CC aarch64-softmmu/target/arm/iwmmxt_helper.o
CC aarch64-softmmu/target/arm/vec_helper.o
CC aarch64-softmmu/target/arm/gdbstub.o
CC aarch64-softmmu/target/arm/cpu64.o
CC aarch64-softmmu/target/arm/translate-a64.o
CC aarch64-softmmu/target/arm/helper-a64.o
CC aarch64-softmmu/target/arm/gdbstub64.o
CC aarch64-softmmu/target/arm/crypto_helper.o
CC aarch64-softmmu/target/arm/arm-powerctl.o
GEN aarch64-softmmu/target/arm/decode-sve.inc.c
CC aarch64-softmmu/target/arm/sve_helper.o
GEN trace/generated-helpers.c
CC aarch64-softmmu/trace/control-target.o
CC aarch64-softmmu/gdbstub-xml.o
CC aarch64-softmmu/trace/generated-helpers.o
CC aarch64-softmmu/target/arm/translate-sve.o
LINK aarch64-softmmu/qemu-system-aarch64
TEST tests/qapi-schema/alternate-any.out
TEST tests/qapi-schema/alternate-array.out
TEST tests/qapi-schema/alternate-base.out
TEST tests/qapi-schema/alternate-clash.out
TEST tests/qapi-schema/alternate-conflict-dict.out
TEST tests/qapi-schema/alternate-conflict-enum-bool.out
TEST tests/qapi-schema/alternate-conflict-enum-int.out
TEST tests/qapi-schema/alternate-conflict-string.out
TEST tests/qapi-schema/alternate-conflict-bool-string.out
TEST tests/qapi-schema/alternate-conflict-num-string.out
TEST tests/qapi-schema/alternate-empty.out
TEST tests/qapi-schema/alternate-unknown.out
TEST tests/qapi-schema/args-alternate.out
TEST tests/qapi-schema/alternate-nested.out
TEST tests/qapi-schema/args-array-empty.out
TEST tests/qapi-schema/args-any.out
TEST tests/qapi-schema/args-array-unknown.out
TEST tests/qapi-schema/args-bad-boxed.out
TEST tests/qapi-schema/args-boxed-anon.out
TEST tests/qapi-schema/args-boxed-empty.out
TEST tests/qapi-schema/args-boxed-string.out
TEST tests/qapi-schema/args-int.out
TEST tests/qapi-schema/args-invalid.out
TEST tests/qapi-schema/args-member-array-bad.out
TEST tests/qapi-schema/args-member-case.out
TEST tests/qapi-schema/args-member-unknown.out
TEST tests/qapi-schema/args-name-clash.out
TEST tests/qapi-schema/args-union.out
TEST tests/qapi-schema/args-unknown.out
TEST tests/qapi-schema/bad-base.out
TEST tests/qapi-schema/bad-data.out
TEST tests/qapi-schema/bad-ident.out
TEST tests/qapi-schema/bad-if.out
TEST tests/qapi-schema/bad-if-empty.out
TEST tests/qapi-schema/bad-if-empty-list.out
TEST tests/qapi-schema/bad-if-list.out
TEST tests/qapi-schema/bad-type-bool.out
TEST tests/qapi-schema/bad-type-dict.out
TEST tests/qapi-schema/bad-type-int.out
TEST tests/qapi-schema/base-cycle-direct.out
TEST tests/qapi-schema/base-cycle-indirect.out
TEST tests/qapi-schema/command-int.out
TEST tests/qapi-schema/comments.out
TEST tests/qapi-schema/doc-bad-command-arg.out
TEST tests/qapi-schema/doc-bad-alternate-member.out
TEST tests/qapi-schema/doc-bad-section.out
TEST tests/qapi-schema/doc-bad-symbol.out
TEST tests/qapi-schema/doc-bad-union-member.out
TEST tests/qapi-schema/doc-before-include.out
TEST tests/qapi-schema/doc-before-pragma.out
TEST tests/qapi-schema/doc-duplicated-arg.out
TEST tests/qapi-schema/doc-duplicated-return.out
TEST tests/qapi-schema/doc-duplicated-since.out
TEST tests/qapi-schema/doc-empty-arg.out
TEST tests/qapi-schema/doc-empty-section.out
TEST tests/qapi-schema/doc-empty-symbol.out
TEST tests/qapi-schema/doc-good.out
TEST tests/qapi-schema/doc-interleaved-section.out
TEST tests/qapi-schema/doc-invalid-end.out
TEST tests/qapi-schema/doc-invalid-end2.out
TEST tests/qapi-schema/doc-invalid-return.out
TEST tests/qapi-schema/doc-invalid-section.out
TEST tests/qapi-schema/doc-invalid-start.out
TEST tests/qapi-schema/doc-missing-colon.out
TEST tests/qapi-schema/doc-missing-expr.out
TEST tests/qapi-schema/doc-missing-space.out
TEST tests/qapi-schema/doc-missing.out
TEST tests/qapi-schema/doc-no-symbol.out
TEST tests/qapi-schema/double-data.out
TEST tests/qapi-schema/double-type.out
TEST tests/qapi-schema/duplicate-key.out
TEST tests/qapi-schema/empty.out
TEST tests/qapi-schema/enum-bad-name.out
TEST tests/qapi-schema/enum-bad-prefix.out
TEST tests/qapi-schema/enum-clash-member.out
TEST tests/qapi-schema/enum-dict-member.out
TEST tests/qapi-schema/enum-int-member.out
TEST tests/qapi-schema/enum-member-case.out
TEST tests/qapi-schema/enum-missing-data.out
TEST tests/qapi-schema/enum-wrong-data.out
TEST tests/qapi-schema/escape-outside-string.out
TEST tests/qapi-schema/escape-too-big.out
TEST tests/qapi-schema/escape-too-short.out
TEST tests/qapi-schema/event-boxed-empty.out
TEST tests/qapi-schema/event-case.out
TEST tests/qapi-schema/event-nest-struct.out
TEST tests/qapi-schema/flat-union-array-branch.out
TEST tests/qapi-schema/flat-union-bad-base.out
TEST tests/qapi-schema/flat-union-bad-discriminator.out
TEST tests/qapi-schema/flat-union-base-any.out
TEST tests/qapi-schema/flat-union-base-union.out
TEST tests/qapi-schema/flat-union-clash-member.out
TEST tests/qapi-schema/flat-union-empty.out
TEST tests/qapi-schema/flat-union-inline.out
TEST tests/qapi-schema/flat-union-int-branch.out
TEST tests/qapi-schema/flat-union-invalid-branch-key.out
TEST tests/qapi-schema/flat-union-invalid-discriminator.out
TEST tests/qapi-schema/flat-union-no-base.out
TEST tests/qapi-schema/flat-union-optional-discriminator.out
TEST tests/qapi-schema/flat-union-string-discriminator.out
TEST tests/qapi-schema/funny-char.out
TEST tests/qapi-schema/ident-with-escape.out
TEST tests/qapi-schema/include-before-err.out
TEST tests/qapi-schema/include-cycle.out
TEST tests/qapi-schema/include-extra-junk.out
TEST tests/qapi-schema/include-format-err.out
TEST tests/qapi-schema/include-nested-err.out
TEST tests/qapi-schema/include-no-file.out
TEST tests/qapi-schema/include-non-file.out
TEST tests/qapi-schema/include-repetition.out
TEST tests/qapi-schema/include-relpath.out
TEST tests/qapi-schema/include-self-cycle.out
TEST tests/qapi-schema/include-simple.out
TEST tests/qapi-schema/indented-expr.out
TEST tests/qapi-schema/leading-comma-list.out
TEST tests/qapi-schema/leading-comma-object.out
TEST tests/qapi-schema/missing-comma-list.out
TEST tests/qapi-schema/missing-colon.out
TEST tests/qapi-schema/missing-comma-object.out
TEST tests/qapi-schema/missing-type.out
TEST tests/qapi-schema/nested-struct-data.out
TEST tests/qapi-schema/non-objects.out
TEST tests/qapi-schema/oob-test.out
TEST tests/qapi-schema/allow-preconfig-test.out
TEST tests/qapi-schema/pragma-doc-required-crap.out
TEST tests/qapi-schema/pragma-extra-junk.out
TEST tests/qapi-schema/pragma-non-dict.out
TEST tests/qapi-schema/pragma-name-case-whitelist-crap.out
TEST tests/qapi-schema/pragma-returns-whitelist-crap.out
TEST tests/qapi-schema/qapi-schema-test.out
TEST tests/qapi-schema/quoted-structural-chars.out
TEST tests/qapi-schema/redefined-builtin.out
TEST tests/qapi-schema/redefined-command.out
TEST tests/qapi-schema/redefined-event.out
TEST tests/qapi-schema/redefined-type.out
TEST tests/qapi-schema/reserved-command-q.out
TEST tests/qapi-schema/reserved-enum-q.out
TEST tests/qapi-schema/reserved-member-has.out
TEST tests/qapi-schema/reserved-member-q.out
TEST tests/qapi-schema/reserved-member-u.out
TEST tests/qapi-schema/reserved-member-underscore.out
TEST tests/qapi-schema/reserved-type-kind.out
TEST tests/qapi-schema/reserved-type-list.out
TEST tests/qapi-schema/returns-alternate.out
TEST tests/qapi-schema/returns-array-bad.out
TEST tests/qapi-schema/returns-dict.out
TEST tests/qapi-schema/returns-unknown.out
TEST tests/qapi-schema/returns-whitelist.out
TEST tests/qapi-schema/struct-base-clash-deep.out
TEST tests/qapi-schema/struct-base-clash.out
TEST tests/qapi-schema/struct-data-invalid.out
TEST tests/qapi-schema/struct-member-invalid.out
TEST tests/qapi-schema/trailing-comma-list.out
TEST tests/qapi-schema/trailing-comma-object.out
TEST tests/qapi-schema/type-bypass-bad-gen.out
TEST tests/qapi-schema/unclosed-object.out
TEST tests/qapi-schema/unclosed-list.out
TEST tests/qapi-schema/unclosed-string.out
TEST tests/qapi-schema/unicode-str.out
TEST tests/qapi-schema/union-base-empty.out
TEST tests/qapi-schema/union-base-no-discriminator.out
TEST tests/qapi-schema/union-branch-case.out
TEST tests/qapi-schema/union-clash-branches.out
TEST tests/qapi-schema/union-empty.out
TEST tests/qapi-schema/union-invalid-base.out
TEST tests/qapi-schema/union-optional-branch.out
TEST tests/qapi-schema/union-unknown.out
TEST tests/qapi-schema/unknown-escape.out
TEST tests/qapi-schema/unknown-expr-key.out
GEN tests/qapi-schema/doc-good.test.texi
CC tests/check-block-qdict.o
CC tests/test-char.o
CC tests/check-qnum.o
CC tests/check-qstring.o
CC tests/check-qlist.o
CC tests/check-qnull.o
CC tests/check-qobject.o
CC tests/check-qjson.o
CC tests/check-qlit.o
CC tests/test-qobject-output-visitor.o
CC tests/test-qapi-visit.o
CC tests/test-qapi-types.o
CC tests/test-qapi-events.o
CC tests/test-qapi-introspect.o
CC tests/test-clone-visitor.o
CC tests/test-qobject-input-visitor.o
CC tests/test-qmp-cmds.o
CC tests/test-qapi-commands.o
CC tests/test-string-input-visitor.o
CC tests/test-string-output-visitor.o
CC tests/test-qmp-event.o
CC tests/test-opts-visitor.o
CC tests/test-coroutine.o
CC tests/iothread.o
CC tests/test-visitor-serialization.o
CC tests/test-iov.o
CC tests/test-aio.o
CC tests/test-aio-multithread.o
CC tests/test-throttle.o
CC tests/test-thread-pool.o
CC tests/test-hbitmap.o
CC tests/test-bdrv-drain.o
CC tests/test-blockjob.o
CC tests/test-x86-cpuid.o
CC tests/test-block-backend.o
CC tests/test-xbzrle.o
CC tests/test-blockjob-txn.o
CC tests/test-vmstate.o
CC tests/test-cutils.o
CC tests/test-shift128.o
CC tests/test-mul64.o
CC tests/test-int128.o
CC tests/rcutorture.o
CC tests/test-rcu-list.o
CC tests/test-qdist.o
CC tests/test-qht.o
CC tests/qht-bench.o
CC tests/test-bitops.o
CC tests/test-qht-par.o
CC tests/test-bitcnt.o
CC tests/test-qdev-global-props.o
CC tests/check-qom-interface.o
CC tests/check-qom-proplist.o
CC tests/test-qemu-opts.o
CC tests/test-keyval.o
CC tests/test-write-threshold.o
CC tests/test-crypto-hash.o
CC tests/test-crypto-hmac.o
CC tests/test-crypto-cipher.o
CC tests/test-crypto-secret.o
CC tests/libqtest.o
CC tests/test-qga.o
CC tests/test-timed-average.o
CC tests/test-util-sockets.o
CC tests/socket-helpers.o
CC tests/test-io-task.o
CC tests/test-io-channel-socket.o
CC tests/io-channel-helpers.o
CC tests/test-io-channel-file.o
CC tests/test-io-channel-command.o
CC tests/test-io-channel-buffer.o
CC tests/test-base64.o
CC tests/test-crypto-ivgen.o
CC tests/test-crypto-afsplit.o
CC tests/test-crypto-xts.o
CC tests/test-crypto-block.o
CC tests/test-logging.o
CC tests/test-replication.o
CC tests/test-bufferiszero.o
CC tests/test-uuid.o
CC tests/ptimer-test.o
CC tests/ptimer-test-stubs.o
CC tests/test-qapi-util.o
CC tests/vhost-user-test.o
CC tests/libqos/pci.o
CC tests/libqos/fw_cfg.o
CC tests/libqos/malloc.o
CC tests/libqos/i2c.o
CC tests/libqos/libqos.o
CC tests/libqos/malloc-spapr.o
CC tests/libqos/libqos-spapr.o
CC tests/libqos/rtas.o
CC tests/libqos/pci-spapr.o
CC tests/libqos/pci-pc.o
CC tests/libqos/malloc-pc.o
CC tests/libqos/libqos-pc.o
CC tests/libqos/ahci.o
CC tests/libqos/virtio.o
CC tests/libqos/virtio-pci.o
CC tests/libqos/virtio-mmio.o
CC tests/libqos/malloc-generic.o
CC tests/endianness-test.o
CC tests/fdc-test.o
CC tests/ahci-test.o
CC tests/ide-test.o
CC tests/hd-geo-test.o
CC tests/boot-order-test.o
CC tests/bios-tables-test.o
CC tests/boot-sector.o
CC tests/acpi-utils.o
CC tests/boot-serial-test.o
CC tests/pxe-test.o
CC tests/rtc-test.o
CC tests/ipmi-kcs-test.o
CC tests/ipmi-bt-test.o
CC tests/i440fx-test.o
CC tests/fw_cfg-test.o
CC tests/drive_del-test.o
CC tests/wdt_ib700-test.o
CC tests/tco-test.o
CC tests/e1000-test.o
CC tests/e1000e-test.o
CC tests/rtl8139-test.o
CC tests/pcnet-test.o
CC tests/ne2000-test.o
CC tests/eepro100-test.o
CC tests/nvme-test.o
CC tests/ac97-test.o
CC tests/es1370-test.o
CC tests/virtio-net-test.o
CC tests/virtio-balloon-test.o
CC tests/virtio-blk-test.o
CC tests/virtio-rng-test.o
CC tests/virtio-scsi-test.o
CC tests/virtio-serial-test.o
CC tests/virtio-console-test.o
CC tests/tpci200-test.o
CC tests/ipoctal232-test.o
CC tests/display-vga-test.o
CC tests/intel-hda-test.o
CC tests/ivshmem-test.o
CC tests/megasas-test.o
CC tests/vmxnet3-test.o
CC tests/pvpanic-test.o
CC tests/i82801b11-test.o
CC tests/ioh3420-test.o
CC tests/usb-hcd-ohci-test.o
CC tests/libqos/usb.o
CC tests/usb-hcd-uhci-test.o
CC tests/usb-hcd-ehci-test.o
CC tests/usb-hcd-xhci-test.o
CC tests/cpu-plug-test.o
CC tests/q35-test.o
CC tests/vmgenid-test.o
CC tests/tpm-crb-swtpm-test.o
CC tests/tpm-emu.o
CC tests/tpm-util.o
CC tests/tpm-tests.o
CC tests/tpm-crb-test.o
CC tests/tpm-tis-swtpm-test.o
CC tests/tpm-tis-test.o
CC tests/test-netfilter.o
CC tests/test-filter-mirror.o
CC tests/test-filter-redirector.o
CC tests/migration-test.o
CC tests/test-x86-cpuid-compat.o
CC tests/sdhci-test.o
CC tests/numa-test.o
CC tests/qmp-test.o
CC tests/device-introspect-test.o
CC tests/cdrom-test.o
CC tests/machine-none-test.o
CC tests/qom-test.o
CC tests/test-hmp.o
TEST decodetree.py
LINK tests/check-block-qdict
LINK tests/test-char
LINK tests/check-qnum
LINK tests/check-qstring
LINK tests/check-qlist
LINK tests/check-qnull
LINK tests/check-qobject
LINK tests/check-qjson
LINK tests/check-qlit
LINK tests/test-qobject-output-visitor
LINK tests/test-clone-visitor
LINK tests/test-qobject-input-visitor
LINK tests/test-qmp-cmds
LINK tests/test-string-input-visitor
LINK tests/test-string-output-visitor
LINK tests/test-qmp-event
LINK tests/test-opts-visitor
LINK tests/test-coroutine
LINK tests/test-visitor-serialization
LINK tests/test-iov
LINK tests/test-aio
LINK tests/test-aio-multithread
LINK tests/test-throttle
LINK tests/test-thread-pool
LINK tests/test-hbitmap
LINK tests/test-bdrv-drain
LINK tests/test-blockjob
LINK tests/test-blockjob-txn
LINK tests/test-block-backend
LINK tests/test-x86-cpuid
LINK tests/test-xbzrle
LINK tests/test-vmstate
LINK tests/test-cutils
LINK tests/test-shift128
LINK tests/test-mul64
LINK tests/test-int128
LINK tests/rcutorture
LINK tests/test-rcu-list
LINK tests/test-qdist
migration/qemu-file.o: In function `qemu_fclose':
/tmp/qemu-test/src/migration/qemu-file.c:331: undefined reference to `migrate_get_current'
/tmp/qemu-test/src/migration/qemu-file.c:333: undefined reference to `migrate_get_current'
collect2: error: ld returned 1 exit status
make: *** [tests/test-vmstate] Error 1
make: *** Waiting for unfinished jobs....
make: *** wait: No child processes. Stop.
Traceback (most recent call last):
File "./tests/docker/docker.py", line 565, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 562, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 308, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 276, in run
quiet=quiet)
File "./tests/docker/docker.py", line 183, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=43a48876970111e8b8c652540069c830', '-u', '1000', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-jlamec93/src/docker-src.2018-08-03-05.40.22.11421:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:213: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-jlamec93/src'
make: *** [tests/docker/Makefile.include:247: docker-run-test-quick@centos7] Error 2
real 2m23.263s
user 0m4.298s
sys 0m3.686s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
` (12 preceding siblings ...)
2018-08-03 9:42 ` [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration no-reply
@ 2018-08-06 6:09 ` 858585 jemmy
13 siblings, 0 replies; 16+ messages in thread
From: 858585 jemmy @ 2018-08-06 6:09 UTC (permalink / raw)
To: zhang.zhanghailiang, Juan Quintela, Dave Gilbert; +Cc: qemu-devel, Lidong Chen
There is one compile error, please ignore those patch, I will send a
new version patch.
On Fri, Aug 3, 2018 at 5:13 PM, Lidong Chen <jemmy858585@gmail.com> wrote:
> The RDMA QIOChannel does not support bi-directional communication, so when RDMA
> live migration with postcopy enabled, the source qemu return path get qemu file
> error.
>
> These patches implement bi-directional communication for RDMA QIOChannel and
> disable the RDMA WRITE during the postcopy phase.
>
> This patch just make postcopy works, and will improve performance later.
>
> [v6]
> - rebase
> - add the check whether release rdma resource has finished(David)
> - remove unnecessary RDMA_CONTROL_ERROR when cleanup(David)
> - poll the cm event for destination qemu
>
> [v5]
> - rebase
> - fix bug for create a dedicated thread to release rdma resource(David)
> - fix bug for poll the cm event while wait RDMA work request completion(David,Gal)
>
> [v4]
> - not wait RDMA_CM_EVENT_DISCONNECTED event after rdma_disconnect
> - implement io_set_aio_fd_handler function for RDMA QIOChannel (Juan Quintela)
> - invoke qio_channel_yield only when qemu_in_coroutine() (Juan Quintela)
> - create a dedicated thread to release rdma resource
> - poll the cm event while wait RDMA work request completion
> - implement the shutdown function for RDMA QIOChannel
>
> [v3]
> - add a mutex in QEMUFile struct to avoid concurrent channel close (Daniel)
> - destroy the mutex before free QEMUFile (David)
> - use rdmain and rmdaout instead of rdma->return_path (Daniel)
>
> [v2]
> - does not update bytes_xfer when disable RDMA WRITE (David)
> - implement bi-directional communication for RDMA QIOChannel (Daniel)
>
> Lidong Chen (12):
> migration: disable RDMA WRITE after postcopy started
> migration: create a dedicated connection for rdma return path
> migration: avoid concurrent invoke channel_close by different threads
> migration: implement bi-directional RDMA QIOChannel
> migration: Stop rdma yielding during incoming postcopy
> migration: implement io_set_aio_fd_handler function for RDMA
> QIOChannel
> migration: invoke qio_channel_yield only when qemu_in_coroutine()
> migration: poll the cm event while wait RDMA work request completion
> migration: implement the shutdown for RDMA QIOChannel
> migration: poll the cm event for destination qemu
> migration: remove the unnecessary RDMA_CONTROL_ERROR message
> migration: create a dedicated thread to release rdma resource
>
> migration/colo.c | 2 +
> migration/migration.c | 13 +-
> migration/migration.h | 10 +
> migration/postcopy-ram.c | 2 +
> migration/qemu-file-channel.c | 12 +-
> migration/qemu-file.c | 14 +-
> migration/ram.c | 4 +
> migration/rdma.c | 448 ++++++++++++++++++++++++++++++++++++++----
> migration/savevm.c | 3 +
> 9 files changed, 458 insertions(+), 50 deletions(-)
>
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH v6 03/12] migration: avoid concurrent invoke channel_close by different threads
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 03/12] migration: avoid concurrent invoke channel_close by different threads Lidong Chen
@ 2018-08-06 9:07 ` 858585 jemmy
0 siblings, 0 replies; 16+ messages in thread
From: 858585 jemmy @ 2018-08-06 9:07 UTC (permalink / raw)
To: zhang.zhanghailiang, Juan Quintela, Dave Gilbert, Daniel P. Berrange
Cc: qemu-devel, Lidong Chen, Lidong Chen
This patch causes compile error when make check.
LINK tests/test-qdist
migration/qemu-file.o: In function `qemu_fclose':
/tmp/qemu-test/src/migration/qemu-file.c:331: undefined reference to
`migrate_get_current'
/tmp/qemu-test/src/migration/qemu-file.c:333: undefined reference to
`migrate_get_current'
collect2: error: ld returned 1 exit status
make: *** [tests/test-vmstate] Error 1
but I don't find an efficient way to fix it.
so I prefer to remove this patch from this series.
maybe it's not the best way to protect by
migrate_get_current()->qemu_file_close_lock.
On Fri, Aug 3, 2018 at 5:13 PM, Lidong Chen <jemmy858585@gmail.com> wrote:
> From: Lidong Chen <jemmy858585@gmail.com>
>
> The channel_close maybe invoked by different threads. For example, source
> qemu invokes qemu_fclose in main thread, migration thread and return path
> thread. Destination qemu invokes qemu_fclose in main thread, listen thread
> and COLO incoming thread.
>
> Signed-off-by: Lidong Chen <lidongchen@tencent.com>
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
> migration/migration.c | 2 ++
> migration/migration.h | 7 +++++++
> migration/qemu-file.c | 6 ++++--
> 3 files changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/migration/migration.c b/migration/migration.c
> index b7d9854..a3a0756 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -3200,6 +3200,7 @@ static void migration_instance_finalize(Object *obj)
> qemu_sem_destroy(&ms->postcopy_pause_sem);
> qemu_sem_destroy(&ms->postcopy_pause_rp_sem);
> qemu_sem_destroy(&ms->rp_state.rp_sem);
> + qemu_mutex_destroy(&ms->qemu_file_close_lock);
> error_free(ms->error);
> }
>
> @@ -3236,6 +3237,7 @@ static void migration_instance_init(Object *obj)
> qemu_sem_init(&ms->rp_state.rp_sem, 0);
> qemu_sem_init(&ms->rate_limit_sem, 0);
> qemu_mutex_init(&ms->qemu_file_lock);
> + qemu_mutex_init(&ms->qemu_file_close_lock);
> }
>
> /*
> diff --git a/migration/migration.h b/migration/migration.h
> index 64a7b33..a50c2de 100644
> --- a/migration/migration.h
> +++ b/migration/migration.h
> @@ -122,6 +122,13 @@ struct MigrationState
> QemuMutex qemu_file_lock;
>
> /*
> + * The to_src_file and from_dst_file point to one QIOChannelRDMA,
> + * And qemu_fclose maybe invoked by different threads. use this lock
> + * to avoid concurrent invoke channel_close by different threads.
> + */
> + QemuMutex qemu_file_close_lock;
> +
> + /*
> * Used to allow urgent requests to override rate limiting.
> */
> QemuSemaphore rate_limit_sem;
> diff --git a/migration/qemu-file.c b/migration/qemu-file.c
> index 977b9ae..74c48e0 100644
> --- a/migration/qemu-file.c
> +++ b/migration/qemu-file.c
> @@ -323,12 +323,14 @@ void qemu_update_position(QEMUFile *f, size_t size)
> */
> int qemu_fclose(QEMUFile *f)
> {
> - int ret;
> + int ret, ret2;
> qemu_fflush(f);
> ret = qemu_file_get_error(f);
>
> if (f->ops->close) {
> - int ret2 = f->ops->close(f->opaque);
> + qemu_mutex_lock(&migrate_get_current()->qemu_file_close_lock);
> + ret2 = f->ops->close(f->opaque);
> + qemu_mutex_unlock(&migrate_get_current()->qemu_file_close_lock);
> if (ret >= 0) {
> ret = ret2;
> }
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2018-08-06 17:32 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-03 9:13 [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 01/12] migration: disable RDMA WRITE after postcopy started Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 02/12] migration: create a dedicated connection for rdma return path Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 03/12] migration: avoid concurrent invoke channel_close by different threads Lidong Chen
2018-08-06 9:07 ` 858585 jemmy
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 04/12] migration: implement bi-directional RDMA QIOChannel Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 05/12] migration: Stop rdma yielding during incoming postcopy Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 06/12] migration: implement io_set_aio_fd_handler function for RDMA QIOChannel Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 07/12] migration: invoke qio_channel_yield only when qemu_in_coroutine() Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 08/12] migration: poll the cm event while wait RDMA work request completion Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 09/12] migration: implement the shutdown for RDMA QIOChannel Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 10/12] migration: poll the cm event for destination qemu Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 11/12] migration: remove the unnecessary RDMA_CONTROL_ERROR message Lidong Chen
2018-08-03 9:13 ` [Qemu-devel] [PATCH v6 12/12] migration: create a dedicated thread to release rdma resource Lidong Chen
2018-08-03 9:42 ` [Qemu-devel] [PATCH v6 00/12] Enable postcopy RDMA live migration no-reply
2018-08-06 6:09 ` 858585 jemmy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.