From: Zhimin Feng <fengzhimin1@huawei.com>
To: <quintela@redhat.com>, <dgilbert@redhat.com>, <armbru@redhat.com>,
<eblake@redhat.com>
Cc: jemmy858585@gmail.com, fengzhimin <fengzhimin1@huawei.com>,
qemu-devel@nongnu.org, zhang.zhanghailiang@huawei.com
Subject: [PATCH RFC 09/12] migration/rdma: Wait for all multiRDMA to complete registration
Date: Thu, 9 Jan 2020 12:59:19 +0800 [thread overview]
Message-ID: <20200109045922.904-10-fengzhimin1@huawei.com> (raw)
In-Reply-To: <20200109045922.904-1-fengzhimin1@huawei.com>
From: fengzhimin <fengzhimin1@huawei.com>
Signed-off-by: fengzhimin <fengzhimin1@huawei.com>
---
migration/rdma.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/migration/rdma.c b/migration/rdma.c
index 6ecc870844..425dfa709d 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -439,6 +439,10 @@ struct {
MultiRDMASendParams *params;
/* number of created threads */
int count;
+ /* this mutex protects the following parameters */
+ QemuMutex mutex_sync;
+ /* number of registered multiRDMA channels */
+ unsigned int reg_mr_channels;
/* syncs main thread and channels */
QemuSemaphore sem_sync;
} *multiRDMA_send_state;
@@ -3998,6 +4002,11 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque,
rdma->dest_blocks[i].remote_host_addr;
local->block[i].remote_rkey = rdma->dest_blocks[i].remote_rkey;
}
+
+ /* Wait for all multiRDMA channels to complete registration */
+ if (migrate_use_multiRDMA()) {
+ qemu_sem_wait(&multiRDMA_send_state->sem_sync);
+ }
}
trace_qemu_rdma_registration_stop(flags);
@@ -4562,6 +4571,17 @@ static void *multiRDMA_send_thread(void *opaque)
}
}
+ /*
+ * Inform the main RDMA thread to run when multiRDMA
+ * threads have completed registration.
+ */
+ qemu_mutex_lock(&multiRDMA_send_state->mutex_sync);
+ if (++multiRDMA_send_state->reg_mr_channels ==
+ migrate_multiRDMA_channels()) {
+ qemu_sem_post(&multiRDMA_send_state->sem_sync);
+ }
+ qemu_mutex_unlock(&multiRDMA_send_state->mutex_sync);
+
while (true) {
qemu_sem_wait(&p->sem);
@@ -4616,6 +4636,8 @@ static int multiRDMA_save_setup(const char *host_port, Error **errp)
multiRDMA_send_state->params = g_new0(MultiRDMASendParams,
thread_count);
atomic_set(&multiRDMA_send_state->count, 0);
+ atomic_set(&multiRDMA_send_state->reg_mr_channels, 0);
+ qemu_mutex_init(&multiRDMA_send_state->mutex_sync);
qemu_sem_init(&multiRDMA_send_state->sem_sync, 0);
for (i = 0; i < thread_count; i++) {
@@ -4714,6 +4736,7 @@ int multiRDMA_save_cleanup(void)
g_free(multiRDMA_send_state->params[i].rdma);
}
+ qemu_mutex_destroy(&multiRDMA_send_state->mutex_sync);
qemu_sem_destroy(&multiRDMA_send_state->sem_sync);
g_free(multiRDMA_send_state);
multiRDMA_send_state = NULL;
--
2.19.1
next prev parent reply other threads:[~2020-01-09 10:01 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-09 4:59 [PATCH RFC 00/12] *** mulitple RDMA channels for migration *** Zhimin Feng
2020-01-09 4:59 ` [PATCH RFC 01/12] migration: Add multiRDMA capability support Zhimin Feng
2020-01-13 15:30 ` Markus Armbruster
2020-01-15 1:55 ` fengzhimin
2020-01-13 16:26 ` Eric Blake
2020-01-15 2:04 ` fengzhimin
2020-01-15 18:09 ` Dr. David Alan Gilbert
2020-01-16 13:18 ` Juan Quintela
2020-01-17 1:30 ` fengzhimin
2020-01-09 4:59 ` [PATCH RFC 02/12] migration: Export the 'migration_incoming_setup' function and add the 'migrate_use_rdma_pin_all' function Zhimin Feng
2020-01-15 18:57 ` Dr. David Alan Gilbert
2020-01-16 13:19 ` Juan Quintela
2020-01-09 4:59 ` [PATCH RFC 03/12] migration: Create the multi-rdma-channels parameter Zhimin Feng
2020-01-13 15:34 ` Markus Armbruster
2020-01-15 1:57 ` fengzhimin
2020-01-16 13:20 ` Juan Quintela
2020-01-09 4:59 ` [PATCH RFC 04/12] migration/rdma: Create multiRDMA migration threads Zhimin Feng
2020-01-16 13:25 ` Juan Quintela
2020-01-17 1:32 ` fengzhimin
2020-01-09 4:59 ` [PATCH RFC 05/12] migration/rdma: Create the multiRDMA channels Zhimin Feng
2020-01-15 19:54 ` Dr. David Alan Gilbert
2020-01-16 13:30 ` Juan Quintela
2020-01-09 4:59 ` [PATCH RFC 06/12] migration/rdma: Transmit initial package Zhimin Feng
2020-01-15 18:36 ` Dr. David Alan Gilbert
2020-01-09 4:59 ` [PATCH RFC 07/12] migration/rdma: Be sure all channels are created Zhimin Feng
2020-01-09 4:59 ` [PATCH RFC 08/12] migration/rdma: register memory for multiRDMA channels Zhimin Feng
2020-01-09 4:59 ` Zhimin Feng [this message]
2020-01-09 4:59 ` [PATCH RFC 10/12] migration/rdma: use multiRDMA to send RAM block for rdma-pin-all mode Zhimin Feng
2020-01-09 4:59 ` [PATCH RFC 11/12] migration/rdma: use multiRDMA to send RAM block for NOT " Zhimin Feng
2020-01-09 4:59 ` [PATCH RFC 12/12] migration/rdma: only register the virt-ram block for MultiRDMA Zhimin Feng
2020-01-17 18:52 ` Dr. David Alan Gilbert
2020-01-19 1:44 ` fengzhimin
2020-01-20 9:05 ` Dr. David Alan Gilbert
2020-01-21 1:30 ` fengzhimin
2020-01-09 10:38 ` [PATCH RFC 00/12] *** mulitple RDMA channels for migration *** no-reply
2020-01-15 19:57 ` Dr. David Alan Gilbert
2020-01-16 1:37 ` fengzhimin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200109045922.904-10-fengzhimin1@huawei.com \
--to=fengzhimin1@huawei.com \
--cc=armbru@redhat.com \
--cc=dgilbert@redhat.com \
--cc=eblake@redhat.com \
--cc=jemmy858585@gmail.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=zhang.zhanghailiang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).