From: Zheng Chuan <zhengchuan@huawei.com>
To: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: yubihong@huawei.com, berrange@redhat.com,
zhang.zhanghailiang@huawei.com, quintela@redhat.com,
qemu-devel@nongnu.org, xiexiangyou@huawei.com,
alex.chen@huawei.com, wanghao232@huawei.com
Subject: Re: [PATCH v4 12/18] migration/rdma: Create the multifd send channels for RDMA
Date: Mon, 1 Mar 2021 20:26:54 +0800 [thread overview]
Message-ID: <0e2cd012-a48b-ffd5-0c6f-0a5b146c84d6@huawei.com> (raw)
In-Reply-To: <20210203195212.GV2950@work-vm>
On 2021/2/4 3:52, Dr. David Alan Gilbert wrote:
> * Chuan Zheng (zhengchuan@huawei.com) wrote:
>> Signed-off-by: Chuan Zheng <zhengchuan@huawei.com>
>> ---
>> migration/multifd.c | 4 ++--
>> migration/multifd.h | 2 ++
>> migration/rdma.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 61 insertions(+), 2 deletions(-)
>>
>> diff --git a/migration/multifd.c b/migration/multifd.c
>> index ae0b7f0..919a414 100644
>> --- a/migration/multifd.c
>> +++ b/migration/multifd.c
>> @@ -176,7 +176,7 @@ void multifd_register_ops(int method, MultiFDMethods *ops)
>> multifd_ops[method] = ops;
>> }
>>
>> -static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
>> +int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp)
>> {
>> MultiFDInit_t msg = {};
>> int ret;
>> @@ -503,7 +503,7 @@ int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset)
>> return 1;
>> }
>>
>> -static void multifd_send_terminate_threads(Error *err)
>> +void multifd_send_terminate_threads(Error *err)
>> {
>> int i;
>>
>> diff --git a/migration/multifd.h b/migration/multifd.h
>> index b17a2c1..26d4489 100644
>> --- a/migration/multifd.h
>> +++ b/migration/multifd.h
>> @@ -184,6 +184,8 @@ typedef struct {
>> #ifdef CONFIG_RDMA
>> extern MultiFDSetup multifd_rdma_ops;
>> #endif
>> +void multifd_send_terminate_threads(Error *err);
>> +int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp);
>> int get_multifd_send_param(int id, MultiFDSendParams **param);
>> int get_multifd_recv_param(int id, MultiFDRecvParams **param);
>> MultiFDSetup *multifd_setup_ops_init(void);
>> diff --git a/migration/rdma.c b/migration/rdma.c
>> index 9654b87..cff9446 100644
>> --- a/migration/rdma.c
>> +++ b/migration/rdma.c
>> @@ -4261,9 +4261,54 @@ err:
>> g_free(rdma_return_path);
>> }
>>
>> +static int multifd_channel_rdma_connect(void *opaque)
>> +{
>> + MultiFDSendParams *p = opaque;
>> + Error *local_err = NULL;
>> + int ret = 0;
>> + MigrationState *s = migrate_get_current();
>> +
>> + p->rdma = qemu_rdma_data_init(s->host_port, &local_err);
>> + if (p->rdma == NULL) {
>> + goto out;
>> + }
>> +
>> + ret = qemu_rdma_source_init(p->rdma,
>> + migrate_rdma_pin_all(),
>> + &local_err);
>> + if (ret) {
>> + goto out;
>> + }
>> +
>> + ret = qemu_rdma_connect(p->rdma, &local_err);
>> + if (ret) {
>> + goto out;
>> + }
>> +
>> + p->file = qemu_fopen_rdma(p->rdma, "wb");
>> + if (p->file == NULL) {
>> + goto out;
>> + }
>> +
>> + p->c = QIO_CHANNEL(getQIOChannel(p->file));
>> +
>> +out:
>> + if (local_err) {
>> + trace_multifd_send_error(p->id);
>> + }
>
> If any of the previous steps have failed, but not the first step,
> what cleans up?
>
Yes,Sorry for that. I'll add cleanup in next v5.
>> + return ret;
>> +}
>> +
>> static void *multifd_rdma_send_thread(void *opaque)
>> {
>> MultiFDSendParams *p = opaque;
>> + Error *local_err = NULL;
>> +
>> + trace_multifd_send_thread_start(p->id);
>> + if (multifd_send_initial_packet(p, &local_err) < 0) {
>> + goto out;
>> + }
>>
>> while (true) {
>> WITH_QEMU_LOCK_GUARD(&p->mutex) {
>> @@ -4274,6 +4319,12 @@ static void *multifd_rdma_send_thread(void *opaque)
>> qemu_sem_wait(&p->sem);
>> }
>>
>> +out:
>> + if (local_err) {
>> + trace_multifd_send_error(p->id);
>> + multifd_send_terminate_threads(local_err);
>> + }
>> +
>> WITH_QEMU_LOCK_GUARD(&p->mutex) {
>> p->running = false;
>> }
>> @@ -4285,6 +4336,12 @@ static void multifd_rdma_send_channel_setup(MultiFDSendParams *p)
>> {
>> Error *local_err = NULL;
>>
>> + if (multifd_channel_rdma_connect(p)) {
>> + error_setg(&local_err, "multifd: rdma channel %d not established",
>> + p->id);
>> + return ;
>> + }
>> +
>> if (p->quit) {
>> error_setg(&local_err, "multifd: send id %d already quit", p->id);
>> return ;
>> --
>> 1.8.3.1
>>
--
Regards.
Chuan
next prev parent reply other threads:[~2021-03-01 12:31 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-03 8:01 [PATCH v4 00/18] Support Multifd for RDMA migration Chuan Zheng
2021-02-03 8:01 ` [PATCH v4 01/18] migration/rdma: add the 'migrate_rdma_pin_all' function Chuan Zheng
2021-02-03 8:01 ` [PATCH v4 02/18] migration/rdma: judge whether or not the RDMA is used for migration Chuan Zheng
2021-02-03 17:49 ` Dr. David Alan Gilbert
2021-03-01 12:25 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 03/18] migration/rdma: create multifd_setup_ops for Tx/Rx thread Chuan Zheng
2021-02-03 8:01 ` [PATCH v4 04/18] migration/rdma: add multifd_setup_ops for rdma Chuan Zheng
2021-02-03 17:58 ` Dr. David Alan Gilbert
2021-02-03 8:01 ` [PATCH v4 05/18] migration/rdma: do not need sync main " Chuan Zheng
2021-02-03 18:10 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 06/18] migration/rdma: export MultiFDSendParams/MultiFDRecvParams Chuan Zheng
2021-02-03 18:23 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 07/18] migration/rdma: add rdma field into multifd send/recv param Chuan Zheng
2021-02-03 18:32 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 08/18] migration/rdma: export getQIOChannel to get QIOchannel in rdma Chuan Zheng
2021-02-03 18:49 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 09/18] migration/rdma: add multifd_rdma_load_setup() to setup multifd rdma Chuan Zheng
2021-02-03 8:01 ` [PATCH v4 10/18] migration/rdma: Create the multifd recv channels for RDMA Chuan Zheng
2021-02-03 18:59 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 11/18] migration/rdma: record host_port for multifd RDMA Chuan Zheng
2021-02-03 19:04 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 12/18] migration/rdma: Create the multifd send channels for RDMA Chuan Zheng
2021-02-03 19:52 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan [this message]
2021-02-03 8:01 ` [PATCH v4 13/18] migration/rdma: Add the function for dynamic page registration Chuan Zheng
2021-02-03 20:06 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 14/18] migration/rdma: register memory for multifd RDMA channels Chuan Zheng
2021-02-03 20:12 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 15/18] migration/rdma: only register the memory for multifd channels Chuan Zheng
2021-02-04 10:09 ` Dr. David Alan Gilbert
2021-02-03 8:01 ` [PATCH v4 16/18] migration/rdma: add rdma_channel into Migrationstate field Chuan Zheng
2021-02-03 20:19 ` Dr. David Alan Gilbert
2021-03-01 12:27 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 17/18] migration/rdma: send data for both rdma-pin-all and NOT rdma-pin-all mode Chuan Zheng
2021-02-04 10:18 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 18/18] migration/rdma: RDMA cleanup for multifd migration Chuan Zheng
2021-02-04 10:32 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0e2cd012-a48b-ffd5-0c6f-0a5b146c84d6@huawei.com \
--to=zhengchuan@huawei.com \
--cc=alex.chen@huawei.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=wanghao232@huawei.com \
--cc=xiexiangyou@huawei.com \
--cc=yubihong@huawei.com \
--cc=zhang.zhanghailiang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).