From: Chuan Zheng <zhengchuan@huawei.com>
To: <quintela@redhat.com>, <dgilbert@redhat.com>, <berrange@redhat.com>
Cc: yubihong@huawei.com, zhang.zhanghailiang@huawei.com,
qemu-devel@nongnu.org, xiexiangyou@huawei.com,
alex.chen@huawei.com, wanghao232@huawei.com
Subject: [PATCH v4 00/18] Support Multifd for RDMA migration
Date: Wed, 3 Feb 2021 16:01:33 +0800 [thread overview]
Message-ID: <1612339311-114805-1-git-send-email-zhengchuan@huawei.com> (raw)
The RDMA bandwidth is not fully utilized for over 25Gigabit NIC because
of single channel for RDMA migration. This patch series is going to support
multifd for RDMA migration based on multifd framework.
Comparsion is between origion and multifd RDMA migration is re-tested for v3.
The VM specifications for migration are as follows:
- VM use 4k page;
- the number of VCPU is 4;
- the total memory is 16Gigabit;
- use 'mempress' tool to pressurize VM(mempress 8000 500);
- use 25Gigabit network card to migrate;
For origin RDMA and MultiRDMA migration, the total migration times of
VM are as follows:
+++++++++++++++++++++++++++++++++++++++++++++++++
| | NOT rdma-pin-all | rdma-pin-all |
+++++++++++++++++++++++++++++++++++++++++++++++++
| origin RDMA | 26 s | 29 s |
-------------------------------------------------
| MultiRDMA | 16 s | 17 s |
+++++++++++++++++++++++++++++++++++++++++++++++++
Test the multifd RDMA migration like this:
virsh migrate --live --parallel --migrateuri
rdma://192.168.1.100 [VM] --listen-address 0.0.0.0 qemu+tcp://192.168.1.100/system --verbose
v3 -> v4:
modify some function names
export multifd_rdma_ops instead of a function
fix minior codestyle issues
v2 -> v3:
create multifd ops for both tcp and rdma
do not export rdma to avoid multifd code in mess
fix build issue for non-rdma
fix some codestyle and buggy code
Chuan Zheng (18):
migration/rdma: add the 'migrate_rdma_pin_all' function
migration/rdma: judge whether or not the RDMA is used for migration
migration/rdma: create multifd_setup_ops for Tx/Rx thread
migration/rdma: add multifd_setup_ops for rdma
migration/rdma: do not need sync main for rdma
migration/rdma: export MultiFDSendParams/MultiFDRecvParams
migration/rdma: add rdma field into multifd send/recv param
migration/rdma: export getQIOChannel to get QIOchannel in rdma
migration/rdma: add multifd_rdma_load_setup() to setup multifd rdma
migration/rdma: Create the multifd recv channels for RDMA
migration/rdma: record host_port for multifd RDMA
migration/rdma: Create the multifd send channels for RDMA
migration/rdma: Add the function for dynamic page registration
migration/rdma: register memory for multifd RDMA channels
migration/rdma: only register the memory for multifd channels
migration/rdma: add rdma_channel into Migrationstate field
migration/rdma: send data for both rdma-pin-all and NOT rdma-pin-all
mode
migration/rdma: RDMA cleanup for multifd migration
migration/migration.c | 24 +++
migration/migration.h | 11 ++
migration/multifd.c | 97 +++++++++-
migration/multifd.h | 25 +++
migration/qemu-file.c | 5 +
migration/qemu-file.h | 1 +
migration/rdma.c | 490 +++++++++++++++++++++++++++++++++++++++++++++++++-
7 files changed, 641 insertions(+), 12 deletions(-)
--
1.8.3.1
next reply other threads:[~2021-02-03 7:46 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-03 8:01 Chuan Zheng [this message]
2021-02-03 8:01 ` [PATCH v4 01/18] migration/rdma: add the 'migrate_rdma_pin_all' function Chuan Zheng
2021-02-03 8:01 ` [PATCH v4 02/18] migration/rdma: judge whether or not the RDMA is used for migration Chuan Zheng
2021-02-03 17:49 ` Dr. David Alan Gilbert
2021-03-01 12:25 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 03/18] migration/rdma: create multifd_setup_ops for Tx/Rx thread Chuan Zheng
2021-02-03 8:01 ` [PATCH v4 04/18] migration/rdma: add multifd_setup_ops for rdma Chuan Zheng
2021-02-03 17:58 ` Dr. David Alan Gilbert
2021-02-03 8:01 ` [PATCH v4 05/18] migration/rdma: do not need sync main " Chuan Zheng
2021-02-03 18:10 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 06/18] migration/rdma: export MultiFDSendParams/MultiFDRecvParams Chuan Zheng
2021-02-03 18:23 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 07/18] migration/rdma: add rdma field into multifd send/recv param Chuan Zheng
2021-02-03 18:32 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 08/18] migration/rdma: export getQIOChannel to get QIOchannel in rdma Chuan Zheng
2021-02-03 18:49 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 09/18] migration/rdma: add multifd_rdma_load_setup() to setup multifd rdma Chuan Zheng
2021-02-03 8:01 ` [PATCH v4 10/18] migration/rdma: Create the multifd recv channels for RDMA Chuan Zheng
2021-02-03 18:59 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 11/18] migration/rdma: record host_port for multifd RDMA Chuan Zheng
2021-02-03 19:04 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 12/18] migration/rdma: Create the multifd send channels for RDMA Chuan Zheng
2021-02-03 19:52 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 13/18] migration/rdma: Add the function for dynamic page registration Chuan Zheng
2021-02-03 20:06 ` Dr. David Alan Gilbert
2021-03-01 12:26 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 14/18] migration/rdma: register memory for multifd RDMA channels Chuan Zheng
2021-02-03 20:12 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 15/18] migration/rdma: only register the memory for multifd channels Chuan Zheng
2021-02-04 10:09 ` Dr. David Alan Gilbert
2021-02-03 8:01 ` [PATCH v4 16/18] migration/rdma: add rdma_channel into Migrationstate field Chuan Zheng
2021-02-03 20:19 ` Dr. David Alan Gilbert
2021-03-01 12:27 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 17/18] migration/rdma: send data for both rdma-pin-all and NOT rdma-pin-all mode Chuan Zheng
2021-02-04 10:18 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
2021-02-03 8:01 ` [PATCH v4 18/18] migration/rdma: RDMA cleanup for multifd migration Chuan Zheng
2021-02-04 10:32 ` Dr. David Alan Gilbert
2021-03-06 8:45 ` Zheng Chuan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1612339311-114805-1-git-send-email-zhengchuan@huawei.com \
--to=zhengchuan@huawei.com \
--cc=alex.chen@huawei.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=wanghao232@huawei.com \
--cc=xiexiangyou@huawei.com \
--cc=yubihong@huawei.com \
--cc=zhang.zhanghailiang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).