From: huangy81@chinatelecom.cn
To: qemu-devel <qemu-devel@nongnu.org>
Cc: "Peter Xu" <peterx@redhat.com>,
"Juan Quintela" <quintela@redhat.com>,
"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
"Eric Blake" <eblake@redhat.com>,
"Markus Armbruster" <armbru@redhat.com>,
"Thomas Huth" <thuth@redhat.com>,
"Laurent Vivier" <lvivier@redhat.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Daniel P. Berrange" <berrange@redhat.com>,
"Hyman Huang(黄勇)" <huangy81@chinatelecom.cn>
Subject: [PATCH v1 0/8] migration: introduce dirtylimit capability
Date: Fri, 2 Sep 2022 01:22:28 +0800 [thread overview]
Message-ID: <cover.1662052189.git.huangy81@chinatelecom.cn> (raw)
From: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
v1:
- make parameter vcpu-dirty-limit experimental
- switch dirty limit off when cancel migrate
- add cancel logic in migration test
Please review, thanks,
Yong
Abstract
========
This series added a new migration capability called "dirtylimit". It can
be enabled when dirty ring is enabled, and it'll improve the vCPU performance
during the process of migration. It is based on the previous patchset:
https://lore.kernel.org/qemu-devel/cover.1656177590.git.huangy81@chinatelecom.cn/
As mentioned in patchset "support dirty restraint on vCPU", dirtylimit way of
migration can make the read-process not be penalized. This series wires up the
vcpu dirty limit and wrappers as dirtylimit capability of migration. I introduce
two parameters vcpu-dirtylimit-period and vcpu-dirtylimit to implement the setup
of dirtylimit during live migration.
To validate the implementation, i tested a 32 vCPU vm live migration with such
model:
Only dirty vcpu0, vcpu1 with heavy memory workoad and leave the rest vcpus
untouched, running unixbench on the vpcu8-vcpu15 by setup the cpu affinity as
the following command:
taskset -c 8-15 ./Run -i 2 -c 8 {unixbench test item}
The following are results:
host cpu: Intel(R) Xeon(R) Platinum 8378A
host interface speed: 1000Mb/s
|---------------------+--------+------------+---------------|
| UnixBench test item | Normal | Dirtylimit | Auto-converge |
|---------------------+--------+------------+---------------|
| dhry2reg | 32800 | 32786 | 25292 |
| whetstone-double | 10326 | 10315 | 9847 |
| pipe | 15442 | 15271 | 14506 |
| context1 | 7260 | 6235 | 4514 |
| spawn | 3663 | 3317 | 3249 |
| syscall | 4669 | 4667 | 3841 |
|---------------------+--------+------------+---------------|
From the data above we can draw a conclusion that vcpus that do not dirty memory
in vm are almost unaffected during the dirtylimit migration, but the auto converge
way does.
I also tested the total time of dirtylimit migration with variable dirty memory
size in vm.
senario 1:
host cpu: Intel(R) Xeon(R) Platinum 8378A
host interface speed: 1000Mb/s
|-----------------------+----------------+-------------------|
| dirty memory size(MB) | Dirtylimit(ms) | Auto-converge(ms) |
|-----------------------+----------------+-------------------|
| 60 | 2014 | 2131 |
| 70 | 5381 | 12590 |
| 90 | 6037 | 33545 |
| 110 | 7660 | [*] |
|-----------------------+----------------+-------------------|
[*]: This case means migration is not convergent.
senario 2:
host cpu: Intel(R) Xeon(R) CPU E5-2650
host interface speed: 10000Mb/s
|-----------------------+----------------+-------------------|
| dirty memory size(MB) | Dirtylimit(ms) | Auto-converge(ms) |
|-----------------------+----------------+-------------------|
| 1600 | 15842 | 27548 |
| 2000 | 19026 | 38447 |
| 2400 | 19897 | 46381 |
| 2800 | 22338 | 57149 |
|-----------------------+----------------+-------------------|
Above data shows that dirtylimit way of migration can also reduce the total
time of migration and it achieves convergence more easily in some case.
In addition to implement dirtylimit capability itself, this series
add 3 tests for migration, aiming at playing around for developer simply:
1. qtest for dirty limit migration
2. support dirty ring way of migration for guestperf tool
3. support dirty limit migration for guestperf tool
Please review, thanks !
Hyman Huang (8):
qapi/migration: Introduce x-vcpu-dirty-limit-period parameter
qapi/migration: Introduce x-vcpu-dirty-limit parameters
migration: Introduce dirty-limit capability
migration: Implement dirty-limit convergence algo
migration: Export dirty-limit time info
tests: Add migration dirty-limit capability test
tests/migration: Introduce dirty-ring-size option into guestperf
tests/migration: Introduce dirty-limit into guestperf
include/sysemu/dirtylimit.h | 2 +
migration/migration.c | 51 +++++++++++
migration/migration.h | 1 +
migration/ram.c | 53 ++++++++---
migration/trace-events | 1 +
monitor/hmp-cmds.c | 26 ++++++
qapi/migration.json | 57 ++++++++++--
softmmu/dirtylimit.c | 33 ++++++-
tests/migration/guestperf/comparison.py | 24 +++++
tests/migration/guestperf/engine.py | 33 ++++++-
tests/migration/guestperf/hardware.py | 8 +-
tests/migration/guestperf/progress.py | 17 +++-
tests/migration/guestperf/scenario.py | 11 ++-
tests/migration/guestperf/shell.py | 25 +++++-
tests/qtest/migration-test.c | 154 ++++++++++++++++++++++++++++++++
15 files changed, 465 insertions(+), 31 deletions(-)
--
1.8.3.1
next reply other threads:[~2022-09-01 17:45 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-01 17:22 huangy81 [this message]
2022-09-01 17:22 ` [PATCH v1 1/8] qapi/migration: Introduce x-vcpu-dirty-limit-period parameter huangy81
2022-09-02 8:02 ` Markus Armbruster
2022-09-01 17:22 ` [PATCH v1 2/8] qapi/migration: Introduce x-vcpu-dirty-limit parameters huangy81
2022-09-02 8:03 ` Markus Armbruster
2022-09-02 13:27 ` Hyman Huang
2022-09-01 17:22 ` [PATCH v1 3/8] migration: Introduce dirty-limit capability huangy81
2022-09-02 8:07 ` Markus Armbruster
2022-09-02 14:15 ` Hyman Huang
2022-09-05 9:32 ` Markus Armbruster
2022-09-05 13:13 ` Hyman Huang
2022-09-06 8:02 ` Markus Armbruster
2022-09-01 17:22 ` [PATCH v1 4/8] migration: Implement dirty-limit convergence algo huangy81
2022-09-06 20:37 ` Peter Xu
2022-09-08 14:35 ` Hyman
2022-09-08 14:47 ` Peter Xu
2022-09-08 14:59 ` Hyman Huang
2022-09-01 17:22 ` [PATCH v1 5/8] migration: Export dirty-limit time info huangy81
2022-10-01 18:31 ` Markus Armbruster
2022-10-02 1:13 ` Hyman Huang
2022-10-07 15:09 ` Markus Armbruster
2022-10-07 16:22 ` Hyman Huang
2022-09-01 17:22 ` [PATCH v1 6/8] tests: Add migration dirty-limit capability test huangy81
2022-09-01 17:22 ` [PATCH v1 7/8] tests/migration: Introduce dirty-ring-size option into guestperf huangy81
2022-09-01 17:22 ` [PATCH v1 8/8] tests/migration: Introduce dirty-limit " huangy81
2022-09-06 20:46 ` [PATCH v1 0/8] migration: introduce dirtylimit capability Peter Xu
2022-09-07 14:52 ` Hyman
2022-10-01 14:37 ` Markus Armbruster
2022-10-01 15:01 ` Hyman Huang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1662052189.git.huangy81@chinatelecom.cn \
--to=huangy81@chinatelecom.cn \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=eblake@redhat.com \
--cc=lvivier@redhat.com \
--cc=pbonzini@redhat.com \
--cc=peterx@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.