[RFC 0/6] migration: introduce dirtylimit capability

[huangy81@chinatelecom.cn]

From: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>

Abstract
========

This series added a new migration capability called "dirtylimit".  It can
be enabled when dirty ring is enabled, and it'll improve the vCPU performance
during the process of migration. It is based on the previous patchset:
https://lore.kernel.org/qemu-devel/cover.1648748793.git.huangy81@chinatelecom.cn/

As mentioned in patchset "support dirty restraint on vCPU", dirtylimit way of
migration can make the read-process not be penalized. This series wires up the
vcpu dirty limit and wrappers as dirtylimit capability of migration. I introduce
two parameters vcpu-dirtylimit-period and vcpu-dirtylimit to implement the setup 
of dirtylimit during live migration.

To validate the implementation, i tested a 32 vCPU vm live migration with such 
model:
Only dirty vcpu0, vcpu1 with heavy memory workoad and leave the rest vcpus
untouched, running unixbench on the vpcu8-vcpu15 by setup the cpu affinity as
the following command:
taskset -c 8-15 ./Run -i 2 -c 8 {unixbench test item}

The following are results:

host cpu: Intel(R) Xeon(R) Platinum 8378A
host interface speed: 1000Mb/s
  |---------------------+--------+------------+---------------|
  | UnixBench test item | Normal | Dirtylimit | Auto-converge |
  |---------------------+--------+------------+---------------|
  | dhry2reg            | 32800  | 32786      | 25292         |
  | whetstone-double    | 10326  | 10315      | 9847          |
  | pipe                | 15442  | 15271      | 14506         |
  | context1            | 7260   | 6235       | 4514          |
  | spawn               | 3663   | 3317       | 3249          |
  | syscall             | 4669   | 4667       | 3841          |
  |---------------------+--------+------------+---------------|
From the data above we can draw a conclusion that vcpus that do not dirty memory
in vm are almost unaffected during the dirtylimit migration, but the auto converge
way does. 

I also tested the total time of dirtylimit migration with variable dirty memory
size in vm.

senario 1:
host cpu: Intel(R) Xeon(R) Platinum 8378A
host interface speed: 1000Mb/s
  |-----------------------+----------------+-------------------|
  | dirty memory size(MB) | Dirtylimit(ms) | Auto-converge(ms) |
  |-----------------------+----------------+-------------------|
  | 60                    | 2014           | 2131              |
  | 70                    | 5381           | 12590             |
  | 90                    | 6037           | 33545             |
  | 110                   | 7660           | [*]               |
  |-----------------------+----------------+-------------------|
  [*]: This case means migration is not convergent. 

senario 2:
host cpu: Intel(R) Xeon(R) CPU E5-2650
host interface speed: 10000Mb/s
  |-----------------------+----------------+-------------------|
  | dirty memory size(MB) | Dirtylimit(ms) | Auto-converge(ms) |
  |-----------------------+----------------+-------------------|
  | 1600                  | 15842          | 27548             |
  | 2000                  | 19026          | 38447             |
  | 2400                  | 19897          | 46381             |
  | 2800                  | 22338          | 57149             |
  |-----------------------+----------------+-------------------|
Above data shows that dirtylimit way of migration can also reduce the total
time of migration and it achieves convergence more easily in some case.

Since the dependent patchset has not been merged yet, i post the series in the
the following link if anyone want to have a try, once the depenent patchset is
ready and merged, the dirtylimit can be availiable by patching series:
https://github.com/newfriday/qemu/tree/dirtylimit_cap_v1

This series is for RFC, comments and reviews about the API, code logic,
algo implementation or anything else are welcomed and appreciated

Please review, thanks !

Yong 

Hyman Huang (6):
  qapi/migration: Introduce vcpu-dirtylimit-period parameters
  qapi/migration: Introduce vcpu-dirtylimit parameters
  migration: Implement dirtylimit convergence algo
  migration: Introduce dirtylimit capability
  migration: Add dirtylimit data into migration info
  tests: Add migration dirtylimit capability test

 include/sysemu/dirtylimit.h  |  2 +
 migration/migration.c        | 48 ++++++++++++++++++++++++
 migration/migration.h        |  1 +
 migration/ram.c              | 53 +++++++++++++++++++-------
 migration/trace-events       |  1 +
 monitor/hmp-cmds.c           | 15 ++++++++
 qapi/migration.json          | 47 ++++++++++++++++++++---
 softmmu/dirtylimit.c         | 33 +++++++++++++++-
 tests/qtest/migration-test.c | 89 ++++++++++++++++++++++++++++++++++++++++++++
 9 files changed, 269 insertions(+), 20 deletions(-)

-- 
1.8.3.1