* [PATCH v2 0/6] migration: Multi-thread compression method support
@ 2020-11-27 9:36 Zeyu Jin
2020-11-30 16:42 ` Eric Blake
0 siblings, 1 reply; 3+ messages in thread
From: Zeyu Jin @ 2020-11-27 9:36 UTC (permalink / raw)
To: quintela, dgilbert; +Cc: qemu-devel, Zeyu Jin, zhang.zhanghailiang
Currently we have both multi-thread compression and multifd to optimize
live migration in Qemu. Mulit-thread compression deals with the situation
where network bandwith is limited but cpu resource adequate. Multifd instead
aims to take full advantage of network bandwith. Moreover it supports both
zlib and zstd compression on each channel.
In this patch series, we did some code refactoring on multi-thread compression
live migration and bring zstd compression method support for it.
Below is the test result of multi-thread compression live migration
with different compress methods. Test result shows that zstd outperforms
zlib by about 70%.
Migration Configuration:
Guest 8U 32G
compress-threads 8
decompress-threads 2
compress-level 1
bandwidth-limit 100Mbps
Test Result:
+---------------------+--------------+-------------+
| compress method | zlib | zstd |
+---------------------+--------------+-------------+
| total time (ms) | 75256 | 44187 |
+---------------------+--------------+-------------+
| downtime(ms) | 128 | 81 |
+---------------------+--------------+-------------+
| transferred ram(kB)| 1576866 | 736117 |
+---------------------+--------------+-------------+
| throughput(mbps) | 172.06 | 137.16 |
+---------------------+--------------+-------------+
| total ram(kB) | 33685952 | 33685952 |
+---------------------+--------------+-------------+
Zeyu Jin (6):
migration: Add multi-thread compress method
migration: Refactoring multi-thread compress migration
migration: Add multi-thread compress ops
migration: Add zstd support in multi-thread compression
migration: Add compress_level sanity check
doc: Update multi-thread compression doc
docs/multi-thread-compression.txt | 31 ++-
hw/core/qdev-properties-system.c | 11 +
include/hw/qdev-properties.h | 4 +
migration/migration.c | 56 ++++-
migration/migration.h | 1 +
migration/qemu-file.c | 62 +----
migration/qemu-file.h | 4 +-
migration/ram.c | 381 +++++++++++++++++++++++++-----
monitor/hmp-cmds.c | 12 +
qapi/migration.json | 26 +-
10 files changed, 465 insertions(+), 123 deletions(-)
--
2.27.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2 0/6] migration: Multi-thread compression method support
2020-11-27 9:36 [PATCH v2 0/6] migration: Multi-thread compression method support Zeyu Jin
@ 2020-11-30 16:42 ` Eric Blake
2020-12-01 6:04 ` Zeyu Jin
0 siblings, 1 reply; 3+ messages in thread
From: Eric Blake @ 2020-11-30 16:42 UTC (permalink / raw)
To: Zeyu Jin, quintela, dgilbert; +Cc: qemu-devel, zhang.zhanghailiang
On 11/27/20 3:36 AM, Zeyu Jin wrote:
Meta-comment: you appear to be having problems threading your series;
I've now seen three separate cover letters (RFC v1, v2 with no subject,
v2 with subject) and two series where each patch was a separate thread.
It is difficult to follow which messages are related when reading in a
mail client that sorts by most-recently-active thread first. You may
want to investigate why your threading is not working, although I'd wait
to send v3 until you have actual changes to incorporate.
> Currently we have both multi-thread compression and multifd to optimize
> live migration in Qemu. Mulit-thread compression deals with the situation
> where network bandwith is limited but cpu resource adequate. Multifd instead
Not that typos in the cover letter matter, but this should be 'bandwidth'
> aims to take full advantage of network bandwith. Moreover it supports both
> zlib and zstd compression on each channel.
>
> In this patch series, we did some code refactoring on multi-thread compression
> live migration and bring zstd compression method support for it.
>
> Below is the test result of multi-thread compression live migration
> with different compress methods. Test result shows that zstd outperforms
> zlib by about 70%.
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2 0/6] migration: Multi-thread compression method support
2020-11-30 16:42 ` Eric Blake
@ 2020-12-01 6:04 ` Zeyu Jin
0 siblings, 0 replies; 3+ messages in thread
From: Zeyu Jin @ 2020-12-01 6:04 UTC (permalink / raw)
To: Eric Blake, quintela, dgilbert; +Cc: qemu-devel, zhang.zhanghailiang
On 2020/12/1 0:42, Eric Blake wrote:
> On 11/27/20 3:36 AM, Zeyu Jin wrote:
>
> Meta-comment: you appear to be having problems threading your series;
> I've now seen three separate cover letters (RFC v1, v2 with no subject,
> v2 with subject) and two series where each patch was a separate thread.
> It is difficult to follow which messages are related when reading in a
> mail client that sorts by most-recently-active thread first. You may
> want to investigate why your threading is not working, although I'd wait
> to send v3 until you have actual changes to incorporate.
>
Thank you for noticing that. It`s my mistake when sending patches.
Everything will be fine in v3.
>> Currently we have both multi-thread compression and multifd to optimize
>> live migration in Qemu. Mulit-thread compression deals with the situation
>> where network bandwith is limited but cpu resource adequate. Multifd instead
>
> Not that typos in the cover letter matter, but this should be 'bandwidth'
>
Yes, I will fix that.
>> aims to take full advantage of network bandwith. Moreover it supports both
>> zlib and zstd compression on each channel.
>>
>> In this patch series, we did some code refactoring on multi-thread compression
>> live migration and bring zstd compression method support for it.
>>
>> Below is the test result of multi-thread compression live migration
>> with different compress methods. Test result shows that zstd outperforms
>> zlib by about 70%.
>>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-12-01 6:05 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-27 9:36 [PATCH v2 0/6] migration: Multi-thread compression method support Zeyu Jin
2020-11-30 16:42 ` Eric Blake
2020-12-01 6:04 ` Zeyu Jin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).