From: guangrong.xiao@gmail.com
To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com
Cc: kvm@vger.kernel.org, quintela@redhat.com,
Xiao Guangrong <xiaoguangrong@tencent.com>,
qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com,
wei.w.wang@intel.com, cota@braap.org, jiang.biao2@zte.com.cn
Subject: [PATCH v3 0/5] migration: improve multithreads
Date: Thu, 22 Nov 2018 15:20:23 +0800 [thread overview]
Message-ID: <20181122072028.22819-1-xiaoguangrong@tencent.com> (raw)
From: Xiao Guangrong <xiaoguangrong@tencent.com>
Changelog in v3:
Thanks to Emilio's comments and his example code, the changes in
this version are:
1. move @requests from the shared data struct to each single thread
2. move completion ev from the shared data struct to each single thread
3. move bitmaps from the shared data struct to each single thread
4. limit the number of request that each thread need handle to 64, then
use uint64_t instead of bitmap pointer.
The performance is measured by using the benchmark we introduced in
this pachset:
./tests/threaded-workqueue-bench -c 20 -m 16 -t N
the data is as followings:
The baseline of v2:
Thread #, Throughput
1, 0.428024
4, 1.668876
8, 3.501940
12, 5.026403
16, 1.912374
20, 1.174771
24, 1.074085
28, 0.747920
32, 0.651409
36, 0.533240
40, 0.517421
44, 0.482153
48, 0.525176
52, 0.492677
56, 0.798679
60, 0.733868
64, 0.751396
After this patchset:
Thread #, Throughput
1, 0.449192
4, 1.849271
8, 3.644339
12, 4.809391
16, 4.709095
20, 4.942153
24, 5.116967
28, 4.921542
32, 5.008816
36, 5.408070
40, 5.166064
44, 4.994953
48, 4.853351
52, 4.797540
56, 4.815153
60, 4.793704
64, 4.913544
To see more detailed compression by each step, please refer to
https://ibb.co/hq7u5V
Xiao Guangrong (5):
bitops: introduce change_bit_atomic
util: introduce threaded workqueue
migration: use threaded workqueue for compression
migration: use threaded workqueue for decompression
tests: add threaded-workqueue-bench
include/qemu/bitops.h | 13 +
include/qemu/threaded-workqueue.h | 106 ++++++++
migration/ram.c | 530 ++++++++++++++------------------------
tests/Makefile.include | 5 +-
tests/threaded-workqueue-bench.c | 255 ++++++++++++++++++
util/Makefile.objs | 1 +
util/threaded-workqueue.c | 463 +++++++++++++++++++++++++++++++++
7 files changed, 1029 insertions(+), 344 deletions(-)
create mode 100644 include/qemu/threaded-workqueue.h
create mode 100644 tests/threaded-workqueue-bench.c
create mode 100644 util/threaded-workqueue.c
--
2.14.5
next reply other threads:[~2018-11-22 7:20 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-22 7:20 guangrong.xiao [this message]
2018-11-22 7:20 ` [PATCH v3 1/5] bitops: introduce change_bit_atomic guangrong.xiao
2018-11-23 10:23 ` Dr. David Alan Gilbert
2018-11-28 9:35 ` Juan Quintela
2018-11-22 7:20 ` [PATCH v3 2/5] util: introduce threaded workqueue guangrong.xiao
2018-11-23 11:02 ` Dr. David Alan Gilbert
2018-11-26 7:57 ` Xiao Guangrong
2018-11-26 10:56 ` Dr. David Alan Gilbert
2018-11-27 7:17 ` Xiao Guangrong
2018-11-26 18:55 ` Emilio G. Cota
2018-11-27 8:30 ` Xiao Guangrong
2018-11-24 0:12 ` Emilio G. Cota
2018-11-26 8:06 ` Xiao Guangrong
2018-11-26 18:49 ` Emilio G. Cota
2018-11-27 8:29 ` Xiao Guangrong
2018-11-24 0:17 ` Emilio G. Cota
2018-11-26 8:18 ` Xiao Guangrong
2018-11-26 10:28 ` Paolo Bonzini
2018-11-27 8:31 ` Xiao Guangrong
2018-11-27 12:49 ` Christophe de Dinechin
2018-11-27 13:51 ` Paolo Bonzini
2018-12-04 15:49 ` Christophe de Dinechin
2018-12-04 17:16 ` Paolo Bonzini
2018-12-10 3:23 ` Xiao Guangrong
2018-11-27 17:39 ` Emilio G. Cota
2018-11-28 8:55 ` Xiao Guangrong
2018-11-22 7:20 ` [PATCH v3 3/5] migration: use threaded workqueue for compression guangrong.xiao
2018-11-23 18:17 ` Dr. David Alan Gilbert
2018-11-23 18:22 ` Paolo Bonzini
2018-11-23 18:29 ` Dr. David Alan Gilbert
2018-11-26 8:00 ` Xiao Guangrong
2018-11-22 7:20 ` [PATCH v3 4/5] migration: use threaded workqueue for decompression guangrong.xiao
2018-11-22 7:20 ` [PATCH v3 5/5] tests: add threaded-workqueue-bench guangrong.xiao
2018-11-22 21:25 ` [PATCH v3 0/5] migration: improve multithreads no-reply
2018-11-22 21:35 ` no-reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181122072028.22819-1-xiaoguangrong@tencent.com \
--to=guangrong.xiao@gmail.com \
--cc=cota@braap.org \
--cc=dgilbert@redhat.com \
--cc=jiang.biao2@zte.com.cn \
--cc=kvm@vger.kernel.org \
--cc=mst@redhat.com \
--cc=mtosatti@redhat.com \
--cc=pbonzini@redhat.com \
--cc=peterx@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=wei.w.wang@intel.com \
--cc=xiaoguangrong@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).