From: guangrong.xiao@gmail.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Cc: kvm@vger.kernel.org, quintela@redhat.com, Xiao Guangrong <xiaoguangrong@tencent.com>, qemu-devel@nongnu.org, peterx@redhat.com, dgilbert@redhat.com, wei.w.wang@intel.com, cota@braap.org, jiang.biao2@zte.com.cn Subject: [PATCH v3 0/5] migration: improve multithreads Date: Thu, 22 Nov 2018 15:20:23 +0800 [thread overview] Message-ID: <20181122072028.22819-1-xiaoguangrong@tencent.com> (raw) From: Xiao Guangrong <xiaoguangrong@tencent.com> Changelog in v3: Thanks to Emilio's comments and his example code, the changes in this version are: 1. move @requests from the shared data struct to each single thread 2. move completion ev from the shared data struct to each single thread 3. move bitmaps from the shared data struct to each single thread 4. limit the number of request that each thread need handle to 64, then use uint64_t instead of bitmap pointer. The performance is measured by using the benchmark we introduced in this pachset: ./tests/threaded-workqueue-bench -c 20 -m 16 -t N the data is as followings: The baseline of v2: Thread #, Throughput 1, 0.428024 4, 1.668876 8, 3.501940 12, 5.026403 16, 1.912374 20, 1.174771 24, 1.074085 28, 0.747920 32, 0.651409 36, 0.533240 40, 0.517421 44, 0.482153 48, 0.525176 52, 0.492677 56, 0.798679 60, 0.733868 64, 0.751396 After this patchset: Thread #, Throughput 1, 0.449192 4, 1.849271 8, 3.644339 12, 4.809391 16, 4.709095 20, 4.942153 24, 5.116967 28, 4.921542 32, 5.008816 36, 5.408070 40, 5.166064 44, 4.994953 48, 4.853351 52, 4.797540 56, 4.815153 60, 4.793704 64, 4.913544 To see more detailed compression by each step, please refer to https://ibb.co/hq7u5V Xiao Guangrong (5): bitops: introduce change_bit_atomic util: introduce threaded workqueue migration: use threaded workqueue for compression migration: use threaded workqueue for decompression tests: add threaded-workqueue-bench include/qemu/bitops.h | 13 + include/qemu/threaded-workqueue.h | 106 ++++++++ migration/ram.c | 530 ++++++++++++++------------------------ tests/Makefile.include | 5 +- tests/threaded-workqueue-bench.c | 255 ++++++++++++++++++ util/Makefile.objs | 1 + util/threaded-workqueue.c | 463 +++++++++++++++++++++++++++++++++ 7 files changed, 1029 insertions(+), 344 deletions(-) create mode 100644 include/qemu/threaded-workqueue.h create mode 100644 tests/threaded-workqueue-bench.c create mode 100644 util/threaded-workqueue.c -- 2.14.5
WARNING: multiple messages have this Message-ID (diff)
From: guangrong.xiao@gmail.com To: pbonzini@redhat.com, mst@redhat.com, mtosatti@redhat.com Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, dgilbert@redhat.com, peterx@redhat.com, wei.w.wang@intel.com, jiang.biao2@zte.com.cn, eblake@redhat.com, quintela@redhat.com, cota@braap.org, Xiao Guangrong <xiaoguangrong@tencent.com> Subject: [Qemu-devel] [PATCH v3 0/5] migration: improve multithreads Date: Thu, 22 Nov 2018 15:20:23 +0800 [thread overview] Message-ID: <20181122072028.22819-1-xiaoguangrong@tencent.com> (raw) From: Xiao Guangrong <xiaoguangrong@tencent.com> Changelog in v3: Thanks to Emilio's comments and his example code, the changes in this version are: 1. move @requests from the shared data struct to each single thread 2. move completion ev from the shared data struct to each single thread 3. move bitmaps from the shared data struct to each single thread 4. limit the number of request that each thread need handle to 64, then use uint64_t instead of bitmap pointer. The performance is measured by using the benchmark we introduced in this pachset: ./tests/threaded-workqueue-bench -c 20 -m 16 -t N the data is as followings: The baseline of v2: Thread #, Throughput 1, 0.428024 4, 1.668876 8, 3.501940 12, 5.026403 16, 1.912374 20, 1.174771 24, 1.074085 28, 0.747920 32, 0.651409 36, 0.533240 40, 0.517421 44, 0.482153 48, 0.525176 52, 0.492677 56, 0.798679 60, 0.733868 64, 0.751396 After this patchset: Thread #, Throughput 1, 0.449192 4, 1.849271 8, 3.644339 12, 4.809391 16, 4.709095 20, 4.942153 24, 5.116967 28, 4.921542 32, 5.008816 36, 5.408070 40, 5.166064 44, 4.994953 48, 4.853351 52, 4.797540 56, 4.815153 60, 4.793704 64, 4.913544 To see more detailed compression by each step, please refer to https://ibb.co/hq7u5V Xiao Guangrong (5): bitops: introduce change_bit_atomic util: introduce threaded workqueue migration: use threaded workqueue for compression migration: use threaded workqueue for decompression tests: add threaded-workqueue-bench include/qemu/bitops.h | 13 + include/qemu/threaded-workqueue.h | 106 ++++++++ migration/ram.c | 530 ++++++++++++++------------------------ tests/Makefile.include | 5 +- tests/threaded-workqueue-bench.c | 255 ++++++++++++++++++ util/Makefile.objs | 1 + util/threaded-workqueue.c | 463 +++++++++++++++++++++++++++++++++ 7 files changed, 1029 insertions(+), 344 deletions(-) create mode 100644 include/qemu/threaded-workqueue.h create mode 100644 tests/threaded-workqueue-bench.c create mode 100644 util/threaded-workqueue.c -- 2.14.5
next reply other threads:[~2018-11-22 7:20 UTC|newest] Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-11-22 7:20 guangrong.xiao [this message] 2018-11-22 7:20 ` [Qemu-devel] [PATCH v3 0/5] migration: improve multithreads guangrong.xiao 2018-11-22 7:20 ` [PATCH v3 1/5] bitops: introduce change_bit_atomic guangrong.xiao 2018-11-22 7:20 ` [Qemu-devel] " guangrong.xiao 2018-11-23 10:23 ` Dr. David Alan Gilbert 2018-11-23 10:23 ` [Qemu-devel] " Dr. David Alan Gilbert 2018-11-28 9:35 ` Juan Quintela 2018-11-28 9:35 ` [Qemu-devel] " Juan Quintela 2018-11-22 7:20 ` [PATCH v3 2/5] util: introduce threaded workqueue guangrong.xiao 2018-11-22 7:20 ` [Qemu-devel] " guangrong.xiao 2018-11-23 11:02 ` Dr. David Alan Gilbert 2018-11-23 11:02 ` [Qemu-devel] " Dr. David Alan Gilbert 2018-11-26 7:57 ` Xiao Guangrong 2018-11-26 7:57 ` [Qemu-devel] " Xiao Guangrong 2018-11-26 10:56 ` Dr. David Alan Gilbert 2018-11-26 10:56 ` [Qemu-devel] " Dr. David Alan Gilbert 2018-11-27 7:17 ` Xiao Guangrong 2018-11-27 7:17 ` [Qemu-devel] " Xiao Guangrong 2018-11-26 18:55 ` Emilio G. Cota 2018-11-26 18:55 ` [Qemu-devel] " Emilio G. Cota 2018-11-27 8:30 ` Xiao Guangrong 2018-11-27 8:30 ` [Qemu-devel] " Xiao Guangrong 2018-11-24 0:12 ` Emilio G. Cota 2018-11-24 0:12 ` [Qemu-devel] " Emilio G. Cota 2018-11-26 8:06 ` Xiao Guangrong 2018-11-26 8:06 ` [Qemu-devel] " Xiao Guangrong 2018-11-26 18:49 ` Emilio G. Cota 2018-11-26 18:49 ` [Qemu-devel] " Emilio G. Cota 2018-11-27 8:29 ` Xiao Guangrong 2018-11-27 8:29 ` [Qemu-devel] " Xiao Guangrong 2018-11-24 0:17 ` Emilio G. Cota 2018-11-24 0:17 ` [Qemu-devel] " Emilio G. Cota 2018-11-26 8:18 ` Xiao Guangrong 2018-11-26 8:18 ` [Qemu-devel] " Xiao Guangrong 2018-11-26 10:28 ` Paolo Bonzini 2018-11-26 10:28 ` [Qemu-devel] " Paolo Bonzini 2018-11-27 8:31 ` Xiao Guangrong 2018-11-27 8:31 ` [Qemu-devel] " Xiao Guangrong 2018-11-27 12:49 ` Christophe de Dinechin 2018-11-27 12:49 ` [Qemu-devel] " Christophe de Dinechin 2018-11-27 13:51 ` Paolo Bonzini 2018-11-27 13:51 ` [Qemu-devel] " Paolo Bonzini 2018-12-04 15:49 ` Christophe de Dinechin 2018-12-04 15:49 ` [Qemu-devel] " Christophe de Dinechin 2018-12-04 17:16 ` Paolo Bonzini 2018-12-04 17:16 ` [Qemu-devel] " Paolo Bonzini 2018-12-10 3:23 ` Xiao Guangrong 2018-12-10 3:23 ` [Qemu-devel] " Xiao Guangrong 2018-11-27 17:39 ` Emilio G. Cota 2018-11-27 17:39 ` [Qemu-devel] " Emilio G. Cota 2018-11-28 8:55 ` Xiao Guangrong 2018-11-28 8:55 ` [Qemu-devel] " Xiao Guangrong 2018-11-22 7:20 ` [PATCH v3 3/5] migration: use threaded workqueue for compression guangrong.xiao 2018-11-22 7:20 ` [Qemu-devel] " guangrong.xiao 2018-11-23 18:17 ` Dr. David Alan Gilbert 2018-11-23 18:17 ` [Qemu-devel] " Dr. David Alan Gilbert 2018-11-23 18:22 ` Paolo Bonzini 2018-11-23 18:22 ` [Qemu-devel] " Paolo Bonzini 2018-11-23 18:29 ` Dr. David Alan Gilbert 2018-11-23 18:29 ` [Qemu-devel] " Dr. David Alan Gilbert 2018-11-26 8:00 ` Xiao Guangrong 2018-11-26 8:00 ` [Qemu-devel] " Xiao Guangrong 2018-11-22 7:20 ` [PATCH v3 4/5] migration: use threaded workqueue for decompression guangrong.xiao 2018-11-22 7:20 ` [Qemu-devel] " guangrong.xiao 2018-11-22 7:20 ` [PATCH v3 5/5] tests: add threaded-workqueue-bench guangrong.xiao 2018-11-22 7:20 ` [Qemu-devel] " guangrong.xiao 2018-11-22 21:25 ` [PATCH v3 0/5] migration: improve multithreads no-reply 2018-11-22 21:25 ` [Qemu-devel] " no-reply 2018-11-22 21:35 ` no-reply 2018-11-22 21:35 ` [Qemu-devel] " no-reply
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20181122072028.22819-1-xiaoguangrong@tencent.com \ --to=guangrong.xiao@gmail.com \ --cc=cota@braap.org \ --cc=dgilbert@redhat.com \ --cc=jiang.biao2@zte.com.cn \ --cc=kvm@vger.kernel.org \ --cc=mst@redhat.com \ --cc=mtosatti@redhat.com \ --cc=pbonzini@redhat.com \ --cc=peterx@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=quintela@redhat.com \ --cc=wei.w.wang@intel.com \ --cc=xiaoguangrong@tencent.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.