From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 88C0DC433F5 for ; Wed, 23 Mar 2022 03:26:21 +0000 (UTC) Received: from localhost ([::1]:36606 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nWrdc-0007HQ-Hx for qemu-devel@archiver.kernel.org; Tue, 22 Mar 2022 23:26:20 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50516) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nWrWg-0002i7-GO for qemu-devel@nongnu.org; Tue, 22 Mar 2022 23:19:10 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.222]:48618 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nWrWe-0006h6-F4 for qemu-devel@nongnu.org; Tue, 22 Mar 2022 23:19:10 -0400 HMM_SOURCE_IP: 172.18.0.218:52666.2058481772 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP Received: from clientip-36.111.64.85 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 3AD7228047A; Wed, 23 Mar 2022 11:19:01 +0800 (CST) X-189-SAVE-TO-SEND: +wucy11@chinatelecom.cn Received: from ([172.18.0.218]) by app0025 with ESMTP id ce0e06a26c37436f838f61d4d3a3b9e0 for qemu-devel@nongnu.org; Wed, 23 Mar 2022 11:19:06 CST X-Transaction-ID: ce0e06a26c37436f838f61d4d3a3b9e0 X-Real-From: wucy11@chinatelecom.cn X-Receive-IP: 172.18.0.218 X-MEDUSA-Status: 0 From: wucy11@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v1 3/5] kvm: Dirty ring autoconverge optmization for kvm_cpu_synchronize_kick_all Date: Wed, 23 Mar 2022 11:18:36 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=42.123.76.222; envelope-from=wucy11@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: baiyw2@chinatelecom.cn, yuanmh12@chinatelecom.cn, tugy@chinatelecom.cn, David Hildenbrand , huangy81@chinatelecom.cn, Juan Quintela , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , yubin1@chinatelecom.cn, dengpc12@chinatelecom.cn, Paolo Bonzini , wucy11@chinatelecom.cn Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chongyun Wu Dirty ring feature need call kvm_cpu_synchronize_kick_all to flush hardware buffers into KVMslots, but when aucoverge run kvm_cpu_synchronize_kick_all calling will become more and more time consuming. This will significantly reduce the efficiency of dirty page queries, especially when memory pressure is high and the speed limit is high. When the CPU speed limit is high and kvm_cpu_synchronize_kick_all is time-consuming, the rate of dirty pages generated by the VM will also be significantly reduced, so it is not necessary to call kvm_cpu_synchronize_kick_all at this time, just call it once before stopping the VM. This will significantly improve the efficiency of dirty page queries under high pressure. Signed-off-by: Chongyun Wu --- accel/kvm/kvm-all.c | 25 +++++-------------------- include/exec/cpu-common.h | 2 ++ migration/migration.c | 11 +++++++++++ softmmu/cpus.c | 18 ++++++++++++++++++ 4 files changed, 36 insertions(+), 20 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 65a4de8..5e02700 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -48,6 +48,8 @@ #include "hw/boards.h" +#include "sysemu/cpu-throttle.h" + /* This check must be after config-host.h is included */ #ifdef CONFIG_EVENTFD #include @@ -839,25 +841,6 @@ static uint64_t kvm_dirty_ring_reap(KVMState *s) return total; } -static void do_kvm_cpu_synchronize_kick(CPUState *cpu, run_on_cpu_data arg) -{ - /* No need to do anything */ -} - -/* - * Kick all vcpus out in a synchronized way. When returned, we - * guarantee that every vcpu has been kicked and at least returned to - * userspace once. - */ -static void kvm_cpu_synchronize_kick_all(void) -{ - CPUState *cpu; - - CPU_FOREACH(cpu) { - run_on_cpu(cpu, do_kvm_cpu_synchronize_kick, RUN_ON_CPU_NULL); - } -} - /* * Flush all the existing dirty pages to the KVM slot buffers. When * this call returns, we guarantee that all the touched dirty pages @@ -879,7 +862,9 @@ static void kvm_dirty_ring_flush(void) * First make sure to flush the hardware buffers by kicking all * vcpus out in a synchronous way. */ - kvm_cpu_synchronize_kick_all(); + if (!cpu_throttle_get_percentage()) { + qemu_kvm_cpu_synchronize_kick_all(); + } kvm_dirty_ring_reap(kvm_state); trace_kvm_dirty_ring_flush(1); } diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 50a7d29..13045b3 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -160,4 +160,6 @@ extern int singlestep; void list_cpus(const char *optarg); +void qemu_kvm_cpu_synchronize_kick_all(void); + #endif /* CPU_COMMON_H */ diff --git a/migration/migration.c b/migration/migration.c index 695f0f2..ca1db88 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -61,6 +61,7 @@ #include "sysemu/cpus.h" #include "yank_functions.h" #include "sysemu/qtest.h" +#include "sysemu/kvm.h" #define MAX_THROTTLE (128 << 20) /* Migration transfer speed throttling */ @@ -3183,6 +3184,16 @@ static void migration_completion(MigrationState *s) if (!ret) { bool inactivate = !migrate_colo_enabled(); + /* + * Before stop vm do qemu_kvm_cpu_synchronize_kick_all to + * fulsh hardware buffer into KVMslots for dirty ring + * optmiaztion, If qemu_kvm_cpu_synchronize_kick_all is not + * called when the CPU speed is limited to improve efficiency + */ + if (kvm_dirty_ring_enabled() + && cpu_throttle_get_percentage()) { + qemu_kvm_cpu_synchronize_kick_all(); + } ret = vm_stop_force_state(RUN_STATE_FINISH_MIGRATE); trace_migration_completion_vm_stop(ret); if (ret >= 0) { diff --git a/softmmu/cpus.c b/softmmu/cpus.c index 7b75bb6..d028d83 100644 --- a/softmmu/cpus.c +++ b/softmmu/cpus.c @@ -810,3 +810,21 @@ void qmp_inject_nmi(Error **errp) nmi_monitor_handle(monitor_get_cpu_index(monitor_cur()), errp); } +static void do_kvm_cpu_synchronize_kick(CPUState *cpu, run_on_cpu_data arg) +{ + /* No need to do anything */ +} + +/* + * Kick all vcpus out in a synchronized way. When returned, we + * guarantee that every vcpu has been kicked and at least returned to + * userspace once. + */ +void qemu_kvm_cpu_synchronize_kick_all(void) +{ + CPUState *cpu; + + CPU_FOREACH(cpu) { + run_on_cpu(cpu, do_kvm_cpu_synchronize_kick, RUN_ON_CPU_NULL); + } +} -- 1.8.3.1