From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39F41C433F5 for ; Wed, 23 Mar 2022 03:27:45 +0000 (UTC) Received: from localhost ([::1]:39558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nWrey-0000t6-8i for qemu-devel@archiver.kernel.org; Tue, 22 Mar 2022 23:27:44 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50530) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nWrWk-0002xC-Sp for qemu-devel@nongnu.org; Tue, 22 Mar 2022 23:19:14 -0400 Received: from prt-mail.chinatelecom.cn ([42.123.76.222]:48659 helo=chinatelecom.cn) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nWrWi-0006hP-ML for qemu-devel@nongnu.org; Tue, 22 Mar 2022 23:19:14 -0400 HMM_SOURCE_IP: 172.18.0.218:52666.2058481772 HMM_ATTACHE_NUM: 0000 HMM_SOURCE_TYPE: SMTP Received: from clientip-36.111.64.85 (unknown [172.18.0.218]) by chinatelecom.cn (HERMES) with SMTP id 6F516280485; Wed, 23 Mar 2022 11:19:06 +0800 (CST) X-189-SAVE-TO-SEND: +wucy11@chinatelecom.cn Received: from ([172.18.0.218]) by app0025 with ESMTP id 95e67f6018bb47a69a4e4cd53159b7aa for qemu-devel@nongnu.org; Wed, 23 Mar 2022 11:19:11 CST X-Transaction-ID: 95e67f6018bb47a69a4e4cd53159b7aa X-Real-From: wucy11@chinatelecom.cn X-Receive-IP: 172.18.0.218 X-MEDUSA-Status: 0 From: wucy11@chinatelecom.cn To: qemu-devel@nongnu.org Subject: [PATCH v1 4/5] kvm: Introduce a dirty rate calculation method based on dirty ring Date: Wed, 23 Mar 2022 11:18:37 +0800 Message-Id: <7593aa50f8595cf06df7675032ccbe877c1823f4.1648002360.git.wucy11@chinatelecom.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=42.123.76.222; envelope-from=wucy11@chinatelecom.cn; helo=chinatelecom.cn X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: baiyw2@chinatelecom.cn, yuanmh12@chinatelecom.cn, tugy@chinatelecom.cn, David Hildenbrand , huangy81@chinatelecom.cn, Juan Quintela , Richard Henderson , "Dr. David Alan Gilbert" , Peter Xu , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , yubin1@chinatelecom.cn, dengpc12@chinatelecom.cn, Paolo Bonzini , wucy11@chinatelecom.cn Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chongyun Wu A new structure KVMDirtyRingDirtyCounter is introduced in KVMDirtyRingReaper to record the number of dirty pages within a period of time. When kvm_dirty_ring_mark_page collects dirty pages, if it finds that the current dirty pages are not duplicates, it increases the dirty_pages_period count. Divide the dirty_pages_period count by the interval to get the dirty page rate for this period. And use dirty_pages_period_peak_rate to count the highest dirty page rate, to solve the problem that the dirty page collection rate may change greatly during a period of time, resulting in a large change in the dirty page rate. Through sufficient testing, it is found that the dirty rate calculated after kvm_dirty_ring_flush usually matches the actual pressure, and the dirty rate counted per second may change in the subsequent seconds, so record the peak dirty rate as the real dirty pages rate. This dirty pages rate is mainly used as the subsequent autoconverge calculation speed limit throttle. Signed-off-by: Chongyun Wu --- accel/kvm/kvm-all.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++- include/sysemu/kvm.h | 2 ++ 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 5e02700..a158b1c 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -114,6 +114,13 @@ enum KVMDirtyRingReaperSpeedControl { KVM_DIRTY_RING_REAPER_SPEED_CONTROL_DOWN }; +struct KVMDirtyRingDirtyCounter { + int64_t time_last_count; + uint64_t dirty_pages_period; + int64_t dirty_pages_rate; + int64_t dirty_pages_period_peak_rate; +}; + /* * KVM reaper instance, responsible for collecting the KVM dirty bits * via the dirty ring. @@ -128,6 +135,7 @@ struct KVMDirtyRingReaper { uint64_t ring_full_cnt; float ratio_adjust_threshold; int stable_count_threshold; + struct KVMDirtyRingDirtyCounter counter; /* Calculate dirty pages rate */ }; struct KVMState @@ -739,7 +747,9 @@ static void kvm_dirty_ring_mark_page(KVMState *s, uint32_t as_id, return; } - set_bit(offset, mem->dirty_bmap); + if (!test_and_set_bit(offset, mem->dirty_bmap)) { + s->reaper.counter.dirty_pages_period++; + } } static bool dirty_gfn_is_dirtied(struct kvm_dirty_gfn *gfn) @@ -783,6 +793,56 @@ static uint32_t kvm_dirty_ring_reap_one(KVMState *s, CPUState *cpu) return count; } +int64_t kvm_dirty_ring_get_rate(void) +{ + return kvm_state->reaper.counter.dirty_pages_rate; +} + +int64_t kvm_dirty_ring_get_peak_rate(void) +{ + return kvm_state->reaper.counter.dirty_pages_period_peak_rate; +} + +static void kvm_dirty_ring_reap_count(KVMState *s) +{ + int64_t spend_time = 0; + int64_t end_time; + + if (!s->reaper.counter.time_last_count) { + s->reaper.counter.time_last_count = + qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + } + + end_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); + spend_time = end_time - s->reaper.counter.time_last_count; + + if (!s->reaper.counter.dirty_pages_period || + !spend_time) { + return; + } + + /* + * More than 1 second = 1000 millisecons, + * or trigger by kvm_log_sync_global which spend time + * more than 300 milliscons. + */ + if (spend_time > 1000) { + /* Count the dirty page rate during this period */ + s->reaper.counter.dirty_pages_rate = + s->reaper.counter.dirty_pages_period * 1000 / spend_time; + /* Update the peak dirty page rate at this period */ + if (s->reaper.counter.dirty_pages_rate > + s->reaper.counter.dirty_pages_period_peak_rate) { + s->reaper.counter.dirty_pages_period_peak_rate = + s->reaper.counter.dirty_pages_rate; + } + + /* Reset counters */ + s->reaper.counter.dirty_pages_period = 0; + s->reaper.counter.time_last_count = 0; + } +} + /* Must be with slots_lock held */ static uint64_t kvm_dirty_ring_reap_locked(KVMState *s) { @@ -793,6 +853,8 @@ static uint64_t kvm_dirty_ring_reap_locked(KVMState *s) stamp = get_clock(); + kvm_dirty_ring_reap_count(s); + CPU_FOREACH(cpu) { total += kvm_dirty_ring_reap_one(s, cpu); } diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index a783c78..05846f9 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -582,4 +582,6 @@ bool kvm_cpu_check_are_resettable(void); bool kvm_arch_cpu_check_are_resettable(void); bool kvm_dirty_ring_enabled(void); +int64_t kvm_dirty_ring_get_rate(void); +int64_t kvm_dirty_ring_get_peak_rate(void); #endif -- 1.8.3.1