From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 213E9ECAAD3 for ; Mon, 19 Sep 2022 09:24:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5/dvL3YfgaKLVtFtzp6XCkg8njdwEs2Wp6GRpU3BGek=; b=iZ0r4HP4QDCa7A 6fgeOErxRRVtorWTMjKVeHOIQsuItCT5yZKdYf4aflUCOESY2kvotyeVCkbdqgtwg7kSaOZkHRN7O iAX6ip2avHt6pHgDTkzs6YghG/JJs6twJ9ikmDbvqfG9HariEr3fpuujWmZh6Yaf1rEnReJmb1Ths Or2xSHbovV02nDFu/JKAMmM0uPaNtdJIbW9+7dfGtPctUr9thnS03WId9aa9svcBF/CwW+BSANwtc zqEvzU5wR0BmyPBuoAOnkYZXNhrd+BiBFkixggwR9eGYTL5XeI+IMtvBaF/3PIvYrFa4kTMfzN5/T PqLyBsGLGXJKPB4076Iw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oaCzc-00AieK-Lc; Mon, 19 Sep 2022 09:23:08 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oaCzV-00AibG-QO for linux-arm-kernel@lists.infradead.org; Mon, 19 Sep 2022 09:23:06 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2A1D861490; Mon, 19 Sep 2022 09:23:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 847DEC433D6; Mon, 19 Sep 2022 09:22:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1663579379; bh=T2rgL1BDoyjbQ5CdqCib48y34QtH3LVMM+evRnLrKCc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=N+s9CEuUJrcus3WX5pXi2KpSY+FIS+746DMDxXKmrc6y5vOyr7PN+qGKmvP4OFCdf clTmyW7PcTepmUtpL6ZsQ7R/9Un5uc1RD5L2cQeSqU5J+TvNtxIAJ7jmfcIF4tO6TV iAwQmuIGemMmlgIBumr2yXxJ7O/Cjz4Ni68SE9x/eQhApEQRS3ibupVgsFVILee0ZV +ciKnEO9c+2SuSybMM6rMfgRonDZMtZ+9J17wC0f002jF3Zw8hAXKOLIhMarFVQb0R mD0N/of+l8AyPZrZ1D9A5Km4LrHjggwD0qCGXTUA6itlFZNcZF7FXQe5F9vNwRSX6I siYqKJDXc4Ciw== Received: from 185-176-101-241.host.sccbroadband.ie ([185.176.101.241] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oaCzR-00B3lO-5W; Mon, 19 Sep 2022 10:22:57 +0100 Date: Mon, 19 Sep 2022 10:22:55 +0100 Message-ID: <87czbrlnv4.wl-maz@kernel.org> From: Marc Zyngier To: Gavin Shan Cc: Peter Xu , kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-doc@vger.kernel.org, catalin.marinas@arm.com, linux-kselftest@vger.kernel.org, bgardon@google.com, shuah@kernel.org, corbet@lwn.net, Andrew Jones , will@kernel.org, zhenyzha@redhat.com, dmatlack@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, oliver.upton@linux.dev, shan.gavin@gmail.com Subject: Re: [PATCH v2 1/5] KVM: x86: Introduce KVM_REQ_RING_SOFT_FULL In-Reply-To: References: <20220916045135.154505-1-gshan@redhat.com> <20220916045135.154505-2-gshan@redhat.com> <87illlkqfu.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.176.101.241 X-SA-Exim-Rcpt-To: gshan@redhat.com, peterx@redhat.com, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-doc@vger.kernel.org, catalin.marinas@arm.com, linux-kselftest@vger.kernel.org, bgardon@google.com, shuah@kernel.org, corbet@lwn.net, andrew.jones@linux.dev, will@kernel.org, zhenyzha@redhat.com, dmatlack@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, oliver.upton@linux.dev, shan.gavin@gmail.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220919_022302_059047_71272760 X-CRM114-Status: GOOD ( 53.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 19 Sep 2022 00:58:10 +0100, Gavin Shan wrote: > > On 9/18/22 7:00 PM, Marc Zyngier wrote: > > On Fri, 16 Sep 2022 19:09:52 +0100, > > Peter Xu wrote: > >> > >> On Fri, Sep 16, 2022 at 12:51:31PM +0800, Gavin Shan wrote: > >>> This adds KVM_REQ_RING_SOFT_FULL, which is raised when the dirty > >>> ring of the specific VCPU becomes softly full in kvm_dirty_ring_push(). > >>> The VCPU is enforced to exit when the request is raised and its > >>> dirty ring is softly full on its entrance. > >>> > >>> Suggested-by: Marc Zyngier > >>> Signed-off-by: Gavin Shan > >>> --- > >>> arch/x86/kvm/x86.c | 5 +++-- > >>> include/linux/kvm_host.h | 1 + > >>> virt/kvm/dirty_ring.c | 4 ++++ > >>> 3 files changed, 8 insertions(+), 2 deletions(-) > >>> > >>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > >>> index 43a6a7efc6ec..7f368f59f033 100644 > >>> --- a/arch/x86/kvm/x86.c > >>> +++ b/arch/x86/kvm/x86.c > >>> @@ -10265,8 +10265,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) > >>> bool req_immediate_exit = false; > >>> /* Forbid vmenter if vcpu dirty ring is soft-full */ > >>> - if (unlikely(vcpu->kvm->dirty_ring_size && > >>> - kvm_dirty_ring_soft_full(&vcpu->dirty_ring))) { > >>> + if (kvm_check_request(KVM_REQ_RING_SOFT_FULL, vcpu) && > >>> + kvm_dirty_ring_soft_full(&vcpu->dirty_ring)) { > >>> + kvm_make_request(KVM_REQ_RING_SOFT_FULL, vcpu); > >>> vcpu->run->exit_reason = KVM_EXIT_DIRTY_RING_FULL; > >>> trace_kvm_dirty_ring_exit(vcpu); > >>> r = 0; > >> > >> As commented previously - can we use kvm_test_request() instead? because we > >> don't want to unconditionally clear the bit. Instead of making the request > >> again, we can clear request only if !full. > > > > I have the feeling that this is a micro-optimisation that won't lead > > to much benefit in practice. You already have the cache line, just not > > in exclusive mode, and given that this is per-vcpu, you'd only see the > > cost if someone else is setting a request to this vcpu while > > evaluating the local requests. > > > > And now you need extra barriers... > > > > Also, can we please refrain from changing things without data showing > > that this actually is worse than what we had before? The point below > > makes me think that this is actually beneficial as is. > > > > I think Marc's explanation makes sense. It won't make difference in terms > of performance. We need to explicitly handle barrier when kvm_test_request() > is used. So I prefer to keep the code if Peter agrees. > > >> We can also safely move this into the block of below kvm_request_pending() > >> as Marc used to suggest. > > > > This, on the other hand, makes sure that we share the cost across all > > requests. Requests should be extremely rare anyway (and if they > > aren't, you have a whole lot of performance issues on your hands > > anyway). > > > > Yeah, We shouldn't have too much requests. I missed the comment from Marc > to move this chunk to kvm_request_pending(). I will fix it in v3. > > >> > >> To explicitly use kvm_clear_request(), we may need to be careful on the > >> memory barriers. I'm wondering whether we should have moved > >> smp_mb__after_atomic() into kvm_clear_request() because kvm_clear_request() > >> is used outside kvm_check_request() and IIUC all the call sites should > >> better have that barrier too to be safe. > >> > >> Side note: when I read the code around I also see some mis-use of clear > >> request where it can be omitted, e.g.: > >> > >> if (kvm_check_request(KVM_REQ_UNHALT, vcpu)) { > >> kvm_clear_request(KVM_REQ_UNHALT, vcpu); > >> vcpu->run->exit_reason = KVM_EXIT_IRQ_WINDOW_OPEN; > >> } > >> > >> Maybe it's a sign of bad naming, so we should renamed kvm_check_request() > >> to kvm_test_clear_request() too to show that clearing after that is not > >> needed? > > > > Yeah, this kvm_clear_request() is superfluous. But this is rather well > > documented, for once, and I don't think we should repaint it based on > > a sample of one. > > > > Yeah, I think Peter is correct that smp_mb__after_atomic() would be > part of kvm_clear_request(). Otherwise, the following two cases aren't > in same order: > > // kvm_check_request() // test and clear > kvm_test_request() kvm_test_request() > kvm_clear_request() kvm_clear_request() > smp_mb__after_atomic() [fixing Drew's email, again] Oh, I totally agree that a standalone use of kvm_clear_request() must come with a barrier. It is just that without additional data, it isn't obvious to me that there is any need for kvm_clear_request() itself to have the barrier. In a number of cases, kvm_clear_request() is used on in the context of the same vcpu, and this should be enough to ensure visibility (for example, I don't think kvm_vcpu_wfi() should require this barrier). But maybe I'm missing something. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel