From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E46A4C7EE23 for ; Tue, 23 May 2023 14:19:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6EAD0900003; Tue, 23 May 2023 10:19:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 69AA5900002; Tue, 23 May 2023 10:19:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 562EA900003; Tue, 23 May 2023 10:19:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 46319900002 for ; Tue, 23 May 2023 10:19:48 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1C632ADFB9 for ; Tue, 23 May 2023 14:19:48 +0000 (UTC) X-FDA: 80821728456.01.DF1745C Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf05.hostedemail.com (Postfix) with ESMTP id 1D2FF100010 for ; Tue, 23 May 2023 14:19:45 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=brwx0PSO; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3gMtsZAYKCPMnZVieXbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--seanjc.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3gMtsZAYKCPMnZVieXbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--seanjc.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684851586; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YJTOtttasiWLDEUcufe2rn3c2B61VJIjpPGMO/P/Yds=; b=XoEDn46VgSpPiH7NYiKaoYA4soWIxJR4t8Nh6oDSrLWkJj7NYDg+hZN+RmTeMIs7aLUfNk sogtCuYYWCglAQq9lj4xky+rIB04MZWG7BAQxq0AAPYRdjj+DZgJLiQvt+rLQLj51IAEtW oeS/hfAV4Do/GDu/ceD6gLdHW0zmD5E= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=brwx0PSO; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3gMtsZAYKCPMnZVieXbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--seanjc.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3gMtsZAYKCPMnZVieXbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--seanjc.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684851586; a=rsa-sha256; cv=none; b=mG2b0om/No8ZESpbmnh3HlMFIkmAkwGmmIiuU2tHSzf+9aYQYz5rLndI+JARfYF788kkjp Y45AUbx7OdzA9Nj7KZW2yl+4ctqIdFjkzvTahrSyRu54ges41ULVudX/yOGNKm362/B0wO fEBL8znRuqf60Wu4Y5hJqq3ZmHuNIY8= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1ae469d881fso40949475ad.0 for ; Tue, 23 May 2023 07:19:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684851584; x=1687443584; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YJTOtttasiWLDEUcufe2rn3c2B61VJIjpPGMO/P/Yds=; b=brwx0PSOMA0wdDk4Q3ZV6Zhimz9MGX1rc3oTrPLZdLDxR99oxRU+kgTiRQZeIUGp/B dyf6U+6YZAZ0UTzApeHQKfgEnBhOiOJ/qXVvQ9J1F3RjGUiuExrD9+BmRrRlbu21nlVD Wx38qijldoSLQu16rAmMxw27fIDqOXmDcc7YNTYCcS5xe37Q9LSgSZUY1x8lcrA4BzR8 Gn6XlWuegWj0mDFi1OGEJ901Pjqo4OoenFzAOsc9erWa0V37gRvkLtOzMhqrjLYhuzgg EJnTJnwVBYHzHNPBMQ/b1Fhyc00bJU+3iH49CfEJT1yRwLmSMHit03ZKsytirD36EFwd gllQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684851584; x=1687443584; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YJTOtttasiWLDEUcufe2rn3c2B61VJIjpPGMO/P/Yds=; b=a00lOhCpCRSZD/+lgboKcOZMo/T+boaz00jvnBOc3vharsfKu/I7LE8GH4OVSFZqIu hy9SVEmqJNjfCAogwg8zM8T8s5akavM0WIsfYy0nQjAK7hMLJUGts7l72FdbW+p1Wk7+ iBjReI7nKSbF8XVKSv2UBh0k/LnmTHuSMA2M57kZPKrCUeIwdlXEhCz9jLsvvIaGGsof EUY7HqhcbbFAeDGca4ZP0SfxaweTqm0BZiNRePdahiCd1cP1SDtvjD9KeXgjX6rc7ncH Eb6Id0Ll1J0+l/qunhwd2cYcm5kpVueoFk0x0rduJiqXILtPUrGDW0QKj3OLtfAwZAbY xUIw== X-Gm-Message-State: AC+VfDyVih/ZM4oVbJEy94a5++ipf+GEukATt4GbD51EgKh78prVm5IK 7770dSsNqVhHYuKnL+nli/ercrM1nDM= X-Google-Smtp-Source: ACHHUZ7XDfHM+pb9CMgdhxQCTf7TNLGglKW4K4aM789UbMGnxCZDh8BKaFM+lAod2VMVzlBmEBluGi8tvqw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:eb11:b0:1a1:b318:2776 with SMTP id l17-20020a170902eb1100b001a1b3182776mr3427854plb.0.1684851584603; Tue, 23 May 2023 07:19:44 -0700 (PDT) Date: Tue, 23 May 2023 07:19:43 -0700 In-Reply-To: Mime-Version: 1.0 References: <20220706082016.2603916-1-chao.p.peng@linux.intel.com> <20220706082016.2603916-9-chao.p.peng@linux.intel.com> Message-ID: Subject: Re: [PATCH v7 08/14] KVM: Rename mmu_notifier_* From: Sean Christopherson To: Kautuk Consul Cc: Chao Peng , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, qemu-devel@nongnu.org, linux-kselftest@vger.kernel.org, Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Shuah Khan , Mike Rapoport , Steven Price , "Maciej S . Szmigiero" , Vlastimil Babka , Vishal Annapurve , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com, aarcange@redhat.com, ddutile@redhat.com, dhildenb@redhat.com, Quentin Perret , Michael Roth , mhocko@suse.com, Muchun Song Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: 1D2FF100010 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: oo8mpm3sygqn1frdg65mebuhk89dtfam X-HE-Tag: 1684851585-704882 X-HE-Meta: U2FsdGVkX19mh2bxEkgnG+2yV5ar+RKinyTst6RwNP0G13FLyOxbrzNR1Ufg4YK3p08Lax+uXzqiu49D3FtowMGJVECOqcGTqgPz2JLl+XL9zT4A2OSu+CyYumV6NxOgsF/7fBJvNOYntNtAibbot8CsQKOCd3A6JV0IuVuZ/24Mf2wjI7o9F5dCzEbtwWoaQpkAaMOHILhh1VbyvBVjnKaMBmpBiQ9xLjr3fkB8USEJl6accvbXjgAReR+vSiU8MYzmDObKasHROeh8GzBEvSizbwLIere5D0YRChq+qeSq7gH7SOqfu0fMkdYQfPfq/2wGmb1KEaT+ZNqcuETJ/WTAmf8TlMJnqrQqHUt1jH7IKBZ+sWSN8RU2jjlpVslbH4wLbseJWHsYLZuHadLnxptGLZXbtGw+CHs4Yh79QFrLGN6OwYjWe3eZ9xCLEimRPEszszZFarFs/1vFbm5ML6H7vmHSZf49CFGtG6Pcf6VAJOeCO2nczM/gJYWsWFiKEN+5C0DYTrIs5Bjg3u6qQHV5EkMMr2GLBXiRv4uuUwRprL+Opd+IQsnONYO0LU02qK3RyRLbquO+XdgHNpO3oSTSPvPy1R6ZNEUEFPei6rf3uK7Kqdl8Zboov0RMLX4iJorgvcnzzCGoTzFqh8BNdSDCCRoTiYhYSZsg6+VItkaGJ0Hj0oHVTGwH0QqmKY+zB9rhnR8YF4qTfIY6x/K3cWoMCgKXCQSlxL/kMw3gnDU1eMtES7G5kWOlzX8IbAaK5nqCTVAt7pkiBem+jWacyHxJc9A7xFxFdJ9u7jUypqKqJqUZzAyLDZAfqxSTEYyL1Q3OngLBZZzJ92uF+j6YQWBsdMyAn2wMG5hSR85jSGABrkLKfu3GzFp8EW5IYYL+yvenLUfWCGmVYUy3XGkR+HkP9F0rTdNdRyra6yolU2lb8/dZI8ygLqRegaE6D5AaOchW4k0UwLMuTqckggU KBEe0PrU /gLEPgdMMzW5ul7aKbntydMOXECZbD2/G/6QDBuzaMYTHsuwJAeOwu+anvA201cCN1UAPXU4Vtu3KoneBwCsxVWHaz6VMWTKCZy54h7mJZux0m5oZ9uoMKiZxKtxpoY+tJp57mDa7Rw4Up+oZPTAIPo1EQ27e1ZOfq5gyNTYm7NE+Ox5cWh3nQLZAmEoMQKZQVGpnmaW1IzJGdnBiw4lroCBn+WEPRv70xDoNPmaTbd20BJUdXGYgRyIbUYvJCw7vC7pl+c9Oi7eevJJwpghl2Z+yM8ATNBN/5C9x5mpEMXNVp6xdRoFOqHghUIS08quqysXX1bYXO57HinQENAnSZ+z9xLxJg1LAPuFnsYvlq9NDIwbek+K51FXW1wkEwmjMa5mfq/xD9SwxqKiuFUbCoVKWjxa4X4KsWjlLRqku4tTUuv09lmpjrcCZk16UgM0ik7Lz2DIt4XRjrNsf4hJF+89ZmuV5/3i5Fi5t1ss8MW8CpwLvA+KLYQ5KYZUvApBe0x0ICTpA6lVbFRoXATlO8WrFMQPNwKqF8q5yXGEXF24Odw3mxDbGoCWthvl4yw8hR30PUxBz5odiDSkVK6px3W0Nz1m2ZEYCzBzVhNInlzqxJrFK9Dro+665fdLgZnXmI6nA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 23, 2023, Kautuk Consul wrote: > On 2022-07-06 16:20:10, Chao Peng wrote: > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > > index e9153b54e2a4..c262ebb168a7 100644 > > --- a/include/linux/kvm_host.h > > +++ b/include/linux/kvm_host.h > > @@ -765,10 +765,10 @@ struct kvm { > > > > #if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER) > > struct mmu_notifier mmu_notifier; > > - unsigned long mmu_notifier_seq; > > - long mmu_notifier_count; > > - gfn_t mmu_notifier_range_start; > > - gfn_t mmu_notifier_range_end; > > + unsigned long mmu_updating_seq; > > + long mmu_updating_count; > > Can we convert mmu_updating_seq and mmu_updating_count to atomic_t ? Heh, can we? Yes. Should we? No. > I see that not all accesses to these are under the kvm->mmu_lock > spinlock. Ya, working as intended. Ignoring gfn_to_pfn_cache for the moment, all accesses to mmu_invalidate_in_progress (was mmu_notifier_count / mmu_updating_count above) are done under mmu_lock. And for for mmu_notifier_seq (mmu_updating_seq above), all writes and some reads are done under mmu_lock. The only reads that are done outside of mmu_lock are the initial snapshots of the sequence number. gfn_to_pfn_cache uses a different locking scheme, the comments in mmu_notifier_retry_cache() do a good job explaining the ordering. > This will also remove the need for putting separate smp_wmb() and > smp_rmb() memory barriers while accessing these structure members. No, the memory barriers aren't there to provide any kind of atomicity. The barriers exist to ensure that stores and loads to/from the sequence and invalidate in-progress counts are ordered relative to the invalidation (stores to counts) and creation (loads) of SPTEs. Making the counts atomic changes nothing because atomic operations don't guarantee the necessary ordering. E.g. when handling a page fault, KVM snapshots the sequence outside of mmu_lock _before_ touching any state that is involved in resolving the host pfn, e.g. primary MMU state (VMAs, host page tables, etc.). After the page fault task acquires mmu_lock, KVM checks that there are no in-progress invalidations and that the sequence count is the same. This ensures that if there is a concurrent page fault and invalidation event, the page fault task will either acquire mmu_lock and create SPTEs _before_ the invalidation is processed, or the page fault task will observe either an elevated mmu_invalidate_in_progress or a different sequence count, and thus retry the page fault, if the page fault task acquires mmu_lock after the invalidation event.