Re: [PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes

From: Dan Williams <dan.j.williams@intel.com>
To: David Hildenbrand <david@redhat.com>
Cc: "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
	"Linux MM" <linux-mm@kvack.org>,
	"Michal Hocko" <mhocko@kernel.org>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	kvm-ppc@vger.kernel.org,
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	"KVM list" <kvm@vger.kernel.org>,
	linux-hyperv@vger.kernel.org, devel@driverdev.osuosl.org,
	xen-devel <xen-devel@lists.xenproject.org>,
	"X86 ML" <x86@kernel.org>,
	"Alexander Duyck" <alexander.duyck@gmail.com>,
	"Alexander Duyck" <alexander.h.duyck@linux.intel.com>,
	"Alex Williamson" <alex.williamson@redhat.com>,
	"Allison Randal" <allison@lohutok.net>,
	"Andy Lutomirski" <luto@kernel.org>,
	"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
	"Anshuman Khandual" <anshuman.khandual@arm.com>,
	"Anthony Yznaga" <anthony.yznaga@oracle.com>,
	"Benjamin Herrenschmidt" <benh@kernel.crashing.org>,
	"Borislav Petkov" <bp@alien8.de>,
	"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
	"Christophe Leroy" <christophe.leroy@c-s.fr>,
	"Cornelia Huck" <cohuck@redhat.com>,
	"Dave Hansen" <dave.hansen@linux.intel.com>,
	"Haiyang Zhang" <haiyangz@microsoft.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Isaac J. Manjarres" <isaacm@codeaurora.org>,
	"Jim Mattson" <jmattson@google.com>,
	"Joerg Roedel" <joro@8bytes.org>,
	"Johannes Weiner" <hannes@cmpxchg.org>,
	"Juergen Gross" <jgross@suse.com>,
	"KarimAllah Ahmed" <karahmed@amazon.de>,
	"Kees Cook" <keescook@chromium.org>,
	"K. Y. Srinivasan" <kys@microsoft.com>,
	"Matthew Wilcox (Oracle)" <willy@infradead.org>,
	"Matt Sickler" <Matt.Sickler@daktronics.com>,
	"Mel Gorman" <mgorman@techsingularity.net>,
	"Michael Ellerman" <mpe@ellerman.id.au>,
	"Michal Hocko" <mhocko@suse.com>,
	"Mike Rapoport" <rppt@linux.ibm.com>,
	"Mike Rapoport" <rppt@linux.vnet.ibm.com>,
	"Nicholas Piggin" <npiggin@gmail.com>,
	"Oscar Salvador" <osalvador@suse.de>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Paul Mackerras" <paulus@ozlabs.org>,
	"Paul Mackerras" <paulus@samba.org>,
	"Pavel Tatashin" <pasha.tatashin@soleen.com>,
	"Pavel Tatashin" <pavel.tatashin@microsoft.com>,
	"Peter Zijlstra" <peterz@infradead.org>, "Qian Cai" <cai@lca.pw>,
	"Radim Krčmář" <rkrcmar@redhat.com>,
	"Sasha Levin" <sashal@kernel.org>,
	"Sean Christopherson" <sean.j.christopherson@intel.com>,
	"Stefano Stabellini" <sstabellini@kernel.org>,
	"Stephen Hemminger" <sthemmin@microsoft.com>,
	"Thomas Gleixner" <tglx@linutronix.de>,
	"Vitaly Kuznetsov" <vkuznets@redhat.com>,
	"Vlastimil Babka" <vbabka@suse.cz>,
	"Wanpeng Li" <wanpengli@tencent.com>,
	YueHaibing <yuehaibing@huawei.com>,
	"Adam Borowski" <kilobyte@angband.pl>
Subject: Re: [PATCH v1 03/10] KVM: Prepare kvm_is_reserved_pfn() for PG_reserved changes
Date: Mon, 4 Nov 2019 20:38:56 -0800	[thread overview]
Message-ID: <CAPcyv4jyTxEpw5ep5Noy0YRV7Dybzj+8OTVMwRK_zeFigF-LsQ@mail.gmail.com> (raw)
In-Reply-To: <20191024120938.11237-4-david@redhat.com>

On Thu, Oct 24, 2019 at 5:11 AM David Hildenbrand <david@redhat.com> wrote:
>
> Right now, ZONE_DEVICE memory is always set PG_reserved. We want to
> change that.
>
> KVM has this weird use case that you can map anything from /dev/mem
> into the guest. pfn_valid() is not a reliable check whether the memmap
> was initialized and can be touched. pfn_to_online_page() makes sure
> that we have an initialized memmap (and don't have ZONE_DEVICE memory).
>
> Rewrite kvm_is_reserved_pfn() to make sure the function produces the
> same result once we stop setting ZONE_DEVICE pages PG_reserved.
>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: "Radim Krčmář" <rkrcmar@redhat.com>
> Cc: Michal Hocko <mhocko@kernel.org>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: KarimAllah Ahmed <karahmed@amazon.de>
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  virt/kvm/kvm_main.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index e9eb666eb6e8..9d18cc67d124 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -151,9 +151,15 @@ __weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm,
>
>  bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
>  {
> -       if (pfn_valid(pfn))
> -               return PageReserved(pfn_to_page(pfn));
> +       struct page *page = pfn_to_online_page(pfn);
>
> +       /*
> +        * We treat any pages that are not online (not managed by the buddy)
> +        * as reserved - this includes ZONE_DEVICE pages and pages without
> +        * a memmap (e.g., mapped via /dev/mem).
> +        */
> +       if (page)
> +               return PageReserved(page);
>         return true;
>  }

So after this all the pfn_valid() usage in kvm_main.c is replaced with
pfn_to_online_page()? Looks correct to me.

However, I'm worried that kvm is taking reference on ZONE_DEVICE pages
through some other path resulting in this:

    https://lore.kernel.org/linux-nvdimm/20190919154708.GA24650@angband.pl/

I'll see if this patch set modulates or maintains that failure mode.