Re: [PATCH 1/9] vfio: Make vfio_(un)register_notifier accept a vfio_device

From: Jason Gunthorpe <jgg@nvidia.com>
To: Christoph Hellwig <hch@lst.de>
Cc: kvm@vger.kernel.org, linux-doc@vger.kernel.org,
	David Airlie <airlied@linux.ie>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	dri-devel@lists.freedesktop.org,
	Kirti Wankhede <kwankhede@nvidia.com>,
	Vineeth Vijayan <vneethv@linux.ibm.com>,
	Alexander Gordeev <agordeev@linux.ibm.com>,
	linux-s390@vger.kernel.org, "Liu, Yi L" <yi.l.liu@intel.com>,
	Matthew Rosato <mjrosato@linux.ibm.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Halil Pasic <pasic@linux.ibm.com>,
	Christian Borntraeger <borntraeger@linux.ibm.com>,
	intel-gfx@lists.freedesktop.org, Zhi Wang <zhi.a.wang@intel.com>,
	Jason Herne <jjherne@linux.ibm.com>,
	Eric Farman <farman@linux.ibm.com>,
	Vasily Gorbik <gor@linux.ibm.com>,
	Heiko Carstens <hca@linux.ibm.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	Harald Freudenberger <freude@linux.ibm.com>,
	Rodrigo Vivi <rodrigo.vivi@intel.com>,
	intel-gvt-dev@lists.freedesktop.org,
	Tony Krowiak <akrowiak@linux.ibm.com>,
	Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
	Cornelia Huck <cohuck@redhat.com>,
	Peter Oberparleiter <oberpar@linux.ibm.com>,
	Sven Schnelle <svens@linux.ibm.com>
Subject: Re: [PATCH 1/9] vfio: Make vfio_(un)register_notifier accept a vfio_device
Date: Wed, 13 Apr 2022 14:37:27 -0300	[thread overview]
Message-ID: <20220413173727.GU2120790@nvidia.com> (raw)
In-Reply-To: <20220413162946.GB31053@lst.de>

On Wed, Apr 13, 2022 at 06:29:46PM +0200, Christoph Hellwig wrote:
> On Wed, Apr 13, 2022 at 01:18:14PM -0300, Jason Gunthorpe wrote:
> > Yeah, I was thinking about that too, but on the other hand I think it
> > is completely wrong that gvt requires kvm at all. A vfio_device is not
> > supposed to be tightly linked to KVM - the only exception possibly
> > being s390..
> 
> So i915/gvt uses it for:
> 
>  - poking into the KVM GFN translations
>  - using the KVM page track notifier
> 
> No idea how these could be solved in a more generic way.

TBH I'm not sure how any of this works fully correctly..

I see this code getting something it calls a GFN and then passing
them to vfio - which makes no sense. Either a value is a GFN - the
physical memory address of the VM, or it is an IOVA. VFIO only takes
in IOVA and kvm only takes in GFN. So these are probably IOVAs really..

But then, I see this code taking GFNs (which are probably IOVAs?) and
passing them to the kvm page track notifier? That can't be right, VFIO
needs to translate the IOVA to a GFN, not assume 1:1...

It seems the purpose is to shadow a page table, and it is capturing
user space CPU writes to this page table memory I guess?

GFN's seems to come from gen8_gtt_get_pfn which seems to be parsing
some guest page table?

Jason