RE: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

From: "Kasireddy, Vivek" <vivek.kasireddy@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: "dri-devel@lists.freedesktop.org"
	<dri-devel@lists.freedesktop.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	David Hildenbrand <david@redhat.com>,
	Mike Kravetz <mike.kravetz@oracle.com>,
	Hugh Dickins <hughd@google.com>, Peter Xu <peterx@redhat.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	"Kim, Dongwon" <dongwon.kim@intel.com>,
	"Chang, Junxiao" <junxiao.chang@intel.com>
Subject: RE: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)
Date: Wed, 19 Jul 2023 00:05:29 +0000	[thread overview]
Message-ID: <IA0PR11MB71852354956EA85210AE5350F839A@IA0PR11MB7185.namprd11.prod.outlook.com> (raw)
In-Reply-To: <ZLaxfmrVo7sygM3d@nvidia.com>

Hi Jason,

> 
> On Tue, Jul 18, 2023 at 01:28:56AM -0700, Vivek Kasireddy wrote:
> > Currently, there does not appear to be any mechanism for letting
> > drivers or other kernel entities know about updates made in a
> > mapping particularly when a new page is faulted in. Providing
> > notifications for such situations is really useful when using
> > memfds backed by ram-based filesystems such as shmem or hugetlbfs
> > that also allow FALLOC_FL_PUNCH_HOLE.
> 
> Huh? You get an invalidate when this happens and the address becomes
> non-present.
Yes, we do get an invalidate (range) but it is not going to help given my
use-case. This is because the invalidate only indicates that the old pages
are gone (and not about the availability of new pages). IIUC, after a hole
gets punched, it appears the new pages are faulted in only when there
are writes made to that region where the hole was punched. So, I think
what would really help is to get notified when a new page becomes part
of the mapping at a given offset. 

> 
> > More specifically, when a hole is punched in a memfd (that is
> > backed by shmem or hugetlbfs), a driver can register for
> > notifications associated with range invalidations. However, it
> > would also be useful to have notifications when new pages are
> > faulted in as a result of writes made to the mapping region that
> > overlaps with a previously punched hole.
> 
> If there is no change to the PTEs then it is hard to see why this
> would be part of a mmu_notifier.
IIUC, the PTEs do get changed but only when a new page is faulted in.
For shmem, it looks like the PTEs are updated in handle_pte_fault()
after shmem_fault() gets called and for hugetlbfs, this seems to
happen in hugetlb_fault().

Instead of introducing a new notifier, I did think about reusing
(or overloading) .change_pte() but I did not fully understand the impact
it would have on KVM, the only user of .change_pte(). 

Thanks,
Vivek

> 
> Jason