From: Alistair Popple <apopple@nvidia.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: rcampbell@nvidia.com, linux-doc@vger.kernel.org,
nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org,
dri-devel@lists.freedesktop.org, hch@infradead.org,
linux-mm@kvack.org, bskeggs@redhat.com, daniel@ffwll.ch,
akpm@linux-foundation.org
Subject: Re: [Nouveau] [PATCH v3 5/8] mm: Device exclusive memory access
Date: Tue, 2 Mar 2021 19:57:58 +1100 [thread overview]
Message-ID: <2758096.Z30Q8iEM0t@nvdebian> (raw)
In-Reply-To: <20210302000559.GA763995@nvidia.com>
On Tuesday, 2 March 2021 11:05:59 AM AEDT Jason Gunthorpe wrote:
> On Fri, Feb 26, 2021 at 06:18:29PM +1100, Alistair Popple wrote:
>
> > +/**
> > + * make_device_exclusive_range() - Mark a range for exclusive use by a
device
> > + * @mm: mm_struct of assoicated target process
> > + * @start: start of the region to mark for exclusive device access
> > + * @end: end address of region
> > + * @pages: returns the pages which were successfully mark for exclusive
acces
> > + *
> > + * Returns: number of pages successfully marked for exclusive access
> > + *
> > + * This function finds the ptes mapping page(s) to the given address
range and
> > + * replaces them with special swap entries preventing userspace CPU
access. On
> > + * fault these entries are replaced with the original mapping after
calling MMU
> > + * notifiers.
> > + */
> > +int make_device_exclusive_range(struct mm_struct *mm, unsigned long
start,
> > + unsigned long end, struct page **pages)
> > +{
> > + long npages = (end - start) >> PAGE_SHIFT;
> > + long i;
> > +
> > + npages = get_user_pages_remote(mm, start, npages,
> > + FOLL_GET | FOLL_WRITE | FOLL_SPLIT_PMD,
> > + pages, NULL, NULL);
> > + for (i = 0; i < npages; i++) {
> > + if (!trylock_page(pages[i])) {
> > + put_page(pages[i]);
> > + pages[i] = NULL;
> > + continue;
> > + }
> > +
> > + if (!try_to_protect(pages[i])) {
>
> Isn't this racy? get_user_pages returns the ptes at an instant in
> time, they could have already been changed to something else?
Right. On it's own this does not guarantee that the page is mapped at the
given location, only that a mapping won't get established without an mmu
notifier callback to clear the swap entry.
The intent was a driver could use HMM or some other mechanism to keep PTEs
synchronised if required. However I just looked at patch 8 in the series again
and it appears I got this wrong when converting from the old migration
approach:
+ mutex_unlock(&svmm->mutex);
+ ret = nouveau_atomic_range_fault(svmm, drm, args,
+ size, hmm_flags, mm);
The mutex needs to be unlocked after the range fault to ensure the PTE hasn't
changed. But this ends up being a problem because try_to_protect() calls
notifiers which need to take that mutex and hence deadlocks.
> I would think you'd want to switch to the swap entry atomically under
> th PTLs?
That is one approach, but the reuse of get_user_pages() to walk the page
tables and fault/gather the pages is a nice simplification and adding a new
FOLL flag/mode to atomically swap entries doesn't seem right.
However try_to_protect() scans the PTEs again under the PTL so checking the
mapping of interest actually gets replaced during the rmap walk seems like a
reasonable solution. Thanks for the comments.
- Alistair
> Jason
>
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau
next prev parent reply other threads:[~2021-03-02 8:58 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-26 7:18 [Nouveau] [PATCH v3 0/8] Add support for SVM atomics in Nouveau Alistair Popple
2021-02-26 7:18 ` [Nouveau] [PATCH v3 1/8] mm: Remove special swap entry functions Alistair Popple
2021-02-26 15:59 ` Christoph Hellwig
2021-03-02 8:52 ` Alistair Popple
2021-03-02 12:02 ` Alistair Popple
2021-03-01 17:46 ` Jason Gunthorpe
2021-03-02 0:21 ` Alistair Popple
2021-02-26 7:18 ` [Nouveau] [PATCH v3 2/8] mm/swapops: Rework swap entry manipulation code Alistair Popple
2021-02-26 16:00 ` Christoph Hellwig
2021-03-01 17:47 ` Jason Gunthorpe
2021-02-26 7:18 ` [Nouveau] [PATCH v3 3/8] mm/rmap: Split try_to_munlock from try_to_unmap Alistair Popple
2021-02-26 16:01 ` Christoph Hellwig
2021-03-01 16:10 ` Jason Gunthorpe
2021-03-04 4:27 ` Alistair Popple
2021-02-26 7:18 ` [Nouveau] [PATCH v3 4/8] mm/rmap: Split migration into its own function Alistair Popple
2021-02-26 16:03 ` Christoph Hellwig
2021-03-02 22:08 ` Zi Yan
2021-03-04 23:54 ` Alistair Popple
2021-02-26 7:18 ` [Nouveau] [PATCH v3 5/8] mm: Device exclusive memory access Alistair Popple
2021-03-01 17:54 ` Jason Gunthorpe
2021-03-01 22:55 ` Ralph Campbell
2021-03-02 0:05 ` Jason Gunthorpe
2021-03-02 8:57 ` Alistair Popple [this message]
2021-03-02 12:41 ` Jason Gunthorpe
2021-03-04 5:20 ` Alistair Popple
2021-02-26 7:18 ` [Nouveau] [PATCH v3 6/8] mm: Selftests for exclusive device memory Alistair Popple
2021-03-01 17:55 ` Jason Gunthorpe
2021-03-01 18:07 ` Ralph Campbell
2021-03-01 23:14 ` Ralph Campbell
2021-03-02 9:12 ` Alistair Popple
2021-02-26 7:18 ` [Nouveau] [PATCH v3 7/8] nouveau/svm: Refactor nouveau_range_fault Alistair Popple
2021-02-26 7:18 ` [Nouveau] [PATCH v3 8/8] nouveau/svm: Implement atomic SVM access Alistair Popple
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2758096.Z30Q8iEM0t@nvdebian \
--to=apopple@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=bskeggs@redhat.com \
--cc=daniel@ffwll.ch \
--cc=dri-devel@lists.freedesktop.org \
--cc=hch@infradead.org \
--cc=jgg@nvidia.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=nouveau@lists.freedesktop.org \
--cc=rcampbell@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).