From: Alistair Popple <apopple@nvidia.com>
To: Peter Xu <peterx@redhat.com>
Cc: <linux-mm@kvack.org>, <nouveau@lists.freedesktop.org>,
<bskeggs@redhat.com>, <akpm@linux-foundation.org>,
<linux-doc@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<dri-devel@lists.freedesktop.org>, <jhubbard@nvidia.com>,
<rcampbell@nvidia.com>, <jglisse@redhat.com>, <jgg@nvidia.com>,
<hch@infradead.org>, <daniel@ffwll.ch>, <willy@infradead.org>,
<bsingharora@gmail.com>, Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH v8 5/8] mm: Device exclusive memory access
Date: Fri, 21 May 2021 16:53:29 +1000 [thread overview]
Message-ID: <1959488.yZHLR0KveG@nvdebian> (raw)
In-Reply-To: <47694715.suB6H4Uo8R@nvdebian>
On Tuesday, 18 May 2021 11:19:05 PM AEST Alistair Popple wrote:
[...]
> > > +/*
> > > + * Restore a potential device exclusive pte to a working pte entry
> > > + */
> > > +static vm_fault_t remove_device_exclusive_entry(struct vm_fault *vmf)
> > > +{
> > > + struct page *page = vmf->page;
> > > + struct vm_area_struct *vma = vmf->vma;
> > > + struct page_vma_mapped_walk pvmw = {
> > > + .page = page,
> > > + .vma = vma,
> > > + .address = vmf->address,
> > > + .flags = PVMW_SYNC,
> > > + };
> > > + vm_fault_t ret = 0;
> > > + struct mmu_notifier_range range;
> > > +
> > > + if (!lock_page_or_retry(page, vma->vm_mm, vmf->flags))
> > > + return VM_FAULT_RETRY;
> > > + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma,
> > > vma->vm_mm,
> > > + vmf->address & PAGE_MASK,
> > > + (vmf->address & PAGE_MASK) + PAGE_SIZE);
> > > + mmu_notifier_invalidate_range_start(&range);
> >
> > I looked at MMU_NOTIFIER_CLEAR document and it tells me:
> >
> > * @MMU_NOTIFY_CLEAR: clear page table entry (many reasons for this like
> > * madvise() or replacing a page by another one, ...).
> >
> > Does MMU_NOTIFIER_CLEAR suite for this case? Normally I think for such a
> > case (existing pte is invalid) we don't need to notify at all. However
> > from what I read from the whole series, this seems to be a critical point
> > where we'd like to kick the owner/driver to synchronously stop doing
> > atomic
> > operations from the device. Not sure whether we'd like a new notifier
> > type, or maybe at least comment on why to use CLEAR?
>
> Right, notifying the owner/driver when it no longer has exclusive access to
> the page and allowing it to stop atomic operations is the critical point and
> why it notifies when we ordinarily wouldn't (ie. invalid -> valid).
>
> I did consider adding a new type, but in the driver implementation it ends
> up
> being treated the same as a CLEAR notification anyway so didn't think it was
> necessary. But I suppose adding a different type would allow other listening
> notifiers to filter these which might be worthwhile.
>
> > > +
> > > + while (page_vma_mapped_walk(&pvmw)) {
> >
> > IIUC a while loop of page_vma_mapped_walk() handles thps, however here
> > it's
> > already in do_swap_page() so it's small pte only? Meanwhile...
> >
> > > + if (unlikely(!pte_same(*pvmw.pte, vmf->orig_pte))) {
> > > + page_vma_mapped_walk_done(&pvmw);
> > > + break;
> > > + }
> > > +
> > > + restore_exclusive_pte(vma, page, pvmw.address, pvmw.pte);
> >
> > ... I'm not sure whether passing in page would work for thp after all, as
> > iiuc we may need to pass in the subpage rather than the head page if so.
>
> Hmm, I need to check this and follow up.
Thanks Peter for pointing that out. I needed to follow this up because I had
slightly misunderstood page_vma_mapped_walk(). As you say this is actually a
small pte and restore_exclusive_pte() needs the actual page from the fault. So
I should be able to drop the page_vma_mapped_walk() and use
pte_offset_map_lock() to call restore_exclusive_pte directly.
- Alistair
WARNING: multiple messages have this Message-ID (diff)
From: Alistair Popple <apopple@nvidia.com>
To: Peter Xu <peterx@redhat.com>
Cc: rcampbell@nvidia.com, willy@infradead.org, daniel@ffwll.ch,
linux-doc@vger.kernel.org, nouveau@lists.freedesktop.org,
bsingharora@gmail.com, linux-kernel@vger.kernel.org,
dri-devel@lists.freedesktop.org, hch@infradead.org,
linux-mm@kvack.org, bskeggs@redhat.com, jgg@nvidia.com,
akpm@linux-foundation.org, Christoph Hellwig <hch@lst.de>
Subject: Re: [Nouveau] [PATCH v8 5/8] mm: Device exclusive memory access
Date: Fri, 21 May 2021 16:53:29 +1000 [thread overview]
Message-ID: <1959488.yZHLR0KveG@nvdebian> (raw)
In-Reply-To: <47694715.suB6H4Uo8R@nvdebian>
On Tuesday, 18 May 2021 11:19:05 PM AEST Alistair Popple wrote:
[...]
> > > +/*
> > > + * Restore a potential device exclusive pte to a working pte entry
> > > + */
> > > +static vm_fault_t remove_device_exclusive_entry(struct vm_fault *vmf)
> > > +{
> > > + struct page *page = vmf->page;
> > > + struct vm_area_struct *vma = vmf->vma;
> > > + struct page_vma_mapped_walk pvmw = {
> > > + .page = page,
> > > + .vma = vma,
> > > + .address = vmf->address,
> > > + .flags = PVMW_SYNC,
> > > + };
> > > + vm_fault_t ret = 0;
> > > + struct mmu_notifier_range range;
> > > +
> > > + if (!lock_page_or_retry(page, vma->vm_mm, vmf->flags))
> > > + return VM_FAULT_RETRY;
> > > + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma,
> > > vma->vm_mm,
> > > + vmf->address & PAGE_MASK,
> > > + (vmf->address & PAGE_MASK) + PAGE_SIZE);
> > > + mmu_notifier_invalidate_range_start(&range);
> >
> > I looked at MMU_NOTIFIER_CLEAR document and it tells me:
> >
> > * @MMU_NOTIFY_CLEAR: clear page table entry (many reasons for this like
> > * madvise() or replacing a page by another one, ...).
> >
> > Does MMU_NOTIFIER_CLEAR suite for this case? Normally I think for such a
> > case (existing pte is invalid) we don't need to notify at all. However
> > from what I read from the whole series, this seems to be a critical point
> > where we'd like to kick the owner/driver to synchronously stop doing
> > atomic
> > operations from the device. Not sure whether we'd like a new notifier
> > type, or maybe at least comment on why to use CLEAR?
>
> Right, notifying the owner/driver when it no longer has exclusive access to
> the page and allowing it to stop atomic operations is the critical point and
> why it notifies when we ordinarily wouldn't (ie. invalid -> valid).
>
> I did consider adding a new type, but in the driver implementation it ends
> up
> being treated the same as a CLEAR notification anyway so didn't think it was
> necessary. But I suppose adding a different type would allow other listening
> notifiers to filter these which might be worthwhile.
>
> > > +
> > > + while (page_vma_mapped_walk(&pvmw)) {
> >
> > IIUC a while loop of page_vma_mapped_walk() handles thps, however here
> > it's
> > already in do_swap_page() so it's small pte only? Meanwhile...
> >
> > > + if (unlikely(!pte_same(*pvmw.pte, vmf->orig_pte))) {
> > > + page_vma_mapped_walk_done(&pvmw);
> > > + break;
> > > + }
> > > +
> > > + restore_exclusive_pte(vma, page, pvmw.address, pvmw.pte);
> >
> > ... I'm not sure whether passing in page would work for thp after all, as
> > iiuc we may need to pass in the subpage rather than the head page if so.
>
> Hmm, I need to check this and follow up.
Thanks Peter for pointing that out. I needed to follow this up because I had
slightly misunderstood page_vma_mapped_walk(). As you say this is actually a
small pte and restore_exclusive_pte() needs the actual page from the fault. So
I should be able to drop the page_vma_mapped_walk() and use
pte_offset_map_lock() to call restore_exclusive_pte directly.
- Alistair
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau
WARNING: multiple messages have this Message-ID (diff)
From: Alistair Popple <apopple@nvidia.com>
To: Peter Xu <peterx@redhat.com>
Cc: rcampbell@nvidia.com, willy@infradead.org,
linux-doc@vger.kernel.org, nouveau@lists.freedesktop.org,
bsingharora@gmail.com, linux-kernel@vger.kernel.org,
dri-devel@lists.freedesktop.org, hch@infradead.org,
linux-mm@kvack.org, jglisse@redhat.com, bskeggs@redhat.com,
jgg@nvidia.com, jhubbard@nvidia.com, akpm@linux-foundation.org,
Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH v8 5/8] mm: Device exclusive memory access
Date: Fri, 21 May 2021 16:53:29 +1000 [thread overview]
Message-ID: <1959488.yZHLR0KveG@nvdebian> (raw)
In-Reply-To: <47694715.suB6H4Uo8R@nvdebian>
On Tuesday, 18 May 2021 11:19:05 PM AEST Alistair Popple wrote:
[...]
> > > +/*
> > > + * Restore a potential device exclusive pte to a working pte entry
> > > + */
> > > +static vm_fault_t remove_device_exclusive_entry(struct vm_fault *vmf)
> > > +{
> > > + struct page *page = vmf->page;
> > > + struct vm_area_struct *vma = vmf->vma;
> > > + struct page_vma_mapped_walk pvmw = {
> > > + .page = page,
> > > + .vma = vma,
> > > + .address = vmf->address,
> > > + .flags = PVMW_SYNC,
> > > + };
> > > + vm_fault_t ret = 0;
> > > + struct mmu_notifier_range range;
> > > +
> > > + if (!lock_page_or_retry(page, vma->vm_mm, vmf->flags))
> > > + return VM_FAULT_RETRY;
> > > + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma,
> > > vma->vm_mm,
> > > + vmf->address & PAGE_MASK,
> > > + (vmf->address & PAGE_MASK) + PAGE_SIZE);
> > > + mmu_notifier_invalidate_range_start(&range);
> >
> > I looked at MMU_NOTIFIER_CLEAR document and it tells me:
> >
> > * @MMU_NOTIFY_CLEAR: clear page table entry (many reasons for this like
> > * madvise() or replacing a page by another one, ...).
> >
> > Does MMU_NOTIFIER_CLEAR suite for this case? Normally I think for such a
> > case (existing pte is invalid) we don't need to notify at all. However
> > from what I read from the whole series, this seems to be a critical point
> > where we'd like to kick the owner/driver to synchronously stop doing
> > atomic
> > operations from the device. Not sure whether we'd like a new notifier
> > type, or maybe at least comment on why to use CLEAR?
>
> Right, notifying the owner/driver when it no longer has exclusive access to
> the page and allowing it to stop atomic operations is the critical point and
> why it notifies when we ordinarily wouldn't (ie. invalid -> valid).
>
> I did consider adding a new type, but in the driver implementation it ends
> up
> being treated the same as a CLEAR notification anyway so didn't think it was
> necessary. But I suppose adding a different type would allow other listening
> notifiers to filter these which might be worthwhile.
>
> > > +
> > > + while (page_vma_mapped_walk(&pvmw)) {
> >
> > IIUC a while loop of page_vma_mapped_walk() handles thps, however here
> > it's
> > already in do_swap_page() so it's small pte only? Meanwhile...
> >
> > > + if (unlikely(!pte_same(*pvmw.pte, vmf->orig_pte))) {
> > > + page_vma_mapped_walk_done(&pvmw);
> > > + break;
> > > + }
> > > +
> > > + restore_exclusive_pte(vma, page, pvmw.address, pvmw.pte);
> >
> > ... I'm not sure whether passing in page would work for thp after all, as
> > iiuc we may need to pass in the subpage rather than the head page if so.
>
> Hmm, I need to check this and follow up.
Thanks Peter for pointing that out. I needed to follow this up because I had
slightly misunderstood page_vma_mapped_walk(). As you say this is actually a
small pte and restore_exclusive_pte() needs the actual page from the fault. So
I should be able to drop the page_vma_mapped_walk() and use
pte_offset_map_lock() to call restore_exclusive_pte directly.
- Alistair
next prev parent reply other threads:[~2021-05-21 6:53 UTC|newest]
Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-07 8:42 [PATCH v8 0/8] Add support for SVM atomics in Nouveau Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-04-07 8:42 ` [PATCH v8 1/8] mm: Remove special swap entry functions Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-05-18 2:17 ` Peter Xu
2021-05-18 2:17 ` Peter Xu
2021-05-18 2:17 ` [Nouveau] " Peter Xu
2021-05-18 11:58 ` Alistair Popple
2021-05-18 11:58 ` Alistair Popple
2021-05-18 11:58 ` [Nouveau] " Alistair Popple
2021-05-18 14:17 ` Peter Xu
2021-05-18 14:17 ` Peter Xu
2021-05-18 14:17 ` [Nouveau] " Peter Xu
2021-04-07 8:42 ` [PATCH v8 2/8] mm/swapops: Rework swap entry manipulation code Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-04-07 8:42 ` [PATCH v8 3/8] mm/rmap: Split try_to_munlock from try_to_unmap Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-05-18 20:04 ` Liam Howlett
2021-05-18 20:04 ` Liam Howlett
2021-05-18 20:04 ` [Nouveau] " Liam Howlett
2021-05-19 12:38 ` Alistair Popple
2021-05-19 12:38 ` Alistair Popple
2021-05-19 12:38 ` [Nouveau] " Alistair Popple
2021-05-20 20:24 ` Liam Howlett
2021-05-20 20:24 ` Liam Howlett
2021-05-20 20:24 ` [Nouveau] " Liam Howlett
2021-05-21 2:23 ` Alistair Popple
2021-05-21 2:23 ` Alistair Popple
2021-05-21 2:23 ` [Nouveau] " Alistair Popple
2021-04-07 8:42 ` [PATCH v8 4/8] mm/rmap: Split migration into its own function Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-04-07 8:42 ` [PATCH v8 5/8] mm: Device exclusive memory access Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-05-18 2:08 ` Peter Xu
2021-05-18 2:08 ` Peter Xu
2021-05-18 2:08 ` [Nouveau] " Peter Xu
2021-05-18 13:19 ` Alistair Popple
2021-05-18 13:19 ` Alistair Popple
2021-05-18 13:19 ` [Nouveau] " Alistair Popple
2021-05-18 17:27 ` Peter Xu
2021-05-18 17:27 ` Peter Xu
2021-05-18 17:27 ` [Nouveau] " Peter Xu
2021-05-18 17:33 ` Jason Gunthorpe
2021-05-18 17:33 ` Jason Gunthorpe
2021-05-18 17:33 ` [Nouveau] " Jason Gunthorpe
2021-05-18 18:01 ` Peter Xu
2021-05-18 18:01 ` Peter Xu
2021-05-18 18:01 ` [Nouveau] " Peter Xu
2021-05-18 19:45 ` Jason Gunthorpe
2021-05-18 19:45 ` Jason Gunthorpe
2021-05-18 19:45 ` [Nouveau] " Jason Gunthorpe
2021-05-18 20:29 ` Peter Xu
2021-05-18 20:29 ` Peter Xu
2021-05-18 20:29 ` [Nouveau] " Peter Xu
2021-05-18 23:03 ` Jason Gunthorpe
2021-05-18 23:03 ` Jason Gunthorpe
2021-05-18 23:03 ` [Nouveau] " Jason Gunthorpe
2021-05-18 23:45 ` Peter Xu
2021-05-18 23:45 ` Peter Xu
2021-05-18 23:45 ` [Nouveau] " Peter Xu
2021-05-19 11:04 ` Alistair Popple
2021-05-19 11:04 ` Alistair Popple
2021-05-19 11:04 ` [Nouveau] " Alistair Popple
2021-05-19 12:15 ` Peter Xu
2021-05-19 12:15 ` Peter Xu
2021-05-19 12:15 ` [Nouveau] " Peter Xu
2021-05-19 13:11 ` Alistair Popple
2021-05-19 13:11 ` Alistair Popple
2021-05-19 13:11 ` [Nouveau] " Alistair Popple
2021-05-19 14:04 ` Peter Xu
2021-05-19 14:04 ` Peter Xu
2021-05-19 14:04 ` [Nouveau] " Peter Xu
2021-05-19 13:28 ` Jason Gunthorpe
2021-05-19 13:28 ` Jason Gunthorpe
2021-05-19 13:28 ` [Nouveau] " Jason Gunthorpe
2021-05-19 14:09 ` Peter Xu
2021-05-19 14:09 ` Peter Xu
2021-05-19 14:09 ` [Nouveau] " Peter Xu
2021-05-19 18:11 ` Jason Gunthorpe
2021-05-19 18:11 ` Jason Gunthorpe
2021-05-19 18:11 ` [Nouveau] " Jason Gunthorpe
2021-05-19 11:35 ` Alistair Popple
2021-05-19 11:35 ` Alistair Popple
2021-05-19 11:35 ` [Nouveau] " Alistair Popple
2021-05-19 12:21 ` Peter Xu
2021-05-19 12:21 ` Peter Xu
2021-05-19 12:21 ` [Nouveau] " Peter Xu
2021-05-19 12:46 ` Alistair Popple
2021-05-19 12:46 ` Alistair Popple
2021-05-19 12:46 ` [Nouveau] " Alistair Popple
2021-05-21 6:53 ` Alistair Popple [this message]
2021-05-21 6:53 ` Alistair Popple
2021-05-21 6:53 ` [Nouveau] " Alistair Popple
2021-05-18 21:16 ` Peter Xu
2021-05-18 21:16 ` Peter Xu
2021-05-18 21:16 ` [Nouveau] " Peter Xu
2021-05-19 10:49 ` Alistair Popple
2021-05-19 10:49 ` Alistair Popple
2021-05-19 10:49 ` [Nouveau] " Alistair Popple
2021-05-19 12:24 ` Peter Xu
2021-05-19 12:24 ` Peter Xu
2021-05-19 12:24 ` [Nouveau] " Peter Xu
2021-05-19 12:46 ` Alistair Popple
2021-05-19 12:46 ` Alistair Popple
2021-05-19 12:46 ` [Nouveau] " Alistair Popple
2021-04-07 8:42 ` [PATCH v8 6/8] mm: Selftests for exclusive device memory Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-04-07 8:42 ` [PATCH v8 7/8] nouveau/svm: Refactor nouveau_range_fault Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-04-07 8:42 ` [PATCH v8 8/8] nouveau/svm: Implement atomic SVM access Alistair Popple
2021-04-07 8:42 ` Alistair Popple
2021-04-07 8:42 ` [Nouveau] " Alistair Popple
2021-05-21 4:04 ` Ben Skeggs
2021-05-21 4:04 ` Ben Skeggs
2021-05-21 4:04 ` [Nouveau] " Ben Skeggs
2021-05-21 4:04 ` Ben Skeggs
2021-05-06 7:43 ` [PATCH v8 0/8] Add support for SVM atomics in Nouveau Alistair Popple
2021-05-06 7:43 ` Alistair Popple
2021-05-06 7:43 ` [Nouveau] " Alistair Popple
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1959488.yZHLR0KveG@nvdebian \
--to=apopple@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=bsingharora@gmail.com \
--cc=bskeggs@redhat.com \
--cc=daniel@ffwll.ch \
--cc=dri-devel@lists.freedesktop.org \
--cc=hch@infradead.org \
--cc=hch@lst.de \
--cc=jgg@nvidia.com \
--cc=jglisse@redhat.com \
--cc=jhubbard@nvidia.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=nouveau@lists.freedesktop.org \
--cc=peterx@redhat.com \
--cc=rcampbell@nvidia.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.