From: Ralph Campbell <rcampbell@nvidia.com>
To: Alistair Popple <apopple@nvidia.com>, <linux-mm@kvack.org>,
<nouveau@lists.freedesktop.org>, <bskeggs@redhat.com>,
<akpm@linux-foundation.org>
Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
dri-devel@lists.freedesktop.org, kvm-ppc@vger.kernel.org
Subject: Re: [Nouveau] [PATCH v4 5/8] mm: Device exclusive memory access
Date: Mon, 8 Mar 2021 11:44:41 -0800 [thread overview]
Message-ID: <ac380c1c-20f4-7c5b-dc5b-be6e1e970921@nvidia.com> (raw)
In-Reply-To: <20210304061645.29747-6-apopple@nvidia.com>
On 3/3/21 10:16 PM, Alistair Popple wrote:
> Some devices require exclusive write access to shared virtual
> memory (SVM) ranges to perform atomic operations on that memory. This
> requires CPU page tables to be updated to deny access whilst atomic
> operations are occurring.
>
> In order to do this introduce a new swap entry
> type (SWP_DEVICE_EXCLUSIVE). When a SVM range needs to be marked for
> exclusive access by a device all page table mappings for the particular
> range are replaced with device exclusive swap entries. This causes any
> CPU access to the page to result in a fault.
>
> Faults are resovled by replacing the faulting entry with the original
> mapping. This results in MMU notifiers being called which a driver uses
> to update access permissions such as revoking atomic access. After
> notifiers have been called the device will no longer have exclusive
> access to the region.
>
> Signed-off-by: Alistair Popple <apopple@nvidia.com>
I see in the next two patches how make_device_exclusive_entry() and
check_device_exclusive_range() are used. This points out a similar
problem that migrate_vma_setup() had before I added the
mmu_notifier_range_init_migrate() helper to pass a cookie from
migrate_vma_setup() to the invalidation callback so the device driver
could ignore an invalidation callback triggered by the caller and thus
resulting in a deadlock or having to invalidate device PTEs that
wouldn't be migrating.
I think you can eliminate the need for check_device_exclusive_range() in
the same way by adding a "void *" pointer to make_device_exclusive_entry()
and passing that through to try_to_protect(), setting rmap_walk_control rwc.arg
and then passing arg to mmu_notifier_range_init_migrate().
Although, maybe it would be better to define a new
mmu_notifier_range_init_exclusive() and event type MMU_NOTIFY_EXCLUSIVE so
that a device driver can revoke atomic/exclusive access but keep read/write
access to other parts of the page.
I thought about how make_device_exclusive_entry() is similar to hmm_range_fault()
and whether it would be possible to add a new HMM_PFN_REQ_EXCLUSIVE flag but I
see that make_device_exclusive_entry() returns the pages locked and with an
additional get_page() reference. This doesn't fit well with the other
hmm_range_fault() entries being returned as a "snapshot" so having a different
API makes sense. I think it would be useful to add a HMM_PFN_EXCLUSIVE flag so
that snapshots of the page tables can at least report that a page is exclusively
being accessed by *some* device. Unfortunately, there is no pgmap pointer to be
able to tell which device has exclusive access (since any struct page could be
exclusively accessed, not just device private ones).
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau
next prev parent reply other threads:[~2021-03-08 19:44 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-04 6:16 [Nouveau] [PATCH v4 0/8] Add support for SVM atomics in Nouveau Alistair Popple
2021-03-04 6:16 ` [Nouveau] [PATCH v4 1/8] mm: Remove special swap entry functions Alistair Popple
2021-03-08 18:24 ` Ralph Campbell
2021-03-04 6:16 ` [Nouveau] [PATCH v4 2/8] mm/swapops: Rework swap entry manipulation code Alistair Popple
2021-03-08 18:30 ` Ralph Campbell
2021-03-04 6:16 ` [Nouveau] [PATCH v4 3/8] mm/rmap: Split try_to_munlock from try_to_unmap Alistair Popple
2021-03-08 18:39 ` Ralph Campbell
2021-03-04 6:16 ` [Nouveau] [PATCH v4 4/8] mm/rmap: Split migration into its own function Alistair Popple
2021-03-08 18:58 ` Ralph Campbell
2021-03-09 0:01 ` Alistair Popple
2021-03-04 6:16 ` [Nouveau] [PATCH v4 5/8] mm: Device exclusive memory access Alistair Popple
2021-03-08 19:44 ` Ralph Campbell [this message]
2021-03-09 1:17 ` Alistair Popple
2021-03-04 6:16 ` [Nouveau] [PATCH v4 6/8] mm: Selftests for exclusive device memory Alistair Popple
2021-03-04 6:16 ` [Nouveau] [PATCH v4 7/8] nouveau/svm: Refactor nouveau_range_fault Alistair Popple
2021-03-04 6:16 ` [Nouveau] [PATCH v4 8/8] nouveau/svm: Implement atomic SVM access Alistair Popple
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ac380c1c-20f4-7c5b-dc5b-be6e1e970921@nvidia.com \
--to=rcampbell@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=apopple@nvidia.com \
--cc=bskeggs@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=kvm-ppc@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=nouveau@lists.freedesktop.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).