From: Jason Gunthorpe <jgg@ziepe.ca>
To: linux-mm@kvack.org, Jerome Glisse <jglisse@redhat.com>,
Ralph Campbell <rcampbell@nvidia.com>,
John Hubbard <jhubbard@nvidia.com>,
Felix.Kuehling@amd.com
Cc: "Juergen Gross" <jgross@suse.com>,
"David Zhou" <David1.Zhou@amd.com>,
"Mike Marciniszyn" <mike.marciniszyn@intel.com>,
"Stefano Stabellini" <sstabellini@kernel.org>,
"Philip Yang" <Philip.Yang@amd.com>,
"Oleksandr Andrushchenko" <oleksandr_andrushchenko@epam.com>,
linux-rdma@vger.kernel.org, nouveau@lists.freedesktop.org,
"Dennis Dalessandro" <dennis.dalessandro@intel.com>,
amd-gfx@lists.freedesktop.org,
"Christoph Hellwig" <hch@infradead.org>,
"Jason Gunthorpe" <jgg@mellanox.com>,
dri-devel@lists.freedesktop.org,
"Alex Deucher" <alexander.deucher@amd.com>,
xen-devel@lists.xenproject.org,
"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
"Petr Cvek" <petrcvekcz@gmail.com>,
"Christian König" <christian.koenig@amd.com>,
"Ben Skeggs" <bskeggs@redhat.com>
Subject: [PATCH v3 03/14] mm/hmm: allow hmm_range to be used with a mmu_interval_notifier or hmm_mirror
Date: Tue, 12 Nov 2019 16:22:20 -0400 [thread overview]
Message-ID: <20191112202231.3856-4-jgg@ziepe.ca> (raw)
Message-ID: <20191112202220.JhkSQHdAulome_lQRc8h5H8AklxtmneX8ZLJmPqlrXQ@z> (raw)
In-Reply-To: <20191112202231.3856-1-jgg@ziepe.ca>
From: Jason Gunthorpe <jgg@mellanox.com>
hmm_mirror's handling of ranges does not use a sequence count which
results in this bug:
CPU0 CPU1
hmm_range_wait_until_valid(range)
valid == true
hmm_range_fault(range)
hmm_invalidate_range_start()
range->valid = false
hmm_invalidate_range_end()
range->valid = true
hmm_range_valid(range)
valid == true
Where the hmm_range_valid() should not have succeeded.
Adding the required sequence count would make it nearly identical to the
new mmu_interval_notifier. Instead replace the hmm_mirror stuff with
mmu_interval_notifier.
Co-existence of the two APIs is the first step.
Reviewed-by: Jérôme Glisse <jglisse@redhat.com>
Tested-by: Philip Yang <Philip.Yang@amd.com>
Tested-by: Ralph Campbell <rcampbell@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
---
include/linux/hmm.h | 5 +++++
mm/hmm.c | 25 +++++++++++++++++++------
2 files changed, 24 insertions(+), 6 deletions(-)
diff --git a/include/linux/hmm.h b/include/linux/hmm.h
index 3fec513b9c00f1..fbb35c78637e57 100644
--- a/include/linux/hmm.h
+++ b/include/linux/hmm.h
@@ -145,6 +145,9 @@ enum hmm_pfn_value_e {
/*
* struct hmm_range - track invalidation lock on virtual address range
*
+ * @notifier: an optional mmu_interval_notifier
+ * @notifier_seq: when notifier is used this is the result of
+ * mmu_interval_read_begin()
* @hmm: the core HMM structure this range is active against
* @vma: the vm area struct for the range
* @list: all range lock are on a list
@@ -159,6 +162,8 @@ enum hmm_pfn_value_e {
* @valid: pfns array did not change since it has been fill by an HMM function
*/
struct hmm_range {
+ struct mmu_interval_notifier *notifier;
+ unsigned long notifier_seq;
struct hmm *hmm;
struct list_head list;
unsigned long start;
diff --git a/mm/hmm.c b/mm/hmm.c
index 6b0136665407a3..8d060c5dabe37b 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -858,6 +858,14 @@ void hmm_range_unregister(struct hmm_range *range)
}
EXPORT_SYMBOL(hmm_range_unregister);
+static bool needs_retry(struct hmm_range *range)
+{
+ if (range->notifier)
+ return mmu_interval_check_retry(range->notifier,
+ range->notifier_seq);
+ return !range->valid;
+}
+
static const struct mm_walk_ops hmm_walk_ops = {
.pud_entry = hmm_vma_walk_pud,
.pmd_entry = hmm_vma_walk_pmd,
@@ -898,18 +906,23 @@ long hmm_range_fault(struct hmm_range *range, unsigned int flags)
const unsigned long device_vma = VM_IO | VM_PFNMAP | VM_MIXEDMAP;
unsigned long start = range->start, end;
struct hmm_vma_walk hmm_vma_walk;
- struct hmm *hmm = range->hmm;
+ struct mm_struct *mm;
struct vm_area_struct *vma;
int ret;
- lockdep_assert_held(&hmm->mmu_notifier.mm->mmap_sem);
+ if (range->notifier)
+ mm = range->notifier->mm;
+ else
+ mm = range->hmm->mmu_notifier.mm;
+
+ lockdep_assert_held(&mm->mmap_sem);
do {
/* If range is no longer valid force retry. */
- if (!range->valid)
+ if (needs_retry(range))
return -EBUSY;
- vma = find_vma(hmm->mmu_notifier.mm, start);
+ vma = find_vma(mm, start);
if (vma == NULL || (vma->vm_flags & device_vma))
return -EFAULT;
@@ -939,7 +952,7 @@ long hmm_range_fault(struct hmm_range *range, unsigned int flags)
start = hmm_vma_walk.last;
/* Keep trying while the range is valid. */
- } while (ret == -EBUSY && range->valid);
+ } while (ret == -EBUSY && !needs_retry(range));
if (ret) {
unsigned long i;
@@ -997,7 +1010,7 @@ long hmm_range_dma_map(struct hmm_range *range, struct device *device,
continue;
/* Check if range is being invalidated */
- if (!range->valid) {
+ if (needs_retry(range)) {
ret = -EBUSY;
goto unmap;
}
--
2.24.0
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2019-11-12 20:46 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-12 20:22 [PATCH hmm v3 00/14] Consolidate the mmu notifier interval_tree and locking Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 01/14] mm/mmu_notifier: define the header pre-processor parts even if disabled Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-13 13:52 ` Christoph Hellwig
2019-11-13 13:52 ` Christoph Hellwig
2019-11-12 20:22 ` [PATCH v3 02/14] mm/mmu_notifier: add an interval tree notifier Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-13 13:59 ` Christoph Hellwig
2019-11-13 13:59 ` Christoph Hellwig
2019-11-13 16:46 ` Jason Gunthorpe
2019-11-13 16:46 ` Jason Gunthorpe
2019-11-23 0:54 ` Ralph Campbell
2019-11-23 0:54 ` Ralph Campbell
2019-11-23 23:59 ` Jason Gunthorpe
2019-11-23 23:59 ` Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe [this message]
2019-11-12 20:22 ` [PATCH v3 03/14] mm/hmm: allow hmm_range to be used with a mmu_interval_notifier or hmm_mirror Jason Gunthorpe
2019-11-13 14:00 ` Christoph Hellwig
2019-11-13 14:00 ` Christoph Hellwig
2019-11-12 20:22 ` [PATCH v3 04/14] mm/hmm: define the pre-processor related parts of hmm.h even if disabled Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-13 14:01 ` Christoph Hellwig
2019-11-13 14:01 ` Christoph Hellwig
2019-11-12 20:22 ` [PATCH v3 05/14] RDMA/odp: Use mmu_interval_notifier_insert() Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 06/14] RDMA/hfi1: Use mmu_interval_notifier_insert for user_exp_rcv Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 07/14] drm/radeon: use mmu_interval_notifier_insert Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 08/14] nouveau: use mmu_notifier directly for invalidate_range_start Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 09/14] nouveau: use mmu_interval_notifier instead of hmm_mirror Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 10/14] drm/amdgpu: Call find_vma under mmap_sem Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 11/14] drm/amdgpu: Use mmu_interval_insert instead of hmm_mirror Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-12 20:22 ` [PATCH v3 12/14] drm/amdgpu: Use mmu_interval_notifier " Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-19 19:59 ` Philip Yang
2019-11-19 19:59 ` Philip Yang
2019-11-12 20:22 ` [PATCH v3 13/14] mm/hmm: remove hmm_mirror and related Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
2019-11-13 14:02 ` Christoph Hellwig
2019-11-13 14:02 ` Christoph Hellwig
2019-11-12 20:22 ` [PATCH v3 14/14] xen/gntdev: use mmu_interval_notifier_insert Jason Gunthorpe
2019-11-12 20:22 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191112202231.3856-4-jgg@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=David1.Zhou@amd.com \
--cc=Felix.Kuehling@amd.com \
--cc=Philip.Yang@amd.com \
--cc=alexander.deucher@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=boris.ostrovsky@oracle.com \
--cc=bskeggs@redhat.com \
--cc=christian.koenig@amd.com \
--cc=dennis.dalessandro@intel.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=hch@infradead.org \
--cc=jgg@mellanox.com \
--cc=jglisse@redhat.com \
--cc=jgross@suse.com \
--cc=jhubbard@nvidia.com \
--cc=linux-mm@kvack.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mike.marciniszyn@intel.com \
--cc=nouveau@lists.freedesktop.org \
--cc=oleksandr_andrushchenko@epam.com \
--cc=petrcvekcz@gmail.com \
--cc=rcampbell@nvidia.com \
--cc=sstabellini@kernel.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).