From: Davidlohr Bueso <dave@stgolabs.net>
To: jgg@ziepe.ca, akpm@linux-foundation.org
Cc: dledford@redhat.com, jgg@mellanox.com, jack@suse.cz,
willy@infradead.org, ira.weiny@intel.com,
linux-rdma@vger.kernel.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org, dave@stgolabs.net,
dennis.dalessandro@intel.com, mike.marciniszyn@intel.com,
Davidlohr Bueso <dbueso@suse.de>
Subject: [PATCH 3/6] drivers/IB,qib: optimize mmap_sem usage
Date: Wed, 6 Feb 2019 09:59:17 -0800 [thread overview]
Message-ID: <20190206175920.31082-4-dave@stgolabs.net> (raw)
In-Reply-To: <20190206175920.31082-1-dave@stgolabs.net>
The driver uses mmap_sem for both pinned_vm accounting and
get_user_pages(). Because rdma drivers might want to use
gup_longterm() in the future we still need some sort of
mmap_sem serialization (as opposed to removing it entirely
by using gup_fast()). Now that pinned_vm is atomic the
writer lock can therefore be converted to reader.
This also fixes a bug that __qib_get_user_pages was not
taking into account the current value of pinned_vm.
Cc: dennis.dalessandro@intel.com
Cc: mike.marciniszyn@intel.com
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
drivers/infiniband/hw/qib/qib_user_pages.c | 73 +++++++++++-------------------
1 file changed, 27 insertions(+), 46 deletions(-)
diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c
index c6c81022d313..ef8bcf366ddc 100644
--- a/drivers/infiniband/hw/qib/qib_user_pages.c
+++ b/drivers/infiniband/hw/qib/qib_user_pages.c
@@ -49,43 +49,6 @@ static void __qib_release_user_pages(struct page **p, size_t num_pages,
}
}
-/*
- * Call with current->mm->mmap_sem held.
- */
-static int __qib_get_user_pages(unsigned long start_page, size_t num_pages,
- struct page **p)
-{
- unsigned long lock_limit;
- size_t got;
- int ret;
-
- lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
-
- if (num_pages > lock_limit && !capable(CAP_IPC_LOCK)) {
- ret = -ENOMEM;
- goto bail;
- }
-
- for (got = 0; got < num_pages; got += ret) {
- ret = get_user_pages_longterm(start_page + got * PAGE_SIZE,
- num_pages - got,
- FOLL_WRITE | FOLL_FORCE,
- p + got, NULL);
- if (ret < 0)
- goto bail_release;
- }
-
- atomic64_add(num_pages, ¤t->mm->pinned_vm);
-
- ret = 0;
- goto bail;
-
-bail_release:
- __qib_release_user_pages(p, got, 0);
-bail:
- return ret;
-}
-
/**
* qib_map_page - a safety wrapper around pci_map_page()
*
@@ -137,26 +100,44 @@ int qib_map_page(struct pci_dev *hwdev, struct page *page, dma_addr_t *daddr)
int qib_get_user_pages(unsigned long start_page, size_t num_pages,
struct page **p)
{
+ unsigned long locked, lock_limit;
+ size_t got;
int ret;
- down_write(¤t->mm->mmap_sem);
+ lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
+ locked = atomic64_add_return(num_pages, ¤t->mm->pinned_vm);
- ret = __qib_get_user_pages(start_page, num_pages, p);
+ if (num_pages > lock_limit && !capable(CAP_IPC_LOCK)) {
+ ret = -ENOMEM;
+ goto bail;
+ }
- up_write(¤t->mm->mmap_sem);
+ down_read(¤t->mm->mmap_sem);
+ for (got = 0; got < num_pages; got += ret) {
+ ret = get_user_pages_longterm(start_page + got * PAGE_SIZE,
+ num_pages - got,
+ FOLL_WRITE | FOLL_FORCE,
+ p + got, NULL);
+ if (ret < 0) {
+ up_read(¤t->mm->mmap_sem);
+ goto bail_release;
+ }
+ }
+ up_read(¤t->mm->mmap_sem);
+ return 0;
+bail_release:
+ __qib_release_user_pages(p, got, 0);
+bail:
+ atomic64_sub(num_pages, ¤t->mm->pinned_vm);
return ret;
}
void qib_release_user_pages(struct page **p, size_t num_pages)
{
- if (current->mm) /* during close after signal, mm can be NULL */
- down_write(¤t->mm->mmap_sem);
-
__qib_release_user_pages(p, num_pages, 1);
- if (current->mm) {
+ /* during close after signal, mm can be NULL */
+ if (current->mm)
atomic64_sub(num_pages, ¤t->mm->pinned_vm);
- up_write(¤t->mm->mmap_sem);
- }
}
--
2.16.4
next prev parent reply other threads:[~2019-02-06 18:00 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-06 17:59 [PATCH v3 0/6] mm: make pinned_vm atomic and simplify users Davidlohr Bueso
2019-02-06 17:59 ` [PATCH 1/6] mm: make mm->pinned_vm an atomic64 counter Davidlohr Bueso
2019-02-06 17:59 ` [PATCH 2/6] drivers/mic/scif: do not use mmap_sem Davidlohr Bueso
2019-02-06 17:59 ` Davidlohr Bueso [this message]
2019-02-06 17:59 ` [PATCH 4/6] drivers/IB,hfi1: do not se mmap_sem Davidlohr Bueso
2019-02-06 17:59 ` [PATCH 5/6] drivers/IB,usnic: reduce scope of mmap_sem Davidlohr Bueso
2019-02-06 17:59 ` [PATCH 6/6] drivers/IB,core: " Davidlohr Bueso
2019-02-07 1:31 ` [PATCH 7/6] Documentation/infiniband: update from locked to pinned_vm Davidlohr Bueso
2019-02-12 0:35 ` Weiny, Ira
2019-02-07 20:12 ` [PATCH v3 0/6] mm: make pinned_vm atomic and simplify users Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190206175920.31082-4-dave@stgolabs.net \
--to=dave@stgolabs.net \
--cc=akpm@linux-foundation.org \
--cc=dbueso@suse.de \
--cc=dennis.dalessandro@intel.com \
--cc=dledford@redhat.com \
--cc=ira.weiny@intel.com \
--cc=jack@suse.cz \
--cc=jgg@mellanox.com \
--cc=jgg@ziepe.ca \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mike.marciniszyn@intel.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).