From: Peter Xu <peterx@redhat.com>
To: linux-kernel@vger.kernel.org, linux-mm@kvack.org
Cc: peterx@redhat.com, Jason Gunthorpe <jgg@ziepe.ca>,
John Hubbard <jhubbard@nvidia.com>,
Andrew Morton <akpm@linux-foundation.org>,
Christoph Hellwig <hch@lst.de>, Yang Shi <shy828301@gmail.com>,
Oleg Nesterov <oleg@redhat.com>,
Kirill Tkhai <ktkhai@virtuozzo.com>,
Kirill Shutemov <kirill@shutemov.name>,
Hugh Dickins <hughd@google.com>, Jann Horn <jannh@google.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Michal Hocko <mhocko@suse.com>, Jan Kara <jack@suse.cz>,
Andrea Arcangeli <aarcange@redhat.com>,
Leon Romanovsky <leonro@nvidia.com>
Subject: [PATCH v2 1/4] mm: Introduce mm_struct.has_pinned
Date: Fri, 25 Sep 2020 18:25:57 -0400 [thread overview]
Message-ID: <20200925222600.6832-2-peterx@redhat.com> (raw)
In-Reply-To: <20200925222600.6832-1-peterx@redhat.com>
(Commit message majorly collected from Jason Gunthorpe)
Reduce the chance of false positive from page_maybe_dma_pinned() by keeping
track if the mm_struct has ever been used with pin_user_pages(). This allows
cases that might drive up the page ref_count to avoid any penalty from handling
dma_pinned pages.
Future work is planned, to provide a more sophisticated solution, likely to
turn it into a real counter. For now, make it atomic_t but use it as a boolean
for simplicity.
Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
include/linux/mm_types.h | 10 ++++++++++
kernel/fork.c | 1 +
mm/gup.c | 6 ++++++
3 files changed, 17 insertions(+)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 496c3ff97cce..ed028af3cb19 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -436,6 +436,16 @@ struct mm_struct {
*/
atomic_t mm_count;
+ /**
+ * @has_pinned: Whether this mm has pinned any pages. This can
+ * be either replaced in the future by @pinned_vm when it
+ * becomes stable, or grow into a counter on its own. We're
+ * aggresive on this bit now - even if the pinned pages were
+ * unpinned later on, we'll still keep this bit set for the
+ * lifecycle of this mm just for simplicity.
+ */
+ atomic_t has_pinned;
+
#ifdef CONFIG_MMU
atomic_long_t pgtables_bytes; /* PTE page table pages */
#endif
diff --git a/kernel/fork.c b/kernel/fork.c
index 49677d668de4..e65d8192d080 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1011,6 +1011,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p,
mm_pgtables_bytes_init(mm);
mm->map_count = 0;
mm->locked_vm = 0;
+ atomic_set(&mm->has_pinned, 0);
atomic64_set(&mm->pinned_vm, 0);
memset(&mm->rss_stat, 0, sizeof(mm->rss_stat));
spin_lock_init(&mm->page_table_lock);
diff --git a/mm/gup.c b/mm/gup.c
index e5739a1974d5..238667445337 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1255,6 +1255,9 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm,
BUG_ON(*locked != 1);
}
+ if (flags & FOLL_PIN)
+ atomic_set(¤t->mm->has_pinned, 1);
+
/*
* FOLL_PIN and FOLL_GET are mutually exclusive. Traditional behavior
* is to set FOLL_GET if the caller wants pages[] filled in (but has
@@ -2660,6 +2663,9 @@ static int internal_get_user_pages_fast(unsigned long start, int nr_pages,
FOLL_FAST_ONLY)))
return -EINVAL;
+ if (gup_flags & FOLL_PIN)
+ atomic_set(¤t->mm->has_pinned, 1);
+
if (!(gup_flags & FOLL_FAST_ONLY))
might_lock_read(¤t->mm->mmap_lock);
--
2.26.2
next prev parent reply other threads:[~2020-09-25 22:26 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-25 22:25 [PATCH v2 0/4] mm: Break COW for pinned pages during fork() Peter Xu
2020-09-25 22:25 ` Peter Xu [this message]
2020-09-25 22:25 ` [PATCH v2 2/4] mm/fork: Pass new vma pointer into copy_page_range() Peter Xu
2020-09-30 13:30 ` Kirill A. Shutemov
2020-09-30 17:05 ` Peter Xu
2020-09-25 22:25 ` [PATCH v2 3/4] mm: Do early cow for pinned pages during fork() for ptes Peter Xu
2020-09-26 23:23 ` Jason Gunthorpe
2020-09-27 0:04 ` Linus Torvalds
2020-09-25 22:26 ` [PATCH v2 4/4] mm/thp: Split huge pmds/puds if they're pinned when fork() Peter Xu
2020-09-27 19:35 ` [PATCH v2 0/4] mm: Break COW for pinned pages during fork() Linus Torvalds
2020-09-29 11:02 ` Leon Romanovsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200925222600.6832-2-peterx@redhat.com \
--to=peterx@redhat.com \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=hch@lst.de \
--cc=hughd@google.com \
--cc=jack@suse.cz \
--cc=jannh@google.com \
--cc=jgg@ziepe.ca \
--cc=jhubbard@nvidia.com \
--cc=kirill@shutemov.name \
--cc=ktkhai@virtuozzo.com \
--cc=leonro@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=oleg@redhat.com \
--cc=shy828301@gmail.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).