From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, alexander.shishkin@linux.intel.com,
aneesh.kumar@linux.ibm.com, benh@kernel.crashing.org,
jhubbard@nvidia.com, jolsa@redhat.com, jrdr.linux@gmail.com,
linux-mm@kvack.org, mark.rutland@arm.com, mingo@redhat.com,
mm-commits@vger.kernel.org, mpe@ellerman.id.au,
msuchanek@suse.de, namhyung@kernel.org, paulus@ozlabs.org,
pbonzini@redhat.com, peterz@infradead.org, rppt@linux.ibm.com,
sfr@canb.auug.org.au, torvalds@linux-foundation.org,
willy@infradead.org
Subject: [patch 16/54] mm/gup.c: convert to use get_user_{page|pages}_fast_only()
Date: Sun, 07 Jun 2020 21:40:55 -0700 [thread overview]
Message-ID: <20200608044055.D5U9kex7x%akpm@linux-foundation.org> (raw)
In-Reply-To: <20200607212615.b050e41fac139a1e16fe00bd@linux-foundation.org>
From: Souptick Joarder <jrdr.linux@gmail.com>
Subject: mm/gup.c: convert to use get_user_{page|pages}_fast_only()
API __get_user_pages_fast() renamed to get_user_pages_fast_only() to align
with pin_user_pages_fast_only().
As part of this we will get rid of write parameter. Instead caller will
pass FOLL_WRITE to get_user_pages_fast_only(). This will not change any
existing functionality of the API.
All the callers are changed to pass FOLL_WRITE.
Also introduce get_user_page_fast_only(), and use it in a few places that
hard-code nr_pages to 1.
Updated the documentation of the API.
Link: http://lkml.kernel.org/r/1590396812-31277-1-git-send-email-jrdr.linux@gmail.com
Signed-off-by: Souptick Joarder <jrdr.linux@gmail.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Paul Mackerras <paulus@ozlabs.org> [arch/powerpc/kvm]
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Mike Rapoport <rppt@linux.ibm.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Michal Suchanek <msuchanek@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 -
arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 -
arch/powerpc/perf/callchain_64.c | 4 ---
include/linux/mm.h | 10 ++++++-
kernel/events/core.c | 4 +--
mm/gup.c | 29 ++++++++++++-----------
virt/kvm/kvm_main.c | 8 ++----
7 files changed, 32 insertions(+), 27 deletions(-)
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c~mm-gupc-convert-to-use-get_user_pagepages_fast_only
+++ a/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -581,7 +581,7 @@ int kvmppc_book3s_hv_page_fault(struct k
* We always ask for write permission since the common case
* is that the page is writable.
*/
- if (__get_user_pages_fast(hva, 1, 1, &page) == 1) {
+ if (get_user_page_fast_only(hva, FOLL_WRITE, &page)) {
write_ok = true;
} else {
/* Call KVM generic code to do the slow-path check */
--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c~mm-gupc-convert-to-use-get_user_pagepages_fast_only
+++ a/arch/powerpc/kvm/book3s_64_mmu_radix.c
@@ -795,7 +795,7 @@ int kvmppc_book3s_instantiate_page(struc
* is that the page is writable.
*/
hva = gfn_to_hva_memslot(memslot, gfn);
- if (!kvm_ro && __get_user_pages_fast(hva, 1, 1, &page) == 1) {
+ if (!kvm_ro && get_user_page_fast_only(hva, FOLL_WRITE, &page)) {
upgrade_write = true;
} else {
unsigned long pfn;
--- a/arch/powerpc/perf/callchain_64.c~mm-gupc-convert-to-use-get_user_pagepages_fast_only
+++ a/arch/powerpc/perf/callchain_64.c
@@ -30,11 +30,9 @@ int read_user_stack_slow(void __user *pt
unsigned long addr = (unsigned long) ptr;
unsigned long offset;
struct page *page;
- int nrpages;
void *kaddr;
- nrpages = __get_user_pages_fast(addr, 1, 1, &page);
- if (nrpages == 1) {
+ if (get_user_page_fast_only(addr, FOLL_WRITE, &page)) {
kaddr = page_address(page);
/* align address to page boundary */
--- a/include/linux/mm.h~mm-gupc-convert-to-use-get_user_pagepages_fast_only
+++ a/include/linux/mm.h
@@ -1824,10 +1824,16 @@ extern int mprotect_fixup(struct vm_area
/*
* doesn't attempt to fault and will return short.
*/
-int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
- struct page **pages);
+int get_user_pages_fast_only(unsigned long start, int nr_pages,
+ unsigned int gup_flags, struct page **pages);
int pin_user_pages_fast_only(unsigned long start, int nr_pages,
unsigned int gup_flags, struct page **pages);
+
+static inline bool get_user_page_fast_only(unsigned long addr,
+ unsigned int gup_flags, struct page **pagep)
+{
+ return get_user_pages_fast_only(addr, 1, gup_flags, pagep) == 1;
+}
/*
* per-process(per-mm_struct) statistics.
*/
--- a/kernel/events/core.c~mm-gupc-convert-to-use-get_user_pagepages_fast_only
+++ a/kernel/events/core.c
@@ -6934,12 +6934,12 @@ static u64 perf_virt_to_phys(u64 virt)
* Walking the pages tables for user address.
* Interrupts are disabled, so it prevents any tear down
* of the page tables.
- * Try IRQ-safe __get_user_pages_fast first.
+ * Try IRQ-safe get_user_page_fast_only first.
* If failed, leave phys_addr as 0.
*/
if (current->mm != NULL) {
pagefault_disable();
- if (__get_user_pages_fast(virt, 1, 0, &p) == 1)
+ if (get_user_page_fast_only(virt, 0, &p))
phys_addr = page_to_phys(p) + virt % PAGE_SIZE;
pagefault_enable();
}
--- a/mm/gup.c~mm-gupc-convert-to-use-get_user_pagepages_fast_only
+++ a/mm/gup.c
@@ -2294,7 +2294,7 @@ pte_unmap:
* to be special.
*
* For a futex to be placed on a THP tail page, get_futex_key requires a
- * __get_user_pages_fast implementation that can pin pages. Thus it's still
+ * get_user_pages_fast_only implementation that can pin pages. Thus it's still
* useful to have gup_huge_pmd even if we can't operate on ptes.
*/
static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
@@ -2699,7 +2699,7 @@ static inline void gup_pgd_range(unsigne
#ifndef gup_fast_permitted
/*
- * Check if it's allowed to use __get_user_pages_fast() for the range, or
+ * Check if it's allowed to use get_user_pages_fast_only() for the range, or
* we need to fall back to the slow version:
*/
static bool gup_fast_permitted(unsigned long start, unsigned long end)
@@ -2811,8 +2811,14 @@ static int internal_get_user_pages_fast(
return ret;
}
-
-/*
+/**
+ * get_user_pages_fast_only() - pin user pages in memory
+ * @start: starting user address
+ * @nr_pages: number of pages from start to pin
+ * @gup_flags: flags modifying pin behaviour
+ * @pages: array that receives pointers to the pages pinned.
+ * Should be at least nr_pages long.
+ *
* Like get_user_pages_fast() except it's IRQ-safe in that it won't fall back to
* the regular GUP.
* Note a difference with get_user_pages_fast: this always returns the
@@ -2825,8 +2831,8 @@ static int internal_get_user_pages_fast(
* access can get ambiguous page results. If you call this function without
* 'write' set, you'd better be sure that you're ok with that ambiguity.
*/
-int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
- struct page **pages)
+int get_user_pages_fast_only(unsigned long start, int nr_pages,
+ unsigned int gup_flags, struct page **pages)
{
int nr_pinned;
/*
@@ -2836,10 +2842,7 @@ int __get_user_pages_fast(unsigned long
* FOLL_FAST_ONLY is required in order to match the API description of
* this routine: no fall back to regular ("slow") GUP.
*/
- unsigned int gup_flags = FOLL_GET | FOLL_FAST_ONLY;
-
- if (write)
- gup_flags |= FOLL_WRITE;
+ gup_flags |= FOLL_GET | FOLL_FAST_ONLY;
nr_pinned = internal_get_user_pages_fast(start, nr_pages, gup_flags,
pages);
@@ -2855,7 +2858,7 @@ int __get_user_pages_fast(unsigned long
return nr_pinned;
}
-EXPORT_SYMBOL_GPL(__get_user_pages_fast);
+EXPORT_SYMBOL_GPL(get_user_pages_fast_only);
/**
* get_user_pages_fast() - pin user pages in memory
@@ -2926,8 +2929,8 @@ int pin_user_pages_fast(unsigned long st
EXPORT_SYMBOL_GPL(pin_user_pages_fast);
/*
- * This is the FOLL_PIN equivalent of __get_user_pages_fast(). Behavior is the
- * same, except that this one sets FOLL_PIN instead of FOLL_GET.
+ * This is the FOLL_PIN equivalent of get_user_pages_fast_only(). Behavior
+ * is the same, except that this one sets FOLL_PIN instead of FOLL_GET.
*
* The API rules are the same, too: no negative values may be returned.
*/
--- a/virt/kvm/kvm_main.c~mm-gupc-convert-to-use-get_user_pagepages_fast_only
+++ a/virt/kvm/kvm_main.c
@@ -1740,7 +1740,6 @@ static bool hva_to_pfn_fast(unsigned lon
bool *writable, kvm_pfn_t *pfn)
{
struct page *page[1];
- int npages;
/*
* Fast pin a writable pfn only if it is a write fault request
@@ -1750,8 +1749,7 @@ static bool hva_to_pfn_fast(unsigned lon
if (!(write_fault || writable))
return false;
- npages = __get_user_pages_fast(addr, 1, 1, page);
- if (npages == 1) {
+ if (get_user_page_fast_only(addr, FOLL_WRITE, page)) {
*pfn = page_to_pfn(page[0]);
if (writable)
@@ -1791,7 +1789,7 @@ static int hva_to_pfn_slow(unsigned long
if (unlikely(!write_fault) && writable) {
struct page *wpage;
- if (__get_user_pages_fast(addr, 1, 1, &wpage) == 1) {
+ if (get_user_page_fast_only(addr, FOLL_WRITE, &wpage)) {
*writable = true;
put_page(page);
page = wpage;
@@ -2003,7 +2001,7 @@ int gfn_to_page_many_atomic(struct kvm_m
if (entry < nr_pages)
return 0;
- return __get_user_pages_fast(addr, nr_pages, 1, pages);
+ return get_user_pages_fast_only(addr, nr_pages, FOLL_WRITE, pages);
}
EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic);
_
next prev parent reply other threads:[~2020-06-08 4:40 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20200607212615.b050e41fac139a1e16fe00bd@linux-foundation.org>
2020-06-08 4:40 ` [patch 01/54] mm/page_idle.c: skip offline pages Andrew Morton
2020-06-08 4:40 ` [patch 02/54] ipc/msg: add missing annotation for freeque() Andrew Morton
2020-06-08 4:40 ` [patch 03/54] ipc/namespace.c: use a work queue to free_ipc Andrew Morton
2020-06-08 4:40 ` [patch 04/54] dynamic_debug: add an option to enable dynamic debug for modules only Andrew Morton
2020-06-08 4:58 ` 答复: " 翟京 (Orson Zhai)
2020-06-08 4:40 ` [patch 05/54] kernel: add panic_on_taint Andrew Morton
2020-06-08 4:40 ` [patch 06/54] xarray.h: correct return code documentation for xa_store_{bh,irq}() Andrew Morton
2020-06-08 4:40 ` [patch 07/54] kernel/sysctl: support setting sysctl parameters from kernel command line Andrew Morton
2020-06-08 4:40 ` [patch 08/54] kernel/sysctl: support handling command line aliases Andrew Morton
2020-06-08 4:40 ` [patch 09/54] kernel/hung_task convert hung_task_panic boot parameter to sysctl Andrew Morton
2020-06-08 4:40 ` [patch 10/54] tools/testing/selftests/sysctl/sysctl.sh: support CONFIG_TEST_SYSCTL=y Andrew Morton
2020-06-08 4:40 ` [patch 11/54] lib/test_sysctl: support testing of sysctl. boot parameter Andrew Morton
2020-06-08 4:40 ` [patch 12/54] kernel/watchdog.c: convert {soft/hard}lockup boot parameters to sysctl aliases Andrew Morton
2020-06-08 4:40 ` [patch 13/54] kernel/hung_task.c: introduce sysctl to print all traces when a hung task is detected Andrew Morton
2020-06-08 4:40 ` [patch 14/54] panic: add sysctl to dump all CPUs backtraces on oops event Andrew Morton
2020-06-08 4:40 ` [patch 15/54] kernel/sysctl.c: ignore out-of-range taint bits introduced via kernel.tainted Andrew Morton
2020-06-08 4:40 ` Andrew Morton [this message]
2020-06-08 4:40 ` [patch 17/54] mm/gup: update pin_user_pages.rst for "case 3" (mmu notifiers) Andrew Morton
2020-06-08 4:41 ` [patch 18/54] mm/gup: introduce pin_user_pages_locked() Andrew Morton
2020-06-08 4:41 ` [patch 19/54] mm/gup: frame_vector: convert get_user_pages() --> pin_user_pages() Andrew Morton
2020-06-08 4:41 ` [patch 20/54] mm/gup: documentation fix for pin_user_pages*() APIs Andrew Morton
2020-06-08 4:41 ` [patch 21/54] docs: mm/gup: pin_user_pages.rst: add a "case 5" Andrew Morton
2020-06-08 4:41 ` [patch 22/54] vhost: convert get_user_pages() --> pin_user_pages() Andrew Morton
2020-06-08 4:41 ` [patch 23/54] mm/mmap.c: add more sanity checks to get_unmapped_area() Andrew Morton
2020-06-08 4:41 ` [patch 24/54] mm/mmap.c: do not allow mappings outside of allowed limits Andrew Morton
2020-06-08 17:50 ` Linus Torvalds
2020-06-08 17:55 ` Linus Torvalds
2020-06-08 4:41 ` [patch 25/54] arm: fix the flush_icache_range arguments in set_fiq_handler Andrew Morton
2020-06-08 4:41 ` [patch 26/54] nds32: unexport flush_icache_page Andrew Morton
2020-06-08 4:41 ` [patch 27/54] powerpc: unexport flush_icache_user_range Andrew Morton
2020-06-08 4:41 ` [patch 28/54] unicore32: remove flush_cache_user_range Andrew Morton
2020-06-08 4:41 ` [patch 29/54] asm-generic: fix the inclusion guards for cacheflush.h Andrew Morton
2020-06-08 4:41 ` [patch 30/54] asm-generic: don't include <linux/mm.h> in cacheflush.h Andrew Morton
2020-06-08 4:41 ` [patch 31/54] asm-generic: improve the flush_dcache_page stub Andrew Morton
2020-06-08 4:41 ` [patch 32/54] alpha: use asm-generic/cacheflush.h Andrew Morton
2020-06-08 4:41 ` [patch 33/54] arm64: " Andrew Morton
2020-06-08 4:41 ` [patch 34/54] c6x: " Andrew Morton
2020-06-08 4:41 ` [patch 35/54] hexagon: " Andrew Morton
2020-06-08 4:42 ` [patch 36/54] ia64: " Andrew Morton
2020-06-08 4:42 ` [patch 37/54] microblaze: " Andrew Morton
2020-06-08 4:42 ` [patch 38/54] m68knommu: " Andrew Morton
2020-06-08 4:42 ` [patch 39/54] openrisc: " Andrew Morton
2020-06-08 4:42 ` [patch 40/54] powerpc: " Andrew Morton
2020-06-08 4:42 ` [patch 41/54] riscv: " Andrew Morton
2020-06-08 4:42 ` [patch 42/54] arm,sparc,unicore32: remove flush_icache_user_range Andrew Morton
2020-06-08 4:42 ` [patch 43/54] mm: rename flush_icache_user_range to flush_icache_user_page Andrew Morton
2020-06-08 4:42 ` [patch 44/54] asm-generic: add a flush_icache_user_range stub Andrew Morton
2020-06-08 4:42 ` [patch 45/54] sh: implement flush_icache_user_range Andrew Morton
2020-06-08 4:42 ` [patch 46/54] xtensa: " Andrew Morton
2020-06-08 4:42 ` [patch 47/54] arm: rename flush_cache_user_range to flush_icache_user_range Andrew Morton
2020-06-08 4:42 ` [patch 48/54] m68k: implement flush_icache_user_range Andrew Morton
2020-06-08 4:42 ` [patch 49/54] exec: only build read_code when needed Andrew Morton
2020-06-08 4:42 ` [patch 50/54] exec: use flush_icache_user_range in read_code Andrew Morton
2020-06-08 4:42 ` [patch 51/54] binfmt_flat: use flush_icache_user_range Andrew Morton
2020-06-08 4:42 ` [patch 52/54] nommu: use flush_icache_user_range in brk and mmap Andrew Morton
2020-06-08 4:42 ` [patch 53/54] module: move the set_fs hack for flush_icache_range to m68k Andrew Morton
2020-06-08 4:42 ` [patch 54/54] doc: cgroup: update note about conditions when oom killer is invoked Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200608044055.D5U9kex7x%akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=aneesh.kumar@linux.ibm.com \
--cc=benh@kernel.crashing.org \
--cc=jhubbard@nvidia.com \
--cc=jolsa@redhat.com \
--cc=jrdr.linux@gmail.com \
--cc=linux-mm@kvack.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=mm-commits@vger.kernel.org \
--cc=mpe@ellerman.id.au \
--cc=msuchanek@suse.de \
--cc=namhyung@kernel.org \
--cc=paulus@ozlabs.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=rppt@linux.ibm.com \
--cc=sfr@canb.auug.org.au \
--cc=torvalds@linux-foundation.org \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).