From: Leonardo Bras <leonardo@linux.ibm.com>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Paul Mackerras <paulus@samba.org>,
Michael Ellerman <mpe@ellerman.id.au>,
Arnd Bergmann <arnd@arndb.de>,
Andrew Morton <akpm@linux-foundation.org>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
Nicholas Piggin <npiggin@gmail.com>,
Christophe Leroy <christophe.leroy@c-s.fr>,
Steven Price <steven.price@arm.com>,
Robin Murphy <robin.murphy@arm.com>,
Leonardo Bras <leonardo@linux.ibm.com>,
Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>,
Balbir Singh <bsingharora@gmail.com>,
Reza Arbab <arbab@linux.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Allison Randal <allison@lohutok.net>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Mike Rapoport <rppt@linux.ibm.com>,
Michal Suchanek <msuchanek@suse.de>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
kvm-ppc@vger.kernel.org, linux-arch@vger.kernel.org,
linux-mm@kvack.org
Subject: [PATCH v6 02/11] mm/gup: Use functions to track lockless pgtbl walks on gup_pgd_range
Date: Thu, 6 Feb 2020 00:08:51 -0300 [thread overview]
Message-ID: <20200206030900.147032-3-leonardo@linux.ibm.com> (raw)
In-Reply-To: <20200206030900.147032-1-leonardo@linux.ibm.com>
As described, gup_pgd_range is a lockless pagetable walk. So, in order to
track against THP split/collapse, it disables/enables irq around it.
To make use of the new tracking functions, it replaces irq disable/enable
by {begin,end}_lockless_pgtbl_walk().
As local_irq_{save,restore} is present inside
{begin,end}_lockless_pgtbl_walk, there should be no change in the
workings.
Signed-off-by: Leonardo Bras <leonardo@linux.ibm.com>
---
mm/gup.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index 1b521e0ac1de..04e6f46993b6 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2369,7 +2369,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
unsigned long len, end;
- unsigned long flags;
+ unsigned long irq_mask;
int nr = 0;
start = untagged_addr(start) & PAGE_MASK;
@@ -2395,9 +2395,9 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_save(flags);
+ irq_mask = begin_lockless_pgtbl_walk();
gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr);
- local_irq_restore(flags);
+ end_lockless_pgtbl_walk(irq_mask);
}
return nr;
@@ -2450,9 +2450,9 @@ static int internal_get_user_pages_fast(unsigned long start, int nr_pages,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_disable();
+ begin_lockless_pgtbl_walk();
gup_pgd_range(addr, end, gup_flags, pages, &nr);
- local_irq_enable();
+ end_lockless_pgtbl_walk(IRQS_ENABLED);
ret = nr;
}
--
2.24.1
WARNING: multiple messages have this Message-ID (diff)
From: Leonardo Bras <leonardo@linux.ibm.com>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Paul Mackerras <paulus@samba.org>,
Michael Ellerman <mpe@ellerman.id.au>,
Arnd Bergmann <arnd@arndb.de>,
Andrew Morton <akpm@linux-foundation.org>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
Nicholas Piggin <npiggin@gmail.com>,
Christophe Leroy <christophe.leroy@c-s.fr>,
Steven Price <steven.price@arm.com>,
Robin Murphy <robin.murphy@arm.com>,
Leonardo Bras <leonardo@linux.ibm.com>,
Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>,
Balbir Singh <bsingharora@gmail.com>,
Reza Arbab <arbab@linux.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Allison Randal <allison@lohutok.net>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Mike Rapoport <rppt@linux.ibm.com>,
Michal Suchanek <msuchanek@su>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
kvm-ppc@vger.kernel.org, linux-arch@vger.kernel.org,
linux-mm@kvack.org
Subject: [PATCH v6 02/11] mm/gup: Use functions to track lockless pgtbl walks on gup_pgd_range
Date: Thu, 6 Feb 2020 00:08:51 -0300 [thread overview]
Message-ID: <20200206030900.147032-3-leonardo@linux.ibm.com> (raw)
In-Reply-To: <20200206030900.147032-1-leonardo@linux.ibm.com>
As described, gup_pgd_range is a lockless pagetable walk. So, in order to
track against THP split/collapse, it disables/enables irq around it.
To make use of the new tracking functions, it replaces irq disable/enable
by {begin,end}_lockless_pgtbl_walk().
As local_irq_{save,restore} is present inside
{begin,end}_lockless_pgtbl_walk, there should be no change in the
workings.
Signed-off-by: Leonardo Bras <leonardo@linux.ibm.com>
---
mm/gup.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index 1b521e0ac1de..04e6f46993b6 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2369,7 +2369,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
unsigned long len, end;
- unsigned long flags;
+ unsigned long irq_mask;
int nr = 0;
start = untagged_addr(start) & PAGE_MASK;
@@ -2395,9 +2395,9 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_save(flags);
+ irq_mask = begin_lockless_pgtbl_walk();
gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr);
- local_irq_restore(flags);
+ end_lockless_pgtbl_walk(irq_mask);
}
return nr;
@@ -2450,9 +2450,9 @@ static int internal_get_user_pages_fast(unsigned long start, int nr_pages,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_disable();
+ begin_lockless_pgtbl_walk();
gup_pgd_range(addr, end, gup_flags, pages, &nr);
- local_irq_enable();
+ end_lockless_pgtbl_walk(IRQS_ENABLED);
ret = nr;
}
--
2.24.1
WARNING: multiple messages have this Message-ID (diff)
From: Leonardo Bras <leonardo@linux.ibm.com>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Paul Mackerras <paulus@samba.org>,
Michael Ellerman <mpe@ellerman.id.au>,
Arnd Bergmann <arnd@arndb.de>,
Andrew Morton <akpm@linux-foundation.org>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
Nicholas Piggin <npiggin@gmail.com>,
Christophe Leroy <christophe.leroy@c-s.fr>,
Steven Price <steven.price@arm.com>,
Robin Murphy <robin.murphy@arm.com>,
Leonardo Bras <leonardo@linux.ibm.com>,
Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>,
Balbir Singh <bsingharora@gmail.com>,
Reza Arbab <arbab@linux.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Allison Randal <allison@lohutok.net>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Mike Rapoport <rppt@linux.ibm.com>,
Michal Suchanek <msuchanek@suse.de>
Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org,
linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
kvm-ppc@vger.kernel.org
Subject: [PATCH v6 02/11] mm/gup: Use functions to track lockless pgtbl walks on gup_pgd_range
Date: Thu, 6 Feb 2020 00:08:51 -0300 [thread overview]
Message-ID: <20200206030900.147032-3-leonardo@linux.ibm.com> (raw)
In-Reply-To: <20200206030900.147032-1-leonardo@linux.ibm.com>
As described, gup_pgd_range is a lockless pagetable walk. So, in order to
track against THP split/collapse, it disables/enables irq around it.
To make use of the new tracking functions, it replaces irq disable/enable
by {begin,end}_lockless_pgtbl_walk().
As local_irq_{save,restore} is present inside
{begin,end}_lockless_pgtbl_walk, there should be no change in the
workings.
Signed-off-by: Leonardo Bras <leonardo@linux.ibm.com>
---
mm/gup.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index 1b521e0ac1de..04e6f46993b6 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2369,7 +2369,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
unsigned long len, end;
- unsigned long flags;
+ unsigned long irq_mask;
int nr = 0;
start = untagged_addr(start) & PAGE_MASK;
@@ -2395,9 +2395,9 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_save(flags);
+ irq_mask = begin_lockless_pgtbl_walk();
gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr);
- local_irq_restore(flags);
+ end_lockless_pgtbl_walk(irq_mask);
}
return nr;
@@ -2450,9 +2450,9 @@ static int internal_get_user_pages_fast(unsigned long start, int nr_pages,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_disable();
+ begin_lockless_pgtbl_walk();
gup_pgd_range(addr, end, gup_flags, pages, &nr);
- local_irq_enable();
+ end_lockless_pgtbl_walk(IRQS_ENABLED);
ret = nr;
}
--
2.24.1
WARNING: multiple messages have this Message-ID (diff)
From: Leonardo Bras <leonardo@linux.ibm.com>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Paul Mackerras <paulus@samba.org>,
Michael Ellerman <mpe@ellerman.id.au>,
Arnd Bergmann <arnd@arndb.de>,
Andrew Morton <akpm@linux-foundation.org>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
Nicholas Piggin <npiggin@gmail.com>,
Christophe Leroy <christophe.leroy@c-s.fr>,
Steven Price <steven.price@arm.com>,
Robin Murphy <robin.murphy@arm.com>,
Leonardo Bras <leonardo@linux.ibm.com>,
Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>,
Balbir Singh <bsingharora@gmail.com>,
Reza Arbab <arbab@linux.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Allison Randal <allison@lohutok.net>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Mike Rapoport <rppt@linux.ibm.com>,
Michal Suchanek <msuchanek@suse.de>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
kvm-ppc@vger.kernel.org, linux-arch@vger.kernel.org,
linux-mm@kvack.org
Subject: [PATCH v6 02/11] mm/gup: Use functions to track lockless pgtbl walks on gup_pgd_range
Date: Thu, 06 Feb 2020 03:08:51 +0000 [thread overview]
Message-ID: <20200206030900.147032-3-leonardo@linux.ibm.com> (raw)
In-Reply-To: <20200206030900.147032-1-leonardo@linux.ibm.com>
As described, gup_pgd_range is a lockless pagetable walk. So, in order to
track against THP split/collapse, it disables/enables irq around it.
To make use of the new tracking functions, it replaces irq disable/enable
by {begin,end}_lockless_pgtbl_walk().
As local_irq_{save,restore} is present inside
{begin,end}_lockless_pgtbl_walk, there should be no change in the
workings.
Signed-off-by: Leonardo Bras <leonardo@linux.ibm.com>
---
mm/gup.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index 1b521e0ac1de..04e6f46993b6 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2369,7 +2369,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages)
{
unsigned long len, end;
- unsigned long flags;
+ unsigned long irq_mask;
int nr = 0;
start = untagged_addr(start) & PAGE_MASK;
@@ -2395,9 +2395,9 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_save(flags);
+ irq_mask = begin_lockless_pgtbl_walk();
gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr);
- local_irq_restore(flags);
+ end_lockless_pgtbl_walk(irq_mask);
}
return nr;
@@ -2450,9 +2450,9 @@ static int internal_get_user_pages_fast(unsigned long start, int nr_pages,
if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
gup_fast_permitted(start, end)) {
- local_irq_disable();
+ begin_lockless_pgtbl_walk();
gup_pgd_range(addr, end, gup_flags, pages, &nr);
- local_irq_enable();
+ end_lockless_pgtbl_walk(IRQS_ENABLED);
ret = nr;
}
--
2.24.1
next prev parent reply other threads:[~2020-02-06 3:10 UTC|newest]
Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-06 3:08 [PATCH v6 00/11] Introduces new functions for tracking lockless pagetable walks Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 01/11] asm-generic/pgtable: Adds generic functions to track lockless pgtable walks Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 5:54 ` Christophe Leroy
2020-02-06 5:54 ` Christophe Leroy
2020-02-06 5:54 ` Christophe Leroy
2020-02-07 2:19 ` Leonardo Bras
2020-02-07 2:19 ` Leonardo Bras
2020-02-07 2:19 ` Leonardo Bras
2020-02-07 5:39 ` kbuild test robot
2020-02-07 5:39 ` kbuild test robot
2020-02-07 5:39 ` kbuild test robot
2020-02-07 5:39 ` kbuild test robot
2020-02-07 5:39 ` kbuild test robot
2020-02-07 5:39 ` kbuild test robot
2020-02-06 3:08 ` Leonardo Bras [this message]
2020-02-06 3:08 ` [PATCH v6 02/11] mm/gup: Use functions to track lockless pgtbl walks on gup_pgd_range Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:25 ` Leonardo Bras
2020-02-06 3:25 ` Leonardo Bras
2020-02-07 22:54 ` John Hubbard
2020-02-07 22:54 ` John Hubbard
2020-02-17 20:55 ` Leonardo Bras
2020-02-17 20:55 ` Leonardo Bras
2020-02-17 20:55 ` Leonardo Bras
2020-10-15 14:46 ` Michal Suchánek
2020-10-15 14:46 ` Michal Suchánek
2020-10-15 14:46 ` Michal Suchánek
2020-10-16 3:27 ` Aneesh Kumar K.V
2020-10-16 3:39 ` Aneesh Kumar K.V
2020-10-16 3:27 ` Aneesh Kumar K.V
2020-02-07 1:19 ` kbuild test robot
2020-02-07 1:19 ` kbuild test robot
2020-02-07 1:19 ` kbuild test robot
2020-02-07 1:19 ` kbuild test robot
2020-02-07 1:19 ` kbuild test robot
2020-02-07 1:19 ` kbuild test robot
2020-02-07 8:01 ` kbuild test robot
2020-02-07 8:01 ` kbuild test robot
2020-02-07 8:01 ` kbuild test robot
2020-02-07 8:01 ` kbuild test robot
2020-02-07 8:01 ` kbuild test robot
2020-02-07 8:01 ` kbuild test robot
2020-02-06 3:08 ` [PATCH v6 03/11] powerpc/mm: Adds arch-specificic functions to track lockless pgtable walks Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 5:46 ` Christophe Leroy
2020-02-06 5:46 ` Christophe Leroy
2020-02-06 5:46 ` Christophe Leroy
2020-02-07 4:38 ` Leonardo Bras
2020-02-07 4:38 ` Leonardo Bras
2020-02-07 4:38 ` Leonardo Bras
2020-02-17 20:32 ` Leonardo Bras
2020-02-17 20:32 ` Leonardo Bras
2020-02-17 20:32 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 04/11] powerpc/mce_power: Use functions to track lockless pgtbl walks Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 5:48 ` Christophe Leroy
2020-02-06 5:48 ` Christophe Leroy
2020-02-06 5:48 ` Christophe Leroy
2020-02-07 4:00 ` Leonardo Bras
2020-02-07 4:00 ` Leonardo Bras
2020-02-07 4:00 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 05/11] powerpc/perf: " Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 06/11] powerpc/mm/book3s64/hash: " Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 6:06 ` Christophe Leroy
2020-02-06 6:06 ` Christophe Leroy
2020-02-06 6:06 ` Christophe Leroy
2020-02-07 3:49 ` Leonardo Bras
2020-02-07 3:49 ` Leonardo Bras
2020-02-07 3:49 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 07/11] powerpc/kvm/e500: " Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 6:18 ` Christophe Leroy
2020-02-06 6:18 ` Christophe Leroy
2020-02-06 6:18 ` Christophe Leroy
2020-02-07 3:10 ` Leonardo Bras
2020-02-07 3:10 ` Leonardo Bras
2020-02-07 3:10 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 08/11] powerpc/kvm/book3s_hv: " Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 09/11] powerpc/kvm/book3s_64: " Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` [PATCH v6 10/11] powerpc/mm: Adds counting method to track lockless pagetable walks Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 3:08 ` Leonardo Bras
2020-02-06 6:23 ` Christophe Leroy
2020-02-06 6:23 ` Christophe Leroy
2020-02-06 6:23 ` Christophe Leroy
2020-02-07 1:56 ` Leonardo Bras
2020-02-07 1:56 ` Leonardo Bras
2020-02-06 3:09 ` [PATCH v6 11/11] powerpc/mm/book3s64/pgtable: Uses counting method to skip serializing Leonardo Bras
2020-02-06 3:09 ` Leonardo Bras
2020-02-06 3:09 ` Leonardo Bras
2020-02-06 3:09 ` Leonardo Bras
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200206030900.147032-3-leonardo@linux.ibm.com \
--to=leonardo@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=allison@lohutok.net \
--cc=aneesh.kumar@linux.ibm.com \
--cc=arbab@linux.ibm.com \
--cc=arnd@arndb.de \
--cc=benh@kernel.crashing.org \
--cc=bsingharora@gmail.com \
--cc=christophe.leroy@c-s.fr \
--cc=gregkh@linuxfoundation.org \
--cc=kvm-ppc@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mahesh@linux.vnet.ibm.com \
--cc=mpe@ellerman.id.au \
--cc=msuchanek@suse.de \
--cc=npiggin@gmail.com \
--cc=paulus@samba.org \
--cc=robin.murphy@arm.com \
--cc=rppt@linux.ibm.com \
--cc=steven.price@arm.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.