From: Christophe Leroy <christophe.leroy@csgroup.eu>
To: Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
Jason Gunthorpe <jgg@ziepe.ca>,
John Hubbard <jhubbard@nvidia.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
linux-mm <linux-mm@kvack.org>, Paul Mackerras <paulus@samba.org>,
linux-sparc <sparclinux@vger.kernel.org>,
Alexander Gordeev <agordeev@linux.ibm.com>,
Claudio Imbrenda <imbrenda@linux.ibm.com>,
Will Deacon <will@kernel.org>,
linux-arch <linux-arch@vger.kernel.org>,
linux-s390 <linux-s390@vger.kernel.org>,
Vasily Gorbik <gor@linux.ibm.com>,
Richard Weinberger <richard@nod.at>, linux-x86 <x86@kernel.org>,
Russell King <linux@armlinux.org.uk>,
Christian Borntraeger <borntraeger@de.ibm.com>,
Ingo Molnar <mingo@redhat.com>,
Andrey Ryabinin <aryabinin@virtuozzo.com>,
Jeff Dike <jdike@addtoit.com>, Arnd Bergmann <arnd@arndb.de>,
Heiko Carstens <hca@linux.ibm.com>,
linux-um <linux-um@lists.infradead.org>,
Borislav Petkov <bp@alien8.de>, Andy Lutomirski <luto@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
linux-arm <linux-arm-kernel@lists.infradead.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
LKML <linux-kernel@vger.kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
linux-power <linuxppc-dev@lists.ozlabs.org>,
Mike Rapoport <rppt@kernel.org>
Subject: Re: [RFC PATCH v2 1/3] mm/gup: fix gup_fast with dynamic page table folding
Date: Tue, 8 Sep 2020 07:06:23 +0200 [thread overview]
Message-ID: <82fbe8f9-f199-5fc2-4168-eb43ad0b0346@csgroup.eu> (raw)
In-Reply-To: <20200907180058.64880-2-gerald.schaefer@linux.ibm.com>
Le 07/09/2020 à 20:00, Gerald Schaefer a écrit :
> From: Alexander Gordeev <agordeev@linux.ibm.com>
>
> Commit 1a42010cdc26 ("s390/mm: convert to the generic get_user_pages_fast
> code") introduced a subtle but severe bug on s390 with gup_fast, due to
> dynamic page table folding.
>
> The question "What would it require for the generic code to work for s390"
> has already been discussed here
> https://lkml.kernel.org/r/20190418100218.0a4afd51@mschwideX1
> and ended with a promising approach here
> https://lkml.kernel.org/r/20190419153307.4f2911b5@mschwideX1
> which in the end unfortunately didn't quite work completely.
>
> We tried to mimic static level folding by changing pgd_offset to always
> calculate top level page table offset, and do nothing in folded pXd_offset.
> What has been overlooked is that PxD_SIZE/MASK and thus pXd_addr_end do
> not reflect this dynamic behaviour, and still act like static 5-level
> page tables.
>
[...]
>
> Fix this by introducing new pXd_addr_end_folded helpers, which take an
> additional pXd entry value parameter, that can be used on s390
> to determine the correct page table level and return corresponding
> end / boundary. With that, the pointer iteration will always
> happen in gup_pgd_range for s390. No change for other architectures
> introduced.
Not sure pXd_addr_end_folded() is the best understandable name,
allthough I don't have any alternative suggestion at the moment.
Maybe could be something like pXd_addr_end_fixup() as it will disappear
in the next patch, or pXd_addr_end_gup() ?
Also, if it happens to be acceptable to get patch 2 in stable, I think
you should switch patch 1 and patch 2 to avoid the step through
pXd_addr_end_folded()
>
> Fixes: 1a42010cdc26 ("s390/mm: convert to the generic get_user_pages_fast code")
> Cc: <stable@vger.kernel.org> # 5.2+
> Reviewed-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
> Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
> Signed-off-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
> ---
> arch/s390/include/asm/pgtable.h | 42 +++++++++++++++++++++++++++++++++
> include/linux/pgtable.h | 16 +++++++++++++
> mm/gup.c | 8 +++----
> 3 files changed, 62 insertions(+), 4 deletions(-)
>
> diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
> index 7eb01a5459cd..027206e4959d 100644
> --- a/arch/s390/include/asm/pgtable.h
> +++ b/arch/s390/include/asm/pgtable.h
> @@ -512,6 +512,48 @@ static inline bool mm_pmd_folded(struct mm_struct *mm)
> }
> #define mm_pmd_folded(mm) mm_pmd_folded(mm)
>
> +/*
> + * With dynamic page table levels on s390, the static pXd_addr_end() functions
> + * will not return corresponding dynamic boundaries. This is no problem as long
> + * as only pXd pointers are passed down during page table walk, because
> + * pXd_offset() will simply return the given pointer for folded levels, and the
> + * pointer iteration over a range simply happens at the correct page table
> + * level.
> + * It is however a problem with gup_fast, or other places walking the page
> + * tables w/o locks using READ_ONCE(), and passing down the pXd values instead
> + * of pointers. In this case, the pointer given to pXd_offset() is a pointer to
> + * a stack variable, which cannot be used for pointer iteration at the correct
> + * level. Instead, the iteration then has to happen by going up to pgd level
> + * again. To allow this, provide pXd_addr_end_folded() functions with an
> + * additional pXd value parameter, which can be used on s390 to determine the
> + * folding level and return the corresponding boundary.
> + */
> +static inline unsigned long rste_addr_end_folded(unsigned long rste, unsigned long addr, unsigned long end)
What does 'rste' stands for ?
Isn't this line a bit long ?
> +{
> + unsigned long type = (rste & _REGION_ENTRY_TYPE_MASK) >> 2;
> + unsigned long size = 1UL << (_SEGMENT_SHIFT + type * 11);
> + unsigned long boundary = (addr + size) & ~(size - 1);
> +
> + /*
> + * FIXME The below check is for internal testing only, to be removed
> + */
> + VM_BUG_ON(type < (_REGION_ENTRY_TYPE_R3 >> 2));
> +
> + return (boundary - 1) < (end - 1) ? boundary : end;
> +}
> +
> +#define pgd_addr_end_folded pgd_addr_end_folded
> +static inline unsigned long pgd_addr_end_folded(pgd_t pgd, unsigned long addr, unsigned long end)
> +{
> + return rste_addr_end_folded(pgd_val(pgd), addr, end);
> +}
> +
> +#define p4d_addr_end_folded p4d_addr_end_folded
> +static inline unsigned long p4d_addr_end_folded(p4d_t p4d, unsigned long addr, unsigned long end)
> +{
> + return rste_addr_end_folded(p4d_val(p4d), addr, end);
> +}
> +
> static inline int mm_has_pgste(struct mm_struct *mm)
> {
> #ifdef CONFIG_PGSTE
> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
> index e8cbc2e795d5..981c4c2a31fe 100644
> --- a/include/linux/pgtable.h
> +++ b/include/linux/pgtable.h
> @@ -681,6 +681,22 @@ static inline int arch_unmap_one(struct mm_struct *mm,
> })
> #endif
>
> +#ifndef pgd_addr_end_folded
> +#define pgd_addr_end_folded(pgd, addr, end) pgd_addr_end(addr, end)
> +#endif
> +
> +#ifndef p4d_addr_end_folded
> +#define p4d_addr_end_folded(p4d, addr, end) p4d_addr_end(addr, end)
> +#endif
> +
> +#ifndef pud_addr_end_folded
> +#define pud_addr_end_folded(pud, addr, end) pud_addr_end(addr, end)
> +#endif
> +
> +#ifndef pmd_addr_end_folded
> +#define pmd_addr_end_folded(pmd, addr, end) pmd_addr_end(addr, end)
> +#endif
> +
> /*
> * When walking page tables, we usually want to skip any p?d_none entries;
> * and any p?d_bad entries - reporting the error before resetting to none.
> diff --git a/mm/gup.c b/mm/gup.c
> index bd883a112724..ba4aace5d0f4 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -2521,7 +2521,7 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end,
> do {
> pmd_t pmd = READ_ONCE(*pmdp);
>
> - next = pmd_addr_end(addr, end);
> + next = pmd_addr_end_folded(pmd, addr, end);
> if (!pmd_present(pmd))
> return 0;
>
> @@ -2564,7 +2564,7 @@ static int gup_pud_range(p4d_t p4d, unsigned long addr, unsigned long end,
> do {
> pud_t pud = READ_ONCE(*pudp);
>
> - next = pud_addr_end(addr, end);
> + next = pud_addr_end_folded(pud, addr, end);
> if (unlikely(!pud_present(pud)))
> return 0;
> if (unlikely(pud_huge(pud))) {
> @@ -2592,7 +2592,7 @@ static int gup_p4d_range(pgd_t pgd, unsigned long addr, unsigned long end,
> do {
> p4d_t p4d = READ_ONCE(*p4dp);
>
> - next = p4d_addr_end(addr, end);
> + next = p4d_addr_end_folded(p4d, addr, end);
> if (p4d_none(p4d))
> return 0;
> BUILD_BUG_ON(p4d_huge(p4d));
> @@ -2617,7 +2617,7 @@ static void gup_pgd_range(unsigned long addr, unsigned long end,
> do {
> pgd_t pgd = READ_ONCE(*pgdp);
>
> - next = pgd_addr_end(addr, end);
> + next = pgd_addr_end_folded(pgd, addr, end);
> if (pgd_none(pgd))
> return;
> if (unlikely(pgd_huge(pgd))) {
>
Christophe
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-09-08 5:08 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-07 18:00 [RFC PATCH v2 0/3] mm/gup: fix gup_fast with dynamic page table folding Gerald Schaefer
2020-09-07 18:00 ` [RFC PATCH v2 1/3] " Gerald Schaefer
2020-09-08 5:06 ` Christophe Leroy [this message]
2020-09-08 12:09 ` Christian Borntraeger
2020-09-08 12:40 ` Christophe Leroy
2020-09-08 13:38 ` Gerald Schaefer
2020-09-08 14:30 ` Dave Hansen
2020-09-08 17:59 ` Gerald Schaefer
2020-09-09 12:29 ` Gerald Schaefer
2020-09-09 16:18 ` Dave Hansen
2020-09-09 17:25 ` Gerald Schaefer
2020-09-09 18:03 ` Jason Gunthorpe
2020-09-10 9:39 ` Alexander Gordeev
2020-09-10 13:02 ` Jason Gunthorpe
2020-09-10 13:28 ` Gerald Schaefer
2020-09-10 15:10 ` Jason Gunthorpe
2020-09-10 17:07 ` Gerald Schaefer
2020-09-10 17:19 ` Jason Gunthorpe
2020-09-10 17:57 ` Gerald Schaefer
2020-09-10 23:21 ` Jason Gunthorpe
2020-09-10 17:35 ` Linus Torvalds
2020-09-10 18:13 ` Jason Gunthorpe
2020-09-10 18:33 ` Linus Torvalds
2020-09-10 19:10 ` Gerald Schaefer
2020-09-10 19:32 ` Linus Torvalds
2020-09-10 21:59 ` Jason Gunthorpe
2020-09-11 7:09 ` peterz
2020-09-11 11:19 ` Jason Gunthorpe
2020-09-11 19:03 ` [PATCH] " Vasily Gorbik
2020-09-11 19:09 ` Linus Torvalds
2020-09-11 19:40 ` Jason Gunthorpe
2020-09-11 20:05 ` Jason Gunthorpe
2020-09-11 20:36 ` [PATCH v2] " Vasily Gorbik
2020-09-15 17:09 ` Vasily Gorbik
2020-09-15 17:14 ` Jason Gunthorpe
2020-09-15 17:18 ` Mike Rapoport
2020-09-15 17:31 ` John Hubbard
2020-09-10 21:22 ` [RFC PATCH v2 1/3] " John Hubbard
2020-09-10 22:11 ` Jason Gunthorpe
2020-09-10 22:17 ` John Hubbard
2020-09-11 12:19 ` Alexander Gordeev
2020-09-11 16:45 ` Linus Torvalds
2020-09-10 13:11 ` Gerald Schaefer
2020-09-07 18:00 ` [RFC PATCH v2 2/3] mm: make pXd_addr_end() functions page-table entry aware Gerald Schaefer
2020-09-08 5:14 ` Christophe Leroy
2020-09-08 7:46 ` Alexander Gordeev
2020-09-08 8:16 ` Christophe Leroy
2020-09-08 14:15 ` Alexander Gordeev
2020-09-09 8:38 ` Christophe Leroy
2020-09-08 14:25 ` Alexander Gordeev
2020-09-08 13:26 ` Jason Gunthorpe
2020-09-08 14:33 ` Dave Hansen
2020-09-07 18:00 ` [RFC PATCH v2 3/3] mm: make generic pXd_addr_end() macros inline functions Gerald Schaefer
2020-09-07 20:15 ` Mike Rapoport
2020-09-08 5:19 ` Christophe Leroy
2020-09-08 15:48 ` Alexander Gordeev
2020-09-08 17:20 ` Christophe Leroy
2020-09-07 20:12 ` [RFC PATCH v2 0/3] mm/gup: fix gup_fast with dynamic page table folding Mike Rapoport
2020-09-08 5:22 ` Christophe Leroy
2020-09-08 17:36 ` Gerald Schaefer
2020-09-09 16:12 ` Gerald Schaefer
2020-09-08 4:42 ` Christophe Leroy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=82fbe8f9-f199-5fc2-4168-eb43ad0b0346@csgroup.eu \
--to=christophe.leroy@csgroup.eu \
--cc=agordeev@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=aryabinin@virtuozzo.com \
--cc=borntraeger@de.ibm.com \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=dave.hansen@linux.intel.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=imbrenda@linux.ibm.com \
--cc=jdike@addtoit.com \
--cc=jgg@ziepe.ca \
--cc=jhubbard@nvidia.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-um@lists.infradead.org \
--cc=linux@armlinux.org.uk \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=paulus@samba.org \
--cc=peterz@infradead.org \
--cc=richard@nod.at \
--cc=rppt@kernel.org \
--cc=sparclinux@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).