From: Jason Gunthorpe <jgg@ziepe.ca>
To: Vasily Gorbik <gor@linux.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Dave Hansen <dave.hansen@linux.intel.com>,
linux-mm <linux-mm@kvack.org>, Paul Mackerras <paulus@samba.org>,
linux-sparc <sparclinux@vger.kernel.org>,
Alexander Gordeev <agordeev@linux.ibm.com>,
Claudio Imbrenda <imbrenda@linux.ibm.com>,
Will Deacon <will@kernel.org>,
linux-arch <linux-arch@vger.kernel.org>,
linux-s390 <linux-s390@vger.kernel.org>,
Richard Weinberger <richard@nod.at>, linux-x86 <x86@kernel.org>,
Russell King <linux@armlinux.org.uk>,
Christian Borntraeger <borntraeger@de.ibm.com>,
Ingo Molnar <mingo@redhat.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Andrey Ryabinin <aryabinin@virtuozzo.com>,
Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
Heiko Carstens <hca@linux.ibm.com>, Arnd Bergmann <arnd@arndb.de>,
John Hubbard <jhubbard@nvidia.com>, Jeff Dike <jdike@addtoit.com>,
linux-um <linux-um@lists.infradead.org>,
Borislav Petkov <bp@alien8.de>, Andy Lutomirski <luto@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
linux-arm <linux-arm-kernel@lists.infradead.org>,
Dave Hansen <dave.hansen@intel.com>,
linux-power <linuxppc-dev@lists.ozlabs.org>,
LKML <linux-kernel@vger.kernel.org>,
Michael Ellerman <mpe@ellerman.id.au>,
Andrew Morton <akpm@linux-foundation.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Mike Rapoport <rppt@kernel.org>
Subject: Re: [PATCH v2] mm/gup: fix gup_fast with dynamic page table folding
Date: Tue, 15 Sep 2020 14:14:20 -0300 [thread overview]
Message-ID: <20200915171420.GK1221970@ziepe.ca> (raw)
In-Reply-To: <patch.git-943f1e5dcff2.your-ad-here.call-01599856292-ext-8676@work.hours>
On Fri, Sep 11, 2020 at 10:36:43PM +0200, Vasily Gorbik wrote:
> Currently to make sure that every page table entry is read just once
> gup_fast walks perform READ_ONCE and pass pXd value down to the next
> gup_pXd_range function by value e.g.:
>
> static int gup_pud_range(p4d_t p4d, unsigned long addr, unsigned long end,
> unsigned int flags, struct page **pages, int *nr)
> ...
> pudp = pud_offset(&p4d, addr);
>
> This function passes a reference on that local value copy to pXd_offset,
> and might get the very same pointer in return. This happens when the
> level is folded (on most arches), and that pointer should not be iterated.
>
> On s390 due to the fact that each task might have different 5,4 or
> 3-level address translation and hence different levels folded the logic
> is more complex and non-iteratable pointer to a local copy leads to
> severe problems.
>
> Here is an example of what happens with gup_fast on s390, for a task
> with 3-levels paging, crossing a 2 GB pud boundary:
>
> // addr = 0x1007ffff000, end = 0x10080001000
> static int gup_pud_range(p4d_t p4d, unsigned long addr, unsigned long end,
> unsigned int flags, struct page **pages, int *nr)
> {
> unsigned long next;
> pud_t *pudp;
>
> // pud_offset returns &p4d itself (a pointer to a value on stack)
> pudp = pud_offset(&p4d, addr);
> do {
> // on second iteratation reading "random" stack value
> pud_t pud = READ_ONCE(*pudp);
>
> // next = 0x10080000000, due to PUD_SIZE/MASK != PGDIR_SIZE/MASK on s390
> next = pud_addr_end(addr, end);
> ...
> } while (pudp++, addr = next, addr != end); // pudp++ iterating over stack
>
> return 1;
> }
>
> This happens since s390 moved to common gup code with
> commit d1874a0c2805 ("s390/mm: make the pxd_offset functions more robust")
> and commit 1a42010cdc26 ("s390/mm: convert to the generic
> get_user_pages_fast code"). s390 tried to mimic static level folding by
> changing pXd_offset primitives to always calculate top level page table
> offset in pgd_offset and just return the value passed when pXd_offset
> has to act as folded.
>
> What is crucial for gup_fast and what has been overlooked is
> that PxD_SIZE/MASK and thus pXd_addr_end should also change
> correspondingly. And the latter is not possible with dynamic folding.
>
> To fix the issue in addition to pXd values pass original
> pXdp pointers down to gup_pXd_range functions. And introduce
> pXd_offset_lockless helpers, which take an additional pXd
> entry value parameter. This has already been discussed in
> https://lkml.kernel.org/r/20190418100218.0a4afd51@mschwideX1
>
> Cc: <stable@vger.kernel.org> # 5.2+
> Fixes: 1a42010cdc26 ("s390/mm: convert to the generic get_user_pages_fast code")
> Reviewed-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
> Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
> ---
> v2: added brackets &pgd -> &(pgd)
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Regards,
Jason
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-09-15 17:16 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-07 18:00 [RFC PATCH v2 0/3] mm/gup: fix gup_fast with dynamic page table folding Gerald Schaefer
2020-09-07 18:00 ` [RFC PATCH v2 1/3] " Gerald Schaefer
2020-09-08 5:06 ` Christophe Leroy
2020-09-08 12:09 ` Christian Borntraeger
2020-09-08 12:40 ` Christophe Leroy
2020-09-08 13:38 ` Gerald Schaefer
2020-09-08 14:30 ` Dave Hansen
2020-09-08 17:59 ` Gerald Schaefer
2020-09-09 12:29 ` Gerald Schaefer
2020-09-09 16:18 ` Dave Hansen
2020-09-09 17:25 ` Gerald Schaefer
2020-09-09 18:03 ` Jason Gunthorpe
2020-09-10 9:39 ` Alexander Gordeev
2020-09-10 13:02 ` Jason Gunthorpe
2020-09-10 13:28 ` Gerald Schaefer
2020-09-10 15:10 ` Jason Gunthorpe
2020-09-10 17:07 ` Gerald Schaefer
2020-09-10 17:19 ` Jason Gunthorpe
2020-09-10 17:57 ` Gerald Schaefer
2020-09-10 23:21 ` Jason Gunthorpe
2020-09-10 17:35 ` Linus Torvalds
2020-09-10 18:13 ` Jason Gunthorpe
2020-09-10 18:33 ` Linus Torvalds
2020-09-10 19:10 ` Gerald Schaefer
2020-09-10 19:32 ` Linus Torvalds
2020-09-10 21:59 ` Jason Gunthorpe
2020-09-11 7:09 ` peterz
2020-09-11 11:19 ` Jason Gunthorpe
2020-09-11 19:03 ` [PATCH] " Vasily Gorbik
2020-09-11 19:09 ` Linus Torvalds
2020-09-11 19:40 ` Jason Gunthorpe
2020-09-11 20:05 ` Jason Gunthorpe
2020-09-11 20:36 ` [PATCH v2] " Vasily Gorbik
2020-09-15 17:09 ` Vasily Gorbik
2020-09-15 17:14 ` Jason Gunthorpe [this message]
2020-09-15 17:18 ` Mike Rapoport
2020-09-15 17:31 ` John Hubbard
2020-09-10 21:22 ` [RFC PATCH v2 1/3] " John Hubbard
2020-09-10 22:11 ` Jason Gunthorpe
2020-09-10 22:17 ` John Hubbard
2020-09-11 12:19 ` Alexander Gordeev
2020-09-11 16:45 ` Linus Torvalds
2020-09-10 13:11 ` Gerald Schaefer
2020-09-07 18:00 ` [RFC PATCH v2 2/3] mm: make pXd_addr_end() functions page-table entry aware Gerald Schaefer
2020-09-08 5:14 ` Christophe Leroy
2020-09-08 7:46 ` Alexander Gordeev
2020-09-08 8:16 ` Christophe Leroy
2020-09-08 14:15 ` Alexander Gordeev
2020-09-09 8:38 ` Christophe Leroy
2020-09-08 14:25 ` Alexander Gordeev
2020-09-08 13:26 ` Jason Gunthorpe
2020-09-08 14:33 ` Dave Hansen
2020-09-07 18:00 ` [RFC PATCH v2 3/3] mm: make generic pXd_addr_end() macros inline functions Gerald Schaefer
2020-09-07 20:15 ` Mike Rapoport
2020-09-08 5:19 ` Christophe Leroy
2020-09-08 15:48 ` Alexander Gordeev
2020-09-08 17:20 ` Christophe Leroy
2020-09-07 20:12 ` [RFC PATCH v2 0/3] mm/gup: fix gup_fast with dynamic page table folding Mike Rapoport
2020-09-08 5:22 ` Christophe Leroy
2020-09-08 17:36 ` Gerald Schaefer
2020-09-09 16:12 ` Gerald Schaefer
2020-09-08 4:42 ` Christophe Leroy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200915171420.GK1221970@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=agordeev@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=aryabinin@virtuozzo.com \
--cc=benh@kernel.crashing.org \
--cc=borntraeger@de.ibm.com \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=dave.hansen@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=imbrenda@linux.ibm.com \
--cc=jdike@addtoit.com \
--cc=jhubbard@nvidia.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-um@lists.infradead.org \
--cc=linux@armlinux.org.uk \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=mpe@ellerman.id.au \
--cc=paulus@samba.org \
--cc=peterz@infradead.org \
--cc=richard@nod.at \
--cc=rppt@kernel.org \
--cc=sparclinux@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).