From: "Liam R. Howlett" <Liam.Howlett@oracle.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, stable@vger.kernel.org
Cc: maple-tree@lists.infradead.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org,
"Liam R. Howlett" <Liam.Howlett@Oracle.com>,
"Liam R . Howlett" <Liam.Howlett@oracle.com>,
syzbot+8d95422d3537159ca390@syzkaller.appspotmail.com
Subject: [PATCH 6.1 14/14] mm: enable maple tree RCU mode by default.
Date: Tue, 11 Apr 2023 11:10:55 -0400 [thread overview]
Message-ID: <20230411151055.2910579-15-Liam.Howlett@oracle.com> (raw)
In-Reply-To: <20230411151055.2910579-1-Liam.Howlett@oracle.com>
From: "Liam R. Howlett" <Liam.Howlett@Oracle.com>
commit 3dd4432549415f3c65dd52d5c687629efbf4ece1 upstream.
Use the maple tree in RCU mode for VMA tracking.
The maple tree tracks the stack and is able to update the pivot
(lower/upper boundary) in-place to allow the page fault handler to write
to the tree while holding just the mmap read lock. This is safe as the
writes to the stack have a guard VMA which ensures there will always be
a NULL in the direction of the growth and thus will only update a pivot.
It is possible, but not recommended, to have VMAs that grow up/down
without guard VMAs. syzbot has constructed a testcase which sets up a
VMA to grow and consume the empty space. Overwriting the entire NULL
entry causes the tree to be altered in a way that is not safe for
concurrent readers; the readers may see a node being rewritten or one
that does not match the maple state they are using.
Enabling RCU mode allows the concurrent readers to see a stable node and
will return the expected result.
Link: https://lkml.kernel.org/r/20230227173632.3292573-9-surenb@google.com
Cc: stable@vger.kernel.org
Fixes: d4af56c5c7c6 ("mm: start tracking VMAs with maple tree")
Signed-off-by: Liam R. Howlett <Liam.Howlett@oracle.com>
Reported-by: syzbot+8d95422d3537159ca390@syzkaller.appspotmail.com
---
include/linux/mm_types.h | 3 ++-
kernel/fork.c | 3 +++
mm/mmap.c | 3 ++-
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 500e536796ca..247aedb18d5c 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -725,7 +725,8 @@ struct mm_struct {
unsigned long cpu_bitmap[];
};
-#define MM_MT_FLAGS (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_LOCK_EXTERN)
+#define MM_MT_FLAGS (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_LOCK_EXTERN | \
+ MT_FLAGS_USE_RCU)
extern struct mm_struct init_mm;
/* Pointer magic because the dynamic array size confuses some compilers. */
diff --git a/kernel/fork.c b/kernel/fork.c
index a6d243a50be3..ec913b13c5ed 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -617,6 +617,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm,
if (retval)
goto out;
+ mt_clear_in_rcu(mas.tree);
mas_for_each(&old_mas, mpnt, ULONG_MAX) {
struct file *file;
@@ -703,6 +704,8 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm,
retval = arch_dup_mmap(oldmm, mm);
loop_out:
mas_destroy(&mas);
+ if (!retval)
+ mt_set_in_rcu(mas.tree);
out:
mmap_write_unlock(mm);
flush_tlb_mm(oldmm);
diff --git a/mm/mmap.c b/mm/mmap.c
index 177714886849..fe1db604dc49 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -2308,7 +2308,7 @@ do_mas_align_munmap(struct ma_state *mas, struct vm_area_struct *vma,
int count = 0;
int error = -ENOMEM;
MA_STATE(mas_detach, &mt_detach, 0, 0);
- mt_init_flags(&mt_detach, MT_FLAGS_LOCK_EXTERN);
+ mt_init_flags(&mt_detach, mas->tree->ma_flags & MT_FLAGS_LOCK_MASK);
mt_set_external_lock(&mt_detach, &mm->mmap_lock);
if (mas_preallocate(mas, vma, GFP_KERNEL))
@@ -3095,6 +3095,7 @@ void exit_mmap(struct mm_struct *mm)
*/
set_bit(MMF_OOM_SKIP, &mm->flags);
mmap_write_lock(mm);
+ mt_clear_in_rcu(&mm->mm_mt);
free_pgtables(&tlb, &mm->mm_mt, vma, FIRST_USER_ADDRESS,
USER_PGTABLES_CEILING);
tlb_finish_mmu(&tlb);
--
2.39.2
next prev parent reply other threads:[~2023-04-11 15:12 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-11 15:10 [PATCH 6.1 00/14] Backport of maple tree fixes for 6.1/6.2 Liam R. Howlett
2023-04-11 15:10 ` [PATCH 6.1 01/14] maple_tree: remove GFP_ZERO from kmem_cache_alloc() and kmem_cache_alloc_bulk() Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: remove GFP_ZERO from kmem_cache_alloc() and kmem_cache_alloc_bulk()" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: remove GFP_ZERO from kmem_cache_alloc() and kmem_cache_alloc_bulk()" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 02/14] maple_tree: fix potential rcu issue Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: fix potential rcu issue" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: fix potential rcu issue" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 03/14] maple_tree: reduce user error potential Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: reduce user error potential" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: reduce user error potential" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 04/14] maple_tree: fix handle of invalidated state in mas_wr_store_setup() Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: fix handle of invalidated state in mas_wr_store_setup()" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: fix handle of invalidated state in mas_wr_store_setup()" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 05/14] maple_tree: fix mas_prev() and mas_find() state handling Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: fix mas_prev() and mas_find() state handling" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: fix mas_prev() and mas_find() state handling" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 06/14] maple_tree: fix mas_skip_node() end slot detection Liam R. Howlett
2023-04-11 15:10 ` [PATCH 6.1 07/14] maple_tree: be more cautious about dead nodes Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: be more cautious about dead nodes" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: be more cautious about dead nodes" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 08/14] maple_tree: refine ma_state init from mas_start() Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: refine ma_state init from mas_start()" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: refine ma_state init from mas_start()" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 09/14] maple_tree: detect dead nodes in mas_start() Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: detect dead nodes in mas_start()" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: detect dead nodes in mas_start()" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 10/14] maple_tree: fix freeing of nodes in rcu mode Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: fix freeing of nodes in rcu mode" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: fix freeing of nodes in rcu mode" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 11/14] maple_tree: remove extra smp_wmb() from mas_dead_leaves() Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: remove extra smp_wmb() from mas_dead_leaves()" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: remove extra smp_wmb() from mas_dead_leaves()" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 12/14] maple_tree: add smp_rmb() to dead node detection Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: add smp_rmb() to dead node detection" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: add smp_rmb() to dead node detection" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` [PATCH 6.1 13/14] maple_tree: add RCU lock checking to rcu callback functions Liam R. Howlett
2023-04-12 8:12 ` Patch "maple_tree: add RCU lock checking to rcu callback functions" has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "maple_tree: add RCU lock checking to rcu callback functions" has been added to the 6.1-stable tree gregkh
2023-04-11 15:10 ` Liam R. Howlett [this message]
2023-04-12 8:12 ` Patch "mm: enable maple tree RCU mode by default." has been added to the 6.2-stable tree gregkh
2023-04-12 8:13 ` Patch "mm: enable maple tree RCU mode by default." has been added to the 6.1-stable tree gregkh
2023-04-12 8:12 ` [PATCH 6.1 00/14] Backport of maple tree fixes for 6.1/6.2 Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230411151055.2910579-15-Liam.Howlett@oracle.com \
--to=liam.howlett@oracle.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=maple-tree@lists.infradead.org \
--cc=stable@vger.kernel.org \
--cc=syzbot+8d95422d3537159ca390@syzkaller.appspotmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).