From: Peter Zijlstra <peterz@infradead.org> To: Andrew Morton <akpm@linux-foundation.org> Cc: linux-kernel@vger.kernel.org, luto@amacapital.net, axboe@kernel.dk, keescook@chromium.org, torvalds@linux-foundation.org, jannh@google.com, will@kernel.org Subject: [PATCH] mm/tlb: Fix use_mm() vs TLB invalidate Date: Fri, 21 Feb 2020 12:11:38 +0100 Message-ID: <20200221111138.GX14897@hirez.programming.kicks-ass.net> (raw) For SMP systems using IPI based TLB invalidation, looking at current->active_mm is entirely reasonable. This then presents the following race condition: CPU0 CPU1 flush_tlb_mm(mm) use_mm(mm) <send-IPI> tsk->active_mm = mm; <IPI> if (tsk->active_mm == mm) // flush TLBs </IPI> switch_mm(old_mm,mm,tsk); Where it is possible the IPI flushed the TLBs for @old_mm, not @mm, because the IPI lands before we actually switched. Avoid this by disabling IRQs across changing ->active_mm and switch_mm(). [ There are all sorts of reasons this might be harmless for various architecture specific reasons, but best not leave the door open at all. ] Cc: stable@kernel.org Reported-by: Andy Lutomirski <luto@amacapital.net> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> --- Index: linux-2.6/mm/mmu_context.c =================================================================== --- linux-2.6.orig/mm/mmu_context.c +++ linux-2.6/mm/mmu_context.c @@ -24,14 +24,19 @@ void use_mm(struct mm_struct *mm) struct mm_struct *active_mm; struct task_struct *tsk = current; + BUG_ON(!(tsk->flags & PF_KTHREAD)); + BUG_ON(tsk->mm != NULL); + task_lock(tsk); + local_irq_disable(); active_mm = tsk->active_mm; if (active_mm != mm) { mmgrab(mm); tsk->active_mm = mm; } tsk->mm = mm; - switch_mm(active_mm, mm, tsk); + switch_mm_irqs_off(active_mm, mm, tsk); + local_irq_enable(); task_unlock(tsk); #ifdef finish_arch_post_lock_switch finish_arch_post_lock_switch(); @@ -54,11 +59,15 @@ void unuse_mm(struct mm_struct *mm) { struct task_struct *tsk = current; + BUG_ON(!(tsk->flags & PF_KTHREAD)); + task_lock(tsk); sync_mm_rss(mm); + local_irq_disable(); tsk->mm = NULL; /* active_mm is still 'mm' */ enter_lazy_tlb(mm, tsk); + local_irq_enable(); task_unlock(tsk); } EXPORT_SYMBOL_GPL(unuse_mm);
next reply index Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-02-21 11:11 Peter Zijlstra [this message] 2020-02-21 19:19 ` Linus Torvalds 2020-02-21 19:22 ` Andy Lutomirski 2020-02-21 19:26 ` Linus Torvalds 2020-02-21 23:10 ` Kees Cook 2020-02-21 23:57 ` Linus Torvalds 2020-02-22 0:29 ` Kees Cook
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200221111138.GX14897@hirez.programming.kicks-ass.net \ --to=peterz@infradead.org \ --cc=akpm@linux-foundation.org \ --cc=axboe@kernel.dk \ --cc=jannh@google.com \ --cc=keescook@chromium.org \ --cc=linux-kernel@vger.kernel.org \ --cc=luto@amacapital.net \ --cc=torvalds@linux-foundation.org \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
LKML Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \ linux-kernel@vger.kernel.org public-inbox-index lkml Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel AGPL code for this site: git clone https://public-inbox.org/public-inbox.git