All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Nicholas Piggin <npiggin@gmail.com>,
	"David S. Miller" <davem@davemloft.net>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Sasha Levin <sashal@kernel.org>
Subject: [PATCH 4.4 12/86] sparc64: remove mm_cpumask clearing to fix kthread_use_mm race
Date: Mon,  9 Nov 2020 13:54:19 +0100	[thread overview]
Message-ID: <20201109125021.455742885@linuxfoundation.org> (raw)
In-Reply-To: <20201109125020.852643676@linuxfoundation.org>

From: Nicholas Piggin <npiggin@gmail.com>

[ Upstream commit bafb056ce27940c9994ea905336aa8f27b4f7275 ]

The de facto (and apparently uncommented) standard for using an mm had,
thanks to this code in sparc if nothing else, been that you must have a
reference on mm_users *and that reference must have been obtained with
mmget()*, i.e., from a thread with a reference to mm_users that had used
the mm.

The introduction of mmget_not_zero() in commit d2005e3f41d4
("userfaultfd: don't pin the user memory in userfaultfd_file_create()")
allowed mm_count holders to aoperate on user mappings asynchronously
from the actual threads using the mm, but they were not to load those
mappings into their TLB (i.e., walking vmas and page tables is okay,
kthread_use_mm() is not).

io_uring 2b188cc1bb857 ("Add io_uring IO interface") added code which
does a kthread_use_mm() from a mmget_not_zero() refcount.

The problem with this is code which previously assumed mm == current->mm
and mm->mm_users == 1 implies the mm will remain single-threaded at
least until this thread creates another mm_users reference, has now
broken.

arch/sparc/kernel/smp_64.c:

    if (atomic_read(&mm->mm_users) == 1) {
        cpumask_copy(mm_cpumask(mm), cpumask_of(cpu));
        goto local_flush_and_out;
    }

vs fs/io_uring.c

    if (unlikely(!(ctx->flags & IORING_SETUP_SQPOLL) ||
                 !mmget_not_zero(ctx->sqo_mm)))
        return -EFAULT;
    kthread_use_mm(ctx->sqo_mm);

mmget_not_zero() could come in right after the mm_users == 1 test, then
kthread_use_mm() which sets its CPU in the mm_cpumask. That update could
be lost if cpumask_copy() occurs afterward.

I propose we fix this by allowing mmget_not_zero() to be a first-class
reference, and not have this obscure undocumented and unchecked
restriction.

The basic fix for sparc64 is to remove its mm_cpumask clearing code. The
optimisation could be effectively restored by sending IPIs to mm_cpumask
members and having them remove themselves from mm_cpumask. This is more
tricky so I leave it as an exercise for someone with a sparc64 SMP.
powerpc has a (currently similarly broken) example.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Link: https://lore.kernel.org/r/20200914045219.3736466-4-npiggin@gmail.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/sparc/kernel/smp_64.c | 65 ++++++++------------------------------
 1 file changed, 14 insertions(+), 51 deletions(-)

diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c
index 46866b2097e81..bd27e86ce36db 100644
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -1030,38 +1030,9 @@ void smp_fetch_global_pmu(void)
  * are flush_tlb_*() routines, and these run after flush_cache_*()
  * which performs the flushw.
  *
- * The SMP TLB coherency scheme we use works as follows:
- *
- * 1) mm->cpu_vm_mask is a bit mask of which cpus an address
- *    space has (potentially) executed on, this is the heuristic
- *    we use to avoid doing cross calls.
- *
- *    Also, for flushing from kswapd and also for clones, we
- *    use cpu_vm_mask as the list of cpus to make run the TLB.
- *
- * 2) TLB context numbers are shared globally across all processors
- *    in the system, this allows us to play several games to avoid
- *    cross calls.
- *
- *    One invariant is that when a cpu switches to a process, and
- *    that processes tsk->active_mm->cpu_vm_mask does not have the
- *    current cpu's bit set, that tlb context is flushed locally.
- *
- *    If the address space is non-shared (ie. mm->count == 1) we avoid
- *    cross calls when we want to flush the currently running process's
- *    tlb state.  This is done by clearing all cpu bits except the current
- *    processor's in current->mm->cpu_vm_mask and performing the
- *    flush locally only.  This will force any subsequent cpus which run
- *    this task to flush the context from the local tlb if the process
- *    migrates to another cpu (again).
- *
- * 3) For shared address spaces (threads) and swapping we bite the
- *    bullet for most cases and perform the cross call (but only to
- *    the cpus listed in cpu_vm_mask).
- *
- *    The performance gain from "optimizing" away the cross call for threads is
- *    questionable (in theory the big win for threads is the massive sharing of
- *    address space state across processors).
+ * mm->cpu_vm_mask is a bit mask of which cpus an address
+ * space has (potentially) executed on, this is the heuristic
+ * we use to limit cross calls.
  */
 
 /* This currently is only used by the hugetlb arch pre-fault
@@ -1071,18 +1042,13 @@ void smp_fetch_global_pmu(void)
 void smp_flush_tlb_mm(struct mm_struct *mm)
 {
 	u32 ctx = CTX_HWBITS(mm->context);
-	int cpu = get_cpu();
 
-	if (atomic_read(&mm->mm_users) == 1) {
-		cpumask_copy(mm_cpumask(mm), cpumask_of(cpu));
-		goto local_flush_and_out;
-	}
+	get_cpu();
 
 	smp_cross_call_masked(&xcall_flush_tlb_mm,
 			      ctx, 0, 0,
 			      mm_cpumask(mm));
 
-local_flush_and_out:
 	__flush_tlb_mm(ctx, SECONDARY_CONTEXT);
 
 	put_cpu();
@@ -1105,17 +1071,15 @@ void smp_flush_tlb_pending(struct mm_struct *mm, unsigned long nr, unsigned long
 {
 	u32 ctx = CTX_HWBITS(mm->context);
 	struct tlb_pending_info info;
-	int cpu = get_cpu();
+
+	get_cpu();
 
 	info.ctx = ctx;
 	info.nr = nr;
 	info.vaddrs = vaddrs;
 
-	if (mm == current->mm && atomic_read(&mm->mm_users) == 1)
-		cpumask_copy(mm_cpumask(mm), cpumask_of(cpu));
-	else
-		smp_call_function_many(mm_cpumask(mm), tlb_pending_func,
-				       &info, 1);
+	smp_call_function_many(mm_cpumask(mm), tlb_pending_func,
+			       &info, 1);
 
 	__flush_tlb_pending(ctx, nr, vaddrs);
 
@@ -1125,14 +1089,13 @@ void smp_flush_tlb_pending(struct mm_struct *mm, unsigned long nr, unsigned long
 void smp_flush_tlb_page(struct mm_struct *mm, unsigned long vaddr)
 {
 	unsigned long context = CTX_HWBITS(mm->context);
-	int cpu = get_cpu();
 
-	if (mm == current->mm && atomic_read(&mm->mm_users) == 1)
-		cpumask_copy(mm_cpumask(mm), cpumask_of(cpu));
-	else
-		smp_cross_call_masked(&xcall_flush_tlb_page,
-				      context, vaddr, 0,
-				      mm_cpumask(mm));
+	get_cpu();
+
+	smp_cross_call_masked(&xcall_flush_tlb_page,
+			      context, vaddr, 0,
+			      mm_cpumask(mm));
+
 	__flush_tlb_page(context, vaddr);
 
 	put_cpu();
-- 
2.27.0




  parent reply	other threads:[~2020-11-09 12:57 UTC|newest]

Thread overview: 92+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-09 12:54 [PATCH 4.4 00/86] 4.4.242-rc1 review Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 01/86] SUNRPC: ECONNREFUSED should cause a rebind Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 02/86] scripts/setlocalversion: make git describe output more reliable Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 03/86] powerpc/powernv/opal-dump : Use IRQ_HANDLED instead of numbers in interrupt handler Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 04/86] efivarfs: Replace invalid slashes with exclamation marks in dentries Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 05/86] ravb: Fix bit fields checking in ravb_hwtstamp_get() Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 06/86] tipc: fix memory leak caused by tipc_buf_append() Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 07/86] mtd: lpddr: Fix bad logic in print_drs_error Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 08/86] ata: sata_rcar: Fix DMA boundary mask Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 09/86] fscrypt: return -EXDEV for incompatible rename or link into encrypted dir Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 10/86] f2fs crypto: avoid unneeded memory allocation in ->readdir Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 11/86] powerpc/powernv/smp: Fix spurious DBG() warning Greg Kroah-Hartman
2020-11-09 12:54 ` Greg Kroah-Hartman [this message]
2020-11-09 12:54 ` [PATCH 4.4 13/86] f2fs: fix to check segment boundary during SIT page readahead Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 14/86] um: change sigio_spinlock to a mutex Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 15/86] xfs: fix realtime bitmap/summary file truncation when growing rt volume Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 16/86] video: fbdev: pvr2fb: initialize variables Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 17/86] ath10k: fix VHT NSS calculation when STBC is enabled Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 18/86] mmc: via-sdmmc: Fix data race bug Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 19/86] printk: reduce LOG_BUF_SHIFT range for H8300 Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 20/86] kgdb: Make "kgdbcon" work properly with "kgdb_earlycon" Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 21/86] USB: adutux: fix debugging Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 22/86] drivers/net/wan/hdlc_fr: Correctly handle special skb->protocol values Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 23/86] power: supply: test_power: add missing newlines when printing parameters by sysfs Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 24/86] md/bitmap: md_bitmap_get_counter returns wrong blocks Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 25/86] clk: ti: clockdomain: fix static checker warning Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 26/86] net: 9p: initialize sun_server.sun_path to have addrs value only when addr is valid Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 27/86] drivers: watchdog: rdc321x_wdt: Fix race condition bugs Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 28/86] ext4: Detect already used quota file early Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 29/86] gfs2: add validation checks for size of superblock Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 30/86] memory: emif: Remove bogus debugfs error handling Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 31/86] ARM: dts: s5pv210: move PMU node out of clock controller Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 32/86] ARM: dts: s5pv210: remove dedicated audio-subsystem node Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 33/86] md/raid5: fix oops during stripe resizing Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 34/86] leds: bcm6328, bcm6358: use devres LED registering function Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 35/86] NFS: fix nfs_path in case of a rename retry Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 36/86] ACPI / extlog: Check for RDMSR failure Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 37/86] ACPI: video: use ACPI backlight for HP 635 Notebook Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 38/86] acpi-cpufreq: Honor _PSD table setting on new AMD CPUs Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 39/86] w1: mxc_w1: Fix timeout resolution problem leading to bus error Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 40/86] scsi: mptfusion: Fix null pointer dereferences in mptscsih_remove() Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 41/86] btrfs: reschedule if necessary when logging directory items Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 42/86] vt: keyboard, simplify vt_kdgkbsent Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 43/86] vt: keyboard, extend func_buf_lock to readers Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 44/86] dmaengine: dma-jz4780: Fix race in jz4780_dma_tx_status Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 45/86] iio:gyro:itg3200: Fix timestamp alignment and prevent data leak Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 46/86] powerpc/powernv/elog: Fix race while processing OPAL error log event Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 47/86] ubifs: dent: Fix some potential memory leaks while iterating entries Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 48/86] ubi: check kthread_should_stop() after the setting of task state Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 49/86] ia64: fix build error with !COREDUMP Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 50/86] ceph: promote to unsigned long long before shifting Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 51/86] libceph: clear con->out_msg on Policy::stateful_server faults Greg Kroah-Hartman
2020-11-09 12:54 ` [PATCH 4.4 52/86] 9P: Cast to loff_t before multiplying Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 53/86] ring-buffer: Return 0 on success from ring_buffer_resize() Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 54/86] vringh: fix __vringh_iov() when riov and wiov are different Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 55/86] tty: make FONTX ioctl use the tty pointer they were actually passed Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 56/86] arm64: berlin: Select DW_APB_TIMER_OF Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 57/86] cachefiles: Handle readpage error correctly Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 58/86] hil/parisc: Disable HIL driver when it gets stuck Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 59/86] ARM: samsung: fix PM debug build with DEBUG_LL but !MMU Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 60/86] ARM: s3c24xx: fix missing system reset Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 61/86] device property: Keep secondary firmware node secondary by type Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 62/86] device property: Dont clear secondary pointer for shared primary firmware node Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 63/86] staging: comedi: cb_pcidas: Allow 2-channel commands for AO subdevice Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 64/86] xen/events: dont use chip_data for legacy IRQs Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 65/86] tipc: fix use-after-free in tipc_bcast_get_mode Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 66/86] gianfar: Replace skb_realloc_headroom with skb_cow_head for PTP Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 67/86] gianfar: Account for Tx PTP timestamp in the skb headroom Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 68/86] Fonts: Replace discarded const qualifier Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 69/86] ALSA: usb-audio: Add implicit feedback quirk for Qu-16 Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 70/86] ftrace: Fix recursion check for NMI test Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 71/86] ftrace: Handle tracing when switching between context Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 72/86] ARM: dts: sun4i-a10: fix cpu_alert temperature Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 73/86] x86/kexec: Use up-to-dated screen_info copy to fill boot params Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 74/86] of: Fix reserved-memory overlap detection Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 75/86] scsi: core: Dont start concurrent async scan on same host Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 76/86] vsock: use ns_capable_noaudit() on socket create Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 77/86] vt: Disable KD_FONT_OP_COPY Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 78/86] fork: fix copy_process(CLONE_PARENT) race with the exiting ->real_parent Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 79/86] serial: 8250_mtk: Fix uart_get_baud_rate warning Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 80/86] serial: txx9: add missing platform_driver_unregister() on error in serial_txx9_init Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 81/86] USB: serial: cyberjack: fix write-URB completion race Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 82/86] USB: serial: option: add LE910Cx compositions 0x1203, 0x1230, 0x1231 Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 83/86] USB: serial: option: add Telit FN980 composition 0x1055 Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 84/86] USB: Add NO_LPM quirk for Kingston flash drive Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 85/86] ARC: stack unwinding: avoid indefinite looping Greg Kroah-Hartman
2020-11-09 12:55 ` [PATCH 4.4 86/86] Revert "ARC: entry: fix potential EFA clobber when TIF_SYSCALL_TRACE" Greg Kroah-Hartman
2020-11-09 15:39 ` [PATCH 4.4 00/86] 4.4.242-rc1 review Jon Hunter
2020-11-09 18:51 ` Pavel Machek
2020-11-09 23:04 ` Guenter Roeck
2020-11-09 23:27 ` Shuah Khan
2020-11-10 10:36 ` Naresh Kamboju

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201109125021.455742885@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=davem@davemloft.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mpe@ellerman.id.au \
    --cc=npiggin@gmail.com \
    --cc=sashal@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.