From: Ben Hutchings <ben@decadent.org.uk>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: akpm@linux-foundation.org, Denis Kirjanov <kda@linux-powerpc.org>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Konstantin Khlebnikov" <khlebnikov@yandex-team.ru>
Subject: [PATCH 3.16 91/99] clocksource: Prevent double add_timer_on() for watchdog_timer
Date: Wed, 20 May 2020 15:14:59 +0100 [thread overview]
Message-ID: <lsq.1589984009.153990885@decadent.org.uk> (raw)
In-Reply-To: <lsq.1589984008.673931885@decadent.org.uk>
3.16.84-rc1 review patch. If anyone has any objections, please let me know.
------------------
From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
commit febac332a819f0e764aa4da62757ba21d18c182b upstream.
Kernel crashes inside QEMU/KVM are observed:
kernel BUG at kernel/time/timer.c:1154!
BUG_ON(timer_pending(timer) || !timer->function) in add_timer_on().
At the same time another cpu got:
general protection fault: 0000 [#1] SMP PTI of poinson pointer 0xdead000000000200 in:
__hlist_del at include/linux/list.h:681
(inlined by) detach_timer at kernel/time/timer.c:818
(inlined by) expire_timers at kernel/time/timer.c:1355
(inlined by) __run_timers at kernel/time/timer.c:1686
(inlined by) run_timer_softirq at kernel/time/timer.c:1699
Unfortunately kernel logs are badly scrambled, stacktraces are lost.
Printing the timer->function before the BUG_ON() pointed to
clocksource_watchdog().
The execution of clocksource_watchdog() can race with a sequence of
clocksource_stop_watchdog() .. clocksource_start_watchdog():
expire_timers()
detach_timer(timer, true);
timer->entry.pprev = NULL;
raw_spin_unlock_irq(&base->lock);
call_timer_fn
clocksource_watchdog()
clocksource_watchdog_kthread() or
clocksource_unbind()
spin_lock_irqsave(&watchdog_lock, flags);
clocksource_stop_watchdog();
del_timer(&watchdog_timer);
watchdog_running = 0;
spin_unlock_irqrestore(&watchdog_lock, flags);
spin_lock_irqsave(&watchdog_lock, flags);
clocksource_start_watchdog();
add_timer_on(&watchdog_timer, ...);
watchdog_running = 1;
spin_unlock_irqrestore(&watchdog_lock, flags);
spin_lock(&watchdog_lock);
add_timer_on(&watchdog_timer, ...);
BUG_ON(timer_pending(timer) || !timer->function);
timer_pending() -> true
BUG()
I.e. inside clocksource_watchdog() watchdog_timer could be already armed.
Check timer_pending() before calling add_timer_on(). This is sufficient as
all operations are synchronized by watchdog_lock.
Fixes: 75c5158f70c0 ("timekeeping: Update clocksource with stop_machine")
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/158048693917.4378.13823603769948933793.stgit@buzz
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
---
kernel/time/clocksource.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -343,8 +343,15 @@ static void clocksource_watchdog(unsigne
next_cpu = cpumask_next(raw_smp_processor_id(), cpu_online_mask);
if (next_cpu >= nr_cpu_ids)
next_cpu = cpumask_first(cpu_online_mask);
- watchdog_timer.expires += WATCHDOG_INTERVAL;
- add_timer_on(&watchdog_timer, next_cpu);
+
+ /*
+ * Arm timer if not already pending: could race with concurrent
+ * pair clocksource_stop_watchdog() clocksource_start_watchdog().
+ */
+ if (!timer_pending(&watchdog_timer)) {
+ watchdog_timer.expires += WATCHDOG_INTERVAL;
+ add_timer_on(&watchdog_timer, next_cpu);
+ }
out:
spin_unlock(&watchdog_lock);
}
next prev parent reply other threads:[~2020-05-20 14:27 UTC|newest]
Thread overview: 112+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-20 14:13 [PATCH 3.16 00/99] 3.16.84-rc1 review Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 01/99] fs/namespace.c: fix mountpoint reference counter race Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 02/99] propagate_one(): mnt_set_mountpoint() needs mount_lock Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 03/99] spi: spi-dw: Add lock protect dw_spi rx/tx to prevent concurrent calls Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 04/99] padata: Remove unused but set variables Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 05/99] padata: avoid race in reordering Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 06/99] padata: get_next is never NULL Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 07/99] padata: ensure the reorder timer callback runs on the correct CPU Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 08/99] padata: ensure padata_do_serial() " Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 09/99] padata: Replace delayed timer with immediate workqueue in padata_reorder Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 10/99] padata: initialize pd->cpu with effective cpumask Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 11/99] padata: Remove broken queue flushing Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 12/99] padata: purge get_cpu and reorder_via_wq from padata_do_serial Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 13/99] crypto: pcrypt - Fix user-after-free on module unload Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 14/99] crypto: pcrypt - Do not clear MAY_SLEEP flag in original request Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 15/99] padata: always acquire cpu_hotplug_lock before pinst->lock Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 16/99] crypto: af_alg - Use bh_lock_sock in sk_destruct Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 17/99] crypto: api - Check spawn->alg under lock in crypto_drop_spawn Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 18/99] crypto: api - Fix race condition in crypto_spawn_alg Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 19/99] mmc: spi: Toggle SPI polarity, do not hardcode it Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 20/99] reiserfs: Fix memory leak of journal device string Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 21/99] reiserfs: Fix spurious unlock in reiserfs_fill_super() error handling Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 22/99] ath9k: fix storage endpoint lookup Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 23/99] rsi: fix use-after-free on failed probe and unbind Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 24/99] brcmfmac: Fix use after free in brcmf_sdio_readframes() Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 25/99] brcmfmac: abort and release host after error Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 26/99] brcmfmac: fix interface sanity check Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 27/99] orinoco_usb: " Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 28/99] rsi_91x_usb: " Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 29/99] zd1211rw: fix storage endpoint lookup Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 30/99] brcmfmac: Fix memory leak in brcmf_usbdev_qinit Ben Hutchings
2020-05-20 14:13 ` [PATCH 3.16 31/99] crypto: picoxcell - adjust the position of tasklet_init and fix missed tasklet_kill Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 32/99] scsi: qla2xxx: Fix mtcp dump collection failure Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 33/99] rtc: hym8563: Return -EINVAL if the time is known to be invalid Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 34/99] gianfar: Fix TX timestamping with a stacked DSA driver Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 35/99] pxa168fb: Fix the function used to release some memory in an error handling path Ben Hutchings
2020-05-21 14:09 ` Marion & Christophe JAILLET
2020-05-21 14:31 ` Marion & Christophe JAILLET
2020-05-21 20:28 ` Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 36/99] ALSA: sh: Fix compile warning wrt const Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 37/99] clk: tegra: Mark fuse clock as critical Ben Hutchings
2020-05-20 15:51 ` Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 38/99] ARM: tegra: Enable PLLP bypass during Tegra124 LP1 Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 39/99] media: iguanair: add sanity checks Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 40/99] media: iguanair: fix endpoint sanity check Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 41/99] ARM: dts: at91: sama5d3: fix maximum peripheral clock rates Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 42/99] ARM: dts: at91: sama5d3: define clock rate range for tcb1 Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 43/99] efi: Use early_mem*() instead of early_io*() Ben Hutchings
2020-05-20 15:53 ` Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 44/99] efi/x86: Map the entire EFI vendor string before copying it Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 45/99] PCI: Don't disable bridge BARs when assigning bus resources Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 46/99] power: supply: sbs-battery: Fix a signedness bug in sbs_get_battery_capacity() Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 47/99] dm space map common: fix to ensure new block isn't already in use Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 48/99] usb: dwc3: turn off VBUS when leaving host mode Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 49/99] usb: gadget: f_ncm: Use atomic_t to track in-flight request Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 50/99] usb: gadget: f_ecm: " Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 51/99] staging: wlan-ng: ensure error return is actually returned Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 52/99] nfs: NFS_SWAP should depend on SWAP Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 53/99] ubifs: Fix deadlock in concurrent bulk-read and writepage Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 54/99] x86/cpu: Update cached HLE state on write to TSX_CTRL_CPUID_CLEAR Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 55/99] jbd2: clear JBD2_ABORT flag before journal_reset to update log tail info when load journal Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 56/99] KVM: arm64: Only sign-extend MMIO up to register width Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 57/99] sparc32: fix struct ipc64_perm type definition Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 58/99] KVM: x86: Don't let userspace set host-reserved cr4 bits Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 59/99] KVM: nVMX: vmread should not set rflags to specify success in case of #PF Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 60/99] x86: kvm: avoid unused variable warning Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 61/99] KVM: x86/mmu: Apply max PA check for MMIO sptes to 32-bit KVM Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 62/99] USB: serial: ir-usb: add missing endpoint sanity check Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 63/99] USB: serial: ir-usb: fix link-speed handling Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 64/99] USB: serial: ir-usb: fix IrLAP framing Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 65/99] media: uvcvideo: Avoid cyclic entity chains due to malformed USB descriptors Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 66/99] KVM: PPC: Book3S HV: Uninit vCPU if vcore creation fails Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 67/99] KVM: PPC: Book3S PR: Free shared page if mmu initialization fails Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 68/99] KVM: x86: Free wbinvd_dirty_mask if vCPU creation fails Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 69/99] tracing: Fix very unlikely race of registering two stat tracers Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 70/99] tracing: Fix tracing_stat return values in error handling paths Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 71/99] jbd2: switch to use jbd2_journal_abort() when failed to submit the commit record Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 72/99] ext4, jbd2: ensure panic when aborting with zero errno Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 73/99] iwlegacy: ensure loop counter addr does not wrap and cause an infinite loop Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 74/99] CIFS: Fix task struct use-after-free on reconnect Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 75/99] net_sched: ematch: reject invalid TCF_EM_SIMPLE Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 76/99] KVM: x86: Protect x86_decode_insn from Spectre-v1/L1TF attacks Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 77/99] KVM: x86: Refactor picdev_write() to prevent " Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 78/99] KVM: x86: Protect ioapic_read_indirect() from " Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 79/99] KVM: x86: Protect ioapic_write_indirect() " Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 80/99] KVM: x86: Protect kvm_lapic_reg_write() " Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 81/99] kvm: x86: use macros to compute bank MSRs Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 82/99] KVM: x86: Protect MSR-based index computations from Spectre-v1/L1TF attacks in x86.c Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 83/99] KVM: x86: Protect DR-based index computations from Spectre-v1/L1TF attacks Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 84/99] KVM: Check for a bad hva before dropping into the ghc slow path Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 85/99] of: Add OF_DMA_DEFAULT_COHERENT & select it on powerpc Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 86/99] Btrfs: fix race between adding and putting tree mod seq elements and nodes Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 87/99] mm/mempolicy.c: fix out of bounds write in mpol_parse_str() Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 88/99] media/v4l2-core: set pages dirty upon releasing DMA buffers Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 89/99] tcp: clear tp->total_retrans in tcp_disconnect() Ben Hutchings
2020-05-20 14:14 ` [PATCH 3.16 90/99] ALSA: dummy: Fix PCM format loop in proc output Ben Hutchings
2020-05-20 14:14 ` Ben Hutchings [this message]
2020-05-20 14:15 ` [PATCH 3.16 92/99] cls_rsvp: fix rsvp_policy Ben Hutchings
2020-05-20 14:15 ` [PATCH 3.16 93/99] kconfig: fix broken dependency in randconfig-generated .config Ben Hutchings
2020-05-20 14:15 ` [PATCH 3.16 94/99] nfs: use kmap/kunmap directly Ben Hutchings
2020-05-20 14:15 ` [PATCH 3.16 95/99] NFS: Fix memory leaks and corruption in readdir Ben Hutchings
2020-05-20 14:15 ` [PATCH 3.16 96/99] NFS: Directory page cache pages need to be locked when read Ben Hutchings
2020-05-20 14:15 ` [PATCH 3.16 97/99] cifs: fail i/o on soft mounts if sessionsetup errors out Ben Hutchings
2020-05-20 14:15 ` [PATCH 3.16 98/99] bonding/alb: properly access headers in bond_alb_xmit() Ben Hutchings
2020-05-20 14:15 ` [PATCH 3.16 99/99] sunrpc: expiry_time should be seconds not timeval Ben Hutchings
2020-05-20 21:23 ` [PATCH 3.16 00/99] 3.16.84-rc1 review Guenter Roeck
2020-05-21 2:47 ` Chen-Yu Tsai
2020-05-21 7:40 ` Guenter Roeck
2020-05-21 20:22 ` Ben Hutchings
2020-05-21 20:20 ` Ben Hutchings
2020-05-21 22:37 ` Guenter Roeck
2020-05-22 0:00 ` Ben Hutchings
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=lsq.1589984009.153990885@decadent.org.uk \
--to=ben@decadent.org.uk \
--cc=akpm@linux-foundation.org \
--cc=kda@linux-powerpc.org \
--cc=khlebnikov@yandex-team.ru \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).