From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>,
Paul Mackerras <paulus@samba.org>,
Steven Rostedt <rostedt@goodmis.org>,
Stephane Eranian <eranian@google.com>,
Dave Jones <davej@redhat.com>, Ingo Molnar <mingo@kernel.org>
Subject: [PATCH 3.4 27/99] perf/x86: Fix event scheduling
Date: Fri, 7 Mar 2014 17:07:23 -0800 [thread overview]
Message-ID: <20140308010612.494332755@linuxfoundation.org> (raw)
In-Reply-To: <20140308010611.468206150@linuxfoundation.org>
3.4-stable review patch. If anyone has any objections, please let me know.
------------------
From: Peter Zijlstra <peterz@infradead.org>
commit 26e61e8939b1fe8729572dabe9a9e97d930dd4f6 upstream.
Vince "Super Tester" Weaver reported a new round of syscall fuzzing (Trinity) failures,
with perf WARN_ON()s triggering. He also provided traces of the failures.
This is I think the relevant bit:
> pec_1076_warn-2804 [000] d... 147.926153: x86_pmu_disable: x86_pmu_disable
> pec_1076_warn-2804 [000] d... 147.926153: x86_pmu_state: Events: {
> pec_1076_warn-2804 [000] d... 147.926156: x86_pmu_state: 0: state: .R config: ffffffffffffffff ( (null))
> pec_1076_warn-2804 [000] d... 147.926158: x86_pmu_state: 33: state: AR config: 0 (ffff88011ac99800)
> pec_1076_warn-2804 [000] d... 147.926159: x86_pmu_state: }
> pec_1076_warn-2804 [000] d... 147.926160: x86_pmu_state: n_events: 1, n_added: 0, n_txn: 1
> pec_1076_warn-2804 [000] d... 147.926161: x86_pmu_state: Assignment: {
> pec_1076_warn-2804 [000] d... 147.926162: x86_pmu_state: 0->33 tag: 1 config: 0 (ffff88011ac99800)
> pec_1076_warn-2804 [000] d... 147.926163: x86_pmu_state: }
> pec_1076_warn-2804 [000] d... 147.926166: collect_events: Adding event: 1 (ffff880119ec8800)
So we add the insn:p event (fd[23]).
At this point we should have:
n_events = 2, n_added = 1, n_txn = 1
> pec_1076_warn-2804 [000] d... 147.926170: collect_events: Adding event: 0 (ffff8800c9e01800)
> pec_1076_warn-2804 [000] d... 147.926172: collect_events: Adding event: 4 (ffff8800cbab2c00)
We try and add the {BP,cycles,br_insn} group (fd[3], fd[4], fd[15]).
These events are 0:cycles and 4:br_insn, the BP event isn't x86_pmu so
that's not visible.
group_sched_in()
pmu->start_txn() /* nop - BP pmu */
event_sched_in()
event->pmu->add()
So here we should end up with:
0: n_events = 3, n_added = 2, n_txn = 2
4: n_events = 4, n_added = 3, n_txn = 3
But seeing the below state on x86_pmu_enable(), the must have failed,
because the 0 and 4 events aren't there anymore.
Looking at group_sched_in(), since the BP is the leader, its
event_sched_in() must have succeeded, for otherwise we would not have
seen the sibling adds.
But since neither 0 or 4 are in the below state; their event_sched_in()
must have failed; but I don't see why, the complete state: 0,0,1:p,4
fits perfectly fine on a core2.
However, since we try and schedule 4 it means the 0 event must have
succeeded! Therefore the 4 event must have failed, its failure will
have put group_sched_in() into the fail path, which will call:
event_sched_out()
event->pmu->del()
on 0 and the BP event.
Now x86_pmu_del() will reduce n_events; but it will not reduce n_added;
giving what we see below:
n_event = 2, n_added = 2, n_txn = 2
> pec_1076_warn-2804 [000] d... 147.926177: x86_pmu_enable: x86_pmu_enable
> pec_1076_warn-2804 [000] d... 147.926177: x86_pmu_state: Events: {
> pec_1076_warn-2804 [000] d... 147.926179: x86_pmu_state: 0: state: .R config: ffffffffffffffff ( (null))
> pec_1076_warn-2804 [000] d... 147.926181: x86_pmu_state: 33: state: AR config: 0 (ffff88011ac99800)
> pec_1076_warn-2804 [000] d... 147.926182: x86_pmu_state: }
> pec_1076_warn-2804 [000] d... 147.926184: x86_pmu_state: n_events: 2, n_added: 2, n_txn: 2
> pec_1076_warn-2804 [000] d... 147.926184: x86_pmu_state: Assignment: {
> pec_1076_warn-2804 [000] d... 147.926186: x86_pmu_state: 0->33 tag: 1 config: 0 (ffff88011ac99800)
> pec_1076_warn-2804 [000] d... 147.926188: x86_pmu_state: 1->0 tag: 1 config: 1 (ffff880119ec8800)
> pec_1076_warn-2804 [000] d... 147.926188: x86_pmu_state: }
> pec_1076_warn-2804 [000] d... 147.926190: x86_pmu_enable: S0: hwc->idx: 33, hwc->last_cpu: 0, hwc->last_tag: 1 hwc->state: 0
So the problem is that x86_pmu_del(), when called from a
group_sched_in() that fails (for whatever reason), and without x86_pmu
TXN support (because the leader is !x86_pmu), will corrupt the n_added
state.
Reported-and-Tested-by: Vince Weaver <vincent.weaver@maine.edu>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Dave Jones <davej@redhat.com>
Link: http://lkml.kernel.org/r/20140221150312.GF3104@twins.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
arch/x86/kernel/cpu/perf_event.c | 3 +++
1 file changed, 3 insertions(+)
--- a/arch/x86/kernel/cpu/perf_event.c
+++ b/arch/x86/kernel/cpu/perf_event.c
@@ -1165,6 +1165,9 @@ static void x86_pmu_del(struct perf_even
for (i = 0; i < cpuc->n_events; i++) {
if (event == cpuc->event_list[i]) {
+ if (i >= cpuc->n_events - cpuc->n_added)
+ --cpuc->n_added;
+
if (x86_pmu.put_event_constraints)
x86_pmu.put_event_constraints(cpuc, event);
next prev parent reply other threads:[~2014-03-08 1:07 UTC|newest]
Thread overview: 123+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-03-08 1:06 [PATCH 3.4 00/99] 3.4.83-stable review Greg Kroah-Hartman
2014-03-08 1:06 ` [PATCH 3.4 01/99] ext4: dont try to modify s_flags if the the file system is read-only Greg Kroah-Hartman
2014-03-08 1:06 ` [PATCH 3.4 02/99] ext4: fix online resize with a non-standard blocks per group setting Greg Kroah-Hartman
2014-03-08 1:06 ` [PATCH 3.4 03/99] ext4: dont leave i_crtime.tv_sec uninitialized Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 04/99] ARM: 7953/1: mm: ensure TLB invalidation is complete before enabling MMU Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 05/99] ARM: 7957/1: add DSB after icache flush in __flush_icache_all() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 06/99] avr32: fix missing module.h causing build failure in mimc200/fram.c Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 07/99] avr32: Makefile: add -D__linux__ flag for gcc-4.4.7 use Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 08/99] cifs: ensure that uncached writes handle unmapped areas correctly Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 09/99] rtl8187: fix regression on MIPS without coherent DMA Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 10/99] rtlwifi: Fix incorrect return from rtl_ps_enable_nic() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 11/99] rtlwifi: rtl8192ce: Fix too long disable of IRQs Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 13/99] tg3: Fix deadlock in tg3_change_mtu() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 14/99] bonding: 802.3ad: make aggregator_identifier bond-private Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 15/99] usbnet: remove generic hard_header_len check Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 16/99] net: sctp: fix sctp_connectx abi for ia32 emulation/compat mode Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 17/99] net: add and use skb_gso_transport_seglen() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 18/99] net: ip, ipv6: handle gso skbs in forwarding path Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 19/99] ALSA: usb-audio: work around KEF X300A firmware bug Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 20/99] ASoC: wm8770: Fix wrong number of enum items Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 22/99] ASoC: sta32x: Fix array access overflow Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 23/99] ASoC: wm8958-dsp: Fix firmware block loading Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 24/99] SUNRPC: Fix races in xs_nospace() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 25/99] powerpc/le: Ensure that the stop-self RTAS token is handled correctly Greg Kroah-Hartman
2014-03-10 10:40 ` Luís Henriques
2014-03-11 23:08 ` Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 26/99] powerpc/crashdump : Fix page frame number check in copy_oldmem_page Greg Kroah-Hartman
2014-03-08 1:07 ` Greg Kroah-Hartman [this message]
2014-03-08 1:07 ` [PATCH 3.4 28/99] ata: enable quirk from jmicron JMB350 for JMB394 Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 29/99] sata_sil: apply MOD15WRITE quirk to TOSHIBA MK2561GSYN Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 30/99] PCI: Enable INTx if BIOS left them disabled Greg Kroah-Hartman
2014-03-08 13:50 ` Bjorn Helgaas
2014-03-11 23:08 ` Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 31/99] i7core_edac: Fix PCI device reference count Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 32/99] ACPI / video: Filter the _BCL table for duplicate brightness values Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 33/99] ACPI / processor: Rework processor throttling with work_on_cpu() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 34/99] USB: serial: option: blacklist interface 4 for Cinterion PHS8 and PXS8 Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 35/99] USB: ftdi_sio: add Cressi Leonardo PID Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 36/99] hwmon: (max1668) Fix writing the minimum temperature Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 37/99] workqueue: ensure @task is valid across kthread_stop() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 38/99] perf: Fix hotplug splat Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 39/99] SELinux: bigendian problems with filename trans rules Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 40/99] quota: Fix race between dqput() and dquot_scan_active() Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 41/99] dma: ste_dma40: dont dereference free:d descriptor Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 42/99] dm mpath: fix stalls when handling invalid ioctls Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 43/99] mm: vmscan: fix endless loop in kswapd balancing Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 44/99] cgroup: cgroup_subsys->fork() should be called after the task is added to css_set Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 45/99] KVM: s390: move kvm_guest_enter,exit closer to sie Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 46/99] s390/kvm: dont announce RRBM support Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 47/99] KVM: PPC: Emulate dcbf Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 48/99] KVM: IOMMU: hva align mapping page size Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 49/99] proc connector: reject unprivileged listener bumps Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 50/99] cgroup: fix RCU accesses to task->cgroups Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 51/99] mm/hotplug: correctly add new zone to all other nodes zone lists Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 52/99] perf tools: Remove extraneous newline when parsing hardware cache events Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 53/99] perf tools: Fix cache event name generation Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 54/99] nilfs2: fix issue with race condition of competition between segments for dirty blocks Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 55/99] fuse: readdir: check for slash in names Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 56/99] fuse: hotfix truncate_pagecache() issue Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 57/99] libceph: unregister request in __map_request failed and nofail == false Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 58/99] cifs: dont instantiate new dentries in readdir for inodes that need to be revalidated immediately Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 59/99] ncpfs: fix rmdir returns Device or resource busy Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 60/99] ext4/jbd2: dont wait (forever) for stale tid caused by wraparound Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 61/99] UBIFS: fix double free of ubifs_orphan objects Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 62/99] ext4: fix possible use-after-free with AIO Greg Kroah-Hartman
2014-03-08 1:07 ` [PATCH 3.4 63/99] cifs: adjust sequence number downward after signing NT_CANCEL request Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 64/99] nbd: correct disconnect behavior Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 65/99] block: Dont access request after it might be freed Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 66/99] ext4: return ENOMEM if sb_getblk() fails Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 67/99] [media] saa7134: Fix unlocked snd_pcm_stop() call Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 68/99] xen/boot: Disable BIOS SMP MP table search Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 69/99] xen/smp: Fix leakage of timer interrupt line for every CPU online/offline Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 70/99] xen/smp/spinlock: Fix leakage of the spinlock " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 71/99] xen-netback: fix sparse warning Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 72/99] xen-netback: coalesce slots in TX path and fix regressions Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 73/99] xen-netback: dont disconnect frontend when seeing oversize packet Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 74/99] xen/io/ring.h: new macro to detect whether there are too many requests on the ring Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 75/99] xen/blkback: Check for insane amounts of request on the ring (v6) Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 76/99] xen/events: mask events when changing their VCPU binding Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 77/99] sunrpc: clarify comments on rpc_make_runnable Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 78/99] SUNRPC: Prevent an rpc_task wakeup race Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 79/99] ASoC: imx-ssi: Fix occasional AC97 reset failure Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 80/99] ASoC: sglt5000: Fix the default value of CHIP_SSS_CTRL Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 81/99] ALSA: atiixp: Fix unlocked snd_pcm_stop() call Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 82/99] ALSA: 6fire: " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 83/99] ALSA: ua101: " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 84/99] ALSA: usx2y: " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 85/99] ALSA: pxa2xx: " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 86/99] ASoC: s6000: " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 87/99] staging: line6: " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 88/99] ALSA: asihpi: " Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 89/99] iwlwifi: fix flow handler debug code Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 90/99] iwlwifi: protect SRAM debugfs Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 91/99] iwlwifi: dont handle masked interrupt Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 92/99] iwlwifi: handle DMA mapping failures Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 93/99] iwlwifi: always copy first 16 bytes of commands Greg Kroah-Hartman
2014-03-22 14:19 ` Andreas Sturmlechner
2014-03-22 16:25 ` Greg Kroah-Hartman
2014-03-22 16:28 ` Andreas Sturmlechner
2014-03-22 16:51 ` Greg Kroah-Hartman
2014-03-22 17:38 ` Ben Hutchings
2014-03-22 18:43 ` Grumbach, Emmanuel
2014-03-22 21:01 ` Andreas Sturmlechner
2014-03-25 2:55 ` Ben Hutchings
2014-03-25 9:29 ` Andreas Sturmlechner
2014-03-25 12:05 ` Jianguo Wu
2014-03-25 17:28 ` [PATCH 3.4] iwlwifi: Complete backport of "iwlwifi: always copy first 16 bytes of commands" Ben Hutchings
2014-03-25 18:16 ` Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 94/99] iwlwifi: dvm: dont send BT_CONFIG on devices w/o Bluetooth Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 95/99] iwlwifi: dvm: fix calling ieee80211_chswitch_done() with NULL Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 96/99] iwlwifi: pcie: add SKUs for 6000, 6005 and 6235 series Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 97/99] rtlwifi: Fix endian error in extracting packet type Greg Kroah-Hartman
2014-03-08 1:08 ` [PATCH 3.4 98/99] net: asix: handle packets crossing URB boundaries Greg Kroah-Hartman
2014-03-08 9:47 ` [PATCH 3.4 00/99] 3.4.83-stable review Satoru Takeuchi
2014-03-08 14:35 ` Guenter Roeck
2014-03-08 16:18 ` Greg Kroah-Hartman
2014-03-08 17:10 ` Guenter Roeck
2014-03-08 20:50 ` Satoru Takeuchi
2014-03-09 4:18 ` Shuah Khan
2014-03-12 0:05 ` Greg Kroah-Hartman
2014-03-12 0:05 ` Greg Kroah-Hartman
2014-03-12 0:04 ` Greg Kroah-Hartman
2014-03-12 2:34 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140308010612.494332755@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=davej@redhat.com \
--cc=eranian@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulus@samba.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).