All of lore.kernel.org
 help / color / mirror / Atom feed
From: Peter Zijlstra <peterz@infradead.org>
To: Vince Weaver <vincent.weaver@maine.edu>
Cc: Ingo Molnar <mingo@kernel.org>,
	linux-kernel@vger.kernel.org,
	Thomas Gleixner <tglx@linutronix.de>,
	Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [perf] more perf_fuzzer memory corruption
Date: Thu, 1 May 2014 17:09:48 +0200	[thread overview]
Message-ID: <20140501150948.GR11096@twins.programming.kicks-ass.net> (raw)
In-Reply-To: <alpine.DEB.2.10.1405011023400.18142@vincent-weaver-1.umelst.maine.edu>

On Thu, May 01, 2014 at 10:27:45AM -0400, Vince Weaver wrote:
> On Thu, 1 May 2014, Vince Weaver wrote:
> 
> > On Wed, 30 Apr 2014, Peter Zijlstra wrote:
> > 
> > > Vince, could you add the below to whatever tracing muck you already
> > > have?
> 
> and this might be what you're looking for.  This is with a different 
> random seed than the one I've used for other traces, your patch changes 
> the syscall behavior enough that the one I was using before wasn't going 
> down the same path.
> 
> This WARNING is
> WARN_ON(event->hlist_entry.pprev && event->hlist_entry.pprev != LIST_POISON2);
> 
> [ 1554.910867] ------------[ cut here ]------------
> [ 1554.919535] WARNING: CPU: 5 PID: 16431 at kernel/events/core.c:3232 __free_event+0x86/0x90()
> [ 1554.931534] Modules linked in: fuse snd_hda_codec_hdmi x86_pkg_temp_thermal intel_powerclamp coretemp i915 kvm snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller crct10dif_pclmul drm_kms_helper snd_hda_codec crc32_pclmul ghash_clmulni_intel aesni_intel snd_hwdep snd_pcm drm snd_seq aes_x86_64 lrw evdev parport_pc iTCO_wdt gf128mul snd_timer tpm_tis snd_seq_device glue_helper i2c_i801 snd iTCO_vendor_support ablk_helper cryptd parport psmouse pcspkr serio_raw button processor video battery i2c_algo_bit i2c_core tpm wmi lpc_ich mfd_core mei_me mei soundcore sg sd_mod sr_mod crc_t10dif crct10dif_common cdrom ahci libahci ehci_pci xhci_hcd ehci_hcd libata e1000e scsi_mod ptp usbcore crc32c_intel pps_core usb_common fan thermal thermal_sys
> [ 1555.010748] CPU: 5 PID: 16431 Comm: perf_fuzzer Tainted: G        W     3.15.0-rc1+ #92
> [ 1555.020142] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014
> [ 1555.028924]  0000000000000009 ffff880117147b78 ffffffff81649790 0000000000000000
> [ 1555.037771]  ffff880117147bb0 ffffffff810646ad ffff880116daf800 0000000000000000
> [ 1555.046611]  ffff880036e45a10 ffff8800cde07588 ffff880116dafaa0 ffff880117147bc0
> [ 1555.055490] Call Trace:
> [ 1555.058963]  [<ffffffff81649790>] dump_stack+0x45/0x56
> [ 1555.065357]  [<ffffffff810646ad>] warn_slowpath_common+0x7d/0xa0
> [ 1555.072646]  [<ffffffff8106478a>] warn_slowpath_null+0x1a/0x20
> [ 1555.079740]  [<ffffffff81132e56>] __free_event+0x86/0x90
> [ 1555.086298]  [<ffffffff811339c9>] _free_event+0xc9/0x200
> [ 1555.092859]  [<ffffffff81133c78>] put_event+0x178/0x1f0
> [ 1555.099293]  [<ffffffff81133b68>] ? put_event+0x68/0x1f0
> [ 1555.105853]  [<ffffffff81133d12>] perf_release+0x12/0x20
> [ 1555.112422]  [<ffffffff811b64ec>] __fput+0xdc/0x1e0
> [ 1555.118576]  [<ffffffff811b663e>] ____fput+0xe/0x10
> [ 1555.124630]  [<ffffffff81085137>] task_work_run+0xa7/0xe0
> [ 1555.131307]  [<ffffffff81066d5c>] do_exit+0x2cc/0xa50
> [ 1555.137606]  [<ffffffff81076949>] ? get_signal_to_deliver+0x249/0x650
> [ 1555.145356]  [<ffffffff8106756c>] do_group_exit+0x4c/0xc0
> [ 1555.151970]  [<ffffffff81076991>] get_signal_to_deliver+0x291/0x650
> [ 1555.159542]  [<ffffffff81012438>] do_signal+0x48/0x990
> [ 1555.165869]  [<ffffffff81655592>] ? do_page_fault+0x22/0x30
> [ 1555.172645]  [<ffffffff81012df0>] do_notify_resume+0x70/0xa0
> [ 1555.179484]  [<ffffffff81651abc>] retint_signal+0x48/0x8c
> [ 1555.185991] ---[ end trace 41ec7a21bb260463 ]---
> [ 1556.112904] Slab corruption (Tainted: G        W    ): kmalloc-2048 start=ffff880116daf800, len=2048
> [ 1556.126221] 040: 6b 6b 6b 6b 6b 6b 6b 6b a8 75 36 ca 00 88 ff ff  kkkkkkkk.u6.....
> [ 1556.137243] Prev obj: start=ffff880116daf000, len=2048
> [ 1556.145566] 000: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
> [ 1556.156292] 010: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b  kkkkkkkkkkkkkkkk
> 
> I can try slapping ftrace on and getting a trace if you want.
> 
> I've been using
> trace-cmd record -e kmem -e raw_syscalls -p function -l '*perf*' -n 'perf_event_task_tick'
> 
> which is a compromise between log size and info, but as you've seen it 
> loses useful info, especially all the things in core.c that don't
> include *perf*.
> 

/proc/sys/kernel/traceoff_on_warning

Is also useful, it disables tracing the moment a warn/bug hits.

But yes please!

  reply	other threads:[~2014-05-01 15:09 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-04-15 21:37 [perf] more perf_fuzzer memory corruption Vince Weaver
2014-04-15 21:49 ` Thomas Gleixner
2014-04-16  3:21   ` Vince Weaver
2014-04-16  4:18     ` Vince Weaver
2014-04-16 14:15 ` Peter Zijlstra
2014-04-16 17:30   ` Vince Weaver
2014-04-16 17:43     ` Vince Weaver
2014-04-16 17:47       ` Peter Zijlstra
2014-04-17  9:48       ` Ingo Molnar
2014-04-17 11:45         ` Peter Zijlstra
2014-04-17 14:22           ` Ingo Molnar
2014-04-17 14:42             ` Vince Weaver
2014-04-17 14:54               ` Peter Zijlstra
2014-04-17 15:35                 ` Vince Weaver
2014-04-18 14:45                 ` Vince Weaver
2014-04-18 14:51                   ` Vince Weaver
2014-04-18 15:23                   ` Peter Zijlstra
2014-04-18 16:59                     ` Peter Zijlstra
2014-04-18 17:15                       ` Peter Zijlstra
2014-04-23 20:58                         ` Vince Weaver
2014-04-25  2:51                           ` Vince Weaver
2014-04-28 14:21                             ` Vince Weaver
2014-04-28 19:38                               ` Vince Weaver
2014-04-29  9:46                                 ` Peter Zijlstra
2014-04-29 18:21                                   ` Vince Weaver
2014-04-29 19:01                                     ` Peter Zijlstra
2014-04-29 20:59                                       ` Vince Weaver
2014-04-30 18:44                                         ` Peter Zijlstra
2014-04-30 21:08                                           ` Vince Weaver
2014-04-30 22:51                                             ` Thomas Gleixner
2014-05-01 10:26                                               ` Peter Zijlstra
2014-05-01 11:50                                                 ` Peter Zijlstra
2014-05-01 12:35                                                   ` Thomas Gleixner
2014-05-01 13:12                                                     ` Peter Zijlstra
2014-05-01 13:29                                                     ` Thomas Gleixner
2014-05-01 13:22                                                 ` Vince Weaver
2014-05-01 14:07                                           ` Vince Weaver
2014-05-01 14:27                                             ` Vince Weaver
2014-05-01 15:09                                               ` Peter Zijlstra [this message]
2014-05-01 15:50                                                 ` Vince Weaver
2014-05-01 16:31                                                   ` Thomas Gleixner
2014-05-01 17:18                                                     ` Vince Weaver
2014-05-01 18:49                                                       ` Vince Weaver
2014-05-01 21:32                                                         ` Vince Weaver
2014-05-02 11:15                                                         ` Peter Zijlstra
2014-05-02 15:42                                                         ` Peter Zijlstra
2014-05-02 16:22                                                           ` Vince Weaver
2014-05-02 16:22                                                             ` Peter Zijlstra
2014-05-02 16:43                                                               ` Vince Weaver
2014-05-02 17:27                                                                 ` Peter Zijlstra
2014-05-02 17:46                                                                   ` Vince Weaver
2014-05-02 19:12                                                                     ` Thomas Gleixner
2014-05-02 20:15                                                                       ` Vince Weaver
2014-05-02 20:45                                                                         ` Thomas Gleixner
2014-05-03  2:32                                                                           ` Vince Weaver
2014-05-03  3:02                                                                             ` Vince Weaver
2014-05-03  7:33                                                                               ` Peter Zijlstra
2014-05-05  9:31                                                                               ` Peter Zijlstra
2014-05-05 16:00                                                                                 ` Vince Weaver
2014-05-05 17:10                                                                                   ` Vince Weaver
2014-05-05 17:14                                                                                     ` Peter Zijlstra
2014-05-05 18:47                                                                                       ` Vince Weaver
2014-05-05 19:36                                                                                         ` Peter Zijlstra
2014-05-05 19:51                                                                                           ` Vince Weaver
2014-05-06  1:06                                                                                         ` Vince Weaver
2014-05-06 16:57                                                                                           ` Vince Weaver
2014-05-07 16:45                                                                                             ` Peter Zijlstra
2014-05-08 10:40                                                                                       ` [tip:perf/core] perf: Fix perf_event_init_context() tip-bot for Peter Zijlstra
2014-05-05 17:29                                                                                   ` [perf] more perf_fuzzer memory corruption Ingo Molnar
2014-05-06  4:51                                                                                     ` Vince Weaver
2014-05-06 17:06                                                                                       ` Vince Weaver
2014-05-07 19:12                                                                                         ` Ingo Molnar
2014-05-07 19:11                                                                                       ` Ingo Molnar
2014-05-08 10:40                                                                                 ` [tip:perf/core] perf: Fix race in removing an event tip-bot for Peter Zijlstra
2014-05-02 17:06                                                           ` [perf] more perf_fuzzer memory corruption Vince Weaver
2014-05-02 17:04                                                             ` Peter Zijlstra
2014-04-29 19:26                                     ` Steven Rostedt
2014-04-29  8:52                               ` Peter Zijlstra
2014-04-29 18:11                                 ` Vince Weaver
2014-04-29 19:21                                   ` Steven Rostedt
2014-04-28 17:48                             ` Thomas Gleixner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140501150948.GR11096@twins.programming.kicks-ass.net \
    --to=peterz@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    --cc=vincent.weaver@maine.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.