From: Eran Liberty <liberty@extricom.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>,
linux-kernel@vger.kernel.org, linuxppc-dev@ozlabs.org,
Steven Rostedt <srostedt@redhat.com>,
Scott Wood <scottwood@freescale.com>,
Alan Modra <amodra@au1.ibm.com>,
Segher Boessenkool <segher@kernel.crashing.org>
Subject: Re: ftrace introduces instability into kernel 2.6.27(-rc2,-rc3)
Date: Wed, 20 Aug 2008 17:02:28 +0300 [thread overview]
Message-ID: <48AC23F4.80900@extricom.com> (raw)
In-Reply-To: <alpine.DEB.1.10.0808200941560.13132@gandalf.stny.rr.com>
Steven Rostedt wrote:
> On Wed, 20 Aug 2008, Eran Liberty wrote:
>
>
>> Steven Rostedt wrote:
>>
>>> On Wed, 20 Aug 2008, Steven Rostedt wrote:
>>>
>>>
>>>
>>>> On Wed, 20 Aug 2008, Benjamin Herrenschmidt wrote:
>>>>
>>>>
>>>>
>>>>> Found the problem (or at least -a- problem), it's a gcc bug.
>>>>>
>>>>> Well, first I must say the code generated by -pg is just plain
>>>>> horrible :-)
>>>>>
>>>>> Appart from that, look at the exit of, for example, __d_lookup, as
>>>>> generated by gcc when ftrace is enabled:
>>>>>
>>>>> c00c0498: 38 60 00 00 li r3,0
>>>>> c00c049c: 81 61 00 00 lwz r11,0(r1)
>>>>> c00c04a0: 80 0b 00 04 lwz r0,4(r11)
>>>>> c00c04a4: 7d 61 5b 78 mr r1,r11
>>>>> c00c04a8: bb 0b ff e0 lmw r24,-32(r11)
>>>>> c00c04ac: 7c 08 03 a6 mtlr r0
>>>>> c00c04b0: 4e 80 00 20 blr
>>>>>
>>>>> As you can see, it restores r1 -before- it pops r24..r31 off
>>>>> the stack ! I let you imagine what happens if an interrupt happens
>>>>> just in between those two instructions (mr and lmw). We don't do
>>>>> redzones on our ABI, so basically, the registers end up corrupted
>>>>> by the interrupt.
>>>>>
>>>>>
>>>> Ouch! You've disassembled this without -pg too, and it does not have this
>>>> bug? What version of gcc do you have?
>>>>
>>>>
>>>>
>>> I have:
>>> gcc (Debian 4.3.1-2) 4.3.1
>>>
>>> c00c64c8: 81 61 00 00 lwz r11,0(r1)
>>> c00c64cc: 7f 83 e3 78 mr r3,r28
>>> c00c64d0: 80 0b 00 04 lwz r0,4(r11)
>>> c00c64d4: ba eb ff dc lmw r23,-36(r11)
>>> c00c64d8: 7d 61 5b 78 mr r1,r11
>>> c00c64dc: 7c 08 03 a6 mtlr r0
>>> c00c64e0: 4e 80 00 20 blr
>>>
>>>
>>> My version looks fine. I'm thinking that this is a separate issue than what
>>> Eran is seeing.
>>>
>>> Eran, can you do an "objdump -dr vmlinux" and search for __d_lookup, and
>>> print out the end of the function dump.
>>>
>>> Thanks,
>>>
>>> -- Steve
>>>
>>>
>>>
>>>
>>>
>> powerpc-linux-gnu-objdump -dr --start-address=0xc00bb584 vmlinux | head -n 100
>>
>> vmlinux: file format elf32-powerpc
>>
>> Disassembly of section .text:
>>
>> c00bb584 <__d_lookup>:
>>
>
> [...]
>
>
>> c00bb670: 41 9e 00 50 beq- cr7,c00bb6c0 <__d_lookup+0x13c>
>> c00bb674: 83 de 00 00 lwz r30,0(r30)
>> c00bb678: 2f 9e 00 00 cmpwi cr7,r30,0
>> c00bb67c: 40 9e ff 98 bne+ cr7,c00bb614 <__d_lookup+0x90>
>> c00bb680: 38 60 00 00 li r3,0
>> c00bb684: 81 61 00 00 lwz r11,0(r1)
>> c00bb688: 80 0b 00 04 lwz r0,4(r11)
>> c00bb68c: 7d 61 5b 78 mr r1,r11
>>
>
> [ BUG HERE IF INTERRUPT HAPPENS ]
>
>
>> c00bb690: bb 0b ff e0 lmw r24,-32(r11)
>> c00bb694: 7c 08 03 a6 mtlr r0
>> c00bb698: 4e 80 00 20 blr
>>
>
> Yep, you have the same bug in your compiler.
>
> -- Steve
>
Hmm... so whats now?
Is there a way to prove this scenario is indeed the one that caused the
opps?
-- Liberty
next prev parent reply other threads:[~2008-08-20 14:04 UTC|newest]
Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-06-18 14:18 2.6.24-rc4: pci_remove_bus_device() => pci_scan_child_bus() => pci_bus_add_devices bug? Eran Liberty
2008-07-20 10:31 ` [PATCH 2.6.24-rc4] PCI: refuse to re-add a device to a bus upon pci_scan_child_bus() Eran Liberty
2008-07-20 16:48 ` Eran Liberty
2008-07-21 19:18 ` [PATCH 2.6.26] " Eran Liberty
2008-07-21 19:49 ` Matthew Wilcox
2008-07-22 8:21 ` eran liberty
2008-07-22 11:49 ` Matthew Wilcox
2008-07-22 13:08 ` Eran Liberty
2008-07-22 13:14 ` Eran Liberty
2008-07-22 14:13 ` Matthew Wilcox
2008-07-22 15:25 ` Eran Liberty
2008-07-22 16:52 ` Matthew Wilcox
2008-07-22 17:41 ` Eran Liberty
2008-07-22 18:11 ` Matthew Wilcox
2008-07-23 18:31 ` Eran Liberty
2008-07-27 11:01 ` Eran Liberty
2008-07-27 15:08 ` Matthew Wilcox
2008-08-18 8:08 ` ftrace introduces instability into kernel 2.6.27(-rc2,-rc3) Eran Liberty
2008-08-18 15:07 ` Steven Rostedt
2008-08-18 15:47 ` Mathieu Desnoyers
2008-08-18 16:12 ` Steven Rostedt
2008-08-18 17:04 ` Mathieu Desnoyers
2008-08-18 17:21 ` Scott Wood
2008-08-18 18:27 ` Steven Rostedt
2008-08-18 18:29 ` Scott Wood
2008-08-19 1:53 ` Benjamin Herrenschmidt
2008-08-19 2:28 ` Steven Rostedt
2008-08-19 2:39 ` Benjamin Herrenschmidt
2008-08-19 2:41 ` Steven Rostedt
2008-08-19 2:47 ` Mathieu Desnoyers
2008-08-19 3:32 ` Steven Rostedt
2008-08-19 3:36 ` Mathieu Desnoyers
2008-08-19 4:00 ` Steven Rostedt
2008-08-19 16:47 ` Steven Rostedt
2008-08-19 17:34 ` Mathieu Desnoyers
2008-08-19 21:08 ` Steven Rostedt
2008-08-20 9:40 ` Nick Piggin
2008-08-19 21:47 ` Benjamin Herrenschmidt
2008-08-19 23:58 ` Jeremy Fitzhardinge
2008-08-20 1:17 ` Benjamin Herrenschmidt
2008-08-19 2:56 ` Benjamin Herrenschmidt
2008-08-19 3:12 ` Steven Rostedt
2008-08-19 4:17 ` Benjamin Herrenschmidt
2008-08-20 7:18 ` Benjamin Herrenschmidt
2008-08-20 13:14 ` Steven Rostedt
2008-08-20 13:19 ` Steven Rostedt
2008-08-20 13:36 ` Eran Liberty
2008-08-20 13:43 ` Steven Rostedt
2008-08-20 14:02 ` Eran Liberty [this message]
2008-08-20 14:55 ` Jon Smirl
2008-08-20 15:23 ` Steven Rostedt
2008-08-20 18:23 ` Eran Liberty
2008-08-20 18:33 ` Steven Rostedt
2008-08-20 15:27 ` Steven Rostedt
2008-08-20 21:37 ` Benjamin Herrenschmidt
2008-08-20 14:16 ` Josh Boyer
2008-08-20 14:22 ` Steven Rostedt
2008-08-20 14:50 ` Josh Boyer
2008-09-15 16:30 ` [PATCH 2.6.26] SERIAL DRIVER: Handle Multiple consecutive sysrq from the serial Eran Liberty
2008-09-17 23:46 ` Andrew Morton
2008-09-18 6:58 ` Eran Liberty
2008-08-20 21:36 ` ftrace introduces instability into kernel 2.6.27(-rc2,-rc3) Benjamin Herrenschmidt
2008-08-20 21:44 ` Steven Rostedt
2008-08-18 18:47 ` Steven Rostedt
2008-08-18 18:56 ` Scott Wood
2008-08-18 19:28 ` Steven Rostedt
2008-08-18 18:25 ` Eran Liberty
2008-08-18 18:41 ` Mathieu Desnoyers
2008-08-19 1:54 ` Benjamin Herrenschmidt
2008-08-19 9:56 ` Eran Liberty
2008-08-19 13:02 ` Mathieu Desnoyers
2008-08-19 21:46 ` Benjamin Herrenschmidt
2008-08-18 18:50 ` Steven Rostedt
2008-08-19 12:09 ` Eran Liberty
2008-08-19 13:05 ` Mathieu Desnoyers
2008-08-19 14:21 ` Eran Liberty
2008-08-19 14:42 ` Mathieu Desnoyers
2008-08-19 20:15 ` Steven Rostedt
2008-08-20 11:18 ` Eran Liberty
2008-08-20 13:12 ` Steven Rostedt
2008-08-19 1:51 ` Benjamin Herrenschmidt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=48AC23F4.80900@extricom.com \
--to=liberty@extricom.com \
--cc=amodra@au1.ibm.com \
--cc=benh@kernel.crashing.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@ozlabs.org \
--cc=mathieu.desnoyers@polymtl.ca \
--cc=paulmck@linux.vnet.ibm.com \
--cc=rostedt@goodmis.org \
--cc=scottwood@freescale.com \
--cc=segher@kernel.crashing.org \
--cc=srostedt@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).