From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756702AbYHTNis (ORCPT ); Wed, 20 Aug 2008 09:38:48 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751593AbYHTNij (ORCPT ); Wed, 20 Aug 2008 09:38:39 -0400 Received: from smtp1.extricom.com ([212.235.24.249]:46488 "HELO smtp.extricom.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with SMTP id S1750935AbYHTNij (ORCPT ); Wed, 20 Aug 2008 09:38:39 -0400 Message-ID: <48AC1DD8.9080702@extricom.com> Date: Wed, 20 Aug 2008 16:36:24 +0300 From: Eran Liberty User-Agent: Thunderbird 2.0.0.14 (X11/20080502) MIME-Version: 1.0 To: Steven Rostedt CC: Benjamin Herrenschmidt , "Paul E. McKenney" , Mathieu Desnoyers , linux-kernel@vger.kernel.org, linuxppc-dev@ozlabs.org, Steven Rostedt , Scott Wood , Alan Modra , Segher Boessenkool Subject: Re: ftrace introduces instability into kernel 2.6.27(-rc2,-rc3) References: <48591941.4070408@extricom.com> <48A92E15.2080709@extricom.com> <48A9901B.1080900@redhat.com> <20080818154746.GA26835@Krystal> <48A9AFA7.8080508@freescale.com> <1219110814.8062.2.camel@pasglop> <1219113549.8062.13.camel@pasglop> <1219114600.8062.15.camel@pasglop> <1219119431.8062.35.camel@pasglop> <1219216705.21386.46.camel@pasglop> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Steven Rostedt wrote: > On Wed, 20 Aug 2008, Steven Rostedt wrote: > > >> On Wed, 20 Aug 2008, Benjamin Herrenschmidt wrote: >> >> >>> Found the problem (or at least -a- problem), it's a gcc bug. >>> >>> Well, first I must say the code generated by -pg is just plain >>> horrible :-) >>> >>> Appart from that, look at the exit of, for example, __d_lookup, as >>> generated by gcc when ftrace is enabled: >>> >>> c00c0498: 38 60 00 00 li r3,0 >>> c00c049c: 81 61 00 00 lwz r11,0(r1) >>> c00c04a0: 80 0b 00 04 lwz r0,4(r11) >>> c00c04a4: 7d 61 5b 78 mr r1,r11 >>> c00c04a8: bb 0b ff e0 lmw r24,-32(r11) >>> c00c04ac: 7c 08 03 a6 mtlr r0 >>> c00c04b0: 4e 80 00 20 blr >>> >>> As you can see, it restores r1 -before- it pops r24..r31 off >>> the stack ! I let you imagine what happens if an interrupt happens >>> just in between those two instructions (mr and lmw). We don't do >>> redzones on our ABI, so basically, the registers end up corrupted >>> by the interrupt. >>> >> Ouch! You've disassembled this without -pg too, and it does not have this >> bug? What version of gcc do you have? >> >> > > I have: > gcc (Debian 4.3.1-2) 4.3.1 > > c00c64c8: 81 61 00 00 lwz r11,0(r1) > c00c64cc: 7f 83 e3 78 mr r3,r28 > c00c64d0: 80 0b 00 04 lwz r0,4(r11) > c00c64d4: ba eb ff dc lmw r23,-36(r11) > c00c64d8: 7d 61 5b 78 mr r1,r11 > c00c64dc: 7c 08 03 a6 mtlr r0 > c00c64e0: 4e 80 00 20 blr > > > My version looks fine. I'm thinking that this is a separate issue than > what Eran is seeing. > > Eran, can you do an "objdump -dr vmlinux" and search for __d_lookup, and > print out the end of the function dump. > > Thanks, > > -- Steve > > > > powerpc-linux-gnu-objdump -dr --start-address=0xc00bb584 vmlinux | head -n 100 vmlinux: file format elf32-powerpc Disassembly of section .text: c00bb584 <__d_lookup>: c00bb584: 7c 08 02 a6 mflr r0 c00bb588: 90 01 00 04 stw r0,4(r1) c00bb58c: 4b f5 5c 51 bl c00111dc <_mcount> c00bb590: 94 21 ff d0 stwu r1,-48(r1) c00bb594: 7c 08 02 a6 mflr r0 c00bb598: 3d 20 9e 37 lis r9,-25033 c00bb59c: bf 01 00 10 stmw r24,16(r1) c00bb5a0: 61 29 00 01 ori r9,r9,1 c00bb5a4: 3d 60 c0 38 lis r11,-16328 c00bb5a8: 90 01 00 34 stw r0,52(r1) c00bb5ac: 7c 60 4a 78 xor r0,r3,r9 c00bb5b0: 54 00 d9 7e rlwinm r0,r0,27,5,31 c00bb5b4: 83 84 00 00 lwz r28,0(r4) c00bb5b8: 7c 3f 0b 78 mr r31,r1 c00bb5bc: 81 0b 1a 2c lwz r8,6700(r11) c00bb5c0: 39 6b 1a 2c addi r11,r11,6700 c00bb5c4: 7c 00 e2 14 add r0,r0,r28 c00bb5c8: 81 4b 00 04 lwz r10,4(r11) c00bb5cc: 7c 09 4a 78 xor r9,r0,r9 c00bb5d0: 83 24 00 04 lwz r25,4(r4) c00bb5d4: 7d 29 44 30 srw r9,r9,r8 c00bb5d8: 81 0b 00 08 lwz r8,8(r11) c00bb5dc: 7d 29 02 78 xor r9,r9,r0 c00bb5e0: 83 04 00 08 lwz r24,8(r4) c00bb5e4: 7d 29 50 38 and r9,r9,r10 c00bb5e8: 55 29 10 3a rlwinm r9,r9,2,0,29 c00bb5ec: 7c 09 40 2e lwzx r0,r9,r8 c00bb5f0: 7c 9a 23 78 mr r26,r4 c00bb5f4: 7c 7b 1b 78 mr r27,r3 c00bb5f8: 2f 80 00 00 cmpwi cr7,r0,0 c00bb5fc: 7c 1e 03 78 mr r30,r0 c00bb600: 40 be 00 14 bne+ cr7,c00bb614 <__d_lookup+0x90> c00bb604: 48 00 00 7c b c00bb680 <__d_lookup+0xfc> c00bb608: 83 de 00 00 lwz r30,0(r30) c00bb60c: 2f 9e 00 00 cmpwi cr7,r30,0 c00bb610: 41 9e 00 70 beq- cr7,c00bb680 <__d_lookup+0xfc> c00bb614: 80 1e 00 00 lwz r0,0(r30) c00bb618: 2f 80 00 00 cmpwi cr7,r0,0 c00bb61c: 41 9e 00 08 beq- cr7,c00bb624 <__d_lookup+0xa0> c00bb620: 7c 00 02 2c dcbt r0,r0 c00bb624: 3b be ff f4 addi r29,r30,-12 c00bb628: 80 1d 00 18 lwz r0,24(r29) c00bb62c: 7f 80 e0 00 cmpw cr7,r0,r28 c00bb630: 40 9e ff d8 bne+ cr7,c00bb608 <__d_lookup+0x84> c00bb634: 80 1d 00 14 lwz r0,20(r29) c00bb638: 7f 80 d8 00 cmpw cr7,r0,r27 c00bb63c: 40 9e ff cc bne+ cr7,c00bb608 <__d_lookup+0x84> c00bb640: 81 3b 00 48 lwz r9,72(r27) c00bb644: 38 9d 00 18 addi r4,r29,24 c00bb648: 2f 89 00 00 cmpwi cr7,r9,0 c00bb64c: 41 9e 00 50 beq- cr7,c00bb69c <__d_lookup+0x118> c00bb650: 80 09 00 08 lwz r0,8(r9) c00bb654: 2f 80 00 00 cmpwi cr7,r0,0 c00bb658: 41 9e 00 44 beq- cr7,c00bb69c <__d_lookup+0x118> c00bb65c: 7f 63 db 78 mr r3,r27 c00bb660: 7c 09 03 a6 mtctr r0 c00bb664: 7f 45 d3 78 mr r5,r26 c00bb668: 4e 80 04 21 bctrl c00bb66c: 2f 83 00 00 cmpwi cr7,r3,0 c00bb670: 41 9e 00 50 beq- cr7,c00bb6c0 <__d_lookup+0x13c> c00bb674: 83 de 00 00 lwz r30,0(r30) c00bb678: 2f 9e 00 00 cmpwi cr7,r30,0 c00bb67c: 40 9e ff 98 bne+ cr7,c00bb614 <__d_lookup+0x90> c00bb680: 38 60 00 00 li r3,0 c00bb684: 81 61 00 00 lwz r11,0(r1) c00bb688: 80 0b 00 04 lwz r0,4(r11) c00bb68c: 7d 61 5b 78 mr r1,r11 c00bb690: bb 0b ff e0 lmw r24,-32(r11) c00bb694: 7c 08 03 a6 mtlr r0 c00bb698: 4e 80 00 20 blr c00bb69c: 80 04 00 04 lwz r0,4(r4) c00bb6a0: 7f 80 c8 00 cmpw cr7,r0,r25 c00bb6a4: 40 9e ff 64 bne+ cr7,c00bb608 <__d_lookup+0x84> c00bb6a8: 80 64 00 08 lwz r3,8(r4) c00bb6ac: 7f 25 cb 78 mr r5,r25 c00bb6b0: 7f 04 c3 78 mr r4,r24 c00bb6b4: 4b f5 ab 65 bl c0016218 c00bb6b8: 2f 83 00 00 cmpwi cr7,r3,0 c00bb6bc: 40 9e ff 4c bne+ cr7,c00bb608 <__d_lookup+0x84> c00bb6c0: 80 1d 00 04 lwz r0,4(r29) c00bb6c4: 70 09 00 10 andi. r9,r0,16 c00bb6c8: 40 a2 ff b8 bne- c00bb680 <__d_lookup+0xfc> c00bb6cc: 7c 00 e8 28 lwarx r0,0,r29 c00bb6d0: 30 00 00 01 addic r0,r0,1 c00bb6d4: 7c 00 e9 2d stwcx. r0,0,r29 c00bb6d8: 40 a2 ff f4 bne- c00bb6cc <__d_lookup+0x148> c00bb6dc: 7f a3 eb 78 mr r3,r29 c00bb6e0: 4b ff ff a4 b c00bb684 <__d_lookup+0x100> c00bb6e4 : c00bb6e4: 7c 08 02 a6 mflr r0 c00bb6e8: 90 01 00 04 stw r0,4(r1)