All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-15  4:04 Anton Blanchard
  2013-07-15  4:39 ` Rusty Russell
  2013-07-15  8:47   ` Benjamin Herrenschmidt
  0 siblings, 2 replies; 28+ messages in thread
From: Anton Blanchard @ 2013-07-15  4:04 UTC (permalink / raw)
  To: Benjamin Herrenschmidt, Paul Mackerras, Rusty Russell, Neil Horman
  Cc: linuxppc-dev, linux-kernel


Module CRCs are implemented as absolute symbols that get resolved by
a linker script. We build an intermediate .o that contains an
unresolved symbol for each CRC. genksysms parses this .o, calculates
the CRCs and writes a linker script that "resolves" the symbols to
the calculated CRC.

Unfortunately the ppc64 relocatable kernel sees these CRCs as symbols
that need relocating and relocates them at boot. Commit d4703aef
(module: handle ppc64 relocating kcrctabs when CONFIG_RELOCATABLE=y)
added a hook to reverse the bogus relocations. Part of this patch
created a symbol at 0x0:

# head -2 /proc/kallsyms 
0000000000000000 T reloc_start
c000000000000000 T .__start

This reloc_start symbol is causing lots of confusion to perf. It
thinks reloc_start is a massive function that stretches from 0x0 to
0xc000000000000000 and we get various cryptic errors out of perf,
including:

problem incrementing symbol count, skipping event

This patch removes the  reloc_start linker script label and instead
defines it as PHYSICAL_START. We also need to wrap it with
CONFIG_PPC64 because the ppc32 kernel can set a non zero
PHYSICAL_START at compile time and we wouldn't want to subtract
it from the CRCs in that case.

Signed-off-by: Anton Blanchard <anton@samba.org>
Cc: <stable@kernel.org>
---

This bug was originally reported on Fedora 19 (3.9.x), so I've marked
it for stable.

Index: b/arch/powerpc/include/asm/module.h
===================================================================
--- a/arch/powerpc/include/asm/module.h
+++ b/arch/powerpc/include/asm/module.h
@@ -82,10 +82,9 @@ struct exception_table_entry;
 void sort_ex_table(struct exception_table_entry *start,
 		   struct exception_table_entry *finish);
 
-#ifdef CONFIG_MODVERSIONS
+#if defined(CONFIG_MODVERSIONS) && defined(CONFIG_PPC64)
 #define ARCH_RELOCATES_KCRCTAB
-
-extern const unsigned long reloc_start[];
+#define reloc_start PHYSICAL_START
 #endif
 #endif /* __KERNEL__ */
 #endif	/* _ASM_POWERPC_MODULE_H */
Index: b/arch/powerpc/kernel/vmlinux.lds.S
===================================================================
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -38,9 +38,6 @@ jiffies = jiffies_64 + 4;
 #endif
 SECTIONS
 {
-	. = 0;
-	reloc_start = .;
-
 	. = KERNELBASE;
 
 /*

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-15  4:04 [PATCH] module: ppc64 module CRC relocation fix causes perf issues Anton Blanchard
@ 2013-07-15  4:39 ` Rusty Russell
  2013-07-15  8:47   ` Benjamin Herrenschmidt
  1 sibling, 0 replies; 28+ messages in thread
From: Rusty Russell @ 2013-07-15  4:39 UTC (permalink / raw)
  To: Anton Blanchard, Benjamin Herrenschmidt, Paul Mackerras, Neil Horman
  Cc: linuxppc-dev, linux-kernel

Anton Blanchard <anton@samba.org> writes:
> Module CRCs are implemented as absolute symbols that get resolved by
> a linker script. We build an intermediate .o that contains an
> unresolved symbol for each CRC. genksysms parses this .o, calculates
> the CRCs and writes a linker script that "resolves" the symbols to
> the calculated CRC.
>
> Unfortunately the ppc64 relocatable kernel sees these CRCs as symbols
> that need relocating and relocates them at boot. Commit d4703aef
> (module: handle ppc64 relocating kcrctabs when CONFIG_RELOCATABLE=y)
> added a hook to reverse the bogus relocations. Part of this patch
> created a symbol at 0x0:
>
> # head -2 /proc/kallsyms 
> 0000000000000000 T reloc_start
> c000000000000000 T .__start
>
> This reloc_start symbol is causing lots of confusion to perf. It
> thinks reloc_start is a massive function that stretches from 0x0 to
> 0xc000000000000000 and we get various cryptic errors out of perf,
> including:
>
> problem incrementing symbol count, skipping event
>
> This patch removes the  reloc_start linker script label and instead
> defines it as PHYSICAL_START. We also need to wrap it with
> CONFIG_PPC64 because the ppc32 kernel can set a non zero
> PHYSICAL_START at compile time and we wouldn't want to subtract
> it from the CRCs in that case.
>
> Signed-off-by: Anton Blanchard <anton@samba.org>
> Cc: <stable@kernel.org>

Acked-by: Rusty Russell <rusty@rustcorp.com.au>

Ben?

Cheers,
Rusty.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-15  4:04 [PATCH] module: ppc64 module CRC relocation fix causes perf issues Anton Blanchard
@ 2013-07-15  8:47   ` Benjamin Herrenschmidt
  2013-07-15  8:47   ` Benjamin Herrenschmidt
  1 sibling, 0 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2013-07-15  8:47 UTC (permalink / raw)
  To: Scott Wood
  Cc: Paul Mackerras, Rusty Russell, Neil Horman, linuxppc-dev,
	linux-kernel, Anton Blanchard

On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> Module CRCs are implemented as absolute symbols that get resolved by
> a linker script. We build an intermediate .o that contains an
> unresolved symbol for each CRC. genksysms parses this .o, calculates
> the CRCs and writes a linker script that "resolves" the symbols to
> the calc

Scott, can somebody from FSL test that on 32-bit and Ack it ?

Thanks !

Cheers,
Ben.

> Unfortunately the ppc64 relocatable kernel sees these CRCs as symbols
> that need relocating and relocates them at boot. Commit d4703aef
> (module: handle ppc64 relocating kcrctabs when CONFIG_RELOCATABLE=y)
> added a hook to reverse the bogus relocations. Part of this patch
> created a symbol at 0x0:
> 
> # head -2 /proc/kallsyms 
> 0000000000000000 T reloc_start
> c000000000000000 T .__start
> 
> This reloc_start symbol is causing lots of confusion to perf. It
> thinks reloc_start is a massive function that stretches from 0x0 to
> 0xc000000000000000 and we get various cryptic errors out of perf,
> including:
> 
> problem incrementing symbol count, skipping event
> 
> This patch removes the  reloc_start linker script label and instead
> defines it as PHYSICAL_START. We also need to wrap it with
> CONFIG_PPC64 because the ppc32 kernel can set a non zero
> PHYSICAL_START at compile time and we wouldn't want to subtract
> it from the CRCs in that case.
> 
> Signed-off-by: Anton Blanchard <anton@samba.org>
> Cc: <stable@kernel.org>
> ---
> 
> This bug was originally reported on Fedora 19 (3.9.x), so I've marked
> it for stable.
> 
> Index: b/arch/powerpc/include/asm/module.h
> ===================================================================
> --- a/arch/powerpc/include/asm/module.h
> +++ b/arch/powerpc/include/asm/module.h
> @@ -82,10 +82,9 @@ struct exception_table_entry;
>  void sort_ex_table(struct exception_table_entry *start,
>  		   struct exception_table_entry *finish);
>  
> -#ifdef CONFIG_MODVERSIONS
> +#if defined(CONFIG_MODVERSIONS) && defined(CONFIG_PPC64)
>  #define ARCH_RELOCATES_KCRCTAB
> -
> -extern const unsigned long reloc_start[];
> +#define reloc_start PHYSICAL_START
>  #endif
>  #endif /* __KERNEL__ */
>  #endif	/* _ASM_POWERPC_MODULE_H */
> Index: b/arch/powerpc/kernel/vmlinux.lds.S
> ===================================================================
> --- a/arch/powerpc/kernel/vmlinux.lds.S
> +++ b/arch/powerpc/kernel/vmlinux.lds.S
> @@ -38,9 +38,6 @@ jiffies = jiffies_64 + 4;
>  #endif
>  SECTIONS
>  {
> -	. = 0;
> -	reloc_start = .;
> -
>  	. = KERNELBASE;
>  
>  /*



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-15  8:47   ` Benjamin Herrenschmidt
  0 siblings, 0 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2013-07-15  8:47 UTC (permalink / raw)
  To: Scott Wood
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras,
	Anton Blanchard, linuxppc-dev

On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> Module CRCs are implemented as absolute symbols that get resolved by
> a linker script. We build an intermediate .o that contains an
> unresolved symbol for each CRC. genksysms parses this .o, calculates
> the CRCs and writes a linker script that "resolves" the symbols to
> the calc

Scott, can somebody from FSL test that on 32-bit and Ack it ?

Thanks !

Cheers,
Ben.

> Unfortunately the ppc64 relocatable kernel sees these CRCs as symbols
> that need relocating and relocates them at boot. Commit d4703aef
> (module: handle ppc64 relocating kcrctabs when CONFIG_RELOCATABLE=y)
> added a hook to reverse the bogus relocations. Part of this patch
> created a symbol at 0x0:
> 
> # head -2 /proc/kallsyms 
> 0000000000000000 T reloc_start
> c000000000000000 T .__start
> 
> This reloc_start symbol is causing lots of confusion to perf. It
> thinks reloc_start is a massive function that stretches from 0x0 to
> 0xc000000000000000 and we get various cryptic errors out of perf,
> including:
> 
> problem incrementing symbol count, skipping event
> 
> This patch removes the  reloc_start linker script label and instead
> defines it as PHYSICAL_START. We also need to wrap it with
> CONFIG_PPC64 because the ppc32 kernel can set a non zero
> PHYSICAL_START at compile time and we wouldn't want to subtract
> it from the CRCs in that case.
> 
> Signed-off-by: Anton Blanchard <anton@samba.org>
> Cc: <stable@kernel.org>
> ---
> 
> This bug was originally reported on Fedora 19 (3.9.x), so I've marked
> it for stable.
> 
> Index: b/arch/powerpc/include/asm/module.h
> ===================================================================
> --- a/arch/powerpc/include/asm/module.h
> +++ b/arch/powerpc/include/asm/module.h
> @@ -82,10 +82,9 @@ struct exception_table_entry;
>  void sort_ex_table(struct exception_table_entry *start,
>  		   struct exception_table_entry *finish);
>  
> -#ifdef CONFIG_MODVERSIONS
> +#if defined(CONFIG_MODVERSIONS) && defined(CONFIG_PPC64)
>  #define ARCH_RELOCATES_KCRCTAB
> -
> -extern const unsigned long reloc_start[];
> +#define reloc_start PHYSICAL_START
>  #endif
>  #endif /* __KERNEL__ */
>  #endif	/* _ASM_POWERPC_MODULE_H */
> Index: b/arch/powerpc/kernel/vmlinux.lds.S
> ===================================================================
> --- a/arch/powerpc/kernel/vmlinux.lds.S
> +++ b/arch/powerpc/kernel/vmlinux.lds.S
> @@ -38,9 +38,6 @@ jiffies = jiffies_64 + 4;
>  #endif
>  SECTIONS
>  {
> -	. = 0;
> -	reloc_start = .;
> -
>  	. = KERNELBASE;
>  
>  /*

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-15  8:47   ` Benjamin Herrenschmidt
@ 2013-07-16 22:40     ` Scott Wood
  -1 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-16 22:40 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: Paul Mackerras, Rusty Russell, Neil Horman, linuxppc-dev,
	linux-kernel, Anton Blanchard

On 07/15/2013 03:47:06 AM, Benjamin Herrenschmidt wrote:
> On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> > Module CRCs are implemented as absolute symbols that get resolved by
> > a linker script. We build an intermediate .o that contains an
> > unresolved symbol for each CRC. genksysms parses this .o, calculates
> > the CRCs and writes a linker script that "resolves" the symbols to
> > the calc
> 
> Scott, can somebody from FSL test that on 32-bit and Ack it ?
> 
> Thanks !
> 
> Cheers,
> Ben.

It boots for me on e500mc and I can insert modules.

-Scott

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-16 22:40     ` Scott Wood
  0 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-16 22:40 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras,
	Anton Blanchard, linuxppc-dev

On 07/15/2013 03:47:06 AM, Benjamin Herrenschmidt wrote:
> On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> > Module CRCs are implemented as absolute symbols that get resolved by
> > a linker script. We build an intermediate .o that contains an
> > unresolved symbol for each CRC. genksysms parses this .o, calculates
> > the CRCs and writes a linker script that "resolves" the symbols to
> > the calc
>=20
> Scott, can somebody from FSL test that on 32-bit and Ack it ?
>=20
> Thanks !
>=20
> Cheers,
> Ben.

It boots for me on e500mc and I can insert modules.

-Scott=

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-16 22:40     ` Scott Wood
@ 2013-07-17  0:04       ` Benjamin Herrenschmidt
  -1 siblings, 0 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2013-07-17  0:04 UTC (permalink / raw)
  To: Scott Wood
  Cc: Paul Mackerras, Rusty Russell, Neil Horman, linuxppc-dev,
	linux-kernel, Anton Blanchard

On Tue, 2013-07-16 at 17:40 -0500, Scott Wood wrote:
> On 07/15/2013 03:47:06 AM, Benjamin Herrenschmidt wrote:
> > On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> > > Module CRCs are implemented as absolute symbols that get resolved by
> > > a linker script. We build an intermediate .o that contains an
> > > unresolved symbol for each CRC. genksysms parses this .o, calculates
> > > the CRCs and writes a linker script that "resolves" the symbols to
> > > the calc
> > 
> > Scott, can somebody from FSL test that on 32-bit and Ack it ?
> > 
> > Thanks !
> > 
> > Cheers,
> > Ben.
> 
> It boots for me on e500mc and I can insert modules.

But does perf work ? :-)

Cheers,
Ben.



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-17  0:04       ` Benjamin Herrenschmidt
  0 siblings, 0 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2013-07-17  0:04 UTC (permalink / raw)
  To: Scott Wood
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras,
	Anton Blanchard, linuxppc-dev

On Tue, 2013-07-16 at 17:40 -0500, Scott Wood wrote:
> On 07/15/2013 03:47:06 AM, Benjamin Herrenschmidt wrote:
> > On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> > > Module CRCs are implemented as absolute symbols that get resolved by
> > > a linker script. We build an intermediate .o that contains an
> > > unresolved symbol for each CRC. genksysms parses this .o, calculates
> > > the CRCs and writes a linker script that "resolves" the symbols to
> > > the calc
> > 
> > Scott, can somebody from FSL test that on 32-bit and Ack it ?
> > 
> > Thanks !
> > 
> > Cheers,
> > Ben.
> 
> It boots for me on e500mc and I can insert modules.

But does perf work ? :-)

Cheers,
Ben.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-17  0:04       ` Benjamin Herrenschmidt
@ 2013-07-17  0:08         ` Scott Wood
  -1 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-17  0:08 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: Paul Mackerras, Rusty Russell, Neil Horman, linuxppc-dev,
	linux-kernel, Anton Blanchard

On 07/16/2013 07:04:05 PM, Benjamin Herrenschmidt wrote:
> On Tue, 2013-07-16 at 17:40 -0500, Scott Wood wrote:
> > On 07/15/2013 03:47:06 AM, Benjamin Herrenschmidt wrote:
> > > On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> > > > Module CRCs are implemented as absolute symbols that get  
> resolved by
> > > > a linker script. We build an intermediate .o that contains an
> > > > unresolved symbol for each CRC. genksysms parses this .o,  
> calculates
> > > > the CRCs and writes a linker script that "resolves" the symbols  
> to
> > > > the calc
> > >
> > > Scott, can somebody from FSL test that on 32-bit and Ack it ?
> > >
> > > Thanks !
> > >
> > > Cheers,
> > > Ben.
> >
> > It boots for me on e500mc and I can insert modules.
> 
> But does perf work ? :-)

What specifically should I do to test it?

-Scott

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-17  0:08         ` Scott Wood
  0 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-17  0:08 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras,
	Anton Blanchard, linuxppc-dev

On 07/16/2013 07:04:05 PM, Benjamin Herrenschmidt wrote:
> On Tue, 2013-07-16 at 17:40 -0500, Scott Wood wrote:
> > On 07/15/2013 03:47:06 AM, Benjamin Herrenschmidt wrote:
> > > On Mon, 2013-07-15 at 14:04 +1000, Anton Blanchard wrote:
> > > > Module CRCs are implemented as absolute symbols that get =20
> resolved by
> > > > a linker script. We build an intermediate .o that contains an
> > > > unresolved symbol for each CRC. genksysms parses this .o, =20
> calculates
> > > > the CRCs and writes a linker script that "resolves" the symbols =20
> to
> > > > the calc
> > >
> > > Scott, can somebody from FSL test that on 32-bit and Ack it ?
> > >
> > > Thanks !
> > >
> > > Cheers,
> > > Ben.
> >
> > It boots for me on e500mc and I can insert modules.
>=20
> But does perf work ? :-)

What specifically should I do to test it?

-Scott=

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-17  0:08         ` Scott Wood
@ 2013-07-18  4:00           ` Anton Blanchard
  -1 siblings, 0 replies; 28+ messages in thread
From: Anton Blanchard @ 2013-07-18  4:00 UTC (permalink / raw)
  To: Scott Wood
  Cc: Benjamin Herrenschmidt, Paul Mackerras, Rusty Russell,
	Neil Horman, linuxppc-dev, linux-kernel


Hi Scott,

> What specifically should I do to test it?

Could you double check perf annotate works? I'm 99% sure it will but
that is what was failing on ppc64.

Anton

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-18  4:00           ` Anton Blanchard
  0 siblings, 0 replies; 28+ messages in thread
From: Anton Blanchard @ 2013-07-18  4:00 UTC (permalink / raw)
  To: Scott Wood
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras, linuxppc-dev


Hi Scott,

> What specifically should I do to test it?

Could you double check perf annotate works? I'm 99% sure it will but
that is what was failing on ppc64.

Anton

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-18  4:00           ` Anton Blanchard
@ 2013-07-19 22:59             ` Scott Wood
  -1 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-19 22:59 UTC (permalink / raw)
  To: Anton Blanchard
  Cc: Benjamin Herrenschmidt, Paul Mackerras, Rusty Russell,
	Neil Horman, linuxppc-dev, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

On 07/17/2013 11:00:45 PM, Anton Blanchard wrote:
> 
> Hi Scott,
> 
> > What specifically should I do to test it?
> 
> Could you double check perf annotate works? I'm 99% sure it will but
> that is what was failing on ppc64.

I'm not really sure what it's supposed to look like when "perf  
annotate" works.  It spits a bunch of unreadable[1] dark-blue-on-black  
assembly code at me, all with "0.00 :" in the left column.

Oh, wait -- some lines have "100.00 : " on the left, in  
even-more-unreadable dark-red-on-black.

Apart from the annoying colors, is there anything specific I should be  
looking for?  Some sort of error message, or output that actually makes  
sense?

I've attached the output from "perf annotate" and "perf report".   
perf.data was generated by "perf record find /usr > /dev/null" on an  
NFS root (which took a few seconds to complete), so the large amount of  
__alloc_skb makes some sense, but the way perf annotate shows 100% on  
one instruction in each function seems odd.

-Scott

[1] ...unless I crank the brightness up on my monitor to the point  
where whites are blinding, or redirect the output to a file so the  
colors go away.

[-- Attachment #2: annotate --]
[-- Type: text/plain, Size: 37045 bytes --]

 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0097510 <perf_event_comm>:
    0.00 :	c0097510:       94 21 ff a0     stwu    r1,-96(r1)
    0.00 :	c0097514:       7c 08 02 a6     mflr    r0
    0.00 :	c0097518:       bf 01 00 40     stmw    r24,64(r1)
    0.00 :	c009751c:       7c 7e 1b 78     mr      r30,r3
    0.00 :	c0097520:       90 01 00 64     stw     r0,100(r1)
    0.00 :	c0097524:       3b 80 00 00     li      r28,0
    0.00 :	c0097528:       3b 63 04 68     addi    r27,r3,1128
    0.00 :	c009752c:       3b 40 00 00     li      r26,0
    0.00 :	c0097530:       87 bb 00 04     lwzu    r29,4(r27)
    0.00 :	c0097534:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c0097538:       41 9e 00 1c     beq-    cr7,c0097554 <perf_event_comm+0x44>
    0.00 :	c009753c:       7f 00 00 a6     mfmsr   r24
    0.00 :	c0097540:       7c 00 01 46     .long 0x7c000146
    0.00 :	c0097544:       80 1d 00 3c     lwz     r0,60(r29)
    0.00 :	c0097548:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c009754c:       40 9e 00 c4     bne-    cr7,c0097610 <perf_event_comm+0x100>
    0.00 :	c0097550:       7f 00 01 06     .long 0x7f000106
  100.00 :	c0097554:       2f 9c 00 01     cmpwi   cr7,r28,1
    0.00 :	c0097558:       3b 9c 00 01     addi    r28,r28,1
    0.00 :	c009755c:       40 be ff d4     bne-    cr7,c0097530 <perf_event_comm+0x20>
    0.00 :	c0097560:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0097564:       80 09 99 50     lwz     r0,-26288(r9)
    0.00 :	c0097568:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c009756c:       41 be 00 88     beq+    cr7,c00975f4 <perf_event_comm+0xe4>
    0.00 :	c0097570:       3b e1 00 08     addi    r31,r1,8
    0.00 :	c0097574:       38 00 00 00     li      r0,0
    0.00 :	c0097578:       39 20 00 03     li      r9,3
    0.00 :	c009757c:       38 9e 01 d8     addi    r4,r30,472
    0.00 :	c0097580:       38 a0 00 10     li      r5,16
    0.00 :	c0097584:       7f e3 fb 78     mr      r3,r31
    0.00 :	c0097588:       90 01 00 1c     stw     r0,28(r1)
    0.00 :	c009758c:       90 01 00 20     stw     r0,32(r1)
    0.00 :	c0097590:       90 01 00 28     stw     r0,40(r1)
    0.00 :	c0097594:       90 01 00 2c     stw     r0,44(r1)
    0.00 :	c0097598:       90 01 00 30     stw     r0,48(r1)
    0.00 :	c009759c:       91 21 00 24     stw     r9,36(r1)
    0.00 :	c00975a0:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c00975a4:       90 01 00 0c     stw     r0,12(r1)
    0.00 :	c00975a8:       90 01 00 10     stw     r0,16(r1)
    0.00 :	c00975ac:       90 01 00 14     stw     r0,20(r1)
    0.00 :	c00975b0:       93 c1 00 18     stw     r30,24(r1)
    0.00 :	c00975b4:       48 1a 05 8d     bl      c0237b40 <strlcpy>
    0.00 :	c00975b8:       7f e3 fb 78     mr      r3,r31
    0.00 :	c00975bc:       4b f8 0e 89     bl      c0018444 <strlen>
    0.00 :	c00975c0:       3c 80 c0 09     lis     r4,-16375
    0.00 :	c00975c4:       39 23 00 08     addi    r9,r3,8
    0.00 :	c00975c8:       93 e1 00 1c     stw     r31,28(r1)
    0.00 :	c00975cc:       55 29 00 38     rlwinm  r9,r9,0,0,28
    0.00 :	c00975d0:       3c 60 c0 09     lis     r3,-16375
    0.00 :	c00975d4:       38 09 00 10     addi    r0,r9,16
    0.00 :	c00975d8:       91 21 00 20     stw     r9,32(r1)
    0.00 :	c00975dc:       38 63 fd 80     addi    r3,r3,-640
    0.00 :	c00975e0:       38 84 65 50     addi    r4,r4,25936
    0.00 :	c00975e4:       38 a1 00 18     addi    r5,r1,24
    0.00 :	c00975e8:       b0 01 00 2a     sth     r0,42(r1)
    0.00 :	c00975ec:       38 c0 00 00     li      r6,0
    0.00 :	c00975f0:       4b ff 86 61     bl      c008fc50 <perf_event_aux>
    0.00 :	c00975f4:       80 01 00 64     lwz     r0,100(r1)
    0.00 :	c00975f8:       bb 01 00 40     lmw     r24,64(r1)
    0.00 :	c00975fc:       38 21 00 60     addi    r1,r1,96
    0.00 :	c0097600:       7c 08 03 a6     mtlr    r0
    0.00 :	c0097604:       4e 80 00 20     blr
    0.00 :	c0097608:       60 00 00 00     nop
    0.00 :	c009760c:       60 00 00 00     nop
    0.00 :	c0097610:       38 7d 00 08     addi    r3,r29,8
    0.00 :	c0097614:       7f b9 eb 78     mr      r25,r29
    0.00 :	c0097618:       48 48 98 a9     bl      c0520ec0 <_raw_spin_lock>
    0.00 :	c009761c:       7f a3 eb 78     mr      r3,r29
    0.00 :	c0097620:       4b ff ce 81     bl      c00944a0 <task_ctx_sched_out>
    0.00 :	c0097624:       87 f9 00 34     lwzu    r31,52(r25)
    0.00 :	c0097628:       7f 99 f8 00     cmpw    cr7,r25,r31
    0.00 :	c009762c:       3b ff ff f8     addi    r31,r31,-8
    0.00 :	c0097630:       41 9e 00 60     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c0097634:       39 60 00 00     li      r11,0
    0.00 :	c0097638:       60 00 00 00     nop
    0.00 :	c009763c:       60 00 00 00     nop
    0.00 :	c0097640:       80 1f 00 a0     lwz     r0,160(r31)
    0.00 :	c0097644:       7c 09 03 78     mr      r9,r0
    0.00 :	c0097648:       74 0a 00 08     andis.  r10,r0,8
    0.00 :	c009764c:       53 49 9b 18     rlwimi  r9,r26,19,12,12
    0.00 :	c0097650:       41 82 00 14     beq-    c0097664 <perf_event_comm+0x154>
    0.00 :	c0097654:       80 1f 00 30     lwz     r0,48(r31)
    0.00 :	c0097658:       91 3f 00 a0     stw     r9,160(r31)
    0.00 :	c009765c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0097660:       41 9c 00 50     blt-    cr7,c00976b0 <perf_event_comm+0x1a0>
    0.00 :	c0097664:       83 ff 00 08     lwz     r31,8(r31)
    0.00 :	c0097668:       7f 99 f8 00     cmpw    cr7,r25,r31
    0.00 :	c009766c:       3b ff ff f8     addi    r31,r31,-8
    0.00 :	c0097670:       40 9e ff d0     bne+    cr7,c0097640 <perf_event_comm+0x130>
    0.00 :	c0097674:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c0097678:       41 9e 00 18     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c009767c:       80 7d 00 70     lwz     r3,112(r29)
    0.00 :	c0097680:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0097684:       41 9e 00 0c     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c0097688:       4b ff a6 69     bl      c0091cf0 <put_ctx>
    0.00 :	c009768c:       93 5d 00 70     stw     r26,112(r29)
    0.00 :	c0097690:       7c 00 04 ac     sync    
    0.00 :	c0097694:       93 5d 00 08     stw     r26,8(r29)
    0.00 :	c0097698:       7f a3 eb 78     mr      r3,r29
    0.00 :	c009769c:       4b ff e1 75     bl      c0095810 <perf_event_context_sched_in.clone.38>
    0.00 :	c00976a0:       4b ff fe b0     b       c0097550 <perf_event_comm+0x40>
    0.00 :	c00976a4:       60 00 00 00     nop
    0.00 :	c00976a8:       60 00 00 00     nop
    0.00 :	c00976ac:       60 00 00 00     nop
    0.00 :	c00976b0:       7f e3 fb 78     mr      r3,r31
    0.00 :	c00976b4:       4b ff 83 cd     bl      c008fa80 <__perf_event_mark_enabled>
    0.00 :	c00976b8:       39 60 00 01     li      r11,1
    0.00 :	c00976bc:       4b ff ff a8     b       c0097664 <perf_event_comm+0x154>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c00368c0 <mod_timer>:
  100.00 :	c00368c0:       94 21 ff d0     stwu    r1,-48(r1)
    0.00 :	c00368c4:       7c 08 02 a6     mflr    r0
    0.00 :	c00368c8:       bf 61 00 1c     stmw    r27,28(r1)
    0.00 :	c00368cc:       7c 7f 1b 78     mr      r31,r3
    0.00 :	c00368d0:       90 01 00 34     stw     r0,52(r1)
    0.00 :	c00368d4:       7c 9e 23 78     mr      r30,r4
    0.00 :	c00368d8:       83 a3 00 18     lwz     r29,24(r3)
    0.00 :	c00368dc:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c00368e0:       41 9c 01 70     blt-    cr7,c0036a50 <mod_timer+0x190>
    0.00 :	c00368e4:       7f bd f2 14     add     r29,r29,r30
    0.00 :	c00368e8:       7f a0 f2 78     xor     r0,r29,r30
    0.00 :	c00368ec:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c00368f0:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c00368f4:       40 9e 01 8c     bne-    cr7,c0036a80 <mod_timer+0x1c0>
    0.00 :	c00368f8:       80 1f 00 00     lwz     r0,0(r31)
    0.00 :	c00368fc:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036900:       41 9e 01 ac     beq-    cr7,c0036aac <mod_timer+0x1ec>
    0.00 :	c0036904:       80 1f 00 08     lwz     r0,8(r31)
    0.00 :	c0036908:       3b 80 00 01     li      r28,1
    0.00 :	c003690c:       7f 80 f0 00     cmpw    cr7,r0,r30
    0.00 :	c0036910:       41 9e 00 f0     beq-    cr7,c0036a00 <mod_timer+0x140>
    0.00 :	c0036914:       80 1f 00 10     lwz     r0,16(r31)
    0.00 :	c0036918:       7c 00 00 34     cntlzw  r0,r0
    0.00 :	c003691c:       54 00 d9 7e     rlwinm  r0,r0,27,5,31
    0.00 :	c0036920:       0f 00 00 00     twnei   r0,0
    0.00 :	c0036924:       38 7f 00 0c     addi    r3,r31,12
    0.00 :	c0036928:       38 81 00 08     addi    r4,r1,8
    0.00 :	c003692c:       4b ff f9 75     bl      c00362a0 <lock_timer_base.clone.24>
    0.00 :	c0036930:       81 3f 00 00     lwz     r9,0(r31)
    0.00 :	c0036934:       7c 7b 1b 78     mr      r27,r3
    0.00 :	c0036938:       3b 80 00 00     li      r28,0
    0.00 :	c003693c:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0036940:       41 9e 00 2c     beq-    cr7,c003696c <mod_timer+0xac>
    0.00 :	c0036944:       81 7f 00 04     lwz     r11,4(r31)
    0.00 :	c0036948:       3c 00 00 20     lis     r0,32
    0.00 :	c003694c:       60 00 02 00     ori     r0,r0,512
    0.00 :	c0036950:       3b 80 00 01     li      r28,1
    0.00 :	c0036954:       91 69 00 04     stw     r11,4(r9)
    0.00 :	c0036958:       91 2b 00 00     stw     r9,0(r11)
    0.00 :	c003695c:       90 1f 00 04     stw     r0,4(r31)
    0.00 :	c0036960:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c0036964:       70 09 00 01     andi.   r9,r0,1
    0.00 :	c0036968:       41 82 00 b8     beq-    c0036a20 <mod_timer+0x160>
    0.00 :	c003696c:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0036970:       80 09 98 ec     lwz     r0,-26388(r9)
    0.00 :	c0036974:       54 29 00 24     rlwinm  r9,r1,0,0,18
    0.00 :	c0036978:       83 a9 00 08     lwz     r29,8(r9)
    0.00 :	c003697c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036980:       40 9e 01 40     bne-    cr7,c0036ac0 <mod_timer+0x200>
    0.00 :	c0036984:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0036988:       57 bd 10 3a     rlwinm  r29,r29,2,0,29
    0.00 :	c003698c:       39 29 9b 40     addi    r9,r9,-25792
    0.00 :	c0036990:       7d 69 e8 2e     lwzx    r11,r9,r29
    0.00 :	c0036994:       3d 20 c0 6d     lis     r9,-16275
    0.00 :	c0036998:       39 29 04 70     addi    r9,r9,1136
    0.00 :	c003699c:       7f ab 48 2e     lwzx    r29,r11,r9
    0.00 :	c00369a0:       7f 9b e8 00     cmpw    cr7,r27,r29
    0.00 :	c00369a4:       41 9e 00 40     beq-    cr7,c00369e4 <mod_timer+0x124>
    0.00 :	c00369a8:       80 1b 00 04     lwz     r0,4(r27)
    0.00 :	c00369ac:       7f 9f 00 00     cmpw    cr7,r31,r0
    0.00 :	c00369b0:       41 9e 01 2c     beq-    cr7,c0036adc <mod_timer+0x21c>
    0.00 :	c00369b4:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c00369b8:       54 00 07 be     clrlwi  r0,r0,30
    0.00 :	c00369bc:       90 1f 00 0c     stw     r0,12(r31)
    0.00 :	c00369c0:       7c 00 04 ac     sync    
    0.00 :	c00369c4:       38 00 00 00     li      r0,0
    0.00 :	c00369c8:       90 1b 00 00     stw     r0,0(r27)
    0.00 :	c00369cc:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369d0:       48 4e a4 f1     bl      c0520ec0 <_raw_spin_lock>
    0.00 :	c00369d4:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c00369d8:       54 00 07 be     clrlwi  r0,r0,30
    0.00 :	c00369dc:       7f a0 03 78     or      r0,r29,r0
    0.00 :	c00369e0:       90 1f 00 0c     stw     r0,12(r31)
    0.00 :	c00369e4:       93 df 00 08     stw     r30,8(r31)
    0.00 :	c00369e8:       7f e4 fb 78     mr      r4,r31
    0.00 :	c00369ec:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369f0:       4b ff f1 d1     bl      c0035bc0 <internal_add_timer>
    0.00 :	c00369f4:       80 81 00 08     lwz     r4,8(r1)
    0.00 :	c00369f8:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369fc:       48 4e 9e 65     bl      c0520860 <_raw_spin_unlock_irqrestore>
    0.00 :	c0036a00:       80 01 00 34     lwz     r0,52(r1)
    0.00 :	c0036a04:       7f 83 e3 78     mr      r3,r28
    0.00 :	c0036a08:       bb 61 00 1c     lmw     r27,28(r1)
    0.00 :	c0036a0c:       38 21 00 30     addi    r1,r1,48
    0.00 :	c0036a10:       7c 08 03 a6     mtlr    r0
    0.00 :	c0036a14:       4e 80 00 20     blr
    0.00 :	c0036a18:       60 00 00 00     nop
    0.00 :	c0036a1c:       60 00 00 00     nop
    0.00 :	c0036a20:       81 23 00 10     lwz     r9,16(r3)
    0.00 :	c0036a24:       80 03 00 0c     lwz     r0,12(r3)
    0.00 :	c0036a28:       39 29 ff ff     addi    r9,r9,-1
    0.00 :	c0036a2c:       91 23 00 10     stw     r9,16(r3)
    0.00 :	c0036a30:       81 3f 00 08     lwz     r9,8(r31)
    0.00 :	c0036a34:       7f 89 00 00     cmpw    cr7,r9,r0
    0.00 :	c0036a38:       40 9e ff 34     bne+    cr7,c003696c <mod_timer+0xac>
    0.00 :	c0036a3c:       80 03 00 08     lwz     r0,8(r3)
    0.00 :	c0036a40:       90 03 00 0c     stw     r0,12(r3)
    0.00 :	c0036a44:       4b ff ff 28     b       c003696c <mod_timer+0xac>
    0.00 :	c0036a48:       60 00 00 00     nop
    0.00 :	c0036a4c:       60 00 00 00     nop
    0.00 :	c0036a50:       3d 20 c0 71     lis     r9,-16271
    0.00 :	c0036a54:       83 a9 30 c4     lwz     r29,12484(r9)
    0.00 :	c0036a58:       7f bd 20 50     subf    r29,r29,r4
    0.00 :	c0036a5c:       2f 9d 00 ff     cmpwi   cr7,r29,255
    0.00 :	c0036a60:       40 bd fe 98     ble-    cr7,c00368f8 <mod_timer+0x38>
    0.00 :	c0036a64:       7f bd 46 70     srawi   r29,r29,8
    0.00 :	c0036a68:       7f bd f2 14     add     r29,r29,r30
    0.00 :	c0036a6c:       7f a0 f2 78     xor     r0,r29,r30
    0.00 :	c0036a70:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036a74:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c0036a78:       41 9e fe 80     beq+    cr7,c00368f8 <mod_timer+0x38>
    0.00 :	c0036a7c:       60 00 00 00     nop
    0.00 :	c0036a80:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0036a84:       38 80 00 20     li      r4,32
    0.00 :	c0036a88:       48 20 94 39     bl      c023fec0 <find_last_bit>
    0.00 :	c0036a8c:       80 1f 00 00     lwz     r0,0(r31)
    0.00 :	c0036a90:       3b c0 00 01     li      r30,1
    0.00 :	c0036a94:       7f de 18 30     slw     r30,r30,r3
    0.00 :	c0036a98:       3b de ff ff     addi    r30,r30,-1
    0.00 :	c0036a9c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036aa0:       93 c1 00 08     stw     r30,8(r1)
    0.00 :	c0036aa4:       7f be f0 78     andc    r30,r29,r30
    0.00 :	c0036aa8:       40 9e fe 5c     bne+    cr7,c0036904 <mod_timer+0x44>
    0.00 :	c0036aac:       81 3f 00 10     lwz     r9,16(r31)
    0.00 :	c0036ab0:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0036ab4:       39 20 00 01     li      r9,1
    0.00 :	c0036ab8:       7c 09 07 9e     .long 0x7c09079e
    0.00 :	c0036abc:       4b ff fe 64     b       c0036920 <mod_timer+0x60>
    0.00 :	c0036ac0:       7f a3 eb 78     mr      r3,r29
    0.00 :	c0036ac4:       48 02 29 bd     bl      c0059480 <idle_cpu>
    0.00 :	c0036ac8:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0036acc:       41 9e fe b8     beq+    cr7,c0036984 <mod_timer+0xc4>
    0.00 :	c0036ad0:       48 02 2c d1     bl      c00597a0 <get_nohz_timer_target>
    0.00 :	c0036ad4:       7c 7d 1b 78     mr      r29,r3
    0.00 :	c0036ad8:       4b ff fe ac     b       c0036984 <mod_timer+0xc4>
    0.00 :	c0036adc:       7f 7d db 78     mr      r29,r27
    0.00 :	c0036ae0:       4b ff ff 04     b       c00369e4 <mod_timer+0x124>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0096550 <perf_event_comm_output>:
  100.00 :	c0096550:       94 21 ff 30     stwu    r1,-208(r1)
    0.00 :	c0096554:       7c 08 02 a6     mflr    r0
    0.00 :	c0096558:       bf 21 00 b4     stmw    r25,180(r1)
    0.00 :	c009655c:       7c 7b 1b 78     mr      r27,r3
    0.00 :	c0096560:       90 01 00 d4     stw     r0,212(r1)
    0.00 :	c0096564:       3b e4 00 0c     addi    r31,r4,12
    0.00 :	c0096568:       7c 9c 23 78     mr      r28,r4
    0.00 :	c009656c:       7f e3 fb 78     mr      r3,r31
    0.00 :	c0096570:       38 81 00 20     addi    r4,r1,32
    0.00 :	c0096574:       7f 65 db 78     mr      r5,r27
    0.00 :	c0096578:       a3 3c 00 12     lhz     r25,18(r28)
    0.00 :	c009657c:       4b ff e9 c5     bl      c0094f40 <perf_event_header__init_id>
    0.00 :	c0096580:       a0 bc 00 12     lhz     r5,18(r28)
    0.00 :	c0096584:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0096588:       7f 64 db 78     mr      r4,r27
    0.00 :	c009658c:       48 00 37 e5     bl      c0099d70 <perf_output_begin>
    0.00 :	c0096590:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0096594:       41 9e 00 1c     beq-    cr7,c00965b0 <perf_event_comm_output+0x60>
    0.00 :	c0096598:       b3 3c 00 12     sth     r25,18(r28)
    0.00 :	c009659c:       80 01 00 d4     lwz     r0,212(r1)
    0.00 :	c00965a0:       bb 21 00 b4     lmw     r25,180(r1)
    0.00 :	c00965a4:       38 21 00 d0     addi    r1,r1,208
    0.00 :	c00965a8:       7c 08 03 a6     mtlr    r0
    0.00 :	c00965ac:       4e 80 00 20     blr
    0.00 :	c00965b0:       80 9c 00 00     lwz     r4,0(r28)
    0.00 :	c00965b4:       7f 63 db 78     mr      r3,r27
    0.00 :	c00965b8:       3b 40 10 00     li      r26,4096
    0.00 :	c00965bc:       4b ff b9 d5     bl      c0091f90 <perf_event_pid>
    0.00 :	c00965c0:       80 9c 00 00     lwz     r4,0(r28)
    0.00 :	c00965c4:       90 7c 00 14     stw     r3,20(r28)
    0.00 :	c00965c8:       7f 63 db 78     mr      r3,r27
    0.00 :	c00965cc:       4b ff ba 05     bl      c0091fd0 <perf_event_tid>
    0.00 :	c00965d0:       7f e4 fb 78     mr      r4,r31
    0.00 :	c00965d4:       90 7c 00 18     stw     r3,24(r28)
    0.00 :	c00965d8:       38 a0 00 10     li      r5,16
    0.00 :	c00965dc:       38 61 00 08     addi    r3,r1,8
    0.00 :	c00965e0:       48 00 36 d1     bl      c0099cb0 <perf_output_copy>
    0.00 :	c00965e4:       83 dc 00 08     lwz     r30,8(r28)
    0.00 :	c00965e8:       83 bc 00 04     lwz     r29,4(r28)
    0.00 :	c00965ec:       60 00 00 00     nop
    0.00 :	c00965f0:       83 e1 00 14     lwz     r31,20(r1)
    0.00 :	c00965f4:       7f a4 eb 78     mr      r4,r29
    0.00 :	c00965f8:       80 61 00 18     lwz     r3,24(r1)
    0.00 :	c00965fc:       7f 9e f8 40     cmplw   cr7,r30,r31
    0.00 :	c0096600:       7f fe ff 1e     .long 0x7ffeff1e
    0.00 :	c0096604:       7f e5 fb 78     mr      r5,r31
    0.00 :	c0096608:       7f df f0 50     subf    r30,r31,r30
    0.00 :	c009660c:       4b f8 24 4d     bl      c0018a58 <memcpy>
    0.00 :	c0096610:       80 01 00 14     lwz     r0,20(r1)
    0.00 :	c0096614:       81 21 00 18     lwz     r9,24(r1)
    0.00 :	c0096618:       2f 9e 00 00     cmpwi   cr7,r30,0
    0.00 :	c009661c:       7f bd fa 14     add     r29,r29,r31
    0.00 :	c0096620:       7c 1f 00 50     subf    r0,r31,r0
    0.00 :	c0096624:       7d 29 fa 14     add     r9,r9,r31
    0.00 :	c0096628:       2f 00 00 00     cmpwi   cr6,r0,0
    0.00 :	c009662c:       91 21 00 18     stw     r9,24(r1)
    0.00 :	c0096630:       90 01 00 14     stw     r0,20(r1)
    0.00 :	c0096634:       40 9a 00 3c     bne-    cr6,c0096670 <perf_event_comm_output+0x120>
    0.00 :	c0096638:       81 61 00 1c     lwz     r11,28(r1)
    0.00 :	c009663c:       81 21 00 0c     lwz     r9,12(r1)
    0.00 :	c0096640:       38 0b 00 01     addi    r0,r11,1
    0.00 :	c0096644:       90 01 00 1c     stw     r0,28(r1)
    0.00 :	c0096648:       81 69 00 0c     lwz     r11,12(r9)
    0.00 :	c009664c:       39 6b ff ff     addi    r11,r11,-1
    0.00 :	c0096650:       7d 6b 00 38     and     r11,r11,r0
    0.00 :	c0096654:       91 61 00 1c     stw     r11,28(r1)
    0.00 :	c0096658:       39 6b 00 10     addi    r11,r11,16
    0.00 :	c009665c:       55 6b 10 3a     rlwinm  r11,r11,2,0,29
    0.00 :	c0096660:       7d 29 5a 14     add     r9,r9,r11
    0.00 :	c0096664:       80 09 00 0c     lwz     r0,12(r9)
    0.00 :	c0096668:       93 41 00 14     stw     r26,20(r1)
    0.00 :	c009666c:       90 01 00 18     stw     r0,24(r1)
    0.00 :	c0096670:       40 9e ff 80     bne+    cr7,c00965f0 <perf_event_comm_output+0xa0>
    0.00 :	c0096674:       7f 63 db 78     mr      r3,r27
    0.00 :	c0096678:       38 81 00 08     addi    r4,r1,8
    0.00 :	c009667c:       38 a1 00 20     addi    r5,r1,32
    0.00 :	c0096680:       4b ff e8 e1     bl      c0094f60 <perf_event__output_id_sample>
    0.00 :	c0096684:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0096688:       48 00 39 c9     bl      c009a050 <perf_output_end>
    0.00 :	c009668c:       b3 3c 00 12     sth     r25,18(r28)
    0.00 :	c0096690:       80 01 00 d4     lwz     r0,212(r1)
    0.00 :	c0096694:       bb 21 00 b4     lmw     r25,180(r1)
    0.00 :	c0096698:       38 21 00 d0     addi    r1,r1,208
    0.00 :	c009669c:       7c 08 03 a6     mtlr    r0
    0.00 :	c00966a0:       4e 80 00 20     blr
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0235580 <__rb_insert_augmented>:
    0.00 :	c0235580:       94 21 ff e0     stwu    r1,-32(r1)
    0.00 :	c0235584:       7c 08 02 a6     mflr    r0
    0.00 :	c0235588:       bf 81 00 10     stmw    r28,16(r1)
    0.00 :	c023558c:       7c 9f 23 78     mr      r31,r4
    0.00 :	c0235590:       90 01 00 24     stw     r0,36(r1)
    0.00 :	c0235594:       90 a1 00 08     stw     r5,8(r1)
    0.00 :	c0235598:       81 23 00 00     lwz     r9,0(r3)
    0.00 :	c023559c:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c02355a0:       41 9e 01 9c     beq-    cr7,c023573c <__rb_insert_augmented+0x1bc>
    0.00 :	c02355a4:       83 c9 00 00     lwz     r30,0(r9)
    0.00 :	c02355a8:       73 c0 00 01     andi.   r0,r30,1
    0.00 :	c02355ac:       40 82 00 8c     bne-    c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c02355b0:       81 7e 00 04     lwz     r11,4(r30)
    0.00 :	c02355b4:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c02355b8:       7f c8 f3 78     mr      r8,r30
    0.00 :	c02355bc:       7f 89 58 00     cmpw    cr7,r9,r11
  100.00 :	c02355c0:       2f 0b 00 00     cmpwi   cr6,r11,0
    0.00 :	c02355c4:       41 9e 00 8c     beq-    cr7,c0235650 <__rb_insert_augmented+0xd0>
    0.00 :	c02355c8:       41 9a 00 10     beq-    cr6,c02355d8 <__rb_insert_augmented+0x58>
    0.00 :	c02355cc:       81 4b 00 00     lwz     r10,0(r11)
    0.00 :	c02355d0:       71 47 00 01     andi.   r7,r10,1
    0.00 :	c02355d4:       41 82 00 9c     beq-    c0235670 <__rb_insert_augmented+0xf0>
    0.00 :	c02355d8:       83 a9 00 04     lwz     r29,4(r9)
    0.00 :	c02355dc:       7d 3c 4b 78     mr      r28,r9
    0.00 :	c02355e0:       7f 9d 18 00     cmpw    cr7,r29,r3
    0.00 :	c02355e4:       41 9e 00 fc     beq-    cr7,c02356e0 <__rb_insert_augmented+0x160>
    0.00 :	c02355e8:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c02355ec:       93 be 00 08     stw     r29,8(r30)
    0.00 :	c02355f0:       93 c9 00 04     stw     r30,4(r9)
    0.00 :	c02355f4:       41 9e 00 0c     beq-    cr7,c0235600 <__rb_insert_augmented+0x80>
    0.00 :	c02355f8:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c02355fc:       90 1d 00 00     stw     r0,0(r29)
    0.00 :	c0235600:       80 1e 00 00     lwz     r0,0(r30)
    0.00 :	c0235604:       54 0b 00 3b     rlwinm. r11,r0,0,0,29
    0.00 :	c0235608:       90 09 00 00     stw     r0,0(r9)
    0.00 :	c023560c:       93 9e 00 00     stw     r28,0(r30)
    0.00 :	c0235610:       41 82 00 c0     beq-    c02356d0 <__rb_insert_augmented+0x150>
    0.00 :	c0235614:       80 0b 00 08     lwz     r0,8(r11)
    0.00 :	c0235618:       7f 80 f0 00     cmpw    cr7,r0,r30
    0.00 :	c023561c:       41 9e 01 14     beq-    cr7,c0235730 <__rb_insert_augmented+0x1b0>
    0.00 :	c0235620:       91 2b 00 04     stw     r9,4(r11)
    0.00 :	c0235624:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235628:       7f c3 f3 78     mr      r3,r30
    0.00 :	c023562c:       7d 24 4b 78     mr      r4,r9
    0.00 :	c0235630:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235634:       4e 80 04 21     bctrl
    0.00 :	c0235638:       80 01 00 24     lwz     r0,36(r1)
    0.00 :	c023563c:       bb 81 00 10     lmw     r28,16(r1)
    0.00 :	c0235640:       38 21 00 20     addi    r1,r1,32
    0.00 :	c0235644:       7c 08 03 a6     mtlr    r0
    0.00 :	c0235648:       4e 80 00 20     blr
    0.00 :	c023564c:       60 00 00 00     nop
    0.00 :	c0235650:       81 7e 00 08     lwz     r11,8(r30)
    0.00 :	c0235654:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c0235658:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c023565c:       41 9e 00 44     beq-    cr7,c02356a0 <__rb_insert_augmented+0x120>
    0.00 :	c0235660:       81 4b 00 00     lwz     r10,0(r11)
    0.00 :	c0235664:       71 47 00 01     andi.   r7,r10,1
    0.00 :	c0235668:       40 82 00 38     bne-    c02356a0 <__rb_insert_augmented+0x120>
    0.00 :	c023566c:       60 00 00 00     nop
    0.00 :	c0235670:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c0235674:       90 09 00 00     stw     r0,0(r9)
    0.00 :	c0235678:       81 3e 00 00     lwz     r9,0(r30)
    0.00 :	c023567c:       55 29 00 3a     rlwinm  r9,r9,0,0,29
    0.00 :	c0235680:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0235684:       91 3e 00 00     stw     r9,0(r30)
    0.00 :	c0235688:       41 9e 00 b0     beq-    cr7,c0235738 <__rb_insert_augmented+0x1b8>
    0.00 :	c023568c:       83 c9 00 00     lwz     r30,0(r9)
    0.00 :	c0235690:       7d 03 43 78     mr      r3,r8
    0.00 :	c0235694:       73 c7 00 01     andi.   r7,r30,1
    0.00 :	c0235698:       41 82 ff 18     beq+    c02355b0 <__rb_insert_augmented+0x30>
    0.00 :	c023569c:       4b ff ff 9c     b       c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c02356a0:       83 a9 00 08     lwz     r29,8(r9)
    0.00 :	c02356a4:       7d 3c 4b 78     mr      r28,r9
    0.00 :	c02356a8:       7f 9d 18 00     cmpw    cr7,r29,r3
    0.00 :	c02356ac:       41 9e 00 a4     beq-    cr7,c0235750 <__rb_insert_augmented+0x1d0>
    0.00 :	c02356b0:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c02356b4:       93 be 00 04     stw     r29,4(r30)
    0.00 :	c02356b8:       93 c9 00 08     stw     r30,8(r9)
    0.00 :	c02356bc:       40 9e ff 3c     bne+    cr7,c02355f8 <__rb_insert_augmented+0x78>
    0.00 :	c02356c0:       4b ff ff 40     b       c0235600 <__rb_insert_augmented+0x80>
    0.00 :	c02356c4:       60 00 00 00     nop
    0.00 :	c02356c8:       60 00 00 00     nop
    0.00 :	c02356cc:       60 00 00 00     nop
    0.00 :	c02356d0:       91 3f 00 00     stw     r9,0(r31)
    0.00 :	c02356d4:       4b ff ff 50     b       c0235624 <__rb_insert_augmented+0xa4>
    0.00 :	c02356d8:       60 00 00 00     nop
    0.00 :	c02356dc:       60 00 00 00     nop
    0.00 :	c02356e0:       81 63 00 08     lwz     r11,8(r3)
    0.00 :	c02356e4:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c02356e8:       91 69 00 04     stw     r11,4(r9)
    0.00 :	c02356ec:       91 23 00 08     stw     r9,8(r3)
    0.00 :	c02356f0:       41 9e 00 0c     beq-    cr7,c02356fc <__rb_insert_augmented+0x17c>
    0.00 :	c02356f4:       61 20 00 01     ori     r0,r9,1
    0.00 :	c02356f8:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c02356fc:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235700:       7d 23 4b 78     mr      r3,r9
    0.00 :	c0235704:       93 a9 00 00     stw     r29,0(r9)
    0.00 :	c0235708:       7f a4 eb 78     mr      r4,r29
    0.00 :	c023570c:       7f bc eb 78     mr      r28,r29
    0.00 :	c0235710:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235714:       4e 80 04 21     bctrl
    0.00 :	c0235718:       7f a9 eb 78     mr      r9,r29
    0.00 :	c023571c:       83 bd 00 04     lwz     r29,4(r29)
    0.00 :	c0235720:       4b ff fe c8     b       c02355e8 <__rb_insert_augmented+0x68>
    0.00 :	c0235724:       60 00 00 00     nop
    0.00 :	c0235728:       60 00 00 00     nop
    0.00 :	c023572c:       60 00 00 00     nop
    0.00 :	c0235730:       91 2b 00 08     stw     r9,8(r11)
    0.00 :	c0235734:       4b ff fe f0     b       c0235624 <__rb_insert_augmented+0xa4>
    0.00 :	c0235738:       7f c3 f3 78     mr      r3,r30
    0.00 :	c023573c:       38 00 00 01     li      r0,1
    0.00 :	c0235740:       90 03 00 00     stw     r0,0(r3)
    0.00 :	c0235744:       4b ff fe f4     b       c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c0235748:       60 00 00 00     nop
    0.00 :	c023574c:       60 00 00 00     nop
    0.00 :	c0235750:       81 63 00 04     lwz     r11,4(r3)
    0.00 :	c0235754:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c0235758:       91 69 00 08     stw     r11,8(r9)
    0.00 :	c023575c:       91 23 00 04     stw     r9,4(r3)
    0.00 :	c0235760:       41 9e 00 0c     beq-    cr7,c023576c <__rb_insert_augmented+0x1ec>
    0.00 :	c0235764:       61 20 00 01     ori     r0,r9,1
    0.00 :	c0235768:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c023576c:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235770:       7d 23 4b 78     mr      r3,r9
    0.00 :	c0235774:       93 a9 00 00     stw     r29,0(r9)
    0.00 :	c0235778:       7f a4 eb 78     mr      r4,r29
    0.00 :	c023577c:       7f bc eb 78     mr      r28,r29
    0.00 :	c0235780:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235784:       4e 80 04 21     bctrl
    0.00 :	c0235788:       7f a9 eb 78     mr      r9,r29
    0.00 :	c023578c:       83 bd 00 08     lwz     r29,8(r29)
    0.00 :	c0235790:       4b ff ff 20     b       c02356b0 <__rb_insert_augmented+0x130>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c040b270 <__alloc_skb>:
  100.00 :	c040b270:       94 21 ff d0     stwu    r1,-48(r1)
    0.00 :	c040b274:       7c 08 02 a6     mflr    r0
    0.00 :	c040b278:       7d 80 00 26     mfcr    r12
    0.00 :	c040b27c:       90 01 00 34     stw     r0,52(r1)
    0.00 :	c040b280:       54 a0 07 fe     clrlwi  r0,r5,31
    0.00 :	c040b284:       2e 00 00 00     cmpwi   cr4,r0,0
    0.00 :	c040b288:       bf 81 00 20     stmw    r28,32(r1)
    0.00 :	c040b28c:       91 81 00 1c     stw     r12,28(r1)
    0.00 :	c040b290:       7c 7c 1b 78     mr      r28,r3
    0.00 :	c040b294:       7c 9e 23 78     mr      r30,r4
    0.00 :	c040b298:       41 92 01 18     beq-    cr4,c040b3b0 <__alloc_skb+0x140>
    0.00 :	c040b29c:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b2a0:       83 a9 9e 50     lwz     r29,-25008(r9)
    0.00 :	c040b2a4:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b2a8:       80 09 c4 3c     lwz     r0,-15300(r9)
    0.00 :	c040b2ac:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c040b2b0:       40 9e 01 24     bne-    cr7,c040b3d4 <__alloc_skb+0x164>
    0.00 :	c040b2b4:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b2b8:       57 c4 00 3c     rlwinm  r4,r30,0,0,30
    0.00 :	c040b2bc:       4b cd 04 25     bl      c00db6e0 <kmem_cache_alloc>
    0.00 :	c040b2c0:       7c 7f 1b 79     mr.     r31,r3
    0.00 :	c040b2c4:       41 82 00 c4     beq-    c040b388 <__alloc_skb+0x118>
    0.00 :	c040b2c8:       7c 00 f9 ec     dcbtst  r0,r31
    0.00 :	c040b2cc:       3b 9c 00 3f     addi    r28,r28,63
    0.00 :	c040b2d0:       7f c4 f3 78     mr      r4,r30
    0.00 :	c040b2d4:       57 83 00 32     rlwinm  r3,r28,0,0,25
    0.00 :	c040b2d8:       38 a1 00 08     addi    r5,r1,8
    0.00 :	c040b2dc:       38 63 00 c0     addi    r3,r3,192
    0.00 :	c040b2e0:       4b ff ff 01     bl      c040b1e0 <__kmalloc_reserve.clone.56>
    0.00 :	c040b2e4:       7c 7e 1b 79     mr.     r30,r3
    0.00 :	c040b2e8:       41 82 00 d8     beq-    c040b3c0 <__alloc_skb+0x150>
    0.00 :	c040b2ec:       4b cc ec 45     bl      c00d9f30 <ksize>
    0.00 :	c040b2f0:       3b 83 ff 40     addi    r28,r3,-192
    0.00 :	c040b2f4:       7f be e2 15     add.    r29,r30,r28
    0.00 :	c040b2f8:       41 82 00 08     beq-    c040b300 <__alloc_skb+0x90>
    0.00 :	c040b2fc:       7c 00 e9 ec     dcbtst  r0,r29
    0.00 :	c040b300:       7f e3 fb 78     mr      r3,r31
    0.00 :	c040b304:       38 80 00 00     li      r4,0
    0.00 :	c040b308:       38 a0 00 98     li      r5,152
    0.00 :	c040b30c:       3b 9c 01 80     addi    r28,r28,384
    0.00 :	c040b310:       4b c0 d5 9d     bl      c00188ac <memset>
    0.00 :	c040b314:       89 21 00 08     lbz     r9,8(r1)
    0.00 :	c040b318:       38 00 00 00     li      r0,0
    0.00 :	c040b31c:       93 9f 00 a8     stw     r28,168(r31)
    0.00 :	c040b320:       3b 80 00 01     li      r28,1
    0.00 :	c040b324:       51 20 6c a4     rlwimi  r0,r9,13,18,18
    0.00 :	c040b328:       90 1f 00 78     stw     r0,120(r31)
    0.00 :	c040b32c:       93 9f 00 ac     stw     r28,172(r31)
    0.00 :	c040b330:       93 df 00 a0     stw     r30,160(r31)
    0.00 :	c040b334:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b338:       93 df 00 a4     stw     r30,164(r31)
    0.00 :	c040b33c:       38 80 00 00     li      r4,0
    0.00 :	c040b340:       93 df 00 98     stw     r30,152(r31)
    0.00 :	c040b344:       38 a0 00 24     li      r5,36
    0.00 :	c040b348:       93 bf 00 9c     stw     r29,156(r31)
    0.00 :	c040b34c:       4b c0 d5 61     bl      c00188ac <memset>
    0.00 :	c040b350:       93 9d 00 24     stw     r28,36(r29)
    0.00 :	c040b354:       41 92 00 34     beq-    cr4,c040b388 <__alloc_skb+0x118>
    0.00 :	c040b358:       80 1f 00 64     lwz     r0,100(r31)
    0.00 :	c040b35c:       53 80 9a d8     rlwimi  r0,r28,19,11,12
    0.00 :	c040b360:       90 1f 00 64     stw     r0,100(r31)
    0.00 :	c040b364:       93 9f 01 60     stw     r28,352(r31)
    0.00 :	c040b368:       81 7f 01 14     lwz     r11,276(r31)
    0.00 :	c040b36c:       39 40 00 00     li      r10,0
    0.00 :	c040b370:       80 1f 01 28     lwz     r0,296(r31)
    0.00 :	c040b374:       51 4b 9a d8     rlwimi  r11,r10,19,11,12
    0.00 :	c040b378:       91 7f 01 14     stw     r11,276(r31)
    0.00 :	c040b37c:       89 61 00 08     lbz     r11,8(r1)
    0.00 :	c040b380:       51 60 6c a4     rlwimi  r0,r11,13,18,18
    0.00 :	c040b384:       90 1f 01 28     stw     r0,296(r31)
    0.00 :	c040b388:       80 01 00 34     lwz     r0,52(r1)
    0.00 :	c040b38c:       7f e3 fb 78     mr      r3,r31
    0.00 :	c040b390:       81 81 00 1c     lwz     r12,28(r1)
    0.00 :	c040b394:       bb 81 00 20     lmw     r28,32(r1)
    0.00 :	c040b398:       38 21 00 30     addi    r1,r1,48
    0.00 :	c040b39c:       7c 08 03 a6     mtlr    r0
    0.00 :	c040b3a0:       7d 80 81 20     mtcrf   8,r12
    0.00 :	c040b3a4:       4e 80 00 20     blr
    0.00 :	c040b3a8:       60 00 00 00     nop
    0.00 :	c040b3ac:       60 00 00 00     nop
    0.00 :	c040b3b0:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b3b4:       83 a9 9e 4c     lwz     r29,-25012(r9)
    0.00 :	c040b3b8:       4b ff fe ec     b       c040b2a4 <__alloc_skb+0x34>
    0.00 :	c040b3bc:       60 00 00 00     nop
    0.00 :	c040b3c0:       7f e4 fb 78     mr      r4,r31
    0.00 :	c040b3c4:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b3c8:       4b cc f0 c9     bl      c00da490 <kmem_cache_free>
    0.00 :	c040b3cc:       3b e0 00 00     li      r31,0
    0.00 :	c040b3d0:       4b ff ff b8     b       c040b388 <__alloc_skb+0x118>
    0.00 :	c040b3d4:       70 a0 00 02     andi.   r0,r5,2
    0.00 :	c040b3d8:       63 c0 20 00     ori     r0,r30,8192
    0.00 :	c040b3dc:       7f de 00 9e     .long 0x7fde009e
    0.00 :	c040b3e0:       4b ff fe d4     b       c040b2b4 <__alloc_skb+0x44>

[-- Attachment #3: report --]
[-- Type: text/plain, Size: 525 bytes --]

# Events: 8  cycles
#
# Overhead  Command      Shared Object                  Symbol
# ........  .......  .................  ......................
#
    97.04%     find  [kernel.kallsyms]  [k] __alloc_skb
     2.32%     find  [kernel.kallsyms]  [k] mod_timer
     0.62%     find  [kernel.kallsyms]  [k] __rb_insert_augmented
     0.01%     find  [kernel.kallsyms]  [k] perf_event_comm_output
     0.00%     find  [kernel.kallsyms]  [k] perf_event_comm


#
# (For a higher level overview, try: perf report --sort comm,dso)
#

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-19 22:59             ` Scott Wood
  0 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-19 22:59 UTC (permalink / raw)
  To: Anton Blanchard
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras, linuxppc-dev

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

On 07/17/2013 11:00:45 PM, Anton Blanchard wrote:
> 
> Hi Scott,
> 
> > What specifically should I do to test it?
> 
> Could you double check perf annotate works? I'm 99% sure it will but
> that is what was failing on ppc64.

I'm not really sure what it's supposed to look like when "perf  
annotate" works.  It spits a bunch of unreadable[1] dark-blue-on-black  
assembly code at me, all with "0.00 :" in the left column.

Oh, wait -- some lines have "100.00 : " on the left, in  
even-more-unreadable dark-red-on-black.

Apart from the annoying colors, is there anything specific I should be  
looking for?  Some sort of error message, or output that actually makes  
sense?

I've attached the output from "perf annotate" and "perf report".   
perf.data was generated by "perf record find /usr > /dev/null" on an  
NFS root (which took a few seconds to complete), so the large amount of  
__alloc_skb makes some sense, but the way perf annotate shows 100% on  
one instruction in each function seems odd.

-Scott

[1] ...unless I crank the brightness up on my monitor to the point  
where whites are blinding, or redirect the output to a file so the  
colors go away.

[-- Attachment #2: annotate --]
[-- Type: text/plain, Size: 37045 bytes --]

 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0097510 <perf_event_comm>:
    0.00 :	c0097510:       94 21 ff a0     stwu    r1,-96(r1)
    0.00 :	c0097514:       7c 08 02 a6     mflr    r0
    0.00 :	c0097518:       bf 01 00 40     stmw    r24,64(r1)
    0.00 :	c009751c:       7c 7e 1b 78     mr      r30,r3
    0.00 :	c0097520:       90 01 00 64     stw     r0,100(r1)
    0.00 :	c0097524:       3b 80 00 00     li      r28,0
    0.00 :	c0097528:       3b 63 04 68     addi    r27,r3,1128
    0.00 :	c009752c:       3b 40 00 00     li      r26,0
    0.00 :	c0097530:       87 bb 00 04     lwzu    r29,4(r27)
    0.00 :	c0097534:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c0097538:       41 9e 00 1c     beq-    cr7,c0097554 <perf_event_comm+0x44>
    0.00 :	c009753c:       7f 00 00 a6     mfmsr   r24
    0.00 :	c0097540:       7c 00 01 46     .long 0x7c000146
    0.00 :	c0097544:       80 1d 00 3c     lwz     r0,60(r29)
    0.00 :	c0097548:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c009754c:       40 9e 00 c4     bne-    cr7,c0097610 <perf_event_comm+0x100>
    0.00 :	c0097550:       7f 00 01 06     .long 0x7f000106
  100.00 :	c0097554:       2f 9c 00 01     cmpwi   cr7,r28,1
    0.00 :	c0097558:       3b 9c 00 01     addi    r28,r28,1
    0.00 :	c009755c:       40 be ff d4     bne-    cr7,c0097530 <perf_event_comm+0x20>
    0.00 :	c0097560:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0097564:       80 09 99 50     lwz     r0,-26288(r9)
    0.00 :	c0097568:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c009756c:       41 be 00 88     beq+    cr7,c00975f4 <perf_event_comm+0xe4>
    0.00 :	c0097570:       3b e1 00 08     addi    r31,r1,8
    0.00 :	c0097574:       38 00 00 00     li      r0,0
    0.00 :	c0097578:       39 20 00 03     li      r9,3
    0.00 :	c009757c:       38 9e 01 d8     addi    r4,r30,472
    0.00 :	c0097580:       38 a0 00 10     li      r5,16
    0.00 :	c0097584:       7f e3 fb 78     mr      r3,r31
    0.00 :	c0097588:       90 01 00 1c     stw     r0,28(r1)
    0.00 :	c009758c:       90 01 00 20     stw     r0,32(r1)
    0.00 :	c0097590:       90 01 00 28     stw     r0,40(r1)
    0.00 :	c0097594:       90 01 00 2c     stw     r0,44(r1)
    0.00 :	c0097598:       90 01 00 30     stw     r0,48(r1)
    0.00 :	c009759c:       91 21 00 24     stw     r9,36(r1)
    0.00 :	c00975a0:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c00975a4:       90 01 00 0c     stw     r0,12(r1)
    0.00 :	c00975a8:       90 01 00 10     stw     r0,16(r1)
    0.00 :	c00975ac:       90 01 00 14     stw     r0,20(r1)
    0.00 :	c00975b0:       93 c1 00 18     stw     r30,24(r1)
    0.00 :	c00975b4:       48 1a 05 8d     bl      c0237b40 <strlcpy>
    0.00 :	c00975b8:       7f e3 fb 78     mr      r3,r31
    0.00 :	c00975bc:       4b f8 0e 89     bl      c0018444 <strlen>
    0.00 :	c00975c0:       3c 80 c0 09     lis     r4,-16375
    0.00 :	c00975c4:       39 23 00 08     addi    r9,r3,8
    0.00 :	c00975c8:       93 e1 00 1c     stw     r31,28(r1)
    0.00 :	c00975cc:       55 29 00 38     rlwinm  r9,r9,0,0,28
    0.00 :	c00975d0:       3c 60 c0 09     lis     r3,-16375
    0.00 :	c00975d4:       38 09 00 10     addi    r0,r9,16
    0.00 :	c00975d8:       91 21 00 20     stw     r9,32(r1)
    0.00 :	c00975dc:       38 63 fd 80     addi    r3,r3,-640
    0.00 :	c00975e0:       38 84 65 50     addi    r4,r4,25936
    0.00 :	c00975e4:       38 a1 00 18     addi    r5,r1,24
    0.00 :	c00975e8:       b0 01 00 2a     sth     r0,42(r1)
    0.00 :	c00975ec:       38 c0 00 00     li      r6,0
    0.00 :	c00975f0:       4b ff 86 61     bl      c008fc50 <perf_event_aux>
    0.00 :	c00975f4:       80 01 00 64     lwz     r0,100(r1)
    0.00 :	c00975f8:       bb 01 00 40     lmw     r24,64(r1)
    0.00 :	c00975fc:       38 21 00 60     addi    r1,r1,96
    0.00 :	c0097600:       7c 08 03 a6     mtlr    r0
    0.00 :	c0097604:       4e 80 00 20     blr
    0.00 :	c0097608:       60 00 00 00     nop
    0.00 :	c009760c:       60 00 00 00     nop
    0.00 :	c0097610:       38 7d 00 08     addi    r3,r29,8
    0.00 :	c0097614:       7f b9 eb 78     mr      r25,r29
    0.00 :	c0097618:       48 48 98 a9     bl      c0520ec0 <_raw_spin_lock>
    0.00 :	c009761c:       7f a3 eb 78     mr      r3,r29
    0.00 :	c0097620:       4b ff ce 81     bl      c00944a0 <task_ctx_sched_out>
    0.00 :	c0097624:       87 f9 00 34     lwzu    r31,52(r25)
    0.00 :	c0097628:       7f 99 f8 00     cmpw    cr7,r25,r31
    0.00 :	c009762c:       3b ff ff f8     addi    r31,r31,-8
    0.00 :	c0097630:       41 9e 00 60     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c0097634:       39 60 00 00     li      r11,0
    0.00 :	c0097638:       60 00 00 00     nop
    0.00 :	c009763c:       60 00 00 00     nop
    0.00 :	c0097640:       80 1f 00 a0     lwz     r0,160(r31)
    0.00 :	c0097644:       7c 09 03 78     mr      r9,r0
    0.00 :	c0097648:       74 0a 00 08     andis.  r10,r0,8
    0.00 :	c009764c:       53 49 9b 18     rlwimi  r9,r26,19,12,12
    0.00 :	c0097650:       41 82 00 14     beq-    c0097664 <perf_event_comm+0x154>
    0.00 :	c0097654:       80 1f 00 30     lwz     r0,48(r31)
    0.00 :	c0097658:       91 3f 00 a0     stw     r9,160(r31)
    0.00 :	c009765c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0097660:       41 9c 00 50     blt-    cr7,c00976b0 <perf_event_comm+0x1a0>
    0.00 :	c0097664:       83 ff 00 08     lwz     r31,8(r31)
    0.00 :	c0097668:       7f 99 f8 00     cmpw    cr7,r25,r31
    0.00 :	c009766c:       3b ff ff f8     addi    r31,r31,-8
    0.00 :	c0097670:       40 9e ff d0     bne+    cr7,c0097640 <perf_event_comm+0x130>
    0.00 :	c0097674:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c0097678:       41 9e 00 18     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c009767c:       80 7d 00 70     lwz     r3,112(r29)
    0.00 :	c0097680:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0097684:       41 9e 00 0c     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c0097688:       4b ff a6 69     bl      c0091cf0 <put_ctx>
    0.00 :	c009768c:       93 5d 00 70     stw     r26,112(r29)
    0.00 :	c0097690:       7c 00 04 ac     sync    
    0.00 :	c0097694:       93 5d 00 08     stw     r26,8(r29)
    0.00 :	c0097698:       7f a3 eb 78     mr      r3,r29
    0.00 :	c009769c:       4b ff e1 75     bl      c0095810 <perf_event_context_sched_in.clone.38>
    0.00 :	c00976a0:       4b ff fe b0     b       c0097550 <perf_event_comm+0x40>
    0.00 :	c00976a4:       60 00 00 00     nop
    0.00 :	c00976a8:       60 00 00 00     nop
    0.00 :	c00976ac:       60 00 00 00     nop
    0.00 :	c00976b0:       7f e3 fb 78     mr      r3,r31
    0.00 :	c00976b4:       4b ff 83 cd     bl      c008fa80 <__perf_event_mark_enabled>
    0.00 :	c00976b8:       39 60 00 01     li      r11,1
    0.00 :	c00976bc:       4b ff ff a8     b       c0097664 <perf_event_comm+0x154>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c00368c0 <mod_timer>:
  100.00 :	c00368c0:       94 21 ff d0     stwu    r1,-48(r1)
    0.00 :	c00368c4:       7c 08 02 a6     mflr    r0
    0.00 :	c00368c8:       bf 61 00 1c     stmw    r27,28(r1)
    0.00 :	c00368cc:       7c 7f 1b 78     mr      r31,r3
    0.00 :	c00368d0:       90 01 00 34     stw     r0,52(r1)
    0.00 :	c00368d4:       7c 9e 23 78     mr      r30,r4
    0.00 :	c00368d8:       83 a3 00 18     lwz     r29,24(r3)
    0.00 :	c00368dc:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c00368e0:       41 9c 01 70     blt-    cr7,c0036a50 <mod_timer+0x190>
    0.00 :	c00368e4:       7f bd f2 14     add     r29,r29,r30
    0.00 :	c00368e8:       7f a0 f2 78     xor     r0,r29,r30
    0.00 :	c00368ec:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c00368f0:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c00368f4:       40 9e 01 8c     bne-    cr7,c0036a80 <mod_timer+0x1c0>
    0.00 :	c00368f8:       80 1f 00 00     lwz     r0,0(r31)
    0.00 :	c00368fc:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036900:       41 9e 01 ac     beq-    cr7,c0036aac <mod_timer+0x1ec>
    0.00 :	c0036904:       80 1f 00 08     lwz     r0,8(r31)
    0.00 :	c0036908:       3b 80 00 01     li      r28,1
    0.00 :	c003690c:       7f 80 f0 00     cmpw    cr7,r0,r30
    0.00 :	c0036910:       41 9e 00 f0     beq-    cr7,c0036a00 <mod_timer+0x140>
    0.00 :	c0036914:       80 1f 00 10     lwz     r0,16(r31)
    0.00 :	c0036918:       7c 00 00 34     cntlzw  r0,r0
    0.00 :	c003691c:       54 00 d9 7e     rlwinm  r0,r0,27,5,31
    0.00 :	c0036920:       0f 00 00 00     twnei   r0,0
    0.00 :	c0036924:       38 7f 00 0c     addi    r3,r31,12
    0.00 :	c0036928:       38 81 00 08     addi    r4,r1,8
    0.00 :	c003692c:       4b ff f9 75     bl      c00362a0 <lock_timer_base.clone.24>
    0.00 :	c0036930:       81 3f 00 00     lwz     r9,0(r31)
    0.00 :	c0036934:       7c 7b 1b 78     mr      r27,r3
    0.00 :	c0036938:       3b 80 00 00     li      r28,0
    0.00 :	c003693c:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0036940:       41 9e 00 2c     beq-    cr7,c003696c <mod_timer+0xac>
    0.00 :	c0036944:       81 7f 00 04     lwz     r11,4(r31)
    0.00 :	c0036948:       3c 00 00 20     lis     r0,32
    0.00 :	c003694c:       60 00 02 00     ori     r0,r0,512
    0.00 :	c0036950:       3b 80 00 01     li      r28,1
    0.00 :	c0036954:       91 69 00 04     stw     r11,4(r9)
    0.00 :	c0036958:       91 2b 00 00     stw     r9,0(r11)
    0.00 :	c003695c:       90 1f 00 04     stw     r0,4(r31)
    0.00 :	c0036960:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c0036964:       70 09 00 01     andi.   r9,r0,1
    0.00 :	c0036968:       41 82 00 b8     beq-    c0036a20 <mod_timer+0x160>
    0.00 :	c003696c:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0036970:       80 09 98 ec     lwz     r0,-26388(r9)
    0.00 :	c0036974:       54 29 00 24     rlwinm  r9,r1,0,0,18
    0.00 :	c0036978:       83 a9 00 08     lwz     r29,8(r9)
    0.00 :	c003697c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036980:       40 9e 01 40     bne-    cr7,c0036ac0 <mod_timer+0x200>
    0.00 :	c0036984:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0036988:       57 bd 10 3a     rlwinm  r29,r29,2,0,29
    0.00 :	c003698c:       39 29 9b 40     addi    r9,r9,-25792
    0.00 :	c0036990:       7d 69 e8 2e     lwzx    r11,r9,r29
    0.00 :	c0036994:       3d 20 c0 6d     lis     r9,-16275
    0.00 :	c0036998:       39 29 04 70     addi    r9,r9,1136
    0.00 :	c003699c:       7f ab 48 2e     lwzx    r29,r11,r9
    0.00 :	c00369a0:       7f 9b e8 00     cmpw    cr7,r27,r29
    0.00 :	c00369a4:       41 9e 00 40     beq-    cr7,c00369e4 <mod_timer+0x124>
    0.00 :	c00369a8:       80 1b 00 04     lwz     r0,4(r27)
    0.00 :	c00369ac:       7f 9f 00 00     cmpw    cr7,r31,r0
    0.00 :	c00369b0:       41 9e 01 2c     beq-    cr7,c0036adc <mod_timer+0x21c>
    0.00 :	c00369b4:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c00369b8:       54 00 07 be     clrlwi  r0,r0,30
    0.00 :	c00369bc:       90 1f 00 0c     stw     r0,12(r31)
    0.00 :	c00369c0:       7c 00 04 ac     sync    
    0.00 :	c00369c4:       38 00 00 00     li      r0,0
    0.00 :	c00369c8:       90 1b 00 00     stw     r0,0(r27)
    0.00 :	c00369cc:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369d0:       48 4e a4 f1     bl      c0520ec0 <_raw_spin_lock>
    0.00 :	c00369d4:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c00369d8:       54 00 07 be     clrlwi  r0,r0,30
    0.00 :	c00369dc:       7f a0 03 78     or      r0,r29,r0
    0.00 :	c00369e0:       90 1f 00 0c     stw     r0,12(r31)
    0.00 :	c00369e4:       93 df 00 08     stw     r30,8(r31)
    0.00 :	c00369e8:       7f e4 fb 78     mr      r4,r31
    0.00 :	c00369ec:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369f0:       4b ff f1 d1     bl      c0035bc0 <internal_add_timer>
    0.00 :	c00369f4:       80 81 00 08     lwz     r4,8(r1)
    0.00 :	c00369f8:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369fc:       48 4e 9e 65     bl      c0520860 <_raw_spin_unlock_irqrestore>
    0.00 :	c0036a00:       80 01 00 34     lwz     r0,52(r1)
    0.00 :	c0036a04:       7f 83 e3 78     mr      r3,r28
    0.00 :	c0036a08:       bb 61 00 1c     lmw     r27,28(r1)
    0.00 :	c0036a0c:       38 21 00 30     addi    r1,r1,48
    0.00 :	c0036a10:       7c 08 03 a6     mtlr    r0
    0.00 :	c0036a14:       4e 80 00 20     blr
    0.00 :	c0036a18:       60 00 00 00     nop
    0.00 :	c0036a1c:       60 00 00 00     nop
    0.00 :	c0036a20:       81 23 00 10     lwz     r9,16(r3)
    0.00 :	c0036a24:       80 03 00 0c     lwz     r0,12(r3)
    0.00 :	c0036a28:       39 29 ff ff     addi    r9,r9,-1
    0.00 :	c0036a2c:       91 23 00 10     stw     r9,16(r3)
    0.00 :	c0036a30:       81 3f 00 08     lwz     r9,8(r31)
    0.00 :	c0036a34:       7f 89 00 00     cmpw    cr7,r9,r0
    0.00 :	c0036a38:       40 9e ff 34     bne+    cr7,c003696c <mod_timer+0xac>
    0.00 :	c0036a3c:       80 03 00 08     lwz     r0,8(r3)
    0.00 :	c0036a40:       90 03 00 0c     stw     r0,12(r3)
    0.00 :	c0036a44:       4b ff ff 28     b       c003696c <mod_timer+0xac>
    0.00 :	c0036a48:       60 00 00 00     nop
    0.00 :	c0036a4c:       60 00 00 00     nop
    0.00 :	c0036a50:       3d 20 c0 71     lis     r9,-16271
    0.00 :	c0036a54:       83 a9 30 c4     lwz     r29,12484(r9)
    0.00 :	c0036a58:       7f bd 20 50     subf    r29,r29,r4
    0.00 :	c0036a5c:       2f 9d 00 ff     cmpwi   cr7,r29,255
    0.00 :	c0036a60:       40 bd fe 98     ble-    cr7,c00368f8 <mod_timer+0x38>
    0.00 :	c0036a64:       7f bd 46 70     srawi   r29,r29,8
    0.00 :	c0036a68:       7f bd f2 14     add     r29,r29,r30
    0.00 :	c0036a6c:       7f a0 f2 78     xor     r0,r29,r30
    0.00 :	c0036a70:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036a74:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c0036a78:       41 9e fe 80     beq+    cr7,c00368f8 <mod_timer+0x38>
    0.00 :	c0036a7c:       60 00 00 00     nop
    0.00 :	c0036a80:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0036a84:       38 80 00 20     li      r4,32
    0.00 :	c0036a88:       48 20 94 39     bl      c023fec0 <find_last_bit>
    0.00 :	c0036a8c:       80 1f 00 00     lwz     r0,0(r31)
    0.00 :	c0036a90:       3b c0 00 01     li      r30,1
    0.00 :	c0036a94:       7f de 18 30     slw     r30,r30,r3
    0.00 :	c0036a98:       3b de ff ff     addi    r30,r30,-1
    0.00 :	c0036a9c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036aa0:       93 c1 00 08     stw     r30,8(r1)
    0.00 :	c0036aa4:       7f be f0 78     andc    r30,r29,r30
    0.00 :	c0036aa8:       40 9e fe 5c     bne+    cr7,c0036904 <mod_timer+0x44>
    0.00 :	c0036aac:       81 3f 00 10     lwz     r9,16(r31)
    0.00 :	c0036ab0:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0036ab4:       39 20 00 01     li      r9,1
    0.00 :	c0036ab8:       7c 09 07 9e     .long 0x7c09079e
    0.00 :	c0036abc:       4b ff fe 64     b       c0036920 <mod_timer+0x60>
    0.00 :	c0036ac0:       7f a3 eb 78     mr      r3,r29
    0.00 :	c0036ac4:       48 02 29 bd     bl      c0059480 <idle_cpu>
    0.00 :	c0036ac8:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0036acc:       41 9e fe b8     beq+    cr7,c0036984 <mod_timer+0xc4>
    0.00 :	c0036ad0:       48 02 2c d1     bl      c00597a0 <get_nohz_timer_target>
    0.00 :	c0036ad4:       7c 7d 1b 78     mr      r29,r3
    0.00 :	c0036ad8:       4b ff fe ac     b       c0036984 <mod_timer+0xc4>
    0.00 :	c0036adc:       7f 7d db 78     mr      r29,r27
    0.00 :	c0036ae0:       4b ff ff 04     b       c00369e4 <mod_timer+0x124>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0096550 <perf_event_comm_output>:
  100.00 :	c0096550:       94 21 ff 30     stwu    r1,-208(r1)
    0.00 :	c0096554:       7c 08 02 a6     mflr    r0
    0.00 :	c0096558:       bf 21 00 b4     stmw    r25,180(r1)
    0.00 :	c009655c:       7c 7b 1b 78     mr      r27,r3
    0.00 :	c0096560:       90 01 00 d4     stw     r0,212(r1)
    0.00 :	c0096564:       3b e4 00 0c     addi    r31,r4,12
    0.00 :	c0096568:       7c 9c 23 78     mr      r28,r4
    0.00 :	c009656c:       7f e3 fb 78     mr      r3,r31
    0.00 :	c0096570:       38 81 00 20     addi    r4,r1,32
    0.00 :	c0096574:       7f 65 db 78     mr      r5,r27
    0.00 :	c0096578:       a3 3c 00 12     lhz     r25,18(r28)
    0.00 :	c009657c:       4b ff e9 c5     bl      c0094f40 <perf_event_header__init_id>
    0.00 :	c0096580:       a0 bc 00 12     lhz     r5,18(r28)
    0.00 :	c0096584:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0096588:       7f 64 db 78     mr      r4,r27
    0.00 :	c009658c:       48 00 37 e5     bl      c0099d70 <perf_output_begin>
    0.00 :	c0096590:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0096594:       41 9e 00 1c     beq-    cr7,c00965b0 <perf_event_comm_output+0x60>
    0.00 :	c0096598:       b3 3c 00 12     sth     r25,18(r28)
    0.00 :	c009659c:       80 01 00 d4     lwz     r0,212(r1)
    0.00 :	c00965a0:       bb 21 00 b4     lmw     r25,180(r1)
    0.00 :	c00965a4:       38 21 00 d0     addi    r1,r1,208
    0.00 :	c00965a8:       7c 08 03 a6     mtlr    r0
    0.00 :	c00965ac:       4e 80 00 20     blr
    0.00 :	c00965b0:       80 9c 00 00     lwz     r4,0(r28)
    0.00 :	c00965b4:       7f 63 db 78     mr      r3,r27
    0.00 :	c00965b8:       3b 40 10 00     li      r26,4096
    0.00 :	c00965bc:       4b ff b9 d5     bl      c0091f90 <perf_event_pid>
    0.00 :	c00965c0:       80 9c 00 00     lwz     r4,0(r28)
    0.00 :	c00965c4:       90 7c 00 14     stw     r3,20(r28)
    0.00 :	c00965c8:       7f 63 db 78     mr      r3,r27
    0.00 :	c00965cc:       4b ff ba 05     bl      c0091fd0 <perf_event_tid>
    0.00 :	c00965d0:       7f e4 fb 78     mr      r4,r31
    0.00 :	c00965d4:       90 7c 00 18     stw     r3,24(r28)
    0.00 :	c00965d8:       38 a0 00 10     li      r5,16
    0.00 :	c00965dc:       38 61 00 08     addi    r3,r1,8
    0.00 :	c00965e0:       48 00 36 d1     bl      c0099cb0 <perf_output_copy>
    0.00 :	c00965e4:       83 dc 00 08     lwz     r30,8(r28)
    0.00 :	c00965e8:       83 bc 00 04     lwz     r29,4(r28)
    0.00 :	c00965ec:       60 00 00 00     nop
    0.00 :	c00965f0:       83 e1 00 14     lwz     r31,20(r1)
    0.00 :	c00965f4:       7f a4 eb 78     mr      r4,r29
    0.00 :	c00965f8:       80 61 00 18     lwz     r3,24(r1)
    0.00 :	c00965fc:       7f 9e f8 40     cmplw   cr7,r30,r31
    0.00 :	c0096600:       7f fe ff 1e     .long 0x7ffeff1e
    0.00 :	c0096604:       7f e5 fb 78     mr      r5,r31
    0.00 :	c0096608:       7f df f0 50     subf    r30,r31,r30
    0.00 :	c009660c:       4b f8 24 4d     bl      c0018a58 <memcpy>
    0.00 :	c0096610:       80 01 00 14     lwz     r0,20(r1)
    0.00 :	c0096614:       81 21 00 18     lwz     r9,24(r1)
    0.00 :	c0096618:       2f 9e 00 00     cmpwi   cr7,r30,0
    0.00 :	c009661c:       7f bd fa 14     add     r29,r29,r31
    0.00 :	c0096620:       7c 1f 00 50     subf    r0,r31,r0
    0.00 :	c0096624:       7d 29 fa 14     add     r9,r9,r31
    0.00 :	c0096628:       2f 00 00 00     cmpwi   cr6,r0,0
    0.00 :	c009662c:       91 21 00 18     stw     r9,24(r1)
    0.00 :	c0096630:       90 01 00 14     stw     r0,20(r1)
    0.00 :	c0096634:       40 9a 00 3c     bne-    cr6,c0096670 <perf_event_comm_output+0x120>
    0.00 :	c0096638:       81 61 00 1c     lwz     r11,28(r1)
    0.00 :	c009663c:       81 21 00 0c     lwz     r9,12(r1)
    0.00 :	c0096640:       38 0b 00 01     addi    r0,r11,1
    0.00 :	c0096644:       90 01 00 1c     stw     r0,28(r1)
    0.00 :	c0096648:       81 69 00 0c     lwz     r11,12(r9)
    0.00 :	c009664c:       39 6b ff ff     addi    r11,r11,-1
    0.00 :	c0096650:       7d 6b 00 38     and     r11,r11,r0
    0.00 :	c0096654:       91 61 00 1c     stw     r11,28(r1)
    0.00 :	c0096658:       39 6b 00 10     addi    r11,r11,16
    0.00 :	c009665c:       55 6b 10 3a     rlwinm  r11,r11,2,0,29
    0.00 :	c0096660:       7d 29 5a 14     add     r9,r9,r11
    0.00 :	c0096664:       80 09 00 0c     lwz     r0,12(r9)
    0.00 :	c0096668:       93 41 00 14     stw     r26,20(r1)
    0.00 :	c009666c:       90 01 00 18     stw     r0,24(r1)
    0.00 :	c0096670:       40 9e ff 80     bne+    cr7,c00965f0 <perf_event_comm_output+0xa0>
    0.00 :	c0096674:       7f 63 db 78     mr      r3,r27
    0.00 :	c0096678:       38 81 00 08     addi    r4,r1,8
    0.00 :	c009667c:       38 a1 00 20     addi    r5,r1,32
    0.00 :	c0096680:       4b ff e8 e1     bl      c0094f60 <perf_event__output_id_sample>
    0.00 :	c0096684:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0096688:       48 00 39 c9     bl      c009a050 <perf_output_end>
    0.00 :	c009668c:       b3 3c 00 12     sth     r25,18(r28)
    0.00 :	c0096690:       80 01 00 d4     lwz     r0,212(r1)
    0.00 :	c0096694:       bb 21 00 b4     lmw     r25,180(r1)
    0.00 :	c0096698:       38 21 00 d0     addi    r1,r1,208
    0.00 :	c009669c:       7c 08 03 a6     mtlr    r0
    0.00 :	c00966a0:       4e 80 00 20     blr
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0235580 <__rb_insert_augmented>:
    0.00 :	c0235580:       94 21 ff e0     stwu    r1,-32(r1)
    0.00 :	c0235584:       7c 08 02 a6     mflr    r0
    0.00 :	c0235588:       bf 81 00 10     stmw    r28,16(r1)
    0.00 :	c023558c:       7c 9f 23 78     mr      r31,r4
    0.00 :	c0235590:       90 01 00 24     stw     r0,36(r1)
    0.00 :	c0235594:       90 a1 00 08     stw     r5,8(r1)
    0.00 :	c0235598:       81 23 00 00     lwz     r9,0(r3)
    0.00 :	c023559c:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c02355a0:       41 9e 01 9c     beq-    cr7,c023573c <__rb_insert_augmented+0x1bc>
    0.00 :	c02355a4:       83 c9 00 00     lwz     r30,0(r9)
    0.00 :	c02355a8:       73 c0 00 01     andi.   r0,r30,1
    0.00 :	c02355ac:       40 82 00 8c     bne-    c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c02355b0:       81 7e 00 04     lwz     r11,4(r30)
    0.00 :	c02355b4:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c02355b8:       7f c8 f3 78     mr      r8,r30
    0.00 :	c02355bc:       7f 89 58 00     cmpw    cr7,r9,r11
  100.00 :	c02355c0:       2f 0b 00 00     cmpwi   cr6,r11,0
    0.00 :	c02355c4:       41 9e 00 8c     beq-    cr7,c0235650 <__rb_insert_augmented+0xd0>
    0.00 :	c02355c8:       41 9a 00 10     beq-    cr6,c02355d8 <__rb_insert_augmented+0x58>
    0.00 :	c02355cc:       81 4b 00 00     lwz     r10,0(r11)
    0.00 :	c02355d0:       71 47 00 01     andi.   r7,r10,1
    0.00 :	c02355d4:       41 82 00 9c     beq-    c0235670 <__rb_insert_augmented+0xf0>
    0.00 :	c02355d8:       83 a9 00 04     lwz     r29,4(r9)
    0.00 :	c02355dc:       7d 3c 4b 78     mr      r28,r9
    0.00 :	c02355e0:       7f 9d 18 00     cmpw    cr7,r29,r3
    0.00 :	c02355e4:       41 9e 00 fc     beq-    cr7,c02356e0 <__rb_insert_augmented+0x160>
    0.00 :	c02355e8:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c02355ec:       93 be 00 08     stw     r29,8(r30)
    0.00 :	c02355f0:       93 c9 00 04     stw     r30,4(r9)
    0.00 :	c02355f4:       41 9e 00 0c     beq-    cr7,c0235600 <__rb_insert_augmented+0x80>
    0.00 :	c02355f8:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c02355fc:       90 1d 00 00     stw     r0,0(r29)
    0.00 :	c0235600:       80 1e 00 00     lwz     r0,0(r30)
    0.00 :	c0235604:       54 0b 00 3b     rlwinm. r11,r0,0,0,29
    0.00 :	c0235608:       90 09 00 00     stw     r0,0(r9)
    0.00 :	c023560c:       93 9e 00 00     stw     r28,0(r30)
    0.00 :	c0235610:       41 82 00 c0     beq-    c02356d0 <__rb_insert_augmented+0x150>
    0.00 :	c0235614:       80 0b 00 08     lwz     r0,8(r11)
    0.00 :	c0235618:       7f 80 f0 00     cmpw    cr7,r0,r30
    0.00 :	c023561c:       41 9e 01 14     beq-    cr7,c0235730 <__rb_insert_augmented+0x1b0>
    0.00 :	c0235620:       91 2b 00 04     stw     r9,4(r11)
    0.00 :	c0235624:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235628:       7f c3 f3 78     mr      r3,r30
    0.00 :	c023562c:       7d 24 4b 78     mr      r4,r9
    0.00 :	c0235630:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235634:       4e 80 04 21     bctrl
    0.00 :	c0235638:       80 01 00 24     lwz     r0,36(r1)
    0.00 :	c023563c:       bb 81 00 10     lmw     r28,16(r1)
    0.00 :	c0235640:       38 21 00 20     addi    r1,r1,32
    0.00 :	c0235644:       7c 08 03 a6     mtlr    r0
    0.00 :	c0235648:       4e 80 00 20     blr
    0.00 :	c023564c:       60 00 00 00     nop
    0.00 :	c0235650:       81 7e 00 08     lwz     r11,8(r30)
    0.00 :	c0235654:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c0235658:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c023565c:       41 9e 00 44     beq-    cr7,c02356a0 <__rb_insert_augmented+0x120>
    0.00 :	c0235660:       81 4b 00 00     lwz     r10,0(r11)
    0.00 :	c0235664:       71 47 00 01     andi.   r7,r10,1
    0.00 :	c0235668:       40 82 00 38     bne-    c02356a0 <__rb_insert_augmented+0x120>
    0.00 :	c023566c:       60 00 00 00     nop
    0.00 :	c0235670:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c0235674:       90 09 00 00     stw     r0,0(r9)
    0.00 :	c0235678:       81 3e 00 00     lwz     r9,0(r30)
    0.00 :	c023567c:       55 29 00 3a     rlwinm  r9,r9,0,0,29
    0.00 :	c0235680:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0235684:       91 3e 00 00     stw     r9,0(r30)
    0.00 :	c0235688:       41 9e 00 b0     beq-    cr7,c0235738 <__rb_insert_augmented+0x1b8>
    0.00 :	c023568c:       83 c9 00 00     lwz     r30,0(r9)
    0.00 :	c0235690:       7d 03 43 78     mr      r3,r8
    0.00 :	c0235694:       73 c7 00 01     andi.   r7,r30,1
    0.00 :	c0235698:       41 82 ff 18     beq+    c02355b0 <__rb_insert_augmented+0x30>
    0.00 :	c023569c:       4b ff ff 9c     b       c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c02356a0:       83 a9 00 08     lwz     r29,8(r9)
    0.00 :	c02356a4:       7d 3c 4b 78     mr      r28,r9
    0.00 :	c02356a8:       7f 9d 18 00     cmpw    cr7,r29,r3
    0.00 :	c02356ac:       41 9e 00 a4     beq-    cr7,c0235750 <__rb_insert_augmented+0x1d0>
    0.00 :	c02356b0:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c02356b4:       93 be 00 04     stw     r29,4(r30)
    0.00 :	c02356b8:       93 c9 00 08     stw     r30,8(r9)
    0.00 :	c02356bc:       40 9e ff 3c     bne+    cr7,c02355f8 <__rb_insert_augmented+0x78>
    0.00 :	c02356c0:       4b ff ff 40     b       c0235600 <__rb_insert_augmented+0x80>
    0.00 :	c02356c4:       60 00 00 00     nop
    0.00 :	c02356c8:       60 00 00 00     nop
    0.00 :	c02356cc:       60 00 00 00     nop
    0.00 :	c02356d0:       91 3f 00 00     stw     r9,0(r31)
    0.00 :	c02356d4:       4b ff ff 50     b       c0235624 <__rb_insert_augmented+0xa4>
    0.00 :	c02356d8:       60 00 00 00     nop
    0.00 :	c02356dc:       60 00 00 00     nop
    0.00 :	c02356e0:       81 63 00 08     lwz     r11,8(r3)
    0.00 :	c02356e4:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c02356e8:       91 69 00 04     stw     r11,4(r9)
    0.00 :	c02356ec:       91 23 00 08     stw     r9,8(r3)
    0.00 :	c02356f0:       41 9e 00 0c     beq-    cr7,c02356fc <__rb_insert_augmented+0x17c>
    0.00 :	c02356f4:       61 20 00 01     ori     r0,r9,1
    0.00 :	c02356f8:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c02356fc:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235700:       7d 23 4b 78     mr      r3,r9
    0.00 :	c0235704:       93 a9 00 00     stw     r29,0(r9)
    0.00 :	c0235708:       7f a4 eb 78     mr      r4,r29
    0.00 :	c023570c:       7f bc eb 78     mr      r28,r29
    0.00 :	c0235710:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235714:       4e 80 04 21     bctrl
    0.00 :	c0235718:       7f a9 eb 78     mr      r9,r29
    0.00 :	c023571c:       83 bd 00 04     lwz     r29,4(r29)
    0.00 :	c0235720:       4b ff fe c8     b       c02355e8 <__rb_insert_augmented+0x68>
    0.00 :	c0235724:       60 00 00 00     nop
    0.00 :	c0235728:       60 00 00 00     nop
    0.00 :	c023572c:       60 00 00 00     nop
    0.00 :	c0235730:       91 2b 00 08     stw     r9,8(r11)
    0.00 :	c0235734:       4b ff fe f0     b       c0235624 <__rb_insert_augmented+0xa4>
    0.00 :	c0235738:       7f c3 f3 78     mr      r3,r30
    0.00 :	c023573c:       38 00 00 01     li      r0,1
    0.00 :	c0235740:       90 03 00 00     stw     r0,0(r3)
    0.00 :	c0235744:       4b ff fe f4     b       c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c0235748:       60 00 00 00     nop
    0.00 :	c023574c:       60 00 00 00     nop
    0.00 :	c0235750:       81 63 00 04     lwz     r11,4(r3)
    0.00 :	c0235754:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c0235758:       91 69 00 08     stw     r11,8(r9)
    0.00 :	c023575c:       91 23 00 04     stw     r9,4(r3)
    0.00 :	c0235760:       41 9e 00 0c     beq-    cr7,c023576c <__rb_insert_augmented+0x1ec>
    0.00 :	c0235764:       61 20 00 01     ori     r0,r9,1
    0.00 :	c0235768:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c023576c:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235770:       7d 23 4b 78     mr      r3,r9
    0.00 :	c0235774:       93 a9 00 00     stw     r29,0(r9)
    0.00 :	c0235778:       7f a4 eb 78     mr      r4,r29
    0.00 :	c023577c:       7f bc eb 78     mr      r28,r29
    0.00 :	c0235780:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235784:       4e 80 04 21     bctrl
    0.00 :	c0235788:       7f a9 eb 78     mr      r9,r29
    0.00 :	c023578c:       83 bd 00 08     lwz     r29,8(r29)
    0.00 :	c0235790:       4b ff ff 20     b       c02356b0 <__rb_insert_augmented+0x130>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c040b270 <__alloc_skb>:
  100.00 :	c040b270:       94 21 ff d0     stwu    r1,-48(r1)
    0.00 :	c040b274:       7c 08 02 a6     mflr    r0
    0.00 :	c040b278:       7d 80 00 26     mfcr    r12
    0.00 :	c040b27c:       90 01 00 34     stw     r0,52(r1)
    0.00 :	c040b280:       54 a0 07 fe     clrlwi  r0,r5,31
    0.00 :	c040b284:       2e 00 00 00     cmpwi   cr4,r0,0
    0.00 :	c040b288:       bf 81 00 20     stmw    r28,32(r1)
    0.00 :	c040b28c:       91 81 00 1c     stw     r12,28(r1)
    0.00 :	c040b290:       7c 7c 1b 78     mr      r28,r3
    0.00 :	c040b294:       7c 9e 23 78     mr      r30,r4
    0.00 :	c040b298:       41 92 01 18     beq-    cr4,c040b3b0 <__alloc_skb+0x140>
    0.00 :	c040b29c:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b2a0:       83 a9 9e 50     lwz     r29,-25008(r9)
    0.00 :	c040b2a4:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b2a8:       80 09 c4 3c     lwz     r0,-15300(r9)
    0.00 :	c040b2ac:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c040b2b0:       40 9e 01 24     bne-    cr7,c040b3d4 <__alloc_skb+0x164>
    0.00 :	c040b2b4:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b2b8:       57 c4 00 3c     rlwinm  r4,r30,0,0,30
    0.00 :	c040b2bc:       4b cd 04 25     bl      c00db6e0 <kmem_cache_alloc>
    0.00 :	c040b2c0:       7c 7f 1b 79     mr.     r31,r3
    0.00 :	c040b2c4:       41 82 00 c4     beq-    c040b388 <__alloc_skb+0x118>
    0.00 :	c040b2c8:       7c 00 f9 ec     dcbtst  r0,r31
    0.00 :	c040b2cc:       3b 9c 00 3f     addi    r28,r28,63
    0.00 :	c040b2d0:       7f c4 f3 78     mr      r4,r30
    0.00 :	c040b2d4:       57 83 00 32     rlwinm  r3,r28,0,0,25
    0.00 :	c040b2d8:       38 a1 00 08     addi    r5,r1,8
    0.00 :	c040b2dc:       38 63 00 c0     addi    r3,r3,192
    0.00 :	c040b2e0:       4b ff ff 01     bl      c040b1e0 <__kmalloc_reserve.clone.56>
    0.00 :	c040b2e4:       7c 7e 1b 79     mr.     r30,r3
    0.00 :	c040b2e8:       41 82 00 d8     beq-    c040b3c0 <__alloc_skb+0x150>
    0.00 :	c040b2ec:       4b cc ec 45     bl      c00d9f30 <ksize>
    0.00 :	c040b2f0:       3b 83 ff 40     addi    r28,r3,-192
    0.00 :	c040b2f4:       7f be e2 15     add.    r29,r30,r28
    0.00 :	c040b2f8:       41 82 00 08     beq-    c040b300 <__alloc_skb+0x90>
    0.00 :	c040b2fc:       7c 00 e9 ec     dcbtst  r0,r29
    0.00 :	c040b300:       7f e3 fb 78     mr      r3,r31
    0.00 :	c040b304:       38 80 00 00     li      r4,0
    0.00 :	c040b308:       38 a0 00 98     li      r5,152
    0.00 :	c040b30c:       3b 9c 01 80     addi    r28,r28,384
    0.00 :	c040b310:       4b c0 d5 9d     bl      c00188ac <memset>
    0.00 :	c040b314:       89 21 00 08     lbz     r9,8(r1)
    0.00 :	c040b318:       38 00 00 00     li      r0,0
    0.00 :	c040b31c:       93 9f 00 a8     stw     r28,168(r31)
    0.00 :	c040b320:       3b 80 00 01     li      r28,1
    0.00 :	c040b324:       51 20 6c a4     rlwimi  r0,r9,13,18,18
    0.00 :	c040b328:       90 1f 00 78     stw     r0,120(r31)
    0.00 :	c040b32c:       93 9f 00 ac     stw     r28,172(r31)
    0.00 :	c040b330:       93 df 00 a0     stw     r30,160(r31)
    0.00 :	c040b334:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b338:       93 df 00 a4     stw     r30,164(r31)
    0.00 :	c040b33c:       38 80 00 00     li      r4,0
    0.00 :	c040b340:       93 df 00 98     stw     r30,152(r31)
    0.00 :	c040b344:       38 a0 00 24     li      r5,36
    0.00 :	c040b348:       93 bf 00 9c     stw     r29,156(r31)
    0.00 :	c040b34c:       4b c0 d5 61     bl      c00188ac <memset>
    0.00 :	c040b350:       93 9d 00 24     stw     r28,36(r29)
    0.00 :	c040b354:       41 92 00 34     beq-    cr4,c040b388 <__alloc_skb+0x118>
    0.00 :	c040b358:       80 1f 00 64     lwz     r0,100(r31)
    0.00 :	c040b35c:       53 80 9a d8     rlwimi  r0,r28,19,11,12
    0.00 :	c040b360:       90 1f 00 64     stw     r0,100(r31)
    0.00 :	c040b364:       93 9f 01 60     stw     r28,352(r31)
    0.00 :	c040b368:       81 7f 01 14     lwz     r11,276(r31)
    0.00 :	c040b36c:       39 40 00 00     li      r10,0
    0.00 :	c040b370:       80 1f 01 28     lwz     r0,296(r31)
    0.00 :	c040b374:       51 4b 9a d8     rlwimi  r11,r10,19,11,12
    0.00 :	c040b378:       91 7f 01 14     stw     r11,276(r31)
    0.00 :	c040b37c:       89 61 00 08     lbz     r11,8(r1)
    0.00 :	c040b380:       51 60 6c a4     rlwimi  r0,r11,13,18,18
    0.00 :	c040b384:       90 1f 01 28     stw     r0,296(r31)
    0.00 :	c040b388:       80 01 00 34     lwz     r0,52(r1)
    0.00 :	c040b38c:       7f e3 fb 78     mr      r3,r31
    0.00 :	c040b390:       81 81 00 1c     lwz     r12,28(r1)
    0.00 :	c040b394:       bb 81 00 20     lmw     r28,32(r1)
    0.00 :	c040b398:       38 21 00 30     addi    r1,r1,48
    0.00 :	c040b39c:       7c 08 03 a6     mtlr    r0
    0.00 :	c040b3a0:       7d 80 81 20     mtcrf   8,r12
    0.00 :	c040b3a4:       4e 80 00 20     blr
    0.00 :	c040b3a8:       60 00 00 00     nop
    0.00 :	c040b3ac:       60 00 00 00     nop
    0.00 :	c040b3b0:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b3b4:       83 a9 9e 4c     lwz     r29,-25012(r9)
    0.00 :	c040b3b8:       4b ff fe ec     b       c040b2a4 <__alloc_skb+0x34>
    0.00 :	c040b3bc:       60 00 00 00     nop
    0.00 :	c040b3c0:       7f e4 fb 78     mr      r4,r31
    0.00 :	c040b3c4:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b3c8:       4b cc f0 c9     bl      c00da490 <kmem_cache_free>
    0.00 :	c040b3cc:       3b e0 00 00     li      r31,0
    0.00 :	c040b3d0:       4b ff ff b8     b       c040b388 <__alloc_skb+0x118>
    0.00 :	c040b3d4:       70 a0 00 02     andi.   r0,r5,2
    0.00 :	c040b3d8:       63 c0 20 00     ori     r0,r30,8192
    0.00 :	c040b3dc:       7f de 00 9e     .long 0x7fde009e
    0.00 :	c040b3e0:       4b ff fe d4     b       c040b2b4 <__alloc_skb+0x44>

[-- Attachment #3: report --]
[-- Type: text/plain, Size: 525 bytes --]

# Events: 8  cycles
#
# Overhead  Command      Shared Object                  Symbol
# ........  .......  .................  ......................
#
    97.04%     find  [kernel.kallsyms]  [k] __alloc_skb
     2.32%     find  [kernel.kallsyms]  [k] mod_timer
     0.62%     find  [kernel.kallsyms]  [k] __rb_insert_augmented
     0.01%     find  [kernel.kallsyms]  [k] perf_event_comm_output
     0.00%     find  [kernel.kallsyms]  [k] perf_event_comm


#
# (For a higher level overview, try: perf report --sort comm,dso)
#

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-19 22:59             ` Scott Wood
@ 2013-07-23 13:30               ` Michael Ellerman
  -1 siblings, 0 replies; 28+ messages in thread
From: Michael Ellerman @ 2013-07-23 13:30 UTC (permalink / raw)
  To: Scott Wood
  Cc: Anton Blanchard, Neil Horman, Rusty Russell, linux-kernel,
	Paul Mackerras, linuxppc-dev

On Fri, Jul 19, 2013 at 05:59:30PM -0500, Scott Wood wrote:
> On 07/17/2013 11:00:45 PM, Anton Blanchard wrote:
> >
> >Hi Scott,
> >
> >> What specifically should I do to test it?
> >
> >Could you double check perf annotate works? I'm 99% sure it will but
> >that is what was failing on ppc64.
> 
> I'm not really sure what it's supposed to look like when "perf
> annotate" works.  It spits a bunch of unreadable[1]
> dark-blue-on-black assembly code at me, all with "0.00 :" in the
> left column.
> 
> Oh, wait -- some lines have "100.00 : " on the left, in
> even-more-unreadable dark-red-on-black.
> 
> Apart from the annoying colors, is there anything specific I should
> be looking for?  Some sort of error message, or output that actually
> makes sense?

The colours look fine on my terminal, so I don't know what you've done
there. If you care you can use "--stdio" to use the plainer interface,
though it still uses colours.

That output looks fine in terms of the bug Anton was chasing. As far as
only ever hitting one instruction that does look weird.

cheers

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-23 13:30               ` Michael Ellerman
  0 siblings, 0 replies; 28+ messages in thread
From: Michael Ellerman @ 2013-07-23 13:30 UTC (permalink / raw)
  To: Scott Wood
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras,
	Anton Blanchard, linuxppc-dev

On Fri, Jul 19, 2013 at 05:59:30PM -0500, Scott Wood wrote:
> On 07/17/2013 11:00:45 PM, Anton Blanchard wrote:
> >
> >Hi Scott,
> >
> >> What specifically should I do to test it?
> >
> >Could you double check perf annotate works? I'm 99% sure it will but
> >that is what was failing on ppc64.
> 
> I'm not really sure what it's supposed to look like when "perf
> annotate" works.  It spits a bunch of unreadable[1]
> dark-blue-on-black assembly code at me, all with "0.00 :" in the
> left column.
> 
> Oh, wait -- some lines have "100.00 : " on the left, in
> even-more-unreadable dark-red-on-black.
> 
> Apart from the annoying colors, is there anything specific I should
> be looking for?  Some sort of error message, or output that actually
> makes sense?

The colours look fine on my terminal, so I don't know what you've done
there. If you care you can use "--stdio" to use the plainer interface,
though it still uses colours.

That output looks fine in terms of the bug Anton was chasing. As far as
only ever hitting one instruction that does look weird.

cheers

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-23 13:30               ` Michael Ellerman
@ 2013-07-24 19:22                 ` Scott Wood
  -1 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-24 19:22 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Anton Blanchard, Neil Horman, Rusty Russell, linux-kernel,
	Paul Mackerras, linuxppc-dev

On 07/23/2013 08:30:32 AM, Michael Ellerman wrote:
> On Fri, Jul 19, 2013 at 05:59:30PM -0500, Scott Wood wrote:
> > On 07/17/2013 11:00:45 PM, Anton Blanchard wrote:
> > >
> > >Hi Scott,
> > >
> > >> What specifically should I do to test it?
> > >
> > >Could you double check perf annotate works? I'm 99% sure it will  
> but
> > >that is what was failing on ppc64.
> >
> > I'm not really sure what it's supposed to look like when "perf
> > annotate" works.  It spits a bunch of unreadable[1]
> > dark-blue-on-black assembly code at me, all with "0.00 :" in the
> > left column.
> >
> > Oh, wait -- some lines have "100.00 : " on the left, in
> > even-more-unreadable dark-red-on-black.
> >
> > Apart from the annoying colors, is there anything specific I should
> > be looking for?  Some sort of error message, or output that actually
> > makes sense?
> 
> The colours look fine on my terminal, so I don't know what you've done
> there.

It probably looks better if the terminal is configured to have a light  
background (which of course makes some other programs look worse), or  
(as I noted) if you've got your monitor set to be very bright.  I now  
see that xfce4-terminal lets me redefine the standard colors, though,  
so that should help.

> If you care you can use "--stdio" to use the plainer interface,
> though it still uses colours.
> 
> That output looks fine in terms of the bug Anton was chasing. As far  
> as
> only ever hitting one instruction that does look weird.

OK.  I'll add "investigate weird e500 perf annotate results" to the  
TODO list...

-Scott

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-24 19:22                 ` Scott Wood
  0 siblings, 0 replies; 28+ messages in thread
From: Scott Wood @ 2013-07-24 19:22 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras,
	Anton Blanchard, linuxppc-dev

On 07/23/2013 08:30:32 AM, Michael Ellerman wrote:
> On Fri, Jul 19, 2013 at 05:59:30PM -0500, Scott Wood wrote:
> > On 07/17/2013 11:00:45 PM, Anton Blanchard wrote:
> > >
> > >Hi Scott,
> > >
> > >> What specifically should I do to test it?
> > >
> > >Could you double check perf annotate works? I'm 99% sure it will =20
> but
> > >that is what was failing on ppc64.
> >
> > I'm not really sure what it's supposed to look like when "perf
> > annotate" works.  It spits a bunch of unreadable[1]
> > dark-blue-on-black assembly code at me, all with "0.00 :" in the
> > left column.
> >
> > Oh, wait -- some lines have "100.00 : " on the left, in
> > even-more-unreadable dark-red-on-black.
> >
> > Apart from the annoying colors, is there anything specific I should
> > be looking for?  Some sort of error message, or output that actually
> > makes sense?
>=20
> The colours look fine on my terminal, so I don't know what you've done
> there.

It probably looks better if the terminal is configured to have a light =20
background (which of course makes some other programs look worse), or =20
(as I noted) if you've got your monitor set to be very bright.  I now =20
see that xfce4-terminal lets me redefine the standard colors, though, =20
so that should help.

> If you care you can use "--stdio" to use the plainer interface,
> though it still uses colours.
>=20
> That output looks fine in terms of the bug Anton was chasing. As far =20
> as
> only ever hitting one instruction that does look weird.

OK.  I'll add "investigate weird e500 perf annotate results" to the =20
TODO list...

-Scott=

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-19 22:59             ` Scott Wood
@ 2013-07-24 22:34               ` Anton Blanchard
  -1 siblings, 0 replies; 28+ messages in thread
From: Anton Blanchard @ 2013-07-24 22:34 UTC (permalink / raw)
  To: Scott Wood
  Cc: Benjamin Herrenschmidt, Paul Mackerras, Rusty Russell,
	Neil Horman, linuxppc-dev, linux-kernel


Hi Scott,

> I'm not really sure what it's supposed to look like when "perf  
> annotate" works.  It spits a bunch of unreadable[1]
> dark-blue-on-black assembly code at me, all with "0.00 :" in the left
> column.
> 
> Oh, wait -- some lines have "100.00 : " on the left, in  
> even-more-unreadable dark-red-on-black.
> 
> Apart from the annoying colors, is there anything specific I should
> be looking for?  Some sort of error message, or output that actually
> makes sense?

Thanks for testing! Ben, I think the patch is good to go.

Anton

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-24 22:34               ` Anton Blanchard
  0 siblings, 0 replies; 28+ messages in thread
From: Anton Blanchard @ 2013-07-24 22:34 UTC (permalink / raw)
  To: Scott Wood
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras, linuxppc-dev


Hi Scott,

> I'm not really sure what it's supposed to look like when "perf  
> annotate" works.  It spits a bunch of unreadable[1]
> dark-blue-on-black assembly code at me, all with "0.00 :" in the left
> column.
> 
> Oh, wait -- some lines have "100.00 : " on the left, in  
> even-more-unreadable dark-red-on-black.
> 
> Apart from the annoying colors, is there anything specific I should
> be looking for?  Some sort of error message, or output that actually
> makes sense?

Thanks for testing! Ben, I think the patch is good to go.

Anton

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-24 22:34               ` Anton Blanchard
@ 2013-07-24 23:14                 ` Benjamin Herrenschmidt
  -1 siblings, 0 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2013-07-24 23:14 UTC (permalink / raw)
  To: Anton Blanchard
  Cc: Scott Wood, Paul Mackerras, Rusty Russell, Neil Horman,
	linuxppc-dev, linux-kernel

On Thu, 2013-07-25 at 08:34 +1000, Anton Blanchard wrote:
> > Apart from the annoying colors, is there anything specific I should
> > be looking for?  Some sort of error message, or output that actually
> > makes sense?
> 
> Thanks for testing! Ben, I think the patch is good to go.

Sent it yesterday to Linus, it's upstream already :-)

Cheers,
Ben.



^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-24 23:14                 ` Benjamin Herrenschmidt
  0 siblings, 0 replies; 28+ messages in thread
From: Benjamin Herrenschmidt @ 2013-07-24 23:14 UTC (permalink / raw)
  To: Anton Blanchard
  Cc: Neil Horman, Rusty Russell, linux-kernel, Paul Mackerras,
	Scott Wood, linuxppc-dev

On Thu, 2013-07-25 at 08:34 +1000, Anton Blanchard wrote:
> > Apart from the annoying colors, is there anything specific I should
> > be looking for?  Some sort of error message, or output that actually
> > makes sense?
> 
> Thanks for testing! Ben, I think the patch is good to go.

Sent it yesterday to Linus, it's upstream already :-)

Cheers,
Ben.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-24 23:14                 ` Benjamin Herrenschmidt
@ 2013-07-25 13:02                   ` Neil Horman
  -1 siblings, 0 replies; 28+ messages in thread
From: Neil Horman @ 2013-07-25 13:02 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: Anton Blanchard, Scott Wood, Paul Mackerras, Rusty Russell,
	linuxppc-dev, linux-kernel

On Thu, Jul 25, 2013 at 09:14:25AM +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2013-07-25 at 08:34 +1000, Anton Blanchard wrote:
> > > Apart from the annoying colors, is there anything specific I should
> > > be looking for?  Some sort of error message, or output that actually
> > > makes sense?
> > 
> > Thanks for testing! Ben, I think the patch is good to go.
> 
> Sent it yesterday to Linus, it's upstream already :-)
> 
> Cheers,
> Ben.
> 
Sorry I'm a bit late to the thread, I've ben swamped.  Has someone tested this
with kexec/kdump?  Thats why the origional patch was created, because when kexec
loads the kernel at a different physical address, the relocations messed with
the module crc's, and modules couldn't load during the kexec boot.  Assuming
that kernaddr_start gets set appropriately during boot, using PHYSICAL_START
should be fine, but I wanted to check, and don't currently have access to a
powerpc system to do so.
Neil

> 
> 
> 

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-25 13:02                   ` Neil Horman
  0 siblings, 0 replies; 28+ messages in thread
From: Neil Horman @ 2013-07-25 13:02 UTC (permalink / raw)
  To: Benjamin Herrenschmidt
  Cc: Rusty Russell, linux-kernel, Paul Mackerras, Anton Blanchard,
	Scott Wood, linuxppc-dev

On Thu, Jul 25, 2013 at 09:14:25AM +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2013-07-25 at 08:34 +1000, Anton Blanchard wrote:
> > > Apart from the annoying colors, is there anything specific I should
> > > be looking for?  Some sort of error message, or output that actually
> > > makes sense?
> > 
> > Thanks for testing! Ben, I think the patch is good to go.
> 
> Sent it yesterday to Linus, it's upstream already :-)
> 
> Cheers,
> Ben.
> 
Sorry I'm a bit late to the thread, I've ben swamped.  Has someone tested this
with kexec/kdump?  Thats why the origional patch was created, because when kexec
loads the kernel at a different physical address, the relocations messed with
the module crc's, and modules couldn't load during the kexec boot.  Assuming
that kernaddr_start gets set appropriately during boot, using PHYSICAL_START
should be fine, but I wanted to check, and don't currently have access to a
powerpc system to do so.
Neil

> 
> 
> 

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-25 13:02                   ` Neil Horman
@ 2013-07-26  1:19                     ` Anton Blanchard
  -1 siblings, 0 replies; 28+ messages in thread
From: Anton Blanchard @ 2013-07-26  1:19 UTC (permalink / raw)
  To: Neil Horman
  Cc: Benjamin Herrenschmidt, Scott Wood, Paul Mackerras,
	Rusty Russell, linuxppc-dev, linux-kernel


Hi Neil,

> Sorry I'm a bit late to the thread, I've ben swamped.  Has someone
> tested this with kexec/kdump?  Thats why the origional patch was
> created, because when kexec loads the kernel at a different physical
> address, the relocations messed with the module crc's, and modules
> couldn't load during the kexec boot.  Assuming that kernaddr_start
> gets set appropriately during boot, using PHYSICAL_START should be
> fine, but I wanted to check, and don't currently have access to a
> powerpc system to do so. Neil

I tested a relocatable kernel forced to run at a non zero physical
address (ie basically kdump). I verified CRCs were bad with your
original patch backed out, and were good with this patch applied.

Anton

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-26  1:19                     ` Anton Blanchard
  0 siblings, 0 replies; 28+ messages in thread
From: Anton Blanchard @ 2013-07-26  1:19 UTC (permalink / raw)
  To: Neil Horman
  Cc: Rusty Russell, linux-kernel, Paul Mackerras, Scott Wood, linuxppc-dev


Hi Neil,

> Sorry I'm a bit late to the thread, I've ben swamped.  Has someone
> tested this with kexec/kdump?  Thats why the origional patch was
> created, because when kexec loads the kernel at a different physical
> address, the relocations messed with the module crc's, and modules
> couldn't load during the kexec boot.  Assuming that kernaddr_start
> gets set appropriately during boot, using PHYSICAL_START should be
> fine, but I wanted to check, and don't currently have access to a
> powerpc system to do so. Neil

I tested a relocatable kernel forced to run at a non zero physical
address (ie basically kdump). I verified CRCs were bad with your
original patch backed out, and were good with this patch applied.

Anton

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
  2013-07-26  1:19                     ` Anton Blanchard
@ 2013-07-26 13:11                       ` Neil Horman
  -1 siblings, 0 replies; 28+ messages in thread
From: Neil Horman @ 2013-07-26 13:11 UTC (permalink / raw)
  To: Anton Blanchard
  Cc: Benjamin Herrenschmidt, Scott Wood, Paul Mackerras,
	Rusty Russell, linuxppc-dev, linux-kernel

On Fri, Jul 26, 2013 at 11:19:13AM +1000, Anton Blanchard wrote:
> 
> Hi Neil,
> 
> > Sorry I'm a bit late to the thread, I've ben swamped.  Has someone
> > tested this with kexec/kdump?  Thats why the origional patch was
> > created, because when kexec loads the kernel at a different physical
> > address, the relocations messed with the module crc's, and modules
> > couldn't load during the kexec boot.  Assuming that kernaddr_start
> > gets set appropriately during boot, using PHYSICAL_START should be
> > fine, but I wanted to check, and don't currently have access to a
> > powerpc system to do so. Neil
> 
> I tested a relocatable kernel forced to run at a non zero physical
> address (ie basically kdump). I verified CRCs were bad with your
> original patch backed out, and were good with this patch applied.
> 
> Anton
> 

Perfect, sounds like a sufficient test to me.

Acked-by: Neil Horman <nhorman@tuxdriver.com>


^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH] module: ppc64 module CRC relocation fix causes perf issues
@ 2013-07-26 13:11                       ` Neil Horman
  0 siblings, 0 replies; 28+ messages in thread
From: Neil Horman @ 2013-07-26 13:11 UTC (permalink / raw)
  To: Anton Blanchard
  Cc: Rusty Russell, linux-kernel, Paul Mackerras, Scott Wood, linuxppc-dev

On Fri, Jul 26, 2013 at 11:19:13AM +1000, Anton Blanchard wrote:
> 
> Hi Neil,
> 
> > Sorry I'm a bit late to the thread, I've ben swamped.  Has someone
> > tested this with kexec/kdump?  Thats why the origional patch was
> > created, because when kexec loads the kernel at a different physical
> > address, the relocations messed with the module crc's, and modules
> > couldn't load during the kexec boot.  Assuming that kernaddr_start
> > gets set appropriately during boot, using PHYSICAL_START should be
> > fine, but I wanted to check, and don't currently have access to a
> > powerpc system to do so. Neil
> 
> I tested a relocatable kernel forced to run at a non zero physical
> address (ie basically kdump). I verified CRCs were bad with your
> original patch backed out, and were good with this patch applied.
> 
> Anton
> 

Perfect, sounds like a sufficient test to me.

Acked-by: Neil Horman <nhorman@tuxdriver.com>

^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2013-07-26 13:11 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-07-15  4:04 [PATCH] module: ppc64 module CRC relocation fix causes perf issues Anton Blanchard
2013-07-15  4:39 ` Rusty Russell
2013-07-15  8:47 ` Benjamin Herrenschmidt
2013-07-15  8:47   ` Benjamin Herrenschmidt
2013-07-16 22:40   ` Scott Wood
2013-07-16 22:40     ` Scott Wood
2013-07-17  0:04     ` Benjamin Herrenschmidt
2013-07-17  0:04       ` Benjamin Herrenschmidt
2013-07-17  0:08       ` Scott Wood
2013-07-17  0:08         ` Scott Wood
2013-07-18  4:00         ` Anton Blanchard
2013-07-18  4:00           ` Anton Blanchard
2013-07-19 22:59           ` Scott Wood
2013-07-19 22:59             ` Scott Wood
2013-07-23 13:30             ` Michael Ellerman
2013-07-23 13:30               ` Michael Ellerman
2013-07-24 19:22               ` Scott Wood
2013-07-24 19:22                 ` Scott Wood
2013-07-24 22:34             ` Anton Blanchard
2013-07-24 22:34               ` Anton Blanchard
2013-07-24 23:14               ` Benjamin Herrenschmidt
2013-07-24 23:14                 ` Benjamin Herrenschmidt
2013-07-25 13:02                 ` Neil Horman
2013-07-25 13:02                   ` Neil Horman
2013-07-26  1:19                   ` Anton Blanchard
2013-07-26  1:19                     ` Anton Blanchard
2013-07-26 13:11                     ` Neil Horman
2013-07-26 13:11                       ` Neil Horman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.