Re: [PATCH perf/core 1/2] [BUGFIX] perf-probe: Fix to handle optimized not-inlined but has no instance

[Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>]

(2015/01/30 23:32), Arnaldo Carvalho de Melo wrote:
> Em Fri, Jan 30, 2015 at 11:21:35AM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Fri, Jan 30, 2015 at 06:37:44PM +0900, Masami Hiramatsu escreveu:
>>> Fix to handle optimized no-inline functions which have
>>> only function definition but no actual instance at
>>> that point. To fix this problem, we need to find actual
>>> instance of the function.
>>>
>>> Without this patch:
>>>   ----
>>>   # perf probe -a __up
>>>   Failed to get entry address of __up.
>>>     Error: Failed to add events.
>>>   # perf probe -L __up
>>>   Specified source line is not found.
>>>     Error: Failed to show lines.
>>>   ----
>>>
>>> With this patch:
>>>   ----
>>>   # perf probe -a __up
>>>   Added new event:
>>>     probe:__up           (on __up)
>>>
>>>   You can now use it in all perf tools, such as:
>>>
>>>           perf record -e probe:__up -aR sleep 1
>>>
>>>   # perf probe -L __up
>>>   <__up@/home/fedora/ksrc/linux-3/kernel/locking/semaphore.c:0>
>>>         0  static noinline void __sched __up(struct semaphore *sem)
>>>            {
>>>                   struct semaphore_waiter *waiter = list_first_entry(&sem->wait_
>>>                                                           struct semaphore_waite
>>>         4         list_del(&waiter->list);
>>>         5         waiter->up = true;
>>>         6         wake_up_process(waiter->task);
>>>         7  }
>>>   ----
>>
>> Since __up here was built in some other way, I looked for another
> 
> Nah, I saw you trying to add "perf probe -a __up" and stopped there,
> hadn't seen that it fails _as well_ for just listing the source code...
> 
> Yes, I managed to reproduce the problem and test your fix, it works:

Thanks for testing!

> 
>   [root@zoo ~]# perf probe --del __up
>   Removed event: probe:__up
>   [root@zoo ~]# perf probe -L __up
>   Specified source line is not found.
>     Error: Failed to show lines.
> 
> ---->>>>>> APPLY THE PATCH, rebuild and try again:
> 
>   [root@zoo ~]# 
>   [root@zoo ~]# perf probe -L __up
>   <__up@/home/git/linux/kernel/locking/semaphore.c:0>
>         0  static noinline void __sched __up(struct semaphore *sem)
>            {
>                   struct semaphore_waiter *waiter = list_first_entry(&sem->wait_list,
>                                                           struct semaphore_waiter, list);
>         4         list_del(&waiter->list);
>         5         waiter->up = true;
>         6         wake_up_process(waiter->task);
>         7  }
>   [root@zoo ~]#
> 
> --------------------------------------------------------------
> 
> So its just this other case that fails, vmalloc_fault, can you check
> this one? Does it happen in your machine as well?

OK, I'll try reproducing this and dig deeper.

> 
> Applying your patch, thanks,

Thank you!

> 
> - Arnaldo
> 
>> 'noinline' function to try, and it failed here:
>>
>>   [root@zoo ~]# perf probe -L vmalloc_fault | head -5
>>   <vmalloc_fault@/home/git/linux/arch/x86/mm/fault.c:0>
>>         0  static noinline int vmalloc_fault(unsigned long address)
>>         1  {
>>            	pgd_t *pgd, *pgd_ref;
>>            	pud_t *pud, *pud_ref;
>>   [root@zoo ~]# perf probe vmalloc_fault
>>   Added new event:
>>   Failed to write event: Invalid argument
>>     Error: Failed to add events.
>>
>> ---->>>>>> APPLY THE PATCH, rebuild and try again:
>>
>>   [root@zoo ~]# perf probe vmalloc_fault
>>   Added new event:
>>   Failed to write event: Invalid argument
>>     Error: Failed to add events.
>>   [root@zoo ~]# perf probe -v vmalloc_fault
>>   probe-definition(0): vmalloc_fault 
>>   symbol:vmalloc_fault file:(null) line:0 offset:0 return:0 lazy:(null)
>>   0 arguments
>>   Looking at the vmlinux_path (7 entries long)
>>   Using /lib/modules/3.19.0-rc6+/build/vmlinux for symbols
>>   Open Debuginfo file: /lib/modules/3.19.0-rc6+/build/vmlinux
>>   Try to find probe point from debuginfo.
>>   Probe point found: vmalloc_fault+0
>>   Found 1 probe_trace_events.
>>   Opening /sys/kernel/debug/tracing/kprobe_events write=1
>>   Added new event:
>>   Writing event: p:probe/vmalloc_fault _text+289600
>>   Failed to write event: Invalid argument
>>     Error: Failed to add events. Reason: Invalid argument (Code: -22)
>>   [root@zoo ~]# grep -w vmalloc_fault /proc/kallsyms
>>   ffffffff81046b40 t vmalloc_fault
>>   [root@zoo ~]# grep -w _text /proc/kallsyms 
>>   ffffffff81000000 T _text
>>   [root@zoo ~]# python 
>>   Python 2.7.5 (default, Nov  3 2014, 14:26:24) 
>>   [GCC 4.8.3 20140911 (Red Hat 4.8.3-7)] on linux2
>>   Type "help", "copyright", "credits" or "license" for more information.
>>   >>> 0xffffffff81046b40 - 0xffffffff81000000
>>   289600L
>>   >>> 
>>   [root@zoo ~]# readelf -s /lib/modules/3.19.0-rc6+/build/vmlinux | egrep -w _text\|vmalloc_fault
>>     3499: ffffffff81046b40   410 FUNC    LOCAL  DEFAULT    1 vmalloc_fault
>>    48873: ffffffff81000000     0 NOTYPE  GLOBAL DEFAULT    1 _text
>>   [root@zoo ~]# 
>>
>> ---------------------------------------------------------------------------
>>
>> So perhaps I need a 'noinline' _and_ '__sched', so that I have a "optimized
>> no-inline functions which have only function definition but no actual instance
>> at that point"?
>>
>> Looking for one, will post here after objdump -dS finishes...
>>
>> - Arnaldo
> 


-- 
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com