From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757395AbbCCQuZ (ORCPT ); Tue, 3 Mar 2015 11:50:25 -0500 Received: from mail.kernel.org ([198.145.29.136]:60594 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757345AbbCCQto (ORCPT ); Tue, 3 Mar 2015 11:49:44 -0500 Date: Tue, 3 Mar 2015 13:49:40 -0300 From: Arnaldo Carvalho de Melo To: David Ahern , Namhyung Kim , Jiri Olsa , Ingo Molnar , =?iso-8859-1?Q?Fr=E9d=E9ric?= Weisbecker Cc: Linux Kernel Mailing List Subject: heads up/RFC: 'perf trace' using ordered_events Message-ID: <20150303164940.GM5187@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Url: http://acmel.wordpress.com User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, Just a preview, but this is something David had mentioned at some point, a major problem with 'perf trace' was that it wasn't using 'perf_session' event reordering mechanism, so I've been working on making it use it, refactoring the ordered_events code so that it can be used by tools that don't deal with perf.data files. I still have to investigate why there are so many sched_wakeup at the beginning, but probably they are really being generated by the kernel and 'perf trace' has to make this output more compact, perhaps noticing that a number of similar events appeared on the stream and instead of writing N lines, print some prefix ([N*] perhaps) and then the line. Also its keeping pointers to the ring buffer, probably we'll need to copy or store the perf_sample somehow in the 'ordered_event' instance. But as this is touching things that Namhyung is touching as well, I thought about sharing this stuff, its on my tree, branch tmp.perf/trace_ordered_events. Lunchtime, bbl! - Arnaldo [root@ssdandy ~]# trace --ev sched:* usleep 1 0.000 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 0.001 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 0.002 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 0.002 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 0.003 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 0.003 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 0.003 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 0.004 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=001) 1.367 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.368 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.369 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.369 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.369 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.370 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.370 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.371 ( ): sched:sched_wakeup:comm=trace pid=4677 prio=120 success=1 target_cpu=003) 1.615 ( ): sched:sched_process_exec:filename=/bin/usleep pid=4677 old_pid=4677) 1.621 ( ): sched:sched_wakeup:comm=rcuop/3 pid=16 prio=120 success=1 target_cpu=005) 1.640 ( 0.001 ms): usleep/4677 brk( ) = 0x64e000 1.662 ( 0.005 ms): usleep/4677 mmap(len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x7f3d1d7ef000 1.675 ( 0.005 ms): usleep/4677 access(filename: 0x7f3d1d5edbe0, mode: R ) = -1 ENOENT No such file or directory 1.687 ( 0.004 ms): usleep/4677 open(filename: 0x7f3d1d5ec4d8, flags: CLOEXEC ) = 3 1.689 ( 0.001 ms): usleep/4677 fstat(fd: 3, statbuf: 0x7fff28219d10 ) = 0 1.695 ( 0.005 ms): usleep/4677 mmap(len: 68635, prot: READ, flags: PRIVATE, fd: 3 ) = 0x7f3d1d7de000 1.696 ( 0.001 ms): usleep/4677 close(fd: 3 ) = 0 1.714 ( 0.007 ms): usleep/4677 open(filename: 0x7f3d1d7e867d, flags: CLOEXEC ) = 3 1.716 ( 0.002 ms): usleep/4677 read(fd: 3, buf: 0x7fff28219eb0, count: 832 ) = 832 1.719 ( 0.001 ms): usleep/4677 fstat(fd: 3, statbuf: 0x7fff28219d60 ) = 0 1.727 ( 0.006 ms): usleep/4677 mmap(len: 2135088, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3 ) = 0x7f3d1d3c6000 1.734 ( 0.007 ms): usleep/4677 mprotect(start: 0x7f3d1d3cf000, len: 2093056 ) = 0 1.741 ( 0.007 ms): usleep/4677 mmap(addr: 0x7f3d1d5ce000, len: 8192, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 32768) = 0x7f3d1d5ce000 1.748 ( 0.001 ms): usleep/4677 close(fd: 3 ) = 0 1.763 ( 0.006 ms): usleep/4677 open(filename: 0x7f3d1d7ed5fa, flags: CLOEXEC ) = 3 1.765 ( 0.002 ms): usleep/4677 read(fd: 3, buf: 0x7fff28219e80, count: 832 ) = 832 1.767 ( 0.001 ms): usleep/4677 fstat(fd: 3, statbuf: 0x7fff28219d30 ) = 0 1.775 ( 0.006 ms): usleep/4677 mmap(len: 3932736, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3 ) = 0x7f3d1d005000 1.782 ( 0.007 ms): usleep/4677 mprotect(start: 0x7f3d1d1bb000, len: 2097152 ) = 0 1.790 ( 0.008 ms): usleep/4677 mmap(addr: 0x7f3d1d3bb000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1794048) ... 1.790 ( ): sched:sched_stat_runtime:comm=usleep pid=4677 runtime=426124 [ns] vruntime=569867840059 [ns]) 1.793 ( ): sched:sched_wakeup:comm=rcuc/3 pid=56 prio=97 success=1 target_cpu=003) 1.794 ( ): sched:sched_wakeup:comm=ksoftirqd/3 pid=58 prio=98 success=1 target_cpu=003) 1.797 ( ): sched:sched_stat_runtime:comm=usleep pid=4677 runtime=2371 [ns] vruntime=569867842430 [ns]) 1.798 ( ): sched:sched_switch:prev_comm=usleep prev_pid=4677 prev_prio=120 prev_state=R+ ==> next_comm=rcuc/3 next_pid=56 next_prio=97) 1.832 ( 0.049 ms): usleep/4677 ... [continued]: mmap()) = 0x7f3d1d3bb000 1.844 ( 0.007 ms): usleep/4677 mmap(addr: 0x7f3d1d3c1000, len: 16960, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: -1) = 0x7f3d1d3c1000 1.849 ( 0.001 ms): usleep/4677 close(fd: 3 ) = 0 1.859 ( 0.005 ms): usleep/4677 mmap(len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x7f3d1d7dd000 1.873 ( 0.005 ms): usleep/4677 mmap(len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x7f3d1d7db000 1.879 ( 0.001 ms): usleep/4677 arch_prctl(option: 4098, arg2: 139900464510784, arg3: 139900464513104, arg4: 34, arg5: 4294967295) = 0 1.969 ( 0.009 ms): usleep/4677 mprotect(start: 0x7f3d1d3bb000, len: 16384, prot: READ ) = 0 1.982 ( 0.007 ms): usleep/4677 mprotect(start: 0x7f3d1d5ce000, len: 4096, prot: READ ) = 0 1.991 ( 0.006 ms): usleep/4677 mprotect(start: 0x601000, len: 4096, prot: READ ) = 0 2.001 ( 0.007 ms): usleep/4677 mprotect(start: 0x7f3d1d7f0000, len: 4096, prot: READ ) = 0 2.009 ( 0.007 ms): usleep/4677 munmap(addr: 0x7f3d1d7de000, len: 68635 ) = 0 2.049 ( ): sched:sched_stat_runtime:comm=usleep pid=4677 runtime=220367 [ns] vruntime=569868062797 [ns]) 2.050 ( ): sched:sched_switch:prev_comm=usleep prev_pid=4677 prev_prio=120 prev_state=R ==> next_comm=irq/51-enp0s25 next_pid=1164 next_prio=49) 2.097 ( ): sched:sched_stat_runtime:comm=usleep pid=4677 runtime=29987 [ns] vruntime=569868092784 [ns]) 2.099 ( ): sched:sched_switch:prev_comm=usleep prev_pid=4677 prev_prio=120 prev_state=R+ ==> next_comm=irq/51-enp0s25 next_pid=1164 next_prio=49) 2.121 ( 0.001 ms): usleep/4677 brk( ) = 0x64e000 2.126 ( 0.005 ms): usleep/4677 brk(brk: 0x66f000 ) = 0x66f000 2.129 ( 0.001 ms): usleep/4677 brk( ) = 0x66f000 2.148 ( ): sched:sched_stat_runtime:comm=usleep pid=4677 runtime=34634 [ns] vruntime=569868127418 [ns]) 2.149 ( ): sched:sched_switch:prev_comm=usleep prev_pid=4677 prev_prio=120 prev_state=R+ ==> next_comm=irq/51-enp0s25 next_pid=1164 next_prio=49) 2.173 ( 0.002 ms): usleep/4677 nanosleep(rqtp: 0x7fff2821a4e0 ) ... 2.173 ( ): sched:sched_stat_runtime:comm=usleep pid=4677 runtime=11440 [ns] vruntime=569868138858 [ns]) 2.174 ( ): sched:sched_switch:prev_comm=usleep prev_pid=4677 prev_prio=120 prev_state=S ==> next_comm=swapper/3 next_pid=0 next_prio=120) 2.227 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.228 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.228 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.229 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.230 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.230 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.230 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.231 ( ): sched:sched_wakeup:comm=usleep pid=4677 prio=120 success=1 target_cpu=003) 2.237 ( 0.067 ms): usleep/4677 ... [continued]: nanosleep()) = 0 2.243 ( 0.000 ms): usleep/4677 exit_group( 2.289 ( ): sched:sched_process_exit:comm=usleep pid=4677 prio=120) [root@ssdandy ~]#