* perf & rasd integration plan @ 2014-09-30 9:06 Jean Pihet 2014-09-30 13:24 ` Arnaldo Carvalho de Melo ` (2 more replies) 0 siblings, 3 replies; 44+ messages in thread From: Jean Pihet @ 2014-09-30 9:06 UTC (permalink / raw) To: Borislav Petkov, linux-kernel Cc: Fu Wei, Robert Richter, Arnaldo Carvalho de Melo, Jiri Olsa, David Ahern, Ingo Molnar The RAS Deamon (rasd) as available at [1] and the development version at [2], allows to enable kernel tracepoints and outputs the tracepoints fields according to the kernel format. [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ [2] https://git.linaro.org/people/jean.pihet/rasd.git rasd currently is a duplicate of the perf tool code, ultimately perf and rasd will use the same common code. The goal is to factor out the common code from perf and place it in small librairies in tools/lib. Here is the code that rasd currently uses and that should be moved to tools/lib: - debugfs: already in tools/lib/api/fs . mount and retrieve path - evlist: tools/perf/util/evlist.[ch] . create and init new evlist, . set cpu and thread maps, . add events to evlist, . init and use internal event id, . alloc and mmap events buffers, manage file descriptors, . enable events, . read events buffers, parse data, . unmap and free buffers - evsel: tools/perf/util/evsel.[ch] . create and init new tracepoints events, . init and use internal event id, . open events, manage fds, . close and free events - trace-event: tools/perf/util/trace-event.[ch] and tools/perf/util/trace-event-parse.c . retrieve and parse events format (using event-parse), . print out events fields - event-parse: already in tools/lib/traceevent/event-parse.[ch] . retrieve and parse events format, . parse events format and print out events fields - trace-seq: already in tools/lib/traceevent/trace-seq.c . format output string for event fields - events plugins: already in tools/lib/traceevent/event-plugin.c - util: tools/perf/util/util.[ch] . files open/read, . manage events attributes, . various macros - test events attributes: tools/perf/tests/attr.c . test_attr__open() - thread: tools/perf/util/thread_map.[ch] and - cpu: tools/perf/util/cpumap.[ch] . init and manage process maps - xyarray: tools/perf/util/xyarray.[ch] - syscall: tools/perf/perf-sys.h - cgroup: tools/perf/util/cgroup.[ch] The plan is to move the small and generic functions first: util, xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, trace-event-parse; and finally integrate rasd into the tools/ dir. Any thought? Can evlist, evsel etc. be moved at once? Patches should come soon, when time allows. Regards, Jean ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-09-30 9:06 perf & rasd integration plan Jean Pihet @ 2014-09-30 13:24 ` Arnaldo Carvalho de Melo 2014-10-05 17:48 ` Borislav Petkov 2014-10-06 9:07 ` Robert Richter 2014-10-06 15:07 ` Arnaldo Carvalho de Melo 2014-10-06 21:26 ` [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded Arnaldo Carvalho de Melo 2 siblings, 2 replies; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-09-30 13:24 UTC (permalink / raw) To: Jean Pihet Cc: Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > The RAS Deamon (rasd) as available at [1] and the development version > at [2], allows to enable kernel tracepoints and outputs the > tracepoints fields according to the kernel format. > > [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ > [2] https://git.linaro.org/people/jean.pihet/rasd.git > > rasd currently is a duplicate of the perf tool code, ultimately perf > and rasd will use the same common code. The goal is to factor out the > common code from perf and place it in small librairies in tools/lib. > > Here is the code that rasd currently uses and that should be moved to tools/lib: > > - debugfs: already in tools/lib/api/fs > . mount and retrieve path > > - evlist: tools/perf/util/evlist.[ch] > . create and init new evlist, > . set cpu and thread maps, > . add events to evlist, > . init and use internal event id, > . alloc and mmap events buffers, manage file descriptors, > . enable events, > . read events buffers, parse data, > . unmap and free buffers > > - evsel: tools/perf/util/evsel.[ch] > . create and init new tracepoints events, > . init and use internal event id, > . open events, manage fds, > . close and free events > > - trace-event: tools/perf/util/trace-event.[ch] and > tools/perf/util/trace-event-parse.c > . retrieve and parse events format (using event-parse), > . print out events fields > > - event-parse: already in tools/lib/traceevent/event-parse.[ch] > . retrieve and parse events format, > . parse events format and print out events fields > > - trace-seq: already in tools/lib/traceevent/trace-seq.c > . format output string for event fields > > - events plugins: already in tools/lib/traceevent/event-plugin.c > > - util: tools/perf/util/util.[ch] > . files open/read, > . manage events attributes, > . various macros > > - test events attributes: tools/perf/tests/attr.c > . test_attr__open() > > - thread: tools/perf/util/thread_map.[ch] and > - cpu: tools/perf/util/cpumap.[ch] > . init and manage process maps > > - xyarray: tools/perf/util/xyarray.[ch] > > - syscall: tools/perf/perf-sys.h > > - cgroup: tools/perf/util/cgroup.[ch] > > The plan is to move the small and generic functions first: util, > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > trace-event-parse; and finally integrate rasd into the tools/ dir. > > Any thought? Can evlist, evsel etc. be moved at once? > > Patches should come soon, when time allows. Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just go on and add tools/perf/util/evlist.o et all to be linked directly, as a first step. Then, as a second step, we can create a tools/lib/perf/evlist.c having what is currently used by both tools/perf/ and tools/rasd/, i.e. what is proven to be useful for something other than perf. As the need arises, we go on moving things into tools/lib/perf/evlist.c et all from wherever it appeared first, be it from tools/rasd/, tools/perf/util/evlist.c or anywhere else. Initial rule being that once it is used by multiple tools living in tools/, then it deserves a place in tools/lib/perf/. Ditto for other stuff currently living in tools/perf/util/. - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-09-30 13:24 ` Arnaldo Carvalho de Melo @ 2014-10-05 17:48 ` Borislav Petkov 2014-10-05 18:24 ` Jiri Olsa 2014-10-06 9:07 ` Robert Richter 1 sibling, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-05 17:48 UTC (permalink / raw) To: Jiri Olsa Cc: Arnaldo Carvalho de Melo, Jean Pihet, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar Top-posting on purpose: Btw, jolsa, if you get your LCE proposal for the perf splitting approved, please post the time here so people can come. Thanks. On Tue, Sep 30, 2014 at 10:24:16AM -0300, Arnaldo Carvalho de Melo wrote: > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > > The RAS Deamon (rasd) as available at [1] and the development version > > at [2], allows to enable kernel tracepoints and outputs the > > tracepoints fields according to the kernel format. > > > > [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ > > [2] https://git.linaro.org/people/jean.pihet/rasd.git > > > > rasd currently is a duplicate of the perf tool code, ultimately perf > > and rasd will use the same common code. The goal is to factor out the > > common code from perf and place it in small librairies in tools/lib. > > > > Here is the code that rasd currently uses and that should be moved to tools/lib: > > > > - debugfs: already in tools/lib/api/fs > > . mount and retrieve path > > > > - evlist: tools/perf/util/evlist.[ch] > > . create and init new evlist, > > . set cpu and thread maps, > > . add events to evlist, > > . init and use internal event id, > > . alloc and mmap events buffers, manage file descriptors, > > . enable events, > > . read events buffers, parse data, > > . unmap and free buffers > > > > - evsel: tools/perf/util/evsel.[ch] > > . create and init new tracepoints events, > > . init and use internal event id, > > . open events, manage fds, > > . close and free events > > > > - trace-event: tools/perf/util/trace-event.[ch] and > > tools/perf/util/trace-event-parse.c > > . retrieve and parse events format (using event-parse), > > . print out events fields > > > > - event-parse: already in tools/lib/traceevent/event-parse.[ch] > > . retrieve and parse events format, > > . parse events format and print out events fields > > > > - trace-seq: already in tools/lib/traceevent/trace-seq.c > > . format output string for event fields > > > > - events plugins: already in tools/lib/traceevent/event-plugin.c > > > > - util: tools/perf/util/util.[ch] > > . files open/read, > > . manage events attributes, > > . various macros > > > > - test events attributes: tools/perf/tests/attr.c > > . test_attr__open() > > > > - thread: tools/perf/util/thread_map.[ch] and > > - cpu: tools/perf/util/cpumap.[ch] > > . init and manage process maps > > > > - xyarray: tools/perf/util/xyarray.[ch] > > > > - syscall: tools/perf/perf-sys.h > > > > - cgroup: tools/perf/util/cgroup.[ch] > > > > The plan is to move the small and generic functions first: util, > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > > trace-event-parse; and finally integrate rasd into the tools/ dir. > > > > Any thought? Can evlist, evsel etc. be moved at once? > > > > Patches should come soon, when time allows. > > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just > go on and add tools/perf/util/evlist.o et all to be linked directly, as > a first step. > > Then, as a second step, we can create a tools/lib/perf/evlist.c having > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is > proven to be useful for something other than perf. > > As the need arises, we go on moving things into tools/lib/perf/evlist.c > et all from wherever it appeared first, be it from tools/rasd/, > tools/perf/util/evlist.c or anywhere else. > > Initial rule being that once it is used by multiple tools living in > tools/, then it deserves a place in tools/lib/perf/. > > Ditto for other stuff currently living in tools/perf/util/. > > - Arnaldo > -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-05 17:48 ` Borislav Petkov @ 2014-10-05 18:24 ` Jiri Olsa 2014-10-05 18:45 ` Borislav Petkov ` (2 more replies) 0 siblings, 3 replies; 44+ messages in thread From: Jiri Olsa @ 2014-10-05 18:24 UTC (permalink / raw) To: Borislav Petkov Cc: Arnaldo Carvalho de Melo, Jean Pihet, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > Top-posting on purpose: > > Btw, jolsa, if you get your LCE proposal for the perf splitting > approved, please post the time here so people can come. yep, it got accepted, ther schedule is: Friday, October 17, 2014 from 9:00am – Noon Room 2 https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 jirka ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-05 18:24 ` Jiri Olsa @ 2014-10-05 18:45 ` Borislav Petkov 2014-10-05 19:24 ` Chuck Ebbert 2014-10-06 6:53 ` Jean Pihet 2 siblings, 0 replies; 44+ messages in thread From: Borislav Petkov @ 2014-10-05 18:45 UTC (permalink / raw) To: Jiri Olsa Cc: Arnaldo Carvalho de Melo, Jean Pihet, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Sun, Oct 05, 2014 at 08:24:42PM +0200, Jiri Olsa wrote: > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > > Top-posting on purpose: > > > > Btw, jolsa, if you get your LCE proposal for the perf splitting > > approved, please post the time here so people can come. > > yep, it got accepted, ther schedule is: > Friday, October 17, 2014 from 9:00am – Noon > Room 2 > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 Cool, thanks. I'll add it to the calendar - lot to do this conf :-) -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-05 18:24 ` Jiri Olsa 2014-10-05 18:45 ` Borislav Petkov @ 2014-10-05 19:24 ` Chuck Ebbert 2014-10-05 19:28 ` Jiri Olsa 2014-10-06 6:53 ` Jean Pihet 2 siblings, 1 reply; 44+ messages in thread From: Chuck Ebbert @ 2014-10-05 19:24 UTC (permalink / raw) To: Jiri Olsa Cc: Borislav Petkov, Arnaldo Carvalho de Melo, Jean Pihet, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Sun, 5 Oct 2014 20:24:42 +0200 Jiri Olsa <jolsa@redhat.com> wrote: > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > > Top-posting on purpose: > > > > Btw, jolsa, if you get your LCE proposal for the perf splitting > > approved, please post the time here so people can come. > > yep, it got accepted, ther schedule is: > Friday, October 17, 2014 from 9:00am – Noon > Room 2 > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 > The SSL cert for that site is comic. 1. Expired over three years ago. 2. Was issued to a different domain. 3. Issued by an un-verifiable CA. This is a joke, right? ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-05 19:24 ` Chuck Ebbert @ 2014-10-05 19:28 ` Jiri Olsa 0 siblings, 0 replies; 44+ messages in thread From: Jiri Olsa @ 2014-10-05 19:28 UTC (permalink / raw) To: Chuck Ebbert Cc: Borislav Petkov, Arnaldo Carvalho de Melo, Jean Pihet, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Sun, Oct 05, 2014 at 02:24:01PM -0500, Chuck Ebbert wrote: > On Sun, 5 Oct 2014 20:24:42 +0200 > Jiri Olsa <jolsa@redhat.com> wrote: > > > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > > > Top-posting on purpose: > > > > > > Btw, jolsa, if you get your LCE proposal for the perf splitting > > > approved, please post the time here so people can come. > > > > yep, it got accepted, ther schedule is: > > Friday, October 17, 2014 from 9:00am – Noon > > Room 2 > > > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 > > > > The SSL cert for that site is comic. > > 1. Expired over three years ago. > > 2. Was issued to a different domain. > > 3. Issued by an un-verifiable CA. > > This is a joke, right? ;-) it's the link from this page: http://www.linuxplumbersconf.org/2014/ocw/events/LPC2014/schedule jirka ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-05 18:24 ` Jiri Olsa 2014-10-05 18:45 ` Borislav Petkov 2014-10-05 19:24 ` Chuck Ebbert @ 2014-10-06 6:53 ` Jean Pihet 2014-10-08 6:59 ` Jiri Olsa 2 siblings, 1 reply; 44+ messages in thread From: Jean Pihet @ 2014-10-06 6:53 UTC (permalink / raw) To: Jiri Olsa Cc: Borislav Petkov, Arnaldo Carvalho de Melo, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar Hi Jiri, On 5 October 2014 20:24, Jiri Olsa <jolsa@redhat.com> wrote: > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: >> Top-posting on purpose: >> >> Btw, jolsa, if you get your LCE proposal for the perf splitting >> approved, please post the time here so people can come. > > yep, it got accepted, ther schedule is: > Friday, October 17, 2014 from 9:00am – Noon > Room 2 > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 Great! Are there slides that we can download? I have a couple of weeks to work on this, I can prepare the first set of patches. Note: I will not be at ELCE/LPC but could attend remotely or have a call to discuss the points. Cheers, Jean > > jirka ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 6:53 ` Jean Pihet @ 2014-10-08 6:59 ` Jiri Olsa 0 siblings, 0 replies; 44+ messages in thread From: Jiri Olsa @ 2014-10-08 6:59 UTC (permalink / raw) To: Jean Pihet Cc: Borislav Petkov, Arnaldo Carvalho de Melo, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Mon, Oct 06, 2014 at 08:53:51AM +0200, Jean Pihet wrote: > Hi Jiri, > > > On 5 October 2014 20:24, Jiri Olsa <jolsa@redhat.com> wrote: > > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > >> Top-posting on purpose: > >> > >> Btw, jolsa, if you get your LCE proposal for the perf splitting > >> approved, please post the time here so people can come. > > > > yep, it got accepted, ther schedule is: > > Friday, October 17, 2014 from 9:00am – Noon > > Room 2 > > > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 > Great! Are there slides that we can download? there'll be just one.. I'm on vacation this week, so I'll try send some till end of the week jirka ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-09-30 13:24 ` Arnaldo Carvalho de Melo 2014-10-05 17:48 ` Borislav Petkov @ 2014-10-06 9:07 ` Robert Richter 2014-10-06 13:44 ` Jean Pihet 2014-10-06 14:58 ` Arnaldo Carvalho de Melo 1 sibling, 2 replies; 44+ messages in thread From: Robert Richter @ 2014-10-06 9:07 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, Borislav Petkov, linux-kernel, Fu Wei, Jiri Olsa, David Ahern, Ingo Molnar On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > > The plan is to move the small and generic functions first: util, > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > > trace-event-parse; and finally integrate rasd into the tools/ dir. > > > > Any thought? Can evlist, evsel etc. be moved at once? > > > > Patches should come soon, when time allows. > > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just > go on and add tools/perf/util/evlist.o et all to be linked directly, as > a first step. > > Then, as a second step, we can create a tools/lib/perf/evlist.c having > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is > proven to be useful for something other than perf. It would be good to have tools/lib/perf or so with some base implementation to setup and connect to perf buffers. This is useful for tools not only in tools/. The rasd would be a good reference for this regardless if it is in tools/ or not. I am not sure whether and when rasd will be moved there. > As the need arises, we go on moving things into tools/lib/perf/evlist.c > et all from wherever it appeared first, be it from tools/rasd/, > tools/perf/util/evlist.c or anywhere else. > > Initial rule being that once it is used by multiple tools living in > tools/, then it deserves a place in tools/lib/perf/. So this wouldn't quite work well as it excludes tools not in tools/. -Robert > > Ditto for other stuff currently living in tools/perf/util/. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 9:07 ` Robert Richter @ 2014-10-06 13:44 ` Jean Pihet 2014-10-06 14:58 ` Arnaldo Carvalho de Melo 1 sibling, 0 replies; 44+ messages in thread From: Jean Pihet @ 2014-10-06 13:44 UTC (permalink / raw) To: Robert Richter, Arnaldo Carvalho de Melo, Borislav Petkov, Jiri Olsa Cc: linux-kernel, Fu Wei, David Ahern, Ingo Molnar Hi, On 6 October 2014 11:07, Robert Richter <rric@kernel.org> wrote: > On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: >> Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: >> > The plan is to move the small and generic functions first: util, >> > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, >> > trace-event-parse; and finally integrate rasd into the tools/ dir. >> > >> > Any thought? Can evlist, evsel etc. be moved at once? >> > >> > Patches should come soon, when time allows. >> >> Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just >> go on and add tools/perf/util/evlist.o et all to be linked directly, as >> a first step. >> >> Then, as a second step, we can create a tools/lib/perf/evlist.c having >> what is currently used by both tools/perf/ and tools/rasd/, i.e. what is >> proven to be useful for something other than perf. > > It would be good to have tools/lib/perf or so with some base > implementation to setup and connect to perf buffers. This is useful > for tools not only in tools/. The rasd would be a good reference for > this regardless if it is in tools/ or not. I am not sure whether and > when rasd will be moved there. Agree. Having a lib (or a set of libs) that external tools can use is a must. This will encourage tools developers to use this API instead of re-invening the wheel every time, and to provide standard tools that can be merged in the kernel source if needed. > >> As the need arises, we go on moving things into tools/lib/perf/evlist.c >> et all from wherever it appeared first, be it from tools/rasd/, >> tools/perf/util/evlist.c or anywhere else. >> >> Initial rule being that once it is used by multiple tools living in >> tools/, then it deserves a place in tools/lib/perf/. > > So this wouldn't quite work well as it excludes tools not in tools/. Ditto I am willing to propose a first set of patches to factor out the common code from perf, but an agreement is needed on the direction to take. My plan is to provide patches for the proposed integration plan (as in the initial e-mail), is it worth doing so or is it purely wasted time and effort? What do you think? Thx, Jean > > -Robert > >> >> Ditto for other stuff currently living in tools/perf/util/. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 9:07 ` Robert Richter 2014-10-06 13:44 ` Jean Pihet @ 2014-10-06 14:58 ` Arnaldo Carvalho de Melo 2014-10-06 15:01 ` Borislav Petkov 2014-10-06 15:02 ` Jean Pihet 1 sibling, 2 replies; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-06 14:58 UTC (permalink / raw) To: Robert Richter Cc: Jean Pihet, Borislav Petkov, linux-kernel, Fu Wei, Jiri Olsa, David Ahern, Ingo Molnar Em Mon, Oct 06, 2014 at 11:07:26AM +0200, Robert Richter escreveu: > On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: > > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > > > The plan is to move the small and generic functions first: util, > > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > > > trace-event-parse; and finally integrate rasd into the tools/ dir. > > > Any thought? Can evlist, evsel etc. be moved at once? > > > Patches should come soon, when time allows. > > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just > > go on and add tools/perf/util/evlist.o et all to be linked directly, as > > a first step. > > Then, as a second step, we can create a tools/lib/perf/evlist.c having > > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is > > proven to be useful for something other than perf. > It would be good to have tools/lib/perf or so with some base > implementation to setup and connect to perf buffers. This is useful > for tools not only in tools/. The rasd would be a good reference for Agreed, we can use rasd as a reference, I'm now looking for the git repo url to see what it is using to prototype how I think this should be done. - Arnaldo > this regardless if it is in tools/ or not. I am not sure whether and > when rasd will be moved there. - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 14:58 ` Arnaldo Carvalho de Melo @ 2014-10-06 15:01 ` Borislav Petkov 2014-10-06 15:08 ` Arnaldo Carvalho de Melo 2014-10-06 15:02 ` Jean Pihet 1 sibling, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-06 15:01 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Robert Richter, Jean Pihet, linux-kernel, Fu Wei, Jiri Olsa, David Ahern, Ingo Molnar On Mon, Oct 06, 2014 at 11:58:56AM -0300, Arnaldo Carvalho de Melo wrote: > Agreed, we can use rasd as a reference, I'm now looking for the git repo > url to see what it is using to prototype how I think this should be > done. https://git.kernel.org/cgit/utils/ras/rasd.git -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 15:01 ` Borislav Petkov @ 2014-10-06 15:08 ` Arnaldo Carvalho de Melo 2014-10-06 15:16 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-06 15:08 UTC (permalink / raw) To: Borislav Petkov Cc: Robert Richter, Jean Pihet, linux-kernel, Fu Wei, Jiri Olsa, David Ahern, Ingo Molnar Em Mon, Oct 06, 2014 at 05:01:47PM +0200, Borislav Petkov escreveu: > On Mon, Oct 06, 2014 at 11:58:56AM -0300, Arnaldo Carvalho de Melo wrote: > > Agreed, we can use rasd as a reference, I'm now looking for the git repo > > url to see what it is using to prototype how I think this should be > > done. > > https://git.kernel.org/cgit/utils/ras/rasd.git I got Jean's "development" repo, should I get the g.k.o one? - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 15:08 ` Arnaldo Carvalho de Melo @ 2014-10-06 15:16 ` Borislav Petkov 0 siblings, 0 replies; 44+ messages in thread From: Borislav Petkov @ 2014-10-06 15:16 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Robert Richter, Jean Pihet, linux-kernel, Fu Wei, Jiri Olsa, David Ahern, Ingo Molnar On Mon, Oct 06, 2014 at 12:08:38PM -0300, Arnaldo Carvalho de Melo wrote: > Em Mon, Oct 06, 2014 at 05:01:47PM +0200, Borislav Petkov escreveu: > > On Mon, Oct 06, 2014 at 11:58:56AM -0300, Arnaldo Carvalho de Melo wrote: > > > Agreed, we can use rasd as a reference, I'm now looking for the git repo > > > url to see what it is using to prototype how I think this should be > > > done. > > > > https://git.kernel.org/cgit/utils/ras/rasd.git > > I got Jean's "development" repo, should I get the g.k.o one? Right, this one above is the official rasd one and Jean's stuff goes ontop. So you can look at both. :-) -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 14:58 ` Arnaldo Carvalho de Melo 2014-10-06 15:01 ` Borislav Petkov @ 2014-10-06 15:02 ` Jean Pihet 1 sibling, 0 replies; 44+ messages in thread From: Jean Pihet @ 2014-10-06 15:02 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Robert Richter, Borislav Petkov, linux-kernel, Fu Wei, Jiri Olsa, David Ahern, Ingo Molnar Hi Arnaldo, On 6 October 2014 16:58, Arnaldo Carvalho de Melo <acme@kernel.org> wrote: > Em Mon, Oct 06, 2014 at 11:07:26AM +0200, Robert Richter escreveu: >> On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: >> > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: >> > > The plan is to move the small and generic functions first: util, >> > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, >> > > trace-event-parse; and finally integrate rasd into the tools/ dir. > >> > > Any thought? Can evlist, evsel etc. be moved at once? > >> > > Patches should come soon, when time allows. > >> > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just >> > go on and add tools/perf/util/evlist.o et all to be linked directly, as >> > a first step. > >> > Then, as a second step, we can create a tools/lib/perf/evlist.c having >> > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is >> > proven to be useful for something other than perf. > >> It would be good to have tools/lib/perf or so with some base >> implementation to setup and connect to perf buffers. This is useful >> for tools not only in tools/. The rasd would be a good reference for > > Agreed, we can use rasd as a reference, I'm now looking for the git repo > url to see what it is using to prototype how I think this should be > done. The latest patches have been submitted to linux-edac [1] and also are available in the dev git tree [2]. [1] http://marc.info/?l=linux-edac&m=141234802311271&w=2 [2] https://git.linaro.org/people/jean.pihet/rasd.git Thx, Jean > > - Arnaldo > >> this regardless if it is in tools/ or not. I am not sure whether and >> when rasd will be moved there. > > - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-09-30 9:06 perf & rasd integration plan Jean Pihet 2014-09-30 13:24 ` Arnaldo Carvalho de Melo @ 2014-10-06 15:07 ` Arnaldo Carvalho de Melo 2014-10-06 15:16 ` Borislav Petkov 2014-10-06 21:26 ` [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded Arnaldo Carvalho de Melo 2 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-06 15:07 UTC (permalink / raw) To: Jean Pihet Cc: Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > The RAS Deamon (rasd) as available at [1] and the development version > at [2], allows to enable kernel tracepoints and outputs the > tracepoints fields according to the kernel format. > > [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ > [2] https://git.linaro.org/people/jean.pihet/rasd.git > > rasd currently is a duplicate of the perf tool code, ultimately perf > and rasd will use the same common code. The goal is to factor out the > common code from perf and place it in small librairies in tools/lib. > > Here is the code that rasd currently uses and that should be moved to tools/lib: > > - debugfs: already in tools/lib/api/fs > . mount and retrieve path > > - evlist: tools/perf/util/evlist.[ch] > . create and init new evlist, > . set cpu and thread maps, > . add events to evlist, > . init and use internal event id, > . alloc and mmap events buffers, manage file descriptors, > . enable events, > . read events buffers, parse data, > . unmap and free buffers > > - evsel: tools/perf/util/evsel.[ch] > . create and init new tracepoints events, > . init and use internal event id, > . open events, manage fds, > . close and free events So, why have you commented out the perf_missing_features fallbacks? Are they getting in the way somehow, what can we do upstream, i.e. in tools/perf/util/ so that it gets closer to what you want to have? Ditto for all the other ifdef'ed code, can you elaborate on why each is needed? That will help in moving what is used by rasd from tools/perf/util/ to tools/perf/lib/ so that we can eliminate the duplicity. - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 15:07 ` Arnaldo Carvalho de Melo @ 2014-10-06 15:16 ` Borislav Petkov 2014-10-06 19:12 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-06 15:16 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar On Mon, Oct 06, 2014 at 12:07:41PM -0300, Arnaldo Carvalho de Melo wrote: > So, why have you commented out the perf_missing_features fallbacks? Are > they getting in the way somehow, what can we do upstream, i.e. in > tools/perf/util/ so that it gets closer to what you want to have? > > Ditto for all the other ifdef'ed code, can you elaborate on why each is > needed? That will help in moving what is used by rasd from > tools/perf/util/ to tools/perf/lib/ so that we can eliminate the > duplicity. Oh, the reason is very simple: so that it builds. We don't need those facilities yet. We will enable them when needed but we wanted to concentrate on rasd first and later clean up stuff as we go. -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 15:16 ` Borislav Petkov @ 2014-10-06 19:12 ` Arnaldo Carvalho de Melo 2014-10-06 19:53 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-06 19:12 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Em Mon, Oct 06, 2014 at 05:16:06PM +0200, Borislav Petkov escreveu: > On Mon, Oct 06, 2014 at 12:07:41PM -0300, Arnaldo Carvalho de Melo wrote: > > So, why have you commented out the perf_missing_features fallbacks? Are > > they getting in the way somehow, what can we do upstream, i.e. in > > tools/perf/util/ so that it gets closer to what you want to have? > > > > Ditto for all the other ifdef'ed code, can you elaborate on why each is > > needed? That will help in moving what is used by rasd from > > tools/perf/util/ to tools/perf/lib/ so that we can eliminate the > > duplicity. > > Oh, the reason is very simple: so that it builds. We don't need those > facilities yet. We will enable them when needed but we wanted to > concentrate on rasd first and later clean up stuff as we go. Right, and it is being a great exercise, thanks for the patience so far ;-) Looking at those ifdefs we see things that are specific to tools/perf/, like perf_evsel having a struct hists embedded... I.e. that is of no interest whatsoever (so far) to rasd, and in turn pulls other object files. So I think that right now we need to look at those ifdefs and go on making what is in tools/perf/util/ stop using it somehow, so that what then gets moved to tools/lib/api/perf/ (I guess this is where it should be, opinions?) have that sorted out. I.e. what goes to tools/lib/api/perf/ is what is common to the needs of tools/perf/ and rasd (wherever it may roam). Perhaps something like I did for sock/inet_sock/inet_connection_sock ages ago... I.e. the tools that want a hists will have a struct hists_evsel { struct perf_evsel evsel; struct hists hists; }; etc. Experimenting with that, as it is the only thing ifdefed out in rasd's copy of evsel.h... - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 19:12 ` Arnaldo Carvalho de Melo @ 2014-10-06 19:53 ` Borislav Petkov 2014-10-06 21:22 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-06 19:53 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming On Mon, Oct 06, 2014 at 04:12:27PM -0300, Arnaldo Carvalho de Melo wrote: > Right, and it is being a great exercise, thanks for the patience so far > ;-) I know, this is your secret agenda to keep me at bay working on this! :-P > Looking at those ifdefs we see things that are specific to tools/perf/, > like perf_evsel having a struct hists embedded... I.e. that is of no > interest whatsoever (so far) to rasd, and in turn pulls other object > files. > > So I think that right now we need to look at those ifdefs and go on > making what is in tools/perf/util/ stop using it somehow, so that what > then gets moved to tools/lib/api/perf/ (I guess this is where it should > be, opinions?) have that sorted out. > > I.e. what goes to tools/lib/api/perf/ is what is common to the needs of > tools/perf/ and rasd (wherever it may roam). > > Perhaps something like I did for sock/inet_sock/inet_connection_sock > ages ago... I.e. the tools that want a hists will have a > > struct hists_evsel { > struct perf_evsel evsel; > struct hists hists; > }; > > etc. > > Experimenting with that, as it is the only thing ifdefed out in rasd's > copy of evsel.h... Right, this all reads like it is going in the right direction but the more important question IMO is are we doing a libperf or are we still doing tools/lib/api/perf/ of single object files to which people can link? Because if it is the second - and it sounds to me like it is - how do you propose we link with external projects? IOW, if I want to have rasd build *without* the kernel tree, is a simple cp -r tools/lib/api/ <my_project>/path/to/perf/lib/ work? I mean, I don't know and I'm just asking. Is that the proposed way? Are we fine with that? Do we want something different, maybe even a lib? Is it time for a lib even? Because the whole perf functionality is being cravingly ogled by other users - I know Matt wants it for cache QoS or whatever it is called and there are probably a whole lot of projects which would like to process events programatically in userspace. So what I'm saying is, we probably should have a nice clean way to be able to share that code with external projects instead of external projects duplicating and building a whole library around perf_event_open() and the likes... Just a couple of thoughts... -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 19:53 ` Borislav Petkov @ 2014-10-06 21:22 ` Arnaldo Carvalho de Melo 2014-10-07 11:23 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-06 21:22 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming Em Mon, Oct 06, 2014 at 09:53:49PM +0200, Borislav Petkov escreveu: > On Mon, Oct 06, 2014 at 04:12:27PM -0300, Arnaldo Carvalho de Melo wrote: > > Right, and it is being a great exercise, thanks for the patience so far > > ;-) > > I know, this is your secret agenda to keep me at bay working on this! :-P > > > Looking at those ifdefs we see things that are specific to tools/perf/, > > like perf_evsel having a struct hists embedded... I.e. that is of no > > interest whatsoever (so far) to rasd, and in turn pulls other object > > files. > > > > So I think that right now we need to look at those ifdefs and go on > > making what is in tools/perf/util/ stop using it somehow, so that what > > then gets moved to tools/lib/api/perf/ (I guess this is where it should > > be, opinions?) have that sorted out. > > I.e. what goes to tools/lib/api/perf/ is what is common to the needs of > > tools/perf/ and rasd (wherever it may roam). > > Perhaps something like I did for sock/inet_sock/inet_connection_sock > > ages ago... I.e. the tools that want a hists will have a > > struct hists_evsel { > > struct perf_evsel evsel; > > struct hists hists; > > }; > > etc. > > Experimenting with that, as it is the only thing ifdefed out in rasd's > > copy of evsel.h... > Right, this all reads like it is going in the right direction but the > more important question IMO is are we doing a libperf or are we still > doing tools/lib/api/perf/ of single object files to which people can > link? My preference would be for single object files, but the pressure to have a written in stone library seems to just build up... > Because if it is the second - and it sounds to me like it is - how do > you propose we link with external projects? IOW, if I want to have rasd > build *without* the kernel tree, is a simple > cp -r tools/lib/api/ <my_project>/path/to/perf/lib/ > work? It should, no matter if we end up with a library. What I'm doing these days to test if perf builds on multiple distros is: #!/bin/sh remotebuild() { target=$1 perfpkg=$2 cmd="rm -rf ${perfpkg} ; tar xf ${perfpkg}.tar.gz && cd ${perfpkg} && rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf && make -C tools/perf O=/tmp/build/perf install && make -C tools/perf build-test" scp ${perfpkg}.tar.gz gita@${target}:. && \ ssh gita@${target} "${cmd}" } targets="fedora14 opensuse12.3 fedora19 ubuntu13.10" perfpkg=perf-3.17.0-rc4 for target in ${targets} ; do remotebuild ${target} ${perfpkg} done After doing a 'make perf-targz-src-pkg' I.e. no kernel sources involved on the machines where I build test. IOW, it is untangled from the kernel sources. As tools/lib/api/ should as well. > I mean, I don't know and I'm just asking. Is that the proposed way? Are > we fine with that? Do we want something different, maybe even a lib? Is > it time for a lib even? Well, the rasd experience is serving to show areas where there is unnecessary entanglement (hists inside perf_event, etc, the ifdefs you put in place). I'm working to remove the ones that are in rasd.c, aiming to have a tools/lib/api/ tree that can be used to build rasd and tools/perf/. What I don't want to do is to simply straight more tools/perf/util/evlist.c to tools/lib/api/perf/, some untanglement work is needed. > Because the whole perf functionality is being cravingly ogled by other > users - I know Matt wants it for cache QoS or whatever it is called and > there are probably a whole lot of projects which would like to process > events programatically in userspace. > > So what I'm saying is, we probably should have a nice clean way > to be able to share that code with external projects instead of > external projects duplicating and building a whole library around > perf_event_open() and the likes... Agreed, hopefully we'll get that, finally. > Just a couple of thoughts... > > -- > Regards/Gruss, > Boris. > > Sent from a fat crate under my desk. Formatting is fine. > -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-06 21:22 ` Arnaldo Carvalho de Melo @ 2014-10-07 11:23 ` Borislav Petkov 2014-10-07 13:40 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-07 11:23 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming On Mon, Oct 06, 2014 at 06:22:43PM -0300, Arnaldo Carvalho de Melo wrote: > My preference would be for single object files, but the pressure to have > a written in stone library seems to just build up... Yeah, I think if we have an alternative way to easily integrate that functionality into external projects, we can avoid the need for a lib. ... > After doing a 'make perf-targz-src-pkg' > > I.e. no kernel sources involved on the machines where I build test. > > IOW, it is untangled from the kernel sources. As tools/lib/api/ should > as well. Ok, that actually sounds like something we should try: $ make perf-api-src-pkg which would spit a tgz with all the generic stuff perf carries. People would simply unpack it in their projects and build it. That would require somewhat stable function names so that updating is relatively painless and if visible functions change, some sort of a README file should say what and why. Sounds like a good alternative to a lib to me... > Well, the rasd experience is serving to show areas where there is > unnecessary entanglement (hists inside perf_event, etc, the ifdefs you > put in place). Yep. > I'm working to remove the ones that are in rasd.c, aiming to have a > tools/lib/api/ tree that can be used to build rasd and tools/perf/. > > What I don't want to do is to simply straight more > tools/perf/util/evlist.c to tools/lib/api/perf/, some untanglement work > is needed. Yep. Thanks acme! -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-07 11:23 ` Borislav Petkov @ 2014-10-07 13:40 ` Arnaldo Carvalho de Melo 2014-10-07 13:49 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-07 13:40 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming Em Tue, Oct 07, 2014 at 01:23:42PM +0200, Borislav Petkov escreveu: > On Mon, Oct 06, 2014 at 06:22:43PM -0300, Arnaldo Carvalho de Melo wrote: > > My preference would be for single object files, but the pressure to have > > a written in stone library seems to just build up... > Yeah, I think if we have an alternative way to easily integrate that > functionality into external projects, we can avoid the need for a lib. > ... > > After doing a 'make perf-targz-src-pkg' > > I.e. no kernel sources involved on the machines where I build test. > > IOW, it is untangled from the kernel sources. As tools/lib/api/ should > > as well. > Ok, that actually sounds like something we should try: > $ make perf-api-src-pkg > which would spit a tgz with all the generic stuff perf carries. People > would simply unpack it in their projects and build it. > That would require somewhat stable function names so that updating is > relatively painless and if visible functions change, some sort of a Right, we now should always think that the great advantage of having this code and its users in the same repo tree is going away, i.e. we will not be able to improve the code by fixing mistakes that require changing its users. So we should export the bare minimum that tools advocate exporting, like you guys are doing now by providing the code that shows how you use it. > README file should say what and why. Well, at that point we can't do that anymore, we will need instead to add new methods while keeping the existing ones in place and with the same semantics, at most for something really annoying to maintain we will have to add annoying warnings telling that in a year or two something will go away, it was deprecated, annoy the maintainer of your tool to update their project. > Sounds like a good alternative to a lib to me... Yes, should be interesting to experiment with this, at the very least it can be considered a step into the direction of a full blown library, if we ever feel that it is strictly needed. > > Well, the rasd experience is serving to show areas where there is > > unnecessary entanglement (hists inside perf_event, etc, the ifdefs you > > put in place). > Yep. > > I'm working to remove the ones that are in rasd.c, aiming to have a > > tools/lib/api/ tree that can be used to build rasd and tools/perf/. > > What I don't want to do is to simply straight more s/more/move/g > > tools/perf/util/evlist.c to tools/lib/api/perf/, some untanglement work > > is needed. > Yep. - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-07 13:40 ` Arnaldo Carvalho de Melo @ 2014-10-07 13:49 ` Borislav Petkov 2014-10-07 13:55 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-07 13:49 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming On Tue, Oct 07, 2014 at 10:40:21AM -0300, Arnaldo Carvalho de Melo wrote: > Right, we now should always think that the great advantage of having > this code and its users in the same repo tree is going away, i.e. we > will not be able to improve the code by fixing mistakes that require > changing its users. Why? I mean, in the rasd case, changing function calling conventions as part of updating the perf libraries is no act. I mean, we won't link against a shared lib but build a static executable and thus the perf src lib (let's call it that - a source library) we're using is a don't care. Thanks. -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-07 13:49 ` Borislav Petkov @ 2014-10-07 13:55 ` Arnaldo Carvalho de Melo 2014-10-07 14:02 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-07 13:55 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming Em Tue, Oct 07, 2014 at 03:49:12PM +0200, Borislav Petkov escreveu: > On Tue, Oct 07, 2014 at 10:40:21AM -0300, Arnaldo Carvalho de Melo wrote: > > Right, we now should always think that the great advantage of having > > this code and its users in the same repo tree is going away, i.e. we > > will not be able to improve the code by fixing mistakes that require > > changing its users. > > Why? I mean, in the rasd case, changing function calling conventions as > part of updating the perf libraries is no act. I mean, we won't link > against a shared lib but build a static executable and thus the perf src > lib (let's call it that - a source library) we're using is a don't care. So you say that if we change a method, say, perf_evlist__open() to accept a new argument, or change the type of one of its arguments, with a good reason, it is acceptable and with just a change in the README explaining why the build fails, which helps in finding how to fix it in a particular project using these "source libraries" then it would be ok? That would be fine with me. We should take care to make sure that the build _fails_ in such cases, i.e.avoid changing the semantic of the Nth argument but keeping its type kind of changes. At some point there will be no more reasons to change things, and that will be noticed by how long since the last change was made to a particular class, at that point we may well think about making library type promises. - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-07 13:55 ` Arnaldo Carvalho de Melo @ 2014-10-07 14:02 ` Borislav Petkov 2014-10-07 14:13 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-07 14:02 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming On Tue, Oct 07, 2014 at 10:55:31AM -0300, Arnaldo Carvalho de Melo wrote: > So you say that if we change a method, say, perf_evlist__open() to > accept a new argument, or change the type of one of its arguments, > with a good reason, it is acceptable and with just a change in the > README explaining why the build fails, which helps in finding how to fix > it in a particular project using these "source libraries" then it would > be ok? Right, exactly. Updating the perf bits used in another project will require a recompilation anyway so adjusting stuff here and there should not be of issue. > That would be fine with me. Yeah. > We should take care to make sure that the build _fails_ in such cases, > i.e.avoid changing the semantic of the Nth argument but keeping its > type kind of changes. Yep. > At some point there will be no more reasons to change things, and that > will be noticed by how long since the last change was made to a > particular class, at that point we may well think about making library > type promises. I was about to say the same thing - once the code doesn't change anymore and gets even boring, doing the library promise is probably pretty easy. See, no need for you to get over to LCE: we're practically done :-) We'll run the proposal in jolsa's timeslot again and make sure to poke holes in it. But it looks good from where I'm standing. Btw, did we just solve the compatibility problem of libraries by delivering source libraries first!?! Haha. -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: perf & rasd integration plan 2014-10-07 14:02 ` Borislav Petkov @ 2014-10-07 14:13 ` Arnaldo Carvalho de Melo 0 siblings, 0 replies; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-07 14:13 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar, Thomas Gleixner, Matt Fleming Em Tue, Oct 07, 2014 at 04:02:29PM +0200, Borislav Petkov escreveu: > On Tue, Oct 07, 2014 at 10:55:31AM -0300, Arnaldo Carvalho de Melo wrote: > > So you say that if we change a method, say, perf_evlist__open() to > > accept a new argument, or change the type of one of its arguments, > > with a good reason, it is acceptable and with just a change in the > > README explaining why the build fails, which helps in finding how to fix > > it in a particular project using these "source libraries" then it would > > be ok? > Right, exactly. > Updating the perf bits used in another project will require a > recompilation anyway so adjusting stuff here and there should not be of > issue. > > That would be fine with me. > Yeah. > > We should take care to make sure that the build _fails_ in such cases, > > i.e.avoid changing the semantic of the Nth argument but keeping its > > type kind of changes. > Yep. > > At some point there will be no more reasons to change things, and that > > will be noticed by how long since the last change was made to a > > particular class, at that point we may well think about making library > > type promises. > I was about to say the same thing - once the code doesn't change anymore > and gets even boring, doing the library promise is probably pretty easy. Ok, at some point we need to make sure this gets into tools/lib/api/README. > See, no need for you to get over to LCE: we're practically done :-) Nah, there is much more than this at LCE, I'm sure, I'll try again next time. > We'll run the proposal in jolsa's timeslot again and make sure to poke > holes in it. But it looks good from where I'm standing. Thanks! > Btw, did we just solve the compatibility problem of libraries by > delivering source libraries first!?! For me one of the points of having tools/ is to try and make kernel developers try to do userspace muck, dirtying the hands trying to use the same discipline used in the kernel proper, using code as unchanged as possible from the kernel proper, etc. If we now are moving to share code in a way that keeps what is nice in having code and its users in the same repo, like the kernel guys have with subsystems and its drivers while allowing to do it in different repos, that is a good thing indeed. > Haha. While having fun even! Good deal. - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-09-30 9:06 perf & rasd integration plan Jean Pihet 2014-09-30 13:24 ` Arnaldo Carvalho de Melo 2014-10-06 15:07 ` Arnaldo Carvalho de Melo @ 2014-10-06 21:26 ` Arnaldo Carvalho de Melo 2014-10-07 8:45 ` Jean Pihet 2 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-06 21:26 UTC (permalink / raw) To: Jean Pihet Cc: Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Heya, please check if this is OK. This was while looking the set of methods used by rasd, trying to reduce it to the bare minimum. Perhaps even that cpu_map__new() one can be ditched, leaving to use the default of a NULL cpumap that will end up being one with -1, i.e. all cpus. - Arnaldo >From 8dc34bd2824c7843182f4fc6deabaf573e42e806 Mon Sep 17 00:00:00 2001 From: Arnaldo Carvalho de Melo <acme@redhat.com> Date: Mon, 6 Oct 2014 15:43:42 -0300 Subject: [PATCH] rasd: Use perf_evlist__open() instead of open coded equivalent Cc: Borislav Petkov <bp@suse.de> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> --- src/rasd.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/src/rasd.c b/src/rasd.c index 06ccdcdd9d56..fb33fca131fa 100644 --- a/src/rasd.c +++ b/src/rasd.c @@ -241,7 +241,6 @@ static void daemonize(void) int main() { - struct perf_evsel *c; struct thread_map *threads; struct cpu_map *cpus; int i; @@ -278,12 +277,9 @@ int main() perf_evlist__set_maps(evlist, cpus, threads); /* Open events */ - evlist__for_each(evlist, c) { - /* On all online cpus by default, system wide tracing */ - if (perf_evsel__open(c, evlist->cpus, NULL) < 0) - err("opening tracepoint, are you root?"); - } - perf_evlist__set_id_pos(evlist); + /* On all online cpus by default, system wide tracing */ + if (perf_evlist__open(evlist) < 0) + err("opening tracepoint, are you root?"); /* mmap buffers */ if (perf_evlist__mmap(evlist, 4 /* opts->mmap_pages */, false) < 0) -- 1.9.3 ^ permalink raw reply related [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-06 21:26 ` [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded Arnaldo Carvalho de Melo @ 2014-10-07 8:45 ` Jean Pihet 2014-10-07 13:32 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 44+ messages in thread From: Jean Pihet @ 2014-10-07 8:45 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Hi Arnaldo, On 6 October 2014 23:26, Arnaldo Carvalho de Melo <acme@kernel.org> wrote: > Heya, please check if this is OK. > > This was while looking the set of methods used by rasd, trying to reduce > it to the bare minimum. > > Perhaps even that cpu_map__new() one can be ditched, leaving to use the > default of a NULL cpumap that will end up being one with -1, i.e. all > cpus. > > - Arnaldo > > From 8dc34bd2824c7843182f4fc6deabaf573e42e806 Mon Sep 17 00:00:00 2001 > From: Arnaldo Carvalho de Melo <acme@redhat.com> > Date: Mon, 6 Oct 2014 15:43:42 -0300 > Subject: [PATCH] rasd: Use perf_evlist__open() instead of open coded > equivalent > > Cc: Borislav Petkov <bp@suse.de> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> > --- > src/rasd.c | 10 +++------- > 1 file changed, 3 insertions(+), 7 deletions(-) > > diff --git a/src/rasd.c b/src/rasd.c > index 06ccdcdd9d56..fb33fca131fa 100644 > --- a/src/rasd.c > +++ b/src/rasd.c > @@ -241,7 +241,6 @@ static void daemonize(void) > > int main() > { > - struct perf_evsel *c; > struct thread_map *threads; > struct cpu_map *cpus; > int i; > @@ -278,12 +277,9 @@ int main() > perf_evlist__set_maps(evlist, cpus, threads); > > /* Open events */ > - evlist__for_each(evlist, c) { > - /* On all online cpus by default, system wide tracing */ > - if (perf_evsel__open(c, evlist->cpus, NULL) < 0) > - err("opening tracepoint, are you root?"); > - } > - perf_evlist__set_id_pos(evlist); > + /* On all online cpus by default, system wide tracing */ > + if (perf_evlist__open(evlist) < 0) > + err("opening tracepoint, are you root?"); That way the system wide tracing does not work. perf_evlist__open uses a non-NULL thread mapping and so it only traces the events generated by the daemon itself. perf top uses a similar approach but uses a lot of code in machine.c (machine__synthesize_threads) and util/event.c (perf_event__synthesize_threads) to synthesize the threads etc., which we want to avoid in the minimalistic approach of rasd. Maybe I missed something about the system wide tracing, any suggestion is welcome. Thanks for looking! Jean > > /* mmap buffers */ > if (perf_evlist__mmap(evlist, 4 /* opts->mmap_pages */, false) < 0) > -- > 1.9.3 > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-07 8:45 ` Jean Pihet @ 2014-10-07 13:32 ` Arnaldo Carvalho de Melo 2014-10-07 14:04 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-07 13:32 UTC (permalink / raw) To: Jean Pihet Cc: Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Em Tue, Oct 07, 2014 at 10:45:14AM +0200, Jean Pihet escreveu: > > --- a/src/rasd.c > > - evlist__for_each(evlist, c) { > > - /* On all online cpus by default, system wide tracing */ > > - if (perf_evsel__open(c, evlist->cpus, NULL) < 0) > > - err("opening tracepoint, are you root?"); > > - } > > - perf_evlist__set_id_pos(evlist); > > + /* On all online cpus by default, system wide tracing */ > > + if (perf_evlist__open(evlist) < 0) > > + err("opening tracepoint, are you root?"); > That way the system wide tracing does not work. perf_evlist__open uses > a non-NULL thread mapping and so it only traces the events generated > by the daemon itself. > perf top uses a similar approach but uses a lot of code in machine.c > (machine__synthesize_threads) and util/event.c > (perf_event__synthesize_threads) to synthesize the threads etc., which > we want to avoid in the minimalistic approach of rasd. Humm, I will look into making it support this usecase, the point of evsel/evlist is to try to abstract away as much as possible, leaving just a few methods to be used by tools. I think the default for perf_evlist__open() should be the most useful for the majority of tools, which I thought would serve rasd well. It is not the case, so I'll into how the tools that currently use perf_evlist__open() behave and try to get a sane default in place. Hopefully we will completely remove the need to set up any thread or cpu map, as what you want is syswide tracing, right? - Arnaldo > Maybe I missed something about the system wide tracing, any suggestion > is welcome. > > Thanks for looking! > > Jean > > > > > /* mmap buffers */ > > if (perf_evlist__mmap(evlist, 4 /* opts->mmap_pages */, false) < 0) > > -- > > 1.9.3 > > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-07 13:32 ` Arnaldo Carvalho de Melo @ 2014-10-07 14:04 ` Borislav Petkov 2014-10-07 14:17 ` Arnaldo Carvalho de Melo 2014-10-10 20:07 ` Arnaldo Carvalho de Melo 0 siblings, 2 replies; 44+ messages in thread From: Borislav Petkov @ 2014-10-07 14:04 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar On Tue, Oct 07, 2014 at 10:32:36AM -0300, Arnaldo Carvalho de Melo wrote: > Hopefully we will completely remove the need to set up any thread or cpu > map, as what you want is syswide tracing, right? Yeah, I was questioning the need to open at least one thread even for system-wide tracing. See, source libraries are the new great thing for collaborative development - f*ck "cloud"! :-D -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-07 14:04 ` Borislav Petkov @ 2014-10-07 14:17 ` Arnaldo Carvalho de Melo 2014-10-10 20:07 ` Arnaldo Carvalho de Melo 1 sibling, 0 replies; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-07 14:17 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Em Tue, Oct 07, 2014 at 04:04:33PM +0200, Borislav Petkov escreveu: > On Tue, Oct 07, 2014 at 10:32:36AM -0300, Arnaldo Carvalho de Melo wrote: > > Hopefully we will completely remove the need to set up any thread or cpu > > map, as what you want is syswide tracing, right? > Yeah, I was questioning the need to open at least one thread even for > system-wide tracing. Right, evsel/evlist evolved out of tons of refactoring steps moving stuff out of open coded tools (record, report, top) while trying to make it work for those tools, sometimes this resulted in sub-optimal defaults (or plain bugs) like needing a thread_map and/or a cpu_map to do syswide tracing, need to get that clarified and sorted out. - Arnaldo > See, source libraries are the new great thing for collaborative > development - f*ck "cloud"! > :-D :-) - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-07 14:04 ` Borislav Petkov 2014-10-07 14:17 ` Arnaldo Carvalho de Melo @ 2014-10-10 20:07 ` Arnaldo Carvalho de Melo 2014-10-10 20:28 ` Borislav Petkov 1 sibling, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-10 20:07 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Em Tue, Oct 07, 2014 at 04:04:33PM +0200, Borislav Petkov escreveu: > On Tue, Oct 07, 2014 at 10:32:36AM -0300, Arnaldo Carvalho de Melo wrote: > > Hopefully we will completely remove the need to set up any thread or cpu > > map, as what you want is syswide tracing, right? > > Yeah, I was questioning the need to open at least one thread even for > system-wide tracing. Ok, so what is now at my perf/hists branch at: git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git should do the trick, the patch below makes rasd.c use it. I'll try building rasd.c with it and checking if it works. Never having tried this, what are the requisites to test it? Some specific hardware and a kernel with the right tracepoint? I guess some recent 3.17-rc kernel is all that I need? - Arnaldo --- [PATCH] rasd: Use perf_evlist__open() instead of open coded equivalent Cc: Borislav Petkov <bp@suse.de> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> --- src/rasd.c | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) diff --git a/src/rasd.c b/src/rasd.c index 06ccdcdd9d56..d3df9868388c 100644 --- a/src/rasd.c +++ b/src/rasd.c @@ -241,9 +241,6 @@ static void daemonize(void) int main() { - struct perf_evsel *c; - struct thread_map *threads; - struct cpu_map *cpus; int i; /* Run the process as a daemon */ @@ -266,24 +263,10 @@ int main() if (read_config_file()) err("error reading config file"); - /* Set cpu and threads maps */ - threads = thread_map__new(-1, getpid(), UINT_MAX); - if (threads == NULL) - err("allocating threads_map\n"); - - cpus = cpu_map__new(NULL); - if (cpus == NULL) - err("allocating cpu_map\n"); - - perf_evlist__set_maps(evlist, cpus, threads); - /* Open events */ - evlist__for_each(evlist, c) { - /* On all online cpus by default, system wide tracing */ - if (perf_evsel__open(c, evlist->cpus, NULL) < 0) - err("opening tracepoint, are you root?"); - } - perf_evlist__set_id_pos(evlist); + /* On all online cpus by default, system wide tracing */ + if (perf_evlist__open(evlist) < 0) + err("opening tracepoint, are you root?"); /* mmap buffers */ if (perf_evlist__mmap(evlist, 4 /* opts->mmap_pages */, false) < 0) -- 1.9.3 ^ permalink raw reply related [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-10 20:07 ` Arnaldo Carvalho de Melo @ 2014-10-10 20:28 ` Borislav Petkov 2014-10-10 20:41 ` Arnaldo Carvalho de Melo 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-10 20:28 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar On Fri, Oct 10, 2014 at 05:07:08PM -0300, Arnaldo Carvalho de Melo wrote: > Ok, so what is now at my perf/hists branch at: > > git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git > > should do the trick, the patch below makes rasd.c use it. Cool. > I'll try lding rasd.c with it and checking if it works. > > Never having tried this, what are the requisites to test it? Some > specific hardware and a kernel with the right tracepoint? I guess some > recent 3.17-rc kernel is all that I need? Well, you'd need the part of Jean's patches which adds the event to evlist: https://lkml.kernel.org/r/1412933690-25576-1-git-send-email-jean.pihet@linaro.org AFAICT, you could apply patches 1-5 and replace 6 with yours. Now, rasd.cfg has the mce:mce_record tracepoint which rasd opens but you probably want to put a tracepoint which is much easier to exercise, maybe some syscall or whatever. I think that should do it but we won't know until we've tried it. HTH and thanks a lot for doing this! -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-10 20:28 ` Borislav Petkov @ 2014-10-10 20:41 ` Arnaldo Carvalho de Melo 2014-10-10 20:44 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-10 20:41 UTC (permalink / raw) To: Borislav Petkov Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Em Fri, Oct 10, 2014 at 10:28:54PM +0200, Borislav Petkov escreveu: > On Fri, Oct 10, 2014 at 05:07:08PM -0300, Arnaldo Carvalho de Melo wrote: > > I'll try lding rasd.c with it and checking if it works. > > Never having tried this, what are the requisites to test it? Some > > specific hardware and a kernel with the right tracepoint? I guess some > > recent 3.17-rc kernel is all that I need? > Well, you'd need the part of Jean's patches which adds the event to evlist: > https://lkml.kernel.org/r/1412933690-25576-1-git-send-email-jean.pihet@linaro.org > AFAICT, you could apply patches 1-5 and replace 6 with yours. Now, > rasd.cfg has the mce:mce_record tracepoint which rasd opens but you > probably want to put a tracepoint which is much easier to exercise, > maybe some syscall or whatever. Right, stoopid me, no need for some specific tracepoint, just to see that whatever tp it is, it will show up in "rasd"'s event loop. Ok, I'll try that later. Next stuff I probably will do is to move the bare minimum used by rasd to tools/lib/api/perf/, i.e. there will be: tools/lib/api/perf/evsel.c tools/perf/util/evsel.c Both will share the perf_evsel__ namespace (which I thought at some point to make just: evsel__<METHOD_NAME>, wdyt?). That way we just make public the bare minimum that already proved to be useful outside tools/perf/ and over time we move stuff from tools/perf/util/evsel.c (and from other tools in or out perf's repo) into the lib. > I think that should do it but we won't know until we've tried it. > > HTH and thanks a lot for doing this! Np, had to be done at some point :) - Arnaldo > -- > Regards/Gruss, > Boris. > > Sent from a fat crate under my desk. Formatting is fine. > -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-10 20:41 ` Arnaldo Carvalho de Melo @ 2014-10-10 20:44 ` Borislav Petkov 2014-10-13 7:29 ` Jean Pihet 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-10 20:44 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar On Fri, Oct 10, 2014 at 05:41:58PM -0300, Arnaldo Carvalho de Melo wrote: > Right, stoopid me, no need for some specific tracepoint, just to see > that whatever tp it is, it will show up in "rasd"'s event loop. Ok, I'll > try that later. > > Next stuff I probably will do is to move the bare minimum used by rasd > to tools/lib/api/perf/, i.e. there will be: > > tools/lib/api/perf/evsel.c > tools/perf/util/evsel.c > > Both will share the perf_evsel__ namespace (which I thought at some > point to make just: evsel__<METHOD_NAME>, wdyt?). > > That way we just make public the bare minimum that already proved to be > useful outside tools/perf/ and over time we move stuff from > tools/perf/util/evsel.c (and from other tools in or out perf's repo) > into the lib. Abso-f*cking-lutely! :-D > Np, had to be done at some point :) Yeah, cool. Let me know what I should do to help out. Thanks. -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-10 20:44 ` Borislav Petkov @ 2014-10-13 7:29 ` Jean Pihet 2014-10-14 13:56 ` Jiri Olsa 0 siblings, 1 reply; 44+ messages in thread From: Jean Pihet @ 2014-10-13 7:29 UTC (permalink / raw) To: Borislav Petkov, Arnaldo Carvalho de Melo Cc: linux-kernel, Fu Wei, Robert Richter, Jiri Olsa, David Ahern, Ingo Molnar Hi Arnaldo, Borislav, On 10 October 2014 22:44, Borislav Petkov <bp@alien8.de> wrote: > On Fri, Oct 10, 2014 at 05:41:58PM -0300, Arnaldo Carvalho de Melo wrote: >> Right, stoopid me, no need for some specific tracepoint, just to see >> that whatever tp it is, it will show up in "rasd"'s event loop. Ok, I'll >> try that later. >> >> Next stuff I probably will do is to move the bare minimum used by rasd >> to tools/lib/api/perf/, i.e. there will be: >> >> tools/lib/api/perf/evsel.c >> tools/perf/util/evsel.c >> >> Both will share the perf_evsel__ namespace (which I thought at some >> point to make just: evsel__<METHOD_NAME>, wdyt?). Makes perfectly sense >> >> That way we just make public the bare minimum that already proved to be >> useful outside tools/perf/ and over time we move stuff from >> tools/perf/util/evsel.c (and from other tools in or out perf's repo) >> into the lib. That is great! About rasd usage, there are some details at https://wiki.linaro.org/LEG/Engineering/Kernel/perfAndRAS#rasd_implementation. Thx, Jean > > Abso-f*cking-lutely! > > :-D > >> Np, had to be done at some point :) > > Yeah, cool. Let me know what I should do to help out. > > Thanks. > > -- > Regards/Gruss, > Boris. > > Sent from a fat crate under my desk. Formatting is fine. > -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-13 7:29 ` Jean Pihet @ 2014-10-14 13:56 ` Jiri Olsa 2014-10-14 14:02 ` Arnaldo Carvalho de Melo 2014-10-14 14:19 ` David Ahern 0 siblings, 2 replies; 44+ messages in thread From: Jiri Olsa @ 2014-10-14 13:56 UTC (permalink / raw) To: Jean Pihet Cc: Borislav Petkov, Arnaldo Carvalho de Melo, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Mon, Oct 13, 2014 at 09:29:57AM +0200, Jean Pihet wrote: > Hi Arnaldo, Borislav, > > On 10 October 2014 22:44, Borislav Petkov <bp@alien8.de> wrote: > > On Fri, Oct 10, 2014 at 05:41:58PM -0300, Arnaldo Carvalho de Melo wrote: > >> Right, stoopid me, no need for some specific tracepoint, just to see > >> that whatever tp it is, it will show up in "rasd"'s event loop. Ok, I'll > >> try that later. > >> > >> Next stuff I probably will do is to move the bare minimum used by rasd > >> to tools/lib/api/perf/, i.e. there will be: > >> > >> tools/lib/api/perf/evsel.c > >> tools/perf/util/evsel.c > >> > >> Both will share the perf_evsel__ namespace (which I thought at some > >> point to make just: evsel__<METHOD_NAME>, wdyt?). > Makes perfectly sense > > >> > >> That way we just make public the bare minimum that already proved to be > >> useful outside tools/perf/ and over time we move stuff from > >> tools/perf/util/evsel.c (and from other tools in or out perf's repo) > >> into the lib. > That is great! > > About rasd usage, there are some details at > https://wiki.linaro.org/LEG/Engineering/Kernel/perfAndRAS#rasd_implementation. heya, sorry for late reply.. I was on vacation last week.. I read the rasd sources and realized we could poke this from another angle.. AFAIU the work the rasd does is following: - reads config file and opens configured tracepoints - reads samples comming from those tracepoints and displays/writes this data to the console/file - is there more? If I'm not missing anything, this is quite usefull/common usage pattern which would deserve new perf command. I can see the analogy with ftrace debugfs interface - choose/enable tracepoints - cat .../tracing/trace-pipe and there could be '-d' for the command to act as daemon. thoughts? ;-) jirka ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-14 13:56 ` Jiri Olsa @ 2014-10-14 14:02 ` Arnaldo Carvalho de Melo 2014-10-14 14:22 ` Jiri Olsa 2014-10-14 14:19 ` David Ahern 1 sibling, 1 reply; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-14 14:02 UTC (permalink / raw) To: Jiri Olsa Cc: Jean Pihet, Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar Em Tue, Oct 14, 2014 at 03:56:31PM +0200, Jiri Olsa escreveu: > On Mon, Oct 13, 2014 at 09:29:57AM +0200, Jean Pihet wrote: > > On 10 October 2014 22:44, Borislav Petkov <bp@alien8.de> wrote: > > > On Fri, Oct 10, 2014 at 05:41:58PM -0300, Arnaldo Carvalho de Melo wrote: > > >> That way we just make public the bare minimum that already proved to be > > >> useful outside tools/perf/ and over time we move stuff from > > >> tools/perf/util/evsel.c (and from other tools in or out perf's repo) > > >> into the lib. > > That is great! > > About rasd usage, there are some details at > > https://wiki.linaro.org/LEG/Engineering/Kernel/perfAndRAS#rasd_implementation. > sorry for late reply.. I was on vacation last week.. > I read the rasd sources and realized we could poke this > from another angle.. AFAIU the work the rasd does is following: > - reads config file and opens configured tracepoints > - reads samples comming from those tracepoints and displays/writes > this data to the console/file > - is there more? > If I'm not missing anything, this is quite usefull/common usage > pattern which would deserve new perf command. > I can see the analogy with ftrace debugfs interface > - choose/enable tracepoints > - cat .../tracing/trace-pipe > and there could be '-d' for the command to act as daemon. Right, this is to be integrated into 'trace', i.e. to be able to ask for more events, some with callchains, some without, etc. We may even have something really bare bones that does what rasd.c does right now, to show how one can write a tool using the exported interfaces in tools/lib/api/, but I think that what rasd is _right now_, is just an attempt to use the libraries using as few as possible apis, right? I.e. more "meat" will be added there, no? Anyway, its beeing an interesting exercise and will result in a lot of untanglement, which is good in any case. - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-14 14:02 ` Arnaldo Carvalho de Melo @ 2014-10-14 14:22 ` Jiri Olsa 2014-10-14 15:17 ` Borislav Petkov 0 siblings, 1 reply; 44+ messages in thread From: Jiri Olsa @ 2014-10-14 14:22 UTC (permalink / raw) To: Arnaldo Carvalho de Melo Cc: Jean Pihet, Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Tue, Oct 14, 2014 at 11:02:23AM -0300, Arnaldo Carvalho de Melo wrote: > Em Tue, Oct 14, 2014 at 03:56:31PM +0200, Jiri Olsa escreveu: > > On Mon, Oct 13, 2014 at 09:29:57AM +0200, Jean Pihet wrote: > > > On 10 October 2014 22:44, Borislav Petkov <bp@alien8.de> wrote: > > > > On Fri, Oct 10, 2014 at 05:41:58PM -0300, Arnaldo Carvalho de Melo wrote: > > > >> That way we just make public the bare minimum that already proved to be > > > >> useful outside tools/perf/ and over time we move stuff from > > > >> tools/perf/util/evsel.c (and from other tools in or out perf's repo) > > > >> into the lib. > > > That is great! > > > > About rasd usage, there are some details at > > > https://wiki.linaro.org/LEG/Engineering/Kernel/perfAndRAS#rasd_implementation. > > > sorry for late reply.. I was on vacation last week.. > > > I read the rasd sources and realized we could poke this > > from another angle.. AFAIU the work the rasd does is following: > > - reads config file and opens configured tracepoints > > - reads samples comming from those tracepoints and displays/writes > > this data to the console/file > > - is there more? > > > If I'm not missing anything, this is quite usefull/common usage > > pattern which would deserve new perf command. > > > I can see the analogy with ftrace debugfs interface > > - choose/enable tracepoints > > - cat .../tracing/trace-pipe > > > and there could be '-d' for the command to act as daemon. > > Right, this is to be integrated into 'trace', i.e. to be able to ask for > more events, some with callchains, some without, etc. > > We may even have something really bare bones that does what rasd.c does > right now, to show how one can write a tool using the exported > interfaces in tools/lib/api/, but I think that what rasd is _right now_, > is just an attempt to use the libraries using as few as possible apis, > right? I.e. more "meat" will be added there, no? right.. is there more planned for RAS daemon to do? because at this point it looks to me more like new perf functionality (in trace or some other new command) > > Anyway, its beeing an interesting exercise and will result in a lot of > untanglement, which is good in any case. yea, I dont mind untangling the interface.. I noticed in the thread people want this anyway - simple 'events open' interface.. I often need something simple for test programs jirka ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-14 14:22 ` Jiri Olsa @ 2014-10-14 15:17 ` Borislav Petkov 2014-10-14 15:20 ` Jean Pihet 0 siblings, 1 reply; 44+ messages in thread From: Borislav Petkov @ 2014-10-14 15:17 UTC (permalink / raw) To: Jiri Olsa Cc: Arnaldo Carvalho de Melo, Jean Pihet, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On Tue, Oct 14, 2014 at 04:22:13PM +0200, Jiri Olsa wrote: > right.. is there more planned for RAS daemon to do? because at this > point it looks to me more like new perf functionality (in trace or > some other new command) > > > > > Anyway, its beeing an interesting exercise and will result in a lot of > > untanglement, which is good in any case. > > yea, I dont mind untangling the interface.. I noticed in the thread > people want this anyway - simple 'events open' interface.. I often > need something simple for test programs Right, we simply need a way to get the tracepoint data into userspace. Nothing else. And right now I cannot think about needing any other perf tool functionality. But who knows.., if we need more, we'll untangle more. :-) -- Regards/Gruss, Boris. -- ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-14 15:17 ` Borislav Petkov @ 2014-10-14 15:20 ` Jean Pihet 0 siblings, 0 replies; 44+ messages in thread From: Jean Pihet @ 2014-10-14 15:20 UTC (permalink / raw) To: Borislav Petkov Cc: Jiri Olsa, Arnaldo Carvalho de Melo, linux-kernel, Fu Wei, Robert Richter, David Ahern, Ingo Molnar On 14 October 2014 17:17, Borislav Petkov <bp@alien8.de> wrote: > On Tue, Oct 14, 2014 at 04:22:13PM +0200, Jiri Olsa wrote: >> right.. is there more planned for RAS daemon to do? because at this >> point it looks to me more like new perf functionality (in trace or >> some other new command) >> >> > >> > Anyway, its beeing an interesting exercise and will result in a lot of >> > untanglement, which is good in any case. >> >> yea, I dont mind untangling the interface.. I noticed in the thread >> people want this anyway - simple 'events open' interface.. I often >> need something simple for test programs > > Right, we simply need a way to get the tracepoint data into userspace. > Nothing else. And right now I cannot think about needing any other perf > tool functionality. But who knows.., if we need more, we'll untangle > more. The persistent events stuff still needs some work and is needed by rasd. > > :-) ;-P > > -- > Regards/Gruss, > Boris. > -- Cheers, Jean ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-14 13:56 ` Jiri Olsa 2014-10-14 14:02 ` Arnaldo Carvalho de Melo @ 2014-10-14 14:19 ` David Ahern 2014-10-14 17:09 ` Arnaldo Carvalho de Melo 1 sibling, 1 reply; 44+ messages in thread From: David Ahern @ 2014-10-14 14:19 UTC (permalink / raw) To: Jiri Olsa, Jean Pihet Cc: Borislav Petkov, Arnaldo Carvalho de Melo, linux-kernel, Fu Wei, Robert Richter, Ingo Molnar On 10/14/14, 7:56 AM, Jiri Olsa wrote: > I read the rasd sources and realized we could poke this > from another angle.. AFAIU the work the rasd does is following: > - reads config file and opens configured tracepoints > - reads samples comming from those tracepoints and displays/writes > this data to the console/file > - is there more? > > If I'm not missing anything, this is quite usefull/common usage > pattern which would deserve new perf command. > > I can see the analogy with ftrace debugfs interface > - choose/enable tracepoints > - cat .../tracing/trace-pipe > > and there could be '-d' for the command to act as daemon. Yes, this parallels a new use case on my end. Right now I am running perf record ... | perf script. With the tracepoints and filters involved it is a LOT of typing - and still collects more than is needed (I don't need MMAP events for example, only COMM events). I am leaning towards a new perf sub-command but from my scheduling timehist and daemon commands I know there is a lot of overhead that goes with that. A perf library with a stable API would make this a lot easier. (AFAIK the python bindings do not currently support opening events, it is mainly an analysis option.) David ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded 2014-10-14 14:19 ` David Ahern @ 2014-10-14 17:09 ` Arnaldo Carvalho de Melo 0 siblings, 0 replies; 44+ messages in thread From: Arnaldo Carvalho de Melo @ 2014-10-14 17:09 UTC (permalink / raw) To: David Ahern Cc: Jiri Olsa, Jean Pihet, Borislav Petkov, linux-kernel, Fu Wei, Robert Richter, Ingo Molnar Em Tue, Oct 14, 2014 at 08:19:52AM -0600, David Ahern escreveu: > On 10/14/14, 7:56 AM, Jiri Olsa wrote: > >I read the rasd sources and realized we could poke this > >from another angle.. AFAIU the work the rasd does is following: > > - reads config file and opens configured tracepoints > > - reads samples comming from those tracepoints and displays/writes > > this data to the console/file > > - is there more? > > > >If I'm not missing anything, this is quite usefull/common usage > >pattern which would deserve new perf command. > > > >I can see the analogy with ftrace debugfs interface > > - choose/enable tracepoints > > - cat .../tracing/trace-pipe > > > >and there could be '-d' for the command to act as daemon. > > Yes, this parallels a new use case on my end. Right now I am running perf > record ... | perf script. With the tracepoints and filters involved it is a > LOT of typing - and still collects more than is needed (I don't need MMAP > events for example, only COMM events). I am leaning towards a new perf > sub-command but from my scheduling timehist and daemon commands I know there > is a lot of overhead that goes with that. A perf library with a stable API > would make this a lot easier. (AFAIK the python bindings do not currently > support opening events, it is mainly an analysis option.) Yeah, please take a look at this branch: https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/hists Trying to use the rasd.c opportunity to go over the boilerplate to diminish it by having defaults for syswide tracing, etc. I'm now working on moving the most minimalistic set of evsel.c/evlist.c into tools/lib/api/perf/ - Arnaldo ^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2014-10-14 17:10 UTC | newest] Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-09-30 9:06 perf & rasd integration plan Jean Pihet 2014-09-30 13:24 ` Arnaldo Carvalho de Melo 2014-10-05 17:48 ` Borislav Petkov 2014-10-05 18:24 ` Jiri Olsa 2014-10-05 18:45 ` Borislav Petkov 2014-10-05 19:24 ` Chuck Ebbert 2014-10-05 19:28 ` Jiri Olsa 2014-10-06 6:53 ` Jean Pihet 2014-10-08 6:59 ` Jiri Olsa 2014-10-06 9:07 ` Robert Richter 2014-10-06 13:44 ` Jean Pihet 2014-10-06 14:58 ` Arnaldo Carvalho de Melo 2014-10-06 15:01 ` Borislav Petkov 2014-10-06 15:08 ` Arnaldo Carvalho de Melo 2014-10-06 15:16 ` Borislav Petkov 2014-10-06 15:02 ` Jean Pihet 2014-10-06 15:07 ` Arnaldo Carvalho de Melo 2014-10-06 15:16 ` Borislav Petkov 2014-10-06 19:12 ` Arnaldo Carvalho de Melo 2014-10-06 19:53 ` Borislav Petkov 2014-10-06 21:22 ` Arnaldo Carvalho de Melo 2014-10-07 11:23 ` Borislav Petkov 2014-10-07 13:40 ` Arnaldo Carvalho de Melo 2014-10-07 13:49 ` Borislav Petkov 2014-10-07 13:55 ` Arnaldo Carvalho de Melo 2014-10-07 14:02 ` Borislav Petkov 2014-10-07 14:13 ` Arnaldo Carvalho de Melo 2014-10-06 21:26 ` [PATCH 1/1] rasd: Use perf_evlist__open() instead of open coded Arnaldo Carvalho de Melo 2014-10-07 8:45 ` Jean Pihet 2014-10-07 13:32 ` Arnaldo Carvalho de Melo 2014-10-07 14:04 ` Borislav Petkov 2014-10-07 14:17 ` Arnaldo Carvalho de Melo 2014-10-10 20:07 ` Arnaldo Carvalho de Melo 2014-10-10 20:28 ` Borislav Petkov 2014-10-10 20:41 ` Arnaldo Carvalho de Melo 2014-10-10 20:44 ` Borislav Petkov 2014-10-13 7:29 ` Jean Pihet 2014-10-14 13:56 ` Jiri Olsa 2014-10-14 14:02 ` Arnaldo Carvalho de Melo 2014-10-14 14:22 ` Jiri Olsa 2014-10-14 15:17 ` Borislav Petkov 2014-10-14 15:20 ` Jean Pihet 2014-10-14 14:19 ` David Ahern 2014-10-14 17:09 ` Arnaldo Carvalho de Melo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).