All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCHv5 00/11] perf record: Add support to store data in directory
@ 2019-03-08 13:47 Jiri Olsa
  2019-03-08 13:47 ` [PATCH 01/11] perf data: Add directory support Jiri Olsa
                   ` (10 more replies)
  0 siblings, 11 replies; 27+ messages in thread
From: Jiri Olsa @ 2019-03-08 13:47 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: lkml, Ingo Molnar, Namhyung Kim, Alexander Shishkin,
	Peter Zijlstra, Adrian Hunter, Andi Kleen, Stephane Eranian,
	Alexey Budankov

hi,
this patchset adds the --dir option to record command (and all
the other record command that overload cmd_record) that allows
the data to be stored in directory with multiple data files.

It's next step for multiple threads implementation in record.
It's now possible to make directory data via --dir option, like:

  $ perf record --dir perf bench sched messaging
  $ ls -l perf.data
  total 344
  -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0
  -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1
  -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2
  -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3
  -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4
  -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5
  -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6
  -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7
  -rw-------. 1 jolsa jolsa  8832 Jan 20 22:26 header

There's a data file created for every cpu and it's storing
data for those cpu maps. The report command will read it
transparently, sort it and display as single file data.

There's new DIR_FORMAT feature to describe directory data
layout/format. In future we can describe different data files
layout according to special needs.

It's possible to transform directory data into standard
perf.data file via simple inject command:
    
  $ perf inject -o perf.data.file -i perf.data

The old perf fails over the directory data with following message:
  $ perf report
  incompatible file format (rerun with -v to learn more)

I'm now testing the record threads support, so I'd like to
have some agreement on the directory data support before.

v5 changes:
  - rebased to latest Arnaldo's perf/core

v4 changes:
  - some of the patches already taken
  - removing up perf.data directory on perf_data__open
    error path

v3 changes:
  - add rm_rf_perf_data to safely remove file/directory perf data
  - allocation fix in perf_data__create_dir

v2 changes:
  - rm_rf changes are already accepted with requested changes
  - updated doc/man plus adding perf.data-directory-format.txt
    to describe directory format/layout
  - the --switch-output options now works over directory data
  - data rollback is not part of this patchset, updated my TODO though ;-)
  - added --output-dir option to combine -o and --dir
  - added DIR_FORMAT feature to describe directory data
  - disabling directory output for aio for now

It's also available in here:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  perf/dir

thanks,
jirka


---
Jiri Olsa (11):
      perf data: Add directory support
      perf data: Don't store auxtrace index for directory data file
      perf data: Add perf_data__update_dir function
      perf data: Make perf_data__size to work over directory
      perf header: Add DIR_FORMAT feature to describe directory data
      perf session: Add process callback to reader object
      perf session: Add __perf_session__process_dir_events function
      perf session: Add path to reader object
      perf record: Add --dir option to store data in directory
      perf record: Add --output-dir option to store data in directory
      perf record: Describe perf.data directory format

 tools/lib/subcmd/parse-options.h                        |   4 ++++
 tools/perf/Documentation/perf-record.txt                |   6 ++++++
 tools/perf/Documentation/perf.data-directory-format.txt |  54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/builtin-record.c                             |  95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 tools/perf/util/data.c                                  |  96 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 tools/perf/util/data.h                                  |  12 ++++++++----
 tools/perf/util/header.c                                |  44 +++++++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/header.h                                |   5 +++++
 tools/perf/util/mmap.h                                  |  23 ++++++++++++-----------
 tools/perf/util/session.c                               | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 10 files changed, 430 insertions(+), 34 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt

^ permalink raw reply	[flat|nested] 27+ messages in thread
* [PATCHv4 00/11] perf record: Add support to store data in directory
@ 2019-02-26  8:48 Jiri Olsa
  2019-02-26  8:48 ` [PATCH 04/11] perf data: Make perf_data__size to work over directory Jiri Olsa
  0 siblings, 1 reply; 27+ messages in thread
From: Jiri Olsa @ 2019-02-26  8:48 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: lkml, Ingo Molnar, Namhyung Kim, Alexander Shishkin,
	Peter Zijlstra, Adrian Hunter, Andi Kleen, Stephane Eranian,
	Alexey Budankov

hi,
this patchset adds the --dir option to record command (and all
the other record command that overload cmd_record) that allows
the data to be stored in directory with multiple data files.

It's next step for multiple threads implementation in record.
It's now possible to make directory data via --dir option, like:

  $ perf record --dir perf bench sched messaging
  $ ls -l perf.data
  total 344
  -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0
  -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1
  -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2
  -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3
  -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4
  -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5
  -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6
  -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7
  -rw-------. 1 jolsa jolsa  8832 Jan 20 22:26 header

There's a data file created for every cpu and it's storing
data for those cpu maps. The report command will read it
transparently, sort it and display as single file data.

There's new DIR_FORMAT feature to describe directory data
layout/format. In future we can describe different data files
layout according to special needs.

It's possible to transform directory data into standard
perf.data file via simple inject command:
    
  $ perf inject -o perf.data.file -i perf.data

The old perf fails over the directory data with following message:
  $ perf report
  incompatible file format (rerun with -v to learn more)

I'm now testing the record threads support, so I'd like to
have some agreement on the directory data support before.

v4 changes:
  - some of the patches already taken
  - removing up perf.data directory on perf_data__open
    error path

v3 changes:
  - add rm_rf_perf_data to safely remove file/directory perf data
  - allocation fix in perf_data__create_dir

v2 changes:
  - rm_rf changes are already accepted with requested changes
  - updated doc/man plus adding perf.data-directory-format.txt
    to describe directory format/layout
  - the --switch-output options now works over directory data
  - data rollback is not part of this patchset, updated my TODO though ;-)
  - added --output-dir option to combine -o and --dir
  - added DIR_FORMAT feature to describe directory data
  - disabling directory output for aio for now

It's also available in here:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  perf/dir

thanks,
jirka


---
Jiri Olsa (11):
      perf data: Add directory support
      perf data: Don't store auxtrace index for directory data file
      perf data: Add perf_data__update_dir function
      perf data: Make perf_data__size to work over directory
      perf header: Add DIR_FORMAT feature to describe directory data
      perf session: Add process callback to reader object
      perf session: Add __perf_session__process_dir_events function
      perf session: Add path to reader object
      perf record: Add --dir option to store data in directory
      perf record: Add --output-dir option to store data in directory
      perf record: Describe perf.data directory format

 tools/lib/subcmd/parse-options.h                        |   4 ++++
 tools/perf/Documentation/perf-record.txt                |   6 ++++++
 tools/perf/Documentation/perf.data-directory-format.txt |  54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/builtin-record.c                             |  95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 tools/perf/util/data.c                                  |  96 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 tools/perf/util/data.h                                  |  12 ++++++++----
 tools/perf/util/header.c                                |  44 +++++++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/header.h                                |   5 +++++
 tools/perf/util/mmap.h                                  |  23 ++++++++++++-----------
 tools/perf/util/session.c                               | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 10 files changed, 430 insertions(+), 34 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt

^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2019-03-22 22:06 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-08 13:47 [PATCHv5 00/11] perf record: Add support to store data in directory Jiri Olsa
2019-03-08 13:47 ` [PATCH 01/11] perf data: Add directory support Jiri Olsa
2019-03-08 17:26   ` Arnaldo Carvalho de Melo
2019-03-22 22:03   ` [tip:perf/urgent] perf data: Support having perf.data stored as a directory tip-bot for Jiri Olsa
2019-03-08 13:47 ` [PATCH 02/11] perf data: Don't store auxtrace index for directory data file Jiri Olsa
2019-03-22 22:03   ` [tip:perf/urgent] " tip-bot for Jiri Olsa
2019-03-08 13:47 ` [PATCH 03/11] perf data: Add perf_data__update_dir function Jiri Olsa
2019-03-22 22:04   ` [tip:perf/urgent] perf data: Add perf_data__update_dir() function tip-bot for Jiri Olsa
2019-03-08 13:47 ` [PATCH 04/11] perf data: Make perf_data__size to work over directory Jiri Olsa
2019-03-08 17:33   ` Arnaldo Carvalho de Melo
2019-03-09 17:21     ` Jiri Olsa
2019-03-22 22:05   ` [tip:perf/urgent] perf data: Make perf_data__size() " tip-bot for Jiri Olsa
2019-03-08 13:47 ` [PATCH 05/11] perf header: Add DIR_FORMAT feature to describe directory data Jiri Olsa
2019-03-08 18:24   ` Arnaldo Carvalho de Melo
2019-03-09 17:21     ` Jiri Olsa
2019-03-22 22:05   ` [tip:perf/urgent] " tip-bot for Jiri Olsa
2019-03-08 13:47 ` [PATCH 06/11] perf session: Add process callback to reader object Jiri Olsa
2019-03-22 22:06   ` [tip:perf/urgent] " tip-bot for Jiri Olsa
2019-03-08 13:47 ` [PATCH 07/11] perf session: Add __perf_session__process_dir_events function Jiri Olsa
2019-03-08 18:38   ` Arnaldo Carvalho de Melo
2019-03-08 18:39     ` Arnaldo Carvalho de Melo
2019-03-09 17:21     ` Jiri Olsa
2019-03-08 13:47 ` [PATCH 08/11] perf session: Add path to reader object Jiri Olsa
2019-03-08 13:47 ` [PATCH 09/11] perf record: Add --dir option to store data in directory Jiri Olsa
2019-03-08 13:47 ` [PATCH 10/11] perf record: Add --output-dir " Jiri Olsa
2019-03-08 13:47 ` [PATCH 11/11] perf record: Describe perf.data directory format Jiri Olsa
  -- strict thread matches above, loose matches on Subject: below --
2019-02-26  8:48 [PATCHv4 00/11] perf record: Add support to store data in directory Jiri Olsa
2019-02-26  8:48 ` [PATCH 04/11] perf data: Make perf_data__size to work over directory Jiri Olsa

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.