All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v6 00/16] perf tool: add meta-data header support for pipe-mode
@ 2017-07-18  4:25 David Carrillo-Cisneros
  2017-07-18  4:25 ` [PATCH v6 01/16] perf header: encapsulate read and swap David Carrillo-Cisneros
                   ` (16 more replies)
  0 siblings, 17 replies; 34+ messages in thread
From: David Carrillo-Cisneros @ 2017-07-18  4:25 UTC (permalink / raw)
  To: linux-kernel
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Alexander Shishkin, David Ahern, Andi Kleen, Simon Que, Wang Nan,
	Jiri Olsa, He Kuang, Masami Hiramatsu, David Ahern, Namhyung Kim,
	Stephane Eranian, Paul Turner, David Carrillo-Cisneros

v6: - Avoid buffer resize that would overflow event's header size.
v5: - Fix buffer leaking and size changes in
      perf_event__synthesize_features.
    - Remove unnecessary renaming.
    - Remove extra tabs in do_write_string.
v4: - Limit write_* functions to page_size.
    - Fixed bugs spotted by Jiri.
    - Add information about pipe-mode to some error messages. 
v3: - Fix header output for event aux record.
    - Uniformize variable naming and other cleanup.
    - Other fixes suggested by Jiri and Nahmyung.

v2: - Finer patch splitting.
    - Add only one record type with a feature id instead of one
      record per new feature (as suggested by Jiri).
    - Add perf.data documentation.

(This is a rebased and updated version of Stephane Eranian's version
 in https://patchwork.kernel.org/patch/1499081/)

Up until now, meta-data was only available when perf record
was used in "regular" mode, i.e., generating a perf.data file.
For users depending on pipe mode, neither host or event header
information were gathered. This patch addresses this limitation.

The difficulty in pipe mode is that information needs to be written
sequentially to the pipe. Meta data headers are usually generated
(and also expected) at the beginning of the file (or piped output).
To solve this problem, we introduce new synthetic record types,
one for each meta-data type. The approach is similar to what
is *ALREADY* used for BUILD_ID and TRACING_DATA.

We have modified util/header.c such that the same routines are used
to generate and read the meta-data information regardless of pipe-mode
vs. regular mode. To make this work, we added a new struct called
feat_fd which encapsulates all the information necessary to read or
write meta-data information to a file/pipe or from a file/pipe.

With this series, it is possible to get:
  $ perf record -o - -e cycles sleep 1 | perf report --stdio --header
  # ========
  # captured on: Mon May 22 16:33:43 2017
  # ========
  #
  # hostname : my_hostname
  # os release : 4.11.0-dbx-up_perf
  # perf version : 4.11.rc6.g6277c80
  # arch : x86_64
  # nrcpus online : 72
  # nrcpus avail : 72
  # cpudesc : Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz
  # cpuid : GenuineIntel,6,63,2
  # total memory : 263457192 kB
  # cmdline : /root/perf record -e cycles sleep 1 
  # event : name = cycles, , size = 112, { sample_period, sample_freq } = 4000, sample_type = IP|TID|TIME|PERIOD, disabled = 1, inherit = 1, mmap = 1, comm = 1, freq = 1, enable_on_exec = 1, task = 1, sample_id_all = 1, exclude_guest = 1, mmap2 = 1, comm_exec = 1
  # CPU_TOPOLOGY info available, use -I to display
  # NUMA_TOPOLOGY info available, use -I to display
  # pmu mappings: intel_bts = 6, cpu = 4, msr = 49, uncore_cbox_10 = 36, [SNIP]
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.000 MB - ]
  ...

Only patches 14 to 16 affect user observed behavior. All other are
preparatory changes or bug fixes.


David Carrillo-Cisneros (16):
  perf header: encapsulate read and swap
  perf header: add PROCESS_STR_FUN macro
  perf header: fail on write_padded error
  perf util: add const modifier to buf in "writen" function
  perf header: revamp do_write
  perf header: add struct feat_fd for write
  perf header: use struct feat_fd for print
  perf header: use struct feat_fd to process header records
  perf header: don't pass struct perf_file_section to process_##_feat
  perf header: use struct feat_fd in read header records
  perf header: make write_pmu_mappings pipe-mode friendly
  perf header: add a buffer to struct feat_fd
  perf header: change FEAT_OP* macros
  perf tool: add show_feature_header to perf_tool
  perf tools: add feature header record to pipe-mode
  perf header: add event desc to pipe-mode header

 tools/perf/Documentation/perf.data-file-format.txt |   10 +-
 tools/perf/builtin-annotate.c                      |    1 +
 tools/perf/builtin-inject.c                        |    1 +
 tools/perf/builtin-record.c                        |    7 +
 tools/perf/builtin-report.c                        |    5 +
 tools/perf/builtin-script.c                        |    4 +
 tools/perf/util/build-id.c                         |   10 +-
 tools/perf/util/build-id.h                         |    4 +-
 tools/perf/util/event.c                            |    1 +
 tools/perf/util/event.h                            |    8 +
 tools/perf/util/header.c                           | 1015 +++++++++++---------
 tools/perf/util/header.h                           |   16 +-
 tools/perf/util/session.c                          |    4 +
 tools/perf/util/tool.h                             |   10 +-
 tools/perf/util/util.c                             |    6 +-
 tools/perf/util/util.h                             |    2 +-
 16 files changed, 636 insertions(+), 468 deletions(-)

-- 
2.13.2.932.g7449e964c-goog

^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, other threads:[~2017-07-20  9:09 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-07-18  4:25 [PATCH v6 00/16] perf tool: add meta-data header support for pipe-mode David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 01/16] perf header: encapsulate read and swap David Carrillo-Cisneros
2017-07-20  8:58   ` [tip:perf/core] perf header: Encapsulate " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 02/16] perf header: add PROCESS_STR_FUN macro David Carrillo-Cisneros
2017-07-20  8:58   ` [tip:perf/core] perf header: Add " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 03/16] perf header: fail on write_padded error David Carrillo-Cisneros
2017-07-20  8:59   ` [tip:perf/core] perf header: Fail " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 04/16] perf util: add const modifier to buf in "writen" function David Carrillo-Cisneros
2017-07-20  8:59   ` [tip:perf/core] perf util: Add " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 05/16] perf header: revamp do_write David Carrillo-Cisneros
2017-07-20  8:59   ` [tip:perf/core] perf header: Revamp do_write() tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 06/16] perf header: add struct feat_fd for write David Carrillo-Cisneros
2017-07-20  9:00   ` [tip:perf/core] perf header: Add " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 07/16] perf header: use struct feat_fd for print David Carrillo-Cisneros
2017-07-20  9:00   ` [tip:perf/core] perf header: Use " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 08/16] perf header: use struct feat_fd to process header records David Carrillo-Cisneros
2017-07-20  9:00   ` [tip:perf/core] perf header: Use " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 09/16] perf header: don't pass struct perf_file_section to process_##_feat David Carrillo-Cisneros
2017-07-20  9:01   ` [tip:perf/core] perf header: Don't " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 10/16] perf header: use struct feat_fd in read header records David Carrillo-Cisneros
2017-07-20  9:01   ` [tip:perf/core] perf header: Use " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 11/16] perf header: make write_pmu_mappings pipe-mode friendly David Carrillo-Cisneros
2017-07-20  9:01   ` [tip:perf/core] perf header: Make " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 12/16] perf header: add a buffer to struct feat_fd David Carrillo-Cisneros
2017-07-20  9:02   ` [tip:perf/core] perf header: Add " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 13/16] perf header: change FEAT_OP* macros David Carrillo-Cisneros
2017-07-20  9:02   ` [tip:perf/core] perf header: Change " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 14/16] perf tool: add show_feature_header to perf_tool David Carrillo-Cisneros
2017-07-20  9:02   ` [tip:perf/core] perf tool: Add " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 15/16] perf tools: add feature header record to pipe-mode David Carrillo-Cisneros
2017-07-20  9:03   ` [tip:perf/core] perf tools: Add " tip-bot for David Carrillo-Cisneros
2017-07-18  4:25 ` [PATCH v6 16/16] perf header: add event desc to pipe-mode header David Carrillo-Cisneros
2017-07-20  9:03   ` [tip:perf/core] perf header: Add " tip-bot for David Carrillo-Cisneros
2017-07-18  7:28 ` [PATCH v6 00/16] perf tool: add meta-data header support for pipe-mode Jiri Olsa

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.