From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B427C4338F for ; Tue, 17 Aug 2021 08:25:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E84E60FBF for ; Tue, 17 Aug 2021 08:25:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238847AbhHQI0Q (ORCPT ); Tue, 17 Aug 2021 04:26:16 -0400 Received: from mga04.intel.com ([192.55.52.120]:18151 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239409AbhHQIZV (ORCPT ); Tue, 17 Aug 2021 04:25:21 -0400 X-IronPort-AV: E=McAfee;i="6200,9189,10078"; a="214171984" X-IronPort-AV: E=Sophos;i="5.84,328,1620716400"; d="scan'208";a="214171984" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Aug 2021 01:24:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,328,1620716400"; d="scan'208";a="471080460" Received: from nntpat99-84.inn.intel.com ([10.125.99.84]) by orsmga008.jf.intel.com with ESMTP; 17 Aug 2021 01:24:44 -0700 From: Alexey Bayduraev To: Arnaldo Carvalho de Melo Cc: Jiri Olsa , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Ingo Molnar , linux-kernel , Andi Kleen , Adrian Hunter , Alexander Antonov , Alexei Budankov , Riccardo Mancini Subject: [PATCH v11 24/24] perf session: Load data directory files for analysis Date: Tue, 17 Aug 2021 11:23:27 +0300 Message-Id: <03e17b23b46647db7a71b9255ed2e3cd0cfd76e2.1629186429.git.alexey.v.bayduraev@linux.intel.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Load data directory files and provide basic raw dump and aggregated analysis support of data directories in report mode, still with no memory consumption optimizations. Design and implementation are based on the prototype [1], [2]. [1] git clone https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git -b perf/record_threads [2] https://lore.kernel.org/lkml/20180913125450.21342-1-jolsa@kernel.org/ Suggested-by: Jiri Olsa Acked-by: Namhyung Kim Reviewed-by: Riccardo Mancini Tested-by: Riccardo Mancini Signed-off-by: Alexey Bayduraev --- tools/perf/util/session.c | 131 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c index b49b52768681..f5a82f66e36b 100644 --- a/tools/perf/util/session.c +++ b/tools/perf/util/session.c @@ -48,6 +48,13 @@ #define NUM_MMAPS 128 #endif +/* + * Processing 10MBs of data from each reader in sequence, + * because that's the way the ordered events sorting works + * most efficiently. + */ +#define READER_MAX_SIZE (10 * 1024 * 1024) + struct reader; typedef s64 (*reader_cb_t)(struct perf_session *session, @@ -65,6 +72,7 @@ struct reader_state { u64 data_size; u64 head; bool eof; + u64 size; }; enum { @@ -2328,6 +2336,7 @@ reader__read_event(struct reader *rd, struct perf_session *session, if (skip) size += skip; + st->size += size; st->head += size; st->file_pos += size; @@ -2427,6 +2436,125 @@ static int __perf_session__process_events(struct perf_session *session) return err; } +/* + * This function reads, merge and process directory data. + * It assumens the version 1 of directory data, where each + * data file holds per-cpu data, already sorted by kernel. + */ +static int __perf_session__process_dir_events(struct perf_session *session) +{ + struct perf_data *data = session->data; + struct perf_tool *tool = session->tool; + int i, ret = 0, readers = 1; + struct ui_progress prog; + u64 total_size = perf_data__size(session->data); + struct reader *rd; + + perf_tool__fill_defaults(tool); + + ui_progress__init_size(&prog, total_size, "Sorting events..."); + + for (i = 0; i < data->dir.nr; i++) { + if (data->dir.files[i].size) + readers++; + } + + rd = session->readers = zalloc(readers * sizeof(struct reader)); + if (!rd) + return -ENOMEM; + session->nr_readers = readers; + readers = 0; + + rd[readers] = (struct reader) { + .fd = perf_data__fd(session->data), + .path = session->data->file.path, + .data_size = session->header.data_size, + .data_offset = session->header.data_offset, + .in_place_update = session->data->in_place_update, + }; + ret = reader__init(&rd[readers], NULL); + if (ret) + goto out_err; + ret = reader__mmap(&rd[readers], session); + if (ret != READER_OK) { + if (ret == READER_EOF) + ret = -EINVAL; + goto out_err; + } + readers++; + + for (i = 0; i < data->dir.nr; i++) { + if (!data->dir.files[i].size) + continue; + rd[readers] = (struct reader) { + .fd = data->dir.files[i].fd, + .path = data->dir.files[i].path, + .data_size = data->dir.files[i].size, + .data_offset = 0, + .in_place_update = session->data->in_place_update, + }; + ret = reader__init(&rd[readers], NULL); + if (ret) + goto out_err; + ret = reader__mmap(&rd[readers], session); + if (ret != READER_OK) { + if (ret == READER_EOF) + ret = -EINVAL; + goto out_err; + } + readers++; + } + + i = 0; + + while ((ret >= 0) && readers) { + if (session_done()) + return 0; + + if (rd[i].state.eof) { + i = (i + 1) % session->nr_readers; + continue; + } + + ret = reader__read_event(&rd[i], session, &prog); + if (ret < 0) + break; + if (ret == READER_NODATA) { + ret = reader__mmap(&rd[i], session); + if (ret < 0) + goto out_err; + if (ret == READER_EOF) + readers--; + } + + if (rd[i].state.size >= READER_MAX_SIZE) { + rd[i].state.size = 0; + i = (i + 1) % session->nr_readers; + } + } + + ret = ordered_events__flush(&session->ordered_events, OE_FLUSH__FINAL); + if (ret) + goto out_err; + + ret = perf_session__flush_thread_stacks(session); +out_err: + ui_progress__finish(); + + if (!tool->no_warn) + perf_session__warn_about_errors(session); + + /* + * We may switching perf.data output, make ordered_events + * reusable. + */ + ordered_events__reinit(&session->ordered_events); + + session->one_mmap = false; + + return ret; +} + int perf_session__process_events(struct perf_session *session) { if (perf_session__register_idle_thread(session) < 0) @@ -2435,6 +2563,9 @@ int perf_session__process_events(struct perf_session *session) if (perf_data__is_pipe(session->data)) return __perf_session__process_pipe_events(session); + if (perf_data__is_dir(session->data)) + return __perf_session__process_dir_events(session); + return __perf_session__process_events(session); } -- 2.19.0