[PATCH v5 0/6] perf report/script: Support percent and multiple range in --time option

[PATCH v5 0/6] perf report/script: Support percent and multiple range in --time option

[Jin Yao]

v5:
---
1. There is an issue that the sample walking can only work when
   '--buildid-all' is not enabled. So we need to let the walking
   be able to work even if '--buildid-all' is enabled and let the
   processing skips the dso hit marking for this case.

2. Check if first/last sample time is recorded in perf data file.
   If it's not recorded, return error message to user.

Patched modified in v5:
   perf record: Get the first sample time and last sample time
   perf report: support time percent and multiple time ranges
   perf script: support time percent and multiple time ranges
 
v4:
---
1. Use perf script time style for timestamp printing. Also add with
   the printing of sample duration. For example:

   perf report --header

   time of first sample : 5276531.323099
   time of last sample : 5276555.345625
   sample duration :  24022.526 ms

2. Fix an invalid time string issue. For example,

   perf script --time 10%/10x12321xsdfdasfdsafdsafdsa

   Now in code, it uses strtol to replace atoi.

3. Remove perf_time__skip_sample, only uses perf_time__ranges_skip_sample
   in perf report/perf script.

v3:
---
1. Move the definitions of first_sample_time/last_sample_time from
   perf_session and struct record to perf_evlist and update the
   related code.

v2:
---
1. This patch creates a new header feature type HEADER_SAMPLE_TIME and related
   ops. Save the first sample time and the last sample time to the feature
   section in perf file header.

2. Add checking for last element in time range.

   For example, select the first and second 10% time slices.
   perf report --time 10%/1,10%/2

   Note that now it includes the last element in [10%, 20%] but it
   doesn't include the last element in [0, 10%). It's to avoid
   the overlap.

Following patches are changed:

   perf header: Record first sample time and last sample time in perf file header
   perf record: Get the first sample time and last sample time
   perf util: Create function to perform multiple time range checking

v1: initial post
----------------

Current perf report/script/... have a --time option to limit the time
range of output. But it only supports the absolute time.

The patch series extend this option to let it support percent of time
and support the multiple time ranges.

For example:

1. Select the second 10% time slice
   perf report --time 10%/2

2. Select from 0% to 10% time slice
   perf report --time 0%-10%

It also support the multiple time ranges.

3. Select the first and second 10% time slices
   perf report --time 10%/1,10%/2

4. Select from 0% to 10% and 30% to 40% slices
   perf report --time 0%-10%,30%-40%

Jin Yao (6):
  perf header: Record first sample time and last sample time in perf
    file header
  perf record: Get the first sample time and last sample time
  perf util: Create function to parse time percent
  perf util: Create function to perform multiple time range checking
  perf report: support time percent and multiple time ranges
  perf script: support time percent and multiple time ranges

 tools/perf/Documentation/perf-report.txt           |  16 ++
 tools/perf/Documentation/perf-script.txt           |  16 ++
 tools/perf/Documentation/perf.data-file-format.txt |   4 +
 tools/perf/builtin-record.c                        |  20 +-
 tools/perf/builtin-report.c                        |  34 ++-
 tools/perf/builtin-script.c                        |  31 ++-
 tools/perf/util/evlist.h                           |   2 +
 tools/perf/util/header.c                           |  60 ++++++
 tools/perf/util/header.h                           |   1 +
 tools/perf/util/time-utils.c                       | 233 +++++++++++++++++++--
 tools/perf/util/time-utils.h                       |   6 +
 11 files changed, 394 insertions(+), 29 deletions(-)

-- 
2.7.4

[PATCH v5 1/6] perf header: Record first sample time and last sample time in perf file header

[Jin Yao]

perf report/script/... have a --time option to limit the time range
of output. That's very useful to slice large traces, e.g. when processing
the output of perf script for some analysis.

But right now --time only supports absolute time. Also there is no fast
way to get the start/end times of a given trace except for looking at it.
This makes it hard to e.g. only decode the first half of the trace, which
is useful for parallelization of scripts

Another problem is that perf records are variable size and there is no
synchronization mechanism. So the only way to find the last sample reliably
would be to walk all samples. But we want to avoid that in perf report/...
because it is already quite expensive. That is why storing the first sample
time and last sample time in perf record is better.

This patch creates a new header feature type HEADER_SAMPLE_TIME and related
ops. Save the first sample time and the last sample time to the feature
section in perf file header.

Change log:
-----------
v4: Use perf script time style for timestamp printing. Also add with
    the printing of sample duration.

v3: Remove the definitions of first_sample_time/last_sample_time from
    perf_session. Just define them in perf_evlist

Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
 tools/perf/Documentation/perf.data-file-format.txt |  4 ++
 tools/perf/util/evlist.h                           |  2 +
 tools/perf/util/header.c                           | 60 ++++++++++++++++++++++
 tools/perf/util/header.h                           |  1 +
 4 files changed, 67 insertions(+)

diff --git a/tools/perf/Documentation/perf.data-file-format.txt b/tools/perf/Documentation/perf.data-file-format.txt
index e90c59c..d5e3043 100644
--- a/tools/perf/Documentation/perf.data-file-format.txt
+++ b/tools/perf/Documentation/perf.data-file-format.txt
@@ -238,6 +238,10 @@ struct auxtrace_index {
 	struct auxtrace_index_entry entries[PERF_AUXTRACE_INDEX_ENTRY_COUNT];
 };
 
+	HEADER_SAMPLE_TIME = 21,
+
+Two uint64_t for the time of first sample and the time of last sample.
+
 	other bits are reserved and should ignored for now
 	HEADER_FEAT_BITS	= 256,
 
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index 8c433e9..a08f407 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -50,6 +50,8 @@ struct perf_evlist {
 	struct perf_evsel *selected;
 	struct events_stats stats;
 	struct perf_env	*env;
+	u64		first_sample_time;
+	u64		last_sample_time;
 };
 
 struct perf_evsel_str_handler {
diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
index 605bbd5..ced07ca 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -17,6 +17,7 @@
 #include <sys/types.h>
 #include <sys/utsname.h>
 #include <unistd.h>
+#include <linux/time64.h>
 
 #include "evlist.h"
 #include "evsel.h"
@@ -36,6 +37,7 @@
 #include <api/fs/fs.h>
 #include "asm/bug.h"
 #include "tool.h"
+#include "time-utils.h"
 
 #include "sane_ctype.h"
 
@@ -1181,6 +1183,20 @@ static int write_stat(struct feat_fd *ff __maybe_unused,
 	return 0;
 }
 
+static int write_sample_time(struct feat_fd *ff,
+			     struct perf_evlist *evlist)
+{
+	int ret;
+
+	ret = do_write(ff, &evlist->first_sample_time,
+		       sizeof(evlist->first_sample_time));
+	if (ret < 0)
+		return ret;
+
+	return do_write(ff, &evlist->last_sample_time,
+			sizeof(evlist->last_sample_time));
+}
+
 static void print_hostname(struct feat_fd *ff, FILE *fp)
 {
 	fprintf(fp, "# hostname : %s\n", ff->ph->env.hostname);
@@ -1506,6 +1522,28 @@ static void print_group_desc(struct feat_fd *ff, FILE *fp)
 	}
 }
 
+static void print_sample_time(struct feat_fd *ff, FILE *fp)
+{
+	struct perf_session *session;
+	char time_buf[32];
+	double d;
+
+	session = container_of(ff->ph, struct perf_session, header);
+
+	timestamp__scnprintf_usec(session->evlist->first_sample_time,
+				  time_buf, sizeof(time_buf));
+	fprintf(fp, "# time of first sample : %s\n", time_buf);
+
+	timestamp__scnprintf_usec(session->evlist->last_sample_time,
+				  time_buf, sizeof(time_buf));
+	fprintf(fp, "# time of last sample : %s\n", time_buf);
+
+	d = (double)(session->evlist->last_sample_time -
+		session->evlist->first_sample_time) / NSEC_PER_MSEC;
+
+	fprintf(fp, "# sample duration : %10.3f ms\n", d);
+}
+
 static int __event_process_build_id(struct build_id_event *bev,
 				    char *filename,
 				    struct perf_session *session)
@@ -2147,6 +2185,27 @@ static int process_cache(struct feat_fd *ff, void *data __maybe_unused)
 	return -1;
 }
 
+static int process_sample_time(struct feat_fd *ff, void *data __maybe_unused)
+{
+	struct perf_session *session;
+	u64 first_sample_time, last_sample_time;
+	int ret;
+
+	session = container_of(ff->ph, struct perf_session, header);
+
+	ret = do_read_u64(ff, &first_sample_time);
+	if (ret)
+		return -1;
+
+	ret = do_read_u64(ff, &last_sample_time);
+	if (ret)
+		return -1;
+
+	session->evlist->first_sample_time = first_sample_time;
+	session->evlist->last_sample_time = last_sample_time;
+	return 0;
+}
+
 struct feature_ops {
 	int (*write)(struct feat_fd *ff, struct perf_evlist *evlist);
 	void (*print)(struct feat_fd *ff, FILE *fp);
@@ -2204,6 +2263,7 @@ static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = {
 	FEAT_OPN(AUXTRACE,	auxtrace,	false),
 	FEAT_OPN(STAT,		stat,		false),
 	FEAT_OPN(CACHE,		cache,		true),
+	FEAT_OPR(SAMPLE_TIME,	sample_time,	false),
 };
 
 struct header_print_data {
diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h
index f7a16ee..1911831 100644
--- a/tools/perf/util/header.h
+++ b/tools/perf/util/header.h
@@ -33,6 +33,7 @@ enum {
 	HEADER_AUXTRACE,
 	HEADER_STAT,
 	HEADER_CACHE,
+	HEADER_SAMPLE_TIME,
 	HEADER_LAST_FEATURE,
 	HEADER_FEAT_BITS	= 256,
 };
-- 
2.7.4

[PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jin Yao]

In perf record, it's walked on all samples yet. So it's very easy to get
the first/last samples and save the time to perf file header via the
function write_sample_time().

In later, perf report/script will fetch the time from perf file header.

Change log:
-----------
v5: There is an issue that the sample walking can only work when
    '--buildid-all' is not enabled. So we need to let the walking
    be able to work even if '--buildid-all' is enabled and let the
    processing skips the dso hit marking for this case.

    At first, I want to provide a new option "--record-time-boundaries".
    While after consideration, I think a new option is not very
    necessary.

v3: Remove the definitions of first_sample_time and last_sample_time
    from struct record and directly save them in perf_evlist.

Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
 tools/perf/builtin-record.c | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index a6cbf16..bd711e8 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -391,8 +391,19 @@ static int process_sample_event(struct perf_tool *tool,
 {
 	struct record *rec = container_of(tool, struct record, tool);
 
-	rec->samples++;
+	if (rec->evlist->first_sample_time == 0)
+		rec->evlist->first_sample_time = sample->time;
+
+	rec->evlist->last_sample_time = sample->time;
+
+	/*
+	 * If --buildid-all is given, it marks all DSO regardless of hits,
+	 * so no need to process this sample.
+	 */
+	if (rec->buildid_all)
+		return 0;
 
+	rec->samples++;
 	return build_id__mark_dso_hit(tool, event, sample, evsel, machine);
 }
 
@@ -415,13 +426,6 @@ static int process_buildids(struct record *rec)
 	 */
 	symbol_conf.ignore_vmlinux_buildid = true;
 
-	/*
-	 * If --buildid-all is given, it marks all DSO regardless of hits,
-	 * so no need to process samples.
-	 */
-	if (rec->buildid_all)
-		rec->tool.sample = NULL;
-
 	return perf_session__process_events(session);
 }
 
-- 
2.7.4

[PATCH v5 3/6] perf util: Create function to parse time percent

[Jin Yao]

Current perf report/script/... have a --time option to limit the time
range of output. But right now it only supports absolute time.

For easy using, now it can support a percent of time usage.

For example:

1. Select the second 10% time slice
   perf report --time 10%/2

2. Select from 0% to 10% time slice
   perf report --time 0%-10%

It also support the multiple time ranges.

3. Select the first and second 10% time slices
   perf report --time 10%/1,10%/2

4. Select from 0% to 10% and 30% to 40% slices
   perf report --time 0%-10%,30%-40%

Change log:
-----------
v4: An issue is found. Following passes.
    perf script --time 10%/10x12321xsdfdasfdsafdsafdsa

    Now it uses strtol to replace atoi.

Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
 tools/perf/util/time-utils.c | 205 ++++++++++++++++++++++++++++++++++++++++---
 tools/perf/util/time-utils.h |   3 +
 2 files changed, 196 insertions(+), 12 deletions(-)

diff --git a/tools/perf/util/time-utils.c b/tools/perf/util/time-utils.c
index 5b5d021..79e4281 100644
--- a/tools/perf/util/time-utils.c
+++ b/tools/perf/util/time-utils.c
@@ -5,6 +5,7 @@
 #include <time.h>
 #include <errno.h>
 #include <inttypes.h>
+#include <math.h>
 
 #include "perf.h"
 #include "debug.h"
@@ -59,11 +60,10 @@ static int parse_timestr_sec_nsec(struct perf_time_interval *ptime,
 	return 0;
 }
 
-int perf_time__parse_str(struct perf_time_interval *ptime, const char *ostr)
+static int split_start_end(char **start, char **end, const char *ostr, char ch)
 {
 	char *start_str, *end_str;
 	char *d, *str;
-	int rc = 0;
 
 	if (ostr == NULL || *ostr == '\0')
 		return 0;
@@ -73,25 +73,35 @@ int perf_time__parse_str(struct perf_time_interval *ptime, const char *ostr)
 	if (str == NULL)
 		return -ENOMEM;
 
-	ptime->start = 0;
-	ptime->end = 0;
-
-	/* str has the format: <start>,<stop>
-	 * variations: <start>,
-	 *             ,<stop>
-	 *             ,
-	 */
 	start_str = str;
-	d = strchr(start_str, ',');
+	d = strchr(start_str, ch);
 	if (d) {
 		*d = '\0';
 		++d;
 	}
 	end_str = d;
 
+	*start = start_str;
+	*end = end_str;
+
+	return 0;
+}
+
+int perf_time__parse_str(struct perf_time_interval *ptime, const char *ostr)
+{
+	char *start_str = NULL, *end_str;
+	int rc;
+
+	rc = split_start_end(&start_str, &end_str, ostr, ',');
+	if (rc || !start_str)
+		return rc;
+
+	ptime->start = 0;
+	ptime->end = 0;
+
 	rc = parse_timestr_sec_nsec(ptime, start_str, end_str);
 
-	free(str);
+	free(start_str);
 
 	/* make sure end time is after start time if it was given */
 	if (rc == 0 && ptime->end && ptime->end < ptime->start)
@@ -103,6 +113,177 @@ int perf_time__parse_str(struct perf_time_interval *ptime, const char *ostr)
 	return rc;
 }
 
+static int parse_percent(double *pcnt, char *str)
+{
+	char *c;
+
+	c = strchr(str, '%');
+	if (c)
+		*c = '\0';
+	else
+		return -1;
+
+	*pcnt = atof(str) / 100.0;
+
+	return 0;
+}
+
+static int percent_slash_split(char *str, struct perf_time_interval *ptime,
+			       u64 start, u64 end)
+{
+	char *p, *end_str;
+	double pcnt, start_pcnt, end_pcnt;
+	u64 total = end - start;
+	int i;
+
+	/*
+	 * Example:
+	 * 10%/2: select the second 10% slice and the third 10% slice
+	 */
+
+	/* We can modify this string since the original one is copied */
+	p = strchr(str, '/');
+	if (!p)
+		return -1;
+
+	*p = '\0';
+	if (parse_percent(&pcnt, str) < 0)
+		return -1;
+
+	p++;
+	i = (int)strtol(p, &end_str, 10);
+	if (*end_str)
+		return -1;
+
+	if (pcnt <= 0.0)
+		return -1;
+
+	start_pcnt = pcnt * (i - 1);
+	end_pcnt = pcnt * i;
+
+	if (start_pcnt < 0.0 || start_pcnt > 1.0 ||
+	    end_pcnt < 0.0 || end_pcnt > 1.0) {
+		return -1;
+	}
+
+	ptime->start = start + round(start_pcnt * total);
+	ptime->end = start + round(end_pcnt * total);
+
+	return 0;
+}
+
+static int percent_dash_split(char *str, struct perf_time_interval *ptime,
+			      u64 start, u64 end)
+{
+	char *start_str = NULL, *end_str;
+	double start_pcnt, end_pcnt;
+	u64 total = end - start;
+	int ret;
+
+	/*
+	 * Example: 0%-10%
+	 */
+
+	ret = split_start_end(&start_str, &end_str, str, '-');
+	if (ret || !start_str)
+		return ret;
+
+	if ((parse_percent(&start_pcnt, start_str) != 0) ||
+	    (parse_percent(&end_pcnt, end_str) != 0)) {
+		free(start_str);
+		return -1;
+	}
+
+	free(start_str);
+
+	if (start_pcnt < 0.0 || start_pcnt > 1.0 ||
+	    end_pcnt < 0.0 || end_pcnt > 1.0 ||
+	    start_pcnt > end_pcnt) {
+		return -1;
+	}
+
+	ptime->start = start + round(start_pcnt * total);
+	ptime->end = start + round(end_pcnt * total);
+
+	return 0;
+}
+
+typedef int (*time_pecent_split)(char *, struct perf_time_interval *,
+				 u64 start, u64 end);
+
+static int percent_comma_split(struct perf_time_interval *ptime_buf, int num,
+			       const char *ostr, u64 start, u64 end,
+			       time_pecent_split func)
+{
+	char *str, *p1, *p2;
+	int len, ret, i = 0;
+
+	str = strdup(ostr);
+	if (str == NULL)
+		return -ENOMEM;
+
+	len = strlen(str);
+	p1 = str;
+
+	while (p1 < str + len) {
+		if (i >= num) {
+			free(str);
+			return -1;
+		}
+
+		p2 = strchr(p1, ',');
+		if (p2)
+			*p2 = '\0';
+
+		ret = (func)(p1, &ptime_buf[i], start, end);
+		if (ret < 0) {
+			free(str);
+			return -1;
+		}
+
+		pr_debug("start time %d: %" PRIu64 ", ", i, ptime_buf[i].start);
+		pr_debug("end time %d: %" PRIu64 "\n", i, ptime_buf[i].end);
+
+		i++;
+
+		if (p2)
+			p1 = p2 + 1;
+		else
+			break;
+	}
+
+	free(str);
+	return i;
+}
+
+int perf_time__percent_parse_str(struct perf_time_interval *ptime_buf, int num,
+				 const char *ostr, u64 start, u64 end)
+{
+	char *c;
+
+	/*
+	 * ostr example:
+	 * 10%/2,10%/3: select the second 10% slice and the third 10% slice
+	 * 0%-10%,30%-40%: multiple time range
+	 */
+
+	memset(ptime_buf, 0, sizeof(*ptime_buf) * num);
+
+	c = strchr(ostr, '/');
+	if (c) {
+		return percent_comma_split(ptime_buf, num, ostr, start,
+					   end, percent_slash_split);
+	}
+
+	c = strchr(ostr, '-');
+	if (c) {
+		return percent_comma_split(ptime_buf, num, ostr, start,
+					   end, percent_dash_split);
+	}
+
+	return -1;
+}
+
 bool perf_time__skip_sample(struct perf_time_interval *ptime, u64 timestamp)
 {
 	/* if time is not set don't drop sample */
diff --git a/tools/perf/util/time-utils.h b/tools/perf/util/time-utils.h
index 8656be0..fd018e2 100644
--- a/tools/perf/util/time-utils.h
+++ b/tools/perf/util/time-utils.h
@@ -12,6 +12,9 @@ int parse_nsec_time(const char *str, u64 *ptime);
 
 int perf_time__parse_str(struct perf_time_interval *ptime, const char *ostr);
 
+int perf_time__percent_parse_str(struct perf_time_interval *ptime_buf, int num,
+				 const char *ostr, u64 start, u64 end);
+
 bool perf_time__skip_sample(struct perf_time_interval *ptime, u64 timestamp);
 
 int timestamp__scnprintf_usec(u64 timestamp, char *buf, size_t sz);
-- 
2.7.4

[PATCH v5 4/6] perf util: Create function to perform multiple time range checking

[Jin Yao]

Previous patch supports the multiple time range.

For example, select the first and second 10% time slices.
perf report --time 10%/1,10%/2

We need a function to check if a timestamp is in the ranges of
[0, 10%) and [10%, 20%].

Note that it includes the last element in [10%, 20%] but it
doesn't include the last element in [0, 10%). It's to avoid
the overlap.

This patch implments a new function perf_time__ranges_skip_sample
for this checking.

Change log:
-----------
v4: Let perf_time__ranges_skip_sample be compatible with
    perf_time__skip_sample when only one time range.

Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
 tools/perf/util/time-utils.c | 28 ++++++++++++++++++++++++++++
 tools/perf/util/time-utils.h |  3 +++
 2 files changed, 31 insertions(+)

diff --git a/tools/perf/util/time-utils.c b/tools/perf/util/time-utils.c
index 79e4281..b380356 100644
--- a/tools/perf/util/time-utils.c
+++ b/tools/perf/util/time-utils.c
@@ -299,6 +299,34 @@ bool perf_time__skip_sample(struct perf_time_interval *ptime, u64 timestamp)
 	return false;
 }
 
+bool perf_time__ranges_skip_sample(struct perf_time_interval *ptime_buf,
+				   int num, u64 timestamp)
+{
+	struct perf_time_interval *ptime;
+	int i;
+
+	if ((timestamp == 0) || (num == 0))
+		return false;
+
+	if (num == 1)
+		return perf_time__skip_sample(&ptime_buf[0], timestamp);
+
+	/*
+	 * start/end of multiple time ranges must be valid.
+	 */
+	for (i = 0; i < num; i++) {
+		ptime = &ptime_buf[i];
+
+		if (timestamp >= ptime->start &&
+		    ((timestamp < ptime->end && i < num - 1) ||
+		     (timestamp <= ptime->end && i == num - 1))) {
+			break;
+		}
+	}
+
+	return (i == num) ? true : false;
+}
+
 int timestamp__scnprintf_usec(u64 timestamp, char *buf, size_t sz)
 {
 	u64  sec = timestamp / NSEC_PER_SEC;
diff --git a/tools/perf/util/time-utils.h b/tools/perf/util/time-utils.h
index fd018e2..de279ea 100644
--- a/tools/perf/util/time-utils.h
+++ b/tools/perf/util/time-utils.h
@@ -17,6 +17,9 @@ int perf_time__percent_parse_str(struct perf_time_interval *ptime_buf, int num,
 
 bool perf_time__skip_sample(struct perf_time_interval *ptime, u64 timestamp);
 
+bool perf_time__ranges_skip_sample(struct perf_time_interval *ptime_buf,
+				   int num, u64 timestamp);
+
 int timestamp__scnprintf_usec(u64 timestamp, char *buf, size_t sz);
 
 int fetch_current_timestamp(char *buf, size_t sz);
-- 
2.7.4

[PATCH v5 5/6] perf report: support time percent and multiple time ranges

[Jin Yao]

perf report has a --time option to limit the time range of output.
It only supports absolute time.

Now this option is extended to support multiple time ranges and
support the percent of time.

For example:

1. Select the first and second 10% time slices
perf report --time 10%/1,10%/2

2. Select from 0% to 10% and 30% to 40% slices
perf report --time 0%-10%,30%-40%

Change log:
-----------
v5: Add checking of first/last sample time to detect if it's recorded
    in perf.data. If it's not recorded, returns error message to user.

v4: Remove perf_time__skip_sample, only uses perf_time__ranges_skip_sample

v3: Since the definitions of first_sample_time/last_sample_time
    are moved from perf_session to perf_evlist so change the
    related code.

Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
 tools/perf/Documentation/perf-report.txt | 16 +++++++++++++++
 tools/perf/builtin-report.c              | 34 ++++++++++++++++++++++++++++----
 2 files changed, 46 insertions(+), 4 deletions(-)

diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt
index 383a98d..3a0975c 100644
--- a/tools/perf/Documentation/perf-report.txt
+++ b/tools/perf/Documentation/perf-report.txt
@@ -402,6 +402,22 @@ OPTIONS
 	stop time is not given (i.e, time string is 'x.y,') then analysis goes
 	to end of file.
 
+	Also support time percent with multipe time range. Time string is
+	'a%/n,b%/m,...' or 'a%-b%,c%-%d,...'. The maximum number of slices is 10.
+
+	For example:
+	Select the second 10% time slice
+	perf report --time 10%/2
+
+	Select from 0% to 10% time slice
+	perf report --time 0%-10%
+
+	Select the first and second 10% time slices
+	perf report --time 10%/1,10%/2
+
+	Select from 0% to 10% and 30% to 40% slices
+	perf report --time 0%-10%,30%-40%
+
 --itrace::
 	Options for decoding instruction tracing data. The options are:
 
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index f9dff65..3b5fc1a 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -51,6 +51,8 @@
 #include <sys/stat.h>
 #include <unistd.h>
 
+#define PTIME_RANGE_MAX	10
+
 struct report {
 	struct perf_tool	tool;
 	struct perf_session	*session;
@@ -68,7 +70,8 @@ struct report {
 	const char		*cpu_list;
 	const char		*symbol_filter_str;
 	const char		*time_str;
-	struct perf_time_interval ptime;
+	struct perf_time_interval ptime_range[PTIME_RANGE_MAX];
+	int			range_num;
 	float			min_percent;
 	u64			nr_entries;
 	u64			queue_size;
@@ -185,8 +188,10 @@ static int process_sample_event(struct perf_tool *tool,
 	};
 	int ret = 0;
 
-	if (perf_time__skip_sample(&rep->ptime, sample->time))
+	if (perf_time__ranges_skip_sample(rep->ptime_range, rep->range_num,
+					  sample->time)) {
 		return 0;
+	}
 
 	if (machine__resolve(machine, &al, sample) < 0) {
 		pr_debug("problem processing %d event, skipping it.\n",
@@ -1073,8 +1078,29 @@ int cmd_report(int argc, const char **argv)
 	if (symbol__init(&session->header.env) < 0)
 		goto error;
 
-	if (perf_time__parse_str(&report.ptime, report.time_str) != 0) {
-		pr_err("Invalid time string\n");
+	if (perf_time__parse_str(report.ptime_range, report.time_str) != 0) {
+		if (session->evlist->first_sample_time == 0 &&
+		    session->evlist->last_sample_time == 0) {
+			pr_err("No first/last sample time in perf data\n");
+			return -EINVAL;
+		}
+
+		report.range_num = perf_time__percent_parse_str(
+					report.ptime_range, PTIME_RANGE_MAX,
+					report.time_str,
+					session->evlist->first_sample_time,
+					session->evlist->last_sample_time);
+
+		if (report.range_num < 0) {
+			pr_err("Invalid time string\n");
+			return -EINVAL;
+		}
+	} else {
+		report.range_num = 1;
+	}
+
+	if (report.range_num > 0 && perf_data_file__is_pipe(session->file)) {
+		pr_err("Time percent range is not supported in pipe\n");
 		return -EINVAL;
 	}
 
-- 
2.7.4

[PATCH v5 6/6] perf script: support time percent and multiple time ranges

[Jin Yao]

perf script has a --time option to limit the time range of output.
It only supports absolute time.

Now this option is extended to support multiple time ranges and
support the percent of time.

For example:

1. Select the first and second 10% time slices
   perf script --time 10%/1,10%/2

2. Select from 0% to 10% and 30% to 40% slices
   perf script --time 0%-10%,30%-40%

Change log:
-----------
v5: Add checking of first/last sample time to detect if it's recorded
    in perf.data. If it's not recorded, returns error message to user.

v4: Remove perf_time__skip_sample, only uses perf_time__ranges_skip_sample

v3: Since the definitions of first_sample_time/last_sample_time
    are moved from perf_session to perf_evlist so change the
    related code.

Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
 tools/perf/Documentation/perf-script.txt | 16 ++++++++++++++++
 tools/perf/builtin-script.c              | 31 ++++++++++++++++++++++++++-----
 2 files changed, 42 insertions(+), 5 deletions(-)

diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index bcc1ba3..2c1f2b9 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -321,6 +321,22 @@ include::itrace.txt[]
 	stop time is not given (i.e, time string is 'x.y,') then analysis goes
 	to end of file.
 
+	Also support time percent with multipe time range. Time string is
+	'a%/n,b%/m,...' or 'a%-b%,c%-%d,...'. The maximum number of slices is 10.
+
+	For example:
+	Select the second 10% time slice
+	perf script --time 10%/2
+
+	Select from 0% to 10% time slice
+	perf script --time 0%-10%
+
+	Select the first and second 10% time slices
+	perf script --time 10%/1,10%/2
+
+	Select from 0% to 10% and 30% to 40% slices
+	perf script --time 0%-10%,30%-40%
+
 --max-blocks::
 	Set the maximum number of program blocks to print with brstackasm for
 	each sample.
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 3e83f47..d7debc4 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -1385,6 +1385,8 @@ static int perf_sample__fprintf_synth(struct perf_sample *sample,
 	return 0;
 }
 
+#define PTIME_RANGE_MAX	10
+
 struct perf_script {
 	struct perf_tool	tool;
 	struct perf_session	*session;
@@ -1397,7 +1399,8 @@ struct perf_script {
 	struct thread_map	*threads;
 	int			name_width;
 	const char              *time_str;
-	struct perf_time_interval ptime;
+	struct perf_time_interval ptime_range[PTIME_RANGE_MAX];
+	int			range_num;
 };
 
 static int perf_evlist__max_name_len(struct perf_evlist *evlist)
@@ -1594,8 +1597,10 @@ static int process_sample_event(struct perf_tool *tool,
 	struct perf_script *scr = container_of(tool, struct perf_script, tool);
 	struct addr_location al;
 
-	if (perf_time__skip_sample(&scr->ptime, sample->time))
+	if (perf_time__ranges_skip_sample(scr->ptime_range, scr->range_num,
+					  sample->time)) {
 		return 0;
+	}
 
 	if (debug_mode) {
 		if (sample->time < last_timestamp) {
@@ -3132,9 +3137,25 @@ int cmd_script(int argc, const char **argv)
 		goto out_delete;
 
 	/* needs to be parsed after looking up reference time */
-	if (perf_time__parse_str(&script.ptime, script.time_str) != 0) {
-		pr_err("Invalid time string\n");
-		return -EINVAL;
+	if (perf_time__parse_str(script.ptime_range, script.time_str) != 0) {
+		if (session->evlist->first_sample_time == 0 &&
+		    session->evlist->last_sample_time == 0) {
+			pr_err("No first/last sample time in perf data\n");
+			return -EINVAL;
+		}
+
+		script.range_num = perf_time__percent_parse_str(
+					script.ptime_range, PTIME_RANGE_MAX,
+					script.time_str,
+					session->evlist->first_sample_time,
+					session->evlist->last_sample_time);
+
+		if (script.range_num < 0) {
+			pr_err("Invalid time string\n");
+			return -EINVAL;
+		}
+	} else {
+		script.range_num = 1;
 	}
 
 	err = __cmd_script(&script);
-- 
2.7.4

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jiri Olsa]

On Sat, Oct 21, 2017 at 07:27:50AM +0800, Jin Yao wrote:
> In perf record, it's walked on all samples yet. So it's very easy to get
> the first/last samples and save the time to perf file header via the
> function write_sample_time().
> 
> In later, perf report/script will fetch the time from perf file header.
> 
> Change log:
> -----------
> v5: There is an issue that the sample walking can only work when
>     '--buildid-all' is not enabled. So we need to let the walking
>     be able to work even if '--buildid-all' is enabled and let the
>     processing skips the dso hit marking for this case.
> 
>     At first, I want to provide a new option "--record-time-boundaries".
>     While after consideration, I think a new option is not very
>     necessary.
> 
> v3: Remove the definitions of first_sample_time and last_sample_time
>     from struct record and directly save them in perf_evlist.
> 
> Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
> ---
>  tools/perf/builtin-record.c | 20 ++++++++++++--------
>  1 file changed, 12 insertions(+), 8 deletions(-)
> 
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index a6cbf16..bd711e8 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -391,8 +391,19 @@ static int process_sample_event(struct perf_tool *tool,
>  {
>  	struct record *rec = container_of(tool, struct record, tool);
>  
> -	rec->samples++;
> +	if (rec->evlist->first_sample_time == 0)
> +		rec->evlist->first_sample_time = sample->time;
> +
> +	rec->evlist->last_sample_time = sample->time;
> +
> +	/*
> +	 * If --buildid-all is given, it marks all DSO regardless of hits,
> +	 * so no need to process this sample.
> +	 */
> +	if (rec->buildid_all)
> +		return 0;
>  
> +	rec->samples++;
>  	return build_id__mark_dso_hit(tool, event, sample, evsel, machine);
>  }
>  
> @@ -415,13 +426,6 @@ static int process_buildids(struct record *rec)
>  	 */
>  	symbol_conf.ignore_vmlinux_buildid = true;
>  
> -	/*
> -	 * If --buildid-all is given, it marks all DSO regardless of hits,
> -	 * so no need to process samples.
> -	 */
> -	if (rec->buildid_all)
> -		rec->tool.sample = NULL;

hum, could you still unset the sample if there's no time given?
and keep the speed in this case..

jirka

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jin, Yao]

On 10/23/2017 11:04 PM, Jiri Olsa wrote:
> On Sat, Oct 21, 2017 at 07:27:50AM +0800, Jin Yao wrote:
>> In perf record, it's walked on all samples yet. So it's very easy to get
>> the first/last samples and save the time to perf file header via the
>> function write_sample_time().
>>
>> In later, perf report/script will fetch the time from perf file header.
>>
>> Change log:
>> -----------
>> v5: There is an issue that the sample walking can only work when
>>      '--buildid-all' is not enabled. So we need to let the walking
>>      be able to work even if '--buildid-all' is enabled and let the
>>      processing skips the dso hit marking for this case.
>>
>>      At first, I want to provide a new option "--record-time-boundaries".
>>      While after consideration, I think a new option is not very
>>      necessary.
>>
>> v3: Remove the definitions of first_sample_time and last_sample_time
>>      from struct record and directly save them in perf_evlist.
>>
>> Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
>> ---
>>   tools/perf/builtin-record.c | 20 ++++++++++++--------
>>   1 file changed, 12 insertions(+), 8 deletions(-)
>>
>> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
>> index a6cbf16..bd711e8 100644
>> --- a/tools/perf/builtin-record.c
>> +++ b/tools/perf/builtin-record.c
>> @@ -391,8 +391,19 @@ static int process_sample_event(struct perf_tool *tool,
>>   {
>>   	struct record *rec = container_of(tool, struct record, tool);
>>   
>> -	rec->samples++;
>> +	if (rec->evlist->first_sample_time == 0)
>> +		rec->evlist->first_sample_time = sample->time;
>> +
>> +	rec->evlist->last_sample_time = sample->time;
>> +
>> +	/*
>> +	 * If --buildid-all is given, it marks all DSO regardless of hits,
>> +	 * so no need to process this sample.
>> +	 */
>> +	if (rec->buildid_all)
>> +		return 0;
>>   
>> +	rec->samples++;
>>   	return build_id__mark_dso_hit(tool, event, sample, evsel, machine);
>>   }
>>   
>> @@ -415,13 +426,6 @@ static int process_buildids(struct record *rec)
>>   	 */
>>   	symbol_conf.ignore_vmlinux_buildid = true;
>>   
>> -	/*
>> -	 * If --buildid-all is given, it marks all DSO regardless of hits,
>> -	 * so no need to process samples.
>> -	 */
>> -	if (rec->buildid_all)
>> -		rec->tool.sample = NULL;
> 
> hum, could you still unset the sample if there's no time given?
> and keep the speed in this case..
> 
> jirka
> 

Oh, yes, I should do that. Thanks Jiri!

Hi Arnaldo, do you have any comment for v5?

I will prepare v6 for adding fixes according to all comments.

Thanks
Jin Yao

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Arnaldo Carvalho de Melo]

Em Tue, Oct 24, 2017 at 01:38:26AM +0800, Jin, Yao escreveu:
> 
> 
> On 10/23/2017 11:04 PM, Jiri Olsa wrote:
> > On Sat, Oct 21, 2017 at 07:27:50AM +0800, Jin Yao wrote:
> > > In perf record, it's walked on all samples yet. So it's very easy to get
> > > the first/last samples and save the time to perf file header via the
> > > function write_sample_time().
> > > 
> > > In later, perf report/script will fetch the time from perf file header.
> > > 
> > > Change log:
> > > -----------
> > > v5: There is an issue that the sample walking can only work when
> > >      '--buildid-all' is not enabled. So we need to let the walking
> > >      be able to work even if '--buildid-all' is enabled and let the
> > >      processing skips the dso hit marking for this case.
> > > 
> > >      At first, I want to provide a new option "--record-time-boundaries".
> > >      While after consideration, I think a new option is not very
> > >      necessary.
> > > 
> > > v3: Remove the definitions of first_sample_time and last_sample_time
> > >      from struct record and directly save them in perf_evlist.
> > > 
> > > Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
> > > ---
> > >   tools/perf/builtin-record.c | 20 ++++++++++++--------
> > >   1 file changed, 12 insertions(+), 8 deletions(-)
> > > 
> > > diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> > > index a6cbf16..bd711e8 100644
> > > --- a/tools/perf/builtin-record.c
> > > +++ b/tools/perf/builtin-record.c
> > > @@ -391,8 +391,19 @@ static int process_sample_event(struct perf_tool *tool,
> > >   {
> > >   	struct record *rec = container_of(tool, struct record, tool);
> > > -	rec->samples++;
> > > +	if (rec->evlist->first_sample_time == 0)
> > > +		rec->evlist->first_sample_time = sample->time;
> > > +
> > > +	rec->evlist->last_sample_time = sample->time;
> > > +
> > > +	/*
> > > +	 * If --buildid-all is given, it marks all DSO regardless of hits,
> > > +	 * so no need to process this sample.
> > > +	 */
> > > +	if (rec->buildid_all)
> > > +		return 0;
> > > +	rec->samples++;
> > >   	return build_id__mark_dso_hit(tool, event, sample, evsel, machine);
> > >   }
> > > @@ -415,13 +426,6 @@ static int process_buildids(struct record *rec)
> > >   	 */
> > >   	symbol_conf.ignore_vmlinux_buildid = true;
> > > -	/*
> > > -	 * If --buildid-all is given, it marks all DSO regardless of hits,
> > > -	 * so no need to process samples.
> > > -	 */
> > > -	if (rec->buildid_all)
> > > -		rec->tool.sample = NULL;
> > 
> > hum, could you still unset the sample if there's no time given?
> > and keep the speed in this case..
> > 
> > jirka
> > 
> 
> Oh, yes, I should do that. Thanks Jiri!
> 
> Hi Arnaldo, do you have any comment for v5?

I'll try to look at it, but don't that hold v6, I'd say :-\

- Arnaldo
 
> I will prepare v6 for adding fixes according to all comments.
> 
> Thanks
> Jin Yao

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jin, Yao]

On 10/23/2017 11:04 PM, Jiri Olsa wrote:
> On Sat, Oct 21, 2017 at 07:27:50AM +0800, Jin Yao wrote:
>> In perf record, it's walked on all samples yet. So it's very easy to get
>> the first/last samples and save the time to perf file header via the
>> function write_sample_time().
>>
>> In later, perf report/script will fetch the time from perf file header.
>>
>> Change log:
>> -----------
>> v5: There is an issue that the sample walking can only work when
>>      '--buildid-all' is not enabled. So we need to let the walking
>>      be able to work even if '--buildid-all' is enabled and let the
>>      processing skips the dso hit marking for this case.
>>
>>      At first, I want to provide a new option "--record-time-boundaries".
>>      While after consideration, I think a new option is not very
>>      necessary.
>>
>> v3: Remove the definitions of first_sample_time and last_sample_time
>>      from struct record and directly save them in perf_evlist.
>>
>> Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
>> ---
>>   tools/perf/builtin-record.c | 20 ++++++++++++--------
>>   1 file changed, 12 insertions(+), 8 deletions(-)
>>
>> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
>> index a6cbf16..bd711e8 100644
>> --- a/tools/perf/builtin-record.c
>> +++ b/tools/perf/builtin-record.c
>> @@ -391,8 +391,19 @@ static int process_sample_event(struct perf_tool *tool,
>>   {
>>   	struct record *rec = container_of(tool, struct record, tool);
>>   
>> -	rec->samples++;
>> +	if (rec->evlist->first_sample_time == 0)
>> +		rec->evlist->first_sample_time = sample->time;
>> +
>> +	rec->evlist->last_sample_time = sample->time;
>> +
>> +	/*
>> +	 * If --buildid-all is given, it marks all DSO regardless of hits,
>> +	 * so no need to process this sample.
>> +	 */
>> +	if (rec->buildid_all)
>> +		return 0;
>>   
>> +	rec->samples++;
>>   	return build_id__mark_dso_hit(tool, event, sample, evsel, machine);
>>   }
>>   
>> @@ -415,13 +426,6 @@ static int process_buildids(struct record *rec)
>>   	 */
>>   	symbol_conf.ignore_vmlinux_buildid = true;
>>   
>> -	/*
>> -	 * If --buildid-all is given, it marks all DSO regardless of hits,
>> -	 * so no need to process samples.
>> -	 */
>> -	if (rec->buildid_all)
>> -		rec->tool.sample = NULL;
> 
> hum, could you still unset the sample if there's no time given?
> and keep the speed in this case..
> 
> jirka
> 

Hi Jiri,

I check this question again. The '--time' option is for perf report but 
not for perf record.

For perf record, we have to always walk on all samples to get the time 
of first sample and the time of last sample whatever buildid_all is 
enabled or not enabled. So 'rec->tool.sample = NULL' is removed.

Sorry, the previous mail was replied at midnight, I was drowsy. :(

If my answer is correct, I will not send v6. If my understanding is 
still not correct, please let me know.

Thanks
Jin Yao

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jiri Olsa]

On Tue, Oct 24, 2017 at 10:03:05AM +0800, Jin, Yao wrote:

SNIP

> > hum, could you still unset the sample if there's no time given?
> > and keep the speed in this case..
> > 
> > jirka
> > 
> 
> Hi Jiri,
> 
> I check this question again. The '--time' option is for perf report but not
> for perf record.
> 
> For perf record, we have to always walk on all samples to get the time of
> first sample and the time of last sample whatever buildid_all is enabled or
> not enabled. So 'rec->tool.sample = NULL' is removed.
> 
> Sorry, the previous mail was replied at midnight, I was drowsy. :(
> 
> If my answer is correct, I will not send v6. If my understanding is still
> not correct, please let me know.

right, I did not realize we store this unconditionaly.. then yes, it's ok

I think I've already acked this, anyway for the patchset:

Acked-by: Jiri Olsa <jolsa@kernel.org>

thanks,
jirka

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jin, Yao]

On 10/24/2017 3:16 PM, Jiri Olsa wrote:
> On Tue, Oct 24, 2017 at 10:03:05AM +0800, Jin, Yao wrote:
> 
> SNIP
> 
>>> hum, could you still unset the sample if there's no time given?
>>> and keep the speed in this case..
>>>
>>> jirka
>>>
>>
>> Hi Jiri,
>>
>> I check this question again. The '--time' option is for perf report but not
>> for perf record.
>>
>> For perf record, we have to always walk on all samples to get the time of
>> first sample and the time of last sample whatever buildid_all is enabled or
>> not enabled. So 'rec->tool.sample = NULL' is removed.
>>
>> Sorry, the previous mail was replied at midnight, I was drowsy. :(
>>
>> If my answer is correct, I will not send v6. If my understanding is still
>> not correct, please let me know.
> 
> right, I did not realize we store this unconditionaly.. then yes, it's ok
> 
> I think I've already acked this, anyway for the patchset:
> 
> Acked-by: Jiri Olsa <jolsa@kernel.org>
> 
> thanks,
> jirka
> 

Hi Jiri,

Thanks a lot for your review comments and your helps!

Thanks
Jin Yao

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jin, Yao]

On 10/24/2017 3:16 PM, Jiri Olsa wrote:
> On Tue, Oct 24, 2017 at 10:03:05AM +0800, Jin, Yao wrote:
> 
> SNIP
> 
>>> hum, could you still unset the sample if there's no time given?
>>> and keep the speed in this case..
>>>
>>> jirka
>>>
>>
>> Hi Jiri,
>>
>> I check this question again. The '--time' option is for perf report but not
>> for perf record.
>>
>> For perf record, we have to always walk on all samples to get the time of
>> first sample and the time of last sample whatever buildid_all is enabled or
>> not enabled. So 'rec->tool.sample = NULL' is removed.
>>
>> Sorry, the previous mail was replied at midnight, I was drowsy. :(
>>
>> If my answer is correct, I will not send v6. If my understanding is still
>> not correct, please let me know.
> 
> right, I did not realize we store this unconditionaly.. then yes, it's ok
> 
> I think I've already acked this, anyway for the patchset:
> 
> Acked-by: Jiri Olsa <jolsa@kernel.org>
> 
> thanks,
> jirka
> 

Hi Arnaldo,

Is this patch-set OK for merging or anything I should improve?

Thanks
Jin Yao

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Arnaldo Carvalho de Melo]

Em Tue, Oct 24, 2017 at 09:16:59AM +0200, Jiri Olsa escreveu:
> On Tue, Oct 24, 2017 at 10:03:05AM +0800, Jin, Yao wrote:
> 
> SNIP
> 
> > > hum, could you still unset the sample if there's no time given?
> > > and keep the speed in this case..
> > > 
> > > jirka
> > > 
> > 
> > Hi Jiri,
> > 
> > I check this question again. The '--time' option is for perf report but not
> > for perf record.
> > 
> > For perf record, we have to always walk on all samples to get the time of
> > first sample and the time of last sample whatever buildid_all is enabled or
> > not enabled. So 'rec->tool.sample = NULL' is removed.
> > 
> > Sorry, the previous mail was replied at midnight, I was drowsy. :(
> > 
> > If my answer is correct, I will not send v6. If my understanding is still
> > not correct, please let me know.
> 
> right, I did not realize we store this unconditionaly.. then yes, it's ok

And should we store this unconditionally? What this patch is doing is
making 'perf record' unconditionally slower so that the generated
perf.data file becomes useful for some usecases, but not for all, only
people interested in using 'perf report/script --time' will benefit,
right?

I thought that we could get this sorted out in a different fashion, i.e.
getting the first timestamp is easy, even if we don't process build-ids,
right? To get the last one we could ask the kernel to insert an extra
dummy sample at the end, one that we know the size and thus can to to
the end of the file, rewind that size, get the event and parse the
sample, agreed?

So I suggest that first make this conditional, i.e. 'perf record
--timestamps' will enable the logic you implemented, and as a followup,
if you agree, add the dummy, known size event at the end, and then even
when build-ids are not processed, the cost for getting the timestamps
will be next to zero.

- Arnaldo

- Arnaldo
 
> I think I've already acked this, anyway for the patchset:
> 
> Acked-by: Jiri Olsa <jolsa@kernel.org>
> 
> thanks,
> jirka

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jin, Yao]

On 11/4/2017 12:29 AM, Arnaldo Carvalho de Melo wrote:
> Em Tue, Oct 24, 2017 at 09:16:59AM +0200, Jiri Olsa escreveu:
>> On Tue, Oct 24, 2017 at 10:03:05AM +0800, Jin, Yao wrote:
>>
>> SNIP
>>
>>>> hum, could you still unset the sample if there's no time given?
>>>> and keep the speed in this case..
>>>>
>>>> jirka
>>>>
>>>
>>> Hi Jiri,
>>>
>>> I check this question again. The '--time' option is for perf report but not
>>> for perf record.
>>>
>>> For perf record, we have to always walk on all samples to get the time of
>>> first sample and the time of last sample whatever buildid_all is enabled or
>>> not enabled. So 'rec->tool.sample = NULL' is removed.
>>>
>>> Sorry, the previous mail was replied at midnight, I was drowsy. :(
>>>
>>> If my answer is correct, I will not send v6. If my understanding is still
>>> not correct, please let me know.
>>
>> right, I did not realize we store this unconditionaly.. then yes, it's ok
> 
> And should we store this unconditionally? What this patch is doing is
> making 'perf record' unconditionally slower so that the generated
> perf.data file becomes useful for some usecases, but not for all, only
> people interested in using 'perf report/script --time' will benefit,
> right?
> 

Yes, that makes sense.

> I thought that we could get this sorted out in a different fashion, i.e.
> getting the first timestamp is easy, even if we don't process build-ids,
> right? To get the last one we could ask the kernel to insert an extra
> dummy sample at the end, one that we know the size and thus can to to
> the end of the file, rewind that size, get the event and parse the
> sample, agreed?
> 

Yes, agree.

> So I suggest that first make this conditional, i.e. 'perf record
> --timestamps' will enable the logic you implemented, and as a followup,
> if you agree, add the dummy, known size event at the end, and then even
> when build-ids are not processed, the cost for getting the timestamps
> will be next to zero.
> 

So we will use 2 steps for the implementation.

Step1:
Upgrade this patch series to v6. The change is to add "--timestamps" 
option in perf record.

Step2:
As a followup patch series, let the kernel generate a dummy sample at 
the end. That maybe needs more discussions on what the format we should 
use in dummy sample and how to generate the dummy sample in kernel.

Let me complete the step1 first.

Thanks
Jin Yao

> - Arnaldo
> 
> - Arnaldo
>   
>> I think I've already acked this, anyway for the patchset:
>>
>> Acked-by: Jiri Olsa <jolsa@kernel.org>
>>
>> thanks,
>> jirka

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jiri Olsa]

On Fri, Nov 03, 2017 at 01:29:42PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Tue, Oct 24, 2017 at 09:16:59AM +0200, Jiri Olsa escreveu:
> > On Tue, Oct 24, 2017 at 10:03:05AM +0800, Jin, Yao wrote:
> > 
> > SNIP
> > 
> > > > hum, could you still unset the sample if there's no time given?
> > > > and keep the speed in this case..
> > > > 
> > > > jirka
> > > > 
> > > 
> > > Hi Jiri,
> > > 
> > > I check this question again. The '--time' option is for perf report but not
> > > for perf record.
> > > 
> > > For perf record, we have to always walk on all samples to get the time of
> > > first sample and the time of last sample whatever buildid_all is enabled or
> > > not enabled. So 'rec->tool.sample = NULL' is removed.
> > > 
> > > Sorry, the previous mail was replied at midnight, I was drowsy. :(
> > > 
> > > If my answer is correct, I will not send v6. If my understanding is still
> > > not correct, please let me know.
> > 
> > right, I did not realize we store this unconditionaly.. then yes, it's ok
> 
> And should we store this unconditionally? What this patch is doing is
> making 'perf record' unconditionally slower so that the generated
> perf.data file becomes useful for some usecases, but not for all, only
> people interested in using 'perf report/script --time' will benefit,
> right?

maybe we can also silently enable that when processing buildids,
(which is set by default), there's no big performance hit once
we already go through samples

jirka

> 
> I thought that we could get this sorted out in a different fashion, i.e.
> getting the first timestamp is easy, even if we don't process build-ids,
> right? To get the last one we could ask the kernel to insert an extra
> dummy sample at the end, one that we know the size and thus can to to
> the end of the file, rewind that size, get the event and parse the
> sample, agreed?
> 
> So I suggest that first make this conditional, i.e. 'perf record
> --timestamps' will enable the logic you implemented, and as a followup,
> if you agree, add the dummy, known size event at the end, and then even
> when build-ids are not processed, the cost for getting the timestamps
> will be next to zero.
> 
> - Arnaldo
> 
> - Arnaldo
>  
> > I think I've already acked this, anyway for the patchset:
> > 
> > Acked-by: Jiri Olsa <jolsa@kernel.org>
> > 
> > thanks,
> > jirka

Re: [PATCH v5 2/6] perf record: Get the first sample time and last sample time

[Jin, Yao]

On 11/4/2017 6:24 PM, Jiri Olsa wrote:
> On Fri, Nov 03, 2017 at 01:29:42PM -0300, Arnaldo Carvalho de Melo wrote:
>> Em Tue, Oct 24, 2017 at 09:16:59AM +0200, Jiri Olsa escreveu:
>>> On Tue, Oct 24, 2017 at 10:03:05AM +0800, Jin, Yao wrote:
>>>
>>> SNIP
>>>
>>>>> hum, could you still unset the sample if there's no time given?
>>>>> and keep the speed in this case..
>>>>>
>>>>> jirka
>>>>>
>>>>
>>>> Hi Jiri,
>>>>
>>>> I check this question again. The '--time' option is for perf report but not
>>>> for perf record.
>>>>
>>>> For perf record, we have to always walk on all samples to get the time of
>>>> first sample and the time of last sample whatever buildid_all is enabled or
>>>> not enabled. So 'rec->tool.sample = NULL' is removed.
>>>>
>>>> Sorry, the previous mail was replied at midnight, I was drowsy. :(
>>>>
>>>> If my answer is correct, I will not send v6. If my understanding is still
>>>> not correct, please let me know.
>>>
>>> right, I did not realize we store this unconditionaly.. then yes, it's ok
>>
>> And should we store this unconditionally? What this patch is doing is
>> making 'perf record' unconditionally slower so that the generated
>> perf.data file becomes useful for some usecases, but not for all, only
>> people interested in using 'perf report/script --time' will benefit,
>> right?
> 
> maybe we can also silently enable that when processing buildids,
> (which is set by default), there's no big performance hit once
> we already go through samples
> 
> jirka
> 

It's a good idea. Default enabling --timestamps in perf record since 
buildids is enabled by default as well.

But if buildids is not enabled, then it needs to check if --timestamps 
is enabled. I will follow this rule in v6.

Thanks
Jin Yao

>>
>> I thought that we could get this sorted out in a different fashion, i.e.
>> getting the first timestamp is easy, even if we don't process build-ids,
>> right? To get the last one we could ask the kernel to insert an extra
>> dummy sample at the end, one that we know the size and thus can to to
>> the end of the file, rewind that size, get the event and parse the
>> sample, agreed?
>>
>> So I suggest that first make this conditional, i.e. 'perf record
>> --timestamps' will enable the logic you implemented, and as a followup,
>> if you agree, add the dummy, known size event at the end, and then even
>> when build-ids are not processed, the cost for getting the timestamps
>> will be next to zero.
>>
>> - Arnaldo
>>
>> - Arnaldo
>>   
>>> I think I've already acked this, anyway for the patchset:
>>>
>>> Acked-by: Jiri Olsa <jolsa@kernel.org>
>>>
>>> thanks,
>>> jirka