[PATCH 0/5] perf record: Allow to ignore missing pid

[PATCH 0/5] perf record: Allow to ignore missing pid

[Jiri Olsa]

hi,
this patches allows record's -u option to ignore group of
pids, that 'disappear' before perf opens their event.

This currently leads perf to report error and exit.

With this change we will allow this race and ignore such
failure with warning:

  $ perf record -u
  WARNING: Ignored open failure for pid 8605
  ...

and continue monitoring whatever portion of processes
we managed to open.

Plus one extra perf mem fix.

Available also here:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  perf/fixes

thanks,
jirka


---
Jiri Olsa (5):
      perf mem: Fix --all-user/--all-kernel options
      perf tools: Use variable instead of repeating lengthy FD macro
      perf tools: Add thread_map__remove function
      perf tools: Allow to ignore missing pid
      perf record: Force ignore_missing_thread for uid option

 tools/perf/builtin-mem.c        |  4 ++--
 tools/perf/builtin-record.c     |  3 +++
 tools/perf/perf.h               |  1 +
 tools/perf/tests/builtin-test.c |  4 ++++
 tools/perf/tests/tests.h        |  1 +
 tools/perf/tests/thread-map.c   | 43 +++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/evsel.c         | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 tools/perf/util/evsel.h         |  1 +
 tools/perf/util/thread_map.c    | 22 ++++++++++++++++++++++
 tools/perf/util/thread_map.h    |  1 +
 10 files changed, 131 insertions(+), 10 deletions(-)

[PATCH 1/5] perf mem: Fix --all-user/--all-kernel options

[Jiri Olsa]

Removing extra '--' prefix.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-p8x8lf2ekjpqc2x5ixw61505@git.kernel.org
---
 tools/perf/builtin-mem.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/builtin-mem.c b/tools/perf/builtin-mem.c
index d1ce29be560e..cd7bc4d104e2 100644
--- a/tools/perf/builtin-mem.c
+++ b/tools/perf/builtin-mem.c
@@ -70,8 +70,8 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem)
 	OPT_UINTEGER(0, "ldlat", &perf_mem_events__loads_ldlat, "mem-loads latency"),
 	OPT_INCR('v', "verbose", &verbose,
 		 "be more verbose (show counter open errors, etc)"),
-	OPT_BOOLEAN('U', "--all-user", &all_user, "collect only user level data"),
-	OPT_BOOLEAN('K', "--all-kernel", &all_kernel, "collect only kernel level data"),
+	OPT_BOOLEAN('U', "all-user", &all_user, "collect only user level data"),
+	OPT_BOOLEAN('K', "all-kernel", &all_kernel, "collect only kernel level data"),
 	OPT_END()
 	};
 
-- 
2.7.4

Re: [PATCH 1/5] perf mem: Fix --all-user/--all-kernel options

[Arnaldo Carvalho de Melo]

Em Mon, Dec 12, 2016 at 11:35:39AM +0100, Jiri Olsa escreveu:
> Removing extra '--' prefix.

Please add a Fixes: tag in such cases, as this helps backporters to
figure out what affects packages they published, in this case I'm
adding:

Fixes: ad16511b0e40 ("perf mem: Add -U/-K (--all-user/--all-kernel)
options")

And I just added this to the Todo[1] list:

* Improve git pre-commit hook script to check for OPTION() with -- in
  its name, to fix bugs like ad16511b0e40 ("perf mem: Add -U/-K
  (--all-user/--all-kernel) options") (acme) 

:-)

Thanks,

- Arnaldo

[1] https://perf.wiki.kernel.org/index.php/Todo
 
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Link: http://lkml.kernel.org/n/tip-p8x8lf2ekjpqc2x5ixw61505@git.kernel.org
> ---
>  tools/perf/builtin-mem.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/perf/builtin-mem.c b/tools/perf/builtin-mem.c
> index d1ce29be560e..cd7bc4d104e2 100644
> --- a/tools/perf/builtin-mem.c
> +++ b/tools/perf/builtin-mem.c
> @@ -70,8 +70,8 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem)
>  	OPT_UINTEGER(0, "ldlat", &perf_mem_events__loads_ldlat, "mem-loads latency"),
>  	OPT_INCR('v', "verbose", &verbose,
>  		 "be more verbose (show counter open errors, etc)"),
> -	OPT_BOOLEAN('U', "--all-user", &all_user, "collect only user level data"),
> -	OPT_BOOLEAN('K', "--all-kernel", &all_kernel, "collect only kernel level data"),
> +	OPT_BOOLEAN('U', "all-user", &all_user, "collect only user level data"),
> +	OPT_BOOLEAN('K', "all-kernel", &all_kernel, "collect only kernel level data"),
>  	OPT_END()
>  	};
>  
> -- 
> 2.7.4

Re: [PATCH 1/5] perf mem: Fix --all-user/--all-kernel options

[Jiri Olsa]

On Mon, Dec 12, 2016 at 10:43:11AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Dec 12, 2016 at 11:35:39AM +0100, Jiri Olsa escreveu:
> > Removing extra '--' prefix.
> 
> Please add a Fixes: tag in such cases, as this helps backporters to
> figure out what affects packages they published, in this case I'm
> adding:
> 
> Fixes: ad16511b0e40 ("perf mem: Add -U/-K (--all-user/--all-kernel)
> options")
> 
> And I just added this to the Todo[1] list:
> 
> * Improve git pre-commit hook script to check for OPTION() with -- in
>   its name, to fix bugs like ad16511b0e40 ("perf mem: Add -U/-K
>   (--all-user/--all-kernel) options") (acme) 
> 
> :-)

I forgot ;-) thanks

jirka

> 
> Thanks,
> 
> - Arnaldo
> 
> [1] https://perf.wiki.kernel.org/index.php/Todo
>  
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > Cc: David Ahern <dsahern@gmail.com>
> > Cc: Namhyung Kim <namhyung@kernel.org>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Link: http://lkml.kernel.org/n/tip-p8x8lf2ekjpqc2x5ixw61505@git.kernel.org
> > ---
> >  tools/perf/builtin-mem.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tools/perf/builtin-mem.c b/tools/perf/builtin-mem.c
> > index d1ce29be560e..cd7bc4d104e2 100644
> > --- a/tools/perf/builtin-mem.c
> > +++ b/tools/perf/builtin-mem.c
> > @@ -70,8 +70,8 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem)
> >  	OPT_UINTEGER(0, "ldlat", &perf_mem_events__loads_ldlat, "mem-loads latency"),
> >  	OPT_INCR('v', "verbose", &verbose,
> >  		 "be more verbose (show counter open errors, etc)"),
> > -	OPT_BOOLEAN('U', "--all-user", &all_user, "collect only user level data"),
> > -	OPT_BOOLEAN('K', "--all-kernel", &all_kernel, "collect only kernel level data"),
> > +	OPT_BOOLEAN('U', "all-user", &all_user, "collect only user level data"),
> > +	OPT_BOOLEAN('K', "all-kernel", &all_kernel, "collect only kernel level data"),
> >  	OPT_END()
> >  	};
> >  
> > -- 
> > 2.7.4

[tip:perf/urgent] perf mem: Fix --all-user/--all-kernel options

[tip-bot for Jiri Olsa]

Commit-ID:  631ac41b46d293fb3ee43a809776c1663de8d9c6
Gitweb:     http://git.kernel.org/tip/631ac41b46d293fb3ee43a809776c1663de8d9c6
Author:     Jiri Olsa <jolsa@kernel.org>
AuthorDate: Mon, 12 Dec 2016 11:35:39 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 15 Dec 2016 16:25:45 -0300

perf mem: Fix --all-user/--all-kernel options

Removing extra '--' prefix.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Fixes: ad16511b0e40 ("perf mem: Add -U/-K (--all-user/--all-kernel) options")
Link: http://lkml.kernel.org/r/1481538943-21874-2-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-mem.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/builtin-mem.c b/tools/perf/builtin-mem.c
index d1ce29b..cd7bc4d 100644
--- a/tools/perf/builtin-mem.c
+++ b/tools/perf/builtin-mem.c
@@ -70,8 +70,8 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem)
 	OPT_UINTEGER(0, "ldlat", &perf_mem_events__loads_ldlat, "mem-loads latency"),
 	OPT_INCR('v', "verbose", &verbose,
 		 "be more verbose (show counter open errors, etc)"),
-	OPT_BOOLEAN('U', "--all-user", &all_user, "collect only user level data"),
-	OPT_BOOLEAN('K', "--all-kernel", &all_kernel, "collect only kernel level data"),
+	OPT_BOOLEAN('U', "all-user", &all_user, "collect only user level data"),
+	OPT_BOOLEAN('K', "all-kernel", &all_kernel, "collect only kernel level data"),
 	OPT_END()
 	};
 

[PATCH 2/5] perf tools: Use variable instead of repeating lengthy FD macro

[Jiri Olsa]

It's more readable and will ease up following patches.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-0ta7fwl6pb4xnp8b3vddcbpt@git.kernel.org
---
 tools/perf/util/evsel.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index b2365a63db45..fd61ebd77c26 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -1474,7 +1474,7 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
 	for (cpu = 0; cpu < cpus->nr; cpu++) {
 
 		for (thread = 0; thread < nthreads; thread++) {
-			int group_fd;
+			int fd, group_fd;
 
 			if (!evsel->cgrp && !evsel->system_wide)
 				pid = thread_map__pid(threads, thread);
@@ -1484,21 +1484,22 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
 			pr_debug2("sys_perf_event_open: pid %d  cpu %d  group_fd %d  flags %#lx",
 				  pid, cpus->map[cpu], group_fd, flags);
 
-			FD(evsel, cpu, thread) = sys_perf_event_open(&evsel->attr,
-								     pid,
-								     cpus->map[cpu],
-								     group_fd, flags);
-			if (FD(evsel, cpu, thread) < 0) {
+			fd = sys_perf_event_open(&evsel->attr, pid, cpus->map[cpu],
+						 group_fd, flags);
+
+			FD(evsel, cpu, thread) = fd;
+
+			if (fd < 0) {
 				err = -errno;
 				pr_debug2("\nsys_perf_event_open failed, error %d\n",
 					  err);
 				goto try_fallback;
 			}
 
-			pr_debug2(" = %d\n", FD(evsel, cpu, thread));
+			pr_debug2(" = %d\n", fd);
 
 			if (evsel->bpf_fd >= 0) {
-				int evt_fd = FD(evsel, cpu, thread);
+				int evt_fd = fd;
 				int bpf_fd = evsel->bpf_fd;
 
 				err = ioctl(evt_fd,
-- 
2.7.4

[tip:perf/urgent] perf evsel: Use variable instead of repeating lengthy FD macro

[tip-bot for Jiri Olsa]

Commit-ID:  83c2e4f3968d6871eed295f2f5675d3d70b01afa
Gitweb:     http://git.kernel.org/tip/83c2e4f3968d6871eed295f2f5675d3d70b01afa
Author:     Jiri Olsa <jolsa@kernel.org>
AuthorDate: Mon, 12 Dec 2016 11:35:40 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 15 Dec 2016 16:25:45 -0300

perf evsel: Use variable instead of repeating lengthy FD macro

It's more readable and will ease up following patches.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1481538943-21874-3-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/evsel.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index b2365a63..fd61ebd 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -1474,7 +1474,7 @@ retry_sample_id:
 	for (cpu = 0; cpu < cpus->nr; cpu++) {
 
 		for (thread = 0; thread < nthreads; thread++) {
-			int group_fd;
+			int fd, group_fd;
 
 			if (!evsel->cgrp && !evsel->system_wide)
 				pid = thread_map__pid(threads, thread);
@@ -1484,21 +1484,22 @@ retry_open:
 			pr_debug2("sys_perf_event_open: pid %d  cpu %d  group_fd %d  flags %#lx",
 				  pid, cpus->map[cpu], group_fd, flags);
 
-			FD(evsel, cpu, thread) = sys_perf_event_open(&evsel->attr,
-								     pid,
-								     cpus->map[cpu],
-								     group_fd, flags);
-			if (FD(evsel, cpu, thread) < 0) {
+			fd = sys_perf_event_open(&evsel->attr, pid, cpus->map[cpu],
+						 group_fd, flags);
+
+			FD(evsel, cpu, thread) = fd;
+
+			if (fd < 0) {
 				err = -errno;
 				pr_debug2("\nsys_perf_event_open failed, error %d\n",
 					  err);
 				goto try_fallback;
 			}
 
-			pr_debug2(" = %d\n", FD(evsel, cpu, thread));
+			pr_debug2(" = %d\n", fd);
 
 			if (evsel->bpf_fd >= 0) {
-				int evt_fd = FD(evsel, cpu, thread);
+				int evt_fd = fd;
 				int bpf_fd = evsel->bpf_fd;
 
 				err = ioctl(evt_fd,

[PATCH 3/5] perf tools: Add thread_map__remove function

[Jiri Olsa]

Adding thread_map__remove function to remove thread
from thread map. Adding automated test also.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-qcfxoqfyfyps9hvm234mxna4@git.kernel.org
---
 tools/perf/tests/builtin-test.c |  4 ++++
 tools/perf/tests/tests.h        |  1 +
 tools/perf/tests/thread-map.c   | 43 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/thread_map.c    | 22 +++++++++++++++++++++
 tools/perf/util/thread_map.h    |  1 +
 5 files changed, 71 insertions(+)

diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 23605202d4a1..a77dcc0d24e3 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -186,6 +186,10 @@ static struct test generic_tests[] = {
 		.func = test__thread_map_synthesize,
 	},
 	{
+		.desc = "Remove thread map",
+		.func = test__thread_map_remove,
+	},
+	{
 		.desc = "Synthesize cpu map",
 		.func = test__cpu_map_synthesize,
 	},
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 0d7b251305af..a512f0c8ff5b 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -80,6 +80,7 @@ const char *test__bpf_subtest_get_desc(int subtest);
 int test__bpf_subtest_get_nr(void);
 int test_session_topology(int subtest);
 int test__thread_map_synthesize(int subtest);
+int test__thread_map_remove(int subtest);
 int test__cpu_map_synthesize(int subtest);
 int test__synthesize_stat_config(int subtest);
 int test__synthesize_stat(int subtest);
diff --git a/tools/perf/tests/thread-map.c b/tools/perf/tests/thread-map.c
index cee2a2cdc933..e227e06ed7be 100644
--- a/tools/perf/tests/thread-map.c
+++ b/tools/perf/tests/thread-map.c
@@ -93,3 +93,46 @@ int test__thread_map_synthesize(int subtest __maybe_unused)
 
 	return 0;
 }
+
+int test__thread_map_remove(int subtest __maybe_unused)
+{
+	struct thread_map *threads;
+	char *str;
+	int i;
+
+	TEST_ASSERT_VAL("failed to allocate map string",
+			asprintf(&str, "%d,%d", getpid(), getppid()) >= 0);
+
+	threads = thread_map__new_str(str, NULL, 0);
+
+	TEST_ASSERT_VAL("failed to allocate thread_map",
+			threads);
+
+	if (verbose)
+		thread_map__fprintf(threads, stderr);
+
+	TEST_ASSERT_VAL("failed to remove thread",
+			!thread_map__remove(threads, 0));
+
+	TEST_ASSERT_VAL("thread_map count != 1", threads->nr == 1);
+
+	if (verbose)
+		thread_map__fprintf(threads, stderr);
+
+	TEST_ASSERT_VAL("failed to remove thread",
+			!thread_map__remove(threads, 0));
+
+	TEST_ASSERT_VAL("thread_map count != 0", threads->nr == 0);
+
+	if (verbose)
+		thread_map__fprintf(threads, stderr);
+
+	TEST_ASSERT_VAL("failed to not remove thread",
+			thread_map__remove(threads, 0));
+
+	for (i = 0; i < threads->nr; i++)
+		free(threads->map[i].comm);
+
+	free(threads);
+	return 0;
+}
diff --git a/tools/perf/util/thread_map.c b/tools/perf/util/thread_map.c
index 40585f5b7027..f9eab200fd75 100644
--- a/tools/perf/util/thread_map.c
+++ b/tools/perf/util/thread_map.c
@@ -448,3 +448,25 @@ bool thread_map__has(struct thread_map *threads, pid_t pid)
 
 	return false;
 }
+
+int thread_map__remove(struct thread_map *threads, int idx)
+{
+	int i;
+
+	if (threads->nr < 1)
+		return -EINVAL;
+
+	if (idx >= threads->nr)
+		return -EINVAL;
+
+	/*
+	 * Free the 'idx' item and shift the rest up.
+	 */
+	free(threads->map[idx].comm);
+
+	for (i = idx; i < threads->nr - 1; i++)
+		threads->map[i] = threads->map[i + 1];
+
+	threads->nr--;
+	return 0;
+}
diff --git a/tools/perf/util/thread_map.h b/tools/perf/util/thread_map.h
index bd3b971588da..ea0ef08c6303 100644
--- a/tools/perf/util/thread_map.h
+++ b/tools/perf/util/thread_map.h
@@ -58,4 +58,5 @@ static inline char *thread_map__comm(struct thread_map *map, int thread)
 
 void thread_map__read_comms(struct thread_map *threads);
 bool thread_map__has(struct thread_map *threads, pid_t pid);
+int thread_map__remove(struct thread_map *threads, int idx);
 #endif	/* __PERF_THREAD_MAP_H */
-- 
2.7.4

Re: [PATCH 3/5] perf tools: Add thread_map__remove function

[Arnaldo Carvalho de Melo]

Em Mon, Dec 12, 2016 at 11:35:41AM +0100, Jiri Olsa escreveu:
> Adding thread_map__remove function to remove thread
> from thread map. Adding automated test also.

I'm fixing this, caught on debian:experimental-x-mips64

  CC       /tmp/build/perf/tests/thread-map.o
  CC       /tmp/build/perf/tests/llvm.o
tests/thread-map.c: In function 'test__thread_map_remove':
tests/thread-map.c:134:3: error: implicit declaration of function 'free' [-Werror=implicit-function-declaration]
   free(threads->map[i].comm);
   ^~~~
tests/thread-map.c:134:3: error: incompatible implicit declaration of built-in function 'free' [-Werror]
tests/thread-map.c:134:3: note: include '<stdlib.h>' or provide a declaration of 'free'
tests/thread-map.c:136:2: error: incompatible implicit declaration of built-in function 'free' [-Werror]
  free(threads);
  ^~~~
tests/thread-map.c:136:2: note: include '<stdlib.h>' or provide a declaration of 'free'
  CC       /tmp/build/perf/tests/bpf.o

 
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Link: http://lkml.kernel.org/n/tip-qcfxoqfyfyps9hvm234mxna4@git.kernel.org
> ---
>  tools/perf/tests/builtin-test.c |  4 ++++
>  tools/perf/tests/tests.h        |  1 +
>  tools/perf/tests/thread-map.c   | 43 +++++++++++++++++++++++++++++++++++++++++
>  tools/perf/util/thread_map.c    | 22 +++++++++++++++++++++
>  tools/perf/util/thread_map.h    |  1 +
>  5 files changed, 71 insertions(+)
> 
> diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
> index 23605202d4a1..a77dcc0d24e3 100644
> --- a/tools/perf/tests/builtin-test.c
> +++ b/tools/perf/tests/builtin-test.c
> @@ -186,6 +186,10 @@ static struct test generic_tests[] = {
>  		.func = test__thread_map_synthesize,
>  	},
>  	{
> +		.desc = "Remove thread map",
> +		.func = test__thread_map_remove,
> +	},
> +	{
>  		.desc = "Synthesize cpu map",
>  		.func = test__cpu_map_synthesize,
>  	},
> diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
> index 0d7b251305af..a512f0c8ff5b 100644
> --- a/tools/perf/tests/tests.h
> +++ b/tools/perf/tests/tests.h
> @@ -80,6 +80,7 @@ const char *test__bpf_subtest_get_desc(int subtest);
>  int test__bpf_subtest_get_nr(void);
>  int test_session_topology(int subtest);
>  int test__thread_map_synthesize(int subtest);
> +int test__thread_map_remove(int subtest);
>  int test__cpu_map_synthesize(int subtest);
>  int test__synthesize_stat_config(int subtest);
>  int test__synthesize_stat(int subtest);
> diff --git a/tools/perf/tests/thread-map.c b/tools/perf/tests/thread-map.c
> index cee2a2cdc933..e227e06ed7be 100644
> --- a/tools/perf/tests/thread-map.c
> +++ b/tools/perf/tests/thread-map.c
> @@ -93,3 +93,46 @@ int test__thread_map_synthesize(int subtest __maybe_unused)
>  
>  	return 0;
>  }
> +
> +int test__thread_map_remove(int subtest __maybe_unused)
> +{
> +	struct thread_map *threads;
> +	char *str;
> +	int i;
> +
> +	TEST_ASSERT_VAL("failed to allocate map string",
> +			asprintf(&str, "%d,%d", getpid(), getppid()) >= 0);
> +
> +	threads = thread_map__new_str(str, NULL, 0);
> +
> +	TEST_ASSERT_VAL("failed to allocate thread_map",
> +			threads);
> +
> +	if (verbose)
> +		thread_map__fprintf(threads, stderr);
> +
> +	TEST_ASSERT_VAL("failed to remove thread",
> +			!thread_map__remove(threads, 0));
> +
> +	TEST_ASSERT_VAL("thread_map count != 1", threads->nr == 1);
> +
> +	if (verbose)
> +		thread_map__fprintf(threads, stderr);
> +
> +	TEST_ASSERT_VAL("failed to remove thread",
> +			!thread_map__remove(threads, 0));
> +
> +	TEST_ASSERT_VAL("thread_map count != 0", threads->nr == 0);
> +
> +	if (verbose)
> +		thread_map__fprintf(threads, stderr);
> +
> +	TEST_ASSERT_VAL("failed to not remove thread",
> +			thread_map__remove(threads, 0));
> +
> +	for (i = 0; i < threads->nr; i++)
> +		free(threads->map[i].comm);
> +
> +	free(threads);
> +	return 0;
> +}
> diff --git a/tools/perf/util/thread_map.c b/tools/perf/util/thread_map.c
> index 40585f5b7027..f9eab200fd75 100644
> --- a/tools/perf/util/thread_map.c
> +++ b/tools/perf/util/thread_map.c
> @@ -448,3 +448,25 @@ bool thread_map__has(struct thread_map *threads, pid_t pid)
>  
>  	return false;
>  }
> +
> +int thread_map__remove(struct thread_map *threads, int idx)
> +{
> +	int i;
> +
> +	if (threads->nr < 1)
> +		return -EINVAL;
> +
> +	if (idx >= threads->nr)
> +		return -EINVAL;
> +
> +	/*
> +	 * Free the 'idx' item and shift the rest up.
> +	 */
> +	free(threads->map[idx].comm);
> +
> +	for (i = idx; i < threads->nr - 1; i++)
> +		threads->map[i] = threads->map[i + 1];
> +
> +	threads->nr--;
> +	return 0;
> +}
> diff --git a/tools/perf/util/thread_map.h b/tools/perf/util/thread_map.h
> index bd3b971588da..ea0ef08c6303 100644
> --- a/tools/perf/util/thread_map.h
> +++ b/tools/perf/util/thread_map.h
> @@ -58,4 +58,5 @@ static inline char *thread_map__comm(struct thread_map *map, int thread)
>  
>  void thread_map__read_comms(struct thread_map *threads);
>  bool thread_map__has(struct thread_map *threads, pid_t pid);
> +int thread_map__remove(struct thread_map *threads, int idx);
>  #endif	/* __PERF_THREAD_MAP_H */
> -- 
> 2.7.4

Re: [PATCH 3/5] perf tools: Add thread_map__remove function

[Arnaldo Carvalho de Melo]

Em Mon, Dec 12, 2016 at 05:08:26PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Dec 12, 2016 at 11:35:41AM +0100, Jiri Olsa escreveu:
> > Adding thread_map__remove function to remove thread
> > from thread map. Adding automated test also.
> 
> I'm fixing this, caught on debian:experimental-x-mips64
> 
>   CC       /tmp/build/perf/tests/thread-map.o
>   CC       /tmp/build/perf/tests/llvm.o
> tests/thread-map.c: In function 'test__thread_map_remove':
> tests/thread-map.c:134:3: error: implicit declaration of function 'free' [-Werror=implicit-function-declaration]
>    free(threads->map[i].comm);
>    ^~~~
> tests/thread-map.c:134:3: error: incompatible implicit declaration of built-in function 'free' [-Werror]
> tests/thread-map.c:134:3: note: include '<stdlib.h>' or provide a declaration of 'free'
> tests/thread-map.c:136:2: error: incompatible implicit declaration of built-in function 'free' [-Werror]
>   free(threads);
>   ^~~~
> tests/thread-map.c:136:2: note: include '<stdlib.h>' or provide a declaration of 'free'
>   CC       /tmp/build/perf/tests/bpf.o
> 

Also it looks strange using free() directly on a thread_map object,
don't we have to use thread_map__put()?

Humm, this thread_map__new_event() doesn't follow the refcounting
lifetimes, humpf, adding stdlib.h there and then will try and mae it use
the refcount in this constructor as well...

- Arnaldo
  
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > Cc: David Ahern <dsahern@gmail.com>
> > Cc: Namhyung Kim <namhyung@kernel.org>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Link: http://lkml.kernel.org/n/tip-qcfxoqfyfyps9hvm234mxna4@git.kernel.org
> > ---
> >  tools/perf/tests/builtin-test.c |  4 ++++
> >  tools/perf/tests/tests.h        |  1 +
> >  tools/perf/tests/thread-map.c   | 43 +++++++++++++++++++++++++++++++++++++++++
> >  tools/perf/util/thread_map.c    | 22 +++++++++++++++++++++
> >  tools/perf/util/thread_map.h    |  1 +
> >  5 files changed, 71 insertions(+)
> > 
> > diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
> > index 23605202d4a1..a77dcc0d24e3 100644
> > --- a/tools/perf/tests/builtin-test.c
> > +++ b/tools/perf/tests/builtin-test.c
> > @@ -186,6 +186,10 @@ static struct test generic_tests[] = {
> >  		.func = test__thread_map_synthesize,
> >  	},
> >  	{
> > +		.desc = "Remove thread map",
> > +		.func = test__thread_map_remove,
> > +	},
> > +	{
> >  		.desc = "Synthesize cpu map",
> >  		.func = test__cpu_map_synthesize,
> >  	},
> > diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
> > index 0d7b251305af..a512f0c8ff5b 100644
> > --- a/tools/perf/tests/tests.h
> > +++ b/tools/perf/tests/tests.h
> > @@ -80,6 +80,7 @@ const char *test__bpf_subtest_get_desc(int subtest);
> >  int test__bpf_subtest_get_nr(void);
> >  int test_session_topology(int subtest);
> >  int test__thread_map_synthesize(int subtest);
> > +int test__thread_map_remove(int subtest);
> >  int test__cpu_map_synthesize(int subtest);
> >  int test__synthesize_stat_config(int subtest);
> >  int test__synthesize_stat(int subtest);
> > diff --git a/tools/perf/tests/thread-map.c b/tools/perf/tests/thread-map.c
> > index cee2a2cdc933..e227e06ed7be 100644
> > --- a/tools/perf/tests/thread-map.c
> > +++ b/tools/perf/tests/thread-map.c
> > @@ -93,3 +93,46 @@ int test__thread_map_synthesize(int subtest __maybe_unused)
> >  
> >  	return 0;
> >  }
> > +
> > +int test__thread_map_remove(int subtest __maybe_unused)
> > +{
> > +	struct thread_map *threads;
> > +	char *str;
> > +	int i;
> > +
> > +	TEST_ASSERT_VAL("failed to allocate map string",
> > +			asprintf(&str, "%d,%d", getpid(), getppid()) >= 0);
> > +
> > +	threads = thread_map__new_str(str, NULL, 0);
> > +
> > +	TEST_ASSERT_VAL("failed to allocate thread_map",
> > +			threads);
> > +
> > +	if (verbose)
> > +		thread_map__fprintf(threads, stderr);
> > +
> > +	TEST_ASSERT_VAL("failed to remove thread",
> > +			!thread_map__remove(threads, 0));
> > +
> > +	TEST_ASSERT_VAL("thread_map count != 1", threads->nr == 1);
> > +
> > +	if (verbose)
> > +		thread_map__fprintf(threads, stderr);
> > +
> > +	TEST_ASSERT_VAL("failed to remove thread",
> > +			!thread_map__remove(threads, 0));
> > +
> > +	TEST_ASSERT_VAL("thread_map count != 0", threads->nr == 0);
> > +
> > +	if (verbose)
> > +		thread_map__fprintf(threads, stderr);
> > +
> > +	TEST_ASSERT_VAL("failed to not remove thread",
> > +			thread_map__remove(threads, 0));
> > +
> > +	for (i = 0; i < threads->nr; i++)
> > +		free(threads->map[i].comm);
> > +
> > +	free(threads);
> > +	return 0;
> > +}
> > diff --git a/tools/perf/util/thread_map.c b/tools/perf/util/thread_map.c
> > index 40585f5b7027..f9eab200fd75 100644
> > --- a/tools/perf/util/thread_map.c
> > +++ b/tools/perf/util/thread_map.c
> > @@ -448,3 +448,25 @@ bool thread_map__has(struct thread_map *threads, pid_t pid)
> >  
> >  	return false;
> >  }
> > +
> > +int thread_map__remove(struct thread_map *threads, int idx)
> > +{
> > +	int i;
> > +
> > +	if (threads->nr < 1)
> > +		return -EINVAL;
> > +
> > +	if (idx >= threads->nr)
> > +		return -EINVAL;
> > +
> > +	/*
> > +	 * Free the 'idx' item and shift the rest up.
> > +	 */
> > +	free(threads->map[idx].comm);
> > +
> > +	for (i = idx; i < threads->nr - 1; i++)
> > +		threads->map[i] = threads->map[i + 1];
> > +
> > +	threads->nr--;
> > +	return 0;
> > +}
> > diff --git a/tools/perf/util/thread_map.h b/tools/perf/util/thread_map.h
> > index bd3b971588da..ea0ef08c6303 100644
> > --- a/tools/perf/util/thread_map.h
> > +++ b/tools/perf/util/thread_map.h
> > @@ -58,4 +58,5 @@ static inline char *thread_map__comm(struct thread_map *map, int thread)
> >  
> >  void thread_map__read_comms(struct thread_map *threads);
> >  bool thread_map__has(struct thread_map *threads, pid_t pid);
> > +int thread_map__remove(struct thread_map *threads, int idx);
> >  #endif	/* __PERF_THREAD_MAP_H */
> > -- 
> > 2.7.4

[tip:perf/urgent] perf thread_map: Add thread_map__remove function

[tip-bot for Jiri Olsa]

Commit-ID:  38af91f01de0e160c17ae380acb5bab5d51066f4
Gitweb:     http://git.kernel.org/tip/38af91f01de0e160c17ae380acb5bab5d51066f4
Author:     Jiri Olsa <jolsa@kernel.org>
AuthorDate: Mon, 12 Dec 2016 11:35:41 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 15 Dec 2016 16:25:45 -0300

perf thread_map: Add thread_map__remove function

Add thread_map__remove function to remove thread from thread map.

Add automated test also.

Committer notes:

Testing it:

  # perf test "Remove thread map"
  39: Remove thread map                          : Ok
  # perf test -v "Remove thread map"
  39: Remove thread map                          :
  --- start ---
  test child forked, pid 4483
  2 threads: 4482, 4483
  1 thread: 4483
  0 thread:
  test child finished with 0
  ---- end ----
  Remove thread map: Ok
  #

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1481538943-21874-4-git-send-email-jolsa@kernel.org
[ Added stdlib.h, to get the free() declaration ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/tests/builtin-test.c |  4 ++++
 tools/perf/tests/tests.h        |  1 +
 tools/perf/tests/thread-map.c   | 44 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/thread_map.c    | 22 +++++++++++++++++++++
 tools/perf/util/thread_map.h    |  1 +
 5 files changed, 72 insertions(+)

diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 2360520..a77dcc0 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -186,6 +186,10 @@ static struct test generic_tests[] = {
 		.func = test__thread_map_synthesize,
 	},
 	{
+		.desc = "Remove thread map",
+		.func = test__thread_map_remove,
+	},
+	{
 		.desc = "Synthesize cpu map",
 		.func = test__cpu_map_synthesize,
 	},
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 0d7b251..a512f0c 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -80,6 +80,7 @@ const char *test__bpf_subtest_get_desc(int subtest);
 int test__bpf_subtest_get_nr(void);
 int test_session_topology(int subtest);
 int test__thread_map_synthesize(int subtest);
+int test__thread_map_remove(int subtest);
 int test__cpu_map_synthesize(int subtest);
 int test__synthesize_stat_config(int subtest);
 int test__synthesize_stat(int subtest);
diff --git a/tools/perf/tests/thread-map.c b/tools/perf/tests/thread-map.c
index cee2a2c..a4a4b46 100644
--- a/tools/perf/tests/thread-map.c
+++ b/tools/perf/tests/thread-map.c
@@ -1,3 +1,4 @@
+#include <stdlib.h>
 #include <sys/types.h>
 #include <unistd.h>
 #include <sys/prctl.h>
@@ -93,3 +94,46 @@ int test__thread_map_synthesize(int subtest __maybe_unused)
 
 	return 0;
 }
+
+int test__thread_map_remove(int subtest __maybe_unused)
+{
+	struct thread_map *threads;
+	char *str;
+	int i;
+
+	TEST_ASSERT_VAL("failed to allocate map string",
+			asprintf(&str, "%d,%d", getpid(), getppid()) >= 0);
+
+	threads = thread_map__new_str(str, NULL, 0);
+
+	TEST_ASSERT_VAL("failed to allocate thread_map",
+			threads);
+
+	if (verbose)
+		thread_map__fprintf(threads, stderr);
+
+	TEST_ASSERT_VAL("failed to remove thread",
+			!thread_map__remove(threads, 0));
+
+	TEST_ASSERT_VAL("thread_map count != 1", threads->nr == 1);
+
+	if (verbose)
+		thread_map__fprintf(threads, stderr);
+
+	TEST_ASSERT_VAL("failed to remove thread",
+			!thread_map__remove(threads, 0));
+
+	TEST_ASSERT_VAL("thread_map count != 0", threads->nr == 0);
+
+	if (verbose)
+		thread_map__fprintf(threads, stderr);
+
+	TEST_ASSERT_VAL("failed to not remove thread",
+			thread_map__remove(threads, 0));
+
+	for (i = 0; i < threads->nr; i++)
+		free(threads->map[i].comm);
+
+	free(threads);
+	return 0;
+}
diff --git a/tools/perf/util/thread_map.c b/tools/perf/util/thread_map.c
index 40585f5..f9eab20 100644
--- a/tools/perf/util/thread_map.c
+++ b/tools/perf/util/thread_map.c
@@ -448,3 +448,25 @@ bool thread_map__has(struct thread_map *threads, pid_t pid)
 
 	return false;
 }
+
+int thread_map__remove(struct thread_map *threads, int idx)
+{
+	int i;
+
+	if (threads->nr < 1)
+		return -EINVAL;
+
+	if (idx >= threads->nr)
+		return -EINVAL;
+
+	/*
+	 * Free the 'idx' item and shift the rest up.
+	 */
+	free(threads->map[idx].comm);
+
+	for (i = idx; i < threads->nr - 1; i++)
+		threads->map[i] = threads->map[i + 1];
+
+	threads->nr--;
+	return 0;
+}
diff --git a/tools/perf/util/thread_map.h b/tools/perf/util/thread_map.h
index bd3b971..ea0ef08 100644
--- a/tools/perf/util/thread_map.h
+++ b/tools/perf/util/thread_map.h
@@ -58,4 +58,5 @@ static inline char *thread_map__comm(struct thread_map *map, int thread)
 
 void thread_map__read_comms(struct thread_map *threads);
 bool thread_map__has(struct thread_map *threads, pid_t pid);
+int thread_map__remove(struct thread_map *threads, int idx);
 #endif	/* __PERF_THREAD_MAP_H */

[PATCH 4/5] perf tools: Allow to ignore missing pid

[Jiri Olsa]

Adding perf_evsel::ignore_missing_cpu_thread bool.

When set true, it allows perf to ignore error of missing
pid of perf event syscall.

We remove missing thread id from the thread_map, so the
rest of the processing like ioctl and mmap won't get
disturbed with -1 fd.

The reason for supporting this is to ease up monitoring
group of pids, that 'disappear' before perf opens their
event. This currently leads perf to report error and exit
and makes perf record's -u option unusable under certain
setup.

With this change we will allow this race and ignore such
failure with following warning:

  WARNING: Ignored open failure for pid 8605

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-atmzsjyhrt1j6xjwwmb84upg@git.kernel.org
---
 tools/perf/perf.h       |  1 +
 tools/perf/util/evsel.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/evsel.h |  1 +
 3 files changed, 46 insertions(+)

diff --git a/tools/perf/perf.h b/tools/perf/perf.h
index 9a0236a4cf95..1c27d947c2fe 100644
--- a/tools/perf/perf.h
+++ b/tools/perf/perf.h
@@ -55,6 +55,7 @@ struct record_opts {
 	bool	     all_user;
 	bool	     tail_synthesize;
 	bool	     overwrite;
+	bool	     ignore_missing_thread;
 	unsigned int freq;
 	unsigned int mmap_pages;
 	unsigned int auxtrace_mmap_pages;
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index fd61ebd77c26..ac87f1637ae9 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -990,6 +990,8 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
 	 * it overloads any global configuration.
 	 */
 	apply_config_terms(evsel, opts);
+
+	evsel->ignore_missing_thread = opts->ignore_missing_thread;
 }
 
 static int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
@@ -1419,6 +1421,33 @@ static int __open_attr__fprintf(FILE *fp, const char *name, const char *val,
 	return fprintf(fp, "  %-32s %s\n", name, val);
 }
 
+static bool ignore_missing_thread(struct perf_evsel *evsel,
+				  struct thread_map *threads,
+				  int thread, int err)
+{
+	if (!evsel->ignore_missing_thread)
+		return false;
+
+	/* The system wide setup does not work with threads. */
+	if (!evsel->system_wide)
+		return false;
+
+	/* The -ESRCH is perf event syscall errno for pid's not found. */
+	if (err != -ESRCH)
+		return false;
+
+	/* If there's only one thread, let it fail. */
+	if (threads->nr == 1)
+		return false;
+
+	if (thread_map__remove(threads, thread))
+		return false;
+
+	pr_warning("WARNING: Ignored open failure for pid %d\n",
+		   thread_map__pid(threads, thread));
+	return true;
+}
+
 static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
 			      struct thread_map *threads)
 {
@@ -1491,6 +1520,21 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
 
 			if (fd < 0) {
 				err = -errno;
+
+				if (ignore_missing_thread(evsel, threads, thread, err)) {
+					/*
+					 * We just removed 1 thread, so take a step
+					 * back on thread index and lower the upper
+					 * nthreads limit.
+					 */
+					nthreads--;
+					thread--;
+
+					/* ... and pretend like nothing have happened. */
+					err = 0;
+					continue;
+				}
+
 				pr_debug2("\nsys_perf_event_open failed, error %d\n",
 					  err);
 				goto try_fallback;
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 6abb89cd27f9..06ef6f29efa1 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -120,6 +120,7 @@ struct perf_evsel {
 	bool			tracking;
 	bool			per_pkg;
 	bool			precise_max;
+	bool			ignore_missing_thread;
 	/* parse modifier helper */
 	int			exclude_GH;
 	int			nr_members;
-- 
2.7.4

Re: [PATCH 4/5] perf tools: Allow to ignore missing pid

[Arnaldo Carvalho de Melo]

Em Mon, Dec 12, 2016 at 11:35:42AM +0100, Jiri Olsa escreveu:
> Adding perf_evsel::ignore_missing_cpu_thread bool.
> 
> When set true, it allows perf to ignore error of missing
> pid of perf event syscall.
> 
> We remove missing thread id from the thread_map, so the
> rest of the processing like ioctl and mmap won't get
> disturbed with -1 fd

What was the test performed? I tried it here the following while doing
a 'make -j4 allmodconfig' in the background, i.e. tons of threads coming
and going, and I sometimes get:

[root@jouet ~]# perf record -u acme 
Error:
The sys_perf_event_open() syscall returned with 3 (No such process) for
event (cycles:ppp).
/bin/dmesg may provide additional information.
No CONFIG_PERF_EVENTS=y kernel support configured?
[root@jouet ~]#
 
> The reason for supporting this is to ease up monitoring
> group of pids, that 'disappear' before perf opens their
> event. This currently leads perf to report error and exit
> and makes perf record's -u option unusable under certain
> setup.
> 
> With this change we will allow this race and ignore such
> failure with following warning:
> 
>   WARNING: Ignored open failure for pid 8605
> 
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Link: http://lkml.kernel.org/n/tip-atmzsjyhrt1j6xjwwmb84upg@git.kernel.org
> ---
>  tools/perf/perf.h       |  1 +
>  tools/perf/util/evsel.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
>  tools/perf/util/evsel.h |  1 +
>  3 files changed, 46 insertions(+)
> 
> diff --git a/tools/perf/perf.h b/tools/perf/perf.h
> index 9a0236a4cf95..1c27d947c2fe 100644
> --- a/tools/perf/perf.h
> +++ b/tools/perf/perf.h
> @@ -55,6 +55,7 @@ struct record_opts {
>  	bool	     all_user;
>  	bool	     tail_synthesize;
>  	bool	     overwrite;
> +	bool	     ignore_missing_thread;
>  	unsigned int freq;
>  	unsigned int mmap_pages;
>  	unsigned int auxtrace_mmap_pages;
> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> index fd61ebd77c26..ac87f1637ae9 100644
> --- a/tools/perf/util/evsel.c
> +++ b/tools/perf/util/evsel.c
> @@ -990,6 +990,8 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
>  	 * it overloads any global configuration.
>  	 */
>  	apply_config_terms(evsel, opts);
> +
> +	evsel->ignore_missing_thread = opts->ignore_missing_thread;
>  }
>  
>  static int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
> @@ -1419,6 +1421,33 @@ static int __open_attr__fprintf(FILE *fp, const char *name, const char *val,
>  	return fprintf(fp, "  %-32s %s\n", name, val);
>  }
>  
> +static bool ignore_missing_thread(struct perf_evsel *evsel,
> +				  struct thread_map *threads,
> +				  int thread, int err)
> +{
> +	if (!evsel->ignore_missing_thread)
> +		return false;
> +
> +	/* The system wide setup does not work with threads. */
> +	if (!evsel->system_wide)
> +		return false;
> +
> +	/* The -ESRCH is perf event syscall errno for pid's not found. */
> +	if (err != -ESRCH)
> +		return false;
> +
> +	/* If there's only one thread, let it fail. */
> +	if (threads->nr == 1)
> +		return false;
> +
> +	if (thread_map__remove(threads, thread))
> +		return false;
> +
> +	pr_warning("WARNING: Ignored open failure for pid %d\n",
> +		   thread_map__pid(threads, thread));
> +	return true;
> +}
> +
>  static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
>  			      struct thread_map *threads)
>  {
> @@ -1491,6 +1520,21 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
>  
>  			if (fd < 0) {
>  				err = -errno;
> +
> +				if (ignore_missing_thread(evsel, threads, thread, err)) {
> +					/*
> +					 * We just removed 1 thread, so take a step
> +					 * back on thread index and lower the upper
> +					 * nthreads limit.
> +					 */
> +					nthreads--;
> +					thread--;
> +
> +					/* ... and pretend like nothing have happened. */
> +					err = 0;
> +					continue;
> +				}
> +
>  				pr_debug2("\nsys_perf_event_open failed, error %d\n",
>  					  err);
>  				goto try_fallback;
> diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
> index 6abb89cd27f9..06ef6f29efa1 100644
> --- a/tools/perf/util/evsel.h
> +++ b/tools/perf/util/evsel.h
> @@ -120,6 +120,7 @@ struct perf_evsel {
>  	bool			tracking;
>  	bool			per_pkg;
>  	bool			precise_max;
> +	bool			ignore_missing_thread;
>  	/* parse modifier helper */
>  	int			exclude_GH;
>  	int			nr_members;
> -- 
> 2.7.4

Re: [PATCH 4/5] perf tools: Allow to ignore missing pid

[Arnaldo Carvalho de Melo]

Em Mon, Dec 12, 2016 at 11:32:09AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Dec 12, 2016 at 11:35:42AM +0100, Jiri Olsa escreveu:
> > Adding perf_evsel::ignore_missing_cpu_thread bool.
> > 
> > When set true, it allows perf to ignore error of missing
> > pid of perf event syscall.
> > 
> > We remove missing thread id from the thread_map, so the
> > rest of the processing like ioctl and mmap won't get
> > disturbed with -1 fd
> 
> What was the test performed? I tried it here the following while doing
> a 'make -j4 allmodconfig' in the background, i.e. tons of threads coming
> and going, and I sometimes get:
> 
> [root@jouet ~]# perf record -u acme 
> Error:
> The sys_perf_event_open() syscall returned with 3 (No such process) for
> event (cycles:ppp).
> /bin/dmesg may provide additional information.
> No CONFIG_PERF_EVENTS=y kernel support configured?
> [root@jouet ~]#

So I applied the first three patches, will continue after discussing
this.

- Arnaldo
  
> > The reason for supporting this is to ease up monitoring
> > group of pids, that 'disappear' before perf opens their
> > event. This currently leads perf to report error and exit
> > and makes perf record's -u option unusable under certain
> > setup.
> > 
> > With this change we will allow this race and ignore such
> > failure with following warning:
> > 
> >   WARNING: Ignored open failure for pid 8605
> > 
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > Cc: David Ahern <dsahern@gmail.com>
> > Cc: Namhyung Kim <namhyung@kernel.org>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Link: http://lkml.kernel.org/n/tip-atmzsjyhrt1j6xjwwmb84upg@git.kernel.org
> > ---
> >  tools/perf/perf.h       |  1 +
> >  tools/perf/util/evsel.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
> >  tools/perf/util/evsel.h |  1 +
> >  3 files changed, 46 insertions(+)
> > 
> > diff --git a/tools/perf/perf.h b/tools/perf/perf.h
> > index 9a0236a4cf95..1c27d947c2fe 100644
> > --- a/tools/perf/perf.h
> > +++ b/tools/perf/perf.h
> > @@ -55,6 +55,7 @@ struct record_opts {
> >  	bool	     all_user;
> >  	bool	     tail_synthesize;
> >  	bool	     overwrite;
> > +	bool	     ignore_missing_thread;
> >  	unsigned int freq;
> >  	unsigned int mmap_pages;
> >  	unsigned int auxtrace_mmap_pages;
> > diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> > index fd61ebd77c26..ac87f1637ae9 100644
> > --- a/tools/perf/util/evsel.c
> > +++ b/tools/perf/util/evsel.c
> > @@ -990,6 +990,8 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
> >  	 * it overloads any global configuration.
> >  	 */
> >  	apply_config_terms(evsel, opts);
> > +
> > +	evsel->ignore_missing_thread = opts->ignore_missing_thread;
> >  }
> >  
> >  static int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
> > @@ -1419,6 +1421,33 @@ static int __open_attr__fprintf(FILE *fp, const char *name, const char *val,
> >  	return fprintf(fp, "  %-32s %s\n", name, val);
> >  }
> >  
> > +static bool ignore_missing_thread(struct perf_evsel *evsel,
> > +				  struct thread_map *threads,
> > +				  int thread, int err)
> > +{
> > +	if (!evsel->ignore_missing_thread)
> > +		return false;
> > +
> > +	/* The system wide setup does not work with threads. */
> > +	if (!evsel->system_wide)
> > +		return false;
> > +
> > +	/* The -ESRCH is perf event syscall errno for pid's not found. */
> > +	if (err != -ESRCH)
> > +		return false;
> > +
> > +	/* If there's only one thread, let it fail. */
> > +	if (threads->nr == 1)
> > +		return false;
> > +
> > +	if (thread_map__remove(threads, thread))
> > +		return false;
> > +
> > +	pr_warning("WARNING: Ignored open failure for pid %d\n",
> > +		   thread_map__pid(threads, thread));
> > +	return true;
> > +}
> > +
> >  static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
> >  			      struct thread_map *threads)
> >  {
> > @@ -1491,6 +1520,21 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
> >  
> >  			if (fd < 0) {
> >  				err = -errno;
> > +
> > +				if (ignore_missing_thread(evsel, threads, thread, err)) {
> > +					/*
> > +					 * We just removed 1 thread, so take a step
> > +					 * back on thread index and lower the upper
> > +					 * nthreads limit.
> > +					 */
> > +					nthreads--;
> > +					thread--;
> > +
> > +					/* ... and pretend like nothing have happened. */
> > +					err = 0;
> > +					continue;
> > +				}
> > +
> >  				pr_debug2("\nsys_perf_event_open failed, error %d\n",
> >  					  err);
> >  				goto try_fallback;
> > diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
> > index 6abb89cd27f9..06ef6f29efa1 100644
> > --- a/tools/perf/util/evsel.h
> > +++ b/tools/perf/util/evsel.h
> > @@ -120,6 +120,7 @@ struct perf_evsel {
> >  	bool			tracking;
> >  	bool			per_pkg;
> >  	bool			precise_max;
> > +	bool			ignore_missing_thread;
> >  	/* parse modifier helper */
> >  	int			exclude_GH;
> >  	int			nr_members;
> > -- 
> > 2.7.4

Re: [PATCH 4/5] perf tools: Allow to ignore missing pid

[Jiri Olsa]

On Mon, Dec 12, 2016 at 11:32:09AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Dec 12, 2016 at 11:35:42AM +0100, Jiri Olsa escreveu:
> > Adding perf_evsel::ignore_missing_cpu_thread bool.
> > 
> > When set true, it allows perf to ignore error of missing
> > pid of perf event syscall.
> > 
> > We remove missing thread id from the thread_map, so the
> > rest of the processing like ioctl and mmap won't get
> > disturbed with -1 fd
> 
> What was the test performed? I tried it here the following while doing
> a 'make -j4 allmodconfig' in the background, i.e. tons of threads coming
> and going, and I sometimes get:
> 
> [root@jouet ~]# perf record -u acme 
> Error:
> The sys_perf_event_open() syscall returned with 3 (No such process) for
> event (cycles:ppp).
> /bin/dmesg may provide additional information.
> No CONFIG_PERF_EVENTS=y kernel support configured?

I dont think it's fast enough (start/stop) to hit the race, I've got best
results with having on background this:

  while :; do /usr/bin/true; done

jirka

Re: [PATCH 4/5] perf tools: Allow to ignore missing pid

[Arnaldo Carvalho de Melo]

Em Mon, Dec 12, 2016 at 03:53:12PM +0100, Jiri Olsa escreveu:
> On Mon, Dec 12, 2016 at 11:32:09AM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Dec 12, 2016 at 11:35:42AM +0100, Jiri Olsa escreveu:
> > > Adding perf_evsel::ignore_missing_cpu_thread bool.
> > > 
> > > When set true, it allows perf to ignore error of missing
> > > pid of perf event syscall.
> > > 
> > > We remove missing thread id from the thread_map, so the
> > > rest of the processing like ioctl and mmap won't get
> > > disturbed with -1 fd
> > 
> > What was the test performed? I tried it here the following while doing
> > a 'make -j4 allmodconfig' in the background, i.e. tons of threads coming
> > and going, and I sometimes get:
> > 
> > [root@jouet ~]# perf record -u acme 
> > Error:
> > The sys_perf_event_open() syscall returned with 3 (No such process) for
> > event (cycles:ppp).
> > /bin/dmesg may provide additional information.
> > No CONFIG_PERF_EVENTS=y kernel support configured?
> 
> I dont think it's fast enough (start/stop) to hit the race, I've got best
> results with having on background this:
> 
>   while :; do /usr/bin/true; done

Ok, but the focus of my message wasn't on generating threads, was in
having the above happen with your patch applied.

- Arnaldo

Re: [PATCH 4/5] perf tools: Allow to ignore missing pid

[Jiri Olsa]

On Mon, Dec 12, 2016 at 12:21:49PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Dec 12, 2016 at 03:53:12PM +0100, Jiri Olsa escreveu:
> > On Mon, Dec 12, 2016 at 11:32:09AM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Mon, Dec 12, 2016 at 11:35:42AM +0100, Jiri Olsa escreveu:
> > > > Adding perf_evsel::ignore_missing_cpu_thread bool.
> > > > 
> > > > When set true, it allows perf to ignore error of missing
> > > > pid of perf event syscall.
> > > > 
> > > > We remove missing thread id from the thread_map, so the
> > > > rest of the processing like ioctl and mmap won't get
> > > > disturbed with -1 fd
> > > 
> > > What was the test performed? I tried it here the following while doing
> > > a 'make -j4 allmodconfig' in the background, i.e. tons of threads coming
> > > and going, and I sometimes get:
> > > 
> > > [root@jouet ~]# perf record -u acme 
> > > Error:
> > > The sys_perf_event_open() syscall returned with 3 (No such process) for
> > > event (cycles:ppp).
> > > /bin/dmesg may provide additional information.
> > > No CONFIG_PERF_EVENTS=y kernel support configured?
> > 
> > I dont think it's fast enough (start/stop) to hit the race, I've got best
> > results with having on background this:
> > 
> >   while :; do /usr/bin/true; done
> 
> Ok, but the focus of my message wasn't on generating threads, was in
> having the above happen with your patch applied.

hum, actualy the patch 5 enables that for -u option.. this patch just adds the logic

jirka

Re: [PATCH 4/5] perf tools: Allow to ignore missing pid

[Arnaldo Carvalho de Melo]

Em Mon, Dec 12, 2016 at 04:38:44PM +0100, Jiri Olsa escreveu:
> On Mon, Dec 12, 2016 at 12:21:49PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Dec 12, 2016 at 03:53:12PM +0100, Jiri Olsa escreveu:
> > > On Mon, Dec 12, 2016 at 11:32:09AM -0300, Arnaldo Carvalho de Melo wrote:
> > > > Em Mon, Dec 12, 2016 at 11:35:42AM +0100, Jiri Olsa escreveu:
> > > > > Adding perf_evsel::ignore_missing_cpu_thread bool.
> > > > > 
> > > > > When set true, it allows perf to ignore error of missing
> > > > > pid of perf event syscall.
> > > > > 
> > > > > We remove missing thread id from the thread_map, so the
> > > > > rest of the processing like ioctl and mmap won't get
> > > > > disturbed with -1 fd
> > > > 
> > > > What was the test performed? I tried it here the following while doing
> > > > a 'make -j4 allmodconfig' in the background, i.e. tons of threads coming
> > > > and going, and I sometimes get:
> > > > 
> > > > [root@jouet ~]# perf record -u acme 
> > > > Error:
> > > > The sys_perf_event_open() syscall returned with 3 (No such process) for
> > > > event (cycles:ppp).
> > > > /bin/dmesg may provide additional information.
> > > > No CONFIG_PERF_EVENTS=y kernel support configured?
> > > 
> > > I dont think it's fast enough (start/stop) to hit the race, I've got best
> > > results with having on background this:
> > > 
> > >   while :; do /usr/bin/true; done
> > 
> > Ok, but the focus of my message wasn't on generating threads, was in
> > having the above happen with your patch applied.
> 
> hum, actualy the patch 5 enables that for -u option.. this patch just adds the logic

Sure, I noticed that, and the above is with your patches, all of them,
that is why I asked about this behaviour, how you tested it, etc.

- Arnaldo

Re: [PATCH 4/5] perf tools: Allow to ignore missing pid

[Namhyung Kim]

Hi Jiri,

On Mon, Dec 12, 2016 at 11:35:42AM +0100, Jiri Olsa wrote:
> Adding perf_evsel::ignore_missing_cpu_thread bool.
> 
> When set true, it allows perf to ignore error of missing
> pid of perf event syscall.
> 
> We remove missing thread id from the thread_map, so the
> rest of the processing like ioctl and mmap won't get
> disturbed with -1 fd.
> 
> The reason for supporting this is to ease up monitoring
> group of pids, that 'disappear' before perf opens their
> event. This currently leads perf to report error and exit
> and makes perf record's -u option unusable under certain
> setup.
> 
> With this change we will allow this race and ignore such
> failure with following warning:
> 
>   WARNING: Ignored open failure for pid 8605
> 
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Link: http://lkml.kernel.org/n/tip-atmzsjyhrt1j6xjwwmb84upg@git.kernel.org
> ---
>  tools/perf/perf.h       |  1 +
>  tools/perf/util/evsel.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
>  tools/perf/util/evsel.h |  1 +
>  3 files changed, 46 insertions(+)
> 
> diff --git a/tools/perf/perf.h b/tools/perf/perf.h
> index 9a0236a4cf95..1c27d947c2fe 100644
> --- a/tools/perf/perf.h
> +++ b/tools/perf/perf.h
> @@ -55,6 +55,7 @@ struct record_opts {
>  	bool	     all_user;
>  	bool	     tail_synthesize;
>  	bool	     overwrite;
> +	bool	     ignore_missing_thread;
>  	unsigned int freq;
>  	unsigned int mmap_pages;
>  	unsigned int auxtrace_mmap_pages;
> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> index fd61ebd77c26..ac87f1637ae9 100644
> --- a/tools/perf/util/evsel.c
> +++ b/tools/perf/util/evsel.c
> @@ -990,6 +990,8 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
>  	 * it overloads any global configuration.
>  	 */
>  	apply_config_terms(evsel, opts);
> +
> +	evsel->ignore_missing_thread = opts->ignore_missing_thread;
>  }
>  
>  static int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
> @@ -1419,6 +1421,33 @@ static int __open_attr__fprintf(FILE *fp, const char *name, const char *val,
>  	return fprintf(fp, "  %-32s %s\n", name, val);
>  }
>  
> +static bool ignore_missing_thread(struct perf_evsel *evsel,
> +				  struct thread_map *threads,
> +				  int thread, int err)
> +{
> +	if (!evsel->ignore_missing_thread)
> +		return false;
> +
> +	/* The system wide setup does not work with threads. */
> +	if (!evsel->system_wide)
> +		return false;

I'm bit confused with this.  Shouldn't the condition be positive?

Thanks,
Namhyung


> +
> +	/* The -ESRCH is perf event syscall errno for pid's not found. */
> +	if (err != -ESRCH)
> +		return false;
> +
> +	/* If there's only one thread, let it fail. */
> +	if (threads->nr == 1)
> +		return false;
> +
> +	if (thread_map__remove(threads, thread))
> +		return false;
> +
> +	pr_warning("WARNING: Ignored open failure for pid %d\n",
> +		   thread_map__pid(threads, thread));
> +	return true;
> +}
> +
>  static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
>  			      struct thread_map *threads)
>  {
> @@ -1491,6 +1520,21 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
>  
>  			if (fd < 0) {
>  				err = -errno;
> +
> +				if (ignore_missing_thread(evsel, threads, thread, err)) {
> +					/*
> +					 * We just removed 1 thread, so take a step
> +					 * back on thread index and lower the upper
> +					 * nthreads limit.
> +					 */
> +					nthreads--;
> +					thread--;
> +
> +					/* ... and pretend like nothing have happened. */
> +					err = 0;
> +					continue;
> +				}
> +
>  				pr_debug2("\nsys_perf_event_open failed, error %d\n",
>  					  err);
>  				goto try_fallback;
> diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
> index 6abb89cd27f9..06ef6f29efa1 100644
> --- a/tools/perf/util/evsel.h
> +++ b/tools/perf/util/evsel.h
> @@ -120,6 +120,7 @@ struct perf_evsel {
>  	bool			tracking;
>  	bool			per_pkg;
>  	bool			precise_max;
> +	bool			ignore_missing_thread;
>  	/* parse modifier helper */
>  	int			exclude_GH;
>  	int			nr_members;
> -- 
> 2.7.4
> 

[PATCHv2 4/5] perf tools: Allow to ignore missing pid

[Jiri Olsa]

On Tue, Dec 13, 2016 at 01:58:25AM +0900, Namhyung Kim wrote:

SNIP

> >  }
> >  
> > +static bool ignore_missing_thread(struct perf_evsel *evsel,
> > +				  struct thread_map *threads,
> > +				  int thread, int err)
> > +{
> > +	if (!evsel->ignore_missing_thread)
> > +		return false;
> > +
> > +	/* The system wide setup does not work with threads. */
> > +	if (!evsel->system_wide)
> > +		return false;
> 
> I'm bit confused with this.  Shouldn't the condition be positive?
> 

ugh, right.. that was last minute change, I had this condition
below together with ignore_missing_thread call and moved it up :-\

thanks for spotting this, attaching v2

jirka


---
Adding perf_evsel::ignore_missing_cpu_thread bool.

When set true, it allows perf to ignore error of missing
pid of perf event syscall.

We remove missing thread id from the thread_map, so the
rest of the processing like ioctl and mmap won't get
disturbed with -1 fd.

The reason for supporting this is to ease up monitoring
group of pids, that 'disappear' before perf opens their
event. This currently leads perf to report error and exit
and makes perf record's -u option unusable under certain
setup.

With this change we will allow this race and ignore such
failure with following warning:

  WARNING: Ignored open failure for pid 8605

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-atmzsjyhrt1j6xjwwmb84upg@git.kernel.org
---
 tools/perf/perf.h       |  1 +
 tools/perf/util/evsel.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/evsel.h |  1 +
 3 files changed, 46 insertions(+)

diff --git a/tools/perf/perf.h b/tools/perf/perf.h
index 9a0236a4cf95..1c27d947c2fe 100644
--- a/tools/perf/perf.h
+++ b/tools/perf/perf.h
@@ -55,6 +55,7 @@ struct record_opts {
 	bool	     all_user;
 	bool	     tail_synthesize;
 	bool	     overwrite;
+	bool	     ignore_missing_thread;
 	unsigned int freq;
 	unsigned int mmap_pages;
 	unsigned int auxtrace_mmap_pages;
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index fd61ebd77c26..04e536ae4d88 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -990,6 +990,8 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
 	 * it overloads any global configuration.
 	 */
 	apply_config_terms(evsel, opts);
+
+	evsel->ignore_missing_thread = opts->ignore_missing_thread;
 }
 
 static int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
@@ -1419,6 +1421,33 @@ static int __open_attr__fprintf(FILE *fp, const char *name, const char *val,
 	return fprintf(fp, "  %-32s %s\n", name, val);
 }
 
+static bool ignore_missing_thread(struct perf_evsel *evsel,
+				  struct thread_map *threads,
+				  int thread, int err)
+{
+	if (!evsel->ignore_missing_thread)
+		return false;
+
+	/* The system wide setup does not work with threads. */
+	if (evsel->system_wide)
+		return false;
+
+	/* The -ESRCH is perf event syscall errno for pid's not found. */
+	if (err != -ESRCH)
+		return false;
+
+	/* If there's only one thread, let it fail. */
+	if (threads->nr == 1)
+		return false;
+
+	if (thread_map__remove(threads, thread))
+		return false;
+
+	pr_warning("WARNING: Ignored open failure for pid %d\n",
+		   thread_map__pid(threads, thread));
+	return true;
+}
+
 static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
 			      struct thread_map *threads)
 {
@@ -1491,6 +1520,21 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
 
 			if (fd < 0) {
 				err = -errno;
+
+				if (ignore_missing_thread(evsel, threads, thread, err)) {
+					/*
+					 * We just removed 1 thread, so take a step
+					 * back on thread index and lower the upper
+					 * nthreads limit.
+					 */
+					nthreads--;
+					thread--;
+
+					/* ... and pretend like nothing have happened. */
+					err = 0;
+					continue;
+				}
+
 				pr_debug2("\nsys_perf_event_open failed, error %d\n",
 					  err);
 				goto try_fallback;
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 6abb89cd27f9..06ef6f29efa1 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -120,6 +120,7 @@ struct perf_evsel {
 	bool			tracking;
 	bool			per_pkg;
 	bool			precise_max;
+	bool			ignore_missing_thread;
 	/* parse modifier helper */
 	int			exclude_GH;
 	int			nr_members;
-- 
2.7.4

[tip:perf/urgent] perf evsel: Allow to ignore missing pid

[tip-bot for Jiri Olsa]

Commit-ID:  a359c17a7e1a9c99384499cf7b43d80867080789
Gitweb:     http://git.kernel.org/tip/a359c17a7e1a9c99384499cf7b43d80867080789
Author:     Jiri Olsa <jolsa@redhat.com>
AuthorDate: Tue, 13 Dec 2016 08:46:22 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 15 Dec 2016 16:25:46 -0300

perf evsel: Allow to ignore missing pid

Adding perf_evsel::ignore_missing_cpu_thread bool.

When set true, it allows perf to ignore error of missing pid of perf
event syscall.

We remove missing thread id from the thread_map, so the rest of the
processing like ioctl and mmap won't get disturbed with -1 fd.

The reason for supporting this is to ease up monitoring group of pids,
that 'disappear' before perf opens their event. This currently leads
perf to report error and exit and makes perf record's -u option unusable
under certain setup.

With this change we will allow this race and ignore such failure with
following warning:

  WARNING: Ignored open failure for pid 8605

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20161213074622.GA3084@krava
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/perf.h       |  1 +
 tools/perf/util/evsel.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/evsel.h |  1 +
 3 files changed, 46 insertions(+)

diff --git a/tools/perf/perf.h b/tools/perf/perf.h
index 9a0236a..1c27d94 100644
--- a/tools/perf/perf.h
+++ b/tools/perf/perf.h
@@ -55,6 +55,7 @@ struct record_opts {
 	bool	     all_user;
 	bool	     tail_synthesize;
 	bool	     overwrite;
+	bool	     ignore_missing_thread;
 	unsigned int freq;
 	unsigned int mmap_pages;
 	unsigned int auxtrace_mmap_pages;
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index fd61ebd..04e536a 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -990,6 +990,8 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts,
 	 * it overloads any global configuration.
 	 */
 	apply_config_terms(evsel, opts);
+
+	evsel->ignore_missing_thread = opts->ignore_missing_thread;
 }
 
 static int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
@@ -1419,6 +1421,33 @@ static int __open_attr__fprintf(FILE *fp, const char *name, const char *val,
 	return fprintf(fp, "  %-32s %s\n", name, val);
 }
 
+static bool ignore_missing_thread(struct perf_evsel *evsel,
+				  struct thread_map *threads,
+				  int thread, int err)
+{
+	if (!evsel->ignore_missing_thread)
+		return false;
+
+	/* The system wide setup does not work with threads. */
+	if (evsel->system_wide)
+		return false;
+
+	/* The -ESRCH is perf event syscall errno for pid's not found. */
+	if (err != -ESRCH)
+		return false;
+
+	/* If there's only one thread, let it fail. */
+	if (threads->nr == 1)
+		return false;
+
+	if (thread_map__remove(threads, thread))
+		return false;
+
+	pr_warning("WARNING: Ignored open failure for pid %d\n",
+		   thread_map__pid(threads, thread));
+	return true;
+}
+
 static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
 			      struct thread_map *threads)
 {
@@ -1491,6 +1520,21 @@ retry_open:
 
 			if (fd < 0) {
 				err = -errno;
+
+				if (ignore_missing_thread(evsel, threads, thread, err)) {
+					/*
+					 * We just removed 1 thread, so take a step
+					 * back on thread index and lower the upper
+					 * nthreads limit.
+					 */
+					nthreads--;
+					thread--;
+
+					/* ... and pretend like nothing have happened. */
+					err = 0;
+					continue;
+				}
+
 				pr_debug2("\nsys_perf_event_open failed, error %d\n",
 					  err);
 				goto try_fallback;
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 6abb89c..06ef6f2 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -120,6 +120,7 @@ struct perf_evsel {
 	bool			tracking;
 	bool			per_pkg;
 	bool			precise_max;
+	bool			ignore_missing_thread;
 	/* parse modifier helper */
 	int			exclude_GH;
 	int			nr_members;

[PATCH 5/5] perf record: Force ignore_missing_thread for uid option

[Jiri Olsa]

Enable perf_evsel::ignore_missing_thread for -u option
to ignore complete failure if any of the user's processes
die between its enumeration and time we open the event.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-vjykwntox3u5tlqctjk280ko@git.kernel.org
---
 tools/perf/builtin-record.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index fa26865364b6..74d6a035133a 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1687,6 +1687,9 @@ int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
 		goto out;
 	}
 
+	/* Enable ignoring missing threads when -u option is defined. */
+	rec->opts.ignore_missing_thread = rec->opts.target.uid != UINT_MAX;
+
 	err = -ENOMEM;
 	if (perf_evlist__create_maps(rec->evlist, &rec->opts.target) < 0)
 		usage_with_options(record_usage, record_options);
-- 
2.7.4

[tip:perf/urgent] perf record: Force ignore_missing_thread for uid option

[tip-bot for Jiri Olsa]

Commit-ID:  23dc4f1586159470aac707caae526824dd077e25
Gitweb:     http://git.kernel.org/tip/23dc4f1586159470aac707caae526824dd077e25
Author:     Jiri Olsa <jolsa@kernel.org>
AuthorDate: Mon, 12 Dec 2016 11:35:43 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 15 Dec 2016 16:25:46 -0300

perf record: Force ignore_missing_thread for uid option

Enable perf_evsel::ignore_missing_thread for -u option to ignore
complete failure if any of the user's processes die between its
enumeration and time we open the event.

Committer notes:

While doing a 'make -j4 allmodconfig' we sometimes get into the race:

Before:

  # perf record -u acme
  Error:
  The sys_perf_event_open() syscall returned with 3 (No such process) for event (cycles:ppp).
  /bin/dmesg may provide additional information.
  No CONFIG_PERF_EVENTS=y kernel support configured?
  #

After:

  [root@jouet ~]# perf record -u acme
  WARNING: Ignored open failure for pid 9888
  WARNING: Ignored open failure for pid 18059
  [root@jouet ~]#

Which is an improvement, with the races not preventing the remaining threads
for the specified user from being monitored, but the message probably needs
further clarification.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1481538943-21874-6-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-record.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index fa26865..74d6a03 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1687,6 +1687,9 @@ int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
 		goto out;
 	}
 
+	/* Enable ignoring missing threads when -u option is defined. */
+	rec->opts.ignore_missing_thread = rec->opts.target.uid != UINT_MAX;
+
 	err = -ENOMEM;
 	if (perf_evlist__create_maps(rec->evlist, &rec->opts.target) < 0)
 		usage_with_options(record_usage, record_options);