[BUG] perf: intel_pt won't display kernel function

[BUG] perf: intel_pt won't display kernel function

[Jiri Olsa]

hi,
perf script --call-trace stop working for me recently,
and displays only user space functions

I bisected that to:
  7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs

data from following comands will display user space functions only:
  # perf-with-kcore record pt -e intel_pt// -- ls
  # perf-with-kcore script pt --call-trace

when I disable the bpf synthesizing (patch below), kernel functions are back

I guess the new events mess up with intel_pt decoder somehow


thanks,
jirka


---
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 4e2d953d4bc5..3daa78bc6549 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1114,10 +1114,12 @@ static int record__synthesize(struct record *rec, bool tail)
 		return err;
 	}
 
+#if 0
 	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
 						machine, opts);
 	if (err < 0)
 		pr_warning("Couldn't synthesize bpf events.\n");
+#endif
 
 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
 					    process_synthesized_event, opts->sample_address,

Re: [BUG] perf: intel_pt won't display kernel function

[Arnaldo Carvalho de Melo]

Em Wed, Apr 03, 2019 at 04:37:38PM +0200, Jiri Olsa escreveu:
> hi,
> perf script --call-trace stop working for me recently,
> and displays only user space functions
> 
> I bisected that to:
>   7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
> 
> data from following comands will display user space functions only:
>   # perf-with-kcore record pt -e intel_pt// -- ls
>   # perf-with-kcore script pt --call-trace
> 
> when I disable the bpf synthesizing (patch below), kernel functions are back
> 
> I guess the new events mess up with intel_pt decoder somehow

And also that synthesizing should't take place when we use 

  perf record --no-bpf-event

I.e. I'm adding the patch below to my perf/urgent branch.

The --no-bpf-event option was put in place exactly for that for
explicitely disabling it if one thinks it is messing with something,
like seems to be the case here.

- Arnaldo

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 4e2d953d4bc5..17d772f192ad 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
 		return err;
 	}
 
-	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
-						machine, opts);
-	if (err < 0)
-		pr_warning("Couldn't synthesize bpf events.\n");
+	if (!opts->no_bpf_event) {
+		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
+		if (err < 0)
+			pr_warning("Couldn't synthesize bpf events.\n");
+	}
 
 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
 					    process_synthesized_event, opts->sample_address,

 
> 
> thanks,
> jirka
> 
> 
> ---
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index 4e2d953d4bc5..3daa78bc6549 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -1114,10 +1114,12 @@ static int record__synthesize(struct record *rec, bool tail)
>  		return err;
>  	}
>  
> +#if 0
>  	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
>  						machine, opts);
>  	if (err < 0)
>  		pr_warning("Couldn't synthesize bpf events.\n");
> +#endif
>  
>  	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
>  					    process_synthesized_event, opts->sample_address,

-- 

- Arnaldo

Re: [BUG] perf: intel_pt won't display kernel function

[Arnaldo Carvalho de Melo]

Em Wed, Apr 03, 2019 at 11:53:53AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Apr 03, 2019 at 04:37:38PM +0200, Jiri Olsa escreveu:
> > hi,
> > perf script --call-trace stop working for me recently,
> > and displays only user space functions
> > 
> > I bisected that to:
> >   7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
> > 
> > data from following comands will display user space functions only:
> >   # perf-with-kcore record pt -e intel_pt// -- ls
> >   # perf-with-kcore script pt --call-trace
> > 
> > when I disable the bpf synthesizing (patch below), kernel functions are back
> > 
> > I guess the new events mess up with intel_pt decoder somehow
> 
> I.e. I'm adding the patch below to my perf/urgent branch.

Song, that is what I have, can I have your Acked-by and please consider
taking a look at the bug Jiri reported,

Thanks,

- Arnaldo

commit 011318ccc2024ba03e96c32a06f74ca5d6ab5503
Author: Arnaldo Carvalho de Melo <acme@redhat.com>
Date:   Wed Apr 3 12:05:15 2019 -0300

    perf record: Do not synthesize BPF records when --no-bpf-event is used
    
    By default we synthesize and ask the kernel for BPF events, having a
    --no-bpf-event option to disable that, which can be useful, for
    instance, if there are still bugs in that code, which seems to be the
    case as reported by Jiri Olsa in:
    
      "[BUG] perf: intel_pt won't display kernel function"
      https://lore.kernel.org/lkml/20190403143738.GB32001@krava
    
    So add the check for record_opts.no_bpf_event when considering
    synthesizing BPF events for pre-existing BPF programs in 'perf record'.
    
    The reported bug needs further analysis and is a separate matter.
    
    Cc: Adrian Hunter <adrian.hunter@intel.com>
    Cc: Alexei Starovoitov <ast@kernel.org>
    Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
    Cc: Daniel Borkmann <daniel@iogearbox.net>
    Cc: Jiri Olsa <jolsa@kernel.org>
    Cc: Martin KaFai Lau <kafai@fb.com>
    Cc: Namhyung Kim <namhyung@kernel.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Song Liu <songliubraving@fb.com>
    Cc: Yonghong Song <yhs@fb.com>
    Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
    Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
    Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 4e2d953d4bc5..17d772f192ad 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
 		return err;
 	}
 
-	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
-						machine, opts);
-	if (err < 0)
-		pr_warning("Couldn't synthesize bpf events.\n");
+	if (!opts->no_bpf_event) {
+		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
+		if (err < 0)
+			pr_warning("Couldn't synthesize bpf events.\n");
+	}
 
 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
 					    process_synthesized_event, opts->sample_address,

Re: [BUG] perf: intel_pt won't display kernel function

[Song Liu]

> On Apr 3, 2019, at 8:15 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
> 
> Em Wed, Apr 03, 2019 at 11:53:53AM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Wed, Apr 03, 2019 at 04:37:38PM +0200, Jiri Olsa escreveu:
>>> hi,
>>> perf script --call-trace stop working for me recently,
>>> and displays only user space functions
>>> 
>>> I bisected that to:
>>>  7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
>>> 
>>> data from following comands will display user space functions only:
>>>  # perf-with-kcore record pt -e intel_pt// -- ls
>>>  # perf-with-kcore script pt --call-trace
>>> 
>>> when I disable the bpf synthesizing (patch below), kernel functions are back
>>> 
>>> I guess the new events mess up with intel_pt decoder somehow
>> 
>> I.e. I'm adding the patch below to my perf/urgent branch.
> 
> Song, that is what I have, can I have your Acked-by and please consider
> taking a look at the bug Jiri reported,
> 
> Thanks,
> 
> - Arnaldo

Current logic with --no-bpf-event is to generate PERF_RECORD_KSYMBOL, but not
PERF_RECORD_BPF_EVENT:

https://elixir.bootlin.com/linux/v5.1-rc3/source/tools/perf/util/bpf-event.c#L254

I will look into the intel_pt problem. 

In the meanwhile, let's fix it for now. 

Acked-by: Song Liu <songliubraving@fb.com>

Thanks, 
Song


> commit 011318ccc2024ba03e96c32a06f74ca5d6ab5503
> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
> Date:   Wed Apr 3 12:05:15 2019 -0300
> 
>    perf record: Do not synthesize BPF records when --no-bpf-event is used
> 
>    By default we synthesize and ask the kernel for BPF events, having a
>    --no-bpf-event option to disable that, which can be useful, for
>    instance, if there are still bugs in that code, which seems to be the
>    case as reported by Jiri Olsa in:
> 
>      "[BUG] perf: intel_pt won't display kernel function"
>      https://lore.kernel.org/lkml/20190403143738.GB32001@krava
> 
>    So add the check for record_opts.no_bpf_event when considering
>    synthesizing BPF events for pre-existing BPF programs in 'perf record'.
> 
>    The reported bug needs further analysis and is a separate matter.
> 
>    Cc: Adrian Hunter <adrian.hunter@intel.com>
>    Cc: Alexei Starovoitov <ast@kernel.org>
>    Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
>    Cc: Daniel Borkmann <daniel@iogearbox.net>
>    Cc: Jiri Olsa <jolsa@kernel.org>
>    Cc: Martin KaFai Lau <kafai@fb.com>
>    Cc: Namhyung Kim <namhyung@kernel.org>
>    Cc: Peter Zijlstra <peterz@infradead.org>
>    Cc: Song Liu <songliubraving@fb.com>
>    Cc: Yonghong Song <yhs@fb.com>
>    Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
>    Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
>    Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index 4e2d953d4bc5..17d772f192ad 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
> 		return err;
> 	}
> 
> -	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
> -						machine, opts);
> -	if (err < 0)
> -		pr_warning("Couldn't synthesize bpf events.\n");
> +	if (!opts->no_bpf_event) {
> +		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
> +		if (err < 0)
> +			pr_warning("Couldn't synthesize bpf events.\n");
> +	}
> 
> 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
> 					    process_synthesized_event, opts->sample_address,

Re: [BUG] perf: intel_pt won't display kernel function

[Song Liu]

> On Apr 3, 2019, at 7:37 AM, Jiri Olsa <jolsa@redhat.com> wrote:
> 
> hi,
> perf script --call-trace stop working for me recently,
> and displays only user space functions
> 
> I bisected that to:
>  7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
> 
> data from following comands will display user space functions only:
>  # perf-with-kcore record pt -e intel_pt// -- ls
>  # perf-with-kcore script pt --call-trace

Seems the following are ok (upstream as-is)?

./perf record -e intel_pt// -- ls
./perf script --call-trace

Jiri, could you please verify this? (Sorry I am new to perf-with-kcore). 

Thanks,
Song

> 
> when I disable the bpf synthesizing (patch below), kernel functions are back
> 
> I guess the new events mess up with intel_pt decoder somehow
> 
> 
> thanks,
> jirka
> 
> 
> ---
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index 4e2d953d4bc5..3daa78bc6549 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -1114,10 +1114,12 @@ static int record__synthesize(struct record *rec, bool tail)
> 		return err;
> 	}
> 
> +#if 0
> 	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
> 						machine, opts);
> 	if (err < 0)
> 		pr_warning("Couldn't synthesize bpf events.\n");
> +#endif
> 
> 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
> 					    process_synthesized_event, opts->sample_address,

Re: [BUG] perf: intel_pt won't display kernel function

[Jiri Olsa]

On Wed, Apr 03, 2019 at 05:05:02PM +0000, Song Liu wrote:
> 
> 
> > On Apr 3, 2019, at 7:37 AM, Jiri Olsa <jolsa@redhat.com> wrote:
> > 
> > hi,
> > perf script --call-trace stop working for me recently,
> > and displays only user space functions
> > 
> > I bisected that to:
> >  7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
> > 
> > data from following comands will display user space functions only:
> >  # perf-with-kcore record pt -e intel_pt// -- ls
> >  # perf-with-kcore script pt --call-trace
> 
> Seems the following are ok (upstream as-is)?
> 
> ./perf record -e intel_pt// -- ls
> ./perf script --call-trace

yes, I think it's the --kallsyms otion which is added by
perf-with-kcore that's broken

jirka

> 
> Jiri, could you please verify this? (Sorry I am new to perf-with-kcore). 
> 
> Thanks,
> Song
> 
> > 
> > when I disable the bpf synthesizing (patch below), kernel functions are back
> > 
> > I guess the new events mess up with intel_pt decoder somehow
> > 
> > 
> > thanks,
> > jirka
> > 
> > 
> > ---
> > diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> > index 4e2d953d4bc5..3daa78bc6549 100644
> > --- a/tools/perf/builtin-record.c
> > +++ b/tools/perf/builtin-record.c
> > @@ -1114,10 +1114,12 @@ static int record__synthesize(struct record *rec, bool tail)
> > 		return err;
> > 	}
> > 
> > +#if 0
> > 	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
> > 						machine, opts);
> > 	if (err < 0)
> > 		pr_warning("Couldn't synthesize bpf events.\n");
> > +#endif
> > 
> > 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
> > 					    process_synthesized_event, opts->sample_address,
> 

Re: [BUG] perf: intel_pt won't display kernel function

[Arnaldo Carvalho de Melo]

Em Wed, Apr 03, 2019 at 04:27:56PM +0000, Song Liu escreveu:
> 
> 
> > On Apr 3, 2019, at 8:15 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
> > 
> > Em Wed, Apr 03, 2019 at 11:53:53AM -0300, Arnaldo Carvalho de Melo escreveu:
> >> Em Wed, Apr 03, 2019 at 04:37:38PM +0200, Jiri Olsa escreveu:
> >>> hi,
> >>> perf script --call-trace stop working for me recently,
> >>> and displays only user space functions
> >>> 
> >>> I bisected that to:
> >>>  7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
> >>> 
> >>> data from following comands will display user space functions only:
> >>>  # perf-with-kcore record pt -e intel_pt// -- ls
> >>>  # perf-with-kcore script pt --call-trace
> >>> 
> >>> when I disable the bpf synthesizing (patch below), kernel functions are back
> >>> 
> >>> I guess the new events mess up with intel_pt decoder somehow
> >> 
> >> I.e. I'm adding the patch below to my perf/urgent branch.
> > 
> > Song, that is what I have, can I have your Acked-by and please consider
> > taking a look at the bug Jiri reported,
> > 
> > Thanks,
> > 
> > - Arnaldo
> 
> Current logic with --no-bpf-event is to generate PERF_RECORD_KSYMBOL, but not
> PERF_RECORD_BPF_EVENT:

I see... The opts is done later, after querying the kernel for existing
BPF programs so that at least the KSYMBOL ones can be generated. So I'll
keep it as is, no need for this patch.
 
> https://elixir.bootlin.com/linux/v5.1-rc3/source/tools/perf/util/bpf-event.c#L254
> 
> I will look into the intel_pt problem. 

Ok.
 
> In the meanwhile, let's fix it for now. 

I'll wait a bit.
 
> Acked-by: Song Liu <songliubraving@fb.com>
> 
> Thanks, 
> Song
> 
> 
> > commit 011318ccc2024ba03e96c32a06f74ca5d6ab5503
> > Author: Arnaldo Carvalho de Melo <acme@redhat.com>
> > Date:   Wed Apr 3 12:05:15 2019 -0300
> > 
> >    perf record: Do not synthesize BPF records when --no-bpf-event is used
> > 
> >    By default we synthesize and ask the kernel for BPF events, having a
> >    --no-bpf-event option to disable that, which can be useful, for
> >    instance, if there are still bugs in that code, which seems to be the
> >    case as reported by Jiri Olsa in:
> > 
> >      "[BUG] perf: intel_pt won't display kernel function"
> >      https://lore.kernel.org/lkml/20190403143738.GB32001@krava
> > 
> >    So add the check for record_opts.no_bpf_event when considering
> >    synthesizing BPF events for pre-existing BPF programs in 'perf record'.
> > 
> >    The reported bug needs further analysis and is a separate matter.
> > 
> >    Cc: Adrian Hunter <adrian.hunter@intel.com>
> >    Cc: Alexei Starovoitov <ast@kernel.org>
> >    Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> >    Cc: Daniel Borkmann <daniel@iogearbox.net>
> >    Cc: Jiri Olsa <jolsa@kernel.org>
> >    Cc: Martin KaFai Lau <kafai@fb.com>
> >    Cc: Namhyung Kim <namhyung@kernel.org>
> >    Cc: Peter Zijlstra <peterz@infradead.org>
> >    Cc: Song Liu <songliubraving@fb.com>
> >    Cc: Yonghong Song <yhs@fb.com>
> >    Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
> >    Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
> >    Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > 
> > diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> > index 4e2d953d4bc5..17d772f192ad 100644
> > --- a/tools/perf/builtin-record.c
> > +++ b/tools/perf/builtin-record.c
> > @@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
> > 		return err;
> > 	}
> > 
> > -	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
> > -						machine, opts);
> > -	if (err < 0)
> > -		pr_warning("Couldn't synthesize bpf events.\n");
> > +	if (!opts->no_bpf_event) {
> > +		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
> > +		if (err < 0)
> > +			pr_warning("Couldn't synthesize bpf events.\n");
> > +	}
> > 
> > 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
> > 					    process_synthesized_event, opts->sample_address,

-- 

- Arnaldo

Re: [BUG] perf: intel_pt won't display kernel function

[Song Liu]

> On Apr 3, 2019, at 11:50 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
> 
> Em Wed, Apr 03, 2019 at 04:27:56PM +0000, Song Liu escreveu:
>> 
>> 
>>> On Apr 3, 2019, at 8:15 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
>>> 
>>> Em Wed, Apr 03, 2019 at 11:53:53AM -0300, Arnaldo Carvalho de Melo escreveu:
>>>> Em Wed, Apr 03, 2019 at 04:37:38PM +0200, Jiri Olsa escreveu:
>>>>> hi,
>>>>> perf script --call-trace stop working for me recently,
>>>>> and displays only user space functions
>>>>> 
>>>>> I bisected that to:
>>>>> 7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
>>>>> 
>>>>> data from following comands will display user space functions only:
>>>>> # perf-with-kcore record pt -e intel_pt// -- ls
>>>>> # perf-with-kcore script pt --call-trace
>>>>> 
>>>>> when I disable the bpf synthesizing (patch below), kernel functions are back
>>>>> 
>>>>> I guess the new events mess up with intel_pt decoder somehow
>>>> 
>>>> I.e. I'm adding the patch below to my perf/urgent branch.
>>> 
>>> Song, that is what I have, can I have your Acked-by and please consider
>>> taking a look at the bug Jiri reported,
>>> 
>>> Thanks,
>>> 
>>> - Arnaldo
>> 
>> Current logic with --no-bpf-event is to generate PERF_RECORD_KSYMBOL, but not
>> PERF_RECORD_BPF_EVENT:
> 
> I see... The opts is done later, after querying the kernel for existing
> BPF programs so that at least the KSYMBOL ones can be generated. So I'll
> keep it as is, no need for this patch.
> 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__elixir.bootlin.com_linux_v5.1-2Drc3_source_tools_perf_util_bpf-2Devent.c-23L254&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=dR8692q0_uaizy0jkrBJQM5k2hfm4CiFxYT8KaysFrg&m=8ITYgIrqFBIy1uQFQPHrsX5kBX5RlmhwcknKYJa2eGs&s=MdB2zA9JstwC93qKEonsfTQeQYiw-yeOLbPxm3QMRo8&e=
>> 
>> I will look into the intel_pt problem. 
> 
> Ok.
> 
>> In the meanwhile, let's fix it for now. 
> 
> I'll wait a bit.

I guess something like this might be a better fix? (Sorry for missing 
commit message):

diff --git i/tools/perf/util/map.c w/tools/perf/util/map.c
index e32628cd20a7..741430a35dca 100644
--- i/tools/perf/util/map.c
+++ w/tools/perf/util/map.c
@@ -261,6 +261,12 @@ bool __map__is_extra_kernel_map(const struct map *map)
        return kmap && kmap->name[0];
 }

+bool __map__is_bpf_prog(const struct map *map)
+{
+       const char *name = map->dso->short_name;
+       return name && (strstr(name, "bpf_prog_") == name);
+}
+
 bool map__has_symbols(const struct map *map)
 {
        return dso__has_symbols(map->dso);
diff --git i/tools/perf/util/map.h w/tools/perf/util/map.h
index 0e20749f2c55..01079f1f4375 100644
--- i/tools/perf/util/map.h
+++ w/tools/perf/util/map.h
@@ -159,10 +159,12 @@ int map__set_kallsyms_ref_reloc_sym(struct map *map, const char *symbol_name,

 bool __map__is_kernel(const struct map *map);
 bool __map__is_extra_kernel_map(const struct map *map);
+bool __map__is_bpf_prog(const struct map *map);

 static inline bool __map__is_kmodule(const struct map *map)
 {
-       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map);
+       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map) &&
+               !__map__is_bpf_prog(map);
 }

 bool map__has_symbols(const struct map *map);

Thanks,
Song

> 
>> Acked-by: Song Liu <songliubraving@fb.com>
>> 
>> Thanks, 
>> Song
>> 
>> 
>>> commit 011318ccc2024ba03e96c32a06f74ca5d6ab5503
>>> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
>>> Date:   Wed Apr 3 12:05:15 2019 -0300
>>> 
>>>   perf record: Do not synthesize BPF records when --no-bpf-event is used
>>> 
>>>   By default we synthesize and ask the kernel for BPF events, having a
>>>   --no-bpf-event option to disable that, which can be useful, for
>>>   instance, if there are still bugs in that code, which seems to be the
>>>   case as reported by Jiri Olsa in:
>>> 
>>>     "[BUG] perf: intel_pt won't display kernel function"
>>>     https://lore.kernel.org/lkml/20190403143738.GB32001@krava
>>> 
>>>   So add the check for record_opts.no_bpf_event when considering
>>>   synthesizing BPF events for pre-existing BPF programs in 'perf record'.
>>> 
>>>   The reported bug needs further analysis and is a separate matter.
>>> 
>>>   Cc: Adrian Hunter <adrian.hunter@intel.com>
>>>   Cc: Alexei Starovoitov <ast@kernel.org>
>>>   Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
>>>   Cc: Daniel Borkmann <daniel@iogearbox.net>
>>>   Cc: Jiri Olsa <jolsa@kernel.org>
>>>   Cc: Martin KaFai Lau <kafai@fb.com>
>>>   Cc: Namhyung Kim <namhyung@kernel.org>
>>>   Cc: Peter Zijlstra <peterz@infradead.org>
>>>   Cc: Song Liu <songliubraving@fb.com>
>>>   Cc: Yonghong Song <yhs@fb.com>
>>>   Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
>>>   Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
>>>   Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
>>> 
>>> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
>>> index 4e2d953d4bc5..17d772f192ad 100644
>>> --- a/tools/perf/builtin-record.c
>>> +++ b/tools/perf/builtin-record.c
>>> @@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
>>> 		return err;
>>> 	}
>>> 
>>> -	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
>>> -						machine, opts);
>>> -	if (err < 0)
>>> -		pr_warning("Couldn't synthesize bpf events.\n");
>>> +	if (!opts->no_bpf_event) {
>>> +		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
>>> +		if (err < 0)
>>> +			pr_warning("Couldn't synthesize bpf events.\n");
>>> +	}
>>> 
>>> 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
>>> 					    process_synthesized_event, opts->sample_address,
> 
> -- 
> 
> - Arnaldo

Re: [BUG] perf: intel_pt won't display kernel function

[Song Liu]

> On Apr 3, 2019, at 11:55 AM, Song Liu <songliubraving@fb.com> wrote:
> 
> 
> 
>> On Apr 3, 2019, at 11:50 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
>> 
>> Em Wed, Apr 03, 2019 at 04:27:56PM +0000, Song Liu escreveu:
>>> 
>>> 
>>>> On Apr 3, 2019, at 8:15 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
>>>> 
>>>> Em Wed, Apr 03, 2019 at 11:53:53AM -0300, Arnaldo Carvalho de Melo escreveu:
>>>>> Em Wed, Apr 03, 2019 at 04:37:38PM +0200, Jiri Olsa escreveu:
>>>>>> hi,
>>>>>> perf script --call-trace stop working for me recently,
>>>>>> and displays only user space functions
>>>>>> 
>>>>>> I bisected that to:
>>>>>> 7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
>>>>>> 
>>>>>> data from following comands will display user space functions only:
>>>>>> # perf-with-kcore record pt -e intel_pt// -- ls
>>>>>> # perf-with-kcore script pt --call-trace
>>>>>> 
>>>>>> when I disable the bpf synthesizing (patch below), kernel functions are back
>>>>>> 
>>>>>> I guess the new events mess up with intel_pt decoder somehow
>>>>> 
>>>>> I.e. I'm adding the patch below to my perf/urgent branch.
>>>> 
>>>> Song, that is what I have, can I have your Acked-by and please consider
>>>> taking a look at the bug Jiri reported,
>>>> 
>>>> Thanks,
>>>> 
>>>> - Arnaldo
>>> 
>>> Current logic with --no-bpf-event is to generate PERF_RECORD_KSYMBOL, but not
>>> PERF_RECORD_BPF_EVENT:
>> 
>> I see... The opts is done later, after querying the kernel for existing
>> BPF programs so that at least the KSYMBOL ones can be generated. So I'll
>> keep it as is, no need for this patch.
>> 
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__elixir.bootlin.com_linux_v5.1-2Drc3_source_tools_perf_util_bpf-2Devent.c-23L254&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=dR8692q0_uaizy0jkrBJQM5k2hfm4CiFxYT8KaysFrg&m=8ITYgIrqFBIy1uQFQPHrsX5kBX5RlmhwcknKYJa2eGs&s=MdB2zA9JstwC93qKEonsfTQeQYiw-yeOLbPxm3QMRo8&e=
>>> 
>>> I will look into the intel_pt problem. 
>> 
>> Ok.
>> 
>>> In the meanwhile, let's fix it for now. 
>> 
>> I'll wait a bit.
> 
> I guess something like this might be a better fix? (Sorry for missing 
> commit message):
> 
> diff --git i/tools/perf/util/map.c w/tools/perf/util/map.c
> index e32628cd20a7..741430a35dca 100644
> --- i/tools/perf/util/map.c
> +++ w/tools/perf/util/map.c
> @@ -261,6 +261,12 @@ bool __map__is_extra_kernel_map(const struct map *map)
>        return kmap && kmap->name[0];
> }
> 
> +bool __map__is_bpf_prog(const struct map *map)
> +{
> +       const char *name = map->dso->short_name;
> +       return name && (strstr(name, "bpf_prog_") == name);
> +}
> +
> bool map__has_symbols(const struct map *map)
> {
>        return dso__has_symbols(map->dso);
> diff --git i/tools/perf/util/map.h w/tools/perf/util/map.h
> index 0e20749f2c55..01079f1f4375 100644
> --- i/tools/perf/util/map.h
> +++ w/tools/perf/util/map.h
> @@ -159,10 +159,12 @@ int map__set_kallsyms_ref_reloc_sym(struct map *map, const char *symbol_name,
> 
> bool __map__is_kernel(const struct map *map);
> bool __map__is_extra_kernel_map(const struct map *map);
> +bool __map__is_bpf_prog(const struct map *map);
> 
> static inline bool __map__is_kmodule(const struct map *map)
> {
> -       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map);
> +       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map) &&
> +               !__map__is_bpf_prog(map);
> }
> 
> bool map__has_symbols(const struct map *map);
> 
> Thanks,
> Song

Actually, it should be something like this:

diff --git i/tools/perf/util/map.c w/tools/perf/util/map.c
index e32628cd20a7..31cd23612529 100644
--- i/tools/perf/util/map.c
+++ w/tools/perf/util/map.c
@@ -261,6 +261,11 @@ bool __map__is_extra_kernel_map(const struct map *map)
        return kmap && kmap->name[0];
 }

+bool __map__is_bpf_prog(const struct map *map)
+{
+       return map->dso->binary_type == DSO_BINARY_TYPE__BPF_PROG_INFO;
+}
+
 bool map__has_symbols(const struct map *map)
 {
        return dso__has_symbols(map->dso);
diff --git i/tools/perf/util/map.h w/tools/perf/util/map.h
index 0e20749f2c55..01079f1f4375 100644
--- i/tools/perf/util/map.h
+++ w/tools/perf/util/map.h
@@ -159,10 +159,12 @@ int map__set_kallsyms_ref_reloc_sym(struct map *map, const char *symbol_name,

 bool __map__is_kernel(const struct map *map);
 bool __map__is_extra_kernel_map(const struct map *map);
+bool __map__is_bpf_prog(const struct map *map);

 static inline bool __map__is_kmodule(const struct map *map)
 {
-       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map);
+       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map) &&
+               !__map__is_bpf_prog(map);
 }

 bool map__has_symbols(const struct map *map);


> 
>> 
>>> Acked-by: Song Liu <songliubraving@fb.com>
>>> 
>>> Thanks, 
>>> Song
>>> 
>>> 
>>>> commit 011318ccc2024ba03e96c32a06f74ca5d6ab5503
>>>> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
>>>> Date:   Wed Apr 3 12:05:15 2019 -0300
>>>> 
>>>>  perf record: Do not synthesize BPF records when --no-bpf-event is used
>>>> 
>>>>  By default we synthesize and ask the kernel for BPF events, having a
>>>>  --no-bpf-event option to disable that, which can be useful, for
>>>>  instance, if there are still bugs in that code, which seems to be the
>>>>  case as reported by Jiri Olsa in:
>>>> 
>>>>    "[BUG] perf: intel_pt won't display kernel function"
>>>>    https://lore.kernel.org/lkml/20190403143738.GB32001@krava
>>>> 
>>>>  So add the check for record_opts.no_bpf_event when considering
>>>>  synthesizing BPF events for pre-existing BPF programs in 'perf record'.
>>>> 
>>>>  The reported bug needs further analysis and is a separate matter.
>>>> 
>>>>  Cc: Adrian Hunter <adrian.hunter@intel.com>
>>>>  Cc: Alexei Starovoitov <ast@kernel.org>
>>>>  Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
>>>>  Cc: Daniel Borkmann <daniel@iogearbox.net>
>>>>  Cc: Jiri Olsa <jolsa@kernel.org>
>>>>  Cc: Martin KaFai Lau <kafai@fb.com>
>>>>  Cc: Namhyung Kim <namhyung@kernel.org>
>>>>  Cc: Peter Zijlstra <peterz@infradead.org>
>>>>  Cc: Song Liu <songliubraving@fb.com>
>>>>  Cc: Yonghong Song <yhs@fb.com>
>>>>  Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
>>>>  Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
>>>>  Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
>>>> 
>>>> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
>>>> index 4e2d953d4bc5..17d772f192ad 100644
>>>> --- a/tools/perf/builtin-record.c
>>>> +++ b/tools/perf/builtin-record.c
>>>> @@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
>>>> 		return err;
>>>> 	}
>>>> 
>>>> -	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
>>>> -						machine, opts);
>>>> -	if (err < 0)
>>>> -		pr_warning("Couldn't synthesize bpf events.\n");
>>>> +	if (!opts->no_bpf_event) {
>>>> +		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
>>>> +		if (err < 0)
>>>> +			pr_warning("Couldn't synthesize bpf events.\n");
>>>> +	}
>>>> 
>>>> 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
>>>> 					    process_synthesized_event, opts->sample_address,
>> 
>> -- 
>> 
>> - Arnaldo

Re: [BUG] perf: intel_pt won't display kernel function

[Song Liu]

> On Apr 3, 2019, at 11:59 AM, Song Liu <songliubraving@fb.com> wrote:
> 
> 
> 
>> On Apr 3, 2019, at 11:55 AM, Song Liu <songliubraving@fb.com> wrote:
>> 
>> 
>> 
>>> On Apr 3, 2019, at 11:50 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
>>> 
>>> Em Wed, Apr 03, 2019 at 04:27:56PM +0000, Song Liu escreveu:
>>>> 
>>>> 
>>>>> On Apr 3, 2019, at 8:15 AM, Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
>>>>> 
>>>>> Em Wed, Apr 03, 2019 at 11:53:53AM -0300, Arnaldo Carvalho de Melo escreveu:
>>>>>> Em Wed, Apr 03, 2019 at 04:37:38PM +0200, Jiri Olsa escreveu:
>>>>>>> hi,
>>>>>>> perf script --call-trace stop working for me recently,
>>>>>>> and displays only user space functions
>>>>>>> 
>>>>>>> I bisected that to:
>>>>>>> 7b612e291a5a perf tools: Synthesize PERF_RECORD_* for loaded BPF programs
>>>>>>> 
>>>>>>> data from following comands will display user space functions only:
>>>>>>> # perf-with-kcore record pt -e intel_pt// -- ls
>>>>>>> # perf-with-kcore script pt --call-trace
>>>>>>> 
>>>>>>> when I disable the bpf synthesizing (patch below), kernel functions are back
>>>>>>> 
>>>>>>> I guess the new events mess up with intel_pt decoder somehow
>>>>>> 
>>>>>> I.e. I'm adding the patch below to my perf/urgent branch.
>>>>> 
>>>>> Song, that is what I have, can I have your Acked-by and please consider
>>>>> taking a look at the bug Jiri reported,
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> - Arnaldo
>>>> 
>>>> Current logic with --no-bpf-event is to generate PERF_RECORD_KSYMBOL, but not
>>>> PERF_RECORD_BPF_EVENT:
>>> 
>>> I see... The opts is done later, after querying the kernel for existing
>>> BPF programs so that at least the KSYMBOL ones can be generated. So I'll
>>> keep it as is, no need for this patch.
>>> 
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__elixir.bootlin.com_linux_v5.1-2Drc3_source_tools_perf_util_bpf-2Devent.c-23L254&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=dR8692q0_uaizy0jkrBJQM5k2hfm4CiFxYT8KaysFrg&m=8ITYgIrqFBIy1uQFQPHrsX5kBX5RlmhwcknKYJa2eGs&s=MdB2zA9JstwC93qKEonsfTQeQYiw-yeOLbPxm3QMRo8&e=
>>>> 
>>>> I will look into the intel_pt problem. 
>>> 
>>> Ok.
>>> 
>>>> In the meanwhile, let's fix it for now. 
>>> 
>>> I'll wait a bit.
>> 
>> I guess something like this might be a better fix? (Sorry for missing 
>> commit message):
>> 
>> diff --git i/tools/perf/util/map.c w/tools/perf/util/map.c
>> index e32628cd20a7..741430a35dca 100644
>> --- i/tools/perf/util/map.c
>> +++ w/tools/perf/util/map.c
>> @@ -261,6 +261,12 @@ bool __map__is_extra_kernel_map(const struct map *map)
>>       return kmap && kmap->name[0];
>> }
>> 
>> +bool __map__is_bpf_prog(const struct map *map)
>> +{
>> +       const char *name = map->dso->short_name;
>> +       return name && (strstr(name, "bpf_prog_") == name);
>> +}
>> +
>> bool map__has_symbols(const struct map *map)
>> {
>>       return dso__has_symbols(map->dso);
>> diff --git i/tools/perf/util/map.h w/tools/perf/util/map.h
>> index 0e20749f2c55..01079f1f4375 100644
>> --- i/tools/perf/util/map.h
>> +++ w/tools/perf/util/map.h
>> @@ -159,10 +159,12 @@ int map__set_kallsyms_ref_reloc_sym(struct map *map, const char *symbol_name,
>> 
>> bool __map__is_kernel(const struct map *map);
>> bool __map__is_extra_kernel_map(const struct map *map);
>> +bool __map__is_bpf_prog(const struct map *map);
>> 
>> static inline bool __map__is_kmodule(const struct map *map)
>> {
>> -       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map);
>> +       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map) &&
>> +               !__map__is_bpf_prog(map);
>> }
>> 
>> bool map__has_symbols(const struct map *map);
>> 
>> Thanks,
>> Song
> 
> Actually, it should be something like this:
> 
> diff --git i/tools/perf/util/map.c w/tools/perf/util/map.c
> index e32628cd20a7..31cd23612529 100644
> --- i/tools/perf/util/map.c
> +++ w/tools/perf/util/map.c
> @@ -261,6 +261,11 @@ bool __map__is_extra_kernel_map(const struct map *map)
>        return kmap && kmap->name[0];
> }
> 
> +bool __map__is_bpf_prog(const struct map *map)
> +{
> +       return map->dso->binary_type == DSO_BINARY_TYPE__BPF_PROG_INFO;
> +}
> +
> bool map__has_symbols(const struct map *map)
> {
>        return dso__has_symbols(map->dso);
> diff --git i/tools/perf/util/map.h w/tools/perf/util/map.h
> index 0e20749f2c55..01079f1f4375 100644
> --- i/tools/perf/util/map.h
> +++ w/tools/perf/util/map.h
> @@ -159,10 +159,12 @@ int map__set_kallsyms_ref_reloc_sym(struct map *map, const char *symbol_name,
> 
> bool __map__is_kernel(const struct map *map);
> bool __map__is_extra_kernel_map(const struct map *map);
> +bool __map__is_bpf_prog(const struct map *map);
> 
> static inline bool __map__is_kmodule(const struct map *map)
> {
> -       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map);
> +       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map) &&
> +               !__map__is_bpf_prog(map);
> }
> 
> bool map__has_symbols(const struct map *map);
> 
> 
>> 
>>> 
>>>> Acked-by: Song Liu <songliubraving@fb.com>
>>>> 
>>>> Thanks, 
>>>> Song
>>>> 
>>>> 
>>>>> commit 011318ccc2024ba03e96c32a06f74ca5d6ab5503
>>>>> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
>>>>> Date:   Wed Apr 3 12:05:15 2019 -0300
>>>>> 
>>>>> perf record: Do not synthesize BPF records when --no-bpf-event is used
>>>>> 
>>>>> By default we synthesize and ask the kernel for BPF events, having a
>>>>> --no-bpf-event option to disable that, which can be useful, for
>>>>> instance, if there are still bugs in that code, which seems to be the
>>>>> case as reported by Jiri Olsa in:
>>>>> 
>>>>>   "[BUG] perf: intel_pt won't display kernel function"
>>>>>   https://lore.kernel.org/lkml/20190403143738.GB32001@krava
>>>>> 
>>>>> So add the check for record_opts.no_bpf_event when considering
>>>>> synthesizing BPF events for pre-existing BPF programs in 'perf record'.
>>>>> 
>>>>> The reported bug needs further analysis and is a separate matter.
>>>>> 
>>>>> Cc: Adrian Hunter <adrian.hunter@intel.com>
>>>>> Cc: Alexei Starovoitov <ast@kernel.org>
>>>>> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
>>>>> Cc: Daniel Borkmann <daniel@iogearbox.net>
>>>>> Cc: Jiri Olsa <jolsa@kernel.org>
>>>>> Cc: Martin KaFai Lau <kafai@fb.com>
>>>>> Cc: Namhyung Kim <namhyung@kernel.org>
>>>>> Cc: Peter Zijlstra <peterz@infradead.org>
>>>>> Cc: Song Liu <songliubraving@fb.com>
>>>>> Cc: Yonghong Song <yhs@fb.com>
>>>>> Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
>>>>> Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
>>>>> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
>>>>> 
>>>>> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
>>>>> index 4e2d953d4bc5..17d772f192ad 100644
>>>>> --- a/tools/perf/builtin-record.c
>>>>> +++ b/tools/perf/builtin-record.c
>>>>> @@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
>>>>> 		return err;
>>>>> 	}
>>>>> 
>>>>> -	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
>>>>> -						machine, opts);
>>>>> -	if (err < 0)
>>>>> -		pr_warning("Couldn't synthesize bpf events.\n");
>>>>> +	if (!opts->no_bpf_event) {
>>>>> +		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
>>>>> +		if (err < 0)
>>>>> +			pr_warning("Couldn't synthesize bpf events.\n");
>>>>> +	}
>>>>> 
>>>>> 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
>>>>> 					    process_synthesized_event, opts->sample_address,
>>> 
>>> -- 
>>> 
>>> - Arnaldo

Yet another version, with commit message and a little more logic:


From 8c842b52905e747487447a014cef8c98d0db94ef Mon Sep 17 00:00:00 2001
From: Song Liu <songliubraving@fb.com>
Date: Wed, 3 Apr 2019 14:24:02 -0700
Subject: [PATCH] perf util: check maps for bpf programs

As reported by Jiri Olsa in:

  "[BUG] perf: intel_pt won't display kernel function"
  https://lore.kernel.org/lkml/20190403143738.GB32001@krava

Recent changes to support PERF_RECORD_KSYMBOL and PERF_RECORD_BPF_EVENT
broke --kallsyms option. This is because it broke test __map__is_kmodule.

This patch fixes this by adding check for bpf program, so that these maps
are not mistaken as kernel modules.

Reported-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Fixes: 76193a94522f ("perf, bpf: Introduce PERF_RECORD_KSYMBOL")
Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/util/map.c | 16 ++++++++++++++++
 tools/perf/util/map.h |  4 +++-
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index e32628cd20a7..28d484ef74ae 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -261,6 +261,22 @@ bool __map__is_extra_kernel_map(const struct map *map)
        return kmap && kmap->name[0];
 }

+bool __map__is_bpf_prog(const struct map *map)
+{
+       const char *name;
+
+       if (map->dso->binary_type == DSO_BINARY_TYPE__BPF_PROG_INFO)
+               return true;
+
+       /*
+        * If PERF_RECORD_BPF_EVENT is not included, the dso will not have
+        * type of DSO_BINARY_TYPE__BPF_PROG_INFO. In such cases, we can
+        * guess the type based on name.
+        */
+       name = map->dso->short_name;
+       return name && (strstr(name, "bpf_prog_") == name);
+}
+
 bool map__has_symbols(const struct map *map)
 {
        return dso__has_symbols(map->dso);
diff --git a/tools/perf/util/map.h b/tools/perf/util/map.h
index 0e20749f2c55..01079f1f4375 100644
--- a/tools/perf/util/map.h
+++ b/tools/perf/util/map.h
@@ -159,10 +159,12 @@ int map__set_kallsyms_ref_reloc_sym(struct map *map, const char *symbol_name,

 bool __map__is_kernel(const struct map *map);
 bool __map__is_extra_kernel_map(const struct map *map);
+bool __map__is_bpf_prog(const struct map *map);

 static inline bool __map__is_kmodule(const struct map *map)
 {
-       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map);
+       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map) &&
+               !__map__is_bpf_prog(map);
 }

 bool map__has_symbols(const struct map *map);
--
2.17.1

Re: [BUG] perf: intel_pt won't display kernel function

[Jiri Olsa]

On Wed, Apr 03, 2019 at 09:48:23PM +0000, Song Liu wrote:

SNIP

> >>>>> commit 011318ccc2024ba03e96c32a06f74ca5d6ab5503
> >>>>> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
> >>>>> Date:   Wed Apr 3 12:05:15 2019 -0300
> >>>>> 
> >>>>> perf record: Do not synthesize BPF records when --no-bpf-event is used
> >>>>> 
> >>>>> By default we synthesize and ask the kernel for BPF events, having a
> >>>>> --no-bpf-event option to disable that, which can be useful, for
> >>>>> instance, if there are still bugs in that code, which seems to be the
> >>>>> case as reported by Jiri Olsa in:
> >>>>> 
> >>>>>   "[BUG] perf: intel_pt won't display kernel function"
> >>>>>   https://lore.kernel.org/lkml/20190403143738.GB32001@krava
> >>>>> 
> >>>>> So add the check for record_opts.no_bpf_event when considering
> >>>>> synthesizing BPF events for pre-existing BPF programs in 'perf record'.
> >>>>> 
> >>>>> The reported bug needs further analysis and is a separate matter.
> >>>>> 
> >>>>> Cc: Adrian Hunter <adrian.hunter@intel.com>
> >>>>> Cc: Alexei Starovoitov <ast@kernel.org>
> >>>>> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> >>>>> Cc: Daniel Borkmann <daniel@iogearbox.net>
> >>>>> Cc: Jiri Olsa <jolsa@kernel.org>
> >>>>> Cc: Martin KaFai Lau <kafai@fb.com>
> >>>>> Cc: Namhyung Kim <namhyung@kernel.org>
> >>>>> Cc: Peter Zijlstra <peterz@infradead.org>
> >>>>> Cc: Song Liu <songliubraving@fb.com>
> >>>>> Cc: Yonghong Song <yhs@fb.com>
> >>>>> Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
> >>>>> Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
> >>>>> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> >>>>> 
> >>>>> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> >>>>> index 4e2d953d4bc5..17d772f192ad 100644
> >>>>> --- a/tools/perf/builtin-record.c
> >>>>> +++ b/tools/perf/builtin-record.c
> >>>>> @@ -1114,10 +1114,11 @@ static int record__synthesize(struct record *rec, bool tail)
> >>>>> 		return err;
> >>>>> 	}
> >>>>> 
> >>>>> -	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
> >>>>> -						machine, opts);
> >>>>> -	if (err < 0)
> >>>>> -		pr_warning("Couldn't synthesize bpf events.\n");
> >>>>> +	if (!opts->no_bpf_event) {
> >>>>> +		err = perf_event__synthesize_bpf_events(session, process_synthesized_event, machine, opts);
> >>>>> +		if (err < 0)
> >>>>> +			pr_warning("Couldn't synthesize bpf events.\n");
> >>>>> +	}
> >>>>> 
> >>>>> 	err = __machine__synthesize_threads(machine, tool, &opts->target, rec->evlist->threads,
> >>>>> 					    process_synthesized_event, opts->sample_address,
> >>> 
> >>> -- 
> >>> 
> >>> - Arnaldo
> 
> Yet another version, with commit message and a little more logic:
> 
> 
> From 8c842b52905e747487447a014cef8c98d0db94ef Mon Sep 17 00:00:00 2001
> From: Song Liu <songliubraving@fb.com>
> Date: Wed, 3 Apr 2019 14:24:02 -0700
> Subject: [PATCH] perf util: check maps for bpf programs
> 
> As reported by Jiri Olsa in:
> 
>   "[BUG] perf: intel_pt won't display kernel function"
>   https://lore.kernel.org/lkml/20190403143738.GB32001@krava
> 
> Recent changes to support PERF_RECORD_KSYMBOL and PERF_RECORD_BPF_EVENT
> broke --kallsyms option. This is because it broke test __map__is_kmodule.
> 
> This patch fixes this by adding check for bpf program, so that these maps
> are not mistaken as kernel modules.
> 
> Reported-by: Jiri Olsa <jolsa@kernel.org>
> Cc: Adrian Hunter <adrian.hunter@intel.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: Martin KaFai Lau <kafai@fb.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Song Liu <songliubraving@fb.com>
> Cc: Yonghong Song <yhs@fb.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Fixes: 76193a94522f ("perf, bpf: Introduce PERF_RECORD_KSYMBOL")
> Link: https://lore.kernel.org/lkml/20190403145353.GE32553@kernel.org
> 
> Signed-off-by: Song Liu <songliubraving@fb.com>
> ---
>  tools/perf/util/map.c | 16 ++++++++++++++++
>  tools/perf/util/map.h |  4 +++-
>  2 files changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
> index e32628cd20a7..28d484ef74ae 100644
> --- a/tools/perf/util/map.c
> +++ b/tools/perf/util/map.c
> @@ -261,6 +261,22 @@ bool __map__is_extra_kernel_map(const struct map *map)
>         return kmap && kmap->name[0];
>  }
> 
> +bool __map__is_bpf_prog(const struct map *map)
> +{
> +       const char *name;
> +
> +       if (map->dso->binary_type == DSO_BINARY_TYPE__BPF_PROG_INFO)
> +               return true;
> +
> +       /*
> +        * If PERF_RECORD_BPF_EVENT is not included, the dso will not have
> +        * type of DSO_BINARY_TYPE__BPF_PROG_INFO. In such cases, we can
> +        * guess the type based on name.
> +        */
> +       name = map->dso->short_name;
> +       return name && (strstr(name, "bpf_prog_") == name);
> +}
> +
>  bool map__has_symbols(const struct map *map)
>  {
>         return dso__has_symbols(map->dso);
> diff --git a/tools/perf/util/map.h b/tools/perf/util/map.h
> index 0e20749f2c55..01079f1f4375 100644
> --- a/tools/perf/util/map.h
> +++ b/tools/perf/util/map.h
> @@ -159,10 +159,12 @@ int map__set_kallsyms_ref_reloc_sym(struct map *map, const char *symbol_name,
> 
>  bool __map__is_kernel(const struct map *map);
>  bool __map__is_extra_kernel_map(const struct map *map);
> +bool __map__is_bpf_prog(const struct map *map);
> 
>  static inline bool __map__is_kmodule(const struct map *map)
>  {
> -       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map);
> +       return !__map__is_kernel(map) && !__map__is_extra_kernel_map(map) &&
> +               !__map__is_bpf_prog(map);
>  }
> 
>  bool map__has_symbols(const struct map *map);

it shows the kernel portion now, but it segfaults on me
in ksymbol map processing

recorded data with:

# perf-with-kcore record pt -e intel_pt// -- sleep 1

gdb session:

(gdb) r script -i pt/perf.data --kallsyms=./pt/kcore_dir/kallsyms
Starting program: /home/jolsa/bin/perf script -i pt/perf.data --kallsyms=./pt/kcore_dir/kallsyms

...

Program received signal SIGABRT, Aborted.
0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-29.fc30.x86_64 elfutils-libelf-0.176-1.fc30.x86_64 elfutils-libs-0.176-1.fc30.x86_64 libgcc-9.0.1-0.10.fc30.x86_64 libunwind-1.3.1-2.fc30.x86_64 libxcrypt-4.4.4-2.fc30.x86_64 numactl-libs-2.0.12-2.fc30.x86_64 perl-libs-5.28.1-434.fc30.x86_64 python2-libs-2.7.16-1.fc30.x86_64 slang-2.3.2-5.fc30.x86_64 xz-libs-5.2.4-5.fc30.x86_64 zlib-1.2.11-15.fc30.x86_64
(gdb) bt
#0  0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
#1  0x00007ffff75d0895 in abort () from /lib64/libc.so.6
#2  0x00007ffff75d0769 in __assert_fail_base.cold () from /lib64/libc.so.6
#3  0x00007ffff75de596 in __assert_fail () from /lib64/libc.so.6
#4  0x00000000004fc006 in refcount_sub_and_test (i=1, r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:131
#5  refcount_dec_and_test (r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:148
#6  map__put (map=0x1224df0) at util/map.c:299
#7  0x00000000004fdb95 in __maps__remove (map=0x1224df0, maps=0xb17d80) at util/map.c:953
#8  maps__remove (maps=0xb17d80, map=0x1224df0) at util/map.c:959
#9  0x00000000004f7d8a in map_groups__remove (map=<optimized out>, mg=<optimized out>) at util/map_groups.h:65
#10 machine__process_ksymbol_unregister (sample=<optimized out>, event=0x7ffff7279670, machine=<optimized out>) at util/machine.c:728
#11 machine__process_ksymbol (machine=<optimized out>, event=0x7ffff7279670, sample=<optimized out>) at util/machine.c:741
#12 0x00000000004fffbb in perf_session__deliver_event (session=0xb11390, event=0x7ffff7279670, tool=0x7fffffffc7b0, file_offset=13936) at util/session.c:1362
#13 0x00000000005039bb in do_flush (show_progress=false, oe=0xb17e80) at util/ordered-events.c:243
#14 __ordered_events__flush (oe=0xb17e80, how=OE_FLUSH__ROUND, timestamp=<optimized out>) at util/ordered-events.c:322
#15 0x00000000005005e4 in perf_session__process_user_event (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8,
    file_offset=file_offset@entry=191224) at util/session.c:1402
#16 0x000000000050091e in perf_session__process_event (file_offset=191224, event=0x7ffff72a4af8, session=0xb11390) at util/session.c:1529
#17 process_simple (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8, file_offset=file_offset@entry=191224) at util/session.c:1962
#18 0x0000000000501f44 in reader__process_events (prog=0x7fffffffc680, session=0xb11390, rd=<synthetic pointer>) at util/session.c:1931
#19 __perf_session__process_events (session=0xb11390) at util/session.c:1985
#20 perf_session__process_events (session=0xb11390) at util/session.c:2018
#21 0x000000000045bb42 in __cmd_script (script=0x7fffffffc7b0) at builtin-script.c:2429
#22 cmd_script (argc=<optimized out>, argv=<optimized out>) at builtin-script.c:3770
#23 0x00000000004a65cb in run_builtin (p=p@entry=0x994918 <commands+408>, argc=argc@entry=4, argv=argv@entry=0x7fffffffe160) at perf.c:303
#24 0x000000000042da1e in handle_internal_command (argv=0x7fffffffe160, argc=4) at perf.c:355
#25 run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:399
#26 main (argc=4, argv=0x7fffffffe160) at perf.c:521

jirka

Re: [BUG] perf: intel_pt won't display kernel function

[Jiri Olsa]

On Thu, Apr 04, 2019 at 11:14:38AM +0200, Jiri Olsa wrote:

SNIP

> Program received signal SIGABRT, Aborted.
> 0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-29.fc30.x86_64 elfutils-libelf-0.176-1.fc30.x86_64 elfutils-libs-0.176-1.fc30.x86_64 libgcc-9.0.1-0.10.fc30.x86_64 libunwind-1.3.1-2.fc30.x86_64 libxcrypt-4.4.4-2.fc30.x86_64 numactl-libs-2.0.12-2.fc30.x86_64 perl-libs-5.28.1-434.fc30.x86_64 python2-libs-2.7.16-1.fc30.x86_64 slang-2.3.2-5.fc30.x86_64 xz-libs-5.2.4-5.fc30.x86_64 zlib-1.2.11-15.fc30.x86_64
> (gdb) bt
> #0  0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
> #1  0x00007ffff75d0895 in abort () from /lib64/libc.so.6
> #2  0x00007ffff75d0769 in __assert_fail_base.cold () from /lib64/libc.so.6
> #3  0x00007ffff75de596 in __assert_fail () from /lib64/libc.so.6
> #4  0x00000000004fc006 in refcount_sub_and_test (i=1, r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:131
> #5  refcount_dec_and_test (r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:148
> #6  map__put (map=0x1224df0) at util/map.c:299
> #7  0x00000000004fdb95 in __maps__remove (map=0x1224df0, maps=0xb17d80) at util/map.c:953
> #8  maps__remove (maps=0xb17d80, map=0x1224df0) at util/map.c:959
> #9  0x00000000004f7d8a in map_groups__remove (map=<optimized out>, mg=<optimized out>) at util/map_groups.h:65
> #10 machine__process_ksymbol_unregister (sample=<optimized out>, event=0x7ffff7279670, machine=<optimized out>) at util/machine.c:728
> #11 machine__process_ksymbol (machine=<optimized out>, event=0x7ffff7279670, sample=<optimized out>) at util/machine.c:741
> #12 0x00000000004fffbb in perf_session__deliver_event (session=0xb11390, event=0x7ffff7279670, tool=0x7fffffffc7b0, file_offset=13936) at util/session.c:1362
> #13 0x00000000005039bb in do_flush (show_progress=false, oe=0xb17e80) at util/ordered-events.c:243
> #14 __ordered_events__flush (oe=0xb17e80, how=OE_FLUSH__ROUND, timestamp=<optimized out>) at util/ordered-events.c:322
> #15 0x00000000005005e4 in perf_session__process_user_event (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8,
>     file_offset=file_offset@entry=191224) at util/session.c:1402
> #16 0x000000000050091e in perf_session__process_event (file_offset=191224, event=0x7ffff72a4af8, session=0xb11390) at util/session.c:1529
> #17 process_simple (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8, file_offset=file_offset@entry=191224) at util/session.c:1962
> #18 0x0000000000501f44 in reader__process_events (prog=0x7fffffffc680, session=0xb11390, rd=<synthetic pointer>) at util/session.c:1931
> #19 __perf_session__process_events (session=0xb11390) at util/session.c:1985
> #20 perf_session__process_events (session=0xb11390) at util/session.c:2018
> #21 0x000000000045bb42 in __cmd_script (script=0x7fffffffc7b0) at builtin-script.c:2429
> #22 cmd_script (argc=<optimized out>, argv=<optimized out>) at builtin-script.c:3770
> #23 0x00000000004a65cb in run_builtin (p=p@entry=0x994918 <commands+408>, argc=argc@entry=4, argv=argv@entry=0x7fffffffe160) at perf.c:303
> #24 0x000000000042da1e in handle_internal_command (argv=0x7fffffffe160, argc=4) at perf.c:355
> #25 run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:399
> #26 main (argc=4, argv=0x7fffffffe160) at perf.c:521


we dont increase the map's reference for same name,
which we need to.. I assume we we re able to hit this
path with bpf maps/dso with same name

jirka


---
diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index 58ae5fc1f0e9..cdc49113ba38 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -929,10 +929,11 @@ static void __maps__insert_name(struct maps *maps, struct map *map)
 		else if (rc  > 0)
 			p = &(*p)->rb_right;
 		else
-			return;
+			goto out;
 	}
 	rb_link_node(&map->rb_node_name, parent, p);
 	rb_insert_color(&map->rb_node_name, &maps->names);
+out:
 	map__get(map);
 }
 

Re: [BUG] perf: intel_pt won't display kernel function

[Song Liu]

> On Apr 4, 2019, at 5:25 AM, Jiri Olsa <jolsa@redhat.com> wrote:
> 
> On Thu, Apr 04, 2019 at 11:14:38AM +0200, Jiri Olsa wrote:
> 
> SNIP
> 
>> Program received signal SIGABRT, Aborted.
>> 0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
>> Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-29.fc30.x86_64 elfutils-libelf-0.176-1.fc30.x86_64 elfutils-libs-0.176-1.fc30.x86_64 libgcc-9.0.1-0.10.fc30.x86_64 libunwind-1.3.1-2.fc30.x86_64 libxcrypt-4.4.4-2.fc30.x86_64 numactl-libs-2.0.12-2.fc30.x86_64 perl-libs-5.28.1-434.fc30.x86_64 python2-libs-2.7.16-1.fc30.x86_64 slang-2.3.2-5.fc30.x86_64 xz-libs-5.2.4-5.fc30.x86_64 zlib-1.2.11-15.fc30.x86_64
>> (gdb) bt
>> #0  0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
>> #1  0x00007ffff75d0895 in abort () from /lib64/libc.so.6
>> #2  0x00007ffff75d0769 in __assert_fail_base.cold () from /lib64/libc.so.6
>> #3  0x00007ffff75de596 in __assert_fail () from /lib64/libc.so.6
>> #4  0x00000000004fc006 in refcount_sub_and_test (i=1, r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:131
>> #5  refcount_dec_and_test (r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:148
>> #6  map__put (map=0x1224df0) at util/map.c:299
>> #7  0x00000000004fdb95 in __maps__remove (map=0x1224df0, maps=0xb17d80) at util/map.c:953
>> #8  maps__remove (maps=0xb17d80, map=0x1224df0) at util/map.c:959
>> #9  0x00000000004f7d8a in map_groups__remove (map=<optimized out>, mg=<optimized out>) at util/map_groups.h:65
>> #10 machine__process_ksymbol_unregister (sample=<optimized out>, event=0x7ffff7279670, machine=<optimized out>) at util/machine.c:728
>> #11 machine__process_ksymbol (machine=<optimized out>, event=0x7ffff7279670, sample=<optimized out>) at util/machine.c:741
>> #12 0x00000000004fffbb in perf_session__deliver_event (session=0xb11390, event=0x7ffff7279670, tool=0x7fffffffc7b0, file_offset=13936) at util/session.c:1362
>> #13 0x00000000005039bb in do_flush (show_progress=false, oe=0xb17e80) at util/ordered-events.c:243
>> #14 __ordered_events__flush (oe=0xb17e80, how=OE_FLUSH__ROUND, timestamp=<optimized out>) at util/ordered-events.c:322
>> #15 0x00000000005005e4 in perf_session__process_user_event (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8,
>>    file_offset=file_offset@entry=191224) at util/session.c:1402
>> #16 0x000000000050091e in perf_session__process_event (file_offset=191224, event=0x7ffff72a4af8, session=0xb11390) at util/session.c:1529
>> #17 process_simple (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8, file_offset=file_offset@entry=191224) at util/session.c:1962
>> #18 0x0000000000501f44 in reader__process_events (prog=0x7fffffffc680, session=0xb11390, rd=<synthetic pointer>) at util/session.c:1931
>> #19 __perf_session__process_events (session=0xb11390) at util/session.c:1985
>> #20 perf_session__process_events (session=0xb11390) at util/session.c:2018
>> #21 0x000000000045bb42 in __cmd_script (script=0x7fffffffc7b0) at builtin-script.c:2429
>> #22 cmd_script (argc=<optimized out>, argv=<optimized out>) at builtin-script.c:3770
>> #23 0x00000000004a65cb in run_builtin (p=p@entry=0x994918 <commands+408>, argc=argc@entry=4, argv=argv@entry=0x7fffffffe160) at perf.c:303
>> #24 0x000000000042da1e in handle_internal_command (argv=0x7fffffffe160, argc=4) at perf.c:355
>> #25 run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:399
>> #26 main (argc=4, argv=0x7fffffffe160) at perf.c:521
> 

For some reason, I cannot repro this issue. I tried to load two
bpf programs with same name, but that doesn't trigger it either. 


> 
> we dont increase the map's reference for same name,
> which we need to.. I assume we we re able to hit this
> path with bpf maps/dso with same name
> 
> jirka

Does the following change fix the issue on your side?

Thanks,
Song

> 
> 
> ---
> diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
> index 58ae5fc1f0e9..cdc49113ba38 100644
> --- a/tools/perf/util/map.c
> +++ b/tools/perf/util/map.c
> @@ -929,10 +929,11 @@ static void __maps__insert_name(struct maps *maps, struct map *map)
> 		else if (rc  > 0)
> 			p = &(*p)->rb_right;
> 		else
> -			return;
> +			goto out;
> 	}
> 	rb_link_node(&map->rb_node_name, parent, p);
> 	rb_insert_color(&map->rb_node_name, &maps->names);
> +out:
> 	map__get(map);
> }
> 

Re: [BUG] perf: intel_pt won't display kernel function

[Jiri Olsa]

On Thu, Apr 04, 2019 at 05:08:55PM +0000, Song Liu wrote:
> 
> 
> > On Apr 4, 2019, at 5:25 AM, Jiri Olsa <jolsa@redhat.com> wrote:
> > 
> > On Thu, Apr 04, 2019 at 11:14:38AM +0200, Jiri Olsa wrote:
> > 
> > SNIP
> > 
> >> Program received signal SIGABRT, Aborted.
> >> 0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
> >> Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-29.fc30.x86_64 elfutils-libelf-0.176-1.fc30.x86_64 elfutils-libs-0.176-1.fc30.x86_64 libgcc-9.0.1-0.10.fc30.x86_64 libunwind-1.3.1-2.fc30.x86_64 libxcrypt-4.4.4-2.fc30.x86_64 numactl-libs-2.0.12-2.fc30.x86_64 perl-libs-5.28.1-434.fc30.x86_64 python2-libs-2.7.16-1.fc30.x86_64 slang-2.3.2-5.fc30.x86_64 xz-libs-5.2.4-5.fc30.x86_64 zlib-1.2.11-15.fc30.x86_64
> >> (gdb) bt
> >> #0  0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
> >> #1  0x00007ffff75d0895 in abort () from /lib64/libc.so.6
> >> #2  0x00007ffff75d0769 in __assert_fail_base.cold () from /lib64/libc.so.6
> >> #3  0x00007ffff75de596 in __assert_fail () from /lib64/libc.so.6
> >> #4  0x00000000004fc006 in refcount_sub_and_test (i=1, r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:131
> >> #5  refcount_dec_and_test (r=0x1224e88) at /home/jolsa/linux/tools/include/linux/refcount.h:148
> >> #6  map__put (map=0x1224df0) at util/map.c:299
> >> #7  0x00000000004fdb95 in __maps__remove (map=0x1224df0, maps=0xb17d80) at util/map.c:953
> >> #8  maps__remove (maps=0xb17d80, map=0x1224df0) at util/map.c:959
> >> #9  0x00000000004f7d8a in map_groups__remove (map=<optimized out>, mg=<optimized out>) at util/map_groups.h:65
> >> #10 machine__process_ksymbol_unregister (sample=<optimized out>, event=0x7ffff7279670, machine=<optimized out>) at util/machine.c:728
> >> #11 machine__process_ksymbol (machine=<optimized out>, event=0x7ffff7279670, sample=<optimized out>) at util/machine.c:741
> >> #12 0x00000000004fffbb in perf_session__deliver_event (session=0xb11390, event=0x7ffff7279670, tool=0x7fffffffc7b0, file_offset=13936) at util/session.c:1362
> >> #13 0x00000000005039bb in do_flush (show_progress=false, oe=0xb17e80) at util/ordered-events.c:243
> >> #14 __ordered_events__flush (oe=0xb17e80, how=OE_FLUSH__ROUND, timestamp=<optimized out>) at util/ordered-events.c:322
> >> #15 0x00000000005005e4 in perf_session__process_user_event (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8,
> >>    file_offset=file_offset@entry=191224) at util/session.c:1402
> >> #16 0x000000000050091e in perf_session__process_event (file_offset=191224, event=0x7ffff72a4af8, session=0xb11390) at util/session.c:1529
> >> #17 process_simple (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8, file_offset=file_offset@entry=191224) at util/session.c:1962
> >> #18 0x0000000000501f44 in reader__process_events (prog=0x7fffffffc680, session=0xb11390, rd=<synthetic pointer>) at util/session.c:1931
> >> #19 __perf_session__process_events (session=0xb11390) at util/session.c:1985
> >> #20 perf_session__process_events (session=0xb11390) at util/session.c:2018
> >> #21 0x000000000045bb42 in __cmd_script (script=0x7fffffffc7b0) at builtin-script.c:2429
> >> #22 cmd_script (argc=<optimized out>, argv=<optimized out>) at builtin-script.c:3770
> >> #23 0x00000000004a65cb in run_builtin (p=p@entry=0x994918 <commands+408>, argc=argc@entry=4, argv=argv@entry=0x7fffffffe160) at perf.c:303
> >> #24 0x000000000042da1e in handle_internal_command (argv=0x7fffffffe160, argc=4) at perf.c:355
> >> #25 run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:399
> >> #26 main (argc=4, argv=0x7fffffffe160) at perf.c:521
> > 
> 
> For some reason, I cannot repro this issue. I tried to load two
> bpf programs with same name, but that doesn't trigger it either. 

for some reason I see multiple same name of bpf programs
on my setup.. need to check for the reason

> 
> 
> > 
> > we dont increase the map's reference for same name,
> > which we need to.. I assume we we re able to hit this
> > path with bpf maps/dso with same name
> > 
> > jirka
> 
> Does the following change fix the issue on your side?

yes, I'll post full patch later

jirka