[RFC] Vendor events file/dir names

[RFC] Vendor events file/dir names

[Arnaldo Carvalho de Melo]

Hi,

	Ingo pointed out to me that in the kernel sources we do not use
file/dir names with uppercase chars (look, for instance, at arch/), so I
mostly scripted a conversion to lowercase and what I got is at:

   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/vendor_events
   
   https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/vendor_events

	I did test it on my Lenovo t450s (Broadwell) and got the same
results as before, and the changed file and directory names don't get
presented to the user AFAIK, so its just a matter of consistency with
the existing practices in the kernel sources, no visible difference to
users.

	Please let me know if you have any technical argument against
this move,

Thanks,

- Arnaldo

Re: [RFC] Vendor events file/dir names

[Andi Kleen]

On Mon, Oct 17, 2016 at 04:04:38PM -0300, Arnaldo Carvalho de Melo wrote:
> Hi,
> 
> 	Ingo pointed out to me that in the kernel sources we do not use
> file/dir names with uppercase chars (look, for instance, at arch/), so I
> mostly scripted a conversion to lowercase and what I got is at:
> 
>    git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/vendor_events
>    
>    https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/vendor_events
> 
> 	I did test it on my Lenovo t450s (Broadwell) and got the same
> results as before, and the changed file and directory names don't get
> presented to the user AFAIK, so its just a matter of consistency with
> the existing practices in the kernel sources, no visible difference to
> users.
> 
> 	Please let me know if you have any technical argument against
> this move,

That's fine for me. 

-Andi

Re: [RFC] Vendor events file/dir names

[Ingo Molnar]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Hi,
> 
> 	Ingo pointed out to me that in the kernel sources we do not use
> file/dir names with uppercase chars (look, for instance, at arch/), so I
> mostly scripted a conversion to lowercase and what I got is at:
> 
>    git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/vendor_events
>    
>    https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/vendor_events
> 
> 	I did test it on my Lenovo t450s (Broadwell) and got the same
> results as before, and the changed file and directory names don't get
> presented to the user AFAIK, so its just a matter of consistency with
> the existing practices in the kernel sources, no visible difference to
> users.
> 
> 	Please let me know if you have any technical argument against
> this move,

Looks good to me!

Thanks,

	Ingo

[GIT PULL] Vendor events file/dir names

[Arnaldo Carvalho de Melo]

Em Tue, Oct 18, 2016 at 09:59:01AM +0200, Ingo Molnar escreveu:
> * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > 	Ingo pointed out to me that in the kernel sources we do not use
> > file/dir names with uppercase chars (look, for instance, at arch/), so I
> > mostly scripted a conversion to lowercase and what I got is at:

> >    git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/vendor_events

> > 	Please let me know if you have any technical argument against
> > this move,
> 
> Looks good to me!

So I've made this signed tag available with just what is in this branch,
that was based off tip/perf/urgent, please pull it into the most convenient
branch at this time,

Thanks,

- Arnaldo

tag perf-vendor_events-for-mingo-20161018
Tagger: Arnaldo Carvalho de Melo <acme@redhat.com>
Date:   Tue Oct 18 11:55:04 2016 -0300

perf/vendor_events event tables:

- Add JSON files with vendor event naming for Intel and Power8
  processors,
  allowing users of tools like oprofile to keep using the event names
they
  are used to, as well as people reading vendor documentation, where
such
  naming is used (Andi Kleen, Sukadev Bhattiprolu)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAABCAAGBQJYBj75AAoJENZQFvNTUqpAP6QP/R/p0UPopmJVql2Ht6cTBi04
K8q25fFyBfzILlHGzGenSL4Q2Rv8poBm5fU7bNVWMHyhRxC8iKElRXd/PD9Cq+7c
doRavPLcOTIqFgmH3xS5pAy3RjhxQU5+0/rPgrbcwA2d7jJm0ThF+sjBSlO6Ppf+
xxYE1sUgNW2KJLy92FZneD/EVILWV7FX/+DaF0dzKD+9cG5DvGjFSY8Z6gMNITs7
SUYvP/E8/8vchr/U55iGl9jcUMhv5oGq3BIGVwtKnigNuvCABsEk01vbaGd+tnU4
iaSG6nI1eyiwqPswBEXrdZLvWIZHz7XkWglNQMCYJG8unSylTCTr2p2Aedl0Wmel
HrR7GRNh0nYhgU7Bp6qnuCKx4Q6WMKTLOle0bDqb6hn47/rfFx0cDfcL8PODQZkK
c8vHFV0M3YgrrFeBLGsPs90ocekXsgX0VPql+7RTMX+X9KGTQPV/u0aEXVHMrjVe
9vVWtrl57KERedg9PnVWbcHB2FXQ6ujyaI/liZyjA/UnYQo8LnkmxrFY0d921ikL
ruCUAwBO47CJC3e77IuCMKAteR3Na5RiY6PFaVvt/On9R2VC2sC07Wufzam7nBGF
3i/agyoHLVMtWC5bLDANTVH0COchq4W1TtQ2u88z0LWidQ6V+G94Q62JnWVEzLu3
/Nw/9EBBSBliCBcJRwB0
=IZAQ
-----END PGP SIGNATURE-----

Re: [GIT PULL] Vendor events file/dir names

[Ingo Molnar]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Tue, Oct 18, 2016 at 09:59:01AM +0200, Ingo Molnar escreveu:
> > * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > 	Ingo pointed out to me that in the kernel sources we do not use
> > > file/dir names with uppercase chars (look, for instance, at arch/), so I
> > > mostly scripted a conversion to lowercase and what I got is at:
> 
> > >    git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/vendor_events
> 
> > > 	Please let me know if you have any technical argument against
> > > this move,
> > 
> > Looks good to me!
> 
> So I've made this signed tag available with just what is in this branch,
> that was based off tip/perf/urgent, please pull it into the most convenient
> branch at this time,
> 
> Thanks,
> 
> - Arnaldo
> 
> tag perf-vendor_events-for-mingo-20161018

Ok, I tried this out, and I like it mostly - the event files hierarchy looks good 
and the 'perf list' output looks good, but I found a couple of usability problems 
when trying to actually navigate and search the new hw events:

1)

How do I query individual hw event groups?

'perf list' output is really long now, and for example it gives me:

 cache:
   l1d.replacement                                   
        [L1D data line replacements]
 ...

If I knew that I'm interested in cache related events, I'd have expected to be 
able to do:

	perf list cache

or at least:

	perf list cache:

or something similar to list just - but it does not seem to do the right thing.

2)

Another problem I found is that there does not appear to be an easy way to get the 
long description of events. For example:

  triton:~/tip> perf list longest_lat_cache.miss

  List of pre-defined events (to be used in -e):

  cache:
    longest_lat_cache.miss                            
         [Core-originated cacheable demand requests missed LLC]

But the event table actually includes the following as well:

        "PublicDescription": "This event counts each cache miss condition for references to the last level cache.",

which is not printed anywhere. I tried the obvious 'perf list -v longest_lat_cache.miss'.

3)

If I come with an event from the vendor world, say "LONGEST_LAT_CACHE.MISS", and 
try to list it, 'perf list' does not recognize it:

  triton:~/tip> perf list LONGEST_LAT_CACHE.MISS

  List of pre-defined events (to be used in -e):

I believe the searching of events in perf list should be case insentitive in 
general.

Event parsing correctly recognizes it:

  triton:~/tip> perf stat -e longest_lat_cache.miss -e LONGEST_LAT_CACHE.MISS -e LoNgEsT_lAt_CaChE.MisS sleep 1

   Performance counter stats for 'sleep 1':

             2,669      longest_lat_cache.miss                                      
             2,669      LONGEST_LAT_CACHE.MISS                                      
             2,669      LoNgEsT_lAt_CaChE.MisS                                      

Thanks,

	Ingo

Re: [GIT PULL] Vendor events file/dir names

[Arnaldo Carvalho de Melo]

Em Wed, Oct 19, 2016 at 03:38:33PM +0200, Ingo Molnar escreveu:
> * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > Em Tue, Oct 18, 2016 at 09:59:01AM +0200, Ingo Molnar escreveu:
> > > * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > > 	Ingo pointed out to me that in the kernel sources we do not use
> > > > file/dir names with uppercase chars (look, for instance, at arch/), so I
> > > > mostly scripted a conversion to lowercase and what I got is at:
> > 
> > > >    git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/vendor_events
> > 
> > > > 	Please let me know if you have any technical argument against
> > > > this move,
> > > 
> > > Looks good to me!
> > 
> > So I've made this signed tag available with just what is in this branch,
> > that was based off tip/perf/urgent, please pull it into the most convenient
> > branch at this time,
> > 
> > Thanks,
> > 
> > - Arnaldo
> > 
> > tag perf-vendor_events-for-mingo-20161018
> 
> Ok, I tried this out, and I like it mostly - the event files hierarchy looks good 
> and the 'perf list' output looks good, but I found a couple of usability problems 
> when trying to actually navigate and search the new hw events:

Ok, but can't we work on these usability glitches with patches on top of
what we have now?

We already have queued up patches to list just vendor events, i.e.

  perf list vendor

That is that 'perf list json' respinned, and for usability there is also
a patch to support tab completion for such long event names.

What you suggested below can be done as patches on top of this, that
already allows using those events.

More below:
 
> 1)
> 
> How do I query individual hw event groups?
> 
> 'perf list' output is really long now, and for example it gives me:
> 
>  cache:
>    l1d.replacement                                   
>         [L1D data line replacements]
>  ...
> 
> If I knew that I'm interested in cache related events, I'd have expected to be 
> able to do:
> 
> 	perf list cache
> 
> or at least:
> 
> 	perf list cache:
> 
> or something similar to list just - but it does not seem to do the right thing.

Right, and we could even have a:

       perf list vendor cache

To get just the vendor named events that are in the "cache" group (i.e.
that tools/perf/pmu-events/arch/x86/broadwell/cache.json file)

And we have it now, to some degree, i.e. "cache" is handled as a
keyword, we get just the PERF_TYPE_HW_CACHE events:

[acme@jouet linux]$ perf list cache

List of pre-defined events (to be used in -e):

  L1-dcache-load-misses      [Hardware cache event]
  L1-dcache-loads            [Hardware cache event]
  L1-dcache-stores           [Hardware cache event]
  L1-icache-load-misses      [Hardware cache event]
  LLC-load-misses            [Hardware cache event]
  LLC-loads                  [Hardware cache event]
  LLC-store-misses           [Hardware cache event]
  LLC-stores                 [Hardware cache event]
  branch-load-misses         [Hardware cache event]
  branch-loads               [Hardware cache event]
  dTLB-load-misses           [Hardware cache event]
  dTLB-loads                 [Hardware cache event]
  dTLB-store-misses          [Hardware cache event]
  dTLB-stores                [Hardware cache event]
  iTLB-load-misses           [Hardware cache event]
  iTLB-loads                 [Hardware cache event]
  node-load-misses           [Hardware cache event]
  node-loads                 [Hardware cache event]
  node-store-misses          [Hardware cache event]
  node-stores                [Hardware cache event]

[acme@jouet linux]$

But I played dirty and used "cach" and got something which is closer to what
you suggest, which is to look for a substring in the event name (I expected it
to look for that substring in the event description (and type) as well, that
has to be audited and improved:

[acme@jouet linux]$ perf list cach

List of pre-defined events (to be used in -e):

  cache-misses                                       [Hardware event]
  cache-references                                   [Hardware event]

  L1-dcache-load-misses                              [Hardware cache event]
  L1-dcache-loads                                    [Hardware cache event]
  L1-dcache-stores                                   [Hardware cache event]
  L1-icache-load-misses                              [Hardware cache event]

  cache-misses OR cpu/cache-misses/                  [Kernel PMU event]
  cache-references OR cpu/cache-references/          [Kernel PMU event]

cache:
  lock_cycles.cache_lock_duration                   
       [Cycles when L1D is locked]
  longest_lat_cache.miss                            
       [Core-originated cacheable demand requests missed L3]
  longest_lat_cache.reference                       
       [Core-originated cacheable demand requests that refer to L3]

frontend:
  icache.hit                                        
       [Number of Instruction Cache, Streaming Buffer and Victim Cache Reads. both cacheable and noncacheable, including UC fetches]
  icache.ifdata_stall                               
       [Cycles where a code fetch is stalled due to L1 instruction-cache miss]
  icache.misses                                     
       [Number of Instruction Cache, Streaming Buffer and Victim Cache Misses. Includes Uncacheable accesses]

[acme@jouet linux]$

> 
> 2)
> 
> Another problem I found is that there does not appear to be an easy way to get the 
> long description of events. For example:
> 
>   triton:~/tip> perf list longest_lat_cache.miss
> 
>   List of pre-defined events (to be used in -e):
> 
>   cache:
>     longest_lat_cache.miss                            
>          [Core-originated cacheable demand requests missed LLC]
> 
> But the event table actually includes the following as well:
> 
>         "PublicDescription": "This event counts each cache miss condition for references to the last level cache.",
> 
> which is not printed anywhere. I tried the obvious 'perf list -v longest_lat_cache.miss'.
> 
> 3)
> 
> If I come with an event from the vendor world, say "LONGEST_LAT_CACHE.MISS", and 
> try to list it, 'perf list' does not recognize it:
> 
>   triton:~/tip> perf list LONGEST_LAT_CACHE.MISS
> 
>   List of pre-defined events (to be used in -e):
> 
> I believe the searching of events in perf list should be case insentitive in 
> general.

Right, as it is when parsing, as you noted below, this I think was trying to go
the existing perf way, i.e. all lowercase, and if you do so, it works:

[acme@jouet linux]$ perf list longest_lat_cache.miss

List of pre-defined events (to be used in -e):


cache:
  longest_lat_cache.miss                            
       [Core-originated cacheable demand requests missed L3]

[acme@jouet linux]$ 

So this is another case when we can fix it in a followup patch. I can work on
those, after I reduce the long backlog of patches waiting to go upstream, as
I think 'perf c2c' will get some love from you 8-)

- Arnaldo
 
> Event parsing correctly recognizes it:
> 
>   triton:~/tip> perf stat -e longest_lat_cache.miss -e LONGEST_LAT_CACHE.MISS -e LoNgEsT_lAt_CaChE.MisS sleep 1
> 
>    Performance counter stats for 'sleep 1':
> 
>              2,669      longest_lat_cache.miss                                      
>              2,669      LONGEST_LAT_CACHE.MISS                                      
>              2,669      LoNgEsT_lAt_CaChE.MisS                                      
> 
> Thanks,
> 
> 	Ingo

Re: [GIT PULL] Vendor events file/dir names

[Ingo Molnar]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Wed, Oct 19, 2016 at 03:38:33PM +0200, Ingo Molnar escreveu:
> > * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > Em Tue, Oct 18, 2016 at 09:59:01AM +0200, Ingo Molnar escreveu:
> > > > * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > > > 	Ingo pointed out to me that in the kernel sources we do not use
> > > > > file/dir names with uppercase chars (look, for instance, at arch/), so I
> > > > > mostly scripted a conversion to lowercase and what I got is at:
> > > 
> > > > >    git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/vendor_events
> > > 
> > > > > 	Please let me know if you have any technical argument against
> > > > > this move,
> > > > 
> > > > Looks good to me!
> > > 
> > > So I've made this signed tag available with just what is in this branch,
> > > that was based off tip/perf/urgent, please pull it into the most convenient
> > > branch at this time,
> > > 
> > > Thanks,
> > > 
> > > - Arnaldo
> > > 
> > > tag perf-vendor_events-for-mingo-20161018
> > 
> > Ok, I tried this out, and I like it mostly - the event files hierarchy looks good 
> > and the 'perf list' output looks good, but I found a couple of usability problems 
> > when trying to actually navigate and search the new hw events:
> 
> Ok, but can't we work on these usability glitches with patches on top of
> what we have now?

Yeah, sure - and it seems like we are in agreement about the usability details, so 
I've pulled your branch into tip:perf/core.

Thanks,

	Ingo

Re: [GIT PULL] Vendor events file/dir names

[Andi Kleen]

> How do I query individual hw event groups?
> 
> 'perf list' output is really long now, and for example it gives me:
> 
>  cache:
>    l1d.replacement                                   
>         [L1D data line replacements]
>  ...
> 
> If I knew that I'm interested in cache related events, I'd have expected to be 
> able to do:
> 
> 	perf list cache
> 
> or at least:
> 
> 	perf list cache:
> 
> or something similar to list just - but it does not seem to do the right thing.

perf list doesn't support topic matching, only event name
matching. I'll send a patch to add topic matching too.

>   triton:~/tip> perf list longest_lat_cache.miss
> 
>   List of pre-defined events (to be used in -e):
> 
>   cache:
>     longest_lat_cache.miss                            
>          [Core-originated cacheable demand requests missed LLC]
> 
> But the event table actually includes the following as well:
> 
>         "PublicDescription": "This event counts each cache miss condition for references to the last level cache.",
> 
> which is not printed anywhere. I tried the obvious 'perf list -v longest_lat_cache.miss'.

-v (or --long-desc) should work and it did at some point. I'll investigate.

> 
> 3)
> 
> If I come with an event from the vendor world, say "LONGEST_LAT_CACHE.MISS", and 
> try to list it, 'perf list' does not recognize it:
> 
>   triton:~/tip> perf list LONGEST_LAT_CACHE.MISS
> 
>   List of pre-defined events (to be used in -e):
> 
> I believe the searching of events in perf list should be case insentitive in 
> general.

Right. Will fix.

-Andi

Re: [GIT PULL] Vendor events file/dir names

[Andi Kleen]

On Wed, Oct 19, 2016 at 09:41:37AM -0700, Andi Kleen wrote:
> > How do I query individual hw event groups?
> > 
> > 'perf list' output is really long now, and for example it gives me:
> > 
> >  cache:
> >    l1d.replacement                                   
> >         [L1D data line replacements]
> >  ...
> > 
> > If I knew that I'm interested in cache related events, I'd have expected to be 
> > able to do:
> > 
> > 	perf list cache
> > 
> > or at least:
> > 
> > 	perf list cache:
> > 
> > or something similar to list just - but it does not seem to do the right thing.
> 
> perf list doesn't support topic matching, only event name
> matching. I'll send a patch to add topic matching too.
> 
> >   triton:~/tip> perf list longest_lat_cache.miss
> > 
> >   List of pre-defined events (to be used in -e):
> > 
> >   cache:
> >     longest_lat_cache.miss                            
> >          [Core-originated cacheable demand requests missed LLC]
> > 
> > But the event table actually includes the following as well:
> > 
> >         "PublicDescription": "This event counts each cache miss condition for references to the last level cache.",
> > 
> > which is not printed anywhere. I tried the obvious 'perf list -v longest_lat_cache.miss'.
> 
> -v (or --long-desc) should work and it did at some point. I'll investigate.

I tested this and it works here with -v


% PERF_CPUID=GenuineIntel-6-4E perf list -v longest_lat_cache.miss

List of pre-defined events (to be used in -e):


cache:
  longest_lat_cache.miss                            
       [This event counts core-originated cacheable demand requests that miss the last level cache
        (LLC). Demand requests include loads, RFOs, and hardware prefetches from L1D, and instruction
        fetches from IFU Spec update: SKL057]

% PERF_CPUID=GenuineIntel-6-4E perf list  longest_lat_cache.miss

List of pre-defined events (to be used in -e):


cache:
  longest_lat_cache.miss                            
       [Core-originated cacheable demand requests missed L3 Spec update: SKL057]




-Andi