Lttng add-context perf commands gives me error

Lttng add-context perf commands gives me error

[Hani Nemati]

[-- Attachment #1.1: Type: text/plain, Size: 962 bytes --]

Hi, 
I am trying to count last level cache misses by using lttng add-context command. I use add-context to add perf:cpu:LLC-load-misses to my channel but I see an error. There is no error for other perf events. Just two perf events including perf:cpu:LLC-load-misses and  perf:cpu:LLC-store-misses give me the following error:
Error: perf:cpu:LLC-load-misses: Add kernel context failed
Warning: Some command(s) went wrong

Here is result of "uname -a" on my computer: 
Linux nemati 3.16.0-51-generic #69~14.04.1-Ubuntu SMP Wed Oct 7 15:32:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 


Here is the command that I used: 
#lttng enable-channel -k test
#Kernel channel test enabled for session hh
#lttng add-context -k -t perf:cpu:LLC-load-misses -c test
Error: perf:cpu:LLC-load-misses: Add kernel context failed
Warning: Some command(s) went wrong


I also tried it on other machines.  <<<<<<<    /-/ /\ /\/ / /\/ E /\/\ /\ T /    >>>>>>>>

[-- Attachment #1.2: Type: text/html, Size: 2354 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Lttng add-context perf commands gives me error

[Mathieu Desnoyers]

[-- Attachment #1.1: Type: text/plain, Size: 1418 bytes --]

----- On Nov 17, 2015, at 4:46 PM, Hani Nemati <hani_nemati@yahoo.com> wrote: 

> Hi,
> I am trying to count last level cache misses by using lttng add-context command.
> I use add-context to add perf:cpu:LLC-load-misses to my channel but I see an
> error. There is no error for other perf events. Just two perf events including
> perf:cpu:LLC-load-misses and perf:cpu:LLC-store-misses give me the following
> error:
> Error: perf:cpu:LLC-load-misses: Add kernel context failed
> Warning: Some command(s) went wrong

> Here is result of "uname -a" on my computer:
> Linux nemati 3.16.0-51-generic #69~14.04.1-Ubuntu SMP Wed Oct 7 15:32:41 UTC
> 2015 x86_64 x86_64 x86_64 GNU/Linux

> Here is the command that I used:
> #lttng enable-channel -k test
> #Kernel channel test enabled for session hh
> #lttng add-context -k -t perf:cpu:LLC-load-misses -c test
> Error: perf:cpu:LLC-load-misses: Add kernel context failed
> Warning: Some command(s) went wrong

> I also tried it on other machines.

It is possible that your architecture does not expose this counter to perf. 
Did you try using this PMC with the perf tool ? 

Thanks, 

Mathieu 

> <<<<<<< /-/ /\ /\/ / /\/ E /\/\ /\ T / >>>>>>>>

> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 

[-- Attachment #1.2: Type: text/html, Size: 3887 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Lttng add-context perf commands gives me error

[Hani Nemati]

[-- Attachment #1.1: Type: text/plain, Size: 2007 bytes --]

There is no PERF symbolic name for LLC-load-misses but I can use raw identifier to find out Number of code read, data read that miss the LLC. Here is the command that I used.
 
#perf stat -e r0109 sleep 2
 Performance counter stats for 'sleep 2':

               117      r0109  

Do you have any idea why I don't have the symbolic name? Is there any way to give raw identifier instead of perf:cpu:LLC-load-misses? 







     On Wednesday, November 18, 2015 9:45 AM, Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:
   

 

----- On Nov 17, 2015, at 4:46 PM, Hani Nemati <hani_nemati@yahoo.com> wrote:

Hi, 
I am trying to count last level cache misses by using lttng add-context command. I use add-context to add perf:cpu:LLC-load-misses to my channel but I see an error. There is no error for other perf events. Just two perf events including perf:cpu:LLC-load-misses and  perf:cpu:LLC-store-misses give me the following error:
Error: perf:cpu:LLC-load-misses: Add kernel context failed
Warning: Some command(s) went wrong

Here is result of "uname -a" on my computer: 
Linux nemati 3.16.0-51-generic #69~14.04.1-Ubuntu SMP Wed Oct 7 15:32:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 


Here is the command that I used: 
#lttng enable-channel -k test
#Kernel channel test enabled for session hh
#lttng add-context -k -t perf:cpu:LLC-load-misses -c test
Error: perf:cpu:LLC-load-misses: Add kernel context failed
Warning: Some command(s) went wrong


I also tried it on other machines. 

It is possible that your architecture does not expose this counter to perf.
Did you try using this PMC with the perf tool ?

Thanks,

Mathieu


<<<<<<<    /-/ /\ /\/ / /\/ E /\/\ /\ T /    >>>>>>>>
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

  

[-- Attachment #1.2: Type: text/html, Size: 6783 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Lttng add-context perf commands gives me error

[Hani Nemati]

[-- Attachment #1.1: Type: text/plain, Size: 2446 bytes --]


Hi Mathieu, 
LCC is last level cache and it is shared between CPU cores. Also, I was running several VMs in background. That's why I used sleep command. Based on Intel's manual r0109 is raw identifier for LCC-load-misses, 
The question is how can I fix the problem since Lttng is using perf symbolic name. 

Thanks, 
Hani  

     On Wednesday, November 18, 2015 3:19 PM, Hani Nemati <hani_nemati@yahoo.com> wrote:
 

 There is no PERF symbolic name for LLC-load-misses but I can use raw identifier to find out Number of code read, data read that miss the LLC. Here is the command that I used.
 
#perf stat -e r0109 sleep 2
 Performance counter stats for 'sleep 2':

               117      r0109  

Do you have any idea why I don't have the symbolic name? Is there any way to give raw identifier instead of perf:cpu:LLC-load-misses? 







     On Wednesday, November 18, 2015 9:45 AM, Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:
   

 

----- On Nov 17, 2015, at 4:46 PM, Hani Nemati <hani_nemati@yahoo.com> wrote:

Hi, 
I am trying to count last level cache misses by using lttng add-context command. I use add-context to add perf:cpu:LLC-load-misses to my channel but I see an error. There is no error for other perf events. Just two perf events including perf:cpu:LLC-load-misses and  perf:cpu:LLC-store-misses give me the following error:
Error: perf:cpu:LLC-load-misses: Add kernel context failed
Warning: Some command(s) went wrong

Here is result of "uname -a" on my computer: 
Linux nemati 3.16.0-51-generic #69~14.04.1-Ubuntu SMP Wed Oct 7 15:32:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 


Here is the command that I used: 
#lttng enable-channel -k test
#Kernel channel test enabled for session hh
#lttng add-context -k -t perf:cpu:LLC-load-misses -c test
Error: perf:cpu:LLC-load-misses: Add kernel context failed
Warning: Some command(s) went wrong


I also tried it on other machines. 

It is possible that your architecture does not expose this counter to perf.
Did you try using this PMC with the perf tool ?

Thanks,

Mathieu


<<<<<<<    /-/ /\ /\/ / /\/ E /\/\ /\ T /    >>>>>>>>
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

   

  

[-- Attachment #1.2: Type: text/html, Size: 9760 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Lttng add-context perf commands gives me error

[Mathieu Desnoyers]

[-- Attachment #1.1: Type: text/plain, Size: 2686 bytes --]

Hi, 

Passing raw pmc identifiers is not supported at the moment. This is a feature we 
would like to add to lttng next year. 

Thanks, 

Mathieu 

----- On Nov 19, 2015, at 10:37 AM, Hani Nemati <hani_nemati@yahoo.com> wrote: 

> Hi Mathieu ,
> LCC is last level cache and it is shared between CPU cores. Also, I was running
> several VMs in background. That's why I used sleep command.
> Based on Intel's manual r0109 is raw identifier for LCC-load-misses,
> The question is how can I fix the problem since Lttng is using perf symbolic
> name.

> Thanks,
> Hani

> On Wednesday, November 18, 2015 3:19 PM, Hani Nemati <hani_nemati@yahoo.com>
> wrote:

> There is no PERF symbolic name for LLC-load-misses but I can use raw identifier
> to find out Number of code read, data read that miss the LLC. Here is the
> command that I used.

> #perf stat -e r0109 sleep 2
> Performance counter stats for 'sleep 2':

> 117 r0109

> Do you have any idea why I don't have the symbolic name? Is there any way to
> give raw identifier instead of perf:cpu:LLC-load-misses?

> On Wednesday, November 18, 2015 9:45 AM, Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:

> ----- On Nov 17, 2015, at 4:46 PM, Hani Nemati <hani_nemati@yahoo.com> wrote:

>> Hi,
>> I am trying to count last level cache misses by using lttng add-context command.
>> I use add-context to add perf:cpu:LLC-load-misses to my channel but I see an
>> error. There is no error for other perf events. Just two perf events including
>> perf:cpu:LLC-load-misses and perf:cpu:LLC-store-misses give me the following
>> error:
>> Error: perf:cpu:LLC-load-misses: Add kernel context failed
>> Warning: Some command(s) went wrong

>> Here is result of "uname -a" on my computer:
>> Linux nemati 3.16.0-51-generic #69~14.04.1-Ubuntu SMP Wed Oct 7 15:32:41 UTC
>> 2015 x86_64 x86_64 x86_64 GNU/Linux

>> Here is the command that I used:
>> #lttng enable-channel -k test
>> #Kernel channel test enabled for session hh
>> #lttng add-context -k -t perf:cpu:LLC-load-misses -c test
>> Error: perf:cpu:LLC-load-misses: Add kernel context failed
>> Warning: Some command(s) went wrong

>> I also tried it on other machines.

> It is possible that your architecture does not expose this counter to perf.
> Did you try using this PMC with the perf tool ?

> Thanks,

> Mathieu

>> <<<<<<<
>> /-/ /\ /\/ / /\/ E /\/\ /\ T
>> / >>>>>>>>

>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 

[-- Attachment #1.2: Type: text/html, Size: 11002 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Lttng add-context perf commands gives me error

[David Boles (dboles)]

I’m using the liburcu system (qsbr flavor) with its list structures as well as a variant of a bonsai tree.

In my unit testing code I have a race between the rcu callback invocation thread (currently just the default one) invoking a callback it was given and the main thread tearing down data that callback depends on.

In looking at the call_rcu_thread() function, it looks like that thread sleeps for no more than ~10ms. However, even if I guarantee 20ms elapses between the last time call_rcu() is called and my teardown, I am still seeing the callback thread processing "old" call_rcu() requests. This appears correlated with the callback thread having thousands of free's to process.

In my real application I happened to have set things up so that everything associated with rcu-managed allocation is freed with call_rcu() so I'm not seeing a problem there (so far). However, I suspect the same issue could crop up with abnormal exit paths.

How do I reliably determine that all rcu callbacks registered prior to some point in time have been processed? 

Thanks,

David Boles

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: liburcu qsbr call_rcu quiescence

[Mathieu Desnoyers]

Hi David,

Please create a new discussion thread when posting a question next
time. It will help others following the lttng-dev archives. You will
find the answer to your question below,

----- On Nov 20, 2015, at 5:33 PM, David Boles (dboles) dboles@micron.com wrote:

> I’m using the liburcu system (qsbr flavor) with its list structures as well as a
> variant of a bonsai tree.
> 
> In my unit testing code I have a race between the rcu callback invocation thread
> (currently just the default one) invoking a callback it was given and the main
> thread tearing down data that callback depends on.
> 
> In looking at the call_rcu_thread() function, it looks like that thread sleeps
> for no more than ~10ms. However, even if I guarantee 20ms elapses between the
> last time call_rcu() is called and my teardown, I am still seeing the callback
> thread processing "old" call_rcu() requests. This appears correlated with the
> callback thread having thousands of free's to process.
> 
> In my real application I happened to have set things up so that everything
> associated with rcu-managed allocation is freed with call_rcu() so I'm not
> seeing a problem there (so far). However, I suspect the same issue could crop
> up with abnormal exit paths.
> 
> How do I reliably determine that all rcu callbacks registered prior to some
> point in time have been processed?

There's an API for that! :-)

See rcu_barrier(). It has been introduced in liburcu 0.8.

Ref:
https://github.com/urcu/userspace-rcu/blob/master/doc/rcu-api.md

"void rcu_barrier(void);

Wait for all call_rcu() work initiated prior to rcu_barrier() by any thread
on the system to have completed before rcu_barrier() returns. rcu_barrier()
should never be called from a call_rcu() thread. This function can be used,
for instance, to ensure that all memory reclaim involving a shared object
has completed before allowing dlclose() of this shared object to complete."

Thanks,

Mathieu

> 
> Thanks,
> 
> David Boles
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev