Re: Beginner question: how to inspect scheduling of multi-threaded user application?

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Francis Deslauriers]

[-- Attachment #1.1: Type: text/plain, Size: 1303 bytes --]

Hi David,
If you specifically want to trace the scheduling of the threads of your
app, you don't need custom tracepoints.
Enabling the sched_switch kernel event will give you both of cpu id and
thread id. Look at the cpu_id and next_tid fields.

You can enable the sched_switch event using : lttng enable-event -k
sched_switch

Cheers,
Francis

2016-08-24 3:17 GMT-04:00 David Aldrich <David.Aldrich@emea.nec.com>:

> Hi
>
>
>
> I am new to tracing in Linux and to lttng. I have a multi-threaded user
> application and I want to see:
>
>
>
> 1)      When the threads are scheduled to run
>
> 2)      Which cores the threads are running on.
>
>
>
> I have installed lttng on Ubuntu 14.04 LTS.  I am expecting to visualise
> the trace using TraceCompass.
>
>
>
> I have read the following doc section:
>
>
>
> http://lttng.org/docs/#doc-tracing-your-own-user-application
>
>
>
> In order to collect my trace, must I define custom tracepoint definitions ( in
> a tracepoint provider header file ), and insert tracepoints into my user
> application, or is there a simpler way of achieving my goal?
>
>
>
> Best regards
>
>
>
> David
>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>

[-- Attachment #1.2: Type: text/html, Size: 3173 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Jonathan Rajotte]

[-- Attachment #1.1: Type: text/plain, Size: 2192 bytes --]

Hi,

On Aug 24, 2016 12:18 PM, "Francis Deslauriers" <francis.deslauriers@mail.
utoronto.ca> wrote:
>
> Hi David,
> If you specifically want to trace the scheduling of the threads of your
app, you don't need custom tracepoints.
> Enabling the sched_switch kernel event will give you both of cpu id and
thread id. Look at the cpu_id and next_tid fields.
>
> You can enable the sched_switch event using : lttng enable-event -k
sched_switch

In TraceCompass you can inspect this data with the control flow view and
the Ressource view under the Kernel analysis node under the trace node in
the project explorer.

I'm not sure of the base requirement for those views you can use the safe
enable-event:

lttng enable-event -k 'sched*'


You can also use "lttng track" to limi the gathering of event to a certain
pid.


Another way to reduce the scope would be to filter per procname:


lttng create
lttng add-context -k -t procname
lttng enable-event 'sched*' --filter '$ctx.procname == "PROCNAMEHERE"'

>
> Cheers,
> Francis
>
>
> 2016-08-24 3:17 GMT-04:00 David Aldrich <David.Aldrich@emea.nec.com>:
>>
>> Hi
>>
>>
>>
>> I am new to tracing in Linux and to lttng. I have a multi-threaded user
application and I want to see:
>>
>>
>>
>> 1)      When the threads are scheduled to run
>>
>> 2)      Which cores the threads are running on.
>>
>>
>>
>> I have installed lttng on Ubuntu 14.04 LTS.  I am expecting to visualise
the trace using TraceCompass.
>>
>>
>>
>> I have read the following doc section:
>>
>>
>>
>> http://lttng.org/docs/#doc-tracing-your-own-user-application
>>
>>
>>
>> In order to collect my trace, must I define custom tracepoint
definitions ( in a tracepoint provider header file ), and insert
tracepoints into my user application, or is there a simpler way of
achieving my goal?
>>
>>
>>
>> Best regards
>>
>>
>>
>> David
>>
>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>

[-- Attachment #1.2: Type: text/html, Size: 3704 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Jonathan Rajotte]

[-- Attachment #1.1: Type: text/plain, Size: 2684 bytes --]

Sorry had a sending problem.

Here is the rest.

On Wed, Aug 24, 2016 at 12:35 PM, Jonathan Rajotte <
jonathan.r.julien@gmail.com> wrote:

> Hi,
>
> On Aug 24, 2016 12:18 PM, "Francis Deslauriers" <
> francis.deslauriers@mail.utoronto.ca> wrote:
> >
> > Hi David,
> > If you specifically want to trace the scheduling of the threads of your
> app, you don't need custom tracepoints.
> > Enabling the sched_switch kernel event will give you both of cpu id and
> thread id. Look at the cpu_id and next_tid fields.
> >
> > You can enable the sched_switch event using : lttng enable-event -k
> sched_switch
>
> In TraceCompass you can inspect this data with the control flow view and
> the Ressource view under the Kernel analysis node under the trace node in
> the project explorer.
>
> I'm not sure of the base requirement for those views you can use the safe
> enable-event:
>

replace "safe" by "easiest".


> lttng enable-event -k 'sched*'
>
>
> You can also use "lttng track" to limi the gathering of event to a certain
> pid.
>
>
> Another way to reduce the scope would be to filter per procname:
>
>
> lttng create
> lttng add-context -k -t procname
> lttng enable-event 'sched*' --filter '$ctx.procname == "PROCNAMEHERE"'
>
lttng start


PROCNAMEHERE can contain '*' wildcard. See the man page for more
information.

Cheers



> >
> > Cheers,
> > Francis
> >
> >
> > 2016-08-24 3:17 GMT-04:00 David Aldrich <David.Aldrich@emea.nec.com>:
> >>
> >> Hi
> >>
> >>
> >>
> >> I am new to tracing in Linux and to lttng. I have a multi-threaded user
> application and I want to see:
> >>
> >>
> >>
> >> 1)      When the threads are scheduled to run
> >>
> >> 2)      Which cores the threads are running on.
> >>
> >>
> >>
> >> I have installed lttng on Ubuntu 14.04 LTS.  I am expecting to
> visualise the trace using TraceCompass.
> >>
> >>
> >>
> >> I have read the following doc section:
> >>
> >>
> >>
> >> http://lttng.org/docs/#doc-tracing-your-own-user-application
> >>
> >>
> >>
> >> In order to collect my trace, must I define custom tracepoint
> definitions ( in a tracepoint provider header file ), and insert
> tracepoints into my user application, or is there a simpler way of
> achieving my goal?
> >>
> >>
> >>
> >> Best regards
> >>
> >>
> >>
> >> David
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> lttng-dev mailing list
> >> lttng-dev@lists.lttng.org
> >> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >>
> >
> >
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev@lists.lttng.org
> > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >
>



-- 
Jonathan Rajotte Julien

[-- Attachment #1.2: Type: text/html, Size: 4975 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[David Aldrich]

[-- Attachment #1.1: Type: text/plain, Size: 4271 bytes --]

Hi Jonathan and Francis

Thanks for your helpful replies.  I am having a bit of trouble implementing your suggestions. Here is my session:


$ lttng create

Spawning a session daemon

PERROR [2885/2893]: bind inet: Address already in use (in lttcomm_bind_inet_sock() at inet.c:109)

Warning: An other session daemon is using this JUL port. JUL support will be deactivated not interfering with the tracing.

Session auto-20160826-083823 created.

Traces will be written in <snip>/lttng-traces/auto-20160826-083823



$ sudo lttng enable-event -k sched_switch

Error: Event sched_switch: Session name not found (channel channel0, session auto-20160826-083823)

Warning: Some command(s) went wrong

There seem to be two problems here:


1)      How to kill a lttng daemon that is already running?


2)      How to specify a session?


Sorry that these are basic questions. I would be grateful if you can help please.

Best regards

David

From: Jonathan Rajotte [mailto:jonathan.r.julien@gmail.com]
Sent: 24 August 2016 17:39
To: Francis Deslauriers <francis.deslauriers@mail.utoronto.ca>
Cc: David Aldrich <David.Aldrich@EMEA.NEC.COM>; lttng-dev@lists.lttng.org
Subject: Re: [lttng-dev] Beginner question: how to inspect scheduling of multi-threaded user application?

Sorry had a sending problem.

Here is the rest.

On Wed, Aug 24, 2016 at 12:35 PM, Jonathan Rajotte <jonathan.r.julien@gmail.com<mailto:jonathan.r.julien@gmail.com>> wrote:

Hi,

On Aug 24, 2016 12:18 PM, "Francis Deslauriers" <francis.deslauriers@mail.utoronto.ca<mailto:francis.deslauriers@mail.utoronto.ca>> wrote:
>
> Hi David,
> If you specifically want to trace the scheduling of the threads of your app, you don't need custom tracepoints.
> Enabling the sched_switch kernel event will give you both of cpu id and thread id. Look at the cpu_id and next_tid fields.
>
> You can enable the sched_switch event using : lttng enable-event -k sched_switch

In TraceCompass you can inspect this data with the control flow view and the Ressource view under the Kernel analysis node under the trace node in the project explorer.

I'm not sure of the base requirement for those views you can use the safe enable-event:

replace "safe" by "easiest".


lttng enable-event -k 'sched*'



You can also use "lttng track" to limi the gathering of event to a certain pid.



Another way to reduce the scope would be to filter per procname:



lttng create
lttng add-context -k -t procname
lttng enable-event 'sched*' --filter '$ctx.procname == "PROCNAMEHERE"'
lttng start


PROCNAMEHERE can contain '*' wildcard. See the man page for more information.

Cheers



>
> Cheers,
> Francis
>
>
> 2016-08-24 3:17 GMT-04:00 David Aldrich <David.Aldrich@emea.nec.com<mailto:David.Aldrich@emea.nec.com>>:
>>
>> Hi
>>
>>
>>
>> I am new to tracing in Linux and to lttng. I have a multi-threaded user application and I want to see:
>>
>>
>>
>> 1)      When the threads are scheduled to run
>>
>> 2)      Which cores the threads are running on.
>>
>>
>>
>> I have installed lttng on Ubuntu 14.04 LTS.  I am expecting to visualise the trace using TraceCompass.
>>
>>
>>
>> I have read the following doc section:
>>
>>
>>
>> http://lttng.org/docs/#doc-tracing-your-own-user-application
>>
>>
>>
>> In order to collect my trace, must I define custom tracepoint definitions ( in a tracepoint provider header file ), and insert tracepoints into my user application, or is there a simpler way of achieving my goal?
>>
>>
>>
>> Best regards
>>
>>
>>
>> David
>>
>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org<mailto:lttng-dev@lists.lttng.org>
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org<mailto:lttng-dev@lists.lttng.org>
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>



--
Jonathan Rajotte Julien



Click here<https://www.mailcontrol.com/sr/unEwdsSZGlHGX2PQPOmvUmQbZlJUA6MYaHsjpmEwGrBfobipAMW1ZCPMpsLI18FS3tdTuAJyxqmiyla!1rT5xQ==> to report this email as spam.

[-- Attachment #1.2: Type: text/html, Size: 15805 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Jonathan Rajotte Julien]

Hi David,

> Hi Jonathan and Francis
>
> Thanks for your helpful replies.  I am having a bit of trouble implementing your suggestions. Here is my session:
>
>
> $ lttng create
>
> Spawning a session daemon
>
> PERROR [2885/2893]: bind inet: Address already in use (in lttcomm_bind_inet_sock() at inet.c:109)
>
> Warning: An other session daemon is using this JUL port. JUL support will be deactivated not interfering with the tracing.
>
> Session auto-20160826-083823 created.
>
> Traces will be written in <snip>/lttng-traces/auto-20160826-083823

First let's get a clean state.
Make sure the user is in the "tracing" group if you plan to do kernel 
tracing from the user.

groups UsernameHere

If not make sure to add the user to the 'tracing' group.


Kill all lttng related daemon.

sudo pkill lttng-sessiond
sudo pkill lttng-relayd

If you installed it via apt-get/ppa a lttng-sessiond should spawn 
otherwise start one with sudo:

sudo lttng-sessiond
or
sudo lttng-sessiond -b
or
sudo lttng-sessiond -d

See lttng-sessiond man page for information regarding these options.

>
>
>
> $ sudo lttng enable-event -k sched_switch
>
> Error: Event sched_switch: Session name not found (channel channel0, session auto-20160826-083823)
>
> Warning: Some command(s) went wrong

This is due to the fact that the state (current session) is kept on a 
per-user basis under ~/.lttng. You started the session with another user 
and then used sudo for "lttng enable-event".

If you want to pass the session name most of the commands support the 
'-s SessionNameHere' option.

>
> There seem to be two problems here:
>
>
> 1)      How to kill a lttng daemon that is already running?
>
>
> 2)      How to specify a session?
>
>
> Sorry that these are basic questions. I would be grateful if you can help please.

Np.

Make sure to check http://lttng.org/docs/ if you did not already.

Cheers

>
> Best regards
>
> David
>
> From: Jonathan Rajotte [mailto:jonathan.r.julien@gmail.com]
> Sent: 24 August 2016 17:39
> To: Francis Deslauriers <francis.deslauriers@mail.utoronto.ca>
> Cc: David Aldrich <David.Aldrich@EMEA.NEC.COM>; lttng-dev@lists.lttng.org
> Subject: Re: [lttng-dev] Beginner question: how to inspect scheduling of multi-threaded user application?
>
> Sorry had a sending problem.
>
> Here is the rest.
>
> On Wed, Aug 24, 2016 at 12:35 PM, Jonathan Rajotte <jonathan.r.julien@gmail.com<mailto:jonathan.r.julien@gmail.com>> wrote:
>
> Hi,
>
> On Aug 24, 2016 12:18 PM, "Francis Deslauriers" <francis.deslauriers@mail.utoronto.ca<mailto:francis.deslauriers@mail.utoronto.ca>> wrote:
>> Hi David,
>> If you specifically want to trace the scheduling of the threads of your app, you don't need custom tracepoints.
>> Enabling the sched_switch kernel event will give you both of cpu id and thread id. Look at the cpu_id and next_tid fields.
>>
>> You can enable the sched_switch event using : lttng enable-event -k sched_switch
> In TraceCompass you can inspect this data with the control flow view and the Ressource view under the Kernel analysis node under the trace node in the project explorer.
>
> I'm not sure of the base requirement for those views you can use the safe enable-event:
>
> replace "safe" by "easiest".
>
>
> lttng enable-event -k 'sched*'
>
>
>
> You can also use "lttng track" to limi the gathering of event to a certain pid.
>
>
>
> Another way to reduce the scope would be to filter per procname:
>
>
>
> lttng create
> lttng add-context -k -t procname
> lttng enable-event 'sched*' --filter '$ctx.procname == "PROCNAMEHERE"'
> lttng start
>
>
> PROCNAMEHERE can contain '*' wildcard. See the man page for more information.
>
> Cheers
>
>
>
>> Cheers,
>> Francis
>>
>>
>> 2016-08-24 3:17 GMT-04:00 David Aldrich <David.Aldrich@emea.nec.com<mailto:David.Aldrich@emea.nec.com>>:
>>> Hi
>>>
>>>
>>>
>>> I am new to tracing in Linux and to lttng. I have a multi-threaded user application and I want to see:
>>>
>>>
>>>
>>> 1)      When the threads are scheduled to run
>>>
>>> 2)      Which cores the threads are running on.
>>>
>>>
>>>
>>> I have installed lttng on Ubuntu 14.04 LTS.  I am expecting to visualise the trace using TraceCompass.
>>>
>>>
>>>
>>> I have read the following doc section:
>>>
>>>
>>>
>>> http://lttng.org/docs/#doc-tracing-your-own-user-application
>>>
>>>
>>>
>>> In order to collect my trace, must I define custom tracepoint definitions ( in a tracepoint provider header file ), and insert tracepoints into my user application, or is there a simpler way of achieving my goal?
>>>
>>>
>>>
>>> Best regards
>>>
>>>
>>>
>>> David
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev@lists.lttng.org<mailto:lttng-dev@lists.lttng.org>
>>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org<mailto:lttng-dev@lists.lttng.org>
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>
>
> --
> Jonathan Rajotte Julien
>
>
>
> Click here<https://www.mailcontrol.com/sr/unEwdsSZGlHGX2PQPOmvUmQbZlJUA6MYaHsjpmEwGrBfobipAMW1ZCPMpsLI18FS3tdTuAJyxqmiyla!1rT5xQ==> to report this email as spam.
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Jonathan R. Julien
Efficios

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[David Aldrich]

Hi

I'm afraid I'm still struggling with getting started with lttng. To recap, I want to use lttng + TraceCompass to monitor the scheduling of threads in my multi-threaded C++ application on Ubuntu.

Here is my session:

$ sudo pkill lttng-sessiond
$ sudo lttng-sessiond
Error: Already running daemon.

$ lttng create demo_session Spawning a session daemon Session demo_session created.
Traces will be written in /home/<snip>/lttng-traces/demo_session-20160915-152653
<snip>:~$ PERROR [14956/14964]: bind inet: Address already in use (in lttcomm_bind_inet_sock() at inet.c:109)
Warning: An other session daemon is using this JUL port. JUL support will be deactivated not interfering with the tracing.

$ sudo lttng enable-event -k sched_switch -s demo_session
Error: Event sched_switch: Session name not found (channel channel0, session demo_session)
Warning: Some command(s) went wrong

As you can see, each of these command generated a warning or error.  Please will you suggest where I am going wrong?

Best regards

David

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Jonathan Rajotte Julien]

Hi,


On 2016-09-15 10:36 AM, David Aldrich wrote:
> Hi
>
> I'm afraid I'm still struggling with getting started with lttng. To recap, I want to use lttng + TraceCompass to monitor the scheduling of threads in my multi-threaded C++ application on Ubuntu.
>
> Here is my session:
>
> $ sudo pkill lttng-sessiond
> $ sudo lttng-sessiond
> Error: Already running daemon.

Would you happen to have installed lttng via the ppa or apt-get/equivalent.

If I remember correctly those package install an actual daemon. If you 
kill it, it will get re spawned.

Make sure that add your user to the tracing group then log out/log in.

Cheers
>
> $ lttng create demo_session Spawning a session daemon Session demo_session created.
> Traces will be written in /home/<snip>/lttng-traces/demo_session-20160915-152653
> <snip>:~$ PERROR [14956/14964]: bind inet: Address already in use (in lttcomm_bind_inet_sock() at inet.c:109)
> Warning: An other session daemon is using this JUL port. JUL support will be deactivated not interfering with the tracing.
>
> $ sudo lttng enable-event -k sched_switch -s demo_session
> Error: Event sched_switch: Session name not found (channel channel0, session demo_session)
> Warning: Some command(s) went wrong
>
> As you can see, each of these command generated a warning or error.  Please will you suggest where I am going wrong?
>
> Best regards
>
> David
>

-- 
Jonathan R. Julien
Efficios

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[David Aldrich]

Hi Jonathan

Thanks, I had not logged out/in after joining the tracing group.

I now see:

$ lttng create demo_session
Session demo_session created.
$ sudo lttng enable-event -k sched_switch -s demo_session
Error: Event sched_switch: Kernel tracer not available (channel channel0, session demo_session)
Warning: Some command(s) went wrong

What do I need to install?

Best regards

David
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Jonathan Rajotte Julien]

Hi,

On 2016-09-15 11:41 AM, David Aldrich wrote:
> Hi Jonathan
>
> Thanks, I had not logged out/in after joining the tracing group.
>
> I now see:
>
> $ lttng create demo_session
> Session demo_session created.
> $ sudo lttng enable-event -k sched_switch -s demo_session
> Error: Event sched_switch: Kernel tracer not available (channel channel0, session demo_session)
> Warning: Some command(s) went wrong
>
> What do I need to install?

Did you have a look at http://lttng.org/download/#ubuntu ?

I recommend the use of the ppa if possible. For kernel tracing make sure 
to install the lttng-modules-dkms.

Make sure there is not error during the installation. It might happen 
but should not :) .

If it's not done already make sure to drop by 
http://lttng.org/docs/#doc-getting-started for more informations.

Cheers

>
> Best regards
>
> David

-- 
Jonathan R. Julien
Efficios

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[David Aldrich]

Thank you, things are looking much better now.

David

> -----Original Message-----
> From: Jonathan Rajotte Julien [mailto:Jonathan.rajotte-julien@efficios.com]
> Sent: 15 September 2016 16:50
> To: David Aldrich <David.Aldrich@EMEA.NEC.COM>; lttng-
> dev@lists.lttng.org
> Subject: Re: [lttng-dev] Beginner question: how to inspect scheduling of
> multi-threaded user application?
> 
> Hi,
> 
> On 2016-09-15 11:41 AM, David Aldrich wrote:
> > Hi Jonathan
> >
> > Thanks, I had not logged out/in after joining the tracing group.
> >
> > I now see:
> >
> > $ lttng create demo_session
> > Session demo_session created.
> > $ sudo lttng enable-event -k sched_switch -s demo_session
> > Error: Event sched_switch: Kernel tracer not available (channel
> > channel0, session demo_session)
> > Warning: Some command(s) went wrong
> >
> > What do I need to install?
> 
> Did you have a look at http://lttng.org/download/#ubuntu ?
> 
> I recommend the use of the ppa if possible. For kernel tracing make sure to
> install the lttng-modules-dkms.
> 
> Make sure there is not error during the installation. It might happen but
> should not :) .
> 
> If it's not done already make sure to drop by http://lttng.org/docs/#doc-
> getting-started for more informations.
> 
> Cheers
> 
> >
> > Best regards
> >
> > David
> 
> --
> Jonathan R. Julien
> Efficios
> 
> 
> 
>  Click
> https://www.mailcontrol.com/sr/+PZCEcpjy5nGX2PQPOmvUsaHTTHE2WHLlI
> gN8TKUyPvY+09UnGjaICLUQ+RbaoZl0ti1ys3tAhuJRcENOpaP7A==  to report
> this email as spam.
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[David Aldrich]

Hi

I wonder if I might ask for some more guidance please?

I now have lttng and Trace Compass running on Ubuntu.  I want to inspect the scheduling of threads in my user-space application.  So far, I have successfully captured a trace with the sched_switch event enabled:

$ sudo lttng enable-event -k sched_switch

This indeed shows my app's threads.  However, it also shows all the other threads in the system and does not show clearly (to my understanding) when the threads are scheduled / blocked.

I would like some guidance on how to improve this tracing.

1) Would it be better if I defined my own “tracepoint providers” and then set trace events for when a thread wakes / blocks?

2) How could I show which core a thread is running on?

Best regards

David
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Jonathan Rajotte Julien]

Hi


On 2016-09-16 07:43 AM, David Aldrich wrote:
> Hi
>
> I wonder if I might ask for some more guidance please?
>
> I now have lttng and Trace Compass running on Ubuntu.  I want to inspect the scheduling of threads in my user-space application.  So far, I have successfully captured a trace with the sched_switch event enabled:
>
> $ sudo lttng enable-event -k sched_switch
>
> This indeed shows my app's threads.  However, it also shows all the other threads in the system and does not show clearly (to my understanding) when the threads are scheduled / blocked.
>
> I would like some guidance on how to improve this tracing.
>
> 1) Would it be better if I defined my own “tracepoint providers” and then set trace events for when a thread wakes / blocks?

Could be a solution but you might want the built-in utility of lttng to 
filter out all noise first using the filtering feature.

e.g I'm interested in sched_switch event where chrome is involved

lttng create
lttng enable-event --kernel "sched_switch" --filter 'prev_comm == 
"chrome*" || next_comm == "chrome*"'
lttng start
....

Using TraceCompass you will be mainly interested in the Control Flow 
View and the Resource view (show per cpu timeline activity).

>
> 2) How could I show which core a thread is running on?

The Resource view from TraceCompass should help here.

Cheers

>
> Best regards
>
> David
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Jonathan R. Julien
Efficios

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[David Aldrich]

Hi Jonathan

Thanks for your reply. What is 'comm' in 'prev_comm'?

Best regards

David

> -----Original Message-----
> From: lttng-dev [mailto:lttng-dev-bounces@lists.lttng.org] On Behalf Of
> Jonathan Rajotte Julien
> Sent: 16 September 2016 18:37
> To: lttng-dev@lists.lttng.org
> Subject: Re: [lttng-dev] Beginner question: how to inspect scheduling of
> multi-threaded user application?
> 
> Hi
> 
> 
> On 2016-09-16 07:43 AM, David Aldrich wrote:
> > Hi
> >
> > I wonder if I might ask for some more guidance please?
> >
> > I now have lttng and Trace Compass running on Ubuntu.  I want to inspect
> the scheduling of threads in my user-space application.  So far, I have
> successfully captured a trace with the sched_switch event enabled:
> >
> > $ sudo lttng enable-event -k sched_switch
> >
> > This indeed shows my app's threads.  However, it also shows all the other
> threads in the system and does not show clearly (to my understanding)
> when the threads are scheduled / blocked.
> >
> > I would like some guidance on how to improve this tracing.
> >
> > 1) Would it be better if I defined my own “tracepoint providers” and then
> set trace events for when a thread wakes / blocks?
> 
> Could be a solution but you might want the built-in utility of lttng to filter
> out all noise first using the filtering feature.
> 
> e.g I'm interested in sched_switch event where chrome is involved
> 
> lttng create
> lttng enable-event --kernel "sched_switch" --filter 'prev_comm == "chrome*"
> || next_comm == "chrome*"'
> lttng start
> ....
> 
> Using TraceCompass you will be mainly interested in the Control Flow View
> and the Resource view (show per cpu timeline activity).
> 
> >
> > 2) How could I show which core a thread is running on?
> 
> The Resource view from TraceCompass should help here.
> 
> Cheers
> 
> >
> > Best regards
> >
> > David
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev@lists.lttng.org
> > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 
> --
> Jonathan R. Julien
> Efficios
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 
> 
>  Click
> https://www.mailcontrol.com/sr/YRkaDi7sHfjGX2PQPOmvUu5zZAYN1MosBr
> 5yrM8i6yLTqipdxOLNFC1fulULj1tW45N!9c+2yXxm4uq9tqSB7g==  to report
> this email as spam.
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Jonathan Rajotte Julien]

Hi,

On 2016-09-19 11:38 AM, David Aldrich wrote:
> Hi Jonathan
>
> Thanks for your reply. What is 'comm' in 'prev_comm'?

It correspond to the "comm" member of the task_struct struct [1].

The important part of [1]:

char comm[TASK_COMM_LEN]; /* executable name excluding path
                              - access with [gs]et_task_comm (which lock
                                it with task_lock())
                              - initialized normally by setup_new_exec */

Also know as the executable name.

Cheers.

[1] include/linux/sched.h of the kernel tree


>
> Best regards
>
> David
>
>> -----Original Message-----
>> From: lttng-dev [mailto:lttng-dev-bounces@lists.lttng.org] On Behalf Of
>> Jonathan Rajotte Julien
>> Sent: 16 September 2016 18:37
>> To: lttng-dev@lists.lttng.org
>> Subject: Re: [lttng-dev] Beginner question: how to inspect scheduling of
>> multi-threaded user application?
>>
>> Hi
>>
>>
>> On 2016-09-16 07:43 AM, David Aldrich wrote:
>>> Hi
>>>
>>> I wonder if I might ask for some more guidance please?
>>>
>>> I now have lttng and Trace Compass running on Ubuntu.  I want to inspect
>> the scheduling of threads in my user-space application.  So far, I have
>> successfully captured a trace with the sched_switch event enabled:
>>> $ sudo lttng enable-event -k sched_switch
>>>
>>> This indeed shows my app's threads.  However, it also shows all the other
>> threads in the system and does not show clearly (to my understanding)
>> when the threads are scheduled / blocked.
>>> I would like some guidance on how to improve this tracing.
>>>
>>> 1) Would it be better if I defined my own “tracepoint providers” and then
>> set trace events for when a thread wakes / blocks?
>>
>> Could be a solution but you might want the built-in utility of lttng to filter
>> out all noise first using the filtering feature.
>>
>> e.g I'm interested in sched_switch event where chrome is involved
>>
>> lttng create
>> lttng enable-event --kernel "sched_switch" --filter 'prev_comm == "chrome*"
>> || next_comm == "chrome*"'
>> lttng start
>> ....
>>
>> Using TraceCompass you will be mainly interested in the Control Flow View
>> and the Resource view (show per cpu timeline activity).
>>
>>> 2) How could I show which core a thread is running on?
>> The Resource view from TraceCompass should help here.
>>
>> Cheers
>>
>>> Best regards
>>>
>>> David
>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev@lists.lttng.org
>>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>> --
>> Jonathan R. Julien
>> Efficios
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>>   Click
>> https://www.mailcontrol.com/sr/YRkaDi7sHfjGX2PQPOmvUu5zZAYN1MosBr
>> 5yrM8i6yLTqipdxOLNFC1fulULj1tW45N!9c+2yXxm4uq9tqSB7g==  to report
>> this email as spam.

-- 
Jonathan R. Julien
Efficios

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Beginner question: how to inspect scheduling of multi-threaded user application?

[Philippe Proulx]

On Mon, Sep 19, 2016 at 11:38 AM, David Aldrich
<David.Aldrich@emea.nec.com> wrote:
>
> Hi Jonathan
>
> Thanks for your reply. What is 'comm' in 'prev_comm'?

"comm" stands for "command".

See this in sched.h:

    /* Task command name length */
    #define TASK_COMM_LEN 16

Phil

>
> Best regards
>
> David
>
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Beginner question: how to inspect scheduling of multi-threaded user application?

[David Aldrich]

[-- Attachment #1.1: Type: text/plain, Size: 667 bytes --]

Hi

I am new to tracing in Linux and to lttng. I have a multi-threaded user application and I want to see:


1)      When the threads are scheduled to run

2)      Which cores the threads are running on.

I have installed lttng on Ubuntu 14.04 LTS.  I am expecting to visualise the trace using TraceCompass.

I have read the following doc section:

http://lttng.org/docs/#doc-tracing-your-own-user-application

In order to collect my trace, must I define custom tracepoint definitions ( in a tracepoint provider header file ), and insert tracepoints into my user application, or is there a simpler way of achieving my goal?

Best regards

David


[-- Attachment #1.2: Type: text/html, Size: 5792 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev