Re: No trace files are generated for UST

Re: No trace files are generated for UST

[Thibault, Daniel]

> De : Manikandan Govindaswamy [mailto:magov@vestas.com] 
> Envoyé : 6 décembre 2013 04:49
>
> Now I'm trying with ust examples : demo and easy-ust and I'm facing the same problem of no tracing is generated.
>
> Please find the sessiond log and the execution sequence as below for sample.
>
> I tried the same for demo and it looks the same behaviour
>
> root@/tmp/lttng-build-root/bin # ./lttng create
> Session auto-20131206-113744 created.
> Traces will be written in /root/lttng-traces/auto-20131206-113744
> root@/tmp/lttng-build-root/bin # ./lttng enable-event -u --all
> All UST events are enabled in channel channel0
> root@/tmp/lttng-build-root/bin # ./lttng start
> Tracing started for session auto-20131206-113744
> root@/tmp/lttng-build-root/bin # cd ../../
> root@/tmp # ./sample
> root@/tmp # cd - /tmp/lttng-build-root/bin
> root@/tmp/lttng-build-root/bin # ./lttng stop
> Waiting for data availability
> Tracing stopped for session auto-20131206-113744
> root@/tmp/lttng-build-root/bin # ./lttng destroy
> Session auto-20131206-113744 destroyed
> root@/tmp/lttng-build-root/bin # cd /
> root@/ # cd root/
> root@~ # ls lttng-traces
> root@~ # cd lttng-traces/
> root@~/lttng-traces # ls
> test-20131205-180744
> root@~/lttng-traces #

   That works fine on my system.  You sure lttng-traces/test-20131205-180744 has an empty ust/uid/1000/64-bit/channel0_0 file?

   Did you 'make install' lttng-ust?  liblttng-ust must be in the $PATH, so try to run sample like this during the session:

$ LD_PRELOAD=/usr/local/lib/liblttng-ust ./sample

   Otherwise, we'll take a look at the output of the make.  In lttng-ust/doc/examples/easy-ust:

$ gcc -I. -c -o tp.o -x c -D TRACEPOINT_CREATE_PROBES sample_component_provider.h
$ gcc -I. -c -o sample.o sample.c
$ gcc -o sample sample.o tp.o -ldl -llttng-ust

   (Here I chose to skip tp.c entirely: '-x c -D TRACEPOINT_CREATE_PROBES sample_component_provider.h' is equivalent to 'tp.c' in the tp.o gcc line)

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
[Valcartier]\\vsi-wdut-o980, [Valcartier]\\edge
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>

Re: No trace files are generated for UST

[Thibault, Daniel]

If:

$ gcc -o sample sample.o tp.o -ldl -llttng-ust

Fails with "error: cannot find -llttng-ust", then do this instead:

$ gcc -o sample sample.o tp.o -L/usr/local/lib -ldl -llttng-ust

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC) Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS) R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>

Re: No trace files are generated for UST

[Thibault, Daniel]

> As I'm using the custom image on x86 machine, i've kept the lib in /tmp/lttng-build-root/lib folder and this sample able to see the library as well.
>
> And the same way I able to create the trace files for kernel but not for ust. There were no dmesg as well.
>
> [...]
> root@/tmp # ldd sample
>         linux-vdso.so.1 (0x00007fff2a135000)
>         libdl.so.2 => /lib64/libdl.so.2 (0x00007fe2cccb5000)
>         liblttng-ust.so.0 => /tmp/lttng-build-root/lib/liblttng-ust.so.0 (0x00007fe2cca51000)
>         [...]

   Try running sample using:

LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample

   If the LD_PRELOAD succeeds, we will know at least that liblttng-ust is reachable by sample.

   The modules have nothing to do with UST: kernel and user-space tracing are essentially independent features.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>

Re: No trace files are generated for UST

[Manikandan Govindaswamy]

Yes, with that LD_PRELOAD sample able to run.

root@/tmp/lttng-build-root/bin # ./lttng create app
Session app created.
Traces will be written in /root/lttng-traces/app-20131206-174935
root@/tmp/lttng-build-root/bin # dmesg
root@/tmp/lttng-build-root/bin # ./lttng enable-event -u -a
All UST events are enabled in channel channel0
root@/tmp/lttng-build-root/bin # ./lttng start
Tracing started for session app
root@/tmp/lttng-build-root/bin # cd ..
root@/tmp/lttng-build-root # cd ..
root@/tmp # LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
root@/tmp #
root@/tmp #
root@/tmp # cd lttng-build-root/
root@/tmp/lttng-build-root # cd bin/
root@/tmp/lttng-build-root/bin # ./lttng stop
Waiting for data availability
Tracing stopped for session app
root@/tmp/lttng-build-root/bin # dmesg
root@/tmp/lttng-build-root/bin # ./lttng destroy
Session app destroyed
root@/tmp/lttng-build-root/bin # cd /
root@/ # cd root/
root@~ # ls
lttng-traces
root@~ # cd lttng-traces/
root@~/lttng-traces # ls
root@~/lttng-traces #< no traces are generated>

in ps -ef

28664 root       0:00 ./lttng-sessiond -vvv --consumerd64-path=/tmp/lttng-build-root/lib/lttng/libexec/lttng-consumerd
29027 root       0:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error

Regards,
GMK.



-----Original Message-----
From: Thibault, Daniel [mailto:Daniel.Thibault@drdc-rddc.gc.ca] 
Sent: Friday, December 06, 2013 9:14 PM
To: Manikandan Govindaswamy; Jérémie Galarneau
Cc: lttng-dev@lists.lttng.org
Subject: RE: [lttng-dev] No trace files are generated for UST

> As I'm using the custom image on x86 machine, i've kept the lib in /tmp/lttng-build-root/lib folder and this sample able to see the library as well.
>
> And the same way I able to create the trace files for kernel but not for ust. There were no dmesg as well.
>
> [...]
> root@/tmp # ldd sample
>         linux-vdso.so.1 (0x00007fff2a135000)
>         libdl.so.2 => /lib64/libdl.so.2 (0x00007fe2cccb5000)
>         liblttng-ust.so.0 => /tmp/lttng-build-root/lib/liblttng-ust.so.0 (0x00007fe2cca51000)
>         [...]

   Try running sample using:

LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample

   If the LD_PRELOAD succeeds, we will know at least that liblttng-ust is reachable by sample.

   The modules have nothing to do with UST: kernel and user-space tracing are essentially independent features.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>

Re: No trace files are generated for UST

[Thibault, Daniel]

> Yes, with that LD_PRELOAD sample able to run.
>
> root@/tmp/lttng-build-root/bin # ./lttng create app
> Session app created.
> Traces will be written in /root/lttng-traces/app-20131206-174935
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng enable-event -u -a
> All UST events are enabled in channel channel0
> root@/tmp/lttng-build-root/bin # ./lttng start
> Tracing started for session app
> root@/tmp/lttng-build-root/bin # cd ..
> root@/tmp/lttng-build-root # cd ..
> root@/tmp # LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
> root@/tmp #
> root@/tmp #
> root@/tmp # cd lttng-build-root/
> root@/tmp/lttng-build-root # cd bin/
> root@/tmp/lttng-build-root/bin # ./lttng stop
> Waiting for data availability
> Tracing stopped for session app
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng destroy
> Session app destroyed
> root@/tmp/lttng-build-root/bin # cd /
> root@/ # cd root/
> root@~ # ls lttng-traces
> root@~ # cd lttng-traces/
> root@~/lttng-traces # ls
> root@~/lttng-traces #< no traces are generated>
>
> in ps -ef
>
> 28664 root       0:00 ./lttng-sessiond -vvv --consumerd64-path=/tmp/lttng-build-root/lib/lttng/libexec/lttng-consumerd
> 29027 root       0:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error

   So ./sample runs but you get no trace?  (Sound of head scratching)  Could you check 'readelf -h ./sample' and make sure the Class is ELF64?  (It's unlikely to be ELF32 but let's make sure anyway)

   I also thought that maybe sample being run as root is the problem, but a quick check on my system shows that cannot be the case (it just traces as ust/uid/0).

   Interesting.  I've managed to sort-of-reproduce the problem.  If I run (from a user account which is NOT a tracing group member):

$ sudo -H create a
$ sudo -H enable-event -u --all
$ sudo -H start
$ ./sample
$ sudo -H destroy

   This generates the expected trace in /root/lttng-traces/a-...

   But if I do this instead:

$ sudo create a
$ sudo enable-event -u --all
$ sudo start
$ ./sample
$ sudo destroy

   Then lttng correctly creates the trace folder in /home/username/lttng-traces/a-... when sample is run, but it remains empty.  No ust sub-folder, nothing.

   Something is definitely awry with paths and $HOME values here.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>

Re: No trace files are generated for UST

[Jérémie Galarneau]

Could you launch the session daemon in verbose mode and post the logs?

lttng-sessiond -vvv --verbose-consumer

On Fri, Dec 6, 2013 at 10:51 AM, Manikandan Govindaswamy
<magov@vestas.com> wrote:
> Yes, with that LD_PRELOAD sample able to run.
>
> root@/tmp/lttng-build-root/bin # ./lttng create app
> Session app created.
> Traces will be written in /root/lttng-traces/app-20131206-174935
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng enable-event -u -a
> All UST events are enabled in channel channel0
> root@/tmp/lttng-build-root/bin # ./lttng start
> Tracing started for session app
> root@/tmp/lttng-build-root/bin # cd ..
> root@/tmp/lttng-build-root # cd ..
> root@/tmp # LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
> root@/tmp #
> root@/tmp #
> root@/tmp # cd lttng-build-root/
> root@/tmp/lttng-build-root # cd bin/
> root@/tmp/lttng-build-root/bin # ./lttng stop
> Waiting for data availability
> Tracing stopped for session app
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng destroy
> Session app destroyed
> root@/tmp/lttng-build-root/bin # cd /
> root@/ # cd root/
> root@~ # ls
> lttng-traces
> root@~ # cd lttng-traces/
> root@~/lttng-traces # ls
> root@~/lttng-traces #< no traces are generated>
>
> in ps -ef
>
> 28664 root       0:00 ./lttng-sessiond -vvv --consumerd64-path=/tmp/lttng-build-root/lib/lttng/libexec/lttng-consumerd
> 29027 root       0:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error
>
> Regards,
> GMK.
>
>
>
> -----Original Message-----
> From: Thibault, Daniel [mailto:Daniel.Thibault@drdc-rddc.gc.ca]
> Sent: Friday, December 06, 2013 9:14 PM
> To: Manikandan Govindaswamy; Jérémie Galarneau
> Cc: lttng-dev@lists.lttng.org
> Subject: RE: [lttng-dev] No trace files are generated for UST
>
>> As I'm using the custom image on x86 machine, i've kept the lib in /tmp/lttng-build-root/lib folder and this sample able to see the library as well.
>>
>> And the same way I able to create the trace files for kernel but not for ust. There were no dmesg as well.
>>
>> [...]
>> root@/tmp # ldd sample
>>         linux-vdso.so.1 (0x00007fff2a135000)
>>         libdl.so.2 => /lib64/libdl.so.2 (0x00007fe2cccb5000)
>>         liblttng-ust.so.0 => /tmp/lttng-build-root/lib/liblttng-ust.so.0 (0x00007fe2cca51000)
>>         [...]
>
>    Try running sample using:
>
> LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
>
>    If the LD_PRELOAD succeeds, we will know at least that liblttng-ust is reachable by sample.
>
>    The modules have nothing to do with UST: kernel and user-space tracing are essentially independent features.
>
> Daniel U. Thibault
> Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
> Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
> R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
> 2459 route de la Bravoure
> Québec QC  G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

Re: No trace files are generated for UST

[Manikandan Govindaswamy]

[-- Attachment #1: Type: text/plain, Size: 3908 bytes --]

Please find the log for the --verbose-consumer attached.

Regards,
GMK.

-----Original Message-----
From: jeremie.galarneau@gmail.com [mailto:jeremie.galarneau@gmail.com] On Behalf Of Jérémie Galarneau
Sent: Friday, December 06, 2013 11:23 PM
To: Manikandan Govindaswamy
Cc: Thibault, Daniel; lttng-dev@lists.lttng.org
Subject: Re: [lttng-dev] No trace files are generated for UST

Could you launch the session daemon in verbose mode and post the logs?

lttng-sessiond -vvv --verbose-consumer

On Fri, Dec 6, 2013 at 10:51 AM, Manikandan Govindaswamy
<magov@vestas.com> wrote:
> Yes, with that LD_PRELOAD sample able to run.
>
> root@/tmp/lttng-build-root/bin # ./lttng create app
> Session app created.
> Traces will be written in /root/lttng-traces/app-20131206-174935
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng enable-event -u -a
> All UST events are enabled in channel channel0
> root@/tmp/lttng-build-root/bin # ./lttng start
> Tracing started for session app
> root@/tmp/lttng-build-root/bin # cd ..
> root@/tmp/lttng-build-root # cd ..
> root@/tmp # LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
> root@/tmp #
> root@/tmp #
> root@/tmp # cd lttng-build-root/
> root@/tmp/lttng-build-root # cd bin/
> root@/tmp/lttng-build-root/bin # ./lttng stop
> Waiting for data availability
> Tracing stopped for session app
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng destroy
> Session app destroyed
> root@/tmp/lttng-build-root/bin # cd /
> root@/ # cd root/
> root@~ # ls
> lttng-traces
> root@~ # cd lttng-traces/
> root@~/lttng-traces # ls
> root@~/lttng-traces #< no traces are generated>
>
> in ps -ef
>
> 28664 root       0:00 ./lttng-sessiond -vvv --consumerd64-path=/tmp/lttng-build-root/lib/lttng/libexec/lttng-consumerd
> 29027 root       0:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error
>
> Regards,
> GMK.
>
>
>
> -----Original Message-----
> From: Thibault, Daniel [mailto:Daniel.Thibault@drdc-rddc.gc.ca]
> Sent: Friday, December 06, 2013 9:14 PM
> To: Manikandan Govindaswamy; Jérémie Galarneau
> Cc: lttng-dev@lists.lttng.org
> Subject: RE: [lttng-dev] No trace files are generated for UST
>
>> As I'm using the custom image on x86 machine, i've kept the lib in /tmp/lttng-build-root/lib folder and this sample able to see the library as well.
>>
>> And the same way I able to create the trace files for kernel but not for ust. There were no dmesg as well.
>>
>> [...]
>> root@/tmp # ldd sample
>>         linux-vdso.so.1 (0x00007fff2a135000)
>>         libdl.so.2 => /lib64/libdl.so.2 (0x00007fe2cccb5000)
>>         liblttng-ust.so.0 => /tmp/lttng-build-root/lib/liblttng-ust.so.0 (0x00007fe2cca51000)
>>         [...]
>
>    Try running sample using:
>
> LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
>
>    If the LD_PRELOAD succeeds, we will know at least that liblttng-ust is reachable by sample.
>
>    The modules have nothing to do with UST: kernel and user-space tracing are essentially independent features.
>
> Daniel U. Thibault
> Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
> Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
> R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
> 2459 route de la Bravoure
> Québec QC  G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

[-- Attachment #2: session.log --]
[-- Type: application/octet-stream, Size: 37354 bytes --]

[-- Attachment #3: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: No trace files are generated for UST

[Manikandan Govindaswamy]

Hi !,

Thanks for your mail.

I'm also able to run the easy-ust and generate the traces successfully if I create and run from an user not as root user.

But  the kernel traces are generated fine for even root user but for ust it runs fine only with non-root user in my case.

And the behavior is vice versa if I try the kernel tracer from user

# lttng list -k
Error: Unable to list kernel events: Tracing the kernel requires a root lttng-sessiond daemon, as well as "tracing" group membership or root user ID for the lttng client.
Error: Command error

Will it be possible to run both kernel and ust to run and create the traces from root user itself? And if you could explain why ust expects the user?

Regards,
GMK

-----Original Message-----
From: jeremie.galarneau@gmail.com [mailto:jeremie.galarneau@gmail.com] On Behalf Of Jérémie Galarneau
Sent: Friday, December 06, 2013 11:23 PM
To: Manikandan Govindaswamy
Cc: Thibault, Daniel; lttng-dev@lists.lttng.org
Subject: Re: [lttng-dev] No trace files are generated for UST

Could you launch the session daemon in verbose mode and post the logs?

lttng-sessiond -vvv --verbose-consumer

On Fri, Dec 6, 2013 at 10:51 AM, Manikandan Govindaswamy
<magov@vestas.com> wrote:
> Yes, with that LD_PRELOAD sample able to run.
>
> root@/tmp/lttng-build-root/bin # ./lttng create app
> Session app created.
> Traces will be written in /root/lttng-traces/app-20131206-174935
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng enable-event -u -a
> All UST events are enabled in channel channel0
> root@/tmp/lttng-build-root/bin # ./lttng start
> Tracing started for session app
> root@/tmp/lttng-build-root/bin # cd ..
> root@/tmp/lttng-build-root # cd ..
> root@/tmp # LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
> root@/tmp #
> root@/tmp #
> root@/tmp # cd lttng-build-root/
> root@/tmp/lttng-build-root # cd bin/
> root@/tmp/lttng-build-root/bin # ./lttng stop
> Waiting for data availability
> Tracing stopped for session app
> root@/tmp/lttng-build-root/bin # dmesg
> root@/tmp/lttng-build-root/bin # ./lttng destroy
> Session app destroyed
> root@/tmp/lttng-build-root/bin # cd /
> root@/ # cd root/
> root@~ # ls
> lttng-traces
> root@~ # cd lttng-traces/
> root@~/lttng-traces # ls
> root@~/lttng-traces #< no traces are generated>
>
> in ps -ef
>
> 28664 root       0:00 ./lttng-sessiond -vvv --consumerd64-path=/tmp/lttng-build-root/lib/lttng/libexec/lttng-consumerd
> 29027 root       0:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error
>
> Regards,
> GMK.
>
>
>
> -----Original Message-----
> From: Thibault, Daniel [mailto:Daniel.Thibault@drdc-rddc.gc.ca]
> Sent: Friday, December 06, 2013 9:14 PM
> To: Manikandan Govindaswamy; Jérémie Galarneau
> Cc: lttng-dev@lists.lttng.org
> Subject: RE: [lttng-dev] No trace files are generated for UST
>
>> As I'm using the custom image on x86 machine, i've kept the lib in /tmp/lttng-build-root/lib folder and this sample able to see the library as well.
>>
>> And the same way I able to create the trace files for kernel but not for ust. There were no dmesg as well.
>>
>> [...]
>> root@/tmp # ldd sample
>>         linux-vdso.so.1 (0x00007fff2a135000)
>>         libdl.so.2 => /lib64/libdl.so.2 (0x00007fe2cccb5000)
>>         liblttng-ust.so.0 => /tmp/lttng-build-root/lib/liblttng-ust.so.0 (0x00007fe2cca51000)
>>         [...]
>
>    Try running sample using:
>
> LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
>
>    If the LD_PRELOAD succeeds, we will know at least that liblttng-ust is reachable by sample.
>
>    The modules have nothing to do with UST: kernel and user-space tracing are essentially independent features.
>
> Daniel U. Thibault
> Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
> Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
> R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
> 2459 route de la Bravoure
> Québec QC  G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

Re: No trace files are generated for UST

[Thibault, Daniel]

   A first glance through the log reveals these possibly significant entries:

DEBUG1 [15860/15996]: Sending response (size: 16, retcode: Unknown error code) (in thread_manage_clients() at main.c:3803)

And later

PERROR [16322/16322]: sendmsg: Broken pipe (in lttcomm_send_unix_sock() at unix.c:218)

   What were the client commands used to generate the log?  And what was the running environment (which versions of the LTTng suite were installed, and which session daemon(s) is(are) running? ).

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>

Re: No trace files are generated for UST

[Manikandan Govindaswamy]

The Lttng version installed is " Dominus Vobiscum" 

The lttng tools: 2.3.1
Lttng-ust 2.3.1
Lttng modules 2.3.4

And The session running is

28664 root       0:00 ./lttng-sessiond -vvv --consumerd64-path=/tmp/lttng-build-root/lib/lttng/libexec/lttng-consumerd
29027 root       0:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error


I'm also able to run the easy-ust and generate the traces successfully if I create and run from an user not as root user.

But  the kernel traces are generated fine for even root user but for ust it runs fine only with non-root user in my case.

And the behavior is vice versa if I try the kernel tracer from user

# lttng list -k
Error: Unable to list kernel events: Tracing the kernel requires a root lttng-sessiond daemon, as well as "tracing" group membership or root user ID for the lttng client.
Error: Command error

Will it be possible to run both kernel and ust to run and create the traces from root user itself? And if you could explain why ust expects the user?

Regards,
GMK

-----Original Message-----
From: Thibault, Daniel [mailto:Daniel.Thibault@drdc-rddc.gc.ca] 
Sent: Monday, December 09, 2013 7:24 PM
To: Manikandan Govindaswamy; Jérémie Galarneau
Cc: lttng-dev@lists.lttng.org
Subject: RE: [lttng-dev] No trace files are generated for UST

   A first glance through the log reveals these possibly significant entries:

DEBUG1 [15860/15996]: Sending response (size: 16, retcode: Unknown error code) (in thread_manage_clients() at main.c:3803)

And later

PERROR [16322/16322]: sendmsg: Broken pipe (in lttcomm_send_unix_sock() at unix.c:218)

   What were the client commands used to generate the log?  And what was the running environment (which versions of the LTTng suite were installed, and which session daemon(s) is(are) running? ).

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>

Re: No trace files are generated for UST

[David Goulet]

[-- Attachment #1.1: Type: text/plain, Size: 4760 bytes --]

On 07 Dec (06:16:14), Manikandan Govindaswamy wrote:
> Please find the log for the --verbose-consumer attached.

With the log file you provided, *no* application registered to the
session daemon meaning probably that lttng-ust is either not called or
not linked at all.

By adding this in front, you should have quite an amount of debug being
printed out on stderr. If not, you have a problem with maybe the
LD_PRELOAD path or lttng-ust linking else provide those logs :).

LTTNG_UST_DEBUG=1 ./sample

Cheers!
David

> 
> Regards,
> GMK.
> 
> -----Original Message-----
> From: jeremie.galarneau@gmail.com [mailto:jeremie.galarneau@gmail.com] On Behalf Of Jérémie Galarneau
> Sent: Friday, December 06, 2013 11:23 PM
> To: Manikandan Govindaswamy
> Cc: Thibault, Daniel; lttng-dev@lists.lttng.org
> Subject: Re: [lttng-dev] No trace files are generated for UST
> 
> Could you launch the session daemon in verbose mode and post the logs?
> 
> lttng-sessiond -vvv --verbose-consumer
> 
> On Fri, Dec 6, 2013 at 10:51 AM, Manikandan Govindaswamy
> <magov@vestas.com> wrote:
> > Yes, with that LD_PRELOAD sample able to run.
> >
> > root@/tmp/lttng-build-root/bin # ./lttng create app
> > Session app created.
> > Traces will be written in /root/lttng-traces/app-20131206-174935
> > root@/tmp/lttng-build-root/bin # dmesg
> > root@/tmp/lttng-build-root/bin # ./lttng enable-event -u -a
> > All UST events are enabled in channel channel0
> > root@/tmp/lttng-build-root/bin # ./lttng start
> > Tracing started for session app
> > root@/tmp/lttng-build-root/bin # cd ..
> > root@/tmp/lttng-build-root # cd ..
> > root@/tmp # LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
> > root@/tmp #
> > root@/tmp #
> > root@/tmp # cd lttng-build-root/
> > root@/tmp/lttng-build-root # cd bin/
> > root@/tmp/lttng-build-root/bin # ./lttng stop
> > Waiting for data availability
> > Tracing stopped for session app
> > root@/tmp/lttng-build-root/bin # dmesg
> > root@/tmp/lttng-build-root/bin # ./lttng destroy
> > Session app destroyed
> > root@/tmp/lttng-build-root/bin # cd /
> > root@/ # cd root/
> > root@~ # ls
> > lttng-traces
> > root@~ # cd lttng-traces/
> > root@~/lttng-traces # ls
> > root@~/lttng-traces #< no traces are generated>
> >
> > in ps -ef
> >
> > 28664 root       0:00 ./lttng-sessiond -vvv --consumerd64-path=/tmp/lttng-build-root/lib/lttng/libexec/lttng-consumerd
> > 29027 root       0:00 lttng-consumerd --quiet -u --consumerd-cmd-sock /var/run/lttng/ustconsumerd64/command --consumerd-err-sock /var/run/lttng/ustconsumerd64/error
> >
> > Regards,
> > GMK.
> >
> >
> >
> > -----Original Message-----
> > From: Thibault, Daniel [mailto:Daniel.Thibault@drdc-rddc.gc.ca]
> > Sent: Friday, December 06, 2013 9:14 PM
> > To: Manikandan Govindaswamy; Jérémie Galarneau
> > Cc: lttng-dev@lists.lttng.org
> > Subject: RE: [lttng-dev] No trace files are generated for UST
> >
> >> As I'm using the custom image on x86 machine, i've kept the lib in /tmp/lttng-build-root/lib folder and this sample able to see the library as well.
> >>
> >> And the same way I able to create the trace files for kernel but not for ust. There were no dmesg as well.
> >>
> >> [...]
> >> root@/tmp # ldd sample
> >>         linux-vdso.so.1 (0x00007fff2a135000)
> >>         libdl.so.2 => /lib64/libdl.so.2 (0x00007fe2cccb5000)
> >>         liblttng-ust.so.0 => /tmp/lttng-build-root/lib/liblttng-ust.so.0 (0x00007fe2cca51000)
> >>         [...]
> >
> >    Try running sample using:
> >
> > LD_PRELOAD=/tmp/lttng-build-root/lib/liblttng-ust.so.0 ./sample
> >
> >    If the LD_PRELOAD succeeds, we will know at least that liblttng-ust is reachable by sample.
> >
> >    The modules have nothing to do with UST: kernel and user-space tracing are essentially independent features.
> >
> > Daniel U. Thibault
> > Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
> > Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
> > R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
> > 2459 route de la Bravoure
> > Québec QC  G3J 1X5
> > CANADA
> > Vox : (418) 844-4000 x4245
> > Fax : (418) 844-4538
> > NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> > Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>
> 
> 
> 
> -- 
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com


> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 620 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: No trace files are generated for UST

[Thibault, Daniel]

>   Interesting.  I've managed to sort-of-reproduce the problem.  If I run (from a user account which is NOT a tracing group member):
> [...]
>   Then lttng correctly creates the trace folder in /home/username/lttng-traces/a-... when sample is run, but it remains empty.  No ust sub-folder, nothing.
>
>   Something is definitely awry with paths and $HOME values here.

   Scratch that, my mistake.  The traces were created just fine, but the trace folders were owned by root, so they appeared empty (in Nautilus) to username.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada <http://www.valcartier.drdc-rddc.gc.ca/>