* Re: LTTng buffer mode [not found] <BY2PR15MB0759C952513EFB6E18812650ABAE0@BY2PR15MB0759.namprd15.prod.outlook.com> @ 2016-02-17 1:02 ` Philippe Proulx [not found] ` <CAB4xu_1qu7ezjfafSHc_CmM=YzcNhu9Oxyim=kd7N2U-OTJwZw@mail.gmail.com> 1 sibling, 0 replies; 6+ messages in thread From: Philippe Proulx @ 2016-02-17 1:02 UTC (permalink / raw) To: Jeffrey Chen; +Cc: lttng-dev No you cannot. There's no hidden global buffering scheme option for user space tracing. From the LTTng documentation: > In the user space tracing domain, two buffering schemes are available > when creating a channel: > > Per-PID buffering: keep one ring buffer per process. > Per-UID buffering: keep one ring buffer for all processes of a single user. and: > The Linux kernel tracing domain only has one available buffering scheme > which is to use a single ring buffer for the whole system. Phil On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen <cpthk@hotmail.com> wrote: > Hi LTTng community: > > > I have a question about LTTng buffer mode. According to documentation, LTTng > supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a > global buffer that does not have separate buffers? The reason I asked is > because, LTTng writes different buffer traces to different directory/file. > It is much more difficult for us to post-process separate directories. We > would like to have all traces write into one single buffer and > directory/file, so it is easier to do post-processing. Thanks. > > > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CAB4xu_1qu7ezjfafSHc_CmM=YzcNhu9Oxyim=kd7N2U-OTJwZw@mail.gmail.com>]
* Re: LTTng buffer mode [not found] ` <CAB4xu_1qu7ezjfafSHc_CmM=YzcNhu9Oxyim=kd7N2U-OTJwZw@mail.gmail.com> @ 2016-02-19 9:00 ` Jeffrey Chen [not found] ` <BY2PR15MB07598F61A09D22D0CBF9B785ABA00@BY2PR15MB0759.namprd15.prod.outlook.com> 1 sibling, 0 replies; 6+ messages in thread From: Jeffrey Chen @ 2016-02-19 9:00 UTC (permalink / raw) To: Philippe Proulx; +Cc: lttng-dev Thanks. If we make changes to LTTng to add this feature, would it violate the LTTng design principle in any way? Thanks. ________________________________________ From: Philippe Proulx <eeppeliteloop@gmail.com> Sent: Tuesday, February 16, 2016 5:02 PM To: Jeffrey Chen Cc: lttng-dev@lists.lttng.org Subject: Re: [lttng-dev] LTTng buffer mode No you cannot. There's no hidden global buffering scheme option for user space tracing. From the LTTng documentation: > In the user space tracing domain, two buffering schemes are available > when creating a channel: > > Per-PID buffering: keep one ring buffer per process. > Per-UID buffering: keep one ring buffer for all processes of a single user. and: > The Linux kernel tracing domain only has one available buffering scheme > which is to use a single ring buffer for the whole system. Phil On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen <cpthk@hotmail.com> wrote: > Hi LTTng community: > > > I have a question about LTTng buffer mode. According to documentation, LTTng > supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a > global buffer that does not have separate buffers? The reason I asked is > because, LTTng writes different buffer traces to different directory/file. > It is much more difficult for us to post-process separate directories. We > would like to have all traces write into one single buffer and > directory/file, so it is easier to do post-processing. Thanks. > > > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <BY2PR15MB07598F61A09D22D0CBF9B785ABA00@BY2PR15MB0759.namprd15.prod.outlook.com>]
* Re: LTTng buffer mode [not found] ` <BY2PR15MB07598F61A09D22D0CBF9B785ABA00@BY2PR15MB0759.namprd15.prod.outlook.com> @ 2016-02-20 18:17 ` Mathieu Desnoyers [not found] ` <829984174.3472.1455992223842.JavaMail.zimbra@efficios.com> 1 sibling, 0 replies; 6+ messages in thread From: Mathieu Desnoyers @ 2016-02-20 18:17 UTC (permalink / raw) To: Jeffrey Chen; +Cc: lttng-dev Hi, LTTng implements per-cpu buffers to eliminate false-sharing between CPUs. Babeltrace and Trace Compass implement iterators that allow doing the merge of those per-cpu buffers into a single stream on the fly. Users of Babeltrace and Trace Compass should not have to worry about doing this merge. Since LTTng is inherently multi-buffers, we also use this buffer merging scheme to deal with tracefile rotation, multi-channel, and combining traces gathered from various processes/user IDs/kernel vs userspace. So my question here is: why you say it is much more difficult for you to deal with those multiple buffers ? Since there are already libraries in Babeltrace that do that for you, I would expect this to be a non-issue. Although it would be technically possible to implement a ring buffer shared across CPUs, it would be inefficient, and we would need a good justification for adding this complexity to the tracers. Thanks, Mathieu ----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote: > Thanks. > If we make changes to LTTng to add this feature, would it violate the LTTng > design principle in any way? Thanks. > > > ________________________________________ > From: Philippe Proulx <eeppeliteloop@gmail.com> > Sent: Tuesday, February 16, 2016 5:02 PM > To: Jeffrey Chen > Cc: lttng-dev@lists.lttng.org > Subject: Re: [lttng-dev] LTTng buffer mode > > No you cannot. > > There's no hidden global buffering scheme option for user space tracing. > > From the LTTng documentation: > >> In the user space tracing domain, two buffering schemes are available >> when creating a channel: >> >> Per-PID buffering: keep one ring buffer per process. >> Per-UID buffering: keep one ring buffer for all processes of a single user. > > and: > >> The Linux kernel tracing domain only has one available buffering scheme >> which is to use a single ring buffer for the whole system. > > Phil > > On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen <cpthk@hotmail.com> wrote: >> Hi LTTng community: >> >> >> I have a question about LTTng buffer mode. According to documentation, LTTng >> supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a >> global buffer that does not have separate buffers? The reason I asked is >> because, LTTng writes different buffer traces to different directory/file. >> It is much more difficult for us to post-process separate directories. We >> would like to have all traces write into one single buffer and >> directory/file, so it is easier to do post-processing. Thanks. >> >> >> >> _______________________________________________ >> lttng-dev mailing list >> lttng-dev@lists.lttng.org >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <829984174.3472.1455992223842.JavaMail.zimbra@efficios.com>]
* Re: LTTng buffer mode [not found] ` <829984174.3472.1455992223842.JavaMail.zimbra@efficios.com> @ 2016-02-26 9:12 ` syed zaidi [not found] ` <CAD6O9aryfG_Wh39620c2K_CVB8Oi0ZaVnQbYzGdRq1b1WMEDbQ@mail.gmail.com> 1 sibling, 0 replies; 6+ messages in thread From: syed zaidi @ 2016-02-26 9:12 UTC (permalink / raw) To: Mathieu Desnoyers; +Cc: lttng-dev [-- Attachment #1.1: Type: text/plain, Size: 4477 bytes --] Hi Mathieu, I have few questions related to the questions above and please forgive me for my ignorance in advance. 1. When LTTng creates the buffers lets say per uid, the files that show up in LTTng folder, are those files that LTTng has stopped writing in? What I mean is, are those files still being written to or they are safe to be moved. 2. If I want to collect traces lets say every 5 mins for the last 5 mins while LTTng keeps running. What is the best approach? Do I just run BabelTrace on the folder and remove all files from the folder to create space for next files and keep repeating? 3. I understand that I should delete the corresponding .idx file for each log file that I remove, what about metadata file? Do I remove it or leave it as is? 4. The trace file produced by Babeltrace, does it guarantee that the file has traces from all the users and the core till the time stamp of the last line?' For example if the last line of the trace file looks like this 23:00:00.912089772,...,... Does that mean that the trace file has all the traces until that time? Thanks Ammar On Sat, Feb 20, 2016 at 10:17 AM, Mathieu Desnoyers < mathieu.desnoyers@efficios.com> wrote: > Hi, > > LTTng implements per-cpu buffers to eliminate false-sharing between > CPUs. Babeltrace and Trace Compass implement iterators that allow > doing the merge of those per-cpu buffers into a single stream on > the fly. Users of Babeltrace and Trace Compass should not have to > worry about doing this merge. > > Since LTTng is inherently multi-buffers, we also use this buffer > merging scheme to deal with tracefile rotation, multi-channel, and > combining traces gathered from various processes/user IDs/kernel vs > userspace. > > So my question here is: why you say it is much more difficult for > you to deal with those multiple buffers ? Since there are already > libraries in Babeltrace that do that for you, I would expect this > to be a non-issue. > > Although it would be technically possible to implement a ring > buffer shared across CPUs, it would be inefficient, and we would > need a good justification for adding this complexity to the > tracers. > > Thanks, > > Mathieu > > > ----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote: > > > Thanks. > > If we make changes to LTTng to add this feature, would it violate the > LTTng > > design principle in any way? Thanks. > > > > > > ________________________________________ > > From: Philippe Proulx <eeppeliteloop@gmail.com> > > Sent: Tuesday, February 16, 2016 5:02 PM > > To: Jeffrey Chen > > Cc: lttng-dev@lists.lttng.org > > Subject: Re: [lttng-dev] LTTng buffer mode > > > > No you cannot. > > > > There's no hidden global buffering scheme option for user space tracing. > > > > From the LTTng documentation: > > > >> In the user space tracing domain, two buffering schemes are available > >> when creating a channel: > >> > >> Per-PID buffering: keep one ring buffer per process. > >> Per-UID buffering: keep one ring buffer for all processes of a single > user. > > > > and: > > > >> The Linux kernel tracing domain only has one available buffering scheme > >> which is to use a single ring buffer for the whole system. > > > > Phil > > > > On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen <cpthk@hotmail.com> wrote: > >> Hi LTTng community: > >> > >> > >> I have a question about LTTng buffer mode. According to documentation, > LTTng > >> supports 2 buffer modes, per-pid, and per-uid. Is there any mode to > have a > >> global buffer that does not have separate buffers? The reason I asked is > >> because, LTTng writes different buffer traces to different > directory/file. > >> It is much more difficult for us to post-process separate directories. > We > >> would like to have all traces write into one single buffer and > >> directory/file, so it is easier to do post-processing. Thanks. > >> > >> > >> > >> _______________________________________________ > >> lttng-dev mailing list > >> lttng-dev@lists.lttng.org > >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > >> > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev@lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > [-- Attachment #1.2: Type: text/html, Size: 6346 bytes --] [-- Attachment #2: Type: text/plain, Size: 155 bytes --] _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CAD6O9aryfG_Wh39620c2K_CVB8Oi0ZaVnQbYzGdRq1b1WMEDbQ@mail.gmail.com>]
* Re: LTTng buffer mode [not found] ` <CAD6O9aryfG_Wh39620c2K_CVB8Oi0ZaVnQbYzGdRq1b1WMEDbQ@mail.gmail.com> @ 2016-02-26 18:07 ` Mathieu Desnoyers 0 siblings, 0 replies; 6+ messages in thread From: Mathieu Desnoyers @ 2016-02-26 18:07 UTC (permalink / raw) To: syed zaidi; +Cc: lttng-dev [-- Attachment #1.1: Type: text/plain, Size: 5240 bytes --] ----- On Feb 26, 2016, at 4:12 AM, syed zaidi <zaidi1039@gmail.com> wrote: > Hi Mathieu, > I have few questions related to the questions above and please forgive me for my > ignorance in advance. > 1. When LTTng creates the buffers lets say per uid, the files that show up in > LTTng folder, are those files that LTTng has stopped writing in? What I mean > is, are those files still being written to or they are safe to be moved. They are still being written to. > 2. If I want to collect traces lets say every 5 mins for the last 5 mins while > LTTng keeps running. What is the best approach? Do I just run BabelTrace on the > folder and remove all files from the folder to create space for next files and > keep repeating? I would not use Babeltrace on the target, but rather just move the CTF files. You could swap between two sessions to do that: lttng supports multiple concurrent sessions. Create and start a new session before you stop, destroy, and move away the old session, every 5 minutes... > 3. I understand that I should delete the corresponding .idx file for each log > file that I remove, what about metadata file? Do I remove it or leave it as is? You need to leave it as is. > 4. The trace file produced by Babeltrace, does it guarantee that the file has > traces from all the users and the core till the time stamp of the last line?' > For example if the last line of the trace file looks like this > 23:00:00.912089772,...,... > Does that mean that the trace file has all the traces until that time? It depends on your tracing mode (discard vs snapshot buffers). In discard mode (default if you don't use the --snapshot option on lttng create), Babeltrace will print on stderr whenever it encounters discarded events. Best regards, Mathieu > Thanks > Ammar > On Sat, Feb 20, 2016 at 10:17 AM, Mathieu Desnoyers < > mathieu.desnoyers@efficios.com > wrote: >> Hi, >> LTTng implements per-cpu buffers to eliminate false-sharing between >> CPUs. Babeltrace and Trace Compass implement iterators that allow >> doing the merge of those per-cpu buffers into a single stream on >> the fly. Users of Babeltrace and Trace Compass should not have to >> worry about doing this merge. >> Since LTTng is inherently multi-buffers, we also use this buffer >> merging scheme to deal with tracefile rotation, multi-channel, and >> combining traces gathered from various processes/user IDs/kernel vs >> userspace. >> So my question here is: why you say it is much more difficult for >> you to deal with those multiple buffers ? Since there are already >> libraries in Babeltrace that do that for you, I would expect this >> to be a non-issue. >> Although it would be technically possible to implement a ring >> buffer shared across CPUs, it would be inefficient, and we would >> need a good justification for adding this complexity to the >> tracers. >> Thanks, >> Mathieu >> ----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote: >> > Thanks. >> > If we make changes to LTTng to add this feature, would it violate the LTTng >> > design principle in any way? Thanks. >> > ________________________________________ >> > From: Philippe Proulx < eeppeliteloop@gmail.com > >> > Sent: Tuesday, February 16, 2016 5:02 PM >> > To: Jeffrey Chen >> > Cc: lttng-dev@lists.lttng.org >> > Subject: Re: [lttng-dev] LTTng buffer mode >> > No you cannot. >> > There's no hidden global buffering scheme option for user space tracing. >> > From the LTTng documentation: >> >> In the user space tracing domain, two buffering schemes are available >> >> when creating a channel: >> >> Per-PID buffering: keep one ring buffer per process. >> >> Per-UID buffering: keep one ring buffer for all processes of a single user. >> > and: >> >> The Linux kernel tracing domain only has one available buffering scheme >> >> which is to use a single ring buffer for the whole system. >> > Phil >> > On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen < cpthk@hotmail.com > wrote: >> >> Hi LTTng community: >> >> I have a question about LTTng buffer mode. According to documentation, LTTng >> >> supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a >> >> global buffer that does not have separate buffers? The reason I asked is >> >> because, LTTng writes different buffer traces to different directory/file. >> >> It is much more difficult for us to post-process separate directories. We >> >> would like to have all traces write into one single buffer and >> >> directory/file, so it is easier to do post-processing. Thanks. >> >> _______________________________________________ >> >> lttng-dev mailing list >> >> lttng-dev@lists.lttng.org >> >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> > _______________________________________________ >> > lttng-dev mailing list >> > lttng-dev@lists.lttng.org >> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> -- >> Mathieu Desnoyers >> EfficiOS Inc. >> http://www.efficios.com >> _______________________________________________ >> lttng-dev mailing list >> lttng-dev@lists.lttng.org >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com [-- Attachment #1.2: Type: text/html, Size: 9254 bytes --] [-- Attachment #2: Type: text/plain, Size: 155 bytes --] _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev ^ permalink raw reply [flat|nested] 6+ messages in thread
* LTTng buffer mode @ 2016-02-17 0:55 Jeffrey Chen 0 siblings, 0 replies; 6+ messages in thread From: Jeffrey Chen @ 2016-02-17 0:55 UTC (permalink / raw) To: lttng-dev [-- Attachment #1.1: Type: text/plain, Size: 528 bytes --] Hi LTTng community: I have a question about LTTng buffer mode. According to documentation, LTTng supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a global buffer that does not have separate buffers? The reason I asked is because, LTTng writes different buffer traces to different directory/file. It is much more difficult for us to post-process separate directories. We would like to have all traces write into one single buffer and directory/file, so it is easier to do post-processing. Thanks. [-- Attachment #1.2: Type: text/html, Size: 953 bytes --] [-- Attachment #2: Type: text/plain, Size: 155 bytes --] _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-02-26 18:07 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <BY2PR15MB0759C952513EFB6E18812650ABAE0@BY2PR15MB0759.namprd15.prod.outlook.com>
2016-02-17 1:02 ` LTTng buffer mode Philippe Proulx
[not found] ` <CAB4xu_1qu7ezjfafSHc_CmM=YzcNhu9Oxyim=kd7N2U-OTJwZw@mail.gmail.com>
2016-02-19 9:00 ` Jeffrey Chen
[not found] ` <BY2PR15MB07598F61A09D22D0CBF9B785ABA00@BY2PR15MB0759.namprd15.prod.outlook.com>
2016-02-20 18:17 ` Mathieu Desnoyers
[not found] ` <829984174.3472.1455992223842.JavaMail.zimbra@efficios.com>
2016-02-26 9:12 ` syed zaidi
[not found] ` <CAD6O9aryfG_Wh39620c2K_CVB8Oi0ZaVnQbYzGdRq1b1WMEDbQ@mail.gmail.com>
2016-02-26 18:07 ` Mathieu Desnoyers
2016-02-17 0:55 Jeffrey Chen
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.