From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeffrey Chen Subject: LTTng buffer mode Date: Wed, 17 Feb 2016 00:55:50 +0000 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1985613594==" Return-path: Received: from col004-omc3s18.hotmail.com ([65.55.34.157]) by ltt.polymtl.ca with esmtp (Exim 4.80) (envelope-from ) id 1aVqPP-00088V-Ea for lttng-dev@lists.lttng.org; Tue, 16 Feb 2016 19:56:03 -0500 Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lttng-dev-bounces@lists.lttng.org To: "lttng-dev@lists.lttng.org" List-Id: lttng-dev@lists.lttng.org --===============1985613594== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_BY2PR15MB0759C952513EFB6E18812650ABAE0BY2PR15MB0759namp_" --_000_BY2PR15MB0759C952513EFB6E18812650ABAE0BY2PR15MB0759namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi LTTng community: I have a question about LTTng buffer mode. According to documentation, LTTn= g supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have = a global buffer that does not have separate buffers? The reason I asked is = because, LTTng writes different buffer traces to different directory/file. = It is much more difficult for us to post-process separate directories. We w= ould like to have all traces write into one single buffer and directory/fil= e, so it is easier to do post-processing. Thanks. --_000_BY2PR15MB0759C952513EFB6E18812650ABAE0BY2PR15MB0759namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi LTTng community:

I have a question about LTTng buffer mode. According to documentation, L= TTng supports 2 buffer modes, per-pid, and per-uid. Is there any mode to ha= ve a global buffer that does not have separate buffers? The reason I asked = is because, LTTng writes different buffer traces to different directory/file. It is much more difficult for u= s to post-process separate directories. We would like to have all traces wr= ite into one single buffer and directory/file, so it is easier to do post-p= rocessing. Thanks.

--_000_BY2PR15MB0759C952513EFB6E18812650ABAE0BY2PR15MB0759namp_-- --===============1985613594== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============1985613594==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Philippe Proulx Subject: Re: LTTng buffer mode Date: Tue, 16 Feb 2016 20:02:56 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail-lb0-f180.google.com ([209.85.217.180]) by ltt.polymtl.ca with esmtp (Exim 4.80) (envelope-from ) id 1aVqWh-0008EM-UL for lttng-dev@lists.lttng.org; Tue, 16 Feb 2016 20:03:32 -0500 Received: by mail-lb0-f180.google.com with SMTP id x1so747780lbj.3 for ; Tue, 16 Feb 2016 17:03:31 -0800 (PST) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lttng-dev-bounces@lists.lttng.org To: Jeffrey Chen Cc: "lttng-dev@lists.lttng.org" List-Id: lttng-dev@lists.lttng.org No you cannot. There's no hidden global buffering scheme option for user space tracing. >From the LTTng documentation: > In the user space tracing domain, two buffering schemes are available > when creating a channel: > > Per-PID buffering: keep one ring buffer per process. > Per-UID buffering: keep one ring buffer for all processes of a single user. and: > The Linux kernel tracing domain only has one available buffering scheme > which is to use a single ring buffer for the whole system. Phil On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen wrote: > Hi LTTng community: > > > I have a question about LTTng buffer mode. According to documentation, LTTng > supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a > global buffer that does not have separate buffers? The reason I asked is > because, LTTng writes different buffer traces to different directory/file. > It is much more difficult for us to post-process separate directories. We > would like to have all traces write into one single buffer and > directory/file, so it is easier to do post-processing. Thanks. > > > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeffrey Chen Subject: Re: LTTng buffer mode Date: Fri, 19 Feb 2016 09:00:34 +0000 Message-ID: References: , Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from blu004-omc1s38.hotmail.com ([65.55.116.49]) by ltt.polymtl.ca with esmtp (Exim 4.80) (envelope-from ) id 1aWgvY-0004jK-QF for lttng-dev@lists.lttng.org; Fri, 19 Feb 2016 04:00:44 -0500 In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lttng-dev-bounces@lists.lttng.org To: Philippe Proulx Cc: "lttng-dev@lists.lttng.org" List-Id: lttng-dev@lists.lttng.org Thanks. If we make changes to LTTng to add this feature, would it violate the LTTng design principle in any way? Thanks. ________________________________________ From: Philippe Proulx Sent: Tuesday, February 16, 2016 5:02 PM To: Jeffrey Chen Cc: lttng-dev@lists.lttng.org Subject: Re: [lttng-dev] LTTng buffer mode No you cannot. There's no hidden global buffering scheme option for user space tracing. >From the LTTng documentation: > In the user space tracing domain, two buffering schemes are available > when creating a channel: > > Per-PID buffering: keep one ring buffer per process. > Per-UID buffering: keep one ring buffer for all processes of a single user. and: > The Linux kernel tracing domain only has one available buffering scheme > which is to use a single ring buffer for the whole system. Phil On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen wrote: > Hi LTTng community: > > > I have a question about LTTng buffer mode. According to documentation, LTTng > supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a > global buffer that does not have separate buffers? The reason I asked is > because, LTTng writes different buffer traces to different directory/file. > It is much more difficult for us to post-process separate directories. We > would like to have all traces write into one single buffer and > directory/file, so it is easier to do post-processing. Thanks. > > > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mathieu Desnoyers Subject: Re: LTTng buffer mode Date: Sat, 20 Feb 2016 18:17:03 +0000 (UTC) Message-ID: <829984174.3472.1455992223842.JavaMail.zimbra__26675.3577198417$1455992306$gmane$org@efficios.com> References:

Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail.efficios.com ([78.47.125.74]) by ltt.polymtl.ca with esmtp (Exim 4.80) (envelope-from ) id 1aXC5k-0008KD-Rx for lttng-dev@lists.lttng.org; Sat, 20 Feb 2016 13:17:18 -0500 In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lttng-dev-bounces@lists.lttng.org To: Jeffrey Chen Cc: lttng-dev List-Id: lttng-dev@lists.lttng.org Hi, LTTng implements per-cpu buffers to eliminate false-sharing between CPUs. Babeltrace and Trace Compass implement iterators that allow doing the merge of those per-cpu buffers into a single stream on the fly. Users of Babeltrace and Trace Compass should not have to worry about doing this merge. Since LTTng is inherently multi-buffers, we also use this buffer merging scheme to deal with tracefile rotation, multi-channel, and combining traces gathered from various processes/user IDs/kernel vs userspace. So my question here is: why you say it is much more difficult for you to deal with those multiple buffers ? Since there are already libraries in Babeltrace that do that for you, I would expect this to be a non-issue. Although it would be technically possible to implement a ring buffer shared across CPUs, it would be inefficient, and we would need a good justification for adding this complexity to the tracers. Thanks, Mathieu ----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote: > Thanks. > If we make changes to LTTng to add this feature, would it violate the LTTng > design principle in any way? Thanks. > > > ________________________________________ > From: Philippe Proulx > Sent: Tuesday, February 16, 2016 5:02 PM > To: Jeffrey Chen > Cc: lttng-dev@lists.lttng.org > Subject: Re: [lttng-dev] LTTng buffer mode > > No you cannot. > > There's no hidden global buffering scheme option for user space tracing. > > From the LTTng documentation: > >> In the user space tracing domain, two buffering schemes are available >> when creating a channel: >> >> Per-PID buffering: keep one ring buffer per process. >> Per-UID buffering: keep one ring buffer for all processes of a single user. > > and: > >> The Linux kernel tracing domain only has one available buffering scheme >> which is to use a single ring buffer for the whole system. > > Phil > > On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen wrote: >> Hi LTTng community: >> >> >> I have a question about LTTng buffer mode. According to documentation, LTTng >> supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a >> global buffer that does not have separate buffers? The reason I asked is >> because, LTTng writes different buffer traces to different directory/file. >> It is much more difficult for us to post-process separate directories. We >> would like to have all traces write into one single buffer and >> directory/file, so it is easier to do post-processing. Thanks. >> >> >> >> _______________________________________________ >> lttng-dev mailing list >> lttng-dev@lists.lttng.org >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com From mboxrd@z Thu Jan 1 00:00:00 1970 From: syed zaidi Subject: Re: LTTng buffer mode Date: Fri, 26 Feb 2016 01:12:54 -0800 Message-ID: References:

<829984174.3472.1455992223842.JavaMail.zimbra@efficios.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2050938611==" Return-path: Received: from mail-ig0-f173.google.com ([209.85.213.173]) by ltt.polymtl.ca with esmtp (Exim 4.80) (envelope-from ) id 1aZESK-0003gk-7h for lttng-dev@lists.lttng.org; Fri, 26 Feb 2016 04:13:01 -0500 Received: by mail-ig0-f173.google.com with SMTP id y8so31864681igp.1 for ; Fri, 26 Feb 2016 01:13:00 -0800 (PST) In-Reply-To: <829984174.3472.1455992223842.JavaMail.zimbra@efficios.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lttng-dev-bounces@lists.lttng.org To: Mathieu Desnoyers Cc: lttng-dev List-Id: lttng-dev@lists.lttng.org --===============2050938611== Content-Type: multipart/alternative; boundary=001a113319da4f20cc052ca8b429 --001a113319da4f20cc052ca8b429 Content-Type: text/plain; charset=UTF-8 Hi Mathieu, I have few questions related to the questions above and please forgive me for my ignorance in advance. 1. When LTTng creates the buffers lets say per uid, the files that show up in LTTng folder, are those files that LTTng has stopped writing in? What I mean is, are those files still being written to or they are safe to be moved. 2. If I want to collect traces lets say every 5 mins for the last 5 mins while LTTng keeps running. What is the best approach? Do I just run BabelTrace on the folder and remove all files from the folder to create space for next files and keep repeating? 3. I understand that I should delete the corresponding .idx file for each log file that I remove, what about metadata file? Do I remove it or leave it as is? 4. The trace file produced by Babeltrace, does it guarantee that the file has traces from all the users and the core till the time stamp of the last line?' For example if the last line of the trace file looks like this 23:00:00.912089772,...,... Does that mean that the trace file has all the traces until that time? Thanks Ammar On Sat, Feb 20, 2016 at 10:17 AM, Mathieu Desnoyers < mathieu.desnoyers@efficios.com> wrote: > Hi, > > LTTng implements per-cpu buffers to eliminate false-sharing between > CPUs. Babeltrace and Trace Compass implement iterators that allow > doing the merge of those per-cpu buffers into a single stream on > the fly. Users of Babeltrace and Trace Compass should not have to > worry about doing this merge. > > Since LTTng is inherently multi-buffers, we also use this buffer > merging scheme to deal with tracefile rotation, multi-channel, and > combining traces gathered from various processes/user IDs/kernel vs > userspace. > > So my question here is: why you say it is much more difficult for > you to deal with those multiple buffers ? Since there are already > libraries in Babeltrace that do that for you, I would expect this > to be a non-issue. > > Although it would be technically possible to implement a ring > buffer shared across CPUs, it would be inefficient, and we would > need a good justification for adding this complexity to the > tracers. > > Thanks, > > Mathieu > > > ----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote: > > > Thanks. > > If we make changes to LTTng to add this feature, would it violate the > LTTng > > design principle in any way? Thanks. > > > > > > ________________________________________ > > From: Philippe Proulx > > Sent: Tuesday, February 16, 2016 5:02 PM > > To: Jeffrey Chen > > Cc: lttng-dev@lists.lttng.org > > Subject: Re: [lttng-dev] LTTng buffer mode > > > > No you cannot. > > > > There's no hidden global buffering scheme option for user space tracing. > > > > From the LTTng documentation: > > > >> In the user space tracing domain, two buffering schemes are available > >> when creating a channel: > >> > >> Per-PID buffering: keep one ring buffer per process. > >> Per-UID buffering: keep one ring buffer for all processes of a single > user. > > > > and: > > > >> The Linux kernel tracing domain only has one available buffering scheme > >> which is to use a single ring buffer for the whole system. > > > > Phil > > > > On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen wrote: > >> Hi LTTng community: > >> > >> > >> I have a question about LTTng buffer mode. According to documentation, > LTTng > >> supports 2 buffer modes, per-pid, and per-uid. Is there any mode to > have a > >> global buffer that does not have separate buffers? The reason I asked is > >> because, LTTng writes different buffer traces to different > directory/file. > >> It is much more difficult for us to post-process separate directories. > We > >> would like to have all traces write into one single buffer and > >> directory/file, so it is easier to do post-processing. Thanks. > >> > >> > >> > >> _______________________________________________ > >> lttng-dev mailing list > >> lttng-dev@lists.lttng.org > >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > >> > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev@lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > --001a113319da4f20cc052ca8b429 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi Mathieu,

I have few questions relate= d to the questions above and please forgive me for my ignorance in advance.=

1. When LTTng creates the buffers lets say per uid, the files th= at show up in LTTng folder, are those files that LTTng has stopped writing = in? What I mean is, are those files still being written to or they are safe= to be moved.

2. If I want to collect traces lets = say every 5 mins for the last 5 mins while LTTng keeps running. What is the= best approach? Do I just run BabelTrace on the folder and remove all files= from the folder to create space for next files and keep repeating? =C2=A0<= /div>

3. I understand that I should delete the correspon= ding .idx file for each log file that I remove, what about metadata file? D= o I remove it or leave it as is?

4. The trace file= produced by Babeltrace, does it guarantee that the file has traces from al= l the users and the core till the time stamp of the last line?'

For example if the last line of the trace file looks like this

23:00:00.912089772,...,...

Does that mean that the trace file ha= s all the traces until that time?

Thanks

Ammar

On Sat, Feb 20, 2016 at 10:17 AM, Mathieu Desnoyers <= mathieu= .desnoyers@efficios.com> wrote:

Hi,

LTTng implements per-cpu buffers to eliminate false-sharing between
CPUs. Babeltrace and Trace Compass implement iterators that allow
doing the merge of those per-cpu buffers into a single stream on
the fly. Users of Babeltrace and Trace Compass should not have to
worry about doing this merge.

Since LTTng is inherently multi-buffers, we also use this buffer
merging scheme to deal with tracefile rotation, multi-channel, and
combining traces gathered from various processes/user IDs/kernel vs
userspace.

So my question here is: why you say it is much more difficult for
you to deal with those multiple buffers ? Since there are already
libraries in Babeltrace that do that for you, I would expect this
to be a non-issue.

Although it would be technically possible to implement a ring
buffer shared across CPUs, it would be inefficient, and we would
need a good justification for adding this complexity to the
tracers.

Thanks,

Mathieu

----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote:

> Thanks.
> If we make changes to LTTng to add this feature, would it violate the = LTTng
> design principle in any way? Thanks.
>
>
> ________________________________________
> From: Philippe Proulx <e= eppeliteloop@gmail.com>
> Sent: Tuesday, February 16, 2016 5:02 PM
> To: Jeffrey Chen
> Cc: lttng-dev@lists.lttng= .org
> Subject: Re: [lttng-dev] LTTng buffer mode
>
> No you cannot.
>
> There's no hidden global buffering scheme option for user space tr= acing.
>
> From the LTTng documentation:
>
>> In the user space tracing domain, two buffering schemes are availa= ble
>> when creating a channel:
>>
>> Per-PID buffering: keep one ring buffer per process.
>> Per-UID buffering: keep one ring buffer for all processes of a sin= gle user.
>
> and:
>
>> The Linux kernel tracing domain only has one available buffering s= cheme
>> which is to use a single ring buffer for the whole system.
>
> Phil
>
> On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen <cpthk@hotmail.com> wrote:
>> Hi LTTng community:
>>
>>
>> I have a question about LTTng buffer mode. According to documentat= ion, LTTng
>> supports 2 buffer modes, per-pid, and per-uid. Is there any mode t= o have a
>> global buffer that does not have separate buffers? The reason I as= ked is
>> because, LTTng writes different buffer traces to different directo= ry/file.
>> It is much more difficult for us to post-process separate director= ies. We
>> would like to have all traces write into one single buffer and
>> directory/file, so it is easier to do post-processing. Thanks.
>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng= .org
>> http://lists.lttng.org/cgi-bin/mai= lman/listinfo/lttng-dev
>>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org=
> http://lists.lttng.org/cgi-bin/mailman= /listinfo/lttng-dev

--
Mathieu Desnoyers
EfficiOS Inc.
ht= tp://www.efficios.com

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org<= br> http://lists.lttng.org/cgi-bin/mailman/li= stinfo/lttng-dev

--001a113319da4f20cc052ca8b429-- --===============2050938611== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============2050938611==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mathieu Desnoyers Subject: Re: LTTng buffer mode Date: Fri, 26 Feb 2016 18:07:05 +0000 (UTC) Message-ID: <1263166217.9071.1456510025603.JavaMail.zimbra__37301.7413694926$1456510096$gmane$org@efficios.com> References:

<829984174.3472.1455992223842.JavaMail.zimbra@efficios.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1195619572==" Return-path: Received: from mail.efficios.com ([78.47.125.74]) by ltt.polymtl.ca with esmtp (Exim 4.80) (envelope-from ) id 1aZMnL-0006j8-Kp for lttng-dev@lists.lttng.org; Fri, 26 Feb 2016 13:07:18 -0500 In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lttng-dev-bounces@lists.lttng.org To: syed zaidi Cc: lttng-dev List-Id: lttng-dev@lists.lttng.org --===============1195619572== Content-Type: multipart/alternative; boundary="----=_Part_9070_1269716780.1456510025602" ------=_Part_9070_1269716780.1456510025602 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit ----- On Feb 26, 2016, at 4:12 AM, syed zaidi wrote: > Hi Mathieu, > I have few questions related to the questions above and please forgive me for my > ignorance in advance. > 1. When LTTng creates the buffers lets say per uid, the files that show up in > LTTng folder, are those files that LTTng has stopped writing in? What I mean > is, are those files still being written to or they are safe to be moved. They are still being written to. > 2. If I want to collect traces lets say every 5 mins for the last 5 mins while > LTTng keeps running. What is the best approach? Do I just run BabelTrace on the > folder and remove all files from the folder to create space for next files and > keep repeating? I would not use Babeltrace on the target, but rather just move the CTF files. You could swap between two sessions to do that: lttng supports multiple concurrent sessions. Create and start a new session before you stop, destroy, and move away the old session, every 5 minutes... > 3. I understand that I should delete the corresponding .idx file for each log > file that I remove, what about metadata file? Do I remove it or leave it as is? You need to leave it as is. > 4. The trace file produced by Babeltrace, does it guarantee that the file has > traces from all the users and the core till the time stamp of the last line?' > For example if the last line of the trace file looks like this > 23:00:00.912089772,...,... > Does that mean that the trace file has all the traces until that time? It depends on your tracing mode (discard vs snapshot buffers). In discard mode (default if you don't use the --snapshot option on lttng create), Babeltrace will print on stderr whenever it encounters discarded events. Best regards, Mathieu > Thanks > Ammar > On Sat, Feb 20, 2016 at 10:17 AM, Mathieu Desnoyers < > mathieu.desnoyers@efficios.com > wrote: >> Hi, >> LTTng implements per-cpu buffers to eliminate false-sharing between >> CPUs. Babeltrace and Trace Compass implement iterators that allow >> doing the merge of those per-cpu buffers into a single stream on >> the fly. Users of Babeltrace and Trace Compass should not have to >> worry about doing this merge. >> Since LTTng is inherently multi-buffers, we also use this buffer >> merging scheme to deal with tracefile rotation, multi-channel, and >> combining traces gathered from various processes/user IDs/kernel vs >> userspace. >> So my question here is: why you say it is much more difficult for >> you to deal with those multiple buffers ? Since there are already >> libraries in Babeltrace that do that for you, I would expect this >> to be a non-issue. >> Although it would be technically possible to implement a ring >> buffer shared across CPUs, it would be inefficient, and we would >> need a good justification for adding this complexity to the >> tracers. >> Thanks, >> Mathieu >> ----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote: >> > Thanks. >> > If we make changes to LTTng to add this feature, would it violate the LTTng >> > design principle in any way? Thanks. >> > ________________________________________ >> > From: Philippe Proulx < eeppeliteloop@gmail.com > >> > Sent: Tuesday, February 16, 2016 5:02 PM >> > To: Jeffrey Chen >> > Cc: lttng-dev@lists.lttng.org >> > Subject: Re: [lttng-dev] LTTng buffer mode >> > No you cannot. >> > There's no hidden global buffering scheme option for user space tracing. >> > From the LTTng documentation: >> >> In the user space tracing domain, two buffering schemes are available >> >> when creating a channel: >> >> Per-PID buffering: keep one ring buffer per process. >> >> Per-UID buffering: keep one ring buffer for all processes of a single user. >> > and: >> >> The Linux kernel tracing domain only has one available buffering scheme >> >> which is to use a single ring buffer for the whole system. >> > Phil >> > On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen < cpthk@hotmail.com > wrote: >> >> Hi LTTng community: >> >> I have a question about LTTng buffer mode. According to documentation, LTTng >> >> supports 2 buffer modes, per-pid, and per-uid. Is there any mode to have a >> >> global buffer that does not have separate buffers? The reason I asked is >> >> because, LTTng writes different buffer traces to different directory/file. >> >> It is much more difficult for us to post-process separate directories. We >> >> would like to have all traces write into one single buffer and >> >> directory/file, so it is easier to do post-processing. Thanks. >> >> _______________________________________________ >> >> lttng-dev mailing list >> >> lttng-dev@lists.lttng.org >> >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> > _______________________________________________ >> > lttng-dev mailing list >> > lttng-dev@lists.lttng.org >> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> -- >> Mathieu Desnoyers >> EfficiOS Inc. >> http://www.efficios.com >> _______________________________________________ >> lttng-dev mailing list >> lttng-dev@lists.lttng.org >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ------=_Part_9070_1269716780.1456510025602 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

----- On Feb 26, 2016, at 4:12 AM, syed zaidi <zaidi1039@gmail.com&g= t; wrote:

Hi Mathieu,=
I have few questions related to the questions above and please for= give me for my ignorance in advance.
1. When LTTng creates the bu= ffers lets say per uid, the files that show up in LTTng folder, are those f= iles that LTTng has stopped writing in? What I mean is, are those files sti= ll being written to or they are safe to be moved.

<= div>

They are still being written to.

2. If I want to collec= t traces lets say every 5 mins for the last 5 mins while LTTng keeps runnin= g. What is the best approach? Do I just run BabelTrace on the folder and re= move all files from the folder to create space for next files and keep repe= ating?

I would not use B= abeltrace on the target, but rather just move the CTF files.

You could swap bet= ween two sessions to do that: lttng supports multiple

concurrent sessions. Create and start a new session before yo= u stop,

destroy, and move away the old s= ession, every 5 minutes...

3. I understand that I should delete the corres= ponding .idx file for each log file that I remove, what about metadata file= ? Do I remove it or leave it as is?

You need to leave it as is.

4. The trace file produced by Babeltrace,= does it guarantee that the file has traces from all the users and the core= till the time stamp of the last line?'
For example if the last l= ine of the trace file looks like this
23:00:00.912089772,...,...<= /div>
Does that mean that the trace file has all the traces until that = time?

It depends on your tracing mode (discard= vs snapshot buffers). In discard

mode (default if you don't = use the --snapshot option on lttng create),

<= div>Babeltrace will print on stderr whenever it encounters discarded events= .

Be= st regards,

Mathieu

Thanks
Ammar

On Sat, Feb 20, 2016 at 10:17 AM, Mathie= u Desnoyers <mathieu.desnoyers@efficios.com> wr= ote:
Hi,

LTTng implements per-cpu buffers to eliminate false-sharing between
CPUs. Babeltrace and Trace Compass implement iterators that allow
doing the merge of those per-cpu buffers into a single stream on
the fly. Users of Babeltrace and Trace Compass should not have to
worry about doing this merge.

Since LTTng is inherently multi-buffers, we also use this buffer
merging scheme to deal with tracefile rotation, multi-channel, and
combining traces gathered from various processes/user IDs/kernel vs
userspace.

So my question here is: why you say it is much more difficult for
you to deal with those multiple buffers ? Since there are already
libraries in Babeltrace that do that for you, I would expect this
to be a non-issue.

Although it would be technically possible to implement a ring
buffer shared across CPUs, it would be inefficient, and we would
need a good justification for adding this complexity to the
tracers.

Thanks,

Mathieu

----- On Feb 19, 2016, at 4:00 AM, Jeffrey Chen cpthk@hotmail.com wrote:

> Thanks.
> If we make changes to LTTng to add this feature, would it violate the = LTTng
> design principle in any way? Thanks.
>
>
> ________________________________________
> From: Philippe Proulx <eeppeliteloop@gmail.com>
> Sent: Tuesday, February 16, 2016 5:02 PM
> To: Jeffrey Chen
> Cc: ltt= ng-dev@lists.lttng.org
> Subject: Re: [lttng-dev] LTTng buffer mode
>
> No you cannot.
>
> There's no hidden global buffering scheme option for user space tracin= g.
>
> From the LTTng documentation:
>
>> In the user space tracing domain, two buffering schemes are availa= ble
>> when creating a channel:
>>
>> Per-PID buffering: keep one ring buffer per process.
>> Per-UID buffering: keep one ring buffer for all processes of a sin= gle user.
>
> and:
>
>> The Linux kernel tracing domain only has one available buffering s= cheme
>> which is to use a single ring buffer for the whole system.
>
> Phil
>
> On Tue, Feb 16, 2016 at 7:55 PM, Jeffrey Chen <cpthk@hotmail.com> wrote:
>> Hi LTTng community:
>>
>>
>> I have a question about LTTng buffer mode. According to documentat= ion, LTTng
>> supports 2 buffer modes, per-pid, and per-uid. Is there any mode t= o have a
>> global buffer that does not have separate buffers? The reason I as= ked is
>> because, LTTng writes different buffer traces to different directo= ry/file.
>> It is much more difficult for us to post-process separate director= ies. We
>> would like to have all traces write into one single buffer and
>> directory/file, so it is easier to do post-processing. Thanks.
>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> ltt= ng-dev@lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mai= lman/listinfo/lttng-dev
>>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-d= ev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman= /listinfo/lttng-dev

--
Mathieu Desnoyers
EfficiOS Inc.
ht= tp://www.efficios.com

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
=

Mathieu Desnoyers
EfficiOS= Inc.
http://www.efficios.com

------=_Part_9070_1269716780.1456510025602-- --===============1195619572== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============1195619572==--