From mboxrd@z Thu Jan 1 00:00:00 1970 From: mrx Subject: Re: A question on protocols Date: Tue, 20 Nov 2018 19:23:31 +0100 Message-ID: References: <20181116171557.GA23758@joraj-alpa> <20181119184641.GA8739@joraj-alpa> <20181120171639.GA12088@joraj-alpa> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8094448347426044779==" Return-path: Received: from mail-it1-x12f.google.com (mail-it1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) by lists.lttng.org (Postfix) with ESMTPS id 42zvGm4xVVzr1L for ; Tue, 20 Nov 2018 13:23:44 -0500 (EST) Received: by mail-it1-x12f.google.com with SMTP id a205-v6so4817459itd.4 for ; Tue, 20 Nov 2018 10:23:44 -0800 (PST) In-Reply-To: <20181120171639.GA12088@joraj-alpa> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lttng-dev-bounces@lists.lttng.org Sender: "lttng-dev" To: jonathan.rajotte-julien@efficios.com Cc: lttng-dev@lists.lttng.org List-Id: lttng-dev@lists.lttng.org --===============8094448347426044779== Content-Type: multipart/alternative; boundary="000000000000c4dcd1057b1cbbe3" --000000000000c4dcd1057b1cbbe3 Content-Type: text/plain; charset="UTF-8" On Tue, Nov 20, 2018 at 6:16 PM Jonathan Rajotte-Julien < jonathan.rajotte-julien@efficios.com> wrote: > Hi Patrick, > > > Sure, so given this I cannot use relayd for our purposes. The only other > > option would be to set LTTng to local tracing, which we don't have flash > > for. My hope was that we could somehow stream the CTF records somewhere > > else, over that HTTP proxy, ot be analyzed. As you explained earlier, the > > relayd can be viewed as a FTP server of sorts. So that will most likely > not > > work. > > A possibility would be to spare some RAM (not easy I know), mount a tmpfs, > configure the session to output to the tmpfs, use session rotation based on > size, send the data, remove the data, rinse and repeat. Now the question > is what > is the amount of tracing data generated and the amount of RAM available. > > > We have RAM for the tracing buffer, no issues there. The tracing buffer > is > > just a temporary medium until you get the trace on disc, possibly via > > relayd. Correct? > > Yes. > > The consumerd is responsible for fetching the data from the buffers, then > it > output it locally or send it over the network to lttng-relayd. > > You can take a look at the interaction here: > > https://lttng.org/docs/v2.10/#doc-plumbing Is the contents of the RAM buffer already proper CTF records ready to be consumed, or is there some transformation that needs to take place as well? If so, can i find that in some library or is it part of the consumerd code base? If i were to read the ring buffer instead of consumerd, then i would perhaps be in a situation where I could stream the ring buffer to another machine over that HTTP proxy? That way I wouldn't need any RAM disc or anything, the data is already stored. The consumerd logic that reads the ring buffer, is that part of some library i could take advantage of? > > > > > > I interpretted your RAM disc suggestion as a place where I could store > the > > trace instead of using flash. > > This was the correct interpretation. > > > > > Perhaps i could have been more clear on the situation as well. > > > > We have LTTng tracing capabilites on our devices. During development we > can > > use LTTng on the device and relayd on our dev machines and everything > works > > great. > > Glad to hear it. > We're very pleased with LTTng. > > > > > In production, the situation is different. > > > > The only means we have of transporting anything from the device is a HTTP > > proxy. We're trying to find a way where we can keep a conitnuous LTTng > > trace on the device, tunnel it through the HTTP proxy and analyze the > data > > remotely. > > > > The devices are in the embedded realm, limited amounts of RAM, CPU, and > > flash. This limits what we can do on the device when it comes to storage > > for example. > > > > Hopefully I've made our situation a bit more clear. > > > > Based on our discussion so far, I have to say that LTTng is starting to > > look less and less as a viable way forward for us. > > It would be quite disappointing if you would not be able to reuse the work > done > for instrumenting your applications. > Quite so, which is why we're trying so hard to find a viable way forward. Regards, Patrik --000000000000c4dcd1057b1cbbe3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue= , Nov 20, 2018 at 6:16 PM Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com= > wrote:
Hi Patrick,

> Sure, so given this I cannot use relayd for our purposes. The only oth= er
> option would be to set LTTng to local tracing, which we don't have= flash
> for. My hope was that we could somehow stream the CTF records somewher= e
> else, over that HTTP proxy, ot be analyzed. As you explained earlier, = the
> relayd can be viewed as a FTP server of sorts. So that will most likel= y not
> work.

A possibility would be to spare some RAM (not easy I know), mount a tmpfs,<= br> configure the session to output to the tmpfs, use session rotation based on=
size, send the data, remove the data, rinse and repeat. Now the question is= what
is the amount of tracing data generated and the amount of RAM available.
> We have RAM for the tracing buffer, no issues there. The tracing buffe= r is
> just a temporary medium until you get the trace on disc, possibly via<= br> > relayd. Correct?

Yes.

The consumerd is responsible for fetching the data from the buffers, then i= t
output it locally or send it over the network to lttng-relayd.

You can take a look at the interaction here:

https://lttng.org/docs/v2.10/#doc-plumbing

Is the contents of the RAM buffer already proper CTF r= ecords ready to be consumed, or is there some transformation that needs to = take place as well? If so, can i find that in some library or is it part of= the consumerd code base?

If i were to read the ri= ng buffer instead of consumerd, then i would perhaps be in a situation wher= e I could stream the ring buffer to another machine over that HTTP proxy? T= hat way I wouldn't need any RAM disc or anything, the data is already s= tored.

The consumerd logic that reads the ring= buffer, is that part of some library i could take advantage of?
<= div>=C2=A0


>
> I interpretted your RAM disc suggestion as a place where I could store= the
> trace instead of using flash.

This was the correct interpretation.

>
> Perhaps i could have been more clear on the situation as well.
>
> We have LTTng tracing capabilites on our devices. During development w= e can
> use LTTng on the device and relayd on our dev machines and everything = works
> great.

Glad to hear it.

We're very pleased= with LTTng.
=C2=A0

>
> In production, the situation is different.
>
> The only means we have of transporting anything from the device is a H= TTP
> proxy. We're trying to find a way where we can keep a conitnuous L= TTng
> trace on the device, tunnel it through the HTTP proxy and analyze the = data
> remotely.
>
> The devices are in the embedded realm, limited amounts of RAM, CPU, an= d
> flash. This limits what we can do on the device when it comes to stora= ge
> for example.
>
> Hopefully I've made our situation a bit more clear.
>
> Based on our discussion so far, I have to say that LTTng is starting t= o
> look less and less as a viable way forward for us.

It would be quite disappointing if you would not be able to reuse the work = done
for instrumenting your applications.

Qu= ite so, which is why we're trying so hard to find a viable way forward.=

Regards,
Patrik
--000000000000c4dcd1057b1cbbe3-- --===============8094448347426044779== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============8094448347426044779==--