Hi Maksim,

It's hard to give general advice without knowing more about your constraints.

I can see three main approaches that are easy to reach:
  1. Store the traces in each container and fetch them,
  2. Trace to a shared volume (make sure sessions are stopped or rotated before you read them),
  3. Or stream traces to a "collector" container (perhaps the one doing the analyses) using network streaming

If you want to continuously consume traces, I would suggest you look into "trace rotations". You could setup periodic trace rotations (based on size or time) and ship traces to your processor node as they become available.

Although my demo wasn't running in a container, the basic idea is what I presented at FOSDEM in 2019:

https://archive.fosdem.org/2019/schedule/event/lttng/

I hope that helps a bit,
Jérémie

--
Jérémie Galarneau
EfficiOS Inc.
https://www.efficios.com
From: lttng-dev <lttng-dev-bounces@lists.lttng.org> on behalf of Maksim Khmelevskiy via lttng-dev <lttng-dev@lists.lttng.org>
Sent: October 10, 2022 12:12
To: lttng-dev@lists.lttng.org <lttng-dev@lists.lttng.org>
Subject: [lttng-dev] LTTng and containers.
 
Hi,

I would like to ask regarding the hot topic - container tracing. I've seen a youtube video, have read a message from LTTng mailing list and tried to google more about this topic but didn't find much of info. Could you please direct me where should I continue digging?
My problem: 
I would like to have multiple containers where traces are generated by apps with compiled-in tracepoints. Traces could be stored in these containers as well.
Besides these containers I would have a trace processor container, a master container which could address a container(or a trace session) and fetch traces from it and read with babeltrace or similar tool.
So far, intuitively, remote tracing comes to my mind but before continuing with the task I would be happy to hear an advice from LTTng devs.

Thank you!