events injection / merging traces

events injection / merging traces

[Carlo Caione]

Hi,
is there an easy way to "inject" new events in a LLTng trace? (you can 
also read that as: is there an easy way to merge two LLTng traces?).

I see that I can use [python] babeltrace.TraceCollection() + add_trace() 
to merge multiple traces, but it seems that I cannot output the 
resultant merged trace again as CTF.

Any hint how to do that if possible at all?

Thank you,

--
Carlo Caione

Re: events injection / merging traces

[Jonathan Rajotte-Julien]

Hi Carlo,

On Wed, Jul 03, 2019 at 01:30:58PM +0100, Carlo Caione wrote:
> Hi,
> is there an easy way to "inject" new events in a LLTng trace? (you can also
> read that as: is there an easy way to merge two LLTng traces?).

Injecting would be possible using a plugin (C or python) in babeltrace 2.

> 
> I see that I can use [python] babeltrace.TraceCollection() + add_trace() to
> merge multiple traces, but it seems that I cannot output the resultant
> merged trace again as CTF.

Babelrace 2 does have a default plugin for outputting to CTF but the result
might not be the one you expect. If you have two CTF traces as sources, two CTF
traces will be outputted by the current CTF sink. This is done for multiple
reasons.

The major one is that there is multiple places where clashes (stream class,
event) can happen (IDs etc.). The easy solution would be to generate new IDs but
it might lead to confusion as to why the IDs of an input trace are not found in
a resulting merge of traces or at least not the "expected" one.

There is a special case where all input traces are chunk of the same tracing
session (lttng-tools 2.11 rotation feature) since we have a guaranteed to have
no clash.

People who are more familiar with CTF,please do not restrain yourself on correcting me
here.

From what I understand, not much would prevent someone that knows the tradeoff
to implement such merging plugin. It simply does not exists today.

It also depends on what you mean by a single CTF trace.

This leads to, as always, whats is the end goal here?

Cheers
-- 
Jonathan Rajotte-Julien
EfficiOS

Re: events injection / merging traces

[Carlo Caione]

On 04/07/2019 15:52, Jonathan Rajotte-Julien wrote:
> Hi Carlo,

Hi Jonathan,
thank you for your reply, quite enlightening.

> This leads to, as always, whats is the end goal here?

This is the context: I'm tracing a Linux kernel (using LTTng) while in 
parallel I'm tracing a process running in a bare-metal OS (running on 
the same machine but on different cores using a partitioning hypervisor).

The binary trace data obtained from the bare-metal OS is converted to 
CTF using babeltrace in a format resembling LTTng (to be analyzed with 
TraceCompass).

The goal here is merge the two traces to have a global picture of what's 
going on in the system.

--
Carlo Caione

Re: events injection / merging traces

[Jonathan Rajotte-Julien]

Hi Carlo,

> 
> > This leads to, as always, whats is the end goal here?
> 
> This is the context: I'm tracing a Linux kernel (using LTTng) while in
> parallel I'm tracing a process running in a bare-metal OS (running on the
> same machine but on different cores using a partitioning hypervisor).

Nice.

> 
> The binary trace data obtained from the bare-metal OS is converted to CTF
> using babeltrace in a format resembling LTTng (to be analyzed with
> TraceCompass).

In both case, babeltrace and tracecompass based analysis should leave the trace
merging to the parser/decoder/muxer. There should be no need to merge them in a
single CTF trace.

The thing that is important is to correlate the clock from both traces to allow
the tools to do a proper merge. Then you consume the events and build state etc.
How you correlate those clocks depends on the tracing scenario.

I would recommend you speak to the Tracecompass devs [1] on how they handle multiple
"kernel" traces (linux + converted baremeta) in "experiment" [2] and how clock
correlation works [3] on their side. This is if you plan to mostly use
Tracecompass for analysis. If you plan to use babeltrace, we can discuss this
more as needed.

[1] https://www.eclipse.org/tracecompass/#community
[2] https://archive.eclipse.org/tracecompass/doc/stable/org.eclipse.tracecompass.doc.user/Trace-Compass-Main-Features.html#Creating_an_Experiment
[3] https://archive.eclipse.org/tracecompass/doc/stable/org.eclipse.tracecompass.doc.user/Time-offsetting.html#Time_offsetting

-- 
Jonathan Rajotte-Julien
EfficiOS