Question about filtering bytecode

Question about filtering bytecode

[Amit Margalit]

[-- Attachment #1.1: Type: text/plain, Size: 613 bytes --]

Hi,

I'm considering whether I can write a wrapper for the libbabeltrace ctf 
iterator which provides a filtered "next()" method.

I've started reading the code in the hopes of understanding how to do 
this, and yet, I'd really appreciate a link to some ordered documentation, 
or - any other kind of explanation.

For example, one thing I still don't understand is the exact sequence, and 
where the bytecode is actually executed? What is the meaning and purpose 
of "linking" the filter to an event.

Thanks,

Amit Margalit
IBM XIV - Storage Reinvented
XIV-NAS Development Team
Tel. 03-689-7774
Fax. 03-689-7230

[-- Attachment #1.2: Type: text/html, Size: 1165 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Question about filtering bytecode

[Amit Margalit]

[-- Attachment #1.1: Type: text/plain, Size: 6256 bytes --]

Thanks for the explanations.

I currently feel that the effort to get such a thing working is not going 
to materialize in the timeframe required for my current work, and 
therefore, regrettably, I will have to work around this somehow. I am 
currently thinking of using a combination of Python and Cython to generate 
a wrapper for the babeltrace iterator. We'll see.

Thanks,

Amit Margalit
IBM XIV - Storage Reinvented
XIV-NAS Development Team
Tel. 03-689-7774
Fax. 03-689-7230



From:   Jérémie Galarneau <jeremie.galarneau@efficios.com>
To:     Amit Margalit/Israel/IBM@IBMIL
Cc:     lttng-dev@lists.lttng.org
Date:   07/21/2013 10:07 PM
Subject:        Re: [lttng-dev] Question about filtering bytecode
Sent by:        jeremie.galarneau@gmail.com



On Sun, Jul 21, 2013 at 1:51 AM, Amit Margalit <AMITM@il.ibm.com> wrote:
> Hi,
>
> I was thinking more about a programmer's guide to using this mechanism.
>

Unfortunately there is no programmer's guide on filters, but feel free
to ask about specifics.

> What I'd like to do is to have Babeltrace filter events it is showing me
> from an existing trace.
>
> Consider the following example:
>
> There are 3 types of tracepoints being captured (without filtering):
>
> A debug log message (occasionally) which includes a "debug_level" value
> (1..10 with 1 being highest importance).
> Function entry (via gcc -finstrument-functions or other 
instrumentation).
> Function exit (similarly).
>
> During a 1-hour period, I have 2-3 GB of trace data captured.
> The trace includes events from 4 different processes.
>
>
> If I use Babeltrace to read the trace I will get an enormously big 
result in
> text (~30-40GB at least).
>
> I would like to be able to do something like this:
>
> $ babeltrace --filter '$ctx.procname == "smbd" || debug_level < 3'
> /local/lttng-traces/ust/uid/2140/64-bit
>
> This would then let me see any events from the "smbd" process (since i 
added
> procname to the context using lttng add-context) or debug messages from 
all
> processes where the debug level is 1 or 2.
>

That would be a neat feature! There are currently no plans to support
filtering in Babeltrace, but that contribution would be welcome.
lttng-ust's filtering code is licensed under LGPL v2.1 so it could be
library-fied and used by the ctf-text plug-in. In the meantime, I
guess you could work around this limitation with a combination of AWK
and shell scripting...

> Regarding linking - I am referring to the function
> lttng_enabler_event_link_bytecode(), called from
> lttng-ust/liblttng-ust/lttng-events.c around line 628.
>

Thanks for clarifying!

Keep in mind that the filter's byte code is generated by the lttng
binary without knowing the events' declarations. Thus, the byte code
must go through a linking stage not completely unlike what is done by
a regular linker. One of the objectives of the linking stage is
setting the appropriate offsets to load an event's fields. The byte
code is also "specialized" by resolving each field's type and making
sure the appropriate implementation of each operator is used. For
instance, the FILTER_OP_EQ (check for equality) op-code is not
implemented the same way for integers, doubles and strings (see
lttng_filter_specialize_bytecode in lttng-filter-specialize.c).

You may want to have a look at _lttng_filter_event_link_bytecode
(lttng-ust/liblttng-ust/lttng-filter.c, line 318) to see how each step
of the linking process is implemented.

Regards,
Jérémie

> Thanks,
>
> Amit Margalit
> IBM XIV - Storage Reinvented
> XIV-NAS Development Team
> Tel. 03-689-7774
> Fax. 03-689-7230
>
>
>
> From:        Jérémie Galarneau <jeremie.galarneau@efficios.com>
> To:        Amit Margalit/Israel/IBM@IBMIL
> Cc:        lttng-dev@lists.lttng.org
> Date:        07/18/2013 07:10 PM
> Subject:        Re: [lttng-dev] Question about filtering bytecode
> Sent by:        jeremie.galarneau@gmail.com
> ________________________________
>
>
>
> Hi Amit,
>
> You may find information on the Babeltrace API in babeltrace/doc/API.txt
> The filtering mechanism used by the user space tracer is unrelated to
> Babeltrace.
>
> As for the sequence of events, this is how I understand it.
>
> When invoking the "lttng" binary with --enable-event and a filter
> option, a byte code is generated from the filtering rules
> (lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:704). This byte code is
> then handed off to the session-daemon.
>
> Then, when an instrumented application is launched and connects to the
> session-daemon, it receives the byte code while its events are being
> enabled. To answer your question directly, that byte code will be
> executed by the tracee when the event is "triggered"
> (lttng-ust/liblttng-ust/lttng-filter-interpreter.c:175).
>
> I'm not sure I understand your question about filter/event linking. A
> filter has to be linked to an event since its rules (expressions) are
> applied on that event's values. Could you clarify if that's not what
> you meant?
>
> Regards,
> Jérémie
>
> On Thu, Jul 18, 2013 at 3:12 AM, Amit Margalit <AMITM@il.ibm.com> wrote:
>> Hi,
>>
>> I'm considering whether I can write a wrapper for the libbabeltrace ctf
>> iterator which provides a filtered "next()" method.
>>
>> I've started reading the code in the hopes of understanding how to do
>> this,
>> and yet, I'd really appreciate a link to some ordered documentation, or 
-
>> any other kind of explanation.
>>
>> For example, one thing I still don't understand is the exact sequence, 
and
>> where the bytecode is actually executed? What is the meaning and 
purpose
>> of
>> "linking" the filter to an event.
>>
>> Thanks,
>>
>> Amit Margalit
>> IBM XIV - Storage Reinvented
>> XIV-NAS Development Team
>> Tel. 03-689-7774
>> Fax. 03-689-7230
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>
>
>
> --
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com
>
>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com



[-- Attachment #1.2: Type: text/html, Size: 9122 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Question about filtering bytecode

[Jérémie Galarneau]

On Sun, Jul 21, 2013 at 1:51 AM, Amit Margalit <AMITM@il.ibm.com> wrote:
> Hi,
>
> I was thinking more about a programmer's guide to using this mechanism.
>

Unfortunately there is no programmer's guide on filters, but feel free
to ask about specifics.

> What I'd like to do is to have Babeltrace filter events it is showing me
> from an existing trace.
>
> Consider the following example:
>
> There are 3 types of tracepoints being captured (without filtering):
>
> A debug log message (occasionally) which includes a "debug_level" value
> (1..10 with 1 being highest importance).
> Function entry (via gcc -finstrument-functions or other instrumentation).
> Function exit (similarly).
>
> During a 1-hour period, I have 2-3 GB of trace data captured.
> The trace includes events from 4 different processes.
>
>
> If I use Babeltrace to read the trace I will get an enormously big result in
> text (~30-40GB at least).
>
> I would like to be able to do something like this:
>
> $ babeltrace --filter '$ctx.procname == "smbd" || debug_level < 3'
> /local/lttng-traces/ust/uid/2140/64-bit
>
> This would then let me see any events from the "smbd" process (since i added
> procname to the context using lttng add-context) or debug messages from all
> processes where the debug level is 1 or 2.
>

That would be a neat feature! There are currently no plans to support
filtering in Babeltrace, but that contribution would be welcome.
lttng-ust's filtering code is licensed under LGPL v2.1 so it could be
library-fied and used by the ctf-text plug-in. In the meantime, I
guess you could work around this limitation with a combination of AWK
and shell scripting...

> Regarding linking - I am referring to the function
> lttng_enabler_event_link_bytecode(), called from
> lttng-ust/liblttng-ust/lttng-events.c around line 628.
>

Thanks for clarifying!

Keep in mind that the filter's byte code is generated by the lttng
binary without knowing the events' declarations. Thus, the byte code
must go through a linking stage not completely unlike what is done by
a regular linker. One of the objectives of the linking stage is
setting the appropriate offsets to load an event's fields. The byte
code is also "specialized" by resolving each field's type and making
sure the appropriate implementation of each operator is used. For
instance, the FILTER_OP_EQ (check for equality) op-code is not
implemented the same way for integers, doubles and strings (see
lttng_filter_specialize_bytecode in lttng-filter-specialize.c).

You may want to have a look at _lttng_filter_event_link_bytecode
(lttng-ust/liblttng-ust/lttng-filter.c, line 318) to see how each step
of the linking process is implemented.

Regards,
Jérémie

> Thanks,
>
> Amit Margalit
> IBM XIV - Storage Reinvented
> XIV-NAS Development Team
> Tel. 03-689-7774
> Fax. 03-689-7230
>
>
>
> From:        Jérémie Galarneau <jeremie.galarneau@efficios.com>
> To:        Amit Margalit/Israel/IBM@IBMIL
> Cc:        lttng-dev@lists.lttng.org
> Date:        07/18/2013 07:10 PM
> Subject:        Re: [lttng-dev] Question about filtering bytecode
> Sent by:        jeremie.galarneau@gmail.com
> ________________________________
>
>
>
> Hi Amit,
>
> You may find information on the Babeltrace API in babeltrace/doc/API.txt
> The filtering mechanism used by the user space tracer is unrelated to
> Babeltrace.
>
> As for the sequence of events, this is how I understand it.
>
> When invoking the "lttng" binary with --enable-event and a filter
> option, a byte code is generated from the filtering rules
> (lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:704). This byte code is
> then handed off to the session-daemon.
>
> Then, when an instrumented application is launched and connects to the
> session-daemon, it receives the byte code while its events are being
> enabled. To answer your question directly, that byte code will be
> executed by the tracee when the event is "triggered"
> (lttng-ust/liblttng-ust/lttng-filter-interpreter.c:175).
>
> I'm not sure I understand your question about filter/event linking. A
> filter has to be linked to an event since its rules (expressions) are
> applied on that event's values. Could you clarify if that's not what
> you meant?
>
> Regards,
> Jérémie
>
> On Thu, Jul 18, 2013 at 3:12 AM, Amit Margalit <AMITM@il.ibm.com> wrote:
>> Hi,
>>
>> I'm considering whether I can write a wrapper for the libbabeltrace ctf
>> iterator which provides a filtered "next()" method.
>>
>> I've started reading the code in the hopes of understanding how to do
>> this,
>> and yet, I'd really appreciate a link to some ordered documentation, or -
>> any other kind of explanation.
>>
>> For example, one thing I still don't understand is the exact sequence, and
>> where the bytecode is actually executed? What is the meaning and purpose
>> of
>> "linking" the filter to an event.
>>
>> Thanks,
>>
>> Amit Margalit
>> IBM XIV - Storage Reinvented
>> XIV-NAS Development Team
>> Tel. 03-689-7774
>> Fax. 03-689-7230
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>
>
>
> --
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com
>
>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

Re: Question about filtering bytecode

[Amit Margalit]

[-- Attachment #1.1: Type: text/plain, Size: 3702 bytes --]

Hi,

I was thinking more about a programmer's guide to using this mechanism.

What I'd like to do is to have Babeltrace filter events it is showing me 
from an existing trace.

Consider the following example:
There are 3 types of tracepoints being captured (without filtering):
A debug log message (occasionally) which includes a "debug_level" value 
(1..10 with 1 being highest importance).
Function entry (via gcc -finstrument-functions or other instrumentation).
Function exit (similarly).
During a 1-hour period, I have 2-3 GB of trace data captured.
The trace includes events from 4 different processes.

If I use Babeltrace to read the trace I will get an enormously big result 
in text (~30-40GB at least).

I would like to be able to do something like this:

$ babeltrace --filter '$ctx.procname == "smbd" || debug_level < 3' 
/local/lttng-traces/ust/uid/2140/64-bit

This would then let me see any events from the "smbd" process (since i 
added procname to the context using lttng add-context) or debug messages 
from all processes where the debug level is 1 or 2.

Regarding linking - I am referring to the function 
lttng_enabler_event_link_bytecode(), called from 
lttng-ust/liblttng-ust/lttng-events.c around line 628.

Thanks,

Amit Margalit
IBM XIV - Storage Reinvented
XIV-NAS Development Team
Tel. 03-689-7774
Fax. 03-689-7230



From:   Jérémie Galarneau <jeremie.galarneau@efficios.com>
To:     Amit Margalit/Israel/IBM@IBMIL
Cc:     lttng-dev@lists.lttng.org
Date:   07/18/2013 07:10 PM
Subject:        Re: [lttng-dev] Question about filtering bytecode
Sent by:        jeremie.galarneau@gmail.com



Hi Amit,

You may find information on the Babeltrace API in babeltrace/doc/API.txt
The filtering mechanism used by the user space tracer is unrelated to
Babeltrace.

As for the sequence of events, this is how I understand it.

When invoking the "lttng" binary with --enable-event and a filter
option, a byte code is generated from the filtering rules
(lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:704). This byte code is
then handed off to the session-daemon.

Then, when an instrumented application is launched and connects to the
session-daemon, it receives the byte code while its events are being
enabled. To answer your question directly, that byte code will be
executed by the tracee when the event is "triggered"
(lttng-ust/liblttng-ust/lttng-filter-interpreter.c:175).

I'm not sure I understand your question about filter/event linking. A
filter has to be linked to an event since its rules (expressions) are
applied on that event's values. Could you clarify if that's not what
you meant?

Regards,
Jérémie

On Thu, Jul 18, 2013 at 3:12 AM, Amit Margalit <AMITM@il.ibm.com> wrote:
> Hi,
>
> I'm considering whether I can write a wrapper for the libbabeltrace ctf
> iterator which provides a filtered "next()" method.
>
> I've started reading the code in the hopes of understanding how to do 
this,
> and yet, I'd really appreciate a link to some ordered documentation, or 
-
> any other kind of explanation.
>
> For example, one thing I still don't understand is the exact sequence, 
and
> where the bytecode is actually executed? What is the meaning and purpose 
of
> "linking" the filter to an event.
>
> Thanks,
>
> Amit Margalit
> IBM XIV - Storage Reinvented
> XIV-NAS Development Team
> Tel. 03-689-7774
> Fax. 03-689-7230
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com



[-- Attachment #1.2: Type: text/html, Size: 6321 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Question about filtering bytecode

[Jérémie Galarneau]

Hi Amit,

You may find information on the Babeltrace API in babeltrace/doc/API.txt
The filtering mechanism used by the user space tracer is unrelated to
Babeltrace.

As for the sequence of events, this is how I understand it.

When invoking the "lttng" binary with --enable-event and a filter
option, a byte code is generated from the filtering rules
(lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:704). This byte code is
then handed off to the session-daemon.

Then, when an instrumented application is launched and connects to the
session-daemon, it receives the byte code while its events are being
enabled. To answer your question directly, that byte code will be
executed by the tracee when the event is "triggered"
(lttng-ust/liblttng-ust/lttng-filter-interpreter.c:175).

I'm not sure I understand your question about filter/event linking. A
filter has to be linked to an event since its rules (expressions) are
applied on that event's values. Could you clarify if that's not what
you meant?

Regards,
Jérémie

On Thu, Jul 18, 2013 at 3:12 AM, Amit Margalit <AMITM@il.ibm.com> wrote:
> Hi,
>
> I'm considering whether I can write a wrapper for the libbabeltrace ctf
> iterator which provides a filtered "next()" method.
>
> I've started reading the code in the hopes of understanding how to do this,
> and yet, I'd really appreciate a link to some ordered documentation, or -
> any other kind of explanation.
>
> For example, one thing I still don't understand is the exact sequence, and
> where the bytecode is actually executed? What is the meaning and purpose of
> "linking" the filter to an event.
>
> Thanks,
>
> Amit Margalit
> IBM XIV - Storage Reinvented
> XIV-NAS Development Team
> Tel. 03-689-7774
> Fax. 03-689-7230
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com