Re: [tracecompass-dev] Scalability

Re: [tracecompass-dev] Scalability

[Jonathan Rajotte-Julien]

Hi Doug,

On Tue, Mar 12, 2019 at 02:56:26PM +0000, Doug Schaefer wrote:
> Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.
> 
> My biggest worry is on the read side. Babeltrace blew up on that file. It's a pretty simple trace (for now) with a single event class with 4 ints and a sequence of 32-bit unsigned ints which is usually only 2 elements long.

This is not I expect. Still eithet jgalar or eep might have a more insight on
this. CCing the lttng-dev mailing list.

Is there any way to share us a similar trace? Either with a generator or we can
provide a link for you to upload it. The current limit on bugs.lttng.org is a
bit too small for such trace.

> 
> Aside from that, am very pleased with the how easy CTF is to work with. Looking forward to doing more.
> 
> Doug.
> 
> On Tue, 2019-03-12 at 14:41 +0000, Matthew Khouzam wrote:
> 
> Hi Doug,
> 
> 
> Great to hear you're coming to a standard! I don't know if trace compass will scale properly as I don't know what the trace configuration is.
> 
> 
> > I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.
> 
> 
> 
> My suggestion is to have 1 MB packets. This makes seeking very efficient. If each event is 32 bytes, basically flush every 25k events or so.
> 
> 
> Please keep us posted!
> 
> 
> Matthew.
> 
> ________________________________
> From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> on behalf of Doug Schaefer <dschaefer@blackberry.com>
> Sent: Tuesday, March 12, 2019 10:30:15 AM
> To: tracecompass-dev@eclipse.org
> Subject: [tracecompass-dev] Scalability
> 
> Hey gang,
> 
> We're finally starting to look at converting our custom traces into CTF so we can leverage tools like TraceCompass and, of course, contribute to it. One thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.
> 
> I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.
> 
> But when I use babeltrace to print it out, I ran out of virtual memory. I then hand coded a reader and simply adding the trace to the context caused the memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 18.04 distro), tries to inflate the events into it's internal representation for the entire trace. I need to do more investigation to confirm that.
> 
> So my question for this list, would TraceCompass do better? Does it have it's own parsing libraries?
> 
> Thanks,
> Doug

> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@eclipse.org
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/tracecompass-dev


-- 
Jonathan Rajotte-Julien
EfficiOS

Re: [tracecompass-dev] Scalability

[Matthew Khouzam]

[-- Attachment #1.1: Type: text/plain, Size: 4339 bytes --]

Welcome Jonathan,


Also, for people seeking great babeltrace support, I feel I should plug the lttng-dev mailinglist.


I hope the URL is not censored. https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev if it is, just search for "lttng-dev -- LTTng development list"

lttng-dev Info Page<https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev>
lists.lttng.org
To see the collection of prior postings to the list, visit the lttng-dev Archives.. Using lttng-dev: To post a message to all the list members, send email to lttng-dev@lists.lttng.org. You can subscribe to the list, or change your existing subscription, in the sections below.



________________________________
From: Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com>
Sent: Tuesday, March 12, 2019 11:03:14 AM
To: Doug Schaefer; tracecompass developer discussions
Cc: Matthew Khouzam; lttng-dev
Subject: Re: [tracecompass-dev] Scalability

Hi Doug,

On Tue, Mar 12, 2019 at 02:56:26PM +0000, Doug Schaefer wrote:
> Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.
>
> My biggest worry is on the read side. Babeltrace blew up on that file. It's a pretty simple trace (for now) with a single event class with 4 ints and a sequence of 32-bit unsigned ints which is usually only 2 elements long.

This is not I expect. Still eithet jgalar or eep might have a more insight on
this. CCing the lttng-dev mailing list.

Is there any way to share us a similar trace? Either with a generator or we can
provide a link for you to upload it. The current limit on bugs.lttng.org is a
bit too small for such trace.

>
> Aside from that, am very pleased with the how easy CTF is to work with. Looking forward to doing more.
>
> Doug.
>
> On Tue, 2019-03-12 at 14:41 +0000, Matthew Khouzam wrote:
>
> Hi Doug,
>
>
> Great to hear you're coming to a standard! I don't know if trace compass will scale properly as I don't know what the trace configuration is.
>
>
> > I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.
>
>
>
> My suggestion is to have 1 MB packets. This makes seeking very efficient. If each event is 32 bytes, basically flush every 25k events or so.
>
>
> Please keep us posted!
>
>
> Matthew.
>
> ________________________________
> From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> on behalf of Doug Schaefer <dschaefer@blackberry.com>
> Sent: Tuesday, March 12, 2019 10:30:15 AM
> To: tracecompass-dev@eclipse.org
> Subject: [tracecompass-dev] Scalability
>
> Hey gang,
>
> We're finally starting to look at converting our custom traces into CTF so we can leverage tools like TraceCompass and, of course, contribute to it. One thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.
>
> I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.
>
> But when I use babeltrace to print it out, I ran out of virtual memory. I then hand coded a reader and simply adding the trace to the context caused the memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 18.04 distro), tries to inflate the events into it's internal representation for the entire trace. I need to do more investigation to confirm that.
>
> So my question for this list, would TraceCompass do better? Does it have it's own parsing libraries?
>
> Thanks,
> Doug

> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@eclipse.org
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/tracecompass-dev


--
Jonathan Rajotte-Julien
EfficiOS

[-- Attachment #1.2: Type: text/html, Size: 7881 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Doug Schaefer]

[-- Attachment #1.1: Type: text/plain, Size: 4574 bytes --]

It should be easy to generate a random one with the class I mentioned below and fill it with 16 million events. BTW, two of the int fields are 5 and 10 bits respectively, but I'm not sure that matters (or at least it shouldn't).

I'll also take a look at the babeltrace code and see what I can see.

BTW, my focus on libbabeltrace is to allow for a full range of tooling for our users using the language of their choice. The python binding is particularly interesting. As we move forward into the new world of IDEs, I can see a node.js binding being interesting as well. And it may even make sense to use it in Java tooling like TraceCompass using JNA. That's the vision at least, but first, baby steps ☺.

Doug.

On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,


On Tue, Mar 12, 2019 at 02:56:26PM +0000, Doug Schaefer wrote:

Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.


My biggest worry is on the read side. Babeltrace blew up on that file. It's a pretty simple trace (for now) with a single event class with 4 ints and a sequence of 32-bit unsigned ints which is usually only 2 elements long.


This is not I expect. Still eithet jgalar or eep might have a more insight on

this. CCing the lttng-dev mailing list.


Is there any way to share us a similar trace? Either with a generator or we can

provide a link for you to upload it. The current limit on bugs.lttng.org is a

bit too small for such trace.



Aside from that, am very pleased with the how easy CTF is to work with. Looking forward to doing more.


Doug.


On Tue, 2019-03-12 at 14:41 +0000, Matthew Khouzam wrote:


Hi Doug,



Great to hear you're coming to a standard! I don't know if trace compass will scale properly as I don't know what the trace configuration is.



I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.




My suggestion is to have 1 MB packets. This makes seeking very efficient. If each event is 32 bytes, basically flush every 25k events or so.



Please keep us posted!



Matthew.


________________________________

From: tracecompass-dev-bounces@eclipse.org<mailto:tracecompass-dev-bounces@eclipse.org> <tracecompass-dev-bounces@eclipse.org<mailto:tracecompass-dev-bounces@eclipse.org>> on behalf of Doug Schaefer <dschaefer@blackberry.com<mailto:dschaefer@blackberry.com>>

Sent: Tuesday, March 12, 2019 10:30:15 AM

To: tracecompass-dev@eclipse.org<mailto:tracecompass-dev@eclipse.org>

Subject: [tracecompass-dev] Scalability


Hey gang,


We're finally starting to look at converting our custom traces into CTF so we can leverage tools like TraceCompass and, of course, contribute to it. One thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.


I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.


But when I use babeltrace to print it out, I ran out of virtual memory. I then hand coded a reader and simply adding the trace to the context caused the memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 18.04 distro), tries to inflate the events into it's internal representation for the entire trace. I need to do more investigation to confirm that.


So my question for this list, would TraceCompass do better? Does it have it's own parsing libraries?


Thanks,

Doug


_______________________________________________

tracecompass-dev mailing list

tracecompass-dev@eclipse.org<mailto:tracecompass-dev@eclipse.org>

To change your delivery options, retrieve your password, or unsubscribe from this list, visit

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eclipse.org_mailman_listinfo_tracecompass-2Ddev&d=DwIBAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=ZoCx61VE_sHGa6j3DXehbqjX5P1NGEuDtEFaGORCh9k&s=qi9q1zvDaC9cJCci0y123O-j66M643YwxRJccJCzg_c&e=




[-- Attachment #1.2: Type: text/html, Size: 6760 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Jonathan Rajotte-Julien]

On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:
> It should be easy to generate a random one with the class I mentioned below
> and fill it with 16 million events.

Not if the problem is in the generator code ;).

> BTW, two of the int fields are 5 and 10
> bits respectively, but I'm not sure that matters (or at least it shouldn't).

Make sure to check the endian-ness of those fields.

What class? Not sure I see any reference to any.

Since you are generating CTF the error can be in the generator code. Please
provide either a trace reproducing the issue or a base reproducer. It will be
much easier overall than us trying to reproduce it blindly.

Cheers.

Re: [tracecompass-dev] Scalability

[Doug Schaefer]

[-- Attachment #1.1: Type: text/plain, Size: 1104 bytes --]

In the thread on the tracecompass list, I also mentioned that the babeltrace utility itself ran out of memory when reading the file. That's actually my biggest worry. I got around the generator issue by flushing every 100K events.

I'll create a quick generator that shows the same issue and post it in a bit.

Doug.

On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:

On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:

It should be easy to generate a random one with the class I mentioned below

and fill it with 16 million events.


Not if the problem is in the generator code ;).


BTW, two of the int fields are 5 and 10

bits respectively, but I'm not sure that matters (or at least it shouldn't).


Make sure to check the endian-ness of those fields.


What class? Not sure I see any reference to any.


Since you are generating CTF the error can be in the generator code. Please

provide either a trace reproducing the issue or a base reproducer. It will be

much easier overall than us trying to reproduce it blindly.


Cheers.


[-- Attachment #1.2: Type: text/html, Size: 1975 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Bernd Hufmann]

[-- Attachment #1.1: Type: text/plain, Size: 6532 bytes --]

Hi Doug

Great to see your interest this area.

>> BTW, my focus on libbabeltrace is to allow for a full range of tooling for our users using the language of their choice. The python binding is particularly interesting. As we move forward into the new world of IDEs, I can see a node.js binding being interesting as well. And it may even make sense to use it in Java tooling like TraceCompass using JNA. That's the vision at least, but first, baby steps ☺.

For the integration with Trace Compass in Eclipse, we chose to implement a CTF parser in Java to avoid using JNI/JNA to libbabeltrace. This was for performance reasons over JNI/JNA interface. We had implemented that at the beginning when Trace Compass supported LTTng version 0.x (prior CTF) and the performance was not sufficient enough.

For the integration in next generation IDEs, we are working on defining a Trace Server Protocol (TSP) as an interface between a client and server for exchanging of trace date for visualization purposes. It’s a similar idea as the LSP (Language Server Protocol) or DAP (Debug Server protocol). The TSP interface is on a higher level than what a trace reading interface like the CTF trace reader in Java or over your suggested JNA interface over libbabeltrace would provide.

We have some promising results for a Typescript front-end based on the next generation IDE Theia that uses the TSP to a trace server using Trace Compass core features as backend. We can already visualize various Trace Compass views. It’s still preliminary, but we could demo it if you’re interested.

BR
Bernd


From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> On Behalf Of Doug Schaefer
Sent: March-12-19 11:23 AM
To: tracecompass-dev@eclipse.org; jonathan.rajotte-julien@efficios.com
Cc: lttng-dev@lists.lttng.org
Subject: Re: [tracecompass-dev] Scalability

It should be easy to generate a random one with the class I mentioned below and fill it with 16 million events. BTW, two of the int fields are 5 and 10 bits respectively, but I'm not sure that matters (or at least it shouldn't).

I'll also take a look at the babeltrace code and see what I can see.

BTW, my focus on libbabeltrace is to allow for a full range of tooling for our users using the language of their choice. The python binding is particularly interesting. As we move forward into the new world of IDEs, I can see a node.js binding being interesting as well. And it may even make sense to use it in Java tooling like TraceCompass using JNA. That's the vision at least, but first, baby steps ☺.

Doug.

On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,



On Tue, Mar 12, 2019 at 02:56:26PM +0000, Doug Schaefer wrote:

Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.



My biggest worry is on the read side. Babeltrace blew up on that file. It's a pretty simple trace (for now) with a single event class with 4 ints and a sequence of 32-bit unsigned ints which is usually only 2 elements long.



This is not I expect. Still eithet jgalar or eep might have a more insight on

this. CCing the lttng-dev mailing list.



Is there any way to share us a similar trace? Either with a generator or we can

provide a link for you to upload it. The current limit on bugs.lttng.org is a

bit too small for such trace.





Aside from that, am very pleased with the how easy CTF is to work with. Looking forward to doing more.



Doug.



On Tue, 2019-03-12 at 14:41 +0000, Matthew Khouzam wrote:



Hi Doug,





Great to hear you're coming to a standard! I don't know if trace compass will scale properly as I don't know what the trace configuration is.





I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.







My suggestion is to have 1 MB packets. This makes seeking very efficient. If each event is 32 bytes, basically flush every 25k events or so.





Please keep us posted!





Matthew.



________________________________

From: tracecompass-dev-bounces@eclipse.org<mailto:tracecompass-dev-bounces@eclipse.org> <tracecompass-dev-bounces@eclipse.org<mailto:tracecompass-dev-bounces@eclipse.org>> on behalf of Doug Schaefer <dschaefer@blackberry.com<mailto:dschaefer@blackberry.com>>

Sent: Tuesday, March 12, 2019 10:30:15 AM

To: tracecompass-dev@eclipse.org<mailto:tracecompass-dev@eclipse.org>

Subject: [tracecompass-dev] Scalability



Hey gang,



We're finally starting to look at converting our custom traces into CTF so we can leverage tools like TraceCompass and, of course, contribute to it. One thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.



I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.



But when I use babeltrace to print it out, I ran out of virtual memory. I then hand coded a reader and simply adding the trace to the context caused the memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 18.04 distro), tries to inflate the events into it's internal representation for the entire trace. I need to do more investigation to confirm that.



So my question for this list, would TraceCompass do better? Does it have it's own parsing libraries?



Thanks,

Doug



_______________________________________________

tracecompass-dev mailing list

tracecompass-dev@eclipse.org<mailto:tracecompass-dev@eclipse.org>

To change your delivery options, retrieve your password, or unsubscribe from this list, visit

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eclipse.org_mailman_listinfo_tracecompass-2Ddev&d=DwIBAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=ZoCx61VE_sHGa6j3DXehbqjX5P1NGEuDtEFaGORCh9k&s=qi9q1zvDaC9cJCci0y123O-j66M643YwxRJccJCzg_c&e=







[-- Attachment #1.2: Type: text/html, Size: 13204 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Doug Schaefer]

[-- Attachment #1.1: Type: text/plain, Size: 2590 bytes --]

I have attached the C file for a generator that creates a trace that reproduces the memory issue with babeltrace. The same issue happens with a custom program in the call to add a trace to a context.

I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the version number 1.5.5.

Thanks for the help!

On Tue, 2019-03-12 at 15:47 +0000, Doug Schaefer wrote:
In the thread on the tracecompass list, I also mentioned that the babeltrace utility itself ran out of memory when reading the file. That's actually my biggest worry. I got around the generator issue by flushing every 100K events.

I'll create a quick generator that shows the same issue and post it in a bit.

Doug.

On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:

On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:

It should be easy to generate a random one with the class I mentioned below

and fill it with 16 million events.


Not if the problem is in the generator code ;).


BTW, two of the int fields are 5 and 10

bits respectively, but I'm not sure that matters (or at least it shouldn't).


Make sure to check the endian-ness of those fields.


What class? Not sure I see any reference to any.


Since you are generating CTF the error can be in the generator code. Please

provide either a trace reproducing the issue or a base reproducer. It will be

much easier overall than us trying to reproduce it blindly.


Cheers.


_______________________________________________

lttng-dev mailing list

lttng-dev@lists.lttng.org<mailto:lttng-dev@lists.lttng.org>

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwICAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk&s=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g&e=

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

[-- Attachment #1.2: Type: text/html, Size: 3878 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: scaletest.c --]
[-- Type: text/x-csrc; name="scaletest.c", Size: 5610 bytes --]

#include <babeltrace/context.h>
#include <babeltrace/ctf-ir/clock.h>
#include <babeltrace/ctf-ir/event-class.h>
#include <babeltrace/ctf-ir/event.h>
#include <babeltrace/ctf-ir/field-types.h>
#include <babeltrace/ctf-ir/fields.h>
#include <babeltrace/ctf-ir/stream-class.h>
#include <babeltrace/ctf-writer/event.h>
#include <babeltrace/ctf-writer/event-fields.h>
#include <babeltrace/ctf-writer/stream-class.h>
#include <babeltrace/ctf-writer/stream.h>
#include <babeltrace/ctf-writer/writer.h>
#include <stdlib.h>
#include <stdint.h>
#include <stdio.h>

#define CHECK(call) \
    if ((call) < 0) { \
        fprintf(stderr, "failed %s:%d %s\n", __func__, __LINE__, #call); \
        exit(1); \
    }

#define NULLCHECK(ptr) \
    if ((ptr) == NULL) { \
        fprintf(stderr, "failed %s:%d %s\n", __func__, __LINE__, #ptr); \
        exit(1); \
    }

int main(int argc, char *argv[])
{
    struct bt_ctf_writer *writer = bt_ctf_writer_create("scaletest.ctf");
    NULLCHECK(writer);

    struct bt_ctf_clock *clock = bt_ctf_clock_create("simple_clock");
    NULLCHECK(clock);
    CHECK(bt_ctf_clock_set_description(clock, "simple clock, really"));
    CHECK(bt_ctf_writer_add_clock(writer, clock));

    struct bt_ctf_stream_class *scale_stream_class = bt_ctf_stream_class_create("scale_stream");
    NULLCHECK(scale_stream_class);
    CHECK(bt_ctf_stream_class_set_clock(scale_stream_class, clock));

    // simple class
    struct bt_ctf_event_class *simple_event_class = bt_ctf_event_class_create("simple");
    NULLCHECK(simple_event_class);

    struct bt_ctf_field_type *cpu_type = bt_ctf_field_type_integer_create(6);
    NULLCHECK(cpu_type);
    CHECK(bt_ctf_field_type_integer_set_signed(cpu_type, 0));
    CHECK(bt_ctf_event_class_add_field(simple_event_class, cpu_type, "cpu"));

    struct bt_ctf_field_type *class_type = bt_ctf_field_type_integer_create(5);
    NULLCHECK(class_type);
    CHECK(bt_ctf_field_type_integer_set_signed(class_type, 0));
    CHECK(bt_ctf_event_class_add_field(simple_event_class, class_type, "class"));

    struct bt_ctf_field_type *type_type = bt_ctf_field_type_integer_create(10);
    NULLCHECK(type_type);
    CHECK(bt_ctf_field_type_integer_set_signed(type_type, 0));
    CHECK(bt_ctf_event_class_add_field(simple_event_class, type_type, "type"));

    struct bt_ctf_field_type *data_length_type = bt_ctf_field_type_integer_create(32);
    NULLCHECK(data_length_type);
    CHECK(bt_ctf_field_type_integer_set_signed(data_length_type, 0));
    CHECK(bt_ctf_event_class_add_field(simple_event_class, data_length_type, "data_length"));

    struct bt_ctf_field_type *data_element_type = bt_ctf_field_type_integer_create(32);
    NULLCHECK(data_element_type);
    CHECK(bt_ctf_field_type_integer_set_signed(data_element_type, 0));
    struct bt_ctf_field_type *data_type = bt_ctf_field_type_sequence_create(data_element_type, "data_length");
    NULLCHECK(data_type);
    CHECK(bt_ctf_event_class_add_field(simple_event_class, data_type, "data"));

    CHECK(bt_ctf_stream_class_add_event_class(scale_stream_class, simple_event_class));

    struct bt_ctf_stream *stream = bt_ctf_writer_create_stream(writer, scale_stream_class);
    NULLCHECK(stream);

    uint64_t time = 10000000;

    for (int i = 0; i < 16000000; i++) {
        if (i % 100000 == 0) {
            printf("event %d\n", i);
            CHECK(bt_ctf_stream_flush(stream));
        }

        CHECK(bt_ctf_clock_set_time(clock, time++));

        struct bt_ctf_event *event = bt_ctf_event_create(simple_event_class);
        NULLCHECK(event);

        struct bt_ctf_field *cpu_field = bt_ctf_field_create(cpu_type);
        NULLCHECK(cpu_field);
        CHECK(bt_ctf_field_unsigned_integer_set_value(cpu_field, 0));
        CHECK(bt_ctf_event_set_payload(event, "cpu", cpu_field));
        bt_ctf_field_put(cpu_field);

        struct bt_ctf_field *class_field = bt_ctf_field_create(class_type);
        NULLCHECK(class_field);
        CHECK(bt_ctf_field_unsigned_integer_set_value(class_field, 0));
        CHECK(bt_ctf_event_set_payload(event, "class", class_field));
        bt_ctf_field_put(class_field);

        struct bt_ctf_field *type_field = bt_ctf_field_create(type_type);
        NULLCHECK(type_field);
        CHECK(bt_ctf_field_unsigned_integer_set_value(type_field, 0));
        CHECK(bt_ctf_event_set_payload(event, "type", type_field));
        bt_ctf_field_put(type_field);

        size_t data_length = 2;
        struct bt_ctf_field *data_length_field = bt_ctf_field_create(data_length_type);
        NULLCHECK(data_length_field);
        CHECK(bt_ctf_field_unsigned_integer_set_value(data_length_field, data_length));
        CHECK(bt_ctf_event_set_payload(event, "data_length", data_length_field));

        struct bt_ctf_field *data_field = bt_ctf_field_create(data_type);
        NULLCHECK(data_field);
        CHECK(bt_ctf_field_sequence_set_length(data_field, data_length_field));
        for (uint64_t j = 0; j < data_length; j++) {
            struct bt_ctf_field *data_element_field = bt_ctf_field_sequence_get_field(data_field, j);
            NULLCHECK(data_element_field);
            CHECK(bt_ctf_field_unsigned_integer_set_value(data_element_field, 0));
            bt_ctf_field_put(data_element_field);
        }
        CHECK(bt_ctf_event_set_payload(event, "data", data_field));
        bt_ctf_field_put(data_length_field);
        bt_ctf_field_put(data_field);

        CHECK(bt_ctf_stream_append_event(stream, event));

        bt_ctf_event_put(event);
    }

    CHECK(bt_ctf_stream_flush(stream));
    bt_ctf_writer_flush_metadata(writer);
    return 0;
}

[-- Attachment #3: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Matthew Khouzam]

[-- Attachment #1.1: Type: text/plain, Size: 3448 bytes --]

Hi Doug & Efficios,


I suspect the issue is with libbabeltrace-ctf. When opening the trace in Trace Compass, it failed saying the sequence was 2gb big. When opening in babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I saw 2gb allocations.


On another subject, fixing the trace made it scale well in both babeltrace and Trace Compass.


I don't know why, data_length writes the wrong data. Hope this helps all parties.


BR,

Matthew

________________________________
From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> on behalf of Doug Schaefer <dschaefer@blackberry.com>
Sent: Tuesday, March 12, 2019 2:50:29 PM
To: jonathan.rajotte-julien@efficios.com
Cc: lttng-dev@lists.lttng.org; tracecompass-dev@eclipse.org
Subject: Re: [tracecompass-dev] [lttng-dev] Scalability

I have attached the C file for a generator that creates a trace that reproduces the memory issue with babeltrace. The same issue happens with a custom program in the call to add a trace to a context.

I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the version number 1.5.5.

Thanks for the help!

On Tue, 2019-03-12 at 15:47 +0000, Doug Schaefer wrote:
In the thread on the tracecompass list, I also mentioned that the babeltrace utility itself ran out of memory when reading the file. That's actually my biggest worry. I got around the generator issue by flushing every 100K events.

I'll create a quick generator that shows the same issue and post it in a bit.

Doug.

On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:

On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:

It should be easy to generate a random one with the class I mentioned below

and fill it with 16 million events.


Not if the problem is in the generator code ;).


BTW, two of the int fields are 5 and 10

bits respectively, but I'm not sure that matters (or at least it shouldn't).


Make sure to check the endian-ness of those fields.


What class? Not sure I see any reference to any.


Since you are generating CTF the error can be in the generator code. Please

provide either a trace reproducing the issue or a base reproducer. It will be

much easier overall than us trying to reproduce it blindly.


Cheers.


_______________________________________________

lttng-dev mailing list

lttng-dev@lists.lttng.org<mailto:lttng-dev@lists.lttng.org>

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwICAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk&s=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g&e=


---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

[-- Attachment #1.2: Type: text/html, Size: 5676 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Philippe Proulx]

On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam
<matthew.khouzam@ericsson.com> wrote:
>
> Hi Doug & Efficios,
>
>
> I suspect the issue is with libbabeltrace-ctf. When opening the trace in Trace Compass, it failed saying the sequence was 2gb big. When opening in babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I saw 2gb allocations.

Yes, byte order issues. Here's the fix:
<https://github.com/efficios/babeltrace/pull/103>.

Phil

>
>
> On another subject, fixing the trace made it scale well in both babeltrace and Trace Compass.
>
>
> I don't know why, data_length writes the wrong data. Hope this helps all parties.
>
>
> BR,
>
> Matthew
>
> ________________________________
> From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> on behalf of Doug Schaefer <dschaefer@blackberry.com>
> Sent: Tuesday, March 12, 2019 2:50:29 PM
> To: jonathan.rajotte-julien@efficios.com
> Cc: lttng-dev@lists.lttng.org; tracecompass-dev@eclipse.org
> Subject: Re: [tracecompass-dev] [lttng-dev] Scalability
>
> I have attached the C file for a generator that creates a trace that reproduces the memory issue with babeltrace. The same issue happens with a custom program in the call to add a trace to a context.
>
> I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the version number 1.5.5.
>
> Thanks for the help!
>
> On Tue, 2019-03-12 at 15:47 +0000, Doug Schaefer wrote:
>
> In the thread on the tracecompass list, I also mentioned that the babeltrace utility itself ran out of memory when reading the file. That's actually my biggest worry. I got around the generator issue by flushing every 100K events.
>
> I'll create a quick generator that shows the same issue and post it in a bit.
>
> Doug.
>
> On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:
>
> On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:
>
> It should be easy to generate a random one with the class I mentioned below
>
> and fill it with 16 million events.
>
>
> Not if the problem is in the generator code ;).
>
>
> BTW, two of the int fields are 5 and 10
>
> bits respectively, but I'm not sure that matters (or at least it shouldn't).
>
>
> Make sure to check the endian-ness of those fields.
>
>
> What class? Not sure I see any reference to any.
>
>
> Since you are generating CTF the error can be in the generator code. Please
>
> provide either a trace reproducing the issue or a base reproducer. It will be
>
> much easier overall than us trying to reproduce it blindly.
>
>
> Cheers.
>
>
> _______________________________________________
>
> lttng-dev mailing list
>
> lttng-dev@lists.lttng.org
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwICAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk&s=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g&e=
>
>
> ---------------------------------------------------------------------
> This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Doug Schaefer]

[-- Attachment #1.1: Type: text/plain, Size: 4845 bytes --]

Very cool. Would explicitly setting the byte order for each field be a short term workaround?

Sent from my BlackBerry - the most secure mobile device - via the Rogers Network
From: eeppeliteloop@gmail.com
Sent: March 13, 2019 8:30 AM
To: matthew.khouzam@ericsson.com
Cc: jonathan.rajotte-julien@efficios.com; dschaefer@blackberry.com; tracecompass-dev@eclipse.org; lttng-dev@lists.lttng.org
Subject: Re: [lttng-dev] [tracecompass-dev] Scalability


On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam
<matthew.khouzam@ericsson.com> wrote:
>
> Hi Doug & Efficios,
>
>
> I suspect the issue is with libbabeltrace-ctf. When opening the trace in Trace Compass, it failed saying the sequence was 2gb big. When opening in babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I saw 2gb allocations.

Yes, byte order issues. Here's the fix:
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_efficios_babeltrace_pull_103&d=DwIFaQ&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M&s=0QPaQrHAx605SVgAPeaZnF264NFCA44uOkcXdbV5dpA&e=>.

Phil

>
>
> On another subject, fixing the trace made it scale well in both babeltrace and Trace Compass.
>
>
> I don't know why, data_length writes the wrong data. Hope this helps all parties.
>
>
> BR,
>
> Matthew
>
> ________________________________
> From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> on behalf of Doug Schaefer <dschaefer@blackberry.com>
> Sent: Tuesday, March 12, 2019 2:50:29 PM
> To: jonathan.rajotte-julien@efficios.com
> Cc: lttng-dev@lists.lttng.org; tracecompass-dev@eclipse.org
> Subject: Re: [tracecompass-dev] [lttng-dev] Scalability
>
> I have attached the C file for a generator that creates a trace that reproduces the memory issue with babeltrace. The same issue happens with a custom program in the call to add a trace to a context.
>
> I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the version number 1.5.5.
>
> Thanks for the help!
>
> On Tue, 2019-03-12 at 15:47 +0000, Doug Schaefer wrote:
>
> In the thread on the tracecompass list, I also mentioned that the babeltrace utility itself ran out of memory when reading the file. That's actually my biggest worry. I got around the generator issue by flushing every 100K events.
>
> I'll create a quick generator that shows the same issue and post it in a bit.
>
> Doug.
>
> On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:
>
> On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:
>
> It should be easy to generate a random one with the class I mentioned below
>
> and fill it with 16 million events.
>
>
> Not if the problem is in the generator code ;).
>
>
> BTW, two of the int fields are 5 and 10
>
> bits respectively, but I'm not sure that matters (or at least it shouldn't).
>
>
> Make sure to check the endian-ness of those fields.
>
>
> What class? Not sure I see any reference to any.
>
>
> Since you are generating CTF the error can be in the generator code. Please
>
> provide either a trace reproducing the issue or a base reproducer. It will be
>
> much easier overall than us trying to reproduce it blindly.
>
>
> Cheers.
>
>
> _______________________________________________
>
> lttng-dev mailing list
>
> lttng-dev@lists.lttng.org
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwICAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk&s=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g&e=
>
>
> ---------------------------------------------------------------------
> This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwIFaQ&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M&s=0dykfQKNunY0J3PjBwLhAqvF-WUIJ-gk74D-duicVKI&e=

[-- Attachment #1.2: Type: text/html, Size: 8397 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Philippe Proulx]

On Wed, Mar 13, 2019 at 8:46 AM Doug Schaefer <dschaefer@blackberry.com> wrote:
>
> Very cool. Would explicitly setting the byte order for each field be a short term workaround?

You found a very specific scenario highlighting this bug.

Some field types are copied behind the scenes for resolving purposes.
This is done in bt_ctf_stream_class_add_event_class() and
bt_ctf_writer_create_stream(). The effective byte order of the original
(`data_length_type` for example) remains unset, while the internal copy
is set. However you use the originals to create fields.

Without the patch, a temporary fix would be to put all the field type
references and get the copies (which have the correct byte order) using
bt_ctf_event_class_get_field_by_name() and
bt_ctf_field_sequence_get_field() _after_ calling
bt_ctf_writer_create_stream() in your specific case.

Phil

>
> Sent from my BlackBerry - the most secure mobile device - via the Rogers Network
> From: eeppeliteloop@gmail.com
> Sent: March 13, 2019 8:30 AM
> To: matthew.khouzam@ericsson.com
> Cc: jonathan.rajotte-julien@efficios.com; dschaefer@blackberry.com; tracecompass-dev@eclipse.org; lttng-dev@lists.lttng.org
> Subject: Re: [lttng-dev] [tracecompass-dev] Scalability
>
> On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam
> <matthew.khouzam@ericsson.com> wrote:
> >
> > Hi Doug & Efficios,
> >
> >
> > I suspect the issue is with libbabeltrace-ctf. When opening the trace in Trace Compass, it failed saying the sequence was 2gb big. When opening in babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I saw 2gb allocations.
>
> Yes, byte order issues. Here's the fix:
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_efficios_babeltrace_pull_103&d=DwIFaQ&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M&s=0QPaQrHAx605SVgAPeaZnF264NFCA44uOkcXdbV5dpA&e=>.
>
> Phil
>
> >
> >
> > On another subject, fixing the trace made it scale well in both babeltrace and Trace Compass.
> >
> >
> > I don't know why, data_length writes the wrong data. Hope this helps all parties.
> >
> >
> > BR,
> >
> > Matthew
> >
> > ________________________________
> > From: tracecompass-dev-bounces@eclipse.org <tracecompass-dev-bounces@eclipse.org> on behalf of Doug Schaefer <dschaefer@blackberry.com>
> > Sent: Tuesday, March 12, 2019 2:50:29 PM
> > To: jonathan.rajotte-julien@efficios.com
> > Cc: lttng-dev@lists.lttng.org; tracecompass-dev@eclipse.org
> > Subject: Re: [tracecompass-dev] [lttng-dev] Scalability
> >
> > I have attached the C file for a generator that creates a trace that reproduces the memory issue with babeltrace. The same issue happens with a custom program in the call to add a trace to a context.
> >
> > I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the version number 1.5.5.
> >
> > Thanks for the help!
> >
> > On Tue, 2019-03-12 at 15:47 +0000, Doug Schaefer wrote:
> >
> > In the thread on the tracecompass list, I also mentioned that the babeltrace utility itself ran out of memory when reading the file. That's actually my biggest worry. I got around the generator issue by flushing every 100K events.
> >
> > I'll create a quick generator that shows the same issue and post it in a bit.
> >
> > Doug.
> >
> > On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:
> >
> > On Tue, Mar 12, 2019 at 03:23:23PM +0000, Doug Schaefer wrote:
> >
> > It should be easy to generate a random one with the class I mentioned below
> >
> > and fill it with 16 million events.
> >
> >
> > Not if the problem is in the generator code ;).
> >
> >
> > BTW, two of the int fields are 5 and 10
> >
> > bits respectively, but I'm not sure that matters (or at least it shouldn't).
> >
> >
> > Make sure to check the endian-ness of those fields.
> >
> >
> > What class? Not sure I see any reference to any.
> >
> >
> > Since you are generating CTF the error can be in the generator code. Please
> >
> > provide either a trace reproducing the issue or a base reproducer. It will be
> >
> > much easier overall than us trying to reproduce it blindly.
> >
> >
> > Cheers.
> >
> >
> > _______________________________________________
> >
> > lttng-dev mailing list
> >
> > lttng-dev@lists.lttng.org
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwICAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk&s=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g&e=
> >
> >
> > ---------------------------------------------------------------------
> > This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev@lists.lttng.org
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev&d=DwIFaQ&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M&s=0dykfQKNunY0J3PjBwLhAqvF-WUIJ-gk74D-duicVKI&e=

Re: [tracecompass-dev] Scalability

[Doug Schaefer]

On Wed, 2019-03-13 at 08:59 -0400, Philippe Proulx wrote:
> On Wed, Mar 13, 2019 at 8:46 AM Doug Schaefer <
> dschaefer@blackberry.com> wrote:
> 
> Very cool. Would explicitly setting the byte order for each field be
> a short term workaround?
> 
> You found a very specific scenario highlighting this bug.
> 
> Some field types are copied behind the scenes for resolving purposes.
> This is done in bt_ctf_stream_class_add_event_class() and
> bt_ctf_writer_create_stream(). The effective byte order of the
> original
> (`data_length_type` for example) remains unset, while the internal
> copy
> is set. However you use the originals to create fields.
> 
> Without the patch, a temporary fix would be to put all the field type
> references and get the copies (which have the correct byte order)
> using
> bt_ctf_event_class_get_field_by_name() and
> bt_ctf_field_sequence_get_field() _after_ calling
> bt_ctf_writer_create_stream() in your specific case.
> 

I'll use the patch ☺.

I was somewhat following the CTF Writer API python example which
implied I could use the original. Is that not the case?

Thanks!
Doug

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Philippe Proulx]

On Wed, Mar 13, 2019 at 10:14 AM Doug Schaefer <dschaefer@blackberry.com> wrote:
>
> I'll use the patch .
>
> I was somewhat following the CTF Writer API python example which
> implied I could use the original. Is that not the case?

Yes it's the case, but there was a bug!

Phil

>
> Thanks!
> Doug
>
>
> ----------------------------------------------------------------------
> This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

Re: [tracecompass-dev] Scalability

[Geneviève Bastien]

[-- Attachment #1.1: Type: text/plain, Size: 5439 bytes --]

Hi Doug,

Welcome to the CTF ecosystem!

FYI, one thing I am working now on TraceCompass is to add support for scripting using the Eclipse EASE framework.

In short, one could use python, javascript, ruby or groovy to interact with POJOs. We'll provide a simple API to get the trace events (the parsers are already available in TraceCompass, so all supported trace types will work), fill backend (state systems, segment stores) and create time graph / XY views.

In a few weeks, I should be able to share a prototype for the community to try. I hope you can benefit from it.

Regards,

Geneviève


On 2019-03-12 11:23 a.m., Doug Schaefer wrote:
> It should be easy to generate a random one with the class I mentioned below and fill it with 16 million events. BTW, two of the int fields are 5 and 10 bits respectively, but I'm not sure that matters (or at least it shouldn't).
>
> I'll also take a look at the babeltrace code and see what I can see.
>
> BTW, my focus on libbabeltrace is to allow for a full range of tooling for our users using the language of their choice. The python binding is particularly interesting. As we move forward into the new world of IDEs, I can see a node.js binding being interesting as well. And it may even make sense to use it in Java tooling like TraceCompass using JNA. That's the vision at least, but first, baby steps ☺.
>
> Doug.
>
> On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:
>> Hi Doug,
>> On Tue, Mar 12, 2019 at 02:56:26PM +0000, Doug Schaefer wrote:
>> Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.
>> My biggest worry is on the read side. Babeltrace blew up on that file. It's a pretty simple trace (for now) with a single event class with 4 ints and a sequence of 32-bit unsigned ints which is usually only 2 elements long.
>> This is not I expect. Still eithet jgalar or eep might have a more insight on
>> this. CCing the lttng-dev mailing list.
>> Is there any way to share us a similar trace? Either with a generator or we can
>> provide a link for you to upload it. The current limit on bugs.lttng.org is a
>> bit too small for such trace.
>> Aside from that, am very pleased with the how easy CTF is to work with. Looking forward to doing more.
>> Doug.
>> On Tue, 2019-03-12 at 14:41 +0000, Matthew Khouzam wrote:
>> Hi Doug,
>> Great to hear you're coming to a standard! I don't know if trace compass will scale properly as I don't know what the trace configuration is.
>> I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.
>> My suggestion is to have 1 MB packets. This makes seeking very efficient. If each event is 32 bytes, basically flush every 25k events or so.
>> Please keep us posted!
>> Matthew.
>> ________________________________
>> From: tracecompass-dev-bounces@eclipse.org <mailto:tracecompass-dev-bounces@eclipse.org> <tracecompass-dev-bounces@eclipse.org <mailto:tracecompass-dev-bounces@eclipse.org>> on behalf of Doug Schaefer <dschaefer@blackberry.com <mailto:dschaefer@blackberry.com>>
>> Sent: Tuesday, March 12, 2019 10:30:15 AM
>> To: tracecompass-dev@eclipse.org <mailto:tracecompass-dev@eclipse.org>
>> Subject: [tracecompass-dev] Scalability
>> Hey gang,
>> We're finally starting to look at converting our custom traces into CTF so we can leverage tools like TraceCompass and, of course, contribute to it. One thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.
>> I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.
>> But when I use babeltrace to print it out, I ran out of virtual memory. I then hand coded a reader and simply adding the trace to the context caused the memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 18.04 distro), tries to inflate the events into it's internal representation for the entire trace. I need to do more investigation to confirm that.
>> So my question for this list, would TraceCompass do better? Does it have it's own parsing libraries?
>> Thanks,
>> Doug
>> _______________________________________________
>> tracecompass-dev mailing list
>> tracecompass-dev@eclipse.org <mailto:tracecompass-dev@eclipse.org>
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eclipse.org_mailman_listinfo_tracecompass-2Ddev&d=DwIBAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=ZoCx61VE_sHGa6j3DXehbqjX5P1NGEuDtEFaGORCh9k&s=qi9q1zvDaC9cJCci0y123O-j66M643YwxRJccJCzg_c&e=
>
> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@eclipse.org
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/tracecompass-dev

[-- Attachment #1.2: Type: text/html, Size: 8483 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Doug Schaefer]

[-- Attachment #1.1: Type: text/plain, Size: 6938 bytes --]

Hi Geneviève, thanks for this.

For now, I'm focused on libbabeltrace. I think this is something my users will like to work with. They're mainly C hacks working on our kernel to start. I think they'll also like using the python bindings.

It would be awesome to have a stack that built on top of that that used the same APIs in whatever language. My current thinking is to build a prototype that visualizes using D3 in an Electron app, so JavaScript/TypeScript would be key there. I'd be happy to help contribute to that. But that will probably be in the Babeltrace repo.

I'm still not clear how TraceCompass fits in but will start looking at that once I'm off the ground. At this stage of the game, as you can see from my work on CDT, whatever solution I plan on adopting has to plug into not only the Eclipse IDE, but HTML5 front ends like VS Code or just simple Electron apps or web pages as well. It's a pretty exciting new direction. Imagine visualizing traces using WebGL. We can do some pretty cool things that users will get a lot of value from.

Cheers,
Doug.

On Fri, 2019-03-15 at 09:40 -0400, Geneviève Bastien wrote:

Hi Doug,

Welcome to the CTF ecosystem!

FYI, one thing I am working now on TraceCompass is to add support for scripting using the Eclipse EASE framework.

In short, one could use python, javascript, ruby or groovy to interact with POJOs. We'll provide a simple API to get the trace events (the parsers are already available in TraceCompass, so all supported trace types will work), fill backend (state systems, segment stores) and create time graph / XY views.

In a few weeks, I should be able to share a prototype for the community to try. I hope you can benefit from it.

Regards,

Geneviève


On 2019-03-12 11:23 a.m., Doug Schaefer wrote:
It should be easy to generate a random one with the class I mentioned below and fill it with 16 million events. BTW, two of the int fields are 5 and 10 bits respectively, but I'm not sure that matters (or at least it shouldn't).

I'll also take a look at the babeltrace code and see what I can see.

BTW, my focus on libbabeltrace is to allow for a full range of tooling for our users using the language of their choice. The python binding is particularly interesting. As we move forward into the new world of IDEs, I can see a node.js binding being interesting as well. And it may even make sense to use it in Java tooling like TraceCompass using JNA. That's the vision at least, but first, baby steps ☺.

Doug.

On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,

On Tue, Mar 12, 2019 at 02:56:26PM +0000, Doug Schaefer wrote:

Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.

My biggest worry is on the read side. Babeltrace blew up on that file. It's a pretty simple trace (for now) with a single event class with 4 ints and a sequence of 32-bit unsigned ints which is usually only 2 elements long.

This is not I expect. Still eithet jgalar or eep might have a more insight on

this. CCing the lttng-dev mailing list.

Is there any way to share us a similar trace? Either with a generator or we can

provide a link for you to upload it. The current limit on bugs.lttng.org is a

bit too small for such trace.

Aside from that, am very pleased with the how easy CTF is to work with. Looking forward to doing more.

Doug.

On Tue, 2019-03-12 at 14:41 +0000, Matthew Khouzam wrote:

Hi Doug,

Great to hear you're coming to a standard! I don't know if trace compass will scale properly as I don't know what the trace configuration is.

I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.

My suggestion is to have 1 MB packets. This makes seeking very efficient. If each event is 32 bytes, basically flush every 25k events or so.

Please keep us posted!

Matthew.

________________________________

From: tracecompass-dev-bounces@eclipse.org<mailto:tracecompass-dev-bounces@eclipse.org> <tracecompass-dev-bounces@eclipse.org<mailto:tracecompass-dev-bounces@eclipse.org>> on behalf of Doug Schaefer <dschaefer@blackberry.com<mailto:dschaefer@blackberry.com>>

Sent: Tuesday, March 12, 2019 10:30:15 AM

To: tracecompass-dev@eclipse.org<mailto:tracecompass-dev@eclipse.org>

Subject: [tracecompass-dev] Scalability

Hey gang,

We're finally starting to look at converting our custom traces into CTF so we can leverage tools like TraceCompass and, of course, contribute to it. One thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.

I have one of our trace which has 16 million events, the size of each event is around 32 bytes giving me a 540MB file. My first attempt at simply writing out the CTF events without a flush ran out of virtual memory. I then flushed after every event which made each event take 32K. So I found a middle ground and the resulting stream file is close to the same size.

But when I use babeltrace to print it out, I ran out of virtual memory. I then hand coded a reader and simply adding the trace to the context caused the memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 18.04 distro), tries to inflate the events into it's internal representation for the entire trace. I need to do more investigation to confirm that.

So my question for this list, would TraceCompass do better? Does it have it's own parsing libraries?

Thanks,

Doug

_______________________________________________

tracecompass-dev mailing list

tracecompass-dev@eclipse.org<mailto:tracecompass-dev@eclipse.org>

To change your delivery options, retrieve your password, or unsubscribe from this list, visit

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eclipse.org_mailman_listinfo_tracecompass-2Ddev&d=DwIBAg&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=ZoCx61VE_sHGa6j3DXehbqjX5P1NGEuDtEFaGORCh9k&s=qi9q1zvDaC9cJCci0y123O-j66M643YwxRJccJCzg_c&e=


_______________________________________________

tracecompass-dev mailing list

tracecompass-dev@eclipse.org<mailto:tracecompass-dev@eclipse.org>

To change your delivery options, retrieve your password, or unsubscribe from this list, visit

https://www.eclipse.org/mailman/listinfo/tracecompass-dev<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eclipse.org_mailman_listinfo_tracecompass-2Ddev&d=DwMDaQ&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=AXjlhbxcRzd4fSiGbMwiy9b1nEq-usTDVgMuGARIGig&s=KZTLmhBOxh0wmvcdRm5Wn1xbWXuEtBDoIa8aBaIkHvI&e=>


[-- Attachment #1.2: Type: text/html, Size: 9005 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [tracecompass-dev] Scalability

[Jonathan Rajotte-Julien]

Hi Doug,

On Fri, Mar 15, 2019 at 02:17:04PM +0000, Doug Schaefer wrote:
> Hi Geneviève, thanks for this.
> 
> For now, I'm focused on libbabeltrace. I think this is something my users will like to work with. They're mainly C hacks working on our kernel to start. I think they'll also like using the python bindings.

Did you have a look at these projects?

LTTng-analyses : https://github.com/lttng/lttng-analyses
LTTng-lami (Machine interface spec) : https://github.com/lttng/lami-spec

It might be a good starting point or example for what can be achieved via the
binding. Keep in mind that these projects are *experimental*.

Cheers

Re: [tracecompass-dev] Scalability

[Doug Schaefer]

[-- Attachment #1.1: Type: text/plain, Size: 1320 bytes --]

On Fri, 2019-03-15 at 10:56 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,


On Fri, Mar 15, 2019 at 02:17:04PM +0000, Doug Schaefer wrote:

Hi Geneviève, thanks for this.


For now, I'm focused on libbabeltrace. I think this is something my users will like to work with. They're mainly C hacks working on our kernel to start. I think they'll also like using the python bindings.


Did you have a look at these projects?


LTTng-analyses : https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lttng_lttng-2Danalyses&d=DwIFAw&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=1pK_XMBIe71LlYS3RRDwiTMzbShbPSQ_TIc18jjghtM&s=qbUlWwabJcfJYNVSdT4Bznx-7Nm0zBQ51eZ_QB30veQ&e=

LTTng-lami (Machine interface spec) : https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lttng_lami-2Dspec&d=DwIFAw&c=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE&r=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc&m=1pK_XMBIe71LlYS3RRDwiTMzbShbPSQ_TIc18jjghtM&s=bGKvY6hgxnwYxJJHIqE30Q86cqprybagOZkUEfQV_xM&e=


It might be a good starting point or example for what can be achieved via the

binding. Keep in mind that these projects are *experimental*.

Cool. Thanks! Everything I'm doing at the moment is experimental so these fit right in ☺.



Cheers


[-- Attachment #1.2: Type: text/html, Size: 2673 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev