Re: decoding tracef msg string via babeltrace API

Re: decoding tracef msg string via babeltrace API

[Milian Wolff via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 4675 bytes --]

On Dienstag, 2. April 2019 21:03:41 CEST Milian Wolff wrote:
> Hey all
> 
> I added a tracef tracepoint and want to decode it's `msg` field in the
> BT_EVENT_FIELDS scope. Babeltrace on the command line seems to handle this
> nicely, but whatever I try, it doesn't work with the API.
> 
> I'm using babeltrace from the stable-1.5 branch.
> 
> First, the output from babeltrace on the command line:
> 
> ```
> event {
>         name = "lttng_ust_tracef:event";
>         id = 9;
>         stream_id = 0;
>         loglevel = 14;
>         fields := struct {
>                 integer { size = 32; align = 8; signed = 0; encoding = none;
> base = 10; } __msg_length;
>                 integer { size = 8; align = 8; signed = 1; encoding = UTF8;
> base = 10; } _msg[ __msg_length ];
>         };
> };
> ...
> timestamp = 15:56:15.404973865, delta = +0.000023594, trace = 1000/64-bit,
> trace:hostname = agathebauer, trace:domain = ust, loglevel = TRACE_DEBUG
> (14), name = lttng_ust_tracef:event, stream.packet.context = {
> timestamp_begin = 4382372875047, timestamp_end = 4462741641629,
> content_size = 47177888, packet_size = 47185920, packet_seq_num = 0,
> events_discarded = 0, cpu_id = 3 }, stream.event.header = { id = (
> "compact" : container = 9 ), v = { compact = { timestamp = 70385656 } } },
> event.fields = { _msg_length = 33, msg = "run_event_entry: ImageCache,
> 1000" }
> ...
> ```
> 
> Now, in my code I find the event and then I've tried:
> 
> ```
>             auto definition = bt_ctf_get_field(event, scope, "msg");
>             auto declaration = bt_ctf_get_decl_from_def(definition);
>             auto type = bt_ctf_field_type(declaration);
>             auto encoding = bt_ctf_get_encoding(declaration);
>             auto string = bt_ctf_get_string(definition);
>             auto char_array = bt_ctf_get_char_array(definition);
>             const bt_definition* const* list = nullptr;
>             unsigned num_list = 0;
>             auto field_list_ret = bt_ctf_get_field_list(ctf_event,
> definition, &list, &num_list);
>             fprintf(stderr, "tracef dbg: %p %p | %d %d | %s | %s | %p %u
> %d\n", definition, declaration, type, encoding, string, char_array, list,
> num_list, field_list_ret);
> ```
> 
> the output is:
> 
> ```
> tracef dbg: 0x55e4c5b084f0 0x55e4c5b3f220 | 9 1 | (null) | (null) | (nil) 0
> -1
> 
> ```
> 
> So, it's a sequence (type 9), but I can't get the sequence... Actually, when
> I tried this the firs ttime without patching babeltrace, then it crashed in
> events.c:256 [1]. The def_sequence is non-null, but def_sequence->elems is
> null and that's not checked...
> 
> [1]: https://github.com/efficios/babeltrace/blob/stable-1.5/formats/ctf/
> events.c#L256
> 
> I'm quite stumped - what am I doing wrong? How does babeltrace handle the
> string-decoding of tracef's msg arg?

I've found out that babeltrace seems to not consume the public API. Instead, 
in `sequence.c` it's directly accessing the sequence's string member. If I 
patch `bt_ctf_get_string` like this:

```
diff --git a/formats/ctf/events.c b/formats/ctf/events.c
index bd195b93..f0ea49c8 100644
--- a/formats/ctf/events.c
+++ b/formats/ctf/events.c
@@ -599,12 +599,26 @@ end:
 char *bt_ctf_get_string(const struct bt_definition *field)
 {
 	char *ret = NULL;
+	const struct bt_declaration *decl = field ? 
bt_ctf_get_decl_from_def(field) : NULL;
+	enum ctf_type_id type = decl ? bt_ctf_field_type(decl) : 
CTF_TYPE_UNKNOWN;
 
-	if (field && bt_ctf_field_type(bt_ctf_get_decl_from_def(field)) == 
CTF_TYPE_STRING)
+	if (type == CTF_TYPE_SEQUENCE) {
+		enum ctf_string_encoding encoding = bt_ctf_get_encoding(decl);
+		struct definition_sequence *sequence_definition;
+		if (encoding == CTF_STRING_UTF8 || encoding  == 
CTF_STRING_ASCII) {
+			sequence_definition = container_of(field, struct 
definition_sequence, p);
+			ret = sequence_definition->string->str;
+			goto end;
+		}
+	} else if (type == CTF_TYPE_STRING) {
 		ret = bt_get_string(field);
-	else
-		bt_ctf_field_set_error(-EINVAL);
+		goto end;
+	}
 
+error:
+	bt_ctf_field_set_error(-EINVAL);
+
+end:
 	return ret;
 }
```

it seems to be doing what I was hoping for. Would something like that be 
accepted upstream, or do we need a third string-returning function 
`bt_ctf_get_string_sequence` or similar?

Generally, I already dislike it very much that there is a distinction between 
`bt_ctf_get_string` and `bt_ctf_get_char_array` - both should - imo - be 
handled through `bt_ctf_get_string`.

Cheers
-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: decoding tracef msg string via babeltrace API

[Milian Wolff via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 5415 bytes --]

On Dienstag, 2. April 2019 21:40:02 CEST Milian Wolff wrote:
> On Dienstag, 2. April 2019 21:03:41 CEST Milian Wolff wrote:
> > Hey all
> > 
> > I added a tracef tracepoint and want to decode it's `msg` field in the
> > BT_EVENT_FIELDS scope. Babeltrace on the command line seems to handle this
> > nicely, but whatever I try, it doesn't work with the API.
> > 
> > I'm using babeltrace from the stable-1.5 branch.
> > 
> > First, the output from babeltrace on the command line:
> > 
> > ```
> > event {
> > 
> >         name = "lttng_ust_tracef:event";
> >         id = 9;
> >         stream_id = 0;
> >         loglevel = 14;
> >         fields := struct {
> >         
> >                 integer { size = 32; align = 8; signed = 0; encoding =
> >                 none;
> > 
> > base = 10; } __msg_length;
> > 
> >                 integer { size = 8; align = 8; signed = 1; encoding =
> >                 UTF8;
> > 
> > base = 10; } _msg[ __msg_length ];
> > 
> >         };
> > 
> > };
> > ...
> > timestamp = 15:56:15.404973865, delta = +0.000023594, trace = 1000/64-bit,
> > trace:hostname = agathebauer, trace:domain = ust, loglevel = TRACE_DEBUG
> > (14), name = lttng_ust_tracef:event, stream.packet.context = {
> > timestamp_begin = 4382372875047, timestamp_end = 4462741641629,
> > content_size = 47177888, packet_size = 47185920, packet_seq_num = 0,
> > events_discarded = 0, cpu_id = 3 }, stream.event.header = { id = (
> > "compact" : container = 9 ), v = { compact = { timestamp = 70385656 } } },
> > event.fields = { _msg_length = 33, msg = "run_event_entry: ImageCache,
> > 1000" }
> > ...
> > ```
> > 
> > Now, in my code I find the event and then I've tried:
> > 
> > ```
> > 
> >             auto definition = bt_ctf_get_field(event, scope, "msg");
> >             auto declaration = bt_ctf_get_decl_from_def(definition);
> >             auto type = bt_ctf_field_type(declaration);
> >             auto encoding = bt_ctf_get_encoding(declaration);
> >             auto string = bt_ctf_get_string(definition);
> >             auto char_array = bt_ctf_get_char_array(definition);
> >             const bt_definition* const* list = nullptr;
> >             unsigned num_list = 0;
> >             auto field_list_ret = bt_ctf_get_field_list(ctf_event,
> > 
> > definition, &list, &num_list);
> > 
> >             fprintf(stderr, "tracef dbg: %p %p | %d %d | %s | %s | %p %u
> > 
> > %d\n", definition, declaration, type, encoding, string, char_array, list,
> > num_list, field_list_ret);
> > ```
> > 
> > the output is:
> > 
> > ```
> > tracef dbg: 0x55e4c5b084f0 0x55e4c5b3f220 | 9 1 | (null) | (null) | (nil)
> > 0
> > -1
> > 
> > ```
> > 
> > So, it's a sequence (type 9), but I can't get the sequence... Actually,
> > when I tried this the firs ttime without patching babeltrace, then it
> > crashed in events.c:256 [1]. The def_sequence is non-null, but
> > def_sequence->elems is null and that's not checked...
> > 
> > [1]: https://github.com/efficios/babeltrace/blob/stable-1.5/formats/ctf/
> > events.c#L256
> > 
> > I'm quite stumped - what am I doing wrong? How does babeltrace handle the
> > string-decoding of tracef's msg arg?
> 
> I've found out that babeltrace seems to not consume the public API. Instead,
> in `sequence.c` it's directly accessing the sequence's string member. If I
> patch `bt_ctf_get_string` like this:
> 
> ```
> diff --git a/formats/ctf/events.c b/formats/ctf/events.c
> index bd195b93..f0ea49c8 100644
> --- a/formats/ctf/events.c
> +++ b/formats/ctf/events.c
> @@ -599,12 +599,26 @@ end:
>  char *bt_ctf_get_string(const struct bt_definition *field)
>  {
>  	char *ret = NULL;
> +	const struct bt_declaration *decl = field ?
> bt_ctf_get_decl_from_def(field) : NULL;
> +	enum ctf_type_id type = decl ? bt_ctf_field_type(decl) :
> CTF_TYPE_UNKNOWN;
> 
> -	if (field && bt_ctf_field_type(bt_ctf_get_decl_from_def(field)) ==
> CTF_TYPE_STRING)
> +	if (type == CTF_TYPE_SEQUENCE) {
> +		enum ctf_string_encoding encoding = bt_ctf_get_encoding(decl);
> +		struct definition_sequence *sequence_definition;
> +		if (encoding == CTF_STRING_UTF8 || encoding  ==
> CTF_STRING_ASCII) {
> +			sequence_definition = container_of(field, struct
> definition_sequence, p);
> +			ret = sequence_definition->string->str;
> +			goto end;
> +		}
> +	} else if (type == CTF_TYPE_STRING) {
>  		ret = bt_get_string(field);
> -	else
> -		bt_ctf_field_set_error(-EINVAL);
> +		goto end;
> +	}
> 
> +error:
> +	bt_ctf_field_set_error(-EINVAL);
> +
> +end:
>  	return ret;
>  }
> ```
> 
> it seems to be doing what I was hoping for. Would something like that be
> accepted upstream, or do we need a third string-returning function
> `bt_ctf_get_string_sequence` or similar?
> 
> Generally, I already dislike it very much that there is a distinction
> between `bt_ctf_get_string` and `bt_ctf_get_char_array` - both should - imo
> - be handled through `bt_ctf_get_string`.

And now I found out that this is known and a patch is even available:

https://github.com/efficios/babeltrace/pull/98/commits/
3ab89a428ce2a8a4f76218f79c9eb6c26ae5aeee

It's been nearly 1.5 years and babeltrace 2.0 is still not ready. Can you 
please merge this into babeltrace 1.5 and release a new version of that stable 
branch?

Thanks
-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: decoding tracef msg string via babeltrace API

[Jérémie Galarneau]

Hi Milian,

I have pushed a fix [1] in the stable-1.5 branch that addresses the
issue you have encountered with using bt_ctf_get_field_list().
This makes it possible to consume a character sequence's content.

The commit message contains an example to show how to essentially
perform what the 'ctf-text' format plug-in achieves by accessing the
internal string field.

Let me know if that works for you and I'll release an updated version
of the 1.5 branch.

Thanks,
Jérémie

[1] https://github.com/efficios/babeltrace/commit/e7f1ad36b82bb8651c91c6c6faf8b92f145f32bc

On Tue, 2 Apr 2019 at 15:09, Milian Wolff via lttng-dev
<lttng-dev@lists.lttng.org> wrote:
>
> Hey all
>
> I added a tracef tracepoint and want to decode it's `msg` field in the
> BT_EVENT_FIELDS scope. Babeltrace on the command line seems to handle this
> nicely, but whatever I try, it doesn't work with the API.
>
> I'm using babeltrace from the stable-1.5 branch.
>
> First, the output from babeltrace on the command line:
>
> ```
> event {
>         name = "lttng_ust_tracef:event";
>         id = 9;
>         stream_id = 0;
>         loglevel = 14;
>         fields := struct {
>                 integer { size = 32; align = 8; signed = 0; encoding = none;
> base = 10; } __msg_length;
>                 integer { size = 8; align = 8; signed = 1; encoding = UTF8;
> base = 10; } _msg[ __msg_length ];
>         };
> };
> ...
> timestamp = 15:56:15.404973865, delta = +0.000023594, trace = 1000/64-bit,
> trace:hostname = agathebauer, trace:domain = ust, loglevel = TRACE_DEBUG (14),
> name = lttng_ust_tracef:event, stream.packet.context = { timestamp_begin =
> 4382372875047, timestamp_end = 4462741641629, content_size = 47177888,
> packet_size = 47185920, packet_seq_num = 0, events_discarded = 0, cpu_id = 3
> }, stream.event.header = { id = ( "compact" : container = 9 ), v = { compact =
> { timestamp = 70385656 } } }, event.fields = { _msg_length = 33, msg =
> "run_event_entry: ImageCache, 1000" }
> ...
> ```
>
> Now, in my code I find the event and then I've tried:
>
> ```
>             auto definition = bt_ctf_get_field(event, scope, "msg");
>             auto declaration = bt_ctf_get_decl_from_def(definition);
>             auto type = bt_ctf_field_type(declaration);
>             auto encoding = bt_ctf_get_encoding(declaration);
>             auto string = bt_ctf_get_string(definition);
>             auto char_array = bt_ctf_get_char_array(definition);
>             const bt_definition* const* list = nullptr;
>             unsigned num_list = 0;
>             auto field_list_ret = bt_ctf_get_field_list(ctf_event, definition,
> &list, &num_list);
>             fprintf(stderr, "tracef dbg: %p %p | %d %d | %s | %s | %p %u
> %d\n", definition, declaration, type, encoding, string, char_array, list,
> num_list, field_list_ret);
> ```
>
> the output is:
>
> ```
> tracef dbg: 0x55e4c5b084f0 0x55e4c5b3f220 | 9 1 | (null) | (null) | (nil) 0 -1
>
> ```
>
> So, it's a sequence (type 9), but I can't get the sequence... Actually, when I
> tried this the firs ttime without patching babeltrace, then it crashed in
> events.c:256 [1]. The def_sequence is non-null, but def_sequence->elems is
> null and that's not checked...
>
> [1]: https://github.com/efficios/babeltrace/blob/stable-1.5/formats/ctf/
> events.c#L256
>
> I'm quite stumped - what am I doing wrong? How does babeltrace handle the
> string-decoding of tracef's msg arg?
>
> Thanks
> --
> Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
> KDAB (Deutschland) GmbH, a KDAB Group company
> Tel: +49-30-521325470
> IKDAB - The Qt, C++ and OpenGL Experts_______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: decoding tracef msg string via babeltrace API

[Milian Wolff via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 1245 bytes --]

On Thursday, April 4, 2019 8:25:51 PM CEST Jérémie Galarneau wrote:
> Hi Milian,
> 
> I have pushed a fix [1] in the stable-1.5 branch that addresses the
> issue you have encountered with using bt_ctf_get_field_list().
> This makes it possible to consume a character sequence's content.
> 
> The commit message contains an example to show how to essentially
> perform what the 'ctf-text' format plug-in achieves by accessing the
> internal string field.
> 
> Let me know if that works for you and I'll release an updated version
> of the 1.5 branch.

Thank you, I'll try that out in the next days!

But: why can't we make bt_ctf_get_string return the string for us? Why do we 
need to reinvent the wheel in consumers of the babeltrace API? You point out 
that it's non-trivial to get the strings decoded properly, so I'd really 
appreciate if we get that code implemented once and then can leverage it 
everywhere.

Generally, I don't understand why there's both, bt_ctf_get_string and 
bt_ctf_get_char_array in the first place....

Cheers

-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: decoding tracef msg string via babeltrace API

[Jérémie Galarneau]

[-- Attachment #1.1: Type: text/plain, Size: 2634 bytes --]

On Tue, 9 Apr 2019 at 04:36, Milian Wolff <milian.wolff@kdab.com> wrote:

> On Thursday, April 4, 2019 8:25:51 PM CEST Jérémie Galarneau wrote:
> > Hi Milian,
> >
> > I have pushed a fix [1] in the stable-1.5 branch that addresses the
> > issue you have encountered with using bt_ctf_get_field_list().
> > This makes it possible to consume a character sequence's content.
> >
> > The commit message contains an example to show how to essentially
> > perform what the 'ctf-text' format plug-in achieves by accessing the
> > internal string field.
> >
> > Let me know if that works for you and I'll release an updated version
> > of the 1.5 branch.
>
> Thank you, I'll try that out in the next days!
>
> But: why can't we make bt_ctf_get_string return the string for us? Why do
> we
> need to reinvent the wheel in consumers of the babeltrace API? You point
> out
>

Hi,

Strings, sequences and arrays are three different types in CTF.

bt_ctf_get_string() can only be used on CTF strings. In the babeltrace 1.x
API, those are expressed through 'bt_definition's that have a
'bt_declaration'
of type CTF_TYPE_STRING. It is not a general-purpose accessor to get
a field's content as a C-string.

Changing this in the 1.x release series goes against the design of the
API (dedicated functions per definition type) and would, arguably, be an
ABI breaking change.

As pointed out in the commit message, bt_ctf_get_char_array() is
a convenience function to access a character-array's contents as
a C-string. An equivalent function could be added for sequences.

Since you pointed-out a bug in the existing API, my priority was
in fixing that bug which prevented users from accessing the sequences'
contents at all in the "looks like a string" case. That fix can be
back-ported into existing releases and made available in a point-release.

Adding new APIs is an orthogonal concern. I'm not against a
convenience function that does what you want; but it won't be
shoe-horned into an existing one.

Jérémie



>
> Cheers
>


> that it's non-trivial to get the strings decoded properly, so I'd really
> appreciate if we get that code implemented once and then can leverage it
> everywhere.
>

> Generally, I don't understand why there's both, bt_ctf_get_string and
> bt_ctf_get_char_array in the first place....
>
-- 
> Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
> KDAB (Deutschland) GmbH, a KDAB Group company
> Tel: +49-30-521325470
> KDAB - The Qt, C++ and OpenGL Experts



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

[-- Attachment #1.2: Type: text/html, Size: 4349 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: decoding tracef msg string via babeltrace API

[Milian Wolff via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 3493 bytes --]

On Tuesday, April 9, 2019 7:02:06 PM CEST Jérémie Galarneau wrote:
> On Tue, 9 Apr 2019 at 04:36, Milian Wolff <milian.wolff@kdab.com> wrote:
> > On Thursday, April 4, 2019 8:25:51 PM CEST Jérémie Galarneau wrote:
> > > Hi Milian,
> > > 
> > > I have pushed a fix [1] in the stable-1.5 branch that addresses the
> > > issue you have encountered with using bt_ctf_get_field_list().
> > > This makes it possible to consume a character sequence's content.
> > > 
> > > The commit message contains an example to show how to essentially
> > > perform what the 'ctf-text' format plug-in achieves by accessing the
> > > internal string field.
> > > 
> > > Let me know if that works for you and I'll release an updated version
> > > of the 1.5 branch.
> > 
> > Thank you, I'll try that out in the next days!
> > 
> > But: why can't we make bt_ctf_get_string return the string for us? Why do
> > we
> > need to reinvent the wheel in consumers of the babeltrace API? You point
> > out
> 
> Hi,
> 
> Strings, sequences and arrays are three different types in CTF.
> 
> bt_ctf_get_string() can only be used on CTF strings. In the babeltrace 1.x
> API, those are expressed through 'bt_definition's that have a
> 'bt_declaration'
> of type CTF_TYPE_STRING. It is not a general-purpose accessor to get
> a field's content as a C-string.
> 
> Changing this in the 1.x release series goes against the design of the
> API (dedicated functions per definition type) and would, arguably, be an
> ABI breaking change.
> 
> As pointed out in the commit message, bt_ctf_get_char_array() is
> a convenience function to access a character-array's contents as
> a C-string. An equivalent function could be added for sequences.
> 
> Since you pointed-out a bug in the existing API, my priority was
> in fixing that bug which prevented users from accessing the sequences'
> contents at all in the "looks like a string" case. That fix can be
> back-ported into existing releases and made available in a point-release.
> 
> Adding new APIs is an orthogonal concern. I'm not against a
> convenience function that does what you want; but it won't be
> shoe-horned into an existing one.

I see. I'll have to live with this, but note how the current situation is far 
from perfect:

There's no way to figure out the exact version of babeltrace, neither at 
compile nor at runtime. So any user of babeltrace will potentially crash when 
calling bt_ctf_get_field_list to read a string-sequence.

Looking ahead, babeltrace 2.0 is supposedly ready any time now. Is that really 
true - I've been hearing this for about two years now I think. Is there any 
documentation that shows how one would transition from the 1.0 API to 2.0? I 
notice e.g. that `include/babeltrace/ctf/iterator.h` is gone? I can only find 
Python binding API documentation, but nothing for the C API? There does seem 
to be API documentation in the code though, is the doxygen generated HTML 
hosted somewhere? Would the issue above be resolves with the 2.0 API - i.e. is 
there a single function to call to get strings-data out of either STRING, 
ARRAY or SEQUENCE trace points?

Orthogonally, can someone explain why tracelog/tracef tracepoints use a 
SEQUENCE instead of a STRING type for the data?

Thanks

-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: decoding tracef msg string via babeltrace API

[Milian Wolff via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 3292 bytes --]

On Wednesday, April 10, 2019 9:56:43 AM CEST Milian Wolff wrote:
> On Tuesday, April 9, 2019 7:02:06 PM CEST Jérémie Galarneau wrote:
> > On Tue, 9 Apr 2019 at 04:36, Milian Wolff <milian.wolff@kdab.com> wrote:
> > > On Thursday, April 4, 2019 8:25:51 PM CEST Jérémie Galarneau wrote:
> > > > Hi Milian,
> > > > 
> > > > I have pushed a fix [1] in the stable-1.5 branch that addresses the
> > > > issue you have encountered with using bt_ctf_get_field_list().
> > > > This makes it possible to consume a character sequence's content.
> > > > 
> > > > The commit message contains an example to show how to essentially
> > > > perform what the 'ctf-text' format plug-in achieves by accessing the
> > > > internal string field.
> > > > 
> > > > Let me know if that works for you and I'll release an updated version
> > > > of the 1.5 branch.
> > > 
> > > Thank you, I'll try that out in the next days!
> > > 
> > > But: why can't we make bt_ctf_get_string return the string for us? Why
> > > do
> > > we
> > > need to reinvent the wheel in consumers of the babeltrace API? You point
> > > out
> > 
> > Hi,
> > 
> > Strings, sequences and arrays are three different types in CTF.
> > 
> > bt_ctf_get_string() can only be used on CTF strings. In the babeltrace 1.x
> > API, those are expressed through 'bt_definition's that have a
> > 'bt_declaration'
> > of type CTF_TYPE_STRING. It is not a general-purpose accessor to get
> > a field's content as a C-string.
> > 
> > Changing this in the 1.x release series goes against the design of the
> > API (dedicated functions per definition type) and would, arguably, be an
> > ABI breaking change.
> > 
> > As pointed out in the commit message, bt_ctf_get_char_array() is
> > a convenience function to access a character-array's contents as
> > a C-string. An equivalent function could be added for sequences.
> > 
> > Since you pointed-out a bug in the existing API, my priority was
> > in fixing that bug which prevented users from accessing the sequences'
> > contents at all in the "looks like a string" case. That fix can be
> > back-ported into existing releases and made available in a point-release.
> > 
> > Adding new APIs is an orthogonal concern. I'm not against a
> > convenience function that does what you want; but it won't be
> > shoe-horned into an existing one.
> 
> I see. I'll have to live with this, but note how the current situation is
> far from perfect:
> 
> There's no way to figure out the exact version of babeltrace, neither at
> compile nor at runtime. So any user of babeltrace will potentially crash
> when calling bt_ctf_get_field_list to read a string-sequence.

Additionally, the suggested approach of iterating over the individual chars in 
the sequence doesn't allow for a zero-copy approach as is done with 
bt_ctf_get_char_array or bt_ctf_get_string.

Both could be solved if a bt_ctf_get_string_sequence or similar would be 
introduced. Would this be accepted upstream, if I create a pull request, or is 
1.5 frozen and no new API is allowed to be added?

Thanks

-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: decoding tracef msg string via babeltrace API

[Milian Wolff via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 3536 bytes --]

On Wednesday, April 10, 2019 10:57:35 AM CEST Milian Wolff via lttng-dev 
wrote:
> On Wednesday, April 10, 2019 9:56:43 AM CEST Milian Wolff wrote:
> > On Tuesday, April 9, 2019 7:02:06 PM CEST Jérémie Galarneau wrote:
> > > On Tue, 9 Apr 2019 at 04:36, Milian Wolff <milian.wolff@kdab.com> wrote:
> > > > On Thursday, April 4, 2019 8:25:51 PM CEST Jérémie Galarneau wrote:
> > > > > Hi Milian,
> > > > > 
> > > > > I have pushed a fix [1] in the stable-1.5 branch that addresses the
> > > > > issue you have encountered with using bt_ctf_get_field_list().
> > > > > This makes it possible to consume a character sequence's content.
> > > > > 
> > > > > The commit message contains an example to show how to essentially
> > > > > perform what the 'ctf-text' format plug-in achieves by accessing the
> > > > > internal string field.
> > > > > 
> > > > > Let me know if that works for you and I'll release an updated
> > > > > version
> > > > > of the 1.5 branch.
> > > > 
> > > > Thank you, I'll try that out in the next days!
> > > > 
> > > > But: why can't we make bt_ctf_get_string return the string for us? Why
> > > > do
> > > > we
> > > > need to reinvent the wheel in consumers of the babeltrace API? You
> > > > point
> > > > out
> > > 
> > > Hi,
> > > 
> > > Strings, sequences and arrays are three different types in CTF.
> > > 
> > > bt_ctf_get_string() can only be used on CTF strings. In the babeltrace
> > > 1.x
> > > API, those are expressed through 'bt_definition's that have a
> > > 'bt_declaration'
> > > of type CTF_TYPE_STRING. It is not a general-purpose accessor to get
> > > a field's content as a C-string.
> > > 
> > > Changing this in the 1.x release series goes against the design of the
> > > API (dedicated functions per definition type) and would, arguably, be an
> > > ABI breaking change.
> > > 
> > > As pointed out in the commit message, bt_ctf_get_char_array() is
> > > a convenience function to access a character-array's contents as
> > > a C-string. An equivalent function could be added for sequences.
> > > 
> > > Since you pointed-out a bug in the existing API, my priority was
> > > in fixing that bug which prevented users from accessing the sequences'
> > > contents at all in the "looks like a string" case. That fix can be
> > > back-ported into existing releases and made available in a
> > > point-release.
> > > 
> > > Adding new APIs is an orthogonal concern. I'm not against a
> > > convenience function that does what you want; but it won't be
> > > shoe-horned into an existing one.
> > 
> > I see. I'll have to live with this, but note how the current situation is
> > far from perfect:
> > 
> > There's no way to figure out the exact version of babeltrace, neither at
> > compile nor at runtime. So any user of babeltrace will potentially crash
> > when calling bt_ctf_get_field_list to read a string-sequence.
> 
> Additionally, the suggested approach of iterating over the individual chars
> in the sequence doesn't allow for a zero-copy approach as is done with
> bt_ctf_get_char_array or bt_ctf_get_string.
> 
> Both could be solved if a bt_ctf_get_string_sequence or similar would be
> introduced. Would this be accepted upstream, if I create a pull request, or
> is 1.5 frozen and no new API is allowed to be added?

Ping?

-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

decoding tracef msg string via babeltrace API

[Milian Wolff via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 2838 bytes --]

Hey all

I added a tracef tracepoint and want to decode it's `msg` field in the 
BT_EVENT_FIELDS scope. Babeltrace on the command line seems to handle this 
nicely, but whatever I try, it doesn't work with the API.

I'm using babeltrace from the stable-1.5 branch.

First, the output from babeltrace on the command line:

```
event {
        name = "lttng_ust_tracef:event";
        id = 9;
        stream_id = 0;
        loglevel = 14;
        fields := struct {
                integer { size = 32; align = 8; signed = 0; encoding = none; 
base = 10; } __msg_length;
                integer { size = 8; align = 8; signed = 1; encoding = UTF8; 
base = 10; } _msg[ __msg_length ];
        };
};
...
timestamp = 15:56:15.404973865, delta = +0.000023594, trace = 1000/64-bit, 
trace:hostname = agathebauer, trace:domain = ust, loglevel = TRACE_DEBUG (14), 
name = lttng_ust_tracef:event, stream.packet.context = { timestamp_begin = 
4382372875047, timestamp_end = 4462741641629, content_size = 47177888, 
packet_size = 47185920, packet_seq_num = 0, events_discarded = 0, cpu_id = 3 
}, stream.event.header = { id = ( "compact" : container = 9 ), v = { compact = 
{ timestamp = 70385656 } } }, event.fields = { _msg_length = 33, msg = 
"run_event_entry: ImageCache, 1000" }
...
```

Now, in my code I find the event and then I've tried:

```
            auto definition = bt_ctf_get_field(event, scope, "msg");
            auto declaration = bt_ctf_get_decl_from_def(definition);
            auto type = bt_ctf_field_type(declaration);
            auto encoding = bt_ctf_get_encoding(declaration);
            auto string = bt_ctf_get_string(definition);
            auto char_array = bt_ctf_get_char_array(definition);
            const bt_definition* const* list = nullptr;
            unsigned num_list = 0;
            auto field_list_ret = bt_ctf_get_field_list(ctf_event, definition, 
&list, &num_list);
            fprintf(stderr, "tracef dbg: %p %p | %d %d | %s | %s | %p %u 
%d\n", definition, declaration, type, encoding, string, char_array, list, 
num_list, field_list_ret);
```

the output is:

```
tracef dbg: 0x55e4c5b084f0 0x55e4c5b3f220 | 9 1 | (null) | (null) | (nil) 0 -1

```

So, it's a sequence (type 9), but I can't get the sequence... Actually, when I 
tried this the firs ttime without patching babeltrace, then it crashed in 
events.c:256 [1]. The def_sequence is non-null, but def_sequence->elems is 
null and that's not checked...

[1]: https://github.com/efficios/babeltrace/blob/stable-1.5/formats/ctf/
events.c#L256

I'm quite stumped - what am I doing wrong? How does babeltrace handle the 
string-decoding of tracef's msg arg?

Thanks
-- 
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
IKDAB - The Qt, C++ and OpenGL Experts

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev