[lttng-dev] Community input: Feedback on use of enumeration type within LTTng-UST

[lttng-dev] Community input: Feedback on use of enumeration type within LTTng-UST

[Mathieu Desnoyers via lttng-dev]

Hi,

We are currently working on the CTF2 specification [1] at EfficiOS, and there is
a user metric we would need to help us with a design decision with respect
to enumerations. This is where we would need community input.

The usage metrics in question are with respect to LTTng-UST enumeration types
(TRACEPOINT_ENUM) [2] used by tracepoint events within instrumented applications.

A tracepoint enumeration can be used by many events. What we are looking for is to get
an idea of the common use, and extreme cases as well.

Ideally, what we would need is:

- For each enumeration within your application instrumentation (TRACEPOINT_ENUM), how
  many events refer to each enumeration ? (average, mode, and maximum)
- For each TRACEPOINT_ENUM, how many labels do they possess ? (sum number of
  ctf_enum_value/ctf_enum_range/ctf_enum_auto entries) (average, mode, and maximum)
- For each TRACEPOINT_ENUM, for each label, what is the string length ? (average, mode, and maximum)

Based on this information, we can estimate the data overhead generated by repeating enumeration
labels in the LTTng-UST metadata. We need to decide whether we allow references to a single
enumeration description when it is used by many events, or if we require repeatedly serializing
the entire enumeration description for each event field using the enumeration.

Thanks,

Mathieu

[1] https://lists.lttng.org/pipermail/lttng-dev/2020-November/029777.html
[2] https://lttng.org/man/3/lttng-ust/v2.12/#doc-tracepoint-enum

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] Community input: Feedback on use of enumeration type within LTTng-UST

[Jesper Derehag via lttng-dev]

Hi,

I dont have any data regarding enum usage unfortunatly.

But is it possible to share metadata between multiple tp-providers?
If so, would you not be required anyway to re-generate the enumeration labels given that any trace producer (sharing the same metadata) may have different enumeration definitions?
For example if you would lets say trace different versions of the same app but where the enum definition have changed?
Cases like those I guess would force you to atleast version enum labels or somesuch?

/Jesper

________________________________________
Från: lttng-dev <lttng-dev-bounces@lists.lttng.org> för Mathieu Desnoyers via lttng-dev <lttng-dev@lists.lttng.org>
Skickat: den 26 januari 2021 16:19
Till: lttng-dev
Ämne: [lttng-dev] Community input: Feedback on use of enumeration type within LTTng-UST

Hi,

We are currently working on the CTF2 specification [1] at EfficiOS, and there is
a user metric we would need to help us with a design decision with respect
to enumerations. This is where we would need community input.

The usage metrics in question are with respect to LTTng-UST enumeration types
(TRACEPOINT_ENUM) [2] used by tracepoint events within instrumented applications.

A tracepoint enumeration can be used by many events. What we are looking for is to get
an idea of the common use, and extreme cases as well.

Ideally, what we would need is:

- For each enumeration within your application instrumentation (TRACEPOINT_ENUM), how
  many events refer to each enumeration ? (average, mode, and maximum)
- For each TRACEPOINT_ENUM, how many labels do they possess ? (sum number of
  ctf_enum_value/ctf_enum_range/ctf_enum_auto entries) (average, mode, and maximum)
- For each TRACEPOINT_ENUM, for each label, what is the string length ? (average, mode, and maximum)

Based on this information, we can estimate the data overhead generated by repeating enumeration
labels in the LTTng-UST metadata. We need to decide whether we allow references to a single
enumeration description when it is used by many events, or if we require repeatedly serializing
the entire enumeration description for each event field using the enumeration.

Thanks,

Mathieu

[1] https://lists.lttng.org/pipermail/lttng-dev/2020-November/029777.html
[2] https://lttng.org/man/3/lttng-ust/v2.12/#doc-tracepoint-enum

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] Community input: Feedback on use of enumeration type within LTTng-UST

[Mathieu Desnoyers via lttng-dev]

Hi Jesper,

We currently don't have anything in place to allow sharing a given enumeration
across multiple tracepoint providers. However, you are not the first one to
express the need for it.

One way I figure we could do this would be to introduce the notion of
"probe provider dependency" in lttng-ust. The basic idea is to have the
enumeration within a probe provider which would be referred to as a
"dependency" by other tracepoint probes. We could also do some sharing
of tracepoint event class declarations as well if needed in a similar
way.

The upgrade scenario problem you highlight is not specific to enumerations.
It also applies to events being changed when upgrading a tracepoint probe
provider.

For events, the way we match them within the session daemon (ht_match_event)
is by comparing their entire description for an exact match: they need to
have all the same fields. If two events happen to have the same name but
different fields (due to an upgrade in some applications but not others),
they will be represented by different event IDs within the metadata. This
allows us to make sure each event has a unique event ID within a trace if
they have different content, even if they have the same name (e.g. upgrade
scenario).

When dealing with enumeration types in the ust registry within lttng-sessiond,
we already do something similar (ht_match_enum): we check that all labels match.

So I think that the only part we are missing is really at the UST probe provider
level: adding the ability to express dependency from provider A on an enumeration
defined by provider B. The rest of the logic is already there in the session daemon
to allow reference to an enumeration from various event fields within a single probe
provider.

Thanks,

Mathieu

----- On Jan 29, 2021, at 10:07 AM, jderehag jderehag@hotmail.com wrote:

> Hi,
> 
> I dont have any data regarding enum usage unfortunatly.
> 
> But is it possible to share metadata between multiple tp-providers?
> If so, would you not be required anyway to re-generate the enumeration labels
> given that any trace producer (sharing the same metadata) may have different
> enumeration definitions?
> For example if you would lets say trace different versions of the same app but
> where the enum definition have changed?
> Cases like those I guess would force you to atleast version enum labels or
> somesuch?
> 
> /Jesper
> 
> ________________________________________
> Från: lttng-dev <lttng-dev-bounces@lists.lttng.org> för Mathieu Desnoyers via
> lttng-dev <lttng-dev@lists.lttng.org>
> Skickat: den 26 januari 2021 16:19
> Till: lttng-dev
> Ämne: [lttng-dev] Community input: Feedback on use of enumeration type within
> LTTng-UST
> 
> Hi,
> 
> We are currently working on the CTF2 specification [1] at EfficiOS, and there is
> a user metric we would need to help us with a design decision with respect
> to enumerations. This is where we would need community input.
> 
> The usage metrics in question are with respect to LTTng-UST enumeration types
> (TRACEPOINT_ENUM) [2] used by tracepoint events within instrumented
> applications.
> 
> A tracepoint enumeration can be used by many events. What we are looking for is
> to get
> an idea of the common use, and extreme cases as well.
> 
> Ideally, what we would need is:
> 
> - For each enumeration within your application instrumentation
> (TRACEPOINT_ENUM), how
>  many events refer to each enumeration ? (average, mode, and maximum)
> - For each TRACEPOINT_ENUM, how many labels do they possess ? (sum number of
>  ctf_enum_value/ctf_enum_range/ctf_enum_auto entries) (average, mode, and
>  maximum)
> - For each TRACEPOINT_ENUM, for each label, what is the string length ?
> (average, mode, and maximum)
> 
> Based on this information, we can estimate the data overhead generated by
> repeating enumeration
> labels in the LTTng-UST metadata. We need to decide whether we allow references
> to a single
> enumeration description when it is used by many events, or if we require
> repeatedly serializing
> the entire enumeration description for each event field using the enumeration.
> 
> Thanks,
> 
> Mathieu
> 
> [1] https://lists.lttng.org/pipermail/lttng-dev/2020-November/029777.html
> [2] https://lttng.org/man/3/lttng-ust/v2.12/#doc-tracepoint-enum
> 
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev