Re: Format of trace.dat flyrecord section

[Tzvetomir Stoyanov <tz.stoyanov@gmail.com>]

On Wed, Jun 15, 2022 at 6:19 PM Matteo Bertolino
<matteo.bertolino@huawei.com> wrote:
>
> Dear community,
>
> [Goal] I am trying to understand more about tracing. In one of my experiments, I decided to try to write my own trace.dat file (with some dummy entries).
> To do that, I am following the documentation for a "version-7 .dat" provided in https://github.com/rostedt/trace-cmd/blob/master/Documentation/trace-cmd/trace-cmd.dat.v7.5.txt ,
> and trying to looking at the code of libtraceevent, trace-cmd and kernelshark. I use kernelshark to be sure I am writing a good trace.
>
> [Context] I managed to write a complete header (I double-checked checking in strategical points in trace-cmd, kernelshark and libtraceevent code). In this header, I have three sections:
> - Section 0, with options 16 (header_infos), option 8 (in which I stated to have a single CPU), and option 3. The latter states that I have a cpu whose data starts at offset 4096 of the trace.
> - Section 16: in which I transfers the information of `header_page` and `header_event` files.
> - Section 3: the flyrecord section, whose header is followed by a padding to be one-page aligned. After the padding, there should be the CPU datas.
> Yet another check of the header's correctness are outputs of commands `trace-cmd dump --flyrecord -i mytrace.dat` and `trace-cmd dump --summary -i ../mytrace_v7.dat`.
>
> [Problem] The problems occur there. I don't manage to get the format of flyrecords.
> I understood that timestamps need to follow the structure of ringbuffers (those expressed in `header_event` file), so 5 bits for the type_len, 27 for the time_delta, and an u32 array[].
> But, kernelshark entries contains also: timestamp, CPUs, PID, EVENT, TASK, LATENCY and "INFO".
>
> Through flybuffer schema, I can only provide TIMESTAMP divided by CPUs, but where do I take other fields?

Hi Matteo,
The best source to look for the format of the events in the ring
buffer and how to parse them is
https://github.com/rostedt/libtraceevent/ , and especially the file
https://github.com/rostedt/libtraceevent/blob/libtraceevent/src/kbuffer-parse.c
. The format of the data in the u32 array[] that you mentioned is
event specific and is described in its format file, for example look
at the events/sched/sched_wakeup/format for sched_wakeup event data,
in the tracefs directory. Notice that the first 4 fields of all
events are the same - those with "common_" prefix.

> In addition to this, I didn't understand how I can provide multiple entries.

The events in the ring buffer are recorded just one after another. As
you can find the size of each recorded event when reading and parsing
the data, it is easy to jump to the next one. Be aware that pages of
the ring buffer are always aligned to the size of the system page.
Note that there are also some special events, with types described in
the `header_event` file.

>
> Thanks for you possible help,
> Matteo
>

-- 
Tzvetomir (Ceco) Stoyanov
VMware Open Source Technology Center