how to continuously capture events

how to continuously capture events

[Lin Wang]

Hi all,

I'm new to ftrace and linux programming in general. Forgive me for
asking dumb questions.

I'm trying to continuously capture certain kernel events, convert them
to a different format and then write them to a file.

I understand that trace_pipe should be used for streaming. But I'm not
sure how to read the content of trace_pipe at event boundary (I
currently just read it to a temp buffer which would sometimes cut the
last event in half). I discovered libtraceevent and libtracefs that I
think are meant to help with this kind of tasks, so I'm reading the
source code of trace-cmd to find examples. But so far the progress has
been slow.

Could anyone point me to the right direction, or advise me with a
general outline of what I should do to achieve my task?

Thanks,
Lin

Re: how to continuously capture events

[Steven Rostedt]

On Wed, 5 May 2021 08:21:35 -0400
Lin Wang <wanglinseven@gmail.com> wrote:

> Hi all,
> 
> I'm new to ftrace and linux programming in general. Forgive me for

Hi Lin,

Welcome!

> asking dumb questions.

When first learning something, there are no dumb questions :-)

> 
> I'm trying to continuously capture certain kernel events, convert them
> to a different format and then write them to a file.
> 
> I understand that trace_pipe should be used for streaming. But I'm not
> sure how to read the content of trace_pipe at event boundary (I
> currently just read it to a temp buffer which would sometimes cut the
> last event in half). I discovered libtraceevent and libtracefs that I
> think are meant to help with this kind of tasks, so I'm reading the
> source code of trace-cmd to find examples. But so far the progress has
> been slow.

Yes, libtracefs is what you want.

The man pages are here (I'm still working on a tutorial):

 https://trace-cmd.org/Documentation/libtracefs/libtracefs.html

Although that may be a little out of date. I need to automate that to be
updated whenever I make a new release.

> 
> Could anyone point me to the right direction, or advise me with a
> general outline of what I should do to achieve my task?
> 

I think you may be on the right track.

trace-cmd is hard to read, but we are working on sample code that will make
using libtracefs much easier. For example, I wrote this simple code to read
all files that are opened.

  # ./show-open-files cat /etc/passwd
42727-<...>: file=/etc/ld.so.cache flags=88000 mode=0
42727-<...>: :   addr=0x7f8900123868
42727-<...>: :   addr=0x7f89001100f7
42727-<...>: file=/lib64/libc.so.6 flags=88000 mode=0
42727-<...>: :   addr=0x7f8900123868
42727-<...>: :   addr=0x7f8900110139
42727-<...>: file=/usr/lib/locale/locale-archive flags=88000 mode=0
42727-<...>: :   addr=0x7f890000886c
42727-<...>: :   addr=0x7f88fff448ce
42727-<...>: :   addr=0x7f88fff44268
root:x:0:0:root:/root:/bin/bash
[..]
flatpak:x:963:962:User for flatpak system helper:/:/sbin/nologin
42727-<...>: file=/etc/passwd flags=8000 mode=0
42727-<...>: :   addr=0x7f890000319b
42727-<...>: :   addr=0x4c45485300647773


  http://rostedt.org/code/show-open-files.c

We are adding new API to libtracefs all the time to make it even easier to
access the tracefs file system. Feel free to subscribe to linux-trace-devel
if you want to participate or just want to see what is being worked on.

  http://vger.kernel.org/vger-lists.html#linux-trace-devel

If you have any more questions, feel free to ask!

-- Steve

Re: how to continuously capture events

[Jonathan Rajotte-Julien]

Hi,

> I'm trying to continuously capture certain kernel events, convert them
> to a different format and then write them to a file.

You might also want to have a look at LTTng and LTTng-modules depending on your
final use case. [1][2]

LTTng produce CTF traces [3] that can then be read/converted/manipulated with Babeltrace [4].

[1] https://lttng.org/
[2] https://lttng.org/docs/v2.12/#doc-tracing-the-linux-kernel
[3] https://diamon.org/ctf/
[4] https://babeltrace.org/

Cheers

-- 
Jonathan Rajotte-Julien
EfficiOS

Re: how to continuously capture events

[Lin Wang]

> > Hi all,
> >
> > I'm new to ftrace and linux programming in general. Forgive me for
>
> Hi Lin,
>
> Welcome!
>
> > asking dumb questions.
>
> When first learning something, there are no dumb questions :-)

Thank you, Steve.

> trace-cmd is hard to read, but we are working on sample code that will make
> using libtracefs much easier. For example, I wrote this simple code to read
> all files that are opened.

I will check out the example, and keep digging in trace-cmd.

Thanks,
-Lin

On Wed, May 5, 2021 at 9:18 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Wed, 5 May 2021 08:21:35 -0400
> Lin Wang <wanglinseven@gmail.com> wrote:
>
> > Hi all,
> >
> > I'm new to ftrace and linux programming in general. Forgive me for
>
> Hi Lin,
>
> Welcome!
>
> > asking dumb questions.
>
> When first learning something, there are no dumb questions :-)
>
> >
> > I'm trying to continuously capture certain kernel events, convert them
> > to a different format and then write them to a file.
> >
> > I understand that trace_pipe should be used for streaming. But I'm not
> > sure how to read the content of trace_pipe at event boundary (I
> > currently just read it to a temp buffer which would sometimes cut the
> > last event in half). I discovered libtraceevent and libtracefs that I
> > think are meant to help with this kind of tasks, so I'm reading the
> > source code of trace-cmd to find examples. But so far the progress has
> > been slow.
>
> Yes, libtracefs is what you want.
>
> The man pages are here (I'm still working on a tutorial):
>
>  https://trace-cmd.org/Documentation/libtracefs/libtracefs.html
>
> Although that may be a little out of date. I need to automate that to be
> updated whenever I make a new release.
>
> >
> > Could anyone point me to the right direction, or advise me with a
> > general outline of what I should do to achieve my task?
> >
>
> I think you may be on the right track.
>
> trace-cmd is hard to read, but we are working on sample code that will make
> using libtracefs much easier. For example, I wrote this simple code to read
> all files that are opened.
>
>   # ./show-open-files cat /etc/passwd
> 42727-<...>: file=/etc/ld.so.cache flags=88000 mode=0
> 42727-<...>: :   addr=0x7f8900123868
> 42727-<...>: :   addr=0x7f89001100f7
> 42727-<...>: file=/lib64/libc.so.6 flags=88000 mode=0
> 42727-<...>: :   addr=0x7f8900123868
> 42727-<...>: :   addr=0x7f8900110139
> 42727-<...>: file=/usr/lib/locale/locale-archive flags=88000 mode=0
> 42727-<...>: :   addr=0x7f890000886c
> 42727-<...>: :   addr=0x7f88fff448ce
> 42727-<...>: :   addr=0x7f88fff44268
> root:x:0:0:root:/root:/bin/bash
> [..]
> flatpak:x:963:962:User for flatpak system helper:/:/sbin/nologin
> 42727-<...>: file=/etc/passwd flags=8000 mode=0
> 42727-<...>: :   addr=0x7f890000319b
> 42727-<...>: :   addr=0x4c45485300647773
>
>
>   http://rostedt.org/code/show-open-files.c
>
> We are adding new API to libtracefs all the time to make it even easier to
> access the tracefs file system. Feel free to subscribe to linux-trace-devel
> if you want to participate or just want to see what is being worked on.
>
>   http://vger.kernel.org/vger-lists.html#linux-trace-devel
>
> If you have any more questions, feel free to ask!
>
> -- Steve

Re: how to continuously capture events

[Lin Wang]

> > I'm trying to continuously capture certain kernel events, convert them
> > to a different format and then write them to a file.
>
> You might also want to have a look at LTTng and LTTng-modules depending on your
> final use case. [1][2]

From a cursory look, it might be an overkill for what I want to
achieve. I will keep this in
mind in case I need to do something more complex. Thank you for the pointers.

-Lin

On Wed, May 5, 2021 at 9:47 AM Jonathan Rajotte-Julien
<jonathan.rajotte-julien@efficios.com> wrote:
>
> Hi,
>
> > I'm trying to continuously capture certain kernel events, convert them
> > to a different format and then write them to a file.
>
> You might also want to have a look at LTTng and LTTng-modules depending on your
> final use case. [1][2]
>
> LTTng produce CTF traces [3] that can then be read/converted/manipulated with Babeltrace [4].
>
> [1] https://lttng.org/
> [2] https://lttng.org/docs/v2.12/#doc-tracing-the-linux-kernel
> [3] https://diamon.org/ctf/
> [4] https://babeltrace.org/
>
> Cheers
>
> --
> Jonathan Rajotte-Julien
> EfficiOS