Move "load_plugins" out of the opening code

Move "load_plugins" out of the opening code

[Steven Rostedt]

Tzvetomir,

From bugzilla:

  https://bugzilla.kernel.org/show_bug.cgi?id=211255

It was reported that:

$ trace-cmd report -N -r funcgraph_exit trace.dat 2>/dev/null | grep funcgraph_exit | wc -l
11645

$ trace-cmd report -N -F funcgraph_exit -r funcgraph_exit trace.dat 2>/dev/null | grep funcgraph_exit | wc -l
24826

Where the first should have been as big as the second. The problem with the
above is really that "-N" does not stop the ftrace function event plugins
from being loaded.

Looking at this further, I think it's a mistake to have the plugins loaded
at opening time.

We should have a new function for the library:

int tracecmd_load_plugins(struct tracecmd_handle *handle, int flags);

And this should be called after read_trace_header() in trace-read.c.

We can still have tracecmd_open_head() call this, as the "open_head" is
basically a "do everything" function.

This will allow us to remove the global "tracecmd_disable_plugins" and
"tracecmd_disable_sys_plugins" variables and use the flags input instead.
With the '-N' option, we should probably just skip calling it regardless.

The below patch was just a POC, to see if there was any side-effects for
this. But it didn't have a return value or flags, which should be added for
the final version.

-- Steve

diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c
index 3e67dd61..11bf8e5d 100644
--- a/lib/trace-cmd/trace-input.c
+++ b/lib/trace-cmd/trace-input.c
@@ -3247,11 +3247,6 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd)
 	if (!handle->pevent)
 		goto failed_read;
 
-	/* register default ftrace functions first */
-	tracecmd_ftrace_overrides(handle, &handle->finfo);
-
-	handle->plugin_list = trace_load_plugins(handle->pevent);
-
 	tep_set_file_bigendian(handle->pevent, buf[0]);
 	tep_set_local_bigendian(handle->pevent, tracecmd_host_bigendian());
 
@@ -3344,6 +3339,13 @@ struct tracecmd_input *tracecmd_open(const char *file)
 	return tracecmd_open_fd(fd);
 }
 
+void tracecmd_load_plugins(struct tracecmd_input *handle)
+{
+	/* register default ftrace functions first */
+	tracecmd_ftrace_overrides(handle, &handle->finfo);
+	handle->plugin_list = trace_load_plugins(handle->pevent);
+}
+
 /**
  * tracecmd_open_head - create a tracecmd_handle from a given file, read
  *			and parse only the trace headers from the file
diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c
index d07f4efe..a79bf0aa 100644
--- a/tracecmd/trace-read.c
+++ b/tracecmd/trace-read.c
@@ -1768,10 +1768,14 @@ void trace_report (int argc, char **argv)
 		die("Wakeup tracing can only be done on a single input file");
 
 	list_for_each_entry(inputs, &input_files, list) {
+		void tracecmd_load_plugins(struct tracecmd_input *handle);
 		handle = read_trace_header(inputs->file);
 		if (!handle)
 			die("error reading header for %s", inputs->file);
 
+		if (!tracecmd_disable_plugins)
+			tracecmd_load_plugins(handle);
+
 		/* If used with instances, top instance will have no tag */
 		add_handle(handle, multi_inputs ? inputs->file : NULL);
 

Re: Move "load_plugins" out of the opening code

[Tzvetomir Stoyanov]

On Tue, Jan 19, 2021 at 7:57 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> Tzvetomir,
>
> From bugzilla:
>
>   https://bugzilla.kernel.org/show_bug.cgi?id=211255
>
> It was reported that:
>
> $ trace-cmd report -N -r funcgraph_exit trace.dat 2>/dev/null | grep funcgraph_exit | wc -l
> 11645
>
> $ trace-cmd report -N -F funcgraph_exit -r funcgraph_exit trace.dat 2>/dev/null | grep funcgraph_exit | wc -l
> 24826
>
> Where the first should have been as big as the second. The problem with the
> above is really that "-N" does not stop the ftrace function event plugins
> from being loaded.
>
> Looking at this further, I think it's a mistake to have the plugins loaded
> at opening time.
>
> We should have a new function for the library:
>
> int tracecmd_load_plugins(struct tracecmd_handle *handle, int flags);
>
> And this should be called after read_trace_header() in trace-read.c.
>
> We can still have tracecmd_open_head() call this, as the "open_head" is
> basically a "do everything" function.
>
> This will allow us to remove the global "tracecmd_disable_plugins" and
> "tracecmd_disable_sys_plugins" variables and use the flags input instead.
> With the '-N' option, we should probably just skip calling it regardless.
>
> The below patch was just a POC, to see if there was any side-effects for
> this. But it didn't have a return value or flags, which should be added for
> the final version.
>

Hi Steven,
I sent a patch with a different implementation of this fix. I decided to
reimplement it because of two reasons:
 - There is already a function tracecmd_load_plugins(), which is supposed
   to load trace-cmd specific plugins. As there are no such plugins for now,
   this function is a dead code and is not called. However, it is
still part of the
   libtracecmd.
 - Moving that logic out of tracecmd_alloc_fd() will break these APIs:
    tracecmd_open()
    tracecmd_open_fd()
    tracecmd_open_head()
 KernelShark relies on them for reading trace.dat files.  We can think of
 moving plugins loading in a separate API, which must be called at the
 library initialisation phase, but this will require changes in the current
library users (KernelShark?).

> -- Steve
>
> diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c
> index 3e67dd61..11bf8e5d 100644
> --- a/lib/trace-cmd/trace-input.c
> +++ b/lib/trace-cmd/trace-input.c
> @@ -3247,11 +3247,6 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd)
>         if (!handle->pevent)
>                 goto failed_read;
>
> -       /* register default ftrace functions first */
> -       tracecmd_ftrace_overrides(handle, &handle->finfo);
> -
> -       handle->plugin_list = trace_load_plugins(handle->pevent);
> -
>         tep_set_file_bigendian(handle->pevent, buf[0]);
>         tep_set_local_bigendian(handle->pevent, tracecmd_host_bigendian());
>
> @@ -3344,6 +3339,13 @@ struct tracecmd_input *tracecmd_open(const char *file)
>         return tracecmd_open_fd(fd);
>  }
>
> +void tracecmd_load_plugins(struct tracecmd_input *handle)
> +{
> +       /* register default ftrace functions first */
> +       tracecmd_ftrace_overrides(handle, &handle->finfo);
> +       handle->plugin_list = trace_load_plugins(handle->pevent);
> +}
> +
>  /**
>   * tracecmd_open_head - create a tracecmd_handle from a given file, read
>   *                     and parse only the trace headers from the file
> diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c
> index d07f4efe..a79bf0aa 100644
> --- a/tracecmd/trace-read.c
> +++ b/tracecmd/trace-read.c
> @@ -1768,10 +1768,14 @@ void trace_report (int argc, char **argv)
>                 die("Wakeup tracing can only be done on a single input file");
>
>         list_for_each_entry(inputs, &input_files, list) {
> +               void tracecmd_load_plugins(struct tracecmd_input *handle);
>                 handle = read_trace_header(inputs->file);
>                 if (!handle)
>                         die("error reading header for %s", inputs->file);
>
> +               if (!tracecmd_disable_plugins)
> +                       tracecmd_load_plugins(handle);
> +
>                 /* If used with instances, top instance will have no tag */
>                 add_handle(handle, multi_inputs ? inputs->file : NULL);
>


-- 
Tzvetomir (Ceco) Stoyanov
VMware Open Source Technology Center

Re: Move "load_plugins" out of the opening code

[Steven Rostedt]

On Thu, 4 Feb 2021 13:04:01 +0200
Tzvetomir Stoyanov <tz.stoyanov@gmail.com> wrote:

>  - Moving that logic out of tracecmd_alloc_fd() will break these APIs:
>     tracecmd_open()
>     tracecmd_open_fd()
>     tracecmd_open_head()

Note, because libtracecmd has not been released yet, we still have the
luxury of breaking its APIs ;-)

>  KernelShark relies on them for reading trace.dat files.  We can think of
>  moving plugins loading in a separate API, which must be called at the
>  library initialisation phase, but this will require changes in the current
> library users (KernelShark?).

For libtracecmd, it is still OK to change the APIs even if KernelShark
breaks. As the only KernelShark user is still in the trace-cmd repo, it
should be OK. Once we have v2 out, then we need to have these APIs frozen.
But until then, let's break the API if it makes it a better design.

-- Steve

Re: Move "load_plugins" out of the opening code

[Steven Rostedt]

On Thu, 4 Feb 2021 09:17:27 -0500
Steven Rostedt <rostedt@goodmis.org> wrote:

> Note, because libtracecmd has not been released yet, we still have the
> luxury of breaking its APIs ;-)

Also, the reason I have not yet released libtracecmd, is because I want to
change the current APIs. Even if it breaks KernelShark.

-- Steve