Re: [EXT] Re: [PATCH 2/6] qla2xxx: Add a generic tracing framework

[Arun Easi <aeasi@marvell.com>]

[-- Attachment #1: Type: text/plain, Size: 3074 bytes --]

On Mon, 18 Jul 2022, 12:22pm, Steven Rostedt wrote:

> External Email
> 
> ----------------------------------------------------------------------
> On Mon, 18 Jul 2022 12:02:26 -0700
> Arun Easi <aeasi@marvell.com> wrote:
> 
> > Many times when a problem was reported on the driver, we had to request 
> > for a repro with extended error logging (via driver module parameter) 
> > turned on. With this internal tracing in place, log messages that appear 
> > only with extended error logging are captured by default in the internal 
> > trace buffer.
> > 
> > AFAIK, kernel tracing requires a user initiated action to be turned on, 
> > like enabling individual tracepoints. Though a script (either startup or 
> > udev) can do this job, enabling tracepoints by default for a single 
> > driver, I think, may not be a preferred choice -- particularly when the 
> > trace buffer is shared across the kernel. That also brings the extra 
> > overhead of finding how this could be done across various distros.
> > 
> > For cases where the memory/driver size matters, there is an option to 
> > compile out this feature, plus choosing a lower default trace buffer size.
> 
> You can enable an ftrace instance from inside the kernel, and make it a
> compile time option.
> 
> 	#include <linux/trace_events.h>
> 	#include <linux/trace.h>
> 
> 	struct trace_array *tr;
> 
> 	tr = trace_array_get_by_name("qla2xxx");
> 	trace_array_set_clr_event(tr, "qla", NULL, true);
> 
> And now you have trace events running:
> 
>  # cat /sys/kernel/tracing/instances/qla/trace
> 

Thanks Steve. I was not aware of this relatively newer interface. This 
looks promising.

I have a question on the behavior of this interface.

It appears by calling the above two interfaces, I get a separate instance 
of "qla" only traces, but with only the "qla"-only instance being enabled, 
leaving the main (/sys/kernel/tracing/events/qla/enable) one disabled. No 
issues there, but when I enable both of them, I get garbage values on one.

/sys/kernel/tracing/instances/qla2xxx/trace:
             cat-56106   [012] ..... 2419873.470098: ql_dbg_log: qla2xxx [0000:05:00.0]-1054:14:  Entered (null).
             cat-56106   [012] ..... 2419873.470101: ql_dbg_log: qla2xxx [0000:05:00.0]-1000:14:  Entered ×+<96>²Ü<98>^H.
             cat-56106   [012] ..... 2419873.470102: ql_dbg_log: qla2xxx [0000:05:00.0]-1006:14:  Prepare to issue mbox cmd=0xde589000.

/sys/kernel/tracing/trace:
             cat-56106   [012] ..... 2419873.470097: ql_dbg_log: qla2xxx [0000:05:00.0]-1054:14:  Entered qla2x00_get_firmware_state.
             cat-56106   [012] ..... 2419873.470100: ql_dbg_log: qla2xxx [0000:05:00.0]-1000:14:  Entered qla2x00_mailbox_command.
             cat-56106   [012] ..... 2419873.470102: ql_dbg_log: qla2xxx [0000:05:00.0]-1006:14:  Prepare to issue mbox cmd=0x69.

It appears that only one should be enabled at a time. Per my read of 
Documentation/trace/ftrace.rst, the main directory and instances have 
separate trace buffers, so I am a bit confused with the above output.

Regards,
-Arun