All of lore.kernel.org
 help / color / mirror / Atom feed
From: Masami Hiramatsu <mhiramat@kernel.org>
To: Randy Dunlap <rdunlap@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>,
	Ingo Molnar <mingo@redhat.com>,
	Frank Rowand <frowand.list@gmail.com>,
	Namhyung Kim <namhyung@kernel.org>, Tim Bird <Tim.Bird@sony.com>,
	Jiri Olsa <jolsa@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Tom Zanussi <tom.zanussi@linux.intel.com>,
	Rob Herring <robh+dt@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Alexey Dobriyan <adobriyan@gmail.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v6 22/22] Documentation: tracing: Add boot-time tracing document
Date: Sun, 19 Jan 2020 23:15:40 +0900	[thread overview]
Message-ID: <20200119231540.eaebef64f8a69bc97f4abf25@kernel.org> (raw)
In-Reply-To: <da3c941a-f7a2-537e-9201-862450cb69d9@infradead.org>

Hi Randy,

Thank you for your comments!

On Sat, 18 Jan 2020 10:14:08 -0800
Randy Dunlap <rdunlap@infradead.org> wrote:

> Hi,
> 
> Here are a few editorial comments for you...
> 
> 
> On 1/10/20 8:07 AM, Masami Hiramatsu wrote:
> > Add a documentation about boot-time tracing options in
> > boot config.
> > 
> > Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> > ---
> >  Documentation/admin-guide/bootconfig.rst |    2 
> >  Documentation/trace/boottime-trace.rst   |  184 ++++++++++++++++++++++++++++++
> >  Documentation/trace/index.rst            |    1 
> >  3 files changed, 187 insertions(+)
> >  create mode 100644 Documentation/trace/boottime-trace.rst
> > 
> 
> > diff --git a/Documentation/trace/boottime-trace.rst b/Documentation/trace/boottime-trace.rst
> > new file mode 100644
> > index 000000000000..1d10fdebf1b2
> > --- /dev/null
> > +++ b/Documentation/trace/boottime-trace.rst
> > @@ -0,0 +1,184 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +=================
> > +Boot-time tracing
> > +=================
> > +
> > +:Author: Masami Hiramatsu <mhiramat@kernel.org>
> > +
> > +Overview
> > +========
> > +
> > +Boot-time tracing allows users to trace boot-time process including
> > +device initialization with full features of ftrace including per-event
> > +filter and actions, histograms, kprobe-events and synthetic-events,
> > +and trace instances.
> > +Since kernel cmdline is not enough to control these complex features,
> > +this uses bootconfig file to describe tracing feature programming.
> > +
> > +Options in the Boot Config
> > +==========================
> > +
> > +Here is the list of available options list for boot time tracing in
> > +boot config file [1]_. All options are under "ftrace." or "kernel."
> > +refix. See kernel parameters for the options which starts
> 
>    prefix.

Oops, OK.

> 
> > +with "kernel." prefix [2]_.
> > +
> > +.. [1] See :ref:`Documentation/admin-guide/bootconfig.rst <bootconfig>`
> > +.. [2] See :ref:`Documentation/admin-guide/kernel-parameters.rst <kernelparameters>`
> > +
> > +Ftrace Global Options
> > +---------------------
> > +
> > +Ftrace global options have "kernel." prefix in boot config, which means
> > +these options are passed as a part of kernel legacy command line.
> > +
> > +kernel.tp_printk
> > +   Output trace-event data on printk buffer too.
> > +
> > +kernel.dump_on_oops [= MODE]
> > +   Dump ftrace on Oops. If MODE = 1 or omitted, dump trace buffer
> > +   on all CPUs. If MODE = 2, dump a buffer on a CPU which kicks Oops.
> > +
> > +kernel.traceoff_on_warning
> > +   Stop tracing if WARN_ON() occurs.
> > +
> > +kernel.fgraph_max_depth = MAX_DEPTH
> > +   Set MAX_DEPTH to maximum depth of fgraph tracer.
> > +
> > +kernel.fgraph_filters = FILTER[, FILTER2...]
> > +   Add fgraph tracing function filters.
> > +
> > +kernel.fgraph_notraces = FILTER[, FILTER2...]
> > +   Add fgraph non tracing function filters.
> 
>                  non-tracing

OK.

> 
> > +
> > +
> > +Ftrace Per-instance Options
> > +---------------------------
> > +
> > +These options can be used for each instance including global ftrace node.
> > +
> > +ftrace.[instance.INSTANCE.]options = OPT1[, OPT2[...]]
> > +   Enable given ftrace options.
> > +
> > +ftrace.[instance.INSTANCE.]trace_clock = CLOCK
> > +   Set given CLOCK to ftrace's trace_clock.
> > +
> > +ftrace.[instance.INSTANCE.]buffer_size = SIZE
> > +   Configure ftrace buffer size to SIZE. You can use "KB" or "MB"
> > +   for that SIZE.
> > +
> > +ftrace.[instance.INSTANCE.]alloc_snapshot
> > +   Allocate snapshot buffer.
> > +
> > +ftrace.[instance.INSTANCE.]cpumask = CPUMASK
> > +   Set CPUMASK as trace cpu-mask.
> > +
> > +ftrace.[instance.INSTANCE.]events = EVENT[, EVENT2[...]]
> > +   Enable given events on boot. You can use a wild card in EVENT.
> > +
> > +ftrace.[instance.INSTANCE.]tracer = TRACER
> > +   Set TRACER to current tracer on boot. (e.g. function)
> > +
> > +ftrace.[instance.INSTANCE.]ftrace.filters
> > +   This will take an array of tracing function filter rules
> 
> end with '.' as above descriptions.

Yes, I missed it.

> 
> > +
> > +ftrace.[instance.INSTANCE.]ftrace.notraces
> > +   This will take an array of NON-tracing function filter rules
> 
> ditto

OK.

> 
> > +
> > +
> > +Ftrace Per-Event Options
> > +------------------------
> > +
> > +These options are setting per-event options.
> > +
> > +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.enable
> > +   Enables GROUP:EVENT tracing.
> 
>       Enable

OK.

> 
> > +
> > +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.filter = FILTER
> > +   Set FILTER rule to the GROUP:EVENT.
> > +
> > +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.actions = ACTION[, ACTION2[...]]
> > +   Set ACTIONs to the GROUP:EVENT.
> > +
> > +ftrace.[instance.INSTANCE.]event.kprobes.EVENT.probes = PROBE[, PROBE2[...]]
> > +   Defines new kprobe event based on PROBEs. It is able to define
> > +   multiple probes on one event, but those must have same type of
> > +   arguments. This option is available only for the event which
> > +   group name is "kprobes".
> > +
> > +ftrace.[instance.INSTANCE.]event.synthetic.EVENT.fields = FIELD[, FIELD2[...]]
> > +   Defines new synthetic event with FIELDs. Each field should be
> > +   "type varname".
> > +
> > +Note that kprobe and synthetic event definitions can be written under
> > +instance node, but those are also visible from other instances. So please
> > +take care for event name conflict.
> > +
> > +
> > +Examples
> > +========
> > +
> > +For example, to add filter and actions for each event, define kprobe
> > +events, and synthetic events with histogram, write a boot config like
> > +below::
> > +
> > +  ftrace.event {
> > +        task.task_newtask {
> > +                filter = "pid < 128"
> > +                enable
> > +        }
> > +        kprobes.vfs_read {
> > +                probes = "vfs_read $arg1 $arg2"
> > +                filter = "common_pid < 200"
> > +                enable
> > +        }
> > +        synthetic.initcall_latency {
> > +                fields = "unsigned long func", "u64 lat"
> > +                actions = "hist:keys=func.sym,lat:vals=lat:sort=lat"
> > +        }
> > +        initcall.initcall_start {
> > +                actions = "hist:keys=func:ts0=common_timestamp.usecs"
> > +        }
> > +        initcall.initcall_finish {
> > +                actions = "hist:keys=func:lat=common_timestamp.usecs-$ts0:onmatch(initcall.initcall_start).initcall_latency(func,$lat)"
> > +        }
> > +  }
> > +
> > +Also, boottime tracing supports "instance" node, which allows us to run
> 
>          boot-time  [for consistency]

OK.

> 
> > +several tracers for different purpose at once. For example, one tracer
> > +is for tracing functions start with "user\_", and others tracing "kernel\_"
> 
>                             starting

OK.

> 
> > +functions, you can write boot config as below::
> > +
> > +  ftrace.instance {
> > +        foo {
> > +                tracer = "function"
> > +                ftrace.filters = "user_*"
> > +        }
> > +        bar {
> > +                tracer = "function"
> > +                ftrace.filters = "kernel_*"
> > +        }
> > +  }
> > +
> > +The instance node also accepts event nodes so that each instance
> > +can customize its event tracing.
> > +
> > +This boot-time tracing also supports ftrace kernel parameters via boot
> > +config.
> > +For example, following kernel parameters::
> > +
> > + trace_options=sym-addr trace_event=initcall:* tp_printk trace_buf_size=1M ftrace=function ftrace_filter="vfs*"
> > +
> > +This can be written in boot config like below::
> > +
> > +  kernel {
> > +        trace_options = sym-addr
> > +        trace_event = "initcall:*"
> > +        tp_printk
> > +        trace_buf_size = 1M
> > +        ftrace = function
> > +        ftrace_filter = "vfs*"
> > +  }
> > +
> > +Note that parameters start with "kernel" prefix instead of "ftrace".
> 
> HTH.

Very helpful. Thanks!


-- 
Masami Hiramatsu <mhiramat@kernel.org>

  reply	other threads:[~2020-01-19 14:15 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-10 16:03 [PATCH v6 00/22] tracing: bootconfig: Boot-time tracing and Extra boot config Masami Hiramatsu
2020-01-10 16:03 ` [PATCH v6 01/22] bootconfig: Add Extra Boot Config support Masami Hiramatsu
2020-01-18 18:33   ` Randy Dunlap
2020-01-19 12:23     ` Masami Hiramatsu
2020-01-10 16:03 ` [PATCH v6 02/22] bootconfig: Load boot config from the tail of initrd Masami Hiramatsu
2020-01-10 16:03 ` [PATCH v6 03/22] tools: bootconfig: Add bootconfig command Masami Hiramatsu
2020-01-10 16:04 ` [PATCH v6 04/22] tools: bootconfig: Add bootconfig test script Masami Hiramatsu
2020-01-10 16:04 ` [PATCH v6 05/22] proc: bootconfig: Add /proc/bootconfig to show boot config list Masami Hiramatsu
2020-01-10 16:04 ` [PATCH v6 06/22] init/main.c: Alloc initcall_command_line in do_initcall() and free it Masami Hiramatsu
2020-01-10 16:04 ` [PATCH v6 07/22] bootconfig: init: Allow admin to use bootconfig for kernel command line Masami Hiramatsu
2020-01-10 16:04 ` [PATCH v6 08/22] bootconfig: init: Allow admin to use bootconfig for init " Masami Hiramatsu
2020-02-07 18:03   ` Kees Cook
2020-02-07 19:31     ` Arvind Sankar
2020-02-07 19:46     ` Steven Rostedt
2020-02-08  0:44       ` Kees Cook
2020-08-02  2:33       ` Arvind Sankar
2020-08-03 15:03         ` Masami Hiramatsu
2020-08-03 15:29           ` Arvind Sankar
2020-08-03 17:22             ` Steven Rostedt
2020-08-04  0:29               ` Masami Hiramatsu
2020-01-10 16:05 ` [PATCH v6 09/22] Documentation: bootconfig: Add a doc for extended boot config Masami Hiramatsu
2020-01-18 18:28   ` Randy Dunlap
2020-01-19 13:36     ` Masami Hiramatsu
2020-01-10 16:05 ` [PATCH v6 10/22] tracing: Apply soft-disabled and filter to tracepoints printk Masami Hiramatsu
2020-01-10 16:05 ` [PATCH v6 11/22] tracing: kprobes: Output kprobe event to printk buffer Masami Hiramatsu
2020-01-10 16:05 ` [PATCH v6 12/22] tracing: kprobes: Register to dynevent earlier stage Masami Hiramatsu
2020-01-10 16:05 ` [PATCH v6 13/22] tracing: Accept different type for synthetic event fields Masami Hiramatsu
2020-01-10 16:06 ` [PATCH v6 14/22] tracing: Add NULL trace-array check in print_synth_event() Masami Hiramatsu
2020-01-10 16:06 ` [PATCH v6 15/22] tracing/boot: Add boot-time tracing Masami Hiramatsu
2020-01-10 16:06 ` [PATCH v6 16/22] tracing/boot: Add per-event settings Masami Hiramatsu
2020-01-10 16:06 ` [PATCH v6 17/22] tracing/boot Add kprobe event support Masami Hiramatsu
2020-01-10 16:06 ` [PATCH v6 18/22] tracing/boot: Add synthetic " Masami Hiramatsu
2020-01-10 16:07 ` [PATCH v6 19/22] tracing/boot: Add instance node support Masami Hiramatsu
2020-01-10 16:07 ` [PATCH v6 20/22] tracing/boot: Add cpu_mask option support Masami Hiramatsu
2020-01-10 16:07 ` [PATCH v6 21/22] tracing/boot: Add function tracer filter options Masami Hiramatsu
2020-01-10 16:07 ` [PATCH v6 22/22] Documentation: tracing: Add boot-time tracing document Masami Hiramatsu
2020-01-18 18:14   ` Randy Dunlap
2020-01-19 14:15     ` Masami Hiramatsu [this message]
2020-01-19 14:20 ` [PATCH v6 00/22] tracing: bootconfig: Boot-time tracing and Extra boot config Masami Hiramatsu
2020-01-19 14:59   ` Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200119231540.eaebef64f8a69bc97f4abf25@kernel.org \
    --to=mhiramat@kernel.org \
    --cc=Tim.Bird@sony.com \
    --cc=acme@kernel.org \
    --cc=adobriyan@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=corbet@lwn.net \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jolsa@redhat.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=robh+dt@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    --cc=tom.zanussi@linux.intel.com \
    --cc=torvalds@linux-foundation.org \
    --subject='Re: [PATCH v6 22/22] Documentation: tracing: Add boot-time tracing document' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.