linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 00/32] selftests/ftrace: Improve ftracetest with coverage check
@ 2018-08-16 16:29 Masami Hiramatsu
  2018-08-16 16:29 ` [PATCH v2 01/32] tracing: Allow gcov profiling on only ftrace subsystem Masami Hiramatsu
                   ` (31 more replies)
  0 siblings, 32 replies; 78+ messages in thread
From: Masami Hiramatsu @ 2018-08-16 16:29 UTC (permalink / raw)
  To: Shuah Khan, Steven Rostedt
  Cc: Ingo Molnar, Masami Hiramatsu, linux-kselftest, linux-kernel

Hello,

Here is the 2nd version of the series for improving ftracetest
testcase using gcov/lcov. From previous version, I've added 6
new testcases for some tracers and ftrace basic features.

With this series, ftracetest coverage under kernel/trace/
is improved as below. (Note that this kernel enables
ftrace startup tests)

Without new testcases
Lines:		10563	17268	61.2 %
Functions:	1114	1785	62.4 %

With new testcases
Lines:		11703	17268	67.8 %
Functions:	1262	1785	70.7 %

Of course we don't need to make it 100%, because ftracetest
is a function/specification test, and there are functions and
lines which never be called (like critical error detection :) ).
Also, ftrace has some amount of code for perf and bpf subsystem
which should be tested by perf/bpf tests.
However, GCOV(LCOV) gives us a good insight into which
functions are covered or not covered by the test cases.
We can find which functions are not tested yet from the
coverage information.

This series has some ftracetest framework improvements too,
which are for debugging ftracetest itself.

Tracing/Gcov-kernel:
 - Add CONFIG_GCOV_PROFILE_FRACE, which enables gcov profiling
   only on tracing subsystem (under kernel/trace/). This may not
   needed for everyone (like a debugging feature).
   So it can be dropped anyway.

Ftracetest framework improvements:
 - Add --stop-fail for detecting failure soon.
 - Add --console for interactively debugging a testcase by shell.
 - Add testcase-number prefix to logfile for finding log easier.
 - Improve general init function
 - Call general init function after all testcases.
 - Remove init/cleanup code from all testcase (since general init
   function will fixed it up)

Testcase fixes:
 - Fix to check $comm availability.
 - Make checkbashisms clean

Testcase improvements:
 - Use loopback address instead of localhost
 - Improve kprobe on module testcase to load/unload module
 - Improve kprobe testcase to check log data
 - Improve kretprobe testcase to check log data
 - Test kprobe-event argument with various bitsize
 - Check set_event_pid result

Adding testcases:
 - Add kprobe event with $comm argument testcase
 - Add kprobe profile testcase
 - Add kprobe-event with symbol argument testcase
 - Add nop tracer testcase
 - Add trace_printk sample module testcase
 - Add ringbuffer size changing testcase
 - Add function profiling statistics testcase
 - Add max stack tracer testcase
 - Add function filter on module testcase
 - Add trace_pipe testcase
 - Add stacktrace ftrace filter command testcase
 - Add wakeup tracer testcase
 - Add wakeup_rt tracer testcase
 - Add blktrace testcase
 - Add ftrace cpumask testcase
 - Add ftrace output format testcase 

TBD:
Following testcase improvements are remaining.
 - Checking of various supported types (like u8, u64 etc.)
   testcases

Thank you,

---

Masami Hiramatsu (32):
      tracing: Allow gcov profiling on only ftrace subsystem
      selftests/ftrace: Add --stop-fail hidden option for debug
      selftests/ftrace: Add --console hidden option
      selftests/ftrace: Add case number prefix to logfile
      selftests/ftrace: More initialize features in initialize_ftrace
      selftests/ftrace: Add SPDX License Identifier to template
      selftests/ftrace: Cleanup ftrace after running test
      selftests/ftrace: Remove unneeded per-test init/cleanup ftrace
      selftests/ftrace: Fix to test kprobe $comm arg only if available
      selftests/ftrace: Fix checkbashisms errors
      selftests/ftrace: Use loopback address instead of localhost
      selftests/ftrace: Improve kprobe on module testcase to load/unload module
      selftests/ftrace: Improve kprobe testcase to check log data
      selftests/ftrace: Improve kretprobe testcase to check log data
      selftests/ftrace: Test kprobe-event argument with various bitsize
      selftests/ftrace: Check set_event_pid result
      selftests/ftrace: Add kprobe event with $comm argument testcase
      selftests/ftrace: Add kprobe profile testcase
      selftests/ftrace: Add a testcase for nop tracer
      selftests/ftrace: Add kprobe-event with symbol argument test
      selftests/ftrace: Add trace_printk sample module test
      selftests/ftrace: Add ringbuffer size changing testcase
      selftests/ftrace: Add function profiling stat testcase
      selftests/ftrace: Add max stack tracer testcase
      selftests/ftrace: Add function filter on module testcase
      selftests/ftrace: Add trace_pipe testcase
      selftests/ftrace: Add stacktrace ftrace filter command testcase
      selftests/ftrace: Add wakeup tracer testcase
      selftests/ftrace: Add wakeup_rt tracer testcase
      selftests/ftrace: Add ftrace cpumask testcase
      selftests/ftrace: Add output format testcase
      selftests/ftrace: Add blktrace testcase


 kernel/trace/Kconfig                               |   13 +++
 kernel/trace/Makefile                              |    5 +
 tools/testing/selftests/ftrace/config              |    7 ++
 tools/testing/selftests/ftrace/ftracetest          |   32 ++++++-
 .../ftrace/test.d/00basic/print_format.tc          |   51 +++++++++++
 .../ftrace/test.d/00basic/ringbuffer_size.tc       |   21 +++++
 .../selftests/ftrace/test.d/00basic/trace_pipe.tc  |   15 +++
 .../selftests/ftrace/test.d/event/event-enable.tc  |    8 --
 .../selftests/ftrace/test.d/event/event-pid.tc     |    6 +
 .../ftrace/test.d/event/subsystem-enable.tc        |    8 --
 .../ftrace/test.d/event/toplevel-enable.tc         |    8 --
 .../selftests/ftrace/test.d/event/trace_printk.tc  |   27 ++++++
 .../ftrace/test.d/ftrace/fgraph-filter-stack.tc    |    4 -
 .../ftrace/test.d/ftrace/fgraph-filter.tc          |    9 --
 .../ftrace/test.d/ftrace/func-filter-pid.tc        |    8 --
 .../ftrace/test.d/ftrace/func-filter-stacktrace.tc |   12 +++
 .../selftests/ftrace/test.d/ftrace/func_cpumask.tc |   34 +++++++
 .../ftrace/test.d/ftrace/func_event_triggers.tc    |    3 -
 .../ftrace/test.d/ftrace/func_mod_trace.tc         |   24 +++++
 .../ftrace/test.d/ftrace/func_profile_stat.tc      |   23 +++++
 .../ftrace/test.d/ftrace/func_profiler.tc          |    4 -
 .../ftrace/test.d/ftrace/func_set_ftrace_file.tc   |   13 ---
 .../ftrace/test.d/ftrace/func_stack_tracer.tc      |   39 +++++++++
 .../test.d/ftrace/func_traceonoff_triggers.tc      |   11 --
 tools/testing/selftests/ftrace/test.d/functions    |    5 +
 .../ftrace/test.d/kprobe/add_and_remove.tc         |    3 -
 .../selftests/ftrace/test.d/kprobe/busy_check.tc   |    3 -
 .../selftests/ftrace/test.d/kprobe/kprobe_args.tc  |    8 +-
 .../ftrace/test.d/kprobe/kprobe_args_comm.tc       |   17 ++++
 .../ftrace/test.d/kprobe/kprobe_args_string.tc     |    5 -
 .../ftrace/test.d/kprobe/kprobe_args_symbol.tc     |   39 +++++++++
 .../ftrace/test.d/kprobe/kprobe_args_syntax.tc     |    8 +-
 .../ftrace/test.d/kprobe/kprobe_args_type.tc       |   52 +++++++----
 .../ftrace/test.d/kprobe/kprobe_eventname.tc       |    5 -
 .../ftrace/test.d/kprobe/kprobe_ftrace.tc          |    9 --
 .../ftrace/test.d/kprobe/kprobe_module.tc          |   36 +++++++-
 .../ftrace/test.d/kprobe/kretprobe_args.tc         |    9 +-
 .../ftrace/test.d/kprobe/kretprobe_maxactive.tc    |    4 -
 .../ftrace/test.d/kprobe/multiple_kprobes.tc       |    5 -
 .../selftests/ftrace/test.d/kprobe/probepoint.tc   |    4 -
 .../selftests/ftrace/test.d/kprobe/profile.tc      |   15 +++
 tools/testing/selftests/ftrace/test.d/template     |    1 
 .../testing/selftests/ftrace/test.d/tracer/blk.tc  |   91 ++++++++++++++++++++
 .../testing/selftests/ftrace/test.d/tracer/nop.tc  |   22 +++++
 .../selftests/ftrace/test.d/tracer/wakeup.tc       |   25 +++++
 .../selftests/ftrace/test.d/tracer/wakeup_rt.tc    |   25 +++++
 .../inter-event/trigger-extended-error-support.tc  |   12 ---
 .../inter-event/trigger-field-variable-support.tc  |   15 ---
 .../trigger-inter-event-combined-hist.tc           |   15 ---
 .../inter-event/trigger-multi-actions-accept.tc    |   14 ---
 .../inter-event/trigger-onmatch-action-hist.tc     |   15 ---
 .../trigger-onmatch-onmax-action-hist.tc           |   15 ---
 .../inter-event/trigger-onmax-action-hist.tc       |   15 ---
 .../trigger-synthetic-event-createremove.tc        |   12 ---
 .../ftrace/test.d/trigger/trigger-eventonoff.tc    |   12 ---
 .../ftrace/test.d/trigger/trigger-filter.tc        |   14 ---
 .../ftrace/test.d/trigger/trigger-hist-mod.tc      |   12 ---
 .../ftrace/test.d/trigger/trigger-hist.tc          |   12 ---
 .../ftrace/test.d/trigger/trigger-multihist.tc     |   16 ----
 .../ftrace/test.d/trigger/trigger-snapshot.tc      |   12 ---
 .../ftrace/test.d/trigger/trigger-stacktrace.tc    |   12 ---
 .../test.d/trigger/trigger-trace-marker-hist.tc    |   11 --
 .../trigger/trigger-trace-marker-snapshot.tc       |   16 ----
 .../trigger-trace-marker-synthetic-kernel.tc       |   12 ---
 .../trigger/trigger-trace-marker-synthetic.tc      |   12 ---
 .../ftrace/test.d/trigger/trigger-traceonoff.tc    |   12 ---
 66 files changed, 630 insertions(+), 412 deletions(-)
 create mode 100644 tools/testing/selftests/ftrace/test.d/00basic/print_format.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_size.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/00basic/trace_pipe.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/event/trace_printk.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/func-filter-stacktrace.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/func_cpumask.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/func_profile_stat.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/func_stack_tracer.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_args_comm.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_args_symbol.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/profile.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/tracer/blk.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/tracer/nop.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/tracer/wakeup.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/tracer/wakeup_rt.tc

--
Masami Hiramatsu (Linaro)

^ permalink raw reply	[flat|nested] 78+ messages in thread

end of thread, other threads:[~2018-08-30 23:40 UTC | newest]

Thread overview: 78+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-16 16:29 [PATCH v2 00/32] selftests/ftrace: Improve ftracetest with coverage check Masami Hiramatsu
2018-08-16 16:29 ` [PATCH v2 01/32] tracing: Allow gcov profiling on only ftrace subsystem Masami Hiramatsu
2018-08-20 15:32   ` Steven Rostedt
2018-08-20 15:33     ` Steven Rostedt
2018-08-21  6:53       ` Masami Hiramatsu
2018-08-21  7:27       ` [PATCH v2.1] " Masami Hiramatsu
2018-08-21 13:12         ` Steven Rostedt
2018-08-16 16:30 ` [PATCH v2 02/32] selftests/ftrace: Add --stop-fail hidden option for debug Masami Hiramatsu
2018-08-24 20:38   ` Steven Rostedt
2018-08-16 16:30 ` [PATCH v2 03/32] selftests/ftrace: Add --console hidden option Masami Hiramatsu
2018-08-24 20:37   ` Steven Rostedt
2018-08-25  1:35     ` Masami Hiramatsu
2018-08-25  1:42       ` Steven Rostedt
2018-08-26  7:28         ` Masami Hiramatsu
2018-08-16 16:31 ` [PATCH v2 04/32] selftests/ftrace: Add case number prefix to logfile Masami Hiramatsu
2018-08-24 21:47   ` Steven Rostedt
2018-08-16 16:31 ` [PATCH v2 05/32] selftests/ftrace: More initialize features in initialize_ftrace Masami Hiramatsu
2018-08-24 21:24   ` Steven Rostedt
2018-08-24 21:46   ` Steven Rostedt
2018-08-24 21:46     ` Steven Rostedt
2018-08-16 16:32 ` [PATCH v2 06/32] selftests/ftrace: Add SPDX License Identifier to template Masami Hiramatsu
2018-08-16 16:32 ` [PATCH v2 07/32] selftests/ftrace: Cleanup ftrace after running test Masami Hiramatsu
2018-08-16 16:32 ` [PATCH v2 08/32] selftests/ftrace: Remove unneeded per-test init/cleanup ftrace Masami Hiramatsu
2018-08-16 16:33 ` [PATCH v2 09/32] selftests/ftrace: Fix to test kprobe $comm arg only if available Masami Hiramatsu
2018-08-16 16:33 ` [PATCH v2 10/32] selftests/ftrace: Fix checkbashisms errors Masami Hiramatsu
2018-08-24 21:50   ` Steven Rostedt
2018-08-16 16:34 ` [PATCH v2 11/32] selftests/ftrace: Use loopback address instead of localhost Masami Hiramatsu
2018-08-24 21:52   ` Steven Rostedt
2018-08-25  1:47     ` Masami Hiramatsu
2018-08-16 16:34 ` [PATCH v2 12/32] selftests/ftrace: Improve kprobe on module testcase to load/unload module Masami Hiramatsu
2018-08-24 23:07   ` Steven Rostedt
2018-08-16 16:35 ` [PATCH v2 13/32] selftests/ftrace: Improve kprobe testcase to check log data Masami Hiramatsu
2018-08-16 16:35 ` [PATCH v2 14/32] selftests/ftrace: Improve kretprobe " Masami Hiramatsu
2018-08-16 16:36 ` [PATCH v2 15/32] selftests/ftrace: Test kprobe-event argument with various bitsize Masami Hiramatsu
2018-08-16 16:36 ` [PATCH v2 16/32] selftests/ftrace: Check set_event_pid result Masami Hiramatsu
2018-08-16 16:37 ` [PATCH v2 17/32] selftests/ftrace: Add kprobe event with $comm argument testcase Masami Hiramatsu
2018-08-16 16:37 ` [PATCH v2 18/32] selftests/ftrace: Add kprobe profile testcase Masami Hiramatsu
2018-08-16 16:38 ` [PATCH v2 19/32] selftests/ftrace: Add a testcase for nop tracer Masami Hiramatsu
2018-08-24 22:53   ` Steven Rostedt
2018-08-26  7:30     ` Masami Hiramatsu
2018-08-16 16:38 ` [PATCH v2 20/32] selftests/ftrace: Add kprobe-event with symbol argument test Masami Hiramatsu
2018-08-16 16:39 ` [PATCH v2 21/32] selftests/ftrace: Add trace_printk sample module test Masami Hiramatsu
2018-08-24 23:06   ` Steven Rostedt
2018-08-26  8:43     ` Masami Hiramatsu
2018-08-16 16:39 ` [PATCH v2 22/32] selftests/ftrace: Add ringbuffer size changing testcase Masami Hiramatsu
2018-08-24 23:18   ` Steven Rostedt
2018-08-30  7:12     ` Masami Hiramatsu
2018-08-30 15:50       ` Steven Rostedt
2018-08-30 23:35         ` Masami Hiramatsu
2018-08-30 23:40           ` Steven Rostedt
2018-08-16 16:40 ` [PATCH v2 23/32] selftests/ftrace: Add function profiling stat testcase Masami Hiramatsu
2018-08-24 23:20   ` Steven Rostedt
2018-08-26 11:20     ` Masami Hiramatsu
2018-08-16 16:40 ` [PATCH v2 24/32] selftests/ftrace: Add max stack tracer testcase Masami Hiramatsu
2018-08-24 23:23   ` Steven Rostedt
2018-08-25  1:45     ` Masami Hiramatsu
2018-08-25  2:41       ` Steven Rostedt
2018-08-26  8:42         ` Masami Hiramatsu
2018-08-16 16:40 ` [PATCH v2 25/32] selftests/ftrace: Add function filter on module testcase Masami Hiramatsu
2018-08-24 23:25   ` Steven Rostedt
2018-08-16 16:41 ` [PATCH v2 26/32] selftests/ftrace: Add trace_pipe testcase Masami Hiramatsu
2018-08-24 23:26   ` Steven Rostedt
2018-08-26 11:20     ` Masami Hiramatsu
2018-08-16 16:41 ` [PATCH v2 27/32] selftests/ftrace: Add stacktrace ftrace filter command testcase Masami Hiramatsu
2018-08-24 23:28   ` Steven Rostedt
2018-08-16 16:42 ` [PATCH v2 28/32] selftests/ftrace: Add wakeup tracer testcase Masami Hiramatsu
2018-08-16 16:42 ` [PATCH v2 29/32] selftests/ftrace: Add wakeup_rt " Masami Hiramatsu
2018-08-25  2:09   ` Steven Rostedt
2018-08-26 11:22     ` Masami Hiramatsu
2018-08-16 16:43 ` [PATCH v2 30/32] selftests/ftrace: Add ftrace cpumask testcase Masami Hiramatsu
2018-08-25  2:18   ` Steven Rostedt
2018-08-27 12:16     ` Masami Hiramatsu
2018-08-16 16:43 ` [PATCH v2 31/32] selftests/ftrace: Add output format testcase Masami Hiramatsu
2018-08-25  2:19   ` Steven Rostedt
2018-08-30  7:48     ` Masami Hiramatsu
2018-08-16 16:44 ` [PATCH v2 32/32] selftests/ftrace: Add blktrace testcase Masami Hiramatsu
2018-08-25  2:22   ` Steven Rostedt
2018-08-30  7:53     ` Masami Hiramatsu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).