[GIT PULL 00/31] perf tools: filtering events using eBPF programs

[GIT PULL 00/31] perf tools: filtering events using eBPF programs

[Wang Nan]

Hi Arnaldo,

   I know you don't have enough time to review my code. I send
this patchset to let you and other know what we are working on.

   In this new patchset, we create a new perf cmdline syntax so
perf users are able to pass perf events created by perf to BPF
maps, which makes bpf_perf_event_read() usable. Compare with out
previous solution[1] which embedded 'struct perf_event_attr' to
"maps" section, this solution is easier to use.

   If you or anyone have any different views on this solution,
please let us know so we can stop our further development base
on it as soon as possible.

Thank you.

 [1] http://lkml.kernel.org/r/1440672142-89311-1-git-send-email-xiakaixu@huawei.com

The following changes since commit 31eb4360546b4bd890f349db01295a173c09b0fb:

  perf hists browser: Add 'm' key for context menu display (2015-10-12 23:29:14 -0300)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pi3orama/linux.git tags/perf-ebpf-for-acme-20151014

for you to fetch changes up to 6df036cb7d42a2e0ebf312e127b02425bd57bc55:

  perf tools: Enable BPF object configure syntax (2015-10-14 10:09:17 +0000)

----------------------------------------------------------------
EBPF support for perf

 - Rebase to newest perf/core

 - Bugfix: kprobe events not remove if bpf__probe() failure occur
   after it creates some kprobe points successfully.

 - Bugfix: when multiple BPF functions reside in one BPF object,
   the last BPF program would be attached to all kprobe events.
   This bug is introduced by removal of dummy event placeholder.

 - New function: support BPF program reading counter through
   bpf_perf_event_read() by adding new syntax and support code
   in event selector. The new BPF object configuration mechanism
   can be extended to support BPF data output.

   In this patchset, following BPF function can be used to measure
   cycles a kernel function cost:

   ===== BPF program bpf_program.c =====

   struct bpf_map_def SEC("maps") pmu_map = {
       .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
       .key_size = sizeof(int),
       .value_size = sizeof(u32),
       .max_entries = __NR_CPUS__,
   };

   SEC("func_write=sys_write")
   int func_write(void *ctx)
   {
       unsigned long long val;
       char fmt[] = "sys_write:        pmu=%llu\n";
       val = bpf_perf_event_read(&pmu_map, bpf_get_smp_processor_id());
       bpf_trace_printk(fmt, sizeof(fmt), val);
       return 0;
   }

   SEC("func_write_return=sys_write%return")
   int func_write_return(void *ctx)
   {
       unsigned long long val = 0;
       char fmt[] = "sys_write_return: pmu=%llu\n";
       val = bpf_perf_event_read(&pmu_map, bpf_get_smp_processor_id());
       bpf_trace_printk(fmt, sizeof(fmt), val);
       return 0;
   }

   With cmdline like this:

   ===== cmdline =====
   # echo "" > /sys/kernel/debug/tracing/trace
   # perf record -e evt=cycles/period=0x7fffffffffffffff/ \
                 -e bpf_program.c/maps.pmu_map.event=evt/
                 -a ls
   # cat /sys/kernel/debug/tracing/trace | grep ls
                ls-3363  [003] d... 75475.056190: : sys_write:        pmu=3961415
                ls-3363  [003] dN.. 75475.056212: : sys_write_return: pmu=4051390
                ls-3363  [003] d... 75475.056216: : sys_write:        pmu=4065447
                ls-3363  [003] dN.. 75475.056227: : sys_write_return: pmu=4109760
                ls-3363  [003] d... 75475.056230: : sys_write:        pmu=4120776
                ls-3363  [003] dN.. 75475.056245: : sys_write_return: pmu=4178441
                ...
   # perf report --stdio
   Error:
   The perf.data file has no samples!

Signed-off-by: Wang Nan <wangnan0@huawei.com>

----------------------------------------------------------------
He Kuang (5):
      perf tools: Add prologue for BPF programs for fetching arguments
      perf record: Support custom vmlinux path
      bpf tools: Add helper function for updating bpf maps elements
      perf tools: Support perf event alias name
      perf record: Apply config to BPF objects before recording

Wang Nan (26):
      perf tools: Make perf depend on libbpf
      perf ebpf: Add the libbpf glue
      perf tools: Enable passing bpf object file to --event
      perf record, bpf: Create probe points for BPF programs
      perf record: Load eBPF object into kernel
      perf tools: Collect perf_evsel in BPF object files
      perf tools: Attach eBPF program to perf event
      perf record: Add clang options for compiling BPF scripts
      perf tools: Compile scriptlets to BPF objects when passing '.c' to --event
      perf test: Enforce LLVM test for BPF test
      perf test: Add 'perf test BPF'
      perf probe: Reset args and nargs for probe_trace_event when failure
      bpf tools: Load a program with different instances using preprocessor
      perf tools: Add BPF_PROLOGUE config options for further patches
      perf tools: Compile dwarf-regs.c if CONFIG_BPF_PROLOGUE is on
      perf tools: Generate prologue for BPF programs
      perf tools: Use same BPF program if arguments are identical
      perf tools: Allow BPF program attach to uprobe events
      perf test: Enforce LLVM test, add kbuild test
      perf test: Test BPF prologue
      bpf tools: Collect map definition in bpf_object
      bpf tools: Extract and collect map names from BPF object file
      perf tools: Pass available CPU number to clang compiler
      perf tools: Add API to config maps in bpf object
      perf tools: Add API to apply config to BPF map
      perf tools: Enable BPF object configure syntax

 tools/build/Makefile.feature                |   6 +-
 tools/lib/bpf/bpf.c                         |  14 +
 tools/lib/bpf/bpf.h                         |   2 +
 tools/lib/bpf/libbpf.c                      | 392 +++++++++---
 tools/lib/bpf/libbpf.h                      |  46 ++
 tools/perf/MANIFEST                         |   3 +
 tools/perf/Makefile.perf                    |  21 +-
 tools/perf/arch/x86/util/Build              |   1 +
 tools/perf/builtin-record.c                 |  21 +
 tools/perf/config/Makefile                  |  31 +-
 tools/perf/perf.c                           |   2 +
 tools/perf/tests/Build                      |  24 +-
 tools/perf/tests/bpf-script-example.c       |  48 ++
 tools/perf/tests/bpf-script-test-kbuild.c   |  21 +
 tools/perf/tests/bpf-script-test-prologue.c |  35 ++
 tools/perf/tests/bpf.c                      | 227 +++++++
 tools/perf/tests/builtin-test.c             |  10 +
 tools/perf/tests/llvm.c                     | 210 ++++++-
 tools/perf/tests/llvm.h                     |  29 +
 tools/perf/tests/make                       |   4 +-
 tools/perf/tests/tests.h                    |   5 +
 tools/perf/util/Build                       |   2 +
 tools/perf/util/bpf-loader.c                | 933 ++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.h                | 148 +++++
 tools/perf/util/bpf-prologue.c              | 443 +++++++++++++
 tools/perf/util/bpf-prologue.h              |  34 +
 tools/perf/util/evlist.c                    |  16 +
 tools/perf/util/evlist.h                    |   4 +
 tools/perf/util/evsel.c                     |  17 +
 tools/perf/util/evsel.h                     |   2 +
 tools/perf/util/llvm-utils.c                |  24 +-
 tools/perf/util/parse-events.c              | 221 ++++++-
 tools/perf/util/parse-events.h              |  15 +
 tools/perf/util/parse-events.l              |   6 +
 tools/perf/util/parse-events.y              |  55 +-
 tools/perf/util/probe-finder.c              |   4 +
 36 files changed, 2968 insertions(+), 108 deletions(-)
 create mode 100644 tools/perf/tests/bpf-script-example.c
 create mode 100644 tools/perf/tests/bpf-script-test-kbuild.c
 create mode 100644 tools/perf/tests/bpf-script-test-prologue.c
 create mode 100644 tools/perf/tests/bpf.c
 create mode 100644 tools/perf/tests/llvm.h
 create mode 100644 tools/perf/util/bpf-loader.c
 create mode 100644 tools/perf/util/bpf-loader.h
 create mode 100644 tools/perf/util/bpf-prologue.c
 create mode 100644 tools/perf/util/bpf-prologue.h

-- 
1.8.3.4

[PATCH 01/31] perf tools: Make perf depend on libbpf

[Wang Nan]

By adding libbpf into perf's Makefile, this patch enables perf to build
libbpf during building if libelf is found and neither NO_LIBELF nor
NO_LIBBPF is set. The newly introduced code is similar to libapi and
libtraceevent building in Makefile.perf.

MANIFEST is also updated for 'make perf-*-src-pkg'.

Append make_no_libbpf to tools/perf/tests/make.

'bpf' feature check is appended into default FEATURE_TESTS and
FEATURE_DISPLAY, so perf will check API version of bpf in
/path/to/kernel/include/uapi/linux/bpf.h. Which should not fail except
when we are trying to port this code to an old kernel.

Error messages are also updated to notify users about the disable of BPF
support of 'perf record' if libelf is missed or BPF API check failed.

tools/lib/bpf is added into TAG_FOLDERS to allow us to navigate on
libbpf files when working on perf using tools/perf/tags.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/tip-9tj3h70vyoku9rkrb8xafcij@git.kernel.org
[ Document NO_LIBBPF in Makefile.perf, noted by Jiri Olsa ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/build/Makefile.feature |  6 ++++--
 tools/perf/MANIFEST          |  3 +++
 tools/perf/Makefile.perf     | 21 +++++++++++++++++++--
 tools/perf/config/Makefile   | 19 ++++++++++++++++++-
 tools/perf/tests/make        |  4 +++-
 5 files changed, 47 insertions(+), 6 deletions(-)

diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index 72817e4..37ff4c9 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -53,7 +53,8 @@ FEATURE_TESTS ?=			\
 	libdw-dwarf-unwind		\
 	zlib				\
 	lzma				\
-	get_cpuid
+	get_cpuid			\
+	bpf
 
 FEATURE_DISPLAY ?=			\
 	dwarf				\
@@ -71,7 +72,8 @@ FEATURE_DISPLAY ?=			\
 	libdw-dwarf-unwind		\
 	zlib				\
 	lzma				\
-	get_cpuid
+	get_cpuid			\
+	bpf
 
 # Set FEATURE_CHECK_(C|LD)FLAGS-all for all FEATURE_TESTS features.
 # If in the future we need per-feature checks/flags for features not
diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index 9e6bdf5..39c38cb 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -17,6 +17,7 @@ tools/build
 tools/arch/x86/include/asm/atomic.h
 tools/arch/x86/include/asm/rmwcc.h
 tools/lib/traceevent
+tools/lib/bpf
 tools/lib/api
 tools/lib/bpf
 tools/lib/hweight.c
@@ -69,6 +70,8 @@ arch/*/lib/memset*.S
 include/linux/poison.h
 include/linux/hw_breakpoint.h
 include/uapi/linux/perf_event.h
+include/uapi/linux/bpf.h
+include/uapi/linux/bpf_common.h
 include/uapi/linux/const.h
 include/uapi/linux/swab.h
 include/uapi/linux/hw_breakpoint.h
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 56517d3..1e2e2d1 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -75,6 +75,8 @@ include config/utilities.mak
 # Define NO_LZMA if you do not want to support compressed (xz) kernel modules
 #
 # Define NO_AUXTRACE if you do not want AUX area tracing support
+#
+# Define NO_LIBBPF if you do not want BPF support
 
 # As per kernel Makefile, avoid funny character set dependencies
 unexport LC_ALL
@@ -145,6 +147,7 @@ AWK     = awk
 
 LIB_DIR          = $(srctree)/tools/lib/api/
 TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
+BPF_DIR = $(srctree)/tools/lib/bpf/
 
 # include config/Makefile by default and rule out
 # non-config cases
@@ -180,6 +183,7 @@ strip-libs = $(filter-out -l%,$(1))
 
 ifneq ($(OUTPUT),)
   TE_PATH=$(OUTPUT)
+  BPF_PATH=$(OUTPUT)
 ifneq ($(subdir),)
   LIB_PATH=$(OUTPUT)/../lib/api/
 else
@@ -188,6 +192,7 @@ endif
 else
   TE_PATH=$(TRACE_EVENT_DIR)
   LIB_PATH=$(LIB_DIR)
+  BPF_PATH=$(BPF_DIR)
 endif
 
 LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
@@ -199,6 +204,8 @@ LIBTRACEEVENT_DYNAMIC_LIST_LDFLAGS = -Xlinker --dynamic-list=$(LIBTRACEEVENT_DYN
 LIBAPI = $(LIB_PATH)libapi.a
 export LIBAPI
 
+LIBBPF = $(BPF_PATH)libbpf.a
+
 # python extension build directories
 PYTHON_EXTBUILD     := $(OUTPUT)python_ext_build/
 PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
@@ -251,6 +258,9 @@ export PERL_PATH
 LIB_FILE=$(OUTPUT)libperf.a
 
 PERFLIBS = $(LIB_FILE) $(LIBAPI) $(LIBTRACEEVENT)
+ifndef NO_LIBBPF
+  PERFLIBS += $(LIBBPF)
+endif
 
 # We choose to avoid "if .. else if .. else .. endif endif"
 # because maintaining the nesting to match is a pain.  If
@@ -420,6 +430,13 @@ $(LIBAPI)-clean:
 	$(call QUIET_CLEAN, libapi)
 	$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
 
+$(LIBBPF): FORCE
+	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) $(OUTPUT)libbpf.a
+
+$(LIBBPF)-clean:
+	$(call QUIET_CLEAN, libbpf)
+	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) clean >/dev/null
+
 help:
 	@echo 'Perf make targets:'
 	@echo '  doc		- make *all* documentation (see below)'
@@ -459,7 +476,7 @@ INSTALL_DOC_TARGETS += quick-install-doc quick-install-man quick-install-html
 $(DOC_TARGETS):
 	$(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) $(@:doc=all)
 
-TAG_FOLDERS= . ../lib/traceevent ../lib/api ../lib/symbol ../include
+TAG_FOLDERS= . ../lib/traceevent ../lib/api ../lib/symbol ../include ../lib/bpf
 TAG_FILES= ../../include/uapi/linux/perf_event.h
 
 TAGS:
@@ -567,7 +584,7 @@ config-clean:
 	$(call QUIET_CLEAN, config)
 	$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
 
-clean: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean config-clean
+clean: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean config-clean
 	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf-with-kcore $(LANG_BINDINGS)
 	$(Q)find . -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)$(RM) $(OUTPUT).config-detected
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index ab09ada..de89ec5 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -106,6 +106,7 @@ ifdef LIBBABELTRACE
   FEATURE_CHECK_LDFLAGS-libbabeltrace := $(LIBBABELTRACE_LDFLAGS) -lbabeltrace-ctf
 endif
 
+FEATURE_CHECK_CFLAGS-bpf = -I. -I$(srctree)/tools/include -I$(srctree)/arch/$(ARCH)/include/uapi -I$(srctree)/include/uapi
 # include ARCH specific config
 -include $(src-perf)/arch/$(ARCH)/Makefile
 
@@ -237,6 +238,7 @@ ifdef NO_LIBELF
   NO_DEMANGLE := 1
   NO_LIBUNWIND := 1
   NO_LIBDW_DWARF_UNWIND := 1
+  NO_LIBBPF := 1
 else
   ifeq ($(feature-libelf), 0)
     ifeq ($(feature-glibc), 1)
@@ -246,13 +248,14 @@ else
       LIBC_SUPPORT := 1
     endif
     ifeq ($(LIBC_SUPPORT),1)
-      msg := $(warning No libelf found, disables 'probe' tool, please install elfutils-libelf-devel/libelf-dev);
+      msg := $(warning No libelf found, disables 'probe' tool and BPF support in 'perf record', please install elfutils-libelf-devel/libelf-dev);
 
       NO_LIBELF := 1
       NO_DWARF := 1
       NO_DEMANGLE := 1
       NO_LIBUNWIND := 1
       NO_LIBDW_DWARF_UNWIND := 1
+      NO_LIBBPF := 1
     else
       ifneq ($(filter s% -static%,$(LDFLAGS),),)
         msg := $(error No static glibc found, please install glibc-static);
@@ -309,6 +312,13 @@ ifndef NO_LIBELF
       $(call detected,CONFIG_DWARF)
     endif # PERF_HAVE_DWARF_REGS
   endif # NO_DWARF
+
+  ifndef NO_LIBBPF
+    ifeq ($(feature-bpf), 1)
+      CFLAGS += -DHAVE_LIBBPF_SUPPORT
+      $(call detected,CONFIG_LIBBPF)
+    endif
+  endif # NO_LIBBPF
 endif # NO_LIBELF
 
 ifeq ($(ARCH),powerpc)
@@ -324,6 +334,13 @@ ifndef NO_LIBUNWIND
   endif
 endif
 
+ifndef NO_LIBBPF
+  ifneq ($(feature-bpf), 1)
+    msg := $(warning BPF API too old. Please install recent kernel headers. BPF support in 'perf record' is disabled.)
+    NO_LIBBPF := 1
+  endif
+endif
+
 dwarf-post-unwind := 1
 dwarf-post-unwind-text := BUG
 
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index ba31c4b..2cbd0c6 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -44,6 +44,7 @@ make_no_libnuma     := NO_LIBNUMA=1
 make_no_libaudit    := NO_LIBAUDIT=1
 make_no_libbionic   := NO_LIBBIONIC=1
 make_no_auxtrace    := NO_AUXTRACE=1
+make_no_libbpf	    := NO_LIBBPF=1
 make_tags           := tags
 make_cscope         := cscope
 make_help           := help
@@ -66,7 +67,7 @@ make_static         := LDFLAGS=-static
 make_minimal        := NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1
 make_minimal        += NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1
 make_minimal        += NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1
-make_minimal        += NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1
+make_minimal        += NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1
 
 # $(run) contains all available tests
 run := make_pure
@@ -94,6 +95,7 @@ run += make_no_libnuma
 run += make_no_libaudit
 run += make_no_libbionic
 run += make_no_auxtrace
+run += make_no_libbpf
 run += make_help
 run += make_doc
 run += make_perf_o
-- 
1.8.3.4

[PATCH 02/31] perf ebpf: Add the libbpf glue

[Wang Nan]

The 'bpf-loader.[ch]' files are introduced in this patch. Which will be
the interface between perf and libbpf. bpf__prepare_load() resides in
bpf-loader.c. Following patches will enrich these two files.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/bpf-loader.c | 57 ++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.h | 29 ++++++++++++++++++++++
 2 files changed, 86 insertions(+)
 create mode 100644 tools/perf/util/bpf-loader.c
 create mode 100644 tools/perf/util/bpf-loader.h

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
new file mode 100644
index 0000000..ab56073
--- /dev/null
+++ b/tools/perf/util/bpf-loader.c
@@ -0,0 +1,57 @@
+/*
+ * bpf-loader.c
+ *
+ * Copyright (C) 2015 Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2015 Huawei Inc.
+ */
+
+#include <bpf/libbpf.h>
+#include <linux/err.h>
+#include "perf.h"
+#include "debug.h"
+#include "bpf-loader.h"
+
+#define DEFINE_PRINT_FN(name, level) \
+static int libbpf_##name(const char *fmt, ...)	\
+{						\
+	va_list args;				\
+	int ret;				\
+						\
+	va_start(args, fmt);			\
+	ret = veprintf(level, verbose, pr_fmt(fmt), args);\
+	va_end(args);				\
+	return ret;				\
+}
+
+DEFINE_PRINT_FN(warning, 0)
+DEFINE_PRINT_FN(info, 0)
+DEFINE_PRINT_FN(debug, 1)
+
+struct bpf_object *bpf__prepare_load(const char *filename)
+{
+	struct bpf_object *obj;
+	static bool libbpf_initialized;
+
+	if (!libbpf_initialized) {
+		libbpf_set_print(libbpf_warning,
+				 libbpf_info,
+				 libbpf_debug);
+		libbpf_initialized = true;
+	}
+
+	obj = bpf_object__open(filename);
+	if (!obj) {
+		pr_debug("bpf: failed to load %s\n", filename);
+		return ERR_PTR(-EINVAL);
+	}
+
+	return obj;
+}
+
+void bpf__clear(void)
+{
+	struct bpf_object *obj, *tmp;
+
+	bpf_object__for_each_safe(obj, tmp)
+		bpf_object__close(obj);
+}
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
new file mode 100644
index 0000000..f402d7c
--- /dev/null
+++ b/tools/perf/util/bpf-loader.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright (C) 2015, Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2015, Huawei Inc.
+ */
+#ifndef __BPF_LOADER_H
+#define __BPF_LOADER_H
+
+#include <linux/compiler.h>
+#include <linux/err.h>
+#include <string.h>
+#include "debug.h"
+
+struct bpf_object;
+
+#ifdef HAVE_LIBBPF_SUPPORT
+struct bpf_object *bpf__prepare_load(const char *filename);
+
+void bpf__clear(void);
+#else
+static inline struct bpf_object *
+bpf__prepare_load(const char *filename __maybe_unused)
+{
+	pr_debug("ERROR: eBPF object loading is disabled during compiling.\n");
+	return ERR_PTR(-ENOTSUP);
+}
+
+static inline void bpf__clear(void) { }
+#endif
+#endif
-- 
1.8.3.4

[PATCH 03/31] perf tools: Enable passing bpf object file to --event

[Wang Nan]

By introducing new rules in tools/perf/util/parse-events.[ly], this
patch enables 'perf record --event bpf_file.o' to select events by an
eBPF object file. It calls parse_events_load_bpf() to load that file,
which uses bpf__prepare_load() and finally calls bpf_object__open() for
the object files.

After applying this patch, commands like:

 # perf record --event foo.o sleep

become possible.

However, at this point it is unable to link any useful things onto the
evsel list because the creating of probe points and BPF program
attaching have not been implemented.  Before real events are possible to
be extracted, to avoid perf report error because of empty evsel list,
this patch link a dummy evsel. The dummy event related code will be
removed when probing and extracting code is ready.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/perf.c              |  2 ++
 tools/perf/util/Build          |  1 +
 tools/perf/util/parse-events.c | 57 ++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/parse-events.h |  8 ++++++
 tools/perf/util/parse-events.l |  3 +++
 tools/perf/util/parse-events.y | 18 ++++++++++++-
 6 files changed, 88 insertions(+), 1 deletion(-)

diff --git a/tools/perf/perf.c b/tools/perf/perf.c
index 5437134..3d4c7c0 100644
--- a/tools/perf/perf.c
+++ b/tools/perf/perf.c
@@ -15,6 +15,7 @@
 #include "util/run-command.h"
 #include "util/parse-events.h"
 #include "util/parse-options.h"
+#include "util/bpf-loader.h"
 #include "util/debug.h"
 #include <api/fs/tracing_path.h>
 #include <pthread.h>
@@ -385,6 +386,7 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
 	status = p->fn(argc, argv, prefix);
 	exit_browser(status);
 	perf_env__exit(&perf_env);
+	bpf__clear();
 
 	if (status)
 		return status & 0xff;
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 9217119..591b3fe 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -87,6 +87,7 @@ libperf-$(CONFIG_AUXTRACE) += intel-bts.o
 libperf-y += parse-branch-options.o
 libperf-y += parse-regs-options.o
 
+libperf-$(CONFIG_LIBBPF) += bpf-loader.o
 libperf-$(CONFIG_LIBELF) += symbol-elf.o
 libperf-$(CONFIG_LIBELF) += probe-file.o
 libperf-$(CONFIG_LIBELF) += probe-event.o
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 991bbd4..a02abd3 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -11,6 +11,7 @@
 #include "symbol.h"
 #include "cache.h"
 #include "header.h"
+#include "bpf-loader.h"
 #include "debug.h"
 #include <api/fs/tracing_path.h>
 #include "parse-events-bison.h"
@@ -529,6 +530,62 @@ static int add_tracepoint_multi_sys(struct list_head *list, int *idx,
 	return ret;
 }
 
+int parse_events_load_bpf_obj(struct parse_events_evlist *data,
+			      struct list_head *list,
+			      struct bpf_object *obj)
+{
+	int err;
+	char errbuf[BUFSIZ];
+
+	if (IS_ERR(obj) || !obj) {
+		snprintf(errbuf, sizeof(errbuf),
+			 "Internal error: load bpf obj with NULL");
+		err = -EINVAL;
+		goto errout;
+	}
+
+	/*
+	 * Temporary add a dummy event here so we can check whether
+	 * basic bpf loader works. Following patches will replace
+	 * dummy event by useful evsels.
+	 */
+	return parse_events_add_numeric(data, list, PERF_TYPE_SOFTWARE,
+					PERF_COUNT_SW_DUMMY, NULL);
+errout:
+	data->error->help = strdup("(add -v to see detail)");
+	data->error->str = strdup(errbuf);
+	return err;
+}
+
+int parse_events_load_bpf(struct parse_events_evlist *data,
+			  struct list_head *list,
+			  char *bpf_file_name)
+{
+	struct bpf_object *obj;
+
+	obj = bpf__prepare_load(bpf_file_name);
+	if (IS_ERR(obj) || !obj) {
+		char errbuf[BUFSIZ];
+		int err;
+
+		err = obj ? PTR_ERR(obj) : -EINVAL;
+
+		if (err == -ENOTSUP)
+			snprintf(errbuf, sizeof(errbuf),
+				 "BPF support is not compiled");
+		else
+			snprintf(errbuf, sizeof(errbuf),
+				 "BPF object file '%s' is invalid",
+				 bpf_file_name);
+
+		data->error->help = strdup("(add -v to see detail)");
+		data->error->str = strdup(errbuf);
+		return err;
+	}
+
+	return parse_events_load_bpf_obj(data, list, obj);
+}
+
 static int
 parse_breakpoint_type(const char *type, struct perf_event_attr *attr)
 {
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index f13d3cc..fbb16c7 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -121,6 +121,14 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
 				char *sys, char *event,
 				struct parse_events_error *error,
 				struct list_head *head_config);
+int parse_events_load_bpf(struct parse_events_evlist *data,
+			  struct list_head *list,
+			  char *bpf_file_name);
+/* Provide this function for perf test */
+struct bpf_object;
+int parse_events_load_bpf_obj(struct parse_events_evlist *data,
+			      struct list_head *list,
+			      struct bpf_object *obj);
 int parse_events_add_numeric(struct parse_events_evlist *data,
 			     struct list_head *list,
 			     u32 type, u64 config,
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index be24457..5e5d31a 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -115,6 +115,7 @@ do {							\
 group		[^,{}/]*[{][^}]*[}][^,{}/]*
 event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
 event		[^,{}/]+
+bpf_object	.*\.(o|bpf)
 
 num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
@@ -159,6 +160,7 @@ modifier_bp	[rwx]{1,3}
 		}
 
 {event_pmu}	|
+{bpf_object}	|
 {event}		{
 			BEGIN(INITIAL);
 			REWIND(1);
@@ -264,6 +266,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
 {num_hex}		{ return value(yyscanner, 16); }
 
 {modifier_event}	{ return str(yyscanner, PE_MODIFIER_EVENT); }
+{bpf_object}		{ return str(yyscanner, PE_BPF_OBJECT); }
 {name}			{ return pmu_str_check(yyscanner); }
 "/"			{ BEGIN(config); return '/'; }
 -			{ return '-'; }
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index ae6af26..497f19b 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -42,6 +42,7 @@ static inc_group_count(struct list_head *list,
 %token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
 %token PE_EVENT_NAME
 %token PE_NAME
+%token PE_BPF_OBJECT
 %token PE_MODIFIER_EVENT PE_MODIFIER_BP
 %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
 %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
@@ -53,6 +54,7 @@ static inc_group_count(struct list_head *list,
 %type <num> PE_RAW
 %type <num> PE_TERM
 %type <str> PE_NAME
+%type <str> PE_BPF_OBJECT
 %type <str> PE_NAME_CACHE_TYPE
 %type <str> PE_NAME_CACHE_OP_RESULT
 %type <str> PE_MODIFIER_EVENT
@@ -70,6 +72,7 @@ static inc_group_count(struct list_head *list,
 %type <tracepoint_name> tracepoint_name
 %type <head> event_legacy_numeric
 %type <head> event_legacy_raw
+%type <head> event_bpf_file
 %type <head> event_def
 %type <head> event_mod
 %type <head> event_name
@@ -203,7 +206,8 @@ event_def: event_pmu |
 	   event_legacy_mem |
 	   event_legacy_tracepoint sep_dc |
 	   event_legacy_numeric sep_dc |
-	   event_legacy_raw sep_dc
+	   event_legacy_raw sep_dc |
+	   event_bpf_file
 
 event_pmu:
 PE_NAME '/' event_config '/'
@@ -449,6 +453,18 @@ PE_RAW
 	$$ = list;
 }
 
+event_bpf_file:
+PE_BPF_OBJECT
+{
+	struct parse_events_evlist *data = _data;
+	struct parse_events_error *error = data->error;
+	struct list_head *list;
+
+	ALLOC_LIST(list);
+	ABORT_ON(parse_events_load_bpf(data, list, $1));
+	$$ = list;
+}
+
 start_terms: event_config
 {
 	struct parse_events_terms *data = _data;
-- 
1.8.3.4

[PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Wang Nan]

This patch introduces bpf__{un,}probe() functions to enable callers to
create kprobe points based on section names a BPF program. It parses
the section names in the program and creates corresponding 'struct
perf_probe_event' structures. The parse_perf_probe_command() function is
used to do the main parsing work. The resuling 'struct perf_probe_event'
is stored into program private data for further using.

By utilizing the new probing API, this patch creates probe points during
event parsing.

To ensure probe points be removed correctly, register an atexit hook
so even perf quit through exit() bpf__clear() is still called, so probing
points are cleared. Note that bpf_clear() should be registered before
bpf__probe() is called, so failure of bpf__probe() can still trigger
bpf__clear() to remove probe points which are already probed.

strerror style error reporting scaffold is created by this patch.
bpf__strerror_probe() is the first error reporting function in bpf-loader.c.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/bpf-loader.c   | 222 ++++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/bpf-loader.h   |  30 ++++++
 tools/perf/util/parse-events.c |  17 ++++
 3 files changed, 268 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index ab56073..56f6fe8 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -10,6 +10,8 @@
 #include "perf.h"
 #include "debug.h"
 #include "bpf-loader.h"
+#include "probe-event.h"
+#include "probe-finder.h" // for MAX_PROBES
 
 #define DEFINE_PRINT_FN(name, level) \
 static int libbpf_##name(const char *fmt, ...)	\
@@ -27,6 +29,10 @@ DEFINE_PRINT_FN(warning, 0)
 DEFINE_PRINT_FN(info, 0)
 DEFINE_PRINT_FN(debug, 1)
 
+struct bpf_prog_priv {
+	struct perf_probe_event pev;
+};
+
 struct bpf_object *bpf__prepare_load(const char *filename)
 {
 	struct bpf_object *obj;
@@ -52,6 +58,220 @@ void bpf__clear(void)
 {
 	struct bpf_object *obj, *tmp;
 
-	bpf_object__for_each_safe(obj, tmp)
+	bpf_object__for_each_safe(obj, tmp) {
+		bpf__unprobe(obj);
 		bpf_object__close(obj);
+	}
+}
+
+static void
+bpf_prog_priv__clear(struct bpf_program *prog __maybe_unused,
+		     void *_priv)
+{
+	struct bpf_prog_priv *priv = _priv;
+
+	cleanup_perf_probe_events(&priv->pev, 1);
+	free(priv);
+}
+
+static int
+config_bpf_program(struct bpf_program *prog)
+{
+	struct perf_probe_event *pev = NULL;
+	struct bpf_prog_priv *priv = NULL;
+	const char *config_str;
+	int err;
+
+	config_str = bpf_program__title(prog, false);
+	if (!config_str) {
+		pr_debug("bpf: unable to get title for program\n");
+		return -EINVAL;
+	}
+
+	priv = calloc(sizeof(*priv), 1);
+	if (!priv) {
+		pr_debug("bpf: failed to alloc priv\n");
+		return -ENOMEM;
+	}
+	pev = &priv->pev;
+
+	pr_debug("bpf: config program '%s'\n", config_str);
+	err = parse_perf_probe_command(config_str, pev);
+	if (err < 0) {
+		pr_debug("bpf: '%s' is not a valid config string\n",
+			 config_str);
+		err = -EINVAL;
+		goto errout;
+	}
+
+	if (pev->group && strcmp(pev->group, PERF_BPF_PROBE_GROUP)) {
+		pr_debug("bpf: '%s': group for event is set and not '%s'.\n",
+			 config_str, PERF_BPF_PROBE_GROUP);
+		err = -EINVAL;
+		goto errout;
+	} else if (!pev->group)
+		pev->group = strdup(PERF_BPF_PROBE_GROUP);
+
+	if (!pev->group) {
+		pr_debug("bpf: strdup failed\n");
+		err = -ENOMEM;
+		goto errout;
+	}
+
+	if (!pev->event) {
+		pr_debug("bpf: '%s': event name is missing\n",
+			 config_str);
+		err = -EINVAL;
+		goto errout;
+	}
+	pr_debug("bpf: config '%s' is ok\n", config_str);
+
+	err = bpf_program__set_private(prog, priv, bpf_prog_priv__clear);
+	if (err) {
+		pr_debug("Failed to set priv for program '%s'\n", config_str);
+		goto errout;
+	}
+
+	return 0;
+
+errout:
+	if (pev)
+		clear_perf_probe_event(pev);
+	free(priv);
+	return err;
+}
+
+static int bpf__prepare_probe(void)
+{
+	static int err = 0;
+	static bool initialized = false;
+
+	/*
+	 * Make err static, so if init failed the first, bpf__prepare_probe()
+	 * fails each time without calling init_probe_symbol_maps multiple
+	 * times.
+	 */
+	if (initialized)
+		return err;
+
+	initialized = true;
+	err = init_probe_symbol_maps(false);
+	if (err < 0)
+		pr_debug("Failed to init_probe_symbol_maps\n");
+	probe_conf.max_probes = MAX_PROBES;
+	return err;
+}
+
+int bpf__probe(struct bpf_object *obj)
+{
+	int err = 0;
+	struct bpf_program *prog;
+	struct bpf_prog_priv *priv;
+	struct perf_probe_event *pev;
+
+	err = bpf__prepare_probe();
+	if (err) {
+		pr_debug("bpf__prepare_probe failed\n");
+		return err;
+	}
+
+	bpf_object__for_each_program(prog, obj) {
+		err = config_bpf_program(prog);
+		if (err)
+			goto out;
+
+		err = bpf_program__get_private(prog, (void **)&priv);
+		if (err || !priv)
+			goto out;
+		pev = &priv->pev;
+
+		err = convert_perf_probe_events(pev, 1);
+		if (err < 0) {
+			pr_debug("bpf_probe: failed to convert perf probe events");
+			goto out;
+		}
+
+		err = apply_perf_probe_events(pev, 1);
+		if (err < 0) {
+			pr_debug("bpf_probe: failed to apply perf probe events");
+			goto out;
+		}
+	}
+out:
+	return err < 0 ? err : 0;
+}
+
+#define EVENTS_WRITE_BUFSIZE  4096
+int bpf__unprobe(struct bpf_object *obj)
+{
+	int err, ret = 0;
+	struct bpf_program *prog;
+	struct bpf_prog_priv *priv;
+
+	bpf_object__for_each_program(prog, obj) {
+		int i;
+
+		err = bpf_program__get_private(prog, (void **)&priv);
+		if (err || !priv)
+			continue;
+
+		for (i = 0; i < priv->pev.ntevs; i++) {
+			struct probe_trace_event *tev = &priv->pev.tevs[i];
+			char name_buf[EVENTS_WRITE_BUFSIZE];
+			struct strfilter *delfilter;
+
+			snprintf(name_buf, EVENTS_WRITE_BUFSIZE,
+				 "%s:%s", tev->group, tev->event);
+			name_buf[EVENTS_WRITE_BUFSIZE - 1] = '\0';
+
+			delfilter = strfilter__new(name_buf, NULL);
+			if (!delfilter) {
+				pr_debug("Failed to create filter for unprobing\n");
+				ret = -ENOMEM;
+				continue;
+			}
+
+			err = del_perf_probe_events(delfilter);
+			strfilter__delete(delfilter);
+			if (err) {
+				pr_debug("Failed to delete %s\n", name_buf);
+				ret = err;
+				continue;
+			}
+		}
+	}
+	return ret;
+}
+
+#define bpf__strerror_head(err, buf, size) \
+	char sbuf[STRERR_BUFSIZE], *emsg;\
+	if (!size)\
+		return 0;\
+	if (err < 0)\
+		err = -err;\
+	emsg = strerror_r(err, sbuf, sizeof(sbuf));\
+	switch (err) {\
+	default:\
+		scnprintf(buf, size, "%s", emsg);\
+		break;
+
+#define bpf__strerror_entry(val, fmt...)\
+	case val: {\
+		scnprintf(buf, size, fmt);\
+		break;\
+	}
+
+#define bpf__strerror_end(buf, size)\
+	}\
+	buf[size - 1] = '\0';
+
+int bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
+			int err, char *buf, size_t size)
+{
+	bpf__strerror_head(err, buf, size);
+	bpf__strerror_entry(EEXIST, "Probe point exist. Try use 'perf probe -d \"*\"'");
+	bpf__strerror_entry(EPERM, "You need to be root, and /proc/sys/kernel/kptr_restrict should be 0\n");
+	bpf__strerror_entry(ENOENT, "You need to check probing points in BPF file\n");
+	bpf__strerror_end(buf, size);
+	return 0;
 }
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index f402d7c..b819622 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -11,11 +11,18 @@
 #include "debug.h"
 
 struct bpf_object;
+#define PERF_BPF_PROBE_GROUP "perf_bpf_probe"
 
 #ifdef HAVE_LIBBPF_SUPPORT
 struct bpf_object *bpf__prepare_load(const char *filename);
 
 void bpf__clear(void);
+
+int bpf__probe(struct bpf_object *obj);
+int bpf__unprobe(struct bpf_object *obj);
+int bpf__strerror_probe(struct bpf_object *obj, int err,
+			char *buf, size_t size);
+
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused)
@@ -25,5 +32,28 @@ bpf__prepare_load(const char *filename __maybe_unused)
 }
 
 static inline void bpf__clear(void) { }
+
+static inline int bpf__probe(struct bpf_object *obj __maybe_unused) { return 0;}
+static inline int bpf__unprobe(struct bpf_object *obj __maybe_unused) { return 0;}
+
+static inline int
+__bpf_strerror(char *buf, size_t size)
+{
+	if (!size)
+		return 0;
+	strncpy(buf,
+		"ERROR: eBPF object loading is disabled during compiling.\n",
+		size);
+	buf[size - 1] = '\0';
+	return 0;
+}
+
+static inline int
+bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
+		    int err __maybe_unused,
+		    char *buf, size_t size)
+{
+	return __bpf_strerror(buf, size);
+}
 #endif
 #endif
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index a02abd3..06ff8d6 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -536,6 +536,7 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 {
 	int err;
 	char errbuf[BUFSIZ];
+	static bool registered_unprobe_atexit = false;
 
 	if (IS_ERR(obj) || !obj) {
 		snprintf(errbuf, sizeof(errbuf),
@@ -545,6 +546,22 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 	}
 
 	/*
+	 * Register atexit handler before calling bpf__probe() so
+	 * bpf__probe() don't need to unprobe probe points its already
+	 * created when failure.
+	 */
+	if (!registered_unprobe_atexit) {
+		atexit(bpf__clear);
+		registered_unprobe_atexit = true;
+	}
+
+	err = bpf__probe(obj);
+	if (err) {
+		bpf__strerror_probe(obj, err, errbuf, sizeof(errbuf));
+		goto errout;
+	}
+
+	/*
 	 * Temporary add a dummy event here so we can check whether
 	 * basic bpf loader works. Following patches will replace
 	 * dummy event by useful evsels.
-- 
1.8.3.4

[PATCH 05/31] perf record: Load eBPF object into kernel

[Wang Nan]

This patch utilizes bpf_object__load() provided by libbpf to load all
objects into kernel.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/bpf-loader.c   | 22 ++++++++++++++++++++++
 tools/perf/util/bpf-loader.h   | 11 +++++++++++
 tools/perf/util/parse-events.c |  6 ++++++
 3 files changed, 39 insertions(+)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index 56f6fe8..7279558 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -243,6 +243,18 @@ int bpf__unprobe(struct bpf_object *obj)
 	return ret;
 }
 
+int bpf__load(struct bpf_object *obj)
+{
+	int err;
+
+	err = bpf_object__load(obj);
+	if (err) {
+		pr_debug("bpf: load objects failed\n");
+		return err;
+	}
+	return 0;
+}
+
 #define bpf__strerror_head(err, buf, size) \
 	char sbuf[STRERR_BUFSIZE], *emsg;\
 	if (!size)\
@@ -275,3 +287,13 @@ int bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
 	bpf__strerror_end(buf, size);
 	return 0;
 }
+
+int bpf__strerror_load(struct bpf_object *obj __maybe_unused,
+		       int err, char *buf, size_t size)
+{
+	bpf__strerror_head(err, buf, size);
+	bpf__strerror_entry(EINVAL, "%s: Are you root and runing a CONFIG_BPF_SYSCALL kernel?",
+			    emsg)
+	bpf__strerror_end(buf, size);
+	return 0;
+}
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index b819622..b091ceb 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -23,6 +23,9 @@ int bpf__unprobe(struct bpf_object *obj);
 int bpf__strerror_probe(struct bpf_object *obj, int err,
 			char *buf, size_t size);
 
+int bpf__load(struct bpf_object *obj);
+int bpf__strerror_load(struct bpf_object *obj, int err,
+		       char *buf, size_t size);
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused)
@@ -35,6 +38,7 @@ static inline void bpf__clear(void) { }
 
 static inline int bpf__probe(struct bpf_object *obj __maybe_unused) { return 0;}
 static inline int bpf__unprobe(struct bpf_object *obj __maybe_unused) { return 0;}
+static inline int bpf__load(struct bpf_object *obj __maybe_unused) { return 0; }
 
 static inline int
 __bpf_strerror(char *buf, size_t size)
@@ -55,5 +59,12 @@ bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
 {
 	return __bpf_strerror(buf, size);
 }
+
+static inline int bpf__strerror_load(struct bpf_object *obj __maybe_unused,
+				     int err __maybe_unused,
+				     char *buf, size_t size)
+{
+	return __bpf_strerror(buf, size);
+}
 #endif
 #endif
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 06ff8d6..5b17b88 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -561,6 +561,12 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 		goto errout;
 	}
 
+	err = bpf__load(obj);
+	if (err) {
+		bpf__strerror_load(obj, err, errbuf, sizeof(errbuf));
+		goto errout;
+	}
+
 	/*
 	 * Temporary add a dummy event here so we can check whether
 	 * basic bpf loader works. Following patches will replace
-- 
1.8.3.4

[PATCH 06/31] perf tools: Collect perf_evsel in BPF object files

[Wang Nan]

This patch collects 'struct perf_evsel' for every probing points in BPF
object file(s) and fill 'struct evlist'. The previous introduced dummy
event now removed. After this patch, following command:

 # perf record --event filter.o ls

Can trace on each probing points defined in filter.o.

The core of this patch is bpf__foreach_tev(), which calls a callback
function for each 'struct probe_trace_event' events for a bpf program
with their file descriptors. Callback function add_bpf_event()
creates evsels by calling parse_events_add_tracepoint().

Since bpf-loader.c will not be built if libbpf is turned off, an empty
bpf__foreach_tev() is defined in bpf-loader.h to avoid compiling
error.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/bpf-loader.c   | 40 ++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.h   | 14 ++++++++++++
 tools/perf/util/parse-events.c | 52 ++++++++++++++++++++++++++++++++++++------
 3 files changed, 99 insertions(+), 7 deletions(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index 7279558..aa784a4 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -255,6 +255,46 @@ int bpf__load(struct bpf_object *obj)
 	return 0;
 }
 
+int bpf__foreach_tev(struct bpf_object *obj,
+		     bpf_prog_iter_callback_t func,
+		     void *arg)
+{
+	struct bpf_program *prog;
+	int err;
+
+	bpf_object__for_each_program(prog, obj) {
+		struct probe_trace_event *tev;
+		struct perf_probe_event *pev;
+		struct bpf_prog_priv *priv;
+		int i, fd;
+
+		err = bpf_program__get_private(prog,
+				(void **)&priv);
+		if (err || !priv) {
+			pr_debug("bpf: failed to get private field\n");
+			return -EINVAL;
+		}
+
+		pev = &priv->pev;
+		for (i = 0; i < pev->ntevs; i++) {
+			tev = &pev->tevs[i];
+
+			fd = bpf_program__fd(prog);
+			if (fd < 0) {
+				pr_debug("bpf: failed to get file descriptor\n");
+				return fd;
+			}
+
+			err = (*func)(tev, fd, arg);
+			if (err) {
+				pr_debug("bpf: call back failed, stop iterate\n");
+				return err;
+			}
+		}
+	}
+	return 0;
+}
+
 #define bpf__strerror_head(err, buf, size) \
 	char sbuf[STRERR_BUFSIZE], *emsg;\
 	if (!size)\
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index b091ceb..a8f25ee 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -8,11 +8,15 @@
 #include <linux/compiler.h>
 #include <linux/err.h>
 #include <string.h>
+#include "probe-event.h"
 #include "debug.h"
 
 struct bpf_object;
 #define PERF_BPF_PROBE_GROUP "perf_bpf_probe"
 
+typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
+					int fd, void *arg);
+
 #ifdef HAVE_LIBBPF_SUPPORT
 struct bpf_object *bpf__prepare_load(const char *filename);
 
@@ -26,6 +30,8 @@ int bpf__strerror_probe(struct bpf_object *obj, int err,
 int bpf__load(struct bpf_object *obj);
 int bpf__strerror_load(struct bpf_object *obj, int err,
 		       char *buf, size_t size);
+int bpf__foreach_tev(struct bpf_object *obj,
+		     bpf_prog_iter_callback_t func, void *arg);
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused)
@@ -41,6 +47,14 @@ static inline int bpf__unprobe(struct bpf_object *obj __maybe_unused) { return 0
 static inline int bpf__load(struct bpf_object *obj __maybe_unused) { return 0; }
 
 static inline int
+bpf__foreach_tev(struct bpf_object *obj __maybe_unused,
+		 bpf_prog_iter_callback_t func __maybe_unused,
+		 void *arg __maybe_unused)
+{
+	return 0;
+}
+
+static inline int
 __bpf_strerror(char *buf, size_t size)
 {
 	if (!size)
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 5b17b88..61c7a47 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -530,12 +530,49 @@ static int add_tracepoint_multi_sys(struct list_head *list, int *idx,
 	return ret;
 }
 
+struct __add_bpf_event_param {
+	struct parse_events_evlist *data;
+	struct list_head *list;
+};
+
+static int add_bpf_event(struct probe_trace_event *tev, int fd,
+			 void *_param)
+{
+	LIST_HEAD(new_evsels);
+	struct __add_bpf_event_param *param = _param;
+	struct parse_events_evlist *evlist = param->data;
+	struct list_head *list = param->list;
+	int err;
+
+	pr_debug("add bpf event %s:%s and attach bpf program %d\n",
+		 tev->group, tev->event, fd);
+
+	err = parse_events_add_tracepoint(&new_evsels, &evlist->idx, tev->group,
+					  tev->event, evlist->error, NULL);
+	if (err) {
+		struct perf_evsel *evsel, *tmp;
+
+		pr_debug("Failed to add BPF event %s:%s\n",
+			 tev->group, tev->event);
+		list_for_each_entry_safe(evsel, tmp, &new_evsels, node) {
+			list_del(&evsel->node);
+			perf_evsel__delete(evsel);
+		}
+		return err;
+	}
+	pr_debug("adding %s:%s\n", tev->group, tev->event);
+
+	list_splice(&new_evsels, list);
+	return 0;
+}
+
 int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 			      struct list_head *list,
 			      struct bpf_object *obj)
 {
 	int err;
 	char errbuf[BUFSIZ];
+	struct __add_bpf_event_param param = {data, list};
 	static bool registered_unprobe_atexit = false;
 
 	if (IS_ERR(obj) || !obj) {
@@ -567,13 +604,14 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 		goto errout;
 	}
 
-	/*
-	 * Temporary add a dummy event here so we can check whether
-	 * basic bpf loader works. Following patches will replace
-	 * dummy event by useful evsels.
-	 */
-	return parse_events_add_numeric(data, list, PERF_TYPE_SOFTWARE,
-					PERF_COUNT_SW_DUMMY, NULL);
+	err = bpf__foreach_tev(obj, add_bpf_event, &param);
+	if (err) {
+		snprintf(errbuf, sizeof(errbuf),
+			 "Attach events in BPF object failed");
+		goto errout;
+	}
+
+	return 0;
 errout:
 	data->error->help = strdup("(add -v to see detail)");
 	data->error->str = strdup(errbuf);
-- 
1.8.3.4

[PATCH 07/31] perf tools: Attach eBPF program to perf event

[Wang Nan]

This is the final patch which makes basic BPF filter work. After
applying this patch, users are allowed to use BPF filter like:

 # perf record --event ./hello_world.o ls

A bpf_fd field is appended to 'struct evsel', and setup during the
callback function add_bpf_event() for each 'probe_trace_event'.

PERF_EVENT_IOC_SET_BPF ioctl is used to attach eBPF program to a newly
created perf event. The file descriptor of the eBPF program is passed to
perf record using previous patches, and stored into evsel->bpf_fd.

It is possible that different perf event are created for one kprobe
events for different CPUs. In this case, when trying to call the
ioctl, EEXIST will be return. This patch doesn't treat it as an error.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/evsel.c        | 17 +++++++++++++++++
 tools/perf/util/evsel.h        |  1 +
 tools/perf/util/parse-events.c |  6 ++++++
 3 files changed, 24 insertions(+)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 8be867c..7d12af5 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -208,6 +208,7 @@ void perf_evsel__init(struct perf_evsel *evsel,
 	evsel->unit	   = "";
 	evsel->scale	   = 1.0;
 	evsel->evlist	   = NULL;
+	evsel->bpf_fd	   = -1;
 	INIT_LIST_HEAD(&evsel->node);
 	INIT_LIST_HEAD(&evsel->config_terms);
 	perf_evsel__object.init(evsel);
@@ -1346,6 +1347,22 @@ retry_open:
 					  err);
 				goto try_fallback;
 			}
+
+			if (evsel->bpf_fd >= 0) {
+				int evt_fd = FD(evsel, cpu, thread);
+				int bpf_fd = evsel->bpf_fd;
+
+				err = ioctl(evt_fd,
+					    PERF_EVENT_IOC_SET_BPF,
+					    bpf_fd);
+				if (err && errno != EEXIST) {
+					pr_err("failed to attach bpf fd %d: %s\n",
+					       bpf_fd, strerror(errno));
+					err = -EINVAL;
+					goto out_close;
+				}
+			}
+
 			set_rlimit = NO_CHANGE;
 
 			/*
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 02a5fed..a60b5d5 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -121,6 +121,7 @@ struct perf_evsel {
 	char			*group_name;
 	bool			cmdline_group_boundary;
 	struct list_head	config_terms;
+	int			bpf_fd;
 };
 
 union u64_swap {
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 61c7a47..5087023 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -542,6 +542,7 @@ static int add_bpf_event(struct probe_trace_event *tev, int fd,
 	struct __add_bpf_event_param *param = _param;
 	struct parse_events_evlist *evlist = param->data;
 	struct list_head *list = param->list;
+	struct perf_evsel *pos;
 	int err;
 
 	pr_debug("add bpf event %s:%s and attach bpf program %d\n",
@@ -562,6 +563,11 @@ static int add_bpf_event(struct probe_trace_event *tev, int fd,
 	}
 	pr_debug("adding %s:%s\n", tev->group, tev->event);
 
+	list_for_each_entry(pos, &new_evsels, node) {
+		pr_debug("adding %s:%s to %p\n",
+			 tev->group, tev->event, pos);
+		pos->bpf_fd = fd;
+	}
 	list_splice(&new_evsels, list);
 	return 0;
 }
-- 
1.8.3.4

[PATCH 08/31] perf record: Add clang options for compiling BPF scripts

[Wang Nan]

Although previous patch allows setting BPF compiler related options in
perfconfig, on some ad-hoc situation it still requires passing options
through cmdline. This patch introduces 2 options to 'perf record' for
this propose: --clang-path and --clang-opt.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/builtin-record.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 24ace2f..847cc67 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -31,6 +31,7 @@
 #include "util/auxtrace.h"
 #include "util/parse-branch-options.h"
 #include "util/parse-regs-options.h"
+#include "util/llvm-utils.h"
 
 #include <unistd.h>
 #include <sched.h>
@@ -1117,6 +1118,12 @@ struct option __record_options[] = {
 			"per thread proc mmap processing timeout in ms"),
 	OPT_BOOLEAN(0, "switch-events", &record.opts.record_switch_events,
 		    "Record context switch events"),
+#ifdef HAVE_LIBBPF_SUPPORT
+	OPT_STRING(0, "clang-path", &llvm_param.clang_path, "clang path",
+		   "clang binary to use for compiling BPF scriptlets"),
+	OPT_STRING(0, "clang-opt", &llvm_param.clang_opt, "clang options",
+		   "options passed to clang when compiling BPF scriptlets"),
+#endif
 	OPT_END()
 };
 
-- 
1.8.3.4

[PATCH 09/31] perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

[Wang Nan]

This patch provides infrastructure for passing source files to --event
directly using:

 # perf record --event bpf-file.c command

This patch does following works:

 1) Allow passing '.c' file to '--event'. parse_events_load_bpf() is
    expanded to allow caller tell it whether the passed file is source
    file or object.

 2) llvm__compile_bpf() is called to compile the '.c' file, the result
    is saved into memory. Use bpf_object__open_buffer() to load the
    in-memory object.

Introduces a bpf-script-example.c so we can manually test it:

 # perf record --clang-opt "-DLINUX_VERSION_CODE=0x40200" --event ./bpf-script-example.c sleep 1

Note that '--clang-opt' must put before '--event'.

Futher patches will merge it into a testcase so can be tested automatically.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/tests/bpf-script-example.c | 44 +++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.c          | 17 ++++++++++++--
 tools/perf/util/bpf-loader.h          |  5 ++--
 tools/perf/util/parse-events.c        |  5 ++--
 tools/perf/util/parse-events.h        |  3 ++-
 tools/perf/util/parse-events.l        |  3 +++
 tools/perf/util/parse-events.y        | 15 ++++++++++--
 7 files changed, 83 insertions(+), 9 deletions(-)
 create mode 100644 tools/perf/tests/bpf-script-example.c

diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
new file mode 100644
index 0000000..410a70b
--- /dev/null
+++ b/tools/perf/tests/bpf-script-example.c
@@ -0,0 +1,44 @@
+#ifndef LINUX_VERSION_CODE
+# error Need LINUX_VERSION_CODE
+# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
+#endif
+#define BPF_ANY 0
+#define BPF_MAP_TYPE_ARRAY 2
+#define BPF_FUNC_map_lookup_elem 1
+#define BPF_FUNC_map_update_elem 2
+
+static void *(*bpf_map_lookup_elem)(void *map, void *key) =
+	(void *) BPF_FUNC_map_lookup_elem;
+static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
+	(void *) BPF_FUNC_map_update_elem;
+
+struct bpf_map_def {
+	unsigned int type;
+	unsigned int key_size;
+	unsigned int value_size;
+	unsigned int max_entries;
+};
+
+#define SEC(NAME) __attribute__((section(NAME), used))
+struct bpf_map_def SEC("maps") flip_table = {
+	.type = BPF_MAP_TYPE_ARRAY,
+	.key_size = sizeof(int),
+	.value_size = sizeof(int),
+	.max_entries = 1,
+};
+
+SEC("func=sys_epoll_pwait")
+int bpf_func__sys_epoll_pwait(void *ctx)
+{
+	int ind =0;
+	int *flag = bpf_map_lookup_elem(&flip_table, &ind);
+	int new_flag;
+	if (!flag)
+		return 0;
+	/* flip flag and store back */
+	new_flag = !*flag;
+	bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
+	return new_flag;
+}
+char _license[] SEC("license") = "GPL";
+int _version SEC("version") = LINUX_VERSION_CODE;
diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index aa784a4..ba6f752 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -12,6 +12,7 @@
 #include "bpf-loader.h"
 #include "probe-event.h"
 #include "probe-finder.h" // for MAX_PROBES
+#include "llvm-utils.h"
 
 #define DEFINE_PRINT_FN(name, level) \
 static int libbpf_##name(const char *fmt, ...)	\
@@ -33,7 +34,7 @@ struct bpf_prog_priv {
 	struct perf_probe_event pev;
 };
 
-struct bpf_object *bpf__prepare_load(const char *filename)
+struct bpf_object *bpf__prepare_load(const char *filename, bool source)
 {
 	struct bpf_object *obj;
 	static bool libbpf_initialized;
@@ -45,7 +46,19 @@ struct bpf_object *bpf__prepare_load(const char *filename)
 		libbpf_initialized = true;
 	}
 
-	obj = bpf_object__open(filename);
+	if (source) {
+		int err;
+		void *obj_buf;
+		size_t obj_buf_sz;
+
+		err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
+		if (err)
+			return ERR_PTR(err);
+		obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, filename);
+		free(obj_buf);
+	} else
+		obj = bpf_object__open(filename);
+
 	if (!obj) {
 		pr_debug("bpf: failed to load %s\n", filename);
 		return ERR_PTR(-EINVAL);
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index a8f25ee..ccd8d7f 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -18,7 +18,7 @@ typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
 					int fd, void *arg);
 
 #ifdef HAVE_LIBBPF_SUPPORT
-struct bpf_object *bpf__prepare_load(const char *filename);
+struct bpf_object *bpf__prepare_load(const char *filename, bool source);
 
 void bpf__clear(void);
 
@@ -34,7 +34,8 @@ int bpf__foreach_tev(struct bpf_object *obj,
 		     bpf_prog_iter_callback_t func, void *arg);
 #else
 static inline struct bpf_object *
-bpf__prepare_load(const char *filename __maybe_unused)
+bpf__prepare_load(const char *filename __maybe_unused,
+		  bool source __maybe_unused)
 {
 	pr_debug("ERROR: eBPF object loading is disabled during compiling.\n");
 	return ERR_PTR(-ENOTSUP);
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 5087023..4849dbd 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -626,11 +626,12 @@ errout:
 
 int parse_events_load_bpf(struct parse_events_evlist *data,
 			  struct list_head *list,
-			  char *bpf_file_name)
+			  char *bpf_file_name,
+			  bool source)
 {
 	struct bpf_object *obj;
 
-	obj = bpf__prepare_load(bpf_file_name);
+	obj = bpf__prepare_load(bpf_file_name, source);
 	if (IS_ERR(obj) || !obj) {
 		char errbuf[BUFSIZ];
 		int err;
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index fbb16c7..8f17c83 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -123,7 +123,8 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
 				struct list_head *head_config);
 int parse_events_load_bpf(struct parse_events_evlist *data,
 			  struct list_head *list,
-			  char *bpf_file_name);
+			  char *bpf_file_name,
+			  bool source);
 /* Provide this function for perf test */
 struct bpf_object;
 int parse_events_load_bpf_obj(struct parse_events_evlist *data,
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index 5e5d31a..eeea4e1 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -116,6 +116,7 @@ group		[^,{}/]*[{][^}]*[}][^,{}/]*
 event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
 event		[^,{}/]+
 bpf_object	.*\.(o|bpf)
+bpf_source	.*\.c
 
 num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
@@ -161,6 +162,7 @@ modifier_bp	[rwx]{1,3}
 
 {event_pmu}	|
 {bpf_object}	|
+{bpf_source}	|
 {event}		{
 			BEGIN(INITIAL);
 			REWIND(1);
@@ -267,6 +269,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
 
 {modifier_event}	{ return str(yyscanner, PE_MODIFIER_EVENT); }
 {bpf_object}		{ return str(yyscanner, PE_BPF_OBJECT); }
+{bpf_source}		{ return str(yyscanner, PE_BPF_SOURCE); }
 {name}			{ return pmu_str_check(yyscanner); }
 "/"			{ BEGIN(config); return '/'; }
 -			{ return '-'; }
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index 497f19b..ad37996 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -42,7 +42,7 @@ static inc_group_count(struct list_head *list,
 %token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
 %token PE_EVENT_NAME
 %token PE_NAME
-%token PE_BPF_OBJECT
+%token PE_BPF_OBJECT PE_BPF_SOURCE
 %token PE_MODIFIER_EVENT PE_MODIFIER_BP
 %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
 %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
@@ -55,6 +55,7 @@ static inc_group_count(struct list_head *list,
 %type <num> PE_TERM
 %type <str> PE_NAME
 %type <str> PE_BPF_OBJECT
+%type <str> PE_BPF_SOURCE
 %type <str> PE_NAME_CACHE_TYPE
 %type <str> PE_NAME_CACHE_OP_RESULT
 %type <str> PE_MODIFIER_EVENT
@@ -461,7 +462,17 @@ PE_BPF_OBJECT
 	struct list_head *list;
 
 	ALLOC_LIST(list);
-	ABORT_ON(parse_events_load_bpf(data, list, $1));
+	ABORT_ON(parse_events_load_bpf(data, list, $1, false));
+	$$ = list;
+}
+|
+PE_BPF_SOURCE
+{
+	struct parse_events_evlist *data = _data;
+	struct list_head *list;
+
+	ALLOC_LIST(list);
+	ABORT_ON(parse_events_load_bpf(data, list, $1, true));
 	$$ = list;
 }
 
-- 
1.8.3.4

[PATCH 10/31] perf test: Enforce LLVM test for BPF test

[Wang Nan]

This patch replaces the original toy BPF program with previous introduced
bpf-script-example.c. Dynamically embedded it into 'llvm-src.c'.

The newly introduced BPF program attaches a BPF program at
'sys_epoll_pwait()', and collect half samples from it. perf itself never
use that syscall, so further test can verify their result with it.

Since BPF program require LINUX_VERSION_CODE of runtime kernel, this
patch computes that code from uname.

Since the resuling BPF object is useful for further testcases, this patch
introduces 'prepare' and 'cleanup' method to tests, and makes test__llvm()
create a MAP_SHARED memory array to hold the resulting object.

Signed-off-by: He Kuang <hekuang@huawei.com>
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/tests/Build          |   9 +++-
 tools/perf/tests/builtin-test.c |   6 +++
 tools/perf/tests/llvm.c         | 104 +++++++++++++++++++++++++++++++++++-----
 tools/perf/tests/llvm.h         |  14 ++++++
 tools/perf/tests/tests.h        |   4 ++
 5 files changed, 123 insertions(+), 14 deletions(-)
 create mode 100644 tools/perf/tests/llvm.h

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 50de225..4afc8c8 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -31,9 +31,16 @@ perf-y += sample-parsing.o
 perf-y += parse-no-sample-id-all.o
 perf-y += kmod-path.o
 perf-y += thread-map.o
-perf-y += llvm.o
+perf-y += llvm.o llvm-src.o
 perf-y += topology.o
 
+$(OUTPUT)tests/llvm-src.c: tests/bpf-script-example.c
+	$(call rule_mkdir)
+	$(Q)echo '#include <tests/llvm.h>' > $@
+	$(Q)echo 'const char test_llvm__bpf_prog[] =' >> $@
+	$(Q)sed -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@
+	$(Q)echo ';' >> $@
+
 ifeq ($(ARCH),$(filter $(ARCH),x86 arm arm64))
 perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
 endif
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 66f72d3..e812a0c 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -160,6 +160,8 @@ static struct test generic_tests[] = {
 	{
 		.desc = "Test LLVM searching and compiling",
 		.func = test__llvm,
+		.prepare = test__llvm_prepare,
+		.cleanup = test__llvm_cleanup,
 	},
 	{
 		.desc = "Test topology in session",
@@ -261,7 +263,11 @@ static int __cmd_test(int argc, const char *argv[], struct intlist *skiplist)
 		}
 
 		pr_debug("\n--- start ---\n");
+		if (t->prepare)
+			t->prepare();
 		err = run_test(t);
+		if (t->cleanup)
+			t->cleanup();
 		pr_debug("---- end ----\n%s:", t->desc);
 
 		switch (err) {
diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
index 52d5597..236bf39 100644
--- a/tools/perf/tests/llvm.c
+++ b/tools/perf/tests/llvm.c
@@ -1,9 +1,13 @@
 #include <stdio.h>
+#include <sys/utsname.h>
 #include <bpf/libbpf.h>
 #include <util/llvm-utils.h>
 #include <util/cache.h>
+#include <util/util.h>
+#include <sys/mman.h>
 #include "tests.h"
 #include "debug.h"
+#include "llvm.h"
 
 static int perf_config_cb(const char *var, const char *val,
 			  void *arg __maybe_unused)
@@ -11,16 +15,6 @@ static int perf_config_cb(const char *var, const char *val,
 	return perf_default_config(var, val, arg);
 }
 
-/*
- * Randomly give it a "version" section since we don't really load it
- * into kernel
- */
-static const char test_bpf_prog[] =
-	"__attribute__((section(\"do_fork\"), used)) "
-	"int fork(void *ctx) {return 0;} "
-	"char _license[] __attribute__((section(\"license\"), used)) = \"GPL\";"
-	"int _version __attribute__((section(\"version\"), used)) = 0x40100;";
-
 #ifdef HAVE_LIBBPF_SUPPORT
 static int test__bpf_parsing(void *obj_buf, size_t obj_buf_sz)
 {
@@ -41,12 +35,44 @@ static int test__bpf_parsing(void *obj_buf __maybe_unused,
 }
 #endif
 
+static char *
+compose_source(void)
+{
+	struct utsname utsname;
+	int version, patchlevel, sublevel, err;
+	unsigned long version_code;
+	char *code;
+
+	if (uname(&utsname))
+		return NULL;
+
+	err = sscanf(utsname.release, "%d.%d.%d",
+		     &version, &patchlevel, &sublevel);
+	if (err != 3) {
+		fprintf(stderr, " (Can't get kernel version from uname '%s')",
+			utsname.release);
+		return NULL;
+	}
+
+	version_code = (version << 16) + (patchlevel << 8) + sublevel;
+	err = asprintf(&code, "#define LINUX_VERSION_CODE 0x%08lx;\n%s",
+		       version_code, test_llvm__bpf_prog);
+	if (err < 0)
+		return NULL;
+
+	return code;
+}
+
+#define SHARED_BUF_INIT_SIZE	(1 << 20)
+struct test_llvm__bpf_result *p_test_llvm__bpf_result;
+
 int test__llvm(void)
 {
 	char *tmpl_new, *clang_opt_new;
 	void *obj_buf;
 	size_t obj_buf_sz;
 	int err, old_verbose;
+	char *source;
 
 	perf_config(perf_config_cb, NULL);
 
@@ -73,10 +99,22 @@ int test__llvm(void)
 	if (!llvm_param.clang_opt)
 		llvm_param.clang_opt = strdup("");
 
-	err = asprintf(&tmpl_new, "echo '%s' | %s", test_bpf_prog,
-		       llvm_param.clang_bpf_cmd_template);
-	if (err < 0)
+	source = compose_source();
+	if (!source) {
+		pr_err("Failed to compose source code\n");
+		return -1;
+	}
+
+	/* Quote __EOF__ so strings in source won't be expanded by shell */
+	err = asprintf(&tmpl_new, "cat << '__EOF__' | %s\n%s\n__EOF__\n",
+		       llvm_param.clang_bpf_cmd_template, source);
+	free(source);
+	source = NULL;
+	if (err < 0) {
+		pr_err("Failed to alloc new template\n");
 		return -1;
+	}
+
 	err = asprintf(&clang_opt_new, "-xc %s", llvm_param.clang_opt);
 	if (err < 0)
 		return -1;
@@ -93,6 +131,46 @@ int test__llvm(void)
 	}
 
 	err = test__bpf_parsing(obj_buf, obj_buf_sz);
+	if (!err && p_test_llvm__bpf_result) {
+		if (obj_buf_sz > SHARED_BUF_INIT_SIZE) {
+			pr_err("Resulting object too large\n");
+		} else {
+			p_test_llvm__bpf_result->size = obj_buf_sz;
+			memcpy(p_test_llvm__bpf_result->object,
+			       obj_buf, obj_buf_sz);
+		}
+	}
 	free(obj_buf);
 	return err;
 }
+
+void test__llvm_prepare(void)
+{
+	p_test_llvm__bpf_result = mmap(NULL, SHARED_BUF_INIT_SIZE,
+				       PROT_READ | PROT_WRITE,
+				       MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+	if (!p_test_llvm__bpf_result)
+		return;
+	memset((void *)p_test_llvm__bpf_result, '\0', SHARED_BUF_INIT_SIZE);
+}
+
+void test__llvm_cleanup(void)
+{
+	unsigned long boundary, buf_end;
+
+	if (!p_test_llvm__bpf_result)
+		return;
+	if (p_test_llvm__bpf_result->size == 0) {
+		munmap((void *)p_test_llvm__bpf_result, SHARED_BUF_INIT_SIZE);
+		p_test_llvm__bpf_result = NULL;
+		return;
+	}
+
+	buf_end = (unsigned long)p_test_llvm__bpf_result + SHARED_BUF_INIT_SIZE;
+
+	boundary = (unsigned long)(p_test_llvm__bpf_result);
+	boundary += p_test_llvm__bpf_result->size;
+	boundary = (boundary + (page_size - 1)) &
+			(~((unsigned long)page_size - 1));
+	munmap((void *)boundary, buf_end - boundary);
+}
diff --git a/tools/perf/tests/llvm.h b/tools/perf/tests/llvm.h
new file mode 100644
index 0000000..1e89e46
--- /dev/null
+++ b/tools/perf/tests/llvm.h
@@ -0,0 +1,14 @@
+#ifndef PERF_TEST_LLVM_H
+#define PERF_TEST_LLVM_H
+
+#include <stddef.h> /* for size_t */
+
+struct test_llvm__bpf_result {
+	size_t size;
+	char object[];
+};
+
+extern struct test_llvm__bpf_result *p_test_llvm__bpf_result;
+extern const char test_llvm__bpf_prog[];
+
+#endif
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index c804869..a848802 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -27,6 +27,8 @@ enum {
 struct test {
 	const char *desc;
 	int (*func)(void);
+	void (*prepare)(void);
+	void (*cleanup)(void);
 };
 
 /* Tests */
@@ -66,6 +68,8 @@ int test__fdarray__add(void);
 int test__kmod_path__parse(void);
 int test__thread_map(void);
 int test__llvm(void);
+void test__llvm_prepare(void);
+void test__llvm_cleanup(void);
 int test_session_topology(void);
 
 #if defined(__arm__) || defined(__aarch64__)
-- 
1.8.3.4

[PATCH 11/31] perf test: Add 'perf test BPF'

[Wang Nan]

This patch adds BPF testcase for testing BPF event filtering.

By utilizing the result of 'perf test LLVM', this patch compiles the
eBPF sample program then test it ability. The BPF script in 'perf test
LLVM' collects half of execution of epoll_pwait(). This patch runs 111
times of it, so the resule should contains 56 samples.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/tests/Build          |   1 +
 tools/perf/tests/bpf.c          | 171 ++++++++++++++++++++++++++++++++++++++++
 tools/perf/tests/builtin-test.c |   4 +
 tools/perf/tests/llvm.c         |  19 +++++
 tools/perf/tests/llvm.h         |   1 +
 tools/perf/tests/tests.h        |   1 +
 tools/perf/util/bpf-loader.c    |  14 ++++
 tools/perf/util/bpf-loader.h    |   9 +++
 8 files changed, 220 insertions(+)
 create mode 100644 tools/perf/tests/bpf.c

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 4afc8c8..d0278a9 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -32,6 +32,7 @@ perf-y += parse-no-sample-id-all.o
 perf-y += kmod-path.o
 perf-y += thread-map.o
 perf-y += llvm.o llvm-src.o
+perf-y += bpf.o
 perf-y += topology.o
 
 $(OUTPUT)tests/llvm-src.c: tests/bpf-script-example.c
diff --git a/tools/perf/tests/bpf.c b/tools/perf/tests/bpf.c
new file mode 100644
index 0000000..d7cdc84
--- /dev/null
+++ b/tools/perf/tests/bpf.c
@@ -0,0 +1,171 @@
+#include <stdio.h>
+#include <sys/epoll.h>
+#include <util/bpf-loader.h>
+#include <util/evlist.h>
+#include "tests.h"
+#include "llvm.h"
+#include "debug.h"
+#define NR_ITERS       111
+
+#ifdef HAVE_LIBBPF_SUPPORT
+
+static int epoll_pwait_loop(void)
+{
+	int i;
+
+	/* Should fail NR_ITERS times */
+	for (i = 0; i < NR_ITERS; i++)
+		epoll_pwait(-(i + 1), NULL, 0, 0, NULL);
+	return 0;
+}
+
+static struct bpf_object *prepare_bpf(void *obj_buf, size_t obj_buf_sz)
+{
+	struct bpf_object *obj;
+
+	obj = bpf__prepare_load_buffer(obj_buf, obj_buf_sz, "[buffer]");
+	if (IS_ERR(obj)) {
+		fprintf(stderr, " (compile failed)");
+		return NULL;
+	}
+	return obj;
+}
+
+static int do_test(struct bpf_object *obj)
+{
+	struct record_opts opts = {
+		.target = {
+			.uid = UINT_MAX,
+			.uses_mmap = true,
+		},
+		.freq	      = 0,
+		.mmap_pages   = 256,
+		.default_interval = 1,
+	};
+
+	int i, err = 0, count = 0;
+	char pid[16];
+	char sbuf[STRERR_BUFSIZE];
+	struct perf_evlist *evlist;
+
+	struct parse_events_evlist parse_evlist;
+	struct parse_events_error parse_error;
+
+	bzero(&parse_error, sizeof(parse_error));
+	bzero(&parse_evlist, sizeof(parse_evlist));
+	parse_evlist.error = &parse_error;
+	INIT_LIST_HEAD(&parse_evlist.list);
+
+	err = parse_events_load_bpf_obj(&parse_evlist, &parse_evlist.list, obj);
+	if (err || list_empty(&parse_evlist.list)) {
+		fprintf(stderr, " (Failed to add events selected by BPF)");
+		if (!err)
+			err = -EINVAL;
+		goto out;
+	}
+
+	snprintf(pid, sizeof(pid), "%d", getpid());
+	pid[sizeof(pid) - 1] = '\0';
+	opts.target.tid = opts.target.pid = pid;
+
+	/* Instead of perf_evlist__new_default, don't add default events */
+	evlist = perf_evlist__new();
+	if (!evlist) {
+		pr_debug("No ehough memory to create evlist\n");
+		return -ENOMEM;
+	}
+
+	err = perf_evlist__create_maps(evlist, &opts.target);
+	if (err < 0) {
+		pr_debug("Not enough memory to create thread/cpu maps\n");
+		goto out_delete_evlist;
+	}
+
+	perf_evlist__splice_list_tail(evlist, &parse_evlist.list);
+	evlist->nr_groups = parse_evlist.nr_groups;
+
+	perf_evlist__config(evlist, &opts);
+
+	err = perf_evlist__open(evlist);
+	if (err < 0) {
+		pr_debug("perf_evlist__open: %s\n",
+			 strerror_r(errno, sbuf, sizeof(sbuf)));
+		goto out_delete_evlist;
+	}
+
+	err = perf_evlist__mmap(evlist, opts.mmap_pages, false);
+	if (err < 0) {
+		pr_debug("perf_evlist__mmap: %s\n",
+			 strerror_r(errno, sbuf, sizeof(sbuf)));
+		goto out_delete_evlist;
+	}
+
+	perf_evlist__enable(evlist);
+	epoll_pwait_loop();
+	perf_evlist__disable(evlist);
+
+	for (i = 0; i < evlist->nr_mmaps; i++) {
+		union perf_event *event;
+
+		while ((event = perf_evlist__mmap_read(evlist, i)) != NULL) {
+			const u32 type = event->header.type;
+
+			if (type == PERF_RECORD_SAMPLE)
+				count ++;
+		}
+	}
+
+	if (count != (NR_ITERS + 1) / 2) {
+		fprintf(stderr, " (filter result incorrect)");
+		err = -EBADF;
+	}
+
+out_delete_evlist:
+	perf_evlist__delete(evlist);
+out:
+	if (err)
+		return TEST_FAIL;
+	return 0;
+}
+
+int test__bpf(void)
+{
+	int err;
+	void *obj_buf;
+	size_t obj_buf_sz;
+	struct bpf_object *obj;
+
+	if (geteuid() != 0) {
+		fprintf(stderr, " (try run as root)");
+		return TEST_SKIP;
+	}
+
+	test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz);
+	if (!obj_buf || !obj_buf_sz) {
+		if (verbose == 0)
+			fprintf(stderr, " (fix 'perf test LLVM' first)");
+		return TEST_SKIP;
+	}
+
+	obj = prepare_bpf(obj_buf, obj_buf_sz);
+	if (!obj) {
+		err = -EINVAL;
+		goto out;
+	}
+
+	err = do_test(obj);
+	if (err)
+		goto out;
+out:
+	bpf__clear();
+	if (err)
+		return TEST_FAIL;
+	return 0;
+}
+
+#else
+int test__bpf(void)
+{
+	return TEST_SKIP;
+}
+#endif
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index e812a0c..4828716 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -168,6 +168,10 @@ static struct test generic_tests[] = {
 		.func = test_session_topology,
 	},
 	{
+		.desc = "Test BPF filter",
+		.func = test__bpf,
+	},
+	{
 		.func = NULL,
 	},
 };
diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
index 236bf39..fd5fdb0 100644
--- a/tools/perf/tests/llvm.c
+++ b/tools/perf/tests/llvm.c
@@ -174,3 +174,22 @@ void test__llvm_cleanup(void)
 			(~((unsigned long)page_size - 1));
 	munmap((void *)boundary, buf_end - boundary);
 }
+
+void
+test_llvm__fetch_bpf_obj(void **p_obj_buf, size_t *p_obj_buf_sz)
+{
+	*p_obj_buf = NULL;
+	*p_obj_buf_sz = 0;
+
+	if (!p_test_llvm__bpf_result) {
+		test__llvm_prepare();
+		test__llvm();
+		test__llvm_cleanup();
+	}
+
+	if (!p_test_llvm__bpf_result)
+		return;
+
+	*p_obj_buf = p_test_llvm__bpf_result->object;
+	*p_obj_buf_sz = p_test_llvm__bpf_result->size;
+}
diff --git a/tools/perf/tests/llvm.h b/tools/perf/tests/llvm.h
index 1e89e46..2fd7ed6 100644
--- a/tools/perf/tests/llvm.h
+++ b/tools/perf/tests/llvm.h
@@ -10,5 +10,6 @@ struct test_llvm__bpf_result {
 
 extern struct test_llvm__bpf_result *p_test_llvm__bpf_result;
 extern const char test_llvm__bpf_prog[];
+void test_llvm__fetch_bpf_obj(void **p_obj_buf, size_t *p_obj_buf_sz);
 
 #endif
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index a848802..15c46e1 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -70,6 +70,7 @@ int test__thread_map(void);
 int test__llvm(void);
 void test__llvm_prepare(void);
 void test__llvm_cleanup(void);
+int test__bpf(void);
 int test_session_topology(void);
 
 #if defined(__arm__) || defined(__aarch64__)
diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index ba6f752..f4c690f 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -34,6 +34,20 @@ struct bpf_prog_priv {
 	struct perf_probe_event pev;
 };
 
+struct bpf_object *
+bpf__prepare_load_buffer(void *obj_buf, size_t obj_buf_sz, const char *name)
+{
+	struct bpf_object *obj;
+
+	obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, name);
+	if (!obj) {
+		pr_debug("bpf: failed to load buffer\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	return obj;
+}
+
 struct bpf_object *bpf__prepare_load(const char *filename, bool source)
 {
 	struct bpf_object *obj;
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index ccd8d7f..d8f1945 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -19,6 +19,8 @@ typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
 
 #ifdef HAVE_LIBBPF_SUPPORT
 struct bpf_object *bpf__prepare_load(const char *filename, bool source);
+struct bpf_object *bpf__prepare_load_buffer(void *obj_buf, size_t obj_buf_sz,
+					    const char *name);
 
 void bpf__clear(void);
 
@@ -41,6 +43,13 @@ bpf__prepare_load(const char *filename __maybe_unused,
 	return ERR_PTR(-ENOTSUP);
 }
 
+static inline struct bpf_object *
+bpf__prepare_load_buffer(void *obj_buf __maybe_unused,
+					   size_t obj_buf_sz __maybe_unused)
+{
+	return ERR_PTR(-ENOTSUP);
+}
+
 static inline void bpf__clear(void) { }
 
 static inline int bpf__probe(struct bpf_object *obj __maybe_unused) { return 0;}
-- 
1.8.3.4

[PATCH 12/31] perf probe: Reset args and nargs for probe_trace_event when failure

[Wang Nan]

When failure occures in add_probe_trace_event(), args in
probe_trace_event is incomplete. Since information in it may be used
in futher, this patch frees the allocated memory and set it to NULL
to avoid dangling pointer.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/probe-finder.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index bd8f03d..b1581d7 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -1235,6 +1235,10 @@ static int add_probe_trace_event(Dwarf_Die *sc_die, struct probe_finder *pf)
 
 end:
 	free(args);
+	if (ret) {
+		tev->nargs = 0;
+		zfree(&tev->args);
+	}
 	return ret;
 }
 
-- 
1.8.3.4

[PATCH 13/31] bpf tools: Load a program with different instances using preprocessor

[Wang Nan]

In this patch, caller of libbpf is able to control the loaded programs
by installing a preprocessor callback for a BPF program. With
preprocessor, different instances can be created from one BPF program.

This patch will be used by perf to generate different prologue for
different 'struct probe_trace_event' instances matched by one
'struct perf_probe_event'.

bpf_program__set_prep() is added to support this feature. Caller
should pass libbpf the number of instances should be created and a
preprocessor function which will be called when doing real loading.
The callback should return instructions arrays for each instances.

fd field in bpf_programs is replaced by instance, which has an nr field
and fds array. bpf_program__nth_fd() is introduced for read fd of
instances. Old interface bpf_program__fd() is reimplemented by
returning the first fd.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/lib/bpf/libbpf.c | 143 +++++++++++++++++++++++++++++++++++++++++++++----
 tools/lib/bpf/libbpf.h |  22 ++++++++
 2 files changed, 156 insertions(+), 9 deletions(-)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 4252fc2..6a07b26 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -98,7 +98,11 @@ struct bpf_program {
 	} *reloc_desc;
 	int nr_reloc;
 
-	int fd;
+	struct {
+		int nr;
+		int *fds;
+	} instance;
+	bpf_program_prep_t preprocessor;
 
 	struct bpf_object *obj;
 	void *priv;
@@ -152,10 +156,24 @@ struct bpf_object {
 
 static void bpf_program__unload(struct bpf_program *prog)
 {
+	int i;
+
 	if (!prog)
 		return;
 
-	zclose(prog->fd);
+	/*
+	 * If the object is opened but the program is never loaded,
+	 * it is possible that prog->instance.nr == -1.
+	 */
+	if (prog->instance.nr > 0) {
+		for (i = 0; i < prog->instance.nr; i++)
+			zclose(prog->instance.fds[i]);
+	} else if (prog->instance.nr != -1)
+		pr_warning("Internal error: instance.nr is %d\n",
+			   prog->instance.nr);
+
+	prog->instance.nr = -1;
+	zfree(&prog->instance.fds);
 }
 
 static void bpf_program__exit(struct bpf_program *prog)
@@ -206,7 +224,8 @@ bpf_program__init(void *data, size_t size, char *name, int idx,
 	memcpy(prog->insns, data,
 	       prog->insns_cnt * sizeof(struct bpf_insn));
 	prog->idx = idx;
-	prog->fd = -1;
+	prog->instance.fds = NULL;
+	prog->instance.nr = -1;
 
 	return 0;
 errout:
@@ -795,13 +814,71 @@ static int
 bpf_program__load(struct bpf_program *prog,
 		  char *license, u32 kern_version)
 {
-	int err, fd;
+	int err = 0, fd, i;
+
+	if (prog->instance.nr < 0 || !prog->instance.fds) {
+		if (prog->preprocessor) {
+			pr_warning("Internal error: can't load program '%s'\n",
+				   prog->section_name);
+			return -EINVAL;
+		}
+
+		prog->instance.fds = malloc(sizeof(int));
+		if (!prog->instance.fds) {
+			pr_warning("No enough memory for fds\n");
+			return -ENOMEM;
+		}
+		prog->instance.nr = 1;
+		prog->instance.fds[0] = -1;
+	}
+
+	if (!prog->preprocessor) {
+		if (prog->instance.nr != 1)
+			pr_warning("Program '%s' inconsistent: nr(%d) not 1\n",
+				   prog->section_name, prog->instance.nr);
 
-	err = load_program(prog->insns, prog->insns_cnt,
-			   license, kern_version, &fd);
-	if (!err)
-		prog->fd = fd;
+		err = load_program(prog->insns, prog->insns_cnt,
+				   license, kern_version, &fd);
+		if (!err)
+			prog->instance.fds[0] = fd;
+		goto out;
+	}
+
+	for (i = 0; i < prog->instance.nr; i++) {
+		struct bpf_prog_prep_result result;
+		bpf_program_prep_t preprocessor = prog->preprocessor;
+
+		bzero(&result, sizeof(result));
+		err = preprocessor(prog, i, prog->insns,
+				   prog->insns_cnt, &result);
+		if (err) {
+			pr_warning("Preprocessing %dth instance of program '%s' failed\n",
+					i, prog->section_name);
+			goto out;
+		}
+
+		if (!result.new_insn_ptr || !result.new_insn_cnt) {
+			pr_debug("Skip loading %dth instance of program '%s'\n",
+					i, prog->section_name);
+			prog->instance.fds[i] = -1;
+			continue;
+		}
+
+		err = load_program(result.new_insn_ptr,
+				   result.new_insn_cnt,
+				   license, kern_version, &fd);
+
+		if (err) {
+			pr_warning("Loading %dth instance of program '%s' failed\n",
+					i, prog->section_name);
+			goto out;
+		}
 
+		if (result.pfd)
+			*result.pfd = fd;
+		prog->instance.fds[i] = fd;
+	}
+out:
 	if (err)
 		pr_warning("failed to load program '%s'\n",
 			   prog->section_name);
@@ -1052,5 +1129,53 @@ const char *bpf_program__title(struct bpf_program *prog, bool dup)
 
 int bpf_program__fd(struct bpf_program *prog)
 {
-	return prog->fd;
+	return bpf_program__nth_fd(prog, 0);
+}
+
+int bpf_program__set_prep(struct bpf_program *prog, int nr_instance,
+			  bpf_program_prep_t prep)
+{
+	int *instance_fds;
+
+	if (nr_instance <= 0 || !prep)
+		return -EINVAL;
+
+	if (prog->instance.nr > 0 || prog->instance.fds) {
+		pr_warning("Can't set pre-processor after loading\n");
+		return -EINVAL;
+	}
+
+	instance_fds = malloc(sizeof(int) * nr_instance);
+	if (!instance_fds) {
+		pr_warning("alloc memory failed for instance of fds\n");
+		return -ENOMEM;
+	}
+
+	/* fill all fd with -1 */
+	memset(instance_fds, 0xff, sizeof(int) * nr_instance);
+
+	prog->instance.nr = nr_instance;
+	prog->instance.fds = instance_fds;
+	prog->preprocessor = prep;
+	return 0;
+}
+
+int bpf_program__nth_fd(struct bpf_program *prog, int n)
+{
+	int fd;
+
+	if (n >= prog->instance.nr || n < 0) {
+		pr_warning("Can't get the %dth fd from program %s: only %d instances\n",
+			   n, prog->section_name, prog->instance.nr);
+		return -EINVAL;
+	}
+
+	fd = prog->instance.fds[n];
+	if (fd < 0) {
+		pr_warning("%dth instance of program '%s' is invalid\n",
+			   n, prog->section_name);
+		return -ENOENT;
+	}
+
+	return fd;
 }
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index f16170c..d82b89e 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -67,6 +67,28 @@ const char *bpf_program__title(struct bpf_program *prog, bool dup);
 
 int bpf_program__fd(struct bpf_program *prog);
 
+struct bpf_insn;
+struct bpf_prog_prep_result {
+	/*
+	 * If not NULL, load new instruction array.
+	 * If set to NULL, don't load this instance.
+	 */
+	struct bpf_insn *new_insn_ptr;
+	int new_insn_cnt;
+
+	/* If not NULL, result fd is set to it */
+	int *pfd;
+};
+
+typedef int (*bpf_program_prep_t)(struct bpf_program *, int n,
+				  struct bpf_insn *, int insn_cnt,
+				  struct bpf_prog_prep_result *res);
+
+int bpf_program__set_prep(struct bpf_program *prog, int nr_instance,
+			  bpf_program_prep_t prep);
+
+int bpf_program__nth_fd(struct bpf_program *prog, int n);
+
 /*
  * We don't need __attribute__((packed)) now since it is
  * unnecessary for 'bpf_map_def' because they are all aligned.
-- 
1.8.3.4

[PATCH 14/31] perf tools: Add BPF_PROLOGUE config options for further patches

[Wang Nan]

If both LIBBPF and DWARF are detected, it is possible to create prologue
for eBPF programs to help them accessing kernel data. HAVE_BPF_PROLOGUE
and CONFIG_BPF_PROLOGUE is added as flags for this feature.

PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET indicates an architecture
supports converting name of a register to its offset in
'struct pt_regs'. Without this support, BPF_PROLOGUE should be turned off.

HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET is introduced as the corresponding
CFLAGS of PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/config/Makefile | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index de89ec5..6eb9a95 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -318,6 +318,18 @@ ifndef NO_LIBELF
       CFLAGS += -DHAVE_LIBBPF_SUPPORT
       $(call detected,CONFIG_LIBBPF)
     endif
+
+    ifndef NO_DWARF
+      ifdef PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET
+        CFLAGS += -DHAVE_BPF_PROLOGUE
+        $(call detected,CONFIG_BPF_PROLOGUE)
+      else
+        msg := $(warning BPF prologue is not supported by architecture $(ARCH), missing regs_query_register_offset());
+      endif
+    else
+      msg := $(warning DWARF support is off, BPF prologue is disabled);
+    endif
+
   endif # NO_LIBBPF
 endif # NO_LIBELF
 
-- 
1.8.3.4

[PATCH 15/31] perf tools: Compile dwarf-regs.c if CONFIG_BPF_PROLOGUE is on

[Wang Nan]

regs_query_register_offset() in dwarf-regs.c is required by BPF prologue.
Make it be compiled if CONFIG_BPF_PROLOGUE is on to avoid building failure
when CONFIG_BPF_PROLOGUE is on but CONFIG_DWARF is not set.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/arch/x86/util/Build | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/arch/x86/util/Build b/tools/perf/arch/x86/util/Build
index ff63649..4659703 100644
--- a/tools/perf/arch/x86/util/Build
+++ b/tools/perf/arch/x86/util/Build
@@ -5,6 +5,7 @@ libperf-y += kvm-stat.o
 libperf-y += perf_regs.o
 
 libperf-$(CONFIG_DWARF) += dwarf-regs.o
+libperf-$(CONFIG_BPF_PROLOGUE) += dwarf-regs.o
 
 libperf-$(CONFIG_LIBUNWIND)          += unwind-libunwind.o
 libperf-$(CONFIG_LIBDW_DWARF_UNWIND) += unwind-libdw.o
-- 
1.8.3.4

[PATCH 16/31] perf tools: Add prologue for BPF programs for fetching arguments

[Wang Nan]

From: He Kuang <hekuang@huawei.com>

This patch generates prologue for a BPF program which fetch arguments
for it. With this patch, the program can have arguments as follow:

 SEC("lock_page=__lock_page page->flags")
 int lock_page(struct pt_regs *ctx, int err, unsigned long flags)
 {
	 return 1;
 }

This patch passes at most 3 arguments from r3, r4 and r5. r1 is still
the ctx pointer. r2 is used to indicate the successfulness of
dereferencing.

This patch uses r6 to hold ctx (struct pt_regs) and r7 to hold stack
pointer for result. Result of each arguments first store on stack:

 low address
 BPF_REG_FP - 24  ARG3
 BPF_REG_FP - 16  ARG2
 BPF_REG_FP - 8   ARG1
 BPF_REG_FP
 high address

Then loaded into r3, r4 and r5.

The output prologue for offn(...off2(off1(reg)))) should be:

     r6 <- r1			// save ctx into a callee saved register
     r7 <- fp
     r7 <- r7 - stack_offset	// pointer to result slot
     /* load r3 with the offset in pt_regs of 'reg' */
     (r7) <- r3			// make slot valid
     r3 <- r3 + off1		// prepare to read unsafe pointer
     r2 <- 8
     r1 <- r7			// result put onto stack
     call probe_read		// read unsafe pointer
     jnei r0, 0, err		// error checking
     r3 <- (r7)			// read result
     r3 <- r3 + off2		// prepare to read unsafe pointer
     r2 <- 8
     r1 <- r7
     call probe_read
     jnei r0, 0, err
     ...
     /* load r2, r3, r4 from stack */
     goto success
err:
     r2 <- 1
     /* load r3, r4, r5 with 0 */
     goto usercode
success:
     r2 <- 0
usercode:
     r1 <- r6	// restore ctx
     // original user code

If all of arguments reside in register (dereferencing is not
required), gen_prologue_fastpath() will be used to create
fast prologue:

     r3 <- (r1 + offset of reg1)
     r4 <- (r1 + offset of reg2)
     r5 <- (r1 + offset of reg3)
     r2 <- 0

P.S.

eBPF calling convention is defined as:

* r0		- return value from in-kernel function, and exit value
                  for eBPF program
* r1 - r5	- arguments from eBPF program to in-kernel function
* r6 - r9	- callee saved registers that in-kernel function will
                  preserve
* r10		- read-only frame pointer to access stack

Signed-off-by: He Kuang <hekuang@huawei.com>
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/Build          |   1 +
 tools/perf/util/bpf-prologue.c | 443 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-prologue.h |  34 ++++
 3 files changed, 478 insertions(+)
 create mode 100644 tools/perf/util/bpf-prologue.c
 create mode 100644 tools/perf/util/bpf-prologue.h

diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 591b3fe..b9d56f2 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -88,6 +88,7 @@ libperf-y += parse-branch-options.o
 libperf-y += parse-regs-options.o
 
 libperf-$(CONFIG_LIBBPF) += bpf-loader.o
+libperf-$(CONFIG_BPF_PROLOGUE) += bpf-prologue.o
 libperf-$(CONFIG_LIBELF) += symbol-elf.o
 libperf-$(CONFIG_LIBELF) += probe-file.o
 libperf-$(CONFIG_LIBELF) += probe-event.o
diff --git a/tools/perf/util/bpf-prologue.c b/tools/perf/util/bpf-prologue.c
new file mode 100644
index 0000000..e4adb18
--- /dev/null
+++ b/tools/perf/util/bpf-prologue.c
@@ -0,0 +1,443 @@
+/*
+ * bpf-prologue.c
+ *
+ * Copyright (C) 2015 He Kuang <hekuang@huawei.com>
+ * Copyright (C) 2015 Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2015 Huawei Inc.
+ */
+
+#include <bpf/libbpf.h>
+#include "perf.h"
+#include "debug.h"
+#include "bpf-prologue.h"
+#include "probe-finder.h"
+#include <dwarf-regs.h>
+#include <linux/filter.h>
+
+#define BPF_REG_SIZE		8
+
+#define JMP_TO_ERROR_CODE	-1
+#define JMP_TO_SUCCESS_CODE	-2
+#define JMP_TO_USER_CODE	-3
+
+struct bpf_insn_pos {
+	struct bpf_insn *begin;
+	struct bpf_insn *end;
+	struct bpf_insn *pos;
+};
+
+static inline int
+pos_get_cnt(struct bpf_insn_pos *pos)
+{
+	return pos->pos - pos->begin;
+}
+
+static int
+append_insn(struct bpf_insn new_insn, struct bpf_insn_pos *pos)
+{
+	if (!pos->pos)
+		return -ERANGE;
+
+	if (pos->pos + 1 >= pos->end) {
+		pr_err("bpf prologue: prologue too long\n");
+		pos->pos = NULL;
+		return -ERANGE;
+	}
+
+	*(pos->pos)++ = new_insn;
+	return 0;
+}
+
+static int
+check_pos(struct bpf_insn_pos *pos)
+{
+	if (!pos->pos || pos->pos >= pos->end)
+		return -ERANGE;
+	return 0;
+}
+
+/* Give it a shorter name */
+#define ins(i, p) append_insn((i), (p))
+
+/*
+ * Give a register name (in 'reg'), generate instruction to
+ * load register into an eBPF register rd:
+ *   'ldd target_reg, offset(ctx_reg)', where:
+ * ctx_reg is pre initialized to pointer of 'struct pt_regs'.
+ */
+static int
+gen_ldx_reg_from_ctx(struct bpf_insn_pos *pos, int ctx_reg,
+		     const char *reg, int target_reg)
+{
+	int offset = regs_query_register_offset(reg);
+
+	if (offset < 0) {
+		pr_err("bpf: prologue: failed to get register %s\n",
+		       reg);
+		return -1;
+	}
+	ins(BPF_LDX_MEM(BPF_DW, target_reg, ctx_reg, offset), pos);
+
+	if (check_pos(pos))
+		return -ERANGE;
+	return 0;
+}
+
+/*
+ * Generate a BPF_FUNC_probe_read function call.
+ *
+ * src_base_addr_reg is a register holding base address,
+ * dst_addr_reg is a register holding dest address (on stack),
+ * result is:
+ *
+ *  *[dst_addr_reg] = *([src_base_addr_reg] + offset)
+ *
+ * Arguments of BPF_FUNC_probe_read:
+ *     ARG1: ptr to stack (dest)
+ *     ARG2: size (8)
+ *     ARG3: unsafe ptr (src)
+ */
+static int
+gen_read_mem(struct bpf_insn_pos *pos,
+	     int src_base_addr_reg,
+	     int dst_addr_reg,
+	     long offset)
+{
+	/* mov arg3, src_base_addr_reg */
+	if (src_base_addr_reg != BPF_REG_ARG3)
+		ins(BPF_MOV64_REG(BPF_REG_ARG3, src_base_addr_reg), pos);
+	/* add arg3, #offset */
+	if (offset)
+		ins(BPF_ALU64_IMM(BPF_ADD, BPF_REG_ARG3, offset), pos);
+
+	/* mov arg2, #reg_size */
+	ins(BPF_ALU64_IMM(BPF_MOV, BPF_REG_ARG2, BPF_REG_SIZE), pos);
+
+	/* mov arg1, dst_addr_reg */
+	if (dst_addr_reg != BPF_REG_ARG1)
+		ins(BPF_MOV64_REG(BPF_REG_ARG1, dst_addr_reg), pos);
+
+	/* Call probe_read  */
+	ins(BPF_EMIT_CALL(BPF_FUNC_probe_read), pos);
+	/*
+	 * Error processing: if read fail, goto error code,
+	 * will be relocated. Target should be the start of
+	 * error processing code.
+	 */
+	ins(BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, JMP_TO_ERROR_CODE),
+	    pos);
+
+	if (check_pos(pos))
+		return -ERANGE;
+	return 0;
+}
+
+/*
+ * Each arg should be bare register. Fetch and save them into argument
+ * registers (r3 - r5).
+ *
+ * BPF_REG_1 should have been initialized with pointer to
+ * 'struct pt_regs'.
+ */
+static int
+gen_prologue_fastpath(struct bpf_insn_pos *pos,
+		      struct probe_trace_arg *args, int nargs)
+{
+	int i;
+
+	for (i = 0; i < nargs; i++)
+		if (gen_ldx_reg_from_ctx(pos, BPF_REG_1, args[i].value,
+					 BPF_PROLOGUE_START_ARG_REG + i))
+			goto errout;
+
+	if (check_pos(pos))
+		goto errout;
+	return 0;
+errout:
+	return -1;
+}
+
+/*
+ * Slow path:
+ *   At least one argument has the form of 'offset($rx)'.
+ *
+ * Following code first stores them into stack, then loads all of then
+ * to r2 - r5.
+ * Before final loading, the final result should be:
+ *
+ * low address
+ * BPF_REG_FP - 24  ARG3
+ * BPF_REG_FP - 16  ARG2
+ * BPF_REG_FP - 8   ARG1
+ * BPF_REG_FP
+ * high address
+ *
+ * For each argument (described as: offn(...off2(off1(reg)))),
+ * generates following code:
+ *
+ *  r7 <- fp
+ *  r7 <- r7 - stack_offset  // Ideal code should initialize r7 using
+ *                           // fp before generating args. However,
+ *                           // eBPF won't regard r7 as stack pointer
+ *                           // if it is generated by minus 8 from
+ *                           // another stack pointer except fp.
+ *                           // This is why we have to set r7
+ *                           // to fp for each variable.
+ *  r3 <- value of 'reg'-> generated using gen_ldx_reg_from_ctx()
+ *  (r7) <- r3       // skip following instructions for bare reg
+ *  r3 <- r3 + off1  . // skip if off1 == 0
+ *  r2 <- 8           \
+ *  r1 <- r7           |-> generated by gen_read_mem()
+ *  call probe_read    /
+ *  jnei r0, 0, err  ./
+ *  r3 <- (r7)
+ *  r3 <- r3 + off2  . // skip if off2 == 0
+ *  r2 <- 8           \  // r2 may be broken by probe_read, so set again
+ *  r1 <- r7           |-> generated by gen_read_mem()
+ *  call probe_read    /
+ *  jnei r0, 0, err  ./
+ *  ...
+ */
+static int
+gen_prologue_slowpath(struct bpf_insn_pos *pos,
+		      struct probe_trace_arg *args, int nargs)
+{
+	int i;
+
+	for (i = 0; i < nargs; i++) {
+		struct probe_trace_arg *arg = &args[i];
+		const char *reg = arg->value;
+		struct probe_trace_arg_ref *ref = NULL;
+		int stack_offset = (i + 1) * -8;
+
+		pr_debug("prologue: fetch arg %d, base reg is %s\n",
+			 i, reg);
+
+		/* value of base register is stored into ARG3 */
+		if (gen_ldx_reg_from_ctx(pos, BPF_REG_CTX, reg,
+					 BPF_REG_ARG3)) {
+			pr_err("prologue: failed to get offset of register %s\n",
+			       reg);
+			goto errout;
+		}
+
+		/* Make r7 the stack pointer. */
+		ins(BPF_MOV64_REG(BPF_REG_7, BPF_REG_FP), pos);
+		/* r7 += -8 */
+		ins(BPF_ALU64_IMM(BPF_ADD, BPF_REG_7, stack_offset), pos);
+		/*
+		 * Store r3 (base register) onto stack
+		 * Ensure fp[offset] is set.
+		 * fp is the only valid base register when storing
+		 * into stack. We are not allowed to use r7 as base
+		 * register here.
+		 */
+		ins(BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_ARG3,
+				stack_offset), pos);
+
+		ref = arg->ref;
+		while (ref) {
+			pr_debug("prologue: arg %d: offset %ld\n",
+				 i, ref->offset);
+			if (gen_read_mem(pos, BPF_REG_3, BPF_REG_7,
+					 ref->offset)) {
+				pr_err("prologue: failed to generate probe_read function call\n");
+				goto errout;
+			}
+
+			ref = ref->next;
+			/*
+			 * Load previous result into ARG3. Use
+			 * BPF_REG_FP instead of r7 because verifier
+			 * allows FP based addressing only.
+			 */
+			if (ref)
+				ins(BPF_LDX_MEM(BPF_DW, BPF_REG_ARG3,
+						BPF_REG_FP, stack_offset), pos);
+		}
+	}
+
+	/* Final pass: read to registers */
+	for (i = 0; i < nargs; i++)
+		ins(BPF_LDX_MEM(BPF_DW, BPF_PROLOGUE_START_ARG_REG + i,
+				BPF_REG_FP, -BPF_REG_SIZE * (i + 1)), pos);
+
+	ins(BPF_JMP_IMM(BPF_JA, BPF_REG_0, 0, JMP_TO_SUCCESS_CODE), pos);
+
+	if (check_pos(pos))
+		goto errout;
+	return 0;
+errout:
+	return -1;
+}
+
+static int
+prologue_relocate(struct bpf_insn_pos *pos, struct bpf_insn *error_code,
+	    struct bpf_insn *success_code, struct bpf_insn *user_code)
+{
+	struct bpf_insn *insn;
+
+	if (check_pos(pos))
+		return -ERANGE;
+
+	for (insn = pos->begin; insn < pos->pos; insn++) {
+		u8 class = BPF_CLASS(insn->code);
+		u8 opcode;
+
+		if (class != BPF_JMP)
+			continue;
+		opcode = BPF_OP(insn->code);
+		if (opcode == BPF_CALL)
+			continue;
+
+		switch (insn->off) {
+		case JMP_TO_ERROR_CODE:
+			insn->off = error_code - (insn + 1);
+			break;
+		case JMP_TO_SUCCESS_CODE:
+			insn->off = success_code - (insn + 1);
+			break;
+		case JMP_TO_USER_CODE:
+			insn->off = user_code - (insn + 1);
+			break;
+		default:
+			pr_err("bpf prologue: internal error: relocation failed\n");
+			return -1;
+		}
+	}
+	return 0;
+}
+
+int bpf__gen_prologue(struct probe_trace_arg *args, int nargs,
+		      struct bpf_insn *new_prog, size_t *new_cnt,
+		      size_t cnt_space)
+{
+	struct bpf_insn *success_code = NULL;
+	struct bpf_insn *error_code = NULL;
+	struct bpf_insn *user_code = NULL;
+	struct bpf_insn_pos pos;
+	bool fastpath = true;
+	int i;
+
+	if (!new_prog || !new_cnt)
+		return -EINVAL;
+
+	pos.begin = new_prog;
+	pos.end = new_prog + cnt_space;
+	pos.pos = new_prog;
+
+	if (!nargs) {
+		ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 0),
+		    &pos);
+
+		if (check_pos(&pos))
+			goto errout;
+
+		*new_cnt = pos_get_cnt(&pos);
+		return 0;
+	}
+
+	if (nargs > BPF_PROLOGUE_MAX_ARGS)
+		nargs = BPF_PROLOGUE_MAX_ARGS;
+	if (cnt_space > BPF_MAXINSNS)
+		cnt_space = BPF_MAXINSNS;
+
+	/* First pass: validation */
+	for (i = 0; i < nargs; i++) {
+		struct probe_trace_arg_ref *ref = args[i].ref;
+
+		if (args[i].value[0] == '@') {
+			/* TODO: fetch global variable */
+			pr_err("bpf: prologue: global %s%+ld not support\n",
+				args[i].value, ref ? ref->offset : 0);
+			return -ENOTSUP;
+		}
+
+		while (ref) {
+			/* fastpath is true if all args has ref == NULL */
+			fastpath = false;
+
+			/*
+			 * Instruction encodes immediate value using
+			 * s32, ref->offset is long. On systems which
+			 * can't fill long in s32, refuse to process if
+			 * ref->offset too large (or small).
+			 */
+#ifdef __LP64__
+#define OFFSET_MAX	((1LL << 31) - 1)
+#define OFFSET_MIN	((1LL << 31) * -1)
+			if (ref->offset > OFFSET_MAX ||
+					ref->offset < OFFSET_MIN) {
+				pr_err("bpf: prologue: offset out of bound: %ld\n",
+				       ref->offset);
+				return -E2BIG;
+			}
+#endif
+			ref = ref->next;
+		}
+	}
+	pr_debug("prologue: pass validation\n");
+
+	if (fastpath) {
+		/* If all variables are registers... */
+		pr_debug("prologue: fast path\n");
+		if (gen_prologue_fastpath(&pos, args, nargs))
+			goto errout;
+	} else {
+		pr_debug("prologue: slow path\n");
+
+		/* Initialization: move ctx to a callee saved register. */
+		ins(BPF_MOV64_REG(BPF_REG_CTX, BPF_REG_ARG1), &pos);
+
+		if (gen_prologue_slowpath(&pos, args, nargs))
+			goto errout;
+		/*
+		 * start of ERROR_CODE (only slow pass needs error code)
+		 *   mov r2 <- 1
+		 *   goto usercode
+		 */
+		error_code = pos.pos;
+		ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 1),
+		    &pos);
+
+		for (i = 0; i < nargs; i++)
+			ins(BPF_ALU64_IMM(BPF_MOV,
+					  BPF_PROLOGUE_START_ARG_REG + i,
+					  0),
+			    &pos);
+		ins(BPF_JMP_IMM(BPF_JA, BPF_REG_0, 0, JMP_TO_USER_CODE),
+				&pos);
+	}
+
+	/*
+	 * start of SUCCESS_CODE:
+	 *   mov r2 <- 0
+	 *   goto usercode  // skip
+	 */
+	success_code = pos.pos;
+	ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 0), &pos);
+
+	/*
+	 * start of USER_CODE:
+	 *   Restore ctx to r1
+	 */
+	user_code = pos.pos;
+	if (!fastpath) {
+		/*
+		 * Only slow path needs restoring of ctx. In fast path,
+		 * register are loaded directly from r1.
+		 */
+		ins(BPF_MOV64_REG(BPF_REG_ARG1, BPF_REG_CTX), &pos);
+		if (prologue_relocate(&pos, error_code, success_code,
+				      user_code))
+			goto errout;
+	}
+
+	if (check_pos(&pos))
+		goto errout;
+
+	*new_cnt = pos_get_cnt(&pos);
+	return 0;
+errout:
+	return -ERANGE;
+}
diff --git a/tools/perf/util/bpf-prologue.h b/tools/perf/util/bpf-prologue.h
new file mode 100644
index 0000000..f1e4c5d
--- /dev/null
+++ b/tools/perf/util/bpf-prologue.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2015, He Kuang <hekuang@huawei.com>
+ * Copyright (C) 2015, Huawei Inc.
+ */
+#ifndef __BPF_PROLOGUE_H
+#define __BPF_PROLOGUE_H
+
+#include <linux/compiler.h>
+#include <linux/filter.h>
+#include "probe-event.h"
+
+#define BPF_PROLOGUE_MAX_ARGS 3
+#define BPF_PROLOGUE_START_ARG_REG BPF_REG_3
+#define BPF_PROLOGUE_FETCH_RESULT_REG BPF_REG_2
+
+#ifdef HAVE_BPF_PROLOGUE
+int bpf__gen_prologue(struct probe_trace_arg *args, int nargs,
+		      struct bpf_insn *new_prog, size_t *new_cnt,
+		      size_t cnt_space);
+#else
+static inline int
+bpf__gen_prologue(struct probe_trace_arg *args __maybe_unused,
+		  int nargs __maybe_unused,
+		  struct bpf_insn *new_prog __maybe_unused,
+		  size_t *new_cnt,
+		  size_t cnt_space __maybe_unused)
+{
+	if (!new_cnt)
+		return -EINVAL;
+	*new_cnt = 0;
+	return 0;
+}
+#endif
+#endif /* __BPF_PROLOGUE_H */
-- 
1.8.3.4

[PATCH 17/31] perf tools: Generate prologue for BPF programs

[Wang Nan]

This patch generates prologue for each 'struct probe_trace_event' for
fetching arguments for BPF programs.

After bpf__probe(), iterate over each programs to check whether
prologue is required. If none of 'struct perf_probe_event' a program
will attach to has at least one argument, simply skip preprocessor
hooking. For those who prologue is required, calls bpf__gen_prologue()
and paste original instruction after prologue.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/bpf-loader.c | 120 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 119 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index f4c690f..c363907 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -5,11 +5,14 @@
  * Copyright (C) 2015 Huawei Inc.
  */
 
+#include <linux/bpf.h>
 #include <bpf/libbpf.h>
 #include <linux/err.h>
 #include "perf.h"
 #include "debug.h"
 #include "bpf-loader.h"
+#include "bpf-prologue.h"
+#include "llvm-utils.h"
 #include "probe-event.h"
 #include "probe-finder.h" // for MAX_PROBES
 #include "llvm-utils.h"
@@ -32,6 +35,8 @@ DEFINE_PRINT_FN(debug, 1)
 
 struct bpf_prog_priv {
 	struct perf_probe_event pev;
+	bool need_prologue;
+	struct bpf_insn *insns_buf;
 };
 
 struct bpf_object *
@@ -98,6 +103,7 @@ bpf_prog_priv__clear(struct bpf_program *prog __maybe_unused,
 	struct bpf_prog_priv *priv = _priv;
 
 	cleanup_perf_probe_events(&priv->pev, 1);
+	zfree(&priv->insns_buf);
 	free(priv);
 }
 
@@ -189,6 +195,102 @@ static int bpf__prepare_probe(void)
 	return err;
 }
 
+static int
+preproc_gen_prologue(struct bpf_program *prog, int n,
+		     struct bpf_insn *orig_insns, int orig_insns_cnt,
+		     struct bpf_prog_prep_result *res)
+{
+	struct probe_trace_event *tev;
+	struct perf_probe_event *pev;
+	struct bpf_prog_priv *priv;
+	struct bpf_insn *buf;
+	size_t prologue_cnt = 0;
+	int err;
+
+	err = bpf_program__get_private(prog, (void **)&priv);
+	if (err || !priv)
+		goto errout;
+
+	pev = &priv->pev;
+
+	if (n < 0 || n >= pev->ntevs)
+		goto errout;
+
+	tev = &pev->tevs[n];
+
+	buf = priv->insns_buf;
+	err = bpf__gen_prologue(tev->args, tev->nargs,
+				buf, &prologue_cnt,
+				BPF_MAXINSNS - orig_insns_cnt);
+	if (err) {
+		const char *title;
+
+		title = bpf_program__title(prog, false);
+		if (!title)
+			title = "[unknown]";
+
+		pr_debug("Failed to generate prologue for program %s\n",
+			 title);
+		return err;
+	}
+
+	memcpy(&buf[prologue_cnt], orig_insns,
+	       sizeof(struct bpf_insn) * orig_insns_cnt);
+
+	res->new_insn_ptr = buf;
+	res->new_insn_cnt = prologue_cnt + orig_insns_cnt;
+	res->pfd = NULL;
+	return 0;
+
+errout:
+	pr_debug("Internal error in preproc_gen_prologue\n");
+	return -EINVAL;
+}
+
+static int hook_load_preprocessor(struct bpf_program *prog)
+{
+	struct perf_probe_event *pev;
+	struct bpf_prog_priv *priv;
+	bool need_prologue = false;
+	int err, i;
+
+	err = bpf_program__get_private(prog, (void **)&priv);
+	if (err || !priv) {
+		pr_debug("Internal error when hook preprocessor\n");
+		return -EINVAL;
+	}
+
+	pev = &priv->pev;
+	for (i = 0; i < pev->ntevs; i++) {
+		struct probe_trace_event *tev = &pev->tevs[i];
+
+		if (tev->nargs > 0) {
+			need_prologue = true;
+			break;
+		}
+	}
+
+	/*
+	 * Since all tev doesn't have argument, we don't need generate
+	 * prologue.
+	 */
+	if (!need_prologue) {
+		priv->need_prologue = false;
+		return 0;
+	}
+
+	priv->need_prologue = true;
+	priv->insns_buf = malloc(sizeof(struct bpf_insn) * BPF_MAXINSNS);
+	if (!priv->insns_buf) {
+		pr_debug("No enough memory: alloc insns_buf failed\n");
+		return -ENOMEM;
+	}
+
+	err = bpf_program__set_prep(prog, pev->ntevs,
+				    preproc_gen_prologue);
+	return err;
+}
+
 int bpf__probe(struct bpf_object *obj)
 {
 	int err = 0;
@@ -223,6 +325,18 @@ int bpf__probe(struct bpf_object *obj)
 			pr_debug("bpf_probe: failed to apply perf probe events");
 			goto out;
 		}
+
+		/*
+		 * After probing, let's consider prologue, which
+		 * adds program fetcher to BPF programs.
+		 *
+		 * hook_load_preprocessorr() hooks pre-processor
+		 * to bpf_program, let it generate prologue
+		 * dynamically during loading.
+		 */
+		err = hook_load_preprocessor(prog);
+		if (err)
+			goto out;
 	}
 out:
 	return err < 0 ? err : 0;
@@ -306,7 +420,11 @@ int bpf__foreach_tev(struct bpf_object *obj,
 		for (i = 0; i < pev->ntevs; i++) {
 			tev = &pev->tevs[i];
 
-			fd = bpf_program__fd(prog);
+			if (priv->need_prologue)
+				fd = bpf_program__nth_fd(prog, i);
+			else
+				fd = bpf_program__fd(prog);
+
 			if (fd < 0) {
 				pr_debug("bpf: failed to get file descriptor\n");
 				return fd;
-- 
1.8.3.4

[PATCH 18/31] perf tools: Use same BPF program if arguments are identical

[Wang Nan]

This patch allows creating only one BPF program for different
'probe_trace_event'(tev) generated by one 'perf_probe_event'(pev), if
their prologues are identical.

This is done by comparing argument list of different tev, and maps type
of prologue and tev using a mapping array. This patch utilizes qsort to
sort tevs. After sorting, tevs with identical argument list will be
grouped together.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/bpf-loader.c | 133 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 126 insertions(+), 7 deletions(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index c363907..af549ea 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -37,6 +37,8 @@ struct bpf_prog_priv {
 	struct perf_probe_event pev;
 	bool need_prologue;
 	struct bpf_insn *insns_buf;
+	int nr_types;
+	int *type_mapping;
 };
 
 struct bpf_object *
@@ -104,6 +106,7 @@ bpf_prog_priv__clear(struct bpf_program *prog __maybe_unused,
 
 	cleanup_perf_probe_events(&priv->pev, 1);
 	zfree(&priv->insns_buf);
+	zfree(&priv->type_mapping);
 	free(priv);
 }
 
@@ -205,7 +208,7 @@ preproc_gen_prologue(struct bpf_program *prog, int n,
 	struct bpf_prog_priv *priv;
 	struct bpf_insn *buf;
 	size_t prologue_cnt = 0;
-	int err;
+	int i, err;
 
 	err = bpf_program__get_private(prog, (void **)&priv);
 	if (err || !priv)
@@ -213,10 +216,20 @@ preproc_gen_prologue(struct bpf_program *prog, int n,
 
 	pev = &priv->pev;
 
-	if (n < 0 || n >= pev->ntevs)
+	if (n < 0 || n >= priv->nr_types)
 		goto errout;
 
-	tev = &pev->tevs[n];
+	/* Find a tev belongs to that type */
+	for (i = 0; i < pev->ntevs; i++)
+		if (priv->type_mapping[i] == n)
+			break;
+
+	if (i >= pev->ntevs) {
+		pr_debug("Internal error: prologue type %d not found\n", n);
+		return -ENOENT;
+	}
+
+	tev = &pev->tevs[i];
 
 	buf = priv->insns_buf;
 	err = bpf__gen_prologue(tev->args, tev->nargs,
@@ -247,6 +260,98 @@ errout:
 	return -EINVAL;
 }
 
+/*
+ * compare_tev_args is reflexive, transitive and antisymmetric.
+ * I can show that but this margin is too narrow to contain.
+ */
+static int compare_tev_args(const void *ptev1, const void *ptev2)
+{
+	int i, ret;
+	const struct probe_trace_event *tev1 =
+		*(const struct probe_trace_event **)ptev1;
+	const struct probe_trace_event *tev2 =
+		*(const struct probe_trace_event **)ptev2;
+
+	ret = tev2->nargs - tev1->nargs;
+	if (ret)
+		return ret;
+
+	for (i = 0; i < tev1->nargs; i++) {
+		struct probe_trace_arg *arg1, *arg2;
+		struct probe_trace_arg_ref *ref1, *ref2;
+
+		arg1 = &tev1->args[i];
+		arg2 = &tev2->args[i];
+
+		ret = strcmp(arg1->value, arg2->value);
+		if (ret)
+			return ret;
+
+		ref1 = arg1->ref;
+		ref2 = arg2->ref;
+
+		while (ref1 && ref2) {
+			ret = ref2->offset - ref1->offset;
+			if (ret)
+				return ret;
+
+			ref1 = ref1->next;
+			ref2 = ref2->next;
+		}
+
+		if (ref1 || ref2)
+			return ref2 ? 1 : -1;
+	}
+
+	return 0;
+}
+
+static int map_prologue(struct perf_probe_event *pev, int *mapping,
+			int *nr_types)
+{
+	int i, type = 0;
+	struct {
+		struct probe_trace_event *tev;
+		int idx;
+	} *stevs;
+	size_t array_sz = sizeof(*stevs) * pev->ntevs;
+
+	stevs = malloc(array_sz);
+	if (!stevs) {
+		pr_debug("No ehough memory: alloc stevs failed\n");
+		return -ENOMEM;
+	}
+
+	pr_debug("In map_prologue, ntevs=%d\n", pev->ntevs);
+	for (i = 0; i < pev->ntevs; i++) {
+		stevs[i].tev = &pev->tevs[i];
+		stevs[i].idx = i;
+	}
+	qsort(stevs, pev->ntevs, sizeof(*stevs),
+	      compare_tev_args);
+
+	for (i = 0; i < pev->ntevs; i++) {
+		if (i == 0) {
+			mapping[stevs[i].idx] = type;
+			pr_debug("mapping[%d]=%d\n", stevs[i].idx,
+				 type);
+			continue;
+		}
+
+		if (compare_tev_args(stevs + i, stevs + i - 1) == 0)
+			mapping[stevs[i].idx] = type;
+		else
+			mapping[stevs[i].idx] = ++type;
+
+		pr_debug("mapping[%d]=%d\n", stevs[i].idx,
+			 mapping[stevs[i].idx]);
+	}
+	free(stevs);
+	*nr_types = type + 1;
+
+	return 0;
+}
+
 static int hook_load_preprocessor(struct bpf_program *prog)
 {
 	struct perf_probe_event *pev;
@@ -286,7 +391,19 @@ static int hook_load_preprocessor(struct bpf_program *prog)
 		return -ENOMEM;
 	}
 
-	err = bpf_program__set_prep(prog, pev->ntevs,
+	priv->type_mapping = malloc(sizeof(int) * pev->ntevs);
+	if (!priv->type_mapping) {
+		pr_debug("No enough memory: alloc type_mapping failed\n");
+		return -ENOMEM;
+	}
+	memset(priv->type_mapping, 0xff,
+	       sizeof(int) * pev->ntevs);
+
+	err = map_prologue(pev, priv->type_mapping, &priv->nr_types);
+	if (err)
+		return err;
+
+	err = bpf_program__set_prep(prog, priv->nr_types,
 				    preproc_gen_prologue);
 	return err;
 }
@@ -420,9 +537,11 @@ int bpf__foreach_tev(struct bpf_object *obj,
 		for (i = 0; i < pev->ntevs; i++) {
 			tev = &pev->tevs[i];
 
-			if (priv->need_prologue)
-				fd = bpf_program__nth_fd(prog, i);
-			else
+			if (priv->need_prologue) {
+				int type = priv->type_mapping[i];
+
+				fd = bpf_program__nth_fd(prog, type);
+			} else
 				fd = bpf_program__fd(prog);
 
 			if (fd < 0) {
-- 
1.8.3.4

[PATCH 19/31] perf record: Support custom vmlinux path

[Wang Nan]

From: He Kuang <hekuang@huawei.com>

Make perf-record command support --vmlinux option if BPF_PROLOGUE is on.

'perf record' needs vmlinux as the source of DWARF info to generate
prologue for BPF programs, so path of vmlinux should be specified.

Short name 'k' has been taken by 'clockid'. This patch skips the short
option name and use '--vmlinux' for vmlinux path.

Signed-off-by: He Kuang <hekuang@huawei.com>
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/builtin-record.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 847cc67..200f221 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1123,6 +1123,10 @@ struct option __record_options[] = {
 		   "clang binary to use for compiling BPF scriptlets"),
 	OPT_STRING(0, "clang-opt", &llvm_param.clang_opt, "clang options",
 		   "options passed to clang when compiling BPF scriptlets"),
+#ifdef HAVE_BPF_PROLOGUE
+	OPT_STRING(0, "vmlinux", &symbol_conf.vmlinux_name,
+		   "file", "vmlinux pathname"),
+#endif
 #endif
 	OPT_END()
 };
-- 
1.8.3.4

[PATCH 20/31] perf tools: Allow BPF program attach to uprobe events

[Wang Nan]

This patch appends new syntax to BPF object section name to support
probing at uprobe event. Now we can use BPF program like this:

 SEC(
 "target=/lib64/libc.so.6\n"
 "libcwrite=__write"
 )
 int libcwrite(void *ctx)
 {
     return 1;
 }

Where, in section name of a program, before the main config string,
we can use 'key=value' style options. Now the only option key "target"
is for uprobe probing.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/util/bpf-loader.c | 86 ++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 80 insertions(+), 6 deletions(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index af549ea..73ff9a9 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -111,6 +111,84 @@ bpf_prog_priv__clear(struct bpf_program *prog __maybe_unused,
 }
 
 static int
+do_config(const char *key, const char *value,
+	  struct perf_probe_event *pev)
+{
+	pr_debug("config bpf program: %s=%s\n", key, value);
+	if (strcmp(key, "target") == 0) {
+		pev->uprobes = true;
+		pev->target = strdup(value);
+		return 0;
+	}
+
+	pr_warning("BPF: WARNING: invalid config option in object: %s=%s\n",
+		   key, value);
+	pr_warning("\tHint: Currently only valid option is 'target=<file>'\n");
+	return 0;
+}
+
+static const char *
+parse_config_kvpair(const char *config_str, struct perf_probe_event *pev)
+{
+	char *text = strdup(config_str);
+	char *sep, *line;
+	const char *main_str = NULL;
+	int err = 0;
+
+	if (!text) {
+		pr_debug("No enough memory: dup config_str failed\n");
+		return NULL;
+	}
+
+	line = text;
+	while ((sep = strchr(line, '\n'))) {
+		char *equ;
+
+		*sep = '\0';
+		equ = strchr(line, '=');
+		if (!equ) {
+			pr_warning("WARNING: invalid config in BPF object: %s\n",
+				   line);
+			pr_warning("\tShould be 'key=value'.\n");
+			goto nextline;
+		}
+		*equ = '\0';
+
+		err = do_config(line, equ + 1, pev);
+		if (err)
+			break;
+nextline:
+		line = sep + 1;
+	}
+
+	if (!err)
+		main_str = config_str + (line - text);
+	free(text);
+
+	return main_str;
+}
+
+static int
+parse_config(const char *config_str, struct perf_probe_event *pev)
+{
+	const char *main_str;
+	int err;
+
+	main_str = parse_config_kvpair(config_str, pev);
+	if (!main_str)
+		return -EINVAL;
+
+	err = parse_perf_probe_command(main_str, pev);
+	if (err < 0) {
+		pr_debug("bpf: '%s' is not a valid config string\n",
+			 config_str);
+		/* parse failed, don't need clear pev. */
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int
 config_bpf_program(struct bpf_program *prog)
 {
 	struct perf_probe_event *pev = NULL;
@@ -132,13 +210,9 @@ config_bpf_program(struct bpf_program *prog)
 	pev = &priv->pev;
 
 	pr_debug("bpf: config program '%s'\n", config_str);
-	err = parse_perf_probe_command(config_str, pev);
-	if (err < 0) {
-		pr_debug("bpf: '%s' is not a valid config string\n",
-			 config_str);
-		err = -EINVAL;
+	err = parse_config(config_str, pev);
+	if (err)
 		goto errout;
-	}
 
 	if (pev->group && strcmp(pev->group, PERF_BPF_PROBE_GROUP)) {
 		pr_debug("bpf: '%s': group for event is set and not '%s'.\n",
-- 
1.8.3.4

[PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Wang Nan]

This patch enforces existing LLVM test, makes it compile more than one
BPF source file. The compiled results are stored, can be used for other
testcases. Except the first testcase (named LLVM_TESTCASE_BASE), failures
of other test cases are not considered as failure of the whole test.

Adds a kbuild testcase to check whether kernel headers can be correctly
found.

For example:

 # perf test LLVM

   38: Test LLVM searching and compiling                        : (llvm.kbuild-dir can be fixed) Ok

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/tests/Build                    |  11 ++-
 tools/perf/tests/bpf-script-example.c     |   4 +
 tools/perf/tests/bpf-script-test-kbuild.c |  21 ++++
 tools/perf/tests/bpf.c                    |   3 +-
 tools/perf/tests/llvm.c                   | 154 ++++++++++++++++++++++--------
 tools/perf/tests/llvm.h                   |  10 +-
 6 files changed, 156 insertions(+), 47 deletions(-)
 create mode 100644 tools/perf/tests/bpf-script-test-kbuild.c

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index d0278a9..23e2201 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -31,17 +31,24 @@ perf-y += sample-parsing.o
 perf-y += parse-no-sample-id-all.o
 perf-y += kmod-path.o
 perf-y += thread-map.o
-perf-y += llvm.o llvm-src.o
+perf-y += llvm.o llvm-src-base.o llvm-src-kbuild.o
 perf-y += bpf.o
 perf-y += topology.o
 
-$(OUTPUT)tests/llvm-src.c: tests/bpf-script-example.c
+$(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c
 	$(call rule_mkdir)
 	$(Q)echo '#include <tests/llvm.h>' > $@
 	$(Q)echo 'const char test_llvm__bpf_prog[] =' >> $@
 	$(Q)sed -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@
 	$(Q)echo ';' >> $@
 
+$(OUTPUT)tests/llvm-src-kbuild.c: tests/bpf-script-test-kbuild.c
+	$(call rule_mkdir)
+	$(Q)echo '#include <tests/llvm.h>' > $@
+	$(Q)echo 'const char test_llvm__bpf_test_kbuild_prog[] =' >> $@
+	$(Q)sed -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@
+	$(Q)echo ';' >> $@
+
 ifeq ($(ARCH),$(filter $(ARCH),x86 arm arm64))
 perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
 endif
diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
index 410a70b..0ec9c2c 100644
--- a/tools/perf/tests/bpf-script-example.c
+++ b/tools/perf/tests/bpf-script-example.c
@@ -1,3 +1,7 @@
+/*
+ * bpf-script-example.c
+ * Test basic LLVM building
+ */
 #ifndef LINUX_VERSION_CODE
 # error Need LINUX_VERSION_CODE
 # error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
diff --git a/tools/perf/tests/bpf-script-test-kbuild.c b/tools/perf/tests/bpf-script-test-kbuild.c
new file mode 100644
index 0000000..a11f589
--- /dev/null
+++ b/tools/perf/tests/bpf-script-test-kbuild.c
@@ -0,0 +1,21 @@
+/*
+ * bpf-script-test-kbuild.c
+ * Test include from kernel header
+ */
+#ifndef LINUX_VERSION_CODE
+# error Need LINUX_VERSION_CODE
+# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
+#endif
+#define SEC(NAME) __attribute__((section(NAME), used))
+
+#include <uapi/linux/fs.h>
+#include <uapi/asm/ptrace.h>
+
+SEC("func=vfs_llseek")
+int bpf_func__vfs_llseek(struct pt_regs *ctx)
+{
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
+int _version SEC("version") = LINUX_VERSION_CODE;
diff --git a/tools/perf/tests/bpf.c b/tools/perf/tests/bpf.c
index d7cdc84..4dd701b 100644
--- a/tools/perf/tests/bpf.c
+++ b/tools/perf/tests/bpf.c
@@ -140,7 +140,8 @@ int test__bpf(void)
 		return TEST_SKIP;
 	}
 
-	test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz);
+	test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz, LLVM_TESTCASE_BASE);
+
 	if (!obj_buf || !obj_buf_sz) {
 		if (verbose == 0)
 			fprintf(stderr, " (fix 'perf test LLVM' first)");
diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
index fd5fdb0..75cd99f 100644
--- a/tools/perf/tests/llvm.c
+++ b/tools/perf/tests/llvm.c
@@ -9,6 +9,22 @@
 #include "debug.h"
 #include "llvm.h"
 
+#define SHARED_BUF_INIT_SIZE	(1 << 20)
+struct llvm_testcase {
+	const char *source;
+	const char *errmsg;
+	struct test_llvm__bpf_result *result;
+	bool tried;
+} llvm_testcases[NR_LLVM_TESTCASES + 1] = {
+	[LLVM_TESTCASE_BASE]	= {.source = test_llvm__bpf_prog,
+				   .errmsg = "Basic LLVM compiling failed",
+				   .tried = false},
+	[LLVM_TESTCASE_KBUILD]	= {.source = test_llvm__bpf_test_kbuild_prog,
+				   .errmsg = "llvm.kbuild-dir can be fixed",
+				   .tried = false},
+	{.source = NULL}
+};
+
 static int perf_config_cb(const char *var, const char *val,
 			  void *arg __maybe_unused)
 {
@@ -36,7 +52,7 @@ static int test__bpf_parsing(void *obj_buf __maybe_unused,
 #endif
 
 static char *
-compose_source(void)
+compose_source(const char *raw_source)
 {
 	struct utsname utsname;
 	int version, patchlevel, sublevel, err;
@@ -56,25 +72,27 @@ compose_source(void)
 
 	version_code = (version << 16) + (patchlevel << 8) + sublevel;
 	err = asprintf(&code, "#define LINUX_VERSION_CODE 0x%08lx;\n%s",
-		       version_code, test_llvm__bpf_prog);
+		       version_code, raw_source);
 	if (err < 0)
 		return NULL;
 
 	return code;
 }
 
-#define SHARED_BUF_INIT_SIZE	(1 << 20)
-struct test_llvm__bpf_result *p_test_llvm__bpf_result;
 
-int test__llvm(void)
+static int __test__llvm(int i)
 {
-	char *tmpl_new, *clang_opt_new;
 	void *obj_buf;
 	size_t obj_buf_sz;
 	int err, old_verbose;
-	char *source;
+	const char *tmpl_old, *clang_opt_old;
+	char *tmpl_new, *clang_opt_new, *source;
+	const char *raw_source = llvm_testcases[i].source;
+	struct test_llvm__bpf_result *result = llvm_testcases[i].result;
 
 	perf_config(perf_config_cb, NULL);
+	clang_opt_old = llvm_param.clang_opt;
+	tmpl_old = llvm_param.clang_bpf_cmd_template;
 
 	/*
 	 * Skip this test if user's .perfconfig doesn't set [llvm] section
@@ -99,15 +117,17 @@ int test__llvm(void)
 	if (!llvm_param.clang_opt)
 		llvm_param.clang_opt = strdup("");
 
-	source = compose_source();
+	source = compose_source(raw_source);
 	if (!source) {
 		pr_err("Failed to compose source code\n");
 		return -1;
 	}
 
 	/* Quote __EOF__ so strings in source won't be expanded by shell */
-	err = asprintf(&tmpl_new, "cat << '__EOF__' | %s\n%s\n__EOF__\n",
-		       llvm_param.clang_bpf_cmd_template, source);
+	err = asprintf(&tmpl_new, "cat << '__EOF__' | %s %s \n%s\n__EOF__\n",
+		       llvm_param.clang_bpf_cmd_template,
+		       !old_verbose ? "2>/dev/null" : "",
+		       source);
 	free(source);
 	source = NULL;
 	if (err < 0) {
@@ -123,73 +143,123 @@ int test__llvm(void)
 	llvm_param.clang_opt = clang_opt_new;
 	err = llvm__compile_bpf("-", &obj_buf, &obj_buf_sz);
 
+	free((void *)llvm_param.clang_bpf_cmd_template);
+	free((void *)llvm_param.clang_opt);
+	llvm_param.clang_bpf_cmd_template = tmpl_old;
+	llvm_param.clang_opt = clang_opt_old;
+
 	verbose = old_verbose;
-	if (err) {
-		if (!verbose)
-			fprintf(stderr, " (use -v to see error message)");
+	if (err)
 		return -1;
-	}
 
 	err = test__bpf_parsing(obj_buf, obj_buf_sz);
-	if (!err && p_test_llvm__bpf_result) {
+	if (!err && result) {
 		if (obj_buf_sz > SHARED_BUF_INIT_SIZE) {
 			pr_err("Resulting object too large\n");
 		} else {
-			p_test_llvm__bpf_result->size = obj_buf_sz;
-			memcpy(p_test_llvm__bpf_result->object,
-			       obj_buf, obj_buf_sz);
+			result->size = obj_buf_sz;
+			memcpy(result->object, obj_buf, obj_buf_sz);
 		}
 	}
 	free(obj_buf);
 	return err;
 }
 
+int test__llvm(void)
+{
+	int i, ret;
+
+	for (i = 0; llvm_testcases[i].source; i++) {
+		ret = __test__llvm(i);
+		if (i == 0 && ret) {
+			/*
+			 * First testcase tests basic LLVM compiling. If it
+			 * fails, no need to check others.
+			 */
+			if (!verbose)
+				fprintf(stderr, " (use -v to see error message)");
+			return ret;
+		} else if (ret) {
+			if (!verbose && llvm_testcases[i].errmsg)
+				fprintf(stderr, " (%s)", llvm_testcases[i].errmsg);
+			return 0;
+		}
+	}
+	return 0;
+}
+
 void test__llvm_prepare(void)
 {
-	p_test_llvm__bpf_result = mmap(NULL, SHARED_BUF_INIT_SIZE,
-				       PROT_READ | PROT_WRITE,
-				       MAP_SHARED | MAP_ANONYMOUS, -1, 0);
-	if (!p_test_llvm__bpf_result)
-		return;
-	memset((void *)p_test_llvm__bpf_result, '\0', SHARED_BUF_INIT_SIZE);
+	int i;
+
+	for (i = 0; llvm_testcases[i].source; i++) {
+		struct test_llvm__bpf_result *result;
+
+		result = mmap(NULL, SHARED_BUF_INIT_SIZE,
+			      PROT_READ | PROT_WRITE,
+			      MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+		if (!result)
+			return;
+		memset((void *)result, '\0', SHARED_BUF_INIT_SIZE);
+
+		llvm_testcases[i].result = result;
+	}
 }
 
 void test__llvm_cleanup(void)
 {
-	unsigned long boundary, buf_end;
+	int i;
 
-	if (!p_test_llvm__bpf_result)
-		return;
-	if (p_test_llvm__bpf_result->size == 0) {
-		munmap((void *)p_test_llvm__bpf_result, SHARED_BUF_INIT_SIZE);
-		p_test_llvm__bpf_result = NULL;
-		return;
-	}
+	for (i = 0; llvm_testcases[i].source; i++) {
+		struct test_llvm__bpf_result *result;
+		unsigned long boundary, buf_end;
 
-	buf_end = (unsigned long)p_test_llvm__bpf_result + SHARED_BUF_INIT_SIZE;
+		result = llvm_testcases[i].result;
+		llvm_testcases[i].tried = true;
 
-	boundary = (unsigned long)(p_test_llvm__bpf_result);
-	boundary += p_test_llvm__bpf_result->size;
-	boundary = (boundary + (page_size - 1)) &
+		if (!result)
+			continue;
+
+		if (result->size == 0) {
+			munmap((void *)result, SHARED_BUF_INIT_SIZE);
+			result = NULL;
+			llvm_testcases[i].result = NULL;
+			continue;
+		}
+
+		buf_end = (unsigned long)result + SHARED_BUF_INIT_SIZE;
+
+		boundary = (unsigned long)(result);
+		boundary += result->size;
+		boundary = (boundary + (page_size - 1)) &
 			(~((unsigned long)page_size - 1));
-	munmap((void *)boundary, buf_end - boundary);
+		munmap((void *)boundary, buf_end - boundary);
+	}
 }
 
 void
-test_llvm__fetch_bpf_obj(void **p_obj_buf, size_t *p_obj_buf_sz)
+test_llvm__fetch_bpf_obj(void **p_obj_buf, size_t *p_obj_buf_sz, int index)
 {
+	struct test_llvm__bpf_result *result;
+
 	*p_obj_buf = NULL;
 	*p_obj_buf_sz = 0;
 
-	if (!p_test_llvm__bpf_result) {
+	if (index > NR_LLVM_TESTCASES)
+		return;
+
+	result = llvm_testcases[index].result;
+
+	if (!result && !llvm_testcases[index].tried) {
 		test__llvm_prepare();
 		test__llvm();
 		test__llvm_cleanup();
 	}
 
-	if (!p_test_llvm__bpf_result)
+	result = llvm_testcases[index].result;
+	if (!result)
 		return;
 
-	*p_obj_buf = p_test_llvm__bpf_result->object;
-	*p_obj_buf_sz = p_test_llvm__bpf_result->size;
+	*p_obj_buf = result->object;
+	*p_obj_buf_sz = result->size;
 }
diff --git a/tools/perf/tests/llvm.h b/tools/perf/tests/llvm.h
index 2fd7ed6..78ec01d 100644
--- a/tools/perf/tests/llvm.h
+++ b/tools/perf/tests/llvm.h
@@ -8,8 +8,14 @@ struct test_llvm__bpf_result {
 	char object[];
 };
 
-extern struct test_llvm__bpf_result *p_test_llvm__bpf_result;
 extern const char test_llvm__bpf_prog[];
-void test_llvm__fetch_bpf_obj(void **p_obj_buf, size_t *p_obj_buf_sz);
+extern const char test_llvm__bpf_test_kbuild_prog[];
+
+enum test_llvm__testcase {
+	LLVM_TESTCASE_BASE,
+	LLVM_TESTCASE_KBUILD,
+	NR_LLVM_TESTCASES,
+};
+void test_llvm__fetch_bpf_obj(void **p_obj_buf, size_t *p_obj_buf_sz, int index);
 
 #endif
-- 
1.8.3.4

[PATCH 22/31] perf test: Test BPF prologue

[Wang Nan]

This patch introduces a new BPF script to test BPF prologue. The new
script probes at null_lseek, which is the function pointer when we try
to lseek on '/dev/null'.

null_lseek is chosen because it is a function pointer, so we don't need
to consider inlining and LTP.

By extracting file->f_mode, bpf-script-test-prologue.c should know whether
the file is writable or readonly. According to llseek_loop() and
bpf-script-test-prologue.c, one forth of total lseeks should be collected.

This patch improve test__bpf so it can run multiple BPF programs on
different test functions.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
---
 tools/perf/tests/Build                      |  9 ++-
 tools/perf/tests/bpf-script-test-prologue.c | 35 ++++++++++++
 tools/perf/tests/bpf.c                      | 89 +++++++++++++++++++++++------
 tools/perf/tests/llvm.c                     |  5 ++
 tools/perf/tests/llvm.h                     |  8 +++
 5 files changed, 128 insertions(+), 18 deletions(-)
 create mode 100644 tools/perf/tests/bpf-script-test-prologue.c

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 23e2201..1d69493 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -31,7 +31,7 @@ perf-y += sample-parsing.o
 perf-y += parse-no-sample-id-all.o
 perf-y += kmod-path.o
 perf-y += thread-map.o
-perf-y += llvm.o llvm-src-base.o llvm-src-kbuild.o
+perf-y += llvm.o llvm-src-base.o llvm-src-kbuild.o llvm-src-prologue.o
 perf-y += bpf.o
 perf-y += topology.o
 
@@ -49,6 +49,13 @@ $(OUTPUT)tests/llvm-src-kbuild.c: tests/bpf-script-test-kbuild.c
 	$(Q)sed -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@
 	$(Q)echo ';' >> $@
 
+$(OUTPUT)tests/llvm-src-prologue.c: tests/bpf-script-test-prologue.c
+	$(call rule_mkdir)
+	$(Q)echo '#include <tests/llvm.h>' > $@
+	$(Q)echo 'const char test_llvm__bpf_test_prologue_prog[] =' >> $@
+	$(Q)sed -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@
+	$(Q)echo ';' >> $@
+
 ifeq ($(ARCH),$(filter $(ARCH),x86 arm arm64))
 perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
 endif
diff --git a/tools/perf/tests/bpf-script-test-prologue.c b/tools/perf/tests/bpf-script-test-prologue.c
new file mode 100644
index 0000000..7230e62
--- /dev/null
+++ b/tools/perf/tests/bpf-script-test-prologue.c
@@ -0,0 +1,35 @@
+/*
+ * bpf-script-test-prologue.c
+ * Test BPF prologue
+ */
+#ifndef LINUX_VERSION_CODE
+# error Need LINUX_VERSION_CODE
+# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
+#endif
+#define SEC(NAME) __attribute__((section(NAME), used))
+
+#include <uapi/linux/fs.h>
+
+#define FMODE_READ		0x1
+#define FMODE_WRITE		0x2
+
+static void (*bpf_trace_printk)(const char *fmt, int fmt_size, ...) =
+	(void *) 6;
+
+SEC("func=null_lseek file->f_mode offset orig")
+int bpf_func__null_lseek(void *ctx, int err, unsigned long f_mode,
+			 unsigned long offset, unsigned long orig)
+{
+	if (err)
+		return 0;
+	if (f_mode & FMODE_WRITE)
+		return 0;
+	if (offset & 1)
+		return 0;
+	if (orig == SEEK_CUR)
+		return 0;
+	return 1;
+}
+
+char _license[] SEC("license") = "GPL";
+int _version SEC("version") = LINUX_VERSION_CODE;
diff --git a/tools/perf/tests/bpf.c b/tools/perf/tests/bpf.c
index 4dd701b..5a6290a 100644
--- a/tools/perf/tests/bpf.c
+++ b/tools/perf/tests/bpf.c
@@ -19,11 +19,35 @@ static int epoll_pwait_loop(void)
 	return 0;
 }
 
-static struct bpf_object *prepare_bpf(void *obj_buf, size_t obj_buf_sz)
+#ifdef HAVE_BPF_PROLOGUE
+
+static int llseek_loop(void)
+{
+	int fds[2], i;
+
+	fds[0] = open("/dev/null", O_RDONLY);
+	fds[1] = open("/dev/null", O_RDWR);
+
+	if (fds[0] < 0 || fds[1] < 0)
+		return -1;
+
+	for (i = 0; i < NR_ITERS; i++) {
+		lseek(fds[i % 2], i, (i / 2) % 2 ? SEEK_CUR : SEEK_SET);
+		lseek(fds[(i + 1) % 2], i, (i / 2) % 2 ? SEEK_CUR : SEEK_SET);
+	}
+	close(fds[0]);
+	close(fds[1]);
+	return 0;
+}
+
+#endif
+
+static struct bpf_object *prepare_bpf(const char *name, void *obj_buf,
+				      size_t obj_buf_sz)
 {
 	struct bpf_object *obj;
 
-	obj = bpf__prepare_load_buffer(obj_buf, obj_buf_sz, "[buffer]");
+	obj = bpf__prepare_load_buffer(obj_buf, obj_buf_sz, name);
 	if (IS_ERR(obj)) {
 		fprintf(stderr, " (compile failed)");
 		return NULL;
@@ -31,7 +55,7 @@ static struct bpf_object *prepare_bpf(void *obj_buf, size_t obj_buf_sz)
 	return obj;
 }
 
-static int do_test(struct bpf_object *obj)
+static int do_test(struct bpf_object *obj, int (*func)(void), int expect)
 {
 	struct record_opts opts = {
 		.target = {
@@ -101,7 +125,7 @@ static int do_test(struct bpf_object *obj)
 	}
 
 	perf_evlist__enable(evlist);
-	epoll_pwait_loop();
+	(*func)();
 	perf_evlist__disable(evlist);
 
 	for (i = 0; i < evlist->nr_mmaps; i++) {
@@ -115,8 +139,8 @@ static int do_test(struct bpf_object *obj)
 		}
 	}
 
-	if (count != (NR_ITERS + 1) / 2) {
-		fprintf(stderr, " (filter result incorrect)");
+	if (count != expect) {
+		fprintf(stderr, " (filter result incorrect: %d != %d)", count, expect);
 		err = -EBADF;
 	}
 
@@ -128,33 +152,32 @@ out:
 	return 0;
 }
 
-int test__bpf(void)
+static int __test__bpf(int index, const char *name,
+		       const char *message_compile,
+		       const char *message_load,
+		       int (*func)(void), int expect)
 {
 	int err;
 	void *obj_buf;
 	size_t obj_buf_sz;
 	struct bpf_object *obj;
 
-	if (geteuid() != 0) {
-		fprintf(stderr, " (try run as root)");
-		return TEST_SKIP;
-	}
-
-	test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz, LLVM_TESTCASE_BASE);
-
+	test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz, index);
 	if (!obj_buf || !obj_buf_sz) {
 		if (verbose == 0)
-			fprintf(stderr, " (fix 'perf test LLVM' first)");
+			fprintf(stderr, " (%s)", message_compile);
 		return TEST_SKIP;
 	}
 
-	obj = prepare_bpf(obj_buf, obj_buf_sz);
+	obj = prepare_bpf(name, obj_buf, obj_buf_sz);
 	if (!obj) {
 		err = -EINVAL;
+		if ((verbose == 0) && (message_load[0] != '\0'))
+			fprintf(stderr, " (%s)", message_load);
 		goto out;
 	}
 
-	err = do_test(obj);
+	err = do_test(obj, func, expect);
 	if (err)
 		goto out;
 out:
@@ -164,6 +187,38 @@ out:
 	return 0;
 }
 
+int test__bpf(void)
+{
+	int err;
+
+	if (geteuid() != 0) {
+		fprintf(stderr, " (try run as root)");
+		return TEST_SKIP;
+	}
+
+	err = __test__bpf(LLVM_TESTCASE_BASE,
+			  "[basic_bpf_test]",
+			  "fix 'perf test LLVM' first",
+			  "load bpf object failed",
+			  &epoll_pwait_loop,
+			  (NR_ITERS + 1) / 2);
+	if (err)
+		return err;
+
+#ifdef HAVE_BPF_PROLOGUE
+	err = __test__bpf(LLVM_TESTCASE_BPF_PROLOGUE,
+			  "[bpf_prologue_test]",
+			  "fix kbuild first",
+			  "check your vmlinux setting?",
+			  &llseek_loop,
+			  (NR_ITERS + 1) / 4);
+	return err;
+#else
+	fprintf(stderr, " (skip BPF prologue test)");
+	return TEST_OK;
+#endif
+}
+
 #else
 int test__bpf(void)
 {
diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
index 75cd99f..e722e8a 100644
--- a/tools/perf/tests/llvm.c
+++ b/tools/perf/tests/llvm.c
@@ -22,6 +22,11 @@ struct llvm_testcase {
 	[LLVM_TESTCASE_KBUILD]	= {.source = test_llvm__bpf_test_kbuild_prog,
 				   .errmsg = "llvm.kbuild-dir can be fixed",
 				   .tried = false},
+	/* Don't output if this one fail. */
+	[LLVM_TESTCASE_BPF_PROLOGUE]	= {
+				   .source = test_llvm__bpf_test_prologue_prog,
+				   .errmsg = "failed for unknown reason",
+				   .tried = false},
 	{.source = NULL}
 };
 
diff --git a/tools/perf/tests/llvm.h b/tools/perf/tests/llvm.h
index 78ec01d..c00c1be 100644
--- a/tools/perf/tests/llvm.h
+++ b/tools/perf/tests/llvm.h
@@ -10,10 +10,18 @@ struct test_llvm__bpf_result {
 
 extern const char test_llvm__bpf_prog[];
 extern const char test_llvm__bpf_test_kbuild_prog[];
+extern const char test_llvm__bpf_test_prologue_prog[];
 
 enum test_llvm__testcase {
 	LLVM_TESTCASE_BASE,
 	LLVM_TESTCASE_KBUILD,
+	/*
+	 * We must put LLVM_TESTCASE_BPF_PROLOGUE after
+	 * LLVM_TESTCASE_KBUILD, so if kbuild test failed,
+	 * don't need to try this one, because it depend on
+	 * kernel header.
+	 */
+	LLVM_TESTCASE_BPF_PROLOGUE,
 	NR_LLVM_TESTCASES,
 };
 void test_llvm__fetch_bpf_obj(void **p_obj_buf, size_t *p_obj_buf_sz, int index);
-- 
1.8.3.4

[PATCH 23/31] bpf tools: Add helper function for updating bpf maps elements

[Wang Nan]

From: He Kuang <hekuang@huawei.com>

Add helper function bpf_map_update_elem() which calls sys_bpf syscall
to update elements in bpf maps.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-yflv9p2x75ht7okavpk97m4g@git.kernel.org
---
 tools/lib/bpf/bpf.c | 14 ++++++++++++++
 tools/lib/bpf/bpf.h |  2 ++
 2 files changed, 16 insertions(+)

diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index a633105..5bdc6ea 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -83,3 +83,17 @@ int bpf_load_program(enum bpf_prog_type type, struct bpf_insn *insns,
 	log_buf[0] = 0;
 	return sys_bpf(BPF_PROG_LOAD, &attr, sizeof(attr));
 }
+
+int bpf_map_update_elem(int fd, void *key, void *value,
+			u64 flags)
+{
+	union bpf_attr attr;
+
+	bzero(&attr, sizeof(attr));
+	attr.map_fd = fd;
+	attr.key = ptr_to_u64(key);
+	attr.value = ptr_to_u64(value);
+	attr.flags = flags;
+
+	return sys_bpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr));
+}
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 854b736..a764655 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -20,4 +20,6 @@ int bpf_load_program(enum bpf_prog_type type, struct bpf_insn *insns,
 		     u32 kern_version, char *log_buf,
 		     size_t log_buf_sz);
 
+int bpf_map_update_elem(int fd, void *key, void *value,
+			u64 flags);
 #endif
-- 
1.8.3.4

[PATCH 24/31] bpf tools: Collect map definition in bpf_object

[Wang Nan]

This patch collects more information from maps sections in BPF object
files into 'struct bpf_object', enables later patches access those
information (such as the type and size of the map).

In this patch, a new handler 'struct bpf_map' is extracted in parallel
with bpf_object and bpf_program. Its iterator and accessor is also
created.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-oewgxlae7jpwqou9hba4eskg@git.kernel.org
---
 tools/lib/bpf/libbpf.c | 186 +++++++++++++++++++++++++++++++++----------------
 tools/lib/bpf/libbpf.h |  21 ++++++
 2 files changed, 147 insertions(+), 60 deletions(-)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 6a07b26..8ae501b 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -109,22 +109,24 @@ struct bpf_program {
 	bpf_program_clear_priv_t clear_priv;
 };
 
+struct bpf_map {
+	int fd;
+	struct bpf_map_def def;
+	void *priv;
+	bpf_map_clear_priv_t clear_priv;
+};
+
 static LIST_HEAD(bpf_objects_list);
 
 struct bpf_object {
 	char license[64];
 	u32 kern_version;
-	void *maps_buf;
-	size_t maps_buf_sz;
 
 	struct bpf_program *programs;
 	size_t nr_programs;
-	int *map_fds;
-	/*
-	 * This field is required because maps_buf will be freed and
-	 * maps_buf_sz will be set to 0 after loaded.
-	 */
-	size_t nr_map_fds;
+	struct bpf_map *maps;
+	size_t nr_maps;
+
 	bool loaded;
 
 	/*
@@ -434,21 +436,38 @@ static int
 bpf_object__init_maps(struct bpf_object *obj, void *data,
 		      size_t size)
 {
-	if (size == 0) {
+	size_t nr_maps;
+	int i;
+
+	nr_maps = size / sizeof(struct bpf_map_def);
+	if (!data || !nr_maps) {
 		pr_debug("%s doesn't need map definition\n",
 			 obj->path);
 		return 0;
 	}
 
-	obj->maps_buf = malloc(size);
-	if (!obj->maps_buf) {
-		pr_warning("malloc maps failed: %s\n", obj->path);
+	pr_debug("maps in %s: %ld bytes\n", obj->path, (long)size);
+
+	obj->maps = calloc(1, sizeof(obj->maps[0]) * nr_maps);
+	if (!obj->maps) {
+		pr_warning("alloc maps for object failed\n");
 		return -ENOMEM;
 	}
+	obj->nr_maps = nr_maps;
 
-	obj->maps_buf_sz = size;
-	memcpy(obj->maps_buf, data, size);
-	pr_debug("maps in %s: %ld bytes\n", obj->path, (long)size);
+	for (i = 0; i < nr_maps; i++) {
+		struct bpf_map_def *def = &obj->maps[i].def;
+
+		/*
+		 * fill all fd with -1 so won't close incorrect
+		 * fd (0, stdin) when failure.
+		 */
+		obj->maps[i].fd = -1;
+
+		/* Save map definition into obj->maps */
+		*def = *(struct bpf_map_def *)(data +
+				i * sizeof(struct bpf_map_def));
+	}
 	return 0;
 }
 
@@ -632,37 +651,15 @@ static int
 bpf_object__create_maps(struct bpf_object *obj)
 {
 	unsigned int i;
-	size_t nr_maps;
-	int *pfd;
-
-	nr_maps = obj->maps_buf_sz / sizeof(struct bpf_map_def);
-	if (!obj->maps_buf || !nr_maps) {
-		pr_debug("don't need create maps for %s\n",
-			 obj->path);
-		return 0;
-	}
-
-	obj->map_fds = malloc(sizeof(int) * nr_maps);
-	if (!obj->map_fds) {
-		pr_warning("realloc perf_bpf_map_fds failed\n");
-		return -ENOMEM;
-	}
-	obj->nr_map_fds = nr_maps;
 
-	/* fill all fd with -1 */
-	memset(obj->map_fds, -1, sizeof(int) * nr_maps);
+	for (i = 0; i < obj->nr_maps; i++) {
+		struct bpf_map_def *def = &obj->maps[i].def;
+		int *pfd = &obj->maps[i].fd;
 
-	pfd = obj->map_fds;
-	for (i = 0; i < nr_maps; i++) {
-		struct bpf_map_def def;
-
-		def = *(struct bpf_map_def *)(obj->maps_buf +
-				i * sizeof(struct bpf_map_def));
-
-		*pfd = bpf_create_map(def.type,
-				      def.key_size,
-				      def.value_size,
-				      def.max_entries);
+		*pfd = bpf_create_map(def->type,
+				      def->key_size,
+				      def->value_size,
+				      def->max_entries);
 		if (*pfd < 0) {
 			size_t j;
 			int err = *pfd;
@@ -670,22 +667,17 @@ bpf_object__create_maps(struct bpf_object *obj)
 			pr_warning("failed to create map: %s\n",
 				   strerror(errno));
 			for (j = 0; j < i; j++)
-				zclose(obj->map_fds[j]);
-			obj->nr_map_fds = 0;
-			zfree(&obj->map_fds);
+				zclose(obj->maps[j].fd);
 			return err;
 		}
 		pr_debug("create map: fd=%d\n", *pfd);
-		pfd++;
 	}
 
-	zfree(&obj->maps_buf);
-	obj->maps_buf_sz = 0;
 	return 0;
 }
 
 static int
-bpf_program__relocate(struct bpf_program *prog, int *map_fds)
+bpf_program__relocate(struct bpf_program *prog, struct bpf_object *obj)
 {
 	int i;
 
@@ -705,7 +697,7 @@ bpf_program__relocate(struct bpf_program *prog, int *map_fds)
 			return -ERANGE;
 		}
 		insns[insn_idx].src_reg = BPF_PSEUDO_MAP_FD;
-		insns[insn_idx].imm = map_fds[map_idx];
+		insns[insn_idx].imm = obj->maps[map_idx].fd;
 	}
 
 	zfree(&prog->reloc_desc);
@@ -724,7 +716,7 @@ bpf_object__relocate(struct bpf_object *obj)
 	for (i = 0; i < obj->nr_programs; i++) {
 		prog = &obj->programs[i];
 
-		err = bpf_program__relocate(prog, obj->map_fds);
+		err = bpf_program__relocate(prog, obj);
 		if (err) {
 			pr_warning("failed to relocate '%s'\n",
 				   prog->section_name);
@@ -748,8 +740,7 @@ static int bpf_object__collect_reloc(struct bpf_object *obj)
 		Elf_Data *data = obj->efile.reloc[i].data;
 		int idx = shdr->sh_info;
 		struct bpf_program *prog;
-		size_t nr_maps = obj->maps_buf_sz /
-				 sizeof(struct bpf_map_def);
+		size_t nr_maps = obj->nr_maps;
 
 		if (shdr->sh_type != SHT_REL) {
 			pr_warning("internal error at %d\n", __LINE__);
@@ -986,10 +977,8 @@ int bpf_object__unload(struct bpf_object *obj)
 	if (!obj)
 		return -EINVAL;
 
-	for (i = 0; i < obj->nr_map_fds; i++)
-		zclose(obj->map_fds[i]);
-	zfree(&obj->map_fds);
-	obj->nr_map_fds = 0;
+	for (i = 0; i < obj->nr_maps; i++)
+		zclose(obj->maps[i].fd);
 
 	for (i = 0; i < obj->nr_programs; i++)
 		bpf_program__unload(&obj->programs[i]);
@@ -1032,7 +1021,15 @@ void bpf_object__close(struct bpf_object *obj)
 	bpf_object__elf_finish(obj);
 	bpf_object__unload(obj);
 
-	zfree(&obj->maps_buf);
+	for (i = 0; i < obj->nr_maps; i++) {
+		if (obj->maps[i].clear_priv)
+			obj->maps[i].clear_priv(&obj->maps[i],
+						obj->maps[i].priv);
+		obj->maps[i].priv = NULL;
+		obj->maps[i].clear_priv = NULL;
+	}
+	zfree(&obj->maps);
+	obj->nr_maps = 0;
 
 	if (obj->programs && obj->nr_programs) {
 		for (i = 0; i < obj->nr_programs; i++)
@@ -1179,3 +1176,72 @@ int bpf_program__nth_fd(struct bpf_program *prog, int n)
 
 	return fd;
 }
+
+int bpf_map__get_fd(struct bpf_map *map)
+{
+	if (!map)
+		return -EINVAL;
+
+	return map->fd;
+}
+
+int bpf_map__get_def(struct bpf_map *map, struct bpf_map_def *pdef)
+{
+	if (!map || !pdef)
+		return -EINVAL;
+
+	*pdef = map->def;
+	return 0;
+}
+
+int bpf_map__set_private(struct bpf_map *map, void *priv,
+			 bpf_map_clear_priv_t clear_priv)
+{
+	if (!map)
+		return -EINVAL;
+
+	if (map->priv) {
+		if (map->clear_priv)
+			map->clear_priv(map, map->priv);
+	}
+
+	map->priv = priv;
+	map->clear_priv = clear_priv;
+	return 0;
+}
+
+int bpf_map__get_private(struct bpf_map *map, void **ppriv)
+{
+	if (!map)
+		return -EINVAL;
+
+	if (ppriv)
+		*ppriv = map->priv;
+	return 0;
+}
+
+struct bpf_map *
+bpf_map__next(struct bpf_map *prev, struct bpf_object *obj)
+{
+	size_t idx;
+	struct bpf_map *s, *e;
+
+	if (!obj || !obj->maps)
+		return NULL;
+
+	s = obj->maps;
+	e = obj->maps + obj->nr_maps;
+
+	if (prev == NULL)
+		return s;
+
+	if ((prev < s) || (prev >= e)) {
+		pr_warning("error: map handler doesn't belong to object\n");
+		return NULL;
+	}
+
+	idx = (prev - obj->maps) + 1;
+	if (idx >= obj->nr_maps)
+		return NULL;
+	return &obj->maps[idx];
+}
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index d82b89e..a3bf71e 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -102,4 +102,25 @@ struct bpf_map_def {
 	unsigned int max_entries;
 };
 
+/*
+ * There is another 'struct bpf_map' in include/linux/map.h. However,
+ * it is not a uapi header so no need to consider name confliction.
+ */
+struct bpf_map;
+
+struct bpf_map *
+bpf_map__next(struct bpf_map *map, struct bpf_object *obj);
+#define bpf_map__for_each(pos, obj)		\
+	for ((pos) = bpf_map__next(NULL, (obj));	\
+	     (pos) != NULL;				\
+	     (pos) = bpf_map__next((pos), (obj)))
+
+int bpf_map__get_fd(struct bpf_map *map);
+int bpf_map__get_def(struct bpf_map *map, struct bpf_map_def *pdef);
+
+typedef void (*bpf_map_clear_priv_t)(struct bpf_map *, void *);
+int bpf_map__set_private(struct bpf_map *map, void *priv,
+			 bpf_map_clear_priv_t clear_priv);
+int bpf_map__get_private(struct bpf_map *map, void **ppriv);
+
 #endif
-- 
1.8.3.4

[PATCH 25/31] bpf tools: Extract and collect map names from BPF object file

[Wang Nan]

This patch collect name of map in BPF object files and saves them into
'maps' field in 'struct bpf_object'. 'bpf_object__get_map_by_name' is
introduced to retrive map fd and definitions through its name.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-ccbekxapr4xy91a3z57xctj0@git.kernel.org
---
 tools/lib/bpf/libbpf.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++--
 tools/lib/bpf/libbpf.h |  3 +++
 2 files changed, 64 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 8ae501b..c4283c7 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -111,6 +111,7 @@ struct bpf_program {
 
 struct bpf_map {
 	int fd;
+	char *name;
 	struct bpf_map_def def;
 	void *priv;
 	bpf_map_clear_priv_t clear_priv;
@@ -471,12 +472,46 @@ bpf_object__init_maps(struct bpf_object *obj, void *data,
 	return 0;
 }
 
+static void
+bpf_object__init_maps_name(struct bpf_object *obj, int maps_shndx)
+{
+	int i;
+	Elf_Data *symbols = obj->efile.symbols;
+
+	if (!symbols || maps_shndx < 0)
+		return;
+
+	for (i = 0; i < symbols->d_size / sizeof(GElf_Sym); i++) {
+		GElf_Sym sym;
+		size_t map_idx;
+		const char *map_name;
+
+		if (!gelf_getsym(symbols, i, &sym))
+			continue;
+		if (sym.st_shndx != maps_shndx)
+			continue;
+
+		map_name = elf_strptr(obj->efile.elf,
+				      obj->efile.ehdr.e_shstrndx,
+				      sym.st_name);
+		map_idx = sym.st_value / sizeof(struct bpf_map_def);
+		if (map_idx >= obj->nr_maps) {
+			pr_warning("index of map \"%s\" is buggy: %zu > %zu\n",
+				   map_name, map_idx, obj->nr_maps);
+			continue;
+		}
+		obj->maps[map_idx].name = strdup(map_name);
+		pr_debug("map %zu is \"%s\"\n", map_idx,
+			 obj->maps[map_idx].name);
+	}
+}
+
 static int bpf_object__elf_collect(struct bpf_object *obj)
 {
 	Elf *elf = obj->efile.elf;
 	GElf_Ehdr *ep = &obj->efile.ehdr;
 	Elf_Scn *scn = NULL;
-	int idx = 0, err = 0;
+	int idx = 0, err = 0, maps_shndx = -1;
 
 	/* Elf is corrupted/truncated, avoid calling elf_strptr. */
 	if (!elf_rawdata(elf_getscn(elf, ep->e_shstrndx), NULL)) {
@@ -526,9 +561,11 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
 			err = bpf_object__init_kversion(obj,
 							data->d_buf,
 							data->d_size);
-		else if (strcmp(name, "maps") == 0)
+		else if (strcmp(name, "maps") == 0) {
 			err = bpf_object__init_maps(obj, data->d_buf,
 						    data->d_size);
+			maps_shndx = idx;
+		}
 		else if (sh.sh_type == SHT_SYMTAB) {
 			if (obj->efile.symbols) {
 				pr_warning("bpf: multiple SYMTAB in %s\n",
@@ -569,6 +606,9 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
 		if (err)
 			goto out;
 	}
+
+	if (maps_shndx >= 0)
+		bpf_object__init_maps_name(obj, maps_shndx);
 out:
 	return err;
 }
@@ -1194,6 +1234,13 @@ int bpf_map__get_def(struct bpf_map *map, struct bpf_map_def *pdef)
 	return 0;
 }
 
+const char *bpf_map__get_name(struct bpf_map *map)
+{
+	if (!map)
+		return NULL;
+	return map->name;
+}
+
 int bpf_map__set_private(struct bpf_map *map, void *priv,
 			 bpf_map_clear_priv_t clear_priv)
 {
@@ -1245,3 +1292,15 @@ bpf_map__next(struct bpf_map *prev, struct bpf_object *obj)
 		return NULL;
 	return &obj->maps[idx];
 }
+
+struct bpf_map *
+bpf_object__get_map_by_name(struct bpf_object *obj, const char *name)
+{
+	struct bpf_map *pos;
+
+	bpf_map__for_each(pos, obj) {
+		if (strcmp(pos->name, name) == 0)
+			return pos;
+	}
+	return NULL;
+}
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index a3bf71e..37b8f27 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -107,6 +107,8 @@ struct bpf_map_def {
  * it is not a uapi header so no need to consider name confliction.
  */
 struct bpf_map;
+struct bpf_map *
+bpf_object__get_map_by_name(struct bpf_object *obj, const char *name);
 
 struct bpf_map *
 bpf_map__next(struct bpf_map *map, struct bpf_object *obj);
@@ -117,6 +119,7 @@ bpf_map__next(struct bpf_map *map, struct bpf_object *obj);
 
 int bpf_map__get_fd(struct bpf_map *map);
 int bpf_map__get_def(struct bpf_map *map, struct bpf_map_def *pdef);
+const char *bpf_map__get_name(struct bpf_map *map);
 
 typedef void (*bpf_map_clear_priv_t)(struct bpf_map *, void *);
 int bpf_map__set_private(struct bpf_map *map, void *priv,
-- 
1.8.3.4

[PATCH 26/31] perf tools: Support perf event alias name

[Wang Nan]

From: He Kuang <hekuang@huawei.com>

This patch adds new bison rules for specifying an alias name to a perf
event, which allows cmdline refer to previous defined perf event through
its name. With this patch user can give alias name to a perf event using
following cmdline:

 # perf record -e mypmu=cycles ...

To allow parser refer to existing event selecter, pass event list to
'struct parse_events_evlist'. perf_evlist__find_evsel_by_alias() is
introduced to get evsel through its alias.

Signed-off-by: He Kuang <hekuang@huawei.com>
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-7w1s62o0s6ovqlaqwrmx20v9@git.kernel.org
---
 tools/perf/util/evlist.c       | 16 ++++++++++++++++
 tools/perf/util/evlist.h       |  4 ++++
 tools/perf/util/evsel.h        |  1 +
 tools/perf/util/parse-events.c | 31 ++++++++++++++++++++++++++++---
 tools/perf/util/parse-events.h |  5 +++++
 tools/perf/util/parse-events.y | 15 ++++++++++++++-
 6 files changed, 68 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index d139219..8dd59aa 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -1753,3 +1753,19 @@ void perf_evlist__set_tracking_event(struct perf_evlist *evlist,
 
 	tracking_evsel->tracking = true;
 }
+
+struct perf_evsel *
+perf_evlist__find_evsel_by_alias(struct perf_evlist *evlist,
+				 const char *alias)
+{
+	struct perf_evsel *evsel;
+
+	evlist__for_each(evlist, evsel) {
+		if (!evsel->alias)
+			continue;
+		if (strcmp(alias, evsel->alias) == 0)
+			return evsel;
+	}
+
+	return NULL;
+}
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index a459fe7..4e25342 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -292,4 +292,8 @@ void perf_evlist__set_tracking_event(struct perf_evlist *evlist,
 				     struct perf_evsel *tracking_evsel);
 
 void perf_event_attr__set_max_precise_ip(struct perf_event_attr *attr);
+
+struct perf_evsel *
+perf_evlist__find_evsel_by_alias(struct perf_evlist *evlist, const char *alias);
+
 #endif /* __PERF_EVLIST_H */
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index a60b5d5..9a95e73 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -87,6 +87,7 @@ struct perf_evsel {
 	int			idx;
 	u32			ids;
 	char			*name;
+	char			*alias;
 	double			scale;
 	const char		*unit;
 	struct event_format	*tp_format;
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 4849dbd..06ba5a6 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -1020,6 +1020,30 @@ int parse_events__modifier_group(struct list_head *list,
 	return parse_events__modifier_event(list, event_mod, true);
 }
 
+int parse_events__set_event_alias(struct parse_events_evlist *data,
+				  struct list_head *list,
+				  const char *str,
+				  void *loc_alias_)
+{
+	struct perf_evsel *evsel;
+	YYLTYPE *loc_alias = loc_alias_;
+
+	if (!str)
+		return 0;
+
+	if (!list_is_singular(list)) {
+		struct parse_events_error *err = data->error;
+
+		err->idx = loc_alias->first_column;
+		err->str = strdup("One alias can be applied to one event only");
+		return -EINVAL;
+	}
+
+	evsel = list_first_entry(list, struct perf_evsel, node);
+	evsel->alias = strdup(str);
+	return evsel->alias ? 0 : -ENOMEM;
+}
+
 void parse_events__set_leader(char *name, struct list_head *list)
 {
 	struct perf_evsel *leader;
@@ -1373,9 +1397,10 @@ int parse_events(struct perf_evlist *evlist, const char *str,
 		 struct parse_events_error *err)
 {
 	struct parse_events_evlist data = {
-		.list  = LIST_HEAD_INIT(data.list),
-		.idx   = evlist->nr_entries,
-		.error = err,
+		.list   = LIST_HEAD_INIT(data.list),
+		.idx    = evlist->nr_entries,
+		.error  = err,
+		.evlist = evlist,
 	};
 	int ret;
 
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 8f17c83..b525353 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -96,6 +96,7 @@ struct parse_events_evlist {
 	int			   idx;
 	int			   nr_groups;
 	struct parse_events_error *error;
+	struct perf_evlist	  *evlist;
 };
 
 struct parse_events_terms {
@@ -168,4 +169,8 @@ extern int is_valid_tracepoint(const char *event_string);
 int valid_event_mount(const char *eventfs);
 char *parse_events_formats_error_string(char *additional_terms);
 
+int parse_events__set_event_alias(struct parse_events_evlist *data,
+				  struct list_head *list,
+				  const char *str,
+				  void *loc_alias_);
 #endif /* __PERF_PARSE_EVENTS_H */
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index ad37996..90e382f 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -76,6 +76,7 @@ static inc_group_count(struct list_head *list,
 %type <head> event_bpf_file
 %type <head> event_def
 %type <head> event_mod
+%type <head> event_alias
 %type <head> event_name
 %type <head> event
 %type <head> events
@@ -192,13 +193,25 @@ event_name PE_MODIFIER_EVENT
 event_name
 
 event_name:
-PE_EVENT_NAME event_def
+PE_EVENT_NAME event_alias
 {
 	ABORT_ON(parse_events_name($2, $1));
 	free($1);
 	$$ = $2;
 }
 |
+event_alias
+
+event_alias:
+PE_NAME '=' event_def
+{
+	struct list_head *list = $3;
+	struct parse_events_evlist *data = _data;
+
+	ABORT_ON(parse_events__set_event_alias(data, list, $1, &@1));
+	$$ = list;
+}
+|
 event_def
 
 event_def: event_pmu |
-- 
1.8.3.4

[PATCH 27/31] perf tools: Pass available CPU number to clang compiler

[Wang Nan]

This patch introduces a new macro "__NR_CPUS__" to perf's embedded
clang compiler, which represent the available CPU counters in this
system. BPF program can use this macro to create a map with same
number of system CPUs. For exmaple:

 struct bpf_map_def SEC("maps") pmu_map = {
     .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
     .key_size = sizeof(int),
     .value_size = sizeof(u32),
     .max_entries = __NR_CPUS__,
 };

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-y603iy62s8w4br4t2gxfois4@git.kernel.org
---
 tools/perf/util/llvm-utils.c | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/tools/perf/util/llvm-utils.c b/tools/perf/util/llvm-utils.c
index 4f6a478..80eecef 100644
--- a/tools/perf/util/llvm-utils.c
+++ b/tools/perf/util/llvm-utils.c
@@ -11,10 +11,11 @@
 #include "cache.h"
 
 #define CLANG_BPF_CMD_DEFAULT_TEMPLATE				\
-		"$CLANG_EXEC -D__KERNEL__ $CLANG_OPTIONS "	\
-		"$KERNEL_INC_OPTIONS -Wno-unused-value "	\
-		"-Wno-pointer-sign -working-directory "		\
-		"$WORKING_DIR -c \"$CLANG_SOURCE\" -target bpf -O2 -o -"
+		"$CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS "\
+		"$CLANG_OPTIONS $KERNEL_INC_OPTIONS "		\
+		"-Wno-unused-value -Wno-pointer-sign "		\
+		"-working-directory $WORKING_DIR "		\
+		"-c \"$CLANG_SOURCE\" -target bpf -O2 -o -"
 
 struct llvm_param llvm_param = {
 	.clang_path = "clang",
@@ -326,8 +327,8 @@ get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 int llvm__compile_bpf(const char *path, void **p_obj_buf,
 		      size_t *p_obj_buf_sz)
 {
-	int err;
-	char clang_path[PATH_MAX];
+	int err, nr_cpus_avail;
+	char clang_path[PATH_MAX], nr_cpus_avail_str[64];
 	const char *clang_opt = llvm_param.clang_opt;
 	const char *template = llvm_param.clang_bpf_cmd_template;
 	char *kbuild_dir = NULL, *kbuild_include_opts = NULL;
@@ -354,6 +355,17 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf,
 	 */
 	get_kbuild_opts(&kbuild_dir, &kbuild_include_opts);
 
+	nr_cpus_avail = sysconf(_SC_NPROCESSORS_CONF);
+	if (nr_cpus_avail <= 0) {
+		pr_err(
+"WARNING:\tunable to get available CPUs in this system: %s\n"
+"        \tUse 128 instead.\n", strerror(errno));
+		nr_cpus_avail = 128;
+	}
+	snprintf(nr_cpus_avail_str, sizeof(nr_cpus_avail_str), "%d",
+		 nr_cpus_avail);
+
+	force_set_env("NR_CPUS", nr_cpus_avail_str);
 	force_set_env("CLANG_EXEC", clang_path);
 	force_set_env("CLANG_OPTIONS", clang_opt);
 	force_set_env("KERNEL_INC_OPTIONS", kbuild_include_opts);
-- 
1.8.3.4

[PATCH 28/31] perf tools: Add API to config maps in bpf object

[Wang Nan]

bpf__config_obj() is introduced as a core API to config BPF object
after loading. One configuration option of maps is introduced. After
this patch BPF object can accept configuration like:

 maps.my_pmy.event=evt

Where evt is a predefined event with alias "evt".

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-36xcrahy9n0ayc05mu7aajpk@git.kernel.org
---
 tools/perf/util/bpf-loader.c | 147 +++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.h |  39 ++++++++++++
 2 files changed, 186 insertions(+)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index 73ff9a9..b92c2f7 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -10,6 +10,7 @@
 #include <linux/err.h>
 #include "perf.h"
 #include "debug.h"
+#include "util.h"
 #include "bpf-loader.h"
 #include "bpf-prologue.h"
 #include "llvm-utils.h"
@@ -633,6 +634,139 @@ int bpf__foreach_tev(struct bpf_object *obj,
 	return 0;
 }
 
+struct bpf_map_priv {
+	struct perf_evsel *evsel;
+};
+
+static void
+bpf_map_priv__clear(struct bpf_map *map __maybe_unused,
+		    void *_priv)
+{
+	struct bpf_map_priv *priv = _priv;
+
+	free(priv);
+}
+
+static int
+bpf__config_obj_map_event(struct bpf_map *map, const char *val,
+			  struct perf_evlist *evlist)
+{
+	struct bpf_map_priv *priv;
+	struct perf_evsel *evsel;
+	struct bpf_map_def def;
+	const char *map_name;
+	int err;
+
+	map_name = bpf_map__get_name(map);
+
+	evsel = perf_evlist__find_evsel_by_alias(evlist, val);
+	if (!evsel) {
+		pr_debug("Event '%s' doesn't exist\n", val);
+		return -EINVAL;
+	}
+
+	err = bpf_map__get_def(map, &def);
+	if (err) {
+		pr_debug("Unable to get map definition from '%s'\n",
+			 map_name);
+		return -EINVAL;
+	}
+
+	if (def.type != BPF_MAP_TYPE_PERF_EVENT_ARRAY) {
+		pr_debug("Map %s type is not BPF_MAP_TYPE_PERF_EVENT_ARRAY\n",
+			 map_name);
+		return -EINVAL;
+	}
+
+	priv = calloc(sizeof(*priv), 1);
+	if (!priv) {
+		pr_debug("No enough memory to alloc map private\n");
+		return -ENOMEM;
+	}
+
+	priv->evsel = evsel;
+	return bpf_map__set_private(map, priv, bpf_map_priv__clear);
+}
+
+struct bpf_config_map_func {
+	const char *config_opt;
+	int (*config_func)(struct bpf_map *, const char *,
+			   struct perf_evlist *);
+};
+
+struct bpf_config_map_func bpf_config_map_funcs[] = {
+	{"event", bpf__config_obj_map_event},
+};
+
+static int
+bpf__config_obj_map(struct bpf_object *obj,
+		    const char *key,
+		    const char *val,
+		    struct perf_evlist *evlist)
+{
+	/* key is "maps.<mapname>.<config opt>" */
+	char *map_name = strdup(key + sizeof("maps.") - 1);
+	struct bpf_map *map;
+	int err = -ENOENT;
+	char *map_opt;
+	size_t i;
+
+	if (!map_name)
+		return -ENOMEM;
+
+	map_opt = strchr(map_name, '.');
+	if (!map_opt) {
+		pr_debug("ERROR: Invalid map config: %s\n", map_name);
+		goto out;
+	}
+
+	*map_opt++ = '\0';
+	if (*map_opt == '\0') {
+		pr_debug("ERROR: Invalid map option: %s\n", key);
+		goto out;
+	}
+
+	map = bpf_object__get_map_by_name(obj, map_name);
+	if (!map) {
+		pr_debug("ERROR: Map %s doesn't exist\n", map_name);
+		goto out;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(bpf_config_map_funcs); i++) {
+		struct bpf_config_map_func *func = &bpf_config_map_funcs[i];
+
+		if (strcmp(map_opt, func->config_opt) == 0) {
+			err = func->config_func(map, val, evlist);
+			goto out;
+		}
+	}
+
+	pr_debug("ERROR: invalid config option '%s' for maps\n",
+		 map_opt);
+	err = -ENOENT;
+out:
+	free(map_name);
+	return err;
+}
+
+int bpf__config_obj(struct bpf_object *obj,
+		    const char *key,
+		    struct bpf_config_val *val,
+		    struct perf_evlist *evlist)
+{
+	if (!obj || !key || !val)
+		return -ENODEV;
+
+	if (!prefixcmp(key, "maps.")) {
+		if (val->type != BPF_CONFIG_VAL_STRING) {
+			pr_debug("ERROR: incorrect value type\n");
+			return -EINVAL;
+		}
+		return bpf__config_obj_map(obj, key, val->string, evlist);
+	}
+	return -ENODEV;
+}
+
 #define bpf__strerror_head(err, buf, size) \
 	char sbuf[STRERR_BUFSIZE], *emsg;\
 	if (!size)\
@@ -675,3 +809,16 @@ int bpf__strerror_load(struct bpf_object *obj __maybe_unused,
 	bpf__strerror_end(buf, size);
 	return 0;
 }
+
+int bpf__strerror_config_obj(struct bpf_object *obj __maybe_unused,
+			     const char *key, struct bpf_config_val *val,
+			     struct perf_evlist *evlist __maybe_unused,
+			     int err, char *buf, size_t size)
+{
+	bpf__strerror_head(err, buf, size);
+	bpf__strerror_entry(ENODEV, "Invalid config option: '%s'", key)
+	bpf__strerror_entry(ENOENT, "Config target in '%s' is invalid", key)
+	bpf__strerror_entry(EINVAL, "Invalid config value %s", val)
+	bpf__strerror_end(buf, size);
+	return 0;
+}
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index d8f1945..4c99b21 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -9,6 +9,7 @@
 #include <linux/err.h>
 #include <string.h>
 #include "probe-event.h"
+#include "evlist.h"
 #include "debug.h"
 
 struct bpf_object;
@@ -17,6 +18,17 @@ struct bpf_object;
 typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
 					int fd, void *arg);
 
+struct bpf_config_val {
+	enum {
+		BPF_CONFIG_VAL_STRING,
+		BPF_CONFIG_VAL_NUM,
+	} type;
+	union {
+		const char *string;
+		unsigned long long num;
+	};
+};
+
 #ifdef HAVE_LIBBPF_SUPPORT
 struct bpf_object *bpf__prepare_load(const char *filename, bool source);
 struct bpf_object *bpf__prepare_load_buffer(void *obj_buf, size_t obj_buf_sz,
@@ -34,6 +46,13 @@ int bpf__strerror_load(struct bpf_object *obj, int err,
 		       char *buf, size_t size);
 int bpf__foreach_tev(struct bpf_object *obj,
 		     bpf_prog_iter_callback_t func, void *arg);
+
+int bpf__config_obj(struct bpf_object *obj, const char *key,
+		    struct bpf_config_val *val, struct perf_evlist *evlist);
+int bpf__strerror_config_obj(struct bpf_object *obj,
+			     const char *key, struct bpf_config_val *val,
+			     struct perf_evlist *evlist,
+			     int err, char *buf, size_t size);
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused,
@@ -65,6 +84,15 @@ bpf__foreach_tev(struct bpf_object *obj __maybe_unused,
 }
 
 static inline int
+bpf__config_obj(struct bpf_object *obj __maybe_unused,
+		const char *key __maybe_unused,
+		struct bpf_config_val *val __maybe_unused,
+		struct perf_evlist *evlist __maybe_unused)
+{
+	return 0;
+}
+
+static inline int
 __bpf_strerror(char *buf, size_t size)
 {
 	if (!size)
@@ -90,5 +118,16 @@ static inline int bpf__strerror_load(struct bpf_object *obj __maybe_unused,
 {
 	return __bpf_strerror(buf, size);
 }
+
+static inline int
+bpf__strerror_config_obj(struct bpf_object *obj __maybe_unused,
+			 const char *key __maybe_unused,
+			 struct bpf_config_val *val __maybe_unused,
+			 struct perf_evlist *evlist __maybe_unused,
+			 int err __maybe_unused,
+			 char *buf, size_t size)
+{
+	return __bpf_strerror(buf, size);
+}
 #endif
 #endif
-- 
1.8.3.4

[PATCH 29/31] perf tools: Add API to apply config to BPF map

[Wang Nan]

bpf__apply_config() is introduced as the core CPI to apply config
options to all BPF objects. This patch also does the real work for
setting maps events for BPF_MAP_TYPE_PERF_EVENT_ARRAY maps by inserting
file descriptions of a evsel into the BPF map.

This patch is required because we are unable to set all BPF config
during parsing. Events in BPF_MAP_TYPE_PERF_EVENT_ARRAY maps is an
example: during parsing, fds of events is not ready yet.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-tmg65cm1zaf1zxs7zmvxmxp4@git.kernel.org
---
 tools/perf/util/bpf-loader.c | 109 +++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.h |  15 ++++++
 2 files changed, 124 insertions(+)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index b92c2f7..9d661c0 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -7,6 +7,7 @@
 
 #include <linux/bpf.h>
 #include <bpf/libbpf.h>
+#include <bpf/bpf.h>
 #include <linux/err.h>
 #include "perf.h"
 #include "debug.h"
@@ -767,6 +768,107 @@ int bpf__config_obj(struct bpf_object *obj,
 	return -ENODEV;
 }
 
+static int
+bpf__apply_config_map(struct bpf_map *map)
+{
+	struct bpf_map_priv *priv;
+	struct bpf_map_def def;
+	const char *name;
+	int err, map_fd;
+
+	name = bpf_map__get_name(map);
+	err = bpf_map__get_private(map, (void **)&priv);
+	if (err) {
+		pr_debug("ERROR: failed to get private field from map %s\n",
+			 name);
+		return err;
+	}
+	if (!priv) {
+		pr_debug("INFO: nothing to config for map %s\n", name);
+		return 0;
+	}
+
+	map_fd = bpf_map__get_fd(map);
+	if (map_fd < 0) {
+		pr_debug("ERROR: failed to get fd from map %s\n", name);
+		return map_fd;
+	}
+
+	err = bpf_map__get_def(map, &def);
+	if (err) {
+		pr_debug("ERROR: failed to retrive map def from map %s\n",
+			 name);
+		return err;
+	}
+
+	if (priv->evsel) {
+		struct xyarray *xy = priv->evsel->fd;
+		unsigned int cpus, i;
+
+		if (!xy) {
+			pr_debug("ERROR: event is not ready for map %s\n", name);
+			return -EINVAL;
+		}
+
+		if (xy->row_size / xy->entry_size != 1) {
+			pr_debug("ERROR: Dimension of target event is incorrect for map %s\n",
+				 name);
+			return -EINVAL;
+		}
+
+		cpus = xy->entries / (xy->row_size / xy->entry_size);
+		if (cpus > def.max_entries) {
+			pr_debug("ERROR: map %s needs to be enlarge to %d for its event\n",
+				 name, cpus);
+			return -EINVAL;
+		} else if (cpus < def.max_entries)
+			pr_debug("WARNING: map %s has more entries than required\n",
+				 name);
+
+		for (i = 0; i < cpus; i++) {
+			int *evt_fd = xyarray__entry(xy, i, 0);
+
+			err = bpf_map_update_elem(map_fd, &i, evt_fd,
+						  BPF_ANY);
+
+			if (err) {
+				pr_debug("ERROR: failed to insert fd %d to %s[%d]\n",
+					 *evt_fd, name, i);
+				return -errno;
+			}
+		}
+	}
+	return 0;
+}
+
+static int
+bpf__apply_config_object(struct bpf_object *obj)
+{
+	struct bpf_map *map;
+	int err;
+
+	bpf_map__for_each(map, obj) {
+		err = bpf__apply_config_map(map);
+		if (err)
+			return err;
+	}
+	return 0;
+}
+
+int bpf__apply_config(void)
+{
+	struct bpf_object *obj, *tmp;
+	int err;
+
+	bpf_object__for_each_safe(obj, tmp) {
+		err = bpf__apply_config_object(obj);
+		if (err)
+			return err;
+	}
+
+	return 0;
+}
+
 #define bpf__strerror_head(err, buf, size) \
 	char sbuf[STRERR_BUFSIZE], *emsg;\
 	if (!size)\
@@ -822,3 +924,10 @@ int bpf__strerror_config_obj(struct bpf_object *obj __maybe_unused,
 	bpf__strerror_end(buf, size);
 	return 0;
 }
+
+int bpf__strerror_apply_config(int err, char *buf, size_t size)
+{
+	bpf__strerror_head(err, buf, size);
+	bpf__strerror_end(buf, size);
+	return 0;
+}
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index 4c99b21..3a93ba3 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -53,6 +53,8 @@ int bpf__strerror_config_obj(struct bpf_object *obj,
 			     const char *key, struct bpf_config_val *val,
 			     struct perf_evlist *evlist,
 			     int err, char *buf, size_t size);
+int bpf__apply_config(void);
+int bpf__strerror_apply_config(int err, char *buf, size_t size);
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused,
@@ -93,6 +95,12 @@ bpf__config_obj(struct bpf_object *obj __maybe_unused,
 }
 
 static inline int
+bpf__apply_config(void)
+{
+	return 0;
+}
+
+static inline int
 __bpf_strerror(char *buf, size_t size)
 {
 	if (!size)
@@ -129,5 +137,12 @@ bpf__strerror_config_obj(struct bpf_object *obj __maybe_unused,
 {
 	return __bpf_strerror(buf, size);
 }
+
+static inline int
+bpf__strerror_apply_config(int err __maybe_unused,
+			   char *buf, size_t size)
+{
+	return __bpf_strerror(buf, size);
+}
 #endif
 #endif
-- 
1.8.3.4

[PATCH 30/31] perf record: Apply config to BPF objects before recording

[Wang Nan]

From: He Kuang <hekuang@huawei.com>

In perf record, before start recording, call bpf__apply_config() to
turn on all BPF config options.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-ziazd5s4t9j96d01t5bdbtat@git.kernel.org
---
 tools/perf/builtin-record.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 200f221..a47ce9e 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -32,6 +32,7 @@
 #include "util/parse-branch-options.h"
 #include "util/parse-regs-options.h"
 #include "util/llvm-utils.h"
+#include "util/bpf-loader.h"
 
 #include <unistd.h>
 #include <sched.h>
@@ -524,6 +525,15 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 		goto out_child;
 	}
 
+	err = bpf__apply_config();
+	if (err) {
+		char errbuf[BUFSIZ];
+
+		bpf__strerror_apply_config(err, errbuf, sizeof(errbuf));
+		pr_warning("WARNING: Apply config to BPF failed: %s\n",
+			   errbuf);
+	}
+
 	/*
 	 * Normally perf_session__new would do this, but it doesn't have the
 	 * evlist.
-- 
1.8.3.4

[PATCH 31/31] perf tools: Enable BPF object configure syntax

[Wang Nan]

This patch adds the final step for BPF map configuration. A new syntax
is appended into parser so user can config BPF objects through '/' '/'
enclosed config terms.

After this patch, BPF programs for perf are finally able to utilize
bpf_perf_event_read() introduced in commit 35578d7984003097af2b1e3
(bpf: Implement function bpf_perf_event_read() that get the selected
hardware PMU conuter) by following way:

 ===== BPF program bpf_program.c =====

 struct bpf_map_def SEC("maps") pmu_map = {
     .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
     .key_size = sizeof(int),
     .value_size = sizeof(u32),
     .max_entries = __NR_CPUS__,
 };

 SEC("func_write=sys_write")
 int func_write(void *ctx)
 {
     unsigned long long val;
     char fmt[] = "sys_write:        pmu=%llu\n";
     val = bpf_perf_event_read(&pmu_map, bpf_get_smp_processor_id());
     bpf_trace_printk(fmt, sizeof(fmt), val);
     return 0;
 }

 SEC("func_write_return=sys_write%return")
 int func_write_return(void *ctx)
 {
     unsigned long long val = 0;
     char fmt[] = "sys_write_return: pmu=%llu\n";
     val = bpf_perf_event_read(&pmu_map, bpf_get_smp_processor_id());
     bpf_trace_printk(fmt, sizeof(fmt), val);
     return 0;
 }

 ===== cmdline =====
 # echo "" > /sys/kernel/debug/tracing/trace
 # perf record -e evt=cycles/period=0x7fffffffffffffff/ \
               -e bpf_program.c/maps.pmu_map.event=evt/
               -a ls
 # cat /sys/kernel/debug/tracing/trace | grep ls
              ls-3363  [003] d... 75475.056190: : sys_write:        pmu=3961415
              ls-3363  [003] dN.. 75475.056212: : sys_write_return: pmu=4051390
              ls-3363  [003] d... 75475.056216: : sys_write:        pmu=4065447
              ls-3363  [003] dN.. 75475.056227: : sys_write_return: pmu=4109760
              ls-3363  [003] d... 75475.056230: : sys_write:        pmu=4120776
              ls-3363  [003] dN.. 75475.056245: : sys_write_return: pmu=4178441
              ...
 # perf report --stdio
Error:
The perf.data file has no samples!

Where, setting period of cycles Set a very large value to period
of cycles event because we want to use this event as a counter
only, don't need sampling.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/n/ebpf-2mjd96mowgzslkj8jrwbnwg7@git.kernel.org
---
 tools/perf/util/parse-events.c | 71 ++++++++++++++++++++++++++++++++++++++++--
 tools/perf/util/parse-events.h |  3 +-
 tools/perf/util/parse-events.y | 19 ++++++++---
 3 files changed, 85 insertions(+), 8 deletions(-)

diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 06ba5a6..f8b2bb8 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -624,17 +624,79 @@ errout:
 	return err;
 }
 
+static int
+parse_events_config_bpf(struct parse_events_evlist *data,
+		       struct bpf_object *obj,
+		       struct list_head *head_config)
+{
+	struct parse_events_term *term;
+
+	if (!head_config || list_empty(head_config))
+		return 0;
+
+	list_for_each_entry(term, head_config, list) {
+		struct bpf_config_val val;
+		char errbuf[BUFSIZ];
+		int err;
+
+		if (term->type_term != PARSE_EVENTS__TERM_TYPE_USER) {
+			snprintf(errbuf, sizeof(errbuf),
+				 "Invalid config term for BPF object");
+			errbuf[BUFSIZ - 1] = '\0';
+
+			data->error->idx = term->err_term;
+			data->error->str = strdup(errbuf);
+			return -EINVAL;
+		}
+
+		switch (term->type_val) {
+		case PARSE_EVENTS__TERM_TYPE_NUM:
+			val.type = BPF_CONFIG_VAL_NUM;
+			val.num = (unsigned long long)term->val.num;
+			break;
+		case PARSE_EVENTS__TERM_TYPE_STR:
+			val.type = BPF_CONFIG_VAL_STRING;
+			val.string = term->val.str;
+			break;
+		default:
+			data->error->idx = term->err_val;
+			data->error->str = strdup("Invalid config value");
+			return -EINVAL;
+		}
+
+		err = bpf__config_obj(obj, term->config, &val, data->evlist);
+		if (err) {
+			bpf__strerror_config_obj(obj, term->config, &val,
+						 data->evlist, err, errbuf,
+						 sizeof(errbuf));
+			data->error->help = strdup(
+"Hint:\tValid config term:\n"
+"     \tmaps.<mapname>.event\n"
+"     \t(add -v to see detail)");
+			data->error->str = strdup(errbuf);
+			if (err == -EINVAL)
+				data->error->idx = term->err_val;
+			else
+				data->error->idx = term->err_term;
+			return err;
+		}
+	}
+	return 0;
+
+}
+
 int parse_events_load_bpf(struct parse_events_evlist *data,
 			  struct list_head *list,
 			  char *bpf_file_name,
-			  bool source)
+			  bool source,
+			  struct list_head *head_config)
 {
 	struct bpf_object *obj;
+	int err;
 
 	obj = bpf__prepare_load(bpf_file_name, source);
 	if (IS_ERR(obj) || !obj) {
 		char errbuf[BUFSIZ];
-		int err;
 
 		err = obj ? PTR_ERR(obj) : -EINVAL;
 
@@ -651,7 +713,10 @@ int parse_events_load_bpf(struct parse_events_evlist *data,
 		return err;
 	}
 
-	return parse_events_load_bpf_obj(data, list, obj);
+	err = parse_events_load_bpf_obj(data, list, obj);
+	if (err)
+		return err;
+	return parse_events_config_bpf(data, obj, head_config);
 }
 
 static int
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index b525353..d4aa88e 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -125,7 +125,8 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
 int parse_events_load_bpf(struct parse_events_evlist *data,
 			  struct list_head *list,
 			  char *bpf_file_name,
-			  bool source);
+			  bool source,
+			  struct list_head *head_config);
 /* Provide this function for perf test */
 struct bpf_object;
 int parse_events_load_bpf_obj(struct parse_events_evlist *data,
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index 90e382f..545bce5 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -64,6 +64,7 @@ static inc_group_count(struct list_head *list,
 %type <str> PE_PMU_EVENT_PRE PE_PMU_EVENT_SUF PE_KERNEL_PMU_EVENT
 %type <num> value_sym
 %type <head> event_config
+%type <head> event_bpf_config
 %type <term> event_term
 %type <head> event_pmu
 %type <head> event_legacy_symbol
@@ -468,27 +469,37 @@ PE_RAW
 }
 
 event_bpf_file:
-PE_BPF_OBJECT
+PE_BPF_OBJECT event_bpf_config
 {
 	struct parse_events_evlist *data = _data;
 	struct parse_events_error *error = data->error;
 	struct list_head *list;
 
 	ALLOC_LIST(list);
-	ABORT_ON(parse_events_load_bpf(data, list, $1, false));
+	ABORT_ON(parse_events_load_bpf(data, list, $1, false, $2));
 	$$ = list;
 }
 |
-PE_BPF_SOURCE
+PE_BPF_SOURCE event_bpf_config
 {
 	struct parse_events_evlist *data = _data;
 	struct list_head *list;
 
 	ALLOC_LIST(list);
-	ABORT_ON(parse_events_load_bpf(data, list, $1, true));
+	ABORT_ON(parse_events_load_bpf(data, list, $1, true, $2));
 	$$ = list;
 }
 
+event_bpf_config:
+'/' event_config '/'
+{
+	$$ = $2;
+}
+|
+{
+	$$ = NULL;
+}
+
 start_terms: event_config
 {
 	struct parse_events_terms *data = _data;
-- 
1.8.3.4

Re: [GIT PULL 00/31] perf tools: filtering events using eBPF programs

[Namhyung Kim]

Hello,

On Wed, Oct 14, 2015 at 12:41:11PM +0000, Wang Nan wrote:
> Hi Arnaldo,
> 
>    I know you don't have enough time to review my code. I send
> this patchset to let you and other know what we are working on.
> 
>    In this new patchset, we create a new perf cmdline syntax so
> perf users are able to pass perf events created by perf to BPF
> maps, which makes bpf_perf_event_read() usable. Compare with out
> previous solution[1] which embedded 'struct perf_event_attr' to
> "maps" section, this solution is easier to use.
> 
>    If you or anyone have any different views on this solution,
> please let us know so we can stop our further development base
> on it as soon as possible.

Thanks for working on this.  I've only reviewed the first part and
will continue to review it tomorrow (or later). :)

For patch 01 - 08,

  Acked-by: Namhyung Kim <namhyung@kernel.org>

Thanks,
Namhyung


> 
> Thank you.
> 
>  [1] http://lkml.kernel.org/r/1440672142-89311-1-git-send-email-xiakaixu@huawei.com
> 
> The following changes since commit 31eb4360546b4bd890f349db01295a173c09b0fb:
> 
>   perf hists browser: Add 'm' key for context menu display (2015-10-12 23:29:14 -0300)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/pi3orama/linux.git tags/perf-ebpf-for-acme-20151014
> 
> for you to fetch changes up to 6df036cb7d42a2e0ebf312e127b02425bd57bc55:
> 
>   perf tools: Enable BPF object configure syntax (2015-10-14 10:09:17 +0000)
> 
> ----------------------------------------------------------------
> EBPF support for perf
> 
>  - Rebase to newest perf/core
> 
>  - Bugfix: kprobe events not remove if bpf__probe() failure occur
>    after it creates some kprobe points successfully.
> 
>  - Bugfix: when multiple BPF functions reside in one BPF object,
>    the last BPF program would be attached to all kprobe events.
>    This bug is introduced by removal of dummy event placeholder.
> 
>  - New function: support BPF program reading counter through
>    bpf_perf_event_read() by adding new syntax and support code
>    in event selector. The new BPF object configuration mechanism
>    can be extended to support BPF data output.
> 
>    In this patchset, following BPF function can be used to measure
>    cycles a kernel function cost:
> 
>    ===== BPF program bpf_program.c =====
> 
>    struct bpf_map_def SEC("maps") pmu_map = {
>        .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
>        .key_size = sizeof(int),
>        .value_size = sizeof(u32),
>        .max_entries = __NR_CPUS__,
>    };
> 
>    SEC("func_write=sys_write")
>    int func_write(void *ctx)
>    {
>        unsigned long long val;
>        char fmt[] = "sys_write:        pmu=%llu\n";
>        val = bpf_perf_event_read(&pmu_map, bpf_get_smp_processor_id());
>        bpf_trace_printk(fmt, sizeof(fmt), val);
>        return 0;
>    }
> 
>    SEC("func_write_return=sys_write%return")
>    int func_write_return(void *ctx)
>    {
>        unsigned long long val = 0;
>        char fmt[] = "sys_write_return: pmu=%llu\n";
>        val = bpf_perf_event_read(&pmu_map, bpf_get_smp_processor_id());
>        bpf_trace_printk(fmt, sizeof(fmt), val);
>        return 0;
>    }
> 
>    With cmdline like this:
> 
>    ===== cmdline =====
>    # echo "" > /sys/kernel/debug/tracing/trace
>    # perf record -e evt=cycles/period=0x7fffffffffffffff/ \
>                  -e bpf_program.c/maps.pmu_map.event=evt/
>                  -a ls
>    # cat /sys/kernel/debug/tracing/trace | grep ls
>                 ls-3363  [003] d... 75475.056190: : sys_write:        pmu=3961415
>                 ls-3363  [003] dN.. 75475.056212: : sys_write_return: pmu=4051390
>                 ls-3363  [003] d... 75475.056216: : sys_write:        pmu=4065447
>                 ls-3363  [003] dN.. 75475.056227: : sys_write_return: pmu=4109760
>                 ls-3363  [003] d... 75475.056230: : sys_write:        pmu=4120776
>                 ls-3363  [003] dN.. 75475.056245: : sys_write_return: pmu=4178441
>                 ...
>    # perf report --stdio
>    Error:
>    The perf.data file has no samples!
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> 
> ----------------------------------------------------------------
> He Kuang (5):
>       perf tools: Add prologue for BPF programs for fetching arguments
>       perf record: Support custom vmlinux path
>       bpf tools: Add helper function for updating bpf maps elements
>       perf tools: Support perf event alias name
>       perf record: Apply config to BPF objects before recording
> 
> Wang Nan (26):
>       perf tools: Make perf depend on libbpf
>       perf ebpf: Add the libbpf glue
>       perf tools: Enable passing bpf object file to --event
>       perf record, bpf: Create probe points for BPF programs
>       perf record: Load eBPF object into kernel
>       perf tools: Collect perf_evsel in BPF object files
>       perf tools: Attach eBPF program to perf event
>       perf record: Add clang options for compiling BPF scripts
>       perf tools: Compile scriptlets to BPF objects when passing '.c' to --event
>       perf test: Enforce LLVM test for BPF test
>       perf test: Add 'perf test BPF'
>       perf probe: Reset args and nargs for probe_trace_event when failure
>       bpf tools: Load a program with different instances using preprocessor
>       perf tools: Add BPF_PROLOGUE config options for further patches
>       perf tools: Compile dwarf-regs.c if CONFIG_BPF_PROLOGUE is on
>       perf tools: Generate prologue for BPF programs
>       perf tools: Use same BPF program if arguments are identical
>       perf tools: Allow BPF program attach to uprobe events
>       perf test: Enforce LLVM test, add kbuild test
>       perf test: Test BPF prologue
>       bpf tools: Collect map definition in bpf_object
>       bpf tools: Extract and collect map names from BPF object file
>       perf tools: Pass available CPU number to clang compiler
>       perf tools: Add API to config maps in bpf object
>       perf tools: Add API to apply config to BPF map
>       perf tools: Enable BPF object configure syntax
> 
>  tools/build/Makefile.feature                |   6 +-
>  tools/lib/bpf/bpf.c                         |  14 +
>  tools/lib/bpf/bpf.h                         |   2 +
>  tools/lib/bpf/libbpf.c                      | 392 +++++++++---
>  tools/lib/bpf/libbpf.h                      |  46 ++
>  tools/perf/MANIFEST                         |   3 +
>  tools/perf/Makefile.perf                    |  21 +-
>  tools/perf/arch/x86/util/Build              |   1 +
>  tools/perf/builtin-record.c                 |  21 +
>  tools/perf/config/Makefile                  |  31 +-
>  tools/perf/perf.c                           |   2 +
>  tools/perf/tests/Build                      |  24 +-
>  tools/perf/tests/bpf-script-example.c       |  48 ++
>  tools/perf/tests/bpf-script-test-kbuild.c   |  21 +
>  tools/perf/tests/bpf-script-test-prologue.c |  35 ++
>  tools/perf/tests/bpf.c                      | 227 +++++++
>  tools/perf/tests/builtin-test.c             |  10 +
>  tools/perf/tests/llvm.c                     | 210 ++++++-
>  tools/perf/tests/llvm.h                     |  29 +
>  tools/perf/tests/make                       |   4 +-
>  tools/perf/tests/tests.h                    |   5 +
>  tools/perf/util/Build                       |   2 +
>  tools/perf/util/bpf-loader.c                | 933 ++++++++++++++++++++++++++++
>  tools/perf/util/bpf-loader.h                | 148 +++++
>  tools/perf/util/bpf-prologue.c              | 443 +++++++++++++
>  tools/perf/util/bpf-prologue.h              |  34 +
>  tools/perf/util/evlist.c                    |  16 +
>  tools/perf/util/evlist.h                    |   4 +
>  tools/perf/util/evsel.c                     |  17 +
>  tools/perf/util/evsel.h                     |   2 +
>  tools/perf/util/llvm-utils.c                |  24 +-
>  tools/perf/util/parse-events.c              | 221 ++++++-
>  tools/perf/util/parse-events.h              |  15 +
>  tools/perf/util/parse-events.l              |   6 +
>  tools/perf/util/parse-events.y              |  55 +-
>  tools/perf/util/probe-finder.c              |   4 +
>  36 files changed, 2968 insertions(+), 108 deletions(-)
>  create mode 100644 tools/perf/tests/bpf-script-example.c
>  create mode 100644 tools/perf/tests/bpf-script-test-kbuild.c
>  create mode 100644 tools/perf/tests/bpf-script-test-prologue.c
>  create mode 100644 tools/perf/tests/bpf.c
>  create mode 100644 tools/perf/tests/llvm.h
>  create mode 100644 tools/perf/util/bpf-loader.c
>  create mode 100644 tools/perf/util/bpf-loader.h
>  create mode 100644 tools/perf/util/bpf-prologue.c
>  create mode 100644 tools/perf/util/bpf-prologue.h
> 
> -- 
> 1.8.3.4
> 

Re: [PATCH 09/31] perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

[Namhyung Kim]

On Wed, Oct 14, 2015 at 12:41:20PM +0000, Wang Nan wrote:
> This patch provides infrastructure for passing source files to --event
> directly using:
> 
>  # perf record --event bpf-file.c command
> 
> This patch does following works:
> 
>  1) Allow passing '.c' file to '--event'. parse_events_load_bpf() is
>     expanded to allow caller tell it whether the passed file is source
>     file or object.
> 
>  2) llvm__compile_bpf() is called to compile the '.c' file, the result
>     is saved into memory. Use bpf_object__open_buffer() to load the
>     in-memory object.
> 
> Introduces a bpf-script-example.c so we can manually test it:
> 
>  # perf record --clang-opt "-DLINUX_VERSION_CODE=0x40200" --event ./bpf-script-example.c sleep 1
> 
> Note that '--clang-opt' must put before '--event'.
> 
> Futher patches will merge it into a testcase so can be tested automatically.
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Signed-off-by: He Kuang <hekuang@huawei.com>
> Acked-by: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---
>  tools/perf/tests/bpf-script-example.c | 44 +++++++++++++++++++++++++++++++++++
>  tools/perf/util/bpf-loader.c          | 17 ++++++++++++--
>  tools/perf/util/bpf-loader.h          |  5 ++--
>  tools/perf/util/parse-events.c        |  5 ++--
>  tools/perf/util/parse-events.h        |  3 ++-
>  tools/perf/util/parse-events.l        |  3 +++
>  tools/perf/util/parse-events.y        | 15 ++++++++++--
>  7 files changed, 83 insertions(+), 9 deletions(-)
>  create mode 100644 tools/perf/tests/bpf-script-example.c
> 
> diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
> new file mode 100644
> index 0000000..410a70b
> --- /dev/null
> +++ b/tools/perf/tests/bpf-script-example.c

Shouldn't it be a part of the next patch?

Thanks,
Namhyung


> @@ -0,0 +1,44 @@
> +#ifndef LINUX_VERSION_CODE
> +# error Need LINUX_VERSION_CODE
> +# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
> +#endif
> +#define BPF_ANY 0
> +#define BPF_MAP_TYPE_ARRAY 2
> +#define BPF_FUNC_map_lookup_elem 1
> +#define BPF_FUNC_map_update_elem 2
> +
> +static void *(*bpf_map_lookup_elem)(void *map, void *key) =
> +	(void *) BPF_FUNC_map_lookup_elem;
> +static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
> +	(void *) BPF_FUNC_map_update_elem;
> +
> +struct bpf_map_def {
> +	unsigned int type;
> +	unsigned int key_size;
> +	unsigned int value_size;
> +	unsigned int max_entries;
> +};
> +
> +#define SEC(NAME) __attribute__((section(NAME), used))
> +struct bpf_map_def SEC("maps") flip_table = {
> +	.type = BPF_MAP_TYPE_ARRAY,
> +	.key_size = sizeof(int),
> +	.value_size = sizeof(int),
> +	.max_entries = 1,
> +};
> +
> +SEC("func=sys_epoll_pwait")
> +int bpf_func__sys_epoll_pwait(void *ctx)
> +{
> +	int ind =0;
> +	int *flag = bpf_map_lookup_elem(&flip_table, &ind);
> +	int new_flag;
> +	if (!flag)
> +		return 0;
> +	/* flip flag and store back */
> +	new_flag = !*flag;
> +	bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
> +	return new_flag;
> +}
> +char _license[] SEC("license") = "GPL";
> +int _version SEC("version") = LINUX_VERSION_CODE;
> diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
> index aa784a4..ba6f752 100644
> --- a/tools/perf/util/bpf-loader.c
> +++ b/tools/perf/util/bpf-loader.c
> @@ -12,6 +12,7 @@
>  #include "bpf-loader.h"
>  #include "probe-event.h"
>  #include "probe-finder.h" // for MAX_PROBES
> +#include "llvm-utils.h"
>  
>  #define DEFINE_PRINT_FN(name, level) \
>  static int libbpf_##name(const char *fmt, ...)	\
> @@ -33,7 +34,7 @@ struct bpf_prog_priv {
>  	struct perf_probe_event pev;
>  };
>  
> -struct bpf_object *bpf__prepare_load(const char *filename)
> +struct bpf_object *bpf__prepare_load(const char *filename, bool source)
>  {
>  	struct bpf_object *obj;
>  	static bool libbpf_initialized;
> @@ -45,7 +46,19 @@ struct bpf_object *bpf__prepare_load(const char *filename)
>  		libbpf_initialized = true;
>  	}
>  
> -	obj = bpf_object__open(filename);
> +	if (source) {
> +		int err;
> +		void *obj_buf;
> +		size_t obj_buf_sz;
> +
> +		err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
> +		if (err)
> +			return ERR_PTR(err);
> +		obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, filename);
> +		free(obj_buf);
> +	} else
> +		obj = bpf_object__open(filename);
> +
>  	if (!obj) {
>  		pr_debug("bpf: failed to load %s\n", filename);
>  		return ERR_PTR(-EINVAL);
> diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
> index a8f25ee..ccd8d7f 100644
> --- a/tools/perf/util/bpf-loader.h
> +++ b/tools/perf/util/bpf-loader.h
> @@ -18,7 +18,7 @@ typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
>  					int fd, void *arg);
>  
>  #ifdef HAVE_LIBBPF_SUPPORT
> -struct bpf_object *bpf__prepare_load(const char *filename);
> +struct bpf_object *bpf__prepare_load(const char *filename, bool source);
>  
>  void bpf__clear(void);
>  
> @@ -34,7 +34,8 @@ int bpf__foreach_tev(struct bpf_object *obj,
>  		     bpf_prog_iter_callback_t func, void *arg);
>  #else
>  static inline struct bpf_object *
> -bpf__prepare_load(const char *filename __maybe_unused)
> +bpf__prepare_load(const char *filename __maybe_unused,
> +		  bool source __maybe_unused)
>  {
>  	pr_debug("ERROR: eBPF object loading is disabled during compiling.\n");
>  	return ERR_PTR(-ENOTSUP);
> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
> index 5087023..4849dbd 100644
> --- a/tools/perf/util/parse-events.c
> +++ b/tools/perf/util/parse-events.c
> @@ -626,11 +626,12 @@ errout:
>  
>  int parse_events_load_bpf(struct parse_events_evlist *data,
>  			  struct list_head *list,
> -			  char *bpf_file_name)
> +			  char *bpf_file_name,
> +			  bool source)
>  {
>  	struct bpf_object *obj;
>  
> -	obj = bpf__prepare_load(bpf_file_name);
> +	obj = bpf__prepare_load(bpf_file_name, source);
>  	if (IS_ERR(obj) || !obj) {
>  		char errbuf[BUFSIZ];
>  		int err;
> diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
> index fbb16c7..8f17c83 100644
> --- a/tools/perf/util/parse-events.h
> +++ b/tools/perf/util/parse-events.h
> @@ -123,7 +123,8 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
>  				struct list_head *head_config);
>  int parse_events_load_bpf(struct parse_events_evlist *data,
>  			  struct list_head *list,
> -			  char *bpf_file_name);
> +			  char *bpf_file_name,
> +			  bool source);
>  /* Provide this function for perf test */
>  struct bpf_object;
>  int parse_events_load_bpf_obj(struct parse_events_evlist *data,
> diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
> index 5e5d31a..eeea4e1 100644
> --- a/tools/perf/util/parse-events.l
> +++ b/tools/perf/util/parse-events.l
> @@ -116,6 +116,7 @@ group		[^,{}/]*[{][^}]*[}][^,{}/]*
>  event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
>  event		[^,{}/]+
>  bpf_object	.*\.(o|bpf)
> +bpf_source	.*\.c
>  
>  num_dec		[0-9]+
>  num_hex		0x[a-fA-F0-9]+
> @@ -161,6 +162,7 @@ modifier_bp	[rwx]{1,3}
>  
>  {event_pmu}	|
>  {bpf_object}	|
> +{bpf_source}	|
>  {event}		{
>  			BEGIN(INITIAL);
>  			REWIND(1);
> @@ -267,6 +269,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
>  
>  {modifier_event}	{ return str(yyscanner, PE_MODIFIER_EVENT); }
>  {bpf_object}		{ return str(yyscanner, PE_BPF_OBJECT); }
> +{bpf_source}		{ return str(yyscanner, PE_BPF_SOURCE); }
>  {name}			{ return pmu_str_check(yyscanner); }
>  "/"			{ BEGIN(config); return '/'; }
>  -			{ return '-'; }
> diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
> index 497f19b..ad37996 100644
> --- a/tools/perf/util/parse-events.y
> +++ b/tools/perf/util/parse-events.y
> @@ -42,7 +42,7 @@ static inc_group_count(struct list_head *list,
>  %token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
>  %token PE_EVENT_NAME
>  %token PE_NAME
> -%token PE_BPF_OBJECT
> +%token PE_BPF_OBJECT PE_BPF_SOURCE
>  %token PE_MODIFIER_EVENT PE_MODIFIER_BP
>  %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
>  %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
> @@ -55,6 +55,7 @@ static inc_group_count(struct list_head *list,
>  %type <num> PE_TERM
>  %type <str> PE_NAME
>  %type <str> PE_BPF_OBJECT
> +%type <str> PE_BPF_SOURCE
>  %type <str> PE_NAME_CACHE_TYPE
>  %type <str> PE_NAME_CACHE_OP_RESULT
>  %type <str> PE_MODIFIER_EVENT
> @@ -461,7 +462,17 @@ PE_BPF_OBJECT
>  	struct list_head *list;
>  
>  	ALLOC_LIST(list);
> -	ABORT_ON(parse_events_load_bpf(data, list, $1));
> +	ABORT_ON(parse_events_load_bpf(data, list, $1, false));
> +	$$ = list;
> +}
> +|
> +PE_BPF_SOURCE
> +{
> +	struct parse_events_evlist *data = _data;
> +	struct list_head *list;
> +
> +	ALLOC_LIST(list);
> +	ABORT_ON(parse_events_load_bpf(data, list, $1, true));
>  	$$ = list;
>  }
>  
> -- 
> 1.8.3.4
> 

Re: [PATCH 10/31] perf test: Enforce LLVM test for BPF test

[Namhyung Kim]

On Wed, Oct 14, 2015 at 12:41:21PM +0000, Wang Nan wrote:
> This patch replaces the original toy BPF program with previous introduced
> bpf-script-example.c. Dynamically embedded it into 'llvm-src.c'.
> 
> The newly introduced BPF program attaches a BPF program at
> 'sys_epoll_pwait()', and collect half samples from it. perf itself never
> use that syscall, so further test can verify their result with it.
> 
> Since BPF program require LINUX_VERSION_CODE of runtime kernel, this
> patch computes that code from uname.
> 
> Since the resuling BPF object is useful for further testcases, this patch
> introduces 'prepare' and 'cleanup' method to tests, and makes test__llvm()
> create a MAP_SHARED memory array to hold the resulting object.
> 
> Signed-off-by: He Kuang <hekuang@huawei.com>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---

[SNIP]

> +void test__llvm_prepare(void)
> +{
> +	p_test_llvm__bpf_result = mmap(NULL, SHARED_BUF_INIT_SIZE,
> +				       PROT_READ | PROT_WRITE,
> +				       MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> +	if (!p_test_llvm__bpf_result)

It should check MAP_FAILED instead.


> +		return;
> +	memset((void *)p_test_llvm__bpf_result, '\0', SHARED_BUF_INIT_SIZE);
> +}
> +
> +void test__llvm_cleanup(void)
> +{
> +	unsigned long boundary, buf_end;
> +
> +	if (!p_test_llvm__bpf_result)

Ditto.

Thanks,
Namhyung


> +		return;
> +	if (p_test_llvm__bpf_result->size == 0) {
> +		munmap((void *)p_test_llvm__bpf_result, SHARED_BUF_INIT_SIZE);
> +		p_test_llvm__bpf_result = NULL;
> +		return;
> +	}
> +
> +	buf_end = (unsigned long)p_test_llvm__bpf_result + SHARED_BUF_INIT_SIZE;
> +
> +	boundary = (unsigned long)(p_test_llvm__bpf_result);
> +	boundary += p_test_llvm__bpf_result->size;
> +	boundary = (boundary + (page_size - 1)) &
> +			(~((unsigned long)page_size - 1));
> +	munmap((void *)boundary, buf_end - boundary);
> +}
> diff --git a/tools/perf/tests/llvm.h b/tools/perf/tests/llvm.h
> new file mode 100644
> index 0000000..1e89e46
> --- /dev/null
> +++ b/tools/perf/tests/llvm.h
> @@ -0,0 +1,14 @@
> +#ifndef PERF_TEST_LLVM_H
> +#define PERF_TEST_LLVM_H
> +
> +#include <stddef.h> /* for size_t */
> +
> +struct test_llvm__bpf_result {
> +	size_t size;
> +	char object[];
> +};
> +
> +extern struct test_llvm__bpf_result *p_test_llvm__bpf_result;
> +extern const char test_llvm__bpf_prog[];
> +
> +#endif
> diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
> index c804869..a848802 100644
> --- a/tools/perf/tests/tests.h
> +++ b/tools/perf/tests/tests.h
> @@ -27,6 +27,8 @@ enum {
>  struct test {
>  	const char *desc;
>  	int (*func)(void);
> +	void (*prepare)(void);
> +	void (*cleanup)(void);
>  };
>  
>  /* Tests */
> @@ -66,6 +68,8 @@ int test__fdarray__add(void);
>  int test__kmod_path__parse(void);
>  int test__thread_map(void);
>  int test__llvm(void);
> +void test__llvm_prepare(void);
> +void test__llvm_cleanup(void);
>  int test_session_topology(void);
>  
>  #if defined(__arm__) || defined(__aarch64__)
> -- 
> 1.8.3.4
> 

Re: [PATCH 09/31] perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

[Wangnan (F)]

On 2015/10/14 23:45, Namhyung Kim wrote:
> On Wed, Oct 14, 2015 at 12:41:20PM +0000, Wang Nan wrote:
>> This patch provides infrastructure for passing source files to --event
>> directly using:
>>
>>   # perf record --event bpf-file.c command
>>
>> This patch does following works:
>>
>>   1) Allow passing '.c' file to '--event'. parse_events_load_bpf() is
>>      expanded to allow caller tell it whether the passed file is source
>>      file or object.
>>
>>   2) llvm__compile_bpf() is called to compile the '.c' file, the result
>>      is saved into memory. Use bpf_object__open_buffer() to load the
>>      in-memory object.
>>
>> Introduces a bpf-script-example.c so we can manually test it:
>>
>>   # perf record --clang-opt "-DLINUX_VERSION_CODE=0x40200" --event ./bpf-script-example.c sleep 1
>>
>> Note that '--clang-opt' must put before '--event'.
>>
>> Futher patches will merge it into a testcase so can be tested automatically.
>>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>> Signed-off-by: He Kuang <hekuang@huawei.com>
>> Acked-by: Alexei Starovoitov <ast@plumgrid.com>
>> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
>> Cc: Daniel Borkmann <daniel@iogearbox.net>
>> Cc: David Ahern <dsahern@gmail.com>
>> Cc: Jiri Olsa <jolsa@kernel.org>
>> Cc: Kaixu Xia <xiakaixu@huawei.com>
>> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>> Cc: Namhyung Kim <namhyung@kernel.org>
>> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
>> Cc: Zefan Li <lizefan@huawei.com>
>> Cc: pi3orama@163.com
>> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
>> ---
>>   tools/perf/tests/bpf-script-example.c | 44 +++++++++++++++++++++++++++++++++++
>>   tools/perf/util/bpf-loader.c          | 17 ++++++++++++--
>>   tools/perf/util/bpf-loader.h          |  5 ++--
>>   tools/perf/util/parse-events.c        |  5 ++--
>>   tools/perf/util/parse-events.h        |  3 ++-
>>   tools/perf/util/parse-events.l        |  3 +++
>>   tools/perf/util/parse-events.y        | 15 ++++++++++--
>>   7 files changed, 83 insertions(+), 9 deletions(-)
>>   create mode 100644 tools/perf/tests/bpf-script-example.c
>>
>> diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
>> new file mode 100644
>> index 0000000..410a70b
>> --- /dev/null
>> +++ b/tools/perf/tests/bpf-script-example.c
> Shouldn't it be a part of the next patch?

I think putting the sample file in this patch should be better.

In commit message I show a cmdline to utilize scriptlet compiling. If
we put the sample code into next patch, then people get to this point
have to write his/her own script to test scriptlet compiling manually.

Thank you.

> Thanks,
> Namhyung
>
>
>> @@ -0,0 +1,44 @@
>> +#ifndef LINUX_VERSION_CODE
>> +# error Need LINUX_VERSION_CODE
>> +# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
>> +#endif
>> +#define BPF_ANY 0
>> +#define BPF_MAP_TYPE_ARRAY 2
>> +#define BPF_FUNC_map_lookup_elem 1
>> +#define BPF_FUNC_map_update_elem 2
>> +
>> +static void *(*bpf_map_lookup_elem)(void *map, void *key) =
>> +	(void *) BPF_FUNC_map_lookup_elem;
>> +static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
>> +	(void *) BPF_FUNC_map_update_elem;
>> +
>> +struct bpf_map_def {
>> +	unsigned int type;
>> +	unsigned int key_size;
>> +	unsigned int value_size;
>> +	unsigned int max_entries;
>> +};
>> +
>> +#define SEC(NAME) __attribute__((section(NAME), used))
>> +struct bpf_map_def SEC("maps") flip_table = {
>> +	.type = BPF_MAP_TYPE_ARRAY,
>> +	.key_size = sizeof(int),
>> +	.value_size = sizeof(int),
>> +	.max_entries = 1,
>> +};
>> +
>> +SEC("func=sys_epoll_pwait")
>> +int bpf_func__sys_epoll_pwait(void *ctx)
>> +{
>> +	int ind =0;
>> +	int *flag = bpf_map_lookup_elem(&flip_table, &ind);
>> +	int new_flag;
>> +	if (!flag)
>> +		return 0;
>> +	/* flip flag and store back */
>> +	new_flag = !*flag;
>> +	bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
>> +	return new_flag;
>> +}
>> +char _license[] SEC("license") = "GPL";
>> +int _version SEC("version") = LINUX_VERSION_CODE;
>> diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
>> index aa784a4..ba6f752 100644
>> --- a/tools/perf/util/bpf-loader.c
>> +++ b/tools/perf/util/bpf-loader.c
>> @@ -12,6 +12,7 @@
>>   #include "bpf-loader.h"
>>   #include "probe-event.h"
>>   #include "probe-finder.h" // for MAX_PROBES
>> +#include "llvm-utils.h"
>>   
>>   #define DEFINE_PRINT_FN(name, level) \
>>   static int libbpf_##name(const char *fmt, ...)	\
>> @@ -33,7 +34,7 @@ struct bpf_prog_priv {
>>   	struct perf_probe_event pev;
>>   };
>>   
>> -struct bpf_object *bpf__prepare_load(const char *filename)
>> +struct bpf_object *bpf__prepare_load(const char *filename, bool source)
>>   {
>>   	struct bpf_object *obj;
>>   	static bool libbpf_initialized;
>> @@ -45,7 +46,19 @@ struct bpf_object *bpf__prepare_load(const char *filename)
>>   		libbpf_initialized = true;
>>   	}
>>   
>> -	obj = bpf_object__open(filename);
>> +	if (source) {
>> +		int err;
>> +		void *obj_buf;
>> +		size_t obj_buf_sz;
>> +
>> +		err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
>> +		if (err)
>> +			return ERR_PTR(err);
>> +		obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, filename);
>> +		free(obj_buf);
>> +	} else
>> +		obj = bpf_object__open(filename);
>> +
>>   	if (!obj) {
>>   		pr_debug("bpf: failed to load %s\n", filename);
>>   		return ERR_PTR(-EINVAL);
>> diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
>> index a8f25ee..ccd8d7f 100644
>> --- a/tools/perf/util/bpf-loader.h
>> +++ b/tools/perf/util/bpf-loader.h
>> @@ -18,7 +18,7 @@ typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
>>   					int fd, void *arg);
>>   
>>   #ifdef HAVE_LIBBPF_SUPPORT
>> -struct bpf_object *bpf__prepare_load(const char *filename);
>> +struct bpf_object *bpf__prepare_load(const char *filename, bool source);
>>   
>>   void bpf__clear(void);
>>   
>> @@ -34,7 +34,8 @@ int bpf__foreach_tev(struct bpf_object *obj,
>>   		     bpf_prog_iter_callback_t func, void *arg);
>>   #else
>>   static inline struct bpf_object *
>> -bpf__prepare_load(const char *filename __maybe_unused)
>> +bpf__prepare_load(const char *filename __maybe_unused,
>> +		  bool source __maybe_unused)
>>   {
>>   	pr_debug("ERROR: eBPF object loading is disabled during compiling.\n");
>>   	return ERR_PTR(-ENOTSUP);
>> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
>> index 5087023..4849dbd 100644
>> --- a/tools/perf/util/parse-events.c
>> +++ b/tools/perf/util/parse-events.c
>> @@ -626,11 +626,12 @@ errout:
>>   
>>   int parse_events_load_bpf(struct parse_events_evlist *data,
>>   			  struct list_head *list,
>> -			  char *bpf_file_name)
>> +			  char *bpf_file_name,
>> +			  bool source)
>>   {
>>   	struct bpf_object *obj;
>>   
>> -	obj = bpf__prepare_load(bpf_file_name);
>> +	obj = bpf__prepare_load(bpf_file_name, source);
>>   	if (IS_ERR(obj) || !obj) {
>>   		char errbuf[BUFSIZ];
>>   		int err;
>> diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
>> index fbb16c7..8f17c83 100644
>> --- a/tools/perf/util/parse-events.h
>> +++ b/tools/perf/util/parse-events.h
>> @@ -123,7 +123,8 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
>>   				struct list_head *head_config);
>>   int parse_events_load_bpf(struct parse_events_evlist *data,
>>   			  struct list_head *list,
>> -			  char *bpf_file_name);
>> +			  char *bpf_file_name,
>> +			  bool source);
>>   /* Provide this function for perf test */
>>   struct bpf_object;
>>   int parse_events_load_bpf_obj(struct parse_events_evlist *data,
>> diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
>> index 5e5d31a..eeea4e1 100644
>> --- a/tools/perf/util/parse-events.l
>> +++ b/tools/perf/util/parse-events.l
>> @@ -116,6 +116,7 @@ group		[^,{}/]*[{][^}]*[}][^,{}/]*
>>   event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
>>   event		[^,{}/]+
>>   bpf_object	.*\.(o|bpf)
>> +bpf_source	.*\.c
>>   
>>   num_dec		[0-9]+
>>   num_hex		0x[a-fA-F0-9]+
>> @@ -161,6 +162,7 @@ modifier_bp	[rwx]{1,3}
>>   
>>   {event_pmu}	|
>>   {bpf_object}	|
>> +{bpf_source}	|
>>   {event}		{
>>   			BEGIN(INITIAL);
>>   			REWIND(1);
>> @@ -267,6 +269,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
>>   
>>   {modifier_event}	{ return str(yyscanner, PE_MODIFIER_EVENT); }
>>   {bpf_object}		{ return str(yyscanner, PE_BPF_OBJECT); }
>> +{bpf_source}		{ return str(yyscanner, PE_BPF_SOURCE); }
>>   {name}			{ return pmu_str_check(yyscanner); }
>>   "/"			{ BEGIN(config); return '/'; }
>>   -			{ return '-'; }
>> diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
>> index 497f19b..ad37996 100644
>> --- a/tools/perf/util/parse-events.y
>> +++ b/tools/perf/util/parse-events.y
>> @@ -42,7 +42,7 @@ static inc_group_count(struct list_head *list,
>>   %token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
>>   %token PE_EVENT_NAME
>>   %token PE_NAME
>> -%token PE_BPF_OBJECT
>> +%token PE_BPF_OBJECT PE_BPF_SOURCE
>>   %token PE_MODIFIER_EVENT PE_MODIFIER_BP
>>   %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
>>   %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
>> @@ -55,6 +55,7 @@ static inc_group_count(struct list_head *list,
>>   %type <num> PE_TERM
>>   %type <str> PE_NAME
>>   %type <str> PE_BPF_OBJECT
>> +%type <str> PE_BPF_SOURCE
>>   %type <str> PE_NAME_CACHE_TYPE
>>   %type <str> PE_NAME_CACHE_OP_RESULT
>>   %type <str> PE_MODIFIER_EVENT
>> @@ -461,7 +462,17 @@ PE_BPF_OBJECT
>>   	struct list_head *list;
>>   
>>   	ALLOC_LIST(list);
>> -	ABORT_ON(parse_events_load_bpf(data, list, $1));
>> +	ABORT_ON(parse_events_load_bpf(data, list, $1, false));
>> +	$$ = list;
>> +}
>> +|
>> +PE_BPF_SOURCE
>> +{
>> +	struct parse_events_evlist *data = _data;
>> +	struct list_head *list;
>> +
>> +	ALLOC_LIST(list);
>> +	ABORT_ON(parse_events_load_bpf(data, list, $1, true));
>>   	$$ = list;
>>   }
>>   
>> -- 
>> 1.8.3.4
>>

Re: [PATCH 16/31] perf tools: Add prologue for BPF programs for fetching arguments

[Namhyung Kim]

On Wed, Oct 14, 2015 at 12:41:27PM +0000, Wang Nan wrote:
> From: He Kuang <hekuang@huawei.com>
> 
> This patch generates prologue for a BPF program which fetch arguments
> for it. With this patch, the program can have arguments as follow:
> 
>  SEC("lock_page=__lock_page page->flags")
>  int lock_page(struct pt_regs *ctx, int err, unsigned long flags)
>  {
> 	 return 1;
>  }
> 
> This patch passes at most 3 arguments from r3, r4 and r5. r1 is still
> the ctx pointer. r2 is used to indicate the successfulness of
> dereferencing.
> 
> This patch uses r6 to hold ctx (struct pt_regs) and r7 to hold stack
> pointer for result. Result of each arguments first store on stack:
> 
>  low address
>  BPF_REG_FP - 24  ARG3
>  BPF_REG_FP - 16  ARG2
>  BPF_REG_FP - 8   ARG1
>  BPF_REG_FP
>  high address
> 
> Then loaded into r3, r4 and r5.
> 
> The output prologue for offn(...off2(off1(reg)))) should be:
> 
>      r6 <- r1			// save ctx into a callee saved register
>      r7 <- fp
>      r7 <- r7 - stack_offset	// pointer to result slot
>      /* load r3 with the offset in pt_regs of 'reg' */
>      (r7) <- r3			// make slot valid
>      r3 <- r3 + off1		// prepare to read unsafe pointer
>      r2 <- 8
>      r1 <- r7			// result put onto stack
>      call probe_read		// read unsafe pointer
>      jnei r0, 0, err		// error checking
>      r3 <- (r7)			// read result
>      r3 <- r3 + off2		// prepare to read unsafe pointer
>      r2 <- 8
>      r1 <- r7
>      call probe_read
>      jnei r0, 0, err
>      ...
>      /* load r2, r3, r4 from stack */
>      goto success
> err:
>      r2 <- 1
>      /* load r3, r4, r5 with 0 */
>      goto usercode
> success:
>      r2 <- 0
> usercode:
>      r1 <- r6	// restore ctx
>      // original user code
> 
> If all of arguments reside in register (dereferencing is not
> required), gen_prologue_fastpath() will be used to create
> fast prologue:
> 
>      r3 <- (r1 + offset of reg1)
>      r4 <- (r1 + offset of reg2)
>      r5 <- (r1 + offset of reg3)
>      r2 <- 0
> 
> P.S.
> 
> eBPF calling convention is defined as:
> 
> * r0		- return value from in-kernel function, and exit value
>                   for eBPF program
> * r1 - r5	- arguments from eBPF program to in-kernel function
> * r6 - r9	- callee saved registers that in-kernel function will
>                   preserve
> * r10		- read-only frame pointer to access stack
> 
> Signed-off-by: He Kuang <hekuang@huawei.com>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---

[SNIP]
> +int bpf__gen_prologue(struct probe_trace_arg *args, int nargs,
> +		      struct bpf_insn *new_prog, size_t *new_cnt,
> +		      size_t cnt_space)
> +{
> +	struct bpf_insn *success_code = NULL;
> +	struct bpf_insn *error_code = NULL;
> +	struct bpf_insn *user_code = NULL;
> +	struct bpf_insn_pos pos;
> +	bool fastpath = true;
> +	int i;
> +
> +	if (!new_prog || !new_cnt)
> +		return -EINVAL;
> +
> +	pos.begin = new_prog;
> +	pos.end = new_prog + cnt_space;
> +	pos.pos = new_prog;
> +
> +	if (!nargs) {
> +		ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 0),
> +		    &pos);
> +
> +		if (check_pos(&pos))
> +			goto errout;
> +
> +		*new_cnt = pos_get_cnt(&pos);
> +		return 0;
> +	}
> +
> +	if (nargs > BPF_PROLOGUE_MAX_ARGS)
> +		nargs = BPF_PROLOGUE_MAX_ARGS;

Wouldn't it be better to inform user if it ignored some arguments?

Thanks,
Namhyung


> +	if (cnt_space > BPF_MAXINSNS)
> +		cnt_space = BPF_MAXINSNS;
> +
> +	/* First pass: validation */
> +	for (i = 0; i < nargs; i++) {
> +		struct probe_trace_arg_ref *ref = args[i].ref;
> +
> +		if (args[i].value[0] == '@') {
> +			/* TODO: fetch global variable */
> +			pr_err("bpf: prologue: global %s%+ld not support\n",
> +				args[i].value, ref ? ref->offset : 0);
> +			return -ENOTSUP;
> +		}
> +
> +		while (ref) {
> +			/* fastpath is true if all args has ref == NULL */
> +			fastpath = false;
> +
> +			/*
> +			 * Instruction encodes immediate value using
> +			 * s32, ref->offset is long. On systems which
> +			 * can't fill long in s32, refuse to process if
> +			 * ref->offset too large (or small).
> +			 */
> +#ifdef __LP64__
> +#define OFFSET_MAX	((1LL << 31) - 1)
> +#define OFFSET_MIN	((1LL << 31) * -1)
> +			if (ref->offset > OFFSET_MAX ||
> +					ref->offset < OFFSET_MIN) {
> +				pr_err("bpf: prologue: offset out of bound: %ld\n",
> +				       ref->offset);
> +				return -E2BIG;
> +			}
> +#endif
> +			ref = ref->next;
> +		}
> +	}
> +	pr_debug("prologue: pass validation\n");
> +
> +	if (fastpath) {
> +		/* If all variables are registers... */
> +		pr_debug("prologue: fast path\n");
> +		if (gen_prologue_fastpath(&pos, args, nargs))
> +			goto errout;
> +	} else {
> +		pr_debug("prologue: slow path\n");
> +
> +		/* Initialization: move ctx to a callee saved register. */
> +		ins(BPF_MOV64_REG(BPF_REG_CTX, BPF_REG_ARG1), &pos);
> +
> +		if (gen_prologue_slowpath(&pos, args, nargs))
> +			goto errout;
> +		/*
> +		 * start of ERROR_CODE (only slow pass needs error code)
> +		 *   mov r2 <- 1
> +		 *   goto usercode
> +		 */
> +		error_code = pos.pos;
> +		ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 1),
> +		    &pos);
> +
> +		for (i = 0; i < nargs; i++)
> +			ins(BPF_ALU64_IMM(BPF_MOV,
> +					  BPF_PROLOGUE_START_ARG_REG + i,
> +					  0),
> +			    &pos);
> +		ins(BPF_JMP_IMM(BPF_JA, BPF_REG_0, 0, JMP_TO_USER_CODE),
> +				&pos);
> +	}
> +
> +	/*
> +	 * start of SUCCESS_CODE:
> +	 *   mov r2 <- 0
> +	 *   goto usercode  // skip
> +	 */
> +	success_code = pos.pos;
> +	ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 0), &pos);
> +
> +	/*
> +	 * start of USER_CODE:
> +	 *   Restore ctx to r1
> +	 */
> +	user_code = pos.pos;
> +	if (!fastpath) {
> +		/*
> +		 * Only slow path needs restoring of ctx. In fast path,
> +		 * register are loaded directly from r1.
> +		 */
> +		ins(BPF_MOV64_REG(BPF_REG_ARG1, BPF_REG_CTX), &pos);
> +		if (prologue_relocate(&pos, error_code, success_code,
> +				      user_code))
> +			goto errout;
> +	}
> +
> +	if (check_pos(&pos))
> +		goto errout;
> +
> +	*new_cnt = pos_get_cnt(&pos);
> +	return 0;
> +errout:
> +	return -ERANGE;
> +}
> diff --git a/tools/perf/util/bpf-prologue.h b/tools/perf/util/bpf-prologue.h
> new file mode 100644
> index 0000000..f1e4c5d
> --- /dev/null
> +++ b/tools/perf/util/bpf-prologue.h
> @@ -0,0 +1,34 @@
> +/*
> + * Copyright (C) 2015, He Kuang <hekuang@huawei.com>
> + * Copyright (C) 2015, Huawei Inc.
> + */
> +#ifndef __BPF_PROLOGUE_H
> +#define __BPF_PROLOGUE_H
> +
> +#include <linux/compiler.h>
> +#include <linux/filter.h>
> +#include "probe-event.h"
> +
> +#define BPF_PROLOGUE_MAX_ARGS 3
> +#define BPF_PROLOGUE_START_ARG_REG BPF_REG_3
> +#define BPF_PROLOGUE_FETCH_RESULT_REG BPF_REG_2
> +
> +#ifdef HAVE_BPF_PROLOGUE
> +int bpf__gen_prologue(struct probe_trace_arg *args, int nargs,
> +		      struct bpf_insn *new_prog, size_t *new_cnt,
> +		      size_t cnt_space);
> +#else
> +static inline int
> +bpf__gen_prologue(struct probe_trace_arg *args __maybe_unused,
> +		  int nargs __maybe_unused,
> +		  struct bpf_insn *new_prog __maybe_unused,
> +		  size_t *new_cnt,
> +		  size_t cnt_space __maybe_unused)
> +{
> +	if (!new_cnt)
> +		return -EINVAL;
> +	*new_cnt = 0;
> +	return 0;
> +}
> +#endif
> +#endif /* __BPF_PROLOGUE_H */
> -- 
> 1.8.3.4
> 

Re: [PATCH 16/31] perf tools: Add prologue for BPF programs for fetching arguments

[Wangnan (F)]

On 2015/10/15 13:26, Namhyung Kim wrote:
> On Wed, Oct 14, 2015 at 12:41:27PM +0000, Wang Nan wrote:
>> From: He Kuang <hekuang@huawei.com>
>>
>> This patch generates prologue for a BPF program which fetch arguments
>> for it. With this patch, the program can have arguments as follow:
>>
>>   SEC("lock_page=__lock_page page->flags")
>>   int lock_page(struct pt_regs *ctx, int err, unsigned long flags)
>>   {
>> 	 return 1;
>>   }
>>
>> This patch passes at most 3 arguments from r3, r4 and r5. r1 is still
>> the ctx pointer. r2 is used to indicate the successfulness of
>> dereferencing.
>>
>> This patch uses r6 to hold ctx (struct pt_regs) and r7 to hold stack
>> pointer for result. Result of each arguments first store on stack:
>>
>>   low address
>>   BPF_REG_FP - 24  ARG3
>>   BPF_REG_FP - 16  ARG2
>>   BPF_REG_FP - 8   ARG1
>>   BPF_REG_FP
>>   high address
>>
>> Then loaded into r3, r4 and r5.
>>
>> The output prologue for offn(...off2(off1(reg)))) should be:
>>
>>       r6 <- r1			// save ctx into a callee saved register
>>       r7 <- fp
>>       r7 <- r7 - stack_offset	// pointer to result slot
>>       /* load r3 with the offset in pt_regs of 'reg' */
>>       (r7) <- r3			// make slot valid
>>       r3 <- r3 + off1		// prepare to read unsafe pointer
>>       r2 <- 8
>>       r1 <- r7			// result put onto stack
>>       call probe_read		// read unsafe pointer
>>       jnei r0, 0, err		// error checking
>>       r3 <- (r7)			// read result
>>       r3 <- r3 + off2		// prepare to read unsafe pointer
>>       r2 <- 8
>>       r1 <- r7
>>       call probe_read
>>       jnei r0, 0, err
>>       ...
>>       /* load r2, r3, r4 from stack */
>>       goto success
>> err:
>>       r2 <- 1
>>       /* load r3, r4, r5 with 0 */
>>       goto usercode
>> success:
>>       r2 <- 0
>> usercode:
>>       r1 <- r6	// restore ctx
>>       // original user code
>>
>> If all of arguments reside in register (dereferencing is not
>> required), gen_prologue_fastpath() will be used to create
>> fast prologue:
>>
>>       r3 <- (r1 + offset of reg1)
>>       r4 <- (r1 + offset of reg2)
>>       r5 <- (r1 + offset of reg3)
>>       r2 <- 0
>>
>> P.S.
>>
>> eBPF calling convention is defined as:
>>
>> * r0		- return value from in-kernel function, and exit value
>>                    for eBPF program
>> * r1 - r5	- arguments from eBPF program to in-kernel function
>> * r6 - r9	- callee saved registers that in-kernel function will
>>                    preserve
>> * r10		- read-only frame pointer to access stack
>>
>> Signed-off-by: He Kuang <hekuang@huawei.com>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>> Cc: Alexei Starovoitov <ast@plumgrid.com>
>> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
>> Cc: Daniel Borkmann <daniel@iogearbox.net>
>> Cc: David Ahern <dsahern@gmail.com>
>> Cc: He Kuang <hekuang@huawei.com>
>> Cc: Jiri Olsa <jolsa@kernel.org>
>> Cc: Kaixu Xia <xiakaixu@huawei.com>
>> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>> Cc: Namhyung Kim <namhyung@kernel.org>
>> Cc: Paul Mackerras <paulus@samba.org>
>> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
>> Cc: Zefan Li <lizefan@huawei.com>
>> Cc: pi3orama@163.com
>> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
>> ---
> [SNIP]
>> +int bpf__gen_prologue(struct probe_trace_arg *args, int nargs,
>> +		      struct bpf_insn *new_prog, size_t *new_cnt,
>> +		      size_t cnt_space)
>> +{
>> +	struct bpf_insn *success_code = NULL;
>> +	struct bpf_insn *error_code = NULL;
>> +	struct bpf_insn *user_code = NULL;
>> +	struct bpf_insn_pos pos;
>> +	bool fastpath = true;
>> +	int i;
>> +
>> +	if (!new_prog || !new_cnt)
>> +		return -EINVAL;
>> +
>> +	pos.begin = new_prog;
>> +	pos.end = new_prog + cnt_space;
>> +	pos.pos = new_prog;
>> +
>> +	if (!nargs) {
>> +		ins(BPF_ALU64_IMM(BPF_MOV, BPF_PROLOGUE_FETCH_RESULT_REG, 0),
>> +		    &pos);
>> +
>> +		if (check_pos(&pos))
>> +			goto errout;
>> +
>> +		*new_cnt = pos_get_cnt(&pos);
>> +		return 0;
>> +	}
>> +
>> +	if (nargs > BPF_PROLOGUE_MAX_ARGS)
>> +		nargs = BPF_PROLOGUE_MAX_ARGS;
> Wouldn't it be better to inform user if it ignored some arguments?

Correct. I'd like to add a notification in next version:

diff --git a/tools/perf/util/bpf-prologue.c b/tools/perf/util/bpf-prologue.c
index e4adb18..36093d9 100644
--- a/tools/perf/util/bpf-prologue.c
+++ b/tools/perf/util/bpf-prologue.c
@@ -337,8 +337,10 @@ int bpf__gen_prologue(struct probe_trace_arg *args, 
int nargs,
                 return 0;
         }

-       if (nargs > BPF_PROLOGUE_MAX_ARGS)
+       if (nargs > BPF_PROLOGUE_MAX_ARGS) {
+               pr_warning("bpf: prologue: too many arguments\n");
                 nargs = BPF_PROLOGUE_MAX_ARGS;
+       }
         if (cnt_space > BPF_MAXINSNS)
                 cnt_space = BPF_MAXINSNS;


Thank you.


> Thanks,
> Namhyung

Re: [PATCH 10/31] perf test: Enforce LLVM test for BPF test

[Wangnan (F)]

On 2015/10/14 23:48, Namhyung Kim wrote:
> On Wed, Oct 14, 2015 at 12:41:21PM +0000, Wang Nan wrote:
>> This patch replaces the original toy BPF program with previous introduced
>> bpf-script-example.c. Dynamically embedded it into 'llvm-src.c'.
>>
>> The newly introduced BPF program attaches a BPF program at
>> 'sys_epoll_pwait()', and collect half samples from it. perf itself never
>> use that syscall, so further test can verify their result with it.
>>
>> Since BPF program require LINUX_VERSION_CODE of runtime kernel, this
>> patch computes that code from uname.
>>
>> Since the resuling BPF object is useful for further testcases, this patch
>> introduces 'prepare' and 'cleanup' method to tests, and makes test__llvm()
>> create a MAP_SHARED memory array to hold the resulting object.
>>
>> Signed-off-by: He Kuang <hekuang@huawei.com>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Cc: Alexei Starovoitov <ast@plumgrid.com>
>> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
>> Cc: Daniel Borkmann <daniel@iogearbox.net>
>> Cc: David Ahern <dsahern@gmail.com>
>> Cc: Jiri Olsa <jolsa@kernel.org>
>> Cc: Kaixu Xia <xiakaixu@huawei.com>
>> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>> Cc: Namhyung Kim <namhyung@kernel.org>
>> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
>> Cc: Zefan Li <lizefan@huawei.com>
>> Cc: pi3orama@163.com
>> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
>> ---
> [SNIP]
>
>> +void test__llvm_prepare(void)
>> +{
>> +	p_test_llvm__bpf_result = mmap(NULL, SHARED_BUF_INIT_SIZE,
>> +				       PROT_READ | PROT_WRITE,
>> +				       MAP_SHARED | MAP_ANONYMOUS, -1, 0);
>> +	if (!p_test_llvm__bpf_result)
> It should check MAP_FAILED instead.
>

Fixed by this way:

diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
index e722e8a..25ddeaf 100644
--- a/tools/perf/tests/llvm.c
+++ b/tools/perf/tests/llvm.c
@@ -199,12 +199,15 @@ void test__llvm_prepare(void)

         for (i = 0; llvm_testcases[i].source; i++) {
                 struct test_llvm__bpf_result *result;
+               void *p;

-               result = mmap(NULL, SHARED_BUF_INIT_SIZE,
-                             PROT_READ | PROT_WRITE,
-                             MAP_SHARED | MAP_ANONYMOUS, -1, 0);
-               if (!result)
+               p = mmap(NULL, SHARED_BUF_INIT_SIZE,
+                        PROT_READ | PROT_WRITE,
+                        MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+               if (p == MAP_FAILED)
                         return;
+
+               result = p;
                 memset((void *)result, '\0', SHARED_BUF_INIT_SIZE);

                 llvm_testcases[i].result = result;

Thank you.

Re: [PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Namhyung Kim]

Hi,

On Wed, Oct 14, 2015 at 12:41:32PM +0000, Wang Nan wrote:
> This patch enforces existing LLVM test, makes it compile more than one
> BPF source file. The compiled results are stored, can be used for other
> testcases. Except the first testcase (named LLVM_TESTCASE_BASE), failures
> of other test cases are not considered as failure of the whole test.
> 
> Adds a kbuild testcase to check whether kernel headers can be correctly
> found.
> 
> For example:
> 
>  # perf test LLVM
> 
>    38: Test LLVM searching and compiling                        : (llvm.kbuild-dir can be fixed) Ok

IMHO it'd be better to keep the test result simply as either "Ok" or
"FAILED" and provide details with -v option.

Anyway I found the tracepoint error message is annoying in the current
perf test output.  Will send a fix soon.

Thanks,
Namhyung

Re: [PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Arnaldo Carvalho de Melo]

Em Mon, Oct 19, 2015 at 11:42:31PM +0900, Namhyung Kim escreveu:
> Hi,
> 
> On Wed, Oct 14, 2015 at 12:41:32PM +0000, Wang Nan wrote:
> > This patch enforces existing LLVM test, makes it compile more than one
> > BPF source file. The compiled results are stored, can be used for other
> > testcases. Except the first testcase (named LLVM_TESTCASE_BASE), failures
> > of other test cases are not considered as failure of the whole test.
> > 
> > Adds a kbuild testcase to check whether kernel headers can be correctly
> > found.
> > 
> > For example:
> > 
> >  # perf test LLVM
> > 
> >    38: Test LLVM searching and compiling                        : (llvm.kbuild-dir can be fixed) Ok
> 
> IMHO it'd be better to keep the test result simply as either "Ok" or
> "FAILED" and provide details with -v option.

"Skipped" looks more natural here, with the reason for it skipping being shown
only with -v.

Ingo made some comments about 'perf test' output recently, have you read that?

> Anyway I found the tracepoint error message is annoying in the current

You mean all these warnings:

[root@zoo ~]# perf test "parse event" 
 5: parse events tests                                       :  Warning: [sunrpc:xprt_lookup_rqst] function __builtin_constant_p not defined
  Warning: [sunrpc:xprt_transmit] function __builtin_constant_p not defined
  Warning: [sunrpc:xprt_complete_rqst] function __builtin_constant_p not defined
<SNIP>
  Warning: [xen:xen_mmu_set_pud] function sizeof not defined
  Warning: [xen:xen_mmu_set_pgd] function sizeof not defined
  Warning: [xen:xen_mmu_ptep_modify_prot_start] function sizeof not defined
  Warning: [xen:xen_mmu_ptep_modify_prot_commit] function sizeof not defined
 Ok
[root@zoo ~]# 

i.e. making those warnings appear only under 'perf test -v'? Cool!

> perf test output.  Will send a fix soon.
> 
> Thanks,
> Namhyung

Re: [PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Namhyung Kim]

Hi Arnaldo,

On Mon, Oct 19, 2015 at 12:53:12PM -0200, Arnaldo Carvalho de Melo wrote:
> Em Mon, Oct 19, 2015 at 11:42:31PM +0900, Namhyung Kim escreveu:
> > Hi,
> > 
> > On Wed, Oct 14, 2015 at 12:41:32PM +0000, Wang Nan wrote:
> > > This patch enforces existing LLVM test, makes it compile more than one
> > > BPF source file. The compiled results are stored, can be used for other
> > > testcases. Except the first testcase (named LLVM_TESTCASE_BASE), failures
> > > of other test cases are not considered as failure of the whole test.
> > > 
> > > Adds a kbuild testcase to check whether kernel headers can be correctly
> > > found.
> > > 
> > > For example:
> > > 
> > >  # perf test LLVM
> > > 
> > >    38: Test LLVM searching and compiling                        : (llvm.kbuild-dir can be fixed) Ok
> > 
> > IMHO it'd be better to keep the test result simply as either "Ok" or
> > "FAILED" and provide details with -v option.
> 
> "Skipped" looks more natural here, with the reason for it skipping being shown
> only with -v.

Agreed.

> 
> Ingo made some comments about 'perf test' output recently, have you read that?

Nop, sorry.  I'll check it (and it'd be great it you could give me the
link :)


> 
> > Anyway I found the tracepoint error message is annoying in the current
> 
> You mean all these warnings:
> 
> [root@zoo ~]# perf test "parse event" 
>  5: parse events tests                                       :  Warning: [sunrpc:xprt_lookup_rqst] function __builtin_constant_p not defined
>   Warning: [sunrpc:xprt_transmit] function __builtin_constant_p not defined
>   Warning: [sunrpc:xprt_complete_rqst] function __builtin_constant_p not defined
> <SNIP>
>   Warning: [xen:xen_mmu_set_pud] function sizeof not defined
>   Warning: [xen:xen_mmu_set_pgd] function sizeof not defined
>   Warning: [xen:xen_mmu_ptep_modify_prot_start] function sizeof not defined
>   Warning: [xen:xen_mmu_ptep_modify_prot_commit] function sizeof not defined
>  Ok
> [root@zoo ~]# 
> 
> i.e. making those warnings appear only under 'perf test -v'? Cool!

Will send the fix soon!

Thanks,
Namhyung

Re: [PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Wangnan (F)]

On 2015/10/19 22:53, Arnaldo Carvalho de Melo wrote:
> Em Mon, Oct 19, 2015 at 11:42:31PM +0900, Namhyung Kim escreveu:
>> Hi,
>>
>> On Wed, Oct 14, 2015 at 12:41:32PM +0000, Wang Nan wrote:
>>> This patch enforces existing LLVM test, makes it compile more than one
>>> BPF source file. The compiled results are stored, can be used for other
>>> testcases. Except the first testcase (named LLVM_TESTCASE_BASE), failures
>>> of other test cases are not considered as failure of the whole test.
>>>
>>> Adds a kbuild testcase to check whether kernel headers can be correctly
>>> found.
>>>
>>> For example:
>>>
>>>   # perf test LLVM
>>>
>>>     38: Test LLVM searching and compiling                        : (llvm.kbuild-dir can be fixed) Ok
>> IMHO it'd be better to keep the test result simply as either "Ok" or
>> "FAILED" and provide details with -v option.

I have to say I'm not the first one to output things like this:

  #perf test dummy attr
  14: struct perf_event_attr setup                             : 
(omitted) Ok
  23: Test using a dummy software event to keep tracking       : (not 
supported) Ok

Do you think they also need to be fixed?

> "Skipped" looks more natural here, with the reason for it skipping being shown
> only with -v.

For this specific case, I will change the logic:

  1. When basic test failed (no clang, compiling error...), report failure;

  2. When basic test passed but kbuild test or later test cases failed, 
report
     Skip.

  3. Don't output any other information.

Thank you.

> Ingo made some comments about 'perf test' output recently, have you read that?
>
>> Anyway I found the tracepoint error message is annoying in the current
> You mean all these warnings:
>
> [root@zoo ~]# perf test "parse event"
>   5: parse events tests                                       :  Warning: [sunrpc:xprt_lookup_rqst] function __builtin_constant_p not defined
>    Warning: [sunrpc:xprt_transmit] function __builtin_constant_p not defined
>    Warning: [sunrpc:xprt_complete_rqst] function __builtin_constant_p not defined
> <SNIP>
>    Warning: [xen:xen_mmu_set_pud] function sizeof not defined
>    Warning: [xen:xen_mmu_set_pgd] function sizeof not defined
>    Warning: [xen:xen_mmu_ptep_modify_prot_start] function sizeof not defined
>    Warning: [xen:xen_mmu_ptep_modify_prot_commit] function sizeof not defined
>   Ok
> [root@zoo ~]#
>
> i.e. making those warnings appear only under 'perf test -v'? Cool!
>
>> perf test output.  Will send a fix soon.
>>
>> Thanks,
>> Namhyung

Re: [PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Wangnan (F)]

Hi Namhyung and Arnaldo,

I changed my testing related patches in my local git tree. All
changed combined together is at the end of this mail. I remove
all stderr output, and improve debug message. Please have a look
at it.

Thank you.


diff --git a/tools/perf/tests/bpf.c b/tools/perf/tests/bpf.c
index 5a6290a..453eff8 100644
--- a/tools/perf/tests/bpf.c
+++ b/tools/perf/tests/bpf.c
@@ -49,7 +49,7 @@ static struct bpf_object *prepare_bpf(const char 
*name, void *obj_buf,

      obj = bpf__prepare_load_buffer(obj_buf, obj_buf_sz, name);
      if (IS_ERR(obj)) {
-        fprintf(stderr, " (compile failed)");
+        pr_debug("Compile BPF program failed.\n");
          return NULL;
      }
      return obj;
@@ -82,7 +82,7 @@ static int do_test(struct bpf_object *obj, int 
(*func)(void), int expect)

      err = parse_events_load_bpf_obj(&parse_evlist, &parse_evlist.list, 
obj);
      if (err || list_empty(&parse_evlist.list)) {
-        fprintf(stderr, " (Failed to add events selected by BPF)");
+        pr_debug("Failed to add events selected by BPF\n");
          if (!err)
              err = -EINVAL;
          goto out;
@@ -140,7 +140,7 @@ static int do_test(struct bpf_object *obj, int 
(*func)(void), int expect)
      }

      if (count != expect) {
-        fprintf(stderr, " (filter result incorrect: %d != %d)", count, 
expect);
+        pr_debug("BPF filter result incorrect: %d != %d\n", count, expect);
          err = -EBADF;
      }

@@ -164,16 +164,14 @@ static int __test__bpf(int index, const char *name,

      test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz, index);
      if (!obj_buf || !obj_buf_sz) {
-        if (verbose == 0)
-            fprintf(stderr, " (%s)", message_compile);
+        pr_debug("Failed to compile: %s\n", message_compile);
          return TEST_SKIP;
      }

      obj = prepare_bpf(name, obj_buf, obj_buf_sz);
      if (!obj) {
          err = -EINVAL;
-        if ((verbose == 0) && (message_load[0] != '\0'))
-            fprintf(stderr, " (%s)", message_load);
+        pr_debug("Failed to load: %s", message_load);
          goto out;
      }

@@ -192,7 +190,7 @@ int test__bpf(void)
      int err;

      if (geteuid() != 0) {
-        fprintf(stderr, " (try run as root)");
+        pr_debug("Only root can run BPF test\n");
          return TEST_SKIP;
      }

@@ -214,7 +212,7 @@ int test__bpf(void)
                (NR_ITERS + 1) / 4);
      return err;
  #else
-    fprintf(stderr, " (skip BPF prologue test)");
+    pr_debug("BPF prologue is disabled when compiling, skip this test\n");
      return TEST_OK;
  #endif
  }
diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
index e722e8a..7c3b2c3 100644
--- a/tools/perf/tests/llvm.c
+++ b/tools/perf/tests/llvm.c
@@ -13,19 +13,23 @@
  struct llvm_testcase {
      const char *source;
      const char *errmsg;
+    const char *hintmsg;
      struct test_llvm__bpf_result *result;
      bool tried;
  } llvm_testcases[NR_LLVM_TESTCASES + 1] = {
      [LLVM_TESTCASE_BASE]    = {.source = test_llvm__bpf_prog,
                     .errmsg = "Basic LLVM compiling failed",
+                   .hintmsg = "Check llvm.clang-path option in 
~/.perfconfig",
                     .tried = false},
      [LLVM_TESTCASE_KBUILD]    = {.source = 
test_llvm__bpf_test_kbuild_prog,
-                   .errmsg = "llvm.kbuild-dir can be fixed",
+                   .errmsg = "Unable to find usable kbuild dir",
+                   .hintmsg = "Check llvm.kbuild-dir option in 
~/.perfconfig",
                     .tried = false},
      /* Don't output if this one fail. */
      [LLVM_TESTCASE_BPF_PROLOGUE]    = {
                     .source = test_llvm__bpf_test_prologue_prog,
-                   .errmsg = "failed for unknown reason",
+                   .errmsg = "Unable to compile BPF prologue testing 
program",
+                   .hintmsg = "This is an internal error, please report 
it",
                     .tried = false},
      {.source = NULL}
  };
@@ -43,16 +47,16 @@ static int test__bpf_parsing(void *obj_buf, size_t 
obj_buf_sz)

      obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, NULL);
      if (!obj)
-        return -1;
+        return TEST_FAIL;
      bpf_object__close(obj);
-    return 0;
+    return TEST_OK;
  }
  #else
  static int test__bpf_parsing(void *obj_buf __maybe_unused,
                   size_t obj_buf_sz __maybe_unused)
  {
-    fprintf(stderr, " (skip bpf parsing)");
-    return 0;
+    pr_debug("BPF support is not compiled, skip BPF parsing\n");
+    return TEST_SKIP;
  }
  #endif

@@ -70,8 +74,8 @@ compose_source(const char *raw_source)
      err = sscanf(utsname.release, "%d.%d.%d",
               &version, &patchlevel, &sublevel);
      if (err != 3) {
-        fprintf(stderr, " (Can't get kernel version from uname '%s')",
-            utsname.release);
+        pr_debug("Unablt to get kernel version from uname '%s'\n",
+             utsname.release);
          return NULL;
      }

@@ -104,7 +108,7 @@ static int __test__llvm(int i)
       * and clang is not found in $PATH, and this is not perf test -v
       */
      if (verbose == 0 && !llvm_param.user_set_param && 
llvm__search_clang()) {
-        fprintf(stderr, " (no clang, try 'perf test -v LLVM')");
+        pr_debug("Unable to find workable clang\n");
          return TEST_SKIP;
      }

@@ -155,7 +159,7 @@ static int __test__llvm(int i)

      verbose = old_verbose;
      if (err)
-        return -1;
+        return TEST_FAIL;

      err = test__bpf_parsing(obj_buf, obj_buf_sz);
      if (!err && result) {
@@ -181,13 +185,12 @@ int test__llvm(void)
               * First testcase tests basic LLVM compiling. If it
               * fails, no need to check others.
               */
-            if (!verbose)
-                fprintf(stderr, " (use -v to see error message)");
-            return ret;
+            return TEST_FAIL;
          } else if (ret) {
-            if (!verbose && llvm_testcases[i].errmsg)
-                fprintf(stderr, " (%s)", llvm_testcases[i].errmsg);
-            return 0;
+            pr_debug("%s\n", llvm_testcases[i].errmsg);
+            if (llvm_testcases[i].hintmsg)
+                pr_debug("Hint:\t%s\n", llvm_testcases[i].hintmsg);
+            return TEST_SKIP;
          }
      }
      return 0;
@@ -199,12 +202,15 @@ void test__llvm_prepare(void)

      for (i = 0; llvm_testcases[i].source; i++) {
          struct test_llvm__bpf_result *result;
+        void *p;

-        result = mmap(NULL, SHARED_BUF_INIT_SIZE,
-                  PROT_READ | PROT_WRITE,
-                  MAP_SHARED | MAP_ANONYMOUS, -1, 0);
-        if (!result)
+        p = mmap(NULL, SHARED_BUF_INIT_SIZE,
+             PROT_READ | PROT_WRITE,
+             MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+        if (p == MAP_FAILED)
              return;
+
+        result = p;
          memset((void *)result, '\0', SHARED_BUF_INIT_SIZE);

          llvm_testcases[i].result = result;
diff --git a/tools/perf/util/bpf-prologue.c b/tools/perf/util/bpf-prologue.c
index e4adb18..36093d9 100644
--- a/tools/perf/util/bpf-prologue.c
+++ b/tools/perf/util/bpf-prologue.c
@@ -337,8 +337,10 @@ int bpf__gen_prologue(struct probe_trace_arg *args, 
int nargs,
          return 0;
      }

-    if (nargs > BPF_PROLOGUE_MAX_ARGS)
+    if (nargs > BPF_PROLOGUE_MAX_ARGS) {
+        pr_warning("bpf: prologue: too many arguments\n");
          nargs = BPF_PROLOGUE_MAX_ARGS;
+    }
      if (cnt_space > BPF_MAXINSNS)
          cnt_space = BPF_MAXINSNS;

Re: [PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Arnaldo Carvalho de Melo]

Em Tue, Oct 20, 2015 at 08:06:54PM +0800, Wangnan (F) escreveu:
> Hi Namhyung and Arnaldo,
> 
> I changed my testing related patches in my local git tree. All
> changed combined together is at the end of this mail. I remove
> all stderr output, and improve debug message. Please have a look
> at it.
> 
> Thank you.

Yeah, from a quick look it looks sane, i.e. 'perf test' should state
just 'Ok', "FAIL" or "Skip", -v provides a reason, -vv will provide more
detail and so on, i.e. a verbosity level, for that we have 'pr_debug(),
pr_debug2(), pr_debug3(), etc, and can always, for some specialized use,
like grouping multiple warnings or deciding if we need to call some
other function only when the verbosity level is greater than some level,
check the 'verbose' variable value.

- Arnaldo
 
> 
> diff --git a/tools/perf/tests/bpf.c b/tools/perf/tests/bpf.c
> index 5a6290a..453eff8 100644
> --- a/tools/perf/tests/bpf.c
> +++ b/tools/perf/tests/bpf.c
> @@ -49,7 +49,7 @@ static struct bpf_object *prepare_bpf(const char
> *name, void *obj_buf,
> 
>      obj = bpf__prepare_load_buffer(obj_buf, obj_buf_sz, name);
>      if (IS_ERR(obj)) {
> -        fprintf(stderr, " (compile failed)");
> +        pr_debug("Compile BPF program failed.\n");
>          return NULL;
>      }
>      return obj;
> @@ -82,7 +82,7 @@ static int do_test(struct bpf_object *obj, int
> (*func)(void), int expect)
> 
>      err = parse_events_load_bpf_obj(&parse_evlist,
> &parse_evlist.list, obj);
>      if (err || list_empty(&parse_evlist.list)) {
> -        fprintf(stderr, " (Failed to add events selected by BPF)");
> +        pr_debug("Failed to add events selected by BPF\n");
>          if (!err)
>              err = -EINVAL;
>          goto out;
> @@ -140,7 +140,7 @@ static int do_test(struct bpf_object *obj, int
> (*func)(void), int expect)
>      }
> 
>      if (count != expect) {
> -        fprintf(stderr, " (filter result incorrect: %d != %d)",
> count, expect);
> +        pr_debug("BPF filter result incorrect: %d != %d\n", count, expect);
>          err = -EBADF;
>      }
> 
> @@ -164,16 +164,14 @@ static int __test__bpf(int index, const char *name,
> 
>      test_llvm__fetch_bpf_obj(&obj_buf, &obj_buf_sz, index);
>      if (!obj_buf || !obj_buf_sz) {
> -        if (verbose == 0)
> -            fprintf(stderr, " (%s)", message_compile);
> +        pr_debug("Failed to compile: %s\n", message_compile);
>          return TEST_SKIP;
>      }
> 
>      obj = prepare_bpf(name, obj_buf, obj_buf_sz);
>      if (!obj) {
>          err = -EINVAL;
> -        if ((verbose == 0) && (message_load[0] != '\0'))
> -            fprintf(stderr, " (%s)", message_load);
> +        pr_debug("Failed to load: %s", message_load);
>          goto out;
>      }
> 
> @@ -192,7 +190,7 @@ int test__bpf(void)
>      int err;
> 
>      if (geteuid() != 0) {
> -        fprintf(stderr, " (try run as root)");
> +        pr_debug("Only root can run BPF test\n");
>          return TEST_SKIP;
>      }
> 
> @@ -214,7 +212,7 @@ int test__bpf(void)
>                (NR_ITERS + 1) / 4);
>      return err;
>  #else
> -    fprintf(stderr, " (skip BPF prologue test)");
> +    pr_debug("BPF prologue is disabled when compiling, skip this test\n");
>      return TEST_OK;
>  #endif
>  }
> diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
> index e722e8a..7c3b2c3 100644
> --- a/tools/perf/tests/llvm.c
> +++ b/tools/perf/tests/llvm.c
> @@ -13,19 +13,23 @@
>  struct llvm_testcase {
>      const char *source;
>      const char *errmsg;
> +    const char *hintmsg;
>      struct test_llvm__bpf_result *result;
>      bool tried;
>  } llvm_testcases[NR_LLVM_TESTCASES + 1] = {
>      [LLVM_TESTCASE_BASE]    = {.source = test_llvm__bpf_prog,
>                     .errmsg = "Basic LLVM compiling failed",
> +                   .hintmsg = "Check llvm.clang-path option in
> ~/.perfconfig",
>                     .tried = false},
>      [LLVM_TESTCASE_KBUILD]    = {.source =
> test_llvm__bpf_test_kbuild_prog,
> -                   .errmsg = "llvm.kbuild-dir can be fixed",
> +                   .errmsg = "Unable to find usable kbuild dir",
> +                   .hintmsg = "Check llvm.kbuild-dir option in
> ~/.perfconfig",
>                     .tried = false},
>      /* Don't output if this one fail. */
>      [LLVM_TESTCASE_BPF_PROLOGUE]    = {
>                     .source = test_llvm__bpf_test_prologue_prog,
> -                   .errmsg = "failed for unknown reason",
> +                   .errmsg = "Unable to compile BPF prologue
> testing program",
> +                   .hintmsg = "This is an internal error, please
> report it",
>                     .tried = false},
>      {.source = NULL}
>  };
> @@ -43,16 +47,16 @@ static int test__bpf_parsing(void *obj_buf,
> size_t obj_buf_sz)
> 
>      obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, NULL);
>      if (!obj)
> -        return -1;
> +        return TEST_FAIL;
>      bpf_object__close(obj);
> -    return 0;
> +    return TEST_OK;
>  }
>  #else
>  static int test__bpf_parsing(void *obj_buf __maybe_unused,
>                   size_t obj_buf_sz __maybe_unused)
>  {
> -    fprintf(stderr, " (skip bpf parsing)");
> -    return 0;
> +    pr_debug("BPF support is not compiled, skip BPF parsing\n");
> +    return TEST_SKIP;
>  }
>  #endif
> 
> @@ -70,8 +74,8 @@ compose_source(const char *raw_source)
>      err = sscanf(utsname.release, "%d.%d.%d",
>               &version, &patchlevel, &sublevel);
>      if (err != 3) {
> -        fprintf(stderr, " (Can't get kernel version from uname '%s')",
> -            utsname.release);
> +        pr_debug("Unablt to get kernel version from uname '%s'\n",
> +             utsname.release);
>          return NULL;
>      }
> 
> @@ -104,7 +108,7 @@ static int __test__llvm(int i)
>       * and clang is not found in $PATH, and this is not perf test -v
>       */
>      if (verbose == 0 && !llvm_param.user_set_param &&
> llvm__search_clang()) {
> -        fprintf(stderr, " (no clang, try 'perf test -v LLVM')");
> +        pr_debug("Unable to find workable clang\n");
>          return TEST_SKIP;
>      }
> 
> @@ -155,7 +159,7 @@ static int __test__llvm(int i)
> 
>      verbose = old_verbose;
>      if (err)
> -        return -1;
> +        return TEST_FAIL;
> 
>      err = test__bpf_parsing(obj_buf, obj_buf_sz);
>      if (!err && result) {
> @@ -181,13 +185,12 @@ int test__llvm(void)
>               * First testcase tests basic LLVM compiling. If it
>               * fails, no need to check others.
>               */
> -            if (!verbose)
> -                fprintf(stderr, " (use -v to see error message)");
> -            return ret;
> +            return TEST_FAIL;
>          } else if (ret) {
> -            if (!verbose && llvm_testcases[i].errmsg)
> -                fprintf(stderr, " (%s)", llvm_testcases[i].errmsg);
> -            return 0;
> +            pr_debug("%s\n", llvm_testcases[i].errmsg);
> +            if (llvm_testcases[i].hintmsg)
> +                pr_debug("Hint:\t%s\n", llvm_testcases[i].hintmsg);
> +            return TEST_SKIP;
>          }
>      }
>      return 0;
> @@ -199,12 +202,15 @@ void test__llvm_prepare(void)
> 
>      for (i = 0; llvm_testcases[i].source; i++) {
>          struct test_llvm__bpf_result *result;
> +        void *p;
> 
> -        result = mmap(NULL, SHARED_BUF_INIT_SIZE,
> -                  PROT_READ | PROT_WRITE,
> -                  MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> -        if (!result)
> +        p = mmap(NULL, SHARED_BUF_INIT_SIZE,
> +             PROT_READ | PROT_WRITE,
> +             MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> +        if (p == MAP_FAILED)
>              return;
> +
> +        result = p;
>          memset((void *)result, '\0', SHARED_BUF_INIT_SIZE);
> 
>          llvm_testcases[i].result = result;
> diff --git a/tools/perf/util/bpf-prologue.c b/tools/perf/util/bpf-prologue.c
> index e4adb18..36093d9 100644
> --- a/tools/perf/util/bpf-prologue.c
> +++ b/tools/perf/util/bpf-prologue.c
> @@ -337,8 +337,10 @@ int bpf__gen_prologue(struct probe_trace_arg
> *args, int nargs,
>          return 0;
>      }
> 
> -    if (nargs > BPF_PROLOGUE_MAX_ARGS)
> +    if (nargs > BPF_PROLOGUE_MAX_ARGS) {
> +        pr_warning("bpf: prologue: too many arguments\n");
>          nargs = BPF_PROLOGUE_MAX_ARGS;
> +    }
>      if (cnt_space > BPF_MAXINSNS)
>          cnt_space = BPF_MAXINSNS;
> 
> 

Re: [PATCH 21/31] perf test: Enforce LLVM test, add kbuild test

[Arnaldo Carvalho de Melo]

Em Tue, Oct 20, 2015 at 06:36:33PM +0800, Wangnan (F) escreveu:
> On 2015/10/19 22:53, Arnaldo Carvalho de Melo wrote:
> >Em Mon, Oct 19, 2015 at 11:42:31PM +0900, Namhyung Kim escreveu:
> >>On Wed, Oct 14, 2015 at 12:41:32PM +0000, Wang Nan wrote:
> >>>This patch enforces existing LLVM test, makes it compile more than one
> >>>BPF source file. The compiled results are stored, can be used for other
> >>>testcases. Except the first testcase (named LLVM_TESTCASE_BASE), failures
> >>>of other test cases are not considered as failure of the whole test.

> >>>Adds a kbuild testcase to check whether kernel headers can be correctly
> >>>found.

> >>>For example:
> >>>  # perf test LLVM
> >>>    38: Test LLVM searching and compiling                        : (llvm.kbuild-dir can be fixed) Ok

> >>IMHO it'd be better to keep the test result simply as either "Ok" or
> >>"FAILED" and provide details with -v option.
> 
> I have to say I'm not the first one to output things like this:
> 
>  #perf test dummy attr
>  14: struct perf_event_attr setup                             :
> (omitted) Ok
>  23: Test using a dummy software event to keep tracking       : (not
> supported) Ok
> 
> Do you think they also need to be fixed?

yes
 
> >"Skipped" looks more natural here, with the reason for it skipping being shown
> >only with -v.
> 
> For this specific case, I will change the logic:
> 
>  1. When basic test failed (no clang, compiling error...), report failure;
> 
>  2. When basic test passed but kbuild test or later test cases
> failed, report
>     Skip.
> 
>  3. Don't output any other information.
> 
> Thank you.
> 
> >Ingo made some comments about 'perf test' output recently, have you read that?
> >
> >>Anyway I found the tracepoint error message is annoying in the current
> >You mean all these warnings:
> >
> >[root@zoo ~]# perf test "parse event"
> >  5: parse events tests                                       :  Warning: [sunrpc:xprt_lookup_rqst] function __builtin_constant_p not defined
> >   Warning: [sunrpc:xprt_transmit] function __builtin_constant_p not defined
> >   Warning: [sunrpc:xprt_complete_rqst] function __builtin_constant_p not defined
> ><SNIP>
> >   Warning: [xen:xen_mmu_set_pud] function sizeof not defined
> >   Warning: [xen:xen_mmu_set_pgd] function sizeof not defined
> >   Warning: [xen:xen_mmu_ptep_modify_prot_start] function sizeof not defined
> >   Warning: [xen:xen_mmu_ptep_modify_prot_commit] function sizeof not defined
> >  Ok
> >[root@zoo ~]#
> >
> >i.e. making those warnings appear only under 'perf test -v'? Cool!
> >
> >>perf test output.  Will send a fix soon.
> >>
> >>Thanks,
> >>Namhyung
> 

Re: [PATCH 03/31] perf tools: Enable passing bpf object file to --event

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:14PM +0000, Wang Nan escreveu:
> By introducing new rules in tools/perf/util/parse-events.[ly], this
> patch enables 'perf record --event bpf_file.o' to select events by an
> eBPF object file. It calls parse_events_load_bpf() to load that file,
> which uses bpf__prepare_load() and finally calls bpf_object__open() for
> the object files.
> 
> After applying this patch, commands like:
> 
>  # perf record --event foo.o sleep
> 
> become possible.

So, trying the above command I get almost perfect output:

  [root@felicio ~]# perf record --event foo.o sleep
  libbpf: failed to open foo.o: No such file or directory
  event syntax error: 'foo.o'
                       \___ BPF object file 'foo.o' is invalid

  (add -v to see detail)
  Run 'perf list' for a list of valid events

   Usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list available events
  [root@felicio ~]# 


Good thing would be to not have any message from libbpf and the right error
message from the parser, i.e. the first three lines become these two:


  event syntax error: 'foo.o'
                       \___ BPF object file 'foo.o' not found.o

But that can be fixed up in an upcoming patch, so I am applying this one now in
my new attempt at processing this patchkit.

- Arnaldo

 
> However, at this point it is unable to link any useful things onto the
> evsel list because the creating of probe points and BPF program
> attaching have not been implemented.  Before real events are possible to
> be extracted, to avoid perf report error because of empty evsel list,
> this patch link a dummy evsel. The dummy event related code will be
> removed when probing and extracting code is ready.
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---
>  tools/perf/perf.c              |  2 ++
>  tools/perf/util/Build          |  1 +
>  tools/perf/util/parse-events.c | 57 ++++++++++++++++++++++++++++++++++++++++++
>  tools/perf/util/parse-events.h |  8 ++++++
>  tools/perf/util/parse-events.l |  3 +++
>  tools/perf/util/parse-events.y | 18 ++++++++++++-
>  6 files changed, 88 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/perf.c b/tools/perf/perf.c
> index 5437134..3d4c7c0 100644
> --- a/tools/perf/perf.c
> +++ b/tools/perf/perf.c
> @@ -15,6 +15,7 @@
>  #include "util/run-command.h"
>  #include "util/parse-events.h"
>  #include "util/parse-options.h"
> +#include "util/bpf-loader.h"
>  #include "util/debug.h"
>  #include <api/fs/tracing_path.h>
>  #include <pthread.h>
> @@ -385,6 +386,7 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
>  	status = p->fn(argc, argv, prefix);
>  	exit_browser(status);
>  	perf_env__exit(&perf_env);
> +	bpf__clear();
>  
>  	if (status)
>  		return status & 0xff;
> diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> index 9217119..591b3fe 100644
> --- a/tools/perf/util/Build
> +++ b/tools/perf/util/Build
> @@ -87,6 +87,7 @@ libperf-$(CONFIG_AUXTRACE) += intel-bts.o
>  libperf-y += parse-branch-options.o
>  libperf-y += parse-regs-options.o
>  
> +libperf-$(CONFIG_LIBBPF) += bpf-loader.o
>  libperf-$(CONFIG_LIBELF) += symbol-elf.o
>  libperf-$(CONFIG_LIBELF) += probe-file.o
>  libperf-$(CONFIG_LIBELF) += probe-event.o
> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
> index 991bbd4..a02abd3 100644
> --- a/tools/perf/util/parse-events.c
> +++ b/tools/perf/util/parse-events.c
> @@ -11,6 +11,7 @@
>  #include "symbol.h"
>  #include "cache.h"
>  #include "header.h"
> +#include "bpf-loader.h"
>  #include "debug.h"
>  #include <api/fs/tracing_path.h>
>  #include "parse-events-bison.h"
> @@ -529,6 +530,62 @@ static int add_tracepoint_multi_sys(struct list_head *list, int *idx,
>  	return ret;
>  }
>  
> +int parse_events_load_bpf_obj(struct parse_events_evlist *data,
> +			      struct list_head *list,
> +			      struct bpf_object *obj)
> +{
> +	int err;
> +	char errbuf[BUFSIZ];
> +
> +	if (IS_ERR(obj) || !obj) {
> +		snprintf(errbuf, sizeof(errbuf),
> +			 "Internal error: load bpf obj with NULL");
> +		err = -EINVAL;
> +		goto errout;
> +	}
> +
> +	/*
> +	 * Temporary add a dummy event here so we can check whether
> +	 * basic bpf loader works. Following patches will replace
> +	 * dummy event by useful evsels.
> +	 */
> +	return parse_events_add_numeric(data, list, PERF_TYPE_SOFTWARE,
> +					PERF_COUNT_SW_DUMMY, NULL);
> +errout:
> +	data->error->help = strdup("(add -v to see detail)");
> +	data->error->str = strdup(errbuf);
> +	return err;
> +}
> +
> +int parse_events_load_bpf(struct parse_events_evlist *data,
> +			  struct list_head *list,
> +			  char *bpf_file_name)
> +{
> +	struct bpf_object *obj;
> +
> +	obj = bpf__prepare_load(bpf_file_name);
> +	if (IS_ERR(obj) || !obj) {
> +		char errbuf[BUFSIZ];
> +		int err;
> +
> +		err = obj ? PTR_ERR(obj) : -EINVAL;
> +
> +		if (err == -ENOTSUP)
> +			snprintf(errbuf, sizeof(errbuf),
> +				 "BPF support is not compiled");
> +		else
> +			snprintf(errbuf, sizeof(errbuf),
> +				 "BPF object file '%s' is invalid",
> +				 bpf_file_name);
> +
> +		data->error->help = strdup("(add -v to see detail)");
> +		data->error->str = strdup(errbuf);
> +		return err;
> +	}
> +
> +	return parse_events_load_bpf_obj(data, list, obj);
> +}
> +
>  static int
>  parse_breakpoint_type(const char *type, struct perf_event_attr *attr)
>  {
> diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
> index f13d3cc..fbb16c7 100644
> --- a/tools/perf/util/parse-events.h
> +++ b/tools/perf/util/parse-events.h
> @@ -121,6 +121,14 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
>  				char *sys, char *event,
>  				struct parse_events_error *error,
>  				struct list_head *head_config);
> +int parse_events_load_bpf(struct parse_events_evlist *data,
> +			  struct list_head *list,
> +			  char *bpf_file_name);
> +/* Provide this function for perf test */
> +struct bpf_object;
> +int parse_events_load_bpf_obj(struct parse_events_evlist *data,
> +			      struct list_head *list,
> +			      struct bpf_object *obj);
>  int parse_events_add_numeric(struct parse_events_evlist *data,
>  			     struct list_head *list,
>  			     u32 type, u64 config,
> diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
> index be24457..5e5d31a 100644
> --- a/tools/perf/util/parse-events.l
> +++ b/tools/perf/util/parse-events.l
> @@ -115,6 +115,7 @@ do {							\
>  group		[^,{}/]*[{][^}]*[}][^,{}/]*
>  event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
>  event		[^,{}/]+
> +bpf_object	.*\.(o|bpf)
>  
>  num_dec		[0-9]+
>  num_hex		0x[a-fA-F0-9]+
> @@ -159,6 +160,7 @@ modifier_bp	[rwx]{1,3}
>  		}
>  
>  {event_pmu}	|
> +{bpf_object}	|
>  {event}		{
>  			BEGIN(INITIAL);
>  			REWIND(1);
> @@ -264,6 +266,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
>  {num_hex}		{ return value(yyscanner, 16); }
>  
>  {modifier_event}	{ return str(yyscanner, PE_MODIFIER_EVENT); }
> +{bpf_object}		{ return str(yyscanner, PE_BPF_OBJECT); }
>  {name}			{ return pmu_str_check(yyscanner); }
>  "/"			{ BEGIN(config); return '/'; }
>  -			{ return '-'; }
> diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
> index ae6af26..497f19b 100644
> --- a/tools/perf/util/parse-events.y
> +++ b/tools/perf/util/parse-events.y
> @@ -42,6 +42,7 @@ static inc_group_count(struct list_head *list,
>  %token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
>  %token PE_EVENT_NAME
>  %token PE_NAME
> +%token PE_BPF_OBJECT
>  %token PE_MODIFIER_EVENT PE_MODIFIER_BP
>  %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
>  %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
> @@ -53,6 +54,7 @@ static inc_group_count(struct list_head *list,
>  %type <num> PE_RAW
>  %type <num> PE_TERM
>  %type <str> PE_NAME
> +%type <str> PE_BPF_OBJECT
>  %type <str> PE_NAME_CACHE_TYPE
>  %type <str> PE_NAME_CACHE_OP_RESULT
>  %type <str> PE_MODIFIER_EVENT
> @@ -70,6 +72,7 @@ static inc_group_count(struct list_head *list,
>  %type <tracepoint_name> tracepoint_name
>  %type <head> event_legacy_numeric
>  %type <head> event_legacy_raw
> +%type <head> event_bpf_file
>  %type <head> event_def
>  %type <head> event_mod
>  %type <head> event_name
> @@ -203,7 +206,8 @@ event_def: event_pmu |
>  	   event_legacy_mem |
>  	   event_legacy_tracepoint sep_dc |
>  	   event_legacy_numeric sep_dc |
> -	   event_legacy_raw sep_dc
> +	   event_legacy_raw sep_dc |
> +	   event_bpf_file
>  
>  event_pmu:
>  PE_NAME '/' event_config '/'
> @@ -449,6 +453,18 @@ PE_RAW
>  	$$ = list;
>  }
>  
> +event_bpf_file:
> +PE_BPF_OBJECT
> +{
> +	struct parse_events_evlist *data = _data;
> +	struct parse_events_error *error = data->error;
> +	struct list_head *list;
> +
> +	ALLOC_LIST(list);
> +	ABORT_ON(parse_events_load_bpf(data, list, $1));
> +	$$ = list;
> +}
> +
>  start_terms: event_config
>  {
>  	struct parse_events_terms *data = _data;
> -- 
> 1.8.3.4

Re: [PATCH 03/31] perf tools: Enable passing bpf object file to --event

[Arnaldo Carvalho de Melo]

Em Tue, Oct 20, 2015 at 12:12:55PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Oct 14, 2015 at 12:41:14PM +0000, Wang Nan escreveu:
> > By introducing new rules in tools/perf/util/parse-events.[ly], this
> > patch enables 'perf record --event bpf_file.o' to select events by an
> > eBPF object file. It calls parse_events_load_bpf() to load that file,
> > which uses bpf__prepare_load() and finally calls bpf_object__open() for
> > the object files.
> > 
> > After applying this patch, commands like:
> > 
> >  # perf record --event foo.o sleep
> > 
> > become possible.
> 
> So, trying the above command I get almost perfect output:
> 
>   [root@felicio ~]# perf record --event foo.o sleep
>   libbpf: failed to open foo.o: No such file or directory
>   event syntax error: 'foo.o'
>                        \___ BPF object file 'foo.o' is invalid
> 
>   (add -v to see detail)
>   Run 'perf list' for a list of valid events
> 
>    Usage: perf record [<options>] [<command>]
>       or: perf record [<options>] -- <command> [<options>]
> 
>       -e, --event <event>   event selector. use 'perf list' to list available events
>   [root@felicio ~]# 
> 
> 
> Good thing would be to not have any message from libbpf and the right error
> message from the parser, i.e. the first three lines become these two:
> 
> 
>   event syntax error: 'foo.o'
>                        \___ BPF object file 'foo.o' not found.o
> 
> But that can be fixed up in an upcoming patch, so I am applying this one now in
> my new attempt at processing this patchkit.

Ditto for:

  [acme@felicio linux]$ perf record --event /tmp/build/perf/perf.o sleep
  libbpf: /tmp/build/perf/perf.o is not an eBPF object file
  event syntax error: '/tmp/build/perf/perf.o'
                       \___ BPF object file '/tmp/build/perf/perf.o' is invalid
  
  (add -v to see detail)
  Run 'perf list' for a list of valid events

   Usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list available events
  [acme@felicio linux]$

Now trying to find a _valid_ ebpf object file to test with.

- Arnaldo

Re: [PATCH 03/31] perf tools: Enable passing bpf object file to --event

[Arnaldo Carvalho de Melo]

Em Tue, Oct 20, 2015 at 12:15:58PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Oct 20, 2015 at 12:12:55PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Wed, Oct 14, 2015 at 12:41:14PM +0000, Wang Nan escreveu:
> > > By introducing new rules in tools/perf/util/parse-events.[ly], this
> > > patch enables 'perf record --event bpf_file.o' to select events by an
> > > eBPF object file. It calls parse_events_load_bpf() to load that file,
> > > which uses bpf__prepare_load() and finally calls bpf_object__open() for
> > > the object files.
> > > 
> > > After applying this patch, commands like:
> > > 
> > >  # perf record --event foo.o sleep
> > > 
> > > become possible.
> > 
> > So, trying the above command I get almost perfect output:
> > 
> >   [root@felicio ~]# perf record --event foo.o sleep
> >   libbpf: failed to open foo.o: No such file or directory
> >   event syntax error: 'foo.o'
> >                        \___ BPF object file 'foo.o' is invalid
> > 
> >   (add -v to see detail)
> >   Run 'perf list' for a list of valid events
> > 
> >    Usage: perf record [<options>] [<command>]
> >       or: perf record [<options>] -- <command> [<options>]
> > 
> >       -e, --event <event>   event selector. use 'perf list' to list available events
> >   [root@felicio ~]# 
> > 
> > 
> > Good thing would be to not have any message from libbpf and the right error
> > message from the parser, i.e. the first three lines become these two:
> > 
> > 
> >   event syntax error: 'foo.o'
> >                        \___ BPF object file 'foo.o' not found.o
> > 
> > But that can be fixed up in an upcoming patch, so I am applying this one now in
> > my new attempt at processing this patchkit.
> 
> Ditto for:
> 
>   [acme@felicio linux]$ perf record --event /tmp/build/perf/perf.o sleep
>   libbpf: /tmp/build/perf/perf.o is not an eBPF object file
>   event syntax error: '/tmp/build/perf/perf.o'
>                        \___ BPF object file '/tmp/build/perf/perf.o' is invalid
>   
>   (add -v to see detail)
>   Run 'perf list' for a list of valid events
> 
>    Usage: perf record [<options>] [<command>]
>       or: perf record [<options>] -- <command> [<options>]
> 
>       -e, --event <event>   event selector. use 'perf list' to list available events
>   [acme@felicio linux]$
> 
> Now trying to find a _valid_ ebpf object file to test with.

Managed after running:

 perf test LLVM

copy'n'pasting the output of those "set env" lines, replacing it with
export, etc to get to:

[acme@felicio linux]$ echo '__attribute__((section("do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | $CLANG_EXEC -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o /tmp/foo.o
[acme@felicio linux]$ file /tmp/foo.o
/tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
[acme@felicio linux]$ 

And finally:


[acme@felicio linux]$ file /tmp/foo.o 
/tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
[acme@felicio linux]$ perf record --event /tmp/foo.o sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.013 MB perf.data ]
[acme@felicio linux]$ perf evlist  -v
/tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1
[acme@felicio linux]$ perf evlist
/tmp/foo.o
[acme@felicio linux]$ 

So, type 1 is PERF_TYPE_SOFTWARE, config 0x9 is PERF_COUNT_SW_DUMMY, ok.

And it behaves accordingly, no samples, etc.

Added the sequence of testing with a non-existing file, with a normal ELF file
and then with a valid eBPF ELF file as committer notes.

Continuing.

- Arnaldo

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:15PM +0000, Wang Nan escreveu:
> This patch introduces bpf__{un,}probe() functions to enable callers to
> create kprobe points based on section names a BPF program. It parses
> the section names in the program and creates corresponding 'struct
> perf_probe_event' structures. The parse_perf_probe_command() function is
> used to do the main parsing work. The resuling 'struct perf_probe_event'
> is stored into program private data for further using.
> 
> By utilizing the new probing API, this patch creates probe points during
> event parsing.
> 
> To ensure probe points be removed correctly, register an atexit hook
> so even perf quit through exit() bpf__clear() is still called, so probing
> points are cleared. Note that bpf_clear() should be registered before
> bpf__probe() is called, so failure of bpf__probe() can still trigger
> bpf__clear() to remove probe points which are already probed.
> 
> strerror style error reporting scaffold is created by this patch.
> bpf__strerror_probe() is the first error reporting function in bpf-loader.c.

So, this one, for a non-root user gives me:

[acme@felicio linux]$ perf record --event /tmp/foo.o sleep 1
event syntax error: '/tmp/foo.o'
                     \___ Invalid argument

(add -v to see detail)
Run 'perf list' for a list of valid events

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

    -e, --event <event>   event selector. use 'perf list' to list available events
[acme@felicio linux]$

--------------------

I.e. no libbpf error (good!) but then, just an -EINVAL as the "event syntax
error", which clearly isn't a syntax error, we need to tell the user that he or she
needs special perfmissions for using sys_bpf() :-)

As root:

[root@felicio ~]# perf record --event /tmp/foo.o sleep
event syntax error: '/tmp/foo.o'
                     \___ Invalid argument

(add -v to see detail)
Run 'perf list' for a list of valid events

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

    -e, --event <event>   event selector. use 'perf list' to list available events
[root@felicio ~]# ls -la /tmp/foo.o
-rw-rw-r--. 1 acme acme 824 Oct 20 12:35 /tmp/foo.o
[root@felicio ~]# file /tmp/foo.o
/tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped


Humm, its something else, this is an ancient kernel, 4.2.0, probably without
eBPF support? Nope, its there:

[root@felicio ~]# grep -i sys_bpf /proc/kallsyms
ffffffff811829d0 T SyS_bpf
ffffffff811829d0 T sys_bpf
[root@felicio ~]#

Its something else, we need to improve this error reporting:

[root@felicio ~]# perf record -v --event /tmp/foo.o sleep 1
libbpf: loading /tmp/foo.o
libbpf: section .strtab, size 60, link 0, flags 0, type=3
libbpf: section .text, size 0, link 0, flags 6, type=1
libbpf: section .data, size 0, link 0, flags 3, type=1
libbpf: section .bss, size 0, link 0, flags 3, type=8
libbpf: section do_fork, size 16, link 0, flags 6, type=1
libbpf: found program do_fork
libbpf: section license, size 4, link 0, flags 3, type=1
libbpf: license of /tmp/foo.o is GPL
libbpf: section version, size 4, link 0, flags 3, type=1
libbpf: kernel version of /tmp/foo.o is 40100
libbpf: section .symtab, size 96, link 1, flags 0, type=2
bpf: config program 'do_fork'
symbol:do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
bpf: 'do_fork': event name is missing
event syntax error: '/tmp/foo.o'
                     \___ Invalid argument

(add -v to see detail)
Run 'perf list' for a list of valid events

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

    -e, --event <event>   event selector. use 'perf list' to list available events
[root@felicio ~]#

[root@felicio ~]# grep do_fork /proc/kallsyms 
ffffffff81099ab0 T _do_fork
ffffffff81ccc800 d do_fork_test
[root@felicio ~]#

$ echo '__attribute__((section("_do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c - -target bpf -O2 -o /tmp/foo.o

[root@felicio ~]# perf record  -v --event /tmp/foo.o sleep 1
libbpf: loading /tmp/foo.o
libbpf: section .strtab, size 61, link 0, flags 0, type=3
libbpf: section .text, size 0, link 0, flags 6, type=1
libbpf: section .data, size 0, link 0, flags 3, type=1
libbpf: section .bss, size 0, link 0, flags 3, type=8
libbpf: section _do_fork, size 16, link 0, flags 6, type=1
libbpf: found program _do_fork
libbpf: section license, size 4, link 0, flags 3, type=1
libbpf: license of /tmp/foo.o is GPL
libbpf: section version, size 4, link 0, flags 3, type=1
libbpf: kernel version of /tmp/foo.o is 40100
libbpf: section .symtab, size 96, link 1, flags 0, type=2
bpf: config program '_do_fork'
symbol:_do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
bpf: '_do_fork': event name is missing
event syntax error: '/tmp/foo.o'
                     \___ Invalid argument

(add -v to see detail)
Run 'perf list' for a list of valid events

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

    -e, --event <event>   event selector. use 'perf list' to list available events
[root@felicio ~]#

So it still doesn't work, doesn't look like it is trying to find a vmlinux,
will look at another patch IIRC is in this patchkit allowing us to tell
'perf record' where to find it... But it can as well use kallsyms...

- Arnaldo

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[David Ahern]

On 10/20/15 1:12 PM, Arnaldo Carvalho de Melo wrote:
> I.e. no libbpf error (good!) but then, just an -EINVAL as the "event syntax
> error", which clearly isn't a syntax error, we need to tell the user that he or she
> needs special perfmissions for using sys_bpf():-)

perfmissions? Are you coining new words -- permissions for perf? ;-)

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Arnaldo Carvalho de Melo]

Em Tue, Oct 20, 2015 at 01:16:04PM -0600, David Ahern escreveu:
> On 10/20/15 1:12 PM, Arnaldo Carvalho de Melo wrote:
> >I.e. no libbpf error (good!) but then, just an -EINVAL as the "event syntax
> >error", which clearly isn't a syntax error, we need to tell the user that he or she
> >needs special perfmissions for using sys_bpf():-)
> 
> perfmissions? Are you coining new words -- permissions for perf? ;-)

LOL, totally unintended 8-)

- Arnaldo

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Arnaldo Carvalho de Melo]

Em Tue, Oct 20, 2015 at 04:12:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Oct 14, 2015 at 12:41:15PM +0000, Wang Nan escreveu:
> > This patch introduces bpf__{un,}probe() functions to enable callers to
> > create kprobe points based on section names a BPF program. It parses
> > the section names in the program and creates corresponding 'struct
> > perf_probe_event' structures. The parse_perf_probe_command() function is
> > used to do the main parsing work. The resuling 'struct perf_probe_event'
> > is stored into program private data for further using.
> > 
> > By utilizing the new probing API, this patch creates probe points during
> > event parsing.
> > 
> > To ensure probe points be removed correctly, register an atexit hook
> > so even perf quit through exit() bpf__clear() is still called, so probing
> > points are cleared. Note that bpf_clear() should be registered before
> > bpf__probe() is called, so failure of bpf__probe() can still trigger
> > bpf__clear() to remove probe points which are already probed.
> > 
> > strerror style error reporting scaffold is created by this patch.
> > bpf__strerror_probe() is the first error reporting function in bpf-loader.c.
> 
> So, this one, for a non-root user gives me:
> 
> [acme@felicio linux]$ perf record --event /tmp/foo.o sleep 1
> event syntax error: '/tmp/foo.o'
>                      \___ Invalid argument
> 
> (add -v to see detail)
> Run 'perf list' for a list of valid events
> 
>  Usage: perf record [<options>] [<command>]
>     or: perf record [<options>] -- <command> [<options>]
> 
>     -e, --event <event>   event selector. use 'perf list' to list available events
> [acme@felicio linux]$
> 
> --------------------
> 
> I.e. no libbpf error (good!) but then, just an -EINVAL as the "event syntax
> error", which clearly isn't a syntax error, we need to tell the user that he or she
> needs special perfmissions for using sys_bpf() :-)
> 
> As root:
> 
> [root@felicio ~]# perf record --event /tmp/foo.o sleep
> event syntax error: '/tmp/foo.o'
>                      \___ Invalid argument
> 
> (add -v to see detail)
> Run 'perf list' for a list of valid events
> 
>  Usage: perf record [<options>] [<command>]
>     or: perf record [<options>] -- <command> [<options>]
> 
>     -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]# ls -la /tmp/foo.o
> -rw-rw-r--. 1 acme acme 824 Oct 20 12:35 /tmp/foo.o
> [root@felicio ~]# file /tmp/foo.o
> /tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
> 
> 
> Humm, its something else, this is an ancient kernel, 4.2.0, probably without
> eBPF support? Nope, its there:
> 
> [root@felicio ~]# grep -i sys_bpf /proc/kallsyms
> ffffffff811829d0 T SyS_bpf
> ffffffff811829d0 T sys_bpf
> [root@felicio ~]#
> 
> Its something else, we need to improve this error reporting:
> 
> [root@felicio ~]# perf record -v --event /tmp/foo.o sleep 1
> libbpf: loading /tmp/foo.o
> libbpf: section .strtab, size 60, link 0, flags 0, type=3
> libbpf: section .text, size 0, link 0, flags 6, type=1
> libbpf: section .data, size 0, link 0, flags 3, type=1
> libbpf: section .bss, size 0, link 0, flags 3, type=8
> libbpf: section do_fork, size 16, link 0, flags 6, type=1
> libbpf: found program do_fork
> libbpf: section license, size 4, link 0, flags 3, type=1
> libbpf: license of /tmp/foo.o is GPL
> libbpf: section version, size 4, link 0, flags 3, type=1
> libbpf: kernel version of /tmp/foo.o is 40100
> libbpf: section .symtab, size 96, link 1, flags 0, type=2
> bpf: config program 'do_fork'
> symbol:do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> bpf: 'do_fork': event name is missing
> event syntax error: '/tmp/foo.o'
>                      \___ Invalid argument
> 
> (add -v to see detail)
> Run 'perf list' for a list of valid events
> 
>  Usage: perf record [<options>] [<command>]
>     or: perf record [<options>] -- <command> [<options>]
> 
>     -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]#
> 
> [root@felicio ~]# grep do_fork /proc/kallsyms 
> ffffffff81099ab0 T _do_fork
> ffffffff81ccc800 d do_fork_test
> [root@felicio ~]#
> 
> $ echo '__attribute__((section("_do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c - -target bpf -O2 -o /tmp/foo.o
> 
> [root@felicio ~]# perf record  -v --event /tmp/foo.o sleep 1
> libbpf: loading /tmp/foo.o
> libbpf: section .strtab, size 61, link 0, flags 0, type=3
> libbpf: section .text, size 0, link 0, flags 6, type=1
> libbpf: section .data, size 0, link 0, flags 3, type=1
> libbpf: section .bss, size 0, link 0, flags 3, type=8
> libbpf: section _do_fork, size 16, link 0, flags 6, type=1
> libbpf: found program _do_fork
> libbpf: section license, size 4, link 0, flags 3, type=1
> libbpf: license of /tmp/foo.o is GPL
> libbpf: section version, size 4, link 0, flags 3, type=1
> libbpf: kernel version of /tmp/foo.o is 40100
> libbpf: section .symtab, size 96, link 1, flags 0, type=2
> bpf: config program '_do_fork'
> symbol:_do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> bpf: '_do_fork': event name is missing
> event syntax error: '/tmp/foo.o'
>                      \___ Invalid argument
> 
> (add -v to see detail)
> Run 'perf list' for a list of valid events
> 
>  Usage: perf record [<options>] [<command>]
>     or: perf record [<options>] -- <command> [<options>]
> 
>     -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]#
> 
> So it still doesn't work, doesn't look like it is trying to find a vmlinux,
> will look at another patch IIRC is in this patchkit allowing us to tell
> 'perf record' where to find it... But it can as well use kallsyms...


That doesn't seem to be the problem, the last check after
parse_perf_probe_command() is failing:


+       pr_debug("bpf: config program '%s'\n", config_str);
+       err = parse_perf_probe_command(config_str, pev);
+       if (err < 0) {
+               pr_debug("bpf: '%s' is not a valid config string\n",
+                        config_str);
+               err = -EINVAL;
+               goto errout;
+       }
+
+       if (pev->group && strcmp(pev->group, PERF_BPF_PROBE_GROUP)) {
+               pr_debug("bpf: '%s': group for event is set and not '%s'.\n",
+                        config_str, PERF_BPF_PROBE_GROUP);
+               err = -EINVAL;
+               goto errout;
+       } else if (!pev->group)
+               pev->group = strdup(PERF_BPF_PROBE_GROUP);
+
+       if (!pev->group) {
+               pr_debug("bpf: strdup failed\n");
+               err = -ENOMEM;
+               goto errout;
+       }
+
+       if (!pev->event) {
+               pr_debug("bpf: '%s': event name is missing\n",
+                        config_str);
+               err = -EINVAL;
+               goto errout;
+       }
+       pr_debug("bpf: config '%s' is ok\n", config_str);


I.e. pev->event is NULL, is this expected at this point? I'll continue
investigating later.

What I have is at the perf/ebpf branch in my tree.

- Arnaldo

Re: [PATCH 03/31] perf tools: Enable passing bpf object file to --event

[Wangnan (F)]

On 2015/10/20 23:15, Arnaldo Carvalho de Melo wrote:
> Em Tue, Oct 20, 2015 at 12:12:55PM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Wed, Oct 14, 2015 at 12:41:14PM +0000, Wang Nan escreveu:
>>> By introducing new rules in tools/perf/util/parse-events.[ly], this
>>> patch enables 'perf record --event bpf_file.o' to select events by an
>>> eBPF object file. It calls parse_events_load_bpf() to load that file,
>>> which uses bpf__prepare_load() and finally calls bpf_object__open() for
>>> the object files.
>>>
>>> After applying this patch, commands like:
>>>
>>>   # perf record --event foo.o sleep
>>>
>>> become possible.
>> So, trying the above command I get almost perfect output:
>>
>>    [root@felicio ~]# perf record --event foo.o sleep
>>    libbpf: failed to open foo.o: No such file or directory
>>    event syntax error: 'foo.o'
>>                         \___ BPF object file 'foo.o' is invalid
>>
>>    (add -v to see detail)
>>    Run 'perf list' for a list of valid events
>>
>>     Usage: perf record [<options>] [<command>]
>>        or: perf record [<options>] -- <command> [<options>]
>>
>>        -e, --event <event>   event selector. use 'perf list' to list available events
>>    [root@felicio ~]#
>>
>>
>> Good thing would be to not have any message from libbpf and the right error
>> message from the parser, i.e. the first three lines become these two:
>>
>>
>>    event syntax error: 'foo.o'
>>                         \___ BPF object file 'foo.o' not found.o
>>
>> But that can be fixed up in an upcoming patch, so I am applying this one now in
>> my new attempt at processing this patchkit.

I think fixing that is not very hard. When designing libbpf, I thought
the problem like this so makes libbpf output controled by caller using
'libbpf_set_print()'. What we need to do is to pass different output
functions to libbpf. We can easily disable all output from libbpf if
verbose is set 0. The only question is: do you want me to provide a new
version of patch 'perf ebpf: Add the libbpf glue' or you prefer another
patch to adjust the output functions?

Thank you.

> Ditto for:
>
>    [acme@felicio linux]$ perf record --event /tmp/build/perf/perf.o sleep
>    libbpf: /tmp/build/perf/perf.o is not an eBPF object file
>    event syntax error: '/tmp/build/perf/perf.o'
>                         \___ BPF object file '/tmp/build/perf/perf.o' is invalid
>    
>    (add -v to see detail)
>    Run 'perf list' for a list of valid events
>
>     Usage: perf record [<options>] [<command>]
>        or: perf record [<options>] -- <command> [<options>]
>
>        -e, --event <event>   event selector. use 'perf list' to list available events
>    [acme@felicio linux]$
>
> Now trying to find a _valid_ ebpf object file to test with.
>
> - Arnaldo

Re: [PATCH 03/31] perf tools: Enable passing bpf object file to --event

[Wangnan (F)]

On 2015/10/20 23:42, Arnaldo Carvalho de Melo wrote:
> Em Tue, Oct 20, 2015 at 12:15:58PM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Tue, Oct 20, 2015 at 12:12:55PM -0300, Arnaldo Carvalho de Melo escreveu:
>>> Em Wed, Oct 14, 2015 at 12:41:14PM +0000, Wang Nan escreveu:
>>>
> Managed after running:
>
>   perf test LLVM
>
> copy'n'pasting the output of those "set env" lines, replacing it with
> export, etc to get to:
>
> [acme@felicio linux]$ echo '__attribute__((section("do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | $CLANG_EXEC -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o /tmp/foo.o
> [acme@felicio linux]$ file /tmp/foo.o
> /tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
> [acme@felicio linux]$
>
> And finally:
>
>
> [acme@felicio linux]$ file /tmp/foo.o
> /tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
> [acme@felicio linux]$ perf record --event /tmp/foo.o sleep 1
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.013 MB perf.data ]
> [acme@felicio linux]$ perf evlist  -v
> /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1
> [acme@felicio linux]$ perf evlist
> /tmp/foo.o
> [acme@felicio linux]$
>
> So, type 1 is PERF_TYPE_SOFTWARE, config 0x9 is PERF_COUNT_SW_DUMMY, ok.

You won't see this dummy event after patch 'perf tools: Collect perf_evsel
in BPF object files'. I use dummy event here so we can test basic parsing
and loading before that enabler patch to avoid enable too much code by
one patch.

Thank you, and glad to see you start working on this patch set again!

> And it behaves accordingly, no samples, etc.
>
> Added the sequence of testing with a non-existing file, with a normal ELF file
> and then with a valid eBPF ELF file as committer notes.
>
> Continuing.
>
> - Arnaldo

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Wangnan (F)]

On 2015/10/21 3:12, Arnaldo Carvalho de Melo wrote:
> Em Wed, Oct 14, 2015 at 12:41:15PM +0000, Wang Nan escreveu:
>> This patch introduces bpf__{un,}probe() functions to enable callers to
>> create kprobe points based on section names a BPF program. It parses
>> the section names in the program and creates corresponding 'struct
>> perf_probe_event' structures. The parse_perf_probe_command() function is
>> used to do the main parsing work. The resuling 'struct perf_probe_event'
>> is stored into program private data for further using.
>>
>> By utilizing the new probing API, this patch creates probe points during
>> event parsing.
>>
>> To ensure probe points be removed correctly, register an atexit hook
>> so even perf quit through exit() bpf__clear() is still called, so probing
>> points are cleared. Note that bpf_clear() should be registered before
>> bpf__probe() is called, so failure of bpf__probe() can still trigger
>> bpf__clear() to remove probe points which are already probed.
>>
>> strerror style error reporting scaffold is created by this patch.
>> bpf__strerror_probe() is the first error reporting function in bpf-loader.c.
> So, this one, for a non-root user gives me:
>
> [acme@felicio linux]$ perf record --event /tmp/foo.o sleep 1
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [acme@felicio linux]$

It should be:

  # perf record -e ./test_config_map.o ls
  Failed to init vmlinux path.
  event syntax error: './test_config_map.o'
                       \___ You need to be root, and 
/proc/sys/kernel/kptr_restrict should be 0


  (add -v to see detail)
  Run 'perf list' for a list of valid events

   usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list 
available events

So this is another problem related to probing. I'll have a look at your lot.

Thank you.

> --------------------
>
> I.e. no libbpf error (good!) but then, just an -EINVAL as the "event syntax
> error", which clearly isn't a syntax error, we need to tell the user that he or she
> needs special perfmissions for using sys_bpf() :-)
>
> As root:
>
> [root@felicio ~]# perf record --event /tmp/foo.o sleep
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]# ls -la /tmp/foo.o
> -rw-rw-r--. 1 acme acme 824 Oct 20 12:35 /tmp/foo.o
> [root@felicio ~]# file /tmp/foo.o
> /tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
>
>
> Humm, its something else, this is an ancient kernel, 4.2.0, probably without
> eBPF support? Nope, its there:
>
> [root@felicio ~]# grep -i sys_bpf /proc/kallsyms
> ffffffff811829d0 T SyS_bpf
> ffffffff811829d0 T sys_bpf
> [root@felicio ~]#
>
> Its something else, we need to improve this error reporting:
>
> [root@felicio ~]# perf record -v --event /tmp/foo.o sleep 1
> libbpf: loading /tmp/foo.o
> libbpf: section .strtab, size 60, link 0, flags 0, type=3
> libbpf: section .text, size 0, link 0, flags 6, type=1
> libbpf: section .data, size 0, link 0, flags 3, type=1
> libbpf: section .bss, size 0, link 0, flags 3, type=8
> libbpf: section do_fork, size 16, link 0, flags 6, type=1
> libbpf: found program do_fork
> libbpf: section license, size 4, link 0, flags 3, type=1
> libbpf: license of /tmp/foo.o is GPL
> libbpf: section version, size 4, link 0, flags 3, type=1
> libbpf: kernel version of /tmp/foo.o is 40100
> libbpf: section .symtab, size 96, link 1, flags 0, type=2
> bpf: config program 'do_fork'
> symbol:do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> bpf: 'do_fork': event name is missing
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]#
>
> [root@felicio ~]# grep do_fork /proc/kallsyms
> ffffffff81099ab0 T _do_fork
> ffffffff81ccc800 d do_fork_test
> [root@felicio ~]#
>
> $ echo '__attribute__((section("_do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c - -target bpf -O2 -o /tmp/foo.o
>
> [root@felicio ~]# perf record  -v --event /tmp/foo.o sleep 1
> libbpf: loading /tmp/foo.o
> libbpf: section .strtab, size 61, link 0, flags 0, type=3
> libbpf: section .text, size 0, link 0, flags 6, type=1
> libbpf: section .data, size 0, link 0, flags 3, type=1
> libbpf: section .bss, size 0, link 0, flags 3, type=8
> libbpf: section _do_fork, size 16, link 0, flags 6, type=1
> libbpf: found program _do_fork
> libbpf: section license, size 4, link 0, flags 3, type=1
> libbpf: license of /tmp/foo.o is GPL
> libbpf: section version, size 4, link 0, flags 3, type=1
> libbpf: kernel version of /tmp/foo.o is 40100
> libbpf: section .symtab, size 96, link 1, flags 0, type=2
> bpf: config program '_do_fork'
> symbol:_do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> bpf: '_do_fork': event name is missing
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]#
>
> So it still doesn't work, doesn't look like it is trying to find a vmlinux,
> will look at another patch IIRC is in this patchkit allowing us to tell
> 'perf record' where to find it... But it can as well use kallsyms...
>
> - Arnaldo

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Wangnan (F)]

On 2015/10/21 3:12, Arnaldo Carvalho de Melo wrote:
> Em Wed, Oct 14, 2015 at 12:41:15PM +0000, Wang Nan escreveu:
>> This patch introduces bpf__{un,}probe() functions to enable callers to
>> create kprobe points based on section names a BPF program. It parses
>> the section names in the program and creates corresponding 'struct
>> perf_probe_event' structures. The parse_perf_probe_command() function is
>> used to do the main parsing work. The resuling 'struct perf_probe_event'
>> is stored into program private data for further using.
>>
>> By utilizing the new probing API, this patch creates probe points during
>> event parsing.
>>
>> To ensure probe points be removed correctly, register an atexit hook
>> so even perf quit through exit() bpf__clear() is still called, so probing
>> points are cleared. Note that bpf_clear() should be registered before
>> bpf__probe() is called, so failure of bpf__probe() can still trigger
>> bpf__clear() to remove probe points which are already probed.
>>
>> strerror style error reporting scaffold is created by this patch.
>> bpf__strerror_probe() is the first error reporting function in bpf-loader.c.
> So, this one, for a non-root user gives me:
>
> [acme@felicio linux]$ perf record --event /tmp/foo.o sleep 1
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [acme@felicio linux]$
>
> --------------------
>
> I.e. no libbpf error (good!) but then, just an -EINVAL as the "event syntax
> error", which clearly isn't a syntax error, we need to tell the user that he or she
> needs special perfmissions for using sys_bpf() :-)
>
> As root:
>
> [root@felicio ~]# perf record --event /tmp/foo.o sleep
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]# ls -la /tmp/foo.o
> -rw-rw-r--. 1 acme acme 824 Oct 20 12:35 /tmp/foo.o
> [root@felicio ~]# file /tmp/foo.o
> /tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
>
>
> Humm, its something else, this is an ancient kernel, 4.2.0, probably without
> eBPF support? Nope, its there:
>
> [root@felicio ~]# grep -i sys_bpf /proc/kallsyms
> ffffffff811829d0 T SyS_bpf
> ffffffff811829d0 T sys_bpf
> [root@felicio ~]#
>
> Its something else, we need to improve this error reporting:
>
> [root@felicio ~]# perf record -v --event /tmp/foo.o sleep 1
> libbpf: loading /tmp/foo.o
> libbpf: section .strtab, size 60, link 0, flags 0, type=3
> libbpf: section .text, size 0, link 0, flags 6, type=1
> libbpf: section .data, size 0, link 0, flags 3, type=1
> libbpf: section .bss, size 0, link 0, flags 3, type=8
> libbpf: section do_fork, size 16, link 0, flags 6, type=1
> libbpf: found program do_fork
> libbpf: section license, size 4, link 0, flags 3, type=1
> libbpf: license of /tmp/foo.o is GPL
> libbpf: section version, size 4, link 0, flags 3, type=1
> libbpf: kernel version of /tmp/foo.o is 40100
> libbpf: section .symtab, size 96, link 1, flags 0, type=2
> bpf: config program 'do_fork'
> symbol:do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> bpf: 'do_fork': event name is missing

BPF report the problem, but it is a little bit hard to understand...

> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]#
>
> [root@felicio ~]# grep do_fork /proc/kallsyms
> ffffffff81099ab0 T _do_fork
> ffffffff81ccc800 d do_fork_test
> [root@felicio ~]#
>
> $ echo '__attribute__((section("_do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c - -target bpf -O2 -o /tmp/foo.o

In your program you only provide "do_fork", but we need "key=value" syntax.
"key" will become the name of created kprobe. Please try 
"__attribute__((section("func=do_fork"), used)) "
instead.

I think when event name is missing we'd better construct one name for it
like perf probe, but then we need to deal with perf probe code again. It
may require another patch.

For this patch, I think we can assign a new errorno so bpf__strerror_probe()
can give more information to let user know whether the problem is reside 
in bpf
program or perf configuration. Do you think ENOEXEC is a good choice?

Thank you.

>
> [root@felicio ~]# perf record  -v --event /tmp/foo.o sleep 1
> libbpf: loading /tmp/foo.o
> libbpf: section .strtab, size 61, link 0, flags 0, type=3
> libbpf: section .text, size 0, link 0, flags 6, type=1
> libbpf: section .data, size 0, link 0, flags 3, type=1
> libbpf: section .bss, size 0, link 0, flags 3, type=8
> libbpf: section _do_fork, size 16, link 0, flags 6, type=1
> libbpf: found program _do_fork
> libbpf: section license, size 4, link 0, flags 3, type=1
> libbpf: license of /tmp/foo.o is GPL
> libbpf: section version, size 4, link 0, flags 3, type=1
> libbpf: kernel version of /tmp/foo.o is 40100
> libbpf: section .symtab, size 96, link 1, flags 0, type=2
> bpf: config program '_do_fork'
> symbol:_do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> bpf: '_do_fork': event name is missing
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> [root@felicio ~]#
>
> So it still doesn't work, doesn't look like it is trying to find a vmlinux,
> will look at another patch IIRC is in this patchkit allowing us to tell
> 'perf record' where to find it... But it can as well use kallsyms...
>
> - Arnaldo

Re: [PATCH 26/31] perf tools: Support perf event alias name

[Namhyung Kim]

Hi,

On Wed, Oct 14, 2015 at 12:41:37PM +0000, Wang Nan wrote:
> From: He Kuang <hekuang@huawei.com>
> 
> This patch adds new bison rules for specifying an alias name to a perf
> event, which allows cmdline refer to previous defined perf event through
> its name. With this patch user can give alias name to a perf event using
> following cmdline:
> 
>  # perf record -e mypmu=cycles ...
> 
> To allow parser refer to existing event selecter, pass event list to
> 'struct parse_events_evlist'. perf_evlist__find_evsel_by_alias() is
> introduced to get evsel through its alias.

What about using event name directly?  I guess the alias name is used
only to refer an event so it'd be better to use the event name.
Anyway we need alias as well when event has no name or name is complex.

Thanks
Namhyung


> 
> Signed-off-by: He Kuang <hekuang@huawei.com>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Link: http://lkml.kernel.org/n/ebpf-7w1s62o0s6ovqlaqwrmx20v9@git.kernel.org
> ---
>  tools/perf/util/evlist.c       | 16 ++++++++++++++++
>  tools/perf/util/evlist.h       |  4 ++++
>  tools/perf/util/evsel.h        |  1 +
>  tools/perf/util/parse-events.c | 31 ++++++++++++++++++++++++++++---
>  tools/perf/util/parse-events.h |  5 +++++
>  tools/perf/util/parse-events.y | 15 ++++++++++++++-
>  6 files changed, 68 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
> index d139219..8dd59aa 100644
> --- a/tools/perf/util/evlist.c
> +++ b/tools/perf/util/evlist.c
> @@ -1753,3 +1753,19 @@ void perf_evlist__set_tracking_event(struct perf_evlist *evlist,
>  
>  	tracking_evsel->tracking = true;
>  }
> +
> +struct perf_evsel *
> +perf_evlist__find_evsel_by_alias(struct perf_evlist *evlist,
> +				 const char *alias)
> +{
> +	struct perf_evsel *evsel;
> +
> +	evlist__for_each(evlist, evsel) {
> +		if (!evsel->alias)
> +			continue;
> +		if (strcmp(alias, evsel->alias) == 0)
> +			return evsel;
> +	}
> +
> +	return NULL;
> +}
> diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
> index a459fe7..4e25342 100644
> --- a/tools/perf/util/evlist.h
> +++ b/tools/perf/util/evlist.h
> @@ -292,4 +292,8 @@ void perf_evlist__set_tracking_event(struct perf_evlist *evlist,
>  				     struct perf_evsel *tracking_evsel);
>  
>  void perf_event_attr__set_max_precise_ip(struct perf_event_attr *attr);
> +
> +struct perf_evsel *
> +perf_evlist__find_evsel_by_alias(struct perf_evlist *evlist, const char *alias);
> +
>  #endif /* __PERF_EVLIST_H */
> diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
> index a60b5d5..9a95e73 100644
> --- a/tools/perf/util/evsel.h
> +++ b/tools/perf/util/evsel.h
> @@ -87,6 +87,7 @@ struct perf_evsel {
>  	int			idx;
>  	u32			ids;
>  	char			*name;
> +	char			*alias;
>  	double			scale;
>  	const char		*unit;
>  	struct event_format	*tp_format;
> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
> index 4849dbd..06ba5a6 100644
> --- a/tools/perf/util/parse-events.c
> +++ b/tools/perf/util/parse-events.c
> @@ -1020,6 +1020,30 @@ int parse_events__modifier_group(struct list_head *list,
>  	return parse_events__modifier_event(list, event_mod, true);
>  }
>  
> +int parse_events__set_event_alias(struct parse_events_evlist *data,
> +				  struct list_head *list,
> +				  const char *str,
> +				  void *loc_alias_)
> +{
> +	struct perf_evsel *evsel;
> +	YYLTYPE *loc_alias = loc_alias_;
> +
> +	if (!str)
> +		return 0;
> +
> +	if (!list_is_singular(list)) {
> +		struct parse_events_error *err = data->error;
> +
> +		err->idx = loc_alias->first_column;
> +		err->str = strdup("One alias can be applied to one event only");
> +		return -EINVAL;
> +	}
> +
> +	evsel = list_first_entry(list, struct perf_evsel, node);
> +	evsel->alias = strdup(str);
> +	return evsel->alias ? 0 : -ENOMEM;
> +}
> +
>  void parse_events__set_leader(char *name, struct list_head *list)
>  {
>  	struct perf_evsel *leader;
> @@ -1373,9 +1397,10 @@ int parse_events(struct perf_evlist *evlist, const char *str,
>  		 struct parse_events_error *err)
>  {
>  	struct parse_events_evlist data = {
> -		.list  = LIST_HEAD_INIT(data.list),
> -		.idx   = evlist->nr_entries,
> -		.error = err,
> +		.list   = LIST_HEAD_INIT(data.list),
> +		.idx    = evlist->nr_entries,
> +		.error  = err,
> +		.evlist = evlist,
>  	};
>  	int ret;
>  
> diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
> index 8f17c83..b525353 100644
> --- a/tools/perf/util/parse-events.h
> +++ b/tools/perf/util/parse-events.h
> @@ -96,6 +96,7 @@ struct parse_events_evlist {
>  	int			   idx;
>  	int			   nr_groups;
>  	struct parse_events_error *error;
> +	struct perf_evlist	  *evlist;
>  };
>  
>  struct parse_events_terms {
> @@ -168,4 +169,8 @@ extern int is_valid_tracepoint(const char *event_string);
>  int valid_event_mount(const char *eventfs);
>  char *parse_events_formats_error_string(char *additional_terms);
>  
> +int parse_events__set_event_alias(struct parse_events_evlist *data,
> +				  struct list_head *list,
> +				  const char *str,
> +				  void *loc_alias_);
>  #endif /* __PERF_PARSE_EVENTS_H */
> diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
> index ad37996..90e382f 100644
> --- a/tools/perf/util/parse-events.y
> +++ b/tools/perf/util/parse-events.y
> @@ -76,6 +76,7 @@ static inc_group_count(struct list_head *list,
>  %type <head> event_bpf_file
>  %type <head> event_def
>  %type <head> event_mod
> +%type <head> event_alias
>  %type <head> event_name
>  %type <head> event
>  %type <head> events
> @@ -192,13 +193,25 @@ event_name PE_MODIFIER_EVENT
>  event_name
>  
>  event_name:
> -PE_EVENT_NAME event_def
> +PE_EVENT_NAME event_alias
>  {
>  	ABORT_ON(parse_events_name($2, $1));
>  	free($1);
>  	$$ = $2;
>  }
>  |
> +event_alias
> +
> +event_alias:
> +PE_NAME '=' event_def
> +{
> +	struct list_head *list = $3;
> +	struct parse_events_evlist *data = _data;
> +
> +	ABORT_ON(parse_events__set_event_alias(data, list, $1, &@1));
> +	$$ = list;
> +}
> +|
>  event_def
>  
>  event_def: event_pmu |
> -- 
> 1.8.3.4
> 

Re: [PATCH 26/31] perf tools: Support perf event alias name

[Wangnan (F)]

On 2015/10/21 16:53, Namhyung Kim wrote:
> Hi,
>
> On Wed, Oct 14, 2015 at 12:41:37PM +0000, Wang Nan wrote:
>> From: He Kuang <hekuang@huawei.com>
>>
>> This patch adds new bison rules for specifying an alias name to a perf
>> event, which allows cmdline refer to previous defined perf event through
>> its name. With this patch user can give alias name to a perf event using
>> following cmdline:
>>
>>   # perf record -e mypmu=cycles ...
>>
>> To allow parser refer to existing event selecter, pass event list to
>> 'struct parse_events_evlist'. perf_evlist__find_evsel_by_alias() is
>> introduced to get evsel through its alias.
> What about using event name directly?  I guess the alias name is used
> only to refer an event so it'd be better to use the event name.
> Anyway we need alias as well when event has no name or name is complex.

It is possible to trigger two perf events with same PMU but
different config options:

  # perf record -e cycles/period=9999/ -e cycles/period=99999/ -a sleep 1

In this case the name of events are:

cycles/period=9999/ `
cycles/period=99999/

Using it in perf cmdline is painful:

  # perf record -e cycles/period=9999/ -e cycles/period=99999/ -e 
bpf_program.c/myevent=cycles/period=9999//...

Thank you.

> Thanks
> Namhyung
>
>
>> Signed-off-by: He Kuang <hekuang@huawei.com>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Cc: Alexei Starovoitov <ast@plumgrid.com>
>> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
>> Cc: Daniel Borkmann <daniel@iogearbox.net>
>> Cc: David Ahern <dsahern@gmail.com>
>> Cc: He Kuang <hekuang@huawei.com>
>> Cc: Jiri Olsa <jolsa@kernel.org>
>> Cc: Kaixu Xia <xiakaixu@huawei.com>
>> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>> Cc: Namhyung Kim <namhyung@kernel.org>
>> Cc: Paul Mackerras <paulus@samba.org>
>> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
>> Cc: Zefan Li <lizefan@huawei.com>
>> Cc: pi3orama@163.com
>> Link: http://lkml.kernel.org/n/ebpf-7w1s62o0s6ovqlaqwrmx20v9@git.kernel.org
>> ---
>>   tools/perf/util/evlist.c       | 16 ++++++++++++++++
>>   tools/perf/util/evlist.h       |  4 ++++
>>   tools/perf/util/evsel.h        |  1 +
>>   tools/perf/util/parse-events.c | 31 ++++++++++++++++++++++++++++---
>>   tools/perf/util/parse-events.h |  5 +++++
>>   tools/perf/util/parse-events.y | 15 ++++++++++++++-
>>   6 files changed, 68 insertions(+), 4 deletions(-)
>>
>> diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
>> index d139219..8dd59aa 100644
>> --- a/tools/perf/util/evlist.c
>> +++ b/tools/perf/util/evlist.c
>> @@ -1753,3 +1753,19 @@ void perf_evlist__set_tracking_event(struct perf_evlist *evlist,
>>   
>>   	tracking_evsel->tracking = true;
>>   }
>> +
>> +struct perf_evsel *
>> +perf_evlist__find_evsel_by_alias(struct perf_evlist *evlist,
>> +				 const char *alias)
>> +{
>> +	struct perf_evsel *evsel;
>> +
>> +	evlist__for_each(evlist, evsel) {
>> +		if (!evsel->alias)
>> +			continue;
>> +		if (strcmp(alias, evsel->alias) == 0)
>> +			return evsel;
>> +	}
>> +
>> +	return NULL;
>> +}
>> diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
>> index a459fe7..4e25342 100644
>> --- a/tools/perf/util/evlist.h
>> +++ b/tools/perf/util/evlist.h
>> @@ -292,4 +292,8 @@ void perf_evlist__set_tracking_event(struct perf_evlist *evlist,
>>   				     struct perf_evsel *tracking_evsel);
>>   
>>   void perf_event_attr__set_max_precise_ip(struct perf_event_attr *attr);
>> +
>> +struct perf_evsel *
>> +perf_evlist__find_evsel_by_alias(struct perf_evlist *evlist, const char *alias);
>> +
>>   #endif /* __PERF_EVLIST_H */
>> diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
>> index a60b5d5..9a95e73 100644
>> --- a/tools/perf/util/evsel.h
>> +++ b/tools/perf/util/evsel.h
>> @@ -87,6 +87,7 @@ struct perf_evsel {
>>   	int			idx;
>>   	u32			ids;
>>   	char			*name;
>> +	char			*alias;
>>   	double			scale;
>>   	const char		*unit;
>>   	struct event_format	*tp_format;
>> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
>> index 4849dbd..06ba5a6 100644
>> --- a/tools/perf/util/parse-events.c
>> +++ b/tools/perf/util/parse-events.c
>> @@ -1020,6 +1020,30 @@ int parse_events__modifier_group(struct list_head *list,
>>   	return parse_events__modifier_event(list, event_mod, true);
>>   }
>>   
>> +int parse_events__set_event_alias(struct parse_events_evlist *data,
>> +				  struct list_head *list,
>> +				  const char *str,
>> +				  void *loc_alias_)
>> +{
>> +	struct perf_evsel *evsel;
>> +	YYLTYPE *loc_alias = loc_alias_;
>> +
>> +	if (!str)
>> +		return 0;
>> +
>> +	if (!list_is_singular(list)) {
>> +		struct parse_events_error *err = data->error;
>> +
>> +		err->idx = loc_alias->first_column;
>> +		err->str = strdup("One alias can be applied to one event only");
>> +		return -EINVAL;
>> +	}
>> +
>> +	evsel = list_first_entry(list, struct perf_evsel, node);
>> +	evsel->alias = strdup(str);
>> +	return evsel->alias ? 0 : -ENOMEM;
>> +}
>> +
>>   void parse_events__set_leader(char *name, struct list_head *list)
>>   {
>>   	struct perf_evsel *leader;
>> @@ -1373,9 +1397,10 @@ int parse_events(struct perf_evlist *evlist, const char *str,
>>   		 struct parse_events_error *err)
>>   {
>>   	struct parse_events_evlist data = {
>> -		.list  = LIST_HEAD_INIT(data.list),
>> -		.idx   = evlist->nr_entries,
>> -		.error = err,
>> +		.list   = LIST_HEAD_INIT(data.list),
>> +		.idx    = evlist->nr_entries,
>> +		.error  = err,
>> +		.evlist = evlist,
>>   	};
>>   	int ret;
>>   
>> diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
>> index 8f17c83..b525353 100644
>> --- a/tools/perf/util/parse-events.h
>> +++ b/tools/perf/util/parse-events.h
>> @@ -96,6 +96,7 @@ struct parse_events_evlist {
>>   	int			   idx;
>>   	int			   nr_groups;
>>   	struct parse_events_error *error;
>> +	struct perf_evlist	  *evlist;
>>   };
>>   
>>   struct parse_events_terms {
>> @@ -168,4 +169,8 @@ extern int is_valid_tracepoint(const char *event_string);
>>   int valid_event_mount(const char *eventfs);
>>   char *parse_events_formats_error_string(char *additional_terms);
>>   
>> +int parse_events__set_event_alias(struct parse_events_evlist *data,
>> +				  struct list_head *list,
>> +				  const char *str,
>> +				  void *loc_alias_);
>>   #endif /* __PERF_PARSE_EVENTS_H */
>> diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
>> index ad37996..90e382f 100644
>> --- a/tools/perf/util/parse-events.y
>> +++ b/tools/perf/util/parse-events.y
>> @@ -76,6 +76,7 @@ static inc_group_count(struct list_head *list,
>>   %type <head> event_bpf_file
>>   %type <head> event_def
>>   %type <head> event_mod
>> +%type <head> event_alias
>>   %type <head> event_name
>>   %type <head> event
>>   %type <head> events
>> @@ -192,13 +193,25 @@ event_name PE_MODIFIER_EVENT
>>   event_name
>>   
>>   event_name:
>> -PE_EVENT_NAME event_def
>> +PE_EVENT_NAME event_alias
>>   {
>>   	ABORT_ON(parse_events_name($2, $1));
>>   	free($1);
>>   	$$ = $2;
>>   }
>>   |
>> +event_alias
>> +
>> +event_alias:
>> +PE_NAME '=' event_def
>> +{
>> +	struct list_head *list = $3;
>> +	struct parse_events_evlist *data = _data;
>> +
>> +	ABORT_ON(parse_events__set_event_alias(data, list, $1, &@1));
>> +	$$ = list;
>> +}
>> +|
>>   event_def
>>   
>>   event_def: event_pmu |
>> -- 
>> 1.8.3.4
>>

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Arnaldo Carvalho de Melo]

Em Wed, Oct 21, 2015 at 11:31:57AM +0800, Wangnan (F) escreveu:
> 
> 
> On 2015/10/21 3:12, Arnaldo Carvalho de Melo wrote:
> >Em Wed, Oct 14, 2015 at 12:41:15PM +0000, Wang Nan escreveu:
> >>This patch introduces bpf__{un,}probe() functions to enable callers to
> >>create kprobe points based on section names a BPF program. It parses
> >>the section names in the program and creates corresponding 'struct
> >>perf_probe_event' structures. The parse_perf_probe_command() function is
> >>used to do the main parsing work. The resuling 'struct perf_probe_event'
> >>is stored into program private data for further using.
> >>
> >>By utilizing the new probing API, this patch creates probe points during
> >>event parsing.
> >>
> >>To ensure probe points be removed correctly, register an atexit hook
> >>so even perf quit through exit() bpf__clear() is still called, so probing
> >>points are cleared. Note that bpf_clear() should be registered before
> >>bpf__probe() is called, so failure of bpf__probe() can still trigger
> >>bpf__clear() to remove probe points which are already probed.
> >>
> >>strerror style error reporting scaffold is created by this patch.
> >>bpf__strerror_probe() is the first error reporting function in bpf-loader.c.
> >So, this one, for a non-root user gives me:
> >
> >[acme@felicio linux]$ perf record --event /tmp/foo.o sleep 1
> >event syntax error: '/tmp/foo.o'
> >                      \___ Invalid argument
> >
> >(add -v to see detail)
> >Run 'perf list' for a list of valid events
> >
> >  Usage: perf record [<options>] [<command>]
> >     or: perf record [<options>] -- <command> [<options>]
> >
> >     -e, --event <event>   event selector. use 'perf list' to list available events
> >[acme@felicio linux]$
> >
> >--------------------
> >
> >I.e. no libbpf error (good!) but then, just an -EINVAL as the "event syntax
> >error", which clearly isn't a syntax error, we need to tell the user that he or she
> >needs special perfmissions for using sys_bpf() :-)
> >
> >As root:
> >
> >[root@felicio ~]# perf record --event /tmp/foo.o sleep
> >event syntax error: '/tmp/foo.o'
> >                      \___ Invalid argument
> >
> >(add -v to see detail)
> >Run 'perf list' for a list of valid events
> >
> >  Usage: perf record [<options>] [<command>]
> >     or: perf record [<options>] -- <command> [<options>]
> >
> >     -e, --event <event>   event selector. use 'perf list' to list available events
> >[root@felicio ~]# ls -la /tmp/foo.o
> >-rw-rw-r--. 1 acme acme 824 Oct 20 12:35 /tmp/foo.o
> >[root@felicio ~]# file /tmp/foo.o
> >/tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
> >
> >
> >Humm, its something else, this is an ancient kernel, 4.2.0, probably without
> >eBPF support? Nope, its there:
> >
> >[root@felicio ~]# grep -i sys_bpf /proc/kallsyms
> >ffffffff811829d0 T SyS_bpf
> >ffffffff811829d0 T sys_bpf
> >[root@felicio ~]#
> >
> >Its something else, we need to improve this error reporting:
> >
> >[root@felicio ~]# perf record -v --event /tmp/foo.o sleep 1
> >libbpf: loading /tmp/foo.o
> >libbpf: section .strtab, size 60, link 0, flags 0, type=3
> >libbpf: section .text, size 0, link 0, flags 6, type=1
> >libbpf: section .data, size 0, link 0, flags 3, type=1
> >libbpf: section .bss, size 0, link 0, flags 3, type=8
> >libbpf: section do_fork, size 16, link 0, flags 6, type=1
> >libbpf: found program do_fork
> >libbpf: section license, size 4, link 0, flags 3, type=1
> >libbpf: license of /tmp/foo.o is GPL
> >libbpf: section version, size 4, link 0, flags 3, type=1
> >libbpf: kernel version of /tmp/foo.o is 40100
> >libbpf: section .symtab, size 96, link 1, flags 0, type=2
> >bpf: config program 'do_fork'
> >symbol:do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> >bpf: 'do_fork': event name is missing
> 
> BPF report the problem, but it is a little bit hard to understand...
> 
> >event syntax error: '/tmp/foo.o'
> >                      \___ Invalid argument
> >
> >(add -v to see detail)
> >Run 'perf list' for a list of valid events
> >
> >  Usage: perf record [<options>] [<command>]
> >     or: perf record [<options>] -- <command> [<options>]
> >
> >     -e, --event <event>   event selector. use 'perf list' to list available events
> >[root@felicio ~]#
> >
> >[root@felicio ~]# grep do_fork /proc/kallsyms
> >ffffffff81099ab0 T _do_fork
> >ffffffff81ccc800 d do_fork_test
> >[root@felicio ~]#
> >
> >$ echo '__attribute__((section("_do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c - -target bpf -O2 -o /tmp/foo.o
 
> In your program you only provide "do_fork", but we need "key=value"
> syntax.  "key" will become the name of created kprobe. Please try
> "__attribute__((section("func=do_fork"), used)) " instead.
 
> I think when event name is missing we'd better construct one name for
> it like perf probe, but then we need to deal with perf probe code
> again. It may require another patch.

Nah, lets go with what we have, i.e. I'll take that into account and
test with the expected form.
 
> For this patch, I think we can assign a new errorno so
> bpf__strerror_probe() can give more information to let user know
> whether the problem is reside in bpf program or perf configuration. Do
> you think ENOEXEC is a good choice?

Unsure if we should use existing errno codes in cases like this,
probably better to use a BPF_ERRNO__ENOALIAS or somesuch.

- Arnaldo

Re: [PATCH 26/31] perf tools: Support perf event alias name

[Namhyung Kim]

Hi,

On Wed, Oct 21, 2015 at 10:00 PM, Wangnan (F) <wangnan0@huawei.com> wrote:
>
>
> On 2015/10/21 16:53, Namhyung Kim wrote:
>>
>> Hi,
>>
>> On Wed, Oct 14, 2015 at 12:41:37PM +0000, Wang Nan wrote:
>>>
>>> From: He Kuang <hekuang@huawei.com>
>>>
>>> This patch adds new bison rules for specifying an alias name to a perf
>>> event, which allows cmdline refer to previous defined perf event through
>>> its name. With this patch user can give alias name to a perf event using
>>> following cmdline:
>>>
>>>   # perf record -e mypmu=cycles ...
>>>
>>> To allow parser refer to existing event selecter, pass event list to
>>> 'struct parse_events_evlist'. perf_evlist__find_evsel_by_alias() is
>>> introduced to get evsel through its alias.
>>
>> What about using event name directly?  I guess the alias name is used
>> only to refer an event so it'd be better to use the event name.
>> Anyway we need alias as well when event has no name or name is complex.
>
>
> It is possible to trigger two perf events with same PMU but
> different config options:
>
>  # perf record -e cycles/period=9999/ -e cycles/period=99999/ -a sleep 1
>
> In this case the name of events are:
>
> cycles/period=9999/ `
> cycles/period=99999/
>
> Using it in perf cmdline is painful:
>
>  # perf record -e cycles/period=9999/ -e cycles/period=99999/ -e
> bpf_program.c/myevent=cycles/period=9999//...

I understand the need of using aliases but I think it's more natural
to use event name for simple cases..

Thanks,
Namhyung

Re: [PATCH 26/31] perf tools: Support perf event alias name

[Wangnan (F)]

On 2015/10/22 15:16, Namhyung Kim wrote:
> Hi,
>
> On Wed, Oct 21, 2015 at 10:00 PM, Wangnan (F) <wangnan0@huawei.com> wrote:
>>
>> On 2015/10/21 16:53, Namhyung Kim wrote:
>>> Hi,
>>>
>>> On Wed, Oct 14, 2015 at 12:41:37PM +0000, Wang Nan wrote:
>>>> From: He Kuang <hekuang@huawei.com>
>>>>
>>>> This patch adds new bison rules for specifying an alias name to a perf
>>>> event, which allows cmdline refer to previous defined perf event through
>>>> its name. With this patch user can give alias name to a perf event using
>>>> following cmdline:
>>>>
>>>>    # perf record -e mypmu=cycles ...
>>>>
>>>> To allow parser refer to existing event selecter, pass event list to
>>>> 'struct parse_events_evlist'. perf_evlist__find_evsel_by_alias() is
>>>> introduced to get evsel through its alias.
>>> What about using event name directly?  I guess the alias name is used
>>> only to refer an event so it'd be better to use the event name.
>>> Anyway we need alias as well when event has no name or name is complex.
>>
>> It is possible to trigger two perf events with same PMU but
>> different config options:
>>
>>   # perf record -e cycles/period=9999/ -e cycles/period=99999/ -a sleep 1
>>
>> In this case the name of events are:
>>
>> cycles/period=9999/ `
>> cycles/period=99999/
>>
>> Using it in perf cmdline is painful:
>>
>>   # perf record -e cycles/period=9999/ -e cycles/period=99999/ -e
>> bpf_program.c/myevent=cycles/period=9999//...
> I understand the need of using aliases but I think it's more natural
> to use event name for simple cases..

I will consider this. However, if we allow using event name directly 
like this:

  # perf record -e cycles -e test_pmu.c/myevent=cycles/ ...

Then two '-e' seems redundant, right? Why not directly using:

  # perf record -e test_pmu.c/myevent=cycles/ ...

and make perf creates cycles event for test_pmu.c?

We can make syntax like

  # perf record -e test_pmu.c/myevent=cycles/ ...

as a syntax sugar of

  # perf record -e randomname=cycles -e test_pmu.c/myevent=randomname/ ...

and don't need to find evsel through their names if alias not exist.

So this is a new feature, and worth another patch.

Thought?

> Thanks,
> Namhyung

Re: [PATCH 26/31] perf tools: Support perf event alias name

[Namhyung Kim]

On Thu, Oct 22, 2015 at 4:29 PM, Wangnan (F) <wangnan0@huawei.com> wrote:
>
>
> On 2015/10/22 15:16, Namhyung Kim wrote:
>>
>> Hi,
>>
>> On Wed, Oct 21, 2015 at 10:00 PM, Wangnan (F) <wangnan0@huawei.com> wrote:
>>>
>>>
>>> On 2015/10/21 16:53, Namhyung Kim wrote:
>>>>
>>>> Hi,
>>>>
>>>> On Wed, Oct 14, 2015 at 12:41:37PM +0000, Wang Nan wrote:
>>>>>
>>>>> From: He Kuang <hekuang@huawei.com>
>>>>>
>>>>> This patch adds new bison rules for specifying an alias name to a perf
>>>>> event, which allows cmdline refer to previous defined perf event
>>>>> through
>>>>> its name. With this patch user can give alias name to a perf event
>>>>> using
>>>>> following cmdline:
>>>>>
>>>>>    # perf record -e mypmu=cycles ...
>>>>>
>>>>> To allow parser refer to existing event selecter, pass event list to
>>>>> 'struct parse_events_evlist'. perf_evlist__find_evsel_by_alias() is
>>>>> introduced to get evsel through its alias.
>>>>
>>>> What about using event name directly?  I guess the alias name is used
>>>> only to refer an event so it'd be better to use the event name.
>>>> Anyway we need alias as well when event has no name or name is complex.
>>>
>>>
>>> It is possible to trigger two perf events with same PMU but
>>> different config options:
>>>
>>>   # perf record -e cycles/period=9999/ -e cycles/period=99999/ -a sleep 1
>>>
>>> In this case the name of events are:
>>>
>>> cycles/period=9999/ `
>>> cycles/period=99999/
>>>
>>> Using it in perf cmdline is painful:
>>>
>>>   # perf record -e cycles/period=9999/ -e cycles/period=99999/ -e
>>> bpf_program.c/myevent=cycles/period=9999//...
>>
>> I understand the need of using aliases but I think it's more natural
>> to use event name for simple cases..
>
>
> I will consider this. However, if we allow using event name directly like
> this:
>
>  # perf record -e cycles -e test_pmu.c/myevent=cycles/ ...
>
> Then two '-e' seems redundant, right? Why not directly using:
>
>  # perf record -e test_pmu.c/myevent=cycles/ ...
>
> and make perf creates cycles event for test_pmu.c?
>
> We can make syntax like
>
>  # perf record -e test_pmu.c/myevent=cycles/ ...
>
> as a syntax sugar of
>
>  # perf record -e randomname=cycles -e test_pmu.c/myevent=randomname/ ...
>
> and don't need to find evsel through their names if alias not exist.
>
> So this is a new feature, and worth another patch.
>
> Thought?

Not sure it's worth.  It can confuse users IMHO.

Isn't it enough to give them in a single argument?

  # perf record -e cycles,test_pmu.c/myevent=cycles/

Thanks,
Namhyung

Re: [PATCH 26/31] perf tools: Support perf event alias name

[Wangnan (F)]

On 2015/10/22 15:53, Namhyung Kim wrote:
> On Thu, Oct 22, 2015 at 4:29 PM, Wangnan (F) <wangnan0@huawei.com> wrote:
>>

[SNIP]
>>> I understand the need of using aliases but I think it's more natural
>>> to use event name for simple cases..
>>
>> I will consider this. However, if we allow using event name directly like
>> this:
>>
>>   # perf record -e cycles -e test_pmu.c/myevent=cycles/ ...
>>
>> Then two '-e' seems redundant, right? Why not directly using:
>>
>>   # perf record -e test_pmu.c/myevent=cycles/ ...
>>
>> and make perf creates cycles event for test_pmu.c?
>>
>> We can make syntax like
>>
>>   # perf record -e test_pmu.c/myevent=cycles/ ...
>>
>> as a syntax sugar of
>>
>>   # perf record -e randomname=cycles -e test_pmu.c/myevent=randomname/ ...
>>
>> and don't need to find evsel through their names if alias not exist.
>>
>> So this is a new feature, and worth another patch.
>>
>> Thought?
> Not sure it's worth.  It can confuse users IMHO.
>
> Isn't it enough to give them in a single argument?
>
>    # perf record -e cycles,test_pmu.c/myevent=cycles/

OK. I have put it on my todo-list.

Thank you.
> Thanks,
> Namhyung

Re: [PATCH 04/31] perf record, bpf: Create probe points for BPF programs

[Arnaldo Carvalho de Melo]

Em Wed, Oct 21, 2015 at 10:28:03AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Oct 21, 2015 at 11:31:57AM +0800, Wangnan (F) escreveu:
> > >bpf: config program 'do_fork'
> > >symbol:do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
> > >bpf: 'do_fork': event name is missing
> > 
> > BPF report the problem, but it is a little bit hard to understand...
> > 
> > >event syntax error: '/tmp/foo.o'
> > >                      \___ Invalid argument
> > >
> > >(add -v to see detail)
> > >Run 'perf list' for a list of valid events
> > >
> > >  Usage: perf record [<options>] [<command>]
> > >     or: perf record [<options>] -- <command> [<options>]
> > >
> > >     -e, --event <event>   event selector. use 'perf list' to list available events
> > >[root@felicio ~]#
> > >
> > >[root@felicio ~]# grep do_fork /proc/kallsyms
> > >ffffffff81099ab0 T _do_fork
> > >ffffffff81ccc800 d do_fork_test
> > >[root@felicio ~]#
> > >
> > >$ echo '__attribute__((section("_do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c - -target bpf -O2 -o /tmp/foo.o
>  
> > In your program you only provide "do_fork", but we need "key=value"
> > syntax.  "key" will become the name of created kprobe. Please try
> > "__attribute__((section("func=do_fork"), used)) " instead.
>  
> > I think when event name is missing we'd better construct one name for
> > it like perf probe, but then we need to deal with perf probe code
> > again. It may require another patch.
> 
> Nah, lets go with what we have, i.e. I'll take that into account and
> test with the expected form.

So, pushed this cset with my committer testing notes:

https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/?h=perf/ebpf&id=bd3c477e57233819580f5824dea3a6922a860a24

continuing...

- Arnaldo

Re: [PATCH 05/31] perf record: Load eBPF object into kernel

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:16PM +0000, Wang Nan escreveu:
> This patch utilizes bpf_object__load() provided by libbpf to load all
> objects into kernel.

So, testing this, using that other proggie, I get;

  # perf record --event /tmp/foo.o sleep 1
  libbpf: load bpf program failed: Invalid argument
  libbpf: -- BEGIN DUMP LOG ---
  libbpf: 

  libbpf: -- END LOG --
  libbpf: failed to load program 'fork=_do_fork'
  libbpf: failed to load object '/tmp/foo.o'
  event syntax error: '/tmp/foo.o'
                       \___ Invalid argument: Are you root and runing a CONFIG_BPF_SYSCALL kernel?

  (add -v to see detail)
  Run 'perf list' for a list of valid events

   Usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list available events

But:

[root@felicio ~]# grep -i bpf /proc/kallsyms  | wc -l
117
[root@felicio ~]# grep -i sys_bpf /proc/kallsyms 
ffffffff811829d0 T SyS_bpf
ffffffff811829d0 T sys_bpf
[root@felicio ~]# uname -a
Linux felicio.ghostprotocols.net 4.2.0 #1 SMP Mon Aug 31 12:25:38 BRT
2015 x86_64 x86_64 x86_64 GNU/Linux
[root@felicio ~]# 


I'll try updating it to torvalds/master...

For reference, that foo.o eBPF proggie was built with:

[acme@felicio linux]$ cat ~/bin/hello-ebpf 
# Taken from 'perf test LLVM'
# Thu Oct 22 12:07:26 BRT 2015

export KERNEL_INC_OPTIONS="-nostdinc -isystem
/usr/lib/gcc/x86_64-redhat-linux/4.8.3/include
-I/home/acme/git/linux/arch/x86/include
-Iarch/x86/include/generated/uapi -Iarch/x86/include/generated
-I/home/acme/git/linux/include -Iinclude
-I/home/acme/git/linux/arch/x86/include/uapi
-Iarch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
-Iinclude/generated/uapi -include
/home/acme/git/linux/include/linux/kconfig.h"
export WORKING_DIR=/lib/modules/4.2.0/build
export CLANG_SOURCE=-
export CLANG_OPTIONS=-xc

OBJ=/tmp/foo.o
rm -f $OBJ
echo '__attribute__((section("fork=_do_fork"), used)) int fork(void
*ctx) {return 0;} char _license[] __attribute__((section("license"),
used)) = "GPL";int _version __attribute__((section("version"), used)) =
0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS
-Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c
"$CLANG_SOURCE" -target bpf -O2 -o /tmp/foo.o && file $OBJ

- Arnaldo

Re: [PATCH 05/31] perf record: Load eBPF object into kernel

[Arnaldo Carvalho de Melo]

Em Fri, Oct 23, 2015 at 01:58:40PM -0300, Arnaldo Carvalho de Melo escreveu:
> I'll try updating it to torvalds/master...

Ok, still the same problem, this time with -vv, what does the following message
mean?

libbpf: don't need create maps for /tmp/foo.o

Full output:

[acme@felicio linux]$ sudo ~/bin/perf record -vv --event /tmp/foo.o sleep 1
libbpf: loading /tmp/foo.o
libbpf: section .strtab, size 66, link 0, flags 0, type=3
libbpf: section .text, size 0, link 0, flags 6, type=1
libbpf: section .data, size 0, link 0, flags 3, type=1
libbpf: section .bss, size 0, link 0, flags 3, type=8
libbpf: section fork=_do_fork, size 16, link 0, flags 6, type=1
libbpf: found program fork=_do_fork
libbpf: section license, size 4, link 0, flags 3, type=1
libbpf: license of /tmp/foo.o is GPL
libbpf: section version, size 4, link 0, flags 3, type=1
libbpf: kernel version of /tmp/foo.o is 40100
libbpf: section .symtab, size 96, link 1, flags 0, type=2
bpf: config program 'fork=_do_fork'
symbol:_do_fork file:(null) line:0 offset:0 return:0 lazy:(null)
bpf: config 'fork=_do_fork' is ok
Blacklist: 0xffffffff81016f10-0xffffffff81016f50, do_device_not_available
Blacklist: 0xffffffff810171f0-0xffffffff810173e0, do_debug
Blacklist: 0xffffffff81016eb0-0xffffffff81016f10, fixup_bad_iret
Blacklist: 0xffffffff81016bd0-0xffffffff81016c00, sync_regs
Blacklist: 0xffffffff81016f50-0xffffffff810170a0, do_int3
Blacklist: 0xffffffff810170a0-0xffffffff810171f0, do_general_protection
Blacklist: 0xffffffff81016c00-0xffffffff81016d40, do_trap
Blacklist: 0xffffffff81019a10-0xffffffff81019af0, __die
Blacklist: 0xffffffff810198e0-0xffffffff810199b0, oops_end
Blacklist: 0xffffffff81019830-0xffffffff810198e0, oops_begin
Blacklist: 0xffffffff8101a770-0xffffffff8101a8c0, do_nmi
Blacklist: 0xffffffff8101a680-0xffffffff8101a770, default_do_nmi
Blacklist: 0xffffffff8101a560-0xffffffff8101a5f0, unknown_nmi_error
Blacklist: 0xffffffff8101a5f0-0xffffffff8101a680, io_check_error
Blacklist: 0xffffffff8101a4e0-0xffffffff8101a560, pci_serr_error
Blacklist: 0xffffffff8101a0d0-0xffffffff8101a1f0, nmi_handle
Blacklist: 0xffffffff81027a90-0xffffffff81027b10, debug_stack_reset
Blacklist: 0xffffffff81027a40-0xffffffff81027a90, debug_stack_set_zero
Blacklist: 0xffffffff81027a00-0xffffffff81027a40, is_debug_stack
Blacklist: 0xffffffff8102b400-0xffffffff8102b440, perf_event_nmi_handler
Blacklist: 0xffffffff8104b020-0xffffffff8104b060, perf_ibs_nmi_handler
Blacklist: 0xffffffff810526a0-0xffffffff810526c0, arch_trigger_all_cpu_backtrace_handler
Blacklist: 0xffffffff8105b090-0xffffffff8105b180, longjmp_break_handler
Blacklist: 0xffffffff8105ae21-0xffffffff8105ae30, jprobe_return_end
Blacklist: 0xffffffff8105ae00-0xffffffff8105ae21, jprobe_return
Blacklist: 0xffffffff8105ae70-0xffffffff8105af10, setjmp_pre_handler
Blacklist: 0xffffffff8105b040-0xffffffff8105b090, kprobe_exceptions_notify
Blacklist: 0xffffffff8105af10-0xffffffff8105b040, kprobe_fault_handler
Blacklist: 0xffffffff8105b8f0-0xffffffff8105b9c0, kprobe_debug_handler
Blacklist: 0xffffffff8105b720-0xffffffff8105b8f0, resume_execution
Blacklist: 0xffffffff8105b180-0xffffffff8105b380, trampoline_handler
Blacklist: 0xffffffff8105ada9-0xffffffff8105ae00, kretprobe_trampoline
Blacklist: 0xffffffff8105ada0-0xffffffff8105ada9, kretprobe_trampoline_holder
Blacklist: 0xffffffff8105b5c0-0xffffffff8105b720, kprobe_int3_handler
Blacklist: 0xffffffff8105b520-0xffffffff8105b5c0, reenter_kprobe
Blacklist: 0xffffffff8105b380-0xffffffff8105b520, setup_singlestep
Blacklist: 0xffffffff8105ad80-0xffffffff8105ada0, arch_prepare_kretprobe
Blacklist: 0xffffffff8105ae30-0xffffffff8105ae70, skip_prefixes
Blacklist: 0xffffffff8105ad60-0xffffffff8105ad80, synthesize_relcall
Blacklist: 0xffffffff8105ad40-0xffffffff8105ad60, synthesize_reljump
Blacklist: 0xffffffff8105c0c0-0xffffffff8105c110, setup_detour_execution
Blacklist: 0xffffffff8105c020-0xffffffff8105c0c0, optimized_callback
Blacklist: 0xffffffff8105c860-0xffffffff8105c970, kprobe_ftrace_handler
Blacklist: 0xffffffff8105c970-0xffffffff8105c9c0, skip_singlestep
Blacklist: 0xffffffff8105fec0-0xffffffff8105ff70, do_async_page_fault
Blacklist: 0xffffffff8105f820-0xffffffff8105f850, kvm_read_and_reset_pf_reason
Blacklist: 0xffffffff810607a0-0xffffffff810607b0, native_load_idt
Blacklist: 0xffffffff81060a30-0xffffffff81060a80, native_set_debugreg
Blacklist: 0xffffffff81060a80-0xffffffff81060ad0, native_get_debugreg
Blacklist: 0xffffffff81064f00-0xffffffff81064ff0, trace_do_page_fault
Blacklist: 0xffffffff81064e80-0xffffffff81064f00, do_page_fault
Blacklist: 0xffffffff81064a90-0xffffffff81064e80, __do_page_fault
Blacklist: 0xffffffff81064900-0xffffffff81064a90, spurious_fault
Blacklist: 0xffffffff81063e50-0xffffffff810640d0, vmalloc_fault
Blacklist: 0xffffffff810bb8d0-0xffffffff810bb930, notify_die
Blacklist: 0xffffffff810bb7f0-0xffffffff810bb810, atomic_notifier_call_chain
Blacklist: 0xffffffff810bb7d0-0xffffffff810bb7f0, __atomic_notifier_call_chain
Blacklist: 0xffffffff810bb760-0xffffffff810bb7d0, notifier_call_chain
Blacklist: 0xffffffff8113fb70-0xffffffff8113fbb0, dump_kprobe
Blacklist: 0xffffffff8113fdf0-0xffffffff8113ffa0, pre_handler_kretprobe
Blacklist: 0xffffffff81141290-0xffffffff81141360, cleanup_rp_inst
Blacklist: 0xffffffff811408a0-0xffffffff811409c0, kprobe_flush_task
Blacklist: 0xffffffff8113fce0-0xffffffff8113fd00, kretprobe_table_unlock
Blacklist: 0xffffffff8113fcb0-0xffffffff8113fce0, kretprobe_hash_unlock
Blacklist: 0xffffffff8113fc80-0xffffffff8113fcb0, kretprobe_table_lock
Blacklist: 0xffffffff8113fc40-0xffffffff8113fc80, kretprobe_hash_lock
Blacklist: 0xffffffff81140800-0xffffffff811408a0, recycle_rp_inst
Blacklist: 0xffffffff81140300-0xffffffff81140370, kprobes_inc_nmissed_count
Blacklist: 0xffffffff8113f840-0xffffffff8113f880, aggr_break_handler
Blacklist: 0xffffffff8113f800-0xffffffff8113f840, aggr_fault_handler
Blacklist: 0xffffffff8113fa30-0xffffffff8113fad0, aggr_post_handler
Blacklist: 0xffffffff8113f990-0xffffffff8113fa30, aggr_pre_handler
Blacklist: 0xffffffff8113f900-0xffffffff8113f990, opt_pre_handler
Blacklist: 0xffffffff8113f7b0-0xffffffff8113f800, get_kprobe
Blacklist: 0xffffffff81173100-0xffffffff811731e0, perf_trace_buf_prepare
Blacklist: 0xffffffff81179380-0xffffffff811793e0, kretprobe_dispatcher
Blacklist: 0xffffffff8117a650-0xffffffff8117a6a0, kprobe_dispatcher
Blacklist: 0xffffffff81178280-0xffffffff811784d0, kretprobe_perf_func
Blacklist: 0xffffffff81178030-0xffffffff81178280, kprobe_perf_func
Blacklist: 0xffffffff81178ff0-0xffffffff81179380, kretprobe_trace_func
Blacklist: 0xffffffff8117a2e0-0xffffffff8117a650, kprobe_trace_func
Blacklist: 0xffffffff81178e90-0xffffffff81178eb0, fetch_symbol_string_size
Blacklist: 0xffffffff81178010-0xffffffff81178030, fetch_symbol_string
Blacklist: 0xffffffff8117a210-0xffffffff8117a230, fetch_symbol_u64
Blacklist: 0xffffffff8117a2c0-0xffffffff8117a2e0, fetch_symbol_u32
Blacklist: 0xffffffff8117a160-0xffffffff8117a180, fetch_symbol_u16
Blacklist: 0xffffffff8117a0b0-0xffffffff8117a0d0, fetch_symbol_u8
Blacklist: 0xffffffff81178de0-0xffffffff81178e90, fetch_memory_string_size
Blacklist: 0xffffffff81177fc0-0xffffffff81178010, fetch_memory_string
Blacklist: 0xffffffff8117a180-0xffffffff8117a210, fetch_memory_u64
Blacklist: 0xffffffff8117a230-0xffffffff8117a2c0, fetch_memory_u32
Blacklist: 0xffffffff8117a0d0-0xffffffff8117a160, fetch_memory_u16
Blacklist: 0xffffffff8117a020-0xffffffff8117a0b0, fetch_memory_u8
Blacklist: 0xffffffff81177f90-0xffffffff81177fc0, fetch_stack_u64
Blacklist: 0xffffffff81177f60-0xffffffff81177f90, fetch_stack_u32
Blacklist: 0xffffffff81177f30-0xffffffff81177f60, fetch_stack_u16
Blacklist: 0xffffffff81177f00-0xffffffff81177f30, fetch_stack_u8
Blacklist: 0xffffffff8117e210-0xffffffff8117e220, fetch_user_stack_address
Blacklist: 0xffffffff8117e200-0xffffffff8117e210, fetch_kernel_stack_address
Blacklist: 0xffffffff8117e190-0xffffffff8117e200, fetch_bitfield_u64
Blacklist: 0xffffffff8117e120-0xffffffff8117e190, fetch_bitfield_u32
Blacklist: 0xffffffff8117e0b0-0xffffffff8117e120, fetch_bitfield_u16
Blacklist: 0xffffffff8117e040-0xffffffff8117e0b0, fetch_bitfield_u8
Blacklist: 0xffffffff8117e590-0xffffffff8117e630, free_deref_fetch_param
Blacklist: 0xffffffff8117e500-0xffffffff8117e590, update_deref_fetch_param
Blacklist: 0xffffffff8117dfc0-0xffffffff8117e040, fetch_deref_string_size
Blacklist: 0xffffffff8117df40-0xffffffff8117dfc0, fetch_deref_string
Blacklist: 0xffffffff8117dec0-0xffffffff8117df40, fetch_deref_u64
Blacklist: 0xffffffff8117de40-0xffffffff8117dec0, fetch_deref_u32
Blacklist: 0xffffffff8117ddc0-0xffffffff8117de40, fetch_deref_u16
Blacklist: 0xffffffff8117dd50-0xffffffff8117ddc0, fetch_deref_u8
Blacklist: 0xffffffff8117dd40-0xffffffff8117dd50, fetch_retval_u64
Blacklist: 0xffffffff8117dd30-0xffffffff8117dd40, fetch_retval_u32
Blacklist: 0xffffffff8117dd20-0xffffffff8117dd30, fetch_retval_u16
Blacklist: 0xffffffff8117dd10-0xffffffff8117dd20, fetch_retval_u8
Blacklist: 0xffffffff8117e4e0-0xffffffff8117e500, fetch_reg_u64
Blacklist: 0xffffffff8117e4c0-0xffffffff8117e4e0, fetch_reg_u32
Blacklist: 0xffffffff8117e4a0-0xffffffff8117e4c0, fetch_reg_u16
Blacklist: 0xffffffff8117e480-0xffffffff8117e4a0, fetch_reg_u8
Blacklist: 0xffffffff8117e420-0xffffffff8117e480, print_type_string
Blacklist: 0xffffffff8117e3e0-0xffffffff8117e420, print_type_s64
Blacklist: 0xffffffff8117e3a0-0xffffffff8117e3e0, print_type_s32
Blacklist: 0xffffffff8117e360-0xffffffff8117e3a0, print_type_s16
Blacklist: 0xffffffff8117e320-0xffffffff8117e360, print_type_s8
Blacklist: 0xffffffff8117e2e0-0xffffffff8117e320, print_type_u64
Blacklist: 0xffffffff8117e2a0-0xffffffff8117e2e0, print_type_u32
Blacklist: 0xffffffff8117e260-0xffffffff8117e2a0, print_type_u16
Blacklist: 0xffffffff8117e220-0xffffffff8117e260, print_type_u8
Blacklist: 0xffffffff8119a9a0-0xffffffff8119a9c0, context_tracking_user_exit
Blacklist: 0xffffffff8119a8d0-0xffffffff8119a9a0, context_tracking_exit
Blacklist: 0xffffffff8119aad0-0xffffffff8119aaf0, context_tracking_user_enter
Blacklist: 0xffffffff8119a9c0-0xffffffff8119aad0, context_tracking_enter
Blacklist: 0xffffffff8139f400-0xffffffff8139f470, nmi_cpu_backtrace
Looking at the vmlinux_path (7 entries long)
Using /lib/modules/4.3.0-rc6+/build/vmlinux for symbols
Open Debuginfo file: /lib/modules/4.3.0-rc6+/build/vmlinux
Try to find probe point from debuginfo.
Matched function: _do_fork
Probe point found: _do_fork+0
Found 1 probe_trace_events.
Opening /sys/kernel/debug/tracing//kprobe_events write=1
Writing event: p:perf_bpf_probe/fork _text+638512
libbpf: don't need create maps for /tmp/foo.o
libbpf: load bpf program failed: Invalid argument
libbpf: -- BEGIN DUMP LOG ---
libbpf: 

libbpf: -- END LOG --
libbpf: failed to load program 'fork=_do_fork'
libbpf: failed to load object '/tmp/foo.o'
bpf: load objects failed
event syntax error: '/tmp/foo.o'
                     \___ Invalid argument: Are you root and runing a CONFIG_BPF_SYSCALL kernel?

(add -v to see detail)
Run 'perf list' for a list of valid events

 Usage: perf record [<options>] [<command>]
    or: perf record [<options>] -- <command> [<options>]

    -e, --event <event>   event selector. use 'perf list' to list available events
Opening /sys/kernel/debug/tracing//kprobe_events write=1
Opening /sys/kernel/debug/tracing//uprobe_events write=1
Parsing probe_events: p:perf_bpf_probe/fork _text+638512
Group:perf_bpf_probe Event:fork probe:p
Writing event: -:perf_bpf_probe/fork
[acme@felicio linux]$ uname -a
Linux felicio.ghostprotocols.net 4.3.0-rc6+ #1 SMP Fri Oct 23 16:40:25 BRT 2015 x86_64 x86_64 x86_64 GNU/Linux
[acme@felicio linux]$ 

Re: [PATCH 05/31] perf record: Load eBPF object into kernel

[pi3orama]

发自我的 iPhone

> 在 2015年10月24日，上午12:58，Arnaldo Carvalho de Melo <acme@kernel.org> 写道：
> 
> Em Wed, Oct 14, 2015 at 12:41:16PM +0000, Wang Nan escreveu:
>> This patch utilizes bpf_object__load() provided by libbpf to load all
>> objects into kernel.
> 
> So, testing this, using that other proggie, I get;
> 
>  # perf record --event /tmp/foo.o sleep 1
>  libbpf: load bpf program failed: Invalid argument
>  libbpf: -- BEGIN DUMP LOG ---
>  libbpf: 
> 
>  libbpf: -- END LOG --

This means your kernel version and version section in your BPF object doesn't
match each other. Please try to pass a correct version code when compiling the
object. I think we should output something through error log so by -v we can see
the problem. I told Alexei about this problem once. Let me improve perf's
error message next week.

Thank you.


>  libbpf: failed to load program 'fork=_do_fork'
>  libbpf: failed to load object '/tmp/foo.o'
>  event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument: Are you root and runing a CONFIG_BPF_SYSCALL kernel?
> 
>  (add -v to see detail)
>  Run 'perf list' for a list of valid events
> 
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
> 
>      -e, --event <event>   event selector. use 'perf list' to list available events
> 
> But:
> 
> [root@felicio ~]# grep -i bpf /proc/kallsyms  | wc -l
> 117
> [root@felicio ~]# grep -i sys_bpf /proc/kallsyms 
> ffffffff811829d0 T SyS_bpf
> ffffffff811829d0 T sys_bpf
> [root@felicio ~]# uname -a
> Linux felicio.ghostprotocols.net 4.2.0 #1 SMP Mon Aug 31 12:25:38 BRT
> 2015 x86_64 x86_64 x86_64 GNU/Linux
> [root@felicio ~]# 
> 
> 
> I'll try updating it to torvalds/master...
> 
> For reference, that foo.o eBPF proggie was built with:
> 
> [acme@felicio linux]$ cat ~/bin/hello-ebpf 
> # Taken from 'perf test LLVM'
> # Thu Oct 22 12:07:26 BRT 2015
> 
> export KERNEL_INC_OPTIONS="-nostdinc -isystem
> /usr/lib/gcc/x86_64-redhat-linux/4.8.3/include
> -I/home/acme/git/linux/arch/x86/include
> -Iarch/x86/include/generated/uapi -Iarch/x86/include/generated
> -I/home/acme/git/linux/include -Iinclude
> -I/home/acme/git/linux/arch/x86/include/uapi
> -Iarch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
> -Iinclude/generated/uapi -include
> /home/acme/git/linux/include/linux/kconfig.h"
> export WORKING_DIR=/lib/modules/4.2.0/build
> export CLANG_SOURCE=-
> export CLANG_OPTIONS=-xc
> 
> OBJ=/tmp/foo.o
> rm -f $OBJ
> echo '__attribute__((section("fork=_do_fork"), used)) int fork(void
> *ctx) {return 0;} char _license[] __attribute__((section("license"),
> used)) = "GPL";int _version __attribute__((section("version"), used)) =
> 0x40100;' | clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS
> -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c
> "$CLANG_SOURCE" -target bpf -O2 -o /tmp/foo.o && file $OBJ
> 
> - Arnaldo

Re: [PATCH 05/31] perf record: Load eBPF object into kernel

[Wangnan (F)]

Hi Arnaldo,

On 2015/10/24 8:27, Arnaldo Carvalho de Melo wrote:
> libbpf: load bpf program failed: Invalid argument
> libbpf: -- BEGIN DUMP LOG ---
> libbpf:
>
> libbpf: -- END LOG --
> libbpf: failed to load program 'fork=_do_fork'
> libbpf: failed to load object '/tmp/foo.o'
> bpf: load objects failed
> event syntax error: '/tmp/foo.o'
>                       \___ Invalid argument: Are you root and runing a CONFIG_BPF_SYSCALL kernel?
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
>   Usage: perf record [<options>] [<command>]
>      or: perf record [<options>] -- <command> [<options>]
>
>      -e, --event <event>   event selector. use 'perf list' to list available events
> Opening /sys/kernel/debug/tracing//kprobe_events write=1
> Opening /sys/kernel/debug/tracing//uprobe_events write=1
> Parsing probe_events: p:perf_bpf_probe/fork _text+638512
> Group:perf_bpf_probe Event:fork probe:p
> Writing event: -:perf_bpf_probe/fork
> [acme@felicio linux]$ uname -a
> Linux felicio.ghostprotocols.net 4.3.0-rc6+ #1 SMP Fri Oct 23 16:40:25 BRT 2015 x86_64 x86_64 x86_64 GNU/Linux
> [acme@felicio linux]$
>
Please have a look at patch [1]. With that patch kernel will dump 
something into
log buffer. In your case, you will see:

root@sandybridge:~# /tmp/perf record -e /tmp/test_config_base.o ls
libbpf: load bpf program failed: Invalid argument
libbpf: -- BEGIN DUMP LOG ---
libbpf:
Kernel version mismatch: 0x40400 != 0x40300

libbpf: -- END LOG --
libbpf: failed to load program 'func_write=sys_write'
libbpf: failed to load object '/tmp/test_config_base.o'
event syntax error: '/tmp/test_config_base.o'
                      \___ Invalid argument: Are you root and runing a 
CONFIG_BPF_SYSCALL kernel?

(add -v to see detail)
Run 'perf list' for a list of valid events
...

And I'd like to improve the error message output by perf.

Do you want me to send a patch for fixing this problem or modifying 
existing patches
in my git tree?

Thank you.

[1] 
http://lkml.kernel.org/r/1445843588-143137-1-git-send-email-wangnan0@huawei.com

Re: [PATCH 20/31] perf tools: Allow BPF program attach to uprobe events

[Wangnan (F)]

On 2015/10/14 20:41, Wang Nan wrote:
> This patch appends new syntax to BPF object section name to support
> probing at uprobe event. Now we can use BPF program like this:
>
>   SEC(
>   "target=/lib64/libc.so.6\n"
>   "libcwrite=__write"
>   )
>   int libcwrite(void *ctx)
>   {
>       return 1;
>   }
>
> Where, in section name of a program, before the main config string,
> we can use 'key=value' style options. Now the only option key "target"
> is for uprobe probing.
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---
>   tools/perf/util/bpf-loader.c | 86 ++++++++++++++++++++++++++++++++++++++++----
>   1 file changed, 80 insertions(+), 6 deletions(-)
>
> diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
> index af549ea..73ff9a9 100644
> --- a/tools/perf/util/bpf-loader.c
> +++ b/tools/perf/util/bpf-loader.c
> @@ -111,6 +111,84 @@ bpf_prog_priv__clear(struct bpf_program *prog __maybe_unused,
>   }
>   
>   static int
> +do_config(const char *key, const char *value,
> +	  struct perf_probe_event *pev)
> +{
> +	pr_debug("config bpf program: %s=%s\n", key, value);
> +	if (strcmp(key, "target") == 0) {
> +		pev->uprobes = true;
> +		pev->target = strdup(value);
> +		return 0;
> +	}
> +
> +	pr_warning("BPF: WARNING: invalid config option in object: %s=%s\n",
> +		   key, value);
> +	pr_warning("\tHint: Currently only valid option is 'target=<file>'\n");
> +	return 0;
> +}
>

This part is very easy to be extended to support probing at modules.
I'd like to change "target" to "exec" to make it unify with perf probe,
then add another patch to support module probing in next pull request.

Thank you.

[PATCH] perf tools: Allow BPF program attach to modules

[Wang Nan]

By extending the syntax of BPF object section names, this patch allows
user to attach BPF programs to symbol in modules. For example:

 SEC("module=i915\n"
     "parse_cmds=i915_parse_cmds")
 int parse_cmds(void *ctx)
 {
     return 1;
 }

Implementation is very simple: like what 'perf probe' does, for module,
fill 'uprobe' field in 'struct perf_probe_event'. Other parts would be
done automatically.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/ebpf-x1mxko3tby054o573zd575qp@git.kernel.org
---

This patch is based on commit ad14ba45236a496a7ce25f4ea947d245b1406e40
in my git tree[1], and will be appear in the next pull request of this
perf eBPF support patchset.

[1] https://git.kernel.org/cgit/linux/kernel/git/pi3orama/linux.git

---
 tools/perf/util/bpf-loader.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index b14aff4..a85caae 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -120,10 +120,17 @@ do_config(const char *key, const char *value,
 		pev->target = strdup(value);
 		return 0;
 	}
+	if (strcmp(key, "module") == 0) {
+		pev->uprobes = false;
+		pev->target = strdup(value);
+		return 0;
+	}
 
 	pr_warning("BPF: WARNING: invalid config option in object: %s=%s\n",
 		   key, value);
-	pr_warning("\tHint: Currently only valid option is 'exec=<full patch of file>'\n");
+	pr_warning("\tHint: Currently valid options are:\n");
+	pr_warning("\t          'exec=<full patch of file>'\n");
+	pr_warning("\t          'module=<module name>'\n");
 	return 0;
 }
 
-- 
1.8.3.4

[tip:perf/core] perf tools: Make perf depend on libbpf

[tip-bot for Wang Nan]

Commit-ID:  ed63f34c026e9a60d17fa750ecdfe3f600d49393
Gitweb:     http://git.kernel.org/tip/ed63f34c026e9a60d17fa750ecdfe3f600d49393
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:12 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 28 Oct 2015 12:48:12 -0300

perf tools: Make perf depend on libbpf

By adding libbpf into perf's Makefile, this patch enables perf to build
libbpf if libelf is found and neither NO_LIBELF nor NO_LIBBPF is set.

The newly introduced code is similar to how libapi and libtraceevent
are wired into Makefile.perf.

MANIFEST is also updated for 'make perf-*-src-pkg'.

Append make_no_libbpf to tools/perf/tests/make.

The 'bpf' feature check is appended into default FEATURE_TESTS and
FEATURE_DISPLAY, so perf will check the API version of bpf in
/path/to/kernel/include/uapi/linux/bpf.h. Which should not fail except
when we are trying to port this code to an old kernel.

Error messages are also updated to notify users about the lack of BPF
support in 'perf record' if libelf is missing or the BPF API check
failed.

tools/lib/bpf is added to TAG_FOLDERS to allow us to navigate libbpf
files when working on perf using tools/perf/tags.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-2-git-send-email-wangnan0@huawei.com
[ Document NO_LIBBPF in Makefile.perf, noted by Jiri Olsa ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/build/Makefile.feature |  6 ++++--
 tools/perf/MANIFEST          |  3 +++
 tools/perf/Makefile.perf     | 21 +++++++++++++++++++--
 tools/perf/config/Makefile   | 19 ++++++++++++++++++-
 tools/perf/tests/make        |  4 +++-
 5 files changed, 47 insertions(+), 6 deletions(-)

diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index 72817e4..37ff4c9 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -53,7 +53,8 @@ FEATURE_TESTS ?=			\
 	libdw-dwarf-unwind		\
 	zlib				\
 	lzma				\
-	get_cpuid
+	get_cpuid			\
+	bpf
 
 FEATURE_DISPLAY ?=			\
 	dwarf				\
@@ -71,7 +72,8 @@ FEATURE_DISPLAY ?=			\
 	libdw-dwarf-unwind		\
 	zlib				\
 	lzma				\
-	get_cpuid
+	get_cpuid			\
+	bpf
 
 # Set FEATURE_CHECK_(C|LD)FLAGS-all for all FEATURE_TESTS features.
 # If in the future we need per-feature checks/flags for features not
diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index 9e6bdf5..39c38cb 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -17,6 +17,7 @@ tools/build
 tools/arch/x86/include/asm/atomic.h
 tools/arch/x86/include/asm/rmwcc.h
 tools/lib/traceevent
+tools/lib/bpf
 tools/lib/api
 tools/lib/bpf
 tools/lib/hweight.c
@@ -69,6 +70,8 @@ arch/*/lib/memset*.S
 include/linux/poison.h
 include/linux/hw_breakpoint.h
 include/uapi/linux/perf_event.h
+include/uapi/linux/bpf.h
+include/uapi/linux/bpf_common.h
 include/uapi/linux/const.h
 include/uapi/linux/swab.h
 include/uapi/linux/hw_breakpoint.h
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 56517d3..1e2e2d1 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -75,6 +75,8 @@ include config/utilities.mak
 # Define NO_LZMA if you do not want to support compressed (xz) kernel modules
 #
 # Define NO_AUXTRACE if you do not want AUX area tracing support
+#
+# Define NO_LIBBPF if you do not want BPF support
 
 # As per kernel Makefile, avoid funny character set dependencies
 unexport LC_ALL
@@ -145,6 +147,7 @@ AWK     = awk
 
 LIB_DIR          = $(srctree)/tools/lib/api/
 TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
+BPF_DIR = $(srctree)/tools/lib/bpf/
 
 # include config/Makefile by default and rule out
 # non-config cases
@@ -180,6 +183,7 @@ strip-libs = $(filter-out -l%,$(1))
 
 ifneq ($(OUTPUT),)
   TE_PATH=$(OUTPUT)
+  BPF_PATH=$(OUTPUT)
 ifneq ($(subdir),)
   LIB_PATH=$(OUTPUT)/../lib/api/
 else
@@ -188,6 +192,7 @@ endif
 else
   TE_PATH=$(TRACE_EVENT_DIR)
   LIB_PATH=$(LIB_DIR)
+  BPF_PATH=$(BPF_DIR)
 endif
 
 LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
@@ -199,6 +204,8 @@ LIBTRACEEVENT_DYNAMIC_LIST_LDFLAGS = -Xlinker --dynamic-list=$(LIBTRACEEVENT_DYN
 LIBAPI = $(LIB_PATH)libapi.a
 export LIBAPI
 
+LIBBPF = $(BPF_PATH)libbpf.a
+
 # python extension build directories
 PYTHON_EXTBUILD     := $(OUTPUT)python_ext_build/
 PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
@@ -251,6 +258,9 @@ export PERL_PATH
 LIB_FILE=$(OUTPUT)libperf.a
 
 PERFLIBS = $(LIB_FILE) $(LIBAPI) $(LIBTRACEEVENT)
+ifndef NO_LIBBPF
+  PERFLIBS += $(LIBBPF)
+endif
 
 # We choose to avoid "if .. else if .. else .. endif endif"
 # because maintaining the nesting to match is a pain.  If
@@ -420,6 +430,13 @@ $(LIBAPI)-clean:
 	$(call QUIET_CLEAN, libapi)
 	$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
 
+$(LIBBPF): FORCE
+	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) $(OUTPUT)libbpf.a
+
+$(LIBBPF)-clean:
+	$(call QUIET_CLEAN, libbpf)
+	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) clean >/dev/null
+
 help:
 	@echo 'Perf make targets:'
 	@echo '  doc		- make *all* documentation (see below)'
@@ -459,7 +476,7 @@ INSTALL_DOC_TARGETS += quick-install-doc quick-install-man quick-install-html
 $(DOC_TARGETS):
 	$(QUIET_SUBDIR0)Documentation $(QUIET_SUBDIR1) $(@:doc=all)
 
-TAG_FOLDERS= . ../lib/traceevent ../lib/api ../lib/symbol ../include
+TAG_FOLDERS= . ../lib/traceevent ../lib/api ../lib/symbol ../include ../lib/bpf
 TAG_FILES= ../../include/uapi/linux/perf_event.h
 
 TAGS:
@@ -567,7 +584,7 @@ config-clean:
 	$(call QUIET_CLEAN, config)
 	$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
 
-clean: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean config-clean
+clean: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean config-clean
 	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf-with-kcore $(LANG_BINDINGS)
 	$(Q)find . -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)$(RM) $(OUTPUT).config-detected
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index ab09ada..de89ec5 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -106,6 +106,7 @@ ifdef LIBBABELTRACE
   FEATURE_CHECK_LDFLAGS-libbabeltrace := $(LIBBABELTRACE_LDFLAGS) -lbabeltrace-ctf
 endif
 
+FEATURE_CHECK_CFLAGS-bpf = -I. -I$(srctree)/tools/include -I$(srctree)/arch/$(ARCH)/include/uapi -I$(srctree)/include/uapi
 # include ARCH specific config
 -include $(src-perf)/arch/$(ARCH)/Makefile
 
@@ -237,6 +238,7 @@ ifdef NO_LIBELF
   NO_DEMANGLE := 1
   NO_LIBUNWIND := 1
   NO_LIBDW_DWARF_UNWIND := 1
+  NO_LIBBPF := 1
 else
   ifeq ($(feature-libelf), 0)
     ifeq ($(feature-glibc), 1)
@@ -246,13 +248,14 @@ else
       LIBC_SUPPORT := 1
     endif
     ifeq ($(LIBC_SUPPORT),1)
-      msg := $(warning No libelf found, disables 'probe' tool, please install elfutils-libelf-devel/libelf-dev);
+      msg := $(warning No libelf found, disables 'probe' tool and BPF support in 'perf record', please install elfutils-libelf-devel/libelf-dev);
 
       NO_LIBELF := 1
       NO_DWARF := 1
       NO_DEMANGLE := 1
       NO_LIBUNWIND := 1
       NO_LIBDW_DWARF_UNWIND := 1
+      NO_LIBBPF := 1
     else
       ifneq ($(filter s% -static%,$(LDFLAGS),),)
         msg := $(error No static glibc found, please install glibc-static);
@@ -309,6 +312,13 @@ ifndef NO_LIBELF
       $(call detected,CONFIG_DWARF)
     endif # PERF_HAVE_DWARF_REGS
   endif # NO_DWARF
+
+  ifndef NO_LIBBPF
+    ifeq ($(feature-bpf), 1)
+      CFLAGS += -DHAVE_LIBBPF_SUPPORT
+      $(call detected,CONFIG_LIBBPF)
+    endif
+  endif # NO_LIBBPF
 endif # NO_LIBELF
 
 ifeq ($(ARCH),powerpc)
@@ -324,6 +334,13 @@ ifndef NO_LIBUNWIND
   endif
 endif
 
+ifndef NO_LIBBPF
+  ifneq ($(feature-bpf), 1)
+    msg := $(warning BPF API too old. Please install recent kernel headers. BPF support in 'perf record' is disabled.)
+    NO_LIBBPF := 1
+  endif
+endif
+
 dwarf-post-unwind := 1
 dwarf-post-unwind-text := BUG
 
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index ba31c4b..2cbd0c6 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -44,6 +44,7 @@ make_no_libnuma     := NO_LIBNUMA=1
 make_no_libaudit    := NO_LIBAUDIT=1
 make_no_libbionic   := NO_LIBBIONIC=1
 make_no_auxtrace    := NO_AUXTRACE=1
+make_no_libbpf	    := NO_LIBBPF=1
 make_tags           := tags
 make_cscope         := cscope
 make_help           := help
@@ -66,7 +67,7 @@ make_static         := LDFLAGS=-static
 make_minimal        := NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1
 make_minimal        += NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1
 make_minimal        += NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1
-make_minimal        += NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1
+make_minimal        += NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1
 
 # $(run) contains all available tests
 run := make_pure
@@ -94,6 +95,7 @@ run += make_no_libnuma
 run += make_no_libaudit
 run += make_no_libbionic
 run += make_no_auxtrace
+run += make_no_libbpf
 run += make_help
 run += make_doc
 run += make_perf_o

[tip:perf/core] perf ebpf: Add the libbpf glue

[tip-bot for Wang Nan]

Commit-ID:  69d262a93a25cf475012ea2e00aeb29f4932c028
Gitweb:     http://git.kernel.org/tip/69d262a93a25cf475012ea2e00aeb29f4932c028
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:13 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 28 Oct 2015 12:48:12 -0300

perf ebpf: Add the libbpf glue

The 'bpf-loader.[ch]' files are introduced in this patch. Which will be
the interface between perf and libbpf. bpf__prepare_load() resides in
bpf-loader.c. Following patches will enrich these two files.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/bpf-loader.c | 57 ++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.h | 29 ++++++++++++++++++++++
 2 files changed, 86 insertions(+)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
new file mode 100644
index 0000000..ab56073
--- /dev/null
+++ b/tools/perf/util/bpf-loader.c
@@ -0,0 +1,57 @@
+/*
+ * bpf-loader.c
+ *
+ * Copyright (C) 2015 Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2015 Huawei Inc.
+ */
+
+#include <bpf/libbpf.h>
+#include <linux/err.h>
+#include "perf.h"
+#include "debug.h"
+#include "bpf-loader.h"
+
+#define DEFINE_PRINT_FN(name, level) \
+static int libbpf_##name(const char *fmt, ...)	\
+{						\
+	va_list args;				\
+	int ret;				\
+						\
+	va_start(args, fmt);			\
+	ret = veprintf(level, verbose, pr_fmt(fmt), args);\
+	va_end(args);				\
+	return ret;				\
+}
+
+DEFINE_PRINT_FN(warning, 0)
+DEFINE_PRINT_FN(info, 0)
+DEFINE_PRINT_FN(debug, 1)
+
+struct bpf_object *bpf__prepare_load(const char *filename)
+{
+	struct bpf_object *obj;
+	static bool libbpf_initialized;
+
+	if (!libbpf_initialized) {
+		libbpf_set_print(libbpf_warning,
+				 libbpf_info,
+				 libbpf_debug);
+		libbpf_initialized = true;
+	}
+
+	obj = bpf_object__open(filename);
+	if (!obj) {
+		pr_debug("bpf: failed to load %s\n", filename);
+		return ERR_PTR(-EINVAL);
+	}
+
+	return obj;
+}
+
+void bpf__clear(void)
+{
+	struct bpf_object *obj, *tmp;
+
+	bpf_object__for_each_safe(obj, tmp)
+		bpf_object__close(obj);
+}
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
new file mode 100644
index 0000000..f402d7c
--- /dev/null
+++ b/tools/perf/util/bpf-loader.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright (C) 2015, Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2015, Huawei Inc.
+ */
+#ifndef __BPF_LOADER_H
+#define __BPF_LOADER_H
+
+#include <linux/compiler.h>
+#include <linux/err.h>
+#include <string.h>
+#include "debug.h"
+
+struct bpf_object;
+
+#ifdef HAVE_LIBBPF_SUPPORT
+struct bpf_object *bpf__prepare_load(const char *filename);
+
+void bpf__clear(void);
+#else
+static inline struct bpf_object *
+bpf__prepare_load(const char *filename __maybe_unused)
+{
+	pr_debug("ERROR: eBPF object loading is disabled during compiling.\n");
+	return ERR_PTR(-ENOTSUP);
+}
+
+static inline void bpf__clear(void) { }
+#endif
+#endif

[tip:perf/core] perf tools: Enable passing bpf object file to --event

[tip-bot for Wang Nan]

Commit-ID:  84c86ca12b2189df751eed7b2d67cb63bc8feda5
Gitweb:     http://git.kernel.org/tip/84c86ca12b2189df751eed7b2d67cb63bc8feda5
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:14 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 28 Oct 2015 12:48:12 -0300

perf tools: Enable passing bpf object file to --event

By introducing new rules in tools/perf/util/parse-events.[ly], this
patch enables 'perf record --event bpf_file.o' to select events by an
eBPF object file. It calls parse_events_load_bpf() to load that file,
which uses bpf__prepare_load() and finally calls bpf_object__open() for
the object files.

After applying this patch, commands like:

 # perf record --event foo.o sleep

become possible.

However, at this point it is unable to link any useful things onto the
evsel list because the creating of probe points and BPF program
attaching have not been implemented.  Before real events are possible to
be extracted, to avoid perf report error because of empty evsel list,
this patch link a dummy evsel. The dummy event related code will be
removed when probing and extracting code is ready.

Commiter notes:

Using it:

  $ ls -la foo.o
  ls: cannot access foo.o: No such file or directory
  $ perf record --event foo.o sleep
  libbpf: failed to open foo.o: No such file or directory
  event syntax error: 'foo.o'
                       \___ BPF object file 'foo.o' is invalid

  (add -v to see detail)
  Run 'perf list' for a list of valid events

   Usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list available events
  $

  $ file /tmp/build/perf/perf.o
  /tmp/build/perf/perf.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
  $ perf record --event /tmp/build/perf/perf.o sleep
  libbpf: /tmp/build/perf/perf.o is not an eBPF object file
  event syntax error: '/tmp/build/perf/perf.o'
                       \___ BPF object file '/tmp/build/perf/perf.o' is invalid

  (add -v to see detail)
  Run 'perf list' for a list of valid events

   Usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list available events
  $

  $ file /tmp/foo.o
  /tmp/foo.o: ELF 64-bit LSB relocatable, no machine, version 1 (SYSV), not stripped
  $ perf record --event /tmp/foo.o sleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.013 MB perf.data ]
  $ perf evlist
  /tmp/foo.o
  $ perf evlist  -v
  /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1
  $

So, type 1 is PERF_TYPE_SOFTWARE, config 0x9 is PERF_COUNT_SW_DUMMY, ok.

  $ perf report --stdio
  Error:
  The perf.data file has no samples!
  # To display the perf.data header info, please use --header/--header-only options.
  #
  $

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-4-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/perf.c              |  2 ++
 tools/perf/util/Build          |  1 +
 tools/perf/util/parse-events.c | 57 ++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/parse-events.h |  8 ++++++
 tools/perf/util/parse-events.l |  3 +++
 tools/perf/util/parse-events.y | 18 ++++++++++++-
 6 files changed, 88 insertions(+), 1 deletion(-)

diff --git a/tools/perf/perf.c b/tools/perf/perf.c
index 5437134..3d4c7c0 100644
--- a/tools/perf/perf.c
+++ b/tools/perf/perf.c
@@ -15,6 +15,7 @@
 #include "util/run-command.h"
 #include "util/parse-events.h"
 #include "util/parse-options.h"
+#include "util/bpf-loader.h"
 #include "util/debug.h"
 #include <api/fs/tracing_path.h>
 #include <pthread.h>
@@ -385,6 +386,7 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
 	status = p->fn(argc, argv, prefix);
 	exit_browser(status);
 	perf_env__exit(&perf_env);
+	bpf__clear();
 
 	if (status)
 		return status & 0xff;
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 9217119..591b3fe 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -87,6 +87,7 @@ libperf-$(CONFIG_AUXTRACE) += intel-bts.o
 libperf-y += parse-branch-options.o
 libperf-y += parse-regs-options.o
 
+libperf-$(CONFIG_LIBBPF) += bpf-loader.o
 libperf-$(CONFIG_LIBELF) += symbol-elf.o
 libperf-$(CONFIG_LIBELF) += probe-file.o
 libperf-$(CONFIG_LIBELF) += probe-event.o
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 72abcf2..a9e1d79 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -11,6 +11,7 @@
 #include "symbol.h"
 #include "cache.h"
 #include "header.h"
+#include "bpf-loader.h"
 #include "debug.h"
 #include <api/fs/tracing_path.h>
 #include "parse-events-bison.h"
@@ -529,6 +530,62 @@ static int add_tracepoint_multi_sys(struct list_head *list, int *idx,
 	return ret;
 }
 
+int parse_events_load_bpf_obj(struct parse_events_evlist *data,
+			      struct list_head *list,
+			      struct bpf_object *obj)
+{
+	int err;
+	char errbuf[BUFSIZ];
+
+	if (IS_ERR(obj) || !obj) {
+		snprintf(errbuf, sizeof(errbuf),
+			 "Internal error: load bpf obj with NULL");
+		err = -EINVAL;
+		goto errout;
+	}
+
+	/*
+	 * Temporary add a dummy event here so we can check whether
+	 * basic bpf loader works. Following patches will replace
+	 * dummy event by useful evsels.
+	 */
+	return parse_events_add_numeric(data, list, PERF_TYPE_SOFTWARE,
+					PERF_COUNT_SW_DUMMY, NULL);
+errout:
+	data->error->help = strdup("(add -v to see detail)");
+	data->error->str = strdup(errbuf);
+	return err;
+}
+
+int parse_events_load_bpf(struct parse_events_evlist *data,
+			  struct list_head *list,
+			  char *bpf_file_name)
+{
+	struct bpf_object *obj;
+
+	obj = bpf__prepare_load(bpf_file_name);
+	if (IS_ERR(obj) || !obj) {
+		char errbuf[BUFSIZ];
+		int err;
+
+		err = obj ? PTR_ERR(obj) : -EINVAL;
+
+		if (err == -ENOTSUP)
+			snprintf(errbuf, sizeof(errbuf),
+				 "BPF support is not compiled");
+		else
+			snprintf(errbuf, sizeof(errbuf),
+				 "BPF object file '%s' is invalid",
+				 bpf_file_name);
+
+		data->error->help = strdup("(add -v to see detail)");
+		data->error->str = strdup(errbuf);
+		return err;
+	}
+
+	return parse_events_load_bpf_obj(data, list, obj);
+}
+
 static int
 parse_breakpoint_type(const char *type, struct perf_event_attr *attr)
 {
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 13c9063..765018a 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -123,6 +123,14 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
 				char *sys, char *event,
 				struct parse_events_error *error,
 				struct list_head *head_config);
+int parse_events_load_bpf(struct parse_events_evlist *data,
+			  struct list_head *list,
+			  char *bpf_file_name);
+/* Provide this function for perf test */
+struct bpf_object;
+int parse_events_load_bpf_obj(struct parse_events_evlist *data,
+			      struct list_head *list,
+			      struct bpf_object *obj);
 int parse_events_add_numeric(struct parse_events_evlist *data,
 			     struct list_head *list,
 			     u32 type, u64 config,
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index 8d0de5b..cf330eb 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -115,6 +115,7 @@ do {							\
 group		[^,{}/]*[{][^}]*[}][^,{}/]*
 event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
 event		[^,{}/]+
+bpf_object	.*\.(o|bpf)
 
 num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
@@ -159,6 +160,7 @@ modifier_bp	[rwx]{1,3}
 		}
 
 {event_pmu}	|
+{bpf_object}	|
 {event}		{
 			BEGIN(INITIAL);
 			REWIND(1);
@@ -266,6 +268,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
 {num_hex}		{ return value(yyscanner, 16); }
 
 {modifier_event}	{ return str(yyscanner, PE_MODIFIER_EVENT); }
+{bpf_object}		{ return str(yyscanner, PE_BPF_OBJECT); }
 {name}			{ return pmu_str_check(yyscanner); }
 "/"			{ BEGIN(config); return '/'; }
 -			{ return '-'; }
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index ae6af26..497f19b 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -42,6 +42,7 @@ static inc_group_count(struct list_head *list,
 %token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
 %token PE_EVENT_NAME
 %token PE_NAME
+%token PE_BPF_OBJECT
 %token PE_MODIFIER_EVENT PE_MODIFIER_BP
 %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
 %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
@@ -53,6 +54,7 @@ static inc_group_count(struct list_head *list,
 %type <num> PE_RAW
 %type <num> PE_TERM
 %type <str> PE_NAME
+%type <str> PE_BPF_OBJECT
 %type <str> PE_NAME_CACHE_TYPE
 %type <str> PE_NAME_CACHE_OP_RESULT
 %type <str> PE_MODIFIER_EVENT
@@ -70,6 +72,7 @@ static inc_group_count(struct list_head *list,
 %type <tracepoint_name> tracepoint_name
 %type <head> event_legacy_numeric
 %type <head> event_legacy_raw
+%type <head> event_bpf_file
 %type <head> event_def
 %type <head> event_mod
 %type <head> event_name
@@ -203,7 +206,8 @@ event_def: event_pmu |
 	   event_legacy_mem |
 	   event_legacy_tracepoint sep_dc |
 	   event_legacy_numeric sep_dc |
-	   event_legacy_raw sep_dc
+	   event_legacy_raw sep_dc |
+	   event_bpf_file
 
 event_pmu:
 PE_NAME '/' event_config '/'
@@ -449,6 +453,18 @@ PE_RAW
 	$$ = list;
 }
 
+event_bpf_file:
+PE_BPF_OBJECT
+{
+	struct parse_events_evlist *data = _data;
+	struct parse_events_error *error = data->error;
+	struct list_head *list;
+
+	ALLOC_LIST(list);
+	ABORT_ON(parse_events_load_bpf(data, list, $1));
+	$$ = list;
+}
+
 start_terms: event_config
 {
 	struct parse_events_terms *data = _data;

[tip:perf/core] perf tools: Create probe points for BPF programs

[tip-bot for Wang Nan]

Commit-ID:  aa3abf30bb28addcf593578d37447d42e3f65fc3
Gitweb:     http://git.kernel.org/tip/aa3abf30bb28addcf593578d37447d42e3f65fc3
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:15 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 28 Oct 2015 12:48:13 -0300

perf tools: Create probe points for BPF programs

This patch introduces bpf__{un,}probe() functions to enable callers to
create kprobe points based on section names a BPF program. It parses the
section names in the program and creates corresponding 'struct
perf_probe_event' structures. The parse_perf_probe_command() function is
used to do the main parsing work. The resuling 'struct perf_probe_event'
is stored into program private data for further using.

By utilizing the new probing API, this patch creates probe points during
event parsing.

To ensure probe points be removed correctly, register an atexit hook so
even perf quit through exit() bpf__clear() is still called, so probing
points are cleared. Note that bpf_clear() should be registered before
bpf__probe() is called, so failure of bpf__probe() can still trigger
bpf__clear() to remove probe points which are already probed.

strerror style error reporting scaffold is created by this patch.
bpf__strerror_probe() is the first error reporting function in
bpf-loader.c.

Committer note:

Trying it:

To build a test eBPF object file:

I am testing using a script I built from the 'perf test -v LLVM' output:

  $ cat ~/bin/hello-ebpf
  export KERNEL_INC_OPTIONS="-nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/4.8.3/include -I/home/acme/git/linux/arch/x86/include -Iarch/x86/include/generated/uapi -Iarch/x86/include/generated -I/home/acme/git/linux/include -Iinclude -I/home/acme/git/linux/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -Iinclude/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h"
  export WORKING_DIR=/lib/modules/4.2.0/build
  export CLANG_SOURCE=-
  export CLANG_OPTIONS=-xc

  OBJ=/tmp/foo.o
  rm -f $OBJ
  echo '__attribute__((section("fork=do_fork"), used)) int fork(void *ctx) {return 0;} char _license[] __attribute__((section("license"), used)) = "GPL";int _version __attribute__((section("version"), used)) = 0x40100;' | \
  clang -D__KERNEL__ $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o /tmp/foo.o && file $OBJ

 ---

First asking to put a probe in a function not present in the kernel
(misses the initial _):

  $ perf record --event /tmp/foo.o sleep 1
  Probe point 'do_fork' not found.
  event syntax error: '/tmp/foo.o'
                       \___ You need to check probing points in BPF file

  (add -v to see detail)
  Run 'perf list' for a list of valid events

   Usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list available events
  $

 ---

Now, with "__attribute__((section("fork=_do_fork"), used)):

 $ grep _do_fork /proc/kallsyms
 ffffffff81099ab0 T _do_fork
 $ perf record --event /tmp/foo.o sleep 1
 Failed to open kprobe_events: Permission denied
 event syntax error: '/tmp/foo.o'
                      \___ Permission denied

 ---

Cool, we need to provide some better hints, "kprobe_events" is too low
level, one doesn't strictly need to know the precise details of how
these things are put in place, so something that shows the command
needed to fix the permissions would be more helpful.

Lets try as root instead:

  # perf record --event /tmp/foo.o sleep 1
  Lowering default frequency rate to 1000.
  Please consider tweaking /proc/sys/kernel/perf_event_max_sample_rate.
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.013 MB perf.data ]
  # perf evlist
  /tmp/foo.o
  [root@felicio ~]# perf evlist -v
  /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period,
  sample_freq }: 1000, sample_type: IP|TID|TIME|PERIOD, disabled: 1,
  inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1,
  sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1

 ---

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-5-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/bpf-loader.c   | 222 ++++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/bpf-loader.h   |  30 ++++++
 tools/perf/util/parse-events.c |  17 ++++
 3 files changed, 268 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index ab56073..56f6fe8 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -10,6 +10,8 @@
 #include "perf.h"
 #include "debug.h"
 #include "bpf-loader.h"
+#include "probe-event.h"
+#include "probe-finder.h" // for MAX_PROBES
 
 #define DEFINE_PRINT_FN(name, level) \
 static int libbpf_##name(const char *fmt, ...)	\
@@ -27,6 +29,10 @@ DEFINE_PRINT_FN(warning, 0)
 DEFINE_PRINT_FN(info, 0)
 DEFINE_PRINT_FN(debug, 1)
 
+struct bpf_prog_priv {
+	struct perf_probe_event pev;
+};
+
 struct bpf_object *bpf__prepare_load(const char *filename)
 {
 	struct bpf_object *obj;
@@ -52,6 +58,220 @@ void bpf__clear(void)
 {
 	struct bpf_object *obj, *tmp;
 
-	bpf_object__for_each_safe(obj, tmp)
+	bpf_object__for_each_safe(obj, tmp) {
+		bpf__unprobe(obj);
 		bpf_object__close(obj);
+	}
+}
+
+static void
+bpf_prog_priv__clear(struct bpf_program *prog __maybe_unused,
+		     void *_priv)
+{
+	struct bpf_prog_priv *priv = _priv;
+
+	cleanup_perf_probe_events(&priv->pev, 1);
+	free(priv);
+}
+
+static int
+config_bpf_program(struct bpf_program *prog)
+{
+	struct perf_probe_event *pev = NULL;
+	struct bpf_prog_priv *priv = NULL;
+	const char *config_str;
+	int err;
+
+	config_str = bpf_program__title(prog, false);
+	if (!config_str) {
+		pr_debug("bpf: unable to get title for program\n");
+		return -EINVAL;
+	}
+
+	priv = calloc(sizeof(*priv), 1);
+	if (!priv) {
+		pr_debug("bpf: failed to alloc priv\n");
+		return -ENOMEM;
+	}
+	pev = &priv->pev;
+
+	pr_debug("bpf: config program '%s'\n", config_str);
+	err = parse_perf_probe_command(config_str, pev);
+	if (err < 0) {
+		pr_debug("bpf: '%s' is not a valid config string\n",
+			 config_str);
+		err = -EINVAL;
+		goto errout;
+	}
+
+	if (pev->group && strcmp(pev->group, PERF_BPF_PROBE_GROUP)) {
+		pr_debug("bpf: '%s': group for event is set and not '%s'.\n",
+			 config_str, PERF_BPF_PROBE_GROUP);
+		err = -EINVAL;
+		goto errout;
+	} else if (!pev->group)
+		pev->group = strdup(PERF_BPF_PROBE_GROUP);
+
+	if (!pev->group) {
+		pr_debug("bpf: strdup failed\n");
+		err = -ENOMEM;
+		goto errout;
+	}
+
+	if (!pev->event) {
+		pr_debug("bpf: '%s': event name is missing\n",
+			 config_str);
+		err = -EINVAL;
+		goto errout;
+	}
+	pr_debug("bpf: config '%s' is ok\n", config_str);
+
+	err = bpf_program__set_private(prog, priv, bpf_prog_priv__clear);
+	if (err) {
+		pr_debug("Failed to set priv for program '%s'\n", config_str);
+		goto errout;
+	}
+
+	return 0;
+
+errout:
+	if (pev)
+		clear_perf_probe_event(pev);
+	free(priv);
+	return err;
+}
+
+static int bpf__prepare_probe(void)
+{
+	static int err = 0;
+	static bool initialized = false;
+
+	/*
+	 * Make err static, so if init failed the first, bpf__prepare_probe()
+	 * fails each time without calling init_probe_symbol_maps multiple
+	 * times.
+	 */
+	if (initialized)
+		return err;
+
+	initialized = true;
+	err = init_probe_symbol_maps(false);
+	if (err < 0)
+		pr_debug("Failed to init_probe_symbol_maps\n");
+	probe_conf.max_probes = MAX_PROBES;
+	return err;
+}
+
+int bpf__probe(struct bpf_object *obj)
+{
+	int err = 0;
+	struct bpf_program *prog;
+	struct bpf_prog_priv *priv;
+	struct perf_probe_event *pev;
+
+	err = bpf__prepare_probe();
+	if (err) {
+		pr_debug("bpf__prepare_probe failed\n");
+		return err;
+	}
+
+	bpf_object__for_each_program(prog, obj) {
+		err = config_bpf_program(prog);
+		if (err)
+			goto out;
+
+		err = bpf_program__get_private(prog, (void **)&priv);
+		if (err || !priv)
+			goto out;
+		pev = &priv->pev;
+
+		err = convert_perf_probe_events(pev, 1);
+		if (err < 0) {
+			pr_debug("bpf_probe: failed to convert perf probe events");
+			goto out;
+		}
+
+		err = apply_perf_probe_events(pev, 1);
+		if (err < 0) {
+			pr_debug("bpf_probe: failed to apply perf probe events");
+			goto out;
+		}
+	}
+out:
+	return err < 0 ? err : 0;
+}
+
+#define EVENTS_WRITE_BUFSIZE  4096
+int bpf__unprobe(struct bpf_object *obj)
+{
+	int err, ret = 0;
+	struct bpf_program *prog;
+	struct bpf_prog_priv *priv;
+
+	bpf_object__for_each_program(prog, obj) {
+		int i;
+
+		err = bpf_program__get_private(prog, (void **)&priv);
+		if (err || !priv)
+			continue;
+
+		for (i = 0; i < priv->pev.ntevs; i++) {
+			struct probe_trace_event *tev = &priv->pev.tevs[i];
+			char name_buf[EVENTS_WRITE_BUFSIZE];
+			struct strfilter *delfilter;
+
+			snprintf(name_buf, EVENTS_WRITE_BUFSIZE,
+				 "%s:%s", tev->group, tev->event);
+			name_buf[EVENTS_WRITE_BUFSIZE - 1] = '\0';
+
+			delfilter = strfilter__new(name_buf, NULL);
+			if (!delfilter) {
+				pr_debug("Failed to create filter for unprobing\n");
+				ret = -ENOMEM;
+				continue;
+			}
+
+			err = del_perf_probe_events(delfilter);
+			strfilter__delete(delfilter);
+			if (err) {
+				pr_debug("Failed to delete %s\n", name_buf);
+				ret = err;
+				continue;
+			}
+		}
+	}
+	return ret;
+}
+
+#define bpf__strerror_head(err, buf, size) \
+	char sbuf[STRERR_BUFSIZE], *emsg;\
+	if (!size)\
+		return 0;\
+	if (err < 0)\
+		err = -err;\
+	emsg = strerror_r(err, sbuf, sizeof(sbuf));\
+	switch (err) {\
+	default:\
+		scnprintf(buf, size, "%s", emsg);\
+		break;
+
+#define bpf__strerror_entry(val, fmt...)\
+	case val: {\
+		scnprintf(buf, size, fmt);\
+		break;\
+	}
+
+#define bpf__strerror_end(buf, size)\
+	}\
+	buf[size - 1] = '\0';
+
+int bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
+			int err, char *buf, size_t size)
+{
+	bpf__strerror_head(err, buf, size);
+	bpf__strerror_entry(EEXIST, "Probe point exist. Try use 'perf probe -d \"*\"'");
+	bpf__strerror_entry(EPERM, "You need to be root, and /proc/sys/kernel/kptr_restrict should be 0\n");
+	bpf__strerror_entry(ENOENT, "You need to check probing points in BPF file\n");
+	bpf__strerror_end(buf, size);
+	return 0;
 }
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index f402d7c..b819622 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -11,11 +11,18 @@
 #include "debug.h"
 
 struct bpf_object;
+#define PERF_BPF_PROBE_GROUP "perf_bpf_probe"
 
 #ifdef HAVE_LIBBPF_SUPPORT
 struct bpf_object *bpf__prepare_load(const char *filename);
 
 void bpf__clear(void);
+
+int bpf__probe(struct bpf_object *obj);
+int bpf__unprobe(struct bpf_object *obj);
+int bpf__strerror_probe(struct bpf_object *obj, int err,
+			char *buf, size_t size);
+
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused)
@@ -25,5 +32,28 @@ bpf__prepare_load(const char *filename __maybe_unused)
 }
 
 static inline void bpf__clear(void) { }
+
+static inline int bpf__probe(struct bpf_object *obj __maybe_unused) { return 0;}
+static inline int bpf__unprobe(struct bpf_object *obj __maybe_unused) { return 0;}
+
+static inline int
+__bpf_strerror(char *buf, size_t size)
+{
+	if (!size)
+		return 0;
+	strncpy(buf,
+		"ERROR: eBPF object loading is disabled during compiling.\n",
+		size);
+	buf[size - 1] = '\0';
+	return 0;
+}
+
+static inline int
+bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
+		    int err __maybe_unused,
+		    char *buf, size_t size)
+{
+	return __bpf_strerror(buf, size);
+}
 #endif
 #endif
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index a9e1d79..10a9467 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -536,6 +536,7 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 {
 	int err;
 	char errbuf[BUFSIZ];
+	static bool registered_unprobe_atexit = false;
 
 	if (IS_ERR(obj) || !obj) {
 		snprintf(errbuf, sizeof(errbuf),
@@ -545,6 +546,22 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 	}
 
 	/*
+	 * Register atexit handler before calling bpf__probe() so
+	 * bpf__probe() don't need to unprobe probe points its already
+	 * created when failure.
+	 */
+	if (!registered_unprobe_atexit) {
+		atexit(bpf__clear);
+		registered_unprobe_atexit = true;
+	}
+
+	err = bpf__probe(obj);
+	if (err) {
+		bpf__strerror_probe(obj, err, errbuf, sizeof(errbuf));
+		goto errout;
+	}
+
+	/*
 	 * Temporary add a dummy event here so we can check whether
 	 * basic bpf loader works. Following patches will replace
 	 * dummy event by useful evsels.

[tip:perf/core] perf tools: Load eBPF object into kernel

[tip-bot for Wang Nan]

Commit-ID:  1e5e3ee8ff3877db6943032b54a6ac21c095affd
Gitweb:     http://git.kernel.org/tip/1e5e3ee8ff3877db6943032b54a6ac21c095affd
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:16 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 28 Oct 2015 13:09:50 -0300

perf tools: Load eBPF object into kernel

This patch utilizes bpf_object__load() provided by libbpf to load all
objects into kernel.

Committer notes:

Testing it:

When using an incorrect kernel version number, i.e., having this in your
eBPF proggie:

  int _version __attribute__((section("version"), used)) = 0x40100;

For a 4.3.0-rc6+ kernel, say, this happens and needs checking at event
parsing time, to provide a better error report to the user:

  # perf record --event /tmp/foo.o sleep 1
  libbpf: load bpf program failed: Invalid argument
  libbpf: -- BEGIN DUMP LOG ---
  libbpf:

  libbpf: -- END LOG --
  libbpf: failed to load program 'fork=_do_fork'
  libbpf: failed to load object '/tmp/foo.o'
  event syntax error: '/tmp/foo.o'
                       \___ Invalid argument: Are you root and runing a CONFIG_BPF_SYSCALL kernel?

  (add -v to see detail)
  Run 'perf list' for a list of valid events

   Usage: perf record [<options>] [<command>]
      or: perf record [<options>] -- <command> [<options>]

      -e, --event <event>   event selector. use 'perf list' to list available events

If we instead make it match, i.e. use 0x40300 on this v4.3.0-rc6+
kernel, the whole process goes thru:

  # perf record --event /tmp/foo.o -a usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.202 MB perf.data ]
  # perf evlist -v
  /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period,
  sample_freq }: 4000, sample_type: IP|TID|TIME|CPU|PERIOD, disabled: 1,
  inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, sample_id_all: 1,
  exclude_guest: 1, mmap2: 1, comm_exec: 1
  #

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-6-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/bpf-loader.c   | 22 ++++++++++++++++++++++
 tools/perf/util/bpf-loader.h   | 11 +++++++++++
 tools/perf/util/parse-events.c |  6 ++++++
 3 files changed, 39 insertions(+)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index 56f6fe8..7279558 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -243,6 +243,18 @@ int bpf__unprobe(struct bpf_object *obj)
 	return ret;
 }
 
+int bpf__load(struct bpf_object *obj)
+{
+	int err;
+
+	err = bpf_object__load(obj);
+	if (err) {
+		pr_debug("bpf: load objects failed\n");
+		return err;
+	}
+	return 0;
+}
+
 #define bpf__strerror_head(err, buf, size) \
 	char sbuf[STRERR_BUFSIZE], *emsg;\
 	if (!size)\
@@ -275,3 +287,13 @@ int bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
 	bpf__strerror_end(buf, size);
 	return 0;
 }
+
+int bpf__strerror_load(struct bpf_object *obj __maybe_unused,
+		       int err, char *buf, size_t size)
+{
+	bpf__strerror_head(err, buf, size);
+	bpf__strerror_entry(EINVAL, "%s: Are you root and runing a CONFIG_BPF_SYSCALL kernel?",
+			    emsg)
+	bpf__strerror_end(buf, size);
+	return 0;
+}
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index b819622..b091ceb 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -23,6 +23,9 @@ int bpf__unprobe(struct bpf_object *obj);
 int bpf__strerror_probe(struct bpf_object *obj, int err,
 			char *buf, size_t size);
 
+int bpf__load(struct bpf_object *obj);
+int bpf__strerror_load(struct bpf_object *obj, int err,
+		       char *buf, size_t size);
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused)
@@ -35,6 +38,7 @@ static inline void bpf__clear(void) { }
 
 static inline int bpf__probe(struct bpf_object *obj __maybe_unused) { return 0;}
 static inline int bpf__unprobe(struct bpf_object *obj __maybe_unused) { return 0;}
+static inline int bpf__load(struct bpf_object *obj __maybe_unused) { return 0; }
 
 static inline int
 __bpf_strerror(char *buf, size_t size)
@@ -55,5 +59,12 @@ bpf__strerror_probe(struct bpf_object *obj __maybe_unused,
 {
 	return __bpf_strerror(buf, size);
 }
+
+static inline int bpf__strerror_load(struct bpf_object *obj __maybe_unused,
+				     int err __maybe_unused,
+				     char *buf, size_t size)
+{
+	return __bpf_strerror(buf, size);
+}
 #endif
 #endif
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 10a9467..c3aabeb 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -561,6 +561,12 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 		goto errout;
 	}
 
+	err = bpf__load(obj);
+	if (err) {
+		bpf__strerror_load(obj, err, errbuf, sizeof(errbuf));
+		goto errout;
+	}
+
 	/*
 	 * Temporary add a dummy event here so we can check whether
 	 * basic bpf loader works. Following patches will replace

[tip:perf/core] perf bpf: Collect perf_evsel in BPF object files

[tip-bot for Wang Nan]

Commit-ID:  4edf30e39e6cff32390eaff6a1508969b3cd967b
Gitweb:     http://git.kernel.org/tip/4edf30e39e6cff32390eaff6a1508969b3cd967b
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:17 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 28 Oct 2015 13:11:59 -0300

perf bpf: Collect perf_evsel in BPF object files

This patch creates a 'struct perf_evsel' for every probe in a BPF object
file(s) and fills 'struct evlist' with them. The previously introduced
dummy event is now removed. After this patch, the following command:

 # perf record --event filter.o ls

Can trace on each of the probes defined in filter.o.

The core of this patch is bpf__foreach_tev(), which calls a callback
function for each 'struct probe_trace_event' event for a bpf program
with each associated file descriptors. The add_bpf_event() callback
creates evsels by calling parse_events_add_tracepoint().

Since bpf-loader.c will not be built if libbpf is turned off, an empty
bpf__foreach_tev() is defined in bpf-loader.h to avoid build errors.

Committer notes:

Before:

  # /tmp/oldperf record --event /tmp/foo.o -a usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.198 MB perf.data ]
  # perf evlist
  /tmp/foo.o
  # perf evlist -v
  /tmp/foo.o: type: 1, size: 112, config: 0x9, { sample_period,
  sample_freq }: 4000, sample_type: IP|TID|TIME|CPU|PERIOD, disabled: 1,
  inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, sample_id_all: 1,
  exclude_guest: 1, mmap2: 1, comm_exec: 1

I.e. we create just the PERF_TYPE_SOFTWARE (type: 1),
PERF_COUNT_SW_DUMMY(config 0x9) event, now, with this patch:

  # perf record --event /tmp/foo.o -a usleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.210 MB perf.data ]
  # perf evlist -v
  perf_bpf_probe:fork: type: 2, size: 112, config: 0x6bd, { sample_period,
  sample_freq }: 1, sample_type: IP|TID|TIME|CPU|PERIOD|RAW, disabled: 1,
  inherit: 1, mmap: 1, comm: 1, task: 1, sample_id_all: 1, exclude_guest:
  1, mmap2: 1, comm_exec: 1
  #

We now have a PERF_TYPE_SOFTWARE (type: 1), but the config states 0x6bd,
which is how, after setting up the event via the kprobes interface, the
'perf_bpf_probe:fork' event is accessible via the perf_event_open
syscall. This is all transient, as soon as the 'perf record' session
ends, these probes will go away.

To see how it looks like, lets try doing a neverending session, one that
expects a control+C to end:

  # perf record --event /tmp/foo.o -a

So, with that in place, we can use 'perf probe' to see what is in place:

  # perf probe -l
    perf_bpf_probe:fork  (on _do_fork@acme/git/linux/kernel/fork.c)

We also can use debugfs:

  [root@felicio ~]# cat /sys/kernel/debug/tracing/kprobe_events
  p:perf_bpf_probe/fork _text+638512

Ok, now lets stop and see if we got some forks:

  [root@felicio linux]# perf record --event /tmp/foo.o -a
  ^C[ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.325 MB perf.data (111 samples) ]

  [root@felicio linux]# perf script
      sshd  1271 [003] 81797.507678: perf_bpf_probe:fork: (ffffffff8109be30)
      sshd 18309 [000] 81797.524917: perf_bpf_probe:fork: (ffffffff8109be30)
      sshd 18309 [001] 81799.381603: perf_bpf_probe:fork: (ffffffff8109be30)
      sshd 18309 [001] 81799.408635: perf_bpf_probe:fork: (ffffffff8109be30)
  <SNIP>

Sure enough, we have 111 forks :-)

Callchains seems to work as well:

  # perf report --stdio --no-child
  # To display the perf.data header info, please use --header/--header-only options.
  #
  # Total Lost Samples: 0
  #
  # Samples: 562  of event 'perf_bpf_probe:fork'
  # Event count (approx.): 562
  #
  # Overhead  Command   Shared Object     Symbol
  # ........  ........  ................  ............
  #
      44.66%  sh        [kernel.vmlinux]  [k] _do_fork
                    |
                    ---_do_fork
                       entry_SYSCALL_64_fastpath
                       __libc_fork
                       make_child

    26.16%  make      [kernel.vmlinux]  [k] _do_fork
<SNIP>
  #

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-7-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/bpf-loader.c   | 40 ++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.h   | 14 ++++++++++++
 tools/perf/util/parse-events.c | 52 ++++++++++++++++++++++++++++++++++++------
 3 files changed, 99 insertions(+), 7 deletions(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index 7279558..aa784a4 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -255,6 +255,46 @@ int bpf__load(struct bpf_object *obj)
 	return 0;
 }
 
+int bpf__foreach_tev(struct bpf_object *obj,
+		     bpf_prog_iter_callback_t func,
+		     void *arg)
+{
+	struct bpf_program *prog;
+	int err;
+
+	bpf_object__for_each_program(prog, obj) {
+		struct probe_trace_event *tev;
+		struct perf_probe_event *pev;
+		struct bpf_prog_priv *priv;
+		int i, fd;
+
+		err = bpf_program__get_private(prog,
+				(void **)&priv);
+		if (err || !priv) {
+			pr_debug("bpf: failed to get private field\n");
+			return -EINVAL;
+		}
+
+		pev = &priv->pev;
+		for (i = 0; i < pev->ntevs; i++) {
+			tev = &pev->tevs[i];
+
+			fd = bpf_program__fd(prog);
+			if (fd < 0) {
+				pr_debug("bpf: failed to get file descriptor\n");
+				return fd;
+			}
+
+			err = (*func)(tev, fd, arg);
+			if (err) {
+				pr_debug("bpf: call back failed, stop iterate\n");
+				return err;
+			}
+		}
+	}
+	return 0;
+}
+
 #define bpf__strerror_head(err, buf, size) \
 	char sbuf[STRERR_BUFSIZE], *emsg;\
 	if (!size)\
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index b091ceb..a8f25ee 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -8,11 +8,15 @@
 #include <linux/compiler.h>
 #include <linux/err.h>
 #include <string.h>
+#include "probe-event.h"
 #include "debug.h"
 
 struct bpf_object;
 #define PERF_BPF_PROBE_GROUP "perf_bpf_probe"
 
+typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
+					int fd, void *arg);
+
 #ifdef HAVE_LIBBPF_SUPPORT
 struct bpf_object *bpf__prepare_load(const char *filename);
 
@@ -26,6 +30,8 @@ int bpf__strerror_probe(struct bpf_object *obj, int err,
 int bpf__load(struct bpf_object *obj);
 int bpf__strerror_load(struct bpf_object *obj, int err,
 		       char *buf, size_t size);
+int bpf__foreach_tev(struct bpf_object *obj,
+		     bpf_prog_iter_callback_t func, void *arg);
 #else
 static inline struct bpf_object *
 bpf__prepare_load(const char *filename __maybe_unused)
@@ -41,6 +47,14 @@ static inline int bpf__unprobe(struct bpf_object *obj __maybe_unused) { return 0
 static inline int bpf__load(struct bpf_object *obj __maybe_unused) { return 0; }
 
 static inline int
+bpf__foreach_tev(struct bpf_object *obj __maybe_unused,
+		 bpf_prog_iter_callback_t func __maybe_unused,
+		 void *arg __maybe_unused)
+{
+	return 0;
+}
+
+static inline int
 __bpf_strerror(char *buf, size_t size)
 {
 	if (!size)
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index c3aabeb..d97b037 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -530,12 +530,49 @@ static int add_tracepoint_multi_sys(struct list_head *list, int *idx,
 	return ret;
 }
 
+struct __add_bpf_event_param {
+	struct parse_events_evlist *data;
+	struct list_head *list;
+};
+
+static int add_bpf_event(struct probe_trace_event *tev, int fd,
+			 void *_param)
+{
+	LIST_HEAD(new_evsels);
+	struct __add_bpf_event_param *param = _param;
+	struct parse_events_evlist *evlist = param->data;
+	struct list_head *list = param->list;
+	int err;
+
+	pr_debug("add bpf event %s:%s and attach bpf program %d\n",
+		 tev->group, tev->event, fd);
+
+	err = parse_events_add_tracepoint(&new_evsels, &evlist->idx, tev->group,
+					  tev->event, evlist->error, NULL);
+	if (err) {
+		struct perf_evsel *evsel, *tmp;
+
+		pr_debug("Failed to add BPF event %s:%s\n",
+			 tev->group, tev->event);
+		list_for_each_entry_safe(evsel, tmp, &new_evsels, node) {
+			list_del(&evsel->node);
+			perf_evsel__delete(evsel);
+		}
+		return err;
+	}
+	pr_debug("adding %s:%s\n", tev->group, tev->event);
+
+	list_splice(&new_evsels, list);
+	return 0;
+}
+
 int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 			      struct list_head *list,
 			      struct bpf_object *obj)
 {
 	int err;
 	char errbuf[BUFSIZ];
+	struct __add_bpf_event_param param = {data, list};
 	static bool registered_unprobe_atexit = false;
 
 	if (IS_ERR(obj) || !obj) {
@@ -567,13 +604,14 @@ int parse_events_load_bpf_obj(struct parse_events_evlist *data,
 		goto errout;
 	}
 
-	/*
-	 * Temporary add a dummy event here so we can check whether
-	 * basic bpf loader works. Following patches will replace
-	 * dummy event by useful evsels.
-	 */
-	return parse_events_add_numeric(data, list, PERF_TYPE_SOFTWARE,
-					PERF_COUNT_SW_DUMMY, NULL);
+	err = bpf__foreach_tev(obj, add_bpf_event, &param);
+	if (err) {
+		snprintf(errbuf, sizeof(errbuf),
+			 "Attach events in BPF object failed");
+		goto errout;
+	}
+
+	return 0;
 errout:
 	data->error->help = strdup("(add -v to see detail)");
 	data->error->str = strdup(errbuf);

Re: [PATCH 09/31] perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:20PM +0000, Wang Nan escreveu:
> This patch provides infrastructure for passing source files to --event
> directly using:
> 
>  # perf record --event bpf-file.c command
> 
> This patch does following works:
> 
>  1) Allow passing '.c' file to '--event'. parse_events_load_bpf() is
>     expanded to allow caller tell it whether the passed file is source
>     file or object.
> 
>  2) llvm__compile_bpf() is called to compile the '.c' file, the result
>     is saved into memory. Use bpf_object__open_buffer() to load the
>     in-memory object.
> 
> Introduces a bpf-script-example.c so we can manually test it:
> 
>  # perf record --clang-opt "-DLINUX_VERSION_CODE=0x40200" --event ./bpf-script-example.c sleep 1
> 
> Note that '--clang-opt' must put before '--event'.
> 
> Futher patches will merge it into a testcase so can be tested automatically.

Not working here... humm, perhaps the version again...

[root@felicio linux]# perf record --clang-opt "-DLINUX_VERSION_CODE=0x40200" --event tools/perf/tests/bpf-script-example.c sleep 1
libbpf: load bpf program failed: Invalid argument
libbpf: -- BEGIN DUMP LOG ---
libbpf: 

libbpf: -- END LOG --
libbpf: failed to load program 'func=sys_epoll_pwait'

Yeah, as soon as I used:

# perf record --clang-opt "-DLINUX_VERSION_CODE=0x40300" --event tools/perf/tests/bpf-script-example.c sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.013 MB perf.data ]
[root@felicio linux]# 

I think we should provide some stuff like this LINUX_VERSION_CODE
automagically, right?

And also check the version in a .o file, to make sure we don't ask the
kernel to load something with a mismatch in that version.

And you could've provided something that would actually do something
meaningful, i.e.  a filter that would capture some samples for a given event
while discarding something not that important...

I.e. show the value provided at the time we build that patch.

I'll try to come up with something like that after I have lunch, but so far
so good, I think I'll be able to send this to Ingo today, and that is a major
milestone!

Great work, thanks a lot for doing this!

- Arnaldo

P.S.: If we pass a .c ending string to 'trace', 'record', 'top', etc, I think
we could do away with that pesky --event/-e, i.e.:

	trace cool_stuff.c

Would do the right thing, i.e. be equivalent to:

	trace --event cool_stuff.c

Re: [PATCH 09/31] perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

[Arnaldo Carvalho de Melo]

Em Thu, Oct 29, 2015 at 01:25:53PM -0300, Arnaldo Carvalho de Melo escreveu:
> And you could've provided something that would actually do something
> meaningful, i.e.  a filter that would capture some samples for a given event
> while discarding something not that important...
> 
> I.e. show the value provided at the time we build that patch.
> 
> I'll try to come up with something like that after I have lunch.

Ok, what I have is at my perf/core branch, will continue after having
some food,

- Arnaldo

Re: [PATCH 10/31] perf test: Enforce LLVM test for BPF test

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:21PM +0000, Wang Nan escreveu:
> This patch replaces the original toy BPF program with previous introduced
> bpf-script-example.c. Dynamically embedded it into 'llvm-src.c'.
> 
> The newly introduced BPF program attaches a BPF program at
> 'sys_epoll_pwait()', and collect half samples from it. perf itself never

"collect half samples from it"? Can you rephrase this?

> use that syscall, so further test can verify their result with it.
> 
> Since BPF program require LINUX_VERSION_CODE of runtime kernel, this
> patch computes that code from uname.
> 
> Since the resuling BPF object is useful for further testcases, this patch
> introduces 'prepare' and 'cleanup' method to tests, and makes test__llvm()
> create a MAP_SHARED memory array to hold the resulting object.

I think you're doing lots of things in just one patch, please split it,
for instance, that ->prepare()/->cleanup() should stand on its own, with
a proper changelog, etc.

The LINUX_VERSION_CODE part should probably also be on its own patch,
working much like __FILE__ or __func__, i.e. gcc predefines those, the
bpf code that calls clang to build the provided .c file should do the
same for LINUX_VERSION_CODE, so that bpf scriptlets can rely on it being
defined.

I.e. that compose_source() function, but not just for the 'perf test'
entry, for any bpf scriptlet.
 
> Signed-off-by: He Kuang <hekuang@huawei.com>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---
>  tools/perf/tests/Build          |   9 +++-
>  tools/perf/tests/builtin-test.c |   6 +++
>  tools/perf/tests/llvm.c         | 104 +++++++++++++++++++++++++++++++++++-----
>  tools/perf/tests/llvm.h         |  14 ++++++
>  tools/perf/tests/tests.h        |   4 ++
>  5 files changed, 123 insertions(+), 14 deletions(-)
>  create mode 100644 tools/perf/tests/llvm.h
> 
> diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> index 50de225..4afc8c8 100644
> --- a/tools/perf/tests/Build
> +++ b/tools/perf/tests/Build
> @@ -31,9 +31,16 @@ perf-y += sample-parsing.o
>  perf-y += parse-no-sample-id-all.o
>  perf-y += kmod-path.o
>  perf-y += thread-map.o
> -perf-y += llvm.o
> +perf-y += llvm.o llvm-src.o
>  perf-y += topology.o
>  
> +$(OUTPUT)tests/llvm-src.c: tests/bpf-script-example.c
> +	$(call rule_mkdir)
> +	$(Q)echo '#include <tests/llvm.h>' > $@
> +	$(Q)echo 'const char test_llvm__bpf_prog[] =' >> $@
> +	$(Q)sed -e 's/"/\\"/g' -e 's/\(.*\)/"\1\\n"/g' $< >> $@
> +	$(Q)echo ';' >> $@
> +
>  ifeq ($(ARCH),$(filter $(ARCH),x86 arm arm64))
>  perf-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
>  endif
> diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
> index 66f72d3..e812a0c 100644
> --- a/tools/perf/tests/builtin-test.c
> +++ b/tools/perf/tests/builtin-test.c
> @@ -160,6 +160,8 @@ static struct test generic_tests[] = {
>  	{
>  		.desc = "Test LLVM searching and compiling",
>  		.func = test__llvm,
> +		.prepare = test__llvm_prepare,
> +		.cleanup = test__llvm_cleanup,
>  	},
>  	{
>  		.desc = "Test topology in session",
> @@ -261,7 +263,11 @@ static int __cmd_test(int argc, const char *argv[], struct intlist *skiplist)
>  		}
>  
>  		pr_debug("\n--- start ---\n");
> +		if (t->prepare)
> +			t->prepare();
>  		err = run_test(t);
> +		if (t->cleanup)
> +			t->cleanup();
>  		pr_debug("---- end ----\n%s:", t->desc);
>  
>  		switch (err) {
> diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
> index 52d5597..236bf39 100644
> --- a/tools/perf/tests/llvm.c
> +++ b/tools/perf/tests/llvm.c
> @@ -1,9 +1,13 @@
>  #include <stdio.h>
> +#include <sys/utsname.h>
>  #include <bpf/libbpf.h>
>  #include <util/llvm-utils.h>
>  #include <util/cache.h>
> +#include <util/util.h>
> +#include <sys/mman.h>
>  #include "tests.h"
>  #include "debug.h"
> +#include "llvm.h"
>  
>  static int perf_config_cb(const char *var, const char *val,
>  			  void *arg __maybe_unused)
> @@ -11,16 +15,6 @@ static int perf_config_cb(const char *var, const char *val,
>  	return perf_default_config(var, val, arg);
>  }
>  
> -/*
> - * Randomly give it a "version" section since we don't really load it
> - * into kernel
> - */
> -static const char test_bpf_prog[] =
> -	"__attribute__((section(\"do_fork\"), used)) "
> -	"int fork(void *ctx) {return 0;} "
> -	"char _license[] __attribute__((section(\"license\"), used)) = \"GPL\";"
> -	"int _version __attribute__((section(\"version\"), used)) = 0x40100;";
> -
>  #ifdef HAVE_LIBBPF_SUPPORT
>  static int test__bpf_parsing(void *obj_buf, size_t obj_buf_sz)
>  {
> @@ -41,12 +35,44 @@ static int test__bpf_parsing(void *obj_buf __maybe_unused,
>  }
>  #endif
>  
> +static char *
> +compose_source(void)
> +{
> +	struct utsname utsname;
> +	int version, patchlevel, sublevel, err;
> +	unsigned long version_code;
> +	char *code;
> +
> +	if (uname(&utsname))
> +		return NULL;
> +
> +	err = sscanf(utsname.release, "%d.%d.%d",
> +		     &version, &patchlevel, &sublevel);
> +	if (err != 3) {
> +		fprintf(stderr, " (Can't get kernel version from uname '%s')",
> +			utsname.release);
> +		return NULL;
> +	}
> +
> +	version_code = (version << 16) + (patchlevel << 8) + sublevel;
> +	err = asprintf(&code, "#define LINUX_VERSION_CODE 0x%08lx;\n%s",
> +		       version_code, test_llvm__bpf_prog);
> +	if (err < 0)
> +		return NULL;
> +
> +	return code;
> +}
> +
> +#define SHARED_BUF_INIT_SIZE	(1 << 20)
> +struct test_llvm__bpf_result *p_test_llvm__bpf_result;
> +
>  int test__llvm(void)
>  {
>  	char *tmpl_new, *clang_opt_new;
>  	void *obj_buf;
>  	size_t obj_buf_sz;
>  	int err, old_verbose;
> +	char *source;
>  
>  	perf_config(perf_config_cb, NULL);
>  
> @@ -73,10 +99,22 @@ int test__llvm(void)
>  	if (!llvm_param.clang_opt)
>  		llvm_param.clang_opt = strdup("");
>  
> -	err = asprintf(&tmpl_new, "echo '%s' | %s", test_bpf_prog,
> -		       llvm_param.clang_bpf_cmd_template);
> -	if (err < 0)
> +	source = compose_source();
> +	if (!source) {
> +		pr_err("Failed to compose source code\n");
> +		return -1;
> +	}
> +
> +	/* Quote __EOF__ so strings in source won't be expanded by shell */
> +	err = asprintf(&tmpl_new, "cat << '__EOF__' | %s\n%s\n__EOF__\n",
> +		       llvm_param.clang_bpf_cmd_template, source);
> +	free(source);
> +	source = NULL;
> +	if (err < 0) {
> +		pr_err("Failed to alloc new template\n");
>  		return -1;
> +	}
> +
>  	err = asprintf(&clang_opt_new, "-xc %s", llvm_param.clang_opt);
>  	if (err < 0)
>  		return -1;
> @@ -93,6 +131,46 @@ int test__llvm(void)
>  	}
>  
>  	err = test__bpf_parsing(obj_buf, obj_buf_sz);
> +	if (!err && p_test_llvm__bpf_result) {
> +		if (obj_buf_sz > SHARED_BUF_INIT_SIZE) {
> +			pr_err("Resulting object too large\n");
> +		} else {
> +			p_test_llvm__bpf_result->size = obj_buf_sz;
> +			memcpy(p_test_llvm__bpf_result->object,
> +			       obj_buf, obj_buf_sz);
> +		}
> +	}
>  	free(obj_buf);
>  	return err;
>  }
> +
> +void test__llvm_prepare(void)
> +{
> +	p_test_llvm__bpf_result = mmap(NULL, SHARED_BUF_INIT_SIZE,
> +				       PROT_READ | PROT_WRITE,
> +				       MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> +	if (!p_test_llvm__bpf_result)
> +		return;
> +	memset((void *)p_test_llvm__bpf_result, '\0', SHARED_BUF_INIT_SIZE);
> +}
> +
> +void test__llvm_cleanup(void)
> +{
> +	unsigned long boundary, buf_end;
> +
> +	if (!p_test_llvm__bpf_result)
> +		return;
> +	if (p_test_llvm__bpf_result->size == 0) {
> +		munmap((void *)p_test_llvm__bpf_result, SHARED_BUF_INIT_SIZE);
> +		p_test_llvm__bpf_result = NULL;
> +		return;
> +	}
> +
> +	buf_end = (unsigned long)p_test_llvm__bpf_result + SHARED_BUF_INIT_SIZE;
> +
> +	boundary = (unsigned long)(p_test_llvm__bpf_result);
> +	boundary += p_test_llvm__bpf_result->size;
> +	boundary = (boundary + (page_size - 1)) &
> +			(~((unsigned long)page_size - 1));
> +	munmap((void *)boundary, buf_end - boundary);
> +}
> diff --git a/tools/perf/tests/llvm.h b/tools/perf/tests/llvm.h
> new file mode 100644
> index 0000000..1e89e46
> --- /dev/null
> +++ b/tools/perf/tests/llvm.h
> @@ -0,0 +1,14 @@
> +#ifndef PERF_TEST_LLVM_H
> +#define PERF_TEST_LLVM_H
> +
> +#include <stddef.h> /* for size_t */
> +
> +struct test_llvm__bpf_result {
> +	size_t size;
> +	char object[];
> +};
> +
> +extern struct test_llvm__bpf_result *p_test_llvm__bpf_result;
> +extern const char test_llvm__bpf_prog[];
> +
> +#endif
> diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
> index c804869..a848802 100644
> --- a/tools/perf/tests/tests.h
> +++ b/tools/perf/tests/tests.h
> @@ -27,6 +27,8 @@ enum {
>  struct test {
>  	const char *desc;
>  	int (*func)(void);
> +	void (*prepare)(void);
> +	void (*cleanup)(void);
>  };
>  
>  /* Tests */
> @@ -66,6 +68,8 @@ int test__fdarray__add(void);
>  int test__kmod_path__parse(void);
>  int test__thread_map(void);
>  int test__llvm(void);
> +void test__llvm_prepare(void);
> +void test__llvm_cleanup(void);
>  int test_session_topology(void);
>  
>  #if defined(__arm__) || defined(__aarch64__)
> -- 
> 1.8.3.4

Re: [PATCH 12/31] perf probe: Reset args and nargs for probe_trace_event when failure

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:23PM +0000, Wang Nan escreveu:
> When failure occures in add_probe_trace_event(), args in
> probe_trace_event is incomplete. Since information in it may be used
> in futher, this patch frees the allocated memory and set it to NULL
> to avoid dangling pointer.

Masami, can I have your Acked-by for this one?

Thanks,

- Arnaldo
 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---
>  tools/perf/util/probe-finder.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
> index bd8f03d..b1581d7 100644
> --- a/tools/perf/util/probe-finder.c
> +++ b/tools/perf/util/probe-finder.c
> @@ -1235,6 +1235,10 @@ static int add_probe_trace_event(Dwarf_Die *sc_die, struct probe_finder *pf)
>  
>  end:
>  	free(args);
> +	if (ret) {
> +		tev->nargs = 0;
> +		zfree(&tev->args);
> +	}
>  	return ret;
>  }
>  
> -- 
> 1.8.3.4

Re: [PATCH 13/31] bpf tools: Load a program with different instances using preprocessor

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:24PM +0000, Wang Nan escreveu:
> In this patch, caller of libbpf is able to control the loaded programs
> by installing a preprocessor callback for a BPF program. With
> preprocessor, different instances can be created from one BPF program.

Why would one want to do that? I'm new to eBPF, as many other here, I
guess, so giving some more context as to what will this achieve will
help me in processing these patches.
 
> This patch will be used by perf to generate different prologue for
> different 'struct probe_trace_event' instances matched by one
> 'struct perf_probe_event'.
> 
> bpf_program__set_prep() is added to support this feature. Caller
> should pass libbpf the number of instances should be created and a
                                          that
> preprocessor function which will be called when doing real loading.
                                                      the
> The callback should return instructions arrays for each instances.
                                instruction arrays for each instance.
> 
> fd field in bpf_programs is replaced by instance, which has an nr field

The fd field in the bpf_programs struct is replaced by instance
(instance of what?)

> and fds array. bpf_program__nth_fd() is introduced for read fd of
> instances. Old interface bpf_program__fd() is reimplemented by
> returning the first fd.
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Signed-off-by: He Kuang <hekuang@huawei.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---
>  tools/lib/bpf/libbpf.c | 143 +++++++++++++++++++++++++++++++++++++++++++++----
>  tools/lib/bpf/libbpf.h |  22 ++++++++
>  2 files changed, 156 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 4252fc2..6a07b26 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -98,7 +98,11 @@ struct bpf_program {
>  	} *reloc_desc;
>  	int nr_reloc;
>  
> -	int fd;
> +	struct {
> +		int nr;
> +		int *fds;
> +	} instance;
> +	bpf_program_prep_t preprocessor;
>  
>  	struct bpf_object *obj;
>  	void *priv;
> @@ -152,10 +156,24 @@ struct bpf_object {
>  
>  static void bpf_program__unload(struct bpf_program *prog)
>  {
> +	int i;
> +
>  	if (!prog)
>  		return;
>  
> -	zclose(prog->fd);
> +	/*
> +	 * If the object is opened but the program is never loaded,
> +	 * it is possible that prog->instance.nr == -1.
> +	 */
> +	if (prog->instance.nr > 0) {
> +		for (i = 0; i < prog->instance.nr; i++)
> +			zclose(prog->instance.fds[i]);
> +	} else if (prog->instance.nr != -1)
> +		pr_warning("Internal error: instance.nr is %d\n",
> +			   prog->instance.nr);
> +
> +	prog->instance.nr = -1;
> +	zfree(&prog->instance.fds);
>  }
>  
>  static void bpf_program__exit(struct bpf_program *prog)
> @@ -206,7 +224,8 @@ bpf_program__init(void *data, size_t size, char *name, int idx,
>  	memcpy(prog->insns, data,
>  	       prog->insns_cnt * sizeof(struct bpf_insn));
>  	prog->idx = idx;
> -	prog->fd = -1;
> +	prog->instance.fds = NULL;
> +	prog->instance.nr = -1;
>  
>  	return 0;
>  errout:
> @@ -795,13 +814,71 @@ static int
>  bpf_program__load(struct bpf_program *prog,
>  		  char *license, u32 kern_version)
>  {
> -	int err, fd;
> +	int err = 0, fd, i;
> +
> +	if (prog->instance.nr < 0 || !prog->instance.fds) {
> +		if (prog->preprocessor) {
> +			pr_warning("Internal error: can't load program '%s'\n",
> +				   prog->section_name);
> +			return -EINVAL;
> +		}
> +
> +		prog->instance.fds = malloc(sizeof(int));
> +		if (!prog->instance.fds) {
> +			pr_warning("No enough memory for fds\n");
> +			return -ENOMEM;
> +		}
> +		prog->instance.nr = 1;
> +		prog->instance.fds[0] = -1;
> +	}
> +
> +	if (!prog->preprocessor) {
> +		if (prog->instance.nr != 1)
> +			pr_warning("Program '%s' inconsistent: nr(%d) not 1\n",
> +				   prog->section_name, prog->instance.nr);
>  
> -	err = load_program(prog->insns, prog->insns_cnt,
> -			   license, kern_version, &fd);
> -	if (!err)
> -		prog->fd = fd;
> +		err = load_program(prog->insns, prog->insns_cnt,
> +				   license, kern_version, &fd);
> +		if (!err)
> +			prog->instance.fds[0] = fd;
> +		goto out;
> +	}
> +
> +	for (i = 0; i < prog->instance.nr; i++) {
> +		struct bpf_prog_prep_result result;
> +		bpf_program_prep_t preprocessor = prog->preprocessor;
> +
> +		bzero(&result, sizeof(result));
> +		err = preprocessor(prog, i, prog->insns,
> +				   prog->insns_cnt, &result);
> +		if (err) {
> +			pr_warning("Preprocessing %dth instance of program '%s' failed\n",
> +					i, prog->section_name);
> +			goto out;
> +		}
> +
> +		if (!result.new_insn_ptr || !result.new_insn_cnt) {
> +			pr_debug("Skip loading %dth instance of program '%s'\n",
> +					i, prog->section_name);
> +			prog->instance.fds[i] = -1;
> +			continue;
> +		}
> +
> +		err = load_program(result.new_insn_ptr,
> +				   result.new_insn_cnt,
> +				   license, kern_version, &fd);
> +
> +		if (err) {
> +			pr_warning("Loading %dth instance of program '%s' failed\n",
> +					i, prog->section_name);
> +			goto out;
> +		}
>  
> +		if (result.pfd)
> +			*result.pfd = fd;
> +		prog->instance.fds[i] = fd;
> +	}
> +out:
>  	if (err)
>  		pr_warning("failed to load program '%s'\n",
>  			   prog->section_name);
> @@ -1052,5 +1129,53 @@ const char *bpf_program__title(struct bpf_program *prog, bool dup)
>  
>  int bpf_program__fd(struct bpf_program *prog)
>  {
> -	return prog->fd;
> +	return bpf_program__nth_fd(prog, 0);
> +}
> +
> +int bpf_program__set_prep(struct bpf_program *prog, int nr_instance,
> +			  bpf_program_prep_t prep)
> +{
> +	int *instance_fds;
> +
> +	if (nr_instance <= 0 || !prep)
> +		return -EINVAL;
> +
> +	if (prog->instance.nr > 0 || prog->instance.fds) {
> +		pr_warning("Can't set pre-processor after loading\n");
> +		return -EINVAL;
> +	}
> +
> +	instance_fds = malloc(sizeof(int) * nr_instance);
> +	if (!instance_fds) {
> +		pr_warning("alloc memory failed for instance of fds\n");
> +		return -ENOMEM;
> +	}
> +
> +	/* fill all fd with -1 */
> +	memset(instance_fds, 0xff, sizeof(int) * nr_instance);
> +
> +	prog->instance.nr = nr_instance;
> +	prog->instance.fds = instance_fds;
> +	prog->preprocessor = prep;
> +	return 0;
> +}
> +
> +int bpf_program__nth_fd(struct bpf_program *prog, int n)
> +{
> +	int fd;
> +
> +	if (n >= prog->instance.nr || n < 0) {
> +		pr_warning("Can't get the %dth fd from program %s: only %d instances\n",
> +			   n, prog->section_name, prog->instance.nr);
> +		return -EINVAL;
> +	}
> +
> +	fd = prog->instance.fds[n];
> +	if (fd < 0) {
> +		pr_warning("%dth instance of program '%s' is invalid\n",
> +			   n, prog->section_name);
> +		return -ENOENT;
> +	}
> +
> +	return fd;
>  }
> diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
> index f16170c..d82b89e 100644
> --- a/tools/lib/bpf/libbpf.h
> +++ b/tools/lib/bpf/libbpf.h
> @@ -67,6 +67,28 @@ const char *bpf_program__title(struct bpf_program *prog, bool dup);
>  
>  int bpf_program__fd(struct bpf_program *prog);
>  
> +struct bpf_insn;
> +struct bpf_prog_prep_result {
> +	/*
> +	 * If not NULL, load new instruction array.
> +	 * If set to NULL, don't load this instance.
> +	 */
> +	struct bpf_insn *new_insn_ptr;
> +	int new_insn_cnt;
> +
> +	/* If not NULL, result fd is set to it */
> +	int *pfd;
> +};
> +
> +typedef int (*bpf_program_prep_t)(struct bpf_program *, int n,
> +				  struct bpf_insn *, int insn_cnt,
> +				  struct bpf_prog_prep_result *res);
> +
> +int bpf_program__set_prep(struct bpf_program *prog, int nr_instance,
> +			  bpf_program_prep_t prep);
> +
> +int bpf_program__nth_fd(struct bpf_program *prog, int n);
> +
>  /*
>   * We don't need __attribute__((packed)) now since it is
>   * unnecessary for 'bpf_map_def' because they are all aligned.
> -- 
> 1.8.3.4

Re: [PATCH 14/31] perf tools: Add BPF_PROLOGUE config options for further patches

[Arnaldo Carvalho de Melo]

Em Wed, Oct 14, 2015 at 12:41:25PM +0000, Wang Nan escreveu:
> If both LIBBPF and DWARF are detected, it is possible to create prologue
> for eBPF programs to help them accessing kernel data. HAVE_BPF_PROLOGUE

                                       what kernel data? you mean to be
able to map from variable/function argument name to where they are in
memory/registers?

> and CONFIG_BPF_PROLOGUE is added as flags for this feature.
> 
> PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET indicates an architecture
> supports converting name of a register to its offset in
> 'struct pt_regs'. Without this support, BPF_PROLOGUE should be turned off.
> 
> HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET is introduced as the corresponding
> CFLAGS of PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET.
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Alexei Starovoitov <ast@plumgrid.com>
> Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Kaixu Xia <xiakaixu@huawei.com>
> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Zefan Li <lizefan@huawei.com>
> Cc: pi3orama@163.com
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
> ---
>  tools/perf/config/Makefile | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
> index de89ec5..6eb9a95 100644
> --- a/tools/perf/config/Makefile
> +++ b/tools/perf/config/Makefile
> @@ -318,6 +318,18 @@ ifndef NO_LIBELF
>        CFLAGS += -DHAVE_LIBBPF_SUPPORT
>        $(call detected,CONFIG_LIBBPF)
>      endif
> +
> +    ifndef NO_DWARF
> +      ifdef PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET
> +        CFLAGS += -DHAVE_BPF_PROLOGUE
> +        $(call detected,CONFIG_BPF_PROLOGUE)
> +      else
> +        msg := $(warning BPF prologue is not supported by architecture $(ARCH), missing regs_query_register_offset());
> +      endif
> +    else
> +      msg := $(warning DWARF support is off, BPF prologue is disabled);
> +    endif
> +
>    endif # NO_LIBBPF
>  endif # NO_LIBELF
>  
> -- 
> 1.8.3.4

Re: [PATCH 09/31] perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

[Arnaldo Carvalho de Melo]

Em Thu, Oct 29, 2015 at 01:30:27PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Oct 29, 2015 at 01:25:53PM -0300, Arnaldo Carvalho de Melo escreveu:
> > And you could've provided something that would actually do something
> > meaningful, i.e.  a filter that would capture some samples for a given event
> > while discarding something not that important...
> > 
> > I.e. show the value provided at the time we build that patch.
> > 
> > I'll try to come up with something like that after I have lunch.
> 
> Ok, what I have is at my perf/core branch, will continue after having
> some food,

Oh well, more interesting things require that we access the function
arguments and that is not yet possible, requires that PROLOGUE + dwarf
bits that comes later, reading those patches...

- Arnaldo

[tip:perf/core] perf bpf: Attach eBPF filter to perf event

[tip-bot for Wang Nan]

Commit-ID:  1f45b1d49073541947193bd7dac9e904142576aa
Gitweb:     http://git.kernel.org/tip/1f45b1d49073541947193bd7dac9e904142576aa
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:18 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 29 Oct 2015 17:16:22 -0300

perf bpf: Attach eBPF filter to perf event

This is the final patch which makes basic BPF filter work. After
applying this patch, users are allowed to use BPF filter like:

 # perf record --event ./hello_world.o ls

A bpf_fd field is appended to 'struct evsel', and setup during the
callback function add_bpf_event() for each 'probe_trace_event'.

PERF_EVENT_IOC_SET_BPF ioctl is used to attach eBPF program to a newly
created perf event. The file descriptor of the eBPF program is passed to
perf record using previous patches, and stored into evsel->bpf_fd.

It is possible that different perf event are created for one kprobe
events for different CPUs. In this case, when trying to call the ioctl,
EEXIST will be return. This patch doesn't treat it as an error.

Committer note:

The bpf proggie used so far:

  __attribute__((section("fork=_do_fork"), used))
  int fork(void *ctx)
  {
	  return 0;
  }

  char _license[] __attribute__((section("license"), used)) = "GPL";
  int _version __attribute__((section("version"), used)) = 0x40300;

failed to produce any samples, even with forks happening and it being
running in system wide mode.

That is because now the filter is being associated, and the code above
always returns zero, meaning that all forks will be probed but filtered
away ;-/

Change it to 'return 1;' instead and after that:

  # trace --no-syscalls --event /tmp/foo.o
     0.000 perf_bpf_probe:fork:(ffffffff8109be30))
     2.333 perf_bpf_probe:fork:(ffffffff8109be30))
     3.725 perf_bpf_probe:fork:(ffffffff8109be30))
     4.550 perf_bpf_probe:fork:(ffffffff8109be30))
  ^C#

And it works with all tools, including 'perf trace'.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-8-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/evsel.c        | 17 +++++++++++++++++
 tools/perf/util/evsel.h        |  1 +
 tools/perf/util/parse-events.c |  6 ++++++
 3 files changed, 24 insertions(+)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 3ac4ee9c..397fb4e 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -208,6 +208,7 @@ void perf_evsel__init(struct perf_evsel *evsel,
 	evsel->unit	   = "";
 	evsel->scale	   = 1.0;
 	evsel->evlist	   = NULL;
+	evsel->bpf_fd	   = -1;
 	INIT_LIST_HEAD(&evsel->node);
 	INIT_LIST_HEAD(&evsel->config_terms);
 	perf_evsel__object.init(evsel);
@@ -1356,6 +1357,22 @@ retry_open:
 					  err);
 				goto try_fallback;
 			}
+
+			if (evsel->bpf_fd >= 0) {
+				int evt_fd = FD(evsel, cpu, thread);
+				int bpf_fd = evsel->bpf_fd;
+
+				err = ioctl(evt_fd,
+					    PERF_EVENT_IOC_SET_BPF,
+					    bpf_fd);
+				if (err && errno != EEXIST) {
+					pr_err("failed to attach bpf fd %d: %s\n",
+					       bpf_fd, strerror(errno));
+					err = -EINVAL;
+					goto out_close;
+				}
+			}
+
 			set_rlimit = NO_CHANGE;
 
 			/*
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 1e8ff19..0e49bd7 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -123,6 +123,7 @@ struct perf_evsel {
 	char			*group_name;
 	bool			cmdline_group_boundary;
 	struct list_head	config_terms;
+	int			bpf_fd;
 };
 
 union u64_swap {
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index d97b037..cee8c61 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -542,6 +542,7 @@ static int add_bpf_event(struct probe_trace_event *tev, int fd,
 	struct __add_bpf_event_param *param = _param;
 	struct parse_events_evlist *evlist = param->data;
 	struct list_head *list = param->list;
+	struct perf_evsel *pos;
 	int err;
 
 	pr_debug("add bpf event %s:%s and attach bpf program %d\n",
@@ -562,6 +563,11 @@ static int add_bpf_event(struct probe_trace_event *tev, int fd,
 	}
 	pr_debug("adding %s:%s\n", tev->group, tev->event);
 
+	list_for_each_entry(pos, &new_evsels, node) {
+		pr_debug("adding %s:%s to %p\n",
+			 tev->group, tev->event, pos);
+		pos->bpf_fd = fd;
+	}
 	list_splice(&new_evsels, list);
 	return 0;
 }

[tip:perf/core] perf record: Add clang options for compiling BPF scripts

[tip-bot for Wang Nan]

Commit-ID:  71dc2326252ff1bcdddc05db03c0f831d16c9447
Gitweb:     http://git.kernel.org/tip/71dc2326252ff1bcdddc05db03c0f831d16c9447
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:19 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 29 Oct 2015 17:16:22 -0300

perf record: Add clang options for compiling BPF scripts

Although previous patch allows setting BPF compiler related options in
perfconfig, on some ad-hoc situation it still requires passing options
through cmdline. This patch introduces 2 options to 'perf record' for
this propose: --clang-path and --clang-opt.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-9-git-send-email-wangnan0@huawei.com
[ Add the new options to the 'record' man page ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf-record.txt | 6 ++++++
 tools/perf/builtin-record.c              | 7 +++++++
 2 files changed, 13 insertions(+)

diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index 7ff6a9d..e630a7d 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -314,6 +314,12 @@ This option sets the time out limit. The default value is 500 ms.
 Record context switch events i.e. events of type PERF_RECORD_SWITCH or
 PERF_RECORD_SWITCH_CPU_WIDE.
 
+--clang-path::
+Path to clang binary to use for compiling BPF scriptlets.
+
+--clang-opt::
+Options passed to clang when compiling BPF scriptlets.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index de02267..199fc31 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -31,6 +31,7 @@
 #include "util/auxtrace.h"
 #include "util/parse-branch-options.h"
 #include "util/parse-regs-options.h"
+#include "util/llvm-utils.h"
 
 #include <unistd.h>
 #include <sched.h>
@@ -1112,6 +1113,12 @@ struct option __record_options[] = {
 			"per thread proc mmap processing timeout in ms"),
 	OPT_BOOLEAN(0, "switch-events", &record.opts.record_switch_events,
 		    "Record context switch events"),
+#ifdef HAVE_LIBBPF_SUPPORT
+	OPT_STRING(0, "clang-path", &llvm_param.clang_path, "clang path",
+		   "clang binary to use for compiling BPF scriptlets"),
+	OPT_STRING(0, "clang-opt", &llvm_param.clang_opt, "clang options",
+		   "options passed to clang when compiling BPF scriptlets"),
+#endif
 	OPT_END()
 };
 

[tip:perf/core] perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

[tip-bot for Wang Nan]

Commit-ID:  d509db0473e40134286271b1d1adadccf42ac467
Gitweb:     http://git.kernel.org/tip/d509db0473e40134286271b1d1adadccf42ac467
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 14 Oct 2015 12:41:20 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Thu, 29 Oct 2015 17:16:23 -0300

perf tools: Compile scriptlets to BPF objects when passing '.c' to --event

This patch provides infrastructure for passing source files to --event
directly using:

 # perf record --event bpf-file.c command

This patch does following works:

 1) Allow passing '.c' file to '--event'. parse_events_load_bpf() is
    expanded to allow caller tell it whether the passed file is source
    file or object.

 2) llvm__compile_bpf() is called to compile the '.c' file, the result
    is saved into memory. Use bpf_object__open_buffer() to load the
    in-memory object.

Introduces a bpf-script-example.c so we can manually test it:

 # perf record --clang-opt "-DLINUX_VERSION_CODE=0x40200" --event ./bpf-script-example.c sleep 1

Note that '--clang-opt' must put before '--event'.

Futher patches will merge it into a testcase so can be tested automatically.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Alexei Starovoitov <ast@plumgrid.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kaixu Xia <xiakaixu@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1444826502-49291-10-git-send-email-wangnan0@huawei.com
Signed-off-by: He Kuang <hekuang@huawei.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/tests/bpf-script-example.c | 44 +++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-loader.c          | 17 ++++++++++++--
 tools/perf/util/bpf-loader.h          |  5 ++--
 tools/perf/util/parse-events.c        |  5 ++--
 tools/perf/util/parse-events.h        |  3 ++-
 tools/perf/util/parse-events.l        |  3 +++
 tools/perf/util/parse-events.y        | 15 ++++++++++--
 7 files changed, 83 insertions(+), 9 deletions(-)

diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
new file mode 100644
index 0000000..410a70b
--- /dev/null
+++ b/tools/perf/tests/bpf-script-example.c
@@ -0,0 +1,44 @@
+#ifndef LINUX_VERSION_CODE
+# error Need LINUX_VERSION_CODE
+# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
+#endif
+#define BPF_ANY 0
+#define BPF_MAP_TYPE_ARRAY 2
+#define BPF_FUNC_map_lookup_elem 1
+#define BPF_FUNC_map_update_elem 2
+
+static void *(*bpf_map_lookup_elem)(void *map, void *key) =
+	(void *) BPF_FUNC_map_lookup_elem;
+static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
+	(void *) BPF_FUNC_map_update_elem;
+
+struct bpf_map_def {
+	unsigned int type;
+	unsigned int key_size;
+	unsigned int value_size;
+	unsigned int max_entries;
+};
+
+#define SEC(NAME) __attribute__((section(NAME), used))
+struct bpf_map_def SEC("maps") flip_table = {
+	.type = BPF_MAP_TYPE_ARRAY,
+	.key_size = sizeof(int),
+	.value_size = sizeof(int),
+	.max_entries = 1,
+};
+
+SEC("func=sys_epoll_pwait")
+int bpf_func__sys_epoll_pwait(void *ctx)
+{
+	int ind =0;
+	int *flag = bpf_map_lookup_elem(&flip_table, &ind);
+	int new_flag;
+	if (!flag)
+		return 0;
+	/* flip flag and store back */
+	new_flag = !*flag;
+	bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
+	return new_flag;
+}
+char _license[] SEC("license") = "GPL";
+int _version SEC("version") = LINUX_VERSION_CODE;
diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index aa784a4..ba6f752 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -12,6 +12,7 @@
 #include "bpf-loader.h"
 #include "probe-event.h"
 #include "probe-finder.h" // for MAX_PROBES
+#include "llvm-utils.h"
 
 #define DEFINE_PRINT_FN(name, level) \
 static int libbpf_##name(const char *fmt, ...)	\
@@ -33,7 +34,7 @@ struct bpf_prog_priv {
 	struct perf_probe_event pev;
 };
 
-struct bpf_object *bpf__prepare_load(const char *filename)
+struct bpf_object *bpf__prepare_load(const char *filename, bool source)
 {
 	struct bpf_object *obj;
 	static bool libbpf_initialized;
@@ -45,7 +46,19 @@ struct bpf_object *bpf__prepare_load(const char *filename)
 		libbpf_initialized = true;
 	}
 
-	obj = bpf_object__open(filename);
+	if (source) {
+		int err;
+		void *obj_buf;
+		size_t obj_buf_sz;
+
+		err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
+		if (err)
+			return ERR_PTR(err);
+		obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, filename);
+		free(obj_buf);
+	} else
+		obj = bpf_object__open(filename);
+
 	if (!obj) {
 		pr_debug("bpf: failed to load %s\n", filename);
 		return ERR_PTR(-EINVAL);
diff --git a/tools/perf/util/bpf-loader.h b/tools/perf/util/bpf-loader.h
index a8f25ee..ccd8d7f 100644
--- a/tools/perf/util/bpf-loader.h
+++ b/tools/perf/util/bpf-loader.h
@@ -18,7 +18,7 @@ typedef int (*bpf_prog_iter_callback_t)(struct probe_trace_event *tev,
 					int fd, void *arg);
 
 #ifdef HAVE_LIBBPF_SUPPORT
-struct bpf_object *bpf__prepare_load(const char *filename);
+struct bpf_object *bpf__prepare_load(const char *filename, bool source);
 
 void bpf__clear(void);
 
@@ -34,7 +34,8 @@ int bpf__foreach_tev(struct bpf_object *obj,
 		     bpf_prog_iter_callback_t func, void *arg);
 #else
 static inline struct bpf_object *
-bpf__prepare_load(const char *filename __maybe_unused)
+bpf__prepare_load(const char *filename __maybe_unused,
+		  bool source __maybe_unused)
 {
 	pr_debug("ERROR: eBPF object loading is disabled during compiling.\n");
 	return ERR_PTR(-ENOTSUP);
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index cee8c61..bee6058 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -626,11 +626,12 @@ errout:
 
 int parse_events_load_bpf(struct parse_events_evlist *data,
 			  struct list_head *list,
-			  char *bpf_file_name)
+			  char *bpf_file_name,
+			  bool source)
 {
 	struct bpf_object *obj;
 
-	obj = bpf__prepare_load(bpf_file_name);
+	obj = bpf__prepare_load(bpf_file_name, source);
 	if (IS_ERR(obj) || !obj) {
 		char errbuf[BUFSIZ];
 		int err;
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 765018a..f1a6db1 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -125,7 +125,8 @@ int parse_events_add_tracepoint(struct list_head *list, int *idx,
 				struct list_head *head_config);
 int parse_events_load_bpf(struct parse_events_evlist *data,
 			  struct list_head *list,
-			  char *bpf_file_name);
+			  char *bpf_file_name,
+			  bool source);
 /* Provide this function for perf test */
 struct bpf_object;
 int parse_events_load_bpf_obj(struct parse_events_evlist *data,
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index cf330eb..58c5831 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -116,6 +116,7 @@ group		[^,{}/]*[{][^}]*[}][^,{}/]*
 event_pmu	[^,{}/]+[/][^/]*[/][^,{}/]*
 event		[^,{}/]+
 bpf_object	.*\.(o|bpf)
+bpf_source	.*\.c
 
 num_dec		[0-9]+
 num_hex		0x[a-fA-F0-9]+
@@ -161,6 +162,7 @@ modifier_bp	[rwx]{1,3}
 
 {event_pmu}	|
 {bpf_object}	|
+{bpf_source}	|
 {event}		{
 			BEGIN(INITIAL);
 			REWIND(1);
@@ -269,6 +271,7 @@ r{num_raw_hex}		{ return raw(yyscanner); }
 
 {modifier_event}	{ return str(yyscanner, PE_MODIFIER_EVENT); }
 {bpf_object}		{ return str(yyscanner, PE_BPF_OBJECT); }
+{bpf_source}		{ return str(yyscanner, PE_BPF_SOURCE); }
 {name}			{ return pmu_str_check(yyscanner); }
 "/"			{ BEGIN(config); return '/'; }
 -			{ return '-'; }
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index 497f19b..ad37996 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -42,7 +42,7 @@ static inc_group_count(struct list_head *list,
 %token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
 %token PE_EVENT_NAME
 %token PE_NAME
-%token PE_BPF_OBJECT
+%token PE_BPF_OBJECT PE_BPF_SOURCE
 %token PE_MODIFIER_EVENT PE_MODIFIER_BP
 %token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
 %token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
@@ -55,6 +55,7 @@ static inc_group_count(struct list_head *list,
 %type <num> PE_TERM
 %type <str> PE_NAME
 %type <str> PE_BPF_OBJECT
+%type <str> PE_BPF_SOURCE
 %type <str> PE_NAME_CACHE_TYPE
 %type <str> PE_NAME_CACHE_OP_RESULT
 %type <str> PE_MODIFIER_EVENT
@@ -461,7 +462,17 @@ PE_BPF_OBJECT
 	struct list_head *list;
 
 	ALLOC_LIST(list);
-	ABORT_ON(parse_events_load_bpf(data, list, $1));
+	ABORT_ON(parse_events_load_bpf(data, list, $1, false));
+	$$ = list;
+}
+|
+PE_BPF_SOURCE
+{
+	struct parse_events_evlist *data = _data;
+	struct list_head *list;
+
+	ALLOC_LIST(list);
+	ABORT_ON(parse_events_load_bpf(data, list, $1, true));
 	$$ = list;
 }
 

RE: [PATCH 12/31] perf probe: Reset args and nargs for probe_trace_event when failure

[平松雅巳 / HIRAMATU，MASAMI]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2016 bytes --]

From: Wang Nan [mailto:wangnan0@huawei.com]
>
>When failure occures in add_probe_trace_event(), args in
>probe_trace_event is incomplete. Since information in it may be used
>in futher, this patch frees the allocated memory and set it to NULL
>to avoid dangling pointer.

Sorry for replying late. Please see my comment.

>
>Signed-off-by: Wang Nan <wangnan0@huawei.com>
>Cc: Alexei Starovoitov <ast@plumgrid.com>
>Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
>Cc: Daniel Borkmann <daniel@iogearbox.net>
>Cc: David Ahern <dsahern@gmail.com>
>Cc: He Kuang <hekuang@huawei.com>
>Cc: Jiri Olsa <jolsa@kernel.org>
>Cc: Kaixu Xia <xiakaixu@huawei.com>
>Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
>Cc: Namhyung Kim <namhyung@kernel.org>
>Cc: Paul Mackerras <paulus@samba.org>
>Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
>Cc: Zefan Li <lizefan@huawei.com>
>Cc: pi3orama@163.com
>Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>Link: http://lkml.kernel.org/n/ebpf-6yw9eg0ej3l4jnqhinngkw86@git.kernel.org
>---
> tools/perf/util/probe-finder.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
>diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
>index bd8f03d..b1581d7 100644
>--- a/tools/perf/util/probe-finder.c
>+++ b/tools/perf/util/probe-finder.c
>@@ -1235,6 +1235,10 @@ static int add_probe_trace_event(Dwarf_Die *sc_die, struct probe_finder *pf)
>
> end:
> 	free(args);

Actually, this args is filled with constant strings allocated in libdw, so it doesn't
free args[*].vars.

>+	if (ret) {
>+		tev->nargs = 0;
>+		zfree(&tev->args);

However, all the fields in tev->args[*] are allocated in find_variables. If you'd like
to clear it, you must free the fields too.
So, you should call clear_probe_trace_event(struct probe_trace_event *tev) defined
in util/probe-event.c.

Thank you,

>+	}
> 	return ret;
> }
>
>--
>1.8.3.4

ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH 10/31] perf test: Enforce LLVM test for BPF test

[Wangnan (F)]

On 2015/10/30 6:37, Arnaldo Carvalho de Melo wrote:
> Em Wed, Oct 14, 2015 at 12:41:21PM +0000, Wang Nan escreveu:
>> This patch replaces the original toy BPF program with previous introduced
>> bpf-script-example.c. Dynamically embedded it into 'llvm-src.c'.
>>
>> The newly introduced BPF program attaches a BPF program at
>> 'sys_epoll_pwait()', and collect half samples from it. perf itself never
> "collect half samples from it"? Can you rephrase this?

I mean half of epoll_pwait syscall will be captured. This test
case totally issue 111 calls.

Thank you.

Re: [PATCH 13/31] bpf tools: Load a program with different instances using preprocessor

[Wangnan (F)]

On 2015/10/30 6:44, Arnaldo Carvalho de Melo wrote:
> Em Wed, Oct 14, 2015 at 12:41:24PM +0000, Wang Nan escreveu:
>> In this patch, caller of libbpf is able to control the loaded programs
>> by installing a preprocessor callback for a BPF program. With
>> preprocessor, different instances can be created from one BPF program.
> Why would one want to do that? I'm new to eBPF, as many other here, I
> guess, so giving some more context as to what will this achieve will
> help me in processing these patches.
>   

I'll explain at the bottom of this reply.

>> This patch will be used by perf to generate different prologue for
>> different 'struct probe_trace_event' instances matched by one
>> 'struct perf_probe_event'.
>>
>> bpf_program__set_prep() is added to support this feature. Caller
>> should pass libbpf the number of instances should be created and a
>                                            that
>> preprocessor function which will be called when doing real loading.
>                                                        the
>> The callback should return instructions arrays for each instances.
>                                  instruction arrays for each instance.
>> fd field in bpf_programs is replaced by instance, which has an nr field
> The fd field in the bpf_programs struct is replaced by instance
> (instance of what?)

Instance of program. It is a new concept introduced in
this patch. Before this patch, one program create one fd.
This patch enables one program to create multiple fds. Each
fd is a program instance.

In summary, this and following patches try to solve a important
problem in such tracing tool that: how to access kernel data
in BPF program.

Our core methode on this is BPF prologue that we automatically
generate some BPF asm code for fetching them. However, after we
implement it another problem arises.

We found that there are many cases that multiple entries share
one symbol. Most of the time caused by function inlining. We also
support using glob to match multiple symbol. For example:

SEC("func=my_func_*_abcd arg1")
int func(...) { }

In the above example, it is highly possible that arg1 exist for
all my_func_*_abcd, but in different registers or on different
place in the stack.

When probing, perf would create one tev for each actual probing
point. Due to prologues for fetching arg1 are different, we need
to create multiple BPF prologue (so different BPF fd) for each
probing point. This is why we need this patch: Load a program with
different instances.

I have to admit that the design of this patch is not very easy to
be understand. However I believe it is the best way we can find.
The biggest reason of current design is that I don't want libbpf
be bounded with perf tightly. The prologue generation should be done
by perf, libbpf should never though about it. What it should provide
is to give caller (perf) a way to decorate the BPF binary in multiple
ways for different instances. Therefore I let caller to inject a
helper (preprocessor) for adjusting the code. What the caller need
to tell libbpf is the total instances it want to create, and the
preprocessor should maintain the mapping between the No. of instance
and the binary BPF program it want to create.

For instance, as the caller of libbpf, perf should know how
to generate prologue for my_func_a_abcd and my_func_b_abcd, then it
should maintain a mapping:

No. 0 --> my_func_a_abcd
No. 1 --> my_func_b_abcd

Then it need to tell libbpf that it would create 2 instances of a
BPF program. In the preprocessor perf inject into libbpf, it should
generate prologue for each probing point according to the ID.

Thank you.

Re: [PATCH 10/31] perf test: Enforce LLVM test for BPF test

[Arnaldo Carvalho de Melo]

Em Thu, Oct 15, 2015 at 07:58:38PM +0800, Wangnan (F) escreveu:
> >>+void test__llvm_prepare(void)
> >>+{
> >>+	p_test_llvm__bpf_result = mmap(NULL, SHARED_BUF_INIT_SIZE,
> >>+				       PROT_READ | PROT_WRITE,
> >>+				       MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> >>+	if (!p_test_llvm__bpf_result)
> >It should check MAP_FAILED instead.
> >
> 
> Fixed by this way:

Can you please try refreshing this patchset on top of what is now in
acme/perf/core?

Also why do we need those struct test->{prepare,cleanup} pointers? You
introduced it and then, on the next patch that touches 'perf test' and
uses test__llvm_{prepare,cleanup} you call them directly, which I think
should be enough, i.e. keep them as functions to call from inside
the test called from run_test(), right?

- Arnaldo
 
> diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
> index e722e8a..25ddeaf 100644
> --- a/tools/perf/tests/llvm.c
> +++ b/tools/perf/tests/llvm.c
> @@ -199,12 +199,15 @@ void test__llvm_prepare(void)
> 
>         for (i = 0; llvm_testcases[i].source; i++) {
>                 struct test_llvm__bpf_result *result;
> +               void *p;
> 
> -               result = mmap(NULL, SHARED_BUF_INIT_SIZE,
> -                             PROT_READ | PROT_WRITE,
> -                             MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> -               if (!result)
> +               p = mmap(NULL, SHARED_BUF_INIT_SIZE,
> +                        PROT_READ | PROT_WRITE,
> +                        MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> +               if (p == MAP_FAILED)
>                         return;
> +
> +               result = p;
>                 memset((void *)result, '\0', SHARED_BUF_INIT_SIZE);
> 
>                 llvm_testcases[i].result = result;
> 
> Thank you.

Re: [PATCH 10/31] perf test: Enforce LLVM test for BPF test

[Wangnan (F)]

On 2015/11/4 2:24, Arnaldo Carvalho de Melo wrote:
> Em Thu, Oct 15, 2015 at 07:58:38PM +0800, Wangnan (F) escreveu:
>>>> +void test__llvm_prepare(void)
>>>> +{
>>>> +	p_test_llvm__bpf_result = mmap(NULL, SHARED_BUF_INIT_SIZE,
>>>> +				       PROT_READ | PROT_WRITE,
>>>> +				       MAP_SHARED | MAP_ANONYMOUS, -1, 0);
>>>> +	if (!p_test_llvm__bpf_result)
>>> It should check MAP_FAILED instead.
>>>
>> Fixed by this way:
> Can you please try refreshing this patchset on top of what is now in
> acme/perf/core?
>
> Also why do we need those struct test->{prepare,cleanup} pointers? You
> introduced it and then, on the next patch that touches 'perf test' and
> uses test__llvm_{prepare,cleanup} you call them directly, which I think
> should be enough, i.e. keep them as functions to call from inside
> the test called from run_test(), right?
>
> - Arnaldo
>   

This prepare/cleanup functions are introduced because I want BPF
test reuse the result of LLVM test, so it don't need to compile
those BPF scripts twice. This is the reason I use shared
memory. However, now I think compiling twice is acceptable and
can make things simpler.

I'll update this patchset in my next pull request.

Thank you.

[GIT PULL 00/31] perf tools: filtering events using eBPF programs

[Wang Nan]

Hi Arnaldo and Ingo,

Several small proglems are fixed based on yesterday's pull request. Please
see below. Since patch order is changed (original 20/32 and 32/32 are
dropped), I decide to send all of them again. Sorry for the noisy.

In addition: I collect a cross-compiling fix I posted yesterday into this
cset (the last one).

The following changes since commit 2c07144dfce366e21465cc7b0ada9f0b6dc7b7ed:

  perf evlist: Add backpointer for perf_env to evlist (2015-08-28 14:54:14 -0300)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/pi3orama/linux tags/perf-ebpf-for-acme-20150829

for you to fetch changes up to d4a337392b3724899a084170d9ea36a8e2392097:

  tools lib traceevent: Support function __get_dynamic_array_len (2015-08-29 02:57:40 +0000)

----------------------------------------------------------------
perf BPF related improvements and bugfix:

 - Rebase to Arnaldo's newest perf/core.

 - Fix a missing include in builtin-trace.c.

 - Drop patch 'perf tools: Fix probe-event.h include' since
   the problem has been fixed by commit 5a023b57.

 - Fix a cross compiling error (introduced by inter pt).

 - Drop patch 'bpf: Introduce function for outputing data to
   perf event' because we want to do better.

Signed-off-by: Wang Nan <wangnan0@huawei.com>

----------------------------------------------------------------
He Kuang (4):
      perf tools: Move linux/filter.h to tools/include
      perf tools: Introduce arch_get_reg_info() for x86
      perf record: Support custom vmlinux path
      tools lib traceevent: Support function __get_dynamic_array_len

Wang Nan (27):
      bpf tools: New API to get name from a BPF object
      perf tools: Don't set cmdline_group_boundary if no evsel is collected
      perf tools: Introduce dummy evsel
      perf tools: Make perf depend on libbpf
      perf ebpf: Add the libbpf glue
      perf tools: Enable passing bpf object file to --event
      perf probe: Attach trace_probe_event with perf_probe_event
      perf record, bpf: Parse and probe eBPF programs probe points
      perf bpf: Collect 'struct perf_probe_event' for bpf_program
      perf record: Load all eBPF object into kernel
      perf tools: Add bpf_fd field to evsel and config it
      perf tools: Allow filter option to be applied to bof object
      perf tools: Attach eBPF program to perf event
      perf tools: Suppress probing messages when probing by BPF loading
      perf record: Add clang options for compiling BPF scripts
      perf tools: Infrastructure for compiling scriptlets when passing '.c' to --event
      perf tests: Enforce LLVM test for BPF test
      perf test: Add 'perf test BPF'
      bpf tools: Load a program with different instances using preprocessor
      perf probe: Reset args and nargs for probe_trace_event when failure
      perf tools: Add BPF_PROLOGUE config options for further patches
      perf tools: Add prologue for BPF programs for fetching arguments
      perf tools: Generate prologue for BPF programs
      perf tools: Use same BPF program if arguments are identical
      perf probe: Init symbol as kprobe
      perf tools: Support attach BPF program on uprobe events
      perf tools: Fix cross compiling error

 tools/build/Makefile.feature                       |   6 +-
 tools/include/linux/filter.h                       | 237 +++++++
 tools/lib/bpf/libbpf.c                             | 168 ++++-
 tools/lib/bpf/libbpf.h                             |  26 +-
 tools/lib/traceevent/event-parse.c                 |  56 +-
 tools/lib/traceevent/event-parse.h                 |   1 +
 tools/perf/MANIFEST                                |   4 +
 tools/perf/Makefile.perf                           |  19 +-
 tools/perf/arch/x86/Makefile                       |   1 +
 tools/perf/arch/x86/util/Build                     |   2 +
 tools/perf/arch/x86/util/dwarf-regs.c              | 104 ++-
 tools/perf/builtin-probe.c                         |   4 +-
 tools/perf/builtin-record.c                        |  64 +-
 tools/perf/builtin-stat.c                          |   9 +-
 tools/perf/builtin-top.c                           |  11 +-
 tools/perf/builtin-trace.c                         |   7 +-
 tools/perf/config/Makefile                         |  31 +-
 tools/perf/tests/Build                             |  10 +-
 tools/perf/tests/bpf-script-example.c              |  44 ++
 tools/perf/tests/bpf.c                             | 170 +++++
 tools/perf/tests/builtin-test.c                    |  12 +
 tools/perf/tests/llvm.c                            | 125 +++-
 tools/perf/tests/llvm.h                            |  15 +
 tools/perf/tests/make                              |   4 +-
 tools/perf/tests/tests.h                           |   3 +
 tools/perf/util/Build                              |   4 +-
 tools/perf/util/bpf-loader.c                       | 730 +++++++++++++++++++++
 tools/perf/util/bpf-loader.h                       |  95 +++
 tools/perf/util/bpf-prologue.c                     | 442 +++++++++++++
 tools/perf/util/bpf-prologue.h                     |  34 +
 tools/perf/util/evlist.c                           | 107 +++
 tools/perf/util/evlist.h                           |   2 +
 tools/perf/util/evsel.c                            |  49 ++
 tools/perf/util/evsel.h                            |   7 +
 tools/perf/util/include/dwarf-regs.h               |   7 +
 tools/perf/util/parse-events.c                     |  73 ++-
 tools/perf/util/parse-events.h                     |   4 +
 tools/perf/util/parse-events.l                     |   6 +
 tools/perf/util/parse-events.y                     |  29 +-
 tools/perf/util/probe-event.c                      |  79 ++-
 tools/perf/util/probe-event.h                      |   7 +-
 tools/perf/util/probe-file.c                       |   5 +-
 tools/perf/util/probe-finder.c                     |   4 +
 .../perf/util/scripting-engines/trace-event-perl.c |   1 +
 .../util/scripting-engines/trace-event-python.c    |   1 +
 45 files changed, 2698 insertions(+), 121 deletions(-)
 create mode 100644 tools/include/linux/filter.h
 create mode 100644 tools/perf/tests/bpf-script-example.c
 create mode 100644 tools/perf/tests/bpf.c
 create mode 100644 tools/perf/tests/llvm.h
 create mode 100644 tools/perf/util/bpf-loader.c
 create mode 100644 tools/perf/util/bpf-loader.h
 create mode 100644 tools/perf/util/bpf-prologue.c
 create mode 100644 tools/perf/util/bpf-prologue.h

-- 
2.1.0