[PATCH v2 00/18] perf clang: Support compiling BPF script on the fly

[PATCH v2 00/18] perf clang: Support compiling BPF script on the fly

[Wang Nan]

This patch add builtin clang, allow perf compile BPF scripts on the fly.
This is the first step to implement what I announced at LinuxCon 2016 NA:

http://events.linuxfoundation.org/sites/events/files/slides/Performance%20Monitoring%20and%20Analysis%20Using%20perf%20and%20BPF_1.pdf

Compare with v1:
 1. Fix API usage so can be built at Ubuntu 16.04 (with llvm-dev and
    libclang-dev installed)
 2. Introduce default include files so BPF script writer doesn't need
    define many BPF functions by their own.

Test:

 # cat ./test_bpf_output.c
 /************************ BEGIN **************************/
 #include <uapi/linux/bpf.h>
 struct bpf_map_def SEC("maps") __bpf_stdout__ = {
        .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
        .key_size = sizeof(int),
        .value_size = sizeof(u32),
        .max_entries = __NR_CPUS__,
 };

 static inline int __attribute__((always_inline))
 func(void *ctx, int type)
 {
	char output_str[] = "Raise a BPF event!";

        bpf_perf_event_output(ctx, &__bpf_stdout__, bpf_get_smp_processor_id(),
			  &output_str, sizeof(output_str));
        return 0;
 }
 SEC("func_begin=sys_nanosleep")
 int func_begin(void *ctx) {return func(ctx, 1);}
 SEC("func_end=sys_nanosleep%return")
 int func_end(void *ctx) { return func(ctx, 2);}
 char _license[] SEC("license") = "GPL";
 int _version SEC("version") = LINUX_VERSION_CODE;
 /************************* END ***************************/
 # perf trace --event ./test_bpf_output.c usleep 10
 ...
 0.449 ( 0.002 ms): usleep/827 getuid(                                                               ) = 0
 0.482 ( 0.006 ms): usleep/827 nanosleep(rqtp: 0x7ffecc22fa50                                        ) ...
 0.482 (         ): __bpf_stdout__:Raise a BPF event!..)
 0.555 (         ): __bpf_stdout__:Raise a BPF event!..)
 0.557 ( 0.081 ms): usleep/827  ... [continued]: nanosleep() = 0
 0.562 ( 0.000 ms): usleep/827 exit_group(                                                           )


v1 can be found at:
  https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1238358.html
  (should be http://lkml.kernel.org/g/1474635001-153850-1-git-send-email-wangnan0@huawei.com
   but the link is broken)

Wang Nan (18):
  tools build: Support compiling C++ source file
  perf tools: Add feature detection for g++
  perf tools: Add feature detection for LLVM
  perf tools: Add feature detection for clang
  perf build: Add clang and llvm compile and linking support
  perf clang: Add builtin clang support ant test case
  perf clang: Use real file system for #include
  perf clang: Allow passing CFLAGS to builtin clang
  perf clang: Update test case to use real BPF script
  perf clang: Support compile IR to BPF object and add testcase
  perf tools: Extract helpers in llvm-utils.c
  perf bpf: Compile BPF script use builtin clang support
  perf clang: Pass full path to builtin clang
  perf clang: Pass CFLAGS to builtin clang
  perf clang: Link BPF functions declaration into perf
  perf clang: Declare BPF functions for BPF scripts automatically
  perf clang: Include helpers to BPF scripts
  perf clang: Define PERF_BUILTIN_CLANG for builtin clang compiling

 tools/build/Build.include                     |   1 +
 tools/build/Makefile.build                    |   7 +
 tools/build/Makefile.feature                  |   2 +-
 tools/build/feature/Makefile                  |  28 ++-
 tools/build/feature/test-clang.cpp            |  21 ++
 tools/build/feature/test-cxx.cpp              |  15 ++
 tools/build/feature/test-llvm.cpp             |   8 +
 tools/perf/Makefile.config                    |  62 ++++-
 tools/perf/Makefile.perf                      |  23 +-
 tools/perf/tests/Build                        |   1 +
 tools/perf/tests/bpf-script-example.c         |  20 +-
 tools/perf/tests/bpf-script-test-kbuild.c     |   2 +
 tools/perf/tests/bpf-script-test-prologue.c   |   4 +
 tools/perf/tests/bpf-script-test-relocation.c |  20 +-
 tools/perf/tests/builtin-test.c               |   9 +
 tools/perf/tests/clang.c                      |  46 ++++
 tools/perf/tests/llvm-cxx.h                   |  13 +
 tools/perf/tests/make                         |   2 +
 tools/perf/tests/tests.h                      |   3 +
 tools/perf/util/Build                         |   2 +
 tools/perf/util/bpf-loader.c                  |  15 +-
 tools/perf/util/c++/Build                     |   4 +
 tools/perf/util/c++/bpf-funcs-str.c           | 214 +++++++++++++++++
 tools/perf/util/c++/bpf-helper-str.c          |  13 +
 tools/perf/util/c++/clang-bpf-includes.h      |  13 +
 tools/perf/util/c++/clang-c.h                 |  43 ++++
 tools/perf/util/c++/clang-test.cpp            |  62 +++++
 tools/perf/util/c++/clang.cpp                 | 329 ++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h                   |  26 ++
 tools/perf/util/llvm-utils-cxx.h              |  14 ++
 tools/perf/util/llvm-utils.c                  |  70 +++++-
 tools/perf/util/llvm-utils.h                  |   7 +-
 tools/perf/util/util-cxx.h                    |  26 ++
 33 files changed, 1078 insertions(+), 47 deletions(-)
 create mode 100644 tools/build/feature/test-clang.cpp
 create mode 100644 tools/build/feature/test-cxx.cpp
 create mode 100644 tools/build/feature/test-llvm.cpp
 create mode 100644 tools/perf/tests/clang.c
 create mode 100644 tools/perf/tests/llvm-cxx.h
 create mode 100644 tools/perf/util/c++/Build
 create mode 100644 tools/perf/util/c++/bpf-funcs-str.c
 create mode 100644 tools/perf/util/c++/bpf-helper-str.c
 create mode 100644 tools/perf/util/c++/clang-bpf-includes.h
 create mode 100644 tools/perf/util/c++/clang-c.h
 create mode 100644 tools/perf/util/c++/clang-test.cpp
 create mode 100644 tools/perf/util/c++/clang.cpp
 create mode 100644 tools/perf/util/c++/clang.h
 create mode 100644 tools/perf/util/llvm-utils-cxx.h
 create mode 100644 tools/perf/util/util-cxx.h

Cc: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
-- 
1.8.3.4

[PATCH v2 01/18] tools build: Support compiling C++ source file

[Wang Nan]

Add new rule to compile .cpp file to .o use g++. C++ support is
required for built-in clang and LLVM support.

Linker side support will be introduced by following commits.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/Build.include  | 1 +
 tools/build/Makefile.build | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/tools/build/Build.include b/tools/build/Build.include
index 4d000bc..b9f014d 100644
--- a/tools/build/Build.include
+++ b/tools/build/Build.include
@@ -90,3 +90,4 @@ if_changed = $(if $(strip $(any-prereq) $(arg-check)),             \
 # - per object C flags
 # - BUILD_STR macro to allow '-D"$(variable)"' constructs
 c_flags = -Wp,-MD,$(depfile),-MT,$@ $(CFLAGS) -D"BUILD_STR(s)=\#s" $(CFLAGS_$(basetarget).o) $(CFLAGS_$(obj))
+cxx_flags = -Wp,-MD,$(depfile),-MT,$@ $(CXXFLAGS) -D"BUILD_STR(s)=\#s" $(CXXFLAGS_$(basetarget).o) $(CXXFLAGS_$(obj))
diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
index 27f3583..b84be2e 100644
--- a/tools/build/Makefile.build
+++ b/tools/build/Makefile.build
@@ -58,6 +58,9 @@ quiet_cmd_mkdir = MKDIR    $(dir $@)
 quiet_cmd_cc_o_c = CC       $@
       cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $<
 
+quiet_cmd_cxx_o_c = CXX      $@
+      cmd_cxx_o_c = $(CXX) $(cxx_flags) -c -o $@ $<
+
 quiet_cmd_cpp_i_c = CPP      $@
       cmd_cpp_i_c = $(CC) $(c_flags) -E -o $@ $<
 
@@ -77,6 +80,10 @@ $(OUTPUT)%.o: %.c FORCE
 	$(call rule_mkdir)
 	$(call if_changed_dep,cc_o_c)
 
+$(OUTPUT)%.o: %.cpp FORCE
+	$(call rule_mkdir)
+	$(call if_changed_dep,cxx_o_c)
+
 $(OUTPUT)%.o: %.S FORCE
 	$(call rule_mkdir)
 	$(call if_changed_dep,cc_o_c)
-- 
1.8.3.4

[PATCH v2 02/18] perf tools: Add feature detection for g++

[Wang Nan]

Check if g++ is available. The result will be used by builtin clang and
LLVM support. Since LLVM requires C++11, this feature detector checks
std::move().

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/Makefile.feature     |  2 +-
 tools/build/feature/Makefile     | 10 +++++++++-
 tools/build/feature/test-cxx.cpp | 15 +++++++++++++++
 3 files changed, 25 insertions(+), 2 deletions(-)
 create mode 100644 tools/build/feature/test-cxx.cpp

diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index a120c6b..ae52e02 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -7,7 +7,7 @@ endif
 
 feature_check = $(eval $(feature_check_code))
 define feature_check_code
-  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
+  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
 endef
 
 feature_set = $(eval $(feature_set_code))
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index a0b29a3..ac9c477 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -46,11 +46,13 @@ FILES=					\
 	test-lzma.bin			\
 	test-bpf.bin			\
 	test-get_cpuid.bin		\
-	test-sdt.bin
+	test-sdt.bin			\
+	test-cxx.bin
 
 FILES := $(addprefix $(OUTPUT),$(FILES))
 
 CC := $(CROSS_COMPILE)gcc -MD
+CXX := $(CROSS_COMPILE)g++ -MD
 PKG_CONFIG := $(CROSS_COMPILE)pkg-config
 
 all: $(FILES)
@@ -58,6 +60,9 @@ all: $(FILES)
 __BUILD = $(CC) $(CFLAGS) -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $(LDFLAGS)
   BUILD = $(__BUILD) > $(@:.bin=.make.output) 2>&1
 
+__BUILDXX = $(CXX) $(CXXFLAGS) -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS)
+  BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
+
 ###############################
 
 $(OUTPUT)test-all.bin:
@@ -217,6 +222,9 @@ $(OUTPUT)test-bpf.bin:
 $(OUTPUT)test-sdt.bin:
 	$(BUILD)
 
+$(OUTPUT)test-cxx.bin:
+	$(BUILDXX) -std=gnu++11
+
 -include $(OUTPUT)*.d
 
 ###############################
diff --git a/tools/build/feature/test-cxx.cpp b/tools/build/feature/test-cxx.cpp
new file mode 100644
index 0000000..b1dee9a
--- /dev/null
+++ b/tools/build/feature/test-cxx.cpp
@@ -0,0 +1,15 @@
+#include <iostream>
+#include <memory>
+
+static void print_str(std::string s)
+{
+	std::cout << s << std::endl;
+}
+
+int main()
+{
+	std::string s("Hello World!");
+	print_str(std::move(s));
+	std::cout << "|" << s << "|" << std::endl;
+	return 0;
+}
-- 
1.8.3.4

[PATCH v2 03/18] perf tools: Add feature detection for LLVM

[Wang Nan]

Check if basic LLVM compiling environment is ready.

Use llvm-config to detect include and library directories. Avoid using
'llvm-config --cxxflags' because its result contain some unwanted flags
like --sysroot (if LLVM is built by yocto).

Use '?=' to set LLVM_CONFIG, so explicitly passing LLVM_CONFIG to make
would override it.

Use 'llvm-config --libs BPF' to check if BPF backend is compiled in.
Since now BPF bytecode is the only required backend, no need to waste
time linking llvm and clang if BPF backend is missing. This
also introduce an implicit requirement that LLVM should be new enough.
Old LLVM doesn't support BPF backend.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/feature/Makefile      | 8 ++++++++
 tools/build/feature/test-llvm.cpp | 8 ++++++++
 2 files changed, 16 insertions(+)
 create mode 100644 tools/build/feature/test-llvm.cpp

diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index ac9c477..fd2f3e2 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -54,6 +54,7 @@ FILES := $(addprefix $(OUTPUT),$(FILES))
 CC := $(CROSS_COMPILE)gcc -MD
 CXX := $(CROSS_COMPILE)g++ -MD
 PKG_CONFIG := $(CROSS_COMPILE)pkg-config
+LLVM_CONFIG ?= llvm-config
 
 all: $(FILES)
 
@@ -225,6 +226,13 @@ $(OUTPUT)test-sdt.bin:
 $(OUTPUT)test-cxx.bin:
 	$(BUILDXX) -std=gnu++11
 
+$(OUTPUT)test-llvm.bin:
+	$(BUILDXX) -std=gnu++11 					\
+		-I$(shell $(LLVM_CONFIG) --includedir) 		\
+		-L$(shell $(LLVM_CONFIG) --libdir)		\
+		$(shell $(LLVM_CONFIG) --libs Core BPF)		\
+		$(shell $(LLVM_CONFIG) --system-libs)
+
 -include $(OUTPUT)*.d
 
 ###############################
diff --git a/tools/build/feature/test-llvm.cpp b/tools/build/feature/test-llvm.cpp
new file mode 100644
index 0000000..d8d2cee
--- /dev/null
+++ b/tools/build/feature/test-llvm.cpp
@@ -0,0 +1,8 @@
+#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/raw_ostream.h"
+int main()
+{
+	llvm::errs() << "Hello World!\n";
+	llvm::llvm_shutdown();
+	return 0;
+}
-- 
1.8.3.4

[PATCH v2 04/18] perf tools: Add feature detection for clang

[Wang Nan]

Check if basic clang compiling environment is ready.

Doesn't like 'llvm-config --libs' which can returns llvm libraries in
right order and duplicates some libraries if necessary, there's no
correspondence for clang libraries (-lclangxxx). to avoid extra
complexity and to avoid new clang breaking libraries ordering, use
--start-group and --end-group.

In this test case, manually identify required clang libs and hope it
to be stable. Putting all clang libraries here is possible (use
make's wildcard), but then feature checking becomes very slow.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/feature/Makefile       | 10 ++++++++++
 tools/build/feature/test-clang.cpp | 21 +++++++++++++++++++++
 2 files changed, 31 insertions(+)
 create mode 100644 tools/build/feature/test-clang.cpp

diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index fd2f3e2..0af90dd 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -233,6 +233,16 @@ $(OUTPUT)test-llvm.bin:
 		$(shell $(LLVM_CONFIG) --libs Core BPF)		\
 		$(shell $(LLVM_CONFIG) --system-libs)
 
+$(OUTPUT)test-clang.bin:
+	$(BUILDXX) -std=gnu++11 					\
+		-I$(shell $(LLVM_CONFIG) --includedir) 		\
+		-L$(shell $(LLVM_CONFIG) --libdir)		\
+		-Wl,--start-group -lclangBasic -lclangDriver	\
+		  -lclangFrontend -lclangEdit -lclangLex	\
+		  -lclangAST -Wl,--end-group 			\
+		$(shell $(LLVM_CONFIG) --libs Core option)	\
+		$(shell $(LLVM_CONFIG) --system-libs)
+
 -include $(OUTPUT)*.d
 
 ###############################
diff --git a/tools/build/feature/test-clang.cpp b/tools/build/feature/test-clang.cpp
new file mode 100644
index 0000000..e23c1b1
--- /dev/null
+++ b/tools/build/feature/test-clang.cpp
@@ -0,0 +1,21 @@
+#include "clang/Basic/VirtualFileSystem.h"
+#include "clang/Driver/Driver.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "llvm/ADT/IntrusiveRefCntPtr.h"
+#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace clang;
+using namespace clang::driver;
+
+int main()
+{
+	IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());
+	IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
+
+	DiagnosticsEngine Diags(DiagID, &*DiagOpts);
+	Driver TheDriver("test", "bpf-pc-linux", Diags);
+
+	llvm::llvm_shutdown();
+	return 0;
+}
-- 
1.8.3.4

[PATCH v2 05/18] perf build: Add clang and llvm compile and linking support

[Wang Nan]

Add necessary c++ flags and link libraries to support builtin clang
and LLVM. Add all llvm and clang libraries, so don't need to worry
about clang changes its libraries setting. However, linking perf
would take much longer than usual.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/Makefile.config | 35 +++++++++++++++++++++++++++++++++++
 tools/perf/Makefile.perf   | 23 ++++++++++++++++++++++-
 tools/perf/tests/make      |  2 ++
 3 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 72edf83..b99c597 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -136,6 +136,7 @@ endif
 # Treat warnings as errors unless directed not to
 ifneq ($(WERROR),0)
   CFLAGS += -Werror
+  CXXFLAGS += -Werror
 endif
 
 ifndef DEBUG
@@ -182,6 +183,13 @@ CFLAGS += -Wall
 CFLAGS += -Wextra
 CFLAGS += -std=gnu99
 
+CXXFLAGS += -std=gnu++11 -fno-exceptions -fno-rtti
+CXXFLAGS += -Wall
+CXXFLAGS += -fno-omit-frame-pointer
+CXXFLAGS += -ggdb3
+CXXFLAGS += -funwind-tables
+CXXFLAGS += -Wno-strict-aliasing
+
 # Enforce a non-executable stack, as we may regress (again) in the future by
 # adding assembler files missing the .GNU-stack linker note.
 LDFLAGS += -Wl,-z,noexecstack
@@ -758,6 +766,33 @@ ifndef NO_AUXTRACE
   endif
 endif
 
+USE_CXX = 0
+USE_CLANGLLVM = 0
+ifdef LIBCLANGLLVM
+  $(call feature_check,cxx)
+  ifneq ($(feature-cxx), 1)
+    msg := $(warning No g++ found, disable clang and llvm support. Please install g++)
+  else
+    $(call feature_check,llvm)
+    ifneq ($(feature-llvm), 1)
+      msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev)
+    else
+      $(call feature_check,clang)
+      ifneq ($(feature-clang), 1)
+        msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev)
+      else
+        CFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT
+        CXXFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT -I$(shell $(LLVM_CONFIG) --includedir)
+        $(call detected,CONFIG_CXX)
+        $(call detected,CONFIG_CLANGLLVM)
+	USE_CXX = 1
+	USE_LLVM = 1
+	USE_CLANG = 1
+      endif
+    endif
+  endif
+endif
+
 # Among the variables below, these:
 #   perfexecdir
 #   template_dir
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index d710db1..d666fed 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -86,6 +86,10 @@ include ../scripts/utilities.mak
 #
 # Define FEATURES_DUMP to provide features detection dump file
 # and bypass the feature detection
+#
+# Define LIBCLANGLLVM if you DO want builtin clang and llvm support.
+# When selected, pass LLVM_CONFIG=/path/to/llvm-config to `make' if
+# llvm-config is not in $PATH.
 
 # As per kernel Makefile, avoid funny character set dependencies
 unexport LC_ALL
@@ -141,10 +145,12 @@ endef
 $(call allow-override,CC,$(CROSS_COMPILE)gcc)
 $(call allow-override,AR,$(CROSS_COMPILE)ar)
 $(call allow-override,LD,$(CROSS_COMPILE)ld)
+$(call allow-override,CXX,$(CROSS_COMPILE)g++)
 
 LD += $(EXTRA_LDFLAGS)
 
 PKG_CONFIG = $(CROSS_COMPILE)pkg-config
+LLVM_CONFIG ?= llvm-config
 
 RM      = rm -f
 LN      = ln -f
@@ -326,6 +332,21 @@ endif
 
 LIBS = -Wl,--whole-archive $(PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
 
+ifeq ($(USE_CLANG), 1)
+  CLANGLIBS_LIST = AST Basic CodeGen Driver Frontend Lex Tooling Edit Sema Analysis Parse Serialization
+  LIBCLANG = $(foreach l,$(CLANGLIBS_LIST),$(wildcard $(shell $(LLVM_CONFIG) --libdir)/libclang$(l).a))
+  LIBS += -Wl,--start-group $(LIBCLANG) -Wl,--end-group
+endif
+
+ifeq ($(USE_LLVM), 1)
+  LIBLLVM = $(shell $(LLVM_CONFIG) --libs all) $(shell $(LLVM_CONFIG) --system-libs)
+  LIBS += -L$(shell $(LLVM_CONFIG) --libdir) $(LIBLLVM)
+endif
+
+ifeq ($(USE_CXX), 1)
+  LIBS += -lstdc++
+endif
+
 export INSTALL SHELL_PATH
 
 ### Build rules
@@ -344,7 +365,7 @@ strip: $(PROGRAMS) $(OUTPUT)perf
 
 PERF_IN := $(OUTPUT)perf-in.o
 
-export srctree OUTPUT RM CC LD AR CFLAGS V BISON FLEX AWK
+export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
 include $(srctree)/tools/build/Makefile.include
 
 $(PERF_IN): prepare FORCE
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index 143f4d5..65eaf8f 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -83,6 +83,7 @@ make_no_libbpf	    := NO_LIBBPF=1
 make_no_libcrypto   := NO_LIBCRYPTO=1
 make_with_babeltrace:= LIBBABELTRACE=1
 make_no_sdt	    := NO_SDT=1
+make_with_clangllvm := LIBCLANGLLVM=1
 make_tags           := tags
 make_cscope         := cscope
 make_help           := help
@@ -139,6 +140,7 @@ run += make_no_libbionic
 run += make_no_auxtrace
 run += make_no_libbpf
 run += make_with_babeltrace
+run += make_with_clangllvm
 run += make_help
 run += make_doc
 run += make_perf_o
-- 
1.8.3.4

[PATCH v2 06/18] perf clang: Add builtin clang support ant test case

[Wang Nan]

Add basic clang support in clang.cpp and test__clang() testcase.
The first testcase checks if builtin clang is able to generate LLVM IR.

tests/clang.c is a proxy. Real testcase resides in utils/c++/clang-test.cpp
in c++ and exports C interface to perf test subsystem.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/tests/Build             |  1 +
 tools/perf/tests/builtin-test.c    |  9 ++++
 tools/perf/tests/clang.c           | 42 +++++++++++++++++
 tools/perf/tests/tests.h           |  3 ++
 tools/perf/util/Build              |  2 +
 tools/perf/util/c++/Build          |  2 +
 tools/perf/util/c++/clang-c.h      | 16 +++++++
 tools/perf/util/c++/clang-test.cpp | 31 ++++++++++++
 tools/perf/util/c++/clang.cpp      | 96 ++++++++++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h        | 16 +++++++
 10 files changed, 218 insertions(+)
 create mode 100644 tools/perf/tests/clang.c
 create mode 100644 tools/perf/util/c++/Build
 create mode 100644 tools/perf/util/c++/clang-c.h
 create mode 100644 tools/perf/util/c++/clang-test.cpp
 create mode 100644 tools/perf/util/c++/clang.cpp
 create mode 100644 tools/perf/util/c++/clang.h

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index dc51bc5..6a20004 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -42,6 +42,7 @@ perf-y += backward-ring-buffer.o
 perf-y += sdt.o
 perf-y += is_printable_array.o
 perf-y += bitmap.o
+perf-y += clang.o
 
 $(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build
 	$(call rule_mkdir)
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 778668a..b9b6053 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -230,6 +230,15 @@ static struct test generic_tests[] = {
 		.func = test__bitmap_print,
 	},
 	{
+		.desc = "Test builtin clang support",
+		.func = test__clang,
+		.subtest = {
+			.skip_if_fail	= true,
+			.get_nr		= test__clang_subtest_get_nr,
+			.get_desc	= test__clang_subtest_get_desc,
+		}
+	},
+	{
 		.func = NULL,
 	},
 };
diff --git a/tools/perf/tests/clang.c b/tools/perf/tests/clang.c
new file mode 100644
index 0000000..57ee160
--- /dev/null
+++ b/tools/perf/tests/clang.c
@@ -0,0 +1,42 @@
+#include "tests.h"
+#include "debug.h"
+#include "util.h"
+#include "c++/clang-c.h"
+
+static struct {
+	int (*func)(void);
+	const char *desc;
+} clang_testcase_table[] = {
+#ifdef HAVE_LIBCLANGLLVM_SUPPORT
+	{
+		.func = test__clang_to_IR,
+		.desc = "Test builtin clang compile C source to IR",
+	},
+#endif
+};
+
+int test__clang_subtest_get_nr(void)
+{
+	return (int)ARRAY_SIZE(clang_testcase_table);
+}
+
+const char *test__clang_subtest_get_desc(int i)
+{
+	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
+		return NULL;
+	return clang_testcase_table[i].desc;
+}
+
+#ifndef HAVE_LIBCLANGLLVM_SUPPORT
+int test__clang(int i __maybe_unused)
+{
+	return TEST_SKIP;
+}
+#else
+int test__clang(int i __maybe_unused)
+{
+	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
+		return TEST_FAIL;
+	return clang_testcase_table[i].func();
+}
+#endif
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 7c196c5..39a4887 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -91,6 +91,9 @@ int test__cpu_map_print(int subtest);
 int test__sdt_event(int subtest);
 int test__is_printable_array(int subtest);
 int test__bitmap_print(int subtest);
+int test__clang(int subtest);
+const char *test__clang_subtest_get_desc(int subtest);
+int test__clang_subtest_get_nr(void);
 
 #if defined(__arm__) || defined(__aarch64__)
 #ifdef HAVE_DWARF_UNWIND_SUPPORT
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index eb60e61..20044d0 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -123,6 +123,8 @@ libperf-$(CONFIG_LIBELF) += genelf.o
 libperf-$(CONFIG_LIBELF) += genelf_debug.o
 endif
 
+libperf-$(CONFIG_CXX) += c++/
+
 CFLAGS_config.o   += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 # avoid compiler warnings in 32-bit mode
 CFLAGS_genelf_debug.o  += -Wno-packed
diff --git a/tools/perf/util/c++/Build b/tools/perf/util/c++/Build
new file mode 100644
index 0000000..988fef1
--- /dev/null
+++ b/tools/perf/util/c++/Build
@@ -0,0 +1,2 @@
+libperf-$(CONFIG_CLANGLLVM) += clang.o
+libperf-$(CONFIG_CLANGLLVM) += clang-test.o
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
new file mode 100644
index 0000000..dcde4b5
--- /dev/null
+++ b/tools/perf/util/c++/clang-c.h
@@ -0,0 +1,16 @@
+#ifndef PERF_UTIL_CLANG_C_H
+#define PERF_UTIL_CLANG_C_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+extern void perf_clang__init(void);
+extern void perf_clang__cleanup(void);
+
+extern int test__clang_to_IR(void);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
new file mode 100644
index 0000000..3da6bfa
--- /dev/null
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -0,0 +1,31 @@
+#include "clang.h"
+#include "clang-c.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/LLVMContext.h"
+
+class perf_clang_scope {
+public:
+	explicit perf_clang_scope() {perf_clang__init();}
+	~perf_clang_scope() {perf_clang__cleanup();}
+};
+
+extern "C" {
+
+int test__clang_to_IR(void)
+{
+	perf_clang_scope _scope;
+
+	std::unique_ptr<llvm::Module> M =
+		perf::getModuleFromSource("perf-test.c",
+					  "int myfunc(void) {return 1;}");
+
+	if (!M)
+		return -1;
+
+	for (llvm::Function& F : *M)
+		if (F.getName() == "myfunc")
+			return 0;
+	return -1;
+}
+
+}
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
new file mode 100644
index 0000000..c17b117
--- /dev/null
+++ b/tools/perf/util/c++/clang.cpp
@@ -0,0 +1,96 @@
+/*
+ * llvm C frontend for perf. Support dynamically compile C file
+ *
+ * Inspired by clang example code:
+ * http://llvm.org/svn/llvm-project/cfe/trunk/examples/clang-interpreter/main.cpp
+ *
+ * Copyright (C) 2016 Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2016 Huawei Inc.
+ */
+
+#include "clang/CodeGen/CodeGenAction.h"
+#include "clang/Frontend/CompilerInvocation.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Option/Option.h"
+#include "llvm/Support/ManagedStatic.h"
+#include <memory>
+
+#include "clang.h"
+#include "clang-c.h"
+
+namespace perf {
+
+static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
+
+using namespace clang;
+
+static vfs::InMemoryFileSystem *
+buildVFS(StringRef& Name, StringRef& Content)
+{
+	vfs::InMemoryFileSystem *VFS = new vfs::InMemoryFileSystem(true);
+	VFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
+	return VFS;
+}
+
+static CompilerInvocation *
+createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
+{
+	llvm::opt::ArgStringList CCArgs {
+		"-cc1",
+		"-triple", "bpf-pc-linux",
+		"-fsyntax-only",
+		"-ferror-limit", "19",
+		"-fmessage-length", "127",
+		"-O2",
+		"-nostdsysteminc",
+		"-nobuiltininc",
+		"-vectorize-loops",
+		"-vectorize-slp",
+		"-Wno-unused-value",
+		"-Wno-pointer-sign",
+		"-x", "c"};
+	CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs);
+
+	FrontendOptions& Opts = CI->getFrontendOpts();
+	Opts.Inputs.clear();
+	Opts.Inputs.emplace_back(Path, IK_C);
+	return CI;
+}
+
+std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Name, StringRef Content)
+{
+	CompilerInstance Clang;
+	Clang.createDiagnostics();
+
+	IntrusiveRefCntPtr<vfs::FileSystem> VFS = buildVFS(Name, Content);
+	Clang.setVirtualFileSystem(&*VFS);
+
+	IntrusiveRefCntPtr<CompilerInvocation> CI =
+		createCompilerInvocation(Name, Clang.getDiagnostics());
+	Clang.setInvocation(&*CI);
+
+	std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx));
+	if (!Clang.ExecuteAction(*Act))
+		return std::unique_ptr<llvm::Module>(nullptr);
+
+	return Act->takeModule();
+}
+
+}
+
+extern "C" {
+void perf_clang__init(void)
+{
+	perf::LLVMCtx.reset(new llvm::LLVMContext());
+}
+
+void perf_clang__cleanup(void)
+{
+	perf::LLVMCtx.reset(nullptr);
+	llvm::llvm_shutdown();
+}
+}
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
new file mode 100644
index 0000000..f64483b
--- /dev/null
+++ b/tools/perf/util/c++/clang.h
@@ -0,0 +1,16 @@
+#ifndef PERF_UTIL_CLANG_H
+#define PERF_UTIL_CLANG_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include <memory>
+namespace perf {
+
+using namespace llvm;
+
+std::unique_ptr<Module>
+getModuleFromSource(StringRef Name, StringRef Content);
+
+}
+#endif
-- 
1.8.3.4

[PATCH v2 07/18] perf clang: Use real file system for #include

[Wang Nan]

Utilize clang's OverlayFileSystem facility to enable VFS passed to
CompilerInstance can access files in real file system, so '#include'
directive can be used. Add a new getModuleFromSource for real file.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang.cpp | 44 +++++++++++++++++++++++++++++++------------
 tools/perf/util/c++/clang.h   |  3 +++
 2 files changed, 35 insertions(+), 12 deletions(-)

diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index c17b117..cf96199 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -15,6 +15,7 @@
 #include "clang/Tooling/Tooling.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Option/Option.h"
+#include "llvm/Support/FileSystem.h"
 #include "llvm/Support/ManagedStatic.h"
 #include <memory>
 
@@ -27,14 +28,6 @@ static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
 
 using namespace clang;
 
-static vfs::InMemoryFileSystem *
-buildVFS(StringRef& Name, StringRef& Content)
-{
-	vfs::InMemoryFileSystem *VFS = new vfs::InMemoryFileSystem(true);
-	VFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
-	return VFS;
-}
-
 static CompilerInvocation *
 createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 {
@@ -60,17 +53,17 @@ createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 	return CI;
 }
 
-std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Name, StringRef Content)
+static std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Path,
+		    IntrusiveRefCntPtr<vfs::FileSystem> VFS)
 {
 	CompilerInstance Clang;
 	Clang.createDiagnostics();
 
-	IntrusiveRefCntPtr<vfs::FileSystem> VFS = buildVFS(Name, Content);
 	Clang.setVirtualFileSystem(&*VFS);
 
 	IntrusiveRefCntPtr<CompilerInvocation> CI =
-		createCompilerInvocation(Name, Clang.getDiagnostics());
+		createCompilerInvocation(Path, Clang.getDiagnostics());
 	Clang.setInvocation(&*CI);
 
 	std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx));
@@ -80,6 +73,33 @@ getModuleFromSource(StringRef Name, StringRef Content)
 	return Act->takeModule();
 }
 
+std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Name, StringRef Content)
+{
+	using namespace vfs;
+
+	llvm::IntrusiveRefCntPtr<OverlayFileSystem> OverlayFS(
+			new OverlayFileSystem(getRealFileSystem()));
+	llvm::IntrusiveRefCntPtr<InMemoryFileSystem> MemFS(
+			new InMemoryFileSystem(true));
+
+	/*
+	 * pushOverlay helps setting working dir for MemFS. Must call
+	 * before addFile.
+	 */
+	OverlayFS->pushOverlay(MemFS);
+	MemFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
+
+	return getModuleFromSource(Name, OverlayFS);
+}
+
+std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Path)
+{
+	IntrusiveRefCntPtr<vfs::FileSystem> VFS(vfs::getRealFileSystem());
+	return getModuleFromSource(Path, VFS);
+}
+
 }
 
 extern "C" {
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
index f64483b..90aff01 100644
--- a/tools/perf/util/c++/clang.h
+++ b/tools/perf/util/c++/clang.h
@@ -12,5 +12,8 @@ using namespace llvm;
 std::unique_ptr<Module>
 getModuleFromSource(StringRef Name, StringRef Content);
 
+std::unique_ptr<Module>
+getModuleFromSource(StringRef Path);
+
 }
 #endif
-- 
1.8.3.4

[PATCH v2 08/18] perf clang: Allow passing CFLAGS to builtin clang

[Wang Nan]

Improve getModuleFromSource() API to accept a cflags list. This feature
will be used to pass LINUX_VERSION_CODE and -I flags.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang-test.cpp |  5 +++--
 tools/perf/util/c++/clang.cpp      | 21 +++++++++++++--------
 tools/perf/util/c++/clang.h        |  8 ++++++--
 3 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
index 3da6bfa..0f484fb 100644
--- a/tools/perf/util/c++/clang-test.cpp
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -16,8 +16,9 @@ int test__clang_to_IR(void)
 	perf_clang_scope _scope;
 
 	std::unique_ptr<llvm::Module> M =
-		perf::getModuleFromSource("perf-test.c",
-					  "int myfunc(void) {return 1;}");
+		perf::getModuleFromSource({"-DRESULT=1"},
+					  "perf-test.c",
+					  "int myfunc(void) {return RESULT;}");
 
 	if (!M)
 		return -1;
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index cf96199..715ca0a 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -29,7 +29,8 @@ static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
 using namespace clang;
 
 static CompilerInvocation *
-createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
+createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path,
+			 DiagnosticsEngine& Diags)
 {
 	llvm::opt::ArgStringList CCArgs {
 		"-cc1",
@@ -45,6 +46,8 @@ createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 		"-Wno-unused-value",
 		"-Wno-pointer-sign",
 		"-x", "c"};
+
+	CCArgs.append(CFlags.begin(), CFlags.end());
 	CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs);
 
 	FrontendOptions& Opts = CI->getFrontendOpts();
@@ -54,8 +57,8 @@ createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 }
 
 static std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Path,
-		    IntrusiveRefCntPtr<vfs::FileSystem> VFS)
+getModuleFromSource(llvm::opt::ArgStringList CFlags,
+		    StringRef Path, IntrusiveRefCntPtr<vfs::FileSystem> VFS)
 {
 	CompilerInstance Clang;
 	Clang.createDiagnostics();
@@ -63,7 +66,8 @@ getModuleFromSource(StringRef Path,
 	Clang.setVirtualFileSystem(&*VFS);
 
 	IntrusiveRefCntPtr<CompilerInvocation> CI =
-		createCompilerInvocation(Path, Clang.getDiagnostics());
+		createCompilerInvocation(std::move(CFlags), Path,
+					 Clang.getDiagnostics());
 	Clang.setInvocation(&*CI);
 
 	std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx));
@@ -74,7 +78,8 @@ getModuleFromSource(StringRef Path,
 }
 
 std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Name, StringRef Content)
+getModuleFromSource(llvm::opt::ArgStringList CFlags,
+		    StringRef Name, StringRef Content)
 {
 	using namespace vfs;
 
@@ -90,14 +95,14 @@ getModuleFromSource(StringRef Name, StringRef Content)
 	OverlayFS->pushOverlay(MemFS);
 	MemFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
 
-	return getModuleFromSource(Name, OverlayFS);
+	return getModuleFromSource(std::move(CFlags), Name, OverlayFS);
 }
 
 std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Path)
+getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path)
 {
 	IntrusiveRefCntPtr<vfs::FileSystem> VFS(vfs::getRealFileSystem());
-	return getModuleFromSource(Path, VFS);
+	return getModuleFromSource(std::move(CFlags), Path, VFS);
 }
 
 }
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
index 90aff01..b4fc2a9 100644
--- a/tools/perf/util/c++/clang.h
+++ b/tools/perf/util/c++/clang.h
@@ -4,16 +4,20 @@
 #include "llvm/ADT/StringRef.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
+#include "llvm/Option/Option.h"
 #include <memory>
+
 namespace perf {
 
 using namespace llvm;
 
 std::unique_ptr<Module>
-getModuleFromSource(StringRef Name, StringRef Content);
+getModuleFromSource(opt::ArgStringList CFlags,
+		    StringRef Name, StringRef Content);
 
 std::unique_ptr<Module>
-getModuleFromSource(StringRef Path);
+getModuleFromSource(opt::ArgStringList CFlags,
+		    StringRef Path);
 
 }
 #endif
-- 
1.8.3.4

[PATCH v2 09/18] perf clang: Update test case to use real BPF script

[Wang Nan]

Allow C++ code to use util.h and tests/llvm.h. Let perf test
to compile a real BPF script.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/Makefile.config         | 27 +++++++++++++++------------
 tools/perf/tests/llvm-cxx.h        | 13 +++++++++++++
 tools/perf/util/c++/clang-test.cpp | 17 ++++++++++++++---
 tools/perf/util/util-cxx.h         | 26 ++++++++++++++++++++++++++
 4 files changed, 68 insertions(+), 15 deletions(-)
 create mode 100644 tools/perf/tests/llvm-cxx.h
 create mode 100644 tools/perf/util/util-cxx.h

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index b99c597..3cf3cb5 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -212,24 +212,27 @@ ifeq ($(DEBUG),0)
   endif
 endif
 
-CFLAGS += -I$(src-perf)/util/include
-CFLAGS += -I$(src-perf)/arch/$(ARCH)/include
-CFLAGS += -I$(srctree)/tools/include/uapi
-CFLAGS += -I$(srctree)/tools/include/
-CFLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/uapi
-CFLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/
-CFLAGS += -I$(srctree)/tools/arch/$(ARCH)/
+INC_FLAGS += -I$(src-perf)/util/include
+INC_FLAGS += -I$(src-perf)/arch/$(ARCH)/include
+INC_FLAGS += -I$(srctree)/tools/include/uapi
+INC_FLAGS += -I$(srctree)/tools/include/
+INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/uapi
+INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/
+INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/
 
 # $(obj-perf)      for generated common-cmds.h
 # $(obj-perf)/util for generated bison/flex headers
 ifneq ($(OUTPUT),)
-CFLAGS += -I$(obj-perf)/util
-CFLAGS += -I$(obj-perf)
+INC_FLAGS += -I$(obj-perf)/util
+INC_FLAGS += -I$(obj-perf)
 endif
 
-CFLAGS += -I$(src-perf)/util
-CFLAGS += -I$(src-perf)
-CFLAGS += -I$(srctree)/tools/lib/
+INC_FLAGS += -I$(src-perf)/util
+INC_FLAGS += -I$(src-perf)
+INC_FLAGS += -I$(srctree)/tools/lib/
+
+CFLAGS   += $(INC_FLAGS)
+CXXFLAGS += $(INC_FLAGS)
 
 CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
 
diff --git a/tools/perf/tests/llvm-cxx.h b/tools/perf/tests/llvm-cxx.h
new file mode 100644
index 0000000..d9c98fa
--- /dev/null
+++ b/tools/perf/tests/llvm-cxx.h
@@ -0,0 +1,13 @@
+#ifndef PERF_TEST_LLVM_CXX_H
+#define PERF_TEST_LLVM_CXX_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "tests/llvm.h"
+
+#ifdef __cplusplus
+}
+#endif
+#endif
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
index 0f484fb..c5546fd 100644
--- a/tools/perf/util/c++/clang-test.cpp
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -3,6 +3,10 @@
 #include "llvm/IR/Function.h"
 #include "llvm/IR/LLVMContext.h"
 
+#include <util-cxx.h>
+#include <tests/llvm-cxx.h>
+#include <string>
+
 class perf_clang_scope {
 public:
 	explicit perf_clang_scope() {perf_clang__init();}
@@ -14,17 +18,24 @@ extern "C" {
 int test__clang_to_IR(void)
 {
 	perf_clang_scope _scope;
+	unsigned int kernel_version;
+
+	if (fetch_kernel_version(&kernel_version, NULL, 0))
+		return -1;
+
+	std::string cflag_kver("-DLINUX_VERSION_CODE=" +
+				std::to_string(kernel_version));
 
 	std::unique_ptr<llvm::Module> M =
-		perf::getModuleFromSource({"-DRESULT=1"},
+		perf::getModuleFromSource({cflag_kver.c_str()},
 					  "perf-test.c",
-					  "int myfunc(void) {return RESULT;}");
+					  test_llvm__bpf_base_prog);
 
 	if (!M)
 		return -1;
 
 	for (llvm::Function& F : *M)
-		if (F.getName() == "myfunc")
+		if (F.getName() == "bpf_func__SyS_epoll_wait")
 			return 0;
 	return -1;
 }
diff --git a/tools/perf/util/util-cxx.h b/tools/perf/util/util-cxx.h
new file mode 100644
index 0000000..0e0e019
--- /dev/null
+++ b/tools/perf/util/util-cxx.h
@@ -0,0 +1,26 @@
+/*
+ * Support C++ source use utilities defined in util.h
+ */
+
+#ifndef PERF_UTIL_UTIL_CXX_H
+#define PERF_UTIL_UTIL_CXX_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+ * Now 'new' is the only C++ keyword found in util.h:
+ * in tools/include/linux/rbtree.h
+ *
+ * Other keywords, like class and delete, should be
+ * redefined if necessary.
+ */
+#define new _new
+#include "util.h"
+#undef new
+
+#ifdef __cplusplus
+}
+#endif
+#endif
-- 
1.8.3.4

[PATCH v2 10/18] perf clang: Support compile IR to BPF object and add testcase

[Wang Nan]

getBPFObjectFromModule() is introduced to use LLVM to convert Module
(LLVM IR) to BPF object code. Add new testcase for it.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/tests/clang.c           |  6 ++++-
 tools/perf/util/c++/clang-c.h      |  1 +
 tools/perf/util/c++/clang-test.cpp | 31 ++++++++++++++++++++-----
 tools/perf/util/c++/clang.cpp      | 46 ++++++++++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h        |  3 +++
 5 files changed, 80 insertions(+), 7 deletions(-)

diff --git a/tools/perf/tests/clang.c b/tools/perf/tests/clang.c
index 57ee160..2964c06 100644
--- a/tools/perf/tests/clang.c
+++ b/tools/perf/tests/clang.c
@@ -12,6 +12,10 @@ static struct {
 		.func = test__clang_to_IR,
 		.desc = "Test builtin clang compile C source to IR",
 	},
+	{
+		.func = test__clang_to_obj,
+		.desc = "Test builtin clang compile C source to ELF object",
+	},
 #endif
 };
 
@@ -33,7 +37,7 @@ int test__clang(int i __maybe_unused)
 	return TEST_SKIP;
 }
 #else
-int test__clang(int i __maybe_unused)
+int test__clang(int i)
 {
 	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
 		return TEST_FAIL;
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
index dcde4b5..22b3936 100644
--- a/tools/perf/util/c++/clang-c.h
+++ b/tools/perf/util/c++/clang-c.h
@@ -9,6 +9,7 @@ extern void perf_clang__init(void);
 extern void perf_clang__cleanup(void);
 
 extern int test__clang_to_IR(void);
+extern int test__clang_to_obj(void);
 
 #ifdef __cplusplus
 }
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
index c5546fd..473fe63 100644
--- a/tools/perf/util/c++/clang-test.cpp
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -13,15 +13,13 @@ public:
 	~perf_clang_scope() {perf_clang__cleanup();}
 };
 
-extern "C" {
-
-int test__clang_to_IR(void)
+static std::unique_ptr<llvm::Module>
+__test__clang_to_IR(void)
 {
-	perf_clang_scope _scope;
 	unsigned int kernel_version;
 
 	if (fetch_kernel_version(&kernel_version, NULL, 0))
-		return -1;
+		return std::unique_ptr<llvm::Module>(nullptr);
 
 	std::string cflag_kver("-DLINUX_VERSION_CODE=" +
 				std::to_string(kernel_version));
@@ -30,14 +28,35 @@ int test__clang_to_IR(void)
 		perf::getModuleFromSource({cflag_kver.c_str()},
 					  "perf-test.c",
 					  test_llvm__bpf_base_prog);
+	return M;
+}
+
+extern "C" {
+int test__clang_to_IR(void)
+{
+	perf_clang_scope _scope;
 
+	auto M = __test__clang_to_IR();
 	if (!M)
 		return -1;
-
 	for (llvm::Function& F : *M)
 		if (F.getName() == "bpf_func__SyS_epoll_wait")
 			return 0;
 	return -1;
 }
 
+int test__clang_to_obj(void)
+{
+	perf_clang_scope _scope;
+
+	auto M = __test__clang_to_IR();
+	if (!M)
+		return -1;
+
+	auto Buffer = perf::getBPFObjectFromModule(&*M);
+	if (!Buffer)
+		return -1;
+	return 0;
+}
+
 }
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 715ca0a..ab3a47c 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -13,10 +13,15 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Tooling/Tooling.h"
+#include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Option/Option.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/TargetRegistry.h"
+#include "llvm/Support/TargetSelect.h"
+#include "llvm/Target/TargetMachine.h"
+#include "llvm/Target/TargetOptions.h"
 #include <memory>
 
 #include "clang.h"
@@ -105,6 +110,47 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path)
 	return getModuleFromSource(std::move(CFlags), Path, VFS);
 }
 
+std::unique_ptr<llvm::SmallVectorImpl<char>>
+getBPFObjectFromModule(llvm::Module *Module)
+{
+	using namespace llvm;
+
+	LLVMInitializeBPFTargetInfo();
+	LLVMInitializeBPFTarget();
+	LLVMInitializeBPFTargetMC();
+	LLVMInitializeBPFAsmPrinter();
+
+	std::string TargetTriple("bpf-pc-linux");
+	std::string Error;
+	const Target* Target = TargetRegistry::lookupTarget(TargetTriple, Error);
+	if (!Target) {
+		llvm::errs() << Error;
+		return std::unique_ptr<llvm::SmallVectorImpl<char>>(nullptr);
+	}
+
+	llvm::TargetOptions Opt;
+	TargetMachine *TargetMachine =
+		Target->createTargetMachine(TargetTriple,
+					    "generic", "",
+					    Opt, Reloc::Static);
+
+	Module->setDataLayout(TargetMachine->createDataLayout());
+	Module->setTargetTriple(TargetTriple);
+
+	std::unique_ptr<SmallVectorImpl<char>> Buffer(new SmallVector<char, 0>());
+	raw_svector_ostream ostream(*Buffer);
+
+	legacy::PassManager PM;
+	if (TargetMachine->addPassesToEmitFile(PM, ostream,
+					       TargetMachine::CGFT_ObjectFile)) {
+		llvm::errs() << "TargetMachine can't emit a file of this type\n";
+		return std::unique_ptr<llvm::SmallVectorImpl<char>>(nullptr);;
+	}
+	PM.run(*Module);
+
+	return std::move(Buffer);
+}
+
 }
 
 extern "C" {
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
index b4fc2a9..dd8b042 100644
--- a/tools/perf/util/c++/clang.h
+++ b/tools/perf/util/c++/clang.h
@@ -19,5 +19,8 @@ std::unique_ptr<Module>
 getModuleFromSource(opt::ArgStringList CFlags,
 		    StringRef Path);
 
+std::unique_ptr<llvm::SmallVectorImpl<char>>
+getBPFObjectFromModule(llvm::Module *Module);
+
 }
 #endif
-- 
1.8.3.4

[PATCH v2 11/18] perf tools: Extract helpers in llvm-utils.c

[Wang Nan]

Following commits will use builtin clang to compile BPF script.
llvm__get_kbuild_opts() and llvm__get_nr_cpus() are extracted to help
building '-DKERNEL_VERSION_CODE' and '-D__NR_CPUS__' macros.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/llvm-utils.c | 70 ++++++++++++++++++++++++++++++++++++--------
 tools/perf/util/llvm-utils.h |  5 ++++
 2 files changed, 62 insertions(+), 13 deletions(-)

diff --git a/tools/perf/util/llvm-utils.c b/tools/perf/util/llvm-utils.c
index bf7216b..17dffb7 100644
--- a/tools/perf/util/llvm-utils.c
+++ b/tools/perf/util/llvm-utils.c
@@ -7,6 +7,7 @@
 #include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <linux/err.h>
 #include "debug.h"
 #include "llvm-utils.h"
 #include "config.h"
@@ -282,9 +283,10 @@ static const char *kinc_fetch_script =
 "rm -rf $TMPDIR\n"
 "exit $RET\n";
 
-static inline void
-get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
+void llvm__get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 {
+	static char *saved_kbuild_dir;
+	static char *saved_kbuild_include_opts;
 	int err;
 
 	if (!kbuild_dir || !kbuild_include_opts)
@@ -293,10 +295,28 @@ get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 	*kbuild_dir = NULL;
 	*kbuild_include_opts = NULL;
 
+	if (saved_kbuild_dir && saved_kbuild_include_opts &&
+	    !IS_ERR(saved_kbuild_dir) && !IS_ERR(saved_kbuild_include_opts)) {
+		*kbuild_dir = strdup(saved_kbuild_dir);
+		*kbuild_include_opts = strdup(saved_kbuild_include_opts);
+
+		if (*kbuild_dir && *kbuild_include_opts)
+			return;
+
+		zfree(kbuild_dir);
+		zfree(kbuild_include_opts);
+		/*
+		 * Don't fall through: it may breaks saved_kbuild_dir and
+		 * saved_kbuild_include_opts if detect them again when
+		 * memory is low.
+		 */
+		return;
+	}
+
 	if (llvm_param.kbuild_dir && !llvm_param.kbuild_dir[0]) {
 		pr_debug("[llvm.kbuild-dir] is set to \"\" deliberately.\n");
 		pr_debug("Skip kbuild options detection.\n");
-		return;
+		goto errout;
 	}
 
 	err = detect_kbuild_dir(kbuild_dir);
@@ -306,7 +326,7 @@ get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 "Hint:\tSet correct kbuild directory using 'kbuild-dir' option in [llvm]\n"
 "     \tsection of ~/.perfconfig or set it to \"\" to suppress kbuild\n"
 "     \tdetection.\n\n");
-		return;
+		goto errout;
 	}
 
 	pr_debug("Kernel build dir is set to %s\n", *kbuild_dir);
@@ -325,10 +345,40 @@ get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 
 		free(*kbuild_dir);
 		*kbuild_dir = NULL;
-		return;
+		goto errout;
 	}
 
 	pr_debug("include option is set to %s\n", *kbuild_include_opts);
+
+	saved_kbuild_dir = strdup(*kbuild_dir);
+	saved_kbuild_include_opts = strdup(*kbuild_include_opts);
+
+	if (!saved_kbuild_dir || !saved_kbuild_include_opts) {
+		zfree(&saved_kbuild_dir);
+		zfree(&saved_kbuild_include_opts);
+	}
+	return;
+errout:
+	saved_kbuild_dir = ERR_PTR(-EINVAL);
+	saved_kbuild_include_opts = ERR_PTR(-EINVAL);
+}
+
+int llvm__get_nr_cpus(void)
+{
+	static int nr_cpus_avail = 0;
+	char serr[STRERR_BUFSIZE];
+
+	if (nr_cpus_avail > 0)
+		return nr_cpus_avail;
+
+	nr_cpus_avail = sysconf(_SC_NPROCESSORS_CONF);
+	if (nr_cpus_avail <= 0) {
+		pr_err(
+"WARNING:\tunable to get available CPUs in this system: %s\n"
+"        \tUse 128 instead.\n", str_error_r(errno, serr, sizeof(serr)));
+		nr_cpus_avail = 128;
+	}
+	return nr_cpus_avail;
 }
 
 static void
@@ -406,15 +456,9 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf,
 	 * This is an optional work. Even it fail we can continue our
 	 * work. Needn't to check error return.
 	 */
-	get_kbuild_opts(&kbuild_dir, &kbuild_include_opts);
+	llvm__get_kbuild_opts(&kbuild_dir, &kbuild_include_opts);
 
-	nr_cpus_avail = sysconf(_SC_NPROCESSORS_CONF);
-	if (nr_cpus_avail <= 0) {
-		pr_err(
-"WARNING:\tunable to get available CPUs in this system: %s\n"
-"        \tUse 128 instead.\n", str_error_r(errno, serr, sizeof(serr)));
-		nr_cpus_avail = 128;
-	}
+	nr_cpus_avail = llvm__get_nr_cpus();
 	snprintf(nr_cpus_avail_str, sizeof(nr_cpus_avail_str), "%d",
 		 nr_cpus_avail);
 
diff --git a/tools/perf/util/llvm-utils.h b/tools/perf/util/llvm-utils.h
index 9f501ce..8a7d588 100644
--- a/tools/perf/util/llvm-utils.h
+++ b/tools/perf/util/llvm-utils.h
@@ -50,4 +50,9 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf, size_t *p_obj_buf_sz);
 
 /* This function is for test__llvm() use only */
 int llvm__search_clang(void);
+
+/* Following functions are reused by builtin clang support */
+void llvm__get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts);
+int llvm__get_nr_cpus(void);
+
 #endif
-- 
1.8.3.4

[PATCH v2 12/18] perf bpf: Compile BPF script use builtin clang support

[Wang Nan]

After this patch, perf utilizes builtin clang support to build BPF
script, no longer depend on external clang.

Test:

 $ type clang
 -bash: type: clang: not found
 $ cat ~/.perfconfig
 $ echo '#define LINUX_VERSION_CODE 0x040700' > ./test.c
 $ cat ./tools/perf/tests/bpf-script-example.c >> ./test.c
 $ ./perf record -v --dry-run -e ./test.c 2>&1 | grep builtin
 bpf: builtin compiling successful

Can't pass cflags so unable to include kernel headers now. Will be
fixed by following commits.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/bpf-loader.c  | 15 +++++++++++----
 tools/perf/util/c++/clang-c.h | 26 ++++++++++++++++++++++++++
 tools/perf/util/c++/clang.cpp | 29 +++++++++++++++++++++++++++++
 3 files changed, 66 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index 2b2c9b8..a5ddb8e 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -14,11 +14,11 @@
 #include "debug.h"
 #include "bpf-loader.h"
 #include "bpf-prologue.h"
-#include "llvm-utils.h"
 #include "probe-event.h"
 #include "probe-finder.h" // for MAX_PROBES
 #include "parse-events.h"
 #include "llvm-utils.h"
+#include "c++/clang-c.h"
 
 #define DEFINE_PRINT_FN(name, level) \
 static int libbpf_##name(const char *fmt, ...)	\
@@ -86,9 +86,16 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source)
 		void *obj_buf;
 		size_t obj_buf_sz;
 
-		err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
-		if (err)
-			return ERR_PTR(-BPF_LOADER_ERRNO__COMPILE);
+		perf_clang__init();
+		err = perf_clang__compile_bpf(filename, &obj_buf, &obj_buf_sz);
+		perf_clang__cleanup();
+		if (err) {
+			pr_debug("bpf: builtin compiling failed: %d, try external compiler\n", err);
+			err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
+			if (err)
+				return ERR_PTR(-BPF_LOADER_ERRNO__COMPILE);
+		} else
+			pr_debug("bpf: builtin compiling successful\n");
 		obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, filename);
 		free(obj_buf);
 	} else
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
index 22b3936..0eadd79 100644
--- a/tools/perf/util/c++/clang-c.h
+++ b/tools/perf/util/c++/clang-c.h
@@ -1,16 +1,42 @@
 #ifndef PERF_UTIL_CLANG_C_H
 #define PERF_UTIL_CLANG_C_H
 
+#include <stddef.h>	/* for size_t */
+#include <util-cxx.h>	/* for __maybe_unused */
+
 #ifdef __cplusplus
 extern "C" {
 #endif
 
+#ifdef HAVE_LIBCLANGLLVM_SUPPORT
 extern void perf_clang__init(void);
 extern void perf_clang__cleanup(void);
 
 extern int test__clang_to_IR(void);
 extern int test__clang_to_obj(void);
 
+extern int perf_clang__compile_bpf(const char *filename,
+				   void **p_obj_buf,
+				   size_t *p_obj_buf_sz);
+#else
+
+
+static inline void perf_clang__init(void) { }
+static inline void perf_clang__cleanup(void) { }
+
+static inline int test__clang_to_IR(void) { return -1; }
+static inline int test__clang_to_obj(void) { return -1;}
+
+static inline int
+perf_clang__compile_bpf(const char *filename __maybe_unused,
+			void **p_obj_buf __maybe_unused,
+			size_t *p_obj_buf_sz __maybe_unused)
+{
+	return -ENOTSUP;
+}
+
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index ab3a47c..3d24e50 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -164,4 +164,33 @@ void perf_clang__cleanup(void)
 	perf::LLVMCtx.reset(nullptr);
 	llvm::llvm_shutdown();
 }
+
+int perf_clang__compile_bpf(const char *filename,
+			    void **p_obj_buf,
+			    size_t *p_obj_buf_sz)
+{
+	using namespace perf;
+
+	if (!p_obj_buf || !p_obj_buf_sz)
+		return -EINVAL;
+
+	llvm::opt::ArgStringList CFlags;
+	auto M = getModuleFromSource(std::move(CFlags), filename);
+	if (!M)
+		return  -EINVAL;
+	auto O = getBPFObjectFromModule(&*M);
+	if (!O)
+		return -EINVAL;
+
+	size_t size = O->size_in_bytes();
+	void *buffer;
+
+	buffer = malloc(size);
+	if (!buffer)
+		return -ENOMEM;
+	memcpy(buffer, O->data(), size);
+	*p_obj_buf = buffer;
+	*p_obj_buf_sz = size;
+	return 0;
+}
 }
-- 
1.8.3.4

[PATCH v2 13/18] perf clang: Pass full path to builtin clang

[Wang Nan]

If clang changes its working directory, filename passed to
perf_clang__compile_bpf() becomes invalid. Before running clang,
convert it to absolute path so file can be found even working directory
is changed.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang.cpp | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 3d24e50..9597ff7 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -13,6 +13,7 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Option/Option.h"
@@ -165,7 +166,7 @@ void perf_clang__cleanup(void)
 	llvm::llvm_shutdown();
 }
 
-int perf_clang__compile_bpf(const char *filename,
+int perf_clang__compile_bpf(const char *_filename,
 			    void **p_obj_buf,
 			    size_t *p_obj_buf_sz)
 {
@@ -174,8 +175,11 @@ int perf_clang__compile_bpf(const char *filename,
 	if (!p_obj_buf || !p_obj_buf_sz)
 		return -EINVAL;
 
+	llvm::SmallString<PATH_MAX> FileName(_filename);
+	llvm::sys::fs::make_absolute(FileName);
+
 	llvm::opt::ArgStringList CFlags;
-	auto M = getModuleFromSource(std::move(CFlags), filename);
+	auto M = getModuleFromSource(std::move(CFlags), FileName.data());
 	if (!M)
 		return  -EINVAL;
 	auto O = getBPFObjectFromModule(&*M);
-- 
1.8.3.4

[PATCH v2 14/18] perf clang: Pass CFLAGS to builtin clang

[Wang Nan]

Pass -DKERNEL_VERSION_CODE, -D__NR_CPUS__, llvm.clang-opt config options
and CFLAGS detected by kbuild detector to builtin clang so BPF scripts
can use kernel headers and user defined options like external clang
compiler.

Test:
 # perf record --dry-run -e tools/perf/tests/bpf-script-test-kbuild.c ls 2>&1 | grep built
 bpf: builtin compiling successful

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang.cpp    | 105 +++++++++++++++++++++++++++++++++++++--
 tools/perf/util/llvm-utils-cxx.h |  14 ++++++
 tools/perf/util/llvm-utils.h     |   2 +-
 3 files changed, 116 insertions(+), 5 deletions(-)
 create mode 100644 tools/perf/util/llvm-utils-cxx.h

diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 9597ff7..c1660ab 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -27,6 +27,8 @@
 
 #include "clang.h"
 #include "clang-c.h"
+#include "llvm-utils-cxx.h"
+#include "util-cxx.h"
 
 namespace perf {
 
@@ -152,6 +154,101 @@ getBPFObjectFromModule(llvm::Module *Module)
 	return std::move(Buffer);
 }
 
+class ClangOptions {
+	llvm::SmallString<PATH_MAX> FileName;
+	llvm::SmallString<64> KVerDef;
+	llvm::SmallString<64> NRCpusDef;
+	char *kbuild_dir;
+	char *kbuild_include_opts;
+	char *clang_opt;
+public:
+	ClangOptions(const char *filename) : FileName(filename),
+					     KVerDef(""),
+					     NRCpusDef(""),
+					     kbuild_dir(NULL),
+					     kbuild_include_opts(NULL),
+					     clang_opt(NULL)
+	{
+		llvm::sys::fs::make_absolute(FileName);
+
+		unsigned int kver;
+		if (!fetch_kernel_version(&kver, NULL, 0))
+			KVerDef = "-DLINUX_VERSION_CODE=" + std::to_string(kver);
+
+		int nr_cpus = llvm__get_nr_cpus();
+		if (nr_cpus > 0)
+			NRCpusDef = "-D__NR_CPUS__=" + std::to_string(nr_cpus);
+
+		if (llvm_param.clang_opt)
+			clang_opt = strdup(llvm_param.clang_opt);
+
+		llvm__get_kbuild_opts(&kbuild_dir, &kbuild_include_opts);
+		if (!kbuild_dir || !kbuild_include_opts) {
+			free(kbuild_dir);
+			free(kbuild_include_opts);
+			kbuild_dir = kbuild_include_opts = NULL;
+		}
+	}
+
+	~ClangOptions()
+	{
+		free(kbuild_dir);
+		free(kbuild_include_opts);
+		free(clang_opt);
+	}
+
+	static void fillCFlagsFromString(opt::ArgStringList &CFlags, char *s, bool check = false)
+	{
+		if (!s)
+			return;
+
+		SmallVector<StringRef, 0> Terms;
+		StringRef Opts(s);
+		Opts.split(Terms, ' ');
+
+		for (auto i = Terms.begin(); i != Terms.end(); i++)
+			s[i->end() - Opts.begin()] = '\0';
+
+		for (auto i = Terms.begin(); i != Terms.end(); i++) {
+			if (!check) {
+				CFlags.push_back(i->begin());
+				continue;
+			}
+
+			if (i->startswith("-I"))
+				CFlags.push_back(i->begin());
+			else if (i->startswith("-D"))
+				CFlags.push_back(i->begin());
+			else if (*i == "-include") {
+				CFlags.push_back((i++)->begin());
+				/* Let clang report this error */
+				if (i == Terms.end())
+					break;
+				CFlags.push_back(i->begin());
+			}
+		}
+	}
+
+	void getCFlags(opt::ArgStringList &CFlags)
+	{
+		CFlags.push_back(KVerDef.c_str());
+		CFlags.push_back(NRCpusDef.c_str());
+
+		fillCFlagsFromString(CFlags, clang_opt);
+		fillCFlagsFromString(CFlags, kbuild_include_opts, true);
+
+		if (kbuild_dir) {
+			CFlags.push_back("-working-directory");
+			CFlags.push_back(kbuild_dir);
+		}
+	}
+
+	const char *getFileName(void)
+	{
+		return FileName.c_str();
+	}
+};
+
 }
 
 extern "C" {
@@ -175,11 +272,11 @@ int perf_clang__compile_bpf(const char *_filename,
 	if (!p_obj_buf || !p_obj_buf_sz)
 		return -EINVAL;
 
-	llvm::SmallString<PATH_MAX> FileName(_filename);
-	llvm::sys::fs::make_absolute(FileName);
-
+	ClangOptions Opts(_filename);
 	llvm::opt::ArgStringList CFlags;
-	auto M = getModuleFromSource(std::move(CFlags), FileName.data());
+
+	Opts.getCFlags(CFlags);
+	auto M = getModuleFromSource(std::move(CFlags), Opts.getFileName());
 	if (!M)
 		return  -EINVAL;
 	auto O = getBPFObjectFromModule(&*M);
diff --git a/tools/perf/util/llvm-utils-cxx.h b/tools/perf/util/llvm-utils-cxx.h
new file mode 100644
index 0000000..1202230
--- /dev/null
+++ b/tools/perf/util/llvm-utils-cxx.h
@@ -0,0 +1,14 @@
+#ifndef PERF_UTIL_LLVM_UTILS_CXX_H
+#define PERF_UTIL_LLVM_UTILS_CXX_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "llvm-utils.h"
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/perf/util/llvm-utils.h b/tools/perf/util/llvm-utils.h
index 8a7d588..fbc7a8b 100644
--- a/tools/perf/util/llvm-utils.h
+++ b/tools/perf/util/llvm-utils.h
@@ -5,7 +5,7 @@
 #ifndef __LLVM_UTILS_H
 #define __LLVM_UTILS_H
 
-#include "debug.h"
+#include <errno.h>
 
 struct llvm_param {
 	/* Path of clang executable */
-- 
1.8.3.4

[PATCH v2 15/18] perf clang: Link BPF functions declaration into perf

[Wang Nan]

Use a shell script to generate BPF functions declarations from kernel
source code, embedded the generated header into a C string. Following
commits will utilizes clang's virtual file system to automatically
include this header to all BPF scripts.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/Build                |   1 +
 tools/perf/util/c++/bpf-funcs-str.c      | 214 +++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang-bpf-includes.h |  12 ++
 3 files changed, 227 insertions(+)
 create mode 100644 tools/perf/util/c++/bpf-funcs-str.c
 create mode 100644 tools/perf/util/c++/clang-bpf-includes.h

diff --git a/tools/perf/util/c++/Build b/tools/perf/util/c++/Build
index 988fef1..bd71abf 100644
--- a/tools/perf/util/c++/Build
+++ b/tools/perf/util/c++/Build
@@ -1,2 +1,3 @@
 libperf-$(CONFIG_CLANGLLVM) += clang.o
 libperf-$(CONFIG_CLANGLLVM) += clang-test.o
+libperf-$(CONFIG_CLANGLLVM) += bpf-funcs-str.o
diff --git a/tools/perf/util/c++/bpf-funcs-str.c b/tools/perf/util/c++/bpf-funcs-str.c
new file mode 100644
index 0000000..5e47a11
--- /dev/null
+++ b/tools/perf/util/c++/bpf-funcs-str.c
@@ -0,0 +1,214 @@
+/*
+ * This file is generated by following script:
+ *
+ * #!/bin/bash
+ * TEMP_KBUILD=$(mktemp -d)
+ * KERNEL_DIR=$(pwd)
+ * OUTPUT=tools/perf/util/c++/bpf-funcs-str.c
+ * rm -rf $OUTPUT
+ * echo "Use temp dir: $TEMP_KBUILD"
+ * function finish()
+ * {
+ * 	rm -rf $TEMP_KBUILD
+ * }
+ * trap finish EXIT
+ * SRCLIST=$(find -name "*.c" | xargs grep bpf_func_proto -l)
+ * cd $TEMP_KBUILD
+ * yes '' | make -C $KERNEL_DIR O=`pwd` oldconfig
+ * cat << EOF >> ./.config
+ * CONFIG_BPF=y
+ * CONFIG_BPF_SYSCALL=y
+ * CONFIG_PERF_EVENTS=y
+ * CONFIG_SOCK_CGROUP_DATA=y
+ * EOF
+ * yes '' | make -C $KERNEL_DIR O=`pwd` oldconfig
+ * FIXOBJLIST=""
+ * for src in ${SRCLIST}
+ * do
+ * 	mkdir -p $(dirname $src)
+ * 	cat << EOF > "${src}-fix.c"
+ * #include <linux/init.h>
+ * #undef __init
+ * #define __init __attribute__((constructor))
+ * #include "`basename $src`"
+ * EOF
+ * 	if [ $(basename $src) == "syscall.c" ]
+ * 	then
+ * 		cat << EOF >> "${src}-fix.c"
+ * const struct bpf_verifier_ops *
+ * find_prog_type_export(enum bpf_prog_type type)
+ * {
+ * 	struct bpf_prog_aux aux;
+ * 	struct bpf_prog p = {.aux = &aux };
+ * 	if (find_prog_type(type, &p))
+ * 		return NULL;
+ * 	return p.aux->ops;
+ * }
+ * EOF
+ * 	fi
+ * 	FIXOBJLIST="$FIXOBJLIST ${src}-fix.o"
+ * done
+ * function dolink()
+ * {
+ * 	touch ./syms.c
+ * 	gcc kernel/bpf/main.o ./syms.c $FIXOBJLIST -o ./gen
+ * }
+ * MAIN=kernel/bpf/main.c
+ * cat << EOF > $MAIN
+ * #include <uapi/linux/bpf.h>
+ * #include <linux/bpf.h>
+ * struct bpf_func {
+ *   const char *name;
+ *   int id;
+ * } bpf_funcs[] = {
+ * EOF
+ * grep '^[[:space:]]BPF_FUNC_[^ ]*,' $KERNEL_DIR/include/uapi/linux/bpf.h | \
+ * 	sed -e 's/.*BPF_FUNC_\([^,]*\),.*$/\1/g' | \
+ * 	xargs -n 1 sh -c 'echo {.name = \"$1\", .id = BPF_FUNC_$1}, >> '"$MAIN" sh
+ * cat << EOF >> $MAIN
+ * {NULL, -1},
+ * };
+ * int capable(int x) {return 1;}
+ * int trace_printk_init_buffers(void) {return 0;}
+ * static int x;
+ * void *metadata_dst_alloc_percpu(int a, int b) {return &x;}
+ * int ___ratelimit(void *a, const void *func) {return 0;}
+ * extern const struct bpf_verifier_ops *
+ * find_prog_type_export(enum bpf_prog_type type);
+ * extern int printf(const char *fmt, ...);
+ * int main(int argc, char *argv[])
+ * {
+ * 	struct bpf_func *f = &bpf_funcs[0];
+ * 	printf("#ifndef BPF_FUNCS_DEFINED\n");
+ * 	printf("#define BPF_FUNCS_DEFINED\n");
+ * 	while (f->id != -1) {
+ * 		enum bpf_prog_type t;
+ * 		const enum bpf_arg_type *argt;
+ * 		const struct bpf_verifier_ops *ops = NULL;
+ * 		const struct bpf_func_proto *proto = NULL;
+ * 		if (f->id == 0)
+ * 			goto skip;
+ * 		for (t = BPF_PROG_TYPE_UNSPEC + 1; ; t++) {
+ * 			ops = find_prog_type_export(t);
+ * 			if (!ops)
+ * 				break;
+ * 			proto = ops->get_func_proto(f->id);
+ * 			if (proto)
+ * 				break;
+ * 		}
+ * 		if (!proto) {
+ * 			printf("static void (*%s)(void) = (void *)-1;\n", f->name);
+ * 			continue;
+ * 		}
+ * 		printf("static ");
+ * 		switch (proto->ret_type) {
+ * 		case RET_INTEGER:
+ * 			printf("int ");
+ * 			break;
+ * 		case RET_PTR_TO_MAP_VALUE_OR_NULL:
+ * 			printf("void *");
+ * 			break;
+ * 		default:
+ * 		case RET_VOID:
+ * 			printf("void ");
+ * 			break;
+ * 		}
+ * 		printf("(*bpf_%s)(", f->name);
+ * 		for (argt = &proto->arg1_type; argt <= &proto->arg5_type; argt++) {
+ * 			if (*argt == ARG_DONTCARE) {
+ * 				if (argt == &proto->arg1_type)
+ * 					printf("void");
+ * 				goto finish;
+ * 			}
+ * 			if (argt != &proto->arg1_type)
+ * 				printf(", ");
+ * 			switch (*argt) {
+ * 			case ARG_CONST_MAP_PTR:
+ * 			case ARG_PTR_TO_MAP_KEY:
+ * 			case ARG_PTR_TO_MAP_VALUE:
+ * 			case ARG_PTR_TO_STACK:
+ * 			case ARG_PTR_TO_RAW_STACK:
+ * 			case ARG_PTR_TO_CTX:
+ * 				printf("void *");
+ * 				break;
+ * 			default:
+ * 				printf("unsigned long");
+ * 				break;
+ * 			}
+ * 		}
+ * finish:
+ * 		printf(") = (void *)%d;\n", f->id);
+ * skip:
+ * 		f++;
+ * 	}
+ * 	printf("#endif\n");
+ * 	return 0;
+ * }
+ * EOF
+ * make -j8 kernel/bpf/main.o $FIXOBJLIST
+ * rm -f ./syms.c
+ * export LANG=POSIX
+ * export LC_ALL=POSIX
+ * dolink 2>&1 | \
+ *        grep 'undefined reference' | \
+ *        awk -F "\`" '{print $2}' | \
+ *        sed "s/'$//g" | sort | uniq | \
+ *        xargs -n 1 sh -c 'echo "int $1 __attribute__((weak));" >> ./syms.c' sh
+ * dolink
+ * cd $KERNEL_DIR
+ * unset X
+ * cat << EOF > $OUTPUT
+ * /$X*
+ *  * This file is generated by following script:
+ *  *
+ * EOF
+ * cat $0 | awk '$0 != "" {print " * " $0}' >> $OUTPUT
+ * echo ' *''/' >> $OUTPUT
+ * echo '#include "clang-bpf-includes.h"' >> $OUTPUT
+ * echo 'const char clang_builtin_bpf_funcs_str[] =' >> $OUTPUT
+ * $TEMP_KBUILD/gen | awk '{print "\""$0"\\n\""}' >> $OUTPUT
+ * echo ';' >> $OUTPUT
+ * echo "Finish generating " $OUTPUT
+ */
+#include "clang-bpf-includes.h"
+const char clang_builtin_bpf_funcs_str[] =
+"#ifndef BPF_FUNCS_DEFINED\n"
+"#define BPF_FUNCS_DEFINED\n"
+"static void *(*bpf_map_lookup_elem)(void *, void *) = (void *)1;\n"
+"static int (*bpf_map_update_elem)(void *, void *, void *, unsigned long) = (void *)2;\n"
+"static int (*bpf_map_delete_elem)(void *, void *) = (void *)3;\n"
+"static int (*bpf_probe_read)(void *, unsigned long, unsigned long) = (void *)4;\n"
+"static int (*bpf_ktime_get_ns)(void) = (void *)5;\n"
+"static int (*bpf_trace_printk)(void *, unsigned long) = (void *)6;\n"
+"static int (*bpf_get_prandom_u32)(void) = (void *)7;\n"
+"static int (*bpf_get_smp_processor_id)(void) = (void *)8;\n"
+"static int (*bpf_skb_store_bytes)(void *, unsigned long, void *, unsigned long, unsigned long) = (void *)9;\n"
+"static int (*bpf_l3_csum_replace)(void *, unsigned long, unsigned long, unsigned long, unsigned long) = (void *)10;\n"
+"static int (*bpf_l4_csum_replace)(void *, unsigned long, unsigned long, unsigned long, unsigned long) = (void *)11;\n"
+"static void (*bpf_tail_call)(void *, void *, unsigned long) = (void *)12;\n"
+"static int (*bpf_clone_redirect)(void *, unsigned long, unsigned long) = (void *)13;\n"
+"static int (*bpf_get_current_pid_tgid)(void) = (void *)14;\n"
+"static int (*bpf_get_current_uid_gid)(void) = (void *)15;\n"
+"static int (*bpf_get_current_comm)(void *, unsigned long) = (void *)16;\n"
+"static int (*bpf_get_cgroup_classid)(void *) = (void *)17;\n"
+"static int (*bpf_skb_vlan_push)(void *, unsigned long, unsigned long) = (void *)18;\n"
+"static int (*bpf_skb_vlan_pop)(void *) = (void *)19;\n"
+"static int (*bpf_skb_get_tunnel_key)(void *, void *, unsigned long, unsigned long) = (void *)20;\n"
+"static int (*bpf_skb_set_tunnel_key)(void *, void *, unsigned long, unsigned long) = (void *)21;\n"
+"static int (*bpf_perf_event_read)(void *, unsigned long) = (void *)22;\n"
+"static int (*bpf_redirect)(unsigned long, unsigned long) = (void *)23;\n"
+"static int (*bpf_get_route_realm)(void *) = (void *)24;\n"
+"static int (*bpf_perf_event_output)(void *, void *, unsigned long, void *, unsigned long) = (void *)25;\n"
+"static int (*bpf_skb_load_bytes)(void *, unsigned long, void *, unsigned long) = (void *)26;\n"
+"static int (*bpf_get_stackid)(void *, void *, unsigned long) = (void *)27;\n"
+"static int (*bpf_csum_diff)(void *, unsigned long, void *, unsigned long, unsigned long) = (void *)28;\n"
+"static int (*bpf_skb_get_tunnel_opt)(void *, void *, unsigned long) = (void *)29;\n"
+"static int (*bpf_skb_set_tunnel_opt)(void *, void *, unsigned long) = (void *)30;\n"
+"static int (*bpf_skb_change_proto)(void *, unsigned long, unsigned long) = (void *)31;\n"
+"static int (*bpf_skb_change_type)(void *, unsigned long) = (void *)32;\n"
+"static int (*bpf_skb_under_cgroup)(void *, void *, unsigned long) = (void *)33;\n"
+"static int (*bpf_get_hash_recalc)(void *) = (void *)34;\n"
+"static int (*bpf_get_current_task)(void) = (void *)35;\n"
+"static int (*bpf_probe_write_user)(unsigned long, void *, unsigned long) = (void *)36;\n"
+"#endif\n"
+;
diff --git a/tools/perf/util/c++/clang-bpf-includes.h b/tools/perf/util/c++/clang-bpf-includes.h
new file mode 100644
index 0000000..385a5bb
--- /dev/null
+++ b/tools/perf/util/c++/clang-bpf-includes.h
@@ -0,0 +1,12 @@
+#ifndef CLANG_BPF_INCLUDS_H
+#define CLANG_BPF_INCLUDS_H
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+extern const char clang_builtin_bpf_funcs_str[];
+
+#ifdef __cplusplus
+}
+#endif
+#endif
-- 
1.8.3.4

[PATCH v2 16/18] perf clang: Declare BPF functions for BPF scripts automatically

[Wang Nan]

Use Clang's OverlayFileSystem, add '-include' options to make builtin
clang define BPF functions. After this patch BPF script writer needn't
define BPF functions by their own.

Test cases are updated to avoid redefine these functions.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/tests/bpf-script-example.c         | 18 +++++++++------
 tools/perf/tests/bpf-script-test-prologue.c   |  2 ++
 tools/perf/tests/bpf-script-test-relocation.c | 18 +++++++++------
 tools/perf/util/c++/clang.cpp                 | 32 ++++++++++++++++++++++++++-
 4 files changed, 55 insertions(+), 15 deletions(-)

diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
index 268e5f8..5fb3e66 100644
--- a/tools/perf/tests/bpf-script-example.c
+++ b/tools/perf/tests/bpf-script-example.c
@@ -8,13 +8,6 @@
 #endif
 #define BPF_ANY 0
 #define BPF_MAP_TYPE_ARRAY 2
-#define BPF_FUNC_map_lookup_elem 1
-#define BPF_FUNC_map_update_elem 2
-
-static void *(*bpf_map_lookup_elem)(void *map, void *key) =
-	(void *) BPF_FUNC_map_lookup_elem;
-static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
-	(void *) BPF_FUNC_map_update_elem;
 
 struct bpf_map_def {
 	unsigned int type;
@@ -24,6 +17,17 @@ struct bpf_map_def {
 };
 
 #define SEC(NAME) __attribute__((section(NAME), used))
+
+#ifndef BPF_FUNCS_DEFINED
+#define BPF_FUNC_map_lookup_elem 1
+#define BPF_FUNC_map_update_elem 2
+
+static void *(*bpf_map_lookup_elem)(void *map, void *key) =
+	(void *) BPF_FUNC_map_lookup_elem;
+static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
+	(void *) BPF_FUNC_map_update_elem;
+#endif
+
 struct bpf_map_def SEC("maps") flip_table = {
 	.type = BPF_MAP_TYPE_ARRAY,
 	.key_size = sizeof(int),
diff --git a/tools/perf/tests/bpf-script-test-prologue.c b/tools/perf/tests/bpf-script-test-prologue.c
index 7230e62..e8dba36 100644
--- a/tools/perf/tests/bpf-script-test-prologue.c
+++ b/tools/perf/tests/bpf-script-test-prologue.c
@@ -13,8 +13,10 @@
 #define FMODE_READ		0x1
 #define FMODE_WRITE		0x2
 
+#ifndef BPF_FUNCS_DEFINED
 static void (*bpf_trace_printk)(const char *fmt, int fmt_size, ...) =
 	(void *) 6;
+#endif
 
 SEC("func=null_lseek file->f_mode offset orig")
 int bpf_func__null_lseek(void *ctx, int err, unsigned long f_mode,
diff --git a/tools/perf/tests/bpf-script-test-relocation.c b/tools/perf/tests/bpf-script-test-relocation.c
index 93af774..a9638eb7 100644
--- a/tools/perf/tests/bpf-script-test-relocation.c
+++ b/tools/perf/tests/bpf-script-test-relocation.c
@@ -8,13 +8,6 @@
 #endif
 #define BPF_ANY 0
 #define BPF_MAP_TYPE_ARRAY 2
-#define BPF_FUNC_map_lookup_elem 1
-#define BPF_FUNC_map_update_elem 2
-
-static void *(*bpf_map_lookup_elem)(void *map, void *key) =
-	(void *) BPF_FUNC_map_lookup_elem;
-static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
-	(void *) BPF_FUNC_map_update_elem;
 
 struct bpf_map_def {
 	unsigned int type;
@@ -24,6 +17,17 @@ struct bpf_map_def {
 };
 
 #define SEC(NAME) __attribute__((section(NAME), used))
+
+#ifndef BPF_FUNCS_DEFINED
+#define BPF_FUNC_map_lookup_elem 1
+#define BPF_FUNC_map_update_elem 2
+
+static void *(*bpf_map_lookup_elem)(void *map, void *key) =
+	(void *) BPF_FUNC_map_lookup_elem;
+static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
+	(void *) BPF_FUNC_map_update_elem;
+#endif
+
 struct bpf_map_def SEC("maps") my_table = {
 	.type = BPF_MAP_TYPE_ARRAY,
 	.key_size = sizeof(int),
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index c1660ab..1016d5d 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -27,11 +27,19 @@
 
 #include "clang.h"
 #include "clang-c.h"
+#include "clang-bpf-includes.h"
 #include "llvm-utils-cxx.h"
 #include "util-cxx.h"
 
 namespace perf {
 
+static struct BPFHeader {
+	llvm::StringRef Path;
+	llvm::StringRef Content;
+} BPFHeaders[] = {
+	{"/virtual/bpf-funcs.h", clang_builtin_bpf_funcs_str},
+};
+
 static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
 
 using namespace clang;
@@ -56,6 +64,11 @@ createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path,
 		"-x", "c"};
 
 	CCArgs.append(CFlags.begin(), CFlags.end());
+	for (BPFHeader &h : BPFHeaders) {
+		CCArgs.append(1, "-include");
+		CCArgs.append(1, h.Path.begin());
+	}
+
 	CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs);
 
 	FrontendOptions& Opts = CI->getFrontendOpts();
@@ -64,6 +77,22 @@ createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path,
 	return CI;
 }
 
+static IntrusiveRefCntPtr<vfs::FileSystem>
+addBPFHeaders(IntrusiveRefCntPtr<vfs::FileSystem> VFS)
+{
+	using namespace vfs;
+
+	llvm::IntrusiveRefCntPtr<OverlayFileSystem> OverlayFS(
+			new OverlayFileSystem(VFS));
+	llvm::IntrusiveRefCntPtr<InMemoryFileSystem> MemFS(
+			new InMemoryFileSystem(true));
+	OverlayFS->pushOverlay(MemFS);
+
+	for (BPFHeader &h : BPFHeaders)
+		MemFS->addFile(h.Path, 0, llvm::MemoryBuffer::getMemBuffer(h.Content));
+	return OverlayFS;
+}
+
 static std::unique_ptr<llvm::Module>
 getModuleFromSource(llvm::opt::ArgStringList CFlags,
 		    StringRef Path, IntrusiveRefCntPtr<vfs::FileSystem> VFS)
@@ -71,7 +100,8 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags,
 	CompilerInstance Clang;
 	Clang.createDiagnostics();
 
-	Clang.setVirtualFileSystem(&*VFS);
+	IntrusiveRefCntPtr<vfs::FileSystem> OverlayVFS = addBPFHeaders(VFS);
+	Clang.setVirtualFileSystem(&*OverlayVFS);
 
 	IntrusiveRefCntPtr<CompilerInvocation> CI =
 		createCompilerInvocation(std::move(CFlags), Path,
-- 
1.8.3.4

[PATCH v2 17/18] perf clang: Include helpers to BPF scripts

[Wang Nan]

Automatically include common used macros and struct definitions
into BPF scripts. Script writers are no longer required to define
'SEC' and 'struct bpf_map_def' in each of their scripts.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/tests/bpf-script-example.c         |  2 ++
 tools/perf/tests/bpf-script-test-kbuild.c     |  2 ++
 tools/perf/tests/bpf-script-test-prologue.c   |  2 ++
 tools/perf/tests/bpf-script-test-relocation.c |  2 ++
 tools/perf/util/c++/Build                     |  1 +
 tools/perf/util/c++/bpf-helper-str.c          | 13 +++++++++++++
 tools/perf/util/c++/clang-bpf-includes.h      |  1 +
 tools/perf/util/c++/clang.cpp                 |  1 +
 8 files changed, 24 insertions(+)
 create mode 100644 tools/perf/util/c++/bpf-helper-str.c

diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
index 5fb3e66..c82d67b 100644
--- a/tools/perf/tests/bpf-script-example.c
+++ b/tools/perf/tests/bpf-script-example.c
@@ -9,6 +9,7 @@
 #define BPF_ANY 0
 #define BPF_MAP_TYPE_ARRAY 2
 
+#ifndef BPF_HELPER_DEFINED
 struct bpf_map_def {
 	unsigned int type;
 	unsigned int key_size;
@@ -17,6 +18,7 @@ struct bpf_map_def {
 };
 
 #define SEC(NAME) __attribute__((section(NAME), used))
+#endif
 
 #ifndef BPF_FUNCS_DEFINED
 #define BPF_FUNC_map_lookup_elem 1
diff --git a/tools/perf/tests/bpf-script-test-kbuild.c b/tools/perf/tests/bpf-script-test-kbuild.c
index 3626924..d01126b 100644
--- a/tools/perf/tests/bpf-script-test-kbuild.c
+++ b/tools/perf/tests/bpf-script-test-kbuild.c
@@ -6,7 +6,9 @@
 # error Need LINUX_VERSION_CODE
 # error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
 #endif
+#ifndef BPF_HELPER_DEFINED
 #define SEC(NAME) __attribute__((section(NAME), used))
+#endif
 
 #include <uapi/linux/fs.h>
 #include <uapi/asm/ptrace.h>
diff --git a/tools/perf/tests/bpf-script-test-prologue.c b/tools/perf/tests/bpf-script-test-prologue.c
index e8dba36..e7ecbf1 100644
--- a/tools/perf/tests/bpf-script-test-prologue.c
+++ b/tools/perf/tests/bpf-script-test-prologue.c
@@ -6,7 +6,9 @@
 # error Need LINUX_VERSION_CODE
 # error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
 #endif
+#ifndef BPF_HELPER_DEFINED
 #define SEC(NAME) __attribute__((section(NAME), used))
+#endif
 
 #include <uapi/linux/fs.h>
 
diff --git a/tools/perf/tests/bpf-script-test-relocation.c b/tools/perf/tests/bpf-script-test-relocation.c
index a9638eb7..dfccdf6 100644
--- a/tools/perf/tests/bpf-script-test-relocation.c
+++ b/tools/perf/tests/bpf-script-test-relocation.c
@@ -9,6 +9,7 @@
 #define BPF_ANY 0
 #define BPF_MAP_TYPE_ARRAY 2
 
+#ifndef BPF_HELPER_DEFINED
 struct bpf_map_def {
 	unsigned int type;
 	unsigned int key_size;
@@ -17,6 +18,7 @@ struct bpf_map_def {
 };
 
 #define SEC(NAME) __attribute__((section(NAME), used))
+#endif
 
 #ifndef BPF_FUNCS_DEFINED
 #define BPF_FUNC_map_lookup_elem 1
diff --git a/tools/perf/util/c++/Build b/tools/perf/util/c++/Build
index bd71abf..faa0268 100644
--- a/tools/perf/util/c++/Build
+++ b/tools/perf/util/c++/Build
@@ -1,3 +1,4 @@
 libperf-$(CONFIG_CLANGLLVM) += clang.o
 libperf-$(CONFIG_CLANGLLVM) += clang-test.o
 libperf-$(CONFIG_CLANGLLVM) += bpf-funcs-str.o
+libperf-$(CONFIG_CLANGLLVM) += bpf-helper-str.o
diff --git a/tools/perf/util/c++/bpf-helper-str.c b/tools/perf/util/c++/bpf-helper-str.c
new file mode 100644
index 0000000..285ff01
--- /dev/null
+++ b/tools/perf/util/c++/bpf-helper-str.c
@@ -0,0 +1,13 @@
+#include "clang-bpf-includes.h"
+const char clang_builtin_bpf_helper_str[] =
+"#ifndef BPF_HELPER_DEFINED\n"
+"#define BPF_HELPER_DEFINED\n"
+"struct bpf_map_def {\n"
+"	unsigned int type;\n"
+"	unsigned int key_size;\n"
+"	unsigned int value_size;\n"
+"	unsigned int max_entries;\n"
+"};\n"
+"#define SEC(NAME) __attribute__((section(NAME), used))\n"
+"#endif"
+;
diff --git a/tools/perf/util/c++/clang-bpf-includes.h b/tools/perf/util/c++/clang-bpf-includes.h
index 385a5bb..577b40c 100644
--- a/tools/perf/util/c++/clang-bpf-includes.h
+++ b/tools/perf/util/c++/clang-bpf-includes.h
@@ -5,6 +5,7 @@ extern "C" {
 #endif
 
 extern const char clang_builtin_bpf_funcs_str[];
+extern const char clang_builtin_bpf_helper_str[];
 
 #ifdef __cplusplus
 }
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 1016d5d..a2de788 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -38,6 +38,7 @@ static struct BPFHeader {
 	llvm::StringRef Content;
 } BPFHeaders[] = {
 	{"/virtual/bpf-funcs.h", clang_builtin_bpf_funcs_str},
+	{"/virtual/bpf-helper.h", clang_builtin_bpf_helper_str},
 };
 
 static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
-- 
1.8.3.4

[PATCH v2 18/18] perf clang: Define PERF_BUILTIN_CLANG for builtin clang compiling

[Wang Nan]

By this macro BPF script knows what compiler it is being built with.
Scripts prefer external clang can force fall back to it by:

 #ifdef PERF_BUILTIN_CLANG
 # error I prefer external clang compiler
 #endif

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang.cpp | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index a2de788..dd32f5d 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -62,6 +62,7 @@ createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path,
 		"-vectorize-slp",
 		"-Wno-unused-value",
 		"-Wno-pointer-sign",
+		"-DPERF_BUILTIN_CLANG=1"
 		"-x", "c"};
 
 	CCArgs.append(CFlags.begin(), CFlags.end());
-- 
1.8.3.4

Re: [PATCH v2 00/18] perf clang: Support compiling BPF script on the fly

[Arnaldo Carvalho de Melo]

Em Mon, Sep 26, 2016 at 07:26:54AM +0000, Wang Nan escreveu:
> This patch add builtin clang, allow perf compile BPF scripts on the fly.
> This is the first step to implement what I announced at LinuxCon 2016 NA:

Ok, so I refreshed this series to apply against my latest perf/core and
put it in a tmp.perf/builtin-clang, will continue testing it tomorrow
after checking if fedora24 has those llvm-dev and libclang-dev that can
be used with this, do you know if it works on that distro?

Cool stuff! :-)

- Arnaldo
 
> http://events.linuxfoundation.org/sites/events/files/slides/Performance%20Monitoring%20and%20Analysis%20Using%20perf%20and%20BPF_1.pdf
> 
> Compare with v1:
>  1. Fix API usage so can be built at Ubuntu 16.04 (with llvm-dev and
>     libclang-dev installed)
>  2. Introduce default include files so BPF script writer doesn't need
>     define many BPF functions by their own.
> 
> Test:
> 
>  # cat ./test_bpf_output.c
>  /************************ BEGIN **************************/
>  #include <uapi/linux/bpf.h>
>  struct bpf_map_def SEC("maps") __bpf_stdout__ = {
>         .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
>         .key_size = sizeof(int),
>         .value_size = sizeof(u32),
>         .max_entries = __NR_CPUS__,
>  };
> 
>  static inline int __attribute__((always_inline))
>  func(void *ctx, int type)
>  {
> 	char output_str[] = "Raise a BPF event!";
> 
>         bpf_perf_event_output(ctx, &__bpf_stdout__, bpf_get_smp_processor_id(),
> 			  &output_str, sizeof(output_str));
>         return 0;
>  }
>  SEC("func_begin=sys_nanosleep")
>  int func_begin(void *ctx) {return func(ctx, 1);}
>  SEC("func_end=sys_nanosleep%return")
>  int func_end(void *ctx) { return func(ctx, 2);}
>  char _license[] SEC("license") = "GPL";
>  int _version SEC("version") = LINUX_VERSION_CODE;
>  /************************* END ***************************/
>  # perf trace --event ./test_bpf_output.c usleep 10
>  ...
>  0.449 ( 0.002 ms): usleep/827 getuid(                                                               ) = 0
>  0.482 ( 0.006 ms): usleep/827 nanosleep(rqtp: 0x7ffecc22fa50                                        ) ...
>  0.482 (         ): __bpf_stdout__:Raise a BPF event!..)
>  0.555 (         ): __bpf_stdout__:Raise a BPF event!..)
>  0.557 ( 0.081 ms): usleep/827  ... [continued]: nanosleep() = 0
>  0.562 ( 0.000 ms): usleep/827 exit_group(                                                           )
> 
> 
> v1 can be found at:
>   https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1238358.html
>   (should be http://lkml.kernel.org/g/1474635001-153850-1-git-send-email-wangnan0@huawei.com
>    but the link is broken)
> 
> Wang Nan (18):
>   tools build: Support compiling C++ source file
>   perf tools: Add feature detection for g++
>   perf tools: Add feature detection for LLVM
>   perf tools: Add feature detection for clang
>   perf build: Add clang and llvm compile and linking support
>   perf clang: Add builtin clang support ant test case
>   perf clang: Use real file system for #include
>   perf clang: Allow passing CFLAGS to builtin clang
>   perf clang: Update test case to use real BPF script
>   perf clang: Support compile IR to BPF object and add testcase
>   perf tools: Extract helpers in llvm-utils.c
>   perf bpf: Compile BPF script use builtin clang support
>   perf clang: Pass full path to builtin clang
>   perf clang: Pass CFLAGS to builtin clang
>   perf clang: Link BPF functions declaration into perf
>   perf clang: Declare BPF functions for BPF scripts automatically
>   perf clang: Include helpers to BPF scripts
>   perf clang: Define PERF_BUILTIN_CLANG for builtin clang compiling
> 
>  tools/build/Build.include                     |   1 +
>  tools/build/Makefile.build                    |   7 +
>  tools/build/Makefile.feature                  |   2 +-
>  tools/build/feature/Makefile                  |  28 ++-
>  tools/build/feature/test-clang.cpp            |  21 ++
>  tools/build/feature/test-cxx.cpp              |  15 ++
>  tools/build/feature/test-llvm.cpp             |   8 +
>  tools/perf/Makefile.config                    |  62 ++++-
>  tools/perf/Makefile.perf                      |  23 +-
>  tools/perf/tests/Build                        |   1 +
>  tools/perf/tests/bpf-script-example.c         |  20 +-
>  tools/perf/tests/bpf-script-test-kbuild.c     |   2 +
>  tools/perf/tests/bpf-script-test-prologue.c   |   4 +
>  tools/perf/tests/bpf-script-test-relocation.c |  20 +-
>  tools/perf/tests/builtin-test.c               |   9 +
>  tools/perf/tests/clang.c                      |  46 ++++
>  tools/perf/tests/llvm-cxx.h                   |  13 +
>  tools/perf/tests/make                         |   2 +
>  tools/perf/tests/tests.h                      |   3 +
>  tools/perf/util/Build                         |   2 +
>  tools/perf/util/bpf-loader.c                  |  15 +-
>  tools/perf/util/c++/Build                     |   4 +
>  tools/perf/util/c++/bpf-funcs-str.c           | 214 +++++++++++++++++
>  tools/perf/util/c++/bpf-helper-str.c          |  13 +
>  tools/perf/util/c++/clang-bpf-includes.h      |  13 +
>  tools/perf/util/c++/clang-c.h                 |  43 ++++
>  tools/perf/util/c++/clang-test.cpp            |  62 +++++
>  tools/perf/util/c++/clang.cpp                 | 329 ++++++++++++++++++++++++++
>  tools/perf/util/c++/clang.h                   |  26 ++
>  tools/perf/util/llvm-utils-cxx.h              |  14 ++
>  tools/perf/util/llvm-utils.c                  |  70 +++++-
>  tools/perf/util/llvm-utils.h                  |   7 +-
>  tools/perf/util/util-cxx.h                    |  26 ++
>  33 files changed, 1078 insertions(+), 47 deletions(-)
>  create mode 100644 tools/build/feature/test-clang.cpp
>  create mode 100644 tools/build/feature/test-cxx.cpp
>  create mode 100644 tools/build/feature/test-llvm.cpp
>  create mode 100644 tools/perf/tests/clang.c
>  create mode 100644 tools/perf/tests/llvm-cxx.h
>  create mode 100644 tools/perf/util/c++/Build
>  create mode 100644 tools/perf/util/c++/bpf-funcs-str.c
>  create mode 100644 tools/perf/util/c++/bpf-helper-str.c
>  create mode 100644 tools/perf/util/c++/clang-bpf-includes.h
>  create mode 100644 tools/perf/util/c++/clang-c.h
>  create mode 100644 tools/perf/util/c++/clang-test.cpp
>  create mode 100644 tools/perf/util/c++/clang.cpp
>  create mode 100644 tools/perf/util/c++/clang.h
>  create mode 100644 tools/perf/util/llvm-utils-cxx.h
>  create mode 100644 tools/perf/util/util-cxx.h
> 
> Cc: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Alexei Starovoitov <ast@fb.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> -- 
> 1.8.3.4

[tip:perf/urgent] tools build: Support compiling C++ source file

[tip-bot for Wang Nan]

Commit-ID:  f61bdc3304281e9ff9eec4f1ed9fb15bcf4d8a5e
Gitweb:     http://git.kernel.org/tip/f61bdc3304281e9ff9eec4f1ed9fb15bcf4d8a5e
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Mon, 26 Sep 2016 07:26:55 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 5 Oct 2016 19:58:51 -0300

tools build: Support compiling C++ source file

Add new rule to compile .cpp file to .o use g++. C++ support is required
for built-in clang and LLVM support.

Linker side support will be introduced by following commits.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1474874832-134786-2-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/build/Build.include  | 1 +
 tools/build/Makefile.build | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/tools/build/Build.include b/tools/build/Build.include
index 0248938..1dcb95e 100644
--- a/tools/build/Build.include
+++ b/tools/build/Build.include
@@ -90,6 +90,7 @@ if_changed = $(if $(strip $(any-prereq) $(arg-check)),             \
 # - per object C flags
 # - BUILD_STR macro to allow '-D"$(variable)"' constructs
 c_flags = -Wp,-MD,$(depfile),-MT,$@ $(CFLAGS) -D"BUILD_STR(s)=\#s" $(CFLAGS_$(basetarget).o) $(CFLAGS_$(obj))
+cxx_flags = -Wp,-MD,$(depfile),-MT,$@ $(CXXFLAGS) -D"BUILD_STR(s)=\#s" $(CXXFLAGS_$(basetarget).o) $(CXXFLAGS_$(obj))
 
 ###
 ## HOSTCC C flags
diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
index 190519a..99c0ccd 100644
--- a/tools/build/Makefile.build
+++ b/tools/build/Makefile.build
@@ -61,6 +61,9 @@ quiet_cmd_cc_o_c = CC       $@
 quiet_cmd_host_cc_o_c = HOSTCC   $@
       cmd_host_cc_o_c = $(HOSTCC) $(host_c_flags) -c -o $@ $<
 
+quiet_cmd_cxx_o_c = CXX      $@
+      cmd_cxx_o_c = $(CXX) $(cxx_flags) -c -o $@ $<
+
 quiet_cmd_cpp_i_c = CPP      $@
       cmd_cpp_i_c = $(CC) $(c_flags) -E -o $@ $<
 
@@ -88,6 +91,10 @@ $(OUTPUT)%.o: %.c FORCE
 	$(call rule_mkdir)
 	$(call if_changed_dep,$(host)cc_o_c)
 
+$(OUTPUT)%.o: %.cpp FORCE
+	$(call rule_mkdir)
+	$(call if_changed_dep,cxx_o_c)
+
 $(OUTPUT)%.o: %.S FORCE
 	$(call rule_mkdir)
 	$(call if_changed_dep,$(host)cc_o_c)

[tip:perf/urgent] tools build: Add feature detection for g++

[tip-bot for Wang Nan]

Commit-ID:  87095f7ddeff3038a0cf8e6574922f9c11688619
Gitweb:     http://git.kernel.org/tip/87095f7ddeff3038a0cf8e6574922f9c11688619
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Mon, 26 Sep 2016 07:26:56 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 5 Oct 2016 19:59:35 -0300

tools build: Add feature detection for g++

Check if g++ is available. The result will be used by builtin clang and
LLVM support. Since LLVM requires C++11, this feature detector checks
std::move().

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1474874832-134786-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/build/Makefile.feature     |  2 +-
 tools/build/feature/Makefile     | 10 +++++++++-
 tools/build/feature/test-cxx.cpp | 15 +++++++++++++++
 3 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index a120c6b..ae52e02 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -7,7 +7,7 @@ endif
 
 feature_check = $(eval $(feature_check_code))
 define feature_check_code
-  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
+  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
 endef
 
 feature_set = $(eval $(feature_set_code))
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index a0b29a3..ac9c477 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -46,11 +46,13 @@ FILES=					\
 	test-lzma.bin			\
 	test-bpf.bin			\
 	test-get_cpuid.bin		\
-	test-sdt.bin
+	test-sdt.bin			\
+	test-cxx.bin
 
 FILES := $(addprefix $(OUTPUT),$(FILES))
 
 CC := $(CROSS_COMPILE)gcc -MD
+CXX := $(CROSS_COMPILE)g++ -MD
 PKG_CONFIG := $(CROSS_COMPILE)pkg-config
 
 all: $(FILES)
@@ -58,6 +60,9 @@ all: $(FILES)
 __BUILD = $(CC) $(CFLAGS) -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $(LDFLAGS)
   BUILD = $(__BUILD) > $(@:.bin=.make.output) 2>&1
 
+__BUILDXX = $(CXX) $(CXXFLAGS) -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS)
+  BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
+
 ###############################
 
 $(OUTPUT)test-all.bin:
@@ -217,6 +222,9 @@ $(OUTPUT)test-bpf.bin:
 $(OUTPUT)test-sdt.bin:
 	$(BUILD)
 
+$(OUTPUT)test-cxx.bin:
+	$(BUILDXX) -std=gnu++11
+
 -include $(OUTPUT)*.d
 
 ###############################
diff --git a/tools/build/feature/test-cxx.cpp b/tools/build/feature/test-cxx.cpp
new file mode 100644
index 0000000..b1dee9a
--- /dev/null
+++ b/tools/build/feature/test-cxx.cpp
@@ -0,0 +1,15 @@
+#include <iostream>
+#include <memory>
+
+static void print_str(std::string s)
+{
+	std::cout << s << std::endl;
+}
+
+int main()
+{
+	std::string s("Hello World!");
+	print_str(std::move(s));
+	std::cout << "|" << s << "|" << std::endl;
+	return 0;
+}

Re: [PATCH v2 00/18] perf clang: Support compiling BPF script on the fly

[Wangnan (F)]

On 2016/10/6 7:20, Arnaldo Carvalho de Melo wrote:
> Em Mon, Sep 26, 2016 at 07:26:54AM +0000, Wang Nan escreveu:
>> This patch add builtin clang, allow perf compile BPF scripts on the fly.
>> This is the first step to implement what I announced at LinuxCon 2016 NA:
> Ok, so I refreshed this series to apply against my latest perf/core and
> put it in a tmp.perf/builtin-clang, will continue testing it tomorrow
> after checking if fedora24 has those llvm-dev and libclang-dev that can
> be used with this, do you know if it works on that distro?

Sorry for the late. I only tested on ubuntu.

I can see a llvm-static package for fedora:

https://apps.fedoraproject.org/packages/llvm-static/

but for clang I can find only dynamic libraries:

https://apps.fedoraproject.org/packages/clang-libs/

I think we can make it work by changing 'libclang$(l).a'
to 'libclang$(l).so' in commit 1125e7f6cf29.

Thank you.