linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang
@ 2016-09-23 12:49 Wang Nan
  2016-09-23 12:49 ` [PATCH 01/14] tools build: Support compiling C++ source file Wang Nan
                   ` (14 more replies)
  0 siblings, 15 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

This patch set is the first step to implement features I announced
in LinuxCon NA 2016. See page 31 of:

 http://events.linuxfoundation.org/sites/events/files/slides/Performance%20Monitoring%20and%20Analysis%20Using%20perf%20and%20BPF_1.pdf

This patch set links LLVM and Clang libraries to perf, so perf
is able to compile BPF script to BPF object on the fly.

To make this this patch set work, developer should first install llvm-devel
and clang-devel (libLLVM*.a and libclang*.a, I compile LLVM from source to
get them and I see ubuntu also ships them), then build perf with LIBCLANGLLVM=1,
and provide a valid llvm-config through LLVM_CONFIG if necessary.

The first 4 patches introduce initial building environment. 3 new
feature detectors are added to ensure the dependency. Patch 5 - 10
support compile C file into IR then object, each step can be tested
by newly introduced 'perf test clang'. Patch 11 - 14 fixes missing
features like kbuild detection and some default macros.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>

Wang Nan (14):
  tools build: Support compiling C++ source file
  perf tools: Add feature detection for g++
  perf tools: Add feature detection for LLVM
  perf tools: Add feature detection for clang
  perf build: Add clang and llvm compile and linking support
  perf clang: Add builtin clang support ant test case
  perf clang: Use real file system for #include
  perf clang: Allow passing CFLAGS to builtin clang
  perf clang: Update test case to use real BPF script
  perf clang: Support compile IR to BPF object and add testcase
  perf tools: Extract kernel build option detector as utils
  perf bpf: Compile BPF script use builtin cflags support
  perf clang: Pass fill path compiler
  perf clang: Pass correct CFLAGS to builtin clang

 tools/build/Build.include          |   1 +
 tools/build/Makefile.build         |   7 +
 tools/build/Makefile.feature       |   2 +-
 tools/build/feature/Makefile       |  28 +++-
 tools/build/feature/test-clang.cpp |  21 +++
 tools/build/feature/test-cxx.cpp   |  15 +++
 tools/build/feature/test-llvm.cpp  |   8 ++
 tools/perf/Makefile.config         |  62 +++++++--
 tools/perf/Makefile.perf           |  23 +++-
 tools/perf/tests/Build             |   1 +
 tools/perf/tests/builtin-test.c    |   9 ++
 tools/perf/tests/clang.c           |  46 +++++++
 tools/perf/tests/llvm-cxx.h        |  13 ++
 tools/perf/tests/make              |   2 +
 tools/perf/tests/tests.h           |   3 +
 tools/perf/util/Build              |   2 +
 tools/perf/util/bpf-loader.c       |  15 ++-
 tools/perf/util/c++/Build          |   2 +
 tools/perf/util/c++/clang-c.h      |  43 ++++++
 tools/perf/util/c++/clang-test.cpp |  58 ++++++++
 tools/perf/util/c++/clang.cpp      | 268 +++++++++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h        |  26 ++++
 tools/perf/util/llvm-utils-cxx.h   |  14 ++
 tools/perf/util/llvm-utils.c       |  44 +++++-
 tools/perf/util/llvm-utils.h       |   5 +-
 tools/perf/util/util-cxx.h         |  26 ++++
 26 files changed, 718 insertions(+), 26 deletions(-)
 create mode 100644 tools/build/feature/test-clang.cpp
 create mode 100644 tools/build/feature/test-cxx.cpp
 create mode 100644 tools/build/feature/test-llvm.cpp
 create mode 100644 tools/perf/tests/clang.c
 create mode 100644 tools/perf/tests/llvm-cxx.h
 create mode 100644 tools/perf/util/c++/Build
 create mode 100644 tools/perf/util/c++/clang-c.h
 create mode 100644 tools/perf/util/c++/clang-test.cpp
 create mode 100644 tools/perf/util/c++/clang.cpp
 create mode 100644 tools/perf/util/c++/clang.h
 create mode 100644 tools/perf/util/llvm-utils-cxx.h
 create mode 100644 tools/perf/util/util-cxx.h

-- 
1.8.3.4

^ permalink raw reply	[flat|nested] 20+ messages in thread
* [PATCH 01/14] tools build: Support compiling C++ source file
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-28 14:33   ` Jiri Olsa
  2016-09-23 12:49 ` [PATCH 02/14] perf tools: Add feature detection for g++ Wang Nan
                   ` (13 subsequent siblings)
  14 siblings, 1 reply; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Add new rule to compile .cpp file to .o use g++. C++ support is
required for built-in clang and LLVM support.

Linker side support will be introduced by following commits.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/Build.include  | 1 +
 tools/build/Makefile.build | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/tools/build/Build.include b/tools/build/Build.include
index 4d000bc..b9f014d 100644
--- a/tools/build/Build.include
+++ b/tools/build/Build.include
@@ -90,3 +90,4 @@ if_changed = $(if $(strip $(any-prereq) $(arg-check)),             \
 # - per object C flags
 # - BUILD_STR macro to allow '-D"$(variable)"' constructs
 c_flags = -Wp,-MD,$(depfile),-MT,$@ $(CFLAGS) -D"BUILD_STR(s)=\#s" $(CFLAGS_$(basetarget).o) $(CFLAGS_$(obj))
+cxx_flags = -Wp,-MD,$(depfile),-MT,$@ $(CXXFLAGS) -D"BUILD_STR(s)=\#s" $(CXXFLAGS_$(basetarget).o) $(CXXFLAGS_$(obj))
diff --git a/tools/build/Makefile.build b/tools/build/Makefile.build
index 27f3583..424c0ec 100644
--- a/tools/build/Makefile.build
+++ b/tools/build/Makefile.build
@@ -58,6 +58,9 @@ quiet_cmd_mkdir = MKDIR    $(dir $@)
 quiet_cmd_cc_o_c = CC       $@
       cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $<
 
+quiet_cmd_cxx_o_c = CXX       $@
+      cmd_cxx_o_c = $(CXX) $(cxx_flags) -c -o $@ $<
+
 quiet_cmd_cpp_i_c = CPP      $@
       cmd_cpp_i_c = $(CC) $(c_flags) -E -o $@ $<
 
@@ -77,6 +80,10 @@ $(OUTPUT)%.o: %.c FORCE
 	$(call rule_mkdir)
 	$(call if_changed_dep,cc_o_c)
 
+$(OUTPUT)%.o: %.cpp FORCE
+	$(call rule_mkdir)
+	$(call if_changed_dep,cxx_o_c)
+
 $(OUTPUT)%.o: %.S FORCE
 	$(call rule_mkdir)
 	$(call if_changed_dep,cc_o_c)
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 02/14] perf tools: Add feature detection for g++
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
  2016-09-23 12:49 ` [PATCH 01/14] tools build: Support compiling C++ source file Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 03/14] perf tools: Add feature detection for LLVM Wang Nan
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, Alexei Starovoitov, He Kuang,
	Jiri Olsa

Check if g++ is available. The result will be used by builtin clang and
LLVM support. Since LLVM requires C++11, this feature detector checks
std::move().

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/Makefile.feature     |  2 +-
 tools/build/feature/Makefile     | 10 +++++++++-
 tools/build/feature/test-cxx.cpp | 15 +++++++++++++++
 3 files changed, 25 insertions(+), 2 deletions(-)
 create mode 100644 tools/build/feature/test-cxx.cpp

diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index a120c6b..ae52e02 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -7,7 +7,7 @@ endif
 
 feature_check = $(eval $(feature_check_code))
 define feature_check_code
-  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
+  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
 endef
 
 feature_set = $(eval $(feature_set_code))
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index a0b29a3..ac9c477 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -46,11 +46,13 @@ FILES=					\
 	test-lzma.bin			\
 	test-bpf.bin			\
 	test-get_cpuid.bin		\
-	test-sdt.bin
+	test-sdt.bin			\
+	test-cxx.bin
 
 FILES := $(addprefix $(OUTPUT),$(FILES))
 
 CC := $(CROSS_COMPILE)gcc -MD
+CXX := $(CROSS_COMPILE)g++ -MD
 PKG_CONFIG := $(CROSS_COMPILE)pkg-config
 
 all: $(FILES)
@@ -58,6 +60,9 @@ all: $(FILES)
 __BUILD = $(CC) $(CFLAGS) -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $(LDFLAGS)
   BUILD = $(__BUILD) > $(@:.bin=.make.output) 2>&1
 
+__BUILDXX = $(CXX) $(CXXFLAGS) -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS)
+  BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
+
 ###############################
 
 $(OUTPUT)test-all.bin:
@@ -217,6 +222,9 @@ $(OUTPUT)test-bpf.bin:
 $(OUTPUT)test-sdt.bin:
 	$(BUILD)
 
+$(OUTPUT)test-cxx.bin:
+	$(BUILDXX) -std=gnu++11
+
 -include $(OUTPUT)*.d
 
 ###############################
diff --git a/tools/build/feature/test-cxx.cpp b/tools/build/feature/test-cxx.cpp
new file mode 100644
index 0000000..b1dee9a
--- /dev/null
+++ b/tools/build/feature/test-cxx.cpp
@@ -0,0 +1,15 @@
+#include <iostream>
+#include <memory>
+
+static void print_str(std::string s)
+{
+	std::cout << s << std::endl;
+}
+
+int main()
+{
+	std::string s("Hello World!");
+	print_str(std::move(s));
+	std::cout << "|" << s << "|" << std::endl;
+	return 0;
+}
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 03/14] perf tools: Add feature detection for LLVM
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
  2016-09-23 12:49 ` [PATCH 01/14] tools build: Support compiling C++ source file Wang Nan
  2016-09-23 12:49 ` [PATCH 02/14] perf tools: Add feature detection for g++ Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 04/14] perf tools: Add feature detection for clang Wang Nan
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Check if basic LLVM compiling environment is ready.

Use llvm-config to detect include and library directories. Avoid using
'llvm-config --cxxflags' because its result contain some unwanted flags
like --sysroot (if LLVM is built by yocto).

Use '?=' to set LLVM_CONFIG, so explicitly passing LLVM_CONFIG to make
would override it.

Use 'llvm-config --libs BPF' to check if BPF backend is compiled in.
Since now BPF bytecode is the only required backend, no need to waste
time linking llvm and clang if BPF backend is missing. This
also introduce an implicit requirement that LLVM should be new enough.
Old LLVM doesn't support BPF backend.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/feature/Makefile      | 8 ++++++++
 tools/build/feature/test-llvm.cpp | 8 ++++++++
 2 files changed, 16 insertions(+)
 create mode 100644 tools/build/feature/test-llvm.cpp

diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index ac9c477..fd2f3e2 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -54,6 +54,7 @@ FILES := $(addprefix $(OUTPUT),$(FILES))
 CC := $(CROSS_COMPILE)gcc -MD
 CXX := $(CROSS_COMPILE)g++ -MD
 PKG_CONFIG := $(CROSS_COMPILE)pkg-config
+LLVM_CONFIG ?= llvm-config
 
 all: $(FILES)
 
@@ -225,6 +226,13 @@ $(OUTPUT)test-sdt.bin:
 $(OUTPUT)test-cxx.bin:
 	$(BUILDXX) -std=gnu++11
 
+$(OUTPUT)test-llvm.bin:
+	$(BUILDXX) -std=gnu++11 					\
+		-I$(shell $(LLVM_CONFIG) --includedir) 		\
+		-L$(shell $(LLVM_CONFIG) --libdir)		\
+		$(shell $(LLVM_CONFIG) --libs Core BPF)		\
+		$(shell $(LLVM_CONFIG) --system-libs)
+
 -include $(OUTPUT)*.d
 
 ###############################
diff --git a/tools/build/feature/test-llvm.cpp b/tools/build/feature/test-llvm.cpp
new file mode 100644
index 0000000..d8d2cee
--- /dev/null
+++ b/tools/build/feature/test-llvm.cpp
@@ -0,0 +1,8 @@
+#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/raw_ostream.h"
+int main()
+{
+	llvm::errs() << "Hello World!\n";
+	llvm::llvm_shutdown();
+	return 0;
+}
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 04/14] perf tools: Add feature detection for clang
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (2 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 03/14] perf tools: Add feature detection for LLVM Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 05/14] perf build: Add clang and llvm compile and linking support Wang Nan
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Check if basic clang compiling environment is ready.

Doesn't like 'llvm-config --libs' which can returns llvm libraries in
right order and duplicates some libraries if necessary, there's no
correspondence for clang libraries (-lclangxxx). to avoid extra
complexity and to avoid new clang breaking libraries ordering, use
--start-group and --end-group.

In this test case, manually identify required clang libs and hope it
to be stable. Putting all clang libraries here is possible (use
make's wildcard), but then feature checking becomes very slow.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/build/feature/Makefile       | 10 ++++++++++
 tools/build/feature/test-clang.cpp | 21 +++++++++++++++++++++
 2 files changed, 31 insertions(+)
 create mode 100644 tools/build/feature/test-clang.cpp

diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index fd2f3e2..0af90dd 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -233,6 +233,16 @@ $(OUTPUT)test-llvm.bin:
 		$(shell $(LLVM_CONFIG) --libs Core BPF)		\
 		$(shell $(LLVM_CONFIG) --system-libs)
 
+$(OUTPUT)test-clang.bin:
+	$(BUILDXX) -std=gnu++11 					\
+		-I$(shell $(LLVM_CONFIG) --includedir) 		\
+		-L$(shell $(LLVM_CONFIG) --libdir)		\
+		-Wl,--start-group -lclangBasic -lclangDriver	\
+		  -lclangFrontend -lclangEdit -lclangLex	\
+		  -lclangAST -Wl,--end-group 			\
+		$(shell $(LLVM_CONFIG) --libs Core option)	\
+		$(shell $(LLVM_CONFIG) --system-libs)
+
 -include $(OUTPUT)*.d
 
 ###############################
diff --git a/tools/build/feature/test-clang.cpp b/tools/build/feature/test-clang.cpp
new file mode 100644
index 0000000..1c6ad41
--- /dev/null
+++ b/tools/build/feature/test-clang.cpp
@@ -0,0 +1,21 @@
+#include "clang/Driver/Driver.h"
+#include "clang/Basic/VirtualFileSystem.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "llvm/ADT/IntrusiveRefCntPtr.h"
+#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace clang;
+using namespace clang::driver;
+
+int main()
+{
+	IntrusiveRefCntPtr<DiagnosticIDs> DiagID(new DiagnosticIDs());
+	IntrusiveRefCntPtr<DiagnosticOptions> DiagOpts = new DiagnosticOptions();
+
+	DiagnosticsEngine Diags(DiagID, &*DiagOpts);
+	Driver TheDriver("test", "bpf-pc-linux", Diags);
+
+	llvm::llvm_shutdown();
+	return 0;
+}
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 05/14] perf build: Add clang and llvm compile and linking support
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (3 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 04/14] perf tools: Add feature detection for clang Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 06/14] perf clang: Add builtin clang support ant test case Wang Nan
                   ` (9 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Add necessary c++ flags and link libraries to support builtin clang
and LLVM. Add all llvm and clang libraries, so don't need to worry
about clang changes its libraries setting. However, linking perf
would take much longer than usual.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/Makefile.config | 35 +++++++++++++++++++++++++++++++++++
 tools/perf/Makefile.perf   | 23 ++++++++++++++++++++++-
 tools/perf/tests/make      |  2 ++
 3 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 72edf83..d977d69 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -136,6 +136,7 @@ endif
 # Treat warnings as errors unless directed not to
 ifneq ($(WERROR),0)
   CFLAGS += -Werror
+  CXXFLAGS += -Werror
 endif
 
 ifndef DEBUG
@@ -182,6 +183,13 @@ CFLAGS += -Wall
 CFLAGS += -Wextra
 CFLAGS += -std=gnu99
 
+CXXFLAGS += -std=gnu++11
+CXXFLAGS += -Wall
+CXXFLAGS += -fno-omit-frame-pointer
+CXXFLAGS += -ggdb3
+CXXFLAGS += -funwind-tables
+CXXFLAGS += -Wno-strict-aliasing
+
 # Enforce a non-executable stack, as we may regress (again) in the future by
 # adding assembler files missing the .GNU-stack linker note.
 LDFLAGS += -Wl,-z,noexecstack
@@ -758,6 +766,33 @@ ifndef NO_AUXTRACE
   endif
 endif
 
+USE_CXX = 0
+USE_CLANGLLVM = 0
+ifdef LIBCLANGLLVM
+  $(call feature_check,cxx)
+  ifneq ($(feature-cxx), 1)
+    msg := $(warning No g++ found, disable clang and llvm support)
+  else
+    $(call feature_check,llvm)
+    ifneq ($(feature-llvm), 1)
+      msg := $(warning No libLLVM found, disable clang and llvm support)
+    else
+      $(call feature_check,clang)
+      ifneq ($(feature-clang), 1)
+        msg := $(warning No libclang found, disable clang and llvm support)
+      else
+        CFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT
+        CXXFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT -I$(shell $(LLVM_CONFIG) --includedir)
+        $(call detected,CONFIG_CXX)
+        $(call detected,CONFIG_CLANGLLVM)
+	USE_CXX = 1
+	USE_LLVM = 1
+	USE_CLANG = 1
+      endif
+    endif
+  endif
+endif
+
 # Among the variables below, these:
 #   perfexecdir
 #   template_dir
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index d710db1..a9dc66c 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -86,6 +86,10 @@ include ../scripts/utilities.mak
 #
 # Define FEATURES_DUMP to provide features detection dump file
 # and bypass the feature detection
+#
+# Define LIBCLANGLLVM if you DO want builtin clang and llvm support.
+# When selected, pass LLVM_CONFIG=/path/to/llvm-config to `make' if
+# llvm-config is not in $PATH.
 
 # As per kernel Makefile, avoid funny character set dependencies
 unexport LC_ALL
@@ -141,10 +145,12 @@ endef
 $(call allow-override,CC,$(CROSS_COMPILE)gcc)
 $(call allow-override,AR,$(CROSS_COMPILE)ar)
 $(call allow-override,LD,$(CROSS_COMPILE)ld)
+$(call allow-override,CXX,$(CROSS_COMPILE)g++)
 
 LD += $(EXTRA_LDFLAGS)
 
 PKG_CONFIG = $(CROSS_COMPILE)pkg-config
+LLVM_CONFIG ?= llvm-config
 
 RM      = rm -f
 LN      = ln -f
@@ -326,6 +332,21 @@ endif
 
 LIBS = -Wl,--whole-archive $(PERFLIBS) -Wl,--no-whole-archive -Wl,--start-group $(EXTLIBS) -Wl,--end-group
 
+ifeq ($(USE_CLANG), 1)
+  CLANGLIBS_LIST = AST Basic CodeGen Driver Frontend Lex Tooling Edit Sema Analysis Parse Serialization
+  LIBCLANG = $(foreach l,$(CLANGLIBS_LIST),$(wildcard $(shell $(LLVM_CONFIG) --libdir)/libclang$(l).a))
+  LIBS += -Wl,--start-group $(LIBCLANG) -Wl,--end-group
+endif
+
+ifeq ($(USE_LLVM), 1)
+  LIBLLVM = $(shell $(LLVM_CONFIG) --libs all) $(shell $(LLVM_CONFIG) --system-libs)
+  LIBS += -L$(shell $(LLVM_CONFIG) --libdir) $(LIBLLVM)
+endif
+
+ifeq ($(USE_CXX), 1)
+  LIBS += -lstdc++
+endif
+
 export INSTALL SHELL_PATH
 
 ### Build rules
@@ -344,7 +365,7 @@ strip: $(PROGRAMS) $(OUTPUT)perf
 
 PERF_IN := $(OUTPUT)perf-in.o
 
-export srctree OUTPUT RM CC LD AR CFLAGS V BISON FLEX AWK
+export srctree OUTPUT RM CC CXX LD AR CFLAGS V BISON FLEX AWK
 include $(srctree)/tools/build/Makefile.include
 
 $(PERF_IN): prepare FORCE
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index 143f4d5..65eaf8f 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -83,6 +83,7 @@ make_no_libbpf	    := NO_LIBBPF=1
 make_no_libcrypto   := NO_LIBCRYPTO=1
 make_with_babeltrace:= LIBBABELTRACE=1
 make_no_sdt	    := NO_SDT=1
+make_with_clangllvm := LIBCLANGLLVM=1
 make_tags           := tags
 make_cscope         := cscope
 make_help           := help
@@ -139,6 +140,7 @@ run += make_no_libbionic
 run += make_no_auxtrace
 run += make_no_libbpf
 run += make_with_babeltrace
+run += make_with_clangllvm
 run += make_help
 run += make_doc
 run += make_perf_o
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 06/14] perf clang: Add builtin clang support ant test case
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (4 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 05/14] perf build: Add clang and llvm compile and linking support Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 07/14] perf clang: Use real file system for #include Wang Nan
                   ` (8 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Add basic clang support in clang.cpp and test__clang() testcase.
The first testcase checks if builtin clang is able to generate LLVM IR.

tests/clang.c is a proxy. Real testcase resides in utils/c++/clang-test.cpp
in c++ and exports C interface to perf test subsystem.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/tests/Build             |  1 +
 tools/perf/tests/builtin-test.c    |  9 ++++
 tools/perf/tests/clang.c           | 42 +++++++++++++++++
 tools/perf/tests/tests.h           |  3 ++
 tools/perf/util/Build              |  2 +
 tools/perf/util/c++/Build          |  2 +
 tools/perf/util/c++/clang-c.h      | 16 +++++++
 tools/perf/util/c++/clang-test.cpp | 31 +++++++++++++
 tools/perf/util/c++/clang.cpp      | 95 ++++++++++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h        | 16 +++++++
 10 files changed, 217 insertions(+)
 create mode 100644 tools/perf/tests/clang.c
 create mode 100644 tools/perf/util/c++/Build
 create mode 100644 tools/perf/util/c++/clang-c.h
 create mode 100644 tools/perf/util/c++/clang-test.cpp
 create mode 100644 tools/perf/util/c++/clang.cpp
 create mode 100644 tools/perf/util/c++/clang.h

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index dc51bc5..6a20004 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -42,6 +42,7 @@ perf-y += backward-ring-buffer.o
 perf-y += sdt.o
 perf-y += is_printable_array.o
 perf-y += bitmap.o
+perf-y += clang.o
 
 $(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build
 	$(call rule_mkdir)
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 778668a..b9b6053 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -230,6 +230,15 @@ static struct test generic_tests[] = {
 		.func = test__bitmap_print,
 	},
 	{
+		.desc = "Test builtin clang support",
+		.func = test__clang,
+		.subtest = {
+			.skip_if_fail	= true,
+			.get_nr		= test__clang_subtest_get_nr,
+			.get_desc	= test__clang_subtest_get_desc,
+		}
+	},
+	{
 		.func = NULL,
 	},
 };
diff --git a/tools/perf/tests/clang.c b/tools/perf/tests/clang.c
new file mode 100644
index 0000000..57ee160
--- /dev/null
+++ b/tools/perf/tests/clang.c
@@ -0,0 +1,42 @@
+#include "tests.h"
+#include "debug.h"
+#include "util.h"
+#include "c++/clang-c.h"
+
+static struct {
+	int (*func)(void);
+	const char *desc;
+} clang_testcase_table[] = {
+#ifdef HAVE_LIBCLANGLLVM_SUPPORT
+	{
+		.func = test__clang_to_IR,
+		.desc = "Test builtin clang compile C source to IR",
+	},
+#endif
+};
+
+int test__clang_subtest_get_nr(void)
+{
+	return (int)ARRAY_SIZE(clang_testcase_table);
+}
+
+const char *test__clang_subtest_get_desc(int i)
+{
+	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
+		return NULL;
+	return clang_testcase_table[i].desc;
+}
+
+#ifndef HAVE_LIBCLANGLLVM_SUPPORT
+int test__clang(int i __maybe_unused)
+{
+	return TEST_SKIP;
+}
+#else
+int test__clang(int i __maybe_unused)
+{
+	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
+		return TEST_FAIL;
+	return clang_testcase_table[i].func();
+}
+#endif
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 7c196c5..39a4887 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -91,6 +91,9 @@ int test__cpu_map_print(int subtest);
 int test__sdt_event(int subtest);
 int test__is_printable_array(int subtest);
 int test__bitmap_print(int subtest);
+int test__clang(int subtest);
+const char *test__clang_subtest_get_desc(int subtest);
+int test__clang_subtest_get_nr(void);
 
 #if defined(__arm__) || defined(__aarch64__)
 #ifdef HAVE_DWARF_UNWIND_SUPPORT
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index eb60e61..20044d0 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -123,6 +123,8 @@ libperf-$(CONFIG_LIBELF) += genelf.o
 libperf-$(CONFIG_LIBELF) += genelf_debug.o
 endif
 
+libperf-$(CONFIG_CXX) += c++/
+
 CFLAGS_config.o   += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 # avoid compiler warnings in 32-bit mode
 CFLAGS_genelf_debug.o  += -Wno-packed
diff --git a/tools/perf/util/c++/Build b/tools/perf/util/c++/Build
new file mode 100644
index 0000000..988fef1
--- /dev/null
+++ b/tools/perf/util/c++/Build
@@ -0,0 +1,2 @@
+libperf-$(CONFIG_CLANGLLVM) += clang.o
+libperf-$(CONFIG_CLANGLLVM) += clang-test.o
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
new file mode 100644
index 0000000..dcde4b5
--- /dev/null
+++ b/tools/perf/util/c++/clang-c.h
@@ -0,0 +1,16 @@
+#ifndef PERF_UTIL_CLANG_C_H
+#define PERF_UTIL_CLANG_C_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+extern void perf_clang__init(void);
+extern void perf_clang__cleanup(void);
+
+extern int test__clang_to_IR(void);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
new file mode 100644
index 0000000..3da6bfa
--- /dev/null
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -0,0 +1,31 @@
+#include "clang.h"
+#include "clang-c.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/LLVMContext.h"
+
+class perf_clang_scope {
+public:
+	explicit perf_clang_scope() {perf_clang__init();}
+	~perf_clang_scope() {perf_clang__cleanup();}
+};
+
+extern "C" {
+
+int test__clang_to_IR(void)
+{
+	perf_clang_scope _scope;
+
+	std::unique_ptr<llvm::Module> M =
+		perf::getModuleFromSource("perf-test.c",
+					  "int myfunc(void) {return 1;}");
+
+	if (!M)
+		return -1;
+
+	for (llvm::Function& F : *M)
+		if (F.getName() == "myfunc")
+			return 0;
+	return -1;
+}
+
+}
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
new file mode 100644
index 0000000..d0cba4c
--- /dev/null
+++ b/tools/perf/util/c++/clang.cpp
@@ -0,0 +1,95 @@
+/*
+ * llvm C frontend for perf. Support dynamically compile C file
+ *
+ * Inspired by clang example code:
+ * http://llvm.org/svn/llvm-project/cfe/trunk/examples/clang-interpreter/main.cpp
+ *
+ * Copyright (C) 2016 Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2016 Huawei Inc.
+ */
+
+#include "clang/Frontend/CompilerInvocation.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "clang/Tooling/Tooling.h"
+#include "clang/CodeGen/CodeGenAction.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Option/Option.h"
+#include <memory>
+
+#include "clang.h"
+#include "clang-c.h"
+
+namespace perf {
+
+static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
+
+using namespace clang;
+
+static vfs::InMemoryFileSystem *
+buildVFS(StringRef& Name, StringRef& Content)
+{
+	vfs::InMemoryFileSystem *VFS = new vfs::InMemoryFileSystem(true);
+	VFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
+	return VFS;
+}
+
+static CompilerInvocation *
+createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
+{
+	llvm::opt::ArgStringList CCArgs {
+		"-cc1",
+		"-triple", "bpf-pc-linux",
+		"-fsyntax-only",
+		"-ferror-limit", "19",
+		"-fmessage-length", "127",
+		"-O2",
+		"-nostdsysteminc",
+		"-nobuiltininc",
+		"-vectorize-loops",
+		"-vectorize-slp",
+		"-Wno-unused-value",
+		"-Wno-pointer-sign",
+		"-x", "c"};
+	CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs);
+
+	FrontendOptions& Opts = CI->getFrontendOpts();
+	Opts.Inputs.clear();
+	Opts.Inputs.emplace_back(Path, IK_C);
+	return CI;
+}
+
+std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Name, StringRef Content)
+{
+	CompilerInstance Clang;
+	Clang.createDiagnostics();
+
+	IntrusiveRefCntPtr<vfs::FileSystem> VFS = buildVFS(Name, Content);
+	Clang.setVirtualFileSystem(&*VFS);
+
+	IntrusiveRefCntPtr<CompilerInvocation> CI =
+		createCompilerInvocation(Name, Clang.getDiagnostics());
+	Clang.setInvocation(&*CI);
+
+	std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx));
+	if (!Clang.ExecuteAction(*Act))
+		return std::unique_ptr<llvm::Module>(nullptr);
+
+	return Act->takeModule();
+}
+
+}
+
+extern "C" {
+void perf_clang__init(void)
+{
+	perf::LLVMCtx.reset(new llvm::LLVMContext());
+}
+
+void perf_clang__cleanup(void)
+{
+	perf::LLVMCtx.reset(nullptr);
+	llvm::llvm_shutdown();
+}
+}
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
new file mode 100644
index 0000000..f64483b
--- /dev/null
+++ b/tools/perf/util/c++/clang.h
@@ -0,0 +1,16 @@
+#ifndef PERF_UTIL_CLANG_H
+#define PERF_UTIL_CLANG_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include <memory>
+namespace perf {
+
+using namespace llvm;
+
+std::unique_ptr<Module>
+getModuleFromSource(StringRef Name, StringRef Content);
+
+}
+#endif
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 07/14] perf clang: Use real file system for #include
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (5 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 06/14] perf clang: Add builtin clang support ant test case Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 08/14] perf clang: Allow passing CFLAGS to builtin clang Wang Nan
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Utilize clang's OverlayFileSystem facility to enable VFS passed to
CompilerInstance can access files in real file system, so '#include'
directive can be used. Add a new getModuleFromSource for real file.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang.cpp | 43 +++++++++++++++++++++++++++++++------------
 tools/perf/util/c++/clang.h   |  3 +++
 2 files changed, 34 insertions(+), 12 deletions(-)

diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index d0cba4c..f032ea8 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -26,14 +26,6 @@ static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
 
 using namespace clang;
 
-static vfs::InMemoryFileSystem *
-buildVFS(StringRef& Name, StringRef& Content)
-{
-	vfs::InMemoryFileSystem *VFS = new vfs::InMemoryFileSystem(true);
-	VFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
-	return VFS;
-}
-
 static CompilerInvocation *
 createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 {
@@ -59,17 +51,17 @@ createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 	return CI;
 }
 
-std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Name, StringRef Content)
+static std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Path,
+		    IntrusiveRefCntPtr<vfs::FileSystem> VFS)
 {
 	CompilerInstance Clang;
 	Clang.createDiagnostics();
 
-	IntrusiveRefCntPtr<vfs::FileSystem> VFS = buildVFS(Name, Content);
 	Clang.setVirtualFileSystem(&*VFS);
 
 	IntrusiveRefCntPtr<CompilerInvocation> CI =
-		createCompilerInvocation(Name, Clang.getDiagnostics());
+		createCompilerInvocation(Path, Clang.getDiagnostics());
 	Clang.setInvocation(&*CI);
 
 	std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx));
@@ -79,6 +71,33 @@ getModuleFromSource(StringRef Name, StringRef Content)
 	return Act->takeModule();
 }
 
+std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Name, StringRef Content)
+{
+	using namespace vfs;
+
+	llvm::IntrusiveRefCntPtr<OverlayFileSystem> OverlayFS(
+			new OverlayFileSystem(getRealFileSystem()));
+	llvm::IntrusiveRefCntPtr<InMemoryFileSystem> MemFS(
+			new InMemoryFileSystem(true));
+
+	/*
+	 * pushOverlay helps setting working dir for MemFS. Must call
+	 * before addFile.
+	 */
+	OverlayFS->pushOverlay(MemFS);
+	MemFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
+
+	return getModuleFromSource(Name, OverlayFS);
+}
+
+std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Path)
+{
+	IntrusiveRefCntPtr<vfs::FileSystem> VFS(vfs::getRealFileSystem());
+	return getModuleFromSource(Path, VFS);
+}
+
 }
 
 extern "C" {
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
index f64483b..90aff01 100644
--- a/tools/perf/util/c++/clang.h
+++ b/tools/perf/util/c++/clang.h
@@ -12,5 +12,8 @@ using namespace llvm;
 std::unique_ptr<Module>
 getModuleFromSource(StringRef Name, StringRef Content);
 
+std::unique_ptr<Module>
+getModuleFromSource(StringRef Path);
+
 }
 #endif
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 08/14] perf clang: Allow passing CFLAGS to builtin clang
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (6 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 07/14] perf clang: Use real file system for #include Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 09/14] perf clang: Update test case to use real BPF script Wang Nan
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Improve getModuleFromSource() API to accept a cflags list. This feature
will be used to pass LINUX_VERSION_CODE and -I flags.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang-test.cpp |  5 +++--
 tools/perf/util/c++/clang.cpp      | 21 +++++++++++++--------
 tools/perf/util/c++/clang.h        |  8 ++++++--
 3 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
index 3da6bfa..0f484fb 100644
--- a/tools/perf/util/c++/clang-test.cpp
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -16,8 +16,9 @@ int test__clang_to_IR(void)
 	perf_clang_scope _scope;
 
 	std::unique_ptr<llvm::Module> M =
-		perf::getModuleFromSource("perf-test.c",
-					  "int myfunc(void) {return 1;}");
+		perf::getModuleFromSource({"-DRESULT=1"},
+					  "perf-test.c",
+					  "int myfunc(void) {return RESULT;}");
 
 	if (!M)
 		return -1;
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index f032ea8..f6f6a8c 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -27,7 +27,8 @@ static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
 using namespace clang;
 
 static CompilerInvocation *
-createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
+createCompilerInvocation(llvm::opt::ArgStringList CFlags, StringRef& Path,
+			 DiagnosticsEngine& Diags)
 {
 	llvm::opt::ArgStringList CCArgs {
 		"-cc1",
@@ -43,6 +44,8 @@ createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 		"-Wno-unused-value",
 		"-Wno-pointer-sign",
 		"-x", "c"};
+
+	CCArgs.append(CFlags.begin(), CFlags.end());
 	CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs);
 
 	FrontendOptions& Opts = CI->getFrontendOpts();
@@ -52,8 +55,8 @@ createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
 }
 
 static std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Path,
-		    IntrusiveRefCntPtr<vfs::FileSystem> VFS)
+getModuleFromSource(llvm::opt::ArgStringList CFlags,
+		    StringRef Path, IntrusiveRefCntPtr<vfs::FileSystem> VFS)
 {
 	CompilerInstance Clang;
 	Clang.createDiagnostics();
@@ -61,7 +64,8 @@ getModuleFromSource(StringRef Path,
 	Clang.setVirtualFileSystem(&*VFS);
 
 	IntrusiveRefCntPtr<CompilerInvocation> CI =
-		createCompilerInvocation(Path, Clang.getDiagnostics());
+		createCompilerInvocation(std::move(CFlags), Path,
+					 Clang.getDiagnostics());
 	Clang.setInvocation(&*CI);
 
 	std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx));
@@ -72,7 +76,8 @@ getModuleFromSource(StringRef Path,
 }
 
 std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Name, StringRef Content)
+getModuleFromSource(llvm::opt::ArgStringList CFlags,
+		    StringRef Name, StringRef Content)
 {
 	using namespace vfs;
 
@@ -88,14 +93,14 @@ getModuleFromSource(StringRef Name, StringRef Content)
 	OverlayFS->pushOverlay(MemFS);
 	MemFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
 
-	return getModuleFromSource(Name, OverlayFS);
+	return getModuleFromSource(std::move(CFlags), Name, OverlayFS);
 }
 
 std::unique_ptr<llvm::Module>
-getModuleFromSource(StringRef Path)
+getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path)
 {
 	IntrusiveRefCntPtr<vfs::FileSystem> VFS(vfs::getRealFileSystem());
-	return getModuleFromSource(Path, VFS);
+	return getModuleFromSource(std::move(CFlags), Path, VFS);
 }
 
 }
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
index 90aff01..b4fc2a9 100644
--- a/tools/perf/util/c++/clang.h
+++ b/tools/perf/util/c++/clang.h
@@ -4,16 +4,20 @@
 #include "llvm/ADT/StringRef.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
+#include "llvm/Option/Option.h"
 #include <memory>
+
 namespace perf {
 
 using namespace llvm;
 
 std::unique_ptr<Module>
-getModuleFromSource(StringRef Name, StringRef Content);
+getModuleFromSource(opt::ArgStringList CFlags,
+		    StringRef Name, StringRef Content);
 
 std::unique_ptr<Module>
-getModuleFromSource(StringRef Path);
+getModuleFromSource(opt::ArgStringList CFlags,
+		    StringRef Path);
 
 }
 #endif
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 09/14] perf clang: Update test case to use real BPF script
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (7 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 08/14] perf clang: Allow passing CFLAGS to builtin clang Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 10/14] perf clang: Support compile IR to BPF object and add testcase Wang Nan
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Allow C++ code to use util.h and tests/llvm.h. Let perf test
to compile a real BPF script.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/Makefile.config         | 27 +++++++++++++++------------
 tools/perf/tests/llvm-cxx.h        | 13 +++++++++++++
 tools/perf/util/c++/clang-test.cpp | 17 ++++++++++++++---
 tools/perf/util/util-cxx.h         | 26 ++++++++++++++++++++++++++
 4 files changed, 68 insertions(+), 15 deletions(-)
 create mode 100644 tools/perf/tests/llvm-cxx.h
 create mode 100644 tools/perf/util/util-cxx.h

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index d977d69..778623e 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -212,24 +212,27 @@ ifeq ($(DEBUG),0)
   endif
 endif
 
-CFLAGS += -I$(src-perf)/util/include
-CFLAGS += -I$(src-perf)/arch/$(ARCH)/include
-CFLAGS += -I$(srctree)/tools/include/uapi
-CFLAGS += -I$(srctree)/tools/include/
-CFLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/uapi
-CFLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/
-CFLAGS += -I$(srctree)/tools/arch/$(ARCH)/
+INC_FLAGS += -I$(src-perf)/util/include
+INC_FLAGS += -I$(src-perf)/arch/$(ARCH)/include
+INC_FLAGS += -I$(srctree)/tools/include/uapi
+INC_FLAGS += -I$(srctree)/tools/include/
+INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/uapi
+INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/include/
+INC_FLAGS += -I$(srctree)/tools/arch/$(ARCH)/
 
 # $(obj-perf)      for generated common-cmds.h
 # $(obj-perf)/util for generated bison/flex headers
 ifneq ($(OUTPUT),)
-CFLAGS += -I$(obj-perf)/util
-CFLAGS += -I$(obj-perf)
+INC_FLAGS += -I$(obj-perf)/util
+INC_FLAGS += -I$(obj-perf)
 endif
 
-CFLAGS += -I$(src-perf)/util
-CFLAGS += -I$(src-perf)
-CFLAGS += -I$(srctree)/tools/lib/
+INC_FLAGS += -I$(src-perf)/util
+INC_FLAGS += -I$(src-perf)
+INC_FLAGS += -I$(srctree)/tools/lib/
+
+CFLAGS   += $(INC_FLAGS)
+CXXFLAGS += $(INC_FLAGS)
 
 CFLAGS += -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
 
diff --git a/tools/perf/tests/llvm-cxx.h b/tools/perf/tests/llvm-cxx.h
new file mode 100644
index 0000000..d9c98fa
--- /dev/null
+++ b/tools/perf/tests/llvm-cxx.h
@@ -0,0 +1,13 @@
+#ifndef PERF_TEST_LLVM_CXX_H
+#define PERF_TEST_LLVM_CXX_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "tests/llvm.h"
+
+#ifdef __cplusplus
+}
+#endif
+#endif
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
index 0f484fb..c5546fd 100644
--- a/tools/perf/util/c++/clang-test.cpp
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -3,6 +3,10 @@
 #include "llvm/IR/Function.h"
 #include "llvm/IR/LLVMContext.h"
 
+#include <util-cxx.h>
+#include <tests/llvm-cxx.h>
+#include <string>
+
 class perf_clang_scope {
 public:
 	explicit perf_clang_scope() {perf_clang__init();}
@@ -14,17 +18,24 @@ extern "C" {
 int test__clang_to_IR(void)
 {
 	perf_clang_scope _scope;
+	unsigned int kernel_version;
+
+	if (fetch_kernel_version(&kernel_version, NULL, 0))
+		return -1;
+
+	std::string cflag_kver("-DLINUX_VERSION_CODE=" +
+				std::to_string(kernel_version));
 
 	std::unique_ptr<llvm::Module> M =
-		perf::getModuleFromSource({"-DRESULT=1"},
+		perf::getModuleFromSource({cflag_kver.c_str()},
 					  "perf-test.c",
-					  "int myfunc(void) {return RESULT;}");
+					  test_llvm__bpf_base_prog);
 
 	if (!M)
 		return -1;
 
 	for (llvm::Function& F : *M)
-		if (F.getName() == "myfunc")
+		if (F.getName() == "bpf_func__SyS_epoll_wait")
 			return 0;
 	return -1;
 }
diff --git a/tools/perf/util/util-cxx.h b/tools/perf/util/util-cxx.h
new file mode 100644
index 0000000..0e0e019
--- /dev/null
+++ b/tools/perf/util/util-cxx.h
@@ -0,0 +1,26 @@
+/*
+ * Support C++ source use utilities defined in util.h
+ */
+
+#ifndef PERF_UTIL_UTIL_CXX_H
+#define PERF_UTIL_UTIL_CXX_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+ * Now 'new' is the only C++ keyword found in util.h:
+ * in tools/include/linux/rbtree.h
+ *
+ * Other keywords, like class and delete, should be
+ * redefined if necessary.
+ */
+#define new _new
+#include "util.h"
+#undef new
+
+#ifdef __cplusplus
+}
+#endif
+#endif
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 10/14] perf clang: Support compile IR to BPF object and add testcase
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (8 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 09/14] perf clang: Update test case to use real BPF script Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 11/14] perf tools: Extract kernel build option detector as utils Wang Nan
                   ` (4 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

getBPFObjectFromModule() is introduced to use LLVM to convert Module
(LLVM IR) to BPF object code. Add new testcase for it.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/tests/clang.c           |  6 ++++-
 tools/perf/util/c++/clang-c.h      |  1 +
 tools/perf/util/c++/clang-test.cpp | 33 ++++++++++++++++++--------
 tools/perf/util/c++/clang.cpp      | 47 ++++++++++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h        |  3 +++
 5 files changed, 80 insertions(+), 10 deletions(-)

diff --git a/tools/perf/tests/clang.c b/tools/perf/tests/clang.c
index 57ee160..2964c06 100644
--- a/tools/perf/tests/clang.c
+++ b/tools/perf/tests/clang.c
@@ -12,6 +12,10 @@ static struct {
 		.func = test__clang_to_IR,
 		.desc = "Test builtin clang compile C source to IR",
 	},
+	{
+		.func = test__clang_to_obj,
+		.desc = "Test builtin clang compile C source to ELF object",
+	},
 #endif
 };
 
@@ -33,7 +37,7 @@ int test__clang(int i __maybe_unused)
 	return TEST_SKIP;
 }
 #else
-int test__clang(int i __maybe_unused)
+int test__clang(int i)
 {
 	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
 		return TEST_FAIL;
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
index dcde4b5..22b3936 100644
--- a/tools/perf/util/c++/clang-c.h
+++ b/tools/perf/util/c++/clang-c.h
@@ -9,6 +9,7 @@ extern void perf_clang__init(void);
 extern void perf_clang__cleanup(void);
 
 extern int test__clang_to_IR(void);
+extern int test__clang_to_obj(void);
 
 #ifdef __cplusplus
 }
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
index c5546fd..0f30e4d 100644
--- a/tools/perf/util/c++/clang-test.cpp
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -13,15 +13,13 @@ public:
 	~perf_clang_scope() {perf_clang__cleanup();}
 };
 
-extern "C" {
-
-int test__clang_to_IR(void)
+static std::unique_ptr<llvm::Module>
+__test__clang_to_IR(void)
 {
-	perf_clang_scope _scope;
 	unsigned int kernel_version;
 
 	if (fetch_kernel_version(&kernel_version, NULL, 0))
-		return -1;
+		return std::unique_ptr<llvm::Module>(nullptr);
 
 	std::string cflag_kver("-DLINUX_VERSION_CODE=" +
 				std::to_string(kernel_version));
@@ -30,14 +28,31 @@ int test__clang_to_IR(void)
 		perf::getModuleFromSource({cflag_kver.c_str()},
 					  "perf-test.c",
 					  test_llvm__bpf_base_prog);
+	return M;
+}
+
+extern "C" {
+int test__clang_to_IR(void)
+{
+	perf_clang_scope _scope;
+
+	if (!__test__clang_to_IR())
+		return -1;
+	return 0;
+}
 
+int test__clang_to_obj(void)
+{
+	perf_clang_scope _scope;
+
+	auto M = __test__clang_to_IR();
 	if (!M)
 		return -1;
 
-	for (llvm::Function& F : *M)
-		if (F.getName() == "bpf_func__SyS_epoll_wait")
-			return 0;
-	return -1;
+	auto Buffer = perf::getBPFObjectFromModule(&*M);
+	if (!Buffer)
+		return -1;
+	return 0;
 }
 
 }
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index f6f6a8c..38dcc14 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -15,6 +15,11 @@
 #include "clang/CodeGen/CodeGenAction.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Option/Option.h"
+#include "llvm/Support/TargetSelect.h"
+#include "llvm/Support/TargetRegistry.h"
+#include "llvm/Target/TargetMachine.h"
+#include "llvm/Target/TargetOptions.h"
+#include "llvm/IR/LegacyPassManager.h"
 #include <memory>
 
 #include "clang.h"
@@ -103,6 +108,48 @@ getModuleFromSource(llvm::opt::ArgStringList CFlags, StringRef Path)
 	return getModuleFromSource(std::move(CFlags), Path, VFS);
 }
 
+std::unique_ptr<llvm::SmallVectorImpl<char>>
+getBPFObjectFromModule(llvm::Module *Module)
+{
+	using namespace llvm;
+
+	LLVMInitializeBPFTargetInfo();
+	LLVMInitializeBPFTarget();
+	LLVMInitializeBPFTargetMC();
+	LLVMInitializeBPFAsmPrinter();
+
+	std::string TargetTriple("bpf-pc-linux");
+	std::string Error;
+	const Target* Target = TargetRegistry::lookupTarget(TargetTriple, Error);
+	if (!Target) {
+		llvm::errs() << Error;
+		return std::unique_ptr<llvm::SmallVectorImpl<char>>(nullptr);
+	}
+
+	llvm::TargetOptions Opt;
+	Optional<Reloc::Model> RM = Optional<Reloc::Model>();
+	TargetMachine *TargetMachine =
+		Target->createTargetMachine(TargetTriple,
+					    "generic", "",
+					    Opt, RM);
+
+	Module->setDataLayout(TargetMachine->createDataLayout());
+	Module->setTargetTriple(TargetTriple);
+
+	std::unique_ptr<SmallVectorImpl<char>> Buffer(new SmallVector<char, 0>());
+	raw_svector_ostream ostream(*Buffer);
+
+	legacy::PassManager PM;
+	if (TargetMachine->addPassesToEmitFile(PM, ostream,
+					       TargetMachine::CGFT_ObjectFile)) {
+		llvm::errs() << "TargetMachine can't emit a file of this type\n";
+		return std::unique_ptr<llvm::SmallVectorImpl<char>>(nullptr);;
+	}
+	PM.run(*Module);
+
+	return std::move(Buffer);
+}
+
 }
 
 extern "C" {
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
index b4fc2a9..dd8b042 100644
--- a/tools/perf/util/c++/clang.h
+++ b/tools/perf/util/c++/clang.h
@@ -19,5 +19,8 @@ std::unique_ptr<Module>
 getModuleFromSource(opt::ArgStringList CFlags,
 		    StringRef Path);
 
+std::unique_ptr<llvm::SmallVectorImpl<char>>
+getBPFObjectFromModule(llvm::Module *Module);
+
 }
 #endif
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 11/14] perf tools: Extract kernel build option detector as utils
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (9 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 10/14] perf clang: Support compile IR to BPF object and add testcase Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:49 ` [PATCH 12/14] perf bpf: Compile BPF script use builtin cflags support Wang Nan
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Following commits will use builtin clang to compile BPF script.
llvm__get_kbuild_opts() is extracted to help it detect kbuild dir
and include options.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/llvm-utils.c | 44 ++++++++++++++++++++++++++++++++++++++------
 tools/perf/util/llvm-utils.h |  3 +++
 2 files changed, 41 insertions(+), 6 deletions(-)

diff --git a/tools/perf/util/llvm-utils.c b/tools/perf/util/llvm-utils.c
index bf7216b..d780a82 100644
--- a/tools/perf/util/llvm-utils.c
+++ b/tools/perf/util/llvm-utils.c
@@ -7,6 +7,7 @@
 #include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <linux/err.h>
 #include "debug.h"
 #include "llvm-utils.h"
 #include "config.h"
@@ -282,9 +283,10 @@ static const char *kinc_fetch_script =
 "rm -rf $TMPDIR\n"
 "exit $RET\n";
 
-static inline void
-get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
+void llvm__get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 {
+	static char *saved_kbuild_dir;
+	static char *saved_kbuild_include_opts;
 	int err;
 
 	if (!kbuild_dir || !kbuild_include_opts)
@@ -293,10 +295,28 @@ get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 	*kbuild_dir = NULL;
 	*kbuild_include_opts = NULL;
 
+	if (saved_kbuild_dir && saved_kbuild_include_opts &&
+	    !IS_ERR(saved_kbuild_dir) && !IS_ERR(saved_kbuild_include_opts)) {
+		*kbuild_dir = strdup(saved_kbuild_dir);
+		*kbuild_include_opts = strdup(saved_kbuild_include_opts);
+
+		if (*kbuild_dir && *kbuild_include_opts)
+			return;
+
+		zfree(kbuild_dir);
+		zfree(kbuild_include_opts);
+		/*
+		 * Don't fall through: it may breaks saved_kbuild_dir and
+		 * saved_kbuild_include_opts if detect them again when
+		 * memory is low.
+		 */
+		return;
+	}
+
 	if (llvm_param.kbuild_dir && !llvm_param.kbuild_dir[0]) {
 		pr_debug("[llvm.kbuild-dir] is set to \"\" deliberately.\n");
 		pr_debug("Skip kbuild options detection.\n");
-		return;
+		goto errout;
 	}
 
 	err = detect_kbuild_dir(kbuild_dir);
@@ -306,7 +326,7 @@ get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 "Hint:\tSet correct kbuild directory using 'kbuild-dir' option in [llvm]\n"
 "     \tsection of ~/.perfconfig or set it to \"\" to suppress kbuild\n"
 "     \tdetection.\n\n");
-		return;
+		goto errout;
 	}
 
 	pr_debug("Kernel build dir is set to %s\n", *kbuild_dir);
@@ -325,10 +345,22 @@ get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts)
 
 		free(*kbuild_dir);
 		*kbuild_dir = NULL;
-		return;
+		goto errout;
 	}
 
 	pr_debug("include option is set to %s\n", *kbuild_include_opts);
+
+	saved_kbuild_dir = strdup(*kbuild_dir);
+	saved_kbuild_include_opts = strdup(*kbuild_include_opts);
+
+	if (!saved_kbuild_dir || !saved_kbuild_include_opts) {
+		zfree(&saved_kbuild_dir);
+		zfree(&saved_kbuild_include_opts);
+	}
+	return;
+errout:
+	saved_kbuild_dir = ERR_PTR(-EINVAL);
+	saved_kbuild_include_opts = ERR_PTR(-EINVAL);
 }
 
 static void
@@ -406,7 +438,7 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf,
 	 * This is an optional work. Even it fail we can continue our
 	 * work. Needn't to check error return.
 	 */
-	get_kbuild_opts(&kbuild_dir, &kbuild_include_opts);
+	llvm__get_kbuild_opts(&kbuild_dir, &kbuild_include_opts);
 
 	nr_cpus_avail = sysconf(_SC_NPROCESSORS_CONF);
 	if (nr_cpus_avail <= 0) {
diff --git a/tools/perf/util/llvm-utils.h b/tools/perf/util/llvm-utils.h
index 9f501ce..d3f410d 100644
--- a/tools/perf/util/llvm-utils.h
+++ b/tools/perf/util/llvm-utils.h
@@ -50,4 +50,7 @@ int llvm__compile_bpf(const char *path, void **p_obj_buf, size_t *p_obj_buf_sz);
 
 /* This function is for test__llvm() use only */
 int llvm__search_clang(void);
+
+/* This function is reused by builtin clang support */
+void llvm__get_kbuild_opts(char **kbuild_dir, char **kbuild_include_opts);
 #endif
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 12/14] perf bpf: Compile BPF script use builtin cflags support
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (10 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 11/14] perf tools: Extract kernel build option detector as utils Wang Nan
@ 2016-09-23 12:49 ` Wang Nan
  2016-09-23 12:50 ` [PATCH 13/14] perf clang: Pass fill path compiler Wang Nan
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:49 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

After this patch, perf utilizes builtin cflags support to build BPF
script, no longer depend on external clang.

Test:

 $ type clang
 -bash: type: clang: not found
 $ cat ~/.perfconfig
 $ echo '#define LINUX_VERSION_CODE 0x040700' > ./test.c
 $ cat ./tools/perf/tests/bpf-script-example.c >> ./test.c
 $ ./perf record -v --dry-run -e ./test.c 2>&1 | grep builtin
 bpf: builtin compiling successful

Can't pass cflags so unable to include kernel headers now. Will be
fixed by following commits.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/bpf-loader.c  | 15 +++++++++++----
 tools/perf/util/c++/clang-c.h | 26 ++++++++++++++++++++++++++
 tools/perf/util/c++/clang.cpp | 29 +++++++++++++++++++++++++++++
 3 files changed, 66 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/bpf-loader.c b/tools/perf/util/bpf-loader.c
index 2b2c9b8..a5ddb8e 100644
--- a/tools/perf/util/bpf-loader.c
+++ b/tools/perf/util/bpf-loader.c
@@ -14,11 +14,11 @@
 #include "debug.h"
 #include "bpf-loader.h"
 #include "bpf-prologue.h"
-#include "llvm-utils.h"
 #include "probe-event.h"
 #include "probe-finder.h" // for MAX_PROBES
 #include "parse-events.h"
 #include "llvm-utils.h"
+#include "c++/clang-c.h"
 
 #define DEFINE_PRINT_FN(name, level) \
 static int libbpf_##name(const char *fmt, ...)	\
@@ -86,9 +86,16 @@ struct bpf_object *bpf__prepare_load(const char *filename, bool source)
 		void *obj_buf;
 		size_t obj_buf_sz;
 
-		err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
-		if (err)
-			return ERR_PTR(-BPF_LOADER_ERRNO__COMPILE);
+		perf_clang__init();
+		err = perf_clang__compile_bpf(filename, &obj_buf, &obj_buf_sz);
+		perf_clang__cleanup();
+		if (err) {
+			pr_debug("bpf: builtin compiling failed: %d, try external compiler\n", err);
+			err = llvm__compile_bpf(filename, &obj_buf, &obj_buf_sz);
+			if (err)
+				return ERR_PTR(-BPF_LOADER_ERRNO__COMPILE);
+		} else
+			pr_debug("bpf: builtin compiling successful\n");
 		obj = bpf_object__open_buffer(obj_buf, obj_buf_sz, filename);
 		free(obj_buf);
 	} else
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
index 22b3936..0eadd79 100644
--- a/tools/perf/util/c++/clang-c.h
+++ b/tools/perf/util/c++/clang-c.h
@@ -1,16 +1,42 @@
 #ifndef PERF_UTIL_CLANG_C_H
 #define PERF_UTIL_CLANG_C_H
 
+#include <stddef.h>	/* for size_t */
+#include <util-cxx.h>	/* for __maybe_unused */
+
 #ifdef __cplusplus
 extern "C" {
 #endif
 
+#ifdef HAVE_LIBCLANGLLVM_SUPPORT
 extern void perf_clang__init(void);
 extern void perf_clang__cleanup(void);
 
 extern int test__clang_to_IR(void);
 extern int test__clang_to_obj(void);
 
+extern int perf_clang__compile_bpf(const char *filename,
+				   void **p_obj_buf,
+				   size_t *p_obj_buf_sz);
+#else
+
+
+static inline void perf_clang__init(void) { }
+static inline void perf_clang__cleanup(void) { }
+
+static inline int test__clang_to_IR(void) { return -1; }
+static inline int test__clang_to_obj(void) { return -1;}
+
+static inline int
+perf_clang__compile_bpf(const char *filename __maybe_unused,
+			void **p_obj_buf __maybe_unused,
+			size_t *p_obj_buf_sz __maybe_unused)
+{
+	return -ENOTSUP;
+}
+
+#endif
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 38dcc14..010b91e 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -163,4 +163,33 @@ void perf_clang__cleanup(void)
 	perf::LLVMCtx.reset(nullptr);
 	llvm::llvm_shutdown();
 }
+
+int perf_clang__compile_bpf(const char *filename,
+			    void **p_obj_buf,
+			    size_t *p_obj_buf_sz)
+{
+	using namespace perf;
+
+	if (!p_obj_buf || !p_obj_buf_sz)
+		return -EINVAL;
+
+	llvm::opt::ArgStringList CFlags;
+	auto M = getModuleFromSource(std::move(CFlags), filename);
+	if (!M)
+		return  -EINVAL;
+	auto O = getBPFObjectFromModule(&*M);
+	if (!O)
+		return -EINVAL;
+
+	size_t size = O->size_in_bytes();
+	void *buffer;
+
+	buffer = malloc(size);
+	if (!buffer)
+		return -ENOMEM;
+	memcpy(buffer, O->data(), size);
+	*p_obj_buf = buffer;
+	*p_obj_buf_sz = size;
+	return 0;
+}
 }
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 13/14] perf clang: Pass fill path compiler
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (11 preceding siblings ...)
  2016-09-23 12:49 ` [PATCH 12/14] perf bpf: Compile BPF script use builtin cflags support Wang Nan
@ 2016-09-23 12:50 ` Wang Nan
  2016-09-23 12:50 ` [PATCH 14/14] perf clang: Pass correct CFLAGS to builtin clang Wang Nan
  2016-09-24 15:16 ` [PATCH 00/14] perf clang: Support compiling BPF script use " Alexei Starovoitov
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:50 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

If clang changes its working directory, filename passed to
perf_clang__compile_bpf() becomes invalid. Before running clang,
convert it to absolute path so file can be found even working directory
is different.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang.cpp | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 010b91e..812dd53 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -13,8 +13,10 @@
 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Tooling/Tooling.h"
 #include "clang/CodeGen/CodeGenAction.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Option/Option.h"
+#include "llvm/Support/FileSystem.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Target/TargetMachine.h"
@@ -164,7 +166,7 @@ void perf_clang__cleanup(void)
 	llvm::llvm_shutdown();
 }
 
-int perf_clang__compile_bpf(const char *filename,
+int perf_clang__compile_bpf(const char *_filename,
 			    void **p_obj_buf,
 			    size_t *p_obj_buf_sz)
 {
@@ -173,8 +175,11 @@ int perf_clang__compile_bpf(const char *filename,
 	if (!p_obj_buf || !p_obj_buf_sz)
 		return -EINVAL;
 
+	llvm::SmallString<PATH_MAX> FileName(_filename);
+	llvm::sys::fs::make_absolute(FileName);
+
 	llvm::opt::ArgStringList CFlags;
-	auto M = getModuleFromSource(std::move(CFlags), filename);
+	auto M = getModuleFromSource(std::move(CFlags), FileName.data());
 	if (!M)
 		return  -EINVAL;
 	auto O = getBPFObjectFromModule(&*M);
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH 14/14] perf clang: Pass correct CFLAGS to builtin clang
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (12 preceding siblings ...)
  2016-09-23 12:50 ` [PATCH 13/14] perf clang: Pass fill path compiler Wang Nan
@ 2016-09-23 12:50 ` Wang Nan
  2016-09-24 15:16 ` [PATCH 00/14] perf clang: Support compiling BPF script use " Alexei Starovoitov
  14 siblings, 0 replies; 20+ messages in thread
From: Wang Nan @ 2016-09-23 12:50 UTC (permalink / raw)
  To: acme, ast
  Cc: pi3orama, linux-kernel, lizefan, Wang Nan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

Pass CFLAGS detected by kbuild detector to builtin clang so BPF scripts
can use kernel headers like external clang compiler.

Test:
 # perf record --dry-run -e tools/perf/tests/bpf-script-test-kbuild.c ls 2>&1 | grep built
 bpf: builtin compiling successful

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/c++/clang.cpp    | 76 +++++++++++++++++++++++++++++++++++++---
 tools/perf/util/llvm-utils-cxx.h | 14 ++++++++
 tools/perf/util/llvm-utils.h     |  2 +-
 3 files changed, 87 insertions(+), 5 deletions(-)
 create mode 100644 tools/perf/util/llvm-utils-cxx.h

diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 812dd53..e8363b4 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -26,6 +26,8 @@
 
 #include "clang.h"
 #include "clang-c.h"
+#include "llvm-utils-cxx.h"
+#include "util-cxx.h"
 
 namespace perf {
 
@@ -152,6 +154,72 @@ getBPFObjectFromModule(llvm::Module *Module)
 	return std::move(Buffer);
 }
 
+class ClangOptions {
+	llvm::SmallString<PATH_MAX> FileName;
+	char *kbuild_dir;
+	char *kbuild_include_opts;
+	llvm::SmallString<64> kernel_ver_def;
+public:
+	ClangOptions(const char *filename) : FileName(filename),
+					     kbuild_dir(NULL),
+					     kbuild_include_opts(NULL),
+					     kernel_ver_def("")
+	{
+		llvm::sys::fs::make_absolute(FileName);
+
+		unsigned int kernel_ver;
+		if (!fetch_kernel_version(&kernel_ver, NULL, 0))
+			kernel_ver_def = "-DLINUX_VERSION_CODE=" + std::to_string(kernel_ver);
+
+		llvm__get_kbuild_opts(&kbuild_dir, &kbuild_include_opts);
+		if (!kbuild_dir || !kbuild_include_opts) {
+			free(kbuild_dir);
+			free(kbuild_include_opts);
+			kbuild_dir = kbuild_include_opts = NULL;
+		}
+	}
+
+	~ClangOptions()
+	{
+		free(kbuild_dir);
+		free(kbuild_include_opts);
+	}
+
+	void getCFlags(opt::ArgStringList &CFlags)
+	{
+		CFlags.push_back(kernel_ver_def.c_str());
+
+		if (!kbuild_dir || !kbuild_include_opts)
+			return;
+		CFlags.push_back("-working-directory");
+		CFlags.push_back(kbuild_dir);
+
+		SmallVector<StringRef, 0> Terms;
+		StringRef Opts(kbuild_include_opts);
+		Opts.split(Terms, ' ');
+
+		for (auto i = Terms.begin(); i != Terms.end(); i++)
+			kbuild_include_opts[i->end() - Opts.begin()] = '\0';
+
+		for (auto i = Terms.begin(); i != Terms.end(); i++) {
+			if (i->startswith("-I"))
+				CFlags.push_back(i->begin());
+			else if (*i == "-include") {
+				CFlags.push_back((i++)->begin());
+				/* Let clang report this error */
+				if (i == Terms.end())
+					break;
+				CFlags.push_back(i->begin());
+			}
+		}
+	}
+
+	const char *getFileName(void)
+	{
+		return FileName.c_str();
+	}
+};
+
 }
 
 extern "C" {
@@ -175,11 +243,11 @@ int perf_clang__compile_bpf(const char *_filename,
 	if (!p_obj_buf || !p_obj_buf_sz)
 		return -EINVAL;
 
-	llvm::SmallString<PATH_MAX> FileName(_filename);
-	llvm::sys::fs::make_absolute(FileName);
-
+	ClangOptions Opts(_filename);
 	llvm::opt::ArgStringList CFlags;
-	auto M = getModuleFromSource(std::move(CFlags), FileName.data());
+
+	Opts.getCFlags(CFlags);
+	auto M = getModuleFromSource(std::move(CFlags), Opts.getFileName());
 	if (!M)
 		return  -EINVAL;
 	auto O = getBPFObjectFromModule(&*M);
diff --git a/tools/perf/util/llvm-utils-cxx.h b/tools/perf/util/llvm-utils-cxx.h
new file mode 100644
index 0000000..1202230
--- /dev/null
+++ b/tools/perf/util/llvm-utils-cxx.h
@@ -0,0 +1,14 @@
+#ifndef PERF_UTIL_LLVM_UTILS_CXX_H
+#define PERF_UTIL_LLVM_UTILS_CXX_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "llvm-utils.h"
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/perf/util/llvm-utils.h b/tools/perf/util/llvm-utils.h
index d3f410d..f9f490b 100644
--- a/tools/perf/util/llvm-utils.h
+++ b/tools/perf/util/llvm-utils.h
@@ -5,7 +5,7 @@
 #ifndef __LLVM_UTILS_H
 #define __LLVM_UTILS_H
 
-#include "debug.h"
+#include <errno.h>
 
 struct llvm_param {
 	/* Path of clang executable */
-- 
1.8.3.4

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* Re: [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang
  2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
                   ` (13 preceding siblings ...)
  2016-09-23 12:50 ` [PATCH 14/14] perf clang: Pass correct CFLAGS to builtin clang Wang Nan
@ 2016-09-24 15:16 ` Alexei Starovoitov
  2016-09-26  1:49   ` Wangnan (F)
  14 siblings, 1 reply; 20+ messages in thread
From: Alexei Starovoitov @ 2016-09-24 15:16 UTC (permalink / raw)
  To: Wang Nan
  Cc: acme, ast, pi3orama, linux-kernel, lizefan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

On Fri, Sep 23, 2016 at 12:49:47PM +0000, Wang Nan wrote:
> This patch set is the first step to implement features I announced
> in LinuxCon NA 2016. See page 31 of:
> 
>  http://events.linuxfoundation.org/sites/events/files/slides/Performance%20Monitoring%20and%20Analysis%20Using%20perf%20and%20BPF_1.pdf
> 
> This patch set links LLVM and Clang libraries to perf, so perf
> is able to compile BPF script to BPF object on the fly.

Nice!
So single perf binary won't have llvm external dependency anymore
or both ways will be maintained?
The command line stays the same?
If I understand the patches correctly, this set is establishing
the basic functionality and more complex features coming?

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang
  2016-09-24 15:16 ` [PATCH 00/14] perf clang: Support compiling BPF script use " Alexei Starovoitov
@ 2016-09-26  1:49   ` Wangnan (F)
  2016-09-26 23:47     ` Alexei Starovoitov
  0 siblings, 1 reply; 20+ messages in thread
From: Wangnan (F) @ 2016-09-26  1:49 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: acme, ast, pi3orama, linux-kernel, lizefan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa



On 2016/9/24 23:16, Alexei Starovoitov wrote:
> On Fri, Sep 23, 2016 at 12:49:47PM +0000, Wang Nan wrote:
>> This patch set is the first step to implement features I announced
>> in LinuxCon NA 2016. See page 31 of:
>>
>>   http://events.linuxfoundation.org/sites/events/files/slides/Performance%20Monitoring%20and%20Analysis%20Using%20perf%20and%20BPF_1.pdf
>>
>> This patch set links LLVM and Clang libraries to perf, so perf
>> is able to compile BPF script to BPF object on the fly.
> Nice!
> So single perf binary won't have llvm external dependency anymore
> or both ways will be maintained?
> The command line stays the same?

Yes. This patch set doesn't change interface. It compiles BPF script
with builtin clang, and if it fail, fall back to external clang.

> If I understand the patches correctly, this set is establishing
> the basic functionality and more complex features coming?
>

Yes. Following steps are:

  1. Ease of use improvement: automatically include BPF functions
     declaration and macros.

  2. Perf's hook: compile part of BPF script into native code, run
     them in perf when something happen. Create a channel, coordinate
     BPF and native code use bpf-output event.

  3. Define a new language to support common profiling task. I'm not
     very clear what the new language should be. It may looks like lua,
     perf converts it to C then to LLVM IR with builtin clang.

Thank you.

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang
  2016-09-26  1:49   ` Wangnan (F)
@ 2016-09-26 23:47     ` Alexei Starovoitov
       [not found]       ` <CA+JHD93WWAnyPM8e2MGw4SoJ16f_j52qdWsS=MuEzBkzCdXYEw@mail.gmail.com>
  0 siblings, 1 reply; 20+ messages in thread
From: Alexei Starovoitov @ 2016-09-26 23:47 UTC (permalink / raw)
  To: Wangnan (F)
  Cc: acme, ast, pi3orama, linux-kernel, lizefan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

On Mon, Sep 26, 2016 at 09:49:30AM +0800, Wangnan (F) wrote:
> 
> 
> On 2016/9/24 23:16, Alexei Starovoitov wrote:
> >On Fri, Sep 23, 2016 at 12:49:47PM +0000, Wang Nan wrote:
> >>This patch set is the first step to implement features I announced
> >>in LinuxCon NA 2016. See page 31 of:
> >>
> >>  http://events.linuxfoundation.org/sites/events/files/slides/Performance%20Monitoring%20and%20Analysis%20Using%20perf%20and%20BPF_1.pdf
> >>
> >>This patch set links LLVM and Clang libraries to perf, so perf
> >>is able to compile BPF script to BPF object on the fly.
> >Nice!
> >So single perf binary won't have llvm external dependency anymore
> >or both ways will be maintained?
> >The command line stays the same?
> 
> Yes. This patch set doesn't change interface. It compiles BPF script
> with builtin clang, and if it fail, fall back to external clang.
> 
> >If I understand the patches correctly, this set is establishing
> >the basic functionality and more complex features coming?
> >
> 
> Yes. Following steps are:
> 
>  1. Ease of use improvement: automatically include BPF functions
>     declaration and macros.

+1

>  2. Perf's hook: compile part of BPF script into native code, run
>     them in perf when something happen. Create a channel, coordinate
>     BPF and native code use bpf-output event.

+1

>  3. Define a new language to support common profiling task. I'm not
>     very clear what the new language should be. It may looks like lua,
>     perf converts it to C then to LLVM IR with builtin clang.

Many tracing languages were invented in the past.
At this point I'm not sure what exactly new language will solve.
To make it easier to write bpf programs?
I think it will be more fruitful to tweak clang/llvm to add
good warnings/errors for cases where we know that C is not going
be compiled into the code that the kernel verifier will accept.
Like we can complain about loops, unitialized variables,
non-inlined and unkown helper functions... all from clang/llvm.
imo that would be the better path forward and will help
both tracing and networking users that write in this restricted C.

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang
       [not found]       ` <CA+JHD93WWAnyPM8e2MGw4SoJ16f_j52qdWsS=MuEzBkzCdXYEw@mail.gmail.com>
@ 2016-09-27  1:49         ` Wangnan (F)
  0 siblings, 0 replies; 20+ messages in thread
From: Wangnan (F) @ 2016-09-27  1:49 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo, Alexei Starovoitov
  Cc: Li Zefan, He Kuang, pi3orama, linux-kernel, Jiri Olsa,
	Arnaldo Carvalho de Melo, ast, acme



On 2016/9/27 7:58, Arnaldo Carvalho de Melo wrote:
>
> Le 26 sept. 2016 8:47 PM, "Alexei Starovoitov" 
> <alexei.starovoitov@gmail.com <mailto:alexei.starovoitov@gmail.com>> a 
> écrit :
> >
> > On Mon, Sep 26, 2016 at 09:49:30AM +0800, Wangnan (F) wrote:
> > >
> > >
> > > On 2016/9/24 23:16, Alexei Starovoitov wrote:
> > > >On Fri, Sep 23, 2016 at 12:49:47PM +0000, Wang Nan wrote:
> > > >>This patch set is the first step to implement features I announced
> > > >>in LinuxCon NA 2016. See page 31 of:
> > > >>
> > > >> 
> http://events.linuxfoundation.org/sites/events/files/slides/Performance%20Monitoring%20and%20Analysis%20Using%20perf%20and%20BPF_1.pdf
> > > >>
> > > >>This patch set links LLVM and Clang libraries to perf, so perf
> > > >>is able to compile BPF script to BPF object on the fly.
> > > >Nice!
> > > >So single perf binary won't have llvm external dependency anymore
> > > >or both ways will be maintained?
> > > >The command line stays the same?
> > >
> > > Yes. This patch set doesn't change interface. It compiles BPF script
> > > with builtin clang, and if it fail, fall back to external clang.
> > >
> > > >If I understand the patches correctly, this set is establishing
> > > >the basic functionality and more complex features coming?
> > > >
> > >
> > > Yes. Following steps are:
> > >
> > >  1. Ease of use improvement: automatically include BPF functions
> > >     declaration and macros.
> >
> > +1
> >
> > >  2. Perf's hook: compile part of BPF script into native code, run
> > >     them in perf when something happen. Create a channel, coordinate
> > >     BPF and native code use bpf-output event.
> >
> > +1
> >
> > >  3. Define a new language to support common profiling task. I'm not
> > >     very clear what the new language should be. It may looks like lua,
> > >     perf converts it to C then to LLVM IR with builtin clang.
> >
> > Many tracing languages were invented in the past.
> > At this point I'm not sure what exactly new language will solve.
> > To make it easier to write bpf programs?
> > I think it will be more fruitful to tweak clang/llvm to add
> > good warnings/errors for cases where we know that C is not going
> > be compiled into the code that the kernel verifier will accept.
> > Like we can complain about loops, unitialized variables,
> > non-inlined and unkown helper functions... all from clang/llvm.
> > imo that would be the better path forward and will help
> > both tracing and networking users that write in this restricted C.
>
> ++1
> >
>

OK. Now let's focus on the first two goals. After that let's consider
how to help writing BPF program.

Thank you.

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH 01/14] tools build: Support compiling C++ source file
  2016-09-23 12:49 ` [PATCH 01/14] tools build: Support compiling C++ source file Wang Nan
@ 2016-09-28 14:33   ` Jiri Olsa
  0 siblings, 0 replies; 20+ messages in thread
From: Jiri Olsa @ 2016-09-28 14:33 UTC (permalink / raw)
  To: Wang Nan
  Cc: acme, ast, pi3orama, linux-kernel, lizefan,
	Arnaldo Carvalho de Melo, He Kuang, Jiri Olsa

On Fri, Sep 23, 2016 at 12:49:48PM +0000, Wang Nan wrote:
> Add new rule to compile .cpp file to .o use g++. C++ support is
> required for built-in clang and LLVM support.
> 
> Linker side support will be introduced by following commits.
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Alexei Starovoitov <ast@fb.com>
> Cc: He Kuang <hekuang@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>

patches 1 to 5:

Acked-by: Jiri Olsa <jolsa@kernel.org>

thanks,
jirka

^ permalink raw reply	[flat|nested] 20+ messages in thread
end of thread, other threads:[~2016-09-28 14:33 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-23 12:49 [PATCH 00/14] perf clang: Support compiling BPF script use builtin clang Wang Nan
2016-09-23 12:49 ` [PATCH 01/14] tools build: Support compiling C++ source file Wang Nan
2016-09-28 14:33   ` Jiri Olsa
2016-09-23 12:49 ` [PATCH 02/14] perf tools: Add feature detection for g++ Wang Nan
2016-09-23 12:49 ` [PATCH 03/14] perf tools: Add feature detection for LLVM Wang Nan
2016-09-23 12:49 ` [PATCH 04/14] perf tools: Add feature detection for clang Wang Nan
2016-09-23 12:49 ` [PATCH 05/14] perf build: Add clang and llvm compile and linking support Wang Nan
2016-09-23 12:49 ` [PATCH 06/14] perf clang: Add builtin clang support ant test case Wang Nan
2016-09-23 12:49 ` [PATCH 07/14] perf clang: Use real file system for #include Wang Nan
2016-09-23 12:49 ` [PATCH 08/14] perf clang: Allow passing CFLAGS to builtin clang Wang Nan
2016-09-23 12:49 ` [PATCH 09/14] perf clang: Update test case to use real BPF script Wang Nan
2016-09-23 12:49 ` [PATCH 10/14] perf clang: Support compile IR to BPF object and add testcase Wang Nan
2016-09-23 12:49 ` [PATCH 11/14] perf tools: Extract kernel build option detector as utils Wang Nan
2016-09-23 12:49 ` [PATCH 12/14] perf bpf: Compile BPF script use builtin cflags support Wang Nan
2016-09-23 12:50 ` [PATCH 13/14] perf clang: Pass fill path compiler Wang Nan
2016-09-23 12:50 ` [PATCH 14/14] perf clang: Pass correct CFLAGS to builtin clang Wang Nan
2016-09-24 15:16 ` [PATCH 00/14] perf clang: Support compiling BPF script use " Alexei Starovoitov
2016-09-26  1:49   ` Wangnan (F)
2016-09-26 23:47     ` Alexei Starovoitov
     [not found]       ` <CA+JHD93WWAnyPM8e2MGw4SoJ16f_j52qdWsS=MuEzBkzCdXYEw@mail.gmail.com>
2016-09-27  1:49         ` Wangnan (F)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).