linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Wang Nan <wangnan0@huawei.com>
To: <acme@redhat.com>, <ast@fb.com>
Cc: <lizefan@huawei.com>, <hekuang@huawei.com>,
	<linux-kernel@vger.kernel.org>, <pi3orama@163.com>,
	Wang Nan <wangnan0@huawei.com>, Jiri Olsa <jolsa@kernel.org>
Subject: [PATCH 24/34] perf clang jit: add PerfModule::doJIT to JIT perfhook functions
Date: Tue, 15 Nov 2016 04:06:07 +0000	[thread overview]
Message-ID: <20161115040617.69788-25-wangnan0@huawei.com> (raw)
In-Reply-To: <20161115040617.69788-1-wangnan0@huawei.com>

PerfModule::doJIT JIT compile perfhook functions and saves result into
a map. Add a test case for it.

At this stage perfhook functions can do no useful things because they
can't invoke external functions and can't return value. Following
commits are going to make improvment.

Don't hook functions right after jitted because bpf_object is unavailable
during jitting but it should be the context of jitted functions.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
---
 tools/perf/tests/bpf-script-example.c |  8 ++++
 tools/perf/tests/clang.c              |  4 ++
 tools/perf/util/c++/clang-c.h         |  2 +
 tools/perf/util/c++/clang-test.cpp    | 32 +++++++++++++++-
 tools/perf/util/c++/clang.cpp         | 71 +++++++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h           | 13 +++++++
 6 files changed, 128 insertions(+), 2 deletions(-)

diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
index 268e5f8..265036e 100644
--- a/tools/perf/tests/bpf-script-example.c
+++ b/tools/perf/tests/bpf-script-example.c
@@ -46,3 +46,11 @@ int bpf_func__SyS_epoll_wait(void *ctx)
 }
 char _license[] SEC("license") = "GPL";
 int _version SEC("version") = LINUX_VERSION_CODE;
+
+#ifdef TEST_PERF_HOOK
+SEC("perfhook:test")
+void hook_test(void)
+{
+	return;
+}
+#endif
diff --git a/tools/perf/tests/clang.c b/tools/perf/tests/clang.c
index 2964c06..f274e62 100644
--- a/tools/perf/tests/clang.c
+++ b/tools/perf/tests/clang.c
@@ -16,6 +16,10 @@ static struct {
 		.func = test__clang_to_obj,
 		.desc = "Test builtin clang compile C source to ELF object",
 	},
+	{
+		.func = test__clang_jit,
+		.desc = "Test builtin clang compile mixed BPF and native code",
+	},
 #endif
 };
 
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
index 0eadd79..5ebcb41 100644
--- a/tools/perf/util/c++/clang-c.h
+++ b/tools/perf/util/c++/clang-c.h
@@ -14,6 +14,7 @@ extern void perf_clang__cleanup(void);
 
 extern int test__clang_to_IR(void);
 extern int test__clang_to_obj(void);
+extern int test__clang_jit(void);
 
 extern int perf_clang__compile_bpf(const char *filename,
 				   void **p_obj_buf,
@@ -26,6 +27,7 @@ static inline void perf_clang__cleanup(void) { }
 
 static inline int test__clang_to_IR(void) { return -1; }
 static inline int test__clang_to_obj(void) { return -1;}
+static inline int test__clang_jit(void) { return -1;}
 
 static inline int
 perf_clang__compile_bpf(const char *filename __maybe_unused,
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
index fb05e56..2b4aa8d 100644
--- a/tools/perf/util/c++/clang-test.cpp
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -5,6 +5,7 @@
 
 #include <util-cxx.h>
 #include <tests/llvm.h>
+#include <perf-hooks.h>
 #include <string>
 
 class perf_clang_scope {
@@ -14,7 +15,7 @@ public:
 };
 
 static std::unique_ptr<perf::PerfModule>
-__test__clang_to_IR(void)
+__test__clang_to_IR(bool perfhook)
 {
 	unsigned int kernel_version;
 
@@ -23,14 +24,22 @@ __test__clang_to_IR(void)
 
 	std::string cflag_kver("-DLINUX_VERSION_CODE=" +
 				std::to_string(kernel_version));
+	std::string cflag_perfhook(perfhook ? "-DTEST_PERF_HOOK=1" : "");
 
 	std::unique_ptr<perf::PerfModule> M =
-		perf::getModuleFromSource({cflag_kver.c_str()},
+		perf::getModuleFromSource({cflag_kver.c_str(),
+					   cflag_perfhook.c_str()},
 					  "perf-test.c",
 					  test_llvm__bpf_base_prog);
 	return M;
 }
 
+static std::unique_ptr<perf::PerfModule>
+__test__clang_to_IR(void)
+{
+	return __test__clang_to_IR(false);
+}
+
 extern "C" {
 int test__clang_to_IR(void)
 {
@@ -59,4 +68,23 @@ int test__clang_to_obj(void)
 	return 0;
 }
 
+int test__clang_jit(void)
+{
+	perf_clang_scope _scope;
+
+	auto M = __test__clang_to_IR(true);
+	if (!M)
+		return -1;
+
+	if (M->doJIT())
+		return -1;
+
+	std::unique_ptr<perf::PerfModule::HookMap> hooks(M->copyJITResult());
+	for (auto i : *hooks)
+		perf_hooks__set_hook(i.first.c_str(), i.second, NULL);
+
+	perf_hooks__invoke_test();
+	return 0;
+}
+
 }
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
index 98d05e2..03012b2 100644
--- a/tools/perf/util/c++/clang.cpp
+++ b/tools/perf/util/c++/clang.cpp
@@ -14,9 +14,14 @@
 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Tooling/Tooling.h"
 #include "llvm/ADT/SmallString.h"
+#include "llvm/ExecutionEngine/Orc/CompileUtils.h"
+#include "llvm/ExecutionEngine/Orc/IRCompileLayer.h"
+#include "llvm/ExecutionEngine/Orc/LambdaResolver.h"
+#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Module.h"
 #include "llvm/Option/Option.h"
+#include "llvm/Support/DynamicLibrary.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/TargetRegistry.h"
@@ -24,11 +29,13 @@
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Target/TargetOptions.h"
 #include <memory>
+#include <vector>
 
 #include "clang.h"
 #include "clang-c.h"
 #include "llvm-utils.h"
 #include "util-cxx.h"
+#include "perf-hooks.h"
 
 namespace perf {
 
@@ -187,6 +194,66 @@ PerfModule::toBPFObject(void)
 	return std::move(Buffer);
 }
 
+/*
+ * Use a global memory manager so allocated code and data won't be released
+ * when object destroy.
+ */
+static llvm::SectionMemoryManager JITMemoryManager;
+
+int PerfModule::doJIT(void)
+{
+	using namespace orc;
+
+	prepareJIT();
+
+	std::unique_ptr<TargetMachine> TM(EngineBuilder().selectTarget());
+	if (!TM) {
+		llvm::errs() << "Can't get target machine\n";
+		return -1;
+	}
+	const DataLayout DL(TM->createDataLayout());
+	Module->setDataLayout(DL);
+	Module->setTargetTriple(TM->getTargetTriple().normalize());
+
+	ObjectLinkingLayer<> ObjectLayer;
+	IRCompileLayer<decltype(ObjectLayer)> CompileLayer(ObjectLayer, SimpleCompiler(*TM));
+
+	auto Resolver = createLambdaResolver(
+			[](const std::string &Name) {
+				return RuntimeDyld::SymbolInfo(nullptr);
+			},
+			[](const std::string &Name) {
+				return RuntimeDyld::SymbolInfo(nullptr);
+			});
+
+	std::vector<llvm::Module *> Ms;
+	Ms.push_back(getModule());
+	CompileLayer.addModuleSet(std::move(Ms),
+			&JITMemoryManager,
+			std::move(Resolver));
+
+
+	for (Function *F : JITFunctions) {
+		JITSymbol sym = CompileLayer.findSymbol(F->getName().str(), true);
+
+		/*
+		 * Type of F->getSection() is moving from
+		 * const char * to StringRef.
+		 * Convert it to std::string so we don't need
+		 * consider this API change.
+		 */
+		std::string sec(F->getSection());
+		std::string hook(&sec.c_str()[sizeof("perfhook:") - 1]);
+		perf_hook_func_t func = (perf_hook_func_t)(intptr_t)sym.getAddress();
+
+		if (JITResult[hook])
+			llvm::errs() << "Warning: multiple functions on hook "
+				     << hook << ", only one is used\n";
+		JITResult[hook] = func;
+	}
+	return 0;
+}
+
 class ClangOptions {
 	llvm::SmallString<PATH_MAX> FileName;
 	llvm::SmallString<64> KVerDef;
@@ -292,6 +359,10 @@ void perf_clang__init(void)
 	LLVMInitializeBPFTarget();
 	LLVMInitializeBPFTargetMC();
 	LLVMInitializeBPFAsmPrinter();
+
+	llvm::InitializeNativeTarget();
+	llvm::InitializeNativeTargetAsmPrinter();
+	llvm::InitializeNativeTargetAsmParser();
 }
 
 void perf_clang__cleanup(void)
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
index 1eb71a6..df2eb8f 100644
--- a/tools/perf/util/c++/clang.h
+++ b/tools/perf/util/c++/clang.h
@@ -7,18 +7,26 @@
 #include "llvm/Option/Option.h"
 #include <memory>
 #include <set>
+#include <map>
+
+#include "util/perf-hooks.h"
 
 namespace perf {
 
 using namespace llvm;
 
 class PerfModule {
+public:
+	typedef std::map<std::string, perf_hook_func_t> HookMap;
 private:
 	std::unique_ptr<llvm::Module> Module;
 
 	std::set<llvm::GlobalVariable *> Maps;
 	std::set<llvm::Function *> BPFFunctions;
 	std::set<llvm::Function *> JITFunctions;
+
+	HookMap JITResult;
+
 	void prepareBPF(void);
 	void prepareJIT(void);
 public:
@@ -26,10 +34,15 @@ class PerfModule {
 	{
 		return Module.get();
 	}
+	inline HookMap *copyJITResult(void)
+	{
+		return new HookMap(JITResult);
+	}
 
 	PerfModule(std::unique_ptr<llvm::Module>&& M);
 
 	std::unique_ptr<llvm::SmallVectorImpl<char>> toBPFObject(void);
+	int doJIT(void);
 };
 
 std::unique_ptr<PerfModule>
-- 
2.10.1

  parent reply	other threads:[~2016-11-15  4:11 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-11-15  4:05 [PATCH 00/34] perf clang: Builtin clang and perfhook support Wang Nan
2016-11-15  4:05 ` [PATCH 01/34] perf tools: Fix kernel version error in ubuntu Wang Nan
2016-11-25 17:20   ` [tip:perf/core] " tip-bot for Wang Nan
2016-11-15  4:05 ` [PATCH 02/34] perf record: Fix segfault when running with suid and kptr_restrict is 1 Wang Nan
2016-11-25 17:21   ` [tip:perf/core] " tip-bot for Wang Nan
2016-11-15  4:05 ` [PATCH 03/34] tools perf: Add missing struct defeinition in probe_event.h Wang Nan
2016-11-25 17:21   ` [tip:perf/core] perf tools: Add missing struct definition " tip-bot for Wang Nan
2016-11-15  4:05 ` [PATCH 04/34] tools lib bpf: fix maps resolution Wang Nan
2016-11-25 17:22   ` [tip:perf/core] tools lib bpf: Fix " tip-bot for Eric Leblond
2016-11-15  4:05 ` [PATCH 05/34] tools lib bpf: Add missing bpf map functions Wang Nan
2016-11-17  3:23   ` Wangnan (F)
2016-11-25 14:31     ` Arnaldo Carvalho de Melo
2016-11-15  4:05 ` [PATCH 06/34] tools lib bpf: Add private field for bpf_object Wang Nan
2016-11-15  4:05 ` [PATCH 07/34] tools lib bpf: Retrive bpf_map through offset of bpf_map_def Wang Nan
2016-11-15  4:05 ` [PATCH 08/34] perf tools: Introduce perf hooks Wang Nan
2016-11-15  4:05 ` [PATCH 09/34] perf tools: Pass context to perf hook functions Wang Nan
2016-11-15  4:05 ` [PATCH 10/34] perf llvm: Extract helpers in llvm-utils.c Wang Nan
2016-11-15  4:05 ` [PATCH 11/34] tools build: Add feature detection for LLVM Wang Nan
2016-11-15  4:05 ` [PATCH 12/34] tools build: Add feature detection for clang Wang Nan
2016-11-15  4:05 ` [PATCH 13/34] perf build: Add clang and llvm compile and linking support Wang Nan
2016-11-15  4:05 ` [PATCH 14/34] perf clang: Add builtin clang support ant test case Wang Nan
2016-11-15  4:05 ` [PATCH 15/34] perf clang: Use real file system for #include Wang Nan
2016-11-15  4:05 ` [PATCH 16/34] perf clang: Allow passing CFLAGS to builtin clang Wang Nan
2016-11-15  4:06 ` [PATCH 17/34] perf clang: Update test case to use real BPF script Wang Nan
2016-11-15  4:06 ` [PATCH 18/34] perf clang: Support compile IR to BPF object and add testcase Wang Nan
2016-11-15  4:06 ` [PATCH 19/34] perf clang: Compile BPF script use builtin clang support Wang Nan
2016-11-15  4:06 ` [PATCH 20/34] perf clang: Pass full path to builtin clang Wang Nan
2016-11-15  4:06 ` [PATCH 21/34] perf clang: Pass CFLAGS " Wang Nan
2016-11-15  4:06 ` [PATCH 22/34] perf clang jit: Wrap llvm::Module using PerfModule Wang Nan
2016-11-15  4:06 ` [PATCH 23/34] perf clang jit: Insignt BPF and JIT functions in a Module Wang Nan
2016-11-15  4:06 ` Wang Nan [this message]
2016-11-15  4:06 ` [PATCH 25/34] perf clang jit: Export functions for jitted code Wang Nan
2016-11-15  4:06 ` [PATCH 26/34] perf clang jit: Actually JIT and hook in bpf loader Wang Nan
2016-11-15  4:06 ` [PATCH 27/34] perf clang jit: Collect the lowest address in maps section as map_base Wang Nan
2016-11-15  4:06 ` [PATCH 28/34] perf clang jit: Access BPF map Wang Nan
2016-11-15  4:06 ` [PATCH 29/34] perf clang jit: Allow jitted perf hook access BPF maps Wang Nan
2016-11-15  4:06 ` [PATCH 30/34] perf clang: Link BPF functions declaration into perf Wang Nan
2016-11-15  4:06 ` [PATCH 31/34] perf clang: Declare BPF functions for BPF scripts automatically Wang Nan
2016-11-15  4:06 ` [PATCH 32/34] perf clang: Include helpers to BPF scripts Wang Nan
2016-11-15  4:06 ` [PATCH 33/34] perf clang builtin: Define hook helpers by default Wang Nan
2016-11-15  4:06 ` [PATCH 34/34] perf clang git: Export getpid() to perf hook Wang Nan
2016-11-15  4:32 ` [PATCH 00/34] perf clang: Builtin clang and perfhook support Wangnan (F)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20161115040617.69788-25-wangnan0@huawei.com \
    --to=wangnan0@huawei.com \
    --cc=acme@redhat.com \
    --cc=ast@fb.com \
    --cc=hekuang@huawei.com \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan@huawei.com \
    --cc=pi3orama@163.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).