Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Alexei Starovoitov]

On Tue, Jan 17, 2017 at 5:38 AM, Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> [acme@jouet linux]$ ls -lah /tmp/perf
> -rwxr-xr-x. 1 acme acme 4.4M Jan 17 10:29 /tmp/perf
> [acme@jouet linux]$ size /tmp/perf
>    text    data     bss     dec     hex filename
> 3954488  622440 23912104        28489032        1b2b548 /tmp/perf
> [acme@jouet linux]$ ldd /tmp/perf | egrep -i llvm\|clang
>         libclangAST.so.40 => /usr/local/lib/libclangAST.so.40 (0x00007f4964981000)
...
>         libLLVMSystemZDisassembler.so.40 => /usr/local/lib/libLLVMSystemZDisassembler.so.40 (0x00007f495fe70000)
...
>         libLLVMSparcDisassembler.so.40 => /usr/local/lib/libLLVMSparcDisassembler.so.40 (0x00007f495f14f000)
...
>         libLLVMPowerPCDisassembler.so.40 => /usr/local/lib/libLLVMPowerPCDisassembler.so.40 (0x00007f495de94000)
...
>         libLLVMMSP430CodeGen.so.40 => /usr/local/lib/libLLVMMSP430CodeGen.so.40 (0x00007f495c79d000)
...
>         libLLVMMipsDisassembler.so.40 => /usr/local/lib/libLLVMMipsDisassembler.so.40 (0x00007f495bf73000)

I think the static binary shouldn't be including all llvm backends.
It's not clear to me why .so pulls them all.
fyi I typically configure llvm with:
-DLLVM_TARGETS_TO_BUILD="X86;BPF"

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Tue, Jan 17, 2017 at 10:27:39AM -0800, Alexei Starovoitov escreveu:
> On Tue, Jan 17, 2017 at 5:38 AM, Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> > [acme@jouet linux]$ ls -lah /tmp/perf
> > -rwxr-xr-x. 1 acme acme 4.4M Jan 17 10:29 /tmp/perf
> > [acme@jouet linux]$ size /tmp/perf
> >    text    data     bss     dec     hex filename
> > 3954488  622440 23912104        28489032        1b2b548 /tmp/perf
> > [acme@jouet linux]$ ldd /tmp/perf | egrep -i llvm\|clang
> >         libclangAST.so.40 => /usr/local/lib/libclangAST.so.40 (0x00007f4964981000)
> ...
> >         libLLVMSystemZDisassembler.so.40 => /usr/local/lib/libLLVMSystemZDisassembler.so.40 (0x00007f495fe70000)
> ...
> >         libLLVMSparcDisassembler.so.40 => /usr/local/lib/libLLVMSparcDisassembler.so.40 (0x00007f495f14f000)
> ...
> >         libLLVMPowerPCDisassembler.so.40 => /usr/local/lib/libLLVMPowerPCDisassembler.so.40 (0x00007f495de94000)
> ...
> >         libLLVMMSP430CodeGen.so.40 => /usr/local/lib/libLLVMMSP430CodeGen.so.40 (0x00007f495c79d000)
> ...
> >         libLLVMMipsDisassembler.so.40 => /usr/local/lib/libLLVMMipsDisassembler.so.40 (0x00007f495bf73000)
 
> I think the static binary shouldn't be including all llvm backends.
> It's not clear to me why .so pulls them all.
> fyi I typically configure llvm with:
> -DLLVM_TARGETS_TO_BUILD="X86;BPF"

Humm, so this is a limitation of the LLVM libraries? I.e. people linking
against it should ask for a subset of backends so that would be dlopened
out of a larger set builtin, say, in a generic distro shipped llvm/clang
package?

- Arnaldo

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Alexei Starovoitov]

On Tue, Dec 6, 2016 at 1:02 PM, Arnaldo Carvalho de Melo
<arnaldo.melo@gmail.com> wrote:
 [acme@jouet linux]$ ls -lah /tmp/perf
> -rwxr-xr-x. 1 acme acme 41M Dec  6 17:58 /tmp/perf

awesome. that's reasonable and matches to
what I see with libbcc.so
We had few tricks before to reduce it into <30M range,
but they were too fragile.

[PATCH v3 00/30] perf clang: Builtin clang and perfhook support

[Wang Nan]

This is version 3 of perf builtin clang and perfhook patch series.
Compare to v2 there is only minor changes:
 1. BPF map helpers in perf hooks now called 'perf_map_...',
    instead of 'jit_helper_map_...'.
    (Alexei Starovoitov)
 2. Rename bpf_map_{pin,get} to bpf_obj_{pin,get}, make them consist
    with kernel.
    (Joe Stringer).

Example in v2 should be changed accordingly:

  $ cat ./count_syscalls.c
  typedef unsigned long u64;
  
  #define BPF_MAP_TYPE_HASH 1
  #define BPF_MAP_TYPE_ARRAY 2
  
  enum GVAL {
      G_perf_pid,
      NR_GVALS
  };
  
  struct bpf_map_def SEC("maps") GVALS = {
      .type = BPF_MAP_TYPE_ARRAY,
      .key_size = sizeof(int),
      .value_size = sizeof(u64),
      .max_entries = NR_GVALS,
  };
  
  struct bpf_map_def SEC("maps") syscall_counter = {
      .type = BPF_MAP_TYPE_HASH,
      .key_size = sizeof(u64),
      .value_size = sizeof(u64),
      .max_entries = 512,
  };
  
  SEC("raw_syscalls:sys_enter")
  int func(void *ctx)
  {
      int key = G_perf_pid;
      u64 id = *((u64 *)(ctx + 8));
      int self_pid = bpf_get_current_pid_tgid() & 0xffffffff;
      int *perf_pid = bpf_map_lookup_elem(&GVALS, &key);
      u64 *counter;
  
      if (!perf_pid)
          return 0;
      if (*perf_pid == self_pid)
          return 0;
      counter = bpf_map_lookup_elem(&syscall_counter, &id);
      if (!counter) {
          u64 value = 1;
          bpf_map_update_elem(&syscall_counter, &id, &value, 0);
          return 0;
      }
      __sync_fetch_and_add(counter, 1);
      return 0;
  }
  
  SEC("perfhook:record_start")
  void record_start(void *ctx)
  {
      int perf_pid = getpid(), key = G_perf_pid;
      printf("Start count, perfpid=%d\n", perf_pid);
      perf_map_update_elem(ctx, &GVALS, &key, &perf_pid, 0);
  }
  
  SEC("perfhook:record_end")
  void record_end(void *ctx)
  {
      u64 key = -1, value;
      while (!perf_map_get_next_key(ctx, &syscall_counter, &key, &key)) {
          perf_map_lookup_elem(ctx, &syscall_counter, &key, &value);
          printf("syscall %ld\tcount: %ld\n", (long)key, (long)value);
      }
  }
  char _license[] SEC("license") = "GPL";
  int _version SEC("version") = LINUX_VERSION_CODE;
  $ sudo -s
  # ulimit -l unlimited
  # perf record -e ./count_syscalls.c echo "Haha"
  Start count, perfpid=25209
  Haha
  [ perf record: Woken up 1 times to write data ]
  syscall 293	count: 2
  syscall 8	count: 7
  syscall 11	count: 763
  syscall 4	count: 43
  syscall 21	count: 48
  syscall 86	count: 1
  syscall 5	count: 791
  ...

Wang Nan (30):
  tools lib bpf: Add missing BPF functions
  tools lib bpf: Add private field for bpf_object
  tools lib bpf: Retrive bpf_map through offset of bpf_map_def
  perf tools: Introduce perf hooks
  perf tools: Pass context to perf hook functions
  perf llvm: Extract helpers in llvm-utils.c
  tools build: Add feature detection for LLVM
  tools build: Add feature detection for clang
  perf build: Add clang and llvm compile and linking support
  perf clang: Add builtin clang support ant test case
  perf clang: Use real file system for #include
  perf clang: Allow passing CFLAGS to builtin clang
  perf clang: Update test case to use real BPF script
  perf clang: Support compile IR to BPF object and add testcase
  perf clang: Compile BPF script use builtin clang support
  perf clang: Pass full path to builtin clang
  perf clang: Pass CFLAGS to builtin clang
  perf clang jit: Wrap llvm::Module using PerfModule
  perf clang jit: Insignt BPF and JIT functions in a Module
  perf clang jit: add PerfModule::doJIT to JIT perfhook functions
  perf clang jit: Export functions for jitted code
  perf clang jit: Actually JIT and hook in bpf loader
  perf clang jit: Collect the lowest address in maps section as map_base
  perf clang jit: Retrive fd of BPF map from its offset
  perf clang jit: Allow jitted perf hook access BPF maps
  perf clang: Link BPF functions declaration into perf
  perf clang: Declare BPF functions for BPF scripts automatically
  perf clang: Include helpers to BPF scripts
  perf clang builtin: Define hook helpers by default
  perf clang jit: Export getpid() to perf hook

 tools/build/feature/Makefile                  |  18 +
 tools/build/feature/test-clang.cpp            |  21 ++
 tools/build/feature/test-llvm.cpp             |   8 +
 tools/lib/bpf/bpf.c                           |  56 +++
 tools/lib/bpf/bpf.h                           |   7 +
 tools/lib/bpf/libbpf.c                        |  35 ++
 tools/lib/bpf/libbpf.h                        |  13 +
 tools/perf/Makefile.config                    |  62 +++-
 tools/perf/Makefile.perf                      |  23 +-
 tools/perf/builtin-record.c                   |  11 +
 tools/perf/tests/Build                        |   4 +-
 tools/perf/tests/bpf-script-example.c         |  30 +-
 tools/perf/tests/bpf-script-test-kbuild.c     |   2 +
 tools/perf/tests/bpf-script-test-prologue.c   |   6 +-
 tools/perf/tests/bpf-script-test-relocation.c |  17 +-
 tools/perf/tests/builtin-test.c               |  13 +
 tools/perf/tests/clang.c                      |  50 +++
 tools/perf/tests/llvm.h                       |   7 +
 tools/perf/tests/make                         |   2 +
 tools/perf/tests/perf-hooks.c                 |  48 +++
 tools/perf/tests/tests.h                      |   4 +
 tools/perf/util/Build                         |   5 +
 tools/perf/util/bpf-loader.c                  | 102 +++++-
 tools/perf/util/bpf-loader.h                  |  19 +
 tools/perf/util/c++/Build                     |   4 +
 tools/perf/util/c++/bpf-funcs-str.c           | 228 ++++++++++++
 tools/perf/util/c++/bpf-helper-str.c          |  23 ++
 tools/perf/util/c++/clang-bpf-includes.h      |  13 +
 tools/perf/util/c++/clang-c.h                 |  63 ++++
 tools/perf/util/c++/clang-test.cpp            |  99 +++++
 tools/perf/util/c++/clang.cpp                 | 497 ++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h                   |  61 ++++
 tools/perf/util/jit-helpers.c                 |  57 +++
 tools/perf/util/jit-helpers.h                 |  28 ++
 tools/perf/util/llvm-utils.c                  |  76 +++-
 tools/perf/util/llvm-utils.h                  |  15 +-
 tools/perf/util/perf-hooks-list.h             |   3 +
 tools/perf/util/perf-hooks.c                  |  88 +++++
 tools/perf/util/perf-hooks.h                  |  39 ++
 tools/perf/util/util-cxx.h                    |  26 ++
 40 files changed, 1830 insertions(+), 53 deletions(-)
 create mode 100644 tools/build/feature/test-clang.cpp
 create mode 100644 tools/build/feature/test-llvm.cpp
 create mode 100644 tools/perf/tests/clang.c
 create mode 100644 tools/perf/tests/perf-hooks.c
 create mode 100644 tools/perf/util/c++/Build
 create mode 100644 tools/perf/util/c++/bpf-funcs-str.c
 create mode 100644 tools/perf/util/c++/bpf-helper-str.c
 create mode 100644 tools/perf/util/c++/clang-bpf-includes.h
 create mode 100644 tools/perf/util/c++/clang-c.h
 create mode 100644 tools/perf/util/c++/clang-test.cpp
 create mode 100644 tools/perf/util/c++/clang.cpp
 create mode 100644 tools/perf/util/c++/clang.h
 create mode 100644 tools/perf/util/jit-helpers.c
 create mode 100644 tools/perf/util/jit-helpers.h
 create mode 100644 tools/perf/util/perf-hooks-list.h
 create mode 100644 tools/perf/util/perf-hooks.c
 create mode 100644 tools/perf/util/perf-hooks.h
 create mode 100644 tools/perf/util/util-cxx.h

Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Joe Stringer <joe@ovn.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com

-- 
2.10.1

[PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Wang Nan]

Add basic clang support in clang.cpp and test__clang() testcase. The
first testcase checks if builtin clang is able to generate LLVM IR.

tests/clang.c is a proxy. Real testcase resides in
utils/c++/clang-test.cpp in c++ and exports C interface to perf test
subsystem.

Test result:

   $ perf test -v clang
   51: Test builtin clang support                               :
   51.1: Test builtin clang compile C source to IR              :
   --- start ---
   test child forked, pid 13215
   test child finished with 0
   ---- end ----
   Test builtin clang support subtest 0: Ok

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
---
 tools/perf/tests/Build             |  1 +
 tools/perf/tests/builtin-test.c    |  9 ++++
 tools/perf/tests/clang.c           | 42 +++++++++++++++++
 tools/perf/tests/tests.h           |  3 ++
 tools/perf/util/Build              |  2 +
 tools/perf/util/c++/Build          |  2 +
 tools/perf/util/c++/clang-c.h      | 16 +++++++
 tools/perf/util/c++/clang-test.cpp | 31 ++++++++++++
 tools/perf/util/c++/clang.cpp      | 96 ++++++++++++++++++++++++++++++++++++++
 tools/perf/util/c++/clang.h        | 16 +++++++
 10 files changed, 218 insertions(+)
 create mode 100644 tools/perf/tests/clang.c
 create mode 100644 tools/perf/util/c++/Build
 create mode 100644 tools/perf/util/c++/clang-c.h
 create mode 100644 tools/perf/util/c++/clang-test.cpp
 create mode 100644 tools/perf/util/c++/clang.cpp
 create mode 100644 tools/perf/util/c++/clang.h

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index af3ec94..6676c2d 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -43,6 +43,7 @@ perf-y += sdt.o
 perf-y += is_printable_array.o
 perf-y += bitmap.o
 perf-y += perf-hooks.o
+perf-y += clang.o
 
 $(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build
 	$(call rule_mkdir)
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index dab83f7..33aaa52 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -234,6 +234,15 @@ static struct test generic_tests[] = {
 		.func = test__perf_hooks,
 	},
 	{
+		.desc = "Test builtin clang support",
+		.func = test__clang,
+		.subtest = {
+			.skip_if_fail	= true,
+			.get_nr		= test__clang_subtest_get_nr,
+			.get_desc	= test__clang_subtest_get_desc,
+		}
+	},
+	{
 		.func = NULL,
 	},
 };
diff --git a/tools/perf/tests/clang.c b/tools/perf/tests/clang.c
new file mode 100644
index 0000000..57ee160
--- /dev/null
+++ b/tools/perf/tests/clang.c
@@ -0,0 +1,42 @@
+#include "tests.h"
+#include "debug.h"
+#include "util.h"
+#include "c++/clang-c.h"
+
+static struct {
+	int (*func)(void);
+	const char *desc;
+} clang_testcase_table[] = {
+#ifdef HAVE_LIBCLANGLLVM_SUPPORT
+	{
+		.func = test__clang_to_IR,
+		.desc = "Test builtin clang compile C source to IR",
+	},
+#endif
+};
+
+int test__clang_subtest_get_nr(void)
+{
+	return (int)ARRAY_SIZE(clang_testcase_table);
+}
+
+const char *test__clang_subtest_get_desc(int i)
+{
+	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
+		return NULL;
+	return clang_testcase_table[i].desc;
+}
+
+#ifndef HAVE_LIBCLANGLLVM_SUPPORT
+int test__clang(int i __maybe_unused)
+{
+	return TEST_SKIP;
+}
+#else
+int test__clang(int i __maybe_unused)
+{
+	if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table))
+		return TEST_FAIL;
+	return clang_testcase_table[i].func();
+}
+#endif
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 3a1f98f..0d7b251 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -92,6 +92,9 @@ int test__sdt_event(int subtest);
 int test__is_printable_array(int subtest);
 int test__bitmap_print(int subtest);
 int test__perf_hooks(int subtest);
+int test__clang(int subtest);
+const char *test__clang_subtest_get_desc(int subtest);
+int test__clang_subtest_get_nr(void);
 
 #if defined(__arm__) || defined(__aarch64__)
 #ifdef HAVE_DWARF_UNWIND_SUPPORT
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index b2a47aa..743a889 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -125,6 +125,8 @@ endif
 
 libperf-y += perf-hooks.o
 
+libperf-$(CONFIG_CXX) += c++/
+
 CFLAGS_config.o   += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 # avoid compiler warnings in 32-bit mode
 CFLAGS_genelf_debug.o  += -Wno-packed
diff --git a/tools/perf/util/c++/Build b/tools/perf/util/c++/Build
new file mode 100644
index 0000000..988fef1
--- /dev/null
+++ b/tools/perf/util/c++/Build
@@ -0,0 +1,2 @@
+libperf-$(CONFIG_CLANGLLVM) += clang.o
+libperf-$(CONFIG_CLANGLLVM) += clang-test.o
diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h
new file mode 100644
index 0000000..dcde4b5
--- /dev/null
+++ b/tools/perf/util/c++/clang-c.h
@@ -0,0 +1,16 @@
+#ifndef PERF_UTIL_CLANG_C_H
+#define PERF_UTIL_CLANG_C_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+extern void perf_clang__init(void);
+extern void perf_clang__cleanup(void);
+
+extern int test__clang_to_IR(void);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp
new file mode 100644
index 0000000..3da6bfa
--- /dev/null
+++ b/tools/perf/util/c++/clang-test.cpp
@@ -0,0 +1,31 @@
+#include "clang.h"
+#include "clang-c.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/LLVMContext.h"
+
+class perf_clang_scope {
+public:
+	explicit perf_clang_scope() {perf_clang__init();}
+	~perf_clang_scope() {perf_clang__cleanup();}
+};
+
+extern "C" {
+
+int test__clang_to_IR(void)
+{
+	perf_clang_scope _scope;
+
+	std::unique_ptr<llvm::Module> M =
+		perf::getModuleFromSource("perf-test.c",
+					  "int myfunc(void) {return 1;}");
+
+	if (!M)
+		return -1;
+
+	for (llvm::Function& F : *M)
+		if (F.getName() == "myfunc")
+			return 0;
+	return -1;
+}
+
+}
diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp
new file mode 100644
index 0000000..c17b117
--- /dev/null
+++ b/tools/perf/util/c++/clang.cpp
@@ -0,0 +1,96 @@
+/*
+ * llvm C frontend for perf. Support dynamically compile C file
+ *
+ * Inspired by clang example code:
+ * http://llvm.org/svn/llvm-project/cfe/trunk/examples/clang-interpreter/main.cpp
+ *
+ * Copyright (C) 2016 Wang Nan <wangnan0@huawei.com>
+ * Copyright (C) 2016 Huawei Inc.
+ */
+
+#include "clang/CodeGen/CodeGenAction.h"
+#include "clang/Frontend/CompilerInvocation.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Option/Option.h"
+#include "llvm/Support/ManagedStatic.h"
+#include <memory>
+
+#include "clang.h"
+#include "clang-c.h"
+
+namespace perf {
+
+static std::unique_ptr<llvm::LLVMContext> LLVMCtx;
+
+using namespace clang;
+
+static vfs::InMemoryFileSystem *
+buildVFS(StringRef& Name, StringRef& Content)
+{
+	vfs::InMemoryFileSystem *VFS = new vfs::InMemoryFileSystem(true);
+	VFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content));
+	return VFS;
+}
+
+static CompilerInvocation *
+createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
+{
+	llvm::opt::ArgStringList CCArgs {
+		"-cc1",
+		"-triple", "bpf-pc-linux",
+		"-fsyntax-only",
+		"-ferror-limit", "19",
+		"-fmessage-length", "127",
+		"-O2",
+		"-nostdsysteminc",
+		"-nobuiltininc",
+		"-vectorize-loops",
+		"-vectorize-slp",
+		"-Wno-unused-value",
+		"-Wno-pointer-sign",
+		"-x", "c"};
+	CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs);
+
+	FrontendOptions& Opts = CI->getFrontendOpts();
+	Opts.Inputs.clear();
+	Opts.Inputs.emplace_back(Path, IK_C);
+	return CI;
+}
+
+std::unique_ptr<llvm::Module>
+getModuleFromSource(StringRef Name, StringRef Content)
+{
+	CompilerInstance Clang;
+	Clang.createDiagnostics();
+
+	IntrusiveRefCntPtr<vfs::FileSystem> VFS = buildVFS(Name, Content);
+	Clang.setVirtualFileSystem(&*VFS);
+
+	IntrusiveRefCntPtr<CompilerInvocation> CI =
+		createCompilerInvocation(Name, Clang.getDiagnostics());
+	Clang.setInvocation(&*CI);
+
+	std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx));
+	if (!Clang.ExecuteAction(*Act))
+		return std::unique_ptr<llvm::Module>(nullptr);
+
+	return Act->takeModule();
+}
+
+}
+
+extern "C" {
+void perf_clang__init(void)
+{
+	perf::LLVMCtx.reset(new llvm::LLVMContext());
+}
+
+void perf_clang__cleanup(void)
+{
+	perf::LLVMCtx.reset(nullptr);
+	llvm::llvm_shutdown();
+}
+}
diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h
new file mode 100644
index 0000000..f64483b
--- /dev/null
+++ b/tools/perf/util/c++/clang.h
@@ -0,0 +1,16 @@
+#ifndef PERF_UTIL_CLANG_H
+#define PERF_UTIL_CLANG_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include <memory>
+namespace perf {
+
+using namespace llvm;
+
+std::unique_ptr<Module>
+getModuleFromSource(StringRef Name, StringRef Content);
+
+}
+#endif
-- 
2.10.1

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Alexei Starovoitov]

On Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan wrote:
> Add basic clang support in clang.cpp and test__clang() testcase. The
> first testcase checks if builtin clang is able to generate LLVM IR.
> 
> tests/clang.c is a proxy. Real testcase resides in
> utils/c++/clang-test.cpp in c++ and exports C interface to perf test
> subsystem.
> 
> Test result:
> 
>    $ perf test -v clang
>    51: Test builtin clang support                               :
>    51.1: Test builtin clang compile C source to IR              :
>    --- start ---
>    test child forked, pid 13215
>    test child finished with 0
>    ---- end ----
>    Test builtin clang support subtest 0: Ok
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
...
> +static CompilerInvocation *
> +createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
> +{
> +	llvm::opt::ArgStringList CCArgs {
> +		"-cc1",
> +		"-triple", "bpf-pc-linux",
> +		"-fsyntax-only",
> +		"-ferror-limit", "19",
> +		"-fmessage-length", "127",

why such limits?

> +		"-O2",
> +		"-nostdsysteminc",
> +		"-nobuiltininc",
> +		"-vectorize-loops",
> +		"-vectorize-slp",

why above two flags are needed?

> +		"-Wno-unused-value",
> +		"-Wno-pointer-sign",

these two -Wno makes sense. please add the comment to explain the reasons.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Wangnan (F)]

On 2016/11/27 1:17, Alexei Starovoitov wrote:
> On Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan wrote:
>> Add basic clang support in clang.cpp and test__clang() testcase. The
>> first testcase checks if builtin clang is able to generate LLVM IR.
>>
>> tests/clang.c is a proxy. Real testcase resides in
>> utils/c++/clang-test.cpp in c++ and exports C interface to perf test
>> subsystem.
>>
>> Test result:
>>
>>     $ perf test -v clang
>>     51: Test builtin clang support                               :
>>     51.1: Test builtin clang compile C source to IR              :
>>     --- start ---
>>     test child forked, pid 13215
>>     test child finished with 0
>>     ---- end ----
>>     Test builtin clang support subtest 0: Ok
>>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> ...
>> +static CompilerInvocation *
>> +createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags)
>> +{
>> +	llvm::opt::ArgStringList CCArgs {
>> +		"-cc1",
>> +		"-triple", "bpf-pc-linux",
>> +		"-fsyntax-only",
>> +		"-ferror-limit", "19",
>> +		"-fmessage-length", "127",
> why such limits?
>
>> +		"-O2",
>> +		"-nostdsysteminc",
>> +		"-nobuiltininc",
>> +		"-vectorize-loops",
>> +		"-vectorize-slp",

Thank you for pointing these out. These arguments are get by analysising
the clang example:

https://llvm.org/svn/llvm-project/cfe/branches/ggreif/CallInst-operands/examples/clang-interpreter/main.cpp

The above example create a C compiler using clang::driver::Driver (bcc 
also uses driver).
I form the argument list according to arglist the driver created for its CI,
and leaves arguments I'm not quite sure unchanged.

> why above two flags are needed?
>
>> +		"-Wno-unused-value",
>> +		"-Wno-pointer-sign",
> these two -Wno makes sense. please add the comment to explain the reasons.
>
They are inherited from samples/bpf/Makefile to suppress some warning
when include kernel headers.

Thank you.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu:
> Add basic clang support in clang.cpp and test__clang() testcase. The
> first testcase checks if builtin clang is able to generate LLVM IR.
> 
> tests/clang.c is a proxy. Real testcase resides in
> utils/c++/clang-test.cpp in c++ and exports C interface to perf test
> subsystem.
> 
> Test result:
> 
>    $ perf test -v clang
>    51: Test builtin clang support                               :
>    51.1: Test builtin clang compile C source to IR              :
>    --- start ---
>    test child forked, pid 13215
>    test child finished with 0
>    ---- end ----
>    Test builtin clang support subtest 0: Ok

While testing this I noticed that the perf binary got huge, can't this
be done in some other way, i.e. using dynamic library?

[root@jouet ~]# size /tmp/perf
   text	   data	    bss	    dec	    hex	filename
75333688	1421584	23962176	100717448	600d388 /tmp/perf
[root@jouet ~]#

I've built it with this:

  make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin

the resulting binary:

[acme@jouet linux]$ 
[acme@jouet linux]$ ls -la ~/bin/perf
-rwxr-xr-x. 2 acme acme 131689136 Dec  2 12:31 /home/acme/bin/perf
[acme@jouet linux]$ ls -lah ~/bin/perf
-rwxr-xr-x. 2 acme acme 126M Dec  2 12:31 /home/acme/bin/perf
[acme@jouet linux]$

Huge, after stripping it:

[acme@jouet linux]$ ls -la /tmp/perf
-rwxr-xr-x. 1 root root 76759056 Dec  2 12:33 /tmp/perf
[acme@jouet linux]$ ls -lah /tmp/perf
-rwxr-xr-x. 1 root root 74M Dec  2 12:33 /tmp/perf
[acme@jouet linux]$

Still huge :-\

Ok, this needs explicitely enabling, but then, for distros, if we want
this feature to be enabled...

I think we should try to help distributions to ship a basic perf binary
in a base package that doesn't brings many libraries, like was done for
the gtk UI, and also we should start providing RPM specfiles and debian
(and others, hopefully contributed by other people with interest in
other distros), to help in getting a sane package split, i.e. want
llvm/bpf? install the 'perf-ebpf' rpm/deb/whatever package and incur in
these costs.

But being a n00b on llvm/clang libraries, etc, my question goes back to:
can we have this using a libllvm.so or libclang.so dynamic libraries?

See this, out of hand :-\ Tell me I'm doing something wrong :-)

[acme@jouet linux]$ cat /etc/fedora-release 
Fedora release 25 (Twenty Five)
[acme@jouet linux]$ rpm -qa | egrep llvm\|clang
llvm-libs-3.8.0-1.fc25.x86_64
[acme@jouet linux]$

[acme@jouet linux]$ nm ~/bin/perf | grep -i clang | wc -l
204757

[acme@jouet linux]$ nm ~/bin/perf | grep -i llvm | wc -l
234036

[acme@jouet linux]$ nm ~/bin/perf | grep -i clang | tail
00000000011741ba t
_ZZZN5clang7CodeGen15CodeGenFunction20EmitOMPTaskDirectiveERKNS_16OMPTaskDirectiveEENKUlRS1_E_clES5_ENKUlvE_clEv
0000000001169fee t
_ZZZN5clang7CodeGen15CodeGenFunction25EmitOMPFirstprivateClauseERKNS_22OMPExecutableDirectiveERNS1_15OMPPrivateScopeEENKUlvE_clEvENKUlNS0_7AddressES8_E_clES8_S8_
00000000038b83e0 r
_ZZZN5clang9ASTReader24FindExternalLexicalDeclsEPKNS_11DeclContextEN4llvm12function_refIFbNS_4Decl4KindEEEERNS4_15SmallVectorImplIPS6_EEENKUlPNS_13serialization10ModuleFileENS4_8ArrayRefINS4_7support6detail31packed_endian_specific_integralIjLNSI_10endiannessE2ELm1EEEEEE_clESG_SN_E19__PRETTY_FUNCTION__
0000000003934140 r
_ZZZN5clang9ASTWriter12WriteASTCoreERNS_4SemaEN4llvm9StringRefERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPNS_6ModuleEENKUljbE0_clEjbE19__PRETTY_FUNCTION__
00000000039319e0 r
_ZZZN5clang9ASTWriter12WriteASTCoreERNS_4SemaEN4llvm9StringRefERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPNS_6ModuleEENKUlPNS_4DeclENS_13serialization17PredefinedDeclIDsEE_clESG_SI_E19__PRETTY_FUNCTION__
000000000393b700 r
_ZZZN5clang9ASTWriter17GetOrCreateTypeIDENS_8QualTypeEENKUlS1_E_clES1_E19__PRETTY_FUNCTION__
000000000393b880 r
_ZZZNK5clang9ASTWriter9getTypeIDENS_8QualTypeEENKUlS1_E_clES1_E19__PRETTY_FUNCTION__
0000000002176480 t
_ZZZZN12_GLOBAL__N_19ASTDumper11dumpLookupsEPKN5clang11DeclContextEbENKUlvE_clEvENKUlvE_clEvENKUlvE_clEv
000000000144ef0a t
_ZZZZN5clang7CodeGen15CGOpenMPRuntime13emitReductionERNS0_15CodeGenFunctionENS_14SourceLocationEN4llvm8ArrayRefIPKNS_4ExprEEESA_SA_SA_bbENKUlS3_S9_S9_S9_E1_clES3_S9_S9_S9_ENKUlNS0_6RValueEE_clESC_ENKUlvE_clEv
000000000217640c t
_ZZZZZN12_GLOBAL__N_19ASTDumper11dumpLookupsEPKN5clang11DeclContextEbENKUlvE_clEvENKUlvE_clEvENKUlvE_clEvENKUlPNS1_4DeclEE_clES9_
[acme@jouet linux]$

Thanks,

- Arnaldo

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Wangnan (F)]

On 2016/12/2 23:44, Arnaldo Carvalho de Melo wrote:
> Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu:
>> Add basic clang support in clang.cpp and test__clang() testcase. The
>> first testcase checks if builtin clang is able to generate LLVM IR.
>>
>> tests/clang.c is a proxy. Real testcase resides in
>> utils/c++/clang-test.cpp in c++ and exports C interface to perf test
>> subsystem.
>>
>> Test result:
>>
>>     $ perf test -v clang
>>     51: Test builtin clang support                               :
>>     51.1: Test builtin clang compile C source to IR              :
>>     --- start ---
>>     test child forked, pid 13215
>>     test child finished with 0
>>     ---- end ----
>>     Test builtin clang support subtest 0: Ok
> While testing this I noticed that the perf binary got huge, can't this
> be done in some other way, i.e. using dynamic library?

I intentionally use statically linking because it is good for
smartphone: we can simply 'adb push' a statically linked perf
to Android.

The resulting ELF executable would be even larger if LLVM is
built with default setting.

In my setting the resuling 'perf' is less than 60MB:

  $ ls -s  ~/perf -h
  58M /home/wn/perf

  $ size  ~/perf
    text       data        bss        dec        hex    filename
  56931273    2950808    24108632    83990713    50198b9 /home/wn/perf

It is reasonable for me.

I think using dynamic clang and llvm libraries is possible but I
never tried it before. It depend on LLVM compiling. I think if
distro provides shared libraries then perf can utilize them
automatically. Let me check it today.

About the file size, I discussed with Alexei, he taught me a lot
on it. Maybe he or his ioVisor ex-colleagues can provide some
new idea?

Thank you.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Wangnan (F)]

On 2016/12/5 10:36, Wangnan (F) wrote:
>
>
> On 2016/12/2 23:44, Arnaldo Carvalho de Melo wrote:
>> Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu:
>>> Add basic clang support in clang.cpp and test__clang() testcase. The
>>> first testcase checks if builtin clang is able to generate LLVM IR.
>>>
>>> tests/clang.c is a proxy. Real testcase resides in
>>> utils/c++/clang-test.cpp in c++ and exports C interface to perf test
>>> subsystem.
>>>
>>> Test result:
>>>
>>>     $ perf test -v clang
>>>     51: Test builtin clang support                               :
>>>     51.1: Test builtin clang compile C source to IR              :
>>>     --- start ---
>>>     test child forked, pid 13215
>>>     test child finished with 0
>>>     ---- end ----
>>>     Test builtin clang support subtest 0: Ok
>> While testing this I noticed that the perf binary got huge, can't this
>> be done in some other way, i.e. using dynamic library?
>
> I intentionally use statically linking because it is good for
> smartphone: we can simply 'adb push' a statically linked perf
> to Android.
>
> The resulting ELF executable would be even larger if LLVM is
> built with default setting.
>
> In my setting the resuling 'perf' is less than 60MB:
>
>  $ ls -s  ~/perf -h
>  58M /home/wn/perf
>
>  $ size  ~/perf
>    text       data        bss        dec        hex    filename
>  56931273    2950808    24108632    83990713    50198b9 /home/wn/perf
>
> It is reasonable for me.
>
> I think using dynamic clang and llvm libraries is possible but I
> never tried it before. It depend on LLVM compiling. I think if
> distro provides shared libraries then perf can utilize them
> automatically. Let me check it today.
>

Good news: it works.

To enable llvm and clang dynamic libraries, we need to build llvm and 
clang with
cmake option -DBUILD_SHARED_LIBS=ON. Then apply a trivial patch to perf:

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index dfb20dd..e3054f3 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -346,7 +346,7 @@ LIBS = -Wl,--whole-archive $(PERFLIBS) 
-Wl,--no-whole-archive -Wl,--start-group

  ifeq ($(USE_CLANG), 1)
    CLANGLIBS_LIST = AST Basic CodeGen Driver Frontend Lex Tooling Edit 
Sema Analysis Parse Serialization
-  LIBCLANG = $(foreach l,$(CLANGLIBS_LIST),$(wildcard $(shell 
$(LLVM_CONFIG) --libdir)/libclang$(l).a))
+  LIBCLANG = $(foreach l,$(CLANGLIBS_LIST),$(wildcard $(shell 
$(LLVM_CONFIG) --libdir)/libclang$(l).so))
    LIBS += -Wl,--start-group $(LIBCLANG) -Wl,--end-group
  endif

(replace '.a' to '.so')

Resuling perf executable:

$ ls -s -h ~/perf
26M /home/wn/perf

$ size ~/perf
    text       data        bss        dec        hex    filename
4274339     755032    23959984    28989355    1ba57ab /home/wn/perf

$ strip ~/perf
$ ls -s -h
4.9M /home/wn/perf

$ ldd ~/perf | grep 'LLVM\|clang'
     libclangBasic.so => /tmp/oxygen_root/usr/lib64/libclangBasic.so 
(0x00007f24da49f000)
     libclangCodeGen.so => /tmp/oxygen_root/usr/lib64/libclangCodeGen.so 
(0x00007f24d9f72000)
     libclangFrontend.so => 
/tmp/oxygen_root/usr/lib64/libclangFrontend.so (0x00007f24d9c64000)
     libclangTooling.so => /tmp/oxygen_root/usr/lib64/libclangTooling.so 
(0x00007f24d9a01000)
     libLLVMBPFCodeGen.so => 
/tmp/oxygen_root/usr/lib64/libLLVMBPFCodeGen.so (0x00007f24d97e2000)
     libLLVMBPFDesc.so => /tmp/oxygen_root/usr/lib64/libLLVMBPFDesc.so 
(0x00007f24d95da000)
     ....

As I said, if distro provides dynamic libraries then everything would be 
fine.

If you are okay with the above conclusion, in next version I'll use 
'-lclangBasic'
style linking options so it works for both dynamic and static 
LLVM/clang, then let's
wait for distro's action.

Thank you.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Mon, Dec 05, 2016 at 12:41:03PM +0800, Wangnan (F) escreveu:
> On 2016/12/5 10:36, Wangnan (F) wrote:
> > On 2016/12/2 23:44, Arnaldo Carvalho de Melo wrote:
> > > Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu:
> > I intentionally use statically linking because it is good for
> > smartphone: we can simply 'adb push' a statically linked perf
> > to Android.

So I'm trying to get back to this, and built -DBUILD_SHARED_LIBS=ON HEAD
llvm/clang, applied your patch to switch from .a to .so and got a smaller perf
binary that is dinamically linked to tons of libraries from LLVM and clang, see
below, how to proceed?

I think we should default to linking it dinamically, which is what I'll have in
a test branch and add some machinery to autodetect and use the right approach,
can you do that while I try to go on processing the next patches?

Its a huge bump in the number of libraries in such a build:

[acme@jouet linux]$ ldd /tmp/perf  | wc -l
174
[acme@jouet linux]$ ldd /tmp/perf | egrep -i llvm\|clang | wc -l
145
[acme@jouet linux]$ 

But one expected by those explicitely enabling support for eBPF, right? Those
needing a llvm/clang toolchain installed can opt for using the external
toolchain, that we keep in place, those with a need for those libraries can use
this approach and then those just wanting perf+eBPF, say on a smartphone, like
you said, can get everything in a static build.

- Arnaldo

[acme@jouet linux]$ ls -lah /tmp/perf
-rwxr-xr-x. 1 acme acme 4.4M Jan 17 10:29 /tmp/perf
[acme@jouet linux]$ size /tmp/perf
   text	   data	    bss	    dec	    hex	filename
3954488	 622440	23912104	28489032	1b2b548	/tmp/perf
[acme@jouet linux]$ ldd /tmp/perf | egrep -i llvm\|clang
	libclangAST.so.40 => /usr/local/lib/libclangAST.so.40 (0x00007f4964981000)
	libclangBasic.so.40 => /usr/local/lib/libclangBasic.so.40 (0x00007f49644e6000)
	libclangCodeGen.so.40 => /usr/local/lib/libclangCodeGen.so.40 (0x00007f4963f92000)
	libclangDriver.so.40 => /usr/local/lib/libclangDriver.so.40 (0x00007f4963c59000)
	libclangFrontend.so.40 => /usr/local/lib/libclangFrontend.so.40 (0x00007f4963945000)
	libclangLex.so.40 => /usr/local/lib/libclangLex.so.40 (0x00007f496368d000)
	libclangTooling.so.40 => /usr/local/lib/libclangTooling.so.40 (0x00007f4963447000)
	libclangEdit.so.40 => /usr/local/lib/libclangEdit.so.40 (0x00007f4963239000)
	libclangSema.so.40 => /usr/local/lib/libclangSema.so.40 (0x00007f4962a32000)
	libclangAnalysis.so.40 => /usr/local/lib/libclangAnalysis.so.40 (0x00007f496279b000)
	libclangParse.so.40 => /usr/local/lib/libclangParse.so.40 (0x00007f49624b4000)
	libclangSerialization.so.40 => /usr/local/lib/libclangSerialization.so.40 (0x00007f4962156000)
	libLLVMLTO.so.40 => /usr/local/lib/libLLVMLTO.so.40 (0x00007f4961f13000)
	libLLVMPasses.so.40 => /usr/local/lib/libLLVMPasses.so.40 (0x00007f4961c6d000)
	libLLVMObjCARCOpts.so.40 => /usr/local/lib/libLLVMObjCARCOpts.so.40 (0x00007f4961a48000)
	libLLVMMIRParser.so.40 => /usr/local/lib/libLLVMMIRParser.so.40 (0x00007f496181d000)
	libLLVMSymbolize.so.40 => /usr/local/lib/libLLVMSymbolize.so.40 (0x00007f496160e000)
	libLLVMDebugInfoPDB.so.40 => /usr/local/lib/libLLVMDebugInfoPDB.so.40 (0x00007f49613ca000)
	libLLVMDebugInfoDWARF.so.40 => /usr/local/lib/libLLVMDebugInfoDWARF.so.40 (0x00007f496118f000)
	libLLVMCoverage.so.40 => /usr/local/lib/libLLVMCoverage.so.40 (0x00007f4960f6d000)
	libLLVMTableGen.so.40 => /usr/local/lib/libLLVMTableGen.so.40 (0x00007f4960d22000)
	libLLVMOrcJIT.so.40 => /usr/local/lib/libLLVMOrcJIT.so.40 (0x00007f4960aea000)
	libLLVMXCoreDisassembler.so.40 => /usr/local/lib/libLLVMXCoreDisassembler.so.40 (0x00007f49608e4000)
	libLLVMXCoreCodeGen.so.40 => /usr/local/lib/libLLVMXCoreCodeGen.so.40 (0x00007f4960697000)
	libLLVMXCoreDesc.so.40 => /usr/local/lib/libLLVMXCoreDesc.so.40 (0x00007f4960489000)
	libLLVMXCoreInfo.so.40 => /usr/local/lib/libLLVMXCoreInfo.so.40 (0x00007f4960287000)
	libLLVMXCoreAsmPrinter.so.40 => /usr/local/lib/libLLVMXCoreAsmPrinter.so.40 (0x00007f4960081000)
	libLLVMSystemZDisassembler.so.40 => /usr/local/lib/libLLVMSystemZDisassembler.so.40 (0x00007f495fe70000)
	libLLVMSystemZCodeGen.so.40 => /usr/local/lib/libLLVMSystemZCodeGen.so.40 (0x00007f495fbde000)
	libLLVMSystemZAsmParser.so.40 => /usr/local/lib/libLLVMSystemZAsmParser.so.40 (0x00007f495f9c6000)
	libLLVMSystemZDesc.so.40 => /usr/local/lib/libLLVMSystemZDesc.so.40 (0x00007f495f767000)
	libLLVMSystemZInfo.so.40 => /usr/local/lib/libLLVMSystemZInfo.so.40 (0x00007f495f565000)
	libLLVMSystemZAsmPrinter.so.40 => /usr/local/lib/libLLVMSystemZAsmPrinter.so.40 (0x00007f495f356000)
	libLLVMSparcDisassembler.so.40 => /usr/local/lib/libLLVMSparcDisassembler.so.40 (0x00007f495f14f000)
	libLLVMSparcCodeGen.so.40 => /usr/local/lib/libLLVMSparcCodeGen.so.40 (0x00007f495ef02000)
	libLLVMSparcAsmParser.so.40 => /usr/local/lib/libLLVMSparcAsmParser.so.40 (0x00007f495ecf0000)
	libLLVMSparcDesc.so.40 => /usr/local/lib/libLLVMSparcDesc.so.40 (0x00007f495eacb000)
	libLLVMSparcInfo.so.40 => /usr/local/lib/libLLVMSparcInfo.so.40 (0x00007f495e8c9000)
	libLLVMSparcAsmPrinter.so.40 => /usr/local/lib/libLLVMSparcAsmPrinter.so.40 (0x00007f495e6ad000)
	libLLVMRISCVDesc.so.40 => /usr/local/lib/libLLVMRISCVDesc.so.40 (0x00007f495e4a5000)
	libLLVMRISCVCodeGen.so.40 => /usr/local/lib/libLLVMRISCVCodeGen.so.40 (0x00007f495e2a2000)
	libLLVMRISCVInfo.so.40 => /usr/local/lib/libLLVMRISCVInfo.so.40 (0x00007f495e0a0000)
	libLLVMPowerPCDisassembler.so.40 => /usr/local/lib/libLLVMPowerPCDisassembler.so.40 (0x00007f495de94000)
	libLLVMPowerPCCodeGen.so.40 => /usr/local/lib/libLLVMPowerPCCodeGen.so.40 (0x00007f495db82000)
	libLLVMPowerPCAsmParser.so.40 => /usr/local/lib/libLLVMPowerPCAsmParser.so.40 (0x00007f495d962000)
	libLLVMPowerPCDesc.so.40 => /usr/local/lib/libLLVMPowerPCDesc.so.40 (0x00007f495d6f9000)
	libLLVMPowerPCInfo.so.40 => /usr/local/lib/libLLVMPowerPCInfo.so.40 (0x00007f495d4f6000)
	libLLVMPowerPCAsmPrinter.so.40 => /usr/local/lib/libLLVMPowerPCAsmPrinter.so.40 (0x00007f495d2db000)
	libLLVMNVPTXCodeGen.so.40 => /usr/local/lib/libLLVMNVPTXCodeGen.so.40 (0x00007f495d03b000)
	libLLVMNVPTXDesc.so.40 => /usr/local/lib/libLLVMNVPTXDesc.so.40 (0x00007f495cde7000)
	libLLVMNVPTXInfo.so.40 => /usr/local/lib/libLLVMNVPTXInfo.so.40 (0x00007f495cbe3000)
	libLLVMNVPTXAsmPrinter.so.40 => /usr/local/lib/libLLVMNVPTXAsmPrinter.so.40 (0x00007f495c9ca000)
	libLLVMMSP430CodeGen.so.40 => /usr/local/lib/libLLVMMSP430CodeGen.so.40 (0x00007f495c79d000)
	libLLVMMSP430Desc.so.40 => /usr/local/lib/libLLVMMSP430Desc.so.40 (0x00007f495c592000)
	libLLVMMSP430Info.so.40 => /usr/local/lib/libLLVMMSP430Info.so.40 (0x00007f495c390000)
	libLLVMMSP430AsmPrinter.so.40 => /usr/local/lib/libLLVMMSP430AsmPrinter.so.40 (0x00007f495c18c000)
	libLLVMMipsDisassembler.so.40 => /usr/local/lib/libLLVMMipsDisassembler.so.40 (0x00007f495bf73000)
	libLLVMMipsCodeGen.so.40 => /usr/local/lib/libLLVMMipsCodeGen.so.40 (0x00007f495bc88000)
	libLLVMMipsAsmParser.so.40 => /usr/local/lib/libLLVMMipsAsmParser.so.40 (0x00007f495ba4a000)
	libLLVMMipsDesc.so.40 => /usr/local/lib/libLLVMMipsDesc.so.40 (0x00007f495b7bf000)
	libLLVMMipsInfo.so.40 => /usr/local/lib/libLLVMMipsInfo.so.40 (0x00007f495b5bc000)
	libLLVMMipsAsmPrinter.so.40 => /usr/local/lib/libLLVMMipsAsmPrinter.so.40 (0x00007f495b3ad000)
	libLLVMLanaiDisassembler.so.40 => /usr/local/lib/libLLVMLanaiDisassembler.so.40 (0x00007f495b1a7000)
	libLLVMLanaiCodeGen.so.40 => /usr/local/lib/libLLVMLanaiCodeGen.so.40 (0x00007f495af64000)
	libLLVMLanaiAsmParser.so.40 => /usr/local/lib/libLLVMLanaiAsmParser.so.40 (0x00007f495ad58000)
	libLLVMLanaiDesc.so.40 => /usr/local/lib/libLLVMLanaiDesc.so.40 (0x00007f495ab4a000)
	libLLVMLanaiInstPrinter.so.40 => /usr/local/lib/libLLVMLanaiInstPrinter.so.40 (0x00007f495a943000)
	libLLVMLanaiInfo.so.40 => /usr/local/lib/libLLVMLanaiInfo.so.40 (0x00007f495a741000)
	libLLVMHexagonDisassembler.so.40 => /usr/local/lib/libLLVMHexagonDisassembler.so.40 (0x00007f495a526000)
	libLLVMHexagonCodeGen.so.40 => /usr/local/lib/libLLVMHexagonCodeGen.so.40 (0x00007f495a15e000)
	libLLVMHexagonAsmParser.so.40 => /usr/local/lib/libLLVMHexagonAsmParser.so.40 (0x00007f4959f23000)
	libLLVMHexagonDesc.so.40 => /usr/local/lib/libLLVMHexagonDesc.so.40 (0x00007f4959c77000)
	libLLVMHexagonInfo.so.40 => /usr/local/lib/libLLVMHexagonInfo.so.40 (0x00007f4959a75000)
	libLLVMBPFDisassembler.so.40 => /usr/local/lib/libLLVMBPFDisassembler.so.40 (0x00007f4959871000)
	libLLVMBPFCodeGen.so.40 => /usr/local/lib/libLLVMBPFCodeGen.so.40 (0x00007f4959651000)
	libLLVMBPFDesc.so.40 => /usr/local/lib/libLLVMBPFDesc.so.40 (0x00007f4959447000)
	libLLVMBPFInfo.so.40 => /usr/local/lib/libLLVMBPFInfo.so.40 (0x00007f4959244000)
	libLLVMBPFAsmPrinter.so.40 => /usr/local/lib/libLLVMBPFAsmPrinter.so.40 (0x00007f4959040000)
	libLLVMARMDisassembler.so.40 => /usr/local/lib/libLLVMARMDisassembler.so.40 (0x00007f4958e0c000)
	libLLVMARMCodeGen.so.40 => /usr/local/lib/libLLVMARMCodeGen.so.40 (0x00007f4958a8a000)
	libLLVMARMAsmParser.so.40 => /usr/local/lib/libLLVMARMAsmParser.so.40 (0x00007f495880d000)
	libLLVMARMDesc.so.40 => /usr/local/lib/libLLVMARMDesc.so.40 (0x00007f4958533000)
	libLLVMARMInfo.so.40 => /usr/local/lib/libLLVMARMInfo.so.40 (0x00007f4958330000)
	libLLVMARMAsmPrinter.so.40 => /usr/local/lib/libLLVMARMAsmPrinter.so.40 (0x00007f4958111000)
	libLLVMAMDGPUDisassembler.so.40 => /usr/local/lib/libLLVMAMDGPUDisassembler.so.40 (0x00007f4957ef2000)
	libLLVMAMDGPUCodeGen.so.40 => /usr/local/lib/libLLVMAMDGPUCodeGen.so.40 (0x00007f4957b91000)
	libLLVMAMDGPUAsmParser.so.40 => /usr/local/lib/libLLVMAMDGPUAsmParser.so.40 (0x00007f4957922000)
	libLLVMAMDGPUDesc.so.40 => /usr/local/lib/libLLVMAMDGPUDesc.so.40 (0x00007f49575c8000)
	libLLVMAMDGPUInfo.so.40 => /usr/local/lib/libLLVMAMDGPUInfo.so.40 (0x00007f49573c6000)
	libLLVMAMDGPUAsmPrinter.so.40 => /usr/local/lib/libLLVMAMDGPUAsmPrinter.so.40 (0x00007f4957195000)
	libLLVMAMDGPUUtils.so.40 => /usr/local/lib/libLLVMAMDGPUUtils.so.40 (0x00007f4956f79000)
	libLLVMAArch64Disassembler.so.40 => /usr/local/lib/libLLVMAArch64Disassembler.so.40 (0x00007f4956d58000)
	libLLVMAArch64CodeGen.so.40 => /usr/local/lib/libLLVMAArch64CodeGen.so.40 (0x00007f4956a18000)
	libLLVMAArch64AsmParser.so.40 => /usr/local/lib/libLLVMAArch64AsmParser.so.40 (0x00007f49567b9000)
	libLLVMAArch64Desc.so.40 => /usr/local/lib/libLLVMAArch64Desc.so.40 (0x00007f495651d000)
	libLLVMAArch64Info.so.40 => /usr/local/lib/libLLVMAArch64Info.so.40 (0x00007f495631a000)
	libLLVMAArch64AsmPrinter.so.40 => /usr/local/lib/libLLVMAArch64AsmPrinter.so.40 (0x00007f49560bb000)
	libLLVMAArch64Utils.so.40 => /usr/local/lib/libLLVMAArch64Utils.so.40 (0x00007f4955e9c000)
	libLLVMObjectYAML.so.40 => /usr/local/lib/libLLVMObjectYAML.so.40 (0x00007f4955c5d000)
	libLLVMLibDriver.so.40 => /usr/local/lib/libLLVMLibDriver.so.40 (0x00007f4955a57000)
	libLLVMOption.so.40 => /usr/local/lib/libLLVMOption.so.40 (0x00007f4955849000)
	libLLVMX86Disassembler.so.40 => /usr/local/lib/libLLVMX86Disassembler.so.40 (0x00007f49554e6000)
	libLLVMX86AsmParser.so.40 => /usr/local/lib/libLLVMX86AsmParser.so.40 (0x00007f495523e000)
	libLLVMX86CodeGen.so.40 => /usr/local/lib/libLLVMX86CodeGen.so.40 (0x00007f4954dc9000)
	libLLVMGlobalISel.so.40 => /usr/local/lib/libLLVMGlobalISel.so.40 (0x00007f4954bc5000)
	libLLVMSelectionDAG.so.40 => /usr/local/lib/libLLVMSelectionDAG.so.40 (0x00007f4954799000)
	libLLVMAsmPrinter.so.40 => /usr/local/lib/libLLVMAsmPrinter.so.40 (0x00007f49544ff000)
	libLLVMDebugInfoCodeView.so.40 => /usr/local/lib/libLLVMDebugInfoCodeView.so.40 (0x00007f495429b000)
	libLLVMDebugInfoMSF.so.40 => /usr/local/lib/libLLVMDebugInfoMSF.so.40 (0x00007f495408c000)
	libLLVMX86Desc.so.40 => /usr/local/lib/libLLVMX86Desc.so.40 (0x00007f4953ca8000)
	libLLVMMCDisassembler.so.40 => /usr/local/lib/libLLVMMCDisassembler.so.40 (0x00007f4953a9f000)
	libLLVMX86Info.so.40 => /usr/local/lib/libLLVMX86Info.so.40 (0x00007f495389d000)
	libLLVMX86AsmPrinter.so.40 => /usr/local/lib/libLLVMX86AsmPrinter.so.40 (0x00007f4953648000)
	libLLVMX86Utils.so.40 => /usr/local/lib/libLLVMX86Utils.so.40 (0x00007f495343c000)
	libLLVMMCJIT.so.40 => /usr/local/lib/libLLVMMCJIT.so.40 (0x00007f495322e000)
	libLLVMLineEditor.so.40 => /usr/local/lib/libLLVMLineEditor.so.40 (0x00007f495302a000)
	libLLVMInterpreter.so.40 => /usr/local/lib/libLLVMInterpreter.so.40 (0x00007f4952e03000)
	libLLVMExecutionEngine.so.40 => /usr/local/lib/libLLVMExecutionEngine.so.40 (0x00007f4952be2000)
	libLLVMRuntimeDyld.so.40 => /usr/local/lib/libLLVMRuntimeDyld.so.40 (0x00007f4952991000)
	libLLVMCodeGen.so.40 => /usr/local/lib/libLLVMCodeGen.so.40 (0x00007f4952407000)
	libLLVMTarget.so.40 => /usr/local/lib/libLLVMTarget.so.40 (0x00007f49521fb000)
	libLLVMCoroutines.so.40 => /usr/local/lib/libLLVMCoroutines.so.40 (0x00007f4951fdd000)
	libLLVMipo.so.40 => /usr/local/lib/libLLVMipo.so.40 (0x00007f4951d0d000)
	libLLVMInstrumentation.so.40 => /usr/local/lib/libLLVMInstrumentation.so.40 (0x00007f4951a70000)
	libLLVMVectorize.so.40 => /usr/local/lib/libLLVMVectorize.so.40 (0x00007f49517ea000)
	libLLVMScalarOpts.so.40 => /usr/local/lib/libLLVMScalarOpts.so.40 (0x00007f49513d3000)
	libLLVMLinker.so.40 => /usr/local/lib/libLLVMLinker.so.40 (0x00007f49511b7000)
	libLLVMIRReader.so.40 => /usr/local/lib/libLLVMIRReader.so.40 (0x00007f4950fb0000)
	libLLVMAsmParser.so.40 => /usr/local/lib/libLLVMAsmParser.so.40 (0x00007f4950d59000)
	libLLVMInstCombine.so.40 => /usr/local/lib/libLLVMInstCombine.so.40 (0x00007f4950a8b000)
	libLLVMTransformUtils.so.40 => /usr/local/lib/libLLVMTransformUtils.so.40 (0x00007f4950733000)
	libLLVMBitWriter.so.40 => /usr/local/lib/libLLVMBitWriter.so.40 (0x00007f49504fe000)
	libLLVMAnalysis.so.40 => /usr/local/lib/libLLVMAnalysis.so.40 (0x00007f495005b000)
	libLLVMObject.so.40 => /usr/local/lib/libLLVMObject.so.40 (0x00007f494fde8000)
	libLLVMMCParser.so.40 => /usr/local/lib/libLLVMMCParser.so.40 (0x00007f494fbac000)
	libLLVMMC.so.40 => /usr/local/lib/libLLVMMC.so.40 (0x00007f494f919000)
	libLLVMBitReader.so.40 => /usr/local/lib/libLLVMBitReader.so.40 (0x00007f494f6cd000)
	libLLVMProfileData.so.40 => /usr/local/lib/libLLVMProfileData.so.40 (0x00007f494f498000)
	libLLVMCore.so.40 => /usr/local/lib/libLLVMCore.so.40 (0x00007f494f049000)
	libLLVMSupport.so.40 => /usr/local/lib/libLLVMSupport.so.40 (0x00007f494ed49000)
	libLLVMDemangle.so.40 => /usr/local/lib/libLLVMDemangle.so.40 (0x00007f494eb30000)
	libclangASTMatchers.so.40 => /usr/local/lib/../lib/libclangASTMatchers.so.40 (0x00007f494daf9000)
	libclangFormat.so.40 => /usr/local/lib/../lib/libclangFormat.so.40 (0x00007f494d88f000)
	libclangRewrite.so.40 => /usr/local/lib/../lib/libclangRewrite.so.40 (0x00007f494d67e000)
	libclangToolingCore.so.40 => /usr/local/lib/../lib/libclangToolingCore.so.40 (0x00007f494d46d000)
[acme@jouet linux]$

> > The resulting ELF executable would be even larger if LLVM is
> > built with default setting.
> > 
> > In my setting the resuling 'perf' is less than 60MB:
> > 
> >  $ ls -s  ~/perf -h
> >  58M /home/wn/perf
> > 
> >  $ size  ~/perf
> >    text       data        bss        dec        hex    filename
> >  56931273    2950808    24108632    83990713    50198b9 /home/wn/perf
> > 
> > It is reasonable for me.
> > 
> > I think using dynamic clang and llvm libraries is possible but I
> > never tried it before. It depend on LLVM compiling. I think if
> > distro provides shared libraries then perf can utilize them
> > automatically. Let me check it today.
> > 
> 
> Good news: it works.
> 
> To enable llvm and clang dynamic libraries, we need to build llvm and clang
> with
> cmake option -DBUILD_SHARED_LIBS=ON. Then apply a trivial patch to perf:
> 
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index dfb20dd..e3054f3 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -346,7 +346,7 @@ LIBS = -Wl,--whole-archive $(PERFLIBS)
> -Wl,--no-whole-archive -Wl,--start-group
> 
>  ifeq ($(USE_CLANG), 1)
>    CLANGLIBS_LIST = AST Basic CodeGen Driver Frontend Lex Tooling Edit Sema
> Analysis Parse Serialization
> -  LIBCLANG = $(foreach l,$(CLANGLIBS_LIST),$(wildcard $(shell
> $(LLVM_CONFIG) --libdir)/libclang$(l).a))
> +  LIBCLANG = $(foreach l,$(CLANGLIBS_LIST),$(wildcard $(shell
> $(LLVM_CONFIG) --libdir)/libclang$(l).so))
>    LIBS += -Wl,--start-group $(LIBCLANG) -Wl,--end-group
>  endif
> 
> (replace '.a' to '.so')
> 
> Resuling perf executable:
> 
> $ ls -s -h ~/perf
> 26M /home/wn/perf
> 
> $ size ~/perf
>    text       data        bss        dec        hex    filename
> 4274339     755032    23959984    28989355    1ba57ab /home/wn/perf
> 
> $ strip ~/perf
> $ ls -s -h
> 4.9M /home/wn/perf
> 
> $ ldd ~/perf | grep 'LLVM\|clang'
>     libclangBasic.so => /tmp/oxygen_root/usr/lib64/libclangBasic.so
> (0x00007f24da49f000)
>     libclangCodeGen.so => /tmp/oxygen_root/usr/lib64/libclangCodeGen.so
> (0x00007f24d9f72000)
>     libclangFrontend.so => /tmp/oxygen_root/usr/lib64/libclangFrontend.so
> (0x00007f24d9c64000)
>     libclangTooling.so => /tmp/oxygen_root/usr/lib64/libclangTooling.so
> (0x00007f24d9a01000)
>     libLLVMBPFCodeGen.so => /tmp/oxygen_root/usr/lib64/libLLVMBPFCodeGen.so
> (0x00007f24d97e2000)
>     libLLVMBPFDesc.so => /tmp/oxygen_root/usr/lib64/libLLVMBPFDesc.so
> (0x00007f24d95da000)
>     ....
> 
> As I said, if distro provides dynamic libraries then everything would be
> fine.
> 
> If you are okay with the above conclusion, in next version I'll use
> '-lclangBasic'
> style linking options so it works for both dynamic and static LLVM/clang,
> then let's
> wait for distro's action.
> 
> Thank you.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Alexei Starovoitov]

On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote:
> Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu:
> > Add basic clang support in clang.cpp and test__clang() testcase. The
> > first testcase checks if builtin clang is able to generate LLVM IR.
> > 
> > tests/clang.c is a proxy. Real testcase resides in
> > utils/c++/clang-test.cpp in c++ and exports C interface to perf test
> > subsystem.
> > 
> > Test result:
> > 
> >    $ perf test -v clang
> >    51: Test builtin clang support                               :
> >    51.1: Test builtin clang compile C source to IR              :
> >    --- start ---
> >    test child forked, pid 13215
> >    test child finished with 0
> >    ---- end ----
> >    Test builtin clang support subtest 0: Ok
> 
> While testing this I noticed that the perf binary got huge, can't this
> be done in some other way, i.e. using dynamic library?
> 
> [root@jouet ~]# size /tmp/perf
>    text	   data	    bss	    dec	    hex	filename
> 75333688	1421584	23962176	100717448	600d388 /tmp/perf
> [root@jouet ~]#
> 
> I've built it with this:
> 
>   make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin
> 
> the resulting binary:
> 
> [acme@jouet linux]$ 
> [acme@jouet linux]$ ls -la ~/bin/perf
> -rwxr-xr-x. 2 acme acme 131689136 Dec  2 12:31 /home/acme/bin/perf
> [acme@jouet linux]$ ls -lah ~/bin/perf
> -rwxr-xr-x. 2 acme acme 126M Dec  2 12:31 /home/acme/bin/perf
> [acme@jouet linux]$
> 
> Huge, after stripping it:
> 
> [acme@jouet linux]$ ls -la /tmp/perf
> -rwxr-xr-x. 1 root root 76759056 Dec  2 12:33 /tmp/perf
> [acme@jouet linux]$ ls -lah /tmp/perf
> -rwxr-xr-x. 1 root root 74M Dec  2 12:33 /tmp/perf
> [acme@jouet linux]$
> 
> Still huge :-\

yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode.
Try llvm-config --build-mode --assertion-mode
it should be Release OFF

> But being a n00b on llvm/clang libraries, etc, my question goes back to:
> can we have this using a libllvm.so or libclang.so dynamic libraries?

that can also work. The reason we build iovisor/bcc into single binary
is to ease operational headache.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu:
> On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote:
> > Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu:
> > > Add basic clang support in clang.cpp and test__clang() testcase. The
> > > first testcase checks if builtin clang is able to generate LLVM IR.
> > > 
> > > tests/clang.c is a proxy. Real testcase resides in
> > > utils/c++/clang-test.cpp in c++ and exports C interface to perf test
> > > subsystem.
> > > 
> > > Test result:
> > > 
> > >    $ perf test -v clang
> > >    51: Test builtin clang support                               :
> > >    51.1: Test builtin clang compile C source to IR              :
> > >    --- start ---
> > >    test child forked, pid 13215
> > >    test child finished with 0
> > >    ---- end ----
> > >    Test builtin clang support subtest 0: Ok
> > 
> > While testing this I noticed that the perf binary got huge, can't this
> > be done in some other way, i.e. using dynamic library?
> > 
> > [root@jouet ~]# size /tmp/perf
> >    text	   data	    bss	    dec	    hex	filename
> > 75333688	1421584	23962176	100717448	600d388 /tmp/perf
> > [root@jouet ~]#
> > 
> > I've built it with this:
> > 
> >   make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin
> > 
> > the resulting binary:
> > 
> > [acme@jouet linux]$ 
> > [acme@jouet linux]$ ls -la ~/bin/perf
> > -rwxr-xr-x. 2 acme acme 131689136 Dec  2 12:31 /home/acme/bin/perf
> > [acme@jouet linux]$ ls -lah ~/bin/perf
> > -rwxr-xr-x. 2 acme acme 126M Dec  2 12:31 /home/acme/bin/perf
> > [acme@jouet linux]$
> > 
> > Huge, after stripping it:
> > 
> > [acme@jouet linux]$ ls -la /tmp/perf
> > -rwxr-xr-x. 1 root root 76759056 Dec  2 12:33 /tmp/perf
> > [acme@jouet linux]$ ls -lah /tmp/perf
> > -rwxr-xr-x. 1 root root 74M Dec  2 12:33 /tmp/perf
> > [acme@jouet linux]$
> > 
> > Still huge :-\
> 
> yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode.
> Try llvm-config --build-mode --assertion-mode
> it should be Release OFF

Probably this was with 3.9 and built from git, quite a while ago, now I
removed it from /usr/local/ and installed what is in f25, but I fear it
will be insufficient, does 3.8 cuts it for what we're testing? Humm, it
looks like it will:

[root@jouet ~]# llc --version
LLVM (http://llvm.org/):
  LLVM version 3.8.0
  Optimized build.
  Built Mar 10 2016 (01:41:45).
  Default target: x86_64-unknown-linux-gnu
  Host CPU: broadwell

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    amdgcn     - AMD GCN GPUs
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    bpf        - BPF (host endian)
    bpfeb      - BPF (big endian)
    bpfel      - BPF (little endian)
    cpp        - C++ backend
    nvptx      - NVIDIA PTX 32-bit
    nvptx64    - NVIDIA PTX 64-bit
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    r600       - AMD GPUs HD2XXX-HD6XXX
    systemz    - SystemZ
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
[root@jouet ~]# 

But I'm now running the container based tests to send a pull req, will
check later, after that.

- Arnaldo
 
> > But being a n00b on llvm/clang libraries, etc, my question goes back to:
> > can we have this using a libllvm.so or libclang.so dynamic libraries?
> 
> that can also work. The reason we build iovisor/bcc into single binary
> is to ease operational headache.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Mon, Dec 05, 2016 at 07:02:48PM -0200, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu:
> > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode.
> > Try llvm-config --build-mode --assertion-mode
> > it should be Release OFF
> 
> Probably this was with 3.9 and built from git, quite a while ago, now I
> removed it from /usr/local/ and installed what is in f25, but I fear it
> will be insufficient, does 3.8 cuts it for what we're testing? Humm, it
> looks like it will:
> 
> [root@jouet ~]# llc --version
> LLVM (http://llvm.org/):
>   LLVM version 3.8.0
> 
> But I'm now running the container based tests to send a pull req, will
> check later, after that.

Not really, Wang, we need to update that feature detection test to state what
is the minimum required LLVM/clang version, one that has those functions,
which, unfortunately, isn't the one in the latest fedora, fedora 25:

[acme@jouet linux]$ rpm -q llvm-devel clang-devel
llvm-devel-3.8.0-1.fc25.x86_64
clang-devel-3.8.0-2.fc25.x86_64
[acme@jouet linux]$ 


  LINK     /tmp/build/perf/libperf-gtk.so
  INSTALL  GTK UI
/tmp/build/perf/libperf.a(libperf-in.o): In function `perf::createCompilerInvocation(llvm::SmallVector<char const*, 16u>, llvm::StringRef&, clang::DiagnosticsEngine&)':
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:56: undefined reference to `clang::tooling::newInvocation(clang::DiagnosticsEngine*, llvm::SmallVector<char const*, 16u> const&)'
/tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)':
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:68: undefined reference to `clang::CompilerInstance::CompilerInstance(std::shared_ptr<clang::PCHContainerOperations>, bool)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:69: undefined reference to `clang::CompilerInstance::createDiagnostics(clang::DiagnosticConsumer*, bool)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:76: undefined reference to `clang::CompilerInstance::setInvocation(clang::CompilerInvocation*)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:78: undefined reference to `clang::EmitLLVMOnlyAction::EmitLLVMOnlyAction(llvm::LLVMContext*)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:79: undefined reference to `clang::CompilerInstance::ExecuteAction(clang::FrontendAction&)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:82: undefined reference to `clang::CodeGenAction::takeModule()'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:68: undefined reference to `clang::CompilerInstance::~CompilerInstance()'
/tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef, llvm::StringRef)':
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:92: undefined reference to `clang::vfs::getRealFileSystem()'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:92: undefined reference to `clang::vfs::OverlayFileSystem::OverlayFileSystem(llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:94: undefined reference to `clang::vfs::InMemoryFileSystem::InMemoryFileSystem(bool)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:100: undefined reference to `clang::vfs::OverlayFileSystem::pushOverlay(llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)'
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:101: undefined reference to `clang::vfs::InMemoryFileSystem::addFile(llvm::Twine const&, long, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >)'
/tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef)':
/home/acme/git/linux/tools/perf/util/c++/clang.cpp:109: undefined reference to `clang::vfs::getRealFileSystem()'
/tmp/build/perf/libperf.a(libperf-in.o): In function `clang::CompilerInvocation::~CompilerInvocation()':
/usr/include/clang/Frontend/CompilerInvocation.h:102: undefined reference to `clang::CompilerInvocationBase::~CompilerInvocationBase()'
/tmp/build/perf/libperf.a(libperf-in.o): In function `void __gnu_cxx::new_allocator<clang::PCHContainerOperations>::construct<clang::PCHContainerOperations>(clang::PCHContainerOperations*)':
/usr/include/c++/6.2.1/ext/new_allocator.h:120: undefined reference to `clang::PCHContainerOperations::PCHContainerOperations()'
collect2: error: ld returned 1 exit status
Makefile.perf:491: recipe for target '/tmp/build/perf/perf' failed
make[1]: *** [/tmp/build/perf/perf] Error 1
Makefile:108: recipe for target 'install-bin' failed
make: *** [install-bin] Error 2
make: Leaving directory '/home/acme/git/linux/tools/perf'
[acme@jouet linux]$

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Wangnan (F)]

On 2016/12/6 5:48, Arnaldo Carvalho de Melo wrote:
> Em Mon, Dec 05, 2016 at 07:02:48PM -0200, Arnaldo Carvalho de Melo escreveu:
>> Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu:
>>> yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode.
>>> Try llvm-config --build-mode --assertion-mode
>>> it should be Release OFF
>> Probably this was with 3.9 and built from git, quite a while ago, now I
>> removed it from /usr/local/ and installed what is in f25, but I fear it
>> will be insufficient, does 3.8 cuts it for what we're testing? Humm, it
>> looks like it will:
>>
>> [root@jouet ~]# llc --version
>> LLVM (http://llvm.org/):
>>    LLVM version 3.8.0
>>
>> But I'm now running the container based tests to send a pull req, will
>> check later, after that.
> Not really, Wang, we need to update that feature detection test to state what
> is the minimum required LLVM/clang version, one that has those functions,
> which, unfortunately, isn't the one in the latest fedora, fedora 25:

I'll set the minimum required LLVM version to 3.9, and report
warning when LLVM is too old. However, since LLVM interface is
keep changing, finally we will have problem if we want to support
2 or 3 different clang/LLVM. We should keep moving minimum
requirement LLVM version if we don't want to see '#ifdef's
spread in our code.

Thank you.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Tue, Dec 06, 2016 at 10:07:51AM +0800, Wangnan (F) escreveu:
> On 2016/12/6 5:48, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Dec 05, 2016 at 07:02:48PM -0200, Arnaldo Carvalho de Melo escreveu:
> > > Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu:
> > > > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode.
> > > > Try llvm-config --build-mode --assertion-mode
> > > > it should be Release OFF
> > > Probably this was with 3.9 and built from git, quite a while ago, now I
> > > removed it from /usr/local/ and installed what is in f25, but I fear it
> > > will be insufficient, does 3.8 cuts it for what we're testing? Humm, it
> > > looks like it will:

> > > [root@jouet ~]# llc --version
> > > LLVM (http://llvm.org/):
> > >    LLVM version 3.8.0

> > > But I'm now running the container based tests to send a pull req, will
> > > check later, after that.
> > Not really, Wang, we need to update that feature detection test to state what
> > is the minimum required LLVM/clang version, one that has those functions,
> > which, unfortunately, isn't the one in the latest fedora, fedora 25:
 
> I'll set the minimum required LLVM version to 3.9, and report
> warning when LLVM is too old. However, since LLVM interface is
> keep changing, finally we will have problem if we want to support
> 2 or 3 different clang/LLVM. We should keep moving minimum
> requirement LLVM version if we don't want to see '#ifdef's
> spread in our code.

If this area is in that much flux, I see no problem in us both not
enabling this by default, which is the case right now, and go on moving
the minimum required version for LLVM/clang, hopefully at some point
this will get stable and widely available (as in what distros ship),
when we then switch to doing the feature detection automatically.

I see you already submitted the patch to do this test, thanks, will
check and continue processing your series.

- Arnaldo

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu:
> On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote:
> > Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu:
> > > Add basic clang support in clang.cpp and test__clang() testcase. The
> > > first testcase checks if builtin clang is able to generate LLVM IR.
> > > 
> > > tests/clang.c is a proxy. Real testcase resides in
> > > utils/c++/clang-test.cpp in c++ and exports C interface to perf test
> > > subsystem.
> > > 
> > > Test result:
> > > 
> > >    $ perf test -v clang
> > >    51: Test builtin clang support                               :
> > >    51.1: Test builtin clang compile C source to IR              :
> > >    --- start ---
> > >    test child forked, pid 13215
> > >    test child finished with 0
> > >    ---- end ----
> > >    Test builtin clang support subtest 0: Ok
> > 
> > While testing this I noticed that the perf binary got huge, can't this
> > be done in some other way, i.e. using dynamic library?
> > 
> > [root@jouet ~]# size /tmp/perf
> >    text	   data	    bss	    dec	    hex	filename
> > 75333688	1421584	23962176	100717448	600d388 /tmp/perf
> > [root@jouet ~]#
> > 
> > I've built it with this:
> > 
> >   make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin
> > 
> > the resulting binary:
> > 
> > [acme@jouet linux]$ 
> > [acme@jouet linux]$ ls -la ~/bin/perf
> > -rwxr-xr-x. 2 acme acme 131689136 Dec  2 12:31 /home/acme/bin/perf
> > [acme@jouet linux]$ ls -lah ~/bin/perf
> > -rwxr-xr-x. 2 acme acme 126M Dec  2 12:31 /home/acme/bin/perf
> > [acme@jouet linux]$
> > 
> > Huge, after stripping it:
> > 
> > [acme@jouet linux]$ ls -la /tmp/perf
> > -rwxr-xr-x. 1 root root 76759056 Dec  2 12:33 /tmp/perf
> > [acme@jouet linux]$ ls -lah /tmp/perf
> > -rwxr-xr-x. 1 root root 74M Dec  2 12:33 /tmp/perf
> > [acme@jouet linux]$
> > 
> > Still huge :-\
> 
> yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode.
> Try llvm-config --build-mode --assertion-mode
> it should be Release OFF

It was ON, rebuilding it with

$ cmake -DCMAKE_BUILD_TYPE=Release /home/acme/git/llvm
$ make -j4

Will take a (another) while :-)

- Arnaldo
 
> > But being a n00b on llvm/clang libraries, etc, my question goes back to:
> > can we have this using a libllvm.so or libclang.so dynamic libraries?
> 
> that can also work. The reason we build iovisor/bcc into single binary
> is to ease operational headache.

Re: [PATCH v3 10/30] perf clang: Add builtin clang support ant test case

[Arnaldo Carvalho de Melo]

Em Tue, Dec 06, 2016 at 04:19:34PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu:
> > On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote:
> > >   make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin
> > > 
> > > the resulting binary:
> > > 
> > > [acme@jouet linux]$ 
> > > [acme@jouet linux]$ ls -la ~/bin/perf
> > > -rwxr-xr-x. 2 acme acme 131689136 Dec  2 12:31 /home/acme/bin/perf
> > > [acme@jouet linux]$ ls -lah ~/bin/perf
> > > -rwxr-xr-x. 2 acme acme 126M Dec  2 12:31 /home/acme/bin/perf
> > > [acme@jouet linux]$
> > > 
> > > Huge, after stripping it:
> > > 
> > > [acme@jouet linux]$ ls -la /tmp/perf
> > > -rwxr-xr-x. 1 root root 76759056 Dec  2 12:33 /tmp/perf
> > > [acme@jouet linux]$ ls -lah /tmp/perf
> > > -rwxr-xr-x. 1 root root 74M Dec  2 12:33 /tmp/perf
> > > [acme@jouet linux]$
> > > 
> > > Still huge :-\
> > 
> > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode.
> > Try llvm-config --build-mode --assertion-mode
> > it should be Release OFF
> 
> It was ON, rebuilding it with
> 
> $ cmake -DCMAKE_BUILD_TYPE=Release /home/acme/git/llvm
> $ make -j4
> 
> Will take a (another) while :-)

Almost halved the size:

[acme@jouet build]$ llvm-config --build-mode --assertion-mode
Release
OFF
[acme@jouet build]$ clang -v
clang version 4.0.0 (http://llvm.org/git/clang.git b7a0d79a6691813bf7d6ade1b4e8b21fd502f50a) (http://llvm.org/git/llvm.git 3a783f8716bfc621355e8ae61daf3a2093c341fc)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/6.2.1
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/6.2.1
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
[acme@jouet build]$ llvm-config --version
4.0.0svn
[acme@jouet build]$

[acme@jouet linux]$ ls -la /tmp/perf
-rwxr-xr-x. 1 acme acme 63641016 Dec  6 17:58 /tmp/perf
[acme@jouet linux]$ size /tmp/perf
   text	   data	    bss	    dec	    hex	filename
40877921	1577232	24065712	66520865	3f70721
/tmp/perf
[acme@jouet linux]$ strip /tmp/perf
[acme@jouet linux]$ ls -la /tmp/perf
-rwxr-xr-x. 1 acme acme 42460904 Dec  6 17:58 /tmp/perf
[acme@jouet linux]$ ls -lah /tmp/perf
-rwxr-xr-x. 1 acme acme 41M Dec  6 17:58 /tmp/perf
[acme@jouet linux]$ size /tmp/perf
   text	   data	    bss	    dec	    hex	filename
40877921	1577232	24065712	66520865	3f70721
/tmp/perf
[acme@jouet linux]$