* [Buildroot] [PATCH 1/2] package/llvm: add BPF backend
@ 2019-01-13 21:21 Romain Naour
2019-01-13 21:21 ` [Buildroot] [PATCH 2/2] package/bcc: new package Romain Naour
0 siblings, 1 reply; 13+ messages in thread
From: Romain Naour @ 2019-01-13 21:21 UTC (permalink / raw)
To: buildroot
From: Romain Naour <romain.naour@smile.fr>
The BPF backend is needed for bcc tool.
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Valentin Korenblit <valentinkorenblit@gmail.com>
---
package/llvm/Config.in | 6 ++++++
package/llvm/llvm.mk | 5 +++++
2 files changed, 11 insertions(+)
diff --git a/package/llvm/Config.in b/package/llvm/Config.in
index 83e94660eb..72d920e226 100644
--- a/package/llvm/Config.in
+++ b/package/llvm/Config.in
@@ -35,6 +35,12 @@ config BR2_PACKAGE_LLVM_AMDGPU
Build AMDGPU target. Select this option if you are going
to install mesa3d with llvm and use Gallium Radeon driver.
+config BR2_PACKAGE_LLVM_BPF
+ bool "BPF backend"
+ help
+ Build BPF target. Select this option if you are going
+ to install bcc on the target.
+
endif
comment "llvm needs a toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8, dynamic library"
diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk
index 613f5f5d47..9999389651 100644
--- a/package/llvm/llvm.mk
+++ b/package/llvm/llvm.mk
@@ -54,6 +54,11 @@ ifeq ($(BR2_PACKAGE_LLVM_AMDGPU),y)
LLVM_TARGETS_TO_BUILD += AMDGPU
endif
+# Build BPF backend
+ifeq ($(BR2_PACKAGE_LLVM_BPF),y)
+LLVM_TARGETS_TO_BUILD += BPF
+endif
+
# Use native llvm-tblgen from host-llvm (needed for cross-compilation)
LLVM_CONF_OPTS += -DLLVM_TABLEGEN=$(HOST_DIR)/bin/llvm-tblgen
--
2.14.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2019-01-13 21:21 [Buildroot] [PATCH 1/2] package/llvm: add BPF backend Romain Naour
@ 2019-01-13 21:21 ` Romain Naour
2019-01-14 19:40 ` Romain Naour
2019-01-14 20:43 ` Matthew Weber
0 siblings, 2 replies; 13+ messages in thread
From: Romain Naour @ 2019-01-13 21:21 UTC (permalink / raw)
To: buildroot
From: Romain Naour <romain.naour@smile.fr>
Before using bcc, you need to copy the kernel source code to
"/lib/module/<kernel version>/build" on the target.
That's because the clang frontend build eBPF code at runtime.
There are some issues with this frontend due to the hardcoded
architecture support and the use of GNU tuple (x86_64-unknown-linux-gnu).
That's why the bcc package dependency is limited to Glibc toolchain.
bcc needs tracing support, at least CONFIG_KPROBES, CONFIG_BPFILTER
and CONFIG_BPF_JIT. See [1] for details.
bcc needs BPF target support in llvm libraries.
Add a patch disabling man pages.
[1] https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
---
The needs of kernel sources on the target is anoying for small devices.
At least, I'm able to complete the bcc tutorial [1] on my x86_64 target.
[1] https://github.com/iovisor/bcc/blob/master/docs/tutorial.md
---
DEVELOPERS | 1 +
package/Config.in | 1 +
...CMake-Allow-to-disable-building-man-pages.patch | 35 +++++++++++++++
package/bcc/Config.in | 50 ++++++++++++++++++++++
package/bcc/bcc.hash | 3 ++
package/bcc/bcc.mk | 23 ++++++++++
6 files changed, 113 insertions(+)
create mode 100644 package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
create mode 100644 package/bcc/Config.in
create mode 100644 package/bcc/bcc.hash
create mode 100644 package/bcc/bcc.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index 0d4c4f1d60..f5dc078c69 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1831,6 +1831,7 @@ F: package/vnstat/
N: Romain Naour <romain.naour@gmail.com>
F: package/aubio/
+F: package/bcc/
F: package/bullet/
F: package/darktable/
F: package/efl/
diff --git a/package/Config.in b/package/Config.in
index 8d05303472..434427f455 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -79,6 +79,7 @@ menu "Compressors and decompressors"
endmenu
menu "Debugging, profiling and benchmark"
+ source "package/bcc/Config.in"
source "package/blktrace/Config.in"
source "package/bonnie/Config.in"
source "package/cache-calibrator/Config.in"
diff --git a/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch b/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
new file mode 100644
index 0000000000..c5c87d52e9
--- /dev/null
+++ b/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
@@ -0,0 +1,35 @@
+From a99f097aedbf964356e62c0c4bba1257f108c3eb Mon Sep 17 00:00:00 2001
+From: Romain Naour <romain.naour@gmail.com>
+Date: Fri, 7 Dec 2018 22:36:21 +0100
+Subject: [PATCH] CMake: Allow to disable building man pages
+
+Signed-off-by: Romain Naour <romain.naour@gmail.com>
+---
+ CMakeLists.txt | 3 +++
+ 1 file changed, 3 insertions(+)
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index bb3f53b9..5e95d555 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -19,6 +19,7 @@ include(cmake/FindCompilerFlag.cmake)
+ option(ENABLE_LLVM_SHARED "Enable linking LLVM as a shared library" OFF)
+ option(ENABLE_CLANG_JIT "Enable Loading BPF through Clang Frontend" ON)
+ option(ENABLE_USDT "Enable User-level Statically Defined Tracing" ON)
++option(ENABLE_MAN "Build man pages" ON)
+ CMAKE_DEPENDENT_OPTION(ENABLE_CPP_API "Enable C++ API" ON "ENABLE_USDT" OFF)
+
+ set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_SOURCE_DIR}/cmake)
+@@ -88,7 +89,9 @@ add_subdirectory(src)
+ add_subdirectory(introspection)
+ if(ENABLE_CLANG_JIT)
+ add_subdirectory(examples)
++if(ENABLE_MAN)
+ add_subdirectory(man)
++endif(ENABLE_MAN)
+ add_subdirectory(tests)
+ add_subdirectory(tools)
+ endif(ENABLE_CLANG_JIT)
+--
+2.14.5
+
diff --git a/package/bcc/Config.in b/package/bcc/Config.in
new file mode 100644
index 0000000000..7bc681e639
--- /dev/null
+++ b/package/bcc/Config.in
@@ -0,0 +1,50 @@
+config BR2_PACKAGE_BCC
+ bool "bcc"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+ depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
+ depends on BR2_LINUX_KERNEL # needs kernel sources on the target
+ depends on BR2_HOST_GCC_AT_LEAST_4_8 # clang
+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
+ depends on BR2_TOOLCHAIN_HAS_THREADS # clang
+ depends on BR2_INSTALL_LIBSTDCPP # clang
+ depends on !BR2_TOOLCHAIN_HAS_GCC_BUG_64735 # clang
+ depends on !BR2_STATIC_LIBS # clang, luajit
+ depends on BR2_USE_WCHAR # clang
+ select BR2_PACKAGE_CLANG
+ select BR2_PACKAGE_ELFUTILS
+ select BR2_PACKAGE_FLEX # needs FlexLexer.h
+ select BR2_PACKAGE_LLVM_BPF
+ select BR2_PACKAGE_LUAJIT
+ help
+ BPF Compiler Collection (BCC)
+
+ BCC is a toolkit for creating efficient kernel tracing and
+ manipulation programs, and includes several useful tools and
+ examples. It makes use of extended BPF (Berkeley Packet
+ Filters), formally known as eBPF, a new feature that was
+ first added to Linux 3.15. Much of what BCC uses requires
+ Linux 4.1 and above.
+
+ Note: Before using bcc, you need to copy the kernel source
+ code to the target. By default bcc use:
+ "/lib/module/<kernel version>/build". That's because the
+ clang frontend build eBPF code at runtime.
+
+ https://github.com/iovisor/bcc
+
+comment "bcc needs a Linux kernel to be built"
+ depends on !BR2_LINUX_KERNEL
+
+comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8, dynamic library"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_LINUX_KERNEL
+ depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
+ || !BR2_INSTALL_LIBSTDCPP || !BR2_HOST_GCC_AT_LEAST_4_8 \
+ || !BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 || BR2_STATIC_LIBS \
+ || !BR2_USE_WCHAR
+
+comment "bcc needs a toolchain not affected by GCC bug 64735"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_LINUX_KERNEL
+ depends on BR2_TOOLCHAIN_HAS_GCC_BUG_64735
diff --git a/package/bcc/bcc.hash b/package/bcc/bcc.hash
new file mode 100644
index 0000000000..19eccb37ca
--- /dev/null
+++ b/package/bcc/bcc.hash
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 269949a47f9ed2ff01b5ebca20ec4a52341f7ff14b033b9dc1669b72a87af042 bcc-v0.8.0.tar.gz
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 LICENSE.txt
diff --git a/package/bcc/bcc.mk b/package/bcc/bcc.mk
new file mode 100644
index 0000000000..401727c68a
--- /dev/null
+++ b/package/bcc/bcc.mk
@@ -0,0 +1,23 @@
+################################################################################
+#
+# bcc
+#
+################################################################################
+
+BCC_VERSION = v0.8.0
+BCC_SITE = $(call github,iovisor,bcc,$(BCC_VERSION))
+BCC_LICENSE = Apache-2.0
+BCC_LICENSE_FILES = LICENSE.txt
+# libbcc.so and libbpf.so
+BCC_INSTALL_STAGING = YES
+
+BCC_DEPENDENCIES = host-bison host-flex clang elfutils flex llvm luajit
+
+# ENABLE_LLVM_SHARED=ON to use llvm.so.
+# Force REVISION otherwise bcc will use git describe to generate a version number.
+BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
+ -DREVISION=$(BCC_VERSION) \
+ -DENABLE_CLANG_JIT=ON \
+ -DENABLE_MAN=OFF
+
+$(eval $(cmake-package))
--
2.14.5
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2019-01-13 21:21 ` [Buildroot] [PATCH 2/2] package/bcc: new package Romain Naour
@ 2019-01-14 19:40 ` Romain Naour
2019-01-14 20:43 ` Matthew Weber
1 sibling, 0 replies; 13+ messages in thread
From: Romain Naour @ 2019-01-14 19:40 UTC (permalink / raw)
To: buildroot
Hi All,
Le 13/01/2019 ? 22:21, Romain Naour a ?crit?:
> From: Romain Naour <romain.naour@smile.fr>
>
> Before using bcc, you need to copy the kernel source code to
> "/lib/module/<kernel version>/build" on the target.
> That's because the clang frontend build eBPF code at runtime.
This PR seems interesting:
"This patch separates BPFModule clang- and runtime functionality."
https://github.com/iovisor/bcc/pull/2078
Best regards,
Romain
>
> There are some issues with this frontend due to the hardcoded
> architecture support and the use of GNU tuple (x86_64-unknown-linux-gnu).
> That's why the bcc package dependency is limited to Glibc toolchain.
>
> bcc needs tracing support, at least CONFIG_KPROBES, CONFIG_BPFILTER
> and CONFIG_BPF_JIT. See [1] for details.
>
> bcc needs BPF target support in llvm libraries.
> Add a patch disabling man pages.
>
> [1] https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
>
> Signed-off-by: Romain Naour <romain.naour@smile.fr>
> Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
> ---
> The needs of kernel sources on the target is anoying for small devices.
> At least, I'm able to complete the bcc tutorial [1] on my x86_64 target.
>
> [1] https://github.com/iovisor/bcc/blob/master/docs/tutorial.md
> ---
> DEVELOPERS | 1 +
> package/Config.in | 1 +
> ...CMake-Allow-to-disable-building-man-pages.patch | 35 +++++++++++++++
> package/bcc/Config.in | 50 ++++++++++++++++++++++
> package/bcc/bcc.hash | 3 ++
> package/bcc/bcc.mk | 23 ++++++++++
> 6 files changed, 113 insertions(+)
> create mode 100644 package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
> create mode 100644 package/bcc/Config.in
> create mode 100644 package/bcc/bcc.hash
> create mode 100644 package/bcc/bcc.mk
>
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 0d4c4f1d60..f5dc078c69 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -1831,6 +1831,7 @@ F: package/vnstat/
>
> N: Romain Naour <romain.naour@gmail.com>
> F: package/aubio/
> +F: package/bcc/
> F: package/bullet/
> F: package/darktable/
> F: package/efl/
> diff --git a/package/Config.in b/package/Config.in
> index 8d05303472..434427f455 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -79,6 +79,7 @@ menu "Compressors and decompressors"
> endmenu
>
> menu "Debugging, profiling and benchmark"
> + source "package/bcc/Config.in"
> source "package/blktrace/Config.in"
> source "package/bonnie/Config.in"
> source "package/cache-calibrator/Config.in"
> diff --git a/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch b/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
> new file mode 100644
> index 0000000000..c5c87d52e9
> --- /dev/null
> +++ b/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
> @@ -0,0 +1,35 @@
> +From a99f097aedbf964356e62c0c4bba1257f108c3eb Mon Sep 17 00:00:00 2001
> +From: Romain Naour <romain.naour@gmail.com>
> +Date: Fri, 7 Dec 2018 22:36:21 +0100
> +Subject: [PATCH] CMake: Allow to disable building man pages
> +
> +Signed-off-by: Romain Naour <romain.naour@gmail.com>
> +---
> + CMakeLists.txt | 3 +++
> + 1 file changed, 3 insertions(+)
> +
> +diff --git a/CMakeLists.txt b/CMakeLists.txt
> +index bb3f53b9..5e95d555 100644
> +--- a/CMakeLists.txt
> ++++ b/CMakeLists.txt
> +@@ -19,6 +19,7 @@ include(cmake/FindCompilerFlag.cmake)
> + option(ENABLE_LLVM_SHARED "Enable linking LLVM as a shared library" OFF)
> + option(ENABLE_CLANG_JIT "Enable Loading BPF through Clang Frontend" ON)
> + option(ENABLE_USDT "Enable User-level Statically Defined Tracing" ON)
> ++option(ENABLE_MAN "Build man pages" ON)
> + CMAKE_DEPENDENT_OPTION(ENABLE_CPP_API "Enable C++ API" ON "ENABLE_USDT" OFF)
> +
> + set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_SOURCE_DIR}/cmake)
> +@@ -88,7 +89,9 @@ add_subdirectory(src)
> + add_subdirectory(introspection)
> + if(ENABLE_CLANG_JIT)
> + add_subdirectory(examples)
> ++if(ENABLE_MAN)
> + add_subdirectory(man)
> ++endif(ENABLE_MAN)
> + add_subdirectory(tests)
> + add_subdirectory(tools)
> + endif(ENABLE_CLANG_JIT)
> +--
> +2.14.5
> +
> diff --git a/package/bcc/Config.in b/package/bcc/Config.in
> new file mode 100644
> index 0000000000..7bc681e639
> --- /dev/null
> +++ b/package/bcc/Config.in
> @@ -0,0 +1,50 @@
> +config BR2_PACKAGE_BCC
> + bool "bcc"
> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> + depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
> + depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
> + depends on BR2_LINUX_KERNEL # needs kernel sources on the target
> + depends on BR2_HOST_GCC_AT_LEAST_4_8 # clang
> + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
> + depends on BR2_TOOLCHAIN_HAS_THREADS # clang
> + depends on BR2_INSTALL_LIBSTDCPP # clang
> + depends on !BR2_TOOLCHAIN_HAS_GCC_BUG_64735 # clang
> + depends on !BR2_STATIC_LIBS # clang, luajit
> + depends on BR2_USE_WCHAR # clang
> + select BR2_PACKAGE_CLANG
> + select BR2_PACKAGE_ELFUTILS
> + select BR2_PACKAGE_FLEX # needs FlexLexer.h
> + select BR2_PACKAGE_LLVM_BPF
> + select BR2_PACKAGE_LUAJIT
> + help
> + BPF Compiler Collection (BCC)
> +
> + BCC is a toolkit for creating efficient kernel tracing and
> + manipulation programs, and includes several useful tools and
> + examples. It makes use of extended BPF (Berkeley Packet
> + Filters), formally known as eBPF, a new feature that was
> + first added to Linux 3.15. Much of what BCC uses requires
> + Linux 4.1 and above.
> +
> + Note: Before using bcc, you need to copy the kernel source
> + code to the target. By default bcc use:
> + "/lib/module/<kernel version>/build". That's because the
> + clang frontend build eBPF code at runtime.
> +
> + https://github.com/iovisor/bcc
> +
> +comment "bcc needs a Linux kernel to be built"
> + depends on !BR2_LINUX_KERNEL
> +
> +comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8, dynamic library"
> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> + depends on BR2_LINUX_KERNEL
> + depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
> + || !BR2_INSTALL_LIBSTDCPP || !BR2_HOST_GCC_AT_LEAST_4_8 \
> + || !BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 || BR2_STATIC_LIBS \
> + || !BR2_USE_WCHAR
> +
> +comment "bcc needs a toolchain not affected by GCC bug 64735"
> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> + depends on BR2_LINUX_KERNEL
> + depends on BR2_TOOLCHAIN_HAS_GCC_BUG_64735
> diff --git a/package/bcc/bcc.hash b/package/bcc/bcc.hash
> new file mode 100644
> index 0000000000..19eccb37ca
> --- /dev/null
> +++ b/package/bcc/bcc.hash
> @@ -0,0 +1,3 @@
> +# locally calculated
> +sha256 269949a47f9ed2ff01b5ebca20ec4a52341f7ff14b033b9dc1669b72a87af042 bcc-v0.8.0.tar.gz
> +sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 LICENSE.txt
> diff --git a/package/bcc/bcc.mk b/package/bcc/bcc.mk
> new file mode 100644
> index 0000000000..401727c68a
> --- /dev/null
> +++ b/package/bcc/bcc.mk
> @@ -0,0 +1,23 @@
> +################################################################################
> +#
> +# bcc
> +#
> +################################################################################
> +
> +BCC_VERSION = v0.8.0
> +BCC_SITE = $(call github,iovisor,bcc,$(BCC_VERSION))
> +BCC_LICENSE = Apache-2.0
> +BCC_LICENSE_FILES = LICENSE.txt
> +# libbcc.so and libbpf.so
> +BCC_INSTALL_STAGING = YES
> +
> +BCC_DEPENDENCIES = host-bison host-flex clang elfutils flex llvm luajit
> +
> +# ENABLE_LLVM_SHARED=ON to use llvm.so.
> +# Force REVISION otherwise bcc will use git describe to generate a version number.
> +BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
> + -DREVISION=$(BCC_VERSION) \
> + -DENABLE_CLANG_JIT=ON \
> + -DENABLE_MAN=OFF
> +
> +$(eval $(cmake-package))
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2019-01-13 21:21 ` [Buildroot] [PATCH 2/2] package/bcc: new package Romain Naour
2019-01-14 19:40 ` Romain Naour
@ 2019-01-14 20:43 ` Matthew Weber
2019-01-14 21:00 ` Romain Naour
1 sibling, 1 reply; 13+ messages in thread
From: Matthew Weber @ 2019-01-14 20:43 UTC (permalink / raw)
To: buildroot
Romain,
On Sun, Jan 13, 2019 at 3:21 PM Romain Naour <romain.naour@gmail.com> wrote:
>
> From: Romain Naour <romain.naour@smile.fr>
>
> Before using bcc, you need to copy the kernel source code to
> "/lib/module/<kernel version>/build" on the target.
> That's because the clang frontend build eBPF code at runtime.
>
> There are some issues with this frontend due to the hardcoded
> architecture support and the use of GNU tuple (x86_64-unknown-linux-gnu).
> That's why the bcc package dependency is limited to Glibc toolchain.
>
> bcc needs tracing support, at least CONFIG_KPROBES, CONFIG_BPFILTER
> and CONFIG_BPF_JIT. See [1] for details.
>
> bcc needs BPF target support in llvm libraries.
> Add a patch disabling man pages.
>
> [1] https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
>
> Signed-off-by: Romain Naour <romain.naour@smile.fr>
> Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
> ---
> The needs of kernel sources on the target is anoying for small devices.
> At least, I'm able to complete the bcc tutorial [1] on my x86_64 target.
Suggest updating LINUX_KCONFIG_FIXUP_CMDS to include enabling of the
BPF features when this package is selected.
>
> [1] https://github.com/iovisor/bcc/blob/master/docs/tutorial.md
> ---
> DEVELOPERS | 1 +
> package/Config.in | 1 +
> ...CMake-Allow-to-disable-building-man-pages.patch | 35 +++++++++++++++
> package/bcc/Config.in | 50 ++++++++++++++++++++++
> package/bcc/bcc.hash | 3 ++
> package/bcc/bcc.mk | 23 ++++++++++
> 6 files changed, 113 insertions(+)
> create mode 100644 package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
> create mode 100644 package/bcc/Config.in
> create mode 100644 package/bcc/bcc.hash
> create mode 100644 package/bcc/bcc.mk
>
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 0d4c4f1d60..f5dc078c69 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -1831,6 +1831,7 @@ F: package/vnstat/
>
> N: Romain Naour <romain.naour@gmail.com>
> F: package/aubio/
> +F: package/bcc/
> F: package/bullet/
> F: package/darktable/
> F: package/efl/
> diff --git a/package/Config.in b/package/Config.in
> index 8d05303472..434427f455 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -79,6 +79,7 @@ menu "Compressors and decompressors"
> endmenu
>
> menu "Debugging, profiling and benchmark"
> + source "package/bcc/Config.in"
> source "package/blktrace/Config.in"
> source "package/bonnie/Config.in"
> source "package/cache-calibrator/Config.in"
> diff --git a/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch b/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
> new file mode 100644
> index 0000000000..c5c87d52e9
> --- /dev/null
> +++ b/package/bcc/0001-CMake-Allow-to-disable-building-man-pages.patch
> @@ -0,0 +1,35 @@
> +From a99f097aedbf964356e62c0c4bba1257f108c3eb Mon Sep 17 00:00:00 2001
> +From: Romain Naour <romain.naour@gmail.com>
> +Date: Fri, 7 Dec 2018 22:36:21 +0100
> +Subject: [PATCH] CMake: Allow to disable building man pages
> +
> +Signed-off-by: Romain Naour <romain.naour@gmail.com>
> +---
> + CMakeLists.txt | 3 +++
> + 1 file changed, 3 insertions(+)
> +
> +diff --git a/CMakeLists.txt b/CMakeLists.txt
> +index bb3f53b9..5e95d555 100644
> +--- a/CMakeLists.txt
> ++++ b/CMakeLists.txt
> +@@ -19,6 +19,7 @@ include(cmake/FindCompilerFlag.cmake)
> + option(ENABLE_LLVM_SHARED "Enable linking LLVM as a shared library" OFF)
> + option(ENABLE_CLANG_JIT "Enable Loading BPF through Clang Frontend" ON)
> + option(ENABLE_USDT "Enable User-level Statically Defined Tracing" ON)
> ++option(ENABLE_MAN "Build man pages" ON)
> + CMAKE_DEPENDENT_OPTION(ENABLE_CPP_API "Enable C++ API" ON "ENABLE_USDT" OFF)
> +
> + set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_SOURCE_DIR}/cmake)
> +@@ -88,7 +89,9 @@ add_subdirectory(src)
> + add_subdirectory(introspection)
> + if(ENABLE_CLANG_JIT)
> + add_subdirectory(examples)
> ++if(ENABLE_MAN)
> + add_subdirectory(man)
> ++endif(ENABLE_MAN)
> + add_subdirectory(tests)
> + add_subdirectory(tools)
> + endif(ENABLE_CLANG_JIT)
> +--
> +2.14.5
> +
> diff --git a/package/bcc/Config.in b/package/bcc/Config.in
> new file mode 100644
> index 0000000000..7bc681e639
> --- /dev/null
> +++ b/package/bcc/Config.in
> @@ -0,0 +1,50 @@
> +config BR2_PACKAGE_BCC
> + bool "bcc"
> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> + depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
> + depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
> + depends on BR2_LINUX_KERNEL # needs kernel sources on the target
> + depends on BR2_HOST_GCC_AT_LEAST_4_8 # clang
> + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
> + depends on BR2_TOOLCHAIN_HAS_THREADS # clang
> + depends on BR2_INSTALL_LIBSTDCPP # clang
> + depends on !BR2_TOOLCHAIN_HAS_GCC_BUG_64735 # clang
> + depends on !BR2_STATIC_LIBS # clang, luajit
> + depends on BR2_USE_WCHAR # clang
> + select BR2_PACKAGE_CLANG
> + select BR2_PACKAGE_ELFUTILS
> + select BR2_PACKAGE_FLEX # needs FlexLexer.h
> + select BR2_PACKAGE_LLVM_BPF
> + select BR2_PACKAGE_LUAJIT
> + help
> + BPF Compiler Collection (BCC)
> +
> + BCC is a toolkit for creating efficient kernel tracing and
> + manipulation programs, and includes several useful tools and
> + examples. It makes use of extended BPF (Berkeley Packet
> + Filters), formally known as eBPF, a new feature that was
> + first added to Linux 3.15. Much of what BCC uses requires
> + Linux 4.1 and above.
> +
> + Note: Before using bcc, you need to copy the kernel source
> + code to the target. By default bcc use:
> + "/lib/module/<kernel version>/build". That's because the
> + clang frontend build eBPF code at runtime.
Very interesting patch and package. Thanks for sending it in. I know
I've added it to my list to tinker with.
Matt
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2019-01-14 20:43 ` Matthew Weber
@ 2019-01-14 21:00 ` Romain Naour
2019-03-14 17:15 ` Romain Naour
0 siblings, 1 reply; 13+ messages in thread
From: Romain Naour @ 2019-01-14 21:00 UTC (permalink / raw)
To: buildroot
Hi Matt,
Le 14/01/2019 ? 21:43, Matthew Weber a ?crit?:
> Romain,
>
>
> On Sun, Jan 13, 2019 at 3:21 PM Romain Naour <romain.naour@gmail.com> wrote:
>>
>> From: Romain Naour <romain.naour@smile.fr>
>>
>> Before using bcc, you need to copy the kernel source code to
>> "/lib/module/<kernel version>/build" on the target.
>> That's because the clang frontend build eBPF code at runtime.
>>
>> There are some issues with this frontend due to the hardcoded
>> architecture support and the use of GNU tuple (x86_64-unknown-linux-gnu).
>> That's why the bcc package dependency is limited to Glibc toolchain.
>>
>> bcc needs tracing support, at least CONFIG_KPROBES, CONFIG_BPFILTER
>> and CONFIG_BPF_JIT. See [1] for details.
>>
>> bcc needs BPF target support in llvm libraries.
>> Add a patch disabling man pages.
>>
>> [1] https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
>>
>> Signed-off-by: Romain Naour <romain.naour@smile.fr>
>> Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
>> ---
>> The needs of kernel sources on the target is anoying for small devices.
>> At least, I'm able to complete the bcc tutorial [1] on my x86_64 target.
>
> Suggest updating LINUX_KCONFIG_FIXUP_CMDS to include enabling of the
> BPF features when this package is selected.
Indeed, I thought about it but I need to double check the kernel options that
are mandatory.
>
>>
>> [1] https://github.com/iovisor/bcc/blob/master/docs/tutorial.md
>> ---
[...]
>> + help
>> + BPF Compiler Collection (BCC)
>> +
>> + BCC is a toolkit for creating efficient kernel tracing and
>> + manipulation programs, and includes several useful tools and
>> + examples. It makes use of extended BPF (Berkeley Packet
>> + Filters), formally known as eBPF, a new feature that was
>> + first added to Linux 3.15. Much of what BCC uses requires
>> + Linux 4.1 and above.
>> +
>> + Note: Before using bcc, you need to copy the kernel source
>> + code to the target. By default bcc use:
>> + "/lib/module/<kernel version>/build". That's because the
>> + clang frontend build eBPF code at runtime.
>
> Very interesting patch and package. Thanks for sending it in. I know
> I've added it to my list to tinker with.
You're welcome :)
My colleague Jugurtha was working on debugging tools like bcc during his
internship. We are experimenting bcc in the context of Buildroot.
Also, it allow to have (yet) another use case for llvm/clang merged this year
thanks to Valentin's work.
Best regards,
Romain
>
> Matt
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2019-01-14 21:00 ` Romain Naour
@ 2019-03-14 17:15 ` Romain Naour
0 siblings, 0 replies; 13+ messages in thread
From: Romain Naour @ 2019-03-14 17:15 UTC (permalink / raw)
To: buildroot
Hi Jugurtha,
Le 14/01/2019 ? 22:00, Romain Naour a ?crit?:
> Hi Matt,
>
> Le 14/01/2019 ? 21:43, Matthew Weber a ?crit?:
>> Romain,
>>
>>
>> On Sun, Jan 13, 2019 at 3:21 PM Romain Naour <romain.naour@gmail.com> wrote:
>>>
>>> From: Romain Naour <romain.naour@smile.fr>
>>>
>>> Before using bcc, you need to copy the kernel source code to
>>> "/lib/module/<kernel version>/build" on the target.
>>> That's because the clang frontend build eBPF code at runtime.
>>>
>>> There are some issues with this frontend due to the hardcoded
>>> architecture support and the use of GNU tuple (x86_64-unknown-linux-gnu).
>>> That's why the bcc package dependency is limited to Glibc toolchain.
>>>
>>> bcc needs tracing support, at least CONFIG_KPROBES, CONFIG_BPFILTER
>>> and CONFIG_BPF_JIT. See [1] for details.
>>>
>>> bcc needs BPF target support in llvm libraries.
>>> Add a patch disabling man pages.
>>>
>>> [1] https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md
>>>
>>> Signed-off-by: Romain Naour <romain.naour@smile.fr>
>>> Cc: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
>>> ---
>>> The needs of kernel sources on the target is anoying for small devices.
>>> At least, I'm able to complete the bcc tutorial [1] on my x86_64 target.
>>
>> Suggest updating LINUX_KCONFIG_FIXUP_CMDS to include enabling of the
>> BPF features when this package is selected.
>
> Indeed, I thought about it but I need to double check the kernel options that
> are mandatory.
>
>>
>>>
>>> [1] https://github.com/iovisor/bcc/blob/master/docs/tutorial.md
>>> ---
>
> [...]
>
>>> + help
>>> + BPF Compiler Collection (BCC)
>>> +
>>> + BCC is a toolkit for creating efficient kernel tracing and
>>> + manipulation programs, and includes several useful tools and
>>> + examples. It makes use of extended BPF (Berkeley Packet
>>> + Filters), formally known as eBPF, a new feature that was
>>> + first added to Linux 3.15. Much of what BCC uses requires
>>> + Linux 4.1 and above.
>>> +
>>> + Note: Before using bcc, you need to copy the kernel source
>>> + code to the target. By default bcc use:
>>> + "/lib/module/<kernel version>/build". That's because the
>>> + clang frontend build eBPF code at runtime.
>>
>> Very interesting patch and package. Thanks for sending it in. I know
>> I've added it to my list to tinker with.
>
> You're welcome :)
>
> My colleague Jugurtha was working on debugging tools like bcc during his
> internship. We are experimenting bcc in the context of Buildroot.
Jugurtha tried today to build an image including BCC for a rpi3 for aarch64 and ARM.
For now, we can't enable BCC for aarch64 due to the luajit dependency (luajit
2.0.5 doesn't support aarch64 yet).
He tried to build for ARM and get the following error:
https://lists.iovisor.org/g/iovisor-dev/message/1610
It seems that BCC's User-level Statically Defined Tracing (USDT) only support a
limited number of architecture (aarch64, powerpc64 and x86_64).
Also, during the investigation we discover that luajit (from the host) is used
to build bcc.o, even while cross-compiling.
/usr/bin/luajit -bg bcc.lua bcc.o [1]
Which produce x86_64 binary on x86_64 host.
$ file ./lua/bcc.o
./lua/bcc.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
So we got an error while linking BCC.
In the end, BCC doesn't seem cross-compilation friendly :-/
BPFd seems an interesting alternative in the context of Buildroot [2].
Best regards,
Romain
[1] https://github.com/iovisor/bcc/blob/master/src/lua/CMakeLists.txt#L18
[2] https://lwn.net/Articles/744522/
>
> Also, it allow to have (yet) another use case for llvm/clang merged this year
> thanks to Valentin's work.
>
> Best regards,
> Romain
>
>>
>> Matt
>>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2020-11-03 22:34 ` Romain Naour
@ 2020-11-24 16:41 ` Qais Yousef
0 siblings, 0 replies; 13+ messages in thread
From: Qais Yousef @ 2020-11-24 16:41 UTC (permalink / raw)
To: buildroot
Hi Romain
On 11/03/20 23:34, Romain Naour wrote:
> Hello Qais,
>
> Sorry for my late reply.
And sorry for my late reply too. I was off for 2 weeks.
>
> Le 04/10/2020 ??? 20:12, Qais Yousef a ???crit???:
> > Hi Jugurtha
> >
> > I am very late to the party and missed earlier reviews. So apologies in advance
> > if any of my comments was already brought up/addressed.
> >
> > Can you keep me on CC on future posting and any other related discussion
> > please?
>
> I'm reworking the series and I'm adding you in Cc for each patch.
> Jugurtha is not available at the moment, but he is still interested by BCC.
Great. Thanks for that. I will allocate time to review it soon.
>
> >
> > On 08/24/20 20:17, Jugurtha BELKALEM wrote:
> >> bcc is a front-end tool for eBPF :
> >> https://github.com/iovisor/bcc/blob/master/README.md.
> >> eBPF is the most powerful Linux tracer, and bcc
> >> allows to write eBPF scripts in C and PYTHON3.
> >>
> >> bcc can help to troubleshoot issues quickly on
> >> embedded systems (as long as Linux kernel
> >> version >= 4.1).
> >>
> >> bcc can also make it easy to create observabilty tools,
> >> SDN configuration, ddos mitigation, intrusion detection
> >> and secure containers. More information is available at:
> >> http://www.brendangregg.com/ebpf.html.
> >
> > There's the brand new ebpf website too
> >
> > http://www.ebpf.io
>
> Link updated, thanks!
>
> >
> >>
> >> BCC can be tested on the target :
> >> $ mount -t debugfs none /sys/kernel/debug
> >
> > I wonder if this something we can automatically append to /etc/fstab when this
> > package is enabled..
>
> Sure, but this is a fstab customization. I don't think the bcc package should
> change the /etc/fstab.
>
> >
> >> $ mkdir -p /lib/modules/KERNEL_VERSION/build
> >> $ cd /lib/modules/KERNEL_VERSION//build
> >> $ cp /sys/kernel/kheaders.tar.xz .
> >> $ unxz kheaders.tar.xz
> >> $ tar xf kheaders.tar
> >
> > All the above is unnecessary. BCC automatically does this if it detects
> > /sys/kernel/kheaders.tar.xz is present. I had to enable BR2_PACKAGE_TAR for BCC
> > to handle this correctly. I guess busybox::tar is too limited to work with BCC
> > commands.
>
> Indeed, thanks for the info.
> Now I see:
> https://github.com/iovisor/bcc/blob/v0.17.0/src/cc/frontends/clang/kbuild_helper.cc#L172
>
> What was the error you get with busybox's tar ?
Sadly I completely forgot. I can try to reproduce it. If you managed to resolve
this in a different way, it works for me.
> bcc doesn't use advenced tar option that is not supported by busybox's tar.
>
> # tar --help
> BusyBox v1.31.1 (2020-05-01 00:37:37 CEST) multi-call binary.
>
> Usage: tar c|x|t [-hvokO] [-f TARFILE] [-C DIR] [-T FILE] [-X FILE] [--exclude
> PATTERN]... [FILE]...
>
> Create, extract, or list files from a tar file
>
> c Create
> x Extract
> t List
> -f FILE Name of TARFILE ('-' for stdin/out)
> -C DIR Change to DIR before operation
> -v Verbose
> -O Extract to stdout
> -o Don't restore user:group
> -k Don't replace existing files
> -h Follow symlinks
> -T FILE File with names to include
> -X FILE File with glob patterns to exclude
> --exclude PATTERN Glob pattern to exclude
>
[...]
> >> menu "Debugging, profiling and benchmark"
> >> + source "package/bcc/Config.in"
> >> source "package/blktrace/Config.in"
> >> source "package/bonnie/Config.in"
> >> source "package/cache-calibrator/Config.in"
> >> diff --git a/package/bcc/0001-fix-aarch64-cross-compile.patch b/package/bcc/0001-fix-aarch64-cross-compile.patch
> >> new file mode 100644
> >> index 0000000..6b42797
> >> --- /dev/null
> >> +++ b/package/bcc/0001-fix-aarch64-cross-compile.patch
> >> @@ -0,0 +1,65 @@
> >> +From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
> >> +From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
> >> +Date: Thu, 2 May 2019 11:06:23 +0200
> >> +Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
> >> + cross-compiling
> >
> > Is this luajit required to enable Lua front-end to BCC? Given python is the
> > most common front-end, I'd argue to make this an extra option that is only
> > enabled if the user selects it in the menu. Same way BPF backend in LLVM is
> > only compiled in if selected. IMHO that's a better default behavior.
>
> As far I can see in other packaging, BCC is using luajit as lua interpreter
> https://src.fedoraproject.org/rpms/bcc/blob/master/f/bcc.spec
>
> I'm agree that python is the most common front-end. I'll try to add a sub option
> to enable or disable it.
I think a sub option is the better way to go. Thanks!
[...]
> >> diff --git a/package/bcc/Config.in b/package/bcc/Config.in
> >> new file mode 100644
> >> index 0000000..cf03927
> >> --- /dev/null
> >> +++ b/package/bcc/Config.in
> >> @@ -0,0 +1,49 @@
> >> +config BR2_PACKAGE_BCC
> >> + bool "bcc"
> >> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> >> + depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
> >> + depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
> >> + depends on BR2_LINUX_KERNEL # needs kernel sources on the target
> >
> > This dependency on building the kernel in buildroot is not user friendly. For
> > example I never build the kernel in buildroot, and certainly would like to be
> > able to enable this package without having to build the kernel in buildroot.
>
> IIRC this option was adde because we used a previous version of BCC (0.8) with a
> kernel 4.x (at the time CONFIG_IKHEADERS didn't exist).
> But still, if we don't enforce building a kernel to select BCC it's the user
> responsibility to install the kernel sources (manually, rootfsoverlay or by a
> post build script).
>
> I'll remove this constrain.
Thanks. IMO relying on users reading the documentation to enable kernel
support is a better option.
>
> >
> >> + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
> >> + depends on BR2_TOOLCHAIN_HAS_THREADS # clang
> >> + depends on BR2_INSTALL_LIBSTDCPP # clang
> >
> >> + select BR2_PACKAGE_AUDIT # runtime
> >
> > This..
> >
> >> + select BR2_PACKAGE_CLANG
> >> + select BR2_PACKAGE_ELFUTILS
> >> + select BR2_PACKAGE_FLEX # needs FlexLexer.h
> >
> >> + select BR2_PACKAGE_IPERF3 # runtime
> >
> > .. this..
> >
> >> + select BR2_PACKAGE_LLVM_BPF
> >> + select BR2_PACKAGE_LUAJIT
> >
> >> + select BR2_PACKAGE_NETPERF # runtime
> >
> > .. and this look odd to me. Can you expand which runtime requires them?
>
> for example, netperf is one of recommanded program for BCC.
> They are listed in debian/control
> https://github.com/iovisor/bcc/blob/v0.17.0/debian/control#L13
>
> I don't remenber about audit, Jugurtha ?
I think this is a specific use case for a class of people who would like to use
BCC. I don't think we should impose netperf on them as a requirement. They can
enable it independently if they want it, no?
If you see a big value in enabling these packages automatically, can we hide
them behind another sub option at least?
Image bloat is a problem I faced in the past, and being able to have the bare
minimum only is always good to have.
>
> >
> >> + select BR2_PACKAGE_PYTHON3
> >> + select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC
> >
> > If you replace XZ with TAR, you should find that BCC automatically handles the
> > ikheader.tar.xz for you. There's now BPF Type Format (BTF) option in the kernel
> > that could replace the dependency on the headers. Haven't played much with that
> > yet though. BTF requires a tool called pahole. We can handle this support later
> > I guess.
>
> There is no package that provide pahole tool. Indeed this can be done later.
+1
>
> >
> >> + help
> >> + BPF Compiler Collection (BCC)
> >> +
> >> + BCC is a toolkit for creating efficient kernel tracing and
> >> + manipulation programs, and includes several useful tools and
> >> + examples. It makes use of extended BPF (Berkeley Packet
> >> + Filters), formally known as eBPF, a new feature that was
> >> + first added to Linux 3.15. Much of what BCC uses requires
> >> + Linux 4.1 and above.
> >> +
> >> + Note: Before using bcc, you need either need to :
> >> + - For kernel_ver = [4.1, 5.2) : Copy kernel source code
> >> + to target folder /lib/module/<kernel_ver>/build.
> >> + - Or kernel_ver >= 5.2 : Compile kernel with CONFIG_IKHEADERS
> >> + and use generated headers under /sys/kernel/kheaders.tar.xz
> >> + to populate /lib/module/<kernel_ver>/build.
> >> +
> >> + That's because the clang frontend build eBPF code at runtime.
> >> +
> >> + https://github.com/iovisor/bcc
> >> +
> >> +comment "bcc needs a Linux kernel to be built"
> >> + depends on !BR2_LINUX_KERNEL
> >
> > I must remove this to compile this patch. This hard dependency is not required
> > IMHO. You can still auto fix up the kernel if you like if the user is building
> > the kernel and has enabled BCC. Just keep in mind it's a fast moving target, so
> > the set of configs might require a bit of maintenance. Given it's all
> > documented in BCC website, I personally think it's better to leave correct
> > kernel configuration to the user to follow the right instructions from BCC
> > website rather than automatically handle it for them.
>
> You mean the bcc.mk shouldn't handler kernel options for BCC ?
Yep.
>
> Ok to remove this constraint on the Linux kernel.
Thanks!
[...]
> >
> >> +
> >> +define BCC_LINUX_CONFIG_FIXUPS
> >> + # Enable kernel support for eBPF
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
> >> + # [for Linux kernel versions 4.1 through 4.6]
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
> >> + # [for Linux kernel versions 4.7 and later]
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
> >> + # [for Linux kernel versions 5.2 and later]
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
> >> + # For running bcc networking examples on vanilla kernel
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
> >> + $(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
> >> +endef
> >
> > This will be a big maintenance headache. New dependency can come up and configs
> > like CONFIG_NET_* are not essential and seem to be specific to your usage.
> >
> > I'm not in favour of them personally, but if you want them the list need to
> > shrink to the bare minimum and depend on user building the kernel in buildroot
> > IMO.
>
> As you said, bcc is moving fast and requires different kernel option depending
> on the kernel version or tool/example that will be running on the target.
> Agree it's may be a big maintenance headache.
>
> But on otherhand I find userfriendly to enable CONFIG_BPF to CONFIG_IKHEADERS
> option. OK to drop kernel options for networking examples.
>
> Thanks for your review!
Thanks for taking care of the issues and sending a new version too! I'll review
as soon as I can.
Cheers
--
Qais Yousef
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2020-10-04 18:12 ` Qais Yousef
2020-10-08 22:48 ` Qais Yousef
@ 2020-11-03 22:34 ` Romain Naour
2020-11-24 16:41 ` Qais Yousef
1 sibling, 1 reply; 13+ messages in thread
From: Romain Naour @ 2020-11-03 22:34 UTC (permalink / raw)
To: buildroot
Hello Qais,
Sorry for my late reply.
Le 04/10/2020 ? 20:12, Qais Yousef a ?crit?:
> Hi Jugurtha
>
> I am very late to the party and missed earlier reviews. So apologies in advance
> if any of my comments was already brought up/addressed.
>
> Can you keep me on CC on future posting and any other related discussion
> please?
I'm reworking the series and I'm adding you in Cc for each patch.
Jugurtha is not available at the moment, but he is still interested by BCC.
>
> On 08/24/20 20:17, Jugurtha BELKALEM wrote:
>> bcc is a front-end tool for eBPF :
>> https://github.com/iovisor/bcc/blob/master/README.md.
>> eBPF is the most powerful Linux tracer, and bcc
>> allows to write eBPF scripts in C and PYTHON3.
>>
>> bcc can help to troubleshoot issues quickly on
>> embedded systems (as long as Linux kernel
>> version >= 4.1).
>>
>> bcc can also make it easy to create observabilty tools,
>> SDN configuration, ddos mitigation, intrusion detection
>> and secure containers. More information is available at:
>> http://www.brendangregg.com/ebpf.html.
>
> There's the brand new ebpf website too
>
> http://www.ebpf.io
Link updated, thanks!
>
>>
>> BCC can be tested on the target :
>> $ mount -t debugfs none /sys/kernel/debug
>
> I wonder if this something we can automatically append to /etc/fstab when this
> package is enabled..
Sure, but this is a fstab customization. I don't think the bcc package should
change the /etc/fstab.
>
>> $ mkdir -p /lib/modules/KERNEL_VERSION/build
>> $ cd /lib/modules/KERNEL_VERSION//build
>> $ cp /sys/kernel/kheaders.tar.xz .
>> $ unxz kheaders.tar.xz
>> $ tar xf kheaders.tar
>
> All the above is unnecessary. BCC automatically does this if it detects
> /sys/kernel/kheaders.tar.xz is present. I had to enable BR2_PACKAGE_TAR for BCC
> to handle this correctly. I guess busybox::tar is too limited to work with BCC
> commands.
Indeed, thanks for the info.
Now I see:
https://github.com/iovisor/bcc/blob/v0.17.0/src/cc/frontends/clang/kbuild_helper.cc#L172
What was the error you get with busybox's tar ?
bcc doesn't use advenced tar option that is not supported by busybox's tar.
# tar --help
BusyBox v1.31.1 (2020-05-01 00:37:37 CEST) multi-call binary.
Usage: tar c|x|t [-hvokO] [-f TARFILE] [-C DIR] [-T FILE] [-X FILE] [--exclude
PATTERN]... [FILE]...
Create, extract, or list files from a tar file
c Create
x Extract
t List
-f FILE Name of TARFILE ('-' for stdin/out)
-C DIR Change to DIR before operation
-v Verbose
-O Extract to stdout
-o Don't restore user:group
-k Don't replace existing files
-h Follow symlinks
-T FILE File with names to include
-X FILE File with glob patterns to exclude
--exclude PATTERN Glob pattern to exclude
>
>> $ cd /usr/share/bcc/examples/tracing
>> $ ./disksnoop.py
>
> /usr/share/bcc/tools contains actual tools. I recommend picking up an example
> from there.
Fixed.
>
>>
>> V3
>> - Bump to version 0.16.0.
>> - Add required python dependency.
>> - Remove unnecessary and duplicated
>> kernel flags (#Testing section) in bcc.mk.
>> -------------
>> V2
>> - Add eBPF's required Kernel flags.
>> - Fix submodule source fetch problem.
>> - Add toolchain dependency.
>
> Did you intend to include these in the commit message?
No, it's a mistake. Fixed.
>
>>
>> Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
>> ---
>> DEVELOPERS | 1 +
>> package/Config.in | 1 +
>> package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
>> package/bcc/Config.in | 49 ++++++++++++++++++
>> package/bcc/bcc.hash | 3 ++
>> package/bcc/bcc.mk | 57 +++++++++++++++++++++
>> 6 files changed, 176 insertions(+)
>> create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
>> create mode 100644 package/bcc/Config.in
>> create mode 100644 package/bcc/bcc.hash
>> create mode 100644 package/bcc/bcc.mk
>>
>> diff --git a/DEVELOPERS b/DEVELOPERS
>> index 42fa5a4..bd836a6 100644
>> --- a/DEVELOPERS
>> +++ b/DEVELOPERS
>> @@ -1396,6 +1396,7 @@ N: Joshua Henderson <joshua.henderson@microchip.com>
>> F: package/qt5/qt5wayland/
>>
>> N: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
>> +F: package/bcc/
>> F: package/python-cycler/
>> F: package/python-matplotlib/
>>
>> diff --git a/package/Config.in b/package/Config.in
>> index d7e79f4..8ec3932 100644
>> --- a/package/Config.in
>> +++ b/package/Config.in
>> @@ -83,6 +83,7 @@ menu "Compressors and decompressors"
>> endmenu
>>
>> menu "Debugging, profiling and benchmark"
>> + source "package/bcc/Config.in"
>> source "package/blktrace/Config.in"
>> source "package/bonnie/Config.in"
>> source "package/cache-calibrator/Config.in"
>> diff --git a/package/bcc/0001-fix-aarch64-cross-compile.patch b/package/bcc/0001-fix-aarch64-cross-compile.patch
>> new file mode 100644
>> index 0000000..6b42797
>> --- /dev/null
>> +++ b/package/bcc/0001-fix-aarch64-cross-compile.patch
>> @@ -0,0 +1,65 @@
>> +From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
>> +From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
>> +Date: Thu, 2 May 2019 11:06:23 +0200
>> +Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
>> + cross-compiling
>
> Is this luajit required to enable Lua front-end to BCC? Given python is the
> most common front-end, I'd argue to make this an extra option that is only
> enabled if the user selects it in the menu. Same way BPF backend in LLVM is
> only compiled in if selected. IMHO that's a better default behavior.
As far I can see in other packaging, BCC is using luajit as lua interpreter
https://src.fedoraproject.org/rpms/bcc/blob/master/f/bcc.spec
I'm agree that python is the most common front-end. I'll try to add a sub option
to enable or disable it.
>
>> +
>> +Unlike CMAKE_SYSTEM_PROCESSOR which identifies aarch64
>> +as a valid architecture, luajit does not recognize it.
>> +luajit defines aarch64 as arm64.
>> +
>> +LuaJIT doesn't use usual arch naming, so we have to convert
>> +between CMake and Luajit for each architectures while
>> +cross-compiling.
>> +
>> +Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
>> +Signed-off-by: Romain Naour <romain.naour@smile.fr>
>> +---
>> +v2: Do the same for other architecture supported by LuaJIT.
>> +https://github.com/iovisor/bcc/pull/2480
>> +---
>> + src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
>> + 1 file changed, 28 insertions(+), 1 deletion(-)
>> +
>> +diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
>> +index 7541d48df..226e1b1d2 100644
>> +--- a/src/lua/CMakeLists.txt
>> ++++ b/src/lua/CMakeLists.txt
>> +@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
>> + DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
>> + )
>> +
>> ++ # LuaJIT doesn't use usual arch naming, so we have to convert
>> ++ # between CMake and Luajit while cross-compiling.
>> ++ if (CMAKE_CROSSCOMPILING)
>> ++ SET (LUAJIT_TARGET_ARCH "-a")
>> ++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
>> ++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
>> ++ if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
>> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
>> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
>> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
>> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
>> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
>> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
>> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
>> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
>> ++ else()
>> ++ MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
>> ++ endif()
>> ++ endif()
>> ++
>> + ADD_CUSTOM_COMMAND(
>> + OUTPUT bcc.o
>> +- COMMAND ${LUAJIT} -bg bcc.lua bcc.o
>> ++ COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
>> + DEPENDS bcc.lua
>> + )
>> +
>> diff --git a/package/bcc/Config.in b/package/bcc/Config.in
>> new file mode 100644
>> index 0000000..cf03927
>> --- /dev/null
>> +++ b/package/bcc/Config.in
>> @@ -0,0 +1,49 @@
>> +config BR2_PACKAGE_BCC
>> + bool "bcc"
>> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
>> + depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
>> + depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
>> + depends on BR2_LINUX_KERNEL # needs kernel sources on the target
>
> This dependency on building the kernel in buildroot is not user friendly. For
> example I never build the kernel in buildroot, and certainly would like to be
> able to enable this package without having to build the kernel in buildroot.
IIRC this option was adde because we used a previous version of BCC (0.8) with a
kernel 4.x (at the time CONFIG_IKHEADERS didn't exist).
But still, if we don't enforce building a kernel to select BCC it's the user
responsibility to install the kernel sources (manually, rootfsoverlay or by a
post build script).
I'll remove this constrain.
>
>> + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
>> + depends on BR2_TOOLCHAIN_HAS_THREADS # clang
>> + depends on BR2_INSTALL_LIBSTDCPP # clang
>
>> + select BR2_PACKAGE_AUDIT # runtime
>
> This..
>
>> + select BR2_PACKAGE_CLANG
>> + select BR2_PACKAGE_ELFUTILS
>> + select BR2_PACKAGE_FLEX # needs FlexLexer.h
>
>> + select BR2_PACKAGE_IPERF3 # runtime
>
> .. this..
>
>> + select BR2_PACKAGE_LLVM_BPF
>> + select BR2_PACKAGE_LUAJIT
>
>> + select BR2_PACKAGE_NETPERF # runtime
>
> .. and this look odd to me. Can you expand which runtime requires them?
for example, netperf is one of recommanded program for BCC.
They are listed in debian/control
https://github.com/iovisor/bcc/blob/v0.17.0/debian/control#L13
I don't remenber about audit, Jugurtha ?
>
>> + select BR2_PACKAGE_PYTHON3
>> + select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC
>
> If you replace XZ with TAR, you should find that BCC automatically handles the
> ikheader.tar.xz for you. There's now BPF Type Format (BTF) option in the kernel
> that could replace the dependency on the headers. Haven't played much with that
> yet though. BTF requires a tool called pahole. We can handle this support later
> I guess.
There is no package that provide pahole tool. Indeed this can be done later.
>
>> + help
>> + BPF Compiler Collection (BCC)
>> +
>> + BCC is a toolkit for creating efficient kernel tracing and
>> + manipulation programs, and includes several useful tools and
>> + examples. It makes use of extended BPF (Berkeley Packet
>> + Filters), formally known as eBPF, a new feature that was
>> + first added to Linux 3.15. Much of what BCC uses requires
>> + Linux 4.1 and above.
>> +
>> + Note: Before using bcc, you need either need to :
>> + - For kernel_ver = [4.1, 5.2) : Copy kernel source code
>> + to target folder /lib/module/<kernel_ver>/build.
>> + - Or kernel_ver >= 5.2 : Compile kernel with CONFIG_IKHEADERS
>> + and use generated headers under /sys/kernel/kheaders.tar.xz
>> + to populate /lib/module/<kernel_ver>/build.
>> +
>> + That's because the clang frontend build eBPF code at runtime.
>> +
>> + https://github.com/iovisor/bcc
>> +
>> +comment "bcc needs a Linux kernel to be built"
>> + depends on !BR2_LINUX_KERNEL
>
> I must remove this to compile this patch. This hard dependency is not required
> IMHO. You can still auto fix up the kernel if you like if the user is building
> the kernel and has enabled BCC. Just keep in mind it's a fast moving target, so
> the set of configs might require a bit of maintenance. Given it's all
> documented in BCC website, I personally think it's better to leave correct
> kernel configuration to the user to follow the right instructions from BCC
> website rather than automatically handle it for them.
You mean the bcc.mk shouldn't handler kernel options for BCC ?
Ok to remove this constraint on the Linux kernel.
>
>> +
>> +comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8"
>> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
>> + depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
>> + depends on BR2_LINUX_KERNEL
>> + depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
>> + || !BR2_INSTALL_LIBSTDCPP
>> diff --git a/package/bcc/bcc.hash b/package/bcc/bcc.hash
>> new file mode 100644
>> index 0000000..e3fa662
>> --- /dev/null
>> +++ b/package/bcc/bcc.hash
>> @@ -0,0 +1,3 @@
>> +# locally calculated
>> +sha256 38105c6c7e53d6a8d4c2be5c2e6dd46ebeaa1489600501711d85cfe9bc065e9d bcc-v0.16.0.tar.gz
>> +sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 LICENSE.txt
>> diff --git a/package/bcc/bcc.mk b/package/bcc/bcc.mk
>> new file mode 100644
>> index 0000000..4fe69cb
>> --- /dev/null
>> +++ b/package/bcc/bcc.mk
>> @@ -0,0 +1,57 @@
>> +################################################################################
>> +#
>> +# bcc
>> +#
>> +################################################################################
>> +
>> +BCC_VERSION = v0.16.0
>> +BCC_SITE = https://github.com/iovisor/bcc.git
>> +BCC_SITE_METHOD = git
>> +BCC_GIT_SUBMODULES = YES
>> +BCC_LICENSE = Apache-2.0
>> +BCC_LICENSE_FILES = LICENSE.txt
>> +# libbcc.so and libbpf.so
>> +BCC_INSTALL_STAGING = YES
>> +BCC_DEPENDENCIES = host-bison host-flex host-luajit clang elfutils flex llvm \
>> + luajit python3
>> +
>> +# ENABLE_LLVM_SHARED=ON to use llvm.so.
>> +# Force REVISION otherwise bcc will use git describe to generate a version number.
>> +BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
>> + -DREVISION=$(BCC_VERSION) \
>> + -DENABLE_CLANG_JIT=ON \
>> + -DENABLE_MAN=OFF
>> +
>> +# BCC requires python-bcc (shiped under bcc/src/python) to work.
>> +# However, BCC compiles bcc/src/python/setup.py.in into bcc/src/python.
>> +# The generated compiled folder (bcc/src/python/bcc-python/bcc) should
>> +# be copied to target python's site-packages directory.
>> +define BCC_INSTALL_PYTHON_BCC
>> + cp -r $(BUILD_DIR)/bcc-$(BCC_VERSION)/src/python/bcc-python/bcc \
>> + $(TARGET_DIR)/usr/lib/python3.*/site-packages/
>> +endef
>> +BCC_POST_INSTALL_TARGET_HOOKS += BCC_INSTALL_PYTHON_BCC
>
> I think you're hitting a bug in BCC build system which causes the package to be
> installed in the wrong location on the target.
I already told to Jugurtha that was a weird hack.
>
> Does the attached patch help?
Thanks I'll take a look.
>
>> +
>> +define BCC_LINUX_CONFIG_FIXUPS
>> + # Enable kernel support for eBPF
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
>> + # [for Linux kernel versions 4.1 through 4.6]
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
>> + # [for Linux kernel versions 4.7 and later]
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
>> + # [for Linux kernel versions 5.2 and later]
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
>> + # For running bcc networking examples on vanilla kernel
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
>> + $(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
>> +endef
>
> This will be a big maintenance headache. New dependency can come up and configs
> like CONFIG_NET_* are not essential and seem to be specific to your usage.
>
> I'm not in favour of them personally, but if you want them the list need to
> shrink to the bare minimum and depend on user building the kernel in buildroot
> IMO.
As you said, bcc is moving fast and requires different kernel option depending
on the kernel version or tool/example that will be running on the target.
Agree it's may be a big maintenance headache.
But on otherhand I find userfriendly to enable CONFIG_BPF to CONFIG_IKHEADERS
option. OK to drop kernel options for networking examples.
Thanks for your review!
Best regards,
Romain
>
> Thanks!
>
> --
> Qais Yousef
>
>> +
>> +$(eval $(cmake-package))
>>
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2020-10-04 18:12 ` Qais Yousef
@ 2020-10-08 22:48 ` Qais Yousef
2020-11-03 22:34 ` Romain Naour
1 sibling, 0 replies; 13+ messages in thread
From: Qais Yousef @ 2020-10-08 22:48 UTC (permalink / raw)
To: buildroot
Hi Jugurtha
On 10/04/20 19:12, Qais Yousef wrote:
[...]
> > +---
> > +v2: Do the same for other architecture supported by LuaJIT.
> > +https://github.com/iovisor/bcc/pull/2480
> > +---
> > + src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
> > + 1 file changed, 28 insertions(+), 1 deletion(-)
> > +
> > +diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
> > +index 7541d48df..226e1b1d2 100644
> > +--- a/src/lua/CMakeLists.txt
> > ++++ b/src/lua/CMakeLists.txt
> > +@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
> > + DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
> > + )
> > +
> > ++ # LuaJIT doesn't use usual arch naming, so we have to convert
> > ++ # between CMake and Luajit while cross-compiling.
> > ++ if (CMAKE_CROSSCOMPILING)
> > ++ SET (LUAJIT_TARGET_ARCH "-a")
> > ++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
> > ++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
> > ++ if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
> > ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
> > ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
> > ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
> > ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
> > ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
> > ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
> > ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
> > ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
> > ++ else()
> > ++ MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
> > ++ endif()
> > ++ endif()
> > ++
> > + ADD_CUSTOM_COMMAND(
> > + OUTPUT bcc.o
> > +- COMMAND ${LUAJIT} -bg bcc.lua bcc.o
> > ++ COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
> > + DEPENDS bcc.lua
> > + )
> > +
With the attached patch where I remove some of the dependencies I could compile
and use bcc and apply my bpftrace patch on top and use it.
Maybe I missed trying something that you do that could make this dependency
obvious. As it stands, it seems this lua front-end support requires this
functionality + a specific use case. iperf3 dependency particularly looks
strange to me as I can't think why bcc would need that.
I suggest wrapping wrapping the lua front-end support with its own config
option and keep it in a separate patch.
We can remove PYTHON_BCC copying if we apply a fixup patch (included in the
attached patch). I'll try to find a proper solution that can be upstreamed
back. If you know the right solution, please go ahead and do it of course :-)
Thanks!
--
Qais Yousef
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bcc.fixup.patch
Type: text/x-diff
Size: 3757 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20201008/650be8ba/attachment.patch>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2020-08-24 18:17 ` [Buildroot] [PATCH 2/2] package/bcc: new package Jugurtha BELKALEM
@ 2020-10-04 18:12 ` Qais Yousef
2020-10-08 22:48 ` Qais Yousef
2020-11-03 22:34 ` Romain Naour
0 siblings, 2 replies; 13+ messages in thread
From: Qais Yousef @ 2020-10-04 18:12 UTC (permalink / raw)
To: buildroot
Hi Jugurtha
I am very late to the party and missed earlier reviews. So apologies in advance
if any of my comments was already brought up/addressed.
Can you keep me on CC on future posting and any other related discussion
please?
On 08/24/20 20:17, Jugurtha BELKALEM wrote:
> bcc is a front-end tool for eBPF :
> https://github.com/iovisor/bcc/blob/master/README.md.
> eBPF is the most powerful Linux tracer, and bcc
> allows to write eBPF scripts in C and PYTHON3.
>
> bcc can help to troubleshoot issues quickly on
> embedded systems (as long as Linux kernel
> version >= 4.1).
>
> bcc can also make it easy to create observabilty tools,
> SDN configuration, ddos mitigation, intrusion detection
> and secure containers. More information is available at:
> http://www.brendangregg.com/ebpf.html.
There's the brand new ebpf website too
http://www.ebpf.io
>
> BCC can be tested on the target :
> $ mount -t debugfs none /sys/kernel/debug
I wonder if this something we can automatically append to /etc/fstab when this
package is enabled..
> $ mkdir -p /lib/modules/KERNEL_VERSION/build
> $ cd /lib/modules/KERNEL_VERSION//build
> $ cp /sys/kernel/kheaders.tar.xz .
> $ unxz kheaders.tar.xz
> $ tar xf kheaders.tar
All the above is unnecessary. BCC automatically does this if it detects
/sys/kernel/kheaders.tar.xz is present. I had to enable BR2_PACKAGE_TAR for BCC
to handle this correctly. I guess busybox::tar is too limited to work with BCC
commands.
> $ cd /usr/share/bcc/examples/tracing
> $ ./disksnoop.py
/usr/share/bcc/tools contains actual tools. I recommend picking up an example
from there.
>
> V3
> - Bump to version 0.16.0.
> - Add required python dependency.
> - Remove unnecessary and duplicated
> kernel flags (#Testing section) in bcc.mk.
> -------------
> V2
> - Add eBPF's required Kernel flags.
> - Fix submodule source fetch problem.
> - Add toolchain dependency.
Did you intend to include these in the commit message?
>
> Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
> ---
> DEVELOPERS | 1 +
> package/Config.in | 1 +
> package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
> package/bcc/Config.in | 49 ++++++++++++++++++
> package/bcc/bcc.hash | 3 ++
> package/bcc/bcc.mk | 57 +++++++++++++++++++++
> 6 files changed, 176 insertions(+)
> create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
> create mode 100644 package/bcc/Config.in
> create mode 100644 package/bcc/bcc.hash
> create mode 100644 package/bcc/bcc.mk
>
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 42fa5a4..bd836a6 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -1396,6 +1396,7 @@ N: Joshua Henderson <joshua.henderson@microchip.com>
> F: package/qt5/qt5wayland/
>
> N: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
> +F: package/bcc/
> F: package/python-cycler/
> F: package/python-matplotlib/
>
> diff --git a/package/Config.in b/package/Config.in
> index d7e79f4..8ec3932 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -83,6 +83,7 @@ menu "Compressors and decompressors"
> endmenu
>
> menu "Debugging, profiling and benchmark"
> + source "package/bcc/Config.in"
> source "package/blktrace/Config.in"
> source "package/bonnie/Config.in"
> source "package/cache-calibrator/Config.in"
> diff --git a/package/bcc/0001-fix-aarch64-cross-compile.patch b/package/bcc/0001-fix-aarch64-cross-compile.patch
> new file mode 100644
> index 0000000..6b42797
> --- /dev/null
> +++ b/package/bcc/0001-fix-aarch64-cross-compile.patch
> @@ -0,0 +1,65 @@
> +From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
> +From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
> +Date: Thu, 2 May 2019 11:06:23 +0200
> +Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
> + cross-compiling
Is this luajit required to enable Lua front-end to BCC? Given python is the
most common front-end, I'd argue to make this an extra option that is only
enabled if the user selects it in the menu. Same way BPF backend in LLVM is
only compiled in if selected. IMHO that's a better default behavior.
> +
> +Unlike CMAKE_SYSTEM_PROCESSOR which identifies aarch64
> +as a valid architecture, luajit does not recognize it.
> +luajit defines aarch64 as arm64.
> +
> +LuaJIT doesn't use usual arch naming, so we have to convert
> +between CMake and Luajit for each architectures while
> +cross-compiling.
> +
> +Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
> +Signed-off-by: Romain Naour <romain.naour@smile.fr>
> +---
> +v2: Do the same for other architecture supported by LuaJIT.
> +https://github.com/iovisor/bcc/pull/2480
> +---
> + src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
> + 1 file changed, 28 insertions(+), 1 deletion(-)
> +
> +diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
> +index 7541d48df..226e1b1d2 100644
> +--- a/src/lua/CMakeLists.txt
> ++++ b/src/lua/CMakeLists.txt
> +@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
> + DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
> + )
> +
> ++ # LuaJIT doesn't use usual arch naming, so we have to convert
> ++ # between CMake and Luajit while cross-compiling.
> ++ if (CMAKE_CROSSCOMPILING)
> ++ SET (LUAJIT_TARGET_ARCH "-a")
> ++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
> ++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
> ++ if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
> ++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
> ++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
> ++ else()
> ++ MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
> ++ endif()
> ++ endif()
> ++
> + ADD_CUSTOM_COMMAND(
> + OUTPUT bcc.o
> +- COMMAND ${LUAJIT} -bg bcc.lua bcc.o
> ++ COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
> + DEPENDS bcc.lua
> + )
> +
> diff --git a/package/bcc/Config.in b/package/bcc/Config.in
> new file mode 100644
> index 0000000..cf03927
> --- /dev/null
> +++ b/package/bcc/Config.in
> @@ -0,0 +1,49 @@
> +config BR2_PACKAGE_BCC
> + bool "bcc"
> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> + depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
> + depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
> + depends on BR2_LINUX_KERNEL # needs kernel sources on the target
This dependency on building the kernel in buildroot is not user friendly. For
example I never build the kernel in buildroot, and certainly would like to be
able to enable this package without having to build the kernel in buildroot.
> + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
> + depends on BR2_TOOLCHAIN_HAS_THREADS # clang
> + depends on BR2_INSTALL_LIBSTDCPP # clang
> + select BR2_PACKAGE_AUDIT # runtime
This..
> + select BR2_PACKAGE_CLANG
> + select BR2_PACKAGE_ELFUTILS
> + select BR2_PACKAGE_FLEX # needs FlexLexer.h
> + select BR2_PACKAGE_IPERF3 # runtime
.. this..
> + select BR2_PACKAGE_LLVM_BPF
> + select BR2_PACKAGE_LUAJIT
> + select BR2_PACKAGE_NETPERF # runtime
.. and this look odd to me. Can you expand which runtime requires them?
> + select BR2_PACKAGE_PYTHON3
> + select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC
If you replace XZ with TAR, you should find that BCC automatically handles the
ikheader.tar.xz for you. There's now BPF Type Format (BTF) option in the kernel
that could replace the dependency on the headers. Haven't played much with that
yet though. BTF requires a tool called pahole. We can handle this support later
I guess.
> + help
> + BPF Compiler Collection (BCC)
> +
> + BCC is a toolkit for creating efficient kernel tracing and
> + manipulation programs, and includes several useful tools and
> + examples. It makes use of extended BPF (Berkeley Packet
> + Filters), formally known as eBPF, a new feature that was
> + first added to Linux 3.15. Much of what BCC uses requires
> + Linux 4.1 and above.
> +
> + Note: Before using bcc, you need either need to :
> + - For kernel_ver = [4.1, 5.2) : Copy kernel source code
> + to target folder /lib/module/<kernel_ver>/build.
> + - Or kernel_ver >= 5.2 : Compile kernel with CONFIG_IKHEADERS
> + and use generated headers under /sys/kernel/kheaders.tar.xz
> + to populate /lib/module/<kernel_ver>/build.
> +
> + That's because the clang frontend build eBPF code at runtime.
> +
> + https://github.com/iovisor/bcc
> +
> +comment "bcc needs a Linux kernel to be built"
> + depends on !BR2_LINUX_KERNEL
I must remove this to compile this patch. This hard dependency is not required
IMHO. You can still auto fix up the kernel if you like if the user is building
the kernel and has enabled BCC. Just keep in mind it's a fast moving target, so
the set of configs might require a bit of maintenance. Given it's all
documented in BCC website, I personally think it's better to leave correct
kernel configuration to the user to follow the right instructions from BCC
website rather than automatically handle it for them.
> +
> +comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8"
> + depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> + depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
> + depends on BR2_LINUX_KERNEL
> + depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
> + || !BR2_INSTALL_LIBSTDCPP
> diff --git a/package/bcc/bcc.hash b/package/bcc/bcc.hash
> new file mode 100644
> index 0000000..e3fa662
> --- /dev/null
> +++ b/package/bcc/bcc.hash
> @@ -0,0 +1,3 @@
> +# locally calculated
> +sha256 38105c6c7e53d6a8d4c2be5c2e6dd46ebeaa1489600501711d85cfe9bc065e9d bcc-v0.16.0.tar.gz
> +sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 LICENSE.txt
> diff --git a/package/bcc/bcc.mk b/package/bcc/bcc.mk
> new file mode 100644
> index 0000000..4fe69cb
> --- /dev/null
> +++ b/package/bcc/bcc.mk
> @@ -0,0 +1,57 @@
> +################################################################################
> +#
> +# bcc
> +#
> +################################################################################
> +
> +BCC_VERSION = v0.16.0
> +BCC_SITE = https://github.com/iovisor/bcc.git
> +BCC_SITE_METHOD = git
> +BCC_GIT_SUBMODULES = YES
> +BCC_LICENSE = Apache-2.0
> +BCC_LICENSE_FILES = LICENSE.txt
> +# libbcc.so and libbpf.so
> +BCC_INSTALL_STAGING = YES
> +BCC_DEPENDENCIES = host-bison host-flex host-luajit clang elfutils flex llvm \
> + luajit python3
> +
> +# ENABLE_LLVM_SHARED=ON to use llvm.so.
> +# Force REVISION otherwise bcc will use git describe to generate a version number.
> +BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
> + -DREVISION=$(BCC_VERSION) \
> + -DENABLE_CLANG_JIT=ON \
> + -DENABLE_MAN=OFF
> +
> +# BCC requires python-bcc (shiped under bcc/src/python) to work.
> +# However, BCC compiles bcc/src/python/setup.py.in into bcc/src/python.
> +# The generated compiled folder (bcc/src/python/bcc-python/bcc) should
> +# be copied to target python's site-packages directory.
> +define BCC_INSTALL_PYTHON_BCC
> + cp -r $(BUILD_DIR)/bcc-$(BCC_VERSION)/src/python/bcc-python/bcc \
> + $(TARGET_DIR)/usr/lib/python3.*/site-packages/
> +endef
> +BCC_POST_INSTALL_TARGET_HOOKS += BCC_INSTALL_PYTHON_BCC
I think you're hitting a bug in BCC build system which causes the package to be
installed in the wrong location on the target.
Does the attached patch help?
> +
> +define BCC_LINUX_CONFIG_FIXUPS
> + # Enable kernel support for eBPF
> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
> + # [for Linux kernel versions 4.1 through 4.6]
> + $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
> + # [for Linux kernel versions 4.7 and later]
> + $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
> + # [for Linux kernel versions 5.2 and later]
> + $(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
> + # For running bcc networking examples on vanilla kernel
> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
> +endef
This will be a big maintenance headache. New dependency can come up and configs
like CONFIG_NET_* are not essential and seem to be specific to your usage.
I'm not in favour of them personally, but if you want them the list need to
shrink to the bare minimum and depend on user building the kernel in buildroot
IMO.
Thanks!
--
Qais Yousef
> +
> +$(eval $(cmake-package))
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0005-fix-python-installation-path.patch
Type: text/x-diff
Size: 531 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20201004/d99fa0bb/attachment.patch>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2020-08-24 18:17 [Buildroot] [PATCH 1/2] package/llvm: add BPF backend Jugurtha BELKALEM
@ 2020-08-24 18:17 ` Jugurtha BELKALEM
2020-10-04 18:12 ` Qais Yousef
0 siblings, 1 reply; 13+ messages in thread
From: Jugurtha BELKALEM @ 2020-08-24 18:17 UTC (permalink / raw)
To: buildroot
bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.
bcc can help to troubleshoot issues quickly on
embedded systems (as long as Linux kernel
version >= 4.1).
bcc can also make it easy to create observabilty tools,
SDN configuration, ddos mitigation, intrusion detection
and secure containers. More information is available at:
http://www.brendangregg.com/ebpf.html.
BCC can be tested on the target :
$ mount -t debugfs none /sys/kernel/debug
$ mkdir -p /lib/modules/KERNEL_VERSION/build
$ cd /lib/modules/KERNEL_VERSION//build
$ cp /sys/kernel/kheaders.tar.xz .
$ unxz kheaders.tar.xz
$ tar xf kheaders.tar
$ cd /usr/share/bcc/examples/tracing
$ ./disksnoop.py
V3
- Bump to version 0.16.0.
- Add required python dependency.
- Remove unnecessary and duplicated
kernel flags (#Testing section) in bcc.mk.
-------------
V2
- Add eBPF's required Kernel flags.
- Fix submodule source fetch problem.
- Add toolchain dependency.
Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
---
DEVELOPERS | 1 +
package/Config.in | 1 +
package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
package/bcc/Config.in | 49 ++++++++++++++++++
package/bcc/bcc.hash | 3 ++
package/bcc/bcc.mk | 57 +++++++++++++++++++++
6 files changed, 176 insertions(+)
create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
create mode 100644 package/bcc/Config.in
create mode 100644 package/bcc/bcc.hash
create mode 100644 package/bcc/bcc.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index 42fa5a4..bd836a6 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1396,6 +1396,7 @@ N: Joshua Henderson <joshua.henderson@microchip.com>
F: package/qt5/qt5wayland/
N: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+F: package/bcc/
F: package/python-cycler/
F: package/python-matplotlib/
diff --git a/package/Config.in b/package/Config.in
index d7e79f4..8ec3932 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -83,6 +83,7 @@ menu "Compressors and decompressors"
endmenu
menu "Debugging, profiling and benchmark"
+ source "package/bcc/Config.in"
source "package/blktrace/Config.in"
source "package/bonnie/Config.in"
source "package/cache-calibrator/Config.in"
diff --git a/package/bcc/0001-fix-aarch64-cross-compile.patch b/package/bcc/0001-fix-aarch64-cross-compile.patch
new file mode 100644
index 0000000..6b42797
--- /dev/null
+++ b/package/bcc/0001-fix-aarch64-cross-compile.patch
@@ -0,0 +1,65 @@
+From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
+From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+Date: Thu, 2 May 2019 11:06:23 +0200
+Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
+ cross-compiling
+
+Unlike CMAKE_SYSTEM_PROCESSOR which identifies aarch64
+as a valid architecture, luajit does not recognize it.
+luajit defines aarch64 as arm64.
+
+LuaJIT doesn't use usual arch naming, so we have to convert
+between CMake and Luajit for each architectures while
+cross-compiling.
+
+Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+Signed-off-by: Romain Naour <romain.naour@smile.fr>
+---
+v2: Do the same for other architecture supported by LuaJIT.
+https://github.com/iovisor/bcc/pull/2480
+---
+ src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
+ 1 file changed, 28 insertions(+), 1 deletion(-)
+
+diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
+index 7541d48df..226e1b1d2 100644
+--- a/src/lua/CMakeLists.txt
++++ b/src/lua/CMakeLists.txt
+@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
+ DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
+ )
+
++ # LuaJIT doesn't use usual arch naming, so we have to convert
++ # between CMake and Luajit while cross-compiling.
++ if (CMAKE_CROSSCOMPILING)
++ SET (LUAJIT_TARGET_ARCH "-a")
++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
++ if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
++ else()
++ MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
++ endif()
++ endif()
++
+ ADD_CUSTOM_COMMAND(
+ OUTPUT bcc.o
+- COMMAND ${LUAJIT} -bg bcc.lua bcc.o
++ COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
+ DEPENDS bcc.lua
+ )
+
diff --git a/package/bcc/Config.in b/package/bcc/Config.in
new file mode 100644
index 0000000..cf03927
--- /dev/null
+++ b/package/bcc/Config.in
@@ -0,0 +1,49 @@
+config BR2_PACKAGE_BCC
+ bool "bcc"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+ depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
+ depends on BR2_LINUX_KERNEL # needs kernel sources on the target
+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
+ depends on BR2_TOOLCHAIN_HAS_THREADS # clang
+ depends on BR2_INSTALL_LIBSTDCPP # clang
+ select BR2_PACKAGE_AUDIT # runtime
+ select BR2_PACKAGE_CLANG
+ select BR2_PACKAGE_ELFUTILS
+ select BR2_PACKAGE_FLEX # needs FlexLexer.h
+ select BR2_PACKAGE_IPERF3 # runtime
+ select BR2_PACKAGE_LLVM_BPF
+ select BR2_PACKAGE_LUAJIT
+ select BR2_PACKAGE_NETPERF # runtime
+ select BR2_PACKAGE_PYTHON3
+ select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC
+ help
+ BPF Compiler Collection (BCC)
+
+ BCC is a toolkit for creating efficient kernel tracing and
+ manipulation programs, and includes several useful tools and
+ examples. It makes use of extended BPF (Berkeley Packet
+ Filters), formally known as eBPF, a new feature that was
+ first added to Linux 3.15. Much of what BCC uses requires
+ Linux 4.1 and above.
+
+ Note: Before using bcc, you need either need to :
+ - For kernel_ver = [4.1, 5.2) : Copy kernel source code
+ to target folder /lib/module/<kernel_ver>/build.
+ - Or kernel_ver >= 5.2 : Compile kernel with CONFIG_IKHEADERS
+ and use generated headers under /sys/kernel/kheaders.tar.xz
+ to populate /lib/module/<kernel_ver>/build.
+
+ That's because the clang frontend build eBPF code at runtime.
+
+ https://github.com/iovisor/bcc
+
+comment "bcc needs a Linux kernel to be built"
+ depends on !BR2_LINUX_KERNEL
+
+comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+ depends on BR2_LINUX_KERNEL
+ depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
+ || !BR2_INSTALL_LIBSTDCPP
diff --git a/package/bcc/bcc.hash b/package/bcc/bcc.hash
new file mode 100644
index 0000000..e3fa662
--- /dev/null
+++ b/package/bcc/bcc.hash
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 38105c6c7e53d6a8d4c2be5c2e6dd46ebeaa1489600501711d85cfe9bc065e9d bcc-v0.16.0.tar.gz
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 LICENSE.txt
diff --git a/package/bcc/bcc.mk b/package/bcc/bcc.mk
new file mode 100644
index 0000000..4fe69cb
--- /dev/null
+++ b/package/bcc/bcc.mk
@@ -0,0 +1,57 @@
+################################################################################
+#
+# bcc
+#
+################################################################################
+
+BCC_VERSION = v0.16.0
+BCC_SITE = https://github.com/iovisor/bcc.git
+BCC_SITE_METHOD = git
+BCC_GIT_SUBMODULES = YES
+BCC_LICENSE = Apache-2.0
+BCC_LICENSE_FILES = LICENSE.txt
+# libbcc.so and libbpf.so
+BCC_INSTALL_STAGING = YES
+BCC_DEPENDENCIES = host-bison host-flex host-luajit clang elfutils flex llvm \
+ luajit python3
+
+# ENABLE_LLVM_SHARED=ON to use llvm.so.
+# Force REVISION otherwise bcc will use git describe to generate a version number.
+BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
+ -DREVISION=$(BCC_VERSION) \
+ -DENABLE_CLANG_JIT=ON \
+ -DENABLE_MAN=OFF
+
+# BCC requires python-bcc (shiped under bcc/src/python) to work.
+# However, BCC compiles bcc/src/python/setup.py.in into bcc/src/python.
+# The generated compiled folder (bcc/src/python/bcc-python/bcc) should
+# be copied to target python's site-packages directory.
+define BCC_INSTALL_PYTHON_BCC
+ cp -r $(BUILD_DIR)/bcc-$(BCC_VERSION)/src/python/bcc-python/bcc \
+ $(TARGET_DIR)/usr/lib/python3.*/site-packages/
+endef
+BCC_POST_INSTALL_TARGET_HOOKS += BCC_INSTALL_PYTHON_BCC
+
+define BCC_LINUX_CONFIG_FIXUPS
+ # Enable kernel support for eBPF
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
+ # [for Linux kernel versions 4.1 through 4.6]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
+ # [for Linux kernel versions 4.7 and later]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
+ # [for Linux kernel versions 5.2 and later]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
+ # For running bcc networking examples on vanilla kernel
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
+endef
+
+$(eval $(cmake-package))
--
2.7.4
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2020-08-13 13:05 [Buildroot] [PATCH 0/2] bcc front end tool for eBPF Jugurtha BELKALEM
@ 2020-08-13 13:05 ` Jugurtha BELKALEM
0 siblings, 0 replies; 13+ messages in thread
From: Jugurtha BELKALEM @ 2020-08-13 13:05 UTC (permalink / raw)
To: buildroot
bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.
bcc can help to troubleshoot issues quickly on
embedded systems (as long as Linux kernel
version >= 4.1).
bcc can also make it easy to create observabilty tools,
SDN configuration, ddos mitigation, intrusion detection
and secure containers. More information is available at:
http://www.brendangregg.com/ebpf.html.
BCC can be tested on the target :
$ mount -t debugfs none /sys/kernel/debug
$ mkdir -p /lib/modules/KERNEL_VERSION/build
$ cd /lib/modules/KERNEL_VERSION//build
$ cp /sys/kernel/kheaders.tar.xz .
$ unxz kheaders.tar.xz
$ tar xf kheaders.tar
$ cd /usr/share/bcc/examples/tracing
$ ./disksnoop.py
Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
---
DEVELOPERS | 1 +
package/Config.in | 1 +
package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
package/bcc/Config.in | 49 ++++++++++++++++++
package/bcc/bcc.hash | 3 ++
package/bcc/bcc.mk | 63 +++++++++++++++++++++++
6 files changed, 182 insertions(+)
create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
create mode 100644 package/bcc/Config.in
create mode 100644 package/bcc/bcc.hash
create mode 100644 package/bcc/bcc.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index 42fa5a4..bd836a6 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1396,6 +1396,7 @@ N: Joshua Henderson <joshua.henderson@microchip.com>
F: package/qt5/qt5wayland/
N: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+F: package/bcc/
F: package/python-cycler/
F: package/python-matplotlib/
diff --git a/package/Config.in b/package/Config.in
index d7e79f4..8ec3932 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -83,6 +83,7 @@ menu "Compressors and decompressors"
endmenu
menu "Debugging, profiling and benchmark"
+ source "package/bcc/Config.in"
source "package/blktrace/Config.in"
source "package/bonnie/Config.in"
source "package/cache-calibrator/Config.in"
diff --git a/package/bcc/0001-fix-aarch64-cross-compile.patch b/package/bcc/0001-fix-aarch64-cross-compile.patch
new file mode 100644
index 0000000..6b42797
--- /dev/null
+++ b/package/bcc/0001-fix-aarch64-cross-compile.patch
@@ -0,0 +1,65 @@
+From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
+From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+Date: Thu, 2 May 2019 11:06:23 +0200
+Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
+ cross-compiling
+
+Unlike CMAKE_SYSTEM_PROCESSOR which identifies aarch64
+as a valid architecture, luajit does not recognize it.
+luajit defines aarch64 as arm64.
+
+LuaJIT doesn't use usual arch naming, so we have to convert
+between CMake and Luajit for each architectures while
+cross-compiling.
+
+Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+Signed-off-by: Romain Naour <romain.naour@smile.fr>
+---
+v2: Do the same for other architecture supported by LuaJIT.
+https://github.com/iovisor/bcc/pull/2480
+---
+ src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
+ 1 file changed, 28 insertions(+), 1 deletion(-)
+
+diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
+index 7541d48df..226e1b1d2 100644
+--- a/src/lua/CMakeLists.txt
++++ b/src/lua/CMakeLists.txt
+@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
+ DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
+ )
+
++ # LuaJIT doesn't use usual arch naming, so we have to convert
++ # between CMake and Luajit while cross-compiling.
++ if (CMAKE_CROSSCOMPILING)
++ SET (LUAJIT_TARGET_ARCH "-a")
++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
++ if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
++ else()
++ MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
++ endif()
++ endif()
++
+ ADD_CUSTOM_COMMAND(
+ OUTPUT bcc.o
+- COMMAND ${LUAJIT} -bg bcc.lua bcc.o
++ COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
+ DEPENDS bcc.lua
+ )
+
diff --git a/package/bcc/Config.in b/package/bcc/Config.in
new file mode 100644
index 0000000..a3cc402
--- /dev/null
+++ b/package/bcc/Config.in
@@ -0,0 +1,49 @@
+config BR2_PACKAGE_BCC
+ bool "bcc"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+ depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
+ depends on BR2_LINUX_KERNEL # needs kernel sources on the target
+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
+ depends on BR2_TOOLCHAIN_HAS_THREADS # clang
+ depends on BR2_INSTALL_LIBSTDCPP # clang
+ select BR2_PACKAGE_AUDIT # runtime
+ select BR2_PACKAGE_CLANG
+ select BR2_PACKAGE_ELFUTILS
+ select BR2_PACKAGE_FLEX # needs FlexLexer.h
+ select BR2_PACKAGE_IPERF3 # runtime
+ select BR2_PACKAGE_LLVM_BPF
+ select BR2_PACKAGE_LUAJIT
+ select BR2_PACKAGE_NETPERF # runtime
+ select BR2_PACKAGE_PYTHON3
+ select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC
+ help
+ BPF Compiler Collection (BCC)
+
+ BCC is a toolkit for creating efficient kernel tracing and
+ manipulation programs, and includes several useful tools and
+ examples. It makes use of extended BPF (Berkeley Packet
+ Filters), formally known as eBPF, a new feature that was
+ first added to Linux 3.15. Much of what BCC uses requires
+ Linux 4.1 and above.
+
+ Note: Before using bcc, you need either :
+ - Copy the kernel source code to target folder
+ /lib/module/<kernel version>/build.
+ - Compile kernel with CONFIG_IKHEADERS (kernel_ver > 5.2)
+ and use generated headers to populate
+ /lib/module/<kernel version>/build.
+
+ That's because the clang frontend build eBPF code at runtime.
+
+ https://github.com/iovisor/bcc
+
+comment "bcc needs a Linux kernel to be built"
+ depends on !BR2_LINUX_KERNEL
+
+comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+ depends on BR2_LINUX_KERNEL
+ depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
+ || !BR2_INSTALL_LIBSTDCPP
diff --git a/package/bcc/bcc.hash b/package/bcc/bcc.hash
new file mode 100644
index 0000000..e01a2e8
--- /dev/null
+++ b/package/bcc/bcc.hash
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 518f2ee52139dcc69c589be7a6171928dc2c1cf2cdb7d0c1468134166aa84ca1 bcc-v0.15.0.tar.gz
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 LICENSE.txt
diff --git a/package/bcc/bcc.mk b/package/bcc/bcc.mk
new file mode 100644
index 0000000..f6a2e12
--- /dev/null
+++ b/package/bcc/bcc.mk
@@ -0,0 +1,63 @@
+################################################################################
+#
+# bcc
+#
+################################################################################
+
+BCC_VERSION = v0.15.0
+BCC_SITE = https://github.com/iovisor/bcc.git
+BCC_SITE_METHOD = git
+BCC_GIT_SUBMODULES = YES
+BCC_LICENSE = Apache-2.0
+BCC_LICENSE_FILES = LICENSE.txt
+# libbcc.so and libbpf.so
+BCC_INSTALL_STAGING = YES
+BCC_DEPENDENCIES = host-bison host-flex host-luajit clang elfutils flex llvm \
+ luajit
+
+# ENABLE_LLVM_SHARED=ON to use llvm.so.
+# Force REVISION otherwise bcc will use git describe to generate a version number.
+BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
+ -DREVISION=$(BCC_VERSION) \
+ -DENABLE_CLANG_JIT=ON \
+ -DENABLE_MAN=OFF
+
+# BCC compiles src/python/setup.py.in into /src/python.
+# The generated compiled folder (/src/python/bcc-python/bcc) should
+# be copied to target python's site-packages directory.
+define BCC_INSTALL_PYTHON_BCC
+ cp -r $(BUILD_DIR)/bcc-$(BCC_VERSION)/src/python/bcc-python/bcc \
+ $(TARGET_DIR)/usr/lib/python3.*/site-packages/
+endef
+BCC_POST_INSTALL_TARGET_HOOKS += BCC_INSTALL_PYTHON_BCC
+
+define BCC_LINUX_CONFIG_FIXUPS
+ # Enable kernel support for eBPF
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
+ # [for Linux kernel versions 4.1 through 4.6]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
+ # [for Linux kernel versions 4.7 and later]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
+ # [for Linux kernel versions 5.3 and later]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
+ # For running bcc networking examples on vanilla kernel
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
+ # Testing
+ $(call KCONFIG_ENABLE_OPT,CONFIG_KPROBES)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_FTRACE)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPFILTER)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_CGROUP_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_ACT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+endef
+
+$(eval $(cmake-package))
--
2.7.4
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [Buildroot] [PATCH 2/2] package/bcc: new package
2020-08-13 13:02 [Buildroot] [PATCH 0/2] bcc front end tool for eBPF Jugurtha BELKALEM
@ 2020-08-13 13:02 ` Jugurtha BELKALEM
0 siblings, 0 replies; 13+ messages in thread
From: Jugurtha BELKALEM @ 2020-08-13 13:02 UTC (permalink / raw)
To: buildroot
bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.
bcc can help to troubleshoot issues quickly on
embedded systems (as long as Linux kernel
version >= 4.1).
bcc can also make it easy to create observabilty tools,
SDN configuration, ddos mitigation, intrusion detection
and secure containers. More information is available at:
http://www.brendangregg.com/ebpf.html.
BCC can be tested on the target :
$ mount -t debugfs none /sys/kernel/debug
$ mkdir -p /lib/modules/KERNEL_VERSION/build
$ cd /lib/modules/KERNEL_VERSION//build
$ cp /sys/kernel/kheaders.tar.xz .
$ unxz kheaders.tar.xz
$ tar xf kheaders.tar
$ cd /usr/share/bcc/examples/tracing
$ ./disksnoop.py
Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
---
DEVELOPERS | 1 +
package/Config.in | 1 +
package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
package/bcc/Config.in | 49 ++++++++++++++++++
package/bcc/bcc.hash | 3 ++
package/bcc/bcc.mk | 63 +++++++++++++++++++++++
6 files changed, 182 insertions(+)
create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
create mode 100644 package/bcc/Config.in
create mode 100644 package/bcc/bcc.hash
create mode 100644 package/bcc/bcc.mk
diff --git a/DEVELOPERS b/DEVELOPERS
index 42fa5a4..bd836a6 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -1396,6 +1396,7 @@ N: Joshua Henderson <joshua.henderson@microchip.com>
F: package/qt5/qt5wayland/
N: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+F: package/bcc/
F: package/python-cycler/
F: package/python-matplotlib/
diff --git a/package/Config.in b/package/Config.in
index d7e79f4..8ec3932 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -83,6 +83,7 @@ menu "Compressors and decompressors"
endmenu
menu "Debugging, profiling and benchmark"
+ source "package/bcc/Config.in"
source "package/blktrace/Config.in"
source "package/bonnie/Config.in"
source "package/cache-calibrator/Config.in"
diff --git a/package/bcc/0001-fix-aarch64-cross-compile.patch b/package/bcc/0001-fix-aarch64-cross-compile.patch
new file mode 100644
index 0000000..6b42797
--- /dev/null
+++ b/package/bcc/0001-fix-aarch64-cross-compile.patch
@@ -0,0 +1,65 @@
+From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
+From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+Date: Thu, 2 May 2019 11:06:23 +0200
+Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
+ cross-compiling
+
+Unlike CMAKE_SYSTEM_PROCESSOR which identifies aarch64
+as a valid architecture, luajit does not recognize it.
+luajit defines aarch64 as arm64.
+
+LuaJIT doesn't use usual arch naming, so we have to convert
+between CMake and Luajit for each architectures while
+cross-compiling.
+
+Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
+Signed-off-by: Romain Naour <romain.naour@smile.fr>
+---
+v2: Do the same for other architecture supported by LuaJIT.
+https://github.com/iovisor/bcc/pull/2480
+---
+ src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
+ 1 file changed, 28 insertions(+), 1 deletion(-)
+
+diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
+index 7541d48df..226e1b1d2 100644
+--- a/src/lua/CMakeLists.txt
++++ b/src/lua/CMakeLists.txt
+@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
+ DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
+ )
+
++ # LuaJIT doesn't use usual arch naming, so we have to convert
++ # between CMake and Luajit while cross-compiling.
++ if (CMAKE_CROSSCOMPILING)
++ SET (LUAJIT_TARGET_ARCH "-a")
++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
++ # https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
++ if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
++ elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
++ SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
++ else()
++ MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
++ endif()
++ endif()
++
+ ADD_CUSTOM_COMMAND(
+ OUTPUT bcc.o
+- COMMAND ${LUAJIT} -bg bcc.lua bcc.o
++ COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
+ DEPENDS bcc.lua
+ )
+
diff --git a/package/bcc/Config.in b/package/bcc/Config.in
new file mode 100644
index 0000000..a3cc402
--- /dev/null
+++ b/package/bcc/Config.in
@@ -0,0 +1,49 @@
+config BR2_PACKAGE_BCC
+ bool "bcc"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+ depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
+ depends on BR2_LINUX_KERNEL # needs kernel sources on the target
+ depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
+ depends on BR2_TOOLCHAIN_HAS_THREADS # clang
+ depends on BR2_INSTALL_LIBSTDCPP # clang
+ select BR2_PACKAGE_AUDIT # runtime
+ select BR2_PACKAGE_CLANG
+ select BR2_PACKAGE_ELFUTILS
+ select BR2_PACKAGE_FLEX # needs FlexLexer.h
+ select BR2_PACKAGE_IPERF3 # runtime
+ select BR2_PACKAGE_LLVM_BPF
+ select BR2_PACKAGE_LUAJIT
+ select BR2_PACKAGE_NETPERF # runtime
+ select BR2_PACKAGE_PYTHON3
+ select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC
+ help
+ BPF Compiler Collection (BCC)
+
+ BCC is a toolkit for creating efficient kernel tracing and
+ manipulation programs, and includes several useful tools and
+ examples. It makes use of extended BPF (Berkeley Packet
+ Filters), formally known as eBPF, a new feature that was
+ first added to Linux 3.15. Much of what BCC uses requires
+ Linux 4.1 and above.
+
+ Note: Before using bcc, you need either :
+ - Copy the kernel source code to target folder
+ /lib/module/<kernel version>/build.
+ - Compile kernel with CONFIG_IKHEADERS (kernel_ver > 5.2)
+ and use generated headers to populate
+ /lib/module/<kernel version>/build.
+
+ That's because the clang frontend build eBPF code at runtime.
+
+ https://github.com/iovisor/bcc
+
+comment "bcc needs a Linux kernel to be built"
+ depends on !BR2_LINUX_KERNEL
+
+comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8"
+ depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+ depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+ depends on BR2_LINUX_KERNEL
+ depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
+ || !BR2_INSTALL_LIBSTDCPP
diff --git a/package/bcc/bcc.hash b/package/bcc/bcc.hash
new file mode 100644
index 0000000..e01a2e8
--- /dev/null
+++ b/package/bcc/bcc.hash
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 518f2ee52139dcc69c589be7a6171928dc2c1cf2cdb7d0c1468134166aa84ca1 bcc-v0.15.0.tar.gz
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 LICENSE.txt
diff --git a/package/bcc/bcc.mk b/package/bcc/bcc.mk
new file mode 100644
index 0000000..f6a2e12
--- /dev/null
+++ b/package/bcc/bcc.mk
@@ -0,0 +1,63 @@
+################################################################################
+#
+# bcc
+#
+################################################################################
+
+BCC_VERSION = v0.15.0
+BCC_SITE = https://github.com/iovisor/bcc.git
+BCC_SITE_METHOD = git
+BCC_GIT_SUBMODULES = YES
+BCC_LICENSE = Apache-2.0
+BCC_LICENSE_FILES = LICENSE.txt
+# libbcc.so and libbpf.so
+BCC_INSTALL_STAGING = YES
+BCC_DEPENDENCIES = host-bison host-flex host-luajit clang elfutils flex llvm \
+ luajit
+
+# ENABLE_LLVM_SHARED=ON to use llvm.so.
+# Force REVISION otherwise bcc will use git describe to generate a version number.
+BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
+ -DREVISION=$(BCC_VERSION) \
+ -DENABLE_CLANG_JIT=ON \
+ -DENABLE_MAN=OFF
+
+# BCC compiles src/python/setup.py.in into /src/python.
+# The generated compiled folder (/src/python/bcc-python/bcc) should
+# be copied to target python's site-packages directory.
+define BCC_INSTALL_PYTHON_BCC
+ cp -r $(BUILD_DIR)/bcc-$(BCC_VERSION)/src/python/bcc-python/bcc \
+ $(TARGET_DIR)/usr/lib/python3.*/site-packages/
+endef
+BCC_POST_INSTALL_TARGET_HOOKS += BCC_INSTALL_PYTHON_BCC
+
+define BCC_LINUX_CONFIG_FIXUPS
+ # Enable kernel support for eBPF
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
+ # [for Linux kernel versions 4.1 through 4.6]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
+ # [for Linux kernel versions 4.7 and later]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
+ # [for Linux kernel versions 5.3 and later]
+ $(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
+ # For running bcc networking examples on vanilla kernel
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
+ # Testing
+ $(call KCONFIG_ENABLE_OPT,CONFIG_KPROBES)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_FTRACE)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_BPFILTER)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_CGROUP_BPF)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_ACT)
+ $(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+endef
+
+$(eval $(cmake-package))
--
2.7.4
^ permalink raw reply related [flat|nested] 13+ messages in thread
end of thread, other threads:[~2020-11-24 16:41 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-13 21:21 [Buildroot] [PATCH 1/2] package/llvm: add BPF backend Romain Naour
2019-01-13 21:21 ` [Buildroot] [PATCH 2/2] package/bcc: new package Romain Naour
2019-01-14 19:40 ` Romain Naour
2019-01-14 20:43 ` Matthew Weber
2019-01-14 21:00 ` Romain Naour
2019-03-14 17:15 ` Romain Naour
2020-08-13 13:02 [Buildroot] [PATCH 0/2] bcc front end tool for eBPF Jugurtha BELKALEM
2020-08-13 13:02 ` [Buildroot] [PATCH 2/2] package/bcc: new package Jugurtha BELKALEM
2020-08-13 13:05 [Buildroot] [PATCH 0/2] bcc front end tool for eBPF Jugurtha BELKALEM
2020-08-13 13:05 ` [Buildroot] [PATCH 2/2] package/bcc: new package Jugurtha BELKALEM
2020-08-24 18:17 [Buildroot] [PATCH 1/2] package/llvm: add BPF backend Jugurtha BELKALEM
2020-08-24 18:17 ` [Buildroot] [PATCH 2/2] package/bcc: new package Jugurtha BELKALEM
2020-10-04 18:12 ` Qais Yousef
2020-10-08 22:48 ` Qais Yousef
2020-11-03 22:34 ` Romain Naour
2020-11-24 16:41 ` Qais Yousef
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.