All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH v2 0/2] Enable BTF headers for Linux kernel
@ 2021-12-22 17:49 Francis Laniel
  2021-12-22 17:49 ` [Buildroot] [PATCH v2 1/2] package/pahole: new host package Francis Laniel
  2021-12-22 17:49 ` [Buildroot] [PATCH v2 2/2] linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE Francis Laniel
  0 siblings, 2 replies; 6+ messages in thread
From: Francis Laniel @ 2021-12-22 17:49 UTC (permalink / raw)
  To: buildroot; +Cc: Francis Laniel, Francis Laniel, Samuel Martin

From: Francis Laniel <laniel_francis@privacyrequired.com>

Hi.


First of all, I hope you are fine and the same for your relatives.

Extended Berkeley Packet Filter (eBPF) is a virtual machine inside the kernel
which permits executing safe code [1].
This code can, for example, be used to hook kernel events.

BPF Compile Once - Run Everywhere (BPF CO-RE) is thought as an evolution of
conventional BPF program [2].
BPF CO-RE relies on BPF Type Format (BTF) to be executed on different kernels.
This is done thanks to libbpf, the BPF loader, which resolves symbols contained
into the BPF binary against symbols offered by the kernel.

Kernel shares its BTF symbols through /sys/kernel/btf/vmlinux which is enabled
by CONFIG_DEBUG_INFO_BTF.
Nonetheless, CONFIG_DEBUG_INFO_BTF relies on pahole to convert DWARF symbols to
BTF [3].
Thus, this series enable BTF headers for Linux kernel by:
1. First, adding pahole as host package by compiling it from source taken from
its git repository.
2. Second, adding pahole as kernel dependencies if user selected
BR2_PACKAGE_HOST_PAHOLE.

I understand Buildroot and BPF can seem to be far from each other at first
glance.
But Buildroot, while mainly used to generate embedded Linux systems, is also
used to generate "qemu ready" image [4].
More particularly, minikube, a tool used to deploy Kubernetes locally, relies on
buildroot to generate its system image [5, 6].

I use minikube daily to test a tool we develop for Kubernetes and I wrote a
patch to enable CONFIG_DEBUG_INFO_BTF [7, 8].
Thus, I thought it could be a good idea to share this patch upstream as this
feature would be welcomed by the community [9].

Regarding the series itself, I tested it by compiling a system for
qemu_x86_defconfig with BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE.
Then, I booted the system using qemu to check if /sys/kernel/btf/vmlinux exists,
which is the case:
Welcome to Buildroot
buildroot login: root
# [ -f /sys/kernel/btf/vmlinux ] && echo yes
yes

If you have any remarks regarding this series, feel free to share as I would be
happy to handle them to increase this contribution quality.

Changes since v1:
* Added pahole COPYING hash.
* Jumped pahole version from v1.22 to v1.23.
* Added host-elfutils as host-pahole dependencies (it does not build without).
* Built dynamically libppf for pahole instead of statically.
* Added BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE which acts like
BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF. If CONFIG_DEBUG_INFO_BTF is set and this
option not, kernel .config will not be written and an error will be printed.

Francis Laniel (2):
  package/pahole: new host package
  linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE

 DEVELOPERS                    |  3 +++
 linux/Config.in               | 12 ++++++++++++
 linux/linux.mk                | 12 ++++++++++++
 package/Config.in.host        |  1 +
 package/pahole/Config.in.host |  8 ++++++++
 package/pahole/pahole.hash    |  3 +++
 package/pahole/pahole.mk      | 18 ++++++++++++++++++
 7 files changed, 57 insertions(+)
 create mode 100644 package/pahole/Config.in.host
 create mode 100644 package/pahole/pahole.hash
 create mode 100644 package/pahole/pahole.mk


Best regards and thank you in advance for your reviews.

---
[1] https://man7.org/linux/man-pages/man2/bpf.2.html
[2] https://nakryiko.com/posts/bpf-portability-and-co-re/
[3] https://elixir.bootlin.com/linux/v5.15.10/source/lib/Kconfig.debug#L315
[4] https://petermalmgren.com/qemu-buildroot/
[5] https://github.com/kubernetes/minikube
[6] https://github.com/kubernetes/minikube/blob/7dc8836303721c4486c007945beb5272cf28601c/deploy/iso/minikube-iso/configs/minikube_defconfig
[7] https://github.com/kinvolk/inspektor-gadget
[8] https://github.com/kinvolk/minikube/commit/44327414939f8acc08e095e789bc41f7dc73099e
[9] https://github.com/kubernetes/minikube/pull/12707#issuecomment-986990849
--
2.30.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v2 1/2] package/pahole: new host package
  2021-12-22 17:49 [Buildroot] [PATCH v2 0/2] Enable BTF headers for Linux kernel Francis Laniel
@ 2021-12-22 17:49 ` Francis Laniel
  2022-01-13 21:40   ` Thomas Petazzoni
  2021-12-22 17:49 ` [Buildroot] [PATCH v2 2/2] linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE Francis Laniel
  1 sibling, 1 reply; 6+ messages in thread
From: Francis Laniel @ 2021-12-22 17:49 UTC (permalink / raw)
  To: buildroot; +Cc: Francis Laniel, Samuel Martin

pahole is a tool used to show data structure embedded in debugging information
formats like DWARF.
It is notably needed by the Linux kernel to generate BPF Type Format (BTF)
information used by Compile Once - Run Everywhere (CO-RE) BPF tools.

To be built, pahole needs __LIB to be set to lib at stated in its README.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
---
 DEVELOPERS                    |  3 +++
 package/Config.in.host        |  1 +
 package/pahole/Config.in.host |  6 ++++++
 package/pahole/pahole.hash    |  3 +++
 package/pahole/pahole.mk      | 18 ++++++++++++++++++
 5 files changed, 31 insertions(+)
 create mode 100644 package/pahole/Config.in.host
 create mode 100644 package/pahole/pahole.hash
 create mode 100644 package/pahole/pahole.mk

diff --git a/DEVELOPERS b/DEVELOPERS
index 64db6c51d0..70df957415 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -939,6 +939,9 @@ N:	Floris Bos <bos@je-eigen-domein.nl>
 F:	package/ipmitool/
 F:	package/odhcploc/
 
+N:	Francis Laniel <flaniel@linux.microsoft.com>
+F:	package/pahole/
+
 N:	Francisco Gonzalez <gzmorell@gmail.com>
 F:	package/ser2net/
 
diff --git a/package/Config.in.host b/package/Config.in.host
index 6e5a5c5fc5..ae512c5643 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -60,6 +60,7 @@ menu "Host utilities"
 	source "package/omap-u-boot-utils/Config.in.host"
 	source "package/openocd/Config.in.host"
 	source "package/opkg-utils/Config.in.host"
+	source "package/pahole/Config.in.host"
 	source "package/parted/Config.in.host"
 	source "package/patchelf/Config.in.host"
 	source "package/pigz/Config.in.host"
diff --git a/package/pahole/Config.in.host b/package/pahole/Config.in.host
new file mode 100644
index 0000000000..e427629632
--- /dev/null
+++ b/package/pahole/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PAHOLE
+	bool "host pahole"
+	help
+	  Pahole and other DWARF utils.
+
+	  https://git.kernel.org/pub/scm/devel/pahole/pahole.git
diff --git a/package/pahole/pahole.hash b/package/pahole/pahole.hash
new file mode 100644
index 0000000000..9ee2ef9429
--- /dev/null
+++ b/package/pahole/pahole.hash
@@ -0,0 +1,3 @@
+# Locally computed
+sha256 cde85af68b368f50a913be387f94f6b43612a04af6c92387b4dcabb712a668fe  pahole-v1.23-br1.tar.gz
+sha256 ab15fd526bd8dd18a9e77ebc139656bf4d33e97fc7238cd11bf60e2b9b8666c6  COPYING
diff --git a/package/pahole/pahole.mk b/package/pahole/pahole.mk
new file mode 100644
index 0000000000..26347cd8fd
--- /dev/null
+++ b/package/pahole/pahole.mk
@@ -0,0 +1,18 @@
+########################################################################
+#
+# pahole
+#
+########################################################################
+
+PAHOLE_VERSION = v1.23
+PAHOLE_SITE = git://git.kernel.org/pub/scm/devel/pahole/pahole.git
+PAHOLE_SITE_METHOD = git
+# pahole contains git submodule and relies on them to be built.
+PAHOLE_GIT_SUBMODULES = YES
+HOST_PAHOLE_DEPENDENCIES = host-elfutils
+# Defining __LIB is needed to build pahole.
+HOST_PAHOLE_CONF_OPTS = -D__LIB=lib
+PAHOLE_LICENSE = GPL-2.0
+PAHOLE_LICENSE_FILES = COPYING
+
+$(eval $(host-cmake-package))
-- 
2.30.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [Buildroot] [PATCH v2 2/2] linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE
  2021-12-22 17:49 [Buildroot] [PATCH v2 0/2] Enable BTF headers for Linux kernel Francis Laniel
  2021-12-22 17:49 ` [Buildroot] [PATCH v2 1/2] package/pahole: new host package Francis Laniel
@ 2021-12-22 17:49 ` Francis Laniel
  2022-01-13 21:41   ` Thomas Petazzoni
  1 sibling, 1 reply; 6+ messages in thread
From: Francis Laniel @ 2021-12-22 17:49 UTC (permalink / raw)
  To: buildroot; +Cc: Francis Laniel, Samuel Martin

CONFIG_DEBUG_BTF_INFO relies on pahole, so kernel DWARF are converted to BTF.
If CONFIG_DEBUG_BTF_INFO is set and BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE not,
an error message is shown and .config is not written.

Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
---
 linux/Config.in               | 12 ++++++++++++
 linux/linux.mk                | 12 ++++++++++++
 package/pahole/Config.in.host |  2 ++
 3 files changed, 26 insertions(+)

diff --git a/linux/Config.in b/linux/Config.in
index 6c30100921..ef86cbc021 100644
--- a/linux/Config.in
+++ b/linux/Config.in
@@ -470,6 +470,18 @@ config BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF
 	  CONFIG_UNWINDER_ORC=y, please install libelf-dev,
 	  libelf-devel or elfutils-libelf-devel".
 
+config BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE
+	bool "Needs host pahole"
+	help
+	  Some Linux kernel configuration options (such as
+	  CONFIG_DEBUG_INFO_BTF) require building a host
+	  program called pahole. Enabling this option will ensure
+	  host-pahole gets built before the Linux kernel.
+
+	  Enable this option if you get a Linux kernel build failure
+	  such as "BTF: .tmp_vmlinux.btf: pahole (pahole) is not
+	  available".
+
 # Linux extensions
 source "linux/Config.ext.in"
 
diff --git a/linux/linux.mk b/linux/linux.mk
index 61fdc0c76c..acb2464e8d 100644
--- a/linux/linux.mk
+++ b/linux/linux.mk
@@ -114,6 +114,17 @@ ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF),y)
 LINUX_DEPENDENCIES += host-elfutils host-pkgconf
 endif
 
+ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE),y)
+LINUX_DEPENDENCIES += host-pahole
+else
+define LINUX_FIXUP_CONFIG_PAHOLE_CHECK
+        if grep -q "^CONFIG_DEBUG_INFO_BTF=y" $(KCONFIG_DOT_CONFIG); then \
+                echo "To use CONFIG_DEBUG_INFO_BTF, enable host-pahole (BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE)" 1>&2; \
+		exit 1; \
+        fi
+endef
+endif
+
 # If host-uboot-tools is selected by the user, assume it is needed to
 # create a custom image
 ifeq ($(BR2_PACKAGE_HOST_UBOOT_TOOLS),y)
@@ -324,6 +335,7 @@ define LINUX_KCONFIG_FIXUP_CMDS
 		$(call KCONFIG_DISABLE_OPT,$(opt))
 	)
 	$(LINUX_FIXUP_CONFIG_ENDIANNESS)
+	$(LINUX_FIXUP_CONFIG_PAHOLE_CHECK)
 	$(if $(BR2_arm)$(BR2_armeb),
 		$(call KCONFIG_ENABLE_OPT,CONFIG_AEABI))
 	$(if $(BR2_powerpc)$(BR2_powerpc64)$(BR2_powerpc64le),
diff --git a/package/pahole/Config.in.host b/package/pahole/Config.in.host
index e427629632..521874961b 100644
--- a/package/pahole/Config.in.host
+++ b/package/pahole/Config.in.host
@@ -3,4 +3,6 @@ config BR2_PACKAGE_HOST_PAHOLE
 	help
 	  Pahole and other DWARF utils.
 
+	  Select this if you want to build a kernel with CONFIG_DEBUG_INFO_BTF set.
+
 	  https://git.kernel.org/pub/scm/devel/pahole/pahole.git
-- 
2.30.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH v2 1/2] package/pahole: new host package
  2021-12-22 17:49 ` [Buildroot] [PATCH v2 1/2] package/pahole: new host package Francis Laniel
@ 2022-01-13 21:40   ` Thomas Petazzoni
  0 siblings, 0 replies; 6+ messages in thread
From: Thomas Petazzoni @ 2022-01-13 21:40 UTC (permalink / raw)
  To: Francis Laniel; +Cc: Samuel Martin, buildroot

On Wed, 22 Dec 2021 18:49:04 +0100
Francis Laniel <flaniel@linux.microsoft.com> wrote:

> pahole is a tool used to show data structure embedded in debugging information
> formats like DWARF.
> It is notably needed by the Linux kernel to generate BPF Type Format (BTF)
> information used by Compile Once - Run Everywhere (CO-RE) BPF tools.
> 
> To be built, pahole needs __LIB to be set to lib at stated in its README.
> 
> Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
> ---
>  DEVELOPERS                    |  3 +++
>  package/Config.in.host        |  1 +
>  package/pahole/Config.in.host |  6 ++++++
>  package/pahole/pahole.hash    |  3 +++
>  package/pahole/pahole.mk      | 18 ++++++++++++++++++
>  5 files changed, 31 insertions(+)
>  create mode 100644 package/pahole/Config.in.host
>  create mode 100644 package/pahole/pahole.hash
>  create mode 100644 package/pahole/pahole.mk

Applied to master with some minor formatting tweaks reported by "make
check-package". Thanks a lot!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH v2 2/2] linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE
  2021-12-22 17:49 ` [Buildroot] [PATCH v2 2/2] linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE Francis Laniel
@ 2022-01-13 21:41   ` Thomas Petazzoni
  2022-01-17 13:51     ` Francis Laniel
  0 siblings, 1 reply; 6+ messages in thread
From: Thomas Petazzoni @ 2022-01-13 21:41 UTC (permalink / raw)
  To: Francis Laniel; +Cc: Samuel Martin, buildroot

Hello,

On Wed, 22 Dec 2021 18:49:05 +0100
Francis Laniel <flaniel@linux.microsoft.com> wrote:

> CONFIG_DEBUG_BTF_INFO relies on pahole, so kernel DWARF are converted to BTF.
> If CONFIG_DEBUG_BTF_INFO is set and BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE not,
> an error message is shown and .config is not written.
> 
> Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>

Thanks, applied, with one change.

> diff --git a/package/pahole/Config.in.host b/package/pahole/Config.in.host
> index e427629632..521874961b 100644
> --- a/package/pahole/Config.in.host
> +++ b/package/pahole/Config.in.host
> @@ -3,4 +3,6 @@ config BR2_PACKAGE_HOST_PAHOLE
>  	help
>  	  Pahole and other DWARF utils.
>  
> +	  Select this if you want to build a kernel with CONFIG_DEBUG_INFO_BTF set.
> +

I've dropped this, because it's not really this option that should be
enabled if the kernel is configured with CONFIG_DEBUG_INFO_BTF, but the
BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE option.

Thanks for your contribution!

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH v2 2/2] linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE
  2022-01-13 21:41   ` Thomas Petazzoni
@ 2022-01-17 13:51     ` Francis Laniel
  0 siblings, 0 replies; 6+ messages in thread
From: Francis Laniel @ 2022-01-17 13:51 UTC (permalink / raw)
  To: Thomas Petazzoni; +Cc: Samuel Martin, buildroot

Hi.


Le jeudi 13 janvier 2022, 22:41:26 CET Thomas Petazzoni a écrit :
> Hello,
> 
> On Wed, 22 Dec 2021 18:49:05 +0100
> 
> Francis Laniel <flaniel@linux.microsoft.com> wrote:
> > CONFIG_DEBUG_BTF_INFO relies on pahole, so kernel DWARF are converted to
> > BTF. If CONFIG_DEBUG_BTF_INFO is set and
> > BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE not, an error message is shown and
> > .config is not written.
> > 
> > Signed-off-by: Francis Laniel <flaniel@linux.microsoft.com>
> 
> Thanks, applied, with one change.
> 
> > diff --git a/package/pahole/Config.in.host b/package/pahole/Config.in.host
> > index e427629632..521874961b 100644
> > --- a/package/pahole/Config.in.host
> > +++ b/package/pahole/Config.in.host
> > @@ -3,4 +3,6 @@ config BR2_PACKAGE_HOST_PAHOLE
> > 
> >  	help
> >  	
> >  	  Pahole and other DWARF utils.
> > 
> > +	  Select this if you want to build a kernel with CONFIG_DEBUG_INFO_BTF
> > set. +
> 
> I've dropped this, because it's not really this option that should be
> enabled if the kernel is configured with CONFIG_DEBUG_INFO_BTF, but the
> BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE option.

Your modification made the thing clearer, thank you for it!

> Thanks for your contribution!

You are welcome! I thank you for the merge!

> Best regards,

Best regards.
 
> Thomas




_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-01-17 13:51 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-22 17:49 [Buildroot] [PATCH v2 0/2] Enable BTF headers for Linux kernel Francis Laniel
2021-12-22 17:49 ` [Buildroot] [PATCH v2 1/2] package/pahole: new host package Francis Laniel
2022-01-13 21:40   ` Thomas Petazzoni
2021-12-22 17:49 ` [Buildroot] [PATCH v2 2/2] linux: add BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE Francis Laniel
2022-01-13 21:41   ` Thomas Petazzoni
2022-01-17 13:51     ` Francis Laniel

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.