Re: [OE-core] [PATCH] make-mod-scripts: preserve libraries when rm_work is used

[Bruce Ashfield <bruce.ashfield@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 9873 bytes --]

Attached is v2 of the patch. I've consolidated the suggested changes.

I'm soaking it a bit longer, and then will send it as part of my next
consolidated pull request.

Bruce

On Thu, Apr 27, 2023 at 9:26 PM Bruce Ashfield via
lists.openembedded.org
<bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
>
> On Thu, Apr 27, 2023 at 6:32 PM Jose Quaresma <quaresma.jose@gmail.com> wrote:
> >
> > Hi Bruce,
> >
> > I have been testing your patch and have some comments.
> > In some of my kernels I don't have the pkg-config changes and so I have some fails linking the scripts/sign-file
> > because for static linking with the libcrypto we need the -ldl -pthread.
> >
> > To fix my build I need to override the CRYPTO_LIBS in my kernel because they use the hardcoded pkg-config
> > where it is not possible to pass the --static argument.
> >
> > With following change on top of your patch I can build moist of my kernels:
> >
> >         export HOSTLDFLAGS="-lz"
> >
> > +       HOSTPKG_CONFIG="pkg-config --static"
> > +       # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in v5.19-rc1
> > +       # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
> > +       CRYPTO_LIBS="$(pkg-config --static --libs libcrypto 2>/dev/null || echo -lcrypto)"
> > +
> >         unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
> >         for t in prepare scripts_basic scripts; do
> >                 oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
> >                 AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \
> > -               HOSTPKG_CONFIG="pkg-config --static" \
> > +               HOSTPKG_CONFIG="${HOSTPKG_CONFIG}" CRYPTO_LIBS="${CRYPTO_LIBS}" \
> >                 -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
> >         done
> >
> >
> > I think belive that the LIBELF_LIBS needs the same fix for the cases where HOSTPKG_CONFIG is not available.
> > Also I think there is a typo in the LIBELF_LIBS because you first populate it with pkg-config but on the export the variable is redefined
> > with the HOST_LIBELF_LIBS. I made a litle change too on that:
> >
> >         # for pre-5.15 kernels
> > -       LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo -lelf)
> > -       export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz"
> > +       LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || echo -lelf)
> > +       export LIBELF_LIBS="$LIBELF_LIBS -lz"
> >         export HOSTLDFLAGS="-lz"
> >
> > Thanks for you help.
>
> Those are definitely plausible tweaks to the patch, I was providing
> the two techniques for that reason, and you've used them
> appropriately.
>
> Let me roll your changes into my patch, re-test and I'll submit it to
> the mailing list as a v2.
>
> Thanks for the testing, and fixup, I knew there would be things missing! :)
>
> Bruce
>
> >
> > Jose
> >
> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda, 24/04/2023 à(s) 20:25:
> >>
> >> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma <quaresma.jose@gmail.com> wrote:
> >> >
> >> >
> >> >
> >> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, 23/04/2023 à(s) 20:55:
> >> >>
> >> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer
> >> >> <Christoph.Lauer@email.de> wrote:
> >> >> >
> >> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield:
> >> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via
> >> >> > > lists.openembedded.org
> >> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
> >> >> > >>
> >> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie
> >> >> > >> <richard.purdie@linuxfoundation.org> wrote:
> >> >> > >>>
> >> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote:
> >> >> > >>>> Hi,
> >> >> > >>>>
> >> >> > >>>> Not related with the previous discussion but just for
> >> >> > >>>> your information.
> >> >> > >>>> The rm_work.bbclass has an exception for the kernel recipes [1].
> >> >> > >>>> So I don't understand why we can't do the same for the make-mod-
> >> >> > >>>> scripts
> >> >> > >>>> who is the twin brother of all these kernel recipes.
> >> >> > >>>>
> >> >> > >>>> [1]
> >> >> > >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168
> >> >> > >>>
> >> >> > >>> Ideally we wouldn't be doing this for the kernel recipes.
> >> >> > >>>
> >> >> > >>> There is also a big difference to that and the proposed patch. The
> >> >> > >>> proposed patch was preserving a specific directory rather than an
> >> >> > >>> entire recipe. Removing the task stamps but leaving a small piece of
> >> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a
> >> >> > >>> specific recipe. The former is not tested and will break things. The
> >> >> > >>> latter is better tolerated by bitbake.
> >> >> > >>
> >> >> > >> Agreed.
> >> >> > >>
> >> >> > >> Plus, I am working on this now.
> >> >> > >>
> >> >> > >> I have static linking of the scripts/tools working, but what I haven't
> >> >> > >> figured out is how to do that without patching the Makefiles.
> >> >> > >>
> >> >> > >
> >> >> > > It turned out to be quite the battle to get older kernels what was
> >> >> > > required for static linking of the tools.
> >> >> > >
> >> >> > > Attached is my WIP patch. I'm out of the office early next week, but
> >> >> > > will revisit it once I'm back.
> >> >> > >
> >> >> > > Bruce
> >> >> > >
> >> >> > >> Next up will be some rpath trickery.
> >> >> > >>
> >> >> > >> Bruce
> >> >> > >>
> >> >> > >>>
> >> >> > >>> So yes, we could do the same. I'm sure there will be other recipes
> >> >> > >>> people want to preserve for other reasons. Where do we draw the line?
> >> >> > >>> We could preserve everything and drop rm_work, then we wouldn't have
> >> >> > >>> these problems? :)
> >> >> > >>>
> >> >> > >>> Cheers,
> >> >> > >>>
> >> >> > >>> Richard
> >> >> > >>
> >> >> > >>
> >> >> > >>
> >> >> > >> --
> >> >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> >> > >> thee at its end
> >> >> > >> - "Use the force Harry" - Gandalf, Star Trek II
> >> >> > >>
> >> >> > >>
> >> >> > >>
> >> >> > >
> >> >> > >
> >> >> >
> >> >> > Thank you for your work, I see you put some time and effort into it.
> >> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19
> >> >>
> >> >> Yes, I realize that and documented it in the patch ... but I also
> >> >> tested on pre-5.19 kernels and what I have in the patch works. Did it
> >> >> not work in your testing ?
> >> >
> >> >
> >> > I will test the patch on a couple of kernel versions with some of them pre-5.19
> >> > but all in 5 major versions.
> >> > I will say something about my results later this week.
> >>
> >> 5.15-stable also has the pkg-config changes
> >>
> >> Bruce
> >>
> >> >
> >> > Thanks for working on this one.
> >> >
> >> > Jose
> >> >
> >> >>
> >> >>
> >> >> > (see kernel patch [1]), so we need a way to call 'pkg-config --static'
> >> >> > with pre-5.19 kernels. A way without modifying the Makefile would be to
> >> >> > modify openssls pkg-config in recipe-sysroot-native of make-mod-script,
> >> >> > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config
> >> >> > --static --libs', but it's a bit hacky.
> >> >>
> >> >> Already considered, and discarded. That's not going to fly.
> >> >>
> >> >> >
> >> >> > Also fully-static executables still need the same glibc during runtime
> >> >> > that they were built with, which makes them error-prone and is generally
> >> >> > discouraged. As an alternative, we could build dynamic executables that
> >> >> > use the static libcrypto library. The linker links by default against
> >> >> > the shared library, so we could remove them from recipe-sysroot-native
> >> >> > to force linking against the static library (again, somewhat hacky).
> >> >>
> >> >> Also considered and discarded.
> >> >>
> >> >> As do the dynamically linked ones for the c runtime. We aren't talking
> >> >> about using these outside of a single build and they are generated on
> >> >> the fly, so again, there's very little concern about runtimes changing
> >> >> after linking.. There's less risk in static than in the alternatives.
> >> >>
> >> >> Bruce
> >> >>
> >> >>
> >> >> >
> >> >> > [1]
> >> >> > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> >> thee at its end
> >> >> - "Use the force Harry" - Gandalf, Star Trek II
> >> >
> >> >
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> > José Quaresma
> >>
> >>
> >>
> >> --
> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> thee at its end
> >> - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > Best regards,
> >
> > José Quaresma
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#180502): https://lists.openembedded.org/g/openembedded-core/message/180502
> Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: 0001-make-mod-scripts-force-static-linking-and-make-depen.patch --]
[-- Type: application/octet-stream, Size: 4766 bytes --]

From 4cd8f17bac82cb97f53fedf20fa4fca4b227737e Mon Sep 17 00:00:00 2001
From: Bruce Ashfield <bruce.ashfield@gmail.com>
Date: Fri, 21 Apr 2023 12:31:31 -0400
Subject: [PATCH] make-mod-scripts: force static linking and make dependencies
 explicit

When the scripts are prepared from the kernel soure, there are some
executables that can also be created (depending on kernel
configuration). sign-file and objtool are two examples which are
commonly created.

Due to the way that the kernel source is staged, and build artifacts
are stored for reuse and performance we can end up in a scenario where
the build artifacts based executable is linked against shared
libraries in the recipe native sysroot (ssl or crypto).

We could manipulate rpath, install the libraries to build-artifacts
and arrange to have available when the host tools are used or even
created a full native package from the tools generated out of scripts.

Those approaches have drawbacks in complexity, relocatability and/or
synchronization issues with the kernel source.

The chosen approach here is to force static linking on the tools,
which allows them to be placed into build artifacts without any
references to the recipe native sysroot.

kernel's newer than 5.15+ allow us to add the -static parameter
to pkg-config, but older kernels do not have that flexiblity. As
a result, we have two approaches to ensure that the libraries
needed for static linking are detected (one via pkg-config and
the other via host ldflags). In the future we can drop the ldflags
approach (when the oldest supported kernels are > 5.15).

There are also some potentially missing dependencies when tools
like sign-file are built, so we add them explicitly to the recipe
and avoid any races or implicit dependencies.

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb  | 30 +++++++++++++++++--
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
index 28e0807d1d..1c972f0d44 100644
--- a/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
+++ b/meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb
@@ -10,25 +10,49 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"
 
 S = "${WORKDIR}"
 
-do_configure[depends] += "virtual/kernel:do_shared_workdir openssl-native:do_populate_sysroot"
+# zlib is required when module signing is enabled
+do_configure[depends] += "virtual/kernel:do_shared_workdir openssl-native:do_populate_sysroot zlib-native:do_populate_sysroot"
 do_compile[depends] += "virtual/kernel:do_compile_kernelmodules"
 
 DEV_PKG_DEPENDENCY = ""
 
 DEPENDS += "bc-native bison-native"
 DEPENDS += "gmp-native"
+# required for module signing support
+DEPENDS += "elfutils-native"
 
-EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}""
-EXTRA_OEMAKE += " HOSTCXX="${BUILD_CXX} ${BUILD_CXXFLAGS} ${BUILD_LDFLAGS}" CROSS_COMPILE=${TARGET_PREFIX}"
+# we are statically building the support tools, since the output of the build is
+# stored in STAGING_KERNEL_BUILDDIR. We do not want any dynamic references to
+# libraries that are only present in the recipe native sysroot
+EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS} -static" HOSTCPP="${BUILD_CPP}""
+EXTRA_OEMAKE += " HOSTCXX="${BUILD_CXX} ${BUILD_CXXFLAGS} ${BUILD_LDFLAGS} -static" CROSS_COMPILE=${TARGET_PREFIX}"
 
 # Build some host tools under work-shared.  CC, LD, and AR are probably
 # not used, but this is the historical way of invoking "make scripts".
 #
 do_configure() {
+	# setup native pkg-config variables, HOSTPKG_CONFIG is available in newer kernels
+	# but we keep these to support older kernels that may not have the variable to
+	# abstract calls to pkg-config
+	export PKG_CONFIG_DIR="${STAGING_DIR_NATIVE}${libdir_native}/pkgconfig"
+	export PKG_CONFIG_PATH="$PKG_CONFIG_DIR:${STAGING_DATADIR_NATIVE}/pkgconfig"
+	export PKG_CONFIG_LIBDIR="$PKG_CONFIG_DIR"
+	export PKG_CONFIG_SYSROOT_DIR=""
+
+	# override CRYPTO_LIBS to support older kernels without HOSTPKG_CONFIG
+	CRYPTO_LIBS="$(pkg-config --static --libs libcrypto 2>/dev/null || echo -lcrypto)"
+
+	# for pre-5.15 kernels
+	LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || echo -lelf)
+	export LIBELF_LIBS="$LIBELF_LIBS -lz"
+	export HOSTLDFLAGS="-lz"
+
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
 	for t in prepare scripts_basic scripts; do
 		oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
 		AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \
+		HOSTPKG_CONFIG="pkg-config --static" \
+		CRYPTO_LIBS="${CRYPTO_LIBS}" \
 		-C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
 	done
 }
-- 
2.34.1