Re: [OE-core] [PATCH] make-mod-scripts: preserve libraries when rm_work is used

[Bruce Ashfield <bruce.ashfield@gmail.com>]

On Thu, Apr 27, 2023 at 6:32 PM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>
> Hi Bruce,
>
> I have been testing your patch and have some comments.
> In some of my kernels I don't have the pkg-config changes and so I have some fails linking the scripts/sign-file
> because for static linking with the libcrypto we need the -ldl -pthread.
>
> To fix my build I need to override the CRYPTO_LIBS in my kernel because they use the hardcoded pkg-config
> where it is not possible to pass the --static argument.
>
> With following change on top of your patch I can build moist of my kernels:
>
>         export HOSTLDFLAGS="-lz"
>
> +       HOSTPKG_CONFIG="pkg-config --static"
> +       # override CRYPTO_LIBS since HOSTPKG_CONFIG lands only in v5.19-rc1
> +       # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
> +       CRYPTO_LIBS="$(pkg-config --static --libs libcrypto 2>/dev/null || echo -lcrypto)"
> +
>         unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
>         for t in prepare scripts_basic scripts; do
>                 oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
>                 AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}" \
> -               HOSTPKG_CONFIG="pkg-config --static" \
> +               HOSTPKG_CONFIG="${HOSTPKG_CONFIG}" CRYPTO_LIBS="${CRYPTO_LIBS}" \
>                 -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
>         done
>
>
> I think belive that the LIBELF_LIBS needs the same fix for the cases where HOSTPKG_CONFIG is not available.
> Also I think there is a typo in the LIBELF_LIBS because you first populate it with pkg-config but on the export the variable is redefined
> with the HOST_LIBELF_LIBS. I made a litle change too on that:
>
>         # for pre-5.15 kernels
> -       LIBELF_LIBS=$(pkg-config libelf --libs 2>/dev/null || echo -lelf)
> -       export LIBELF_LIBS="$HOST_LIBELF_LIBS -lz"
> +       LIBELF_LIBS=$(pkg-config --static libelf --libs 2>/dev/null || echo -lelf)
> +       export LIBELF_LIBS="$LIBELF_LIBS -lz"
>         export HOSTLDFLAGS="-lz"
>
> Thanks for you help.

Those are definitely plausible tweaks to the patch, I was providing
the two techniques for that reason, and you've used them
appropriately.

Let me roll your changes into my patch, re-test and I'll submit it to
the mailing list as a v2.

Thanks for the testing, and fixup, I knew there would be things missing! :)

Bruce

>
> Jose
>
> Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia segunda, 24/04/2023 à(s) 20:25:
>>
>> On Mon, Apr 24, 2023 at 6:30 AM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>> >
>> >
>> >
>> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia domingo, 23/04/2023 à(s) 20:55:
>> >>
>> >> On Sat, Apr 22, 2023 at 9:06 AM Christoph Lauer
>> >> <Christoph.Lauer@email.de> wrote:
>> >> >
>> >> > Am 21.04.23 um 22:28 schrieb Bruce Ashfield:
>> >> > > On Wed, Apr 19, 2023 at 11:03 PM Bruce Ashfield via
>> >> > > lists.openembedded.org
>> >> > > <bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
>> >> > >>
>> >> > >> On Wed, Apr 19, 2023 at 6:54 PM Richard Purdie
>> >> > >> <richard.purdie@linuxfoundation.org> wrote:
>> >> > >>>
>> >> > >>> On Wed, 2023-04-19 at 23:34 +0100, Jose Quaresma wrote:
>> >> > >>>> Hi,
>> >> > >>>>
>> >> > >>>> Not related with the previous discussion but just for
>> >> > >>>> your information.
>> >> > >>>> The rm_work.bbclass has an exception for the kernel recipes [1].
>> >> > >>>> So I don't understand why we can't do the same for the make-mod-
>> >> > >>>> scripts
>> >> > >>>> who is the twin brother of all these kernel recipes.
>> >> > >>>>
>> >> > >>>> [1]
>> >> > >>>> https://git.openembedded.org/openembedded-core/tree/meta/classes/rm_work.bbclass#n168
>> >> > >>>
>> >> > >>> Ideally we wouldn't be doing this for the kernel recipes.
>> >> > >>>
>> >> > >>> There is also a big difference to that and the proposed patch. The
>> >> > >>> proposed patch was preserving a specific directory rather than an
>> >> > >>> entire recipe. Removing the task stamps but leaving a small piece of
>> >> > >>> WORKDIR is quite different to preserving WORKDIR and STAMPS for a
>> >> > >>> specific recipe. The former is not tested and will break things. The
>> >> > >>> latter is better tolerated by bitbake.
>> >> > >>
>> >> > >> Agreed.
>> >> > >>
>> >> > >> Plus, I am working on this now.
>> >> > >>
>> >> > >> I have static linking of the scripts/tools working, but what I haven't
>> >> > >> figured out is how to do that without patching the Makefiles.
>> >> > >>
>> >> > >
>> >> > > It turned out to be quite the battle to get older kernels what was
>> >> > > required for static linking of the tools.
>> >> > >
>> >> > > Attached is my WIP patch. I'm out of the office early next week, but
>> >> > > will revisit it once I'm back.
>> >> > >
>> >> > > Bruce
>> >> > >
>> >> > >> Next up will be some rpath trickery.
>> >> > >>
>> >> > >> Bruce
>> >> > >>
>> >> > >>>
>> >> > >>> So yes, we could do the same. I'm sure there will be other recipes
>> >> > >>> people want to preserve for other reasons. Where do we draw the line?
>> >> > >>> We could preserve everything and drop rm_work, then we wouldn't have
>> >> > >>> these problems? :)
>> >> > >>>
>> >> > >>> Cheers,
>> >> > >>>
>> >> > >>> Richard
>> >> > >>
>> >> > >>
>> >> > >>
>> >> > >> --
>> >> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> > >> thee at its end
>> >> > >> - "Use the force Harry" - Gandalf, Star Trek II
>> >> > >>
>> >> > >> -=-=-=-=-=-=-=-=-=-=-=-
>> >> > >> Links: You receive all messages sent to this group.
>> >> > >> View/Reply Online (#180233): https://lists.openembedded.org/g/openembedded-core/message/180233
>> >> > >> Mute This Topic: https://lists.openembedded.org/mt/98296212/1050810
>> >> > >> Group Owner: openembedded-core+owner@lists.openembedded.org
>> >> > >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
>> >> > >> -=-=-=-=-=-=-=-=-=-=-=-
>> >> > >>
>> >> > >
>> >> > >
>> >> >
>> >> > Thank you for your work, I see you put some time and effort into it.
>> >> > HOSTPKG_CONFIG is, as you mentioned, available since kernel version 5.19
>> >>
>> >> Yes, I realize that and documented it in the patch ... but I also
>> >> tested on pre-5.19 kernels and what I have in the patch works. Did it
>> >> not work in your testing ?
>> >
>> >
>> > I will test the patch on a couple of kernel versions with some of them pre-5.19
>> > but all in 5 major versions.
>> > I will say something about my results later this week.
>>
>> 5.15-stable also has the pkg-config changes
>>
>> Bruce
>>
>> >
>> > Thanks for working on this one.
>> >
>> > Jose
>> >
>> >>
>> >>
>> >> > (see kernel patch [1]), so we need a way to call 'pkg-config --static'
>> >> > with pre-5.19 kernels. A way without modifying the Makefile would be to
>> >> > modify openssls pkg-config in recipe-sysroot-native of make-mod-script,
>> >> > so 'pkg-config --libs' actually shows the dependencies of 'pkg-config
>> >> > --static --libs', but it's a bit hacky.
>> >>
>> >> Already considered, and discarded. That's not going to fly.
>> >>
>> >> >
>> >> > Also fully-static executables still need the same glibc during runtime
>> >> > that they were built with, which makes them error-prone and is generally
>> >> > discouraged. As an alternative, we could build dynamic executables that
>> >> > use the static libcrypto library. The linker links by default against
>> >> > the shared library, so we could remove them from recipe-sysroot-native
>> >> > to force linking against the static library (again, somewhat hacky).
>> >>
>> >> Also considered and discarded.
>> >>
>> >> As do the dynamically linked ones for the c runtime. We aren't talking
>> >> about using these outside of a single build and they are generated on
>> >> the fly, so again, there's very little concern about runtimes changing
>> >> after linking.. There's less risk in static than in the alternatives.
>> >>
>> >> Bruce
>> >>
>> >>
>> >> >
>> >> > [1]
>> >> > https://github.com/torvalds/linux/commit/d5ea4fece4508bf8e72b659cd22fa4840d8d61e5
>> >>
>> >>
>> >>
>> >> --
>> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> thee at its end
>> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >
>> >
>> >
>> > --
>> > Best regards,
>> >
>> > José Quaresma
>>
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Best regards,
>
> José Quaresma



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II