[PATCH] syslinux: always use the BFD linker

[PATCH] syslinux: always use the BFD linker

[Allen Wild]

Syslinux compiles successfully with Gold, but at runtime Isolinux fails
to boot, getting stuck at the version screen without loading the menu.
Forcing the BFD linker fixes this.

To future-proof, use an inline python replace rather than duplicating
the definition of LD in bitbake.conf

Signed-off-by: Allen Wild <allenwild93@gmail.com>
---
 meta/recipes-devtools/syslinux/syslinux_6.03.bb | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/syslinux/syslinux_6.03.bb b/meta/recipes-devtools/syslinux/syslinux_6.03.bb
index f8b1094096..c61adec577 100644
--- a/meta/recipes-devtools/syslinux/syslinux_6.03.bb
+++ b/meta/recipes-devtools/syslinux/syslinux_6.03.bb
@@ -34,6 +34,11 @@ COMPATIBLE_HOST = '(x86_64|i.86).*-(linux|freebsd.*)'
 INSANE_SKIP_${PN}-misc = "arch"
 INSANE_SKIP_${PN}-chain = "arch"
 
+# isolinux isn't compatible with the Gold linker, so force BFD
+# some python replacing is used here to avoid getting out of sync with the definition in bitbake.conf
+SYSLINUX_LD = "${LD}"
+SYSLINUX_LD_class-target = "${@d.getVar('LD').replace('%sld'%d.getVar('HOST_PREFIX'), '%sld.bfd'%d.getVar('HOST_PREFIX'))}"
+
 EXTRA_OEMAKE = " \
 	BINDIR=${bindir} SBINDIR=${sbindir} LIBDIR=${libdir} \
 	DATADIR=${datadir} MANDIR=${mandir} INCDIR=${includedir} \
@@ -58,11 +63,11 @@ do_compile() {
 
 	# Rebuild only the installer; keep precompiled bootloaders
 	# as per author's request (doc/distrib.txt)
-	oe_runmake CC="${CC} ${CFLAGS}" LD="${LD}" LDFLAGS="${LDFLAGS}" firmware="bios" installer
+	oe_runmake CC="${CC} ${CFLAGS}" LD="${SYSLINUX_LD}" LDFLAGS="${LDFLAGS}" firmware="bios" installer
 }
 
 do_install() {
-	oe_runmake CC="${CC} ${CFLAGS}" LD="${LD}" install INSTALLROOT="${D}" firmware="bios"
+	oe_runmake CC="${CC} ${CFLAGS}" LD="${SYSLINUX_LD}" install INSTALLROOT="${D}" firmware="bios"
 
 	install -d ${D}${datadir}/syslinux/
 	install -m 644 ${S}/bios/core/ldlinux.sys ${D}${datadir}/syslinux/
-- 
2.14.1

Re: [PATCH] syslinux: always use the BFD linker

[André Draszik]

On Mon, 2017-12-11 at 21:49 -0500, Allen Wild wrote:
> Syslinux compiles successfully with Gold, but at runtime Isolinux fails
> to boot, getting stuck at the version screen without loading the menu.
> Forcing the BFD linker fixes this.
> 
> To future-proof, use an inline python replace rather than duplicating
> the definition of LD in bitbake.conf

Hm, other recipes that have this need typically simply do:

LDFLAGS_append = "${@bb.utils.contains('DISTRO_FEATURES', 'ld-is-gold', ' -fuse-ld=bfd ', '', d)}"

Would a similar approach work here, too? Instead of this big change.

Cheers,
Andre'
> 

Re: [PATCH] syslinux: always use the BFD linker

[Allen Wild]

Oops, forgot mail the list when sending my first reply.

I tried appending LDFLAGS, but it doesn't work for syslinux. It's
during do_install, not do_compile, that $(LD) is called directly to
link libutil.elf, libcom32.elf, libgpl.elf, and ldlinux.elf.

A python-free but more intrusive appropach to fix this could be to add
a new variable e.g. HOST_LDNAME in bitbake.conf and incorporate it
into the definition of LD. It would default to "ld" but could be
overridden by recipes like syslinux where appending LDFLAGS doesn't
work.

If we don't expect the bitbake.conf definition of LD to change,
syslinux could also just simply redefine LD =
"${HOST_PREFIX}ld.bfd${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

Thoughts?
--Allen

On Tue, Dec 12, 2017 at 4:46 PM, Allen Wild <allenwild93@gmail.com> wrote:
>> Hm, other recipes that have this need typically simply do:
>>
>> LDFLAGS_append = "${@bb.utils.contains('DISTRO_FEATURES', 'ld-is-gold', ' -fuse-ld=bfd ', '', d)}"
>>
>> Would a similar approach work here, too? Instead of this big change.
>
> I thought about that, but grepping the Makefiles appeared to show many
> usages of $(LD) directly rather than using gcc as the linker, so I
> assumed it wouldn't work.
>
> However, I just looked more closely at my log.do_compile and I don't
> see anything appear to call LD directly, everything is getting run
> through x86_64-oe-linux-gcc.
> Clearly the LD variable passed to make affects this consistently, but
> now I don't know why. I'll try appending LDFLAGS and see if that
> works.
>
> Regards,
> Allen
>
> On Tue, Dec 12, 2017 at 3:55 AM, André Draszik <git@andred.net> wrote:
>> On Mon, 2017-12-11 at 21:49 -0500, Allen Wild wrote:
>>> Syslinux compiles successfully with Gold, but at runtime Isolinux fails
>>> to boot, getting stuck at the version screen without loading the menu.
>>> Forcing the BFD linker fixes this.
>>>
>>> To future-proof, use an inline python replace rather than duplicating
>>> the definition of LD in bitbake.conf
>>
>> Hm, other recipes that have this need typically simply do:
>>
>> LDFLAGS_append = "${@bb.utils.contains('DISTRO_FEATURES', 'ld-is-gold', ' -fuse-ld=bfd ', '', d)}"
>>
>> Would a similar approach work here, too? Instead of this big change.
>>
>> Cheers,
>> Andre'
>>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] syslinux: always use the BFD linker

[Andre McCurdy]

On Tue, Dec 12, 2017 at 6:42 PM, Allen Wild <allenwild93@gmail.com> wrote:
> Oops, forgot mail the list when sending my first reply.
>
> I tried appending LDFLAGS, but it doesn't work for syslinux. It's
> during do_install, not do_compile, that $(LD) is called directly to
> link libutil.elf, libcom32.elf, libgpl.elf, and ldlinux.elf.
>
> A python-free but more intrusive appropach to fix this could be to add
> a new variable e.g. HOST_LDNAME in bitbake.conf and incorporate it
> into the definition of LD. It would default to "ld" but could be
> overridden by recipes like syslinux where appending LDFLAGS doesn't
> work.
>
> If we don't expect the bitbake.conf definition of LD to change,
> syslinux could also just simply redefine LD =
> "${HOST_PREFIX}ld.bfd${TOOLCHAIN_OPTIONS} ${HOST_LD_ARCH}"

If you don't need to modify the value of LD in the bitbake environment
(only the value seen by Make) perhaps try something like this:

  LD='${LD} -fuse-ld=bfd"

passed on the Make command line?

> Thoughts?
> --Allen
>
> On Tue, Dec 12, 2017 at 4:46 PM, Allen Wild <allenwild93@gmail.com> wrote:
>>> Hm, other recipes that have this need typically simply do:
>>>
>>> LDFLAGS_append = "${@bb.utils.contains('DISTRO_FEATURES', 'ld-is-gold', ' -fuse-ld=bfd ', '', d)}"
>>>
>>> Would a similar approach work here, too? Instead of this big change.
>>
>> I thought about that, but grepping the Makefiles appeared to show many
>> usages of $(LD) directly rather than using gcc as the linker, so I
>> assumed it wouldn't work.
>>
>> However, I just looked more closely at my log.do_compile and I don't
>> see anything appear to call LD directly, everything is getting run
>> through x86_64-oe-linux-gcc.
>> Clearly the LD variable passed to make affects this consistently, but
>> now I don't know why. I'll try appending LDFLAGS and see if that
>> works.
>>
>> Regards,
>> Allen
>>
>> On Tue, Dec 12, 2017 at 3:55 AM, André Draszik <git@andred.net> wrote:
>>> On Mon, 2017-12-11 at 21:49 -0500, Allen Wild wrote:
>>>> Syslinux compiles successfully with Gold, but at runtime Isolinux fails
>>>> to boot, getting stuck at the version screen without loading the menu.
>>>> Forcing the BFD linker fixes this.
>>>>
>>>> To future-proof, use an inline python replace rather than duplicating
>>>> the definition of LD in bitbake.conf
>>>
>>> Hm, other recipes that have this need typically simply do:
>>>
>>> LDFLAGS_append = "${@bb.utils.contains('DISTRO_FEATURES', 'ld-is-gold', ' -fuse-ld=bfd ', '', d)}"
>>>
>>> Would a similar approach work here, too? Instead of this big change.
>>>
>>> Cheers,
>>> Andre'
>>>>
>>> --
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] syslinux: always use the BFD linker

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 528 bytes --]

On 13 December 2017 at 02:42, Allen Wild <allenwild93@gmail.com> wrote:

> Oops, forgot mail the list when sending my first reply.
>
> I tried appending LDFLAGS, but it doesn't work for syslinux. It's
> during do_install, not do_compile, that $(LD) is called directly to
> link libutil.elf, libcom32.elf, libgpl.elf, and ldlinux.elf.
>

That's a terrible implementation of "install" and it should be using
LDFLAGS, so that's an upstreamable patch.  As Andre says, adding the flag
to LD directly should work.

Ross

[-- Attachment #2: Type: text/html, Size: 893 bytes --]

Re: [PATCH] syslinux: always use the BFD linker

[Allen Wild]

[-- Attachment #1: Type: text/plain, Size: 2142 bytes --]

>   LD='${LD} -fuse-ld=bfd"

This doesn't work because the -fuse-ld option is only relevant when using
gcc as a wrapper for the linker.  The Makefile calls x86_64-oe-linux-ld
directly (which is a hardlink of x86_64-oe-linux-ld.gold or
x86_64-oe-linux-ld.bfd) so when the linker starts up it's already Gold,
rather than letting gcc call ld automatically.

Syslinux calls LD directly to use a custom linker script and avoids linking
with system libraries (not uncommon for bare-metal programs like a
bootloader). The -nostdlib gcc option can be used to get the same effect
when linking with gcc as a wrapper, but doing that involves patching many
syslinux makefiles to prefix LDFLAGS options with -Wl.

I have a working version of a patch that does this (19 changes in 15
makefiles), I can submit it here after some more testing if others think
it's an acceptable fix. I ignored the EFI targets for now, but such a patch
could be upstreamed after updating and testing the EFI code too.

>That's a terrible implementation of "install"

I agree, and I think that the fact that libgpl.elf and ldlinux.elf are
getting rebuilt at all means the current recipe violates upstream's "don't
recompile the bootloader itself" wishes in doc/distrib.txt. The tarball we
use includes shipped versions of those elf files and a bunch of c32 files,
but for some reason make is deciding to rebuild them from source when we
run make install. I haven't been able to determine why that's the case.

Allen

On Wed, Dec 13, 2017 at 6:53 AM, Burton, Ross <ross.burton@intel.com> wrote:

> On 13 December 2017 at 02:42, Allen Wild <allenwild93@gmail.com> wrote:
>
>> Oops, forgot mail the list when sending my first reply.
>>
>> I tried appending LDFLAGS, but it doesn't work for syslinux. It's
>> during do_install, not do_compile, that $(LD) is called directly to
>> link libutil.elf, libcom32.elf, libgpl.elf, and ldlinux.elf.
>>
>
> That's a terrible implementation of "install" and it should be using
> LDFLAGS, so that's an upstreamable patch.  As Andre says, adding the flag
> to LD directly should work.
>
> Ross
>

[-- Attachment #2: Type: text/html, Size: 3166 bytes --]

Re: [PATCH] syslinux: always use the BFD linker

[Khem Raj]

On 12/16/17 12:57 PM, Allen Wild wrote:
>>   LD='${LD} -fuse-ld=bfd"
> 
> This doesn't work because the -fuse-ld option is only relevant when
> using gcc as a wrapper for the linker.  The Makefile calls
> x86_64-oe-linux-ld directly (which is a hardlink of
> x86_64-oe-linux-ld.gold or x86_64-oe-linux-ld.bfd) so when the linker
> starts up it's already Gold, rather than letting gcc call ld automatically.
> 
> Syslinux calls LD directly to use a custom linker script and avoids
> linking with system libraries (not uncommon for bare-metal programs like
> a bootloader). The -nostdlib gcc option can be used to get the same
> effect when linking with gcc as a wrapper, but doing that involves
> patching many syslinux makefiles to prefix LDFLAGS options with -Wl.

you need to override LD in recipe much like how KERNEL_LD is setup and
then ensure that LD is honored by makefiles
and sets LD to weakly assign to 'ld' to fix it properly.

> 
> I have a working version of a patch that does this (19 changes in 15
> makefiles), I can submit it here after some more testing if others think
> it's an acceptable fix. I ignored the EFI targets for now, but such a
> patch could be upstreamed after updating and testing the EFI code too.
> 
>>That's a terrible implementation of "install"
> 
> I agree, and I think that the fact that libgpl.elf and ldlinux.elf are
> getting rebuilt at all means the current recipe violates upstream's
> "don't recompile the bootloader itself" wishes in doc/distrib.txt. The
> tarball we use includes shipped versions of those elf files and a bunch
> of c32 files, but for some reason make is deciding to rebuild them from
> source when we run make install. I haven't been able to determine why
> that's the case.
> 
> Allen
> 
> On Wed, Dec 13, 2017 at 6:53 AM, Burton, Ross <ross.burton@intel.com
> <mailto:ross.burton@intel.com>> wrote:
> 
>     On 13 December 2017 at 02:42, Allen Wild <allenwild93@gmail.com
>     <mailto:allenwild93@gmail.com>> wrote:
> 
>         Oops, forgot mail the list when sending my first reply.
> 
>         I tried appending LDFLAGS, but it doesn't work for syslinux. It's
>         during do_install, not do_compile, that $(LD) is called directly to
>         link libutil.elf, libcom32.elf, libgpl.elf, and ldlinux.elf.
> 
> 
>     That's a terrible implementation of "install" and it should be using
>     LDFLAGS, so that's an upstreamable patch.  As Andre says, adding the
>     flag to LD directly should work.
> 
>     Ross 
> 
> 
> 

Re: [PATCH] syslinux: always use the BFD linker

[Yang, Zhangle (Eric)]