* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-03 14:21 ` Kirill A. Shutemov
@ 2018-07-03 14:27 ` Thomas Gleixner
2018-07-03 18:03 ` Andi Kleen
` (2 subsequent siblings)
3 siblings, 0 replies; 39+ messages in thread
From: Thomas Gleixner @ 2018-07-03 14:27 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, H. Peter Anvin, X86 ML, bero,
Andi Kleen
On Tue, 3 Jul 2018, Kirill A. Shutemov wrote:
> On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
> > On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
> > > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@redhat.com>:
> > > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
> > > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
> > > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
> > > >> QEMU/KVM. No messages are shown post-GRUB; the VM instantly reboots.
> > > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
> > > >> paging boot if kernel is above 4G") fixes it. I've attached our kernel
> > > >> config for reference, and am happy to test patches, provide sample QCOW
> > > >> images, etc.
> > > >
> > >
> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> > >
> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> > > too with the same symptoms
> >
> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
>
> -flto in LDFLAGS screws up this part of paging_prepare():
>
> /* Copy trampoline code in place */
> memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long),
> &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE);
>
> In particular, relocation for trampoline_32bit_src solved in the wrong
> way. Without -flto, we have rip-realtive address load:
>
> 982d30: 48 8d 35 09 cc ff ff lea -0x33f7(%rip),%rsi # 97f940 <trampoline_32bit_src>
>
> With -flto we have immediate load:
>
> 982cf0: 48 c7 c6 f0 f8 97 00 mov $0x97f8f0,%rsi
>
> It only would be okay if bootloader loads kernel at the address we compile
> it for. But it's not usually the case.
>
> As result we copy garbage into trampoline and crash when trying to execute
> it.
>
> I don't know how to solve it. As far as I know we don't support compiling
> kernel with LTO in mainline.
>
> Any suggestions?
LTO is broken. The boot code is compiled as position independent, so this
'optimization' is pure garbage.
Thanks,
tglx
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-03 14:21 ` Kirill A. Shutemov
2018-07-03 14:27 ` Thomas Gleixner
@ 2018-07-03 18:03 ` Andi Kleen
2018-07-03 20:26 ` Kirill A. Shutemov
2018-07-04 3:10 ` Benjamin Gilbert
2018-07-04 15:08 ` Kirill A. Shutemov
3 siblings, 1 reply; 39+ messages in thread
From: Andi Kleen @ 2018-07-03 18:03 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero
On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
> On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
> > On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
> > > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@redhat.com>:
> > > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
> > > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
> > > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
> > > >> QEMU/KVM. No messages are shown post-GRUB; the VM instantly reboots.
> > > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
> > > >> paging boot if kernel is above 4G") fixes it. I've attached our kernel
> > > >> config for reference, and am happy to test patches, provide sample QCOW
> > > >> images, etc.
> > > >
> > >
> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> > >
> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> > > too with the same symptoms
> >
> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
>
> -flto in LDFLAGS screws up this part of paging_prepare():
Where is that coming from? The LTO patches are not upstream.
And I don't see any LTO usage in the main line.
>
> /* Copy trampoline code in place */
> memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long),
> &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE);
> In particular, relocation for trampoline_32bit_src solved in the wrong
> way. Without -flto, we have rip-realtive address load:
>
> 982d30: 48 8d 35 09 cc ff ff lea -0x33f7(%rip),%rsi # 97f940 <trampoline_32bit_src>
>
> With -flto we have immediate load:
>
> 982cf0: 48 c7 c6 f0 f8 97 00 mov $0x97f8f0,%rsi
Strange.
Can you add some RELOC_HIDE()s and see if that helps?
> It only would be okay if bootloader loads kernel at the address we compile
> it for. But it's not usually the case.
>
> As result we copy garbage into trampoline and crash when trying to execute
> it.
>
> I don't know how to solve it. As far as I know we don't support compiling
> kernel with LTO in mainline.
Right.
-Andi
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-03 18:03 ` Andi Kleen
@ 2018-07-03 20:26 ` Kirill A. Shutemov
2018-07-03 21:00 ` Andi Kleen
0 siblings, 1 reply; 39+ messages in thread
From: Kirill A. Shutemov @ 2018-07-03 20:26 UTC (permalink / raw)
To: Andi Kleen
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero
On Tue, Jul 03, 2018 at 11:03:07AM -0700, Andi Kleen wrote:
> On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
> > On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
> > > On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
> > > > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@redhat.com>:
> > > > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
> > > > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
> > > > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
> > > > >> QEMU/KVM. No messages are shown post-GRUB; the VM instantly reboots.
> > > > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
> > > > >> paging boot if kernel is above 4G") fixes it. I've attached our kernel
> > > > >> config for reference, and am happy to test patches, provide sample QCOW
> > > > >> images, etc.
> > > > >
> > > >
> > > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> > > >
> > > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> > > > too with the same symptoms
> > >
> > > I tracked it down to -flto in LDFLAGS. I'll look more into this.
> >
> > -flto in LDFLAGS screws up this part of paging_prepare():
>
> Where is that coming from? The LTO patches are not upstream.
>
> And I don't see any LTO usage in the main line.
Apparently some distros try to hack it around:
https://bugzilla.kernel.org/show_bug.cgi?id=200385
I'm amazed that it kinda worked for them.
> > /* Copy trampoline code in place */
> > memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long),
> > &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE);
>
>
> > In particular, relocation for trampoline_32bit_src solved in the wrong
> > way. Without -flto, we have rip-realtive address load:
> >
> > 982d30: 48 8d 35 09 cc ff ff lea -0x33f7(%rip),%rsi # 97f940 <trampoline_32bit_src>
> >
> > With -flto we have immediate load:
> >
> > 982cf0: 48 c7 c6 f0 f8 97 00 mov $0x97f8f0,%rsi
>
> Strange.
>
> Can you add some RELOC_HIDE()s and see if that helps?
Nope. No difference in generated code.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-03 20:26 ` Kirill A. Shutemov
@ 2018-07-03 21:00 ` Andi Kleen
0 siblings, 0 replies; 39+ messages in thread
From: Andi Kleen @ 2018-07-03 21:00 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero
On Tue, Jul 03, 2018 at 11:26:09PM +0300, Kirill A. Shutemov wrote:
> On Tue, Jul 03, 2018 at 11:03:07AM -0700, Andi Kleen wrote:
> > On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
> > > On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
> > > > On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
> > > > > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@redhat.com>:
> > > > > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
> > > > > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
> > > > > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
> > > > > >> QEMU/KVM. No messages are shown post-GRUB; the VM instantly reboots.
> > > > > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
> > > > > >> paging boot if kernel is above 4G") fixes it. I've attached our kernel
> > > > > >> config for reference, and am happy to test patches, provide sample QCOW
> > > > > >> images, etc.
> > > > > >
> > > > >
> > > > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> > > > >
> > > > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> > > > > too with the same symptoms
> > > >
> > > > I tracked it down to -flto in LDFLAGS. I'll look more into this.
> > >
> > > -flto in LDFLAGS screws up this part of paging_prepare():
> >
> > Where is that coming from? The LTO patches are not upstream.
> >
> > And I don't see any LTO usage in the main line.
>
> Apparently some distros try to hack it around:
>
> https://bugzilla.kernel.org/show_bug.cgi?id=200385
>
> I'm amazed that it kinda worked for them.
I think it only works on older gccs that don't default to
thin LTO, but always generate a fallback non LTO object.
The kernel directly uses ld in the link step (without my patches), so LTO
shouldn't be able to ever generate code.
The early boot code may be an exception of this, and it's likely
the only code that actually uses LTO in such a set up.
The -fPIC is actually scarier than the -flto. The generated code
must create quite a mess and I'm not sure why you even would want that
because the kernel can be relocatable without it.
BTW I hope to eventually resend the full LTO patches.
They seem to get more and more users recently, mainly for smaller
code size.
>
>
> > > /* Copy trampoline code in place */
> > > memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long),
> > > &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE);
> >
> >
> > > In particular, relocation for trampoline_32bit_src solved in the wrong
> > > way. Without -flto, we have rip-realtive address load:
> > >
> > > 982d30: 48 8d 35 09 cc ff ff lea -0x33f7(%rip),%rsi # 97f940 <trampoline_32bit_src>
> > >
> > > With -flto we have immediate load:
> > >
> > > 982cf0: 48 c7 c6 f0 f8 97 00 mov $0x97f8f0,%rsi
> >
> > Strange.
> >
> > Can you add some RELOC_HIDE()s and see if that helps?
>
> Nope. No difference in generated code.
Ok will need to put together some self contained test case for the compiler people.
I'll try to take a look.
-Andi
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-03 14:21 ` Kirill A. Shutemov
2018-07-03 14:27 ` Thomas Gleixner
2018-07-03 18:03 ` Andi Kleen
@ 2018-07-04 3:10 ` Benjamin Gilbert
2018-07-04 13:21 ` Kirill A. Shutemov
2018-07-04 15:08 ` Kirill A. Shutemov
3 siblings, 1 reply; 39+ messages in thread
From: Benjamin Gilbert @ 2018-07-04 3:10 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, linux-x86_64, LKML, Kirill A. Shutemov, Ingo Molnar,
Thomas Gleixner, H. Peter Anvin, X86 ML, bero, Andi Kleen
On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
> I don't know how to solve it. As far as I know we don't support compiling
> kernel with LTO in mainline.
>
> Any suggestions?
>
> Benjamin, do you change LDFLAGS or CFLAGS when compiling the kernel?
We're using the standard build flags as far as I can tell. In particular,
we don't enable LTO, and I've verified that -flto isn't in the build logs.
Here's a sample image:
https://users.developer.core-os.net/bgilbert/4.17/vmlinuz-4.17.3-coreos
https://users.developer.core-os.net/bgilbert/4.17/vmlinux-4.17.3-coreos
https://users.developer.core-os.net/bgilbert/4.17/System.map
--Benjamin Gilbert
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-04 3:10 ` Benjamin Gilbert
@ 2018-07-04 13:21 ` Kirill A. Shutemov
0 siblings, 0 replies; 39+ messages in thread
From: Kirill A. Shutemov @ 2018-07-04 13:21 UTC (permalink / raw)
To: Benjamin Gilbert
Cc: Gabriel C, linux-x86_64, LKML, Kirill A. Shutemov, Ingo Molnar,
Thomas Gleixner, H. Peter Anvin, X86 ML, bero, Andi Kleen
On Tue, Jul 03, 2018 at 11:10:27PM -0400, Benjamin Gilbert wrote:
> On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
> > I don't know how to solve it. As far as I know we don't support compiling
> > kernel with LTO in mainline.
> >
> > Any suggestions?
> >
> > Benjamin, do you change LDFLAGS or CFLAGS when compiling the kernel?
>
> We're using the standard build flags as far as I can tell. In particular,
> we don't enable LTO, and I've verified that -flto isn't in the build logs.
>
> Here's a sample image:
>
> https://users.developer.core-os.net/bgilbert/4.17/vmlinuz-4.17.3-coreos
> https://users.developer.core-os.net/bgilbert/4.17/vmlinux-4.17.3-coreos
> https://users.developer.core-os.net/bgilbert/4.17/System.map
It's basically the same issue. We have immidiate load instead of
RIP-relative address load.
You can make the vmlinuz bootable with this binary patch:
echo -en "\x8d\x05\xa9\xa9\xff\xff" | dd of=vmlinuz-4.17.3-coreos seek=$((0x005d1fc1)) bs=1 conv=notrunc
Now we need to find out how linker gets it wrong.
Please, *after* complete build of the kernel with your toolchain do this:
touch arch/x86/boot/compressed/pgtable_64.c
make V=1
And share your build log.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-03 14:21 ` Kirill A. Shutemov
` (2 preceding siblings ...)
2018-07-04 3:10 ` Benjamin Gilbert
@ 2018-07-04 15:08 ` Kirill A. Shutemov
2018-07-04 20:42 ` Benjamin Gilbert
2018-07-06 6:37 ` Masahiro Yamada
3 siblings, 2 replies; 39+ messages in thread
From: Kirill A. Shutemov @ 2018-07-04 15:08 UTC (permalink / raw)
To: Gabriel C
Cc: Benjamin Gilbert, linux-x86_64, LKML, Kirill A. Shutemov,
Ingo Molnar, Thomas Gleixner, H. Peter Anvin, X86 ML, bero,
Andi Kleen, Masahiro Yamada, Michal Marek
On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
> On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
> > On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
> > > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@redhat.com>:
> > > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
> > > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
> > > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
> > > >> QEMU/KVM. No messages are shown post-GRUB; the VM instantly reboots.
> > > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
> > > >> paging boot if kernel is above 4G") fixes it. I've attached our kernel
> > > >> config for reference, and am happy to test patches, provide sample QCOW
> > > >> images, etc.
> > > >
> > >
> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> > >
> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> > > too with the same symptoms
> >
> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
>
> -flto in LDFLAGS screws up this part of paging_prepare():
+Masahiro, Michal.
I've got it wrong. *Any* LDFLAGS option passed to make this way:
make LDFLAGS="..."
would cause a issue. Even empty.
It overrides all assignments to the variable in the makefile.
As result the image is built without -pie and linker doesn't generate
position independed code.
Looks like the patch below helps, but my make-fu is poor.
I don't see many override directives in kernel makefiles.
It makes me think that there's a better way to fix this.
Hm?
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index fa42f895fdde..4f24baa8cdeb 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -42,16 +42,16 @@ KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__
GCOV_PROFILE := n
UBSAN_SANITIZE :=n
-LDFLAGS := -m elf_$(UTS_MACHINE)
+override LDFLAGS := -m elf_$(UTS_MACHINE)
# Compressed kernel should be built as PIE since it may be loaded at any
# address by the bootloader.
ifeq ($(CONFIG_X86_32),y)
-LDFLAGS += $(call ld-option, -pie) $(call ld-option, --no-dynamic-linker)
+override LDFLAGS += $(call ld-option, -pie) $(call ld-option, --no-dynamic-linker)
else
# To build 64-bit compressed kernel as PIE, we disable relocation
# overflow check to avoid relocation overflow error with a new linker
# command-line option, -z noreloc-overflow.
-LDFLAGS += $(shell $(LD) --help 2>&1 | grep -q "\-z noreloc-overflow" \
+override LDFLAGS += $(shell $(LD) --help 2>&1 | grep -q "\-z noreloc-overflow" \
&& echo "-z noreloc-overflow -pie --no-dynamic-linker")
endif
LDFLAGS_vmlinux := -T
--
Kirill A. Shutemov
^ permalink raw reply related [flat|nested] 39+ messages in thread* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-04 15:08 ` Kirill A. Shutemov
@ 2018-07-04 20:42 ` Benjamin Gilbert
2018-07-06 6:37 ` Masahiro Yamada
1 sibling, 0 replies; 39+ messages in thread
From: Benjamin Gilbert @ 2018-07-04 20:42 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, linux-x86_64, LKML, Kirill A. Shutemov, Ingo Molnar,
Thomas Gleixner, H. Peter Anvin, X86 ML, bero, Andi Kleen,
Masahiro Yamada, Michal Marek
On Wed, Jul 04, 2018 at 06:08:57PM +0300, Kirill A. Shutemov wrote:
> On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
> > On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
> > > On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
> > > > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@redhat.com>:
> > > > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
> > > > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
> > > > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
> > > > >> QEMU/KVM. No messages are shown post-GRUB; the VM instantly reboots.
> > > > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
> > > > >> paging boot if kernel is above 4G") fixes it. I've attached our kernel
> > > > >> config for reference, and am happy to test patches, provide sample QCOW
> > > > >> images, etc.
> > > > >
> > > >
> > > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> > > >
> > > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> > > > too with the same symptoms
> > >
> > > I tracked it down to -flto in LDFLAGS. I'll look more into this.
> >
> > -flto in LDFLAGS screws up this part of paging_prepare():
>
> I've got it wrong. *Any* LDFLAGS option passed to make this way:
>
> make LDFLAGS="..."
>
> would cause a issue. Even empty.
>
> It overrides all assignments to the variable in the makefile.
> As result the image is built without -pie and linker doesn't generate
> position independed code.
>
> Looks like the patch below helps, but my make-fu is poor.
Sure enough, we're passing LDFLAGS="" to make. Your patch fixes the boot
failure for me.
--Benjamin Gilbert
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-04 15:08 ` Kirill A. Shutemov
2018-07-04 20:42 ` Benjamin Gilbert
@ 2018-07-06 6:37 ` Masahiro Yamada
2018-07-06 10:41 ` Kirill A. Shutemov
1 sibling, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2018-07-06 6:37 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
Hi.
2018-07-05 0:08 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
> On Tue, Jul 03, 2018 at 05:21:50PM +0300, Kirill A. Shutemov wrote:
>> On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
>> > On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
>> > > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@redhat.com>:
>> > > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
>> > > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
>> > > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
>> > > >> QEMU/KVM. No messages are shown post-GRUB; the VM instantly reboots.
>> > > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
>> > > >> paging boot if kernel is above 4G") fixes it. I've attached our kernel
>> > > >> config for reference, and am happy to test patches, provide sample QCOW
>> > > >> images, etc.
>> > > >
>> > >
>> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
>> > >
>> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
>> > > too with the same symptoms
>> >
>> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
>>
>> -flto in LDFLAGS screws up this part of paging_prepare():
>
> +Masahiro, Michal.
>
> I've got it wrong. *Any* LDFLAGS option passed to make this way:
>
> make LDFLAGS="..."
>
> would cause a issue. Even empty.
>
> It overrides all assignments to the variable in the makefile.
> As result the image is built without -pie and linker doesn't generate
> position independed code.
>
> Looks like the patch below helps, but my make-fu is poor.
> I don't see many override directives in kernel makefiles.
> It makes me think that there's a better way to fix this.
>
> Hm?
LDFLAGS is for internal-use.
Please do not override it from the command line.
You want to pass your own linker flags
for building vmlinux and modules,
but do not want to pass them to
the decompressor (arch/x86/boot/compressed).
Correct?
Kbuild provides a way for users
to pass additional linker flags to modules.
(LDFLAGS_MODULE)
But, there is no way to do that for vmlinux.
It is easy to support it, though.
https://patchwork.kernel.org/patch/10510833/
If this is the one you want, I can merge this.
make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
will allow you to append linker flags.
> diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
> index fa42f895fdde..4f24baa8cdeb 100644
> --- a/arch/x86/boot/compressed/Makefile
> +++ b/arch/x86/boot/compressed/Makefile
> @@ -42,16 +42,16 @@ KBUILD_AFLAGS := $(KBUILD_CFLAGS) -D__ASSEMBLY__
> GCOV_PROFILE := n
> UBSAN_SANITIZE :=n
>
> -LDFLAGS := -m elf_$(UTS_MACHINE)
> +override LDFLAGS := -m elf_$(UTS_MACHINE)
> # Compressed kernel should be built as PIE since it may be loaded at any
> # address by the bootloader.
> ifeq ($(CONFIG_X86_32),y)
> -LDFLAGS += $(call ld-option, -pie) $(call ld-option, --no-dynamic-linker)
> +override LDFLAGS += $(call ld-option, -pie) $(call ld-option, --no-dynamic-linker)
> else
> # To build 64-bit compressed kernel as PIE, we disable relocation
> # overflow check to avoid relocation overflow error with a new linker
> # command-line option, -z noreloc-overflow.
> -LDFLAGS += $(shell $(LD) --help 2>&1 | grep -q "\-z noreloc-overflow" \
> +override LDFLAGS += $(shell $(LD) --help 2>&1 | grep -q "\-z noreloc-overflow" \
> && echo "-z noreloc-overflow -pie --no-dynamic-linker")
> endif
> LDFLAGS_vmlinux := -T
> --
> Kirill A. Shutemov
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 6:37 ` Masahiro Yamada
@ 2018-07-06 10:41 ` Kirill A. Shutemov
2018-07-06 14:13 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: Kirill A. Shutemov @ 2018-07-06 10:41 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
On Fri, Jul 06, 2018 at 03:37:58PM +0900, Masahiro Yamada wrote:
> >> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> >> > >
> >> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> >> > > too with the same symptoms
> >> >
> >> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
> >>
> >> -flto in LDFLAGS screws up this part of paging_prepare():
> >
> > +Masahiro, Michal.
> >
> > I've got it wrong. *Any* LDFLAGS option passed to make this way:
> >
> > make LDFLAGS="..."
> >
> > would cause a issue. Even empty.
> >
> > It overrides all assignments to the variable in the makefile.
> > As result the image is built without -pie and linker doesn't generate
> > position independed code.
> >
> > Looks like the patch below helps, but my make-fu is poor.
> > I don't see many override directives in kernel makefiles.
> > It makes me think that there's a better way to fix this.
> >
> > Hm?
>
>
> LDFLAGS is for internal-use.
> Please do not override it from the command line.
Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
other critical internal-use-only variables?
This breakage was rather hard to debug. We need to have some kind of
fail-safe for the future.
> You want to pass your own linker flags
> for building vmlinux and modules,
> but do not want to pass them to
> the decompressor (arch/x86/boot/compressed).
>
> Correct?
I personally don't think that changing compiler/linker options for kernel
build is good idea in general.
> Kbuild provides a way for users
> to pass additional linker flags to modules.
> (LDFLAGS_MODULE)
>
>
> But, there is no way to do that for vmlinux.
>
> It is easy to support it, though.
>
> https://patchwork.kernel.org/patch/10510833/
>
> If this is the one you want, I can merge this.
>
>
> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
> will allow you to append linker flags.
Okay. It makes me wounder if we should taint kernel in such cases?
Custom compiler/linker flags are risky and can lead to weird bugs.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 10:41 ` Kirill A. Shutemov
@ 2018-07-06 14:13 ` Masahiro Yamada
2018-07-06 14:39 ` Gabriel C
2018-07-06 16:29 ` Kirill A. Shutemov
0 siblings, 2 replies; 39+ messages in thread
From: Masahiro Yamada @ 2018-07-06 14:13 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
Hi.
2018-07-06 19:41 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
> On Fri, Jul 06, 2018 at 03:37:58PM +0900, Masahiro Yamada wrote:
>> >> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
>> >> > >
>> >> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
>> >> > > too with the same symptoms
>> >> >
>> >> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
>> >>
>> >> -flto in LDFLAGS screws up this part of paging_prepare():
>> >
>> > +Masahiro, Michal.
>> >
>> > I've got it wrong. *Any* LDFLAGS option passed to make this way:
>> >
>> > make LDFLAGS="..."
>> >
>> > would cause a issue. Even empty.
>> >
>> > It overrides all assignments to the variable in the makefile.
>> > As result the image is built without -pie and linker doesn't generate
>> > position independed code.
>> >
>> > Looks like the patch below helps, but my make-fu is poor.
>> > I don't see many override directives in kernel makefiles.
>> > It makes me think that there's a better way to fix this.
>> >
>> > Hm?
>>
>>
>> LDFLAGS is for internal-use.
>> Please do not override it from the command line.
>
> Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
> other critical internal-use-only variables?
Yes, Make can check where variables came from.
> This breakage was rather hard to debug. We need to have some kind of
> fail-safe for the future.
>
>> You want to pass your own linker flags
>> for building vmlinux and modules,
>> but do not want to pass them to
>> the decompressor (arch/x86/boot/compressed).
>>
>> Correct?
>
> I personally don't think that changing compiler/linker options for kernel
> build is good idea in general.
>
>> Kbuild provides a way for users
>> to pass additional linker flags to modules.
>> (LDFLAGS_MODULE)
>>
>>
>> But, there is no way to do that for vmlinux.
>>
>> It is easy to support it, though.
>>
>> https://patchwork.kernel.org/patch/10510833/
>>
>> If this is the one you want, I can merge this.
>>
>>
>> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
>> will allow you to append linker flags.
>
> Okay. It makes me wounder if we should taint kernel in such cases?
> Custom compiler/linker flags are risky and can lead to weird bugs.
OK.
So, what problem are we discussing?
> I've got it wrong. *Any* LDFLAGS option passed to make this way:
>
> make LDFLAGS="..."
In your previous mail, I thought you were asking me how to pass
custom linker flags.
If not, we do not need to think about that case.
Just say "Do not do that".
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 14:13 ` Masahiro Yamada
@ 2018-07-06 14:39 ` Gabriel C
2018-07-06 16:33 ` Kirill A. Shutemov
2018-07-07 0:55 ` Masahiro Yamada
2018-07-06 16:29 ` Kirill A. Shutemov
1 sibling, 2 replies; 39+ messages in thread
From: Gabriel C @ 2018-07-06 14:39 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Kirill A. Shutemov, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
2018-07-06 16:13 GMT+02:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
> Hi.
>
> 2018-07-06 19:41 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
>> On Fri, Jul 06, 2018 at 03:37:58PM +0900, Masahiro Yamada wrote:
>>> >> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
>>> >> > >
>>> >> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
>>> >> > > too with the same symptoms
>>> >> >
>>> >> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
>>> >>
>>> >> -flto in LDFLAGS screws up this part of paging_prepare():
>>> >
>>> > +Masahiro, Michal.
>>> >
>>> > I've got it wrong. *Any* LDFLAGS option passed to make this way:
>>> >
>>> > make LDFLAGS="..."
>>> >
>>> > would cause a issue. Even empty.
>>> >
>>> > It overrides all assignments to the variable in the makefile.
>>> > As result the image is built without -pie and linker doesn't generate
>>> > position independed code.
>>> >
>>> > Looks like the patch below helps, but my make-fu is poor.
>>> > I don't see many override directives in kernel makefiles.
>>> > It makes me think that there's a better way to fix this.
>>> >
>>> > Hm?
>>>
>>>
>>> LDFLAGS is for internal-use.
>>> Please do not override it from the command line.
>>
>> Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
>> other critical internal-use-only variables?
>
> Yes, Make can check where variables came from.
>
>
>> This breakage was rather hard to debug. We need to have some kind of
>> fail-safe for the future.
>>
>>> You want to pass your own linker flags
>>> for building vmlinux and modules,
>>> but do not want to pass them to
>>> the decompressor (arch/x86/boot/compressed).
>>>
>>> Correct?
>>
>> I personally don't think that changing compiler/linker options for kernel
>> build is good idea in general.
>>
>>> Kbuild provides a way for users
>>> to pass additional linker flags to modules.
>>> (LDFLAGS_MODULE)
>>>
>>>
>>> But, there is no way to do that for vmlinux.
>>>
>>> It is easy to support it, though.
>>>
>>> https://patchwork.kernel.org/patch/10510833/
>>>
>>> If this is the one you want, I can merge this.
>>>
>>>
>>> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
>>> will allow you to append linker flags.
>>
>> Okay. It makes me wounder if we should taint kernel in such cases?
>> Custom compiler/linker flags are risky and can lead to weird bugs.
>
> OK.
> So, what problem are we discussing?
>
>
>> I've got it wrong. *Any* LDFLAGS option passed to make this way:
>>
>> make LDFLAGS="..."
>
> In your previous mail, I thought you were asking me how to pass
> custom linker flags.
>
> If not, we do not need to think about that case.
> Just say "Do not do that".
I am sorry but I have a hard time to get your logic here.
You are saying : the *env* variable LDFLAGS as well passing
LDFLAGS to make , which your build allows should not be use
because is for 'internal usage' .. ?
Well that logic you have here is wrong and wrong for any project
not just for the kernel,
If you know 'parts' need have particular flags then 'you' have to
ensure nothing
overrides these or nothing at all can chage these.
So swap your logic and apped LDFLAGS to your private
'call_it_whatever_you_wish_KERNEL_NEED_BE_THERE_ANY_KIND_FLAGS'
and don't allow these to be changed at all , when you
*know* they have be there.
Teling users to not use LD/C/CXX flags is not really going to work right ?
BR
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 14:39 ` Gabriel C
@ 2018-07-06 16:33 ` Kirill A. Shutemov
2018-07-06 17:31 ` Gabriel C
2018-07-07 0:55 ` Masahiro Yamada
1 sibling, 1 reply; 39+ messages in thread
From: Kirill A. Shutemov @ 2018-07-06 16:33 UTC (permalink / raw)
To: Gabriel C
Cc: Masahiro Yamada, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
On Fri, Jul 06, 2018 at 04:39:28PM +0200, Gabriel C wrote:
> > If not, we do not need to think about that case.
> > Just say "Do not do that".
>
> I am sorry but I have a hard time to get your logic here.
>
> You are saying : the *env* variable LDFLAGS as well passing
> LDFLAGS to make , which your build allows should not be use
> because is for 'internal usage' .. ?
Environment variables do not override make variables. Only passing varible
assignment as make argument will do this.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 16:33 ` Kirill A. Shutemov
@ 2018-07-06 17:31 ` Gabriel C
0 siblings, 0 replies; 39+ messages in thread
From: Gabriel C @ 2018-07-06 17:31 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Masahiro Yamada, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
2018-07-06 18:33 GMT+02:00 Kirill A. Shutemov <kirill@shutemov.name>:
> On Fri, Jul 06, 2018 at 04:39:28PM +0200, Gabriel C wrote:
>> > If not, we do not need to think about that case.
>> > Just say "Do not do that".
>>
>> I am sorry but I have a hard time to get your logic here.
>>
>> You are saying : the *env* variable LDFLAGS as well passing
>> LDFLAGS to make , which your build allows should not be use
>> because is for 'internal usage' .. ?
>
> Environment variables do not override make variables. Only passing varible
> assignment as make argument will do this.
>
Still .. When the build system allows to do 'make FOO=bar' and you know
when using that things will break , the build system should be fixed.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 14:39 ` Gabriel C
2018-07-06 16:33 ` Kirill A. Shutemov
@ 2018-07-07 0:55 ` Masahiro Yamada
1 sibling, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2018-07-07 0:55 UTC (permalink / raw)
To: Gabriel C
Cc: Kirill A. Shutemov, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
2018-07-06 23:39 GMT+09:00 Gabriel C <nix.or.die@gmail.com>:
> 2018-07-06 16:13 GMT+02:00 Masahiro Yamada <yamada.masahiro@socionext.com>:
>> Hi.
>>
>> 2018-07-06 19:41 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
>>> On Fri, Jul 06, 2018 at 03:37:58PM +0900, Masahiro Yamada wrote:
>>>> >> > > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
>>>> >> > >
>>>> >> > > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
>>>> >> > > too with the same symptoms
>>>> >> >
>>>> >> > I tracked it down to -flto in LDFLAGS. I'll look more into this.
>>>> >>
>>>> >> -flto in LDFLAGS screws up this part of paging_prepare():
>>>> >
>>>> > +Masahiro, Michal.
>>>> >
>>>> > I've got it wrong. *Any* LDFLAGS option passed to make this way:
>>>> >
>>>> > make LDFLAGS="..."
>>>> >
>>>> > would cause a issue. Even empty.
>>>> >
>>>> > It overrides all assignments to the variable in the makefile.
>>>> > As result the image is built without -pie and linker doesn't generate
>>>> > position independed code.
>>>> >
>>>> > Looks like the patch below helps, but my make-fu is poor.
>>>> > I don't see many override directives in kernel makefiles.
>>>> > It makes me think that there's a better way to fix this.
>>>> >
>>>> > Hm?
>>>>
>>>>
>>>> LDFLAGS is for internal-use.
>>>> Please do not override it from the command line.
>>>
>>> Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
>>> other critical internal-use-only variables?
>>
>> Yes, Make can check where variables came from.
>>
>>
>>> This breakage was rather hard to debug. We need to have some kind of
>>> fail-safe for the future.
>>>
>>>> You want to pass your own linker flags
>>>> for building vmlinux and modules,
>>>> but do not want to pass them to
>>>> the decompressor (arch/x86/boot/compressed).
>>>>
>>>> Correct?
>>>
>>> I personally don't think that changing compiler/linker options for kernel
>>> build is good idea in general.
>>>
>>>> Kbuild provides a way for users
>>>> to pass additional linker flags to modules.
>>>> (LDFLAGS_MODULE)
>>>>
>>>>
>>>> But, there is no way to do that for vmlinux.
>>>>
>>>> It is easy to support it, though.
>>>>
>>>> https://patchwork.kernel.org/patch/10510833/
>>>>
>>>> If this is the one you want, I can merge this.
>>>>
>>>>
>>>> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
>>>> will allow you to append linker flags.
>>>
>>> Okay. It makes me wounder if we should taint kernel in such cases?
>>> Custom compiler/linker flags are risky and can lead to weird bugs.
>>
>> OK.
>> So, what problem are we discussing?
>>
>>
>>> I've got it wrong. *Any* LDFLAGS option passed to make this way:
>>>
>>> make LDFLAGS="..."
>>
>> In your previous mail, I thought you were asking me how to pass
>> custom linker flags.
>>
>> If not, we do not need to think about that case.
>> Just say "Do not do that".
>
> I am sorry but I have a hard time to get your logic here.
>
> You are saying : the *env* variable LDFLAGS as well passing
> LDFLAGS to make , which your build allows should not be use
> because is for 'internal usage' .. ?
>
> Well that logic you have here is wrong and wrong for any project
> not just for the kernel,
Why 'my logic'?
LDFLAGS has been long used internally since the old days,
before I ever worked on the kernel.
I shared my knowledge about the kernel build system.
The current situation is not nice,
but why are you blaming me for the code I did not add ?
Note:
I have never said 'the *env* variable LDFLAGS'
> If you know 'parts' need have particular flags then 'you' have to
> ensure nothing
> overrides these or nothing at all can chage these.
>
> So swap your logic and apped LDFLAGS to your private
> 'call_it_whatever_you_wish_KERNEL_NEED_BE_THERE_ANY_KIND_FLAGS'
> and don't allow these to be changed at all , when you
> *know* they have be there.
>
>
> Teling users to not use LD/C/CXX flags is not really going to work right ?
>
>
> BR
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 14:13 ` Masahiro Yamada
2018-07-06 14:39 ` Gabriel C
@ 2018-07-06 16:29 ` Kirill A. Shutemov
2018-07-06 18:11 ` Andi Kleen
2018-07-07 1:21 ` Masahiro Yamada
1 sibling, 2 replies; 39+ messages in thread
From: Kirill A. Shutemov @ 2018-07-06 16:29 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
On Fri, Jul 06, 2018 at 11:13:02PM +0900, Masahiro Yamada wrote:
> >> LDFLAGS is for internal-use.
> >> Please do not override it from the command line.
> >
> > Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
> > other critical internal-use-only variables?
>
> Yes, Make can check where variables came from.
I think we should do this.
> >> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
> >> will allow you to append linker flags.
> >
> > Okay. It makes me wounder if we should taint kernel in such cases?
> > Custom compiler/linker flags are risky and can lead to weird bugs.
>
> OK.
> So, what problem are we discussing?
Users set custom LDFLAGS/CFLAGS and break kernel. Then report bug that
hard to debug. See
https://bugzilla.kernel.org/show_bug.cgi?id=200385
and start of this thread:
https://lore.kernel.org/lkml/20180701213243.GA20180@trogon.sfo.coreos.systems/
It took me a while to track down the issue. I blamed linker for a while.
> > I've got it wrong. *Any* LDFLAGS option passed to make this way:
> >
> > make LDFLAGS="..."
>
> In your previous mail, I thought you were asking me how to pass
> custom linker flags.
>
> If not, we do not need to think about that case.
> Just say "Do not do that".
At least we need to make user aware about risk of setting custom flags.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 16:29 ` Kirill A. Shutemov
@ 2018-07-06 18:11 ` Andi Kleen
2018-07-06 19:34 ` Benjamin Gilbert
2018-07-07 1:21 ` Masahiro Yamada
1 sibling, 1 reply; 39+ messages in thread
From: Andi Kleen @ 2018-07-06 18:11 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Masahiro Yamada, Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Michal Marek
> At least we need to make user aware about risk of setting custom flags.
There are valid use cases to override the flags. I use it sometimes too,
and know some other people do to.
But you need to know what you're doing.
Perhaps a warning during build would be reasonable. So if you ask
for a build log you would see it.
-Andi
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 18:11 ` Andi Kleen
@ 2018-07-06 19:34 ` Benjamin Gilbert
0 siblings, 0 replies; 39+ messages in thread
From: Benjamin Gilbert @ 2018-07-06 19:34 UTC (permalink / raw)
To: Andi Kleen
Cc: Kirill A. Shutemov, Masahiro Yamada, Gabriel C, linux-x86_64,
LKML, Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner,
H. Peter Anvin, X86 ML, bero, Michal Marek
On Fri, Jul 06, 2018 at 11:11:10AM -0700, Andi Kleen wrote:
> There are valid use cases to override the flags. I use it sometimes too,
> and know some other people do to.
>
> But you need to know what you're doing.
>
> Perhaps a warning during build would be reasonable. So if you ask
> for a build log you would see it.
In our case, the package is presumably passing LDFLAGS="" to override the
LDFLAGS environment variable already set by the packaging system. This has
worked for years without a problem.
--Benjamin Gilbert
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-06 16:29 ` Kirill A. Shutemov
2018-07-06 18:11 ` Andi Kleen
@ 2018-07-07 1:21 ` Masahiro Yamada
2018-07-09 10:10 ` Kirill A. Shutemov
1 sibling, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2018-07-07 1:21 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
2018-07-07 1:29 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
> On Fri, Jul 06, 2018 at 11:13:02PM +0900, Masahiro Yamada wrote:
>> >> LDFLAGS is for internal-use.
>> >> Please do not override it from the command line.
>> >
>> > Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
>> > other critical internal-use-only variables?
>>
>> Yes, Make can check where variables came from.
>
> I think we should do this.
>
>> >> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
>> >> will allow you to append linker flags.
>> >
>> > Okay. It makes me wounder if we should taint kernel in such cases?
>> > Custom compiler/linker flags are risky and can lead to weird bugs.
>>
>> OK.
>> So, what problem are we discussing?
>
> Users set custom LDFLAGS/CFLAGS and break kernel. Then report bug that
> hard to debug. See
>
> https://bugzilla.kernel.org/show_bug.cgi?id=200385
CFLAGS is only used under tools/.
Passing CFLAGS is probably no effect to the kernel.
In Linux makefiles,
KBUILD_ prefixed variables are used internally.
KBUILD_CFLAGS, KBUILD_CPPFLAGS, KBUILD_AFLAGS, etc.
LDFLAGS is an exception. I do not know why.
Renaming LDFLAGS to KBUILD_LDFLAGS
will make the code consistent.
At least, it will avoid overriding flags by accident.
Of course, users still can change KBUILD_LDFLAGS
if they really want.
The build system could add belt and braces checks for that,
but it is arguable since
there are lots of lots of internal variables.
> and start of this thread:
>
> https://lore.kernel.org/lkml/20180701213243.GA20180@trogon.sfo.coreos.systems/
>
> It took me a while to track down the issue. I blamed linker for a while.
>
>> > I've got it wrong. *Any* LDFLAGS option passed to make this way:
>> >
>> > make LDFLAGS="..."
>>
>> In your previous mail, I thought you were asking me how to pass
>> custom linker flags.
>>
>> If not, we do not need to think about that case.
>> Just say "Do not do that".
>
> At least we need to make user aware about risk of setting custom flags.
>
> --
> Kirill A. Shutemov
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-07 1:21 ` Masahiro Yamada
@ 2018-07-09 10:10 ` Kirill A. Shutemov
2018-07-09 10:37 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: Kirill A. Shutemov @ 2018-07-09 10:10 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
On Sat, Jul 07, 2018 at 10:21:47AM +0900, Masahiro Yamada wrote:
> 2018-07-07 1:29 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
> > On Fri, Jul 06, 2018 at 11:13:02PM +0900, Masahiro Yamada wrote:
> >> >> LDFLAGS is for internal-use.
> >> >> Please do not override it from the command line.
> >> >
> >> > Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
> >> > other critical internal-use-only variables?
> >>
> >> Yes, Make can check where variables came from.
> >
> > I think we should do this.
> >
> >> >> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
> >> >> will allow you to append linker flags.
> >> >
> >> > Okay. It makes me wounder if we should taint kernel in such cases?
> >> > Custom compiler/linker flags are risky and can lead to weird bugs.
> >>
> >> OK.
> >> So, what problem are we discussing?
> >
> > Users set custom LDFLAGS/CFLAGS and break kernel. Then report bug that
> > hard to debug. See
> >
> > https://bugzilla.kernel.org/show_bug.cgi?id=200385
>
>
> CFLAGS is only used under tools/.
> Passing CFLAGS is probably no effect to the kernel.
>
> In Linux makefiles,
> KBUILD_ prefixed variables are used internally.
>
> KBUILD_CFLAGS, KBUILD_CPPFLAGS, KBUILD_AFLAGS, etc.
>
>
> LDFLAGS is an exception. I do not know why.
> Renaming LDFLAGS to KBUILD_LDFLAGS
> will make the code consistent.
>
> At least, it will avoid overriding flags by accident.
>
> Of course, users still can change KBUILD_LDFLAGS
> if they really want.
>
> The build system could add belt and braces checks for that,
> but it is arguable since
> there are lots of lots of internal variables.
I think renaming LDFLAGS to KBUILD_LDFLAGS is good idea.
Would you prepare patch?
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"
2018-07-09 10:10 ` Kirill A. Shutemov
@ 2018-07-09 10:37 ` Masahiro Yamada
0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2018-07-09 10:37 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Gabriel C, Benjamin Gilbert, linux-x86_64, LKML,
Kirill A. Shutemov, Ingo Molnar, Thomas Gleixner, H. Peter Anvin,
X86 ML, bero, Andi Kleen, Michal Marek
2018-07-09 19:10 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
> On Sat, Jul 07, 2018 at 10:21:47AM +0900, Masahiro Yamada wrote:
>> 2018-07-07 1:29 GMT+09:00 Kirill A. Shutemov <kirill@shutemov.name>:
>> > On Fri, Jul 06, 2018 at 11:13:02PM +0900, Masahiro Yamada wrote:
>> >> >> LDFLAGS is for internal-use.
>> >> >> Please do not override it from the command line.
>> >> >
>> >> > Can we generate a build error if a user try to override LDFLAGS, CFLAGS or
>> >> > other critical internal-use-only variables?
>> >>
>> >> Yes, Make can check where variables came from.
>> >
>> > I think we should do this.
>> >
>> >> >> make LDFLAGS_KERNEL=... LDFLAGS_MODULE=...
>> >> >> will allow you to append linker flags.
>> >> >
>> >> > Okay. It makes me wounder if we should taint kernel in such cases?
>> >> > Custom compiler/linker flags are risky and can lead to weird bugs.
>> >>
>> >> OK.
>> >> So, what problem are we discussing?
>> >
>> > Users set custom LDFLAGS/CFLAGS and break kernel. Then report bug that
>> > hard to debug. See
>> >
>> > https://bugzilla.kernel.org/show_bug.cgi?id=200385
>>
>>
>> CFLAGS is only used under tools/.
>> Passing CFLAGS is probably no effect to the kernel.
>>
>> In Linux makefiles,
>> KBUILD_ prefixed variables are used internally.
>>
>> KBUILD_CFLAGS, KBUILD_CPPFLAGS, KBUILD_AFLAGS, etc.
>>
>>
>> LDFLAGS is an exception. I do not know why.
>> Renaming LDFLAGS to KBUILD_LDFLAGS
>> will make the code consistent.
>>
>> At least, it will avoid overriding flags by accident.
>>
>> Of course, users still can change KBUILD_LDFLAGS
>> if they really want.
>>
>> The build system could add belt and braces checks for that,
>> but it is arguable since
>> there are lots of lots of internal variables.
>
> I think renaming LDFLAGS to KBUILD_LDFLAGS is good idea.
> Would you prepare patch?
Yes, targeting for 4.19-rc1.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread