Re: [PATCH] x86/boot: allow a relocatable kernel to be linked with lld

From: Arvind Sankar <nivedita@alum.mit.edu>
To: Fangrui Song <maskray@google.com>
Cc: Dmitry Golovin <dima@golovin.in>, Borislav Petkov <bp@alien8.de>,
	"clang-built-linux@googlegroups.com" 
	<clang-built-linux@googlegroups.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, "x86@kernel.org" <x86@kernel.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Nick Desaulniers <ndesaulniers@google.com>,
	Ard Biesheuvel <ardb@kernel.org>,
	Masahiro Yamada <masahiroy@kernel.org>,
	Daniel Kiper <daniel.kiper@oracle.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] x86/boot: allow a relocatable kernel to be linked with lld
Date: Sun, 17 May 2020 16:25:27 -0400	[thread overview]
Message-ID: <20200517202527.GA2563549@rani.riverdale.lan> (raw)
In-Reply-To: <20200517194429.scwhfr4l4bv4h3ux@google.com>

On Sun, May 17, 2020 at 12:44:29PM -0700, Fangrui Song wrote:
> On 2020-05-16, Dmitry Golovin wrote:
> >15.05.2020, 21:50, "Borislav Petkov" <bp@alien8.de>:
> >>
> >> I need more info here about which segment is read-only?
> >>
> >> Is this something LLD does by default or what's happening?
> >>
> >
> >Probably should have quoted the original error message:
> >
> >    ld.lld: error: can't create dynamic relocation R_386_32 against
> >    symbol: _bss in readonly segment; recompile object files with -fPIC
> >    or pass '-Wl,-z,notext' to allow text relocations in the output
> 
> Do we know where do these R_386_32 come from?
> 
> When linking in -shared mode, the linker assumes the image is a shared
> object and has undetermined image base at runtime. An absolute
> relocation needs a text relocation (a relocation against a readonly
> segment).
> 
> When neither -z notext nor -z text is specified, GNU ld is in an
> indefinite state where it will enable text relocations (DT_TEXTREL
> DF_TEXTREL) on demand. It is not considered a good practice for
> userspace applications to do this.
> 
> Of course the kernel is different....... I know little about the kernel,
> but if there is a way to make the sections containing R_386_32
> relocations writable (SHF_WRITE), that will be a better solution to me.
> In LLD, -z notext is like making every section SHF_WRITE.

The assembly files head_32.S and head_64.S in arch/x86/boot/compressed
create bogus relocations in .head.text and .text.

This is the source of the common warning when using the bfd linker, for
eg on 64-bit:
  LD      arch/x86/boot/compressed/vmlinux
  ld: arch/x86/boot/compressed/head_64.o: warning: relocation in read-only section `.head.text'
  ld: warning: creating a DT_TEXTREL in object

These relocations are "bogus", i.e. they're unwanted and the kernel
won't actually boot if anything were to perform those relocations before
running it. They're also the cause of the 64-bit kernel requiring linker
support for the -z noreloc-overflow option to link it as PIE.

From arch/x86/boot/compressed/Makefile:

# To build 64-bit compressed kernel as PIE, we disable relocation
# overflow check to avoid relocation overflow error with a new linker
# command-line option, -z noreloc-overflow.
KBUILD_LDFLAGS += $(shell $(LD) --help 2>&1 | grep -q "\-z
noreloc-overflow" \
        && echo "-z noreloc-overflow -pie --no-dynamic-linker")

The relocations come from code like
	leal    gdt(%ebp), %eax
which should really be
	leal	(gdt-startup_32)(%ebp), %eax
to be technically correct.

I've played around with fixing the head code to avoid creating the
relocations in the first place, but never got around to submitting
anything: if there is interest in this, I can polish it up and send it
around.

Thanks.