[PATCH v2] x86: Avoid relocation information in final vmlinux

[PATCH v2] x86: Avoid relocation information in final vmlinux

[Petr Pavlu]

The Linux build process on x86 roughly consists of compiling all input
files, statically linking them into a vmlinux ELF file, and then taking
and turning this file into an actual bzImage bootable file.

vmlinux has in this process two main purposes:
1) It is an intermediate build target on the way to produce the final
   bootable image.
2) It is a file that is expected to be used by debuggers and standard
   ELF tooling to work with the built kernel.

For the second purpose, a vmlinux file is typically collected by various
package build recipes, such as distribution spec files, including the
kernel's own binrpm-pkg target.

When building a kernel supporting KASLR with CONFIG_X86_NEED_RELOCS,
vmlinux contains also relocation information produced by using the
--emit-relocs linker option. This is utilized by subsequent build steps
to create vmlinux.relocs and produce a relocatable image. However, the
information is not needed by debuggers and other standard ELF tooling.

The issue is then that the collected vmlinux file and hence distribution
packages end up unnecessarily large because of this extra data. The
following is a size comparison of vmlinux v6.0-rc5 with and without the
relocation information:
| Configuration      | With relocs | Stripped relocs |
| x86_64_defconfig   |       70 MB |           43 MB |
| +CONFIG_DEBUG_INFO |      818 MB |          367 MB |

The patch optimizes a resulting vmlinux by adding a postlink step that
splits the relocation information into vmlinux.relocs and then strips it
from the vmlinux binary.

Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
---

Notes:
    Changes since v1 [1]:
    - Fix the command to remove relocations to work with llvm-objcopy too.
    
    [1] https://lore.kernel.org/lkml/20220913132911.6850-1-petr.pavlu@suse.com/

 .gitignore                        |  1 +
 arch/x86/Makefile.postlink        | 41 +++++++++++++++++++++++++++++++
 arch/x86/boot/compressed/Makefile | 10 +++-----
 3 files changed, 46 insertions(+), 6 deletions(-)
 create mode 100644 arch/x86/Makefile.postlink

diff --git a/.gitignore b/.gitignore
index 265959544978..cd4ef88584ea 100644
--- a/.gitignore
+++ b/.gitignore
@@ -37,6 +37,7 @@
 *.o
 *.o.*
 *.patch
+*.relocs
 *.s
 *.so
 *.so.dbg
diff --git a/arch/x86/Makefile.postlink b/arch/x86/Makefile.postlink
new file mode 100644
index 000000000000..b38ffa4defb3
--- /dev/null
+++ b/arch/x86/Makefile.postlink
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+# ===========================================================================
+# Post-link x86 pass
+# ===========================================================================
+#
+# 1. Separate relocations from vmlinux into vmlinux.relocs.
+# 2. Strip relocations from vmlinux.
+
+PHONY := __archpost
+__archpost:
+
+-include include/config/auto.conf
+include scripts/Kbuild.include
+
+CMD_RELOCS = arch/x86/tools/relocs
+quiet_cmd_relocs = RELOCS  $@.relocs
+      cmd_relocs = $(CMD_RELOCS) $@ > $@.relocs;$(CMD_RELOCS) --abs-relocs $@
+
+quiet_cmd_strip_relocs = RSTRIP  $@
+      cmd_strip_relocs = $(OBJCOPY) --remove-section='.rel.*' --remove-section='.rel__*' --remove-section='.rela.*' --remove-section='.rela__*' $@
+
+# `@true` prevents complaint when there is nothing to be done
+
+vmlinux: FORCE
+	@true
+ifeq ($(CONFIG_X86_NEED_RELOCS),y)
+	$(call cmd,relocs)
+	$(call cmd,strip_relocs)
+endif
+
+%.ko: FORCE
+	@true
+
+clean:
+	@rm -f vmlinux.relocs
+
+PHONY += FORCE clean
+
+FORCE:
+
+.PHONY: $(PHONY)
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 35ce1a64068b..eba7709d75ae 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -120,14 +120,12 @@ $(obj)/vmlinux.bin: vmlinux FORCE
 
 targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relocs
 
-CMD_RELOCS = arch/x86/tools/relocs
-quiet_cmd_relocs = RELOCS  $@
-      cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $<
-$(obj)/vmlinux.relocs: vmlinux FORCE
-	$(call if_changed,relocs)
+# vmlinux.relocs is created by the vmlinux postlink step.
+vmlinux.relocs: vmlinux
+	@true
 
 vmlinux.bin.all-y := $(obj)/vmlinux.bin
-vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
+vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += vmlinux.relocs
 
 $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
 	$(call if_changed,gzip)
-- 
2.35.3

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Borislav Petkov]

On Tue, Sep 27, 2022 at 10:46:32AM +0200, Petr Pavlu wrote:
> When building a kernel supporting KASLR with CONFIG_X86_NEED_RELOCS,
> vmlinux contains also relocation information produced by using the
> --emit-relocs linker option. This is utilized by subsequent build steps
> to create vmlinux.relocs and produce a relocatable image. However, the
> information is not needed by debuggers and other standard ELF tooling.

Hm, my ld manpage says:

       -q
       --emit-relocs
           Leave relocation sections and contents in fully linked executables.  Post
	   									^^^^
           link analysis and optimization tools may need this information in order to
	   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

           perform correct modifications of executables.  This results in larger
           executables.

So what's up?

> The issue is then that the collected vmlinux file and hence distribution
> packages end up unnecessarily large because of this extra data. The
> following is a size comparison of vmlinux v6.0-rc5 with and without the
> relocation information:
> | Configuration      | With relocs | Stripped relocs |
> | x86_64_defconfig   |       70 MB |           43 MB |
> | +CONFIG_DEBUG_INFO |      818 MB |          367 MB |

Hmm, I see a different story with my tailored config here:

   text    data     bss     dec     hex filename
17131605        128673450       37339140        183144195       aea8f03 vmlinux.before
17132217        128677706       37363716        183173639       aeb0207 vmlinux.after

361M vmlinux.before
361M vmlinux.after

and

738K vmlinux.relocs

and before and after .configs simply have RANDOMIZE_BASE =n and =y,
respectively.

So how do you see such a big diff, even with defconfig?

> The patch optimizes a resulting vmlinux by adding a postlink step that

Avoid having "This/The patch" or "This commit" in the commit message. It is
tautologically useless.

Also, do

$ git grep 'This patch' Documentation/process

for more details.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]

On Wed, Nov 23, 2022 at 01:30:21PM +0100, Borislav Petkov wrote:
> > The patch optimizes a resulting vmlinux by adding a postlink step that
> 
> Avoid having "This/The patch" or "This commit" in the commit message. It is
> tautologically useless.
> 
> Also, do
> 
> $ git grep 'This patch' Documentation/process
> 
> for more details.

Why do you say "tautologically useless"? I think even
Documentation/process/submitting-patches.rst doesn't describe why.
Regardless, I'm almost always sticking to the rule.

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Borislav Petkov]

On Wed, Nov 23, 2022 at 08:31:47PM +0700, Bagas Sanjaya wrote:
> Why do you say "tautologically useless"?

Do you want a hint and think about it a bit or should I tell you
directly what this means?

> I think even Documentation/process/submitting-patches.rst doesn't
> describe why.

Execute the grep command above and read the text around the results.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Petr Pavlu]

On 11/23/22 13:30, Borislav Petkov wrote:
> On Tue, Sep 27, 2022 at 10:46:32AM +0200, Petr Pavlu wrote:
>> When building a kernel supporting KASLR with CONFIG_X86_NEED_RELOCS,
>> vmlinux contains also relocation information produced by using the
>> --emit-relocs linker option. This is utilized by subsequent build steps
>> to create vmlinux.relocs and produce a relocatable image. However, the
>> information is not needed by debuggers and other standard ELF tooling.
> 
> Hm, my ld manpage says:
> 
>        -q
>        --emit-relocs
>            Leave relocation sections and contents in fully linked executables.  Post
> 	   									^^^^
>            link analysis and optimization tools may need this information in order to
> 	   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
>            perform correct modifications of executables.  This results in larger
>            executables.
> 
> So what's up?

The only post-link analysis tool in this case should be arch/x86/tools/relocs.
It produces a vmlinux.relocs file which is appended to vmlinux.bin. This is
all internal to the Linux build. I'm not aware of any external tooling, such
as kernel debuggers, that would require this relocation information in
vmlinux.

>> The issue is then that the collected vmlinux file and hence distribution
>> packages end up unnecessarily large because of this extra data. The
>> following is a size comparison of vmlinux v6.0-rc5 with and without the
>> relocation information:
>> | Configuration      | With relocs | Stripped relocs |
>> | x86_64_defconfig   |       70 MB |           43 MB |
>> | +CONFIG_DEBUG_INFO |      818 MB |          367 MB |
> 
> Hmm, I see a different story with my tailored config here:
> 
>    text    data     bss     dec     hex filename
> 17131605        128673450       37339140        183144195       aea8f03 vmlinux.before
> 17132217        128677706       37363716        183173639       aeb0207 vmlinux.after
> 
> 361M vmlinux.before
> 361M vmlinux.after
> 
> and
> 
> 738K vmlinux.relocs
> 
> and before and after .configs simply have RANDOMIZE_BASE =n and =y,
> respectively.
> 
> So how do you see such a big diff, even with defconfig?

The size command used in your example includes only allocatable code, data and
bss sections. It does not show size of any relocation sections.

My measurement was simply with 'ls -lh'. One can compare output of 'readelf
--sections' from the before and after binaries to see what sections are
removed with the patch and what is their size.

The following sections are dropped for defconfig on v6.1-rc6:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 2] .rela.text        RELA             0000000000000000  02adf908
       00000000009457f8  0000000000000018   I      60     1     8
  [ 4] .rela.rodata      RELA             0000000000000000  03425100
       00000000001781d0  0000000000000018   I      60     3     8
  [ 6] .rela.pci_fixup   RELA             0000000000000000  0359d2d0
       0000000000005028  0000000000000018   I      60     5     8
  [ 8] .rela.tracedata   RELA             0000000000000000  035a22f8
       0000000000000120  0000000000000018   I      60     7     8
  [10] .rela__ksymtab    RELA             0000000000000000  035a2418
       000000000006fff0  0000000000000018   I      60     9     8
  [12] .rela__ksymt[...] RELA             0000000000000000  03612408
       00000000000609a8  0000000000000018   I      60    11     8
  [15] .rela__param      RELA             0000000000000000  03672db0
       0000000000009150  0000000000000018   I      60    14     8
  [17] .rela__modver     RELA             0000000000000000  0367bf00
       0000000000000540  0000000000000018   I      60    16     8
  [19] .rela__ex_table   RELA             0000000000000000  0367c440
       000000000000dec0  0000000000000018   I      60    18     8
  [22] .rela.data        RELA             0000000000000000  0368a300
       000000000011e5a8  0000000000000018   I      60    21     8
  [24] .rela__bug_table  RELA             0000000000000000  037a88a8
       0000000000073b30  0000000000000018   I      60    23     8
  [26] .rela.orc_un[...] RELA             0000000000000000  0381c3d8
       0000000000aea9e0  0000000000000018   I      60    25     8
  [31] .rela.data..[...] RELA             0000000000000000  04306db8
       0000000000000180  0000000000000018   I      60    30     8
  [33] .rela.init.text   RELA             0000000000000000  04306f38
       000000000009f8a0  0000000000000018   I      60    32     8
  [35] .rela.altins[...] RELA             0000000000000000  043a67d8
       0000000000009dc8  0000000000000018   I      60    34     8
  [37] .rela.init.data   RELA             0000000000000000  043b05a0
       000000000002ec50  0000000000000018   I      60    36     8
  [39] .rela.x86_cp[...] RELA             0000000000000000  043df1f0
       0000000000000078  0000000000000018   I      60    38     8
  [41] .rela.parain[...] RELA             0000000000000000  043df268
       00000000000015f0  0000000000000018   I      60    40     8
  [43] .rela.retpol[...] RELA             0000000000000000  043e0858
       000000000005c700  0000000000000018   I      60    42     8
  [45] .rela.return[...] RELA             0000000000000000  0443cf58
       0000000000170fe8  0000000000000018   I      60    44     8
  [47] .rela.altins[...] RELA             0000000000000000  045adf40
       000000000001b0f0  0000000000000018   I      60    46     8
  [49] .rela.altins[...] RELA             0000000000000000  045c9030
       0000000000004d28  0000000000000018   I      60    48     8
  [51] .rela.apicdrivers RELA             0000000000000000  045cdd58
       0000000000000030  0000000000000018   I      60    50     8
  [53] .rela.exit.text   RELA             0000000000000000  045cdd88
       0000000000006870  0000000000000018   I      60    52     8
  [55] .rela.smp_locks   RELA             0000000000000000  045d45f8
       00000000000420c0  0000000000000018   I      60    54     8

Total size of these sections is 27.2 MB.

>> The patch optimizes a resulting vmlinux by adding a postlink step that
> 
> Avoid having "This/The patch" or "This commit" in the commit message. It is
> tautologically useless.
> 
> Also, do
> 
> $ git grep 'This patch' Documentation/process
> 
> for more details.

Noted.

Thanks,
Petr

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Borislav Petkov]

On Wed, Nov 23, 2022 at 04:43:20PM +0100, Petr Pavlu wrote:
> The only post-link analysis tool in this case should be arch/x86/tools/relocs.
> It produces a vmlinux.relocs file which is appended to vmlinux.bin. This is
> all internal to the Linux build. I'm not aware of any external tooling, such
> as kernel debuggers, that would require this relocation information in
> vmlinux.

It would be good to know why --emit-relocs was added in the first place
- that might give us a hint. Lemme talk to Micha.

> The size command used in your example includes only allocatable code, data and
> bss sections. It does not show size of any relocation sections.

This:

361M vmlinux.before
361M vmlinux.after

is simple ls output.

Maybe I need something else enabled in my .config which would show this
significant difference *and* *explain* it.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[H. Peter Anvin]

On November 23, 2022 7:54:29 AM PST, Borislav Petkov <bp@alien8.de> wrote:
>On Wed, Nov 23, 2022 at 04:43:20PM +0100, Petr Pavlu wrote:
>> The only post-link analysis tool in this case should be arch/x86/tools/relocs.
>> It produces a vmlinux.relocs file which is appended to vmlinux.bin. This is
>> all internal to the Linux build. I'm not aware of any external tooling, such
>> as kernel debuggers, that would require this relocation information in
>> vmlinux.
>
>It would be good to know why --emit-relocs was added in the first place
>- that might give us a hint. Lemme talk to Micha.
>
>> The size command used in your example includes only allocatable code, data and
>> bss sections. It does not show size of any relocation sections.
>
>This:
>
>361M vmlinux.before
>361M vmlinux.after
>
>is simple ls output.
>
>Maybe I need something else enabled in my .config which would show this
>significant difference *and* *explain* it.
>
>Thx.
>

The real question is: why does anyone care about the size of the vmlinux file specifically?

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Petr Pavlu]

On 11/23/22 18:45, H. Peter Anvin wrote:
> On November 23, 2022 7:54:29 AM PST, Borislav Petkov <bp@alien8.de> wrote:
>> On Wed, Nov 23, 2022 at 04:43:20PM +0100, Petr Pavlu wrote:
>>> The only post-link analysis tool in this case should be arch/x86/tools/relocs.
>>> It produces a vmlinux.relocs file which is appended to vmlinux.bin. This is
>>> all internal to the Linux build. I'm not aware of any external tooling, such
>>> as kernel debuggers, that would require this relocation information in
>>> vmlinux.
>>
>> It would be good to know why --emit-relocs was added in the first place
>> - that might give us a hint. Lemme talk to Micha.
>>
>>> The size command used in your example includes only allocatable code, data and
>>> bss sections. It does not show size of any relocation sections.
>>
>> This:
>>
>> 361M vmlinux.before
>> 361M vmlinux.after
>>
>> is simple ls output.
>>
>> Maybe I need something else enabled in my .config which would show this
>> significant difference *and* *explain* it.
>>
>> Thx.
>>
> 
> The real question is: why does anyone care about the size of the vmlinux file specifically?

The vmlinux file is typically collected by various package build recipes, such
as distribution spec files, including the kernel's own binrpm-pkg target.
Users then have available a version of their kernel in the ELF format that
they can use with debuggers, disassemblers and other standard ELF tooling.

My initial motivation was to reduce size of vmlinux when debuginfo is enabled.
The present .rela.debug_* sections have a significant impact on the size but
they are not actually needed by debuggers. Stripping this data then saves
bandwidth and disk space required to work with vmlinux for the target users.

Petr

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Petr Pavlu]

On 11/23/22 16:54, Borislav Petkov wrote:
> On Wed, Nov 23, 2022 at 04:43:20PM +0100, Petr Pavlu wrote:
>> The only post-link analysis tool in this case should be arch/x86/tools/relocs.
>> It produces a vmlinux.relocs file which is appended to vmlinux.bin. This is
>> all internal to the Linux build. I'm not aware of any external tooling, such
>> as kernel debuggers, that would require this relocation information in
>> vmlinux.
> 
> It would be good to know why --emit-relocs was added in the first place
> - that might give us a hint. Lemme talk to Micha.

My understanding is that option --emit-relocs is needed to provide relocation
data for KASLR support. Produced relocation sections in vmlinux get processed
by arch/x86/tools/relocs which creates vmlinux.relocs with only relocation
data. This file is appended to vmlinux.bin. Upon boot, function
extract_kernel() decompresses the kernel at the target address and relocates
it using the vmlinux.relocs data.

>> The size command used in your example includes only allocatable code, data and
>> bss sections. It does not show size of any relocation sections.
> 
> This:
> 
> 361M vmlinux.before
> 361M vmlinux.after
> 
> is simple ls output.
> 
> Maybe I need something else enabled in my .config which would show this
> significant difference *and* *explain* it.

Option CONFIG_RANDOMIZE_BASE=y needs to be enabled. Switching it on should
automatically select also CONFIG_X86_NEED_RELOCS=y which is what actually
enables use of --emit-relocs in arch/x86/Makefile.

Petr

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Borislav Petkov]

On Thu, Nov 24, 2022 at 10:21:33AM +0100, Petr Pavlu wrote:
> Option CONFIG_RANDOMIZE_BASE=y needs to be enabled. Switching it on should
> automatically select also CONFIG_X86_NEED_RELOCS=y which is what actually
> enables use of --emit-relocs in arch/x86/Makefile.

Yeah, as I said in my previous mail:

"and before and after .configs simply have RANDOMIZE_BASE =n and =y,
respectively."

I just did it again to make sure:

-rwxr-xr-x 1 boris boris 377666112 Nov 24 13:28 vmlinux.before
-rwxr-xr-x 1 boris boris 377718768 Nov 24 13:33 vmlinux.after

With

$ grep -E "(NEED_RELOCS|RANDOMIZE)" .config
CONFIG_RANDOMIZE_BASE=y
CONFIG_X86_NEED_RELOCS=y
CONFIG_RANDOMIZE_MEMORY=y
CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING=0x0
CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
CONFIG_HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET=y
CONFIG_RANDOMIZE_KSTACK_OFFSET=y

that second vmlinux file is even a bit larger (~51K) ...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Petr Pavlu]

On 11/24/22 13:38, Borislav Petkov wrote:
> On Thu, Nov 24, 2022 at 10:21:33AM +0100, Petr Pavlu wrote:
>> Option CONFIG_RANDOMIZE_BASE=y needs to be enabled. Switching it on should
>> automatically select also CONFIG_X86_NEED_RELOCS=y which is what actually
>> enables use of --emit-relocs in arch/x86/Makefile.
> 
> Yeah, as I said in my previous mail:
> 
> "and before and after .configs simply have RANDOMIZE_BASE =n and =y,
> respectively."
> 
> I just did it again to make sure:
> 
> -rwxr-xr-x 1 boris boris 377666112 Nov 24 13:28 vmlinux.before
> -rwxr-xr-x 1 boris boris 377718768 Nov 24 13:33 vmlinux.after
> 
> With
> 
> $ grep -E "(NEED_RELOCS|RANDOMIZE)" .config
> CONFIG_RANDOMIZE_BASE=y
> CONFIG_X86_NEED_RELOCS=y
> CONFIG_RANDOMIZE_MEMORY=y
> CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING=0x0
> CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
> CONFIG_HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET=y
> CONFIG_RANDOMIZE_KSTACK_OFFSET=y
> 
> that second vmlinux file is even a bit larger (~51K) ...

If the before case is with RANDOMIZE_BASE=n and the after case is with
RANDOMIZE_BASE=y then it makes sense the resulting sizes are similar. With
RANDOMIZE_BASE=n, vmlinux is linked without --emit-relocs and so there will be
no relocation sections at all. With RANDOMIZE_BASE=y and my patch, the
sections get created but are stripped eventually. The increased size in the
second case is likely due to the logic to support the relocation process.

The case that the patch improves is with RANDOMIZE_BASE=y. Both the before and
after case need to have this option enabled. Comparison without my patch and
with it should then show that the patch significantly reduces the size of
vmlinux.

Petr

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Borislav Petkov]

On Thu, Nov 24, 2022 at 02:33:28PM +0100, Petr Pavlu wrote:
> If the before case is with RANDOMIZE_BASE=n and the after case is with
> RANDOMIZE_BASE=y then it makes sense the resulting sizes are similar. With
> RANDOMIZE_BASE=n, vmlinux is linked without --emit-relocs and so there will be
> no relocation sections at all. With RANDOMIZE_BASE=y and my patch, the
> sections get created but are stripped eventually. The increased size in the
> second case is likely due to the logic to support the relocation process.

This is in both cases with your patch, once with RANDOMIZE_BASE=y and
once with RANDOMIZE_BASE=n.

IOW, your patch actually makes my vmlinux bigger by 51K.

IOW, I cannot reproduce your

| Configuration      | With relocs | Stripped relocs |
| x86_64_defconfig   |       70 MB |           43 MB |

claim, but not with a defconfig but with my specially tailored config.

I guess the next thing I'll try is without your patch.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Borislav Petkov]

On Thu, Nov 24, 2022 at 04:41:11PM +0100, Borislav Petkov wrote:
> This is in both cases with your patch, once with RANDOMIZE_BASE=y and
> once with RANDOMIZE_BASE=n.
> 
> IOW, your patch actually makes my vmlinux bigger by 51K.
> 
> IOW, I cannot reproduce your
> 
> | Configuration      | With relocs | Stripped relocs |
> | x86_64_defconfig   |       70 MB |           43 MB |
> 
> claim, but not with a defconfig but with my specially tailored config.
> 
> I guess the next thing I'll try is without your patch.

Ok, I tried reproducing your defconfig numbers first:

defconfig:						vmlinux
- with patch: NEED_RELOCS=y				 48053656
- with patch, NEED_RELOCS=n, 				 48000696
- without patch, NEED_RELOCS=n, 			 48000696
- without patch: NEED_RELOCS=y				 83091744

So yes, I can reproduce your observation - defconfig *with* your patch
does slim down vmlinux, because, well, it is moving the reloc sections
into a separate file. Doh.

And with my .config too:

tailored config:
- with patch: 		NEED_RELOCS=y			377718768
- without patch, 	NEED_RELOCS=y			639952576

So I guess we can try it - let's see who complains.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Borislav Petkov]

On Tue, Sep 27, 2022 at 10:46:32AM +0200, Petr Pavlu wrote:
> diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
> index 35ce1a64068b..eba7709d75ae 100644
> --- a/arch/x86/boot/compressed/Makefile
> +++ b/arch/x86/boot/compressed/Makefile
> @@ -120,14 +120,12 @@ $(obj)/vmlinux.bin: vmlinux FORCE
>  
>  targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relocs
>  
> -CMD_RELOCS = arch/x86/tools/relocs
> -quiet_cmd_relocs = RELOCS  $@
> -      cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $<
> -$(obj)/vmlinux.relocs: vmlinux FORCE
> -	$(call if_changed,relocs)
> +# vmlinux.relocs is created by the vmlinux postlink step.
> +vmlinux.relocs: vmlinux
> +	@true
>  
>  vmlinux.bin.all-y := $(obj)/vmlinux.bin
> -vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
> +vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += vmlinux.relocs

Any particular reason why vmlinux.relocs must move out of $(obj)/ where
it resides now?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2] x86: Avoid relocation information in final vmlinux

[Petr Pavlu]

On 11/25/22 20:25, Borislav Petkov wrote:
> On Tue, Sep 27, 2022 at 10:46:32AM +0200, Petr Pavlu wrote:
>> diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
>> index 35ce1a64068b..eba7709d75ae 100644
>> --- a/arch/x86/boot/compressed/Makefile
>> +++ b/arch/x86/boot/compressed/Makefile
>> @@ -120,14 +120,12 @@ $(obj)/vmlinux.bin: vmlinux FORCE
>>  
>>  targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relocs
>>  
>> -CMD_RELOCS = arch/x86/tools/relocs
>> -quiet_cmd_relocs = RELOCS  $@
>> -      cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $<
>> -$(obj)/vmlinux.relocs: vmlinux FORCE
>> -	$(call if_changed,relocs)
>> +# vmlinux.relocs is created by the vmlinux postlink step.
>> +vmlinux.relocs: vmlinux
>> +	@true
>>  
>>  vmlinux.bin.all-y := $(obj)/vmlinux.bin
>> -vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
>> +vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += vmlinux.relocs
> 
> Any particular reason why vmlinux.relocs must move out of $(obj)/ where
> it resides now?

It looked slightly cleaner to me that Makefile.postlink generates
vmlinux.relocs at the same directory level where it is executed instead of
stuffing the file into arch/x86/boot/compressed. Makefile.postlink is invoked
in the context where it has no $(obj) variable set and has no immediate
knowledge about arch/x86/boot/compressed. It would also require that
Makefile.postlink makes first sure that this directory exists in the target
build tree as this step is run prior to arch/x86/boot/compressed/Makefile.

Let me know if this move of vmlinux.relocs is ok or if it would be preferred
to preserve its location.

With the move, the patch should be however at least updated to remove
vmlinux.relocs from arch/x86/boot/compressed/.gitignore, which I missed.

Thanks,
Petr

[tip: x86/build] x86/build: Avoid relocation information in final vmlinux

[tip-bot2 for Petr Pavlu]

The following commit has been merged into the x86/build branch of tip:

Commit-ID:     9d9173e9ceb63660ccad80f41373fd7eb48ff4ac
Gitweb:        https://git.kernel.org/tip/9d9173e9ceb63660ccad80f41373fd7eb48ff4ac
Author:        Petr Pavlu <petr.pavlu@suse.com>
AuthorDate:    Mon, 27 Mar 2023 11:54:06 +02:00
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Wed, 14 Jun 2023 19:54:40 +02:00

x86/build: Avoid relocation information in final vmlinux

The Linux build process on x86 roughly consists of compiling all input
files, statically linking them into a vmlinux ELF file, and then taking
and turning this file into an actual bzImage bootable file.

vmlinux has in this process two main purposes:
1) It is an intermediate build target on the way to produce the final
   bootable image.
2) It is a file that is expected to be used by debuggers and standard
   ELF tooling to work with the built kernel.

For the second purpose, a vmlinux file is typically collected by various
package build recipes, such as distribution spec files, including the
kernel's own tar-pkg target.

When building a kernel supporting KASLR with CONFIG_X86_NEED_RELOCS,
vmlinux contains also relocation information produced by using the
--emit-relocs linker option. This is utilized by subsequent build steps
to create vmlinux.relocs and produce a relocatable image. However, the
information is not needed by debuggers and other standard ELF tooling.

The issue is then that the collected vmlinux file and hence distribution
packages end up unnecessarily large because of this extra data. The
following is a size comparison of vmlinux v6.0 with and without the
relocation information:

  | Configuration      | With relocs | Stripped relocs |
  | x86_64_defconfig   |       70 MB |           43 MB |
  | +CONFIG_DEBUG_INFO |      818 MB |          367 MB |

Optimize a resulting vmlinux by adding a postlink step that splits the
relocation information into vmlinux.relocs and then strips it from the
vmlinux binary.

Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Link: https://lore.kernel.org/r/20220927084632.14531-1-petr.pavlu@suse.com
---
 arch/x86/Makefile.postlink        | 47 ++++++++++++++++++++++++++++++-
 arch/x86/boot/compressed/Makefile |  8 +----
 2 files changed, 50 insertions(+), 5 deletions(-)
 create mode 100644 arch/x86/Makefile.postlink

diff --git a/arch/x86/Makefile.postlink b/arch/x86/Makefile.postlink
new file mode 100644
index 0000000..936093d
--- /dev/null
+++ b/arch/x86/Makefile.postlink
@@ -0,0 +1,47 @@
+# SPDX-License-Identifier: GPL-2.0
+# ===========================================================================
+# Post-link x86 pass
+# ===========================================================================
+#
+# 1. Separate relocations from vmlinux into vmlinux.relocs.
+# 2. Strip relocations from vmlinux.
+
+PHONY := __archpost
+__archpost:
+
+-include include/config/auto.conf
+include $(srctree)/scripts/Kbuild.include
+
+CMD_RELOCS = arch/x86/tools/relocs
+OUT_RELOCS = arch/x86/boot/compressed
+quiet_cmd_relocs = RELOCS  $(OUT_RELOCS)/$@.relocs
+      cmd_relocs = \
+	mkdir -p $(OUT_RELOCS); \
+	$(CMD_RELOCS) $@ > $(OUT_RELOCS)/$@.relocs; \
+	$(CMD_RELOCS) --abs-relocs $@
+
+quiet_cmd_strip_relocs = RSTRIP  $@
+      cmd_strip_relocs = \
+	$(OBJCOPY) --remove-section='.rel.*' --remove-section='.rel__*' \
+		   --remove-section='.rela.*' --remove-section='.rela__*' $@
+
+# `@true` prevents complaint when there is nothing to be done
+
+vmlinux: FORCE
+	@true
+ifeq ($(CONFIG_X86_NEED_RELOCS),y)
+	$(call cmd,relocs)
+	$(call cmd,strip_relocs)
+endif
+
+%.ko: FORCE
+	@true
+
+clean:
+	@rm -f $(OUT_RELOCS)/vmlinux.relocs
+
+PHONY += FORCE clean
+
+FORCE:
+
+.PHONY: $(PHONY)
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index 6b6cfe6..0f78dbb 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -121,11 +121,9 @@ $(obj)/vmlinux.bin: vmlinux FORCE
 
 targets += $(patsubst $(obj)/%,%,$(vmlinux-objs-y)) vmlinux.bin.all vmlinux.relocs
 
-CMD_RELOCS = arch/x86/tools/relocs
-quiet_cmd_relocs = RELOCS  $@
-      cmd_relocs = $(CMD_RELOCS) $< > $@;$(CMD_RELOCS) --abs-relocs $<
-$(obj)/vmlinux.relocs: vmlinux FORCE
-	$(call if_changed,relocs)
+# vmlinux.relocs is created by the vmlinux postlink step.
+$(obj)/vmlinux.relocs: vmlinux
+	@true
 
 vmlinux.bin.all-y := $(obj)/vmlinux.bin
 vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs