* Re: [PATCH] Makefile: support compressed debug info
2020-05-12 19:23 ` Nick Desaulniers
@ 2020-05-12 20:01 ` Fangrui Song
2020-05-12 20:06 ` Sedat Dilek
2020-05-13 19:00 ` Nick Desaulniers
2020-05-12 20:02 ` [PATCH] " Sedat Dilek
2020-05-13 2:51 ` Masahiro Yamada
2 siblings, 2 replies; 32+ messages in thread
From: Fangrui Song @ 2020-05-12 20:01 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Sedat Dilek, Nick Desaulniers, Michal Marek,
Andrew Morton, Changbin Du, Randy Dunlap, Krzysztof Kozlowski,
Linux Kbuild mailing list, Linux Kernel Mailing List,
Clang-Built-Linux ML
On 2020-05-12, Nick Desaulniers wrote:
>On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>>
>> > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
>> > ><nick.desaulniers@gmail.com> wrote:
>> > >>
>> > >> As debug information gets larger and larger, it helps significantly save
>> > >> the size of vmlinux images to compress the information in the debug
>> > >> information sections. Note: this debug info is typically split off from
>> > >> the final compressed kernel image, which is why vmlinux is what's used
>> > >> in conjunction with GDB. Minimizing the debug info size should have no
>> > >> impact on boot times, or final compressed kernel image size.
>> > >>
>> Nick,
>>
>> I am OK with this patch.
>>
>> Fangrui provided the minimal requirement for
>> --compress-debug-sections=zlib
>>
>>
>> Is it worth recording in the help text?
>> Do you want to send v2?
>
>Yes I'd like to record that information. I can also record Sedat's
>Tested-by tag. Thank you for testing Sedat.
>
>I don't know what "linux-image-dbg file" are, or why they would be
>bigger. The size of the debug info is the primary concern with this
>config. It sounds like however that file is created might be
>problematic.
>
>Fangrui, I wasn't able to easily find what version of binutils first
>added support. Can you please teach me how to fish?
I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and
located the sources... I think an easier way is:
% cd binutils-gdb
% git show binutils-2_26:./gas/as.c | grep compress-debug-sections
--compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\
...
GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in
newer versions.
Similarly, for GNU ld:
% git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections
--compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\
(I have spent a lot of time investigating GNU ld's behavior :)
>Another question I had for Fangrui is, if the linker can compress
>these sections, shouldn't we just have the linker do it, not the the
>compiler and assembler? IIUC the debug info can contain relocations,
>so the linker would have to decompress these, perform relocations,
>then recompress these? I guess having the compiler and assembler
>compress the debug info as well would minimize the size of the .o
>files on disk.
The linker will decompress debug info unconditionally. Because
input .debug_info sections need to be concatenated to form the output
.debug_info . Whether the output .debug_info is compressed is controlled
by the linker option --compress-debug-sections=zlib, which is not
affected by the compression state of object files.
Both GNU as and GNU ld name the option --compress-debug-sections=zlib.
In a compiler driver context, an unfamiliar user may find
-Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib
confusing:/
>Otherwise I should add this flag to the assembler invocation, too, in
>v2. Thoughts?
Compressing object files along with the linked output should be fine. It
can save disk space. (It'd be great if you paste the comparison
with and w/o object files compressed)
Feel free to add:
Reviewed-by: Fangrui Song <maskray@google.com>
>I have a patch series that enables dwarf5 support in the kernel that
>I'm working up to. I wanted to send this first. Both roughly reduce
>the debug info size by 20% each, though I haven't measured them
>together, yet. Requires ToT binutils because there have been many
>fixes from reports of mine recently.
This will be awesome! I also heard that enabling DWARF v5 for our object
files can easily make debug info size smaller by 20%. Glad that the
kernel can benefit it as well:)
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH] Makefile: support compressed debug info
2020-05-12 20:01 ` Fangrui Song
@ 2020-05-12 20:06 ` Sedat Dilek
2020-05-13 19:00 ` Nick Desaulniers
1 sibling, 0 replies; 32+ messages in thread
From: Sedat Dilek @ 2020-05-12 20:06 UTC (permalink / raw)
To: Fangrui Song
Cc: Nick Desaulniers, Masahiro Yamada, Nick Desaulniers,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, Clang-Built-Linux ML
On Tue, May 12, 2020 at 10:01 PM Fangrui Song <maskray@google.com> wrote:
>
> On 2020-05-12, Nick Desaulniers wrote:
...
> >I have a patch series that enables dwarf5 support in the kernel that
> >I'm working up to. I wanted to send this first. Both roughly reduce
> >the debug info size by 20% each, though I haven't measured them
> >together, yet. Requires ToT binutils because there have been many
> >fixes from reports of mine recently.
>
> This will be awesome! I also heard that enabling DWARF v5 for our object
> files can easily make debug info size smaller by 20%. Glad that the
> kernel can benefit it as well:)
Please CC me on your pathcset - 20% smaller - I flip out.
- Sedat -
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH] Makefile: support compressed debug info
2020-05-12 20:01 ` Fangrui Song
2020-05-12 20:06 ` Sedat Dilek
@ 2020-05-13 19:00 ` Nick Desaulniers
2020-05-14 11:34 ` Nick Clifton
1 sibling, 1 reply; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-13 19:00 UTC (permalink / raw)
To: Fangrui Song, nickc, H.J. Lu
Cc: Masahiro Yamada, Sedat Dilek, Nick Desaulniers, Michal Marek,
Andrew Morton, Changbin Du, Randy Dunlap, Krzysztof Kozlowski,
Linux Kbuild mailing list, Linux Kernel Mailing List,
Clang-Built-Linux ML
On Tue, May 12, 2020 at 1:01 PM Fangrui Song <maskray@google.com> wrote:
>
> >Fangrui, I wasn't able to easily find what version of binutils first
> >added support. Can you please teach me how to fish?
>
> I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and
> located the sources... I think an easier way is:
>
> % cd binutils-gdb
> % git show binutils-2_26:./gas/as.c | grep compress-debug-sections
This assumes you knew to look at the binutils-2_26 tag, which is
putting the cart before the horse. ;)
I guess:
$ git log gas/as.c
/compress-debug-sections
commit 19a7fe52ae3d ("Make default compression gABI compliant")
looks related
$ git describe --contains "19a7fe52ae3d" | sed 's/~.*//'
users/hjl/linux/release/2.25.51.0.4
so it landed in 2.25.51.0.4.
+ Nick, H.J.
I'm unfamiliar with the git tag conventions of binutils. Does a patch
that landed in 2.25.51.0.4 mean it shipped in the official 2.25
release, or 2.26 release? Specifically, commit 19a7fe52ae3d.
> --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\
> ...
>
> GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in
> newer versions.
>
> Similarly, for GNU ld:
>
> % git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections
> --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\
>
> (I have spent a lot of time investigating GNU ld's behavior :)
>
> >Another question I had for Fangrui is, if the linker can compress
> >these sections, shouldn't we just have the linker do it, not the the
> >compiler and assembler? IIUC the debug info can contain relocations,
> >so the linker would have to decompress these, perform relocations,
> >then recompress these? I guess having the compiler and assembler
> >compress the debug info as well would minimize the size of the .o
> >files on disk.
>
> The linker will decompress debug info unconditionally. Because
> input .debug_info sections need to be concatenated to form the output
> .debug_info . Whether the output .debug_info is compressed is controlled
> by the linker option --compress-debug-sections=zlib, which is not
> affected by the compression state of object files.
>
> Both GNU as and GNU ld name the option --compress-debug-sections=zlib.
> In a compiler driver context, an unfamiliar user may find
> -Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib
> confusing:/
The kernel uses the compiler as the driver for out of line assembly,
as they are all preprocessed first. Most out of line assembly in the
kernel uses the C preprocessor to #include headers that share #defines
of common constants shared between C and asm. #ifdef __ASSEMBLY__ is
used frequently in these headers. But for the linker, the linker
itself is invoked as the driver, though there are a few
inconsistencies we've cleaned up or still have to.
>
> >Otherwise I should add this flag to the assembler invocation, too, in
> >v2. Thoughts?
>
> Compressing object files along with the linked output should be fine. It
> can save disk space. (It'd be great if you paste the comparison
> with and w/o object files compressed)
>
> Feel free to add:
>
> Reviewed-by: Fangrui Song <maskray@google.com>
Thanks, will add that to v2.
>
> >I have a patch series that enables dwarf5 support in the kernel that
> >I'm working up to. I wanted to send this first. Both roughly reduce
> >the debug info size by 20% each, though I haven't measured them
> >together, yet. Requires ToT binutils because there have been many
> >fixes from reports of mine recently.
>
> This will be awesome! I also heard that enabling DWARF v5 for our object
> files can easily make debug info size smaller by 20%. Glad that the
> kernel can benefit it as well:)
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH] Makefile: support compressed debug info
2020-05-13 19:00 ` Nick Desaulniers
@ 2020-05-14 11:34 ` Nick Clifton
2020-05-20 19:36 ` [PATCH v2] " Nick Desaulniers
0 siblings, 1 reply; 32+ messages in thread
From: Nick Clifton @ 2020-05-14 11:34 UTC (permalink / raw)
To: Nick Desaulniers, Fangrui Song, H.J. Lu
Cc: Masahiro Yamada, Sedat Dilek, Nick Desaulniers, Michal Marek,
Andrew Morton, Changbin Du, Randy Dunlap, Krzysztof Kozlowski,
Linux Kbuild mailing list, Linux Kernel Mailing List,
Clang-Built-Linux ML
Hi Nick,
> + Nick, H.J.
> I'm unfamiliar with the git tag conventions of binutils. Does a patch
> that landed in 2.25.51.0.4 mean it shipped in the official 2.25
> release, or 2.26 release? Specifically, commit 19a7fe52ae3d.
2.26.
The convention is that a released form of the binutils has a version
number of X.XX or possible X.XX.N. The current mainline development
sources have a version of X.XX.50 where X.XX is the latest release.
(So the current mainline sources are version 2.34.50). When a release
happens the XX value is incremented by one as part of the release
process, and the .50 is dropped. (So the next binutils release will
be 2.35).
So 2.25.51.0.4 is a development version which will then have been
released as binutils 2.26.
Cheers
Nick
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v2] Makefile: support compressed debug info
2020-05-14 11:34 ` Nick Clifton
@ 2020-05-20 19:36 ` Nick Desaulniers
2020-05-20 23:21 ` Nick Desaulniers
` (2 more replies)
0 siblings, 3 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-20 19:36 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, Nick Desaulniers,
David Blaikie, Michal Marek, Andrew Morton, Changbin Du,
Randy Dunlap, Stephen Rothwell, Mauro Carvalho Chehab,
Anshuman Khandual, Krzysztof Kozlowski, linux-kbuild,
linux-kernel, clang-built-linux
As debug information gets larger and larger, it helps significantly save
the size of vmlinux images to compress the information in the debug
information sections. Note: this debug info is typically split off from
the final compressed kernel image, which is why vmlinux is what's used
in conjunction with GDB. Minimizing the debug info size should have no
impact on boot times, or final compressed kernel image size.
All of the debug sections will have a `C` flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +18 [ = ] 0 [Unmapped]
-73.3% -114Ki [ = ] 0 .debug_aranges
-76.2% -2.01Mi [ = ] 0 .debug_frame
-73.6% -2.89Mi [ = ] 0 .debug_str
-80.7% -4.66Mi [ = ] 0 .debug_abbrev
-82.9% -4.88Mi [ = ] 0 .debug_ranges
-70.5% -9.04Mi [ = ] 0 .debug_line
-79.3% -10.9Mi [ = ] 0 .debug_loc
-39.5% -88.6Mi [ = ] 0 .debug_info
-18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +23 [ = ] 0 [Unmapped]
-65.6% -871 [ = ] 0 .debug_aranges
-77.4% -1.84Mi [ = ] 0 .debug_frame
-82.9% -2.33Mi [ = ] 0 .debug_abbrev
-73.1% -2.43Mi [ = ] 0 .debug_str
-84.8% -3.07Mi [ = ] 0 .debug_ranges
-65.9% -8.62Mi [ = ] 0 .debug_line
-86.2% -40.0Mi [ = ] 0 .debug_loc
-42.0% -64.1Mi [ = ] 0 .debug_info
-22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
Maximum resident set size (kbytes): 1566776
Suggested-by: David Blaikie <blakie@google.com>
Suggested-by: Fangrui Song <maskray@google.com>
Suggested-by: Nick Clifton <nickc@redhat.com>
Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++
lib/Kconfig.debug | 15 +++++++++++++++
2 files changed, 21 insertions(+)
diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif
+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b8f023e054b9..5a423cbfaea4 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)
+ depends on $(as-option,-Wa,--compress-debug-sections=zlib)
+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+, binutils 2.26+, and zlib.
+
+ Users of dpkg-deb via scripts/package/builddeb may
+ wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
+ the debug info again with a different compression scheme, which can
+ result in larger binaries.
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--
2.26.2.761.g0e0b3e54be-goog
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v2] Makefile: support compressed debug info
2020-05-20 19:36 ` [PATCH v2] " Nick Desaulniers
@ 2020-05-20 23:21 ` Nick Desaulniers
2020-05-20 23:22 ` Andrew Morton
2020-05-21 2:47 ` Masahiro Yamada
2 siblings, 0 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-20 23:21 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, Michal Marek,
Andrew Morton, Changbin Du, Randy Dunlap, Stephen Rothwell,
Mauro Carvalho Chehab, Anshuman Khandual, Krzysztof Kozlowski,
Linux Kbuild mailing list, LKML, clang-built-linux,
David Blaikie
On Wed, May 20, 2020 at 12:36 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> As debug information gets larger and larger, it helps significantly save
> the size of vmlinux images to compress the information in the debug
> information sections. Note: this debug info is typically split off from
> the final compressed kernel image, which is why vmlinux is what's used
> in conjunction with GDB. Minimizing the debug info size should have no
> impact on boot times, or final compressed kernel image size.
>
> All of the debug sections will have a `C` flag set.
> $ readelf -S <object file>
>
> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
> vmlinux.gcc75.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +18 [ = ] 0 [Unmapped]
> -73.3% -114Ki [ = ] 0 .debug_aranges
> -76.2% -2.01Mi [ = ] 0 .debug_frame
> -73.6% -2.89Mi [ = ] 0 .debug_str
> -80.7% -4.66Mi [ = ] 0 .debug_abbrev
> -82.9% -4.88Mi [ = ] 0 .debug_ranges
> -70.5% -9.04Mi [ = ] 0 .debug_line
> -79.3% -10.9Mi [ = ] 0 .debug_loc
> -39.5% -88.6Mi [ = ] 0 .debug_info
> -18.2% -123Mi [ = ] 0 TOTAL
>
> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \
> vmlinux.clang11.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +23 [ = ] 0 [Unmapped]
> -65.6% -871 [ = ] 0 .debug_aranges
> -77.4% -1.84Mi [ = ] 0 .debug_frame
> -82.9% -2.33Mi [ = ] 0 .debug_abbrev
> -73.1% -2.43Mi [ = ] 0 .debug_str
> -84.8% -3.07Mi [ = ] 0 .debug_ranges
> -65.9% -8.62Mi [ = ] 0 .debug_line
> -86.2% -40.0Mi [ = ] 0 .debug_loc
> -42.0% -64.1Mi [ = ] 0 .debug_info
> -22.1% -122Mi [ = ] 0 TOTAL
>
> For x86_64 defconfig + LLVM=1 (before):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
> Maximum resident set size (kbytes): 43856
>
> For x86_64 defconfig + LLVM=1 (after):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
> Maximum resident set size (kbytes): 1566776
>
> Suggested-by: David Blaikie <blakie@google.com>
Sorry, I have mistyped David's email, it should be:
Suggested-by: David Blaikie <blaikie@google.com>
(I missed the first `i` in the last name)
> Suggested-by: Fangrui Song <maskray@google.com>
> Suggested-by: Nick Clifton <nickc@redhat.com>
> Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Changes V1 -> V2:
> * rebase on linux-next.
> * Add assembler flags as per Fangrui.
> * Add note about KDEB_COMPRESS+scripts/package/builddeb
> as per Sedat and Masahiro.
> * Add note about bintutils version requirements as per Nick C.
> * Add note about measured increased build time and max RSS.
>
> Makefile | 6 ++++++
> lib/Kconfig.debug | 15 +++++++++++++++
> 2 files changed, 21 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 71687bfe1cd9..be8835296754 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
> $(call cc-option,-fno-var-tracking)
> endif
>
> +ifdef CONFIG_DEBUG_INFO_COMPRESSED
> +DEBUG_CFLAGS += -gz=zlib
> +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
> +KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +endif
> +
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
> export DEBUG_CFLAGS
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index b8f023e054b9..5a423cbfaea4 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_COMPRESSED
> + bool "Compressed debugging information"
> + depends on DEBUG_INFO
> + depends on $(cc-option,-gz=zlib)
> + depends on $(as-option,-Wa,--compress-debug-sections=zlib)
> + depends on $(ld-option,--compress-debug-sections=zlib)
> + help
> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> + 5.0+, binutils 2.26+, and zlib.
> +
> + Users of dpkg-deb via scripts/package/builddeb may
> + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
> + the debug info again with a different compression scheme, which can
> + result in larger binaries.
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on DEBUG_INFO
> --
> 2.26.2.761.g0e0b3e54be-goog
>
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2] Makefile: support compressed debug info
2020-05-20 19:36 ` [PATCH v2] " Nick Desaulniers
2020-05-20 23:21 ` Nick Desaulniers
@ 2020-05-20 23:22 ` Andrew Morton
2020-05-21 2:47 ` Masahiro Yamada
2 siblings, 0 replies; 32+ messages in thread
From: Andrew Morton @ 2020-05-20 23:22 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Sedat Dilek, Fangrui Song, Nick Clifton,
David Blaikie, Michal Marek, Changbin Du, Randy Dunlap,
Stephen Rothwell, Mauro Carvalho Chehab, Anshuman Khandual,
Krzysztof Kozlowski, linux-kbuild, linux-kernel,
clang-built-linux
On Wed, 20 May 2020 12:36:36 -0700 Nick Desaulniers <ndesaulniers@google.com> wrote:
> As debug information gets larger and larger, it helps significantly save
> the size of vmlinux images to compress the information in the debug
> information sections. Note: this debug info is typically split off from
> the final compressed kernel image, which is why vmlinux is what's used
> in conjunction with GDB. Minimizing the debug info size should have no
> impact on boot times, or final compressed kernel image size.
>
> All of the debug sections will have a `C` flag set.
> $ readelf -S <object file>
>
> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
> vmlinux.gcc75.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +18 [ = ] 0 [Unmapped]
> -73.3% -114Ki [ = ] 0 .debug_aranges
> -76.2% -2.01Mi [ = ] 0 .debug_frame
> -73.6% -2.89Mi [ = ] 0 .debug_str
> -80.7% -4.66Mi [ = ] 0 .debug_abbrev
> -82.9% -4.88Mi [ = ] 0 .debug_ranges
> -70.5% -9.04Mi [ = ] 0 .debug_line
> -79.3% -10.9Mi [ = ] 0 .debug_loc
> -39.5% -88.6Mi [ = ] 0 .debug_info
> -18.2% -123Mi [ = ] 0 TOTAL
>
> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \
> vmlinux.clang11.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +23 [ = ] 0 [Unmapped]
> -65.6% -871 [ = ] 0 .debug_aranges
> -77.4% -1.84Mi [ = ] 0 .debug_frame
> -82.9% -2.33Mi [ = ] 0 .debug_abbrev
> -73.1% -2.43Mi [ = ] 0 .debug_str
> -84.8% -3.07Mi [ = ] 0 .debug_ranges
> -65.9% -8.62Mi [ = ] 0 .debug_line
> -86.2% -40.0Mi [ = ] 0 .debug_loc
> -42.0% -64.1Mi [ = ] 0 .debug_info
> -22.1% -122Mi [ = ] 0 TOTAL
>
> For x86_64 defconfig + LLVM=1 (before):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
> Maximum resident set size (kbytes): 43856
>
> For x86_64 defconfig + LLVM=1 (after):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
> Maximum resident set size (kbytes): 1566776
I'm not sure who we're expecting to merge this, but I like shiny things ;)
> --- a/Makefile
> +++ b/Makefile
> @@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
> $(call cc-option,-fno-var-tracking)
> endif
>
> +ifdef CONFIG_DEBUG_INFO_COMPRESSED
> +DEBUG_CFLAGS += -gz=zlib
> +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
> +KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +endif
> +
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
> export DEBUG_CFLAGS
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index b8f023e054b9..5a423cbfaea4 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_COMPRESSED
> + bool "Compressed debugging information"
> + depends on DEBUG_INFO
> + depends on $(cc-option,-gz=zlib)
> + depends on $(as-option,-Wa,--compress-debug-sections=zlib)
> + depends on $(ld-option,--compress-debug-sections=zlib)
huh, I didn't know kbuild/kconfig could do this. Does it all work as
expected when cross-compiling?
> + help
> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> + 5.0+, binutils 2.26+, and zlib.
> +
> + Users of dpkg-deb via scripts/package/builddeb may
> + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
> + the debug info again with a different compression scheme, which can
> + result in larger binaries.
> +
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2] Makefile: support compressed debug info
2020-05-20 19:36 ` [PATCH v2] " Nick Desaulniers
2020-05-20 23:21 ` Nick Desaulniers
2020-05-20 23:22 ` Andrew Morton
@ 2020-05-21 2:47 ` Masahiro Yamada
2020-05-21 21:57 ` Nick Desaulniers
2 siblings, 1 reply; 32+ messages in thread
From: Masahiro Yamada @ 2020-05-21 2:47 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Mauro Carvalho Chehab, Anshuman Khandual,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, clang-built-linux
On Thu, May 21, 2020 at 4:36 AM 'Nick Desaulniers' via Clang Built
Linux <clang-built-linux@googlegroups.com> wrote:
>
> As debug information gets larger and larger, it helps significantly save
> the size of vmlinux images to compress the information in the debug
> information sections. Note: this debug info is typically split off from
> the final compressed kernel image, which is why vmlinux is what's used
> in conjunction with GDB. Minimizing the debug info size should have no
> impact on boot times, or final compressed kernel image size.
>
> All of the debug sections will have a `C` flag set.
> $ readelf -S <object file>
>
> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
> vmlinux.gcc75.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +18 [ = ] 0 [Unmapped]
> -73.3% -114Ki [ = ] 0 .debug_aranges
> -76.2% -2.01Mi [ = ] 0 .debug_frame
> -73.6% -2.89Mi [ = ] 0 .debug_str
> -80.7% -4.66Mi [ = ] 0 .debug_abbrev
> -82.9% -4.88Mi [ = ] 0 .debug_ranges
> -70.5% -9.04Mi [ = ] 0 .debug_line
> -79.3% -10.9Mi [ = ] 0 .debug_loc
> -39.5% -88.6Mi [ = ] 0 .debug_info
> -18.2% -123Mi [ = ] 0 TOTAL
>
> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \
> vmlinux.clang11.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +23 [ = ] 0 [Unmapped]
> -65.6% -871 [ = ] 0 .debug_aranges
> -77.4% -1.84Mi [ = ] 0 .debug_frame
> -82.9% -2.33Mi [ = ] 0 .debug_abbrev
> -73.1% -2.43Mi [ = ] 0 .debug_str
> -84.8% -3.07Mi [ = ] 0 .debug_ranges
> -65.9% -8.62Mi [ = ] 0 .debug_line
> -86.2% -40.0Mi [ = ] 0 .debug_loc
> -42.0% -64.1Mi [ = ] 0 .debug_info
> -22.1% -122Mi [ = ] 0 TOTAL
>
> For x86_64 defconfig + LLVM=1 (before):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
> Maximum resident set size (kbytes): 43856
>
> For x86_64 defconfig + LLVM=1 (after):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
> Maximum resident set size (kbytes): 1566776
>
> Suggested-by: David Blaikie <blakie@google.com>
> Suggested-by: Fangrui Song <maskray@google.com>
Suggested-by -> Reviewed-by
https://patchwork.kernel.org/patch/11524939/#23349551
> Suggested-by: Nick Clifton <nickc@redhat.com>
I do not know where this tag came from.
Nick Clifton taught us the version rule of binutils,but did not state
anything about this patch itself.
https://patchwork.kernel.org/patch/11524939/#23355175
> Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
I do not see the source of this tag, either...
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
snip
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_COMPRESSED
> + bool "Compressed debugging information"
> + depends on DEBUG_INFO
> + depends on $(cc-option,-gz=zlib)
> + depends on $(as-option,-Wa,--compress-debug-sections=zlib)
This does not work. (always false)
You cannot enable this option.
The comma between -Wa and --compress-debug-sections=zlib
is eaten by Kconfig parser because commas are delimiters
of function parameters.
Please write like this.
depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
> + depends on $(ld-option,--compress-debug-sections=zlib)
> + help
> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> + 5.0+, binutils 2.26+, and zlib.
> +
> + Users of dpkg-deb via scripts/package/builddeb may
> + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
> + the debug info again with a different compression scheme, which can
> + result in larger binaries.
No. This is not correct.
CONFIG_DEBUG_INFO_COMPRESSED compresses the only debug info part.
The other parts still get by benefit from the default KDEB_COMPRESS=xz.
The numbers are here:
CONFIG_DEBUG_INFO_COMPRESSED=y
-rw-r--r-- 1 masahiro masahiro 209077584 May 21 11:19
linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-26_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=y and KDEB_COMPRESS=none
-rw-r--r-- 1 masahiro masahiro 643051712 May 21 11:22
linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-27_amd64.deb
CONFIG_DEBUG_INFO_COMPRESSED=n
-rw-r--r-- 1 masahiro masahiro 112200308 May 21 11:40
linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-30_amd64.deb
For the deb package size perspective,
it is better to keep KDEB_COMPRESS as default.
The main motivation for changing KDEB_COMPRESS
is the build speed. (see commit 1a7f0a34ea7d05)
CONFIG_DEBUG_INFO_COMPRESSED has a downside
for the debug deb package, but we need to accept it.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v2] Makefile: support compressed debug info
2020-05-21 2:47 ` Masahiro Yamada
@ 2020-05-21 21:57 ` Nick Desaulniers
2020-05-21 22:00 ` [PATCH v3] " Nick Desaulniers
2020-05-24 3:56 ` [PATCH v2] " Masahiro Yamada
0 siblings, 2 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-21 21:57 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Mauro Carvalho Chehab, Anshuman Khandual,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, clang-built-linux
On Wed, May 20, 2020 at 7:48 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> > Suggested-by: Fangrui Song <maskray@google.com>
>
>
> Suggested-by -> Reviewed-by
>
> https://patchwork.kernel.org/patch/11524939/#23349551
Yes, my mistake.
> > Suggested-by: Nick Clifton <nickc@redhat.com>
>
>
> I do not know where this tag came from.
>
> Nick Clifton taught us the version rule of binutils,but did not state
> anything about this patch itself.
>
> https://patchwork.kernel.org/patch/11524939/#23355175
>
>
> > Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
>
> I do not see the source of this tag, either...
Not all contributions to open source need to be in the form of
patches. Both Sedat and Nick gave suggestions which ultimately
informed the contents of this patch. They should be rewarded for
their efforts, and incentivized to help improve the code base further.
I think suggested by tags are a good way to do that; but if it's
against a written convention or if you still disagree, it's not the
end of the world to me, and you may drop those tags from the v3.
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
> > DEBUG_INFO build and compile times are reduced too.
> > Only works with newer gcc versions.
> >
> > +config DEBUG_INFO_COMPRESSED
> > + bool "Compressed debugging information"
> > + depends on DEBUG_INFO
> > + depends on $(cc-option,-gz=zlib)
> > + depends on $(as-option,-Wa,--compress-debug-sections=zlib)
>
> This does not work. (always false)
Technically, always true. `-Wa` disables all warnings from the
assembler. Also, I did test this via `make menuconfig`.
> You cannot enable this option.
>
> The comma between -Wa and --compress-debug-sections=zlib
> is eaten by Kconfig parser because commas are delimiters
> of function parameters.
>
>
> Please write like this.
>
> depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
You're right, I knew this bug forgot. Will fix in v3.
> > + depends on $(ld-option,--compress-debug-sections=zlib)
> > + help
> > + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> > + 5.0+, binutils 2.26+, and zlib.
> > +
> > + Users of dpkg-deb via scripts/package/builddeb may
> > + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
> > + the debug info again with a different compression scheme, which can
> > + result in larger binaries.
>
> No. This is not correct.
>
> CONFIG_DEBUG_INFO_COMPRESSED compresses the only debug info part.
> The other parts still get by benefit from the default KDEB_COMPRESS=xz.
>
>
> The numbers are here:
>
>
> CONFIG_DEBUG_INFO_COMPRESSED=y
> -rw-r--r-- 1 masahiro masahiro 209077584 May 21 11:19
> linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-26_amd64.deb
>
>
> CONFIG_DEBUG_INFO_COMPRESSED=y and KDEB_COMPRESS=none
> -rw-r--r-- 1 masahiro masahiro 643051712 May 21 11:22
> linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-27_amd64.deb
>
>
> CONFIG_DEBUG_INFO_COMPRESSED=n
> -rw-r--r-- 1 masahiro masahiro 112200308 May 21 11:40
> linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-30_amd64.deb
>
>
>
>
> For the deb package size perspective,
> it is better to keep KDEB_COMPRESS as default.
>
> The main motivation for changing KDEB_COMPRESS
> is the build speed. (see commit 1a7f0a34ea7d05)
>
>
>
>
> CONFIG_DEBUG_INFO_COMPRESSED has a downside
> for the debug deb package, but we need to accept it.
Ah, I see. Thank you for those measurements. I'll send a v3 with
fixed up help text, but ultimately, I don't really care what it says
here. Please feel empowered to reword it should you choose to accept
+ apply it.
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v3] Makefile: support compressed debug info
2020-05-21 21:57 ` Nick Desaulniers
@ 2020-05-21 22:00 ` Nick Desaulniers
2020-05-22 11:56 ` Sedat Dilek
2020-05-26 10:26 ` Masahiro Yamada
2020-05-24 3:56 ` [PATCH v2] " Masahiro Yamada
1 sibling, 2 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-21 22:00 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, Nick Desaulniers,
David Blaikie, Michal Marek, Andrew Morton, Changbin Du,
Randy Dunlap, Stephen Rothwell, Anshuman Khandual,
Mauro Carvalho Chehab, Krzysztof Kozlowski, linux-kbuild,
linux-kernel, clang-built-linux
As debug information gets larger and larger, it helps significantly save
the size of vmlinux images to compress the information in the debug
information sections. Note: this debug info is typically split off from
the final compressed kernel image, which is why vmlinux is what's used
in conjunction with GDB. Minimizing the debug info size should have no
impact on boot times, or final compressed kernel image size.
All of the debug sections will have a `C` flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +18 [ = ] 0 [Unmapped]
-73.3% -114Ki [ = ] 0 .debug_aranges
-76.2% -2.01Mi [ = ] 0 .debug_frame
-73.6% -2.89Mi [ = ] 0 .debug_str
-80.7% -4.66Mi [ = ] 0 .debug_abbrev
-82.9% -4.88Mi [ = ] 0 .debug_ranges
-70.5% -9.04Mi [ = ] 0 .debug_line
-79.3% -10.9Mi [ = ] 0 .debug_loc
-39.5% -88.6Mi [ = ] 0 .debug_info
-18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +23 [ = ] 0 [Unmapped]
-65.6% -871 [ = ] 0 .debug_aranges
-77.4% -1.84Mi [ = ] 0 .debug_frame
-82.9% -2.33Mi [ = ] 0 .debug_abbrev
-73.1% -2.43Mi [ = ] 0 .debug_str
-84.8% -3.07Mi [ = ] 0 .debug_ranges
-65.9% -8.62Mi [ = ] 0 .debug_line
-86.2% -40.0Mi [ = ] 0 .debug_loc
-42.0% -64.1Mi [ = ] 0 .debug_info
-22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
Maximum resident set size (kbytes): 1566776
Suggested-by: David Blaikie <blaikie@google.com>
Suggested-by: Nick Clifton <nickc@redhat.com>
Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
Reviewed-by: Fangrui Song <maskray@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++
lib/Kconfig.debug | 17 +++++++++++++++++
2 files changed, 23 insertions(+)
diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif
+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b8f023e054b9..7fc82dcf814b 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)
+ depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+, binutils 2.26+, and zlib.
+
+ Users of dpkg-deb via scripts/package/builddeb may find an increase in
+ size of their debug .deb packages with this config set, due to the
+ debug info being compressed with zlib, then the object files being
+ recompressed with a different compression scheme. But this is still
+ preferable to setting $KDEB_COMPRESS to "none" which would be even
+ larger.
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--
2.27.0.rc0.183.gde8f92d652-goog
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v3] Makefile: support compressed debug info
2020-05-21 22:00 ` [PATCH v3] " Nick Desaulniers
@ 2020-05-22 11:56 ` Sedat Dilek
2020-05-26 10:26 ` Masahiro Yamada
1 sibling, 0 replies; 32+ messages in thread
From: Sedat Dilek @ 2020-05-22 11:56 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Anshuman Khandual, Mauro Carvalho Chehab,
Krzysztof Kozlowski, linux-kbuild, linux-kernel,
Clang-Built-Linux ML
On Fri, May 22, 2020 at 12:00 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> As debug information gets larger and larger, it helps significantly save
> the size of vmlinux images to compress the information in the debug
> information sections. Note: this debug info is typically split off from
> the final compressed kernel image, which is why vmlinux is what's used
> in conjunction with GDB. Minimizing the debug info size should have no
> impact on boot times, or final compressed kernel image size.
>
> All of the debug sections will have a `C` flag set.
> $ readelf -S <object file>
>
> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
> vmlinux.gcc75.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +18 [ = ] 0 [Unmapped]
> -73.3% -114Ki [ = ] 0 .debug_aranges
> -76.2% -2.01Mi [ = ] 0 .debug_frame
> -73.6% -2.89Mi [ = ] 0 .debug_str
> -80.7% -4.66Mi [ = ] 0 .debug_abbrev
> -82.9% -4.88Mi [ = ] 0 .debug_ranges
> -70.5% -9.04Mi [ = ] 0 .debug_line
> -79.3% -10.9Mi [ = ] 0 .debug_loc
> -39.5% -88.6Mi [ = ] 0 .debug_info
> -18.2% -123Mi [ = ] 0 TOTAL
>
> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \
> vmlinux.clang11.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +23 [ = ] 0 [Unmapped]
> -65.6% -871 [ = ] 0 .debug_aranges
> -77.4% -1.84Mi [ = ] 0 .debug_frame
> -82.9% -2.33Mi [ = ] 0 .debug_abbrev
> -73.1% -2.43Mi [ = ] 0 .debug_str
> -84.8% -3.07Mi [ = ] 0 .debug_ranges
> -65.9% -8.62Mi [ = ] 0 .debug_line
> -86.2% -40.0Mi [ = ] 0 .debug_loc
> -42.0% -64.1Mi [ = ] 0 .debug_info
> -22.1% -122Mi [ = ] 0 TOTAL
>
> For x86_64 defconfig + LLVM=1 (before):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
> Maximum resident set size (kbytes): 43856
>
> For x86_64 defconfig + LLVM=1 (after):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
> Maximum resident set size (kbytes): 1566776
>
> Suggested-by: David Blaikie <blaikie@google.com>
> Suggested-by: Nick Clifton <nickc@redhat.com>
> Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Reviewed-by: Fangrui Song <maskray@google.com>
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Re-tested V3 on top of Linux v5.7-rc6+ with Clang/LLD v10.0.1-rc1.
- Sedat -
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Changes V2 -> V3:
> * Fix blaikie@'s email addr.
> * Fix Fangrui's Reviewed-by tag as per Masahiro.
> * Fix help text as per Masahiro.
> * Fix -Wa$(comma)foo as per Masahiro.
>
> Changes V1 -> V2:
> * rebase on linux-next.
> * Add assembler flags as per Fangrui.
> * Add note about KDEB_COMPRESS+scripts/package/builddeb
> as per Sedat and Masahiro.
> * Add note about bintutils version requirements as per Nick C.
> * Add note about measured increased build time and max RSS.
> Makefile | 6 ++++++
> lib/Kconfig.debug | 17 +++++++++++++++++
> 2 files changed, 23 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 71687bfe1cd9..be8835296754 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
> $(call cc-option,-fno-var-tracking)
> endif
>
> +ifdef CONFIG_DEBUG_INFO_COMPRESSED
> +DEBUG_CFLAGS += -gz=zlib
> +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
> +KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +endif
> +
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
> export DEBUG_CFLAGS
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index b8f023e054b9..7fc82dcf814b 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_COMPRESSED
> + bool "Compressed debugging information"
> + depends on DEBUG_INFO
> + depends on $(cc-option,-gz=zlib)
> + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
> + depends on $(ld-option,--compress-debug-sections=zlib)
> + help
> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> + 5.0+, binutils 2.26+, and zlib.
> +
> + Users of dpkg-deb via scripts/package/builddeb may find an increase in
> + size of their debug .deb packages with this config set, due to the
> + debug info being compressed with zlib, then the object files being
> + recompressed with a different compression scheme. But this is still
> + preferable to setting $KDEB_COMPRESS to "none" which would be even
> + larger.
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on DEBUG_INFO
> --
> 2.27.0.rc0.183.gde8f92d652-goog
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3] Makefile: support compressed debug info
2020-05-21 22:00 ` [PATCH v3] " Nick Desaulniers
2020-05-22 11:56 ` Sedat Dilek
@ 2020-05-26 10:26 ` Masahiro Yamada
2020-05-26 15:53 ` Nick Desaulniers
1 sibling, 1 reply; 32+ messages in thread
From: Masahiro Yamada @ 2020-05-26 10:26 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Anshuman Khandual, Mauro Carvalho Chehab,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, clang-built-linux
On Fri, May 22, 2020 at 7:00 AM 'Nick Desaulniers' via Clang Built
Linux <clang-built-linux@googlegroups.com> wrote:
>
> As debug information gets larger and larger, it helps significantly save
> the size of vmlinux images to compress the information in the debug
> information sections. Note: this debug info is typically split off from
> the final compressed kernel image, which is why vmlinux is what's used
> in conjunction with GDB. Minimizing the debug info size should have no
> impact on boot times, or final compressed kernel image size.
>
> All of the debug sections will have a `C` flag set.
> $ readelf -S <object file>
>
> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
> vmlinux.gcc75.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +18 [ = ] 0 [Unmapped]
> -73.3% -114Ki [ = ] 0 .debug_aranges
> -76.2% -2.01Mi [ = ] 0 .debug_frame
> -73.6% -2.89Mi [ = ] 0 .debug_str
> -80.7% -4.66Mi [ = ] 0 .debug_abbrev
> -82.9% -4.88Mi [ = ] 0 .debug_ranges
> -70.5% -9.04Mi [ = ] 0 .debug_line
> -79.3% -10.9Mi [ = ] 0 .debug_loc
> -39.5% -88.6Mi [ = ] 0 .debug_info
> -18.2% -123Mi [ = ] 0 TOTAL
>
> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \
> vmlinux.clang11.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +23 [ = ] 0 [Unmapped]
> -65.6% -871 [ = ] 0 .debug_aranges
> -77.4% -1.84Mi [ = ] 0 .debug_frame
> -82.9% -2.33Mi [ = ] 0 .debug_abbrev
> -73.1% -2.43Mi [ = ] 0 .debug_str
> -84.8% -3.07Mi [ = ] 0 .debug_ranges
> -65.9% -8.62Mi [ = ] 0 .debug_line
> -86.2% -40.0Mi [ = ] 0 .debug_loc
> -42.0% -64.1Mi [ = ] 0 .debug_info
> -22.1% -122Mi [ = ] 0 TOTAL
>
> For x86_64 defconfig + LLVM=1 (before):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
> Maximum resident set size (kbytes): 43856
>
> For x86_64 defconfig + LLVM=1 (after):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
> Maximum resident set size (kbytes): 1566776
>
> Suggested-by: David Blaikie <blaikie@google.com>
> Suggested-by: Nick Clifton <nickc@redhat.com>
> Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Reviewed-by: Fangrui Song <maskray@google.com>
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Suggested-by seems strange to me, but
I decided to not be worried too much.
Applied to linux-kbuild.
Thanks.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3] Makefile: support compressed debug info
2020-05-26 10:26 ` Masahiro Yamada
@ 2020-05-26 15:53 ` Nick Desaulniers
2020-05-26 16:15 ` Masahiro Yamada
0 siblings, 1 reply; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-26 15:53 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Anshuman Khandual, Mauro Carvalho Chehab,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, clang-built-linux
On Tue, May 26, 2020 at 3:28 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> > Suggested-by: David Blaikie <blaikie@google.com>
> > Suggested-by: Nick Clifton <nickc@redhat.com>
> > Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
> > Reviewed-by: Fangrui Song <maskray@google.com>
> > Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
>
>
>
> Suggested-by seems strange to me, but
> I decided to not be worried too much.
Ah, sorry, Monday was a holiday. I unplugged for the long weekend.
https://en.wikipedia.org/wiki/Memorial_Day
I like the suggestion to simply say "thanks to ..." in the commit
message and will use that next time. I was ok to send a v4 with it.
>
> Applied to linux-kbuild.
Appreciated. I will have a dwarf-5 patch set, too. I'm not thrilled
with how I wired up Kconfig; maybe posting it for feedback is better
than me worrying about it too much. Hopefully will send this week,
assuming there's not too many bugs that require my immediate attention
(there never is...).
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3] Makefile: support compressed debug info
2020-05-26 15:53 ` Nick Desaulniers
@ 2020-05-26 16:15 ` Masahiro Yamada
2020-05-26 17:03 ` Nick Desaulniers
2020-05-26 17:18 ` [PATCH v5] " Nick Desaulniers
0 siblings, 2 replies; 32+ messages in thread
From: Masahiro Yamada @ 2020-05-26 16:15 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Anshuman Khandual, Mauro Carvalho Chehab,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, clang-built-linux
On Wed, May 27, 2020 at 12:53 AM 'Nick Desaulniers' via Clang Built
Linux <clang-built-linux@googlegroups.com> wrote:
>
> On Tue, May 26, 2020 at 3:28 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > > Suggested-by: David Blaikie <blaikie@google.com>
> > > Suggested-by: Nick Clifton <nickc@redhat.com>
> > > Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
> > > Reviewed-by: Fangrui Song <maskray@google.com>
> > > Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> >
> >
> >
> > Suggested-by seems strange to me, but
> > I decided to not be worried too much.
>
> Ah, sorry, Monday was a holiday. I unplugged for the long weekend.
> https://en.wikipedia.org/wiki/Memorial_Day
>
> I like the suggestion to simply say "thanks to ..." in the commit
> message and will use that next time. I was ok to send a v4 with it.
If you send v4, I will replace.
Thanks.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v4] Makefile: support compressed debug info
2020-05-26 16:15 ` Masahiro Yamada
@ 2020-05-26 17:03 ` Nick Desaulniers
2020-05-26 17:18 ` [PATCH v5] " Nick Desaulniers
1 sibling, 0 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-26 17:03 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, Nick Desaulniers,
David Blaikie, Michal Marek, Catalin Marinas, Will Deacon,
Vincenzo Frascino, Thomas Gleixner, Naohiro Aota, Andrew Morton,
Changbin Du, Randy Dunlap, Stephen Rothwell, Anshuman Khandual,
Mauro Carvalho Chehab, Krzysztof Kozlowski, linux-kbuild,
linux-kernel, linux-arm-kernel, clang-built-linux
As debug information gets larger and larger, it helps significantly save
the size of vmlinux images to compress the information in the debug
information sections. Note: this debug info is typically split off from
the final compressed kernel image, which is why vmlinux is what's used
in conjunction with GDB. Minimizing the debug info size should have no
impact on boot times, or final compressed kernel image size.
All of the debug sections will have a `C` flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +18 [ = ] 0 [Unmapped]
-73.3% -114Ki [ = ] 0 .debug_aranges
-76.2% -2.01Mi [ = ] 0 .debug_frame
-73.6% -2.89Mi [ = ] 0 .debug_str
-80.7% -4.66Mi [ = ] 0 .debug_abbrev
-82.9% -4.88Mi [ = ] 0 .debug_ranges
-70.5% -9.04Mi [ = ] 0 .debug_line
-79.3% -10.9Mi [ = ] 0 .debug_loc
-39.5% -88.6Mi [ = ] 0 .debug_info
-18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +23 [ = ] 0 [Unmapped]
-65.6% -871 [ = ] 0 .debug_aranges
-77.4% -1.84Mi [ = ] 0 .debug_frame
-82.9% -2.33Mi [ = ] 0 .debug_abbrev
-73.1% -2.43Mi [ = ] 0 .debug_str
-84.8% -3.07Mi [ = ] 0 .debug_ranges
-65.9% -8.62Mi [ = ] 0 .debug_line
-86.2% -40.0Mi [ = ] 0 .debug_loc
-42.0% -64.1Mi [ = ] 0 .debug_info
-22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
Maximum resident set size (kbytes): 1566776
Thanks to:
Nick Clifton helped us to provide the minimal binutils version.
Sedat Dilet found an increase in size of debug .deb package.
Cc: Nick Clifton <nickc@redhat.com>
Cc: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: David Blaikie <blaikie@google.com>
Reviewed-by: Fangrui Song <maskray@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per
Masahiro.
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++
arch/arm64/kernel/vdso32/Makefile | 2 +-
lib/Kconfig.debug | 17 +++++++++++++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif
+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
index 3964738ebbde..5fd7792d03fc 100644
--- a/arch/arm64/kernel/vdso32/Makefile
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -135,7 +135,7 @@ c-obj-vdso-gettimeofday := vgettimeofday.o
asm-obj-vdso := sigreturn.o
ifneq ($(c-gettimeofday-y),)
-VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) -marm
endif
VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b8f023e054b9..7fc82dcf814b 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)
+ depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+, binutils 2.26+, and zlib.
+
+ Users of dpkg-deb via scripts/package/builddeb may find an increase in
+ size of their debug .deb packages with this config set, due to the
+ debug info being compressed with zlib, then the object files being
+ recompressed with a different compression scheme. But this is still
+ preferable to setting $KDEB_COMPRESS to "none" which would be even
+ larger.
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--
2.27.0.rc0.183.gde8f92d652-goog
^ permalink raw reply related [flat|nested] 32+ messages in thread* [PATCH v4] Makefile: support compressed debug info
@ 2020-05-26 17:03 ` Nick Desaulniers
0 siblings, 0 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-26 17:03 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Nick Clifton, Catalin Marinas, Vincenzo Frascino, Will Deacon,
Naohiro Aota, Stephen Rothwell, Fangrui Song,
Mauro Carvalho Chehab, linux-kbuild, Krzysztof Kozlowski,
clang-built-linux, Anshuman Khandual, Sedat Dilek, David Blaikie,
Thomas Gleixner, linux-arm-kernel, Michal Marek, Randy Dunlap,
Nick Desaulniers, linux-kernel, Changbin Du, Andrew Morton
As debug information gets larger and larger, it helps significantly save
the size of vmlinux images to compress the information in the debug
information sections. Note: this debug info is typically split off from
the final compressed kernel image, which is why vmlinux is what's used
in conjunction with GDB. Minimizing the debug info size should have no
impact on boot times, or final compressed kernel image size.
All of the debug sections will have a `C` flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +18 [ = ] 0 [Unmapped]
-73.3% -114Ki [ = ] 0 .debug_aranges
-76.2% -2.01Mi [ = ] 0 .debug_frame
-73.6% -2.89Mi [ = ] 0 .debug_str
-80.7% -4.66Mi [ = ] 0 .debug_abbrev
-82.9% -4.88Mi [ = ] 0 .debug_ranges
-70.5% -9.04Mi [ = ] 0 .debug_line
-79.3% -10.9Mi [ = ] 0 .debug_loc
-39.5% -88.6Mi [ = ] 0 .debug_info
-18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +23 [ = ] 0 [Unmapped]
-65.6% -871 [ = ] 0 .debug_aranges
-77.4% -1.84Mi [ = ] 0 .debug_frame
-82.9% -2.33Mi [ = ] 0 .debug_abbrev
-73.1% -2.43Mi [ = ] 0 .debug_str
-84.8% -3.07Mi [ = ] 0 .debug_ranges
-65.9% -8.62Mi [ = ] 0 .debug_line
-86.2% -40.0Mi [ = ] 0 .debug_loc
-42.0% -64.1Mi [ = ] 0 .debug_info
-22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
Maximum resident set size (kbytes): 1566776
Thanks to:
Nick Clifton helped us to provide the minimal binutils version.
Sedat Dilet found an increase in size of debug .deb package.
Cc: Nick Clifton <nickc@redhat.com>
Cc: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: David Blaikie <blaikie@google.com>
Reviewed-by: Fangrui Song <maskray@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per
Masahiro.
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++
arch/arm64/kernel/vdso32/Makefile | 2 +-
lib/Kconfig.debug | 17 +++++++++++++++++
3 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif
+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
index 3964738ebbde..5fd7792d03fc 100644
--- a/arch/arm64/kernel/vdso32/Makefile
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -135,7 +135,7 @@ c-obj-vdso-gettimeofday := vgettimeofday.o
asm-obj-vdso := sigreturn.o
ifneq ($(c-gettimeofday-y),)
-VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) -marm
endif
VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b8f023e054b9..7fc82dcf814b 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)
+ depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+, binutils 2.26+, and zlib.
+
+ Users of dpkg-deb via scripts/package/builddeb may find an increase in
+ size of their debug .deb packages with this config set, due to the
+ debug info being compressed with zlib, then the object files being
+ recompressed with a different compression scheme. But this is still
+ preferable to setting $KDEB_COMPRESS to "none" which would be even
+ larger.
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--
2.27.0.rc0.183.gde8f92d652-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v4] Makefile: support compressed debug info
2020-05-26 17:03 ` Nick Desaulniers
@ 2020-05-26 17:06 ` Nick Desaulniers
-1 siblings, 0 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-26 17:06 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Catalin Marinas, Will Deacon, Vincenzo Frascino,
Thomas Gleixner, Naohiro Aota, Andrew Morton, Changbin Du,
Randy Dunlap, Stephen Rothwell, Anshuman Khandual,
Mauro Carvalho Chehab, Krzysztof Kozlowski,
Linux Kbuild mailing list, LKML, Linux ARM, clang-built-linux
On Tue, May 26, 2020 at 10:03 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> As debug information gets larger and larger, it helps significantly save
> the size of vmlinux images to compress the information in the debug
> information sections. Note: this debug info is typically split off from
> the final compressed kernel image, which is why vmlinux is what's used
> in conjunction with GDB. Minimizing the debug info size should have no
> impact on boot times, or final compressed kernel image size.
>
> All of the debug sections will have a `C` flag set.
> $ readelf -S <object file>
>
> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
> vmlinux.gcc75.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +18 [ = ] 0 [Unmapped]
> -73.3% -114Ki [ = ] 0 .debug_aranges
> -76.2% -2.01Mi [ = ] 0 .debug_frame
> -73.6% -2.89Mi [ = ] 0 .debug_str
> -80.7% -4.66Mi [ = ] 0 .debug_abbrev
> -82.9% -4.88Mi [ = ] 0 .debug_ranges
> -70.5% -9.04Mi [ = ] 0 .debug_line
> -79.3% -10.9Mi [ = ] 0 .debug_loc
> -39.5% -88.6Mi [ = ] 0 .debug_info
> -18.2% -123Mi [ = ] 0 TOTAL
>
> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \
> vmlinux.clang11.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +23 [ = ] 0 [Unmapped]
> -65.6% -871 [ = ] 0 .debug_aranges
> -77.4% -1.84Mi [ = ] 0 .debug_frame
> -82.9% -2.33Mi [ = ] 0 .debug_abbrev
> -73.1% -2.43Mi [ = ] 0 .debug_str
> -84.8% -3.07Mi [ = ] 0 .debug_ranges
> -65.9% -8.62Mi [ = ] 0 .debug_line
> -86.2% -40.0Mi [ = ] 0 .debug_loc
> -42.0% -64.1Mi [ = ] 0 .debug_info
> -22.1% -122Mi [ = ] 0 TOTAL
>
> For x86_64 defconfig + LLVM=1 (before):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
> Maximum resident set size (kbytes): 43856
>
> For x86_64 defconfig + LLVM=1 (after):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
> Maximum resident set size (kbytes): 1566776
>
> Thanks to:
> Nick Clifton helped us to provide the minimal binutils version.
> Sedat Dilet found an increase in size of debug .deb package.
>
> Cc: Nick Clifton <nickc@redhat.com>
> Cc: Sedat Dilek <sedat.dilek@gmail.com>
> Suggested-by: David Blaikie <blaikie@google.com>
> Reviewed-by: Fangrui Song <maskray@google.com>
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Changes V3 -> V4:
> * Add thanks line to commit message as per Masahiro.
> * Swap Sugguested-by to Cc for two lines in commit message, as per
> Masahiro.
>
> Changes V2 -> V3:
> * Fix blaikie@'s email addr.
> * Fix Fangrui's Reviewed-by tag as per Masahiro.
> * Fix help text as per Masahiro.
> * Fix -Wa$(comma)foo as per Masahiro.
>
> Changes V1 -> V2:
> * rebase on linux-next.
> * Add assembler flags as per Fangrui.
> * Add note about KDEB_COMPRESS+scripts/package/builddeb
> as per Sedat and Masahiro.
> * Add note about bintutils version requirements as per Nick C.
> * Add note about measured increased build time and max RSS.
>
> Makefile | 6 ++++++
> arch/arm64/kernel/vdso32/Makefile | 2 +-
Sorry, I was wondering why Will and TGLX got cc'ed. My tree was dirty
when I amended ... was carrying another patch to send, please
disregard v4, and sorry for the noise.
> lib/Kconfig.debug | 17 +++++++++++++++++
> 3 files changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 71687bfe1cd9..be8835296754 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
> $(call cc-option,-fno-var-tracking)
> endif
>
> +ifdef CONFIG_DEBUG_INFO_COMPRESSED
> +DEBUG_CFLAGS += -gz=zlib
> +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
> +KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +endif
> +
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
> export DEBUG_CFLAGS
>
> diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
> index 3964738ebbde..5fd7792d03fc 100644
> --- a/arch/arm64/kernel/vdso32/Makefile
> +++ b/arch/arm64/kernel/vdso32/Makefile
> @@ -135,7 +135,7 @@ c-obj-vdso-gettimeofday := vgettimeofday.o
> asm-obj-vdso := sigreturn.o
>
> ifneq ($(c-gettimeofday-y),)
> -VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
> +VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) -marm
> endif
>
> VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index b8f023e054b9..7fc82dcf814b 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_COMPRESSED
> + bool "Compressed debugging information"
> + depends on DEBUG_INFO
> + depends on $(cc-option,-gz=zlib)
> + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
> + depends on $(ld-option,--compress-debug-sections=zlib)
> + help
> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> + 5.0+, binutils 2.26+, and zlib.
> +
> + Users of dpkg-deb via scripts/package/builddeb may find an increase in
> + size of their debug .deb packages with this config set, due to the
> + debug info being compressed with zlib, then the object files being
> + recompressed with a different compression scheme. But this is still
> + preferable to setting $KDEB_COMPRESS to "none" which would be even
> + larger.
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on DEBUG_INFO
> --
> 2.27.0.rc0.183.gde8f92d652-goog
>
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v4] Makefile: support compressed debug info
@ 2020-05-26 17:06 ` Nick Desaulniers
0 siblings, 0 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-26 17:06 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Naohiro Aota, Stephen Rothwell, Michal Marek, Nick Clifton,
Fangrui Song, Anshuman Khandual, Mauro Carvalho Chehab,
Catalin Marinas, Randy Dunlap, LKML, Krzysztof Kozlowski,
Changbin Du, Linux ARM, clang-built-linux, Sedat Dilek,
David Blaikie, Thomas Gleixner, Vincenzo Frascino, Will Deacon,
Andrew Morton, Linux Kbuild mailing list
On Tue, May 26, 2020 at 10:03 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> As debug information gets larger and larger, it helps significantly save
> the size of vmlinux images to compress the information in the debug
> information sections. Note: this debug info is typically split off from
> the final compressed kernel image, which is why vmlinux is what's used
> in conjunction with GDB. Minimizing the debug info size should have no
> impact on boot times, or final compressed kernel image size.
>
> All of the debug sections will have a `C` flag set.
> $ readelf -S <object file>
>
> $ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
> vmlinux.gcc75.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +18 [ = ] 0 [Unmapped]
> -73.3% -114Ki [ = ] 0 .debug_aranges
> -76.2% -2.01Mi [ = ] 0 .debug_frame
> -73.6% -2.89Mi [ = ] 0 .debug_str
> -80.7% -4.66Mi [ = ] 0 .debug_abbrev
> -82.9% -4.88Mi [ = ] 0 .debug_ranges
> -70.5% -9.04Mi [ = ] 0 .debug_line
> -79.3% -10.9Mi [ = ] 0 .debug_loc
> -39.5% -88.6Mi [ = ] 0 .debug_info
> -18.2% -123Mi [ = ] 0 TOTAL
>
> $ bloaty vmlinux.clang11.compressed.dwarf4 -- \
> vmlinux.clang11.uncompressed.dwarf4
>
> FILE SIZE VM SIZE
> -------------- --------------
> +0.0% +23 [ = ] 0 [Unmapped]
> -65.6% -871 [ = ] 0 .debug_aranges
> -77.4% -1.84Mi [ = ] 0 .debug_frame
> -82.9% -2.33Mi [ = ] 0 .debug_abbrev
> -73.1% -2.43Mi [ = ] 0 .debug_str
> -84.8% -3.07Mi [ = ] 0 .debug_ranges
> -65.9% -8.62Mi [ = ] 0 .debug_line
> -86.2% -40.0Mi [ = ] 0 .debug_loc
> -42.0% -64.1Mi [ = ] 0 .debug_info
> -22.1% -122Mi [ = ] 0 TOTAL
>
> For x86_64 defconfig + LLVM=1 (before):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
> Maximum resident set size (kbytes): 43856
>
> For x86_64 defconfig + LLVM=1 (after):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
> Maximum resident set size (kbytes): 1566776
>
> Thanks to:
> Nick Clifton helped us to provide the minimal binutils version.
> Sedat Dilet found an increase in size of debug .deb package.
>
> Cc: Nick Clifton <nickc@redhat.com>
> Cc: Sedat Dilek <sedat.dilek@gmail.com>
> Suggested-by: David Blaikie <blaikie@google.com>
> Reviewed-by: Fangrui Song <maskray@google.com>
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Changes V3 -> V4:
> * Add thanks line to commit message as per Masahiro.
> * Swap Sugguested-by to Cc for two lines in commit message, as per
> Masahiro.
>
> Changes V2 -> V3:
> * Fix blaikie@'s email addr.
> * Fix Fangrui's Reviewed-by tag as per Masahiro.
> * Fix help text as per Masahiro.
> * Fix -Wa$(comma)foo as per Masahiro.
>
> Changes V1 -> V2:
> * rebase on linux-next.
> * Add assembler flags as per Fangrui.
> * Add note about KDEB_COMPRESS+scripts/package/builddeb
> as per Sedat and Masahiro.
> * Add note about bintutils version requirements as per Nick C.
> * Add note about measured increased build time and max RSS.
>
> Makefile | 6 ++++++
> arch/arm64/kernel/vdso32/Makefile | 2 +-
Sorry, I was wondering why Will and TGLX got cc'ed. My tree was dirty
when I amended ... was carrying another patch to send, please
disregard v4, and sorry for the noise.
> lib/Kconfig.debug | 17 +++++++++++++++++
> 3 files changed, 24 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 71687bfe1cd9..be8835296754 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
> $(call cc-option,-fno-var-tracking)
> endif
>
> +ifdef CONFIG_DEBUG_INFO_COMPRESSED
> +DEBUG_CFLAGS += -gz=zlib
> +KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
> +KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +endif
> +
> KBUILD_CFLAGS += $(DEBUG_CFLAGS)
> export DEBUG_CFLAGS
>
> diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
> index 3964738ebbde..5fd7792d03fc 100644
> --- a/arch/arm64/kernel/vdso32/Makefile
> +++ b/arch/arm64/kernel/vdso32/Makefile
> @@ -135,7 +135,7 @@ c-obj-vdso-gettimeofday := vgettimeofday.o
> asm-obj-vdso := sigreturn.o
>
> ifneq ($(c-gettimeofday-y),)
> -VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
> +VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) -marm
> endif
>
> VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index b8f023e054b9..7fc82dcf814b 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_COMPRESSED
> + bool "Compressed debugging information"
> + depends on DEBUG_INFO
> + depends on $(cc-option,-gz=zlib)
> + depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
> + depends on $(ld-option,--compress-debug-sections=zlib)
> + help
> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> + 5.0+, binutils 2.26+, and zlib.
> +
> + Users of dpkg-deb via scripts/package/builddeb may find an increase in
> + size of their debug .deb packages with this config set, due to the
> + debug info being compressed with zlib, then the object files being
> + recompressed with a different compression scheme. But this is still
> + preferable to setting $KDEB_COMPRESS to "none" which would be even
> + larger.
> +
> config DEBUG_INFO_SPLIT
> bool "Produce split debuginfo in .dwo files"
> depends on DEBUG_INFO
> --
> 2.27.0.rc0.183.gde8f92d652-goog
>
--
Thanks,
~Nick Desaulniers
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v5] Makefile: support compressed debug info
2020-05-26 16:15 ` Masahiro Yamada
2020-05-26 17:03 ` Nick Desaulniers
@ 2020-05-26 17:18 ` Nick Desaulniers
1 sibling, 0 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-26 17:18 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, Nick Desaulniers,
David Blaikie, Michal Marek, Andrew Morton, Changbin Du,
Randy Dunlap, Stephen Rothwell, Mauro Carvalho Chehab,
Anshuman Khandual, Krzysztof Kozlowski, linux-kbuild,
linux-kernel, clang-built-linux
As debug information gets larger and larger, it helps significantly save
the size of vmlinux images to compress the information in the debug
information sections. Note: this debug info is typically split off from
the final compressed kernel image, which is why vmlinux is what's used
in conjunction with GDB. Minimizing the debug info size should have no
impact on boot times, or final compressed kernel image size.
All of the debug sections will have a `C` flag set.
$ readelf -S <object file>
$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +18 [ = ] 0 [Unmapped]
-73.3% -114Ki [ = ] 0 .debug_aranges
-76.2% -2.01Mi [ = ] 0 .debug_frame
-73.6% -2.89Mi [ = ] 0 .debug_str
-80.7% -4.66Mi [ = ] 0 .debug_abbrev
-82.9% -4.88Mi [ = ] 0 .debug_ranges
-70.5% -9.04Mi [ = ] 0 .debug_line
-79.3% -10.9Mi [ = ] 0 .debug_loc
-39.5% -88.6Mi [ = ] 0 .debug_info
-18.2% -123Mi [ = ] 0 TOTAL
$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4
FILE SIZE VM SIZE
-------------- --------------
+0.0% +23 [ = ] 0 [Unmapped]
-65.6% -871 [ = ] 0 .debug_aranges
-77.4% -1.84Mi [ = ] 0 .debug_frame
-82.9% -2.33Mi [ = ] 0 .debug_abbrev
-73.1% -2.43Mi [ = ] 0 .debug_str
-84.8% -3.07Mi [ = ] 0 .debug_ranges
-65.9% -8.62Mi [ = ] 0 .debug_line
-86.2% -40.0Mi [ = ] 0 .debug_loc
-42.0% -64.1Mi [ = ] 0 .debug_info
-22.1% -122Mi [ = ] 0 TOTAL
For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
Maximum resident set size (kbytes): 43856
For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
Maximum resident set size (kbytes): 1566776
Thanks to:
Nick Clifton helped us to provide the minimal binutils version.
Sedat Dilet found an increase in size of debug .deb package.
Cc: Nick Clifton <nickc@redhat.com>
Cc: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: David Blaikie <blaikie@google.com>
Reviewed-by: Fangrui Song <maskray@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
Changes V4 -> V5:
* Drop unrelated hunk from a dirty tree.
Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per
Masahiro.
Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++
lib/Kconfig.debug | 17 +++++++++++++++++
2 files changed, 23 insertions(+)
diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif
+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b8f023e054b9..7fc82dcf814b 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.
+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)
+ depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+, binutils 2.26+, and zlib.
+
+ Users of dpkg-deb via scripts/package/builddeb may find an increase in
+ size of their debug .deb packages with this config set, due to the
+ debug info being compressed with zlib, then the object files being
+ recompressed with a different compression scheme. But this is still
+ preferable to setting $KDEB_COMPRESS to "none" which would be even
+ larger.
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--
2.27.0.rc0.183.gde8f92d652-goog
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH v2] Makefile: support compressed debug info
2020-05-21 21:57 ` Nick Desaulniers
2020-05-21 22:00 ` [PATCH v3] " Nick Desaulniers
@ 2020-05-24 3:56 ` Masahiro Yamada
2020-05-24 7:48 ` Sedat Dilek
1 sibling, 1 reply; 32+ messages in thread
From: Masahiro Yamada @ 2020-05-24 3:56 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Sedat Dilek, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Mauro Carvalho Chehab, Anshuman Khandual,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, clang-built-linux
On Fri, May 22, 2020 at 6:57 AM 'Nick Desaulniers' via Clang Built
Linux <clang-built-linux@googlegroups.com> wrote:
>
> On Wed, May 20, 2020 at 7:48 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > > Suggested-by: Fangrui Song <maskray@google.com>
> >
> >
> > Suggested-by -> Reviewed-by
> >
> > https://patchwork.kernel.org/patch/11524939/#23349551
>
> Yes, my mistake.
>
> > > Suggested-by: Nick Clifton <nickc@redhat.com>
> >
> >
> > I do not know where this tag came from.
> >
> > Nick Clifton taught us the version rule of binutils,but did not state
> > anything about this patch itself.
> >
> > https://patchwork.kernel.org/patch/11524939/#23355175
> >
> >
> > > Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
> >
> > I do not see the source of this tag, either...
>
> Not all contributions to open source need to be in the form of
> patches. Both Sedat and Nick gave suggestions which ultimately
> informed the contents of this patch. They should be rewarded for
> their efforts, and incentivized to help improve the code base further.
> I think suggested by tags are a good way to do that; but if it's
> against a written convention or if you still disagree, it's not the
> end of the world to me, and you may drop those tags from the v3.
Documentation/process/submitting-patches.rst
gives the guideline.
"A Suggested-by: tag indicates that the patch idea is suggested by the person
named and ensures credit to the person for the idea. Please note that this
tag should not be added without the reporter's permission, especially if the
idea was not posted in a public forum. That said, if we diligently credit our
idea reporters, they will, hopefully, be inspired to help us again in the
future."
I think this tag should be given to people who
gave the main idea to come up with
the main part of the patch.
Is that David Blaikie who suggested to
compress the debug info ?
If so, definitely he deserves to have Suggested-by tag.
For the others, I do not think Suggested-by is a good fit.
I appreciate their contribution to improve this patch.
So, maybe you can give credit in other form, for example,
mention it in the commit log explicitly?
Nick Clifton helped us to provide the minimal binutils version.
Sedat Dilet found an increase in size of debug .deb package.
Thanks.
>
> > > --- a/lib/Kconfig.debug
> > > +++ b/lib/Kconfig.debug
> > > @@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
> > > DEBUG_INFO build and compile times are reduced too.
> > > Only works with newer gcc versions.
> > >
> > > +config DEBUG_INFO_COMPRESSED
> > > + bool "Compressed debugging information"
> > > + depends on DEBUG_INFO
> > > + depends on $(cc-option,-gz=zlib)
> > > + depends on $(as-option,-Wa,--compress-debug-sections=zlib)
> >
> > This does not work. (always false)
>
> Technically, always true. `-Wa` disables all warnings from the
> assembler. Also, I did test this via `make menuconfig`.
>
> > You cannot enable this option.
> >
> > The comma between -Wa and --compress-debug-sections=zlib
> > is eaten by Kconfig parser because commas are delimiters
> > of function parameters.
> >
> >
> > Please write like this.
> >
> > depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)
>
> You're right, I knew this bug forgot. Will fix in v3.
>
> > > + depends on $(ld-option,--compress-debug-sections=zlib)
> > > + help
> > > + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> > > + 5.0+, binutils 2.26+, and zlib.
> > > +
> > > + Users of dpkg-deb via scripts/package/builddeb may
> > > + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
> > > + the debug info again with a different compression scheme, which can
> > > + result in larger binaries.
> >
> > No. This is not correct.
> >
> > CONFIG_DEBUG_INFO_COMPRESSED compresses the only debug info part.
> > The other parts still get by benefit from the default KDEB_COMPRESS=xz.
> >
> >
> > The numbers are here:
> >
> >
> > CONFIG_DEBUG_INFO_COMPRESSED=y
> > -rw-r--r-- 1 masahiro masahiro 209077584 May 21 11:19
> > linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-26_amd64.deb
> >
> >
> > CONFIG_DEBUG_INFO_COMPRESSED=y and KDEB_COMPRESS=none
> > -rw-r--r-- 1 masahiro masahiro 643051712 May 21 11:22
> > linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-27_amd64.deb
> >
> >
> > CONFIG_DEBUG_INFO_COMPRESSED=n
> > -rw-r--r-- 1 masahiro masahiro 112200308 May 21 11:40
> > linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-30_amd64.deb
> >
> >
> >
> >
> > For the deb package size perspective,
> > it is better to keep KDEB_COMPRESS as default.
> >
> > The main motivation for changing KDEB_COMPRESS
> > is the build speed. (see commit 1a7f0a34ea7d05)
> >
> >
> >
> >
> > CONFIG_DEBUG_INFO_COMPRESSED has a downside
> > for the debug deb package, but we need to accept it.
>
> Ah, I see. Thank you for those measurements. I'll send a v3 with
> fixed up help text, but ultimately, I don't really care what it says
> here. Please feel empowered to reword it should you choose to accept
> + apply it.
> --
> Thanks,
> ~Nick Desaulniers
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOd%3DjOr4ZaLx-dSNTqZnGRATY1PZktUfu4JGWKRwRH%3DUjnw%40mail.gmail.com.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2] Makefile: support compressed debug info
2020-05-24 3:56 ` [PATCH v2] " Masahiro Yamada
@ 2020-05-24 7:48 ` Sedat Dilek
2020-05-26 15:58 ` Nick Desaulniers
0 siblings, 1 reply; 32+ messages in thread
From: Sedat Dilek @ 2020-05-24 7:48 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Nick Desaulniers, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Changbin Du, Randy Dunlap,
Stephen Rothwell, Mauro Carvalho Chehab, Anshuman Khandual,
Krzysztof Kozlowski, Linux Kbuild mailing list,
Linux Kernel Mailing List, clang-built-linux
On Sun, May 24, 2020 at 5:57 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Fri, May 22, 2020 at 6:57 AM 'Nick Desaulniers' via Clang Built
> Linux <clang-built-linux@googlegroups.com> wrote:
> >
> > On Wed, May 20, 2020 at 7:48 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > > Suggested-by: Fangrui Song <maskray@google.com>
> > >
> > >
> > > Suggested-by -> Reviewed-by
> > >
> > > https://patchwork.kernel.org/patch/11524939/#23349551
> >
> > Yes, my mistake.
> >
> > > > Suggested-by: Nick Clifton <nickc@redhat.com>
> > >
> > >
> > > I do not know where this tag came from.
> > >
> > > Nick Clifton taught us the version rule of binutils,but did not state
> > > anything about this patch itself.
> > >
> > > https://patchwork.kernel.org/patch/11524939/#23355175
> > >
> > >
> > > > Suggested-by: Sedat Dilek <sedat.dilek@gmail.com>
> > >
> > > I do not see the source of this tag, either...
> >
> > Not all contributions to open source need to be in the form of
> > patches. Both Sedat and Nick gave suggestions which ultimately
> > informed the contents of this patch. They should be rewarded for
> > their efforts, and incentivized to help improve the code base further.
> > I think suggested by tags are a good way to do that; but if it's
> > against a written convention or if you still disagree, it's not the
> > end of the world to me, and you may drop those tags from the v3.
>
>
> Documentation/process/submitting-patches.rst
> gives the guideline.
>
>
> "A Suggested-by: tag indicates that the patch idea is suggested by the person
> named and ensures credit to the person for the idea. Please note that this
> tag should not be added without the reporter's permission, especially if the
> idea was not posted in a public forum. That said, if we diligently credit our
> idea reporters, they will, hopefully, be inspired to help us again in the
> future."
>
>
> I think this tag should be given to people who
> gave the main idea to come up with
> the main part of the patch.
>
>
> Is that David Blaikie who suggested to
> compress the debug info ?
> If so, definitely he deserves to have Suggested-by tag.
>
> For the others, I do not think Suggested-by is a good fit.
>
> I appreciate their contribution to improve this patch.
> So, maybe you can give credit in other form, for example,
> mention it in the commit log explicitly?
>
> Nick Clifton helped us to provide the minimal binutils version.
> Sedat Dilet found an increase in size of debug .deb package.
>
>
> Thanks.
Hi,
first my last name is Dilek - just for the sake of completeness.
No, it is not my first name as Dilek is a female Turkish first name,
so I do not want to change my gender.
So this discussions come up again and again.
Thus some own words on this - this is my personal opinion.
Like the author of Curl and DOH said at FOSDEM 2019 in Bruessel:
I am doing all this work - first - for myself - in my build and
developing environment.
Very very egoistically!
"Share knowledge aggressively!"
...was Nick's words at First ClangBuiltLinux Meetup in Zurich 2020.
In a 2nd round I share my knowledge and I like this - that's why I am
doing Open Source.
For me it sounds like a "Suggested-by" tag or other credits like
"Reviewed-by" have a higher value than a Tested-by tag.
*** The opposite is the case. ***
Here, I am on a Samsung SandyBridge CPU/GPU aka 2nd generation
ultrabook series runing Debian/testing AMD64.
A slightly modified Debian-kernel linux-config takes me approx. 5 (in
words five) hours of compiling and generating Debian packages.
Plus, testing.
Plus, testing.
Plus, testing.
In Linux-next times I run the whole Linux-Test-Project tests plus some
FIO tests.
Finally, I decide depending from what is new and interesting to me to
attend a full single Linux-kernel release cycle.
The last was Linux v5.3 which was the first release to be
compile/link-able - with no modifications - with LLVM/Clang/LLD v9.0.
For upcoming Linux v5.7 I have built each single RC Linux-kernel and
used it in my daily work!
Since RC1 - for me running on bare metal counts - checking QEMU or
other VM is nice - but showed me that says sometimes nothing.
Plus, I am building llvm-toolchains (LLVM/Clang/LLD) and testing with
them (and report if needed).
"...if we diligently credit our idea reporters, they will, hopefully,
be inspired to help us again in the future."
These are some motivating words...
My Tested-by is like a certificate - like a "Made in Germany" seal :-).
Virtual Greeting from North-West Germany,
- Sedat -
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2] Makefile: support compressed debug info
2020-05-24 7:48 ` Sedat Dilek
@ 2020-05-26 15:58 ` Nick Desaulniers
0 siblings, 0 replies; 32+ messages in thread
From: Nick Desaulniers @ 2020-05-26 15:58 UTC (permalink / raw)
To: Sedat Dilek
Cc: Masahiro Yamada, Fangrui Song, Nick Clifton, David Blaikie,
Michal Marek, Andrew Morton, Randy Dunlap, Stephen Rothwell,
Mauro Carvalho Chehab, Anshuman Khandual, Krzysztof Kozlowski,
Linux Kbuild mailing list, Linux Kernel Mailing List,
clang-built-linux
On Sun, May 24, 2020 at 12:48 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> *** The opposite is the case. ***
>
> Here, I am on a Samsung SandyBridge CPU/GPU aka 2nd generation
> ultrabook series runing Debian/testing AMD64.
>
> A slightly modified Debian-kernel linux-config takes me approx. 5 (in
> words five) hours of compiling and generating Debian packages.
>
> Plus, testing.
> Plus, testing.
> Plus, testing.
>
> In Linux-next times I run the whole Linux-Test-Project tests plus some
> FIO tests.
>
> Finally, I decide depending from what is new and interesting to me to
> attend a full single Linux-kernel release cycle.
> The last was Linux v5.3 which was the first release to be
> compile/link-able - with no modifications - with LLVM/Clang/LLD v9.0.
> For upcoming Linux v5.7 I have built each single RC Linux-kernel and
> used it in my daily work!
> Since RC1 - for me running on bare metal counts - checking QEMU or
> other VM is nice - but showed me that says sometimes nothing.
>
> Plus, I am building llvm-toolchains (LLVM/Clang/LLD) and testing with
> them (and report if needed).
This is a lot of invaluable work. It means the world to me Sedat!
>
> "...if we diligently credit our idea reporters, they will, hopefully,
> be inspired to help us again in the future."
>
> These are some motivating words...
>
> My Tested-by is like a certificate - like a "Made in Germany" seal :-).
I love this, it is. Maybe if folks on this thread are bored, they
could help me with a personal project of mine?
https://github.com/nickdesaulniers/What-Open-Source-Means-To-Me
We could use more German, and Japanese.
>
> Virtual Greeting from North-West Germany,
> - Sedat -
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH] Makefile: support compressed debug info
2020-05-12 19:23 ` Nick Desaulniers
2020-05-12 20:01 ` Fangrui Song
@ 2020-05-12 20:02 ` Sedat Dilek
2020-05-13 2:51 ` Masahiro Yamada
2 siblings, 0 replies; 32+ messages in thread
From: Sedat Dilek @ 2020-05-12 20:02 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Masahiro Yamada, Fangrui Song, Nick Desaulniers, Michal Marek,
Andrew Morton, Changbin Du, Randy Dunlap, Krzysztof Kozlowski,
Linux Kbuild mailing list, Linux Kernel Mailing List,
Clang-Built-Linux ML
On Tue, May 12, 2020 at 9:23 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
> > > ><nick.desaulniers@gmail.com> wrote:
> > > >>
> > > >> As debug information gets larger and larger, it helps significantly save
> > > >> the size of vmlinux images to compress the information in the debug
> > > >> information sections. Note: this debug info is typically split off from
> > > >> the final compressed kernel image, which is why vmlinux is what's used
> > > >> in conjunction with GDB. Minimizing the debug info size should have no
> > > >> impact on boot times, or final compressed kernel image size.
> > > >>
> > Nick,
> >
> > I am OK with this patch.
> >
> > Fangrui provided the minimal requirement for
> > --compress-debug-sections=zlib
> >
> >
> > Is it worth recording in the help text?
> > Do you want to send v2?
>
> Yes I'd like to record that information. I can also record Sedat's
> Tested-by tag. Thank you for testing Sedat.
>
> I don't know what "linux-image-dbg file" are, or why they would be
> bigger. The size of the debug info is the primary concern with this
> config. It sounds like however that file is created might be
> problematic.
>
Hi Nick,
sorry, I try to explain the magic of "linux-image-dbg file".
In my workflow, I use the "scripts/package/{builddeb,mkdebian}"
shipped with the Linux-kernel as a base to build my Debian packages.
With enabled debugging (CONFIG_DEBUG_INFO=y) a
"linux-image-$KERNELRELEASE-dbg" Debian package is created.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/package/mkdebian#n203
As you can see below...
[ CONFIG_DEBUG_INFO_COMPRESSED=n ]
47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
[ CONFIG_DEBUG_INFO_COMPRESSED=y ]
47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
...there is minimal change in the size for the Debian package w/o
debug-infos - approx. 47M.
As said 424M vs. 771M for the dbg packages.
There is another big benefit checking my recorded stats:
$ grep 'cache size' stats/5.7.0-rc4-*/ccache-s.txt
stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:cache size
4.7 GB
stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:max cache size
10.0 GB
stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:cache size
3.4 GB
stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:max cache size
10.0 GB
So the cache of ccache is reduced: 4.7 GB vs. 3.4 GB
If you have any questions, "Don't ask to ask - just ask." :-).
Thanks.
Regards,
- Sedat -
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH] Makefile: support compressed debug info
2020-05-12 19:23 ` Nick Desaulniers
2020-05-12 20:01 ` Fangrui Song
2020-05-12 20:02 ` [PATCH] " Sedat Dilek
@ 2020-05-13 2:51 ` Masahiro Yamada
2020-05-13 16:33 ` Sedat Dilek
2 siblings, 1 reply; 32+ messages in thread
From: Masahiro Yamada @ 2020-05-13 2:51 UTC (permalink / raw)
To: Nick Desaulniers
Cc: Fangrui Song, Sedat Dilek, Nick Desaulniers, Michal Marek,
Andrew Morton, Changbin Du, Randy Dunlap, Krzysztof Kozlowski,
Linux Kbuild mailing list, Linux Kernel Mailing List,
Clang-Built-Linux ML
Nick,
On Wed, May 13, 2020 at 4:23 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
> > > ><nick.desaulniers@gmail.com> wrote:
> > > >>
> > > >> As debug information gets larger and larger, it helps significantly save
> > > >> the size of vmlinux images to compress the information in the debug
> > > >> information sections. Note: this debug info is typically split off from
> > > >> the final compressed kernel image, which is why vmlinux is what's used
> > > >> in conjunction with GDB. Minimizing the debug info size should have no
> > > >> impact on boot times, or final compressed kernel image size.
> > > >>
> > Nick,
> >
> > I am OK with this patch.
> >
> > Fangrui provided the minimal requirement for
> > --compress-debug-sections=zlib
> >
> >
> > Is it worth recording in the help text?
> > Do you want to send v2?
>
> Yes I'd like to record that information. I can also record Sedat's
> Tested-by tag. Thank you for testing Sedat.
>
> I don't know what "linux-image-dbg file" are, or why they would be
> bigger. The size of the debug info is the primary concern with this
> config. It sounds like however that file is created might be
> problematic.
As Sedat explained, deb package data
is compressed by xz, which is default.
You can use another compression method,
or disable compression if you desire.
"man dpkg-deb" says as follows:
-Zcompress-type
Specify which compression type to use when building a package.
Allowed values are gzip, xz (since dpkg 1.15.6), and none
(default is xz).
Kbuild supports KDEB_COMPRESS variable
to change the compression method.
See line 46 of scripts/package/builddeb.
If you are interested,
try "make bindeb-pkg" with/without CONFIG_DEBUG_INFO_COMPRESSED,
and compare the size of the generated debug package.
As Sedat stated,
(plain data) -> compress by gzip -> compress by xz
is often less efficient than
(plain data) -> compress by xz
I hope this is clearer.
> Fangrui, I wasn't able to easily find what version of binutils first
> added support. Can you please teach me how to fish?
>
> Another question I had for Fangrui is, if the linker can compress
> these sections, shouldn't we just have the linker do it, not the the
> compiler and assembler? IIUC the debug info can contain relocations,
> so the linker would have to decompress these, perform relocations,
> then recompress these? I guess having the compiler and assembler
> compress the debug info as well would minimize the size of the .o
> files on disk.
>
> Otherwise I should add this flag to the assembler invocation, too, in
> v2. Thoughts?
>
> I have a patch series that enables dwarf5 support in the kernel that
> I'm working up to. I wanted to send this first. Both roughly reduce
> the debug info size by 20% each, though I haven't measured them
> together, yet. Requires ToT binutils because there have been many
> fixes from reports of mine recently.
> --
> Thanks,
> ~Nick Desaulniers
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH] Makefile: support compressed debug info
2020-05-13 2:51 ` Masahiro Yamada
@ 2020-05-13 16:33 ` Sedat Dilek
0 siblings, 0 replies; 32+ messages in thread
From: Sedat Dilek @ 2020-05-13 16:33 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Nick Desaulniers, Fangrui Song, Nick Desaulniers, Michal Marek,
Andrew Morton, Changbin Du, Randy Dunlap, Krzysztof Kozlowski,
Linux Kbuild mailing list, Linux Kernel Mailing List,
Clang-Built-Linux ML
On Wed, May 13, 2020 at 4:52 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Nick,
>
> On Wed, May 13, 2020 at 4:23 AM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
> > > > ><nick.desaulniers@gmail.com> wrote:
> > > > >>
> > > > >> As debug information gets larger and larger, it helps significantly save
> > > > >> the size of vmlinux images to compress the information in the debug
> > > > >> information sections. Note: this debug info is typically split off from
> > > > >> the final compressed kernel image, which is why vmlinux is what's used
> > > > >> in conjunction with GDB. Minimizing the debug info size should have no
> > > > >> impact on boot times, or final compressed kernel image size.
> > > > >>
> > > Nick,
> > >
> > > I am OK with this patch.
> > >
> > > Fangrui provided the minimal requirement for
> > > --compress-debug-sections=zlib
> > >
> > >
> > > Is it worth recording in the help text?
> > > Do you want to send v2?
> >
> > Yes I'd like to record that information. I can also record Sedat's
> > Tested-by tag. Thank you for testing Sedat.
> >
> > I don't know what "linux-image-dbg file" are, or why they would be
> > bigger. The size of the debug info is the primary concern with this
> > config. It sounds like however that file is created might be
> > problematic.
>
>
>
> As Sedat explained, deb package data
> is compressed by xz, which is default.
>
> You can use another compression method,
> or disable compression if you desire.
>
>
>
> "man dpkg-deb" says as follows:
>
> -Zcompress-type
> Specify which compression type to use when building a package.
> Allowed values are gzip, xz (since dpkg 1.15.6), and none
> (default is xz).
>
>
>
> Kbuild supports KDEB_COMPRESS variable
> to change the compression method.
> See line 46 of scripts/package/builddeb.
>
Hi Masahiro,
thanks for the clarification and the informations.
With next RC I will add KDEB_COMPRESS=gzip to my build-script and report.
Regards,
- Sedat -
>
> If you are interested,
> try "make bindeb-pkg" with/without CONFIG_DEBUG_INFO_COMPRESSED,
> and compare the size of the generated debug package.
>
>
>
>
> As Sedat stated,
>
> (plain data) -> compress by gzip -> compress by xz
>
> is often less efficient than
>
> (plain data) -> compress by xz
>
>
>
> I hope this is clearer.
>
>
>
>
>
>
>
>
> > Fangrui, I wasn't able to easily find what version of binutils first
> > added support. Can you please teach me how to fish?
> >
> > Another question I had for Fangrui is, if the linker can compress
> > these sections, shouldn't we just have the linker do it, not the the
> > compiler and assembler? IIUC the debug info can contain relocations,
> > so the linker would have to decompress these, perform relocations,
> > then recompress these? I guess having the compiler and assembler
> > compress the debug info as well would minimize the size of the .o
> > files on disk.
> >
> > Otherwise I should add this flag to the assembler invocation, too, in
> > v2. Thoughts?
> >
> > I have a patch series that enables dwarf5 support in the kernel that
> > I'm working up to. I wanted to send this first. Both roughly reduce
> > the debug info size by 20% each, though I haven't measured them
> > together, yet. Requires ToT binutils because there have been many
> > fixes from reports of mine recently.
> > --
> > Thanks,
> > ~Nick Desaulniers
>
>
>
> --
> Best Regards
> Masahiro Yamada
^ permalink raw reply [flat|nested] 32+ messages in thread