[PATCH v7 0/2] Kbuild: DWARF v5 support

[PATCH v7 0/2] Kbuild: DWARF v5 support

[Nick Desaulniers]

DWARF v5 is the latest standard of the DWARF debug info format.

DWARF5 wins significantly in terms of size and especially so when mixed
with compression (CONFIG_DEBUG_INFO_COMPRESSED).

Link: http://www.dwarfstd.org/doc/DWARF5.pdf

Patch 1 is a cleanup that lays the ground work and isn't DWARF
v5 specific.
Patch 2 implements Kconfig and Kbuild support for DWARFv5.

Changes from v6:
* Reorder sections from linker script to match order from BFD's internal
  linker script.
* Add .debug_names section, as per Fangrui.
* Drop CONFIG_DEBUG_INFO_DWARF2. Patch 0001 becomes a menu with 1
  choice. GCC's implicit default version of DWARF has been DWARF v4
  since ~4.8.
* Modify the test script to check for the presence of
  https://sourceware.org/bugzilla/show_bug.cgi?id=27195.
* Drop the clang without integrated assembler block in
  0002. Bumps the version requirement for GAS to 2.35.2, which isn't
  released yet (but should be released soon).  Folks looking to test
  with clang but without the integrated assembler should fetch
  binutils-gdb, build it from source, add a symlink to
  binutils-gdb/gas/as-new to binutils-gdb/gas/as, then prefix
  binutils-gdb/gas/as to their $PATH when building the kernel.

Changes from v5:
* Drop previous patch 1, it has been accepted into kbuild:
  https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=kbuild&id=3f4d8ce271c7082be75bacbcbd2048aa78ce2b44
* Trying to set -Wa,-gdwarf-4 in the earlier patch was the source of
  additional complexity. Drop it that part of the patch. We can revisit
  clang without the integrated assembler setting -Wa,-gdwarf-4 later.
  That is a separate problem from generally supporting DWARF v5.
* Rework the final patch for clang without the integrated assembler.
  -Wa,-gdwarf-5 is required for DWARF5 in that case otherwise GAS will
  not accept the assembler directives clang produces from C code when
  generating asm.

Changes from v4:
* drop set -e from script as per Nathan.
* add dependency on !CONFIG_DEBUG_INFO_BTF for DWARF v5 as per Sedat.
* Move LLVM_IAS=1 complexity from patch 2 to patch 3 as per Arvind and
  Masahiro. Sorry it took me a few tries to understand the point (I
  might still not), but it looks much cleaner this way. Sorry Nathan, I
  did not carry forward your previous reviews as a result, but I would
  appreciate if you could look again.
* Add Nathan's reviewed by tag to patch 1.
* Reword commit message for patch 3 to mention LLVM_IAS=1 and -gdwarf-5
  binutils addition later, and BTF issue.
* I still happen to see a pahole related error spew for the combination
  of:
  * LLVM=1
  * LLVM_IAS=1
  * CONFIG_DEBUG_INFO_DWARF4
  * CONFIG_DEBUG_INFO_BTF
  Though they're non-fatal to the build. I'm not sure yet why removing
  any one of the above prevents the warning spew. Maybe we'll need a v6.

Changes from v3:

Changes as per Arvind:
* only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
* add -gdwarf-5 to Kconfig shell script.
* only run Kconfig shell script for Clang.

Apologies to Sedat and Nathan; I appreciate previous testing/review, but
I did no carry forward your Tested-by and Reviewed-by tags, as the
patches have changed too much IMO.

Changes from v2:
* Drop two of the earlier patches that have been accepted already.
* Add measurements with GCC 10.2 to commit message.
* Update help text as per Arvind with help from Caroline.
* Improve case/wording between DWARF Versions as per Masahiro.

Changes from the RFC:
* split patch in 3 patch series, include Fangrui's patch, too.
* prefer `DWARF vX` format, as per Fangrui.
* use spaces between assignment in Makefile as per Masahiro.
* simplify setting dwarf-version-y as per Masahiro.
* indent `prompt` in Kconfig change as per Masahiro.
* remove explicit default in Kconfig as per Masahiro.
* add comments to test_dwarf5_support.sh.
* change echo in test_dwarf5_support.sh as per Masahiro.
* remove -u from test_dwarf5_support.sh as per Masahiro.
* add a -gdwarf-5 cc-option check to Kconfig as per Jakub.

Nick Desaulniers (2):
  Kbuild: make DWARF version a choice
  Kbuild: implement support for DWARF v5

 Makefile                          |  6 +++---
 include/asm-generic/vmlinux.lds.h |  7 +++++-
 lib/Kconfig.debug                 | 36 +++++++++++++++++++++++++------
 scripts/test_dwarf5_support.sh    |  8 +++++++
 4 files changed, 47 insertions(+), 10 deletions(-)
 create mode 100755 scripts/test_dwarf5_support.sh

-- 
2.30.0.365.g02bc693789-goog

[PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
the default. Does so in a way that's forward compatible with existing
configs, and makes adding future versions more straightforward.

GCC since ~4.8 has defaulted to this DWARF version implicitly.

Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
Suggested-by: Fangrui Song <maskray@google.com>
Suggested-by: Nathan Chancellor <nathan@kernel.org>
Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
 Makefile          |  5 ++---
 lib/Kconfig.debug | 16 +++++++++++-----
 2 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/Makefile b/Makefile
index 95ab9856f357..d2b4980807e0 100644
--- a/Makefile
+++ b/Makefile
@@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
 KBUILD_AFLAGS	+= -Wa,-gdwarf-2
 endif
 
-ifdef CONFIG_DEBUG_INFO_DWARF4
-DEBUG_CFLAGS	+= -gdwarf-4
-endif
+dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
+DEBUG_CFLAGS	+= -gdwarf-$(dwarf-version-y)
 
 ifdef CONFIG_DEBUG_INFO_REDUCED
 DEBUG_CFLAGS	+= $(call cc-option, -femit-struct-debug-baseonly) \
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index e906ea906cb7..94c1a7ed6306 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -256,13 +256,19 @@ config DEBUG_INFO_SPLIT
 	  to know about the .dwo files and include them.
 	  Incompatible with older versions of ccache.
 
+choice
+	prompt "DWARF version"
+	help
+	  Which version of DWARF debug info to emit.
+
 config DEBUG_INFO_DWARF4
-	bool "Generate dwarf4 debuginfo"
+	bool "Generate DWARF Version 4 debuginfo"
 	help
-	  Generate dwarf4 debug info. This requires recent versions
-	  of gcc and gdb. It makes the debug information larger.
-	  But it significantly improves the success of resolving
-	  variables in gdb on optimized code.
+	  Generate DWARF v4 debug info. This requires gcc 4.5+ and gdb 7.0+.
+	  It makes the debug information larger, but it significantly
+	  improves the success of resolving variables in gdb on optimized code.
+
+endchoice # "DWARF version"
 
 config DEBUG_INFO_BTF
 	bool "Generate BTF typeinfo"
-- 
2.30.0.365.g02bc693789-goog

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nathan Chancellor]

On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> the default. Does so in a way that's forward compatible with existing
> configs, and makes adding future versions more straightforward.
> 
> GCC since ~4.8 has defaulted to this DWARF version implicitly.
> 
> Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> Suggested-by: Fangrui Song <maskray@google.com>
> Suggested-by: Nathan Chancellor <nathan@kernel.org>
> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>

One comment below:

Reviewed-by: Nathan Chancellor <nathan@kernel.org>

> ---
>  Makefile          |  5 ++---
>  lib/Kconfig.debug | 16 +++++++++++-----
>  2 files changed, 13 insertions(+), 8 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 95ab9856f357..d2b4980807e0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
>  KBUILD_AFLAGS	+= -Wa,-gdwarf-2

It is probably worth a comment somewhere that assembly files will still
have DWARF v2.

>  endif
>  
> -ifdef CONFIG_DEBUG_INFO_DWARF4
> -DEBUG_CFLAGS	+= -gdwarf-4
> -endif
> +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> +DEBUG_CFLAGS	+= -gdwarf-$(dwarf-version-y)
>  
>  ifdef CONFIG_DEBUG_INFO_REDUCED
>  DEBUG_CFLAGS	+= $(call cc-option, -femit-struct-debug-baseonly) \
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index e906ea906cb7..94c1a7ed6306 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -256,13 +256,19 @@ config DEBUG_INFO_SPLIT
>  	  to know about the .dwo files and include them.
>  	  Incompatible with older versions of ccache.
>  
> +choice
> +	prompt "DWARF version"
> +	help
> +	  Which version of DWARF debug info to emit.
> +
>  config DEBUG_INFO_DWARF4
> -	bool "Generate dwarf4 debuginfo"
> +	bool "Generate DWARF Version 4 debuginfo"
>  	help
> -	  Generate dwarf4 debug info. This requires recent versions
> -	  of gcc and gdb. It makes the debug information larger.
> -	  But it significantly improves the success of resolving
> -	  variables in gdb on optimized code.
> +	  Generate DWARF v4 debug info. This requires gcc 4.5+ and gdb 7.0+.
> +	  It makes the debug information larger, but it significantly
> +	  improves the success of resolving variables in gdb on optimized code.
> +
> +endchoice # "DWARF version"
>  
>  config DEBUG_INFO_BTF
>  	bool "Generate BTF typeinfo"
> -- 
> 2.30.0.365.g02bc693789-goog
> 

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Masahiro Yamada]

On Sat, Jan 30, 2021 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > the default. Does so in a way that's forward compatible with existing
> > configs, and makes adding future versions more straightforward.
> >
> > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> >
> > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > Suggested-by: Fangrui Song <maskray@google.com>
> > Suggested-by: Nathan Chancellor <nathan@kernel.org>
> > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
>
> One comment below:
>
> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
>
> > ---
> >  Makefile          |  5 ++---
> >  lib/Kconfig.debug | 16 +++++++++++-----
> >  2 files changed, 13 insertions(+), 8 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index 95ab9856f357..d2b4980807e0 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
> >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
>
> It is probably worth a comment somewhere that assembly files will still
> have DWARF v2.

I agree.
Please noting the reason will be helpful.

Could you summarize Jakub's comment in short?
https://patchwork.kernel.org/project/linux-kbuild/patch/20201022012106.1875129-1-ndesaulniers@google.com/#23727667






One more question.


Can we remove -g option like follows?


 ifdef CONFIG_DEBUG_INFO_SPLIT
 DEBUG_CFLAGS   += -gsplit-dwarf
-else
-DEBUG_CFLAGS   += -g
 endif





In the current mainline code,
-g is the only debug option
if CONFIG_DEBUG_INFO_DWARF4 is disabled.


The GCC manual says:
https://gcc.gnu.org/onlinedocs/gcc-10.2.0/gcc/Debugging-Options.html#Debugging-Options


-g

    Produce debugging information in the operating system’s
    native format (stabs, COFF, XCOFF, or DWARF).
    GDB can work with this debugging information.


Of course, we expect the -g option will produce
the debug info in the DWARF format.





With this patch set applied, it is very explicit.

Only the format type, but also the version.

The compiler will be given either
-gdwarf-4 or -gdwarf-5,
making the -g option redundant, I think.









>
> >  endif
> >
> > -ifdef CONFIG_DEBUG_INFO_DWARF4
> > -DEBUG_CFLAGS += -gdwarf-4
> > -endif
> > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> > +DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)
> >
> >  ifdef CONFIG_DEBUG_INFO_REDUCED
> >  DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index e906ea906cb7..94c1a7ed6306 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -256,13 +256,19 @@ config DEBUG_INFO_SPLIT
> >         to know about the .dwo files and include them.
> >         Incompatible with older versions of ccache.
> >
> > +choice
> > +     prompt "DWARF version"
> > +     help
> > +       Which version of DWARF debug info to emit.
> > +
> >  config DEBUG_INFO_DWARF4
> > -     bool "Generate dwarf4 debuginfo"
> > +     bool "Generate DWARF Version 4 debuginfo"
> >       help
> > -       Generate dwarf4 debug info. This requires recent versions
> > -       of gcc and gdb. It makes the debug information larger.
> > -       But it significantly improves the success of resolving
> > -       variables in gdb on optimized code.
> > +       Generate DWARF v4 debug info. This requires gcc 4.5+ and gdb 7.0+.
> > +       It makes the debug information larger, but it significantly
> > +       improves the success of resolving variables in gdb on optimized code.
> > +
> > +endchoice # "DWARF version"
> >
> >  config DEBUG_INFO_BTF
> >       bool "Generate BTF typeinfo"
> > --
> > 2.30.0.365.g02bc693789-goog
> >



-- 
Best Regards
Masahiro Yamada

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Masahiro Yamada]

On Thu, Feb 4, 2021 at 7:23 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sat, Jan 30, 2021 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > > the default. Does so in a way that's forward compatible with existing
> > > configs, and makes adding future versions more straightforward.
> > >
> > > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> > >
> > > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > > Suggested-by: Fangrui Song <maskray@google.com>
> > > Suggested-by: Nathan Chancellor <nathan@kernel.org>
> > > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> >
> > One comment below:
> >
> > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> >
> > > ---
> > >  Makefile          |  5 ++---
> > >  lib/Kconfig.debug | 16 +++++++++++-----
> > >  2 files changed, 13 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/Makefile b/Makefile
> > > index 95ab9856f357..d2b4980807e0 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
> > >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
> >
> > It is probably worth a comment somewhere that assembly files will still
> > have DWARF v2.
>
> I agree.
> Please noting the reason will be helpful.
>
> Could you summarize Jakub's comment in short?
> https://patchwork.kernel.org/project/linux-kbuild/patch/20201022012106.1875129-1-ndesaulniers@google.com/#23727667
>
>
>
>
>
>
> One more question.
>
>
> Can we remove -g option like follows?
>
>
>  ifdef CONFIG_DEBUG_INFO_SPLIT
>  DEBUG_CFLAGS   += -gsplit-dwarf
> -else
> -DEBUG_CFLAGS   += -g
>  endif
>
>
>
>
>
> In the current mainline code,
> -g is the only debug option
> if CONFIG_DEBUG_INFO_DWARF4 is disabled.
>
>
> The GCC manual says:
> https://gcc.gnu.org/onlinedocs/gcc-10.2.0/gcc/Debugging-Options.html#Debugging-Options
>
>
> -g
>
>     Produce debugging information in the operating system’s
>     native format (stabs, COFF, XCOFF, or DWARF).
>     GDB can work with this debugging information.
>
>
> Of course, we expect the -g option will produce
> the debug info in the DWARF format.
>
>
>
>
>
> With this patch set applied, it is very explicit.
>
> Only the format type, but also the version.


I mean:

Not only the format type, but also the version
is explicit.




> The compiler will be given either
> -gdwarf-4 or -gdwarf-5,
> making the -g option redundant, I think.
>
>
>
>
>
>
-- 
Best Regards
Masahiro Yamada

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Wed, Feb 3, 2021 at 2:24 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sat, Jan 30, 2021 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > > the default. Does so in a way that's forward compatible with existing
> > > configs, and makes adding future versions more straightforward.
> > >
> > > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> > >
> > > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > > Suggested-by: Fangrui Song <maskray@google.com>
> > > Suggested-by: Nathan Chancellor <nathan@kernel.org>
> > > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> >
> > One comment below:
> >
> > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> >
> > > ---
> > >  Makefile          |  5 ++---
> > >  lib/Kconfig.debug | 16 +++++++++++-----
> > >  2 files changed, 13 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/Makefile b/Makefile
> > > index 95ab9856f357..d2b4980807e0 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
> > >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
> >
> > It is probably worth a comment somewhere that assembly files will still
> > have DWARF v2.
>
> I agree.
> Please noting the reason will be helpful.

Via a comment in the source, or in the commit message?

>
> Could you summarize Jakub's comment in short?
> https://patchwork.kernel.org/project/linux-kbuild/patch/20201022012106.1875129-1-ndesaulniers@google.com/#23727667

Via a comment in the source, or in the commit message?

> One more question.
>
>
> Can we remove -g option like follows?
>
>
>  ifdef CONFIG_DEBUG_INFO_SPLIT
>  DEBUG_CFLAGS   += -gsplit-dwarf
> -else
> -DEBUG_CFLAGS   += -g
>  endif
>
>
>
>
>
> In the current mainline code,
> -g is the only debug option
> if CONFIG_DEBUG_INFO_DWARF4 is disabled.
>
>
> The GCC manual says:
> https://gcc.gnu.org/onlinedocs/gcc-10.2.0/gcc/Debugging-Options.html#Debugging-Options
>
>
> -g
>
>     Produce debugging information in the operating system’s
>     native format (stabs, COFF, XCOFF, or DWARF).
>     GDB can work with this debugging information.
>
>
> Of course, we expect the -g option will produce
> the debug info in the DWARF format.
>
>
>
>
>
> With this patch set applied, it is very explicit.
>
> Only the format type, but also the version.
>
> The compiler will be given either
> -gdwarf-4 or -gdwarf-5,
> making the -g option redundant, I think.

Can I provide that as a separate patch?  I don't want any breakage
from that to delay DWARF v5 support further.  If so, should it be the
first patch or last?
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Masahiro Yamada]

On Thu, Feb 4, 2021 at 8:16 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Wed, Feb 3, 2021 at 2:24 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Sat, Jan 30, 2021 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > > > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > > > the default. Does so in a way that's forward compatible with existing
> > > > configs, and makes adding future versions more straightforward.
> > > >
> > > > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> > > >
> > > > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > > > Suggested-by: Fangrui Song <maskray@google.com>
> > > > Suggested-by: Nathan Chancellor <nathan@kernel.org>
> > > > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > >
> > > One comment below:
> > >
> > > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> > >
> > > > ---
> > > >  Makefile          |  5 ++---
> > > >  lib/Kconfig.debug | 16 +++++++++++-----
> > > >  2 files changed, 13 insertions(+), 8 deletions(-)
> > > >
> > > > diff --git a/Makefile b/Makefile
> > > > index 95ab9856f357..d2b4980807e0 100644
> > > > --- a/Makefile
> > > > +++ b/Makefile
> > > > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
> > > >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
> > >
> > > It is probably worth a comment somewhere that assembly files will still
> > > have DWARF v2.
> >
> > I agree.
> > Please noting the reason will be helpful.
>
> Via a comment in the source, or in the commit message?
>
> >
> > Could you summarize Jakub's comment in short?
> > https://patchwork.kernel.org/project/linux-kbuild/patch/20201022012106.1875129-1-ndesaulniers@google.com/#23727667
>
> Via a comment in the source, or in the commit message?


Both in the source if you can summarize it in three lines or so.


If you need to add more detailed explanation,
please provide it in the commit log.




> >
> > With this patch set applied, it is very explicit.
> >
> > Only the format type, but also the version.
> >
> > The compiler will be given either
> > -gdwarf-4 or -gdwarf-5,
> > making the -g option redundant, I think.
>
> Can I provide that as a separate patch?  I don't want any breakage
> from that to delay DWARF v5 support further.  If so, should it be the
> first patch or last?


OK.

At the last to be safe?

I am fine with doing a clean-up work later.






> --
> Thanks,
> ~Nick Desaulniers



--
Best Regards
Masahiro Yamada

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Wed, Feb 3, 2021 at 4:30 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Thu, Feb 4, 2021 at 8:16 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
> >
> > On Wed, Feb 3, 2021 at 2:24 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > On Sat, Jan 30, 2021 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
> > > >
> > > > On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > > > > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > > > > the default. Does so in a way that's forward compatible with existing
> > > > > configs, and makes adding future versions more straightforward.
> > > > >
> > > > > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> > > > >
> > > > > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > > > > Suggested-by: Fangrui Song <maskray@google.com>
> > > > > Suggested-by: Nathan Chancellor <nathan@kernel.org>
> > > > > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > > > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > > >
> > > > One comment below:
> > > >
> > > > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> > > >
> > > > > ---
> > > > >  Makefile          |  5 ++---
> > > > >  lib/Kconfig.debug | 16 +++++++++++-----
> > > > >  2 files changed, 13 insertions(+), 8 deletions(-)
> > > > >
> > > > > diff --git a/Makefile b/Makefile
> > > > > index 95ab9856f357..d2b4980807e0 100644
> > > > > --- a/Makefile
> > > > > +++ b/Makefile
> > > > > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
> > > > >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
> > > >
> > > > It is probably worth a comment somewhere that assembly files will still
> > > > have DWARF v2.
> > >
> > > I agree.
> > > Please noting the reason will be helpful.
> >
> > Via a comment in the source, or in the commit message?
> >
> > >
> > > Could you summarize Jakub's comment in short?
> > > https://patchwork.kernel.org/project/linux-kbuild/patch/20201022012106.1875129-1-ndesaulniers@google.com/#23727667
> >
> > Via a comment in the source, or in the commit message?
>
>
> Both in the source if you can summarize it in three lines or so.
>
>
> If you need to add more detailed explanation,
> please provide it in the commit log.

This information is mostly in the commit log of patch 2.  I will
reword it with additional info from the link you sent, but I find that
such minutiae is resulting in diminishing returns on subsequent
respins of the patch series.  I implore you to reword v8 (which I will
send) as you see fit when applying the series.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Fangrui Song]

On 2021-02-04, Masahiro Yamada wrote:
>On Sat, Jan 30, 2021 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
>>
>> On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
>> > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
>> > the default. Does so in a way that's forward compatible with existing
>> > configs, and makes adding future versions more straightforward.
>> >
>> > GCC since ~4.8 has defaulted to this DWARF version implicitly.
>> >
>> > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
>> > Suggested-by: Fangrui Song <maskray@google.com>
>> > Suggested-by: Nathan Chancellor <nathan@kernel.org>
>> > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
>> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
>>
>> One comment below:
>>
>> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
>>
>> > ---
>> >  Makefile          |  5 ++---
>> >  lib/Kconfig.debug | 16 +++++++++++-----
>> >  2 files changed, 13 insertions(+), 8 deletions(-)
>> >
>> > diff --git a/Makefile b/Makefile
>> > index 95ab9856f357..d2b4980807e0 100644
>> > --- a/Makefile
>> > +++ b/Makefile
>> > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
>> >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
>>
>> It is probably worth a comment somewhere that assembly files will still
>> have DWARF v2.
>
>I agree.
>Please noting the reason will be helpful.
>
>Could you summarize Jakub's comment in short?
>https://patchwork.kernel.org/project/linux-kbuild/patch/20201022012106.1875129-1-ndesaulniers@google.com/#23727667
>
>
>
>
>
>
>One more question.
>
>
>Can we remove -g option like follows?
>
>
> ifdef CONFIG_DEBUG_INFO_SPLIT
> DEBUG_CFLAGS   += -gsplit-dwarf
>-else
>-DEBUG_CFLAGS   += -g
> endif

GCC 11/Clang 12 -gsplit-dwarf no longer imply -g2
(https://reviews.llvm.org/D80391). May be worth checking whether
-gsplit-dwarf is used without a debug info enabling option.

>
>
>
>
>In the current mainline code,
>-g is the only debug option
>if CONFIG_DEBUG_INFO_DWARF4 is disabled.
>
>
>The GCC manual says:
>https://gcc.gnu.org/onlinedocs/gcc-10.2.0/gcc/Debugging-Options.html#Debugging-Options
>
>
>-g
>
>    Produce debugging information in the operating system’s
>    native format (stabs, COFF, XCOFF, or DWARF).
>    GDB can work with this debugging information.
>
>
>Of course, we expect the -g option will produce
>the debug info in the DWARF format.
>
>
>
>
>
>With this patch set applied, it is very explicit.
>
>Only the format type, but also the version.
>
>The compiler will be given either
>-gdwarf-4 or -gdwarf-5,
>making the -g option redundant, I think.

-gdwarf-N does imply -g2 but personally I'd not suggest remove it if it
already exists. The non-orthogonality is the reason Clang has
-fdebug-default-version (https://reviews.llvm.org/D69822).

>
>
>
>
>
>
>
>
>>
>> >  endif
>> >
>> > -ifdef CONFIG_DEBUG_INFO_DWARF4
>> > -DEBUG_CFLAGS += -gdwarf-4
>> > -endif
>> > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
>> > +DEBUG_CFLAGS += -gdwarf-$(dwarf-version-y)
>> >
>> >  ifdef CONFIG_DEBUG_INFO_REDUCED
>> >  DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
>> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
>> > index e906ea906cb7..94c1a7ed6306 100644
>> > --- a/lib/Kconfig.debug
>> > +++ b/lib/Kconfig.debug
>> > @@ -256,13 +256,19 @@ config DEBUG_INFO_SPLIT
>> >         to know about the .dwo files and include them.
>> >         Incompatible with older versions of ccache.
>> >
>> > +choice
>> > +     prompt "DWARF version"
>> > +     help
>> > +       Which version of DWARF debug info to emit.
>> > +
>> >  config DEBUG_INFO_DWARF4
>> > -     bool "Generate dwarf4 debuginfo"
>> > +     bool "Generate DWARF Version 4 debuginfo"
>> >       help
>> > -       Generate dwarf4 debug info. This requires recent versions
>> > -       of gcc and gdb. It makes the debug information larger.
>> > -       But it significantly improves the success of resolving
>> > -       variables in gdb on optimized code.
>> > +       Generate DWARF v4 debug info. This requires gcc 4.5+ and gdb 7.0+.
>> > +       It makes the debug information larger, but it significantly
>> > +       improves the success of resolving variables in gdb on optimized code.
>> > +
>> > +endchoice # "DWARF version"
>> >
>> >  config DEBUG_INFO_BTF
>> >       bool "Generate BTF typeinfo"
>> > --
>> > 2.30.0.365.g02bc693789-goog
>> >
>
>
>
>-- 
>Best Regards
>Masahiro Yamada

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Wed, Feb 3, 2021 at 7:32 PM Fangrui Song <maskray@google.com> wrote:
>
> On 2021-02-04, Masahiro Yamada wrote:
> >On Sat, Jan 30, 2021 at 10:52 AM Nathan Chancellor <nathan@kernel.org> wrote:
> >>
> >> On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> >> > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> >> > the default. Does so in a way that's forward compatible with existing
> >> > configs, and makes adding future versions more straightforward.
> >> >
> >> > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> >> >
> >> > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> >> > Suggested-by: Fangrui Song <maskray@google.com>
> >> > Suggested-by: Nathan Chancellor <nathan@kernel.org>
> >> > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> >> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> >>
> >> One comment below:
> >>
> >> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> >>
> >> > ---
> >> >  Makefile          |  5 ++---
> >> >  lib/Kconfig.debug | 16 +++++++++++-----
> >> >  2 files changed, 13 insertions(+), 8 deletions(-)
> >> >
> >> > diff --git a/Makefile b/Makefile
> >> > index 95ab9856f357..d2b4980807e0 100644
> >> > --- a/Makefile
> >> > +++ b/Makefile
> >> > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
> >> >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
> >>
> >> It is probably worth a comment somewhere that assembly files will still
> >> have DWARF v2.
> >
> >I agree.
> >Please noting the reason will be helpful.
> >
> >Could you summarize Jakub's comment in short?
> >https://patchwork.kernel.org/project/linux-kbuild/patch/20201022012106.1875129-1-ndesaulniers@google.com/#23727667
> >
> >
> >
> >
> >
> >
> >One more question.
> >
> >
> >Can we remove -g option like follows?
> >
> >
> > ifdef CONFIG_DEBUG_INFO_SPLIT
> > DEBUG_CFLAGS   += -gsplit-dwarf
> >-else
> >-DEBUG_CFLAGS   += -g
> > endif
>
> GCC 11/Clang 12 -gsplit-dwarf no longer imply -g2
> (https://reviews.llvm.org/D80391). May be worth checking whether
> -gsplit-dwarf is used without a debug info enabling option.

Indeed, I also remember -g was required for
-fno-eliminate-unused-debug-types, used by libabigail.
https://reviews.llvm.org/D80242
Masahiro, respectfully, I will not touch removing -g in this series.
I suspect it will be its own can of worms.

>
> >
> >
> >
> >
> >In the current mainline code,
> >-g is the only debug option
> >if CONFIG_DEBUG_INFO_DWARF4 is disabled.
> >
> >
> >The GCC manual says:
> >https://gcc.gnu.org/onlinedocs/gcc-10.2.0/gcc/Debugging-Options.html#Debugging-Options
> >
> >
> >-g
> >
> >    Produce debugging information in the operating system’s
> >    native format (stabs, COFF, XCOFF, or DWARF).
> >    GDB can work with this debugging information.
> >
> >
> >Of course, we expect the -g option will produce
> >the debug info in the DWARF format.
> >
> >
> >
> >
> >
> >With this patch set applied, it is very explicit.
> >
> >Only the format type, but also the version.
> >
> >The compiler will be given either
> >-gdwarf-4 or -gdwarf-5,
> >making the -g option redundant, I think.
>
> -gdwarf-N does imply -g2 but personally I'd not suggest remove it if it
> already exists. The non-orthogonality is the reason Clang has
> -fdebug-default-version (https://reviews.llvm.org/D69822).

-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Fri, Jan 29, 2021 at 5:52 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > the default. Does so in a way that's forward compatible with existing
> > configs, and makes adding future versions more straightforward.
> >
> > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> >
> > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > Suggested-by: Fangrui Song <maskray@google.com>
> > Suggested-by: Nathan Chancellor <nathan@kernel.org>
> > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
>
> One comment below:
>
> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
>
> > ---
> >  Makefile          |  5 ++---
> >  lib/Kconfig.debug | 16 +++++++++++-----
> >  2 files changed, 13 insertions(+), 8 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index 95ab9856f357..d2b4980807e0 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -830,9 +830,8 @@ ifneq ($(LLVM_IAS),1)
> >  KBUILD_AFLAGS        += -Wa,-gdwarf-2
>
> It is probably worth a comment somewhere that assembly files will still
> have DWARF v2.

Such a comment should have gone on
commit b8a9092330da ("Kbuild: do not emit debug info for assembly with
LLVM_IAS=1")

It's also more complicated than "assembly files will still have DWARF
v4," due to the LLVM_IAS check.

Further, such a comment would be inappropriate for patch 1 of the
series, which simply changes a Kconfig option into a choice.  I will
add a note to the commit message of patch 2, and carry your reviewed
by tag.  Please nack v8 (which I plan to send imminently) if you
disagree.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Mark Wielaard]

Hi Nick,

On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> the default. Does so in a way that's forward compatible with existing
> configs, and makes adding future versions more straightforward.
> 
> GCC since ~4.8 has defaulted to this DWARF version implicitly.

And since GCC 11 it defaults to DWARF version 5.

It would be better to set the default to the DWARF version that the
compiler generates. So if the user doesn't select any version then it
should default to just -g (or -gdwarf).

Thanks,

Mark

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Thu, Feb 4, 2021 at 2:41 AM Mark Wielaard <mark@klomp.org> wrote:
>
> Hi Nick,
>
> On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > the default. Does so in a way that's forward compatible with existing
> > configs, and makes adding future versions more straightforward.
> >
> > GCC since ~4.8 has defaulted to this DWARF version implicitly.
>
> And since GCC 11 it defaults to DWARF version 5.
>
> It would be better to set the default to the DWARF version that the
> compiler generates. So if the user doesn't select any version then it
> should default to just -g (or -gdwarf).

I disagree.

https://lore.kernel.org/lkml/CAKwvOdk0zxewEOaFuqK0aSMz3vKNzDOgmez=-Dae4+bodsSg5w@mail.gmail.com/
"""
I agree that this patch takes away the compiler vendor's choice as to
what the implicit default choice is for dwarf version for the kernel.
(We, the Linux kernel, do so already for implicit default -std=gnuc*
as well). ...
But I'm
going to suggest we follow the Zen of Python: explicit is better than
implicit.
"""
We have a number of in tree and out of tree DWARF consumers that
aren't ready for DWARF v5.  Kernel developers need a way to disable
DWARF v5 until their dependencies are deployed or more widely
available.

I'm happy to consider eventually moving the default DWARF version for
the kernel to v5, and ideas for how to wean developers off of v4, but
I don't think forcing v5 on kernel developers right now is the most
delicate approach.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Mark Wielaard]

On Thu, 2021-02-04 at 11:18 -0800, Nick Desaulniers wrote:
> On Thu, Feb 4, 2021 at 2:41 AM Mark Wielaard <mark@klomp.org> wrote:
> > On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > > the default. Does so in a way that's forward compatible with existing
> > > configs, and makes adding future versions more straightforward.
> > > 
> > > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> > 
> > And since GCC 11 it defaults to DWARF version 5.
> > 
> > It would be better to set the default to the DWARF version that the
> > compiler generates. So if the user doesn't select any version then it
> > should default to just -g (or -gdwarf).
> 
> I disagree.
> 
> https://lore.kernel.org/lkml/CAKwvOdk0zxewEOaFuqK0aSMz3vKNzDOgmez=-Dae4+bodsSg5w@mail.gmail.com/
> """
> I agree that this patch takes away the compiler vendor's choice as to
> what the implicit default choice is for dwarf version for the kernel.
> (We, the Linux kernel, do so already for implicit default -std=gnuc*
> as well). ...
> But I'm
> going to suggest we follow the Zen of Python: explicit is better than
> implicit.
> """
> We have a number of in tree and out of tree DWARF consumers that
> aren't ready for DWARF v5.  Kernel developers need a way to disable
> DWARF v5 until their dependencies are deployed or more widely
> available.

I agree with Jakub. Now that GCC has defaulted to DWARF5 all the tools
have adopted to the new default version. And DWARF5 has been out for
more than 4 years already. It isn't unreasonable to assume that people
using GCC11 will also be using the rest of the toolchain that has moved
on. Which DWARF consumers are you concerned about not being ready for
GCC defaulting to DWARF5 once GCC11 is released?

Thanks,

Mark

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Thu, Feb 4, 2021 at 11:56 AM Mark Wielaard <mark@klomp.org> wrote:
>
> On Thu, 2021-02-04 at 11:18 -0800, Nick Desaulniers wrote:
> > On Thu, Feb 4, 2021 at 2:41 AM Mark Wielaard <mark@klomp.org> wrote:
> > > On Fri, Jan 29, 2021 at 04:44:00PM -0800, Nick Desaulniers wrote:
> > > > Modifies CONFIG_DEBUG_INFO_DWARF4 to be a member of a choice which is
> > > > the default. Does so in a way that's forward compatible with existing
> > > > configs, and makes adding future versions more straightforward.
> > > >
> > > > GCC since ~4.8 has defaulted to this DWARF version implicitly.
> > >
> > > And since GCC 11 it defaults to DWARF version 5.
> > >
> > > It would be better to set the default to the DWARF version that the
> > > compiler generates. So if the user doesn't select any version then it
> > > should default to just -g (or -gdwarf).
> >
> > I disagree.
> >
> > https://lore.kernel.org/lkml/CAKwvOdk0zxewEOaFuqK0aSMz3vKNzDOgmez=-Dae4+bodsSg5w@mail.gmail.com/
> > """
> > I agree that this patch takes away the compiler vendor's choice as to
> > what the implicit default choice is for dwarf version for the kernel.
> > (We, the Linux kernel, do so already for implicit default -std=gnuc*
> > as well). ...
> > But I'm
> > going to suggest we follow the Zen of Python: explicit is better than
> > implicit.
> > """
> > We have a number of in tree and out of tree DWARF consumers that
> > aren't ready for DWARF v5.  Kernel developers need a way to disable
> > DWARF v5 until their dependencies are deployed or more widely
> > available.
>
> I agree with Jakub. Now that GCC has defaulted to DWARF5 all the tools
> have adopted to the new default version. And DWARF5 has been out for

"all of the tools" ?

> more than 4 years already. It isn't unreasonable to assume that people
> using GCC11 will also be using the rest of the toolchain that has moved
> on. Which DWARF consumers are you concerned about not being ready for
> GCC defaulting to DWARF5 once GCC11 is released?

Folks who don't have top of tree pahole or binutils are the two that
come to mind.  I don't have specifics on out of tree consumers, but
some Aarch64 extensions which had some changes to DWARF for ARMv8.3
PAC support broke some debuggers.

I don't doubt a lot of work has gone into fixing many downstream
projects and then when building everything from ToT that there are no
issues with DWARF v5.  The issue is getting upgrades into developers
hands, and what to default to until then.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Mark Wielaard]

On Thu, 2021-02-04 at 12:04 -0800, Nick Desaulniers wrote:
> On Thu, Feb 4, 2021 at 11:56 AM Mark Wielaard <mark@klomp.org> wrote:
> > I agree with Jakub. Now that GCC has defaulted to DWARF5 all the
> > tools
> > have adopted to the new default version. And DWARF5 has been out
> > for
> 
> "all of the tools" ?

I believe so yes, we did a mass-rebuild of all of Fedora a few weeks
back with a GCC11 pre-release and did find some tools which weren't
ready, but as far as I know all have been fixed now. I did try to
coordinate with the Suse and Debian packagers too, so all the major
distros should have all the necessary updates once switching to GCC11.

> > more than 4 years already. It isn't unreasonable to assume that people
> > using GCC11 will also be using the rest of the toolchain that has moved
> > on. Which DWARF consumers are you concerned about not being ready for
> > GCC defaulting to DWARF5 once GCC11 is released?
> 
> Folks who don't have top of tree pahole or binutils are the two that
> come to mind.

I believe pahole just saw a 1.20 release. I am sure it will be widely
available once GCC11 is released (which will still be 1 or 2 months)
and people are actually using it. Or do you expect distros/people are
going to upgrade to GCC11 without updating their other toolchain tools?
BTW. GCC11 doesn't need top of tree binutils, it will detect the
binutils capabilities (bugs) and adjust its DWARF output based on it.

>   I don't have specifics on out of tree consumers, but
> some Aarch64 extensions which had some changes to DWARF for ARMv8.3
> PAC support broke some debuggers.

It would be really helpful if you could provide some specifics. I did
fix some consumers to handle the PAC operands in CFI last year, but I
don't believe that had anything to do with the default DWARF version,
just with dealing with DW_CFA_AARCH64_negate_ra_state.

> I don't doubt a lot of work has gone into fixing many downstream
> projects and then when building everything from ToT that there are no
> issues with DWARF v5.  The issue is getting upgrades into developers
> hands, and what to default to until then.

I would suggest you simply default to what you already do when the
compiler is given -g. Just like you do already for the implicit default
-std=gnuc*. Once GCC11 is actually released and people upgrade their
toolchain to use it the tools will be ready and in developers hands.

Cheers,

Mark

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Thu, Feb 4, 2021 at 12:28 PM Mark Wielaard <mark@klomp.org> wrote:
>
> On Thu, 2021-02-04 at 12:04 -0800, Nick Desaulniers wrote:
> > On Thu, Feb 4, 2021 at 11:56 AM Mark Wielaard <mark@klomp.org> wrote:
> > > I agree with Jakub. Now that GCC has defaulted to DWARF5 all the
> > > tools
> > > have adopted to the new default version. And DWARF5 has been out
> > > for
> >
> > "all of the tools" ?
>
> I believe so yes, we did a mass-rebuild of all of Fedora a few weeks
> back with a GCC11 pre-release and did find some tools which weren't
> ready, but as far as I know all have been fixed now. I did try to

Congrats, I'm sure that was _a lot_ of work.  Our toolchain folks have
been pouring a lot of effort over getting our internal code all moved
over, and it doesn't look like it's been easy from my perspective.

> coordinate with the Suse and Debian packagers too, so all the major
> distros should have all the necessary updates once switching to GCC11.

That's great for users of the next Fedora release who can and will
upgrade, but I wouldn't assume kernel developers can, or will (or are
even using those distros).

Most recently, there was discussion on the list about upgrading the
minimally required version of GCC for building the kernel to GCC 5.1;
we still had developers complain about abandoning GCC 4.9.  And
Guenter shared with me a list of architectures he tests with where he
cannot upgrade the version of GCC in order to keep building them.
https://github.com/groeck/linux-build-test/blob/master/bin/stable-build-arch.sh
(I hope someone sent bug reports for those)

My intent is very much to allow for users of toolchains that have not
switched the implicit default (such as all of the supported versions
of GCC that have been released ie. up to GCC 10.2, and Clang; so all
toolchains the kernel still supports) to enjoy the size saving of
DWARF v5, and find what other tooling needs to be updated.

> > > more than 4 years already. It isn't unreasonable to assume that people
> > > using GCC11 will also be using the rest of the toolchain that has moved
> > > on. Which DWARF consumers are you concerned about not being ready for
> > > GCC defaulting to DWARF5 once GCC11 is released?
> >
> > Folks who don't have top of tree pahole or binutils are the two that
> > come to mind.
>
> I believe pahole just saw a 1.20 release. I am sure it will be widely
> available once GCC11 is released (which will still be 1 or 2 months)
> and people are actually using it. Or do you expect distros/people are
> going to upgrade to GCC11 without updating their other toolchain tools?

Does no one test linux kernel builds with top of tree GCC built from
source?  Or does that require "updating their other toolchain tools?"
If I build ToT GCC from source, do I need to do the same for
binutils-gdb in order to build the kernel?  Pretty sure I don't.

https://bugzilla.redhat.com/show_bug.cgi?id=1922707 and
https://bugzilla.redhat.com/show_bug.cgi?id=1922698 look like user
reports to me, but hopefully some CI system reported earlier that
builds of the Linux kernel with GCC 11 pre release would produce the
warnings from those bug report.  Otherwise it looks like evidence that
users "are going to upgrade to GCC11 without updating their other
toolchain tools."  In the case of pahole, they could not, because
fixes were not yet written.  "Just upgrade" doesn't work if there's no
fix yet upstream.  (pahole is reported fixed for that specific issue,
FWIW).

> BTW. GCC11 doesn't need top of tree binutils, it will detect the
> binutils capabilities (bugs) and adjust its DWARF output based on it.

Yes, I saw https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=6923255e35a3d54f2083ad0f67edebb3f1b86506
and https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=1aeb7d7d67d167297ca2f4a97ef20f68e7546b4c.
It's nice that GCC can tightly couple to a version of binutils. Clang
without its integrated assembler can make no such assumptions about
which assembler the user will prefer to use instead at runtime, and
without binutils 2.35.1 being widely available as we all would like,
leads to issues shipping DWARF v5 by default.

> >   I don't have specifics on out of tree consumers, but
> > some Aarch64 extensions which had some changes to DWARF for ARMv8.3
> > PAC support broke some debuggers.
>
> It would be really helpful if you could provide some specifics. I did
> fix some consumers to handle the PAC operands in CFI last year, but I
> don't believe that had anything to do with the default DWARF version,
> just with dealing with DW_CFA_AARCH64_negate_ra_state.

Yep, that's the one.  I suspect that the same out of tree consumers of
DWARF that did not see that coming will similarly be stumped when
presented with DWARF v5, but it's hypothetical, so not much of an
argument I'll admit.  I just wouldn't bet that an upgrade to DWARF v5
will be painless at this point in time, as evidenced by how much blood
has been poured into finding what tools out there were broken and
needed to be fixed.  I also recognize we can't drag our heels forever
over this.  It's not the intent of the patch series to do so.

> > I don't doubt a lot of work has gone into fixing many downstream
> > projects and then when building everything from ToT that there are no
> > issues with DWARF v5.  The issue is getting upgrades into developers
> > hands, and what to default to until then.
>
> I would suggest you simply default to what you already do when the
> compiler is given -g. Just like you do already for the implicit default
> -std=gnuc*. Once GCC11 is actually released and people upgrade their
> toolchain to use it the tools will be ready and in developers hands.

Hmmm...thinking about this more over lunch.

I think the linker script additions for the new DWARF v5 sections are
most important.  There's no need to hold those hostage over what we do
with the configs. They are orthogonal, and can go in first.  There's
already one section in the linker script already related to v5, and we
know that DWARF v5 is coming, so there's no reason not to just add
them.  That would resolve
https://bugzilla.redhat.com/show_bug.cgi?id=1922707.

For configs, today the kernel has an explicit opt in for DWARF v4
called CONFIG_DEBUG_INFO_DWARF4.  When it was introduced, I suspect
DWARF v2 (or v3) was the default, so it let kernel developers opt in
to newer versions (say, for testing) than what was the implicit
default version.  With the advent of DWARF v5 and toolchains moving to
producing that by default, maybe there is still a place for
CONFIG_DEBUG_INFO_DWARF4, but as a way to opt in to older versions.
Your installed version of pahole or binutils is too old for DWARF v5?
Great, CONFIG_DEBUG_INFO_DWARF4 is your friend until you can update
your tools.  CONFIG_DEBUG_INFO_DWARF4 becomes "opt backwards" rather
than it's original "opt forwards" (if that makes sense).

One small issue I have with that is that it doesn't help users of GCC
5 through 10.2 (or clang) opt into DWARF v5 (to test, or for the size
savings) similar to the original intent of CONFIG_DEBUG_INFO_DWARF4.
That was my intent with this series; to opt into new versions so we
can begin testing them and finding kinks in other tools.  It was not
intentionally to hold back compilers once they upgrade their default
versions, even if I still think explicit is better than implicit.  And
it would be nice to have the framework to continue to do so for v6
someday.

So opting in to v4 explicitly provides a "pressure relief valve" for
developers that don't have the latest tools.  Opting into v5
explicitly allows for testing of those tools with the latest DWARF
version (and size savings that are worthwhile) for folks with older
GCC and Clang.

What are your thoughts on this?

What if I modified the config to have 3 options:
1. (default) CONFIG_DEBUG_INFO_DWARF_VERSION_DONT_CARE (I'm open to a
better color for the bikeshed). Does not set a -gdwarf-*.  This is the
status quo today when you build the kernel.  You allow toolchain
developers to decide your fate. If that makes you unhappy, congrats,
you have the below options.
2. CONFIG_DEBUG_INFO_DWARF4 (the ship has already sailed on the name
here, sorry) Explicitly sets -gdwarf-4. Use this if you have a good
reason not to upgrade to the recommended -gdwarf-5.  We might
disappear this option soon. (already exists in tree today, and doesn't
need to be removed; instead repurposed)
3. CONFIG_DEBUG_INFO_DWARF5 Explicitly sets -gdwarf-5.  Use this if
you would like to opt into a newer version than what's provided by
default.  Provides significant size savings over DWARF v4. Avoid if
you have broken shit you can't upgrade for whatever reason?

Then I think everyone can be happy? Toolchain vendors can to continue
pursuing updates of implicit default dwarf version (aggressive, but
necessary force adoption, perhaps), BTF/pahole/binutils users can
downgrade until they have the necessary updates, and developers with
older GCC and clang can opt in new features without having to upgrade
the toolchain's implicit default (since that requires significant work
for other codebases, but is making progress) and still realize the
binary size savings.

--
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Fangrui Song]

On 2021-02-04, Nick Desaulniers wrote:
>On Thu, Feb 4, 2021 at 12:28 PM Mark Wielaard <mark@klomp.org> wrote:
>>
>> On Thu, 2021-02-04 at 12:04 -0800, Nick Desaulniers wrote:
>> > On Thu, Feb 4, 2021 at 11:56 AM Mark Wielaard <mark@klomp.org> wrote:
>> > > I agree with Jakub. Now that GCC has defaulted to DWARF5 all the
>> > > tools
>> > > have adopted to the new default version. And DWARF5 has been out
>> > > for
>> >
>> > "all of the tools" ?
>>
>> I believe so yes, we did a mass-rebuild of all of Fedora a few weeks
>> back with a GCC11 pre-release and did find some tools which weren't
>> ready, but as far as I know all have been fixed now. I did try to
>
>Congrats, I'm sure that was _a lot_ of work.  Our toolchain folks have
>been pouring a lot of effort over getting our internal code all moved
>over, and it doesn't look like it's been easy from my perspective.
>
>> coordinate with the Suse and Debian packagers too, so all the major
>> distros should have all the necessary updates once switching to GCC11.
>
>That's great for users of the next Fedora release who can and will
>upgrade, but I wouldn't assume kernel developers can, or will (or are
>even using those distros).
>
>Most recently, there was discussion on the list about upgrading the
>minimally required version of GCC for building the kernel to GCC 5.1;
>we still had developers complain about abandoning GCC 4.9.  And
>Guenter shared with me a list of architectures he tests with where he
>cannot upgrade the version of GCC in order to keep building them.
>https://github.com/groeck/linux-build-test/blob/master/bin/stable-build-arch.sh
>(I hope someone sent bug reports for those)
>
>My intent is very much to allow for users of toolchains that have not
>switched the implicit default (such as all of the supported versions
>of GCC that have been released ie. up to GCC 10.2, and Clang; so all
>toolchains the kernel still supports) to enjoy the size saving of
>DWARF v5, and find what other tooling needs to be updated.
>
>> > > more than 4 years already. It isn't unreasonable to assume that people
>> > > using GCC11 will also be using the rest of the toolchain that has moved
>> > > on. Which DWARF consumers are you concerned about not being ready for
>> > > GCC defaulting to DWARF5 once GCC11 is released?
>> >
>> > Folks who don't have top of tree pahole or binutils are the two that
>> > come to mind.
>>
>> I believe pahole just saw a 1.20 release. I am sure it will be widely
>> available once GCC11 is released (which will still be 1 or 2 months)
>> and people are actually using it. Or do you expect distros/people are
>> going to upgrade to GCC11 without updating their other toolchain tools?
>
>Does no one test linux kernel builds with top of tree GCC built from
>source?  Or does that require "updating their other toolchain tools?"
>If I build ToT GCC from source, do I need to do the same for
>binutils-gdb in order to build the kernel?  Pretty sure I don't.
>
>https://bugzilla.redhat.com/show_bug.cgi?id=1922707 and
>https://bugzilla.redhat.com/show_bug.cgi?id=1922698 look like user
>reports to me, but hopefully some CI system reported earlier that
>builds of the Linux kernel with GCC 11 pre release would produce the
>warnings from those bug report.  Otherwise it looks like evidence that
>users "are going to upgrade to GCC11 without updating their other
>toolchain tools."  In the case of pahole, they could not, because
>fixes were not yet written.  "Just upgrade" doesn't work if there's no
>fix yet upstream.  (pahole is reported fixed for that specific issue,
>FWIW).
>
>> BTW. GCC11 doesn't need top of tree binutils, it will detect the
>> binutils capabilities (bugs) and adjust its DWARF output based on it.
>
>Yes, I saw https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=6923255e35a3d54f2083ad0f67edebb3f1b86506
>and https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=1aeb7d7d67d167297ca2f4a97ef20f68e7546b4c.
>It's nice that GCC can tightly couple to a version of binutils. Clang
>without its integrated assembler can make no such assumptions about
>which assembler the user will prefer to use instead at runtime, and
>without binutils 2.35.1 being widely available as we all would like,
>leads to issues shipping DWARF v5 by default.
>
>> >   I don't have specifics on out of tree consumers, but
>> > some Aarch64 extensions which had some changes to DWARF for ARMv8.3
>> > PAC support broke some debuggers.
>>
>> It would be really helpful if you could provide some specifics. I did
>> fix some consumers to handle the PAC operands in CFI last year, but I
>> don't believe that had anything to do with the default DWARF version,
>> just with dealing with DW_CFA_AARCH64_negate_ra_state.
>
>Yep, that's the one.  I suspect that the same out of tree consumers of
>DWARF that did not see that coming will similarly be stumped when
>presented with DWARF v5, but it's hypothetical, so not much of an
>argument I'll admit.  I just wouldn't bet that an upgrade to DWARF v5
>will be painless at this point in time, as evidenced by how much blood
>has been poured into finding what tools out there were broken and
>needed to be fixed.  I also recognize we can't drag our heels forever
>over this.  It's not the intent of the patch series to do so.
>
>> > I don't doubt a lot of work has gone into fixing many downstream
>> > projects and then when building everything from ToT that there are no
>> > issues with DWARF v5.  The issue is getting upgrades into developers
>> > hands, and what to default to until then.
>>
>> I would suggest you simply default to what you already do when the
>> compiler is given -g. Just like you do already for the implicit default
>> -std=gnuc*. Once GCC11 is actually released and people upgrade their
>> toolchain to use it the tools will be ready and in developers hands.
>
>Hmmm...thinking about this more over lunch.
>
>I think the linker script additions for the new DWARF v5 sections are
>most important.  There's no need to hold those hostage over what we do
>with the configs. They are orthogonal, and can go in first.  There's
>already one section in the linker script already related to v5, and we
>know that DWARF v5 is coming, so there's no reason not to just add
>them.  That would resolve
>https://bugzilla.redhat.com/show_bug.cgi?id=1922707.
>
>For configs, today the kernel has an explicit opt in for DWARF v4
>called CONFIG_DEBUG_INFO_DWARF4.  When it was introduced, I suspect
>DWARF v2 (or v3) was the default, so it let kernel developers opt in
>to newer versions (say, for testing) than what was the implicit
>default version.  With the advent of DWARF v5 and toolchains moving to
>producing that by default, maybe there is still a place for
>CONFIG_DEBUG_INFO_DWARF4, but as a way to opt in to older versions.
>Your installed version of pahole or binutils is too old for DWARF v5?
>Great, CONFIG_DEBUG_INFO_DWARF4 is your friend until you can update
>your tools.  CONFIG_DEBUG_INFO_DWARF4 becomes "opt backwards" rather
>than it's original "opt forwards" (if that makes sense).
>
>One small issue I have with that is that it doesn't help users of GCC
>5 through 10.2 (or clang) opt into DWARF v5 (to test, or for the size
>savings) similar to the original intent of CONFIG_DEBUG_INFO_DWARF4.
>That was my intent with this series; to opt into new versions so we
>can begin testing them and finding kinks in other tools.  It was not
>intentionally to hold back compilers once they upgrade their default
>versions, even if I still think explicit is better than implicit.  And
>it would be nice to have the framework to continue to do so for v6
>someday.
>
>So opting in to v4 explicitly provides a "pressure relief valve" for
>developers that don't have the latest tools.  Opting into v5
>explicitly allows for testing of those tools with the latest DWARF
>version (and size savings that are worthwhile) for folks with older
>GCC and Clang.
>
>What are your thoughts on this?
>
>What if I modified the config to have 3 options:
>1. (default) CONFIG_DEBUG_INFO_DWARF_VERSION_DONT_CARE (I'm open to a
>better color for the bikeshed). Does not set a -gdwarf-*.  This is the
>status quo today when you build the kernel.  You allow toolchain
>developers to decide your fate. If that makes you unhappy, congrats,
>you have the below options.
>2. CONFIG_DEBUG_INFO_DWARF4 (the ship has already sailed on the name
>here, sorry) Explicitly sets -gdwarf-4. Use this if you have a good
>reason not to upgrade to the recommended -gdwarf-5.  We might
>disappear this option soon. (already exists in tree today, and doesn't
>need to be removed; instead repurposed)
>3. CONFIG_DEBUG_INFO_DWARF5 Explicitly sets -gdwarf-5.  Use this if
>you would like to opt into a newer version than what's provided by
>default.  Provides significant size savings over DWARF v4. Avoid if
>you have broken shit you can't upgrade for whatever reason?
>
>Then I think everyone can be happy? Toolchain vendors can to continue
>pursuing updates of implicit default dwarf version (aggressive, but
>necessary force adoption, perhaps), BTF/pahole/binutils users can
>downgrade until they have the necessary updates, and developers with
>older GCC and clang can opt in new features without having to upgrade
>the toolchain's implicit default (since that requires significant work
>for other codebases, but is making progress) and still realize the
>binary size savings.

The points raised by Nick are very reasonable.

A few colleagues and I are involved in upgrading our internal systems to be
DWARF v5 ready. It required to fix a dozen of things.

Externally I watch binutils mailing list closely and in recent months DWARF
fixes are quite regular.  (Mark and H.J. contributed lot of fixes. Big thanks!)
For example, objdump -S did not display source for gcc-11 compiled object
files.  It was just fixed a few days ago
https://sourceware.org/bugzilla/show_bug.cgi?id=27231 , which is included in
binutils 2.35.2
(https://sourceware.org/pipermail/binutils/2021-January/115150.html).

As we all know DWARF v5 has huge size savings and we want to make it the
default, but today is probably a bit early.


(Note about Clang and binutils: I added -fbinutils-version= to Clang 12.
If there are workaround needs we can use that option.)

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Mark Wielaard]

Hi Nick,

On Thu, 2021-02-04 at 14:06 -0800, Nick Desaulniers wrote:
> On Thu, Feb 4, 2021 at 12:28 PM Mark Wielaard <mark@klomp.org> wrote:
> > I believe so yes, we did a mass-rebuild of all of Fedora a few weeks
> > back with a GCC11 pre-release and did find some tools which weren't
> > ready, but as far as I know all have been fixed now. I did try to
> 
> Congrats, I'm sure that was _a lot_ of work.  Our toolchain folks have
> been pouring a lot of effort over getting our internal code all moved
> over, and it doesn't look like it's been easy from my perspective.
> 
> > coordinate with the Suse and Debian packagers too, so all the major
> > distros should have all the necessary updates once switching to GCC11.
> 
> That's great for users of the next Fedora release who can and will
> upgrade, but I wouldn't assume kernel developers can, or will (or are
> even using those distros).

And that is fine, then they will probably also not run GCC11 yet when
it gets released. GCC11 (and Fedora34) will not be released for another
2 months. By then I hope all issues found have made it into upstream
releases.

> My intent is very much to allow for users of toolchains that have not
> switched the implicit default (such as all of the supported versions
> of GCC that have been released ie. up to GCC 10.2, and Clang; so all
> toolchains the kernel still supports) to enjoy the size saving of
> DWARF v5, and find what other tooling needs to be updated.

Which seems a good thing. No complaints about making it easier to
experiment with DWARFv5 for such older compiler releases.

> Does no one test linux kernel builds with top of tree GCC built from
> source?

Fedora rawhide (f34) builds rc kernels against GCC11 ToT now:
https://koji.fedoraproject.org/koji/packageinfo?packageID=8

>   Or does that require "updating their other toolchain tools?"
> If I build ToT GCC from source, do I need to do the same for
> binutils-gdb in order to build the kernel?  Pretty sure I don't.

It isn't required, but then GCC won't produce full DWARF5 (e.g. line
tables are then still emitted as DWARF4).

> https://bugzilla.redhat.com/show_bug.cgi?id=1922707 and
> https://bugzilla.redhat.com/show_bug.cgi?id=1922698 look like user
> reports to me, but hopefully some CI system reported earlier that
> builds of the Linux kernel with GCC 11 pre release would produce the
> warnings from those bug report.  Otherwise it looks like evidence that
> users "are going to upgrade to GCC11 without updating their other
> toolchain tools."  In the case of pahole, they could not, because
> fixes were not yet written.  "Just upgrade" doesn't work if there's no
> fix yet upstream.  (pahole is reported fixed for that specific issue,
> FWIW).

So those are users that are participating in the Fedora rawhide
experiment with the GC11-prerelease. That are precisely the bug reports
we need to make sure we caught all issues before the final release. I
don't think that is any different from users trying out rc kernels and
finding issues that are then fixed.

> > > I don't doubt a lot of work has gone into fixing many downstream
> > > projects and then when building everything from ToT that there are no
> > > issues with DWARF v5.  The issue is getting upgrades into developers
> > > hands, and what to default to until then.
> > 
> > I would suggest you simply default to what you already do when the
> > compiler is given -g. Just like you do already for the implicit default
> > -std=gnuc*. Once GCC11 is actually released and people upgrade their
> > toolchain to use it the tools will be ready and in developers hands.
> 
> Hmmm...thinking about this more over lunch.
> 
> I think the linker script additions for the new DWARF v5 sections are
> most important.  There's no need to hold those hostage over what we do
> with the configs. They are orthogonal, and can go in first.  There's
> already one section in the linker script already related to v5, and we
> know that DWARF v5 is coming, so there's no reason not to just add
> them.  That would resolve
> https://bugzilla.redhat.com/show_bug.cgi?id=1922707.

Yes, that patch seems fine without any of the other config changes. It
would be nice to see that go in.

> For configs, today the kernel has an explicit opt in for DWARF v4
> called CONFIG_DEBUG_INFO_DWARF4.  When it was introduced, I suspect
> DWARF v2 (or v3) was the default, so it let kernel developers opt in
> to newer versions (say, for testing) than what was the implicit
> default version.  With the advent of DWARF v5 and toolchains moving to
> producing that by default, maybe there is still a place for
> CONFIG_DEBUG_INFO_DWARF4, but as a way to opt in to older versions.
> Your installed version of pahole or binutils is too old for DWARF v5?
> Great, CONFIG_DEBUG_INFO_DWARF4 is your friend until you can update
> your tools.  CONFIG_DEBUG_INFO_DWARF4 becomes "opt backwards" rather
> than it's original "opt forwards" (if that makes sense).
> 
> One small issue I have with that is that it doesn't help users of GCC
> 5 through 10.2 (or clang) opt into DWARF v5 (to test, or for the size
> savings) similar to the original intent of CONFIG_DEBUG_INFO_DWARF4.
> That was my intent with this series; to opt into new versions so we
> can begin testing them and finding kinks in other tools.  It was not
> intentionally to hold back compilers once they upgrade their default
> versions, even if I still think explicit is better than implicit.  And
> it would be nice to have the framework to continue to do so for v6
> someday.
> 
> So opting in to v4 explicitly provides a "pressure relief valve" for
> developers that don't have the latest tools.  Opting into v5
> explicitly allows for testing of those tools with the latest DWARF
> version (and size savings that are worthwhile) for folks with older
> GCC and Clang.
> 
> What are your thoughts on this?

Yes, being able to opt into DWARF4 if you have some newer tools like
GCC11 that already default to version 5, but not others, like an old
pahole release that doesn't handle version 5 yet would be nice to have.
But I don't think that is a very likely scenario given that pahole (and
binutils) already got a new releases and GCC11 won't be released for a
couple of months yet.

Likewise having the option to opt into DWARF5 even though some of the
tools don't default to (or fully support) DWARF5 yet seems a good idea.
Just for users to see whether that works with all their tools.

> What if I modified the config to have 3 options:
> 1. (default) CONFIG_DEBUG_INFO_DWARF_VERSION_DONT_CARE (I'm open to a
> better color for the bikeshed). Does not set a -gdwarf-*.  This is the
> status quo today when you build the kernel.  You allow toolchain
> developers to decide your fate. If that makes you unhappy, congrats,
> you have the below options.

My color of the bikeshed would be
CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT.

> 2. CONFIG_DEBUG_INFO_DWARF4 (the ship has already sailed on the name
> here, sorry) Explicitly sets -gdwarf-4. Use this if you have a good
> reason not to upgrade to the recommended -gdwarf-5.  We might
> disappear this option soon. (already exists in tree today, and doesn't
> need to be removed; instead repurposed)
> 3. CONFIG_DEBUG_INFO_DWARF5 Explicitly sets -gdwarf-5.  Use this if
> you would like to opt into a newer version than what's provided by
> default.  Provides significant size savings over DWARF v4. Avoid if
> you have broken shit you can't upgrade for whatever reason?
> 
> Then I think everyone can be happy? Toolchain vendors can to continue
> pursuing updates of implicit default dwarf version (aggressive, but
> necessary force adoption, perhaps), BTF/pahole/binutils users can
> downgrade until they have the necessary updates, and developers with
> older GCC and clang can opt in new features without having to upgrade
> the toolchain's implicit default (since that requires significant work
> for other codebases, but is making progress) and still realize the
> binary size savings.

That would certainly make me happy :)

Thanks,

Mark

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Nick Desaulniers]

On Fri, Feb 5, 2021 at 4:49 AM Mark Wielaard <mark@klomp.org> wrote:
>
> Hi Nick,
>
> On Thu, 2021-02-04 at 14:06 -0800, Nick Desaulniers wrote:
> > On Thu, Feb 4, 2021 at 12:28 PM Mark Wielaard <mark@klomp.org> wrote:
> > > I believe so yes, we did a mass-rebuild of all of Fedora a few weeks
> > > back with a GCC11 pre-release and did find some tools which weren't
> > > ready, but as far as I know all have been fixed now. I did try to

I guess I'm curious whether
https://bugzilla.redhat.com/show_bug.cgi?id=1922707 came up during the
mass rebuild of all of Fedora a few weeks ago?  Assuming the Linux
kernel was part of that test, those warnings would have been both new
and obviously related to changing the implicit default DWARF version.
It's possible it was a kernel build without debug info enabled or a
kernel version old enough to not have that linker warning enabled, I
suppose.  It might be good to check though, that way changes to GCC
that impact the kernel are caught even sooner or ASAP. New diagnostics
added every compiler release come to mind in particular.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 1/2] Kbuild: make DWARF version a choice

[Mark Wielaard]

Hi Nick,

On Fri, Feb 05, 2021 at 01:18:11PM -0800, Nick Desaulniers wrote:
> On Fri, Feb 5, 2021 at 4:49 AM Mark Wielaard <mark@klomp.org> wrote:
> I guess I'm curious whether
> https://bugzilla.redhat.com/show_bug.cgi?id=1922707 came up during the
> mass rebuild of all of Fedora a few weeks ago?  Assuming the Linux
> kernel was part of that test, those warnings would have been both new
> and obviously related to changing the implicit default DWARF version.

Yes, looking at the build.log that warning was also present.  But the
dwarves pahole update to process DWARF5 was more important.  Also at
first it was believed this came from the binutils ld linker
scripts. Which were also updated first. Once your patch is accepted we
can resolve that bug.

Cheers,

Mark

[PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Nick Desaulniers]

DWARF v5 is the latest standard of the DWARF debug info format.

Feature detection of DWARF5 is onerous, especially given that we've
removed $(AS), so we must query $(CC) for DWARF5 assembler directive
support.

The DWARF version of a binary can be validated with:
$ llvm-dwarfdump vmlinux | head -n 4 | grep version
or
$ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version

DWARF5 wins significantly in terms of size when mixed with compression
(CONFIG_DEBUG_INFO_COMPRESSED).

363M    vmlinux.clang12.dwarf5.compressed
434M    vmlinux.clang12.dwarf4.compressed
439M    vmlinux.clang12.dwarf2.compressed
457M    vmlinux.clang12.dwarf5
536M    vmlinux.clang12.dwarf4
548M    vmlinux.clang12.dwarf2

515M    vmlinux.gcc10.2.dwarf5.compressed
599M    vmlinux.gcc10.2.dwarf4.compressed
624M    vmlinux.gcc10.2.dwarf2.compressed
630M    vmlinux.gcc10.2.dwarf5
765M    vmlinux.gcc10.2.dwarf4
809M    vmlinux.gcc10.2.dwarf2

Though the quality of debug info is harder to quantify; size is not a
proxy for quality.

Jakub notes:
  All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
  option that enabled some small DWARF subset (initially only a few
  DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
  after DWARF 5 has been finalized) started emitting DWARF5 section
  headers and got most of the DWARF5 changes in...

Version check GCC so that we don't need to worry about the difference in
command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
validate the DWARF Version in the assembler feature detection script.

GNU `as` only recently gained support for specifying -gdwarf-5, so when
compiling with Clang but without Clang's integrated assembler
(LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.

Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
recognize the new additions to the DWARF debug info. Thanks to Sedat for
the report.

Link: http://www.dwarfstd.org/doc/DWARF5.pdf
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
Suggested-by: Caroline Tice <cmtice@google.com>
Suggested-by: Fangrui Song <maskray@google.com>
Suggested-by: Jakub Jelinek <jakub@redhat.com>
Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
 Makefile                          |  1 +
 include/asm-generic/vmlinux.lds.h |  7 ++++++-
 lib/Kconfig.debug                 | 18 ++++++++++++++++++
 scripts/test_dwarf5_support.sh    |  8 ++++++++
 4 files changed, 33 insertions(+), 1 deletion(-)
 create mode 100755 scripts/test_dwarf5_support.sh

diff --git a/Makefile b/Makefile
index d2b4980807e0..5387a6f2f62d 100644
--- a/Makefile
+++ b/Makefile
@@ -831,6 +831,7 @@ KBUILD_AFLAGS	+= -Wa,-gdwarf-2
 endif
 
 dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
+dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
 DEBUG_CFLAGS	+= -gdwarf-$(dwarf-version-y)
 
 ifdef CONFIG_DEBUG_INFO_REDUCED
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 34b7e0d2346c..1e7cde4bd3f9 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -842,8 +842,13 @@
 		/* DWARF 4 */						\
 		.debug_types	0 : { *(.debug_types) }			\
 		/* DWARF 5 */						\
+		.debug_addr	0 : { *(.debug_addr) }			\
+		.debug_line_str	0 : { *(.debug_line_str) }		\
+		.debug_loclists	0 : { *(.debug_loclists) }		\
 		.debug_macro	0 : { *(.debug_macro) }			\
-		.debug_addr	0 : { *(.debug_addr) }
+		.debug_names	0 : { *(.debug_names) }			\
+		.debug_rnglists	0 : { *(.debug_rnglists) }		\
+		.debug_str_offsets	0 : { *(.debug_str_offsets) }
 
 /* Stabs debugging sections. */
 #define STABS_DEBUG							\
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 94c1a7ed6306..ad6f78989d4f 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
 	  It makes the debug information larger, but it significantly
 	  improves the success of resolving variables in gdb on optimized code.
 
+config DEBUG_INFO_DWARF5
+	bool "Generate DWARF Version 5 debuginfo"
+	depends on GCC_VERSION >= 50000 || CC_IS_CLANG
+	depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
+	depends on !DEBUG_INFO_BTF
+	help
+	  Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
+	  5.0+ accepts the -gdwarf-5 flag but only had partial support for some
+	  draft features until 7.0), and gdb 8.0+.
+
+	  Changes to the structure of debug info in Version 5 allow for around
+	  15-18% savings in resulting image and debug info section sizes as
+	  compared to DWARF Version 4. DWARF Version 5 standardizes previous
+	  extensions such as accelerators for symbol indexing and the format
+	  for fission (.dwo/.dwp) files. Users may not want to select this
+	  config if they rely on tooling that has not yet been updated to
+	  support DWARF Version 5.
+
 endchoice # "DWARF version"
 
 config DEBUG_INFO_BTF
diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
new file mode 100755
index 000000000000..c46e2456b47a
--- /dev/null
+++ b/scripts/test_dwarf5_support.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
+# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
+# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
+echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
+  $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
-- 
2.30.0.365.g02bc693789-goog

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Nathan Chancellor]

On Fri, Jan 29, 2021 at 04:44:01PM -0800, Nick Desaulniers wrote:
> DWARF v5 is the latest standard of the DWARF debug info format.
> 
> Feature detection of DWARF5 is onerous, especially given that we've
> removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> support.
> 
> The DWARF version of a binary can be validated with:
> $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> or
> $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
> 
> DWARF5 wins significantly in terms of size when mixed with compression
> (CONFIG_DEBUG_INFO_COMPRESSED).
> 
> 363M    vmlinux.clang12.dwarf5.compressed
> 434M    vmlinux.clang12.dwarf4.compressed
> 439M    vmlinux.clang12.dwarf2.compressed
> 457M    vmlinux.clang12.dwarf5
> 536M    vmlinux.clang12.dwarf4
> 548M    vmlinux.clang12.dwarf2
> 
> 515M    vmlinux.gcc10.2.dwarf5.compressed
> 599M    vmlinux.gcc10.2.dwarf4.compressed
> 624M    vmlinux.gcc10.2.dwarf2.compressed
> 630M    vmlinux.gcc10.2.dwarf5
> 765M    vmlinux.gcc10.2.dwarf4
> 809M    vmlinux.gcc10.2.dwarf2
> 
> Though the quality of debug info is harder to quantify; size is not a
> proxy for quality.
> 
> Jakub notes:
>   All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
>   option that enabled some small DWARF subset (initially only a few
>   DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
>   after DWARF 5 has been finalized) started emitting DWARF5 section
>   headers and got most of the DWARF5 changes in...
> 
> Version check GCC so that we don't need to worry about the difference in
> command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> validate the DWARF Version in the assembler feature detection script.
> 
> GNU `as` only recently gained support for specifying -gdwarf-5, so when
> compiling with Clang but without Clang's integrated assembler
> (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
> 
> Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> recognize the new additions to the DWARF debug info. Thanks to Sedat for
> the report.
> 
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
> Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> Suggested-by: Caroline Tice <cmtice@google.com>
> Suggested-by: Fangrui Song <maskray@google.com>
> Suggested-by: Jakub Jelinek <jakub@redhat.com>
> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>

Reviewed-by: Nathan Chancellor <nathan@kernel.org>

> ---
>  Makefile                          |  1 +
>  include/asm-generic/vmlinux.lds.h |  7 ++++++-
>  lib/Kconfig.debug                 | 18 ++++++++++++++++++
>  scripts/test_dwarf5_support.sh    |  8 ++++++++
>  4 files changed, 33 insertions(+), 1 deletion(-)
>  create mode 100755 scripts/test_dwarf5_support.sh
> 
> diff --git a/Makefile b/Makefile
> index d2b4980807e0..5387a6f2f62d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -831,6 +831,7 @@ KBUILD_AFLAGS	+= -Wa,-gdwarf-2
>  endif
>  
>  dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
>  DEBUG_CFLAGS	+= -gdwarf-$(dwarf-version-y)
>  
>  ifdef CONFIG_DEBUG_INFO_REDUCED
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 34b7e0d2346c..1e7cde4bd3f9 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -842,8 +842,13 @@
>  		/* DWARF 4 */						\
>  		.debug_types	0 : { *(.debug_types) }			\
>  		/* DWARF 5 */						\
> +		.debug_addr	0 : { *(.debug_addr) }			\
> +		.debug_line_str	0 : { *(.debug_line_str) }		\
> +		.debug_loclists	0 : { *(.debug_loclists) }		\
>  		.debug_macro	0 : { *(.debug_macro) }			\
> -		.debug_addr	0 : { *(.debug_addr) }
> +		.debug_names	0 : { *(.debug_names) }			\
> +		.debug_rnglists	0 : { *(.debug_rnglists) }		\
> +		.debug_str_offsets	0 : { *(.debug_str_offsets) }
>  
>  /* Stabs debugging sections. */
>  #define STABS_DEBUG							\
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 94c1a7ed6306..ad6f78989d4f 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
>  	  It makes the debug information larger, but it significantly
>  	  improves the success of resolving variables in gdb on optimized code.
>  
> +config DEBUG_INFO_DWARF5
> +	bool "Generate DWARF Version 5 debuginfo"
> +	depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> +	depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> +	depends on !DEBUG_INFO_BTF
> +	help
> +	  Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> +	  5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> +	  draft features until 7.0), and gdb 8.0+.
> +
> +	  Changes to the structure of debug info in Version 5 allow for around
> +	  15-18% savings in resulting image and debug info section sizes as
> +	  compared to DWARF Version 4. DWARF Version 5 standardizes previous
> +	  extensions such as accelerators for symbol indexing and the format
> +	  for fission (.dwo/.dwp) files. Users may not want to select this
> +	  config if they rely on tooling that has not yet been updated to
> +	  support DWARF Version 5.
> +
>  endchoice # "DWARF version"
>  
>  config DEBUG_INFO_BTF
> diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> new file mode 100755
> index 000000000000..c46e2456b47a
> --- /dev/null
> +++ b/scripts/test_dwarf5_support.sh
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +
> +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> +  $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> -- 
> 2.30.0.365.g02bc693789-goog
> 

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Sedat Dilek]

On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> DWARF v5 is the latest standard of the DWARF debug info format.
>
> Feature detection of DWARF5 is onerous, especially given that we've
> removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> support.
>
> The DWARF version of a binary can be validated with:
> $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> or
> $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
>
> DWARF5 wins significantly in terms of size when mixed with compression
> (CONFIG_DEBUG_INFO_COMPRESSED).
>
> 363M    vmlinux.clang12.dwarf5.compressed
> 434M    vmlinux.clang12.dwarf4.compressed
> 439M    vmlinux.clang12.dwarf2.compressed
> 457M    vmlinux.clang12.dwarf5
> 536M    vmlinux.clang12.dwarf4
> 548M    vmlinux.clang12.dwarf2
>
> 515M    vmlinux.gcc10.2.dwarf5.compressed
> 599M    vmlinux.gcc10.2.dwarf4.compressed
> 624M    vmlinux.gcc10.2.dwarf2.compressed
> 630M    vmlinux.gcc10.2.dwarf5
> 765M    vmlinux.gcc10.2.dwarf4
> 809M    vmlinux.gcc10.2.dwarf2
>
> Though the quality of debug info is harder to quantify; size is not a
> proxy for quality.
>
> Jakub notes:
>   All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
>   option that enabled some small DWARF subset (initially only a few
>   DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
>   after DWARF 5 has been finalized) started emitting DWARF5 section
>   headers and got most of the DWARF5 changes in...
>
> Version check GCC so that we don't need to worry about the difference in
> command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> validate the DWARF Version in the assembler feature detection script.
>
> GNU `as` only recently gained support for specifying -gdwarf-5, so when
> compiling with Clang but without Clang's integrated assembler
> (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
>
> Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> recognize the new additions to the DWARF debug info. Thanks to Sedat for
> the report.
>
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
> Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> Suggested-by: Caroline Tice <cmtice@google.com>
> Suggested-by: Fangrui Song <maskray@google.com>
> Suggested-by: Jakub Jelinek <jakub@redhat.com>
> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
>  Makefile                          |  1 +
>  include/asm-generic/vmlinux.lds.h |  7 ++++++-
>  lib/Kconfig.debug                 | 18 ++++++++++++++++++
>  scripts/test_dwarf5_support.sh    |  8 ++++++++
>  4 files changed, 33 insertions(+), 1 deletion(-)
>  create mode 100755 scripts/test_dwarf5_support.sh
>
> diff --git a/Makefile b/Makefile
> index d2b4980807e0..5387a6f2f62d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -831,6 +831,7 @@ KBUILD_AFLAGS       += -Wa,-gdwarf-2
>  endif
>
>  dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
>  DEBUG_CFLAGS   += -gdwarf-$(dwarf-version-y)
>
>  ifdef CONFIG_DEBUG_INFO_REDUCED
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 34b7e0d2346c..1e7cde4bd3f9 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -842,8 +842,13 @@
>                 /* DWARF 4 */                                           \
>                 .debug_types    0 : { *(.debug_types) }                 \
>                 /* DWARF 5 */                                           \
> +               .debug_addr     0 : { *(.debug_addr) }                  \
> +               .debug_line_str 0 : { *(.debug_line_str) }              \
> +               .debug_loclists 0 : { *(.debug_loclists) }              \
>                 .debug_macro    0 : { *(.debug_macro) }                 \
> -               .debug_addr     0 : { *(.debug_addr) }
> +               .debug_names    0 : { *(.debug_names) }                 \
> +               .debug_rnglists 0 : { *(.debug_rnglists) }              \
> +               .debug_str_offsets      0 : { *(.debug_str_offsets) }
>

I just looked at binutils 2.36 in the Debian/experimental repositories.

[1] says:

+ PR ld/27230
+ * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.

...

-  /* DWARF Extension.  */
-  .debug_macro    0 : { *(.debug_macro) }
+  /* DWARF 5.  */
   .debug_addr     0 : { *(.debug_addr) }
+  .debug_line_str 0 : { *(.debug_line_str) }
+  .debug_loclists 0 : { *(.debug_loclists) }
+  .debug_macro    0 : { *(.debug_macro) }
+  .debug_names    0 : { *(.debug_names) }
+  .debug_rnglists 0 : { *(.debug_rnglists) }
+  .debug_str_offsets 0 : { *(.debug_str_offsets) }
+  .debug_sup      0 : { *(.debug_sup) }

The list of DWARF-5 .debug_* sections is alphabetically sorted.
AFAICS .debug_sup section is missing?

- Sedat -

[1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430

>  /* Stabs debugging sections. */
>  #define STABS_DEBUG                                                    \
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 94c1a7ed6306..ad6f78989d4f 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
>           It makes the debug information larger, but it significantly
>           improves the success of resolving variables in gdb on optimized code.
>
> +config DEBUG_INFO_DWARF5
> +       bool "Generate DWARF Version 5 debuginfo"
> +       depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> +       depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> +       depends on !DEBUG_INFO_BTF
> +       help
> +         Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> +         5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> +         draft features until 7.0), and gdb 8.0+.
> +
> +         Changes to the structure of debug info in Version 5 allow for around
> +         15-18% savings in resulting image and debug info section sizes as
> +         compared to DWARF Version 4. DWARF Version 5 standardizes previous
> +         extensions such as accelerators for symbol indexing and the format
> +         for fission (.dwo/.dwp) files. Users may not want to select this
> +         config if they rely on tooling that has not yet been updated to
> +         support DWARF Version 5.
> +
>  endchoice # "DWARF version"
>
>  config DEBUG_INFO_BTF
> diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> new file mode 100755
> index 000000000000..c46e2456b47a
> --- /dev/null
> +++ b/scripts/test_dwarf5_support.sh
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +
> +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> +  $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> --
> 2.30.0.365.g02bc693789-goog
>

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Sedat Dilek]

On Sun, Jan 31, 2021 at 12:10 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > DWARF v5 is the latest standard of the DWARF debug info format.
> >
> > Feature detection of DWARF5 is onerous, especially given that we've
> > removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> > support.
> >
> > The DWARF version of a binary can be validated with:
> > $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> > or
> > $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
> >
> > DWARF5 wins significantly in terms of size when mixed with compression
> > (CONFIG_DEBUG_INFO_COMPRESSED).
> >
> > 363M    vmlinux.clang12.dwarf5.compressed
> > 434M    vmlinux.clang12.dwarf4.compressed
> > 439M    vmlinux.clang12.dwarf2.compressed
> > 457M    vmlinux.clang12.dwarf5
> > 536M    vmlinux.clang12.dwarf4
> > 548M    vmlinux.clang12.dwarf2
> >
> > 515M    vmlinux.gcc10.2.dwarf5.compressed
> > 599M    vmlinux.gcc10.2.dwarf4.compressed
> > 624M    vmlinux.gcc10.2.dwarf2.compressed
> > 630M    vmlinux.gcc10.2.dwarf5
> > 765M    vmlinux.gcc10.2.dwarf4
> > 809M    vmlinux.gcc10.2.dwarf2
> >
> > Though the quality of debug info is harder to quantify; size is not a
> > proxy for quality.
> >
> > Jakub notes:
> >   All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> >   option that enabled some small DWARF subset (initially only a few
> >   DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
> >   after DWARF 5 has been finalized) started emitting DWARF5 section
> >   headers and got most of the DWARF5 changes in...
> >
> > Version check GCC so that we don't need to worry about the difference in
> > command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> > validate the DWARF Version in the assembler feature detection script.
> >
> > GNU `as` only recently gained support for specifying -gdwarf-5, so when
> > compiling with Clang but without Clang's integrated assembler
> > (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
> >
> > Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> > recognize the new additions to the DWARF debug info. Thanks to Sedat for
> > the report.
> >
> > Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> > Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
> > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > Suggested-by: Caroline Tice <cmtice@google.com>
> > Suggested-by: Fangrui Song <maskray@google.com>
> > Suggested-by: Jakub Jelinek <jakub@redhat.com>
> > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > ---
> >  Makefile                          |  1 +
> >  include/asm-generic/vmlinux.lds.h |  7 ++++++-
> >  lib/Kconfig.debug                 | 18 ++++++++++++++++++
> >  scripts/test_dwarf5_support.sh    |  8 ++++++++
> >  4 files changed, 33 insertions(+), 1 deletion(-)
> >  create mode 100755 scripts/test_dwarf5_support.sh
> >
> > diff --git a/Makefile b/Makefile
> > index d2b4980807e0..5387a6f2f62d 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -831,6 +831,7 @@ KBUILD_AFLAGS       += -Wa,-gdwarf-2
> >  endif
> >
> >  dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> >  DEBUG_CFLAGS   += -gdwarf-$(dwarf-version-y)
> >
> >  ifdef CONFIG_DEBUG_INFO_REDUCED
> > diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> > index 34b7e0d2346c..1e7cde4bd3f9 100644
> > --- a/include/asm-generic/vmlinux.lds.h
> > +++ b/include/asm-generic/vmlinux.lds.h
> > @@ -842,8 +842,13 @@
> >                 /* DWARF 4 */                                           \
> >                 .debug_types    0 : { *(.debug_types) }                 \
> >                 /* DWARF 5 */                                           \
> > +               .debug_addr     0 : { *(.debug_addr) }                  \
> > +               .debug_line_str 0 : { *(.debug_line_str) }              \
> > +               .debug_loclists 0 : { *(.debug_loclists) }              \
> >                 .debug_macro    0 : { *(.debug_macro) }                 \
> > -               .debug_addr     0 : { *(.debug_addr) }
> > +               .debug_names    0 : { *(.debug_names) }                 \
> > +               .debug_rnglists 0 : { *(.debug_rnglists) }              \
> > +               .debug_str_offsets      0 : { *(.debug_str_offsets) }
> >
>
> I just looked at binutils 2.36 in the Debian/experimental repositories.
>
> [1] says:
>
> + PR ld/27230
> + * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.
>
> ...
>
> -  /* DWARF Extension.  */
> -  .debug_macro    0 : { *(.debug_macro) }
> +  /* DWARF 5.  */
>    .debug_addr     0 : { *(.debug_addr) }
> +  .debug_line_str 0 : { *(.debug_line_str) }
> +  .debug_loclists 0 : { *(.debug_loclists) }
> +  .debug_macro    0 : { *(.debug_macro) }
> +  .debug_names    0 : { *(.debug_names) }
> +  .debug_rnglists 0 : { *(.debug_rnglists) }
> +  .debug_str_offsets 0 : { *(.debug_str_offsets) }
> +  .debug_sup      0 : { *(.debug_sup) }
>
> The list of DWARF-5 .debug_* sections is alphabetically sorted.
> AFAICS .debug_sup section is missing?
>

Here the link to PR ld/27230 ("[PATCH] Add DWARF v5 sections and v4
.debug_types").

- Sedat -

[1] https://sourceware.org/pipermail/binutils/2021-January/115064.html

> - Sedat -
>
> [1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430
>
> >  /* Stabs debugging sections. */
> >  #define STABS_DEBUG                                                    \
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 94c1a7ed6306..ad6f78989d4f 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> >           It makes the debug information larger, but it significantly
> >           improves the success of resolving variables in gdb on optimized code.
> >
> > +config DEBUG_INFO_DWARF5
> > +       bool "Generate DWARF Version 5 debuginfo"
> > +       depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > +       depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> > +       depends on !DEBUG_INFO_BTF
> > +       help
> > +         Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> > +         5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> > +         draft features until 7.0), and gdb 8.0+.
> > +
> > +         Changes to the structure of debug info in Version 5 allow for around
> > +         15-18% savings in resulting image and debug info section sizes as
> > +         compared to DWARF Version 4. DWARF Version 5 standardizes previous
> > +         extensions such as accelerators for symbol indexing and the format
> > +         for fission (.dwo/.dwp) files. Users may not want to select this
> > +         config if they rely on tooling that has not yet been updated to
> > +         support DWARF Version 5.
> > +
> >  endchoice # "DWARF version"
> >
> >  config DEBUG_INFO_BTF
> > diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> > new file mode 100755
> > index 000000000000..c46e2456b47a
> > --- /dev/null
> > +++ b/scripts/test_dwarf5_support.sh
> > @@ -0,0 +1,8 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> > +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> > +  $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> > --
> > 2.30.0.365.g02bc693789-goog
> >

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Fāng-ruì Sòng]

On Sat, Jan 30, 2021 at 3:10 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > DWARF v5 is the latest standard of the DWARF debug info format.
> >
> > Feature detection of DWARF5 is onerous, especially given that we've
> > removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> > support.
> >
> > The DWARF version of a binary can be validated with:
> > $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> > or
> > $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
> >
> > DWARF5 wins significantly in terms of size when mixed with compression
> > (CONFIG_DEBUG_INFO_COMPRESSED).
> >
> > 363M    vmlinux.clang12.dwarf5.compressed
> > 434M    vmlinux.clang12.dwarf4.compressed
> > 439M    vmlinux.clang12.dwarf2.compressed
> > 457M    vmlinux.clang12.dwarf5
> > 536M    vmlinux.clang12.dwarf4
> > 548M    vmlinux.clang12.dwarf2
> >
> > 515M    vmlinux.gcc10.2.dwarf5.compressed
> > 599M    vmlinux.gcc10.2.dwarf4.compressed
> > 624M    vmlinux.gcc10.2.dwarf2.compressed
> > 630M    vmlinux.gcc10.2.dwarf5
> > 765M    vmlinux.gcc10.2.dwarf4
> > 809M    vmlinux.gcc10.2.dwarf2
> >
> > Though the quality of debug info is harder to quantify; size is not a
> > proxy for quality.
> >
> > Jakub notes:
> >   All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> >   option that enabled some small DWARF subset (initially only a few
> >   DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
> >   after DWARF 5 has been finalized) started emitting DWARF5 section
> >   headers and got most of the DWARF5 changes in...
> >
> > Version check GCC so that we don't need to worry about the difference in
> > command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> > validate the DWARF Version in the assembler feature detection script.
> >
> > GNU `as` only recently gained support for specifying -gdwarf-5, so when
> > compiling with Clang but without Clang's integrated assembler
> > (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
> >
> > Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> > recognize the new additions to the DWARF debug info. Thanks to Sedat for
> > the report.
> >
> > Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> > Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
> > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > Suggested-by: Caroline Tice <cmtice@google.com>
> > Suggested-by: Fangrui Song <maskray@google.com>
> > Suggested-by: Jakub Jelinek <jakub@redhat.com>
> > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > ---
> >  Makefile                          |  1 +
> >  include/asm-generic/vmlinux.lds.h |  7 ++++++-
> >  lib/Kconfig.debug                 | 18 ++++++++++++++++++
> >  scripts/test_dwarf5_support.sh    |  8 ++++++++
> >  4 files changed, 33 insertions(+), 1 deletion(-)
> >  create mode 100755 scripts/test_dwarf5_support.sh
> >
> > diff --git a/Makefile b/Makefile
> > index d2b4980807e0..5387a6f2f62d 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -831,6 +831,7 @@ KBUILD_AFLAGS       += -Wa,-gdwarf-2
> >  endif
> >
> >  dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> >  DEBUG_CFLAGS   += -gdwarf-$(dwarf-version-y)
> >
> >  ifdef CONFIG_DEBUG_INFO_REDUCED
> > diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> > index 34b7e0d2346c..1e7cde4bd3f9 100644
> > --- a/include/asm-generic/vmlinux.lds.h
> > +++ b/include/asm-generic/vmlinux.lds.h
> > @@ -842,8 +842,13 @@
> >                 /* DWARF 4 */                                           \
> >                 .debug_types    0 : { *(.debug_types) }                 \
> >                 /* DWARF 5 */                                           \
> > +               .debug_addr     0 : { *(.debug_addr) }                  \
> > +               .debug_line_str 0 : { *(.debug_line_str) }              \
> > +               .debug_loclists 0 : { *(.debug_loclists) }              \
> >                 .debug_macro    0 : { *(.debug_macro) }                 \
> > -               .debug_addr     0 : { *(.debug_addr) }
> > +               .debug_names    0 : { *(.debug_names) }                 \
> > +               .debug_rnglists 0 : { *(.debug_rnglists) }              \
> > +               .debug_str_offsets      0 : { *(.debug_str_offsets) }
> >
>
> I just looked at binutils 2.36 in the Debian/experimental repositories.
>
> [1] says:
>
> + PR ld/27230
> + * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.
>
> ...
>
> -  /* DWARF Extension.  */
> -  .debug_macro    0 : { *(.debug_macro) }
> +  /* DWARF 5.  */
>    .debug_addr     0 : { *(.debug_addr) }
> +  .debug_line_str 0 : { *(.debug_line_str) }
> +  .debug_loclists 0 : { *(.debug_loclists) }
> +  .debug_macro    0 : { *(.debug_macro) }
> +  .debug_names    0 : { *(.debug_names) }
> +  .debug_rnglists 0 : { *(.debug_rnglists) }
> +  .debug_str_offsets 0 : { *(.debug_str_offsets) }
> +  .debug_sup      0 : { *(.debug_sup) }
>
> The list of DWARF-5 .debug_* sections is alphabetically sorted.
> AFAICS .debug_sup section is missing?
>
> - Sedat -

No compiler produces .debug_sup section. It could be from some
post-processing tool which is unrelated to the linker.
Omitting it is fine.

> [1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430
>
> >  /* Stabs debugging sections. */
> >  #define STABS_DEBUG                                                    \
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 94c1a7ed6306..ad6f78989d4f 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> >           It makes the debug information larger, but it significantly
> >           improves the success of resolving variables in gdb on optimized code.
> >
> > +config DEBUG_INFO_DWARF5
> > +       bool "Generate DWARF Version 5 debuginfo"
> > +       depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > +       depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> > +       depends on !DEBUG_INFO_BTF
> > +       help
> > +         Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> > +         5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> > +         draft features until 7.0), and gdb 8.0+.
> > +
> > +         Changes to the structure of debug info in Version 5 allow for around
> > +         15-18% savings in resulting image and debug info section sizes as
> > +         compared to DWARF Version 4. DWARF Version 5 standardizes previous
> > +         extensions such as accelerators for symbol indexing and the format
> > +         for fission (.dwo/.dwp) files. Users may not want to select this
> > +         config if they rely on tooling that has not yet been updated to
> > +         support DWARF Version 5.
> > +
> >  endchoice # "DWARF version"
> >
> >  config DEBUG_INFO_BTF
> > diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> > new file mode 100755
> > index 000000000000..c46e2456b47a
> > --- /dev/null
> > +++ b/scripts/test_dwarf5_support.sh
> > @@ -0,0 +1,8 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> > +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> > +  $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> > --
> > 2.30.0.365.g02bc693789-goog
> >

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Sedat Dilek]

On Sun, Jan 31, 2021 at 1:37 AM Fāng-ruì Sòng <maskray@google.com> wrote:
>
> On Sat, Jan 30, 2021 at 3:10 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
> > <ndesaulniers@google.com> wrote:
> > >
> > > DWARF v5 is the latest standard of the DWARF debug info format.
> > >
> > > Feature detection of DWARF5 is onerous, especially given that we've
> > > removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> > > support.
> > >
> > > The DWARF version of a binary can be validated with:
> > > $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> > > or
> > > $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
> > >
> > > DWARF5 wins significantly in terms of size when mixed with compression
> > > (CONFIG_DEBUG_INFO_COMPRESSED).
> > >
> > > 363M    vmlinux.clang12.dwarf5.compressed
> > > 434M    vmlinux.clang12.dwarf4.compressed
> > > 439M    vmlinux.clang12.dwarf2.compressed
> > > 457M    vmlinux.clang12.dwarf5
> > > 536M    vmlinux.clang12.dwarf4
> > > 548M    vmlinux.clang12.dwarf2
> > >
> > > 515M    vmlinux.gcc10.2.dwarf5.compressed
> > > 599M    vmlinux.gcc10.2.dwarf4.compressed
> > > 624M    vmlinux.gcc10.2.dwarf2.compressed
> > > 630M    vmlinux.gcc10.2.dwarf5
> > > 765M    vmlinux.gcc10.2.dwarf4
> > > 809M    vmlinux.gcc10.2.dwarf2
> > >
> > > Though the quality of debug info is harder to quantify; size is not a
> > > proxy for quality.
> > >
> > > Jakub notes:
> > >   All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
> > >   option that enabled some small DWARF subset (initially only a few
> > >   DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
> > >   after DWARF 5 has been finalized) started emitting DWARF5 section
> > >   headers and got most of the DWARF5 changes in...
> > >
> > > Version check GCC so that we don't need to worry about the difference in
> > > command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> > > validate the DWARF Version in the assembler feature detection script.
> > >
> > > GNU `as` only recently gained support for specifying -gdwarf-5, so when
> > > compiling with Clang but without Clang's integrated assembler
> > > (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
> > >
> > > Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> > > recognize the new additions to the DWARF debug info. Thanks to Sedat for
> > > the report.
> > >
> > > Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> > > Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
> > > Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> > > Suggested-by: Caroline Tice <cmtice@google.com>
> > > Suggested-by: Fangrui Song <maskray@google.com>
> > > Suggested-by: Jakub Jelinek <jakub@redhat.com>
> > > Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> > > Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> > > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> > > ---
> > >  Makefile                          |  1 +
> > >  include/asm-generic/vmlinux.lds.h |  7 ++++++-
> > >  lib/Kconfig.debug                 | 18 ++++++++++++++++++
> > >  scripts/test_dwarf5_support.sh    |  8 ++++++++
> > >  4 files changed, 33 insertions(+), 1 deletion(-)
> > >  create mode 100755 scripts/test_dwarf5_support.sh
> > >
> > > diff --git a/Makefile b/Makefile
> > > index d2b4980807e0..5387a6f2f62d 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -831,6 +831,7 @@ KBUILD_AFLAGS       += -Wa,-gdwarf-2
> > >  endif
> > >
> > >  dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> > > +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
> > >  DEBUG_CFLAGS   += -gdwarf-$(dwarf-version-y)
> > >
> > >  ifdef CONFIG_DEBUG_INFO_REDUCED
> > > diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> > > index 34b7e0d2346c..1e7cde4bd3f9 100644
> > > --- a/include/asm-generic/vmlinux.lds.h
> > > +++ b/include/asm-generic/vmlinux.lds.h
> > > @@ -842,8 +842,13 @@
> > >                 /* DWARF 4 */                                           \
> > >                 .debug_types    0 : { *(.debug_types) }                 \
> > >                 /* DWARF 5 */                                           \
> > > +               .debug_addr     0 : { *(.debug_addr) }                  \
> > > +               .debug_line_str 0 : { *(.debug_line_str) }              \
> > > +               .debug_loclists 0 : { *(.debug_loclists) }              \
> > >                 .debug_macro    0 : { *(.debug_macro) }                 \
> > > -               .debug_addr     0 : { *(.debug_addr) }
> > > +               .debug_names    0 : { *(.debug_names) }                 \
> > > +               .debug_rnglists 0 : { *(.debug_rnglists) }              \
> > > +               .debug_str_offsets      0 : { *(.debug_str_offsets) }
> > >
> >
> > I just looked at binutils 2.36 in the Debian/experimental repositories.
> >
> > [1] says:
> >
> > + PR ld/27230
> > + * scripttempl/DWARF.sc: Add DWARF-5 .debug_* sections.
> >
> > ...
> >
> > -  /* DWARF Extension.  */
> > -  .debug_macro    0 : { *(.debug_macro) }
> > +  /* DWARF 5.  */
> >    .debug_addr     0 : { *(.debug_addr) }
> > +  .debug_line_str 0 : { *(.debug_line_str) }
> > +  .debug_loclists 0 : { *(.debug_loclists) }
> > +  .debug_macro    0 : { *(.debug_macro) }
> > +  .debug_names    0 : { *(.debug_names) }
> > +  .debug_rnglists 0 : { *(.debug_rnglists) }
> > +  .debug_str_offsets 0 : { *(.debug_str_offsets) }
> > +  .debug_sup      0 : { *(.debug_sup) }
> >
> > The list of DWARF-5 .debug_* sections is alphabetically sorted.
> > AFAICS .debug_sup section is missing?
> >
> > - Sedat -
>
> No compiler produces .debug_sup section. It could be from some
> post-processing tool which is unrelated to the linker.
> Omitting it is fine.
>

Thanks for the clarification, Fangrui.

- Sedat -

> > [1] https://salsa.debian.org/toolchain-team/binutils/-/commit/f58f33081035672b01a04326a9c8daadbd09a430
> >
> > >  /* Stabs debugging sections. */
> > >  #define STABS_DEBUG                                                    \
> > > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > > index 94c1a7ed6306..ad6f78989d4f 100644
> > > --- a/lib/Kconfig.debug
> > > +++ b/lib/Kconfig.debug
> > > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> > >           It makes the debug information larger, but it significantly
> > >           improves the success of resolving variables in gdb on optimized code.
> > >
> > > +config DEBUG_INFO_DWARF5
> > > +       bool "Generate DWARF Version 5 debuginfo"
> > > +       depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > > +       depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> > > +       depends on !DEBUG_INFO_BTF
> > > +       help
> > > +         Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> > > +         5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> > > +         draft features until 7.0), and gdb 8.0+.
> > > +
> > > +         Changes to the structure of debug info in Version 5 allow for around
> > > +         15-18% savings in resulting image and debug info section sizes as
> > > +         compared to DWARF Version 4. DWARF Version 5 standardizes previous
> > > +         extensions such as accelerators for symbol indexing and the format
> > > +         for fission (.dwo/.dwp) files. Users may not want to select this
> > > +         config if they rely on tooling that has not yet been updated to
> > > +         support DWARF Version 5.
> > > +
> > >  endchoice # "DWARF version"
> > >
> > >  config DEBUG_INFO_BTF
> > > diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> > > new file mode 100755
> > > index 000000000000..c46e2456b47a
> > > --- /dev/null
> > > +++ b/scripts/test_dwarf5_support.sh
> > > @@ -0,0 +1,8 @@
> > > +#!/bin/sh
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> > > +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> > > +  $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> > > --
> > > 2.30.0.365.g02bc693789-goog
> > >

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Masahiro Yamada]

On Sat, Jan 30, 2021 at 9:44 AM 'Nick Desaulniers' via Clang Built
Linux <clang-built-linux@googlegroups.com> wrote:
>
> DWARF v5 is the latest standard of the DWARF debug info format.
>
> Feature detection of DWARF5 is onerous, especially given that we've
> removed $(AS), so we must query $(CC) for DWARF5 assembler directive
> support.
>
> The DWARF version of a binary can be validated with:
> $ llvm-dwarfdump vmlinux | head -n 4 | grep version
> or
> $ readelf --debug-dump=info vmlinux 2>/dev/null | grep Version
>
> DWARF5 wins significantly in terms of size when mixed with compression
> (CONFIG_DEBUG_INFO_COMPRESSED).
>
> 363M    vmlinux.clang12.dwarf5.compressed
> 434M    vmlinux.clang12.dwarf4.compressed
> 439M    vmlinux.clang12.dwarf2.compressed
> 457M    vmlinux.clang12.dwarf5
> 536M    vmlinux.clang12.dwarf4
> 548M    vmlinux.clang12.dwarf2
>
> 515M    vmlinux.gcc10.2.dwarf5.compressed
> 599M    vmlinux.gcc10.2.dwarf4.compressed
> 624M    vmlinux.gcc10.2.dwarf2.compressed
> 630M    vmlinux.gcc10.2.dwarf5
> 765M    vmlinux.gcc10.2.dwarf4
> 809M    vmlinux.gcc10.2.dwarf2
>
> Though the quality of debug info is harder to quantify; size is not a
> proxy for quality.
>
> Jakub notes:
>   All [GCC] 5.1 - 6.x did was start accepting -gdwarf-5 as experimental
>   option that enabled some small DWARF subset (initially only a few
>   DW_LANG_* codes newly added to DWARF5 drafts).  Only GCC 7 (released
>   after DWARF 5 has been finalized) started emitting DWARF5 section
>   headers and got most of the DWARF5 changes in...
>
> Version check GCC so that we don't need to worry about the difference in
> command line args between GNU readelf and llvm-readelf/llvm-dwarfdump to
> validate the DWARF Version in the assembler feature detection script.
>
> GNU `as` only recently gained support for specifying -gdwarf-5, so when
> compiling with Clang but without Clang's integrated assembler
> (LLVM_IAS=1 is not set), explicitly add -Wa,-gdwarf-5 to DEBUG_CFLAGS.
>
> Disabled for now if CONFIG_DEBUG_INFO_BTF is set; pahole doesn't yet
> recognize the new additions to the DWARF debug info. Thanks to Sedat for
> the report.
>
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
> Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
> Suggested-by: Arvind Sankar <nivedita@alum.mit.edu>
> Suggested-by: Caroline Tice <cmtice@google.com>
> Suggested-by: Fangrui Song <maskray@google.com>
> Suggested-by: Jakub Jelinek <jakub@redhat.com>
> Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
> Suggested-by: Nathan Chancellor <natechancellor@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
>  Makefile                          |  1 +
>  include/asm-generic/vmlinux.lds.h |  7 ++++++-
>  lib/Kconfig.debug                 | 18 ++++++++++++++++++
>  scripts/test_dwarf5_support.sh    |  8 ++++++++
>  4 files changed, 33 insertions(+), 1 deletion(-)
>  create mode 100755 scripts/test_dwarf5_support.sh




Nick, the patch set is getting simpler and simpler,
and almost good enough to be merged.


Please let me ask two questions below.

There has been a lot of discussion, and
I might have missed the context.





> diff --git a/Makefile b/Makefile
> index d2b4980807e0..5387a6f2f62d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -831,6 +831,7 @@ KBUILD_AFLAGS       += -Wa,-gdwarf-2
>  endif
>
>  dwarf-version-$(CONFIG_DEBUG_INFO_DWARF4) := 4
> +dwarf-version-$(CONFIG_DEBUG_INFO_DWARF5) := 5
>  DEBUG_CFLAGS   += -gdwarf-$(dwarf-version-y)
>
>  ifdef CONFIG_DEBUG_INFO_REDUCED
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 34b7e0d2346c..1e7cde4bd3f9 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -842,8 +842,13 @@
>                 /* DWARF 4 */                                           \
>                 .debug_types    0 : { *(.debug_types) }                 \
>                 /* DWARF 5 */                                           \
> +               .debug_addr     0 : { *(.debug_addr) }                  \
> +               .debug_line_str 0 : { *(.debug_line_str) }              \
> +               .debug_loclists 0 : { *(.debug_loclists) }              \
>                 .debug_macro    0 : { *(.debug_macro) }                 \
> -               .debug_addr     0 : { *(.debug_addr) }
> +               .debug_names    0 : { *(.debug_names) }                 \
> +               .debug_rnglists 0 : { *(.debug_rnglists) }              \
> +               .debug_str_offsets      0 : { *(.debug_str_offsets) }
>
>  /* Stabs debugging sections. */
>  #define STABS_DEBUG                                                    \
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 94c1a7ed6306..ad6f78989d4f 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
>           It makes the debug information larger, but it significantly
>           improves the success of resolving variables in gdb on optimized code.
>
> +config DEBUG_INFO_DWARF5
> +       bool "Generate DWARF Version 5 debuginfo"
> +       depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> +       depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))

Q1.

This  "CC_IS_GCC ||" was introduced by v4.

GCC never outputs '.file 0', which is why
this test is only needed for Clang, correct?






> +       depends on !DEBUG_INFO_BTF
> +       help
> +         Generate DWARF v5 debug info. Requires binutils 2.35.2, gcc 5.0+ (gcc
> +         5.0+ accepts the -gdwarf-5 flag but only had partial support for some
> +         draft features until 7.0), and gdb 8.0+.
> +
> +         Changes to the structure of debug info in Version 5 allow for around
> +         15-18% savings in resulting image and debug info section sizes as
> +         compared to DWARF Version 4. DWARF Version 5 standardizes previous
> +         extensions such as accelerators for symbol indexing and the format
> +         for fission (.dwo/.dwp) files. Users may not want to select this
> +         config if they rely on tooling that has not yet been updated to
> +         support DWARF Version 5.
> +
>  endchoice # "DWARF version"
>
>  config DEBUG_INFO_BTF
> diff --git a/scripts/test_dwarf5_support.sh b/scripts/test_dwarf5_support.sh
> new file mode 100755
> index 000000000000..c46e2456b47a
> --- /dev/null
> +++ b/scripts/test_dwarf5_support.sh
> @@ -0,0 +1,8 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +
> +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611


I saw the following links in v6.

https://sourceware.org/bugzilla/show_bug.cgi?id=25612
https://sourceware.org/bugzilla/show_bug.cgi?id=25614

They were dropped in v7. Why?

I just thought they were good to know...



> +echo '.file 0 "filename" md5 0x7a0b65214090b6693bd1dc24dd248245' | \
> +  $* -gdwarf-5 -Wno-unused-command-line-argument -c -x assembler -o /dev/null -
> --
> 2.30.0.365.g02bc693789-goog
>
> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20210130004401.2528717-3-ndesaulniers%40google.com.



-- 
Best Regards
Masahiro Yamada

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Nick Desaulniers]

On Wed, Feb 3, 2021 at 3:07 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Nick, the patch set is getting simpler and simpler,
> and almost good enough to be merged.

I agree.  I think Sedat pointed out a binutils 2.35.2 release; thanks
to Nick Clifton for that.

>
>
> Please let me ask two questions below.
>
> There has been a lot of discussion, and
> I might have missed the context.
>
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> >           It makes the debug information larger, but it significantly
> >           improves the success of resolving variables in gdb on optimized code.
> >
> > +config DEBUG_INFO_DWARF5
> > +       bool "Generate DWARF Version 5 debuginfo"
> > +       depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > +       depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
>
> Q1.
>
> This  "CC_IS_GCC ||" was introduced by v4.
>
> GCC never outputs '.file 0', which is why
> this test is only needed for Clang, correct?

This test script is only needed when compiling with clang but without
its integrated assembler.  It checks that when clang is used as the
driver, but GAS is used as the assembler, that GAS will be able to
decode the DWARF v5 assembler additions Clang will produce without
needing an explicit -Wa,-gdwarf-5 flag passed.

Technically, it is unnecessary for `LLVM=1 LLVM_IAS=1` or `CC=clang
LLVM_IAS=1` (ie. clang+clang's integrated assembler).  But there is no
way to express AS_IS_IAS today in KConfig (similar to
CC_IS_{GCC|CLANG} or LD_IS_LLD).  I don't think that's necessary;
whether or not clang's integrated assembler is used, when using clang,
run the simple check.

> > --- /dev/null
> > +++ b/scripts/test_dwarf5_support.sh
> > @@ -0,0 +1,8 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0
> > +
> > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
>
>
> I saw the following links in v6.
>
> https://sourceware.org/bugzilla/show_bug.cgi?id=25612
> https://sourceware.org/bugzilla/show_bug.cgi?id=25614
>
> They were dropped in v7. Why?
>
> I just thought they were good to know...

While having fixes for those bugs is required, technically
https://sourceware.org/bugzilla/show_bug.cgi?id=25611 is the latest
bug which was fixed.  Testing for a fix of
https://sourceware.org/bugzilla/show_bug.cgi?id=25611 implies that
fixes for 25612 and 25614 exist due to the order they were fixed in
GAS.  Technically, you could argue that this script is quite GAS
centric; given an arbitrary "assembler" the test should check a few
things.  Realistically, I think that's overkill based on what
assemblers are in use today; we can always grow the script should we
identify other tests additional assemblers may need to pass, but until
then, I suspect YAGNI.  Maybe there's a more precise name for the
script to reflect that, but that gets close to "what color shall we
paint the bikeshed?"  Given the number of folks on the thread, plz no.
-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Masahiro Yamada]

On Thu, Feb 4, 2021 at 8:27 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Wed, Feb 3, 2021 at 3:07 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > Nick, the patch set is getting simpler and simpler,
> > and almost good enough to be merged.
>
> I agree.  I think Sedat pointed out a binutils 2.35.2 release; thanks
> to Nick Clifton for that.
>
> >
> >
> > Please let me ask two questions below.
> >
> > There has been a lot of discussion, and
> > I might have missed the context.
> >
> > > --- a/lib/Kconfig.debug
> > > +++ b/lib/Kconfig.debug
> > > @@ -268,6 +268,24 @@ config DEBUG_INFO_DWARF4
> > >           It makes the debug information larger, but it significantly
> > >           improves the success of resolving variables in gdb on optimized code.
> > >
> > > +config DEBUG_INFO_DWARF5
> > > +       bool "Generate DWARF Version 5 debuginfo"
> > > +       depends on GCC_VERSION >= 50000 || CC_IS_CLANG
> > > +       depends on CC_IS_GCC || $(success,$(srctree)/scripts/test_dwarf5_support.sh $(CC) $(CLANG_FLAGS))
> >
> > Q1.
> >
> > This  "CC_IS_GCC ||" was introduced by v4.
> >
> > GCC never outputs '.file 0', which is why
> > this test is only needed for Clang, correct?
>
> This test script is only needed when compiling with clang but without
> its integrated assembler.  It checks that when clang is used as the
> driver, but GAS is used as the assembler, that GAS will be able to
> decode the DWARF v5 assembler additions Clang will produce without
> needing an explicit -Wa,-gdwarf-5 flag passed.
>
> Technically, it is unnecessary for `LLVM=1 LLVM_IAS=1` or `CC=clang
> LLVM_IAS=1` (ie. clang+clang's integrated assembler).  But there is no
> way to express AS_IS_IAS today in KConfig (similar to
> CC_IS_{GCC|CLANG} or LD_IS_LLD).  I don't think that's necessary;
> whether or not clang's integrated assembler is used, when using clang,
> run the simple check.
>
> > > --- /dev/null
> > > +++ b/scripts/test_dwarf5_support.sh
> > > @@ -0,0 +1,8 @@
> > > +#!/bin/sh
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +
> > > +# Test that the assembler doesn't need -Wa,-gdwarf-5 when presented with DWARF
> > > +# v5 input, such as `.file 0` and `md5 0x00`. Should be fixed in GNU binutils
> > > +# 2.35.2. https://sourceware.org/bugzilla/show_bug.cgi?id=25611
> >
> >
> > I saw the following links in v6.
> >
> > https://sourceware.org/bugzilla/show_bug.cgi?id=25612
> > https://sourceware.org/bugzilla/show_bug.cgi?id=25614
> >
> > They were dropped in v7. Why?
> >
> > I just thought they were good to know...
>
> While having fixes for those bugs is required, technically
> https://sourceware.org/bugzilla/show_bug.cgi?id=25611 is the latest
> bug which was fixed.  Testing for a fix of
> https://sourceware.org/bugzilla/show_bug.cgi?id=25611 implies that
> fixes for 25612 and 25614 exist due to the order they were fixed in
> GAS.


It is difficult to know the patch order in the binutils project.

Personally, I prefer having all the three references here.
Otherwise, it is difficult to understand why
this script is doing such complex checks.





> Technically, you could argue that this script is quite GAS
> centric; given an arbitrary "assembler" the test should check a few
> things.  Realistically, I think that's overkill based on what
> assemblers are in use today; we can always grow the script should we
> identify other tests additional assemblers may need to pass, but until
> then, I suspect YAGNI.  Maybe there's a more precise name for the
> script to reflect that, but that gets close to "what color shall we
> paint the bikeshed?"  Given the number of folks on the thread, plz no.


No argument with this regard. I agree with you.




> --
> Thanks,
> ~Nick Desaulniers



-- 
Best Regards
Masahiro Yamada

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Jakub Jelinek]

On Thu, Feb 04, 2021 at 08:06:12AM +0900, Masahiro Yamada wrote:
> GCC never outputs '.file 0', which is why
> this test is only needed for Clang, correct?

No, GCC outputs .file 0 if it during configure time detected assembler that
supports it and doesn't have any of the known bugs related to it.
But that means kernel doesn't need to care because GCC already took care of
that.

	Jakub

Re: [PATCH v7 2/2] Kbuild: implement support for DWARF v5

[Masahiro Yamada]

On Thu, Feb 4, 2021 at 8:36 AM Jakub Jelinek <jakub@redhat.com> wrote:
>
> On Thu, Feb 04, 2021 at 08:06:12AM +0900, Masahiro Yamada wrote:
> > GCC never outputs '.file 0', which is why
> > this test is only needed for Clang, correct?
>
> No, GCC outputs .file 0 if it during configure time detected assembler that
> supports it and doesn't have any of the known bugs related to it.
> But that means kernel doesn't need to care because GCC already took care of
> that.


OK, then I am fine with the current code.

Thanks for the clarification.







--
Best Regards
Masahiro Yamada

Re: [PATCH v7 0/2] Kbuild: DWARF v5 support

[Sedat Dilek]

On Sat, Jan 30, 2021 at 1:44 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> DWARF v5 is the latest standard of the DWARF debug info format.
>
> DWARF5 wins significantly in terms of size and especially so when mixed
> with compression (CONFIG_DEBUG_INFO_COMPRESSED).
>
> Link: http://www.dwarfstd.org/doc/DWARF5.pdf
>
> Patch 1 is a cleanup that lays the ground work and isn't DWARF
> v5 specific.
> Patch 2 implements Kconfig and Kbuild support for DWARFv5.
>
> Changes from v6:
> * Reorder sections from linker script to match order from BFD's internal
>   linker script.
> * Add .debug_names section, as per Fangrui.
> * Drop CONFIG_DEBUG_INFO_DWARF2. Patch 0001 becomes a menu with 1
>   choice. GCC's implicit default version of DWARF has been DWARF v4
>   since ~4.8.
> * Modify the test script to check for the presence of
>   https://sourceware.org/bugzilla/show_bug.cgi?id=27195.

Just as a note: GNU Binutils 2.35.2 Release is now available

- Sedat -

[1] https://sourceware.org/pipermail/binutils/2021-January/115150.html

> * Drop the clang without integrated assembler block in
>   0002. Bumps the version requirement for GAS to 2.35.2, which isn't
>   released yet (but should be released soon).  Folks looking to test
>   with clang but without the integrated assembler should fetch
>   binutils-gdb, build it from source, add a symlink to
>   binutils-gdb/gas/as-new to binutils-gdb/gas/as, then prefix
>   binutils-gdb/gas/as to their $PATH when building the kernel.
>
> Changes from v5:
> * Drop previous patch 1, it has been accepted into kbuild:
>   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=kbuild&id=3f4d8ce271c7082be75bacbcbd2048aa78ce2b44
> * Trying to set -Wa,-gdwarf-4 in the earlier patch was the source of
>   additional complexity. Drop it that part of the patch. We can revisit
>   clang without the integrated assembler setting -Wa,-gdwarf-4 later.
>   That is a separate problem from generally supporting DWARF v5.
> * Rework the final patch for clang without the integrated assembler.
>   -Wa,-gdwarf-5 is required for DWARF5 in that case otherwise GAS will
>   not accept the assembler directives clang produces from C code when
>   generating asm.
>
> Changes from v4:
> * drop set -e from script as per Nathan.
> * add dependency on !CONFIG_DEBUG_INFO_BTF for DWARF v5 as per Sedat.
> * Move LLVM_IAS=1 complexity from patch 2 to patch 3 as per Arvind and
>   Masahiro. Sorry it took me a few tries to understand the point (I
>   might still not), but it looks much cleaner this way. Sorry Nathan, I
>   did not carry forward your previous reviews as a result, but I would
>   appreciate if you could look again.
> * Add Nathan's reviewed by tag to patch 1.
> * Reword commit message for patch 3 to mention LLVM_IAS=1 and -gdwarf-5
>   binutils addition later, and BTF issue.
> * I still happen to see a pahole related error spew for the combination
>   of:
>   * LLVM=1
>   * LLVM_IAS=1
>   * CONFIG_DEBUG_INFO_DWARF4
>   * CONFIG_DEBUG_INFO_BTF
>   Though they're non-fatal to the build. I'm not sure yet why removing
>   any one of the above prevents the warning spew. Maybe we'll need a v6.
>
> Changes from v3:
>
> Changes as per Arvind:
> * only add -Wa,-gdwarf-5 for (LLVM=1|CC=clang)+LLVM_IAS=0 builds.
> * add -gdwarf-5 to Kconfig shell script.
> * only run Kconfig shell script for Clang.
>
> Apologies to Sedat and Nathan; I appreciate previous testing/review, but
> I did no carry forward your Tested-by and Reviewed-by tags, as the
> patches have changed too much IMO.
>
> Changes from v2:
> * Drop two of the earlier patches that have been accepted already.
> * Add measurements with GCC 10.2 to commit message.
> * Update help text as per Arvind with help from Caroline.
> * Improve case/wording between DWARF Versions as per Masahiro.
>
> Changes from the RFC:
> * split patch in 3 patch series, include Fangrui's patch, too.
> * prefer `DWARF vX` format, as per Fangrui.
> * use spaces between assignment in Makefile as per Masahiro.
> * simplify setting dwarf-version-y as per Masahiro.
> * indent `prompt` in Kconfig change as per Masahiro.
> * remove explicit default in Kconfig as per Masahiro.
> * add comments to test_dwarf5_support.sh.
> * change echo in test_dwarf5_support.sh as per Masahiro.
> * remove -u from test_dwarf5_support.sh as per Masahiro.
> * add a -gdwarf-5 cc-option check to Kconfig as per Jakub.
>
> Nick Desaulniers (2):
>   Kbuild: make DWARF version a choice
>   Kbuild: implement support for DWARF v5
>
>  Makefile                          |  6 +++---
>  include/asm-generic/vmlinux.lds.h |  7 +++++-
>  lib/Kconfig.debug                 | 36 +++++++++++++++++++++++++------
>  scripts/test_dwarf5_support.sh    |  8 +++++++
>  4 files changed, 47 insertions(+), 10 deletions(-)
>  create mode 100755 scripts/test_dwarf5_support.sh
>
> --
> 2.30.0.365.g02bc693789-goog
>