[PATCH v2 0/5] kernel hacking: GCC optimization for debug experience (-Og)

[PATCH v2 0/5] kernel hacking: GCC optimization for debug experience (-Og)

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

Hi all,
I know some kernel developers was searching for a method to dissable GCC
optimizations, probably they want to apply GCC '-O0' option. But since Linux
kernel replys on GCC optimization to remove some dead code, so '-O0' just
breaks the build. They do need this because they want to debug kernel with
qemu, simics, kgtp or kgdb.

Thanks for the GCC '-Og' optimization level introduced in GCC 4.8, which
offers a reasonable level of optimization while maintaining fast compilation
and a good debugging experience. It is similar to '-O1' while perfer keeping
debug ability over runtime speed. With '-Og', we can build a kernel with
better debug ability and little performance drop after some simple change.

In this series, firstly introduce a new config CONFIG_NO_AUTO_INLINE after two
fixes for this new option. With this option, only functions explicitly marked
with "inline" will  be inlined. This will allow the function tracer to trace
more functions because it only traces functions that the compiler has not
inlined.

Then introduce new config CONFIG_DEBUG_EXPERIENCE which apply '-Og'
optimization level for whole kernel, with a simple fix in fix_to_virt().
Currently this option is only tested on a QEMU gust and it works fine.


Comparison of vmlinux size: a bit smaller.

    w/o CONFIG_DEBUG_EXPERIENCE
    $ size vmlinux
       text    data     bss     dec     hex filename
    22665554   9709674  2920908 35296136        21a9388 vmlinux

    w/ CONFIG_DEBUG_EXPERIENCE
    $ size vmlinux
       text    data     bss     dec     hex filename
    21499032   10102758 2920908 34522698        20ec64a vmlinux


Comparison of system performance: a bit drop (~6%).
    This benchmark of kernel compilation is suggested by Ingo Molnar.
    https://lkml.org/lkml/2018/5/2/74

    Preparation: Set cpufreq to 'performance'.
    for ((cpu=0; cpu<120; cpu++)); do
      G=/sys/devices/system/cpu/cpu$cpu/cpufreq/scaling_governor
      [ -f $G ] && echo performance > $G
    done

    w/o CONFIG_DEBUG_EXPERIENCE
    $ perf stat --repeat 5 --null --pre                 '\
        cp -a kernel ../kernel.copy.$(date +%s);         \
        rm -rf *;                                        \
        git checkout .;                                  \
        echo 1 > /proc/sys/vm/drop_caches;               \
        find ../kernel* -type f | xargs cat >/dev/null;  \
        make -j kernel >/dev/null;                       \
        make clean >/dev/null 2>&1;                      \
        sync                                            '\
                                                         \
        make -j8 >/dev/null

     Performance counter stats for 'make -j8' (5 runs):

        219.764246652 seconds time elapsed                   ( +-  0.78% )

    w/ CONFIG_DEBUG_EXPERIENCE
    $ perf stat --repeat 5 --null --pre                 '\
        cp -a kernel ../kernel.copy.$(date +%s);         \
        rm -rf *;                                        \
        git checkout .;                                  \
        echo 1 > /proc/sys/vm/drop_caches;               \
        find ../kernel* -type f | xargs cat >/dev/null;  \
        make -j kernel >/dev/null;                       \
        make clean >/dev/null 2>&1;                      \
        sync                                            '\
                                                         \
        make -j8 >/dev/null

    Performance counter stats for 'make -j8' (5 runs):

         233.574187771 seconds time elapsed                  ( +-  0.19% )

Changbin Du (5):
  x86/mm: surround level4_kernel_pgt with #ifdef
    CONFIG_X86_5LEVEL...#endif
  regulator: add dummy function of_find_regulator_by_node
  kernel hacking: new config NO_AUTO_INLINE to disable compiler
    auto-inline optimizations
  kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og
    optimization
  asm-generic: fix build error in fix_to_virt with
    CONFIG_DEBUG_EXPERIENCE

 Makefile                          | 10 ++++++++++
 arch/x86/include/asm/pgtable_64.h |  2 ++
 arch/x86/kernel/head64.c          | 13 ++++++-------
 drivers/regulator/internal.h      |  9 +++++++--
 include/asm-generic/fixmap.h      |  3 ++-
 include/linux/compiler-gcc.h      |  2 +-
 include/linux/compiler.h          |  2 +-
 lib/Kconfig.debug                 | 39 +++++++++++++++++++++++++++++++++++++++
 8 files changed, 68 insertions(+), 12 deletions(-)

-- 
2.7.4

[PATCH v2 1/5] x86/mm: surround level4_kernel_pgt with #ifdef CONFIG_X86_5LEVEL...#endif

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

The level4_kernel_pgt is only defined when X86_5LEVEL is enabled. So
surround level4_kernel_pgt with #ifdef CONFIG_X86_5LEVEL...#endif to
make code correct.

Signed-off-by: Changbin Du <changbin.du@intel.com>
---
 arch/x86/include/asm/pgtable_64.h |  2 ++
 arch/x86/kernel/head64.c          | 13 ++++++-------
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/pgtable_64.h b/arch/x86/include/asm/pgtable_64.h
index 877bc27..9e7f667 100644
--- a/arch/x86/include/asm/pgtable_64.h
+++ b/arch/x86/include/asm/pgtable_64.h
@@ -15,7 +15,9 @@
 #include <linux/bitops.h>
 #include <linux/threads.h>
 
+#ifdef CONFIG_X86_5LEVEL
 extern p4d_t level4_kernel_pgt[512];
+#endif
 extern p4d_t level4_ident_pgt[512];
 extern pud_t level3_kernel_pgt[512];
 extern pud_t level3_ident_pgt[512];
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 0c408f8..775d7a6 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -143,16 +143,15 @@ unsigned long __head __startup_64(unsigned long physaddr,
 
 	pgd = fixup_pointer(&early_top_pgt, physaddr);
 	p = pgd + pgd_index(__START_KERNEL_map);
-	if (la57)
-		*p = (unsigned long)level4_kernel_pgt;
-	else
-		*p = (unsigned long)level3_kernel_pgt;
-	*p += _PAGE_TABLE_NOENC - __START_KERNEL_map + load_delta;
-
+#ifdef CONFIG_X86_5LEVEL
 	if (la57) {
+		*p = (unsigned long)level4_kernel_pgt;
 		p4d = fixup_pointer(&level4_kernel_pgt, physaddr);
 		p4d[511] += load_delta;
-	}
+	} else
+#endif
+		*p = (unsigned long)level3_kernel_pgt;
+	*p += _PAGE_TABLE_NOENC - __START_KERNEL_map + load_delta;
 
 	pud = fixup_pointer(&level3_kernel_pgt, physaddr);
 	pud[510] += load_delta;
-- 
2.7.4

[PATCH v2 2/5] regulator: add dummy function of_find_regulator_by_node

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

If device tree is not enabled, of_find_regulator_by_node() should have
a dummy function since the function call is still there.

This is to fix build error after CONFIG_NO_AUTO_INLINE is introduced.
If this option is enabled, GCC will not auto-inline functions that are
not explicitly marked as inline.

In this case (no CONFIG_OF), the copmiler will report error in function
regulator_dev_lookup().

W/O NO_AUTO_INLINE, function of_get_regulator() is auto-inlined and then
the call to of_find_regulator_by_node() is optimized out since
of_get_regulator() always return NULL.

W/ NO_AUTO_INLINE, the return value of of_get_regulator() is a variable
so the call to of_find_regulator_by_node() cannot be optimized out. So
we need a stub of_find_regulator_by_node().

static struct regulator_dev *regulator_dev_lookup(struct device *dev,
						  const char *supply)
{
	struct regulator_dev *r = NULL;
	struct device_node *node;
	struct regulator_map *map;
	const char *devname = NULL;

	regulator_supply_alias(&dev, &supply);

	/* first do a dt based lookup */
	if (dev && dev->of_node) {
		node = of_get_regulator(dev, supply);
		if (node) {
			r = of_find_regulator_by_node(node);
			if (r)
				return r;
	...

Signed-off-by: Changbin Du <changbin.du@intel.com>
---
 drivers/regulator/internal.h | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/regulator/internal.h b/drivers/regulator/internal.h
index abfd56e..24fde1e 100644
--- a/drivers/regulator/internal.h
+++ b/drivers/regulator/internal.h
@@ -56,14 +56,19 @@ static inline struct regulator_dev *dev_to_rdev(struct device *dev)
 	return container_of(dev, struct regulator_dev, dev);
 }
 
-struct regulator_dev *of_find_regulator_by_node(struct device_node *np);
-
 #ifdef CONFIG_OF
+struct regulator_dev *of_find_regulator_by_node(struct device_node *np);
 struct regulator_init_data *regulator_of_get_init_data(struct device *dev,
 			         const struct regulator_desc *desc,
 				 struct regulator_config *config,
 				 struct device_node **node);
 #else
+static inline struct regulator_dev *
+of_find_regulator_by_node(struct device_node *np)
+{
+	return NULL;
+}
+
 static inline struct regulator_init_data *
 regulator_of_get_init_data(struct device *dev,
 			   const struct regulator_desc *desc,
-- 
2.7.4

[PATCH v2 3/5] kernel hacking: new config NO_AUTO_INLINE to disable compiler auto-inline optimizations

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

This patch add a new kernel hacking option NO_AUTO_INLINE. Selecting
this option will prevent the compiler from optimizing the kernel by
auto-inlining functions not marked with the inline keyword.

With this option, only functions explicitly marked with "inline" will
be inlined. This will allow the function tracer to trace more functions
because it only traces functions that the compiler has not inlined.

Signed-off-by: Changbin Du <changbin.du@intel.com>
Cc: Steven Rostedt <rostedt@goodmis.org>

---
v2: Some grammar updates from Steven.
---
 Makefile          |  6 ++++++
 lib/Kconfig.debug | 18 ++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/Makefile b/Makefile
index 619a85a..eb694f6 100644
--- a/Makefile
+++ b/Makefile
@@ -775,6 +775,12 @@ KBUILD_CFLAGS 	+= $(call cc-option, -femit-struct-debug-baseonly) \
 		   $(call cc-option,-fno-var-tracking)
 endif
 
+ifdef CONFIG_NO_AUTO_INLINE
+KBUILD_CFLAGS   += $(call cc-option, -fno-inline-functions) \
+		   $(call cc-option, -fno-inline-small-functions) \
+		   $(call cc-option, -fno-inline-functions-called-once)
+endif
+
 ifdef CONFIG_FUNCTION_TRACER
 ifndef CC_FLAGS_FTRACE
 CC_FLAGS_FTRACE := -pg
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index c40c7b7..ab55801 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -198,6 +198,24 @@ config GDB_SCRIPTS
 	  instance. See Documentation/dev-tools/gdb-kernel-debugging.rst
 	  for further details.
 
+config NO_AUTO_INLINE
+	bool "Disable compiler auto-inline optimizations"
+	default n
+	help
+	  This will prevent the compiler from optimizing the kernel by
+	  auto-inlining functions not marked with the inline keyword.
+	  With this option, only functions explicitly marked with
+	  "inline" will be inlined. This will allow the function tracer
+	  to trace more functions because it only traces functions that
+	  the compiler has not inlined.
+
+	  Enabling this function can help debugging a kernel if using
+	  the function tracer. But it can also change how the kernel
+	  works, because inlining functions may change the timing,
+	  which could make it difficult while debugging race conditions.
+
+	  If unsure, select N.
+
 config ENABLE_WARN_DEPRECATED
 	bool "Enable __deprecated logic"
 	default y
-- 
2.7.4

Re: [PATCH v2 3/5] kernel hacking: new config NO_AUTO_INLINE to disable compiler auto-inline optimizations

[Steven Rostedt]

On Wed,  2 May 2018 21:44:58 +0800
changbin.du@intel.com wrote:

> From: Changbin Du <changbin.du@intel.com>
> 
> This patch add a new kernel hacking option NO_AUTO_INLINE. Selecting
> this option will prevent the compiler from optimizing the kernel by
> auto-inlining functions not marked with the inline keyword.
> 
> With this option, only functions explicitly marked with "inline" will
> be inlined. This will allow the function tracer to trace more functions
> because it only traces functions that the compiler has not inlined.
> 
> Signed-off-by: Changbin Du <changbin.du@intel.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>

I'm fine with this patch if others are OK with it too.

Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve

> 
> ---
> v2: Some grammar updates from Steven.
> ---
>  Makefile          |  6 ++++++
>  lib/Kconfig.debug | 18 ++++++++++++++++++
>  2 files changed, 24 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 619a85a..eb694f6 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -775,6 +775,12 @@ KBUILD_CFLAGS 	+= $(call cc-option, -femit-struct-debug-baseonly) \
>  		   $(call cc-option,-fno-var-tracking)
>  endif
>  
> +ifdef CONFIG_NO_AUTO_INLINE
> +KBUILD_CFLAGS   += $(call cc-option, -fno-inline-functions) \
> +		   $(call cc-option, -fno-inline-small-functions) \
> +		   $(call cc-option, -fno-inline-functions-called-once)
> +endif
> +
>  ifdef CONFIG_FUNCTION_TRACER
>  ifndef CC_FLAGS_FTRACE
>  CC_FLAGS_FTRACE := -pg
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index c40c7b7..ab55801 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -198,6 +198,24 @@ config GDB_SCRIPTS
>  	  instance. See Documentation/dev-tools/gdb-kernel-debugging.rst
>  	  for further details.
>  
> +config NO_AUTO_INLINE
> +	bool "Disable compiler auto-inline optimizations"
> +	default n
> +	help
> +	  This will prevent the compiler from optimizing the kernel by
> +	  auto-inlining functions not marked with the inline keyword.
> +	  With this option, only functions explicitly marked with
> +	  "inline" will be inlined. This will allow the function tracer
> +	  to trace more functions because it only traces functions that
> +	  the compiler has not inlined.
> +
> +	  Enabling this function can help debugging a kernel if using
> +	  the function tracer. But it can also change how the kernel
> +	  works, because inlining functions may change the timing,
> +	  which could make it difficult while debugging race conditions.
> +
> +	  If unsure, select N.
> +
>  config ENABLE_WARN_DEPRECATED
>  	bool "Enable __deprecated logic"
>  	default y

Re: [PATCH v2 3/5] kernel hacking: new config NO_AUTO_INLINE to disable compiler auto-inline optimizations

[Arnd Bergmann]

On Wed, May 2, 2018 at 9:44 AM,  <changbin.du@intel.com> wrote:
> From: Changbin Du <changbin.du@intel.com>
>
> This patch add a new kernel hacking option NO_AUTO_INLINE. Selecting
> this option will prevent the compiler from optimizing the kernel by
> auto-inlining functions not marked with the inline keyword.
>
> With this option, only functions explicitly marked with "inline" will
> be inlined. This will allow the function tracer to trace more functions
> because it only traces functions that the compiler has not inlined.
>
> Signed-off-by: Changbin Du <changbin.du@intel.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>

Should this be closer to CONFIG_OPTIMIZE_INLINING or
possibly mutually exclusive with it?

       Arnd

Re: [PATCH v2 3/5] kernel hacking: new config NO_AUTO_INLINE to disable compiler auto-inline optimizations

[Du, Changbin]

On Wed, May 02, 2018 at 04:27:47PM -0400, Arnd Bergmann wrote:
> On Wed, May 2, 2018 at 9:44 AM,  <changbin.du@intel.com> wrote:
> > From: Changbin Du <changbin.du@intel.com>
> >
> > This patch add a new kernel hacking option NO_AUTO_INLINE. Selecting
> > this option will prevent the compiler from optimizing the kernel by
> > auto-inlining functions not marked with the inline keyword.
> >
> > With this option, only functions explicitly marked with "inline" will
> > be inlined. This will allow the function tracer to trace more functions
> > because it only traces functions that the compiler has not inlined.
> >
> > Signed-off-by: Changbin Du <changbin.du@intel.com>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> 
> Should this be closer to CONFIG_OPTIMIZE_INLINING or
> possibly mutually exclusive with it?
>
They are not related I think. CONFIG_OPTIMIZE_INLINING only has effect on
functions which are explicitly marked as inline.

>        Arnd

-- 
Thanks,
Changbin Du

Re: [PATCH v2 3/5] kernel hacking: new config NO_AUTO_INLINE to disable compiler auto-inline optimizations

[Steven Rostedt]

On Thu, 3 May 2018 21:00:13 +0800
"Du, Changbin" <changbin.du@intel.com> wrote:

> On Wed, May 02, 2018 at 04:27:47PM -0400, Arnd Bergmann wrote:
> > On Wed, May 2, 2018 at 9:44 AM,  <changbin.du@intel.com> wrote:  
> > > From: Changbin Du <changbin.du@intel.com>
> > >
> > > This patch add a new kernel hacking option NO_AUTO_INLINE. Selecting
> > > this option will prevent the compiler from optimizing the kernel by
> > > auto-inlining functions not marked with the inline keyword.
> > >
> > > With this option, only functions explicitly marked with "inline" will
> > > be inlined. This will allow the function tracer to trace more functions
> > > because it only traces functions that the compiler has not inlined.
> > >
> > > Signed-off-by: Changbin Du <changbin.du@intel.com>
> > > Cc: Steven Rostedt <rostedt@goodmis.org>  
> > 
> > Should this be closer to CONFIG_OPTIMIZE_INLINING or
> > possibly mutually exclusive with it?
> >  
> They are not related I think. CONFIG_OPTIMIZE_INLINING only has effect on
> functions which are explicitly marked as inline.
> 

Agreed, as OPTIMIZE_INLINING is to make functions marked inline not to
be inlined. And I just noticed (doing a git grep), that that config is
now only available in arch/x86. Maybe it always was, but sparc
undefines it for vclock_gettime.c.

$ git grep OPTIMIZE_INLIN
arch/sparc/vdso/vdso32/vclock_gettime.c:#undef  CONFIG_OPTIMIZE_INLINING
arch/x86/Kconfig.debug:config OPTIMIZE_INLINING
arch/x86/configs/i386_defconfig:CONFIG_OPTIMIZE_INLINING=y
arch/x86/configs/x86_64_defconfig:CONFIG_OPTIMIZE_INLINING=y
arch/x86/entry/vdso/vdso32/vclock_gettime.c:#undef CONFIG_OPTIMIZE_INLINING
include/linux/compiler-gcc.h:    !defined(CONFIG_OPTIMIZE_INLINING) || (__GNUC__ < 4)
kernel/configs/tiny.config:CONFIG_OPTIMIZE_INLINING=y

-- Steve

[PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

This will apply GCC '-Og' optimization level which is supported
since GCC 4.8. This optimization level offers a reasonable level
of optimization while maintaining fast compilation and a good
debugging experience. It is similar to '-O1' while perfer keeping
debug ability over runtime speed.

If enabling this option breaks your kernel, you should either
disable this or find a fix (mostly in the arch code). Currently
this option has only be tested on x86_64 platform.

This option can satisfy people who was searching for a method
to disable compiler optimizations so to achieve better kernel
debugging experience with kgdb or qemu.

The main problem of '-Og' is we must not use __attribute__((error(msg))).
The compiler will report error though the call to error function
still can be optimize out. So we must fallback to array tricky.

Comparison of vmlinux size: a bit smaller.

    w/o CONFIG_DEBUG_EXPERIENCE
    $ size vmlinux
       text    data     bss     dec     hex filename
    22665554   9709674  2920908 35296136        21a9388 vmlinux

    w/ CONFIG_DEBUG_EXPERIENCE
    $ size vmlinux
       text    data     bss     dec     hex filename
    21499032   10102758 2920908 34522698        20ec64a vmlinux

Comparison of system performance: a bit drop (~6%).
    This benchmark of kernel compilation is suggested by Ingo Molnar.
    https://lkml.org/lkml/2018/5/2/74

    Preparation: Set cpufreq to 'performance'.
    for ((cpu=0; cpu<120; cpu++)); do
      G=/sys/devices/system/cpu/cpu$cpu/cpufreq/scaling_governor
      [ -f $G ] && echo performance > $G
    done

    w/o CONFIG_DEBUG_EXPERIENCE
    $ perf stat --repeat 5 --null --pre                 '\
        cp -a kernel ../kernel.copy.$(date +%s);         \
        rm -rf *;                                        \
        git checkout .;                                  \
        echo 1 > /proc/sys/vm/drop_caches;               \
        find ../kernel* -type f | xargs cat >/dev/null;  \
        make -j kernel >/dev/null;                       \
        make clean >/dev/null 2>&1;                      \
        sync                                            '\
                                                         \
        make -j8 >/dev/null

     Performance counter stats for 'make -j8' (5 runs):

        219.764246652 seconds time elapsed                   ( +-  0.78% )

    w/ CONFIG_DEBUG_EXPERIENCE
    $ perf stat --repeat 5 --null --pre                 '\
        cp -a kernel ../kernel.copy.$(date +%s);         \
        rm -rf *;                                        \
        git checkout .;                                  \
        echo 1 > /proc/sys/vm/drop_caches;               \
        find ../kernel* -type f | xargs cat >/dev/null;  \
        make -j kernel >/dev/null;                       \
        make clean >/dev/null 2>&1;                      \
        sync                                            '\
                                                         \
        make -j8 >/dev/null

    Performance counter stats for 'make -j8' (5 runs):

         233.574187771 seconds time elapsed                  ( +-  0.19% )

Signed-off-by: Changbin Du <changbin.du@intel.com>

---
v2:
  o Improve performance benchmark as suggested by Ingo.
  o Grammar updates in description. (Randy Dunlap)
---
 Makefile                     |  4 ++++
 include/linux/compiler-gcc.h |  2 +-
 include/linux/compiler.h     |  2 +-
 lib/Kconfig.debug            | 21 +++++++++++++++++++++
 4 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index eb694f6..6a10469 100644
--- a/Makefile
+++ b/Makefile
@@ -639,6 +639,9 @@ KBUILD_CFLAGS	+= $(call cc-disable-warning, format-truncation)
 KBUILD_CFLAGS	+= $(call cc-disable-warning, format-overflow)
 KBUILD_CFLAGS	+= $(call cc-disable-warning, int-in-bool-context)
 
+ifdef CONFIG_DEBUG_EXPERIENCE
+KBUILD_CFLAGS	+= $(call cc-option, -Og)
+else
 ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
 KBUILD_CFLAGS	+= $(call cc-option,-Oz,-Os)
 KBUILD_CFLAGS	+= $(call cc-disable-warning,maybe-uninitialized,)
@@ -649,6 +652,7 @@ else
 KBUILD_CFLAGS   += -O2
 endif
 endif
+endif
 
 KBUILD_CFLAGS += $(call cc-ifversion, -lt, 0409, \
 			$(call cc-disable-warning,maybe-uninitialized,))
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index b4bf73f..b8b3832 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -192,7 +192,7 @@
 
 #define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __COUNTER__)
 
-#ifndef __CHECKER__
+#if !defined(__CHECKER__) && !defined(CONFIG_DEBUG_EXPERIENCE)
 # define __compiletime_warning(message) __attribute__((warning(message)))
 # define __compiletime_error(message) __attribute__((error(message)))
 #endif /* __CHECKER__ */
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index ab4711c..952cc7f 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -301,7 +301,7 @@ unsigned long read_word_at_a_time(const void *addr)
  * sparse see a constant array size without breaking compiletime_assert on old
  * versions of GCC (e.g. 4.2.4), so hide the array from sparse altogether.
  */
-# ifndef __CHECKER__
+# if !defined(__CHECKER__) && !defined(CONFIG_DEBUG_EXPERIENCE)
 #  define __compiletime_error_fallback(condition) \
 	do { ((void)sizeof(char[1 - 2 * condition])); } while (0)
 # endif
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index ab55801..e264199 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -216,6 +216,27 @@ config NO_AUTO_INLINE
 
 	  If unsure, select N.
 
+config DEBUG_EXPERIENCE
+	bool "Optimize for better debugging experience (-Og)"
+	default n
+	select NO_AUTO_INLINE
+	depends on !CC_OPTIMIZE_FOR_SIZE
+	help
+	  This will apply GCC '-Og' optimization level which is supported
+	  since GCC 4.8. This optimization level offers a reasonable level
+	  of optimization while maintaining fast compilation and a good
+	  debugging experience. It is similar to '-O1' while preferring to
+	  keep debug ability over runtime speed. The overall performance
+	  will drop a bit (~6%).
+
+	  Use only if you want to debug the kernel, especially if you want
+	  to have better kernel debugging experience with gdb facilities
+	  like kgdb or qemu. If enabling this option breaks your kernel,
+	  you should either disable this or find a fix (mostly in the arch
+	  code). Currently this option has only be tested on x86_64 platform.
+
+	  If unsure, select N.
+
 config ENABLE_WARN_DEPRECATED
 	bool "Enable __deprecated logic"
 	default y
-- 
2.7.4

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Steven Rostedt]

On Wed,  2 May 2018 21:44:59 +0800
changbin.du@intel.com wrote:

> From: Changbin Du <changbin.du@intel.com>
> 
> This will apply GCC '-Og' optimization level which is supported
> since GCC 4.8. This optimization level offers a reasonable level
> of optimization while maintaining fast compilation and a good
> debugging experience. It is similar to '-O1' while perfer keeping
> debug ability over runtime speed.
> 
> If enabling this option breaks your kernel, you should either
> disable this or find a fix (mostly in the arch code). Currently
> this option has only be tested on x86_64 platform.

If this becomes an issue, you probably need to add an arch config that
it depends on like CONFIG_HAVE_DEBUG_EXPERIENCE (or another name, as I
mention below).

> 
> This option can satisfy people who was searching for a method
> to disable compiler optimizations so to achieve better kernel
> debugging experience with kgdb or qemu.
> 
> The main problem of '-Og' is we must not use __attribute__((error(msg))).
> The compiler will report error though the call to error function
> still can be optimize out. So we must fallback to array tricky.
> 
> Comparison of vmlinux size: a bit smaller.
> 
>     w/o CONFIG_DEBUG_EXPERIENCE

I hate the config name.

I probably can't come up with better ones but let's try:

 CONFIG_DEBUG_OPTIMIZE ?
 CONFIG_OPTIMIZE_DEBUG ?

But "EXPERIENCE" sounds like I'm on some DEBUG LSD.



>     $ size vmlinux
>        text    data     bss     dec     hex filename
>     22665554   9709674  2920908 35296136        21a9388 vmlinux
> 
>     w/ CONFIG_DEBUG_EXPERIENCE
>     $ size vmlinux
>        text    data     bss     dec     hex filename
>     21499032   10102758 2920908 34522698        20ec64a vmlinux
> 
> Comparison of system performance: a bit drop (~6%).
>     This benchmark of kernel compilation is suggested by Ingo Molnar.
>     https://lkml.org/lkml/2018/5/2/74
> 
>     Preparation: Set cpufreq to 'performance'.
>     for ((cpu=0; cpu<120; cpu++)); do
>       G=/sys/devices/system/cpu/cpu$cpu/cpufreq/scaling_governor
>       [ -f $G ] && echo performance > $G
>     done
> 
>     w/o CONFIG_DEBUG_EXPERIENCE
>     $ perf stat --repeat 5 --null --pre                 '\
>         cp -a kernel ../kernel.copy.$(date +%s);         \
>         rm -rf *;                                        \
>         git checkout .;                                  \
>         echo 1 > /proc/sys/vm/drop_caches;               \
>         find ../kernel* -type f | xargs cat >/dev/null;  \
>         make -j kernel >/dev/null;                       \
>         make clean >/dev/null 2>&1;                      \
>         sync                                            '\
>                                                          \
>         make -j8 >/dev/null
> 
>      Performance counter stats for 'make -j8' (5 runs):
> 
>         219.764246652 seconds time elapsed                   ( +-  0.78% )
> 
>     w/ CONFIG_DEBUG_EXPERIENCE
>     $ perf stat --repeat 5 --null --pre                 '\
>         cp -a kernel ../kernel.copy.$(date +%s);         \
>         rm -rf *;                                        \
>         git checkout .;                                  \
>         echo 1 > /proc/sys/vm/drop_caches;               \
>         find ../kernel* -type f | xargs cat >/dev/null;  \
>         make -j kernel >/dev/null;                       \
>         make clean >/dev/null 2>&1;                      \
>         sync                                            '\
>                                                          \
>         make -j8 >/dev/null
> 
>     Performance counter stats for 'make -j8' (5 runs):
> 
>          233.574187771 seconds time elapsed                  ( +-  0.19% )
> 
> Signed-off-by: Changbin Du <changbin.du@intel.com>
> 
> ---
> v2:
>   o Improve performance benchmark as suggested by Ingo.
>   o Grammar updates in description. (Randy Dunlap)
> ---
>  Makefile                     |  4 ++++
>  include/linux/compiler-gcc.h |  2 +-
>  include/linux/compiler.h     |  2 +-
>  lib/Kconfig.debug            | 21 +++++++++++++++++++++
>  4 files changed, 27 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index eb694f6..6a10469 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -639,6 +639,9 @@ KBUILD_CFLAGS	+= $(call cc-disable-warning, format-truncation)
>  KBUILD_CFLAGS	+= $(call cc-disable-warning, format-overflow)
>  KBUILD_CFLAGS	+= $(call cc-disable-warning, int-in-bool-context)
>  
> +ifdef CONFIG_DEBUG_EXPERIENCE
> +KBUILD_CFLAGS	+= $(call cc-option, -Og)
> +else
>  ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
>  KBUILD_CFLAGS	+= $(call cc-option,-Oz,-Os)
>  KBUILD_CFLAGS	+= $(call cc-disable-warning,maybe-uninitialized,)
> @@ -649,6 +652,7 @@ else
>  KBUILD_CFLAGS   += -O2
>  endif
>  endif
> +endif
>  
>  KBUILD_CFLAGS += $(call cc-ifversion, -lt, 0409, \
>  			$(call cc-disable-warning,maybe-uninitialized,))
> diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
> index b4bf73f..b8b3832 100644
> --- a/include/linux/compiler-gcc.h
> +++ b/include/linux/compiler-gcc.h
> @@ -192,7 +192,7 @@
>  
>  #define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __COUNTER__)
>  
> -#ifndef __CHECKER__
> +#if !defined(__CHECKER__) && !defined(CONFIG_DEBUG_EXPERIENCE)
>  # define __compiletime_warning(message) __attribute__((warning(message)))
>  # define __compiletime_error(message) __attribute__((error(message)))
>  #endif /* __CHECKER__ */
> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> index ab4711c..952cc7f 100644
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -301,7 +301,7 @@ unsigned long read_word_at_a_time(const void *addr)
>   * sparse see a constant array size without breaking compiletime_assert on old
>   * versions of GCC (e.g. 4.2.4), so hide the array from sparse altogether.
>   */
> -# ifndef __CHECKER__
> +# if !defined(__CHECKER__) && !defined(CONFIG_DEBUG_EXPERIENCE)
>  #  define __compiletime_error_fallback(condition) \
>  	do { ((void)sizeof(char[1 - 2 * condition])); } while (0)
>  # endif
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index ab55801..e264199 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -216,6 +216,27 @@ config NO_AUTO_INLINE
>  
>  	  If unsure, select N.
>  
> +config DEBUG_EXPERIENCE
> +	bool "Optimize for better debugging experience (-Og)"
> +	default n

You don't need to add "default n" because that's the default if it
isn't specified.

-- Steve

> +	select NO_AUTO_INLINE
> +	depends on !CC_OPTIMIZE_FOR_SIZE
> +	help
> +	  This will apply GCC '-Og' optimization level which is supported
> +	  since GCC 4.8. This optimization level offers a reasonable level
> +	  of optimization while maintaining fast compilation and a good
> +	  debugging experience. It is similar to '-O1' while preferring to
> +	  keep debug ability over runtime speed. The overall performance
> +	  will drop a bit (~6%).
> +
> +	  Use only if you want to debug the kernel, especially if you want
> +	  to have better kernel debugging experience with gdb facilities
> +	  like kgdb or qemu. If enabling this option breaks your kernel,
> +	  you should either disable this or find a fix (mostly in the arch
> +	  code). Currently this option has only be tested on x86_64 platform.
> +
> +	  If unsure, select N.
> +
>  config ENABLE_WARN_DEPRECATED
>  	bool "Enable __deprecated logic"
>  	default y

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Andrew Morton]

On Wed, 2 May 2018 10:17:07 -0400 Steven Rostedt <rostedt@goodmis.org> wrote:

> > Comparison of vmlinux size: a bit smaller.
> > 
> >     w/o CONFIG_DEBUG_EXPERIENCE
> 
> I hate the config name.
> 
> I probably can't come up with better ones but let's try:
> 
>  CONFIG_DEBUG_OPTIMIZE ?
>  CONFIG_OPTIMIZE_DEBUG ?
> 
> But "EXPERIENCE" sounds like I'm on some DEBUG LSD.

Metoo, but the gcc people decided on "-Og: Optimize debugging
experience ..." and I think there are benefits if the kernel is to
align the naming with that.

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Steven Rostedt]

On Wed, 2 May 2018 13:45:58 -0700
Andrew Morton <akpm@linux-foundation.org> wrote:

> On Wed, 2 May 2018 10:17:07 -0400 Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > > Comparison of vmlinux size: a bit smaller.
> > > 
> > >     w/o CONFIG_DEBUG_EXPERIENCE  
> > 
> > I hate the config name.
> > 
> > I probably can't come up with better ones but let's try:
> > 
> >  CONFIG_DEBUG_OPTIMIZE ?
> >  CONFIG_OPTIMIZE_DEBUG ?
> > 
> > But "EXPERIENCE" sounds like I'm on some DEBUG LSD.  
> 
> Metoo, but the gcc people decided on "-Og: Optimize debugging
> experience ..." and I think there are benefits if the kernel is to
> align the naming with that.

I still see that as "Optimize debugging" and "experience" is just the
platform of what was done.

With that gcc comment, I still think CONFIG_OPTIMIZE_DEBUG is more
inline with what it is and understandable than
CONFIG_DEBUG_EXPERIENCE. The "OPTIMIZE" is the key word there.

-- Steve

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Du, Changbin]

On Wed, May 02, 2018 at 09:19:56PM -0400, Steven Rostedt wrote:
> On Wed, 2 May 2018 13:45:58 -0700
> Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > On Wed, 2 May 2018 10:17:07 -0400 Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > > > Comparison of vmlinux size: a bit smaller.
> > > > 
> > > >     w/o CONFIG_DEBUG_EXPERIENCE  
> > > 
> > > I hate the config name.
> > > 
> > > I probably can't come up with better ones but let's try:
> > > 
> > >  CONFIG_DEBUG_OPTIMIZE ?
> > >  CONFIG_OPTIMIZE_DEBUG ?
> > > 
> > > But "EXPERIENCE" sounds like I'm on some DEBUG LSD.  
> > 
> > Metoo, but the gcc people decided on "-Og: Optimize debugging
> > experience ..." and I think there are benefits if the kernel is to
> > align the naming with that.
> 
> I still see that as "Optimize debugging" and "experience" is just the
> platform of what was done.
> 
> With that gcc comment, I still think CONFIG_OPTIMIZE_DEBUG is more
> inline with what it is and understandable than
> CONFIG_DEBUG_EXPERIENCE. The "OPTIMIZE" is the key word there.
> 
> -- Steve
What about CONFIG_CC_OPTIMIZE_FOR_DEBUGGING? We alreay have
CONFIG_CC_OPTIMIZE_FOR_SIZE and CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE.

And do we need to move it to existing configuration menu "General setup->
Compiler optimization level"? But I also want it appear in "kernel hacking"
since this is a debug option.

-- 
Thanks,
Changbin Du

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Steven Rostedt]

On Thu, 3 May 2018 21:45:46 +0800
"Du, Changbin" <changbin.du@intel.com> wrote:

> > With that gcc comment, I still think CONFIG_OPTIMIZE_DEBUG is more
> > inline with what it is and understandable than
> > CONFIG_DEBUG_EXPERIENCE. The "OPTIMIZE" is the key word there.
> > 
> > -- Steve  
> What about CONFIG_CC_OPTIMIZE_FOR_DEBUGGING? We alreay have
> CONFIG_CC_OPTIMIZE_FOR_SIZE and CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE.

Yes I like that much better.

> 
> And do we need to move it to existing configuration menu "General setup->
> Compiler optimization level"? But I also want it appear in "kernel hacking"
> since this is a debug option.

I understand why you would want it by debugging, but I think it does
make more sense to be included with the above two other options, as
they are all mutually exclusive.

This brings up the topic of creating config paradigms. That is, a way
of saying "I want a debug kernel" and select one option that selects
everything you would expect. Or perhaps we should have a:

 make debug_config

that does it.

But that's a different topic. For now, I would just included it in
init/Kconfig, and not worry about it not showing up in kernel hacking.


-- Steve

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Du, Changbin]

On Thu, May 03, 2018 at 10:28:23AM -0400, Steven Rostedt wrote:
> On Thu, 3 May 2018 21:45:46 +0800
> "Du, Changbin" <changbin.du@intel.com> wrote:
> 
> > > With that gcc comment, I still think CONFIG_OPTIMIZE_DEBUG is more
> > > inline with what it is and understandable than
> > > CONFIG_DEBUG_EXPERIENCE. The "OPTIMIZE" is the key word there.
> > > 
> > > -- Steve  
> > What about CONFIG_CC_OPTIMIZE_FOR_DEBUGGING? We alreay have
> > CONFIG_CC_OPTIMIZE_FOR_SIZE and CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE.
> 
> Yes I like that much better.
> 
> > 
> > And do we need to move it to existing configuration menu "General setup->
> > Compiler optimization level"? But I also want it appear in "kernel hacking"
> > since this is a debug option.
> 
> I understand why you would want it by debugging, but I think it does
> make more sense to be included with the above two other options, as
> they are all mutually exclusive.
> 
> This brings up the topic of creating config paradigms. That is, a way
> of saying "I want a debug kernel" and select one option that selects
> everything you would expect. Or perhaps we should have a:
> 
>  make debug_config
> 
Agree, I accomplish this by running script scripts/kconfig/merge_config.sh.

> that does it.
> 
> But that's a different topic. For now, I would just included it in
> init/Kconfig, and not worry about it not showing up in kernel hacking.
> 
> 
> -- Steve

-- 
Thanks,
Changbin Du

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Randy Dunlap]

On 05/05/2018 04:57 PM, Du, Changbin wrote:
> On Thu, May 03, 2018 at 10:28:23AM -0400, Steven Rostedt wrote:
>> On Thu, 3 May 2018 21:45:46 +0800
>> "Du, Changbin" <changbin.du@intel.com> wrote:
>>
>>>> With that gcc comment, I still think CONFIG_OPTIMIZE_DEBUG is more
>>>> inline with what it is and understandable than
>>>> CONFIG_DEBUG_EXPERIENCE. The "OPTIMIZE" is the key word there.
>>>>
>>>> -- Steve  
>>> What about CONFIG_CC_OPTIMIZE_FOR_DEBUGGING? We alreay have
>>> CONFIG_CC_OPTIMIZE_FOR_SIZE and CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE.
>>
>> Yes I like that much better.
>>
>>>
>>> And do we need to move it to existing configuration menu "General setup->
>>> Compiler optimization level"? But I also want it appear in "kernel hacking"
>>> since this is a debug option.
>>
>> I understand why you would want it by debugging, but I think it does
>> make more sense to be included with the above two other options, as
>> they are all mutually exclusive.

mm, sounds good.

>> This brings up the topic of creating config paradigms. That is, a way
>> of saying "I want a debug kernel" and select one option that selects
>> everything you would expect. Or perhaps we should have a:
>>
>>  make debug_config

Sometimes I want to build allmodconfig-minus-debug options ... but not
all debug options.  I still want debugfs but most options that end with
_DEBUG are disabled.  I.e., I don't want the options that cause big
run-time slowdowns. (unless I cause that by printing some debugfs
contents)

Can merge_config.sh (or one of its cousins) help with that?

> Agree, I accomplish this by running script scripts/kconfig/merge_config.sh.
> 
>> that does it.
>>
>> But that's a different topic. For now, I would just included it in
>> init/Kconfig, and not worry about it not showing up in kernel hacking.

Ack.


thanks.
-- 
~Randy

Re: [PATCH v2 4/5] kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og optimization

[Randy Dunlap]

On 05/02/2018 06:44 AM, changbin.du@intel.com wrote:
> From: Changbin Du <changbin.du@intel.com>
> 

Sorry, I missed one:

> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index ab55801..e264199 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -216,6 +216,27 @@ config NO_AUTO_INLINE
>  
>  	  If unsure, select N.
>  
> +config DEBUG_EXPERIENCE
> +	bool "Optimize for better debugging experience (-Og)"
> +	default n
> +	select NO_AUTO_INLINE
> +	depends on !CC_OPTIMIZE_FOR_SIZE
> +	help
> +	  This will apply GCC '-Og' optimization level which is supported
> +	  since GCC 4.8. This optimization level offers a reasonable level
> +	  of optimization while maintaining fast compilation and a good
> +	  debugging experience. It is similar to '-O1' while preferring to
> +	  keep debug ability over runtime speed. The overall performance
> +	  will drop a bit (~6%).
> +
> +	  Use only if you want to debug the kernel, especially if you want
> +	  to have better kernel debugging experience with gdb facilities
> +	  like kgdb or qemu. If enabling this option breaks your kernel,
> +	  you should either disable this or find a fix (mostly in the arch
> +	  code). Currently this option has only be tested on x86_64 platform.

	                                        been tested

> +
> +	  If unsure, select N.
> +
>  config ENABLE_WARN_DEPRECATED
>  	bool "Enable __deprecated logic"
>  	default y
> 


-- 
~Randy

[PATCH v2 5/5] asm-generic: fix build error in fix_to_virt with CONFIG_DEBUG_EXPERIENCE

[changbin.du]

From: Changbin Du <changbin.du@intel.com>

With '-Og' optimization level, GCC would not optimize a count for a loop
as a constant value. But BUILD_BUG_ON() only accept compile-time constant
values.

arch/arm/mm/mmu.o: In function `fix_to_virt':
/home/changbin/work/linux/./include/asm-generic/fixmap.h:31: undefined reference to `__compiletime_assert_31'
Makefile:1051: recipe for target 'vmlinux' failed
make: *** [vmlinux] Error 1

Signed-off-by: Changbin Du <changbin.du@intel.com>
---
 include/asm-generic/fixmap.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/asm-generic/fixmap.h b/include/asm-generic/fixmap.h
index 827e4d3..a6576d4 100644
--- a/include/asm-generic/fixmap.h
+++ b/include/asm-generic/fixmap.h
@@ -28,7 +28,8 @@
  */
 static __always_inline unsigned long fix_to_virt(const unsigned int idx)
 {
-	BUILD_BUG_ON(idx >= __end_of_fixed_addresses);
+	BUILD_BUG_ON(__builtin_constant_p(idx) &&
+		     idx >= __end_of_fixed_addresses);
 	return __fix_to_virt(idx);
 }
 
-- 
2.7.4

Re: [PATCH v2 5/5] asm-generic: fix build error in fix_to_virt with CONFIG_DEBUG_EXPERIENCE

[Steven Rostedt]

On Wed,  2 May 2018 21:45:00 +0800
changbin.du@intel.com wrote:

> From: Changbin Du <changbin.du@intel.com>
> 
> With '-Og' optimization level, GCC would not optimize a count for a loop
> as a constant value. But BUILD_BUG_ON() only accept compile-time constant
> values.
> 
> arch/arm/mm/mmu.o: In function `fix_to_virt':
> /home/changbin/work/linux/./include/asm-generic/fixmap.h:31: undefined reference to `__compiletime_assert_31'
> Makefile:1051: recipe for target 'vmlinux' failed
> make: *** [vmlinux] Error 1
> 
> Signed-off-by: Changbin Du <changbin.du@intel.com>
> ---
>  include/asm-generic/fixmap.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/include/asm-generic/fixmap.h b/include/asm-generic/fixmap.h
> index 827e4d3..a6576d4 100644
> --- a/include/asm-generic/fixmap.h
> +++ b/include/asm-generic/fixmap.h
> @@ -28,7 +28,8 @@
>   */
>  static __always_inline unsigned long fix_to_virt(const unsigned int idx)
>  {
> -	BUILD_BUG_ON(idx >= __end_of_fixed_addresses);
> +	BUILD_BUG_ON(__builtin_constant_p(idx) &&
> +		     idx >= __end_of_fixed_addresses);

Hmm, this changes the check slightly. Perhaps we should only do this
when your config is active:

{
	BUILD_BUG_ON(
/* CONFIG_DEBUG_OPTIMIZE may cause idx not to be constant */
#ifdef CONFIG_DEBUG_OPTIMIZE
		__builtin_constant_p(idx) &&
#endif
		idx >= __end_of_fixed_addresses);

}

-- Steve

>  	return __fix_to_virt(idx);
>  }
>  

Re: [PATCH v2 5/5] asm-generic: fix build error in fix_to_virt with CONFIG_DEBUG_EXPERIENCE

[Du, Changbin]

On Wed, May 02, 2018 at 10:19:30AM -0400, Steven Rostedt wrote:
> On Wed,  2 May 2018 21:45:00 +0800
> changbin.du@intel.com wrote:
> 
> > From: Changbin Du <changbin.du@intel.com>
> > 
> > With '-Og' optimization level, GCC would not optimize a count for a loop
> > as a constant value. But BUILD_BUG_ON() only accept compile-time constant
> > values.
> > 
> > arch/arm/mm/mmu.o: In function `fix_to_virt':
> > /home/changbin/work/linux/./include/asm-generic/fixmap.h:31: undefined reference to `__compiletime_assert_31'
> > Makefile:1051: recipe for target 'vmlinux' failed
> > make: *** [vmlinux] Error 1
> > 
> > Signed-off-by: Changbin Du <changbin.du@intel.com>
> > ---
> >  include/asm-generic/fixmap.h | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/include/asm-generic/fixmap.h b/include/asm-generic/fixmap.h
> > index 827e4d3..a6576d4 100644
> > --- a/include/asm-generic/fixmap.h
> > +++ b/include/asm-generic/fixmap.h
> > @@ -28,7 +28,8 @@
> >   */
> >  static __always_inline unsigned long fix_to_virt(const unsigned int idx)
> >  {
> > -	BUILD_BUG_ON(idx >= __end_of_fixed_addresses);
> > +	BUILD_BUG_ON(__builtin_constant_p(idx) &&
> > +		     idx >= __end_of_fixed_addresses);
> 
> Hmm, this changes the check slightly. Perhaps we should only do this
> when your config is active:
> 
> {
> 	BUILD_BUG_ON(
> /* CONFIG_DEBUG_OPTIMIZE may cause idx not to be constant */
> #ifdef CONFIG_DEBUG_OPTIMIZE
> 		__builtin_constant_p(idx) &&
> #endif
> 		idx >= __end_of_fixed_addresses);
> 
> }
I think fix_to_virt() is designed for constant idx only. So I think we should
fix it at the caller side by replacing it with __fix_to_virt().

--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1599,7 +1599,7 @@ static void __init early_fixmap_shutdown(void)
                pte_t *pte;
                struct map_desc map;

-               map.virtual = fix_to_virt(i);
+               map.virtual = __fix_to_virt(i);
                pte = pte_offset_early_fixmap(pmd_off_k(map.virtual), map.virtual);

> 
> -- Steve
> 
> >  	return __fix_to_virt(idx);
> >  }
> >  
> 

-- 
Thanks,
Changbin Du

Re: [PATCH v2 5/5] asm-generic: fix build error in fix_to_virt with CONFIG_DEBUG_EXPERIENCE

[Steven Rostedt]

On Thu, 3 May 2018 21:25:08 +0800
"Du, Changbin" <changbin.du@intel.com> wrote:

> I think fix_to_virt() is designed for constant idx only. So I think we should
> fix it at the caller side by replacing it with __fix_to_virt().
> 
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1599,7 +1599,7 @@ static void __init early_fixmap_shutdown(void)
>                 pte_t *pte;
>                 struct map_desc map;
> 
> -               map.virtual = fix_to_virt(i);
> +               map.virtual = __fix_to_virt(i);
>                 pte = pte_offset_early_fixmap(pmd_off_k(map.virtual), map.virtual);

That's a better solution than the current patch.

-- Steve

Re: [PATCH v2 5/5] asm-generic: fix build error in fix_to_virt with CONFIG_DEBUG_EXPERIENCE

[weylin]

Dear kernel devs,

After configuring for 2 hours settings, faced below on making :

cc1: error: 
/usr/src/linux-next-master/arch/x86/crypto/serpent-sse2-x86_64-asm_64.S: 
Input/output error

Thank you for your attantion to config matter last days, It is really 
physically  hard every time sort sort out menu again. If it would 
ppossible to have standard -- Conservative Unix style secure / paranoid 
choice alongwith -- allyesconfig .


Thank you !


On 05/03/2018 02:19 PM, Steven Rostedt wrote:
> On Thu, 3 May 2018 21:25:08 +0800
> "Du, Changbin" <changbin.du@intel.com> wrote:
>
>> I think fix_to_virt() is designed for constant idx only. So I think we should
>> fix it at the caller side by replacing it with __fix_to_virt().
>>
>> --- a/arch/arm/mm/mmu.c
>> +++ b/arch/arm/mm/mmu.c
>> @@ -1599,7 +1599,7 @@ static void __init early_fixmap_shutdown(void)
>>                  pte_t *pte;
>>                  struct map_desc map;
>>
>> -               map.virtual = fix_to_virt(i);
>> +               map.virtual = __fix_to_virt(i);
>>                  pte = pte_offset_early_fixmap(pmd_off_k(map.virtual), map.virtual);
> That's a better solution than the current patch.
>
> -- Steve
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 0/5] kernel hacking: GCC optimization for debug experience (-Og)

[Daniel Thompson]

On Wed, May 02, 2018 at 09:44:55PM +0800, changbin.du@intel.com wrote:
> From: Changbin Du <changbin.du@intel.com>
> 
> Hi all,
> I know some kernel developers was searching for a method to dissable GCC
> optimizations, probably they want to apply GCC '-O0' option. But since Linux
> kernel replys on GCC optimization to remove some dead code, so '-O0' just
> breaks the build. They do need this because they want to debug kernel with
> qemu, simics, kgtp or kgdb.
> 
> Thanks for the GCC '-Og' optimization level introduced in GCC 4.8, which
> offers a reasonable level of optimization while maintaining fast compilation
> and a good debugging experience. It is similar to '-O1' while perfer keeping
> debug ability over runtime speed. With '-Og', we can build a kernel with
> better debug ability and little performance drop after some simple change.
> 
> In this series, firstly introduce a new config CONFIG_NO_AUTO_INLINE after two
> fixes for this new option. With this option, only functions explicitly marked
> with "inline" will  be inlined. This will allow the function tracer to trace
> more functions because it only traces functions that the compiler has not
> inlined.
> 
> Then introduce new config CONFIG_DEBUG_EXPERIENCE which apply '-Og'
> optimization level for whole kernel, with a simple fix in fix_to_virt().
> Currently this option is only tested on a QEMU gust and it works fine.
> 
> 
> Comparison of vmlinux size: a bit smaller.
> 
>     w/o CONFIG_DEBUG_EXPERIENCE
>     $ size vmlinux
>        text    data     bss     dec     hex filename
>     22665554   9709674  2920908 35296136        21a9388 vmlinux
> 
>     w/ CONFIG_DEBUG_EXPERIENCE
>     $ size vmlinux
>        text    data     bss     dec     hex filename
>     21499032   10102758 2920908 34522698        20ec64a vmlinux
> 
> 
> Comparison of system performance: a bit drop (~6%).
>     This benchmark of kernel compilation is suggested by Ingo Molnar.
>     https://lkml.org/lkml/2018/5/2/74

In my mind was the opposite question. When running on the same kernel
does a kernel whose config contains CONFIG_DEBUG_EXPERIENCE build faster
than one without (due to the disabled optimization passes).

To be honest this is more curiosity than a review comment though... if
you have the figures please share, if not then don't sweat it on my
account!


Daniel.



> 
>     Preparation: Set cpufreq to 'performance'.
>     for ((cpu=0; cpu<120; cpu++)); do
>       G=/sys/devices/system/cpu/cpu$cpu/cpufreq/scaling_governor
>       [ -f $G ] && echo performance > $G
>     done
> 
>     w/o CONFIG_DEBUG_EXPERIENCE
>     $ perf stat --repeat 5 --null --pre                 '\
>         cp -a kernel ../kernel.copy.$(date +%s);         \
>         rm -rf *;                                        \
>         git checkout .;                                  \
>         echo 1 > /proc/sys/vm/drop_caches;               \
>         find ../kernel* -type f | xargs cat >/dev/null;  \
>         make -j kernel >/dev/null;                       \
>         make clean >/dev/null 2>&1;                      \
>         sync                                            '\
>                                                          \
>         make -j8 >/dev/null
> 
>      Performance counter stats for 'make -j8' (5 runs):
> 
>         219.764246652 seconds time elapsed                   ( +-  0.78% )
> 
>     w/ CONFIG_DEBUG_EXPERIENCE
>     $ perf stat --repeat 5 --null --pre                 '\
>         cp -a kernel ../kernel.copy.$(date +%s);         \
>         rm -rf *;                                        \
>         git checkout .;                                  \
>         echo 1 > /proc/sys/vm/drop_caches;               \
>         find ../kernel* -type f | xargs cat >/dev/null;  \
>         make -j kernel >/dev/null;                       \
>         make clean >/dev/null 2>&1;                      \
>         sync                                            '\
>                                                          \
>         make -j8 >/dev/null
> 
>     Performance counter stats for 'make -j8' (5 runs):
> 
>          233.574187771 seconds time elapsed                  ( +-  0.19% )
> 
> Changbin Du (5):
>   x86/mm: surround level4_kernel_pgt with #ifdef
>     CONFIG_X86_5LEVEL...#endif
>   regulator: add dummy function of_find_regulator_by_node
>   kernel hacking: new config NO_AUTO_INLINE to disable compiler
>     auto-inline optimizations
>   kernel hacking: new config DEBUG_EXPERIENCE to apply GCC -Og
>     optimization
>   asm-generic: fix build error in fix_to_virt with
>     CONFIG_DEBUG_EXPERIENCE
> 
>  Makefile                          | 10 ++++++++++
>  arch/x86/include/asm/pgtable_64.h |  2 ++
>  arch/x86/kernel/head64.c          | 13 ++++++-------
>  drivers/regulator/internal.h      |  9 +++++++--
>  include/asm-generic/fixmap.h      |  3 ++-
>  include/linux/compiler-gcc.h      |  2 +-
>  include/linux/compiler.h          |  2 +-
>  lib/Kconfig.debug                 | 39 +++++++++++++++++++++++++++++++++++++++
>  8 files changed, 68 insertions(+), 12 deletions(-)
> 
> -- 
> 2.7.4
> 

Re: [PATCH v2 0/5] kernel hacking: GCC optimization for debug experience (-Og)

[Du, Changbin]

On Wed, May 02, 2018 at 03:56:31PM +0100, Daniel Thompson wrote:
> On Wed, May 02, 2018 at 09:44:55PM +0800, changbin.du@intel.com wrote:
> > From: Changbin Du <changbin.du@intel.com>
> > 
> > Hi all,
> > I know some kernel developers was searching for a method to dissable GCC
> > optimizations, probably they want to apply GCC '-O0' option. But since Linux
> > kernel replys on GCC optimization to remove some dead code, so '-O0' just
> > breaks the build. They do need this because they want to debug kernel with
> > qemu, simics, kgtp or kgdb.
> > 
> > Thanks for the GCC '-Og' optimization level introduced in GCC 4.8, which
> > offers a reasonable level of optimization while maintaining fast compilation
> > and a good debugging experience. It is similar to '-O1' while perfer keeping
> > debug ability over runtime speed. With '-Og', we can build a kernel with
> > better debug ability and little performance drop after some simple change.
> > 
> > In this series, firstly introduce a new config CONFIG_NO_AUTO_INLINE after two
> > fixes for this new option. With this option, only functions explicitly marked
> > with "inline" will  be inlined. This will allow the function tracer to trace
> > more functions because it only traces functions that the compiler has not
> > inlined.
> > 
> > Then introduce new config CONFIG_DEBUG_EXPERIENCE which apply '-Og'
> > optimization level for whole kernel, with a simple fix in fix_to_virt().
> > Currently this option is only tested on a QEMU gust and it works fine.
> > 
> > 
> > Comparison of vmlinux size: a bit smaller.
> > 
> >     w/o CONFIG_DEBUG_EXPERIENCE
> >     $ size vmlinux
> >        text    data     bss     dec     hex filename
> >     22665554   9709674  2920908 35296136        21a9388 vmlinux
> > 
> >     w/ CONFIG_DEBUG_EXPERIENCE
> >     $ size vmlinux
> >        text    data     bss     dec     hex filename
> >     21499032   10102758 2920908 34522698        20ec64a vmlinux
> > 
> > 
> > Comparison of system performance: a bit drop (~6%).
> >     This benchmark of kernel compilation is suggested by Ingo Molnar.
> >     https://lkml.org/lkml/2018/5/2/74
> 
> In my mind was the opposite question. When running on the same kernel
> does a kernel whose config contains CONFIG_DEBUG_EXPERIENCE build faster
> than one without (due to the disabled optimization passes).
> 
> To be honest this is more curiosity than a review comment though... if
> you have the figures please share, if not then don't sweat it on my
> account!
> 
> 
> Daniel.
>
Sorry I don't have the data yet. Per the comment in GCC, I think it should be a
little faster but not obviously.

> 
-- 
Thanks,
Changbin Du