Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Nicolas Pitre]

Gentle ping...

If possible, I'd like for those patches to be sorted out before the 
opening of the merge window gets too close like last time.


On Tue, 29 Mar 2016, Nicolas Pitre wrote:

> This patch series provides the option to omit exported symbols from
> the kernel and modules that are never referenced by any of the selected
> modules in the current kernel configuration. this allows for optimizing
> the compiled code and reducing final binaries' size. When using LTO the
> binary size reduction is even more effective. It could also be argued
> that this could bring some security advantages.
> 
> The original cover letter with lots of test results can be found here:
> 
> 	https://lkml.org/lkml/2016/2/8/813
> 
> Please consider for merging into your tree.
> 
> Alternately, the following branch can be pulled:
> 
> 	http://git.linaro.org/people/nicolas.pitre/linux.git autoksyms
> 
> Thanks.
> 
> Changes from v6:
> 
> - Rebased on v4.6-rc1, including adjustments to the recently introduced
>   rule_as_o_S.
> 
> - Explicitly ignored more if_changed_dep callers for which no EXPORT_SYMBOL
>   can be parsed.
> 
> Changes from v5:
> 
> - Disable automatic dependency file generation during the preprocessor
>   pass when gathering exported symbol names.  Not only it is redundant
>   but in some conditions this crashes fixdep that might be reading the
>   previous dependency file at the same time. Reported by Michal Marek.
> 
> - Redirect error messages to stderr rather than pass it as a symbol name
>   dependency.
> 
> Changes from v4:
> 
> - Correctness changes plus small cleanup to adjust_autoksyms.sh as
>   suggested by Michal Marek.
> 
> - Changed ksym build dependency generation by re-running the preprocessor
>   rather than collecting those dependencies as warnings through stderr
>   which was too fragile.
> 
> Changes from v3:
> 
> - Shell portability changes to adjust_autoksyms.sh, partly from
>   suggestions by Zev Weiss.
> 
> - Fix sample modules by building them before adjust_autoksyms.sh is run.
> 
> Changes from v2:
> 
> - Generating the build dependencies by parsing the source with fixdep
>   turned out to be unreliable due to all the EXPORT_SYMBOL() variants,
>   and especially their use within macros where the actual symbol name
>   is known only after running the preprocessor. This list of symbol names
>   is now obtained from the preprocessor directly, fixing allmodconfig
>   builds.
> 
> Changes from v1:
> 
> - Replaced "exp" that doesn't convey the right meaning as noted by
>   Sam Ravnborg. The "ksym" identifier is actually what the kernel
>   already uses for this. Therefore:
>   - CONFIG_TRIM_UNUSED_EXPSYMS --> CONFIG_TRIM_UNUSED_KSYMS
>   - include/generated/expsyms.h --> include/generated/autoksyms.h
>   - #define __EXPSYM_* --> #define __KSYM_*
> 
> - Some sed regexp improvements as suggested by Al Viro.
> 
> - Renamed vmlinux_recursive target to autoksyms_recursive.
> 
> - Accept EXPORT_SYMBOL variants with a prefix, e.g. ACPI_EXPORT_SYMBOL.
> 
> - Minor commit log clarifications.
> 
> - Added Rusty's ACK.
> 
> diffstat:
> 
>  Makefile                    |  23 +++++++--
>  include/linux/export.h      |  33 ++++++++++++-
>  init/Kconfig                |  16 ++++++
>  scripts/Kbuild.include      |  32 +++++++++++-
>  scripts/Makefile.build      |  32 ++++++------
>  scripts/adjust_autoksyms.sh | 101 ++++++++++++++++++++++++++++++++++++++
>  scripts/basic/fixdep.c      |  61 +++++++++++++++++------
>  7 files changed, 261 insertions(+), 37 deletions(-)
> 
> 

Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Nicolas Pitre]

Ping !

/me only hears back empty cave echoes ...


On Wed, 6 Apr 2016, Nicolas Pitre wrote:

> 
> Gentle ping...
> 
> If possible, I'd like for those patches to be sorted out before the 
> opening of the merge window gets too close like last time.
> 
> 
> On Tue, 29 Mar 2016, Nicolas Pitre wrote:
> 
> > This patch series provides the option to omit exported symbols from
> > the kernel and modules that are never referenced by any of the selected
> > modules in the current kernel configuration. this allows for optimizing
> > the compiled code and reducing final binaries' size. When using LTO the
> > binary size reduction is even more effective. It could also be argued
> > that this could bring some security advantages.
> > 
> > The original cover letter with lots of test results can be found here:
> > 
> > 	https://lkml.org/lkml/2016/2/8/813
> > 
> > Please consider for merging into your tree.
> > 
> > Alternately, the following branch can be pulled:
> > 
> > 	http://git.linaro.org/people/nicolas.pitre/linux.git autoksyms
> > 
> > Thanks.
> > 
> > Changes from v6:
> > 
> > - Rebased on v4.6-rc1, including adjustments to the recently introduced
> >   rule_as_o_S.
> > 
> > - Explicitly ignored more if_changed_dep callers for which no EXPORT_SYMBOL
> >   can be parsed.
> > 
> > Changes from v5:
> > 
> > - Disable automatic dependency file generation during the preprocessor
> >   pass when gathering exported symbol names.  Not only it is redundant
> >   but in some conditions this crashes fixdep that might be reading the
> >   previous dependency file at the same time. Reported by Michal Marek.
> > 
> > - Redirect error messages to stderr rather than pass it as a symbol name
> >   dependency.
> > 
> > Changes from v4:
> > 
> > - Correctness changes plus small cleanup to adjust_autoksyms.sh as
> >   suggested by Michal Marek.
> > 
> > - Changed ksym build dependency generation by re-running the preprocessor
> >   rather than collecting those dependencies as warnings through stderr
> >   which was too fragile.
> > 
> > Changes from v3:
> > 
> > - Shell portability changes to adjust_autoksyms.sh, partly from
> >   suggestions by Zev Weiss.
> > 
> > - Fix sample modules by building them before adjust_autoksyms.sh is run.
> > 
> > Changes from v2:
> > 
> > - Generating the build dependencies by parsing the source with fixdep
> >   turned out to be unreliable due to all the EXPORT_SYMBOL() variants,
> >   and especially their use within macros where the actual symbol name
> >   is known only after running the preprocessor. This list of symbol names
> >   is now obtained from the preprocessor directly, fixing allmodconfig
> >   builds.
> > 
> > Changes from v1:
> > 
> > - Replaced "exp" that doesn't convey the right meaning as noted by
> >   Sam Ravnborg. The "ksym" identifier is actually what the kernel
> >   already uses for this. Therefore:
> >   - CONFIG_TRIM_UNUSED_EXPSYMS --> CONFIG_TRIM_UNUSED_KSYMS
> >   - include/generated/expsyms.h --> include/generated/autoksyms.h
> >   - #define __EXPSYM_* --> #define __KSYM_*
> > 
> > - Some sed regexp improvements as suggested by Al Viro.
> > 
> > - Renamed vmlinux_recursive target to autoksyms_recursive.
> > 
> > - Accept EXPORT_SYMBOL variants with a prefix, e.g. ACPI_EXPORT_SYMBOL.
> > 
> > - Minor commit log clarifications.
> > 
> > - Added Rusty's ACK.
> > 
> > diffstat:
> > 
> >  Makefile                    |  23 +++++++--
> >  include/linux/export.h      |  33 ++++++++++++-
> >  init/Kconfig                |  16 ++++++
> >  scripts/Kbuild.include      |  32 +++++++++++-
> >  scripts/Makefile.build      |  32 ++++++------
> >  scripts/adjust_autoksyms.sh | 101 ++++++++++++++++++++++++++++++++++++++
> >  scripts/basic/fixdep.c      |  61 +++++++++++++++++------
> >  7 files changed, 261 insertions(+), 37 deletions(-)
> > 
> > 
> 
> 

Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Michal Marek]

Dne 19.4.2016 v 05:32 Nicolas Pitre napsal(a):
> Ping !
> 
> /me only hears back empty cave echoes ...

Pulled now, sorry for the delay.

Michal

Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Nicolas Pitre]

On Wed, 20 Apr 2016, Michal Marek wrote:

> Dne 19.4.2016 v 05:32 Nicolas Pitre napsal(a):
> > Ping !
> > 
> > /me only hears back empty cave echoes ...
> 
> Pulled now, sorry for the delay.

Thanks!


Nicolas

Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Nicolas Pitre]

On Wed, 20 Apr 2016, Nicolas Pitre wrote:

> On Wed, 20 Apr 2016, Michal Marek wrote:
> 
> > Dne 19.4.2016 v 05:32 Nicolas Pitre napsal(a):
> > > Ping !
> > > 
> > > /me only hears back empty cave echoes ...
> > 
> > Pulled now, sorry for the delay.
> 
> Thanks!

Looking at your kbuild branch, I see subsequent changes that require 
minor adjustments. Please merge the following patch:

----- >8
>From a14d6e7a2902af78ee43ab075c7caf829fdfa4b2 Mon Sep 17 00:00:00 2001
From: Nicolas Pitre <nicolas.pitre@linaro.org>
Date: Wed, 20 Apr 2016 15:16:01 -0400
Subject: [PATCH] kbuild: adjust ksym_dep_filter for some cmd renames

The following renames occurred recently:

  cmd_cc_i_c --> cmd_cpp_i_c
  cmd_as_s_S --> cmd_cpp_s_S

The respective cc_*_c and as_*_S patterns no longer match the above
therefore additional patterns are needed.

Signed-off-by: Nicolas Pitre <nico@linaro.org>

diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index a09927e027..bd6bd048cf 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -275,8 +275,10 @@ else
 flags_nodeps = $(filter-out -Wp$(comma)-M%, $($(1)))
 ksym_dep_filter =                                                            \
 	case "$(1)" in                                                       \
-	cc_*_c) $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;    \
-	as_*_S) $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;    \
+	cc_*_c)  $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;   \
+	as_*_S)  $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;   \
+	cpp_i_c) $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;   \
+	cpp_s_S) $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;   \
 	boot*|build*|*cpp_lds_S|dtc|host*|vdso*) : ;;                        \
 	*) echo "Don't know how to preprocess $(1)" >&2; false ;;            \
 	esac | sed -rn 's/^.*=== __KSYM_(.*) ===.*$$/KSYM_\1/p'


Nicolas

Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Michal Marek]

On 2016-04-20 21:26, Nicolas Pitre wrote:
> On Wed, 20 Apr 2016, Nicolas Pitre wrote:
> 
>> On Wed, 20 Apr 2016, Michal Marek wrote:
>>
>>> Dne 19.4.2016 v 05:32 Nicolas Pitre napsal(a):
>>>> Ping !
>>>>
>>>> /me only hears back empty cave echoes ...
>>>
>>> Pulled now, sorry for the delay.
>>
>> Thanks!
> 
> Looking at your kbuild branch, I see subsequent changes that require 
> minor adjustments. Please merge the following patch:
> 
> ----- >8
> From a14d6e7a2902af78ee43ab075c7caf829fdfa4b2 Mon Sep 17 00:00:00 2001
> From: Nicolas Pitre <nicolas.pitre@linaro.org>
> Date: Wed, 20 Apr 2016 15:16:01 -0400
> Subject: [PATCH] kbuild: adjust ksym_dep_filter for some cmd renames
> 
> The following renames occurred recently:
> 
>   cmd_cc_i_c --> cmd_cpp_i_c
>   cmd_as_s_S --> cmd_cpp_s_S
> 
> The respective cc_*_c and as_*_S patterns no longer match the above
> therefore additional patterns are needed.
> 
> Signed-off-by: Nicolas Pitre <nico@linaro.org>
> 
> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> index a09927e027..bd6bd048cf 100644
> --- a/scripts/Kbuild.include
> +++ b/scripts/Kbuild.include
> @@ -275,8 +275,10 @@ else
>  flags_nodeps = $(filter-out -Wp$(comma)-M%, $($(1)))
>  ksym_dep_filter =                                                            \
>  	case "$(1)" in                                                       \
> -	cc_*_c) $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;    \
> -	as_*_S) $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;    \
> +	cc_*_c)  $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;   \
> +	as_*_S)  $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;   \
> +	cpp_i_c) $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;   \
> +	cpp_s_S) $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;   \

Please make it
cc_*_c | cpp_i_c) ...
as_*_S | cpp_s_S) ...

Thanks,
Michal

Re: [PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Nicolas Pitre]

On Tue, 26 Apr 2016, Michal Marek wrote:

> On 2016-04-20 21:26, Nicolas Pitre wrote:
> > On Wed, 20 Apr 2016, Nicolas Pitre wrote:
> > 
> >> On Wed, 20 Apr 2016, Michal Marek wrote:
> >>
> >>> Dne 19.4.2016 v 05:32 Nicolas Pitre napsal(a):
> >>>> Ping !
> >>>>
> >>>> /me only hears back empty cave echoes ...
> >>>
> >>> Pulled now, sorry for the delay.
> >>
> >> Thanks!
> > 
> > Looking at your kbuild branch, I see subsequent changes that require 
> > minor adjustments. Please merge the following patch:
> > 
> > ----- >8
> > From a14d6e7a2902af78ee43ab075c7caf829fdfa4b2 Mon Sep 17 00:00:00 2001
> > From: Nicolas Pitre <nicolas.pitre@linaro.org>
> > Date: Wed, 20 Apr 2016 15:16:01 -0400
> > Subject: [PATCH] kbuild: adjust ksym_dep_filter for some cmd renames
> > 
> > The following renames occurred recently:
> > 
> >   cmd_cc_i_c --> cmd_cpp_i_c
> >   cmd_as_s_S --> cmd_cpp_s_S
> > 
> > The respective cc_*_c and as_*_S patterns no longer match the above
> > therefore additional patterns are needed.
> > 
> > Signed-off-by: Nicolas Pitre <nico@linaro.org>
> > 
> > diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> > index a09927e027..bd6bd048cf 100644
> > --- a/scripts/Kbuild.include
> > +++ b/scripts/Kbuild.include
> > @@ -275,8 +275,10 @@ else
> >  flags_nodeps = $(filter-out -Wp$(comma)-M%, $($(1)))
> >  ksym_dep_filter =                                                            \
> >  	case "$(1)" in                                                       \
> > -	cc_*_c) $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;    \
> > -	as_*_S) $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;    \
> > +	cc_*_c)  $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;   \
> > +	as_*_S)  $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;   \
> > +	cpp_i_c) $(CPP) $(call flags_nodeps,c_flags) -D__KSYM_DEPS__ $< ;;   \
> > +	cpp_s_S) $(CPP) $(call flags_nodeps,a_flags) -D__KSYM_DEPS__ $< ;;   \
> 
> Please make it
> cc_*_c | cpp_i_c) ...
> as_*_S | cpp_s_S) ...

Sure. Patch follows


Nicolas

[PATCH v7 0/8]/[PULL REQUEST] Trim unused exported kernel symbols

[Nicolas Pitre]

This patch series provides the option to omit exported symbols from
the kernel and modules that are never referenced by any of the selected
modules in the current kernel configuration. this allows for optimizing
the compiled code and reducing final binaries' size. When using LTO the
binary size reduction is even more effective. It could also be argued
that this could bring some security advantages.

The original cover letter with lots of test results can be found here:

	https://lkml.org/lkml/2016/2/8/813

Please consider for merging into your tree.

Alternately, the following branch can be pulled:

	http://git.linaro.org/people/nicolas.pitre/linux.git autoksyms

Thanks.

Changes from v6:

- Rebased on v4.6-rc1, including adjustments to the recently introduced
  rule_as_o_S.

- Explicitly ignored more if_changed_dep callers for which no EXPORT_SYMBOL
  can be parsed.

Changes from v5:

- Disable automatic dependency file generation during the preprocessor
  pass when gathering exported symbol names.  Not only it is redundant
  but in some conditions this crashes fixdep that might be reading the
  previous dependency file at the same time. Reported by Michal Marek.

- Redirect error messages to stderr rather than pass it as a symbol name
  dependency.

Changes from v4:

- Correctness changes plus small cleanup to adjust_autoksyms.sh as
  suggested by Michal Marek.

- Changed ksym build dependency generation by re-running the preprocessor
  rather than collecting those dependencies as warnings through stderr
  which was too fragile.

Changes from v3:

- Shell portability changes to adjust_autoksyms.sh, partly from
  suggestions by Zev Weiss.

- Fix sample modules by building them before adjust_autoksyms.sh is run.

Changes from v2:

- Generating the build dependencies by parsing the source with fixdep
  turned out to be unreliable due to all the EXPORT_SYMBOL() variants,
  and especially their use within macros where the actual symbol name
  is known only after running the preprocessor. This list of symbol names
  is now obtained from the preprocessor directly, fixing allmodconfig
  builds.

Changes from v1:

- Replaced "exp" that doesn't convey the right meaning as noted by
  Sam Ravnborg. The "ksym" identifier is actually what the kernel
  already uses for this. Therefore:
  - CONFIG_TRIM_UNUSED_EXPSYMS --> CONFIG_TRIM_UNUSED_KSYMS
  - include/generated/expsyms.h --> include/generated/autoksyms.h
  - #define __EXPSYM_* --> #define __KSYM_*

- Some sed regexp improvements as suggested by Al Viro.

- Renamed vmlinux_recursive target to autoksyms_recursive.

- Accept EXPORT_SYMBOL variants with a prefix, e.g. ACPI_EXPORT_SYMBOL.

- Minor commit log clarifications.

- Added Rusty's ACK.

diffstat:

 Makefile                    |  23 +++++++--
 include/linux/export.h      |  33 ++++++++++++-
 init/Kconfig                |  16 ++++++
 scripts/Kbuild.include      |  32 +++++++++++-
 scripts/Makefile.build      |  32 ++++++------
 scripts/adjust_autoksyms.sh | 101 ++++++++++++++++++++++++++++++++++++++
 scripts/basic/fixdep.c      |  61 +++++++++++++++++------
 7 files changed, 261 insertions(+), 37 deletions(-)