[PATCH 0/2] improve modversions for symbols exported from assembly code

[PATCH 0/2] improve modversions for symbols exported from assembly code

[Nicolas Pitre]

This is a partial repost of the patches I sent last week, without the
ARM part this time, in the hope that this can be considered for the next
merge window so that architecture specific changes can start using it
afterwards.


 include/asm-generic/export.h | 24 ++++++++++++++++++++++++
 scripts/Makefile.build       | 38 +++++++++++++++++++-------------------
 2 files changed, 43 insertions(+), 19 deletions(-)

[PATCH 1/2] kbuild: improve EXPORT_SYMBOL() parsing from asm code

[Nicolas Pitre]

First, make the asm-prototypes.h presence optional.  The next patch will
make it unneeded for modversion support.

Use the -D__GENKSYMS__ like we do for .c files but to expand the
EXPORT_SYMBOL macro using the preprocessor instead of a sed script.
The preprocessor output parsing is then limited to a simpler filtering
and made more robust against multiple assembly statements on a single
line.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
---
 include/asm-generic/export.h |  9 +++++++++
 scripts/Makefile.build       | 20 ++++++--------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 63554e9f6e..39a19dc366 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -28,6 +28,8 @@
 #define KSYM(name) name
 #endif
 
+#if defined(__KERNEL__) && !defined(__GENKSYMS__)
+
 /*
  * note on .section use: @progbits vs %progbits nastiness doesn't matter,
  * since we immediately emit into those sections anyway.
@@ -82,6 +84,13 @@ KSYM(__kcrctab_\name):
 #define __EXPORT_SYMBOL(sym, val, sec) ___EXPORT_SYMBOL sym, val, sec
 #endif
 
+#else /* __GENKSYMS__ */
+
+/* create a preprocessor output suitable for cmd_gensymtypes_S */
+#define __EXPORT_SYMBOL(sym, val, sec) EXPORT_SYMBOL(sym)
+
+#endif /* __GENKSYMS__ */
+
 #define EXPORT_SYMBOL(name)					\
 	__EXPORT_SYMBOL(name, KSYM_FUNC(KSYM(name)),)
 #define EXPORT_SYMBOL_GPL(name) 				\
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 7675d11ee6..ebf6e08ae4 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -325,15 +325,15 @@ $(real-objs-m:.o=.s): modkern_aflags := $(KBUILD_AFLAGS_MODULE) $(AFLAGS_MODULE)
 # This is convoluted. The .S file must first be preprocessed to run guards and
 # expand names, then the resulting exports must be constructed into plain
 # EXPORT_SYMBOL(symbol); to build our dummy C file, and that gets preprocessed
-# to make the genksyms input.
+# to make the genksyms input. See also include/asm-generic/export.h.
 #
 # These mirror gensymtypes_c and co above, keep them in synch.
 cmd_gensymtypes_S =                                                         \
-    (echo "\#include <linux/kernel.h>" ;                                    \
-     echo "\#include <asm/asm-prototypes.h>" ;                              \
-    $(CPP) $(a_flags) $< |                                                  \
-     grep "\<___EXPORT_SYMBOL\>" |                                          \
-     sed 's/.*___EXPORT_SYMBOL[[:space:]]*\([a-zA-Z0-9_]*\)[[:space:]]*,.*/EXPORT_SYMBOL(\1);/' ) | \
+    ( echo "\#include <linux/kernel.h>" ;                                   \
+      if [ -e $(srctree)/arch/$(SRCARCH)/include/asm/asm-prototypes.h ];    \
+      then echo "\#include <asm/asm-prototypes.h>"; fi;                     \
+      $(CPP) -D__GENKSYMS__ $(a_flags) $< | tr ";" "\n" |                   \
+      sed -n -e '/EXPORT_SYMBOL(/s/$$/;/p' ) |                              \
     $(CPP) -D__GENKSYMS__ $(c_flags) -xc - |                                \
     $(GENKSYMS) $(if $(1), -T $(2))                                         \
      $(patsubst y,-s _,$(CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX))             \
@@ -363,13 +363,6 @@ cmd_as_o_S = $(CC) $(a_flags) -c -o $@ $<
 
 else
 
-ASM_PROTOTYPES := $(wildcard $(srctree)/arch/$(SRCARCH)/include/asm/asm-prototypes.h)
-
-ifeq ($(ASM_PROTOTYPES),)
-cmd_as_o_S = $(CC) $(a_flags) -c -o $@ $<
-
-else
-
 # versioning matches the C process described above, with difference that
 # we parse asm-prototypes.h C header to get function definitions.
 
@@ -387,7 +380,6 @@ cmd_modversions_S =								\
 		mv -f $(@D)/.tmp_$(@F) $@;					\
 	fi;
 endif
-endif
 
 $(obj)/%.o: $(src)/%.S $(objtool_obj) FORCE
 	$(call if_changed_rule,as_o_S)
-- 
2.7.4

[PATCH 2/2] kbuild: make modversion for exported asm symbols more convivial

[Nicolas Pitre]

Rather than having an asm-prototypes.h file where C prototypes for exported
asm symbols are centralized, let's have some macros that can be used
directly in the code where those symbols are exported for genksyms
consumption.  Either the prototype is provided directly if no include
files has it, or the include file containing it is specified.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
---
 include/asm-generic/export.h | 15 +++++++++++++++
 scripts/Makefile.build       | 22 +++++++++++++++-------
 2 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 39a19dc366..83dda5f840 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -84,11 +84,26 @@ KSYM(__kcrctab_\name):
 #define __EXPORT_SYMBOL(sym, val, sec) ___EXPORT_SYMBOL sym, val, sec
 #endif
 
+/* in the non genksyms case those are no-ops */
+#define SYMBOL_CPROTO(expr)
+#define SYMBOL_CPROTO_INCLUDE(file)
+
 #else /* __GENKSYMS__ */
 
 /* create a preprocessor output suitable for cmd_gensymtypes_S */
 #define __EXPORT_SYMBOL(sym, val, sec) EXPORT_SYMBOL(sym)
 
+/*
+ * The cmd_gensymtypes_S kbuild command recognizes the following:
+ *
+ * Provide a C prototype for genksyms: SYMBOL_CPROTO(expr)
+ * example: SYMBOL_CPROTO(void foobar(int x, int y))
+ *
+ * Specify an include file that contains C prototypes for genksyms:
+ * SYMBOL_CPROTO_INCLUDE(quoted_filename)
+ * example: SYMBOL_CPROTO_INCLUDE(<linux/string.h>)
+ */
+
 #endif /* __GENKSYMS__ */
 
 #define EXPORT_SYMBOL(name)					\
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index ebf6e08ae4..58ebc4b15d 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -317,24 +317,32 @@ modkern_aflags := $(KBUILD_AFLAGS_KERNEL) $(AFLAGS_KERNEL)
 $(real-objs-m)      : modkern_aflags := $(KBUILD_AFLAGS_MODULE) $(AFLAGS_MODULE)
 $(real-objs-m:.o=.s): modkern_aflags := $(KBUILD_AFLAGS_MODULE) $(AFLAGS_MODULE)
 
-# .S file exports must have their C prototypes defined in asm/asm-prototypes.h
-# or a file that it includes, in order to get versioned symbols. We build a
-# dummy C file that includes asm-prototypes and the EXPORT_SYMBOL lines from
-# the .S file (with trailing ';'), and run genksyms on that, to extract vers.
+# In order to get versioned symbols, .S file exports must have their C prototypes:
+# - defined in asm/asm-prototypes.h or a file that it includes, or
+# - provided in a file specified by SYMBOL_CPROTO_INCLUDE(), or
+# - specified directly by SYMBOL_CPROTO().
+# We build a dummy C file that includes asm-prototypes and the EXPORT_SYMBOL
+# lines from the .S file (with trailing ';'), and run genksyms on that, to
+# extract vers.
 #
 # This is convoluted. The .S file must first be preprocessed to run guards and
 # expand names, then the resulting exports must be constructed into plain
 # EXPORT_SYMBOL(symbol); to build our dummy C file, and that gets preprocessed
 # to make the genksyms input. See also include/asm-generic/export.h.
 #
+# The -Ulinux is necessary otherwise SYMBOL_CPROTO_INCLUDE(<linux/bitops.h>)
+# is turned into #include <1/bitops.h> due to "linux" being defined to 1.
+#
 # These mirror gensymtypes_c and co above, keep them in synch.
 cmd_gensymtypes_S =                                                         \
     ( echo "\#include <linux/kernel.h>" ;                                   \
       if [ -e $(srctree)/arch/$(SRCARCH)/include/asm/asm-prototypes.h ];    \
       then echo "\#include <asm/asm-prototypes.h>"; fi;                     \
-      $(CPP) -D__GENKSYMS__ $(a_flags) $< | tr ";" "\n" |                   \
-      sed -n -e '/EXPORT_SYMBOL(/s/$$/;/p' ) |                              \
-    $(CPP) -D__GENKSYMS__ $(c_flags) -xc - |                                \
+      $(CPP) -D__GENKSYMS__ $(a_flags) -Ulinux $< | tr ";" "\n" |           \
+      sed -n -e '/EXPORT_SYMBOL(/s/$$/;/p'                                  \
+             -e 's/SYMBOL_CPROTO(\(.*\))/\1;/p'                             \
+             -e 's/SYMBOL_CPROTO_INCLUDE(\(.*\))/\#include \1/p' ) |        \
+    $(CPP) -D__GENKSYMS__ -I$(@D) $(c_flags) -xc - |                        \
     $(GENKSYMS) $(if $(1), -T $(2))                                         \
      $(patsubst y,-s _,$(CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX))             \
      $(if $(KBUILD_PRESERVE),-p)                                            \
-- 
2.7.4

Re: [PATCH 2/2] kbuild: make modversion for exported asm symbols more convivial

[Michal Marek]

Dne 8.12.2016 v 05:24 Nicolas Pitre napsal(a):
> Rather than having an asm-prototypes.h file where C prototypes for exported
> asm symbols are centralized, let's have some macros that can be used
> directly in the code where those symbols are exported for genksyms
> consumption.  Either the prototype is provided directly if no include
> files has it, or the include file containing it is specified.
> 
> Signed-off-by: Nicolas Pitre <nico@linaro.org>
> ---
>  include/asm-generic/export.h | 15 +++++++++++++++
>  scripts/Makefile.build       | 22 +++++++++++++++-------
>  2 files changed, 30 insertions(+), 7 deletions(-)
> 
> diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
> index 39a19dc366..83dda5f840 100644
> --- a/include/asm-generic/export.h
> +++ b/include/asm-generic/export.h
> @@ -84,11 +84,26 @@ KSYM(__kcrctab_\name):
>  #define __EXPORT_SYMBOL(sym, val, sec) ___EXPORT_SYMBOL sym, val, sec
>  #endif
>  
> +/* in the non genksyms case those are no-ops */
> +#define SYMBOL_CPROTO(expr)
> +#define SYMBOL_CPROTO_INCLUDE(file)

Do we really _need_ the SYMBOL_CPROTO macro? The exported functions are
called from C files presumably, so there ought to be headers with the
declarations. If these headers can't be included easily, we should fix
them, but having copies of the declarations in the asm files is no big
improvement over the asm-prototypes.h approach, IMO.

Michal

Re: [PATCH 2/2] kbuild: make modversion for exported asm symbols more convivial

[Nicolas Pitre]

On Wed, 14 Dec 2016, Michal Marek wrote:

> Dne 8.12.2016 v 05:24 Nicolas Pitre napsal(a):
> > Rather than having an asm-prototypes.h file where C prototypes for exported
> > asm symbols are centralized, let's have some macros that can be used
> > directly in the code where those symbols are exported for genksyms
> > consumption.  Either the prototype is provided directly if no include
> > files has it, or the include file containing it is specified.
> > 
> > Signed-off-by: Nicolas Pitre <nico@linaro.org>
> > ---
> >  include/asm-generic/export.h | 15 +++++++++++++++
> >  scripts/Makefile.build       | 22 +++++++++++++++-------
> >  2 files changed, 30 insertions(+), 7 deletions(-)
> > 
> > diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
> > index 39a19dc366..83dda5f840 100644
> > --- a/include/asm-generic/export.h
> > +++ b/include/asm-generic/export.h
> > @@ -84,11 +84,26 @@ KSYM(__kcrctab_\name):
> >  #define __EXPORT_SYMBOL(sym, val, sec) ___EXPORT_SYMBOL sym, val, sec
> >  #endif
> >  
> > +/* in the non genksyms case those are no-ops */
> > +#define SYMBOL_CPROTO(expr)
> > +#define SYMBOL_CPROTO_INCLUDE(file)
> 
> Do we really _need_ the SYMBOL_CPROTO macro? The exported functions are
> called from C files presumably, so there ought to be headers with the
> declarations. If these headers can't be included easily, we should fix
> them, but having copies of the declarations in the asm files is no big
> improvement over the asm-prototypes.h approach, IMO.

On ARM there are a few symbols that are part of the gcc support library 
such as division routines and so on.  Those are never called directly 
from C code. It is the compiler that implicitly creates references to 
them.  However, in order to be able to export those symbols, dummy C 
prototypes were used before it was possible to export symbols from asm 
code but those prototypes make no sense otherwise.  So the SYMBOL_CPROTO 
macro is there mainly to maintain backward compatibility with the 
traditional symbol version signature for those symbols.

The SYMBOL_CPROTO macro, being close to the actual code, could mark the 
intended definition for symbols in assembly code and allow for a tool to 
ensure there is no mismatch with the actual declaration located 
elsewhere. That could be useful for all global symbols, not just 
exported ones.  But that's not the primary reason why I created it.


Nicolas