Linux-Modules Archive on lore.kernel.org
 help / color / Atom feed
* Re: [PATCH 2/6] module: add support for symbol namespaces.
       [not found]     ` <20180725155507.umb5oatduquxwlmt@linux-8ccs>
@ 2018-07-25 16:48       ` Lucas De Marchi
  2018-07-26  7:44         ` Martijn Coenen
  0 siblings, 1 reply; 51+ messages in thread
From: Lucas De Marchi @ 2018-07-25 16:48 UTC (permalink / raw)
  To: Jessica Yu
  Cc: maco, lkml, yamada.masahiro, michal.lkml, geert, Thomas Gleixner,
	mingo, H. Peter Anvin, x86, Alan Stern, Greg KH, oneukum,
	Arnd Bergmann, sboyd, pombredanne, kstewart, sam, linux-kbuild,
	linux-m68k, linux-usb, usb-storage, linux-scsi, linux-arch, maco,
	sspatil, malchev, joelaf, linux-modules

On Wed, Jul 25, 2018 at 8:55 AM Jessica Yu <jeyu@kernel.org> wrote:
>
> +++ Martijn Coenen [24/07/18 09:56 +0200]:
> >I did find an issue with my approach:
> >
> >On Mon, Jul 16, 2018 at 2:21 PM, Martijn Coenen <maco@android.com> wrote:
> >> The ELF symbols are renamed to include the namespace with an asm label;
> >> for example, symbol 'usb_stor_suspend' in namespace USB_STORAGE becomes
> >> 'usb_stor_suspend.USB_STORAGE'.  This allows modpost to do namespace
> >> checking, without having to go through all the effort of parsing ELF and
> >> reloction records just to get to the struct kernel_symbols.
> >
> >depmod doesn't like this: if namespaced symbols are built-in to the
> >kernel, they will appear as 'symbol.NS' in the symbol table, whereas
> >modules using those symbols just depend on 'symbol'. This will cause
> >depmod to warn about unknown symbols. I didn't find this previously
> >because all the exports/imports I tested were done from modules
> >themselves.
> >
> >One way to deal with it is to change depmod, but it looks like it
> >hasn't been changed in ages, and it would also introduce a dependency

this might be because you are looking to the wrong project
(module-init-tools) rather than kmod that replaced it some years ago?

> >on userspaces to update it to avoid these warnings. So, we'd have to
> >encode the symbol namespace information in another way for modpost to
> >use. I could just write a separate post-processing tool (much like
> >genksyms), or perhaps encode the information in a discardable section.
> >Any other suggestions welcome.
>
> This seems to be because depmod uses System.map as a source for kernel
> symbols and scans for only the ones prefixed with "__ksymtab" to find
> the exported ones - and those happen to use the '.' to mark the
> namespace it belongs to. It strips that prefix and uses the remainder
> as the actual symbol name. To fix that it'd have to strip the
> namespace suffix in the symbol name as well.
>
> Just scanning the depmod source code, it seems really trivial to just
> have depmod accommodate the new symbol name format. Adding Lucas (kmod
> maintainer) to CC.

Yep, that would be easy and allow depmod to be backward compatible
since we would do nothing if the symbol doesn't have the suffix.
As for dependency on a new version, this seems trivial enough to be
backported to previous releases used on distros so even if they are
not rolling they would get a compatible depmod.

CC'ing linux-modules@vger.kernel.org to keep track of this there


thanks
Lucas De Marchi

>
> Thanks,
>
> Jessica
>
> >
> >>
> >> On x86_64 I saw no difference in binary size (compression), but at
> >> runtime this will require a word of memory per export to hold the
> >> namespace. An alternative could be to store namespaced symbols in their
> >> own section and use a separate 'struct namespaced_kernel_symbol' for
> >> that section, at the cost of making the module loader more complex.
> >>
> >> Signed-off-by: Martijn Coenen <maco@android.com>
> >> ---
> >>  include/asm-generic/export.h |  2 +-
> >>  include/linux/export.h       | 83 +++++++++++++++++++++++++++---------
> >>  include/linux/module.h       | 13 ++++++
> >>  kernel/module.c              | 79 ++++++++++++++++++++++++++++++++++
> >>  4 files changed, 156 insertions(+), 21 deletions(-)
> >>
> >> diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
> >> index 68efb950a918..4c3d1afb702f 100644
> >> --- a/include/asm-generic/export.h
> >> +++ b/include/asm-generic/export.h
> >> @@ -29,7 +29,7 @@
> >>         .section ___ksymtab\sec+\name,"a"
> >>         .balign KSYM_ALIGN
> >>  __ksymtab_\name:
> >> -       __put \val, __kstrtab_\name
> >> +       __put \val, __kstrtab_\name, 0
> >>         .previous
> >>         .section __ksymtab_strings,"a"
> >>  __kstrtab_\name:
> >> diff --git a/include/linux/export.h b/include/linux/export.h
> >> index ad6b8e697b27..9f6e70eeb85f 100644
> >> --- a/include/linux/export.h
> >> +++ b/include/linux/export.h
> >> @@ -22,6 +22,11 @@ struct kernel_symbol
> >>  {
> >>         unsigned long value;
> >>         const char *name;
> >> +       const char *namespace;
> >> +};
> >> +
> >> +struct namespace_import {
> >> +       const char *namespace;
> >>  };
> >>
> >>  #ifdef MODULE
> >> @@ -54,18 +59,28 @@ extern struct module __this_module;
> >>  #define __CRC_SYMBOL(sym, sec)
> >>  #endif
> >>
> >> +#define NS_SEPARATOR "."
> >> +
> >> +#define MODULE_IMPORT_NS(ns)                                           \
> >> +       static const struct namespace_import __knsimport_##ns           \
> >> +       asm("__knsimport_" #ns)                                         \
> >> +       __attribute__((section("__knsimport"), used))                   \
> >> +       = { #ns }
> >> +
> >>  /* For every exported symbol, place a struct in the __ksymtab section */
> >> -#define ___EXPORT_SYMBOL(sym, sec)                                     \
> >> +#define ___EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2)                        \
> >>         extern typeof(sym) sym;                                         \
> >>         __CRC_SYMBOL(sym, sec)                                          \
> >> -       static const char __kstrtab_##sym[]                             \
> >> +       static const char __kstrtab_##sym##nspost[]                     \
> >>         __attribute__((section("__ksymtab_strings"), aligned(1)))       \
> >>         = #sym;                                                         \
> >> -       static const struct kernel_symbol __ksymtab_##sym               \
> >> +       static const struct kernel_symbol __ksymtab_##sym##nspost       \
> >> +       asm("__ksymtab_" #sym nspost2)                                  \
> >>         __used                                                          \
> >> -       __attribute__((section("___ksymtab" sec "+" #sym), used))       \
> >> +       __attribute__((section("___ksymtab" sec "+" #sym #nspost)))     \
> >> +       __attribute__((used))                                           \
> >>         __attribute__((aligned(sizeof(void *))))                        \
> >> -       = { (unsigned long)&sym, __kstrtab_##sym }
> >> +       = { (unsigned long)&sym, __kstrtab_##sym##nspost, ns }
> >>
> >>  #if defined(__KSYM_DEPS__)
> >>
> >> @@ -76,52 +91,80 @@ extern struct module __this_module;
> >>   * system filters out from the preprocessor output (see ksym_dep_filter
> >>   * in scripts/Kbuild.include).
> >>   */
> >> -#define __EXPORT_SYMBOL(sym, sec)      === __KSYM_##sym ===
> >> +#define __EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2) === __KSYM_##sym ===
> >>
> >>  #elif defined(CONFIG_TRIM_UNUSED_KSYMS)
> >>
> >>  #include <generated/autoksyms.h>
> >>
> >> -#define __EXPORT_SYMBOL(sym, sec)                              \
> >> -       __cond_export_sym(sym, sec, __is_defined(__KSYM_##sym))
> >> -#define __cond_export_sym(sym, sec, conf)                      \
> >> -       ___cond_export_sym(sym, sec, conf)
> >> -#define ___cond_export_sym(sym, sec, enabled)                  \
> >> -       __cond_export_sym_##enabled(sym, sec)
> >> -#define __cond_export_sym_1(sym, sec) ___EXPORT_SYMBOL(sym, sec)
> >> -#define __cond_export_sym_0(sym, sec) /* nothing */
> >> +#define __EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2)                 \
> >> +       __cond_export_sym(sym, sec, ns, nspost, nspost2,                \
> >> +                         __is_defined(__KSYM_##sym))
> >> +#define __cond_export_sym(sym, sec, ns, nspost, nspost2, conf)         \
> >> +       ___cond_export_sym(sym, sec, ns, nspost, nspost2, conf)
> >> +#define ___cond_export_sym(sym, sec, ns, nspost, nspost2, enabled)     \
> >> +       __cond_export_sym_##enabled(sym, sec, ns, nspost, nspost2)
> >> +#define __cond_export_sym_1(sym, sec, ns, nspost, nspost2)             \
> >> +       ___EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2)
> >> +#define __cond_export_sym_0(sym, sec, ns, nspost, nspost2) /* nothing */
> >>
> >>  #else
> >>  #define __EXPORT_SYMBOL ___EXPORT_SYMBOL
> >>  #endif
> >>
> >>  #define EXPORT_SYMBOL(sym)                                     \
> >> -       __EXPORT_SYMBOL(sym, "")
> >> +       __EXPORT_SYMBOL(sym, "", NULL, ,)
> >>
> >>  #define EXPORT_SYMBOL_GPL(sym)                                 \
> >> -       __EXPORT_SYMBOL(sym, "_gpl")
> >> +       __EXPORT_SYMBOL(sym, "_gpl", NULL, ,)
> >>
> >>  #define EXPORT_SYMBOL_GPL_FUTURE(sym)                          \
> >> -       __EXPORT_SYMBOL(sym, "_gpl_future")
> >> +       __EXPORT_SYMBOL(sym, "_gpl_future", NULL, ,)
> >> +
> >> +#define EXPORT_SYMBOL_NS(sym, ns)                               \
> >> +       __EXPORT_SYMBOL(sym, "", #ns, __##ns, NS_SEPARATOR #ns)
> >> +
> >> +#define EXPORT_SYMBOL_NS_GPL(sym, ns)                               \
> >> +       __EXPORT_SYMBOL(sym, "_gpl", #ns, __##ns, NS_SEPARATOR #ns)
> >>
> >>  #ifdef CONFIG_UNUSED_SYMBOLS
> >> -#define EXPORT_UNUSED_SYMBOL(sym) __EXPORT_SYMBOL(sym, "_unused")
> >> -#define EXPORT_UNUSED_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_unused_gpl")
> >> +#define EXPORT_UNUSED_SYMBOL(sym) __EXPORT_SYMBOL(sym, "_unused", , ,)
> >> +#define EXPORT_UNUSED_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_unused_gpl", , ,)
> >>  #else
> >>  #define EXPORT_UNUSED_SYMBOL(sym)
> >>  #define EXPORT_UNUSED_SYMBOL_GPL(sym)
> >>  #endif
> >>
> >> -#endif /* __GENKSYMS__ */
> >> +#endif /* __KERNEL__ && !__GENKSYMS__ */
> >> +
> >> +#if defined(__GENKSYMS__)
> >> +/*
> >> + * When we're running genksyms, ignore the namespace and make the _NS
> >> + * variants look like the normal ones. There are two reasons for this:
> >> + * 1) In the normal definition of EXPORT_SYMBOL_NS, the 'ns' macro
> >> + *    argument is itself not expanded because it's always tokenized or
> >> + *    concatenated; but when running genksyms, a blank definition of the
> >> + *    macro does allow the argument to be expanded; if a namespace
> >> + *    happens to collide with a #define, this can cause issues.
> >> + * 2) There's no need to modify genksyms to deal with the _NS variants
> >> + */
> >> +#define EXPORT_SYMBOL_NS(sym, ns)                              \
> >> +       EXPORT_SYMBOL(sym)
> >> +#define EXPORT_SYMBOL_NS_GPL(sym, ns)                          \
> >> +       EXPORT_SYMBOL_GPL(sym)
> >> +#endif
> >>
> >>  #else /* !CONFIG_MODULES... */
> >>
> >>  #define EXPORT_SYMBOL(sym)
> >> +#define EXPORT_SYMBOL_NS(sym, ns)
> >> +#define EXPORT_SYMBOL_NS_GPL(sym, ns)
> >>  #define EXPORT_SYMBOL_GPL(sym)
> >>  #define EXPORT_SYMBOL_GPL_FUTURE(sym)
> >>  #define EXPORT_UNUSED_SYMBOL(sym)
> >>  #define EXPORT_UNUSED_SYMBOL_GPL(sym)
> >>
> >> +#define MODULE_IMPORT_NS(ns)
> >>  #endif /* CONFIG_MODULES */
> >>  #endif /* !__ASSEMBLY__ */
> >>
> >> diff --git a/include/linux/module.h b/include/linux/module.h
> >> index d44df9b2c131..afab4e8fa188 100644
> >> --- a/include/linux/module.h
> >> +++ b/include/linux/module.h
> >> @@ -268,6 +268,12 @@ void *__symbol_get(const char *symbol);
> >>  void *__symbol_get_gpl(const char *symbol);
> >>  #define symbol_get(x) ((typeof(&x))(__symbol_get(VMLINUX_SYMBOL_STR(x))))
> >>
> >> +/* namespace dependencies of the module */
> >> +struct module_ns_dep {
> >> +       struct list_head ns_dep;
> >> +       const char *namespace;
> >> +};
> >> +
> >>  /* modules using other modules: kdb wants to see this. */
> >>  struct module_use {
> >>         struct list_head source_list;
> >> @@ -359,6 +365,13 @@ struct module {
> >>         const struct kernel_symbol *gpl_syms;
> >>         const s32 *gpl_crcs;
> >>
> >> +       /* Namespace imports */
> >> +       unsigned int num_ns_imports;
> >> +       const struct namespace_import *ns_imports;
> >> +
> >> +       /* Namespace dependencies */
> >> +       struct list_head ns_dependencies;
> >> +
> >>  #ifdef CONFIG_UNUSED_SYMBOLS
> >>         /* unused exported symbols. */
> >>         const struct kernel_symbol *unused_syms;
> >> diff --git a/kernel/module.c b/kernel/module.c
> >> index f475f30eed8c..63de0fe849f9 100644
> >> --- a/kernel/module.c
> >> +++ b/kernel/module.c
> >> @@ -1166,6 +1166,51 @@ static inline int module_unload_init(struct module *mod)
> >>  }
> >>  #endif /* CONFIG_MODULE_UNLOAD */
> >>
> >> +static bool module_has_ns_dependency(struct module *mod, const char *ns)
> >> +{
> >> +       struct module_ns_dep *ns_dep;
> >> +
> >> +       list_for_each_entry(ns_dep, &mod->ns_dependencies, ns_dep) {
> >> +               if (strcmp(ns_dep->namespace, ns) == 0)
> >> +                       return true;
> >> +       }
> >> +
> >> +       return false;
> >> +}
> >> +
> >> +static int add_module_ns_dependency(struct module *mod, const char *ns)
> >> +{
> >> +       struct module_ns_dep *ns_dep;
> >> +
> >> +       if (module_has_ns_dependency(mod, ns))
> >> +               return 0;
> >> +
> >> +       ns_dep = kmalloc(sizeof(*ns_dep), GFP_ATOMIC);
> >> +       if (!ns_dep)
> >> +               return -ENOMEM;
> >> +
> >> +       ns_dep->namespace = ns;
> >> +
> >> +       list_add(&ns_dep->ns_dep, &mod->ns_dependencies);
> >> +
> >> +       return 0;
> >> +}
> >> +
> >> +static bool module_imports_ns(struct module *mod, const char *ns)
> >> +{
> >> +       size_t i;
> >> +
> >> +       if (!ns)
> >> +               return true;
> >> +
> >> +       for (i = 0; i < mod->num_ns_imports; ++i) {
> >> +               if (!strcmp(mod->ns_imports[i].namespace, ns))
> >> +                       return true;
> >> +       }
> >> +
> >> +       return false;
> >> +}
> >> +
> >>  static size_t module_flags_taint(struct module *mod, char *buf)
> >>  {
> >>         size_t l = 0;
> >> @@ -1415,6 +1460,18 @@ static const struct kernel_symbol *resolve_symbol(struct module *mod,
> >>                 goto getname;
> >>         }
> >>
> >> +       /*
> >> +        * We can't yet verify that the module actually imports this
> >> +        * namespace, because the imports themselves are only available
> >> +        * after processing the symbol table and doing relocation; so
> >> +        * instead just record the dependency and check later.
> >> +        */
> >> +       if (sym->namespace) {
> >> +               err = add_module_ns_dependency(mod, sym->namespace);
> >> +               if (err)
> >> +                       sym = ERR_PTR(err);
> >> +       }
> >> +
> >>  getname:
> >>         /* We must make copy under the lock if we failed to get ref. */
> >>         strncpy(ownername, module_name(owner), MODULE_NAME_LEN);
> >> @@ -3061,6 +3118,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
> >>                                      sizeof(*mod->gpl_syms),
> >>                                      &mod->num_gpl_syms);
> >>         mod->gpl_crcs = section_addr(info, "__kcrctab_gpl");
> >> +
> >> +       mod->ns_imports = section_objs(info, "__knsimport",
> >> +                                      sizeof(*mod->ns_imports),
> >> +                                      &mod->num_ns_imports);
> >> +
> >>         mod->gpl_future_syms = section_objs(info,
> >>                                             "__ksymtab_gpl_future",
> >>                                             sizeof(*mod->gpl_future_syms),
> >> @@ -3381,6 +3443,19 @@ static int post_relocation(struct module *mod, const struct load_info *info)
> >>         return module_finalize(info->hdr, info->sechdrs, mod);
> >>  }
> >>
> >> +static void verify_namespace_dependencies(struct module *mod)
> >> +{
> >> +       struct module_ns_dep *ns_dep;
> >> +
> >> +       list_for_each_entry(ns_dep, &mod->ns_dependencies, ns_dep) {
> >> +               if (!module_imports_ns(mod, ns_dep->namespace)) {
> >> +                       pr_warn("%s: module uses symbols from namespace %s,"
> >> +                               " but does not import it.\n",
> >> +                               mod->name, ns_dep->namespace);
> >> +               }
> >> +       }
> >> +}
> >> +
> >>  /* Is this module of this name done loading?  No locks held. */
> >>  static bool finished_loading(const char *name)
> >>  {
> >> @@ -3682,6 +3757,8 @@ static int load_module(struct load_info *info, const char __user *uargs,
> >>         if (err)
> >>                 goto free_module;
> >>
> >> +       INIT_LIST_HEAD(&mod->ns_dependencies);
> >> +
> >>  #ifdef CONFIG_MODULE_SIG
> >>         mod->sig_ok = info->sig_ok;
> >>         if (!mod->sig_ok) {
> >> @@ -3730,6 +3807,8 @@ static int load_module(struct load_info *info, const char __user *uargs,
> >>         if (err < 0)
> >>                 goto free_modinfo;
> >>
> >> +       verify_namespace_dependencies(mod);
> >> +
> >>         flush_module_icache(mod);
> >>
> >>         /* Now copy in args */
> >> --
> >> 2.18.0.203.gfac676dfb9-goog
> >>



-- 
Lucas De Marchi

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH 2/6] module: add support for symbol namespaces.
  2018-07-25 16:48       ` [PATCH 2/6] module: add support for symbol namespaces Lucas De Marchi
@ 2018-07-26  7:44         ` Martijn Coenen
  0 siblings, 0 replies; 51+ messages in thread
From: Martijn Coenen @ 2018-07-26  7:44 UTC (permalink / raw)
  To: Lucas De Marchi
  Cc: Jessica Yu, lkml, Masahiro Yamada, Michal Marek,
	Geert Uytterhoeven, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
	the arch/x86 maintainers, Alan Stern, Greg KH, Oliver Neukum,
	Arnd Bergmann, Stephen Boyd, Philippe Ombredanne, Kate Stewart,
	Sam Ravnborg, Linux Kbuild mailing list, linux-m68k, USB list,
	USB Storage list, linux-scsi, Linux-Arch, Martijn Coenen,
	Sandeep Patil, Iliyan Malchev, Joel Fernandes, linux-modules

On Wed, Jul 25, 2018 at 6:48 PM, Lucas De Marchi
<lucas.de.marchi@gmail.com> wrote:
> this might be because you are looking to the wrong project
> (module-init-tools) rather than kmod that replaced it some years ago?

Oh yes indeed, thanks for pointing that out :-)


>> Just scanning the depmod source code, it seems really trivial to just
>> have depmod accommodate the new symbol name format. Adding Lucas (kmod
>> maintainer) to CC.
>
> Yep, that would be easy and allow depmod to be backward compatible
> since we would do nothing if the symbol doesn't have the suffix.
> As for dependency on a new version, this seems trivial enough to be
> backported to previous releases used on distros so even if they are
> not rolling they would get a compatible depmod.

Thanks for the suggestion. I had also considered modifying the script
that generates System.map to strip out the namespace; but that seems a
bit hacky since it will then mismatch with the actual symbol table
that's in the kernel image. The situation with depmod has gotten me a
bit worried that perhaps there are more tools that look at System.map
or the kernel's symbol table that won't be able to understand the new
format. But of course those could be updated, too.

Another alternative I was considering is leaving the namespace out of
the __ksymtab symbols, and instead for the symbols that have a
namespace, have an entry in __ksymtab (without namespace) and an entry
in a new section __ksymtab_ns (with namespace). We can then discard
__ksymtab_ns when building the final executable/modules, since that
information is not needed at runtime (it's encoded in struct
kernel_symbol). What makes this a bit more complex is that modpost
takes the fully linked version of vmlinux which would already have
__ksymtab_ns stripped, so we'd need to use objcopy to copy out that
section so modpost can use it to read vmlinux's namespaced symbols.
Modules wouldn't have that problem, since modpost processes modules
before their final linking step.

This last approach is more complex than just modifying depmod, but it
does have the benefit that we won't be break userspace tools depending
on the kernel __ksymtab section symbols. On the other hand, perhaps
userspace tools would like to have the namespace information at some
point.

For now I'll keep things as is and write up a patch for depmod, but if
others prefer the __ksymtab_ns approach I described above I don't mind
making that work.

Thanks,
Martijn

>
> CC'ing linux-modules@vger.kernel.org to keep track of this there
>
>
> thanks
> Lucas De Marchi
>
>>
>> Thanks,
>>
>> Jessica
>>
>> >
>> >>
>> >> On x86_64 I saw no difference in binary size (compression), but at
>> >> runtime this will require a word of memory per export to hold the
>> >> namespace. An alternative could be to store namespaced symbols in their
>> >> own section and use a separate 'struct namespaced_kernel_symbol' for
>> >> that section, at the cost of making the module loader more complex.
>> >>
>> >> Signed-off-by: Martijn Coenen <maco@android.com>
>> >> ---
>> >>  include/asm-generic/export.h |  2 +-
>> >>  include/linux/export.h       | 83 +++++++++++++++++++++++++++---------
>> >>  include/linux/module.h       | 13 ++++++
>> >>  kernel/module.c              | 79 ++++++++++++++++++++++++++++++++++
>> >>  4 files changed, 156 insertions(+), 21 deletions(-)
>> >>
>> >> diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
>> >> index 68efb950a918..4c3d1afb702f 100644
>> >> --- a/include/asm-generic/export.h
>> >> +++ b/include/asm-generic/export.h
>> >> @@ -29,7 +29,7 @@
>> >>         .section ___ksymtab\sec+\name,"a"
>> >>         .balign KSYM_ALIGN
>> >>  __ksymtab_\name:
>> >> -       __put \val, __kstrtab_\name
>> >> +       __put \val, __kstrtab_\name, 0
>> >>         .previous
>> >>         .section __ksymtab_strings,"a"
>> >>  __kstrtab_\name:
>> >> diff --git a/include/linux/export.h b/include/linux/export.h
>> >> index ad6b8e697b27..9f6e70eeb85f 100644
>> >> --- a/include/linux/export.h
>> >> +++ b/include/linux/export.h
>> >> @@ -22,6 +22,11 @@ struct kernel_symbol
>> >>  {
>> >>         unsigned long value;
>> >>         const char *name;
>> >> +       const char *namespace;
>> >> +};
>> >> +
>> >> +struct namespace_import {
>> >> +       const char *namespace;
>> >>  };
>> >>
>> >>  #ifdef MODULE
>> >> @@ -54,18 +59,28 @@ extern struct module __this_module;
>> >>  #define __CRC_SYMBOL(sym, sec)
>> >>  #endif
>> >>
>> >> +#define NS_SEPARATOR "."
>> >> +
>> >> +#define MODULE_IMPORT_NS(ns)                                           \
>> >> +       static const struct namespace_import __knsimport_##ns           \
>> >> +       asm("__knsimport_" #ns)                                         \
>> >> +       __attribute__((section("__knsimport"), used))                   \
>> >> +       = { #ns }
>> >> +
>> >>  /* For every exported symbol, place a struct in the __ksymtab section */
>> >> -#define ___EXPORT_SYMBOL(sym, sec)                                     \
>> >> +#define ___EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2)                        \
>> >>         extern typeof(sym) sym;                                         \
>> >>         __CRC_SYMBOL(sym, sec)                                          \
>> >> -       static const char __kstrtab_##sym[]                             \
>> >> +       static const char __kstrtab_##sym##nspost[]                     \
>> >>         __attribute__((section("__ksymtab_strings"), aligned(1)))       \
>> >>         = #sym;                                                         \
>> >> -       static const struct kernel_symbol __ksymtab_##sym               \
>> >> +       static const struct kernel_symbol __ksymtab_##sym##nspost       \
>> >> +       asm("__ksymtab_" #sym nspost2)                                  \
>> >>         __used                                                          \
>> >> -       __attribute__((section("___ksymtab" sec "+" #sym), used))       \
>> >> +       __attribute__((section("___ksymtab" sec "+" #sym #nspost)))     \
>> >> +       __attribute__((used))                                           \
>> >>         __attribute__((aligned(sizeof(void *))))                        \
>> >> -       = { (unsigned long)&sym, __kstrtab_##sym }
>> >> +       = { (unsigned long)&sym, __kstrtab_##sym##nspost, ns }
>> >>
>> >>  #if defined(__KSYM_DEPS__)
>> >>
>> >> @@ -76,52 +91,80 @@ extern struct module __this_module;
>> >>   * system filters out from the preprocessor output (see ksym_dep_filter
>> >>   * in scripts/Kbuild.include).
>> >>   */
>> >> -#define __EXPORT_SYMBOL(sym, sec)      === __KSYM_##sym ===
>> >> +#define __EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2) === __KSYM_##sym ===
>> >>
>> >>  #elif defined(CONFIG_TRIM_UNUSED_KSYMS)
>> >>
>> >>  #include <generated/autoksyms.h>
>> >>
>> >> -#define __EXPORT_SYMBOL(sym, sec)                              \
>> >> -       __cond_export_sym(sym, sec, __is_defined(__KSYM_##sym))
>> >> -#define __cond_export_sym(sym, sec, conf)                      \
>> >> -       ___cond_export_sym(sym, sec, conf)
>> >> -#define ___cond_export_sym(sym, sec, enabled)                  \
>> >> -       __cond_export_sym_##enabled(sym, sec)
>> >> -#define __cond_export_sym_1(sym, sec) ___EXPORT_SYMBOL(sym, sec)
>> >> -#define __cond_export_sym_0(sym, sec) /* nothing */
>> >> +#define __EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2)                 \
>> >> +       __cond_export_sym(sym, sec, ns, nspost, nspost2,                \
>> >> +                         __is_defined(__KSYM_##sym))
>> >> +#define __cond_export_sym(sym, sec, ns, nspost, nspost2, conf)         \
>> >> +       ___cond_export_sym(sym, sec, ns, nspost, nspost2, conf)
>> >> +#define ___cond_export_sym(sym, sec, ns, nspost, nspost2, enabled)     \
>> >> +       __cond_export_sym_##enabled(sym, sec, ns, nspost, nspost2)
>> >> +#define __cond_export_sym_1(sym, sec, ns, nspost, nspost2)             \
>> >> +       ___EXPORT_SYMBOL(sym, sec, ns, nspost, nspost2)
>> >> +#define __cond_export_sym_0(sym, sec, ns, nspost, nspost2) /* nothing */
>> >>
>> >>  #else
>> >>  #define __EXPORT_SYMBOL ___EXPORT_SYMBOL
>> >>  #endif
>> >>
>> >>  #define EXPORT_SYMBOL(sym)                                     \
>> >> -       __EXPORT_SYMBOL(sym, "")
>> >> +       __EXPORT_SYMBOL(sym, "", NULL, ,)
>> >>
>> >>  #define EXPORT_SYMBOL_GPL(sym)                                 \
>> >> -       __EXPORT_SYMBOL(sym, "_gpl")
>> >> +       __EXPORT_SYMBOL(sym, "_gpl", NULL, ,)
>> >>
>> >>  #define EXPORT_SYMBOL_GPL_FUTURE(sym)                          \
>> >> -       __EXPORT_SYMBOL(sym, "_gpl_future")
>> >> +       __EXPORT_SYMBOL(sym, "_gpl_future", NULL, ,)
>> >> +
>> >> +#define EXPORT_SYMBOL_NS(sym, ns)                               \
>> >> +       __EXPORT_SYMBOL(sym, "", #ns, __##ns, NS_SEPARATOR #ns)
>> >> +
>> >> +#define EXPORT_SYMBOL_NS_GPL(sym, ns)                               \
>> >> +       __EXPORT_SYMBOL(sym, "_gpl", #ns, __##ns, NS_SEPARATOR #ns)
>> >>
>> >>  #ifdef CONFIG_UNUSED_SYMBOLS
>> >> -#define EXPORT_UNUSED_SYMBOL(sym) __EXPORT_SYMBOL(sym, "_unused")
>> >> -#define EXPORT_UNUSED_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_unused_gpl")
>> >> +#define EXPORT_UNUSED_SYMBOL(sym) __EXPORT_SYMBOL(sym, "_unused", , ,)
>> >> +#define EXPORT_UNUSED_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_unused_gpl", , ,)
>> >>  #else
>> >>  #define EXPORT_UNUSED_SYMBOL(sym)
>> >>  #define EXPORT_UNUSED_SYMBOL_GPL(sym)
>> >>  #endif
>> >>
>> >> -#endif /* __GENKSYMS__ */
>> >> +#endif /* __KERNEL__ && !__GENKSYMS__ */
>> >> +
>> >> +#if defined(__GENKSYMS__)
>> >> +/*
>> >> + * When we're running genksyms, ignore the namespace and make the _NS
>> >> + * variants look like the normal ones. There are two reasons for this:
>> >> + * 1) In the normal definition of EXPORT_SYMBOL_NS, the 'ns' macro
>> >> + *    argument is itself not expanded because it's always tokenized or
>> >> + *    concatenated; but when running genksyms, a blank definition of the
>> >> + *    macro does allow the argument to be expanded; if a namespace
>> >> + *    happens to collide with a #define, this can cause issues.
>> >> + * 2) There's no need to modify genksyms to deal with the _NS variants
>> >> + */
>> >> +#define EXPORT_SYMBOL_NS(sym, ns)                              \
>> >> +       EXPORT_SYMBOL(sym)
>> >> +#define EXPORT_SYMBOL_NS_GPL(sym, ns)                          \
>> >> +       EXPORT_SYMBOL_GPL(sym)
>> >> +#endif
>> >>
>> >>  #else /* !CONFIG_MODULES... */
>> >>
>> >>  #define EXPORT_SYMBOL(sym)
>> >> +#define EXPORT_SYMBOL_NS(sym, ns)
>> >> +#define EXPORT_SYMBOL_NS_GPL(sym, ns)
>> >>  #define EXPORT_SYMBOL_GPL(sym)
>> >>  #define EXPORT_SYMBOL_GPL_FUTURE(sym)
>> >>  #define EXPORT_UNUSED_SYMBOL(sym)
>> >>  #define EXPORT_UNUSED_SYMBOL_GPL(sym)
>> >>
>> >> +#define MODULE_IMPORT_NS(ns)
>> >>  #endif /* CONFIG_MODULES */
>> >>  #endif /* !__ASSEMBLY__ */
>> >>
>> >> diff --git a/include/linux/module.h b/include/linux/module.h
>> >> index d44df9b2c131..afab4e8fa188 100644
>> >> --- a/include/linux/module.h
>> >> +++ b/include/linux/module.h
>> >> @@ -268,6 +268,12 @@ void *__symbol_get(const char *symbol);
>> >>  void *__symbol_get_gpl(const char *symbol);
>> >>  #define symbol_get(x) ((typeof(&x))(__symbol_get(VMLINUX_SYMBOL_STR(x))))
>> >>
>> >> +/* namespace dependencies of the module */
>> >> +struct module_ns_dep {
>> >> +       struct list_head ns_dep;
>> >> +       const char *namespace;
>> >> +};
>> >> +
>> >>  /* modules using other modules: kdb wants to see this. */
>> >>  struct module_use {
>> >>         struct list_head source_list;
>> >> @@ -359,6 +365,13 @@ struct module {
>> >>         const struct kernel_symbol *gpl_syms;
>> >>         const s32 *gpl_crcs;
>> >>
>> >> +       /* Namespace imports */
>> >> +       unsigned int num_ns_imports;
>> >> +       const struct namespace_import *ns_imports;
>> >> +
>> >> +       /* Namespace dependencies */
>> >> +       struct list_head ns_dependencies;
>> >> +
>> >>  #ifdef CONFIG_UNUSED_SYMBOLS
>> >>         /* unused exported symbols. */
>> >>         const struct kernel_symbol *unused_syms;
>> >> diff --git a/kernel/module.c b/kernel/module.c
>> >> index f475f30eed8c..63de0fe849f9 100644
>> >> --- a/kernel/module.c
>> >> +++ b/kernel/module.c
>> >> @@ -1166,6 +1166,51 @@ static inline int module_unload_init(struct module *mod)
>> >>  }
>> >>  #endif /* CONFIG_MODULE_UNLOAD */
>> >>
>> >> +static bool module_has_ns_dependency(struct module *mod, const char *ns)
>> >> +{
>> >> +       struct module_ns_dep *ns_dep;
>> >> +
>> >> +       list_for_each_entry(ns_dep, &mod->ns_dependencies, ns_dep) {
>> >> +               if (strcmp(ns_dep->namespace, ns) == 0)
>> >> +                       return true;
>> >> +       }
>> >> +
>> >> +       return false;
>> >> +}
>> >> +
>> >> +static int add_module_ns_dependency(struct module *mod, const char *ns)
>> >> +{
>> >> +       struct module_ns_dep *ns_dep;
>> >> +
>> >> +       if (module_has_ns_dependency(mod, ns))
>> >> +               return 0;
>> >> +
>> >> +       ns_dep = kmalloc(sizeof(*ns_dep), GFP_ATOMIC);
>> >> +       if (!ns_dep)
>> >> +               return -ENOMEM;
>> >> +
>> >> +       ns_dep->namespace = ns;
>> >> +
>> >> +       list_add(&ns_dep->ns_dep, &mod->ns_dependencies);
>> >> +
>> >> +       return 0;
>> >> +}
>> >> +
>> >> +static bool module_imports_ns(struct module *mod, const char *ns)
>> >> +{
>> >> +       size_t i;
>> >> +
>> >> +       if (!ns)
>> >> +               return true;
>> >> +
>> >> +       for (i = 0; i < mod->num_ns_imports; ++i) {
>> >> +               if (!strcmp(mod->ns_imports[i].namespace, ns))
>> >> +                       return true;
>> >> +       }
>> >> +
>> >> +       return false;
>> >> +}
>> >> +
>> >>  static size_t module_flags_taint(struct module *mod, char *buf)
>> >>  {
>> >>         size_t l = 0;
>> >> @@ -1415,6 +1460,18 @@ static const struct kernel_symbol *resolve_symbol(struct module *mod,
>> >>                 goto getname;
>> >>         }
>> >>
>> >> +       /*
>> >> +        * We can't yet verify that the module actually imports this
>> >> +        * namespace, because the imports themselves are only available
>> >> +        * after processing the symbol table and doing relocation; so
>> >> +        * instead just record the dependency and check later.
>> >> +        */
>> >> +       if (sym->namespace) {
>> >> +               err = add_module_ns_dependency(mod, sym->namespace);
>> >> +               if (err)
>> >> +                       sym = ERR_PTR(err);
>> >> +       }
>> >> +
>> >>  getname:
>> >>         /* We must make copy under the lock if we failed to get ref. */
>> >>         strncpy(ownername, module_name(owner), MODULE_NAME_LEN);
>> >> @@ -3061,6 +3118,11 @@ static int find_module_sections(struct module *mod, struct load_info *info)
>> >>                                      sizeof(*mod->gpl_syms),
>> >>                                      &mod->num_gpl_syms);
>> >>         mod->gpl_crcs = section_addr(info, "__kcrctab_gpl");
>> >> +
>> >> +       mod->ns_imports = section_objs(info, "__knsimport",
>> >> +                                      sizeof(*mod->ns_imports),
>> >> +                                      &mod->num_ns_imports);
>> >> +
>> >>         mod->gpl_future_syms = section_objs(info,
>> >>                                             "__ksymtab_gpl_future",
>> >>                                             sizeof(*mod->gpl_future_syms),
>> >> @@ -3381,6 +3443,19 @@ static int post_relocation(struct module *mod, const struct load_info *info)
>> >>         return module_finalize(info->hdr, info->sechdrs, mod);
>> >>  }
>> >>
>> >> +static void verify_namespace_dependencies(struct module *mod)
>> >> +{
>> >> +       struct module_ns_dep *ns_dep;
>> >> +
>> >> +       list_for_each_entry(ns_dep, &mod->ns_dependencies, ns_dep) {
>> >> +               if (!module_imports_ns(mod, ns_dep->namespace)) {
>> >> +                       pr_warn("%s: module uses symbols from namespace %s,"
>> >> +                               " but does not import it.\n",
>> >> +                               mod->name, ns_dep->namespace);
>> >> +               }
>> >> +       }
>> >> +}
>> >> +
>> >>  /* Is this module of this name done loading?  No locks held. */
>> >>  static bool finished_loading(const char *name)
>> >>  {
>> >> @@ -3682,6 +3757,8 @@ static int load_module(struct load_info *info, const char __user *uargs,
>> >>         if (err)
>> >>                 goto free_module;
>> >>
>> >> +       INIT_LIST_HEAD(&mod->ns_dependencies);
>> >> +
>> >>  #ifdef CONFIG_MODULE_SIG
>> >>         mod->sig_ok = info->sig_ok;
>> >>         if (!mod->sig_ok) {
>> >> @@ -3730,6 +3807,8 @@ static int load_module(struct load_info *info, const char __user *uargs,
>> >>         if (err < 0)
>> >>                 goto free_modinfo;
>> >>
>> >> +       verify_namespace_dependencies(mod);
>> >> +
>> >>         flush_module_icache(mod);
>> >>
>> >>         /* Now copy in args */
>> >> --
>> >> 2.18.0.203.gfac676dfb9-goog
>> >>
>
>
>
> --
> Lucas De Marchi

^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 0/10] Symbol namespaces - RFC
       [not found] <20180716122125.175792-1-maco@android.com>
       [not found] ` <20180716122125.175792-3-maco@android.com>
@ 2019-08-13 12:16 ` Matthias Maennich
  2019-08-13 12:16   ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
                     ` (10 more replies)
  1 sibling, 11 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:16 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Adrian Reber, Andrew Morton,
	Ard Biesheuvel, cocci, Dan Williams, David Howells,
	David S. Miller, Ingo Molnar, Johannes Weiner, Jonathan Cameron,
	Mauro Carvalho Chehab, Michael Ellerman, Nicolas Ferre,
	Patrick Bellasi, Richard Guy Briggs, Will Deacon

As of Linux 5.3-rc4, there are 31203 [1] exported symbols in the kernel.
That is a growth of almost 1000 symbols since 4.17 (30206 [2]).  There
seems to be some consensus amongst kernel devs that the export surface
is too large, and hard to reason about.

Generally, these symbols fall in one of these categories:
1) Symbols actually meant for drivers
2) Symbols that are only exported because functionality is split over
   multiple modules, yet they really shouldn't be used by modules outside
   of their own subsystem
3) Symbols really only meant for in-tree use

When module developers try to upstream their code, it regularly turns
out that they are using exported symbols that they really shouldn't be
using. This problem is even bigger for drivers that are currently
out-of-tree, which may be using many symbols that they shouldn't be
using, and that break when those symbols are removed or modified.

This patch allows subsystem maintainers to partition their exported
symbols into separate namespaces, and module authors to import such
namespaces only when needed.

This allows subsystem maintainers to more easily limit availability of
these namespaced symbols to other parts of the kernel. It can also be
used to partition the set of exported symbols for documentation
purposes; for example, a set of symbols that is really only used for
debugging could be in a "SUBSYSTEM_DEBUG" namespace.

I continued the work mainly done by Martijn Coenen. In this v2 the
following changes have been introduced compared to v1 of this series:

- Rather than adding and evaluating separate sections __knsimport_NS,
  use modinfo tags to declare the namespaces a module introduces.
  Adjust modpost and the module loader accordingly.
- Also add support for reading multiple modinfo values for the same tag
  to allow list-like access to modinfo tags.
- The macros in export.h have been cleaned up to avoid redundancy in the
  macro parameters (ns, nspost, nspost2).
- The introduction of relative references in the ksymtab entries caused
  a rework of the macros to accommodate that configuration as well.
- Alignment of kernel_symbol in the ksymtab needed to be fixed to allow
  growing the kernel_symbol struct.
- Modpost does now also append the namespace suffix to the symbol
  entries in Module.symvers.
- The configuration option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS allows
  relaxing the enforcement of properly declared namespace imports at module
  loading time.
- Symbols can be collectively exported into a namespace by defining
  DEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile.
- The requirement for a very recent coccinelle spatch has been lifted by
  simplifying the script.
- nsdeps does now ensures MODULE_IMPORT_NS statements are sorted when
  patching the module source files.
- Some minor bugs have been addressed in nsdeps to allow it to work with
  modules that have more than one source file.
- The RFC for the usb-storage symbols has been simplified by using
  DEFAULT_SYMBOL_NAMESPACE=USB_STORAGE rather than explicitly exporting each
  and every symbol into that new namespace.

This patch series was developed against v5.3-rc4.

[1] git grep "^EXPORT_SYMBOL\w*(" v5.3-rc4 | wc -l
[2] git grep "^EXPORT_SYMBOL\w*(" v4.17    | wc -l

Matthias Maennich (10):
  module: support reading multiple values per modinfo tag
  export: explicitly align struct kernel_symbol
  module: add support for symbol namespaces.
  modpost: add support for symbol namespaces
  module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
  export: allow definition default namespaces in Makefiles or sources
  modpost: add support for generating namespace dependencies
  scripts: Coccinelle script for namespace dependencies.
  usb-storage: remove single-use define for debugging
  RFC: usb-storage: export symbols in USB_STORAGE namespace

 MAINTAINERS                                 |   5 +
 Makefile                                    |  12 ++
 arch/m68k/include/asm/export.h              |   1 -
 drivers/usb/storage/Makefile                |   2 +
 drivers/usb/storage/alauda.c                |   1 +
 drivers/usb/storage/cypress_atacb.c         |   1 +
 drivers/usb/storage/datafab.c               |   1 +
 drivers/usb/storage/debug.h                 |   2 -
 drivers/usb/storage/ene_ub6250.c            |   1 +
 drivers/usb/storage/freecom.c               |   1 +
 drivers/usb/storage/isd200.c                |   1 +
 drivers/usb/storage/jumpshot.c              |   1 +
 drivers/usb/storage/karma.c                 |   1 +
 drivers/usb/storage/onetouch.c              |   1 +
 drivers/usb/storage/realtek_cr.c            |   1 +
 drivers/usb/storage/scsiglue.c              |   2 +-
 drivers/usb/storage/sddr09.c                |   1 +
 drivers/usb/storage/sddr55.c                |   1 +
 drivers/usb/storage/shuttle_usbat.c         |   1 +
 drivers/usb/storage/uas.c                   |   1 +
 include/asm-generic/export.h                |  14 +-
 include/linux/export.h                      |  92 +++++++++++--
 include/linux/module.h                      |   2 +
 init/Kconfig                                |  14 ++
 kernel/module.c                             |  67 ++++++++-
 scripts/Makefile.modpost                    |   4 +-
 scripts/coccinelle/misc/add_namespace.cocci |  23 ++++
 scripts/mod/modpost.c                       | 144 ++++++++++++++++++--
 scripts/mod/modpost.h                       |   9 ++
 scripts/nsdeps                              |  54 ++++++++
 30 files changed, 421 insertions(+), 40 deletions(-)
 create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
 create mode 100644 scripts/nsdeps

-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 01/10] module: support reading multiple values per modinfo tag
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
@ 2019-08-13 12:16   ` Matthias Maennich
  2019-08-13 12:40     ` Greg KH
  2019-08-13 12:16   ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich
                     ` (9 subsequent siblings)
  10 siblings, 1 reply; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:16 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

Similar to modpost's get_next_modinfo(), introduce get_next_modinfo() in
kernel/module.c to acquire any further values associated with the same
modinfo tag name. That is useful for any tags that have multiple
occurrences (such as 'alias'), but is in particular introduced here as
part of the symbol namespaces patch series to read the (potentially)
multiple namespaces a module is importing.

Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Reviewed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 kernel/module.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/kernel/module.c b/kernel/module.c
index 5933395af9a0..a23067907169 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -2481,7 +2481,8 @@ static char *next_string(char *string, unsigned long *secsize)
 	return string;
 }
 
-static char *get_modinfo(struct load_info *info, const char *tag)
+static char *get_next_modinfo(const struct load_info *info, const char *tag,
+			      char *prev)
 {
 	char *p;
 	unsigned int taglen = strlen(tag);
@@ -2492,13 +2493,25 @@ static char *get_modinfo(struct load_info *info, const char *tag)
 	 * get_modinfo() calls made before rewrite_section_headers()
 	 * must use sh_offset, as sh_addr isn't set!
 	 */
-	for (p = (char *)info->hdr + infosec->sh_offset; p; p = next_string(p, &size)) {
+	char *modinfo = (char *)info->hdr + infosec->sh_offset;
+
+	if (prev) {
+		size -= prev - modinfo;
+		modinfo = next_string(prev, &size);
+	}
+
+	for (p = modinfo; p; p = next_string(p, &size)) {
 		if (strncmp(p, tag, taglen) == 0 && p[taglen] == '=')
 			return p + taglen + 1;
 	}
 	return NULL;
 }
 
+static char *get_modinfo(const struct load_info *info, const char *tag)
+{
+	return get_next_modinfo(info, tag, NULL);
+}
+
 static void setup_modinfo(struct module *mod, struct load_info *info)
 {
 	struct module_attribute *attr;
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 02/10] export: explicitly align struct kernel_symbol
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
  2019-08-13 12:16   ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
@ 2019-08-13 12:16   ` Matthias Maennich
  2019-08-13 12:41     ` Greg KH
  2019-08-13 12:17   ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich
                     ` (8 subsequent siblings)
  10 siblings, 1 reply; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:16 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton, Nicolas Pitre,
	Michael Ellerman, Ard Biesheuvel

This change allows growing struct kernel_symbol without wasting bytes to
alignment. It also concretized the alignment of ksymtab entries if
relative references are used for ksymtab entries.

struct kernel_symbol was already implicitly being aligned to the word
size, except on x86_64 and m68k, where it is aligned to 16 and 2 bytes,
respectively.

As far as I can tell there is no requirement for aligning struct
kernel_symbol to 16 bytes on x86_64, but gcc aligns structs to their
size, and the linker aligns the custom __ksymtab sections to the largest
data type contained within, so setting KSYM_ALIGN to 16 was necessary to
stay consistent with the code generated for non-ASM EXPORT_SYMBOL(). Now
that non-ASM EXPORT_SYMBOL() explicitly aligns to word size (8),
KSYM_ALIGN is no longer necessary.

In case of relative references, the alignment has been changed
accordingly to not waste space when adding new struct members.

As for m68k, struct kernel_symbol is aligned to 2 bytes even though the
structure itself is 8 bytes; using a 4-byte alignment shouldn't hurt.

I manually verified the output of the __ksymtab sections didn't change
on x86, x86_64, arm, arm64 and m68k. As expected, the section contents
didn't change, and the ELF section alignment only changed on x86_64 and
m68k. Feedback from other archs more than welcome.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 arch/m68k/include/asm/export.h | 1 -
 include/asm-generic/export.h   | 8 +++-----
 include/linux/export.h         | 3 ++-
 3 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/arch/m68k/include/asm/export.h b/arch/m68k/include/asm/export.h
index 0af20f48bd07..b53008b67ce1 100644
--- a/arch/m68k/include/asm/export.h
+++ b/arch/m68k/include/asm/export.h
@@ -1,3 +1,2 @@
-#define KSYM_ALIGN 2
 #define KCRC_ALIGN 2
 #include <asm-generic/export.h>
diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 294d6ae785d4..63f54907317b 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -4,15 +4,13 @@
 #ifndef KSYM_FUNC
 #define KSYM_FUNC(x) x
 #endif
-#ifdef CONFIG_64BIT
-#ifndef KSYM_ALIGN
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+#define KSYM_ALIGN 4
+#elif defined(CONFIG_64BIT)
 #define KSYM_ALIGN 8
-#endif
 #else
-#ifndef KSYM_ALIGN
 #define KSYM_ALIGN 4
 #endif
-#endif
 #ifndef KCRC_ALIGN
 #define KCRC_ALIGN 4
 #endif
diff --git a/include/linux/export.h b/include/linux/export.h
index fd8711ed9ac4..28a4d2150689 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -52,7 +52,7 @@ extern struct module __this_module;
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	__ADDRESSABLE(sym)						\
 	asm("	.section \"___ksymtab" sec "+" #sym "\", \"a\"	\n"	\
-	    "	.balign	8					\n"	\
+	    "	.balign 4					\n"	\
 	    "__ksymtab_" #sym ":				\n"	\
 	    "	.long	" #sym "- .				\n"	\
 	    "	.long	__kstrtab_" #sym "- .			\n"	\
@@ -66,6 +66,7 @@ struct kernel_symbol {
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	static const struct kernel_symbol __ksymtab_##sym		\
 	__attribute__((section("___ksymtab" sec "+" #sym), used))	\
+	__aligned(sizeof(void *))					\
 	= { (unsigned long)&sym, __kstrtab_##sym }
 
 struct kernel_symbol {
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 03/10] module: add support for symbol namespaces.
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
  2019-08-13 12:16   ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
  2019-08-13 12:16   ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich
@ 2019-08-13 12:17   ` Matthias Maennich
  2019-08-13 15:26     ` Greg KH
  2019-08-13 12:17   ` [PATCH v2 04/10] modpost: " Matthias Maennich
                     ` (7 subsequent siblings)
  10 siblings, 1 reply; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton,
	Michael Ellerman, Ard Biesheuvel

The EXPORT_SYMBOL_NS() and EXPORT_SYMBOL_NS_GPL() macros can be used to
export a symbol to a specific namespace.  There are no _GPL_FUTURE and
_UNUSED variants because these are currently unused, and I'm not sure
they are necessary.

I didn't add EXPORT_SYMBOL_NS() for ASM exports; this patch sets the
namespace of ASM exports to NULL by default. In case of relative
references, it will be relocatable to NULL. If there's a need, this
should be pretty easy to add.

A module that wants to use a symbol exported to a namespace must add a
MODULE_IMPORT_NS() statement to their module code; otherwise, modpost
will complain when building the module, and the kernel module loader
will emit an error and fail when loading the module.

MODULE_IMPORT_NS() adds a modinfo tag 'import_ns' to the module. That
tag can be observed by the modinfo command, modpost and kernel/module.c
at the time of loading the module.

The ELF symbols are renamed to include the namespace with an asm label;
for example, symbol 'usb_stor_suspend' in namespace USB_STORAGE becomes
'usb_stor_suspend.USB_STORAGE'.  This allows modpost to do namespace
checking, without having to go through all the effort of parsing ELF and
relocation records just to get to the struct kernel_symbols.

On x86_64 I saw no difference in binary size (compression), but at
runtime this will require a word of memory per export to hold the
namespace. An alternative could be to store namespaced symbols in their
own section and use a separate 'struct namespaced_kernel_symbol' for
that section, at the cost of making the module loader more complex.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 include/asm-generic/export.h |  6 +--
 include/linux/export.h       | 85 ++++++++++++++++++++++++++++++------
 include/linux/module.h       |  2 +
 kernel/module.c              | 43 ++++++++++++++++++
 4 files changed, 120 insertions(+), 16 deletions(-)

diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 63f54907317b..e2b5d0f569d3 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -17,11 +17,11 @@
 
 .macro __put, val, name
 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
-	.long	\val - ., \name - .
+	.long	\val - ., \name - ., 0 - .
 #elif defined(CONFIG_64BIT)
-	.quad	\val, \name
+	.quad	\val, \name, 0
 #else
-	.long	\val, \name
+	.long	\val, \name, 0
 #endif
 .endm
 
diff --git a/include/linux/export.h b/include/linux/export.h
index 28a4d2150689..8e12e05444d1 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -20,6 +20,8 @@ extern struct module __this_module;
 
 #ifdef CONFIG_MODULES
 
+#define NS_SEPARATOR "."
+
 #if defined(__KERNEL__) && !defined(__GENKSYMS__)
 #ifdef CONFIG_MODVERSIONS
 /* Mark the CRC weak since genksyms apparently decides not to
@@ -49,6 +51,16 @@ extern struct module __this_module;
  * absolute relocations that require runtime processing on relocatable
  * kernels.
  */
+#define __KSYMTAB_ENTRY_NS(sym, sec, ns)				\
+	__ADDRESSABLE(sym)						\
+	asm("	.section \"___ksymtab" sec "+" #sym "\", \"a\"	\n"	\
+	    "	.balign	4					\n"	\
+	    "__ksymtab_" #sym NS_SEPARATOR #ns ":		\n"	\
+	    "	.long	" #sym "- .				\n"	\
+	    "	.long	__kstrtab_" #sym "- .			\n"	\
+	    "	.long	__kstrtab_ns_" #sym "- .		\n"	\
+	    "	.previous					\n")
+
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	__ADDRESSABLE(sym)						\
 	asm("	.section \"___ksymtab" sec "+" #sym "\", \"a\"	\n"	\
@@ -56,32 +68,53 @@ extern struct module __this_module;
 	    "__ksymtab_" #sym ":				\n"	\
 	    "	.long	" #sym "- .				\n"	\
 	    "	.long	__kstrtab_" #sym "- .			\n"	\
+	    "	.long	0 - .					\n"	\
 	    "	.previous					\n")
 
 struct kernel_symbol {
 	int value_offset;
 	int name_offset;
+	int namespace_offset;
 };
 #else
+#define __KSYMTAB_ENTRY_NS(sym, sec, ns)				\
+	static const struct kernel_symbol __ksymtab_##sym##__##ns	\
+	asm("__ksymtab_" #sym NS_SEPARATOR #ns)				\
+	__attribute__((section("___ksymtab" sec "+" #sym), used))	\
+	__aligned(sizeof(void *))					\
+	= { (unsigned long)&sym, __kstrtab_##sym, __kstrtab_ns_##sym}
+
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	static const struct kernel_symbol __ksymtab_##sym		\
+	asm("__ksymtab_" #sym)						\
 	__attribute__((section("___ksymtab" sec "+" #sym), used))	\
 	__aligned(sizeof(void *))					\
-	= { (unsigned long)&sym, __kstrtab_##sym }
+	= { (unsigned long)&sym, __kstrtab_##sym, NULL }
 
 struct kernel_symbol {
 	unsigned long value;
 	const char *name;
+	const char *namespace;
 };
 #endif
 
-/* For every exported symbol, place a struct in the __ksymtab section */
-#define ___EXPORT_SYMBOL(sym, sec)					\
+#define ___export_symbol_common(sym, sec)				\
 	extern typeof(sym) sym;						\
 	__CRC_SYMBOL(sym, sec)						\
 	static const char __kstrtab_##sym[]				\
 	__attribute__((section("__ksymtab_strings"), used, aligned(1)))	\
-	= #sym;								\
+	= #sym								\
+
+/* For every exported symbol, place a struct in the __ksymtab section */
+#define ___EXPORT_SYMBOL_NS(sym, sec, ns)				\
+	___export_symbol_common(sym, sec);			\
+	static const char __kstrtab_ns_##sym[]				\
+	__attribute__((section("__ksymtab_strings"), used, aligned(1)))	\
+	= #ns;								\
+	__KSYMTAB_ENTRY_NS(sym, sec, ns)
+
+#define ___EXPORT_SYMBOL(sym, sec)					\
+	___export_symbol_common(sym, sec);				\
 	__KSYMTAB_ENTRY(sym, sec)
 
 #if defined(__DISABLE_EXPORTS)
@@ -91,6 +124,7 @@ struct kernel_symbol {
  * be reused in other execution contexts such as the UEFI stub or the
  * decompressor.
  */
+#define __EXPORT_SYMBOL_NS(sym, sec, ns)
 #define __EXPORT_SYMBOL(sym, sec)
 
 #elif defined(CONFIG_TRIM_UNUSED_KSYMS)
@@ -117,18 +151,26 @@ struct kernel_symbol {
 #define __cond_export_sym_1(sym, sec) ___EXPORT_SYMBOL(sym, sec)
 #define __cond_export_sym_0(sym, sec) /* nothing */
 
+#define __EXPORT_SYMBOL_NS(sym, sec, ns)				\
+	__ksym_marker(sym);						\
+	__cond_export_ns_sym(sym, sec, ns, __is_defined(__KSYM_##sym))
+#define __cond_export_ns_sym(sym, sec, ns, conf)			\
+	___cond_export_ns_sym(sym, sec, ns, conf)
+#define ___cond_export_ns_sym(sym, sec, ns, enabled)			\
+	__cond_export_ns_sym_##enabled(sym, sec, ns)
+#define __cond_export_ns_sym_1(sym, sec, ns) ___EXPORT_SYMBOL_NS(sym, sec, ns)
+#define __cond_export_ns_sym_0(sym, sec, ns) /* nothing */
+
 #else
+#define __EXPORT_SYMBOL_NS ___EXPORT_SYMBOL_NS
 #define __EXPORT_SYMBOL ___EXPORT_SYMBOL
 #endif
 
-#define EXPORT_SYMBOL(sym)					\
-	__EXPORT_SYMBOL(sym, "")
-
-#define EXPORT_SYMBOL_GPL(sym)					\
-	__EXPORT_SYMBOL(sym, "_gpl")
-
-#define EXPORT_SYMBOL_GPL_FUTURE(sym)				\
-	__EXPORT_SYMBOL(sym, "_gpl_future")
+#define EXPORT_SYMBOL(sym) __EXPORT_SYMBOL(sym, "")
+#define EXPORT_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_gpl")
+#define EXPORT_SYMBOL_GPL_FUTURE(sym) __EXPORT_SYMBOL(sym, "_gpl_future")
+#define EXPORT_SYMBOL_NS(sym, ns) __EXPORT_SYMBOL_NS(sym, "", ns)
+#define EXPORT_SYMBOL_NS_GPL(sym, ns) __EXPORT_SYMBOL_NS(sym, "_gpl", ns)
 
 #ifdef CONFIG_UNUSED_SYMBOLS
 #define EXPORT_UNUSED_SYMBOL(sym) __EXPORT_SYMBOL(sym, "_unused")
@@ -138,11 +180,28 @@ struct kernel_symbol {
 #define EXPORT_UNUSED_SYMBOL_GPL(sym)
 #endif
 
-#endif	/* __GENKSYMS__ */
+#endif	/* __KERNEL__ && !__GENKSYMS__ */
+
+#if defined(__GENKSYMS__)
+/*
+ * When we're running genksyms, ignore the namespace and make the _NS
+ * variants look like the normal ones. There are two reasons for this:
+ * 1) In the normal definition of EXPORT_SYMBOL_NS, the 'ns' macro
+ *    argument is itself not expanded because it's always tokenized or
+ *    concatenated; but when running genksyms, a blank definition of the
+ *    macro does allow the argument to be expanded; if a namespace
+ *    happens to collide with a #define, this can cause issues.
+ * 2) There's no need to modify genksyms to deal with the _NS variants
+ */
+#define EXPORT_SYMBOL_NS(sym, ns) EXPORT_SYMBOL(sym)
+#define EXPORT_SYMBOL_NS_GPL(sym, ns) EXPORT_SYMBOL_GPL(sym)
+#endif
 
 #else /* !CONFIG_MODULES... */
 
 #define EXPORT_SYMBOL(sym)
+#define EXPORT_SYMBOL_NS(sym, ns)
+#define EXPORT_SYMBOL_NS_GPL(sym, ns)
 #define EXPORT_SYMBOL_GPL(sym)
 #define EXPORT_SYMBOL_GPL_FUTURE(sym)
 #define EXPORT_UNUSED_SYMBOL(sym)
diff --git a/include/linux/module.h b/include/linux/module.h
index 1455812dd325..b3611e749f72 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -280,6 +280,8 @@ struct notifier_block;
 
 #ifdef CONFIG_MODULES
 
+#define MODULE_IMPORT_NS(ns) MODULE_INFO(import_ns, #ns)
+
 extern int modules_disabled; /* for sysctl */
 /* Get/put a kernel symbol (calls must be symmetric) */
 void *__symbol_get(const char *symbol);
diff --git a/kernel/module.c b/kernel/module.c
index a23067907169..57e8253f2251 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -544,6 +544,15 @@ static const char *kernel_symbol_name(const struct kernel_symbol *sym)
 #endif
 }
 
+static const char *kernel_symbol_namespace(const struct kernel_symbol *sym)
+{
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+	return offset_to_ptr(&sym->namespace_offset);
+#else
+	return sym->namespace;
+#endif
+}
+
 static int cmp_name(const void *va, const void *vb)
 {
 	const char *a;
@@ -1379,6 +1388,34 @@ static inline int same_magic(const char *amagic, const char *bmagic,
 }
 #endif /* CONFIG_MODVERSIONS */
 
+static char *get_modinfo(const struct load_info *info, const char *tag);
+static char *get_next_modinfo(const struct load_info *info, const char *tag,
+			      char *prev);
+
+static int verify_namespace_is_imported(const struct load_info *info,
+					const struct kernel_symbol *sym,
+					struct module *mod)
+{
+	const char *namespace;
+	char *imported_namespace;
+
+	namespace = kernel_symbol_namespace(sym);
+	if (namespace) {
+		imported_namespace = get_modinfo(info, "import_ns");
+		while (imported_namespace) {
+			if (strcmp(namespace, imported_namespace) == 0)
+				return 0;
+			imported_namespace = get_next_modinfo(
+				info, "import_ns", imported_namespace);
+		}
+		pr_err("%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
+		       mod->name, kernel_symbol_name(sym), namespace);
+		return -EINVAL;
+	}
+	return 0;
+}
+
+
 /* Resolve a symbol for this module.  I.e. if we find one, record usage. */
 static const struct kernel_symbol *resolve_symbol(struct module *mod,
 						  const struct load_info *info,
@@ -1413,6 +1450,12 @@ static const struct kernel_symbol *resolve_symbol(struct module *mod,
 		goto getname;
 	}
 
+	err = verify_namespace_is_imported(info, sym, mod);
+	if (err) {
+		sym = ERR_PTR(err);
+		goto getname;
+	}
+
 getname:
 	/* We must make copy under the lock if we failed to get ref. */
 	strncpy(ownername, module_name(owner), MODULE_NAME_LEN);
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 04/10] modpost: add support for symbol namespaces
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
                     ` (2 preceding siblings ...)
  2019-08-13 12:17   ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich
@ 2019-08-13 12:17   ` " Matthias Maennich
  2019-08-13 15:27     ` Greg KH
  2019-08-13 12:17   ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
                     ` (6 subsequent siblings)
  10 siblings, 1 reply; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

Add support for symbols that are exported into namespaces. For that,
extract any namespace suffix from the symbol name. In addition, emit a
warning whenever a module refers to an exported symbol without
explicitly importing the namespace that it is defined in. This patch
consistently adds the namespace suffix to symbol names exported into
Module.symvers.

Example warning emitted by modpost in case of the above violation:

 WARNING: module ums-usbat uses symbol usb_stor_resume from namespace
 USB_STORAGE, but does not import it.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 scripts/mod/modpost.c | 91 +++++++++++++++++++++++++++++++++++++------
 scripts/mod/modpost.h |  7 ++++
 2 files changed, 87 insertions(+), 11 deletions(-)

diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index f277e116e0eb..538bb24ffee3 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -164,6 +164,7 @@ struct symbol {
 	struct module *module;
 	unsigned int crc;
 	int crc_valid;
+	const char *namespace;
 	unsigned int weak:1;
 	unsigned int vmlinux:1;    /* 1 if symbol is defined in vmlinux */
 	unsigned int kernel:1;     /* 1 if symbol is from kernel
@@ -233,6 +234,37 @@ static struct symbol *find_symbol(const char *name)
 	return NULL;
 }
 
+static bool contains_namespace(struct namespace_list *list,
+			       const char *namespace)
+{
+	struct namespace_list *ns_entry;
+
+	for (ns_entry = list; ns_entry != NULL; ns_entry = ns_entry->next)
+		if (strcmp(ns_entry->namespace, namespace) == 0)
+			return true;
+
+	return false;
+}
+
+static void add_namespace(struct namespace_list **list, const char *namespace)
+{
+	struct namespace_list *ns_entry;
+
+	if (!contains_namespace(*list, namespace)) {
+		ns_entry = NOFAIL(malloc(sizeof(struct namespace_list) +
+					 strlen(namespace) + 1));
+		strcpy(ns_entry->namespace, namespace);
+		ns_entry->next = *list;
+		*list = ns_entry;
+	}
+}
+
+static bool module_imports_namespace(struct module *module,
+				     const char *namespace)
+{
+	return contains_namespace(module->imported_namespaces, namespace);
+}
+
 static const struct {
 	const char *str;
 	enum export export;
@@ -312,6 +344,22 @@ static enum export export_from_sec(struct elf_info *elf, unsigned int sec)
 		return export_unknown;
 }
 
+static const char *sym_extract_namespace(const char **symname)
+{
+	size_t n;
+	char *dupsymname;
+
+	n = strcspn(*symname, ".");
+	if (n < strlen(*symname) - 1) {
+		dupsymname = NOFAIL(strdup(*symname));
+		dupsymname[n] = '\0';
+		*symname = dupsymname;
+		return dupsymname + n + 1;
+	}
+
+	return NULL;
+}
+
 /**
  * Add an exported symbol - it may have already been added without a
  * CRC, in this case just update the CRC
@@ -319,16 +367,18 @@ static enum export export_from_sec(struct elf_info *elf, unsigned int sec)
 static struct symbol *sym_add_exported(const char *name, struct module *mod,
 				       enum export export)
 {
-	struct symbol *s = find_symbol(name);
+	const char *symbol_name = name;
+	const char *namespace = sym_extract_namespace(&symbol_name);
+	struct symbol *s = find_symbol(symbol_name);
 
 	if (!s) {
-		s = new_symbol(name, mod, export);
+		s = new_symbol(symbol_name, mod, export);
+		s->namespace = namespace;
 	} else {
 		if (!s->preloaded) {
-			warn("%s: '%s' exported twice. Previous export "
-			     "was in %s%s\n", mod->name, name,
-			     s->module->name,
-			     is_vmlinux(s->module->name) ?"":".ko");
+			warn("%s: '%s' exported twice. Previous export was in %s%s\n",
+			     mod->name, symbol_name, s->module->name,
+			     is_vmlinux(s->module->name) ? "" : ".ko");
 		} else {
 			/* In case Module.symvers was out of date */
 			s->module = mod;
@@ -1943,6 +1993,7 @@ static void read_symbols(const char *modname)
 	const char *symname;
 	char *version;
 	char *license;
+	char *namespace;
 	struct module *mod;
 	struct elf_info info = { };
 	Elf_Sym *sym;
@@ -1974,6 +2025,12 @@ static void read_symbols(const char *modname)
 		license = get_next_modinfo(&info, "license", license);
 	}
 
+	namespace = get_modinfo(&info, "import_ns");
+	while (namespace) {
+		add_namespace(&mod->imported_namespaces, namespace);
+		namespace = get_next_modinfo(&info, "import_ns", namespace);
+	}
+
 	for (sym = info.symtab_start; sym < info.symtab_stop; sym++) {
 		symname = remove_dot(info.strtab + sym->st_name);
 
@@ -2118,6 +2175,13 @@ static int check_exports(struct module *mod)
 			basename++;
 		else
 			basename = mod->name;
+
+		if (exp->namespace &&
+		    !module_imports_namespace(mod, exp->namespace)) {
+			warn("module %s uses symbol %s from namespace %s, but does not import it.\n",
+			     basename, exp->name, exp->namespace);
+		}
+
 		if (!mod->gpl_compatible)
 			check_for_gpl_usage(exp->export, basename, exp->name);
 		check_for_unused(exp->export, basename, exp->name);
@@ -2395,16 +2459,21 @@ static void write_dump(const char *fname)
 {
 	struct buffer buf = { };
 	struct symbol *symbol;
+	const char *namespace;
 	int n;
 
 	for (n = 0; n < SYMBOL_HASH_SIZE ; n++) {
 		symbol = symbolhash[n];
 		while (symbol) {
-			if (dump_sym(symbol))
-				buf_printf(&buf, "0x%08x\t%s\t%s\t%s\n",
-					symbol->crc, symbol->name,
-					symbol->module->name,
-					export_str(symbol->export));
+			if (dump_sym(symbol)) {
+				namespace = symbol->namespace;
+				buf_printf(&buf, "0x%08x\t%s%s%s\t%s\t%s\n",
+					   symbol->crc, symbol->name,
+					   namespace ? "." : "",
+					   namespace ? namespace : "",
+					   symbol->module->name,
+					   export_str(symbol->export));
+			}
 			symbol = symbol->next;
 		}
 	}
diff --git a/scripts/mod/modpost.h b/scripts/mod/modpost.h
index 8453d6ac2f77..9626bf3e7424 100644
--- a/scripts/mod/modpost.h
+++ b/scripts/mod/modpost.h
@@ -109,6 +109,11 @@ buf_printf(struct buffer *buf, const char *fmt, ...);
 void
 buf_write(struct buffer *buf, const char *s, int len);
 
+struct namespace_list {
+	struct namespace_list *next;
+	char namespace[0];
+};
+
 struct module {
 	struct module *next;
 	const char *name;
@@ -121,6 +126,8 @@ struct module {
 	struct buffer dev_table_buf;
 	char	     srcversion[25];
 	int is_dot_o;
+	// Actual imported namespaces
+	struct namespace_list *imported_namespaces;
 };
 
 struct elf_info {
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
                     ` (3 preceding siblings ...)
  2019-08-13 12:17   ` [PATCH v2 04/10] modpost: " Matthias Maennich
@ 2019-08-13 12:17   ` Matthias Maennich
  2019-08-13 18:17     ` Greg KH
  2019-08-13 20:15     ` Saravana Kannan
  2019-08-13 12:17   ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
                     ` (5 subsequent siblings)
  10 siblings, 2 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton,
	Johannes Weiner, Peter Zijlstra (Intel),
	Mauro Carvalho Chehab, David Howells, Patrick Bellasi,
	Dan Williams, Adrian Reber, Richard Guy Briggs

If MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is enabled (default=n), the
requirement for modules to import all namespaces that are used by
the module is relaxed.

Enabling this option effectively allows (invalid) modules to be loaded
while only a warning is emitted.

Disabling this option keeps the enforcement at module loading time and
loading is denied if the module's imports are not satisfactory.

Reviewed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 init/Kconfig    | 14 ++++++++++++++
 kernel/module.c | 11 +++++++++--
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index bd7d650d4a99..b3373334cdf1 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2119,6 +2119,20 @@ config MODULE_COMPRESS_XZ
 
 endchoice
 
+config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
+	bool "Allow loading of modules with missing namespace imports"
+	default n
+	help
+	  Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
+	  a namespace. A module that makes use of a symbol exported with such a
+	  namespace is required to import the namespace via MODULE_IMPORT_NS().
+	  This option relaxes this requirement when loading a module. While
+	  technically there is no reason to enforce correct namespace imports,
+	  it creates consistency between symbols defining namespaces and users
+	  importing namespaces they make use of.
+
+	  If unsure, say N.
+
 config TRIM_UNUSED_KSYMS
 	bool "Trim unused exported kernel symbols"
 	depends on MODULES && !UNUSED_SYMBOLS
diff --git a/kernel/module.c b/kernel/module.c
index 57e8253f2251..7c934aaae2d3 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -1408,9 +1408,16 @@ static int verify_namespace_is_imported(const struct load_info *info,
 			imported_namespace = get_next_modinfo(
 				info, "import_ns", imported_namespace);
 		}
-		pr_err("%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
-		       mod->name, kernel_symbol_name(sym), namespace);
+#ifdef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
+		pr_warn(
+#else
+		pr_err(
+#endif
+			"%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
+			mod->name, kernel_symbol_name(sym), namespace);
+#ifndef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
 		return -EINVAL;
+#endif
 	}
 	return 0;
 }
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
                     ` (4 preceding siblings ...)
  2019-08-13 12:17   ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
@ 2019-08-13 12:17   ` Matthias Maennich
  2019-08-13 18:16     ` Greg KH
  2019-08-13 18:16     ` Greg KH
  2019-08-13 12:17   ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich
                     ` (4 subsequent siblings)
  10 siblings, 2 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Ard Biesheuvel,
	Michael Ellerman

To avoid excessive usage of EXPORT_SYMBOL_NS(sym, MY_NAMESPACE), where
MY_NAMESPACE will always be the namespace we are exporting to, allow
exporting all definitions of EXPORT_SYMBOL() and friends by defining
DEFAULT_SYMBOL_NAMESPACE.

For example, to export all symbols defined in usb-common into the
namespace USB_COMMON, add a line like this to drivers/usb/common/Makefile:

  ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=USB_COMMON

That is equivalent to changing all EXPORT_SYMBOL(sym) definitions to
EXPORT_SYMBOL_NS(sym, USB_COMMON). Subsequently all symbol namespaces
functionality will apply.

Another way of making use of this feature is to define the namespace
within source or header files similar to how TRACE_SYSTEM defines are
used:
  #undef DEFAULT_SYMBOL_NAMESPACE
  #define DEFAULT_SYMBOL_NAMESPACE USB_COMMON

Please note that, as opposed to TRACE_SYSTEM, DEFAULT_SYMBOL_NAMESPACE
has to be defined before including include/linux/export.h.

If DEFAULT_SYMBOL_NAMESPACE is defined, a symbol can still be exported
to another namespace by using EXPORT_SYMBOL_NS() and friends with
explicitly specifying the namespace.

Suggested-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 include/linux/export.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/export.h b/include/linux/export.h
index 8e12e05444d1..1fb243abdbc4 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -166,6 +166,12 @@ struct kernel_symbol {
 #define __EXPORT_SYMBOL ___EXPORT_SYMBOL
 #endif
 
+#ifdef DEFAULT_SYMBOL_NAMESPACE
+#undef __EXPORT_SYMBOL
+#define __EXPORT_SYMBOL(sym, sec)				\
+	__EXPORT_SYMBOL_NS(sym, sec, DEFAULT_SYMBOL_NAMESPACE)
+#endif
+
 #define EXPORT_SYMBOL(sym) __EXPORT_SYMBOL(sym, "")
 #define EXPORT_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_gpl")
 #define EXPORT_SYMBOL_GPL_FUTURE(sym) __EXPORT_SYMBOL(sym, "_gpl_future")
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 07/10] modpost: add support for generating namespace dependencies
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
                     ` (5 preceding siblings ...)
  2019-08-13 12:17   ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
@ 2019-08-13 12:17   ` Matthias Maennich
  2019-08-13 18:21     ` Greg KH
  2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
                     ` (3 subsequent siblings)
  10 siblings, 1 reply; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

This patch adds an option to modpost to generate a <module>.ns_deps file
per module, containing the namespace dependencies for that module.

E.g. if the linked module my-module.ko would depend on the symbol
myfunc.MY_NS in the namespace MY_NS, the my-module.ns_deps file created
by modpost would contain the entry MY_NS to express the namespace
dependency of my-module imposed by using the symbol myfunc.

These files can subsequently be used by static analysis tools (like
coccinelle scripts) to address issues with missing namespace imports. A
later patch of this series will introduce such a script 'nsdeps' and a
corresponding make target to automatically add missing
MODULE_IMPORT_NS() definitions to the module's sources. For that it uses
the information provided in the generated .ns_deps files.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 scripts/mod/modpost.c | 61 +++++++++++++++++++++++++++++++++++++++----
 scripts/mod/modpost.h |  2 ++
 2 files changed, 58 insertions(+), 5 deletions(-)

diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 538bb24ffee3..833a7e1bcbb5 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -38,6 +38,8 @@ static int sec_mismatch_count = 0;
 static int sec_mismatch_fatal = 0;
 /* ignore missing files */
 static int ignore_missing_files;
+/* write namespace dependencies */
+static int write_namespace_deps;
 
 enum export {
 	export_plain,      export_unused,     export_gpl,
@@ -2176,10 +2178,15 @@ static int check_exports(struct module *mod)
 		else
 			basename = mod->name;
 
-		if (exp->namespace &&
-		    !module_imports_namespace(mod, exp->namespace)) {
-			warn("module %s uses symbol %s from namespace %s, but does not import it.\n",
-			     basename, exp->name, exp->namespace);
+		if (exp->namespace) {
+			add_namespace(&mod->required_namespaces,
+				      exp->namespace);
+
+			if (!write_namespace_deps &&
+			    !module_imports_namespace(mod, exp->namespace)) {
+				warn("module %s uses symbol %s from namespace %s, but does not import it.\n",
+				     basename, exp->name, exp->namespace);
+			}
 		}
 
 		if (!mod->gpl_compatible)
@@ -2481,6 +2488,38 @@ static void write_dump(const char *fname)
 	free(buf.p);
 }
 
+static void write_namespace_deps_files(void)
+{
+	struct module *mod;
+	struct namespace_list *ns;
+	struct buffer ns_deps_buf = {};
+
+	for (mod = modules; mod; mod = mod->next) {
+		char fname[PATH_MAX];
+		const char *basename;
+
+		if (mod->skip)
+			continue;
+
+		ns_deps_buf.pos = 0;
+
+		for (ns = mod->required_namespaces; ns; ns = ns->next)
+			buf_printf(&ns_deps_buf, "%s\n", ns->namespace);
+
+		if (ns_deps_buf.pos == 0)
+			continue;
+
+		basename = strrchr(mod->name, '/');
+		if (basename)
+			basename++;
+		else
+			basename = mod->name;
+
+		sprintf(fname, ".tmp_versions/%s.ns_deps", basename);
+		write_if_changed(&ns_deps_buf, fname);
+	}
+}
+
 struct ext_sym_list {
 	struct ext_sym_list *next;
 	const char *file;
@@ -2497,7 +2536,7 @@ int main(int argc, char **argv)
 	struct ext_sym_list *extsym_iter;
 	struct ext_sym_list *extsym_start = NULL;
 
-	while ((opt = getopt(argc, argv, "i:I:e:mnsT:o:awE")) != -1) {
+	while ((opt = getopt(argc, argv, "i:I:e:mnsT:o:awEd")) != -1) {
 		switch (opt) {
 		case 'i':
 			kernel_read = optarg;
@@ -2538,6 +2577,9 @@ int main(int argc, char **argv)
 		case 'E':
 			sec_mismatch_fatal = 1;
 			break;
+		case 'd':
+			write_namespace_deps = 1;
+			break;
 		default:
 			exit(1);
 		}
@@ -2572,6 +2614,9 @@ int main(int argc, char **argv)
 
 		err |= check_modname_len(mod);
 		err |= check_exports(mod);
+		if (write_namespace_deps)
+			continue;
+
 		add_header(&buf, mod);
 		add_intree_flag(&buf, !external_module);
 		add_retpoline(&buf);
@@ -2584,6 +2629,12 @@ int main(int argc, char **argv)
 		sprintf(fname, "%s.mod.c", mod->name);
 		write_if_changed(&buf, fname);
 	}
+
+	if (write_namespace_deps) {
+		write_namespace_deps_files();
+		return 0;
+	}
+
 	if (dump_write)
 		write_dump(dump_write);
 	if (sec_mismatch_count && sec_mismatch_fatal)
diff --git a/scripts/mod/modpost.h b/scripts/mod/modpost.h
index 9626bf3e7424..92a926d375d2 100644
--- a/scripts/mod/modpost.h
+++ b/scripts/mod/modpost.h
@@ -126,6 +126,8 @@ struct module {
 	struct buffer dev_table_buf;
 	char	     srcversion[25];
 	int is_dot_o;
+	// Required namespace dependencies
+	struct namespace_list *required_namespaces;
 	// Actual imported namespaces
 	struct namespace_list *imported_namespaces;
 };
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 08/10] scripts: Coccinelle script for namespace dependencies.
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
                     ` (6 preceding siblings ...)
  2019-08-13 12:17   ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich
@ 2019-08-13 12:17   ` " Matthias Maennich
  2019-08-13 12:31     ` Julia Lawall
                       ` (4 more replies)
  2019-08-13 12:17   ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich
                     ` (2 subsequent siblings)
  10 siblings, 5 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Julia Lawall, Gilles Muller,
	Nicolas Palix, Mauro Carvalho Chehab, David S. Miller,
	Jonathan Cameron, Nicolas Ferre, cocci

A script that uses the '<module>.ns_deps' file generated by modpost to
automatically add the required symbol namespace dependencies to each
module.

Usage:
1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
   DEFAULT_SYMBOL_NAMESPACE
2) Run 'make' (or 'make modules') and get warnings about modules not
   importing that namespace.
3) Run 'make nsdeps' to automatically add required import statements
   to said modules.

This makes it easer for subsystem maintainers to introduce and maintain
symbol namespaces into their codebase.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 MAINTAINERS                                 |  5 ++
 Makefile                                    | 12 +++++
 scripts/Makefile.modpost                    |  4 +-
 scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++
 scripts/nsdeps                              | 54 +++++++++++++++++++++
 5 files changed, 97 insertions(+), 1 deletion(-)
 create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
 create mode 100644 scripts/nsdeps

diff --git a/MAINTAINERS b/MAINTAINERS
index e81e60bd7c26..aa169070a052 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11414,6 +11414,11 @@ S:	Maintained
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
 F:	tools/include/nolibc/
 
+NSDEPS
+M:	Matthias Maennich <maennich@google.com>
+S:	Maintained
+F:	scripts/nsdeps
+
 NTB AMD DRIVER
 M:	Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
 L:	linux-ntb@googlegroups.com
diff --git a/Makefile b/Makefile
index 1b23f95db176..c5c3356e133c 100644
--- a/Makefile
+++ b/Makefile
@@ -1500,6 +1500,9 @@ help:
 	@echo  '  headerdep       - Detect inclusion cycles in headers'
 	@echo  '  coccicheck      - Check with Coccinelle'
 	@echo  ''
+	@echo  'Tools:'
+	@echo  '  nsdeps          - Generate missing symbol namespace dependencies'
+	@echo  ''
 	@echo  'Kernel selftest:'
 	@echo  '  kselftest       - Build and run kernel selftest (run as root)'
 	@echo  '                    Build, install, and boot kernel before'
@@ -1687,6 +1690,15 @@ quiet_cmd_tags = GEN     $@
 tags TAGS cscope gtags: FORCE
 	$(call cmd,tags)
 
+# Script to generate missing namespace dependencies
+# ---------------------------------------------------------------------------
+
+PHONY += nsdeps
+
+nsdeps:
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost nsdeps
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/$@
+
 # Scripts to check various things for consistency
 # ---------------------------------------------------------------------------
 
diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
index 26e6574ecd08..743fe3a2e885 100644
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -56,7 +56,8 @@ MODPOST = scripts/mod/modpost						\
 	$(if $(KBUILD_EXTMOD),$(addprefix -e ,$(KBUILD_EXTRA_SYMBOLS)))	\
 	$(if $(KBUILD_EXTMOD),-o $(modulesymfile))			\
 	$(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E)			\
-	$(if $(KBUILD_MODPOST_WARN),-w)
+	$(if $(KBUILD_MODPOST_WARN),-w)					\
+	$(if $(filter nsdeps,$(MAKECMDGOALS)),-d)
 
 ifdef MODPOST_VMLINUX
 
@@ -134,6 +135,7 @@ $(modules): %.ko :%.o %.mod.o FORCE
 
 targets += $(modules)
 
+nsdeps: __modpost
 
 # Add FORCE to the prequisites of a target to force it to be always rebuilt.
 # ---------------------------------------------------------------------------
diff --git a/scripts/coccinelle/misc/add_namespace.cocci b/scripts/coccinelle/misc/add_namespace.cocci
new file mode 100644
index 000000000000..c832bb6445a8
--- /dev/null
+++ b/scripts/coccinelle/misc/add_namespace.cocci
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0-only
+//
+/// Adds missing MODULE_IMPORT_NS statements to source files
+///
+/// This script is usually called from scripts/nsdeps with -D ns=<namespace> to
+/// add a missing namespace tag to a module source file.
+///
+
+@has_ns_import@
+declarer name MODULE_IMPORT_NS;
+identifier virtual.ns;
+@@
+MODULE_IMPORT_NS(ns);
+
+// Add missing imports, but only adjacent to a MODULE_LICENSE statement.
+// That ensures we are adding it only to the main module source file.
+@do_import depends on !has_ns_import@
+declarer name MODULE_LICENSE;
+expression license;
+identifier virtual.ns;
+@@
+MODULE_LICENSE(license);
++ MODULE_IMPORT_NS(ns);
diff --git a/scripts/nsdeps b/scripts/nsdeps
new file mode 100644
index 000000000000..148db65a830f
--- /dev/null
+++ b/scripts/nsdeps
@@ -0,0 +1,54 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Linux kernel symbol namespace import generator
+#
+# This script requires at least spatch
+# version 1.0.4.
+SPATCH_REQ_VERSION="1.0.4"
+
+DIR="$(dirname $(readlink -f $0))/.."
+SPATCH="`which ${SPATCH:=spatch}`"
+if [ ! -x "$SPATCH" ]; then
+    echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/'
+    exit 1
+fi
+
+SPATCH_REQ_VERSION_NUM=$(echo $SPATCH_REQ_VERSION | ${DIR}/scripts/ld-version.sh)
+SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}')
+SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh)
+
+if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then
+    echo 'spatch needs to be version 1.06 or higher'
+    exit 1
+fi
+
+generate_deps_for_ns() {
+    $SPATCH --very-quiet --in-place --sp-file \
+	    $srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2
+}
+
+generate_deps() {
+    local mod_file=`echo $@ | sed -e 's/\.ns_deps/\.mod/'`
+    local mod_name=`cat $mod_file | sed -n 1p | sed -e 's/\/[^.]*$//'`
+    local mod_source_files=`cat $mod_file | sed -n 2p | sed -e 's/\.o/\.c/g'`
+    for ns in `cat $@`; do
+	echo "Adding namespace $ns to module $mod_name (if needed)."
+        generate_deps_for_ns $ns $mod_source_files
+	# sort the imports
+        for source_file in $mod_source_files; do
+            sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp
+            offset=$(wc -l ${source_file}.tmp | awk '{print $1;}')
+            cat $source_file | grep MODULE_IMPORT_NS | sort -u >> ${source_file}.tmp
+            tail -n +$((offset +1)) ${source_file} | grep -v MODULE_IMPORT_NS >> ${source_file}.tmp
+            if ! diff -q ${source_file} ${source_file}.tmp; then
+                mv ${source_file}.tmp ${source_file}
+            else
+                rm ${source_file}.tmp
+            fi
+        done
+    done
+}
+
+for f in `find $srctree/.tmp_versions/ -name *.ns_deps`; do
+    generate_deps $f
+done
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 09/10] usb-storage: remove single-use define for debugging
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
                     ` (7 preceding siblings ...)
  2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
@ 2019-08-13 12:17   ` Matthias Maennich
  2019-08-13 12:42     ` Greg KH
  2019-08-13 12:17   ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
  10 siblings, 1 reply; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

USB_STORAGE was defined as "usb-storage: " and used in a single location
as argument to printk. In order to be able to use the name
'USB_STORAGE', drop the definition and use the string directly for the
printk call.

Signed-off-by: Matthias Maennich <maennich@google.com>
---
 drivers/usb/storage/debug.h    | 2 --
 drivers/usb/storage/scsiglue.c | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/usb/storage/debug.h b/drivers/usb/storage/debug.h
index 6d64f342f587..16ce06039a4d 100644
--- a/drivers/usb/storage/debug.h
+++ b/drivers/usb/storage/debug.h
@@ -29,8 +29,6 @@
 
 #include <linux/kernel.h>
 
-#define USB_STORAGE "usb-storage: "
-
 #ifdef CONFIG_USB_STORAGE_DEBUG
 void usb_stor_show_command(const struct us_data *us, struct scsi_cmnd *srb);
 void usb_stor_show_sense(const struct us_data *us, unsigned char key,
diff --git a/drivers/usb/storage/scsiglue.c b/drivers/usb/storage/scsiglue.c
index 05b80211290d..df4de8323eff 100644
--- a/drivers/usb/storage/scsiglue.c
+++ b/drivers/usb/storage/scsiglue.c
@@ -379,7 +379,7 @@ static int queuecommand_lck(struct scsi_cmnd *srb,
 
 	/* check for state-transition errors */
 	if (us->srb != NULL) {
-		printk(KERN_ERR USB_STORAGE "Error in %s: us->srb = %p\n",
+		printk(KERN_ERR "usb-storage: Error in %s: us->srb = %p\n",
 			__func__, us->srb);
 		return SCSI_MLQUEUE_HOST_BUSY;
 	}
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace
  2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
                     ` (8 preceding siblings ...)
  2019-08-13 12:17   ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich
@ 2019-08-13 12:17   ` Matthias Maennich
  2019-08-13 12:45     ` Greg KH
  2019-08-13 12:47     ` Greg KH
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
  10 siblings, 2 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 12:17 UTC (permalink / raw)
  To: linux-kernel, maco
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

Modules using these symbols are required to explicitly import the
namespace. This patch was generated with the following steps and serves
as a reference to use the symbol namespace feature:

 1) Define DDEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
 2) make  (see warnings during modpost about missing imports)
 3) make nsdeps

Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
variants can be used to explicitly specify the namespace. The advantage
of the method used here is that newly added symbols are automatically
exported and existing ones are exported without touching their
respective EXPORT_SYMBOL macro expansion.

Signed-off-by: Matthias Maennich <maennich@google.com>
---
 drivers/usb/storage/Makefile        | 2 ++
 drivers/usb/storage/alauda.c        | 1 +
 drivers/usb/storage/cypress_atacb.c | 1 +
 drivers/usb/storage/datafab.c       | 1 +
 drivers/usb/storage/ene_ub6250.c    | 1 +
 drivers/usb/storage/freecom.c       | 1 +
 drivers/usb/storage/isd200.c        | 1 +
 drivers/usb/storage/jumpshot.c      | 1 +
 drivers/usb/storage/karma.c         | 1 +
 drivers/usb/storage/onetouch.c      | 1 +
 drivers/usb/storage/realtek_cr.c    | 1 +
 drivers/usb/storage/sddr09.c        | 1 +
 drivers/usb/storage/sddr55.c        | 1 +
 drivers/usb/storage/shuttle_usbat.c | 1 +
 drivers/usb/storage/uas.c           | 1 +
 15 files changed, 16 insertions(+)

diff --git a/drivers/usb/storage/Makefile b/drivers/usb/storage/Makefile
index a67ddcbb4e24..46635fa4a340 100644
--- a/drivers/usb/storage/Makefile
+++ b/drivers/usb/storage/Makefile
@@ -8,6 +8,8 @@
 
 ccflags-y := -I $(srctree)/drivers/scsi
 
+ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=USB_STORAGE
+
 obj-$(CONFIG_USB_UAS)		+= uas.o
 obj-$(CONFIG_USB_STORAGE)	+= usb-storage.o
 
diff --git a/drivers/usb/storage/alauda.c b/drivers/usb/storage/alauda.c
index 6b8edf6178df..ddab2cd3d2e7 100644
--- a/drivers/usb/storage/alauda.c
+++ b/drivers/usb/storage/alauda.c
@@ -36,6 +36,7 @@
 MODULE_DESCRIPTION("Driver for Alauda-based card readers");
 MODULE_AUTHOR("Daniel Drake <dsd@gentoo.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * Status bytes
diff --git a/drivers/usb/storage/cypress_atacb.c b/drivers/usb/storage/cypress_atacb.c
index 4825902377eb..a6f3267bbef6 100644
--- a/drivers/usb/storage/cypress_atacb.c
+++ b/drivers/usb/storage/cypress_atacb.c
@@ -22,6 +22,7 @@
 MODULE_DESCRIPTION("SAT support for Cypress USB/ATA bridges with ATACB");
 MODULE_AUTHOR("Matthieu Castet <castet.matthieu@free.fr>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * The table of devices
diff --git a/drivers/usb/storage/datafab.c b/drivers/usb/storage/datafab.c
index 09353be199be..588818483f4b 100644
--- a/drivers/usb/storage/datafab.c
+++ b/drivers/usb/storage/datafab.c
@@ -54,6 +54,7 @@
 MODULE_DESCRIPTION("Driver for Datafab USB Compact Flash reader");
 MODULE_AUTHOR("Jimmie Mayfield <mayfield+datafab@sackheads.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 struct datafab_info {
 	unsigned long   sectors;	/* total sector count */
diff --git a/drivers/usb/storage/ene_ub6250.c b/drivers/usb/storage/ene_ub6250.c
index c26129d5b943..8b1b73065421 100644
--- a/drivers/usb/storage/ene_ub6250.c
+++ b/drivers/usb/storage/ene_ub6250.c
@@ -26,6 +26,7 @@
 
 MODULE_DESCRIPTION("Driver for ENE UB6250 reader");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 MODULE_FIRMWARE(SD_INIT1_FIRMWARE);
 MODULE_FIRMWARE(SD_INIT2_FIRMWARE);
 MODULE_FIRMWARE(SD_RW_FIRMWARE);
diff --git a/drivers/usb/storage/freecom.c b/drivers/usb/storage/freecom.c
index 4f542df37a44..34e7eaff1174 100644
--- a/drivers/usb/storage/freecom.c
+++ b/drivers/usb/storage/freecom.c
@@ -29,6 +29,7 @@
 MODULE_DESCRIPTION("Driver for Freecom USB/IDE adaptor");
 MODULE_AUTHOR("David Brown <usb-storage@davidb.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 #ifdef CONFIG_USB_STORAGE_DEBUG
 static void pdump(struct us_data *us, void *ibuffer, int length);
diff --git a/drivers/usb/storage/isd200.c b/drivers/usb/storage/isd200.c
index 2b474d60b4db..c4da3fd6eff9 100644
--- a/drivers/usb/storage/isd200.c
+++ b/drivers/usb/storage/isd200.c
@@ -53,6 +53,7 @@
 MODULE_DESCRIPTION("Driver for In-System Design, Inc. ISD200 ASIC");
 MODULE_AUTHOR("Björn Stenberg <bjorn@haxx.se>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 static int isd200_Initialization(struct us_data *us);
 
diff --git a/drivers/usb/storage/jumpshot.c b/drivers/usb/storage/jumpshot.c
index 917f170c4124..229bf0c1afc9 100644
--- a/drivers/usb/storage/jumpshot.c
+++ b/drivers/usb/storage/jumpshot.c
@@ -51,6 +51,7 @@
 MODULE_DESCRIPTION("Driver for Lexar \"Jumpshot\" Compact Flash reader");
 MODULE_AUTHOR("Jimmie Mayfield <mayfield+usb@sackheads.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * The table of devices
diff --git a/drivers/usb/storage/karma.c b/drivers/usb/storage/karma.c
index 395cf8fb5870..05cec81dcd3f 100644
--- a/drivers/usb/storage/karma.c
+++ b/drivers/usb/storage/karma.c
@@ -23,6 +23,7 @@
 MODULE_DESCRIPTION("Driver for Rio Karma");
 MODULE_AUTHOR("Bob Copeland <me@bobcopeland.com>, Keith Bennett <keith@mcs.st-and.ac.uk>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 #define RIO_PREFIX "RIOP\x00"
 #define RIO_PREFIX_LEN 5
diff --git a/drivers/usb/storage/onetouch.c b/drivers/usb/storage/onetouch.c
index 39a5009a41a6..a989fe930e21 100644
--- a/drivers/usb/storage/onetouch.c
+++ b/drivers/usb/storage/onetouch.c
@@ -25,6 +25,7 @@
 MODULE_DESCRIPTION("Maxtor USB OneTouch hard drive button driver");
 MODULE_AUTHOR("Nick Sillik <n.sillik@temple.edu>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 #define ONETOUCH_PKT_LEN        0x02
 #define ONETOUCH_BUTTON         KEY_PROG1
diff --git a/drivers/usb/storage/realtek_cr.c b/drivers/usb/storage/realtek_cr.c
index cc794e25a0b6..edbe419053d6 100644
--- a/drivers/usb/storage/realtek_cr.c
+++ b/drivers/usb/storage/realtek_cr.c
@@ -35,6 +35,7 @@
 MODULE_DESCRIPTION("Driver for Realtek USB Card Reader");
 MODULE_AUTHOR("wwang <wei_wang@realsil.com.cn>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 static int auto_delink_en = 1;
 module_param(auto_delink_en, int, S_IRUGO | S_IWUSR);
diff --git a/drivers/usb/storage/sddr09.c b/drivers/usb/storage/sddr09.c
index bc9da736bdfc..51bcd4a43690 100644
--- a/drivers/usb/storage/sddr09.c
+++ b/drivers/usb/storage/sddr09.c
@@ -47,6 +47,7 @@
 MODULE_DESCRIPTION("Driver for SanDisk SDDR-09 SmartMedia reader");
 MODULE_AUTHOR("Andries Brouwer <aeb@cwi.nl>, Robert Baruch <autophile@starband.net>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 static int usb_stor_sddr09_dpcm_init(struct us_data *us);
 static int sddr09_transport(struct scsi_cmnd *srb, struct us_data *us);
diff --git a/drivers/usb/storage/sddr55.c b/drivers/usb/storage/sddr55.c
index b8527c55335b..ba955d65eb0e 100644
--- a/drivers/usb/storage/sddr55.c
+++ b/drivers/usb/storage/sddr55.c
@@ -29,6 +29,7 @@
 MODULE_DESCRIPTION("Driver for SanDisk SDDR-55 SmartMedia reader");
 MODULE_AUTHOR("Simon Munton");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * The table of devices
diff --git a/drivers/usb/storage/shuttle_usbat.c b/drivers/usb/storage/shuttle_usbat.c
index 854498e1012c..54aa1392c9ca 100644
--- a/drivers/usb/storage/shuttle_usbat.c
+++ b/drivers/usb/storage/shuttle_usbat.c
@@ -48,6 +48,7 @@
 MODULE_DESCRIPTION("Driver for SCM Microsystems (a.k.a. Shuttle) USB-ATAPI cable");
 MODULE_AUTHOR("Daniel Drake <dsd@gentoo.org>, Robert Baruch <autophile@starband.net>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /* Supported device types */
 #define USBAT_DEV_HP8200	0x01
diff --git a/drivers/usb/storage/uas.c b/drivers/usb/storage/uas.c
index 047c5922618f..bf80d6f81f58 100644
--- a/drivers/usb/storage/uas.c
+++ b/drivers/usb/storage/uas.c
@@ -1219,5 +1219,6 @@ static struct usb_driver uas_driver = {
 module_usb_driver(uas_driver);
 
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 MODULE_AUTHOR(
 	"Hans de Goede <hdegoede@redhat.com>, Matthew Wilcox and Sarah Sharp");
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 08/10] scripts: Coccinelle script for namespace dependencies.
  2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
@ 2019-08-13 12:31     ` Julia Lawall
  2019-08-13 12:44     ` Greg KH
                       ` (3 subsequent siblings)
  4 siblings, 0 replies; 51+ messages in thread
From: Julia Lawall @ 2019-08-13 12:31 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, gregkh, hpa, jeyu,
	joel, kstewart, linux-arch, linux-kbuild, linux-m68k,
	linux-modules, linux-scsi, linux-usb, lucas.de.marchi, maco,
	michal.lkml, mingo, oneukum, pombredanne, sam, sboyd, sspatil,
	stern, tglx, usb-storage, x86, yamada.masahiro, Julia Lawall,
	Gilles Muller, Nicolas Palix, Mauro Carvalho Chehab,
	David S. Miller, Jonathan Cameron, Nicolas Ferre, cocci



On Tue, 13 Aug 2019, Matthias Maennich wrote:

> A script that uses the '<module>.ns_deps' file generated by modpost to
> automatically add the required symbol namespace dependencies to each
> module.
>
> Usage:
> 1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
>    DEFAULT_SYMBOL_NAMESPACE
> 2) Run 'make' (or 'make modules') and get warnings about modules not
>    importing that namespace.
> 3) Run 'make nsdeps' to automatically add required import statements
>    to said modules.
>
> This makes it easer for subsystem maintainers to introduce and maintain
> symbol namespaces into their codebase.
>
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>


> ---
>  MAINTAINERS                                 |  5 ++
>  Makefile                                    | 12 +++++
>  scripts/Makefile.modpost                    |  4 +-
>  scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++
>  scripts/nsdeps                              | 54 +++++++++++++++++++++
>  5 files changed, 97 insertions(+), 1 deletion(-)
>  create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
>  create mode 100644 scripts/nsdeps
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e81e60bd7c26..aa169070a052 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -11414,6 +11414,11 @@ S:	Maintained
>  T:	git git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
>  F:	tools/include/nolibc/
>
> +NSDEPS
> +M:	Matthias Maennich <maennich@google.com>
> +S:	Maintained
> +F:	scripts/nsdeps
> +
>  NTB AMD DRIVER
>  M:	Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
>  L:	linux-ntb@googlegroups.com
> diff --git a/Makefile b/Makefile
> index 1b23f95db176..c5c3356e133c 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1500,6 +1500,9 @@ help:
>  	@echo  '  headerdep       - Detect inclusion cycles in headers'
>  	@echo  '  coccicheck      - Check with Coccinelle'
>  	@echo  ''
> +	@echo  'Tools:'
> +	@echo  '  nsdeps          - Generate missing symbol namespace dependencies'
> +	@echo  ''
>  	@echo  'Kernel selftest:'
>  	@echo  '  kselftest       - Build and run kernel selftest (run as root)'
>  	@echo  '                    Build, install, and boot kernel before'
> @@ -1687,6 +1690,15 @@ quiet_cmd_tags = GEN     $@
>  tags TAGS cscope gtags: FORCE
>  	$(call cmd,tags)
>
> +# Script to generate missing namespace dependencies
> +# ---------------------------------------------------------------------------
> +
> +PHONY += nsdeps
> +
> +nsdeps:
> +	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost nsdeps
> +	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/$@
> +
>  # Scripts to check various things for consistency
>  # ---------------------------------------------------------------------------
>
> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> index 26e6574ecd08..743fe3a2e885 100644
> --- a/scripts/Makefile.modpost
> +++ b/scripts/Makefile.modpost
> @@ -56,7 +56,8 @@ MODPOST = scripts/mod/modpost						\
>  	$(if $(KBUILD_EXTMOD),$(addprefix -e ,$(KBUILD_EXTRA_SYMBOLS)))	\
>  	$(if $(KBUILD_EXTMOD),-o $(modulesymfile))			\
>  	$(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E)			\
> -	$(if $(KBUILD_MODPOST_WARN),-w)
> +	$(if $(KBUILD_MODPOST_WARN),-w)					\
> +	$(if $(filter nsdeps,$(MAKECMDGOALS)),-d)
>
>  ifdef MODPOST_VMLINUX
>
> @@ -134,6 +135,7 @@ $(modules): %.ko :%.o %.mod.o FORCE
>
>  targets += $(modules)
>
> +nsdeps: __modpost
>
>  # Add FORCE to the prequisites of a target to force it to be always rebuilt.
>  # ---------------------------------------------------------------------------
> diff --git a/scripts/coccinelle/misc/add_namespace.cocci b/scripts/coccinelle/misc/add_namespace.cocci
> new file mode 100644
> index 000000000000..c832bb6445a8
> --- /dev/null
> +++ b/scripts/coccinelle/misc/add_namespace.cocci
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +//
> +/// Adds missing MODULE_IMPORT_NS statements to source files
> +///
> +/// This script is usually called from scripts/nsdeps with -D ns=<namespace> to
> +/// add a missing namespace tag to a module source file.
> +///
> +
> +@has_ns_import@
> +declarer name MODULE_IMPORT_NS;
> +identifier virtual.ns;
> +@@
> +MODULE_IMPORT_NS(ns);
> +
> +// Add missing imports, but only adjacent to a MODULE_LICENSE statement.
> +// That ensures we are adding it only to the main module source file.
> +@do_import depends on !has_ns_import@
> +declarer name MODULE_LICENSE;
> +expression license;
> +identifier virtual.ns;
> +@@
> +MODULE_LICENSE(license);
> ++ MODULE_IMPORT_NS(ns);
> diff --git a/scripts/nsdeps b/scripts/nsdeps
> new file mode 100644
> index 000000000000..148db65a830f
> --- /dev/null
> +++ b/scripts/nsdeps
> @@ -0,0 +1,54 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Linux kernel symbol namespace import generator
> +#
> +# This script requires at least spatch
> +# version 1.0.4.
> +SPATCH_REQ_VERSION="1.0.4"
> +
> +DIR="$(dirname $(readlink -f $0))/.."
> +SPATCH="`which ${SPATCH:=spatch}`"
> +if [ ! -x "$SPATCH" ]; then
> +    echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/'
> +    exit 1
> +fi
> +
> +SPATCH_REQ_VERSION_NUM=$(echo $SPATCH_REQ_VERSION | ${DIR}/scripts/ld-version.sh)
> +SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}')
> +SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh)
> +
> +if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then
> +    echo 'spatch needs to be version 1.06 or higher'
> +    exit 1
> +fi
> +
> +generate_deps_for_ns() {
> +    $SPATCH --very-quiet --in-place --sp-file \
> +	    $srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2
> +}
> +
> +generate_deps() {
> +    local mod_file=`echo $@ | sed -e 's/\.ns_deps/\.mod/'`
> +    local mod_name=`cat $mod_file | sed -n 1p | sed -e 's/\/[^.]*$//'`
> +    local mod_source_files=`cat $mod_file | sed -n 2p | sed -e 's/\.o/\.c/g'`
> +    for ns in `cat $@`; do
> +	echo "Adding namespace $ns to module $mod_name (if needed)."
> +        generate_deps_for_ns $ns $mod_source_files
> +	# sort the imports
> +        for source_file in $mod_source_files; do
> +            sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp
> +            offset=$(wc -l ${source_file}.tmp | awk '{print $1;}')
> +            cat $source_file | grep MODULE_IMPORT_NS | sort -u >> ${source_file}.tmp
> +            tail -n +$((offset +1)) ${source_file} | grep -v MODULE_IMPORT_NS >> ${source_file}.tmp
> +            if ! diff -q ${source_file} ${source_file}.tmp; then
> +                mv ${source_file}.tmp ${source_file}
> +            else
> +                rm ${source_file}.tmp
> +            fi
> +        done
> +    done
> +}
> +
> +for f in `find $srctree/.tmp_versions/ -name *.ns_deps`; do
> +    generate_deps $f
> +done
> --
> 2.23.0.rc1.153.gdeed80330f-goog
>
>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 01/10] module: support reading multiple values per modinfo tag
  2019-08-13 12:16   ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
@ 2019-08-13 12:40     ` Greg KH
  0 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 12:40 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 01:16:58PM +0100, Matthias Maennich wrote:
> Similar to modpost's get_next_modinfo(), introduce get_next_modinfo() in
> kernel/module.c to acquire any further values associated with the same
> modinfo tag name. That is useful for any tags that have multiple
> occurrences (such as 'alias'), but is in particular introduced here as
> part of the symbol namespaces patch series to read the (potentially)
> multiple namespaces a module is importing.
> 
> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> Reviewed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 02/10] export: explicitly align struct kernel_symbol
  2019-08-13 12:16   ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich
@ 2019-08-13 12:41     ` Greg KH
  0 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 12:41 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton, Nicolas Pitre,
	Michael Ellerman, Ard Biesheuvel

On Tue, Aug 13, 2019 at 01:16:59PM +0100, Matthias Maennich wrote:
> This change allows growing struct kernel_symbol without wasting bytes to
> alignment. It also concretized the alignment of ksymtab entries if
> relative references are used for ksymtab entries.
> 
> struct kernel_symbol was already implicitly being aligned to the word
> size, except on x86_64 and m68k, where it is aligned to 16 and 2 bytes,
> respectively.
> 
> As far as I can tell there is no requirement for aligning struct
> kernel_symbol to 16 bytes on x86_64, but gcc aligns structs to their
> size, and the linker aligns the custom __ksymtab sections to the largest
> data type contained within, so setting KSYM_ALIGN to 16 was necessary to
> stay consistent with the code generated for non-ASM EXPORT_SYMBOL(). Now
> that non-ASM EXPORT_SYMBOL() explicitly aligns to word size (8),
> KSYM_ALIGN is no longer necessary.
> 
> In case of relative references, the alignment has been changed
> accordingly to not waste space when adding new struct members.
> 
> As for m68k, struct kernel_symbol is aligned to 2 bytes even though the
> structure itself is 8 bytes; using a 4-byte alignment shouldn't hurt.
> 
> I manually verified the output of the __ksymtab sections didn't change
> on x86, x86_64, arm, arm64 and m68k. As expected, the section contents
> didn't change, and the ELF section alignment only changed on x86_64 and
> m68k. Feedback from other archs more than welcome.
> 
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>

Ick, messy, nice fix.

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 09/10] usb-storage: remove single-use define for debugging
  2019-08-13 12:17   ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich
@ 2019-08-13 12:42     ` Greg KH
  2019-08-13 13:12       ` Greg KH
  0 siblings, 1 reply; 51+ messages in thread
From: Greg KH @ 2019-08-13 12:42 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 01:17:06PM +0100, Matthias Maennich wrote:
> USB_STORAGE was defined as "usb-storage: " and used in a single location
> as argument to printk. In order to be able to use the name
> 'USB_STORAGE', drop the definition and use the string directly for the
> printk call.
> 
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  drivers/usb/storage/debug.h    | 2 --
>  drivers/usb/storage/scsiglue.c | 2 +-
>  2 files changed, 1 insertion(+), 3 deletions(-)

I'll go take this today.  The module really should just be using
dev_err() there.  It needs to be cleaned up :(

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 08/10] scripts: Coccinelle script for namespace dependencies.
  2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
  2019-08-13 12:31     ` Julia Lawall
@ 2019-08-13 12:44     ` Greg KH
  2019-08-14  6:36     ` [Cocci] " Himanshu Jha
                       ` (2 subsequent siblings)
  4 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 12:44 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Julia Lawall, Gilles Muller,
	Nicolas Palix, Mauro Carvalho Chehab, David S. Miller,
	Jonathan Cameron, Nicolas Ferre, cocci

On Tue, Aug 13, 2019 at 01:17:05PM +0100, Matthias Maennich wrote:
> A script that uses the '<module>.ns_deps' file generated by modpost to
> automatically add the required symbol namespace dependencies to each
> module.
> 
> Usage:
> 1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
>    DEFAULT_SYMBOL_NAMESPACE
> 2) Run 'make' (or 'make modules') and get warnings about modules not
>    importing that namespace.
> 3) Run 'make nsdeps' to automatically add required import statements
>    to said modules.
> 
> This makes it easer for subsystem maintainers to introduce and maintain
> symbol namespaces into their codebase.
> 
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---

I really can't express just how cool this patch is.  I was amazed when I
first saw it in action a long time ago, and still am.

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace
  2019-08-13 12:17   ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
@ 2019-08-13 12:45     ` Greg KH
  2019-08-13 12:47     ` Greg KH
  1 sibling, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 12:45 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 01:17:07PM +0100, Matthias Maennich wrote:
> Modules using these symbols are required to explicitly import the
> namespace. This patch was generated with the following steps and serves
> as a reference to use the symbol namespace feature:
> 
>  1) Define DDEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
>  2) make  (see warnings during modpost about missing imports)
>  3) make nsdeps
> 
> Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
> variants can be used to explicitly specify the namespace. The advantage
> of the method used here is that newly added symbols are automatically
> exported and existing ones are exported without touching their
> respective EXPORT_SYMBOL macro expansion.
> 
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  drivers/usb/storage/Makefile        | 2 ++
>  drivers/usb/storage/alauda.c        | 1 +
>  drivers/usb/storage/cypress_atacb.c | 1 +
>  drivers/usb/storage/datafab.c       | 1 +
>  drivers/usb/storage/ene_ub6250.c    | 1 +
>  drivers/usb/storage/freecom.c       | 1 +
>  drivers/usb/storage/isd200.c        | 1 +
>  drivers/usb/storage/jumpshot.c      | 1 +
>  drivers/usb/storage/karma.c         | 1 +
>  drivers/usb/storage/onetouch.c      | 1 +
>  drivers/usb/storage/realtek_cr.c    | 1 +
>  drivers/usb/storage/sddr09.c        | 1 +
>  drivers/usb/storage/sddr55.c        | 1 +
>  drivers/usb/storage/shuttle_usbat.c | 1 +
>  drivers/usb/storage/uas.c           | 1 +
>  15 files changed, 16 insertions(+)
> 
> diff --git a/drivers/usb/storage/Makefile b/drivers/usb/storage/Makefile
> index a67ddcbb4e24..46635fa4a340 100644
> --- a/drivers/usb/storage/Makefile
> +++ b/drivers/usb/storage/Makefile
> @@ -8,6 +8,8 @@
>  
>  ccflags-y := -I $(srctree)/drivers/scsi
>  
> +ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=USB_STORAGE

Wait, we have to do this for every subsystem?  I thought there was a
macro we could use in the code itself for this.  What changed from
earlier versions, or was this always here?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace
  2019-08-13 12:17   ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
  2019-08-13 12:45     ` Greg KH
@ 2019-08-13 12:47     ` Greg KH
  2019-08-13 15:02       ` Matthias Maennich
  1 sibling, 1 reply; 51+ messages in thread
From: Greg KH @ 2019-08-13 12:47 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 01:17:07PM +0100, Matthias Maennich wrote:
> Modules using these symbols are required to explicitly import the
> namespace. This patch was generated with the following steps and serves
> as a reference to use the symbol namespace feature:
> 
>  1) Define DDEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
>  2) make  (see warnings during modpost about missing imports)
>  3) make nsdeps
> 
> Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
> variants can be used to explicitly specify the namespace. The advantage
> of the method used here is that newly added symbols are automatically
> exported and existing ones are exported without touching their
> respective EXPORT_SYMBOL macro expansion.

Ok, I can't read text, this answers my previous question.

But, as an example, shouldn't we also have some code here that uses the
EXPORT_SYMBOL_NS() macro to ensure that it actually works?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 09/10] usb-storage: remove single-use define for debugging
  2019-08-13 12:42     ` Greg KH
@ 2019-08-13 13:12       ` Greg KH
  0 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 13:12 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 02:42:59PM +0200, Greg KH wrote:
> On Tue, Aug 13, 2019 at 01:17:06PM +0100, Matthias Maennich wrote:
> > USB_STORAGE was defined as "usb-storage: " and used in a single location
> > as argument to printk. In order to be able to use the name
> > 'USB_STORAGE', drop the definition and use the string directly for the
> > printk call.
> > 
> > Signed-off-by: Matthias Maennich <maennich@google.com>
> > ---
> >  drivers/usb/storage/debug.h    | 2 --
> >  drivers/usb/storage/scsiglue.c | 2 +-
> >  2 files changed, 1 insertion(+), 3 deletions(-)
> 
> I'll go take this today.  The module really should just be using
> dev_err() there.  It needs to be cleaned up :(

Now applied to the usb git tree, thanks.

greg k-h

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace
  2019-08-13 12:47     ` Greg KH
@ 2019-08-13 15:02       ` Matthias Maennich
  0 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-13 15:02 UTC (permalink / raw)
  To: Greg KH
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 02:47:08PM +0200, Greg KH wrote:
>On Tue, Aug 13, 2019 at 01:17:07PM +0100, Matthias Maennich wrote:
>> Modules using these symbols are required to explicitly import the
>> namespace. This patch was generated with the following steps and serves
>> as a reference to use the symbol namespace feature:
>>
>>  1) Define DDEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
>>  2) make  (see warnings during modpost about missing imports)
>>  3) make nsdeps
>>
>> Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
>> variants can be used to explicitly specify the namespace. The advantage
>> of the method used here is that newly added symbols are automatically
>> exported and existing ones are exported without touching their
>> respective EXPORT_SYMBOL macro expansion.
>
>Ok, I can't read text, this answers my previous question.
>
>But, as an example, shouldn't we also have some code here that uses the
>EXPORT_SYMBOL_NS() macro to ensure that it actually works?
>
I will create another patch for a different subsystem where the use of
the macros is more appropriate. Then we have both use cases covered.

Cheers,
Matthias


^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 03/10] module: add support for symbol namespaces.
  2019-08-13 12:17   ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich
@ 2019-08-13 15:26     ` Greg KH
  0 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 15:26 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton,
	Michael Ellerman, Ard Biesheuvel

On Tue, Aug 13, 2019 at 01:17:00PM +0100, Matthias Maennich wrote:
> The EXPORT_SYMBOL_NS() and EXPORT_SYMBOL_NS_GPL() macros can be used to
> export a symbol to a specific namespace.  There are no _GPL_FUTURE and
> _UNUSED variants because these are currently unused, and I'm not sure
> they are necessary.
> 
> I didn't add EXPORT_SYMBOL_NS() for ASM exports; this patch sets the
> namespace of ASM exports to NULL by default. In case of relative
> references, it will be relocatable to NULL. If there's a need, this
> should be pretty easy to add.
> 
> A module that wants to use a symbol exported to a namespace must add a
> MODULE_IMPORT_NS() statement to their module code; otherwise, modpost
> will complain when building the module, and the kernel module loader
> will emit an error and fail when loading the module.
> 
> MODULE_IMPORT_NS() adds a modinfo tag 'import_ns' to the module. That
> tag can be observed by the modinfo command, modpost and kernel/module.c
> at the time of loading the module.
> 
> The ELF symbols are renamed to include the namespace with an asm label;
> for example, symbol 'usb_stor_suspend' in namespace USB_STORAGE becomes
> 'usb_stor_suspend.USB_STORAGE'.  This allows modpost to do namespace
> checking, without having to go through all the effort of parsing ELF and
> relocation records just to get to the struct kernel_symbols.
> 
> On x86_64 I saw no difference in binary size (compression), but at
> runtime this will require a word of memory per export to hold the
> namespace. An alternative could be to store namespaced symbols in their
> own section and use a separate 'struct namespaced_kernel_symbol' for
> that section, at the cost of making the module loader more complex.
> 
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 04/10] modpost: add support for symbol namespaces
  2019-08-13 12:17   ` [PATCH v2 04/10] modpost: " Matthias Maennich
@ 2019-08-13 15:27     ` Greg KH
  0 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 15:27 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 01:17:01PM +0100, Matthias Maennich wrote:
> Add support for symbols that are exported into namespaces. For that,
> extract any namespace suffix from the symbol name. In addition, emit a
> warning whenever a module refers to an exported symbol without
> explicitly importing the namespace that it is defined in. This patch
> consistently adds the namespace suffix to symbol names exported into
> Module.symvers.
> 
> Example warning emitted by modpost in case of the above violation:
> 
>  WARNING: module ums-usbat uses symbol usb_stor_resume from namespace
>  USB_STORAGE, but does not import it.
> 
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  scripts/mod/modpost.c | 91 +++++++++++++++++++++++++++++++++++++------
>  scripts/mod/modpost.h |  7 ++++
>  2 files changed, 87 insertions(+), 11 deletions(-)

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources
  2019-08-13 12:17   ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
@ 2019-08-13 18:16     ` Greg KH
  2019-08-13 18:16     ` Greg KH
  1 sibling, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 18:16 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Ard Biesheuvel,
	Michael Ellerman

On Tue, Aug 13, 2019 at 01:17:03PM +0100, Matthias Maennich wrote:
> To avoid excessive usage of EXPORT_SYMBOL_NS(sym, MY_NAMESPACE), where
> MY_NAMESPACE will always be the namespace we are exporting to, allow
> exporting all definitions of EXPORT_SYMBOL() and friends by defining
> DEFAULT_SYMBOL_NAMESPACE.
> 
> For example, to export all symbols defined in usb-common into the
> namespace USB_COMMON, add a line like this to drivers/usb/common/Makefile:
> 
>   ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=USB_COMMON

I thought we were trying to get away from cflags :(

> 
> That is equivalent to changing all EXPORT_SYMBOL(sym) definitions to
> EXPORT_SYMBOL_NS(sym, USB_COMMON). Subsequently all symbol namespaces
> functionality will apply.
> 
> Another way of making use of this feature is to define the namespace
> within source or header files similar to how TRACE_SYSTEM defines are
> used:
>   #undef DEFAULT_SYMBOL_NAMESPACE
>   #define DEFAULT_SYMBOL_NAMESPACE USB_COMMON
> 
> Please note that, as opposed to TRACE_SYSTEM, DEFAULT_SYMBOL_NAMESPACE
> has to be defined before including include/linux/export.h.
> 
> If DEFAULT_SYMBOL_NAMESPACE is defined, a symbol can still be exported
> to another namespace by using EXPORT_SYMBOL_NS() and friends with
> explicitly specifying the namespace.

Ok, good, hopefully the cflags stuff will not be the default for people.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources
  2019-08-13 12:17   ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
  2019-08-13 18:16     ` Greg KH
@ 2019-08-13 18:16     ` Greg KH
  1 sibling, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 18:16 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Ard Biesheuvel,
	Michael Ellerman

On Tue, Aug 13, 2019 at 01:17:03PM +0100, Matthias Maennich wrote:
> To avoid excessive usage of EXPORT_SYMBOL_NS(sym, MY_NAMESPACE), where
> MY_NAMESPACE will always be the namespace we are exporting to, allow
> exporting all definitions of EXPORT_SYMBOL() and friends by defining
> DEFAULT_SYMBOL_NAMESPACE.
> 
> For example, to export all symbols defined in usb-common into the
> namespace USB_COMMON, add a line like this to drivers/usb/common/Makefile:
> 
>   ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=USB_COMMON
> 
> That is equivalent to changing all EXPORT_SYMBOL(sym) definitions to
> EXPORT_SYMBOL_NS(sym, USB_COMMON). Subsequently all symbol namespaces
> functionality will apply.
> 
> Another way of making use of this feature is to define the namespace
> within source or header files similar to how TRACE_SYSTEM defines are
> used:
>   #undef DEFAULT_SYMBOL_NAMESPACE
>   #define DEFAULT_SYMBOL_NAMESPACE USB_COMMON
> 
> Please note that, as opposed to TRACE_SYSTEM, DEFAULT_SYMBOL_NAMESPACE
> has to be defined before including include/linux/export.h.
> 
> If DEFAULT_SYMBOL_NAMESPACE is defined, a symbol can still be exported
> to another namespace by using EXPORT_SYMBOL_NS() and friends with
> explicitly specifying the namespace.
> 
> Suggested-by: Arnd Bergmann <arnd@arndb.de>
> Reviewed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  include/linux/export.h | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/include/linux/export.h b/include/linux/export.h
> index 8e12e05444d1..1fb243abdbc4 100644
> --- a/include/linux/export.h
> +++ b/include/linux/export.h
> @@ -166,6 +166,12 @@ struct kernel_symbol {
>  #define __EXPORT_SYMBOL ___EXPORT_SYMBOL
>  #endif
>  
> +#ifdef DEFAULT_SYMBOL_NAMESPACE
> +#undef __EXPORT_SYMBOL
> +#define __EXPORT_SYMBOL(sym, sec)				\
> +	__EXPORT_SYMBOL_NS(sym, sec, DEFAULT_SYMBOL_NAMESPACE)
> +#endif
> +
>  #define EXPORT_SYMBOL(sym) __EXPORT_SYMBOL(sym, "")
>  #define EXPORT_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_gpl")
>  #define EXPORT_SYMBOL_GPL_FUTURE(sym) __EXPORT_SYMBOL(sym, "_gpl_future")
> -- 
> 2.23.0.rc1.153.gdeed80330f-goog
> 

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
  2019-08-13 12:17   ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
@ 2019-08-13 18:17     ` Greg KH
  2019-08-13 20:15     ` Saravana Kannan
  1 sibling, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 18:17 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton,
	Johannes Weiner, Peter Zijlstra (Intel),
	Mauro Carvalho Chehab, David Howells, Patrick Bellasi,
	Dan Williams, Adrian Reber, Richard Guy Briggs

On Tue, Aug 13, 2019 at 01:17:02PM +0100, Matthias Maennich wrote:
> If MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is enabled (default=n), the
> requirement for modules to import all namespaces that are used by
> the module is relaxed.
> 
> Enabling this option effectively allows (invalid) modules to be loaded
> while only a warning is emitted.
> 
> Disabling this option keeps the enforcement at module loading time and
> loading is denied if the module's imports are not satisfactory.
> 
> Reviewed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  init/Kconfig    | 14 ++++++++++++++
>  kernel/module.c | 11 +++++++++--
>  2 files changed, 23 insertions(+), 2 deletions(-)
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index bd7d650d4a99..b3373334cdf1 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -2119,6 +2119,20 @@ config MODULE_COMPRESS_XZ
>  
>  endchoice
>  
> +config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
> +	bool "Allow loading of modules with missing namespace imports"
> +	default n

the default for config options is always N, no need to list it here.

> +	help
> +	  Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
> +	  a namespace. A module that makes use of a symbol exported with such a
> +	  namespace is required to import the namespace via MODULE_IMPORT_NS().
> +	  This option relaxes this requirement when loading a module. While
> +	  technically there is no reason to enforce correct namespace imports,
> +	  it creates consistency between symbols defining namespaces and users
> +	  importing namespaces they make use of.
> +
> +	  If unsure, say N.
> +
>  config TRIM_UNUSED_KSYMS
>  	bool "Trim unused exported kernel symbols"
>  	depends on MODULES && !UNUSED_SYMBOLS
> diff --git a/kernel/module.c b/kernel/module.c
> index 57e8253f2251..7c934aaae2d3 100644
> --- a/kernel/module.c
> +++ b/kernel/module.c
> @@ -1408,9 +1408,16 @@ static int verify_namespace_is_imported(const struct load_info *info,
>  			imported_namespace = get_next_modinfo(
>  				info, "import_ns", imported_namespace);
>  		}
> -		pr_err("%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
> -		       mod->name, kernel_symbol_name(sym), namespace);
> +#ifdef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
> +		pr_warn(
> +#else
> +		pr_err(
> +#endif
> +			"%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
> +			mod->name, kernel_symbol_name(sym), namespace);
> +#ifndef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
>  		return -EINVAL;
> +#endif

This #ifdef mess is a hack, but oh well :)

If you drop the above default line, feel free to add:

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 07/10] modpost: add support for generating namespace dependencies
  2019-08-13 12:17   ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich
@ 2019-08-13 18:21     ` Greg KH
  0 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-13 18:21 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Tue, Aug 13, 2019 at 01:17:04PM +0100, Matthias Maennich wrote:
> This patch adds an option to modpost to generate a <module>.ns_deps file
> per module, containing the namespace dependencies for that module.
> 
> E.g. if the linked module my-module.ko would depend on the symbol
> myfunc.MY_NS in the namespace MY_NS, the my-module.ns_deps file created
> by modpost would contain the entry MY_NS to express the namespace
> dependency of my-module imposed by using the symbol myfunc.
> 
> These files can subsequently be used by static analysis tools (like
> coccinelle scripts) to address issues with missing namespace imports. A
> later patch of this series will introduce such a script 'nsdeps' and a
> corresponding make target to automatically add missing
> MODULE_IMPORT_NS() definitions to the module's sources. For that it uses
> the information provided in the generated .ns_deps files.
> 
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  scripts/mod/modpost.c | 61 +++++++++++++++++++++++++++++++++++++++----
>  scripts/mod/modpost.h |  2 ++
>  2 files changed, 58 insertions(+), 5 deletions(-)

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
  2019-08-13 12:17   ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
  2019-08-13 18:17     ` Greg KH
@ 2019-08-13 20:15     ` Saravana Kannan
  2019-08-14 12:54       ` Matthias Maennich
  1 sibling, 1 reply; 51+ messages in thread
From: Saravana Kannan @ 2019-08-13 20:15 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: LKML, maco, Android Kernel Team, arnd, geert, Greg Kroah-Hartman,
	hpa, jeyu, Joel Fernandes (Google),
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, sboyd, Sandeep Patil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton,
	Johannes Weiner, Peter Zijlstra (Intel),
	Mauro Carvalho Chehab, David Howells, Patrick Bellasi,
	Dan Williams, Adrian Reber, Richard Guy Briggs

On Tue, Aug 13, 2019 at 5:19 AM 'Matthias Maennich' via kernel-team
<kernel-team@android.com> wrote:
>
> If MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is enabled (default=n), the
> requirement for modules to import all namespaces that are used by
> the module is relaxed.
>
> Enabling this option effectively allows (invalid) modules to be loaded
> while only a warning is emitted.
>
> Disabling this option keeps the enforcement at module loading time and
> loading is denied if the module's imports are not satisfactory.
>
> Reviewed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  init/Kconfig    | 14 ++++++++++++++
>  kernel/module.c | 11 +++++++++--
>  2 files changed, 23 insertions(+), 2 deletions(-)
>
> diff --git a/init/Kconfig b/init/Kconfig
> index bd7d650d4a99..b3373334cdf1 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -2119,6 +2119,20 @@ config MODULE_COMPRESS_XZ
>
>  endchoice
>
> +config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
> +       bool "Allow loading of modules with missing namespace imports"
> +       default n
> +       help
> +         Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
> +         a namespace. A module that makes use of a symbol exported with such a
> +         namespace is required to import the namespace via MODULE_IMPORT_NS().
> +         This option relaxes this requirement when loading a module.

> While
> +         technically there is no reason to enforce correct namespace imports,
> +         it creates consistency between symbols defining namespaces and users
> +         importing namespaces they make use of.

I'm confused by this sentence. It sounds like it's the opposite of
what the config is doing? Can you please reword it for clarify?

-Saravana

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [Cocci] [PATCH v2 08/10] scripts: Coccinelle script for namespace dependencies.
  2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
  2019-08-13 12:31     ` Julia Lawall
  2019-08-13 12:44     ` Greg KH
@ 2019-08-14  6:36     ` " Himanshu Jha
  2019-08-14  8:03       ` Matthias Maennich
  2019-08-14 12:00     ` [v2 " Markus Elfring
  2019-08-15 13:50     ` Markus Elfring
  4 siblings, 1 reply; 51+ messages in thread
From: Himanshu Jha @ 2019-08-14  6:36 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, maco, kstewart, linux-m68k, oneukum,
	Mauro Carvalho Chehab, usb-storage, hpa, joel, sam, cocci,
	linux-arch, linux-scsi, x86, lucas.de.marchi, mingo, geert,
	stern, kernel-team, sspatil, arnd, linux-kbuild, Nicolas Palix,
	jeyu, Jonathan Cameron, tglx, michal.lkml, gregkh, linux-usb,
	sboyd, Nicolas Ferre, maco, pombredanne, David S. Miller,
	linux-modules

On Tue, Aug 13, 2019 at 01:17:05PM +0100, Matthias Maennich wrote:
> A script that uses the '<module>.ns_deps' file generated by modpost to
> automatically add the required symbol namespace dependencies to each
> module.
> 
> Usage:
> 1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
>    DEFAULT_SYMBOL_NAMESPACE
> 2) Run 'make' (or 'make modules') and get warnings about modules not
>    importing that namespace.
> 3) Run 'make nsdeps' to automatically add required import statements
>    to said modules.
> 
> This makes it easer for subsystem maintainers to introduce and maintain
> symbol namespaces into their codebase.
> 
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---

[]

>  MAINTAINERS                                 |  5 ++
>  Makefile                                    | 12 +++++
>  scripts/Makefile.modpost                    |  4 +-
>  scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++
>  scripts/nsdeps                              | 54 +++++++++++++++++++++
>  5 files changed, 97 insertions(+), 1 deletion(-)
>  create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
>  create mode 100644 scripts/nsdeps

[]

> +if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then
> +    echo 'spatch needs to be version 1.06 or higher'

Nitpick: 1.0.6

> +    exit 1
> +fi


-- 
Himanshu Jha
Undergraduate Student
Department of Electronics & Communication
Guru Tegh Bahadur Institute of Technology

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [Cocci] [PATCH v2 08/10] scripts: Coccinelle script for namespace dependencies.
  2019-08-14  6:36     ` [Cocci] " Himanshu Jha
@ 2019-08-14  8:03       ` Matthias Maennich
  0 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-14  8:03 UTC (permalink / raw)
  To: Himanshu Jha
  Cc: linux-kernel, maco, kstewart, linux-m68k, oneukum,
	Mauro Carvalho Chehab, usb-storage, hpa, joel, sam, cocci,
	linux-arch, linux-scsi, x86, lucas.de.marchi, mingo, geert,
	stern, kernel-team, sspatil, arnd, linux-kbuild, Nicolas Palix,
	jeyu, Jonathan Cameron, tglx, michal.lkml, gregkh, linux-usb,
	Nicolas Ferre, maco, pombredanne, David S. Miller, linux-modules

On Wed, Aug 14, 2019 at 12:06:11PM +0530, Himanshu Jha wrote:
>On Tue, Aug 13, 2019 at 01:17:05PM +0100, Matthias Maennich wrote:
>> A script that uses the '<module>.ns_deps' file generated by modpost to
>> automatically add the required symbol namespace dependencies to each
>> module.
>>
>> Usage:
>> 1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
>>    DEFAULT_SYMBOL_NAMESPACE
>> 2) Run 'make' (or 'make modules') and get warnings about modules not
>>    importing that namespace.
>> 3) Run 'make nsdeps' to automatically add required import statements
>>    to said modules.
>>
>> This makes it easer for subsystem maintainers to introduce and maintain
>> symbol namespaces into their codebase.
>>
>> Co-developed-by: Martijn Coenen <maco@android.com>
>> Signed-off-by: Martijn Coenen <maco@android.com>
>> Signed-off-by: Matthias Maennich <maennich@google.com>
>> ---
>
>[]
>
>>  MAINTAINERS                                 |  5 ++
>>  Makefile                                    | 12 +++++
>>  scripts/Makefile.modpost                    |  4 +-
>>  scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++
>>  scripts/nsdeps                              | 54 +++++++++++++++++++++
>>  5 files changed, 97 insertions(+), 1 deletion(-)
>>  create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
>>  create mode 100644 scripts/nsdeps
>
>[]
>
>> +if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then
>> +    echo 'spatch needs to be version 1.06 or higher'
>
>Nitpick: 1.0.6
>

Good catch! Thanks!
Actually it needs to be even 1.0.4, so I will just use
$SPATCH_REQ_VERSION from above.

>> +    exit 1
>> +fi
>
>
>-- 
>Himanshu Jha
>Undergraduate Student
>Department of Electronics & Communication
>Guru Tegh Bahadur Institute of Technology

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [v2 08/10] scripts: Coccinelle script for namespace dependencies
  2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
                       ` (2 preceding siblings ...)
  2019-08-14  6:36     ` [Cocci] " Himanshu Jha
@ 2019-08-14 12:00     ` " Markus Elfring
  2019-08-15 13:50     ` Markus Elfring
  4 siblings, 0 replies; 51+ messages in thread
From: Markus Elfring @ 2019-08-14 12:00 UTC (permalink / raw)
  To: Matthias Maennich, Greg Kroah-Hartman, Julia Lawall,
	Martijn Coenen, Himanshu Jha, cocci, kernel-janitors
  Cc: linux-arch, linux-kbuild, ,linux-kernel, linux-m68k,
	linux-modules, linux-scsi, linux-usb, kernel-team, usb-storage,
	x86, Alan Stern, Arnd Bergmann, David S. Miller,
	Geert Uytterhoeven, Gilles Muller, H. Peter Anvin, Ingo Molnar,
	Jessica Yu, Joel Fernandes, Jonathan Cameron, Kate Stewart,
	Lucas De Marchi, Martijn Coenen, Masahiro Yamada,
	Mauro Carvalho Chehab, Michal Marek, Nicolas Ferre,
	Nicolas Palix, Oliver Neukum, Philippe Ombredanne, Sam Ravnborg,
	Sandeep Patil, Stephen Boyd, Thomas Gleixner

+# This script requires at least spatch
+# version 1.0.4.

How do you think about to avoid the duplicate specification of this identification?

Regards,
Markus

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
  2019-08-13 20:15     ` Saravana Kannan
@ 2019-08-14 12:54       ` Matthias Maennich
  2019-08-14 17:34         ` Saravana Kannan
  0 siblings, 1 reply; 51+ messages in thread
From: Matthias Maennich @ 2019-08-14 12:54 UTC (permalink / raw)
  To: Saravana Kannan
  Cc: LKML, maco, Android Kernel Team, arnd, geert, Greg Kroah-Hartman,
	hpa, jeyu, Joel Fernandes (Google),
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, michal.lkml, mingo,
	oneukum, pombredanne, sam, Sandeep Patil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton,
	Johannes Weiner, Peter Zijlstra (Intel),
	Mauro Carvalho Chehab, David Howells, Patrick Bellasi,
	Dan Williams, Adrian Reber, Richard Guy Briggs

On Tue, Aug 13, 2019 at 01:15:44PM -0700, Saravana Kannan wrote:
>On Tue, Aug 13, 2019 at 5:19 AM 'Matthias Maennich' via kernel-team
><kernel-team@android.com> wrote:
>>
>> If MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is enabled (default=n), the
>> requirement for modules to import all namespaces that are used by
>> the module is relaxed.
>>
>> Enabling this option effectively allows (invalid) modules to be loaded
>> while only a warning is emitted.
>>
>> Disabling this option keeps the enforcement at module loading time and
>> loading is denied if the module's imports are not satisfactory.
>>
>> Reviewed-by: Martijn Coenen <maco@android.com>
>> Signed-off-by: Matthias Maennich <maennich@google.com>
>> ---
>>  init/Kconfig    | 14 ++++++++++++++
>>  kernel/module.c | 11 +++++++++--
>>  2 files changed, 23 insertions(+), 2 deletions(-)
>>
>> diff --git a/init/Kconfig b/init/Kconfig
>> index bd7d650d4a99..b3373334cdf1 100644
>> --- a/init/Kconfig
>> +++ b/init/Kconfig
>> @@ -2119,6 +2119,20 @@ config MODULE_COMPRESS_XZ
>>
>>  endchoice
>>
>> +config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
>> +       bool "Allow loading of modules with missing namespace imports"
>> +       default n
>> +       help
>> +         Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
>> +         a namespace. A module that makes use of a symbol exported with such a
>> +         namespace is required to import the namespace via MODULE_IMPORT_NS().
>> +         This option relaxes this requirement when loading a module.
>
>> While
>> +         technically there is no reason to enforce correct namespace imports,
>> +         it creates consistency between symbols defining namespaces and users
>> +         importing namespaces they make use of.
>
>I'm confused by this sentence. It sounds like it's the opposite of
>what the config is doing? Can you please reword it for clarify?

How about:

  Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
  a namespace. A module that makes use of a symbol exported with such a
  namespace is required to import the namespace via MODULE_IMPORT_NS().
  There is no technical reason to enforce correct namespace imports,
  but it creates consistency between symbols defining namespaces and
  users importing namespaces they make use of. This option relaxes this
  requirement and lifts the enforcement when loading a module.

-- 
Cheers,
Matthias

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
  2019-08-14 12:54       ` Matthias Maennich
@ 2019-08-14 17:34         ` Saravana Kannan
  0 siblings, 0 replies; 51+ messages in thread
From: Saravana Kannan @ 2019-08-14 17:34 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: LKML, maco, Android Kernel Team, arnd, geert, Greg Kroah-Hartman,
	hpa, jeyu, Joel Fernandes (Google),
	Kate Stewart, linux-arch, linux-kbuild, linux-m68k,
	linux-modules, linux-scsi, linux-usb, lucas.de.marchi,
	Martijn Coenen, michal.lkml, mingo, oneukum, Philippe Ombredanne,
	sam, Sandeep Patil, stern, tglx, usb-storage, x86,
	yamada.masahiro, Andrew Morton, Johannes Weiner,
	Peter Zijlstra (Intel),
	Mauro Carvalho Chehab, David Howells, Patrick Bellasi,
	Dan Williams, Adrian Reber, Richard Guy Briggs

On Wed, Aug 14, 2019 at 5:54 AM 'Matthias Maennich' via kernel-team
<kernel-team@android.com> wrote:
>
> On Tue, Aug 13, 2019 at 01:15:44PM -0700, Saravana Kannan wrote:
> >On Tue, Aug 13, 2019 at 5:19 AM 'Matthias Maennich' via kernel-team
> ><kernel-team@android.com> wrote:
> >>
> >> If MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is enabled (default=n), the
> >> requirement for modules to import all namespaces that are used by
> >> the module is relaxed.
> >>
> >> Enabling this option effectively allows (invalid) modules to be loaded
> >> while only a warning is emitted.
> >>
> >> Disabling this option keeps the enforcement at module loading time and
> >> loading is denied if the module's imports are not satisfactory.
> >>
> >> Reviewed-by: Martijn Coenen <maco@android.com>
> >> Signed-off-by: Matthias Maennich <maennich@google.com>
> >> ---
> >>  init/Kconfig    | 14 ++++++++++++++
> >>  kernel/module.c | 11 +++++++++--
> >>  2 files changed, 23 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/init/Kconfig b/init/Kconfig
> >> index bd7d650d4a99..b3373334cdf1 100644
> >> --- a/init/Kconfig
> >> +++ b/init/Kconfig
> >> @@ -2119,6 +2119,20 @@ config MODULE_COMPRESS_XZ
> >>
> >>  endchoice
> >>
> >> +config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
> >> +       bool "Allow loading of modules with missing namespace imports"
> >> +       default n
> >> +       help
> >> +         Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
> >> +         a namespace. A module that makes use of a symbol exported with such a
> >> +         namespace is required to import the namespace via MODULE_IMPORT_NS().
> >> +         This option relaxes this requirement when loading a module.
> >
> >> While
> >> +         technically there is no reason to enforce correct namespace imports,
> >> +         it creates consistency between symbols defining namespaces and users
> >> +         importing namespaces they make use of.
> >
> >I'm confused by this sentence. It sounds like it's the opposite of
> >what the config is doing? Can you please reword it for clarify?
>
> How about:
>
>   Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
>   a namespace. A module that makes use of a symbol exported with such a
>   namespace is required to import the namespace via MODULE_IMPORT_NS().
>   There is no technical reason to enforce correct namespace imports,
>   but it creates consistency between symbols defining namespaces and
>   users importing namespaces they make use of. This option relaxes this
>   requirement and lifts the enforcement when loading a module.

That's a lot better. Especially moving the "This option relaxes..." to
the bottom. Thanks.

-Saravana

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [v2 08/10] scripts: Coccinelle script for namespace dependencies
  2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
                       ` (3 preceding siblings ...)
  2019-08-14 12:00     ` [v2 " Markus Elfring
@ 2019-08-15 13:50     ` Markus Elfring
  4 siblings, 0 replies; 51+ messages in thread
From: Markus Elfring @ 2019-08-15 13:50 UTC (permalink / raw)
  To: Matthias Maennich, Greg Kroah-Hartman, Julia Lawall,
	Martijn Coenen, cocci, kernel-janitors
  Cc: linux-arch, linux-kbuild, linux-kernel, linux-m68k,
	linux-modules, linux-scsi, linux-usb, kernel-team, usb-storage,
	x86, Alan Stern, Arnd Bergmann, David S. Miller,
	Geert Uytterhoeven, Gilles Muller, H. Peter Anvin, Ingo Molnar,
	Jessica Yu, Joel Fernandes, Jonathan Cameron, Kate Stewart,
	Lucas De Marchi, Martijn Coenen, Masahiro Yamada,
	Mauro Carvalho Chehab, Michal Marek, Nicolas Ferre,
	Nicolas Palix, Oliver Neukum, Philippe Ombredanne, Sam Ravnborg,
	Sandeep Patil, Stephen Boyd, Thomas Gleixner

> +generate_deps_for_ns() {
> +    $SPATCH --very-quiet --in-place --sp-file \
> +	    $srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2
> +}

* Where will the variable “srctree” be set for the file “scripts/nsdeps”?

* Would you like to support a separate build directory for desired adjustments?

* How do you think about to check error handling around such commands?


> +generate_deps() {
> +        for source_file in $mod_source_files; do
> +            sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp
…

I suggest to assign the name for the temporary file to a variable
which should be used by subsequent commands.

Regards,
Markus

^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 01/11] module: support reading multiple values per modinfo tag
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
                       ` (8 subsequent siblings)
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

Similar to modpost's get_next_modinfo(), introduce get_next_modinfo() in
kernel/module.c to acquire any further values associated with the same
modinfo tag name. That is useful for any tags that have multiple
occurrences (such as 'alias'), but is in particular introduced here as
part of the symbol namespaces patch series to read the (potentially)
multiple namespaces a module is importing.

Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Reviewed-by: Martijn Coenen <maco@android.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 kernel/module.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/kernel/module.c b/kernel/module.c
index 5933395af9a0..a23067907169 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -2481,7 +2481,8 @@ static char *next_string(char *string, unsigned long *secsize)
 	return string;
 }
 
-static char *get_modinfo(struct load_info *info, const char *tag)
+static char *get_next_modinfo(const struct load_info *info, const char *tag,
+			      char *prev)
 {
 	char *p;
 	unsigned int taglen = strlen(tag);
@@ -2492,13 +2493,25 @@ static char *get_modinfo(struct load_info *info, const char *tag)
 	 * get_modinfo() calls made before rewrite_section_headers()
 	 * must use sh_offset, as sh_addr isn't set!
 	 */
-	for (p = (char *)info->hdr + infosec->sh_offset; p; p = next_string(p, &size)) {
+	char *modinfo = (char *)info->hdr + infosec->sh_offset;
+
+	if (prev) {
+		size -= prev - modinfo;
+		modinfo = next_string(prev, &size);
+	}
+
+	for (p = modinfo; p; p = next_string(p, &size)) {
 		if (strncmp(p, tag, taglen) == 0 && p[taglen] == '=')
 			return p + taglen + 1;
 	}
 	return NULL;
 }
 
+static char *get_modinfo(const struct load_info *info, const char *tag)
+{
+	return get_next_modinfo(info, tag, NULL);
+}
+
 static void setup_modinfo(struct module *mod, struct load_info *info)
 {
 	struct module_attribute *attr;
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 02/11] export: explicitly align struct kernel_symbol
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
  2019-08-21 11:49     ` [PATCH v3 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 03/11] module: add support for symbol namespaces Matthias Maennich
                       ` (7 subsequent siblings)
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Ingo Molnar, Andrew Morton,
	Ard Biesheuvel

This change allows growing struct kernel_symbol without wasting bytes to
alignment. It also concretized the alignment of ksymtab entries if
relative references are used for ksymtab entries.

struct kernel_symbol was already implicitly being aligned to the word
size, except on x86_64 and m68k, where it is aligned to 16 and 2 bytes,
respectively.

As far as I can tell there is no requirement for aligning struct
kernel_symbol to 16 bytes on x86_64, but gcc aligns structs to their
size, and the linker aligns the custom __ksymtab sections to the largest
data type contained within, so setting KSYM_ALIGN to 16 was necessary to
stay consistent with the code generated for non-ASM EXPORT_SYMBOL(). Now
that non-ASM EXPORT_SYMBOL() explicitly aligns to word size (8),
KSYM_ALIGN is no longer necessary.

In case of relative references, the alignment has been changed
accordingly to not waste space when adding new struct members.

As for m68k, struct kernel_symbol is aligned to 2 bytes even though the
structure itself is 8 bytes; using a 4-byte alignment shouldn't hurt.

I manually verified the output of the __ksymtab sections didn't change
on x86, x86_64, arm, arm64 and m68k. As expected, the section contents
didn't change, and the ELF section alignment only changed on x86_64 and
m68k. Feedback from other archs more than welcome.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 arch/m68k/include/asm/export.h | 1 -
 include/asm-generic/export.h   | 8 +++-----
 include/linux/export.h         | 3 ++-
 3 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/arch/m68k/include/asm/export.h b/arch/m68k/include/asm/export.h
index 0af20f48bd07..b53008b67ce1 100644
--- a/arch/m68k/include/asm/export.h
+++ b/arch/m68k/include/asm/export.h
@@ -1,3 +1,2 @@
-#define KSYM_ALIGN 2
 #define KCRC_ALIGN 2
 #include <asm-generic/export.h>
diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 294d6ae785d4..63f54907317b 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -4,15 +4,13 @@
 #ifndef KSYM_FUNC
 #define KSYM_FUNC(x) x
 #endif
-#ifdef CONFIG_64BIT
-#ifndef KSYM_ALIGN
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+#define KSYM_ALIGN 4
+#elif defined(CONFIG_64BIT)
 #define KSYM_ALIGN 8
-#endif
 #else
-#ifndef KSYM_ALIGN
 #define KSYM_ALIGN 4
 #endif
-#endif
 #ifndef KCRC_ALIGN
 #define KCRC_ALIGN 4
 #endif
diff --git a/include/linux/export.h b/include/linux/export.h
index fd8711ed9ac4..28a4d2150689 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -52,7 +52,7 @@ extern struct module __this_module;
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	__ADDRESSABLE(sym)						\
 	asm("	.section \"___ksymtab" sec "+" #sym "\", \"a\"	\n"	\
-	    "	.balign	8					\n"	\
+	    "	.balign 4					\n"	\
 	    "__ksymtab_" #sym ":				\n"	\
 	    "	.long	" #sym "- .				\n"	\
 	    "	.long	__kstrtab_" #sym "- .			\n"	\
@@ -66,6 +66,7 @@ struct kernel_symbol {
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	static const struct kernel_symbol __ksymtab_##sym		\
 	__attribute__((section("___ksymtab" sec "+" #sym), used))	\
+	__aligned(sizeof(void *))					\
 	= { (unsigned long)&sym, __kstrtab_##sym }
 
 struct kernel_symbol {
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 03/11] module: add support for symbol namespaces.
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
  2019-08-21 11:49     ` [PATCH v3 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 04/11] modpost: " Matthias Maennich
                       ` (6 subsequent siblings)
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Michael Ellerman,
	Andrew Morton, Ard Biesheuvel

The EXPORT_SYMBOL_NS() and EXPORT_SYMBOL_NS_GPL() macros can be used to
export a symbol to a specific namespace.  There are no _GPL_FUTURE and
_UNUSED variants because these are currently unused, and I'm not sure
they are necessary.

I didn't add EXPORT_SYMBOL_NS() for ASM exports; this patch sets the
namespace of ASM exports to NULL by default. In case of relative
references, it will be relocatable to NULL. If there's a need, this
should be pretty easy to add.

A module that wants to use a symbol exported to a namespace must add a
MODULE_IMPORT_NS() statement to their module code; otherwise, modpost
will complain when building the module, and the kernel module loader
will emit an error and fail when loading the module.

MODULE_IMPORT_NS() adds a modinfo tag 'import_ns' to the module. That
tag can be observed by the modinfo command, modpost and kernel/module.c
at the time of loading the module.

The ELF symbols are renamed to include the namespace with an asm label;
for example, symbol 'usb_stor_suspend' in namespace USB_STORAGE becomes
'usb_stor_suspend.USB_STORAGE'.  This allows modpost to do namespace
checking, without having to go through all the effort of parsing ELF and
relocation records just to get to the struct kernel_symbols.

On x86_64 I saw no difference in binary size (compression), but at
runtime this will require a word of memory per export to hold the
namespace. An alternative could be to store namespaced symbols in their
own section and use a separate 'struct namespaced_kernel_symbol' for
that section, at the cost of making the module loader more complex.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 include/asm-generic/export.h |  6 +--
 include/linux/export.h       | 85 ++++++++++++++++++++++++++++++------
 include/linux/module.h       |  2 +
 kernel/module.c              | 43 ++++++++++++++++++
 4 files changed, 120 insertions(+), 16 deletions(-)

diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 63f54907317b..e2b5d0f569d3 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -17,11 +17,11 @@
 
 .macro __put, val, name
 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
-	.long	\val - ., \name - .
+	.long	\val - ., \name - ., 0 - .
 #elif defined(CONFIG_64BIT)
-	.quad	\val, \name
+	.quad	\val, \name, 0
 #else
-	.long	\val, \name
+	.long	\val, \name, 0
 #endif
 .endm
 
diff --git a/include/linux/export.h b/include/linux/export.h
index 28a4d2150689..8e12e05444d1 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -20,6 +20,8 @@ extern struct module __this_module;
 
 #ifdef CONFIG_MODULES
 
+#define NS_SEPARATOR "."
+
 #if defined(__KERNEL__) && !defined(__GENKSYMS__)
 #ifdef CONFIG_MODVERSIONS
 /* Mark the CRC weak since genksyms apparently decides not to
@@ -49,6 +51,16 @@ extern struct module __this_module;
  * absolute relocations that require runtime processing on relocatable
  * kernels.
  */
+#define __KSYMTAB_ENTRY_NS(sym, sec, ns)				\
+	__ADDRESSABLE(sym)						\
+	asm("	.section \"___ksymtab" sec "+" #sym "\", \"a\"	\n"	\
+	    "	.balign	4					\n"	\
+	    "__ksymtab_" #sym NS_SEPARATOR #ns ":		\n"	\
+	    "	.long	" #sym "- .				\n"	\
+	    "	.long	__kstrtab_" #sym "- .			\n"	\
+	    "	.long	__kstrtab_ns_" #sym "- .		\n"	\
+	    "	.previous					\n")
+
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	__ADDRESSABLE(sym)						\
 	asm("	.section \"___ksymtab" sec "+" #sym "\", \"a\"	\n"	\
@@ -56,32 +68,53 @@ extern struct module __this_module;
 	    "__ksymtab_" #sym ":				\n"	\
 	    "	.long	" #sym "- .				\n"	\
 	    "	.long	__kstrtab_" #sym "- .			\n"	\
+	    "	.long	0 - .					\n"	\
 	    "	.previous					\n")
 
 struct kernel_symbol {
 	int value_offset;
 	int name_offset;
+	int namespace_offset;
 };
 #else
+#define __KSYMTAB_ENTRY_NS(sym, sec, ns)				\
+	static const struct kernel_symbol __ksymtab_##sym##__##ns	\
+	asm("__ksymtab_" #sym NS_SEPARATOR #ns)				\
+	__attribute__((section("___ksymtab" sec "+" #sym), used))	\
+	__aligned(sizeof(void *))					\
+	= { (unsigned long)&sym, __kstrtab_##sym, __kstrtab_ns_##sym}
+
 #define __KSYMTAB_ENTRY(sym, sec)					\
 	static const struct kernel_symbol __ksymtab_##sym		\
+	asm("__ksymtab_" #sym)						\
 	__attribute__((section("___ksymtab" sec "+" #sym), used))	\
 	__aligned(sizeof(void *))					\
-	= { (unsigned long)&sym, __kstrtab_##sym }
+	= { (unsigned long)&sym, __kstrtab_##sym, NULL }
 
 struct kernel_symbol {
 	unsigned long value;
 	const char *name;
+	const char *namespace;
 };
 #endif
 
-/* For every exported symbol, place a struct in the __ksymtab section */
-#define ___EXPORT_SYMBOL(sym, sec)					\
+#define ___export_symbol_common(sym, sec)				\
 	extern typeof(sym) sym;						\
 	__CRC_SYMBOL(sym, sec)						\
 	static const char __kstrtab_##sym[]				\
 	__attribute__((section("__ksymtab_strings"), used, aligned(1)))	\
-	= #sym;								\
+	= #sym								\
+
+/* For every exported symbol, place a struct in the __ksymtab section */
+#define ___EXPORT_SYMBOL_NS(sym, sec, ns)				\
+	___export_symbol_common(sym, sec);			\
+	static const char __kstrtab_ns_##sym[]				\
+	__attribute__((section("__ksymtab_strings"), used, aligned(1)))	\
+	= #ns;								\
+	__KSYMTAB_ENTRY_NS(sym, sec, ns)
+
+#define ___EXPORT_SYMBOL(sym, sec)					\
+	___export_symbol_common(sym, sec);				\
 	__KSYMTAB_ENTRY(sym, sec)
 
 #if defined(__DISABLE_EXPORTS)
@@ -91,6 +124,7 @@ struct kernel_symbol {
  * be reused in other execution contexts such as the UEFI stub or the
  * decompressor.
  */
+#define __EXPORT_SYMBOL_NS(sym, sec, ns)
 #define __EXPORT_SYMBOL(sym, sec)
 
 #elif defined(CONFIG_TRIM_UNUSED_KSYMS)
@@ -117,18 +151,26 @@ struct kernel_symbol {
 #define __cond_export_sym_1(sym, sec) ___EXPORT_SYMBOL(sym, sec)
 #define __cond_export_sym_0(sym, sec) /* nothing */
 
+#define __EXPORT_SYMBOL_NS(sym, sec, ns)				\
+	__ksym_marker(sym);						\
+	__cond_export_ns_sym(sym, sec, ns, __is_defined(__KSYM_##sym))
+#define __cond_export_ns_sym(sym, sec, ns, conf)			\
+	___cond_export_ns_sym(sym, sec, ns, conf)
+#define ___cond_export_ns_sym(sym, sec, ns, enabled)			\
+	__cond_export_ns_sym_##enabled(sym, sec, ns)
+#define __cond_export_ns_sym_1(sym, sec, ns) ___EXPORT_SYMBOL_NS(sym, sec, ns)
+#define __cond_export_ns_sym_0(sym, sec, ns) /* nothing */
+
 #else
+#define __EXPORT_SYMBOL_NS ___EXPORT_SYMBOL_NS
 #define __EXPORT_SYMBOL ___EXPORT_SYMBOL
 #endif
 
-#define EXPORT_SYMBOL(sym)					\
-	__EXPORT_SYMBOL(sym, "")
-
-#define EXPORT_SYMBOL_GPL(sym)					\
-	__EXPORT_SYMBOL(sym, "_gpl")
-
-#define EXPORT_SYMBOL_GPL_FUTURE(sym)				\
-	__EXPORT_SYMBOL(sym, "_gpl_future")
+#define EXPORT_SYMBOL(sym) __EXPORT_SYMBOL(sym, "")
+#define EXPORT_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_gpl")
+#define EXPORT_SYMBOL_GPL_FUTURE(sym) __EXPORT_SYMBOL(sym, "_gpl_future")
+#define EXPORT_SYMBOL_NS(sym, ns) __EXPORT_SYMBOL_NS(sym, "", ns)
+#define EXPORT_SYMBOL_NS_GPL(sym, ns) __EXPORT_SYMBOL_NS(sym, "_gpl", ns)
 
 #ifdef CONFIG_UNUSED_SYMBOLS
 #define EXPORT_UNUSED_SYMBOL(sym) __EXPORT_SYMBOL(sym, "_unused")
@@ -138,11 +180,28 @@ struct kernel_symbol {
 #define EXPORT_UNUSED_SYMBOL_GPL(sym)
 #endif
 
-#endif	/* __GENKSYMS__ */
+#endif	/* __KERNEL__ && !__GENKSYMS__ */
+
+#if defined(__GENKSYMS__)
+/*
+ * When we're running genksyms, ignore the namespace and make the _NS
+ * variants look like the normal ones. There are two reasons for this:
+ * 1) In the normal definition of EXPORT_SYMBOL_NS, the 'ns' macro
+ *    argument is itself not expanded because it's always tokenized or
+ *    concatenated; but when running genksyms, a blank definition of the
+ *    macro does allow the argument to be expanded; if a namespace
+ *    happens to collide with a #define, this can cause issues.
+ * 2) There's no need to modify genksyms to deal with the _NS variants
+ */
+#define EXPORT_SYMBOL_NS(sym, ns) EXPORT_SYMBOL(sym)
+#define EXPORT_SYMBOL_NS_GPL(sym, ns) EXPORT_SYMBOL_GPL(sym)
+#endif
 
 #else /* !CONFIG_MODULES... */
 
 #define EXPORT_SYMBOL(sym)
+#define EXPORT_SYMBOL_NS(sym, ns)
+#define EXPORT_SYMBOL_NS_GPL(sym, ns)
 #define EXPORT_SYMBOL_GPL(sym)
 #define EXPORT_SYMBOL_GPL_FUTURE(sym)
 #define EXPORT_UNUSED_SYMBOL(sym)
diff --git a/include/linux/module.h b/include/linux/module.h
index 1455812dd325..b3611e749f72 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -280,6 +280,8 @@ struct notifier_block;
 
 #ifdef CONFIG_MODULES
 
+#define MODULE_IMPORT_NS(ns) MODULE_INFO(import_ns, #ns)
+
 extern int modules_disabled; /* for sysctl */
 /* Get/put a kernel symbol (calls must be symmetric) */
 void *__symbol_get(const char *symbol);
diff --git a/kernel/module.c b/kernel/module.c
index a23067907169..57e8253f2251 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -544,6 +544,15 @@ static const char *kernel_symbol_name(const struct kernel_symbol *sym)
 #endif
 }
 
+static const char *kernel_symbol_namespace(const struct kernel_symbol *sym)
+{
+#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
+	return offset_to_ptr(&sym->namespace_offset);
+#else
+	return sym->namespace;
+#endif
+}
+
 static int cmp_name(const void *va, const void *vb)
 {
 	const char *a;
@@ -1379,6 +1388,34 @@ static inline int same_magic(const char *amagic, const char *bmagic,
 }
 #endif /* CONFIG_MODVERSIONS */
 
+static char *get_modinfo(const struct load_info *info, const char *tag);
+static char *get_next_modinfo(const struct load_info *info, const char *tag,
+			      char *prev);
+
+static int verify_namespace_is_imported(const struct load_info *info,
+					const struct kernel_symbol *sym,
+					struct module *mod)
+{
+	const char *namespace;
+	char *imported_namespace;
+
+	namespace = kernel_symbol_namespace(sym);
+	if (namespace) {
+		imported_namespace = get_modinfo(info, "import_ns");
+		while (imported_namespace) {
+			if (strcmp(namespace, imported_namespace) == 0)
+				return 0;
+			imported_namespace = get_next_modinfo(
+				info, "import_ns", imported_namespace);
+		}
+		pr_err("%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
+		       mod->name, kernel_symbol_name(sym), namespace);
+		return -EINVAL;
+	}
+	return 0;
+}
+
+
 /* Resolve a symbol for this module.  I.e. if we find one, record usage. */
 static const struct kernel_symbol *resolve_symbol(struct module *mod,
 						  const struct load_info *info,
@@ -1413,6 +1450,12 @@ static const struct kernel_symbol *resolve_symbol(struct module *mod,
 		goto getname;
 	}
 
+	err = verify_namespace_is_imported(info, sym, mod);
+	if (err) {
+		sym = ERR_PTR(err);
+		goto getname;
+	}
+
 getname:
 	/* We must make copy under the lock if we failed to get ref. */
 	strncpy(ownername, module_name(owner), MODULE_NAME_LEN);
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 04/11] modpost: add support for symbol namespaces
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
                       ` (2 preceding siblings ...)
  2019-08-21 11:49     ` [PATCH v3 03/11] module: add support for symbol namespaces Matthias Maennich
@ 2019-08-21 11:49     ` " Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
                       ` (5 subsequent siblings)
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

Add support for symbols that are exported into namespaces. For that,
extract any namespace suffix from the symbol name. In addition, emit a
warning whenever a module refers to an exported symbol without
explicitly importing the namespace that it is defined in. This patch
consistently adds the namespace suffix to symbol names exported into
Module.symvers.

Example warning emitted by modpost in case of the above violation:

 WARNING: module ums-usbat uses symbol usb_stor_resume from namespace
 USB_STORAGE, but does not import it.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 scripts/mod/modpost.c | 91 +++++++++++++++++++++++++++++++++++++------
 scripts/mod/modpost.h |  7 ++++
 2 files changed, 87 insertions(+), 11 deletions(-)

diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index f277e116e0eb..538bb24ffee3 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -164,6 +164,7 @@ struct symbol {
 	struct module *module;
 	unsigned int crc;
 	int crc_valid;
+	const char *namespace;
 	unsigned int weak:1;
 	unsigned int vmlinux:1;    /* 1 if symbol is defined in vmlinux */
 	unsigned int kernel:1;     /* 1 if symbol is from kernel
@@ -233,6 +234,37 @@ static struct symbol *find_symbol(const char *name)
 	return NULL;
 }
 
+static bool contains_namespace(struct namespace_list *list,
+			       const char *namespace)
+{
+	struct namespace_list *ns_entry;
+
+	for (ns_entry = list; ns_entry != NULL; ns_entry = ns_entry->next)
+		if (strcmp(ns_entry->namespace, namespace) == 0)
+			return true;
+
+	return false;
+}
+
+static void add_namespace(struct namespace_list **list, const char *namespace)
+{
+	struct namespace_list *ns_entry;
+
+	if (!contains_namespace(*list, namespace)) {
+		ns_entry = NOFAIL(malloc(sizeof(struct namespace_list) +
+					 strlen(namespace) + 1));
+		strcpy(ns_entry->namespace, namespace);
+		ns_entry->next = *list;
+		*list = ns_entry;
+	}
+}
+
+static bool module_imports_namespace(struct module *module,
+				     const char *namespace)
+{
+	return contains_namespace(module->imported_namespaces, namespace);
+}
+
 static const struct {
 	const char *str;
 	enum export export;
@@ -312,6 +344,22 @@ static enum export export_from_sec(struct elf_info *elf, unsigned int sec)
 		return export_unknown;
 }
 
+static const char *sym_extract_namespace(const char **symname)
+{
+	size_t n;
+	char *dupsymname;
+
+	n = strcspn(*symname, ".");
+	if (n < strlen(*symname) - 1) {
+		dupsymname = NOFAIL(strdup(*symname));
+		dupsymname[n] = '\0';
+		*symname = dupsymname;
+		return dupsymname + n + 1;
+	}
+
+	return NULL;
+}
+
 /**
  * Add an exported symbol - it may have already been added without a
  * CRC, in this case just update the CRC
@@ -319,16 +367,18 @@ static enum export export_from_sec(struct elf_info *elf, unsigned int sec)
 static struct symbol *sym_add_exported(const char *name, struct module *mod,
 				       enum export export)
 {
-	struct symbol *s = find_symbol(name);
+	const char *symbol_name = name;
+	const char *namespace = sym_extract_namespace(&symbol_name);
+	struct symbol *s = find_symbol(symbol_name);
 
 	if (!s) {
-		s = new_symbol(name, mod, export);
+		s = new_symbol(symbol_name, mod, export);
+		s->namespace = namespace;
 	} else {
 		if (!s->preloaded) {
-			warn("%s: '%s' exported twice. Previous export "
-			     "was in %s%s\n", mod->name, name,
-			     s->module->name,
-			     is_vmlinux(s->module->name) ?"":".ko");
+			warn("%s: '%s' exported twice. Previous export was in %s%s\n",
+			     mod->name, symbol_name, s->module->name,
+			     is_vmlinux(s->module->name) ? "" : ".ko");
 		} else {
 			/* In case Module.symvers was out of date */
 			s->module = mod;
@@ -1943,6 +1993,7 @@ static void read_symbols(const char *modname)
 	const char *symname;
 	char *version;
 	char *license;
+	char *namespace;
 	struct module *mod;
 	struct elf_info info = { };
 	Elf_Sym *sym;
@@ -1974,6 +2025,12 @@ static void read_symbols(const char *modname)
 		license = get_next_modinfo(&info, "license", license);
 	}
 
+	namespace = get_modinfo(&info, "import_ns");
+	while (namespace) {
+		add_namespace(&mod->imported_namespaces, namespace);
+		namespace = get_next_modinfo(&info, "import_ns", namespace);
+	}
+
 	for (sym = info.symtab_start; sym < info.symtab_stop; sym++) {
 		symname = remove_dot(info.strtab + sym->st_name);
 
@@ -2118,6 +2175,13 @@ static int check_exports(struct module *mod)
 			basename++;
 		else
 			basename = mod->name;
+
+		if (exp->namespace &&
+		    !module_imports_namespace(mod, exp->namespace)) {
+			warn("module %s uses symbol %s from namespace %s, but does not import it.\n",
+			     basename, exp->name, exp->namespace);
+		}
+
 		if (!mod->gpl_compatible)
 			check_for_gpl_usage(exp->export, basename, exp->name);
 		check_for_unused(exp->export, basename, exp->name);
@@ -2395,16 +2459,21 @@ static void write_dump(const char *fname)
 {
 	struct buffer buf = { };
 	struct symbol *symbol;
+	const char *namespace;
 	int n;
 
 	for (n = 0; n < SYMBOL_HASH_SIZE ; n++) {
 		symbol = symbolhash[n];
 		while (symbol) {
-			if (dump_sym(symbol))
-				buf_printf(&buf, "0x%08x\t%s\t%s\t%s\n",
-					symbol->crc, symbol->name,
-					symbol->module->name,
-					export_str(symbol->export));
+			if (dump_sym(symbol)) {
+				namespace = symbol->namespace;
+				buf_printf(&buf, "0x%08x\t%s%s%s\t%s\t%s\n",
+					   symbol->crc, symbol->name,
+					   namespace ? "." : "",
+					   namespace ? namespace : "",
+					   symbol->module->name,
+					   export_str(symbol->export));
+			}
 			symbol = symbol->next;
 		}
 	}
diff --git a/scripts/mod/modpost.h b/scripts/mod/modpost.h
index 8453d6ac2f77..9626bf3e7424 100644
--- a/scripts/mod/modpost.h
+++ b/scripts/mod/modpost.h
@@ -109,6 +109,11 @@ buf_printf(struct buffer *buf, const char *fmt, ...);
 void
 buf_write(struct buffer *buf, const char *s, int len);
 
+struct namespace_list {
+	struct namespace_list *next;
+	char namespace[0];
+};
+
 struct module {
 	struct module *next;
 	const char *name;
@@ -121,6 +126,8 @@ struct module {
 	struct buffer dev_table_buf;
 	char	     srcversion[25];
 	int is_dot_o;
+	// Actual imported namespaces
+	struct namespace_list *imported_namespaces;
 };
 
 struct elf_info {
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
                       ` (3 preceding siblings ...)
  2019-08-21 11:49     ` [PATCH v3 04/11] modpost: " Matthias Maennich
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
                       ` (4 subsequent siblings)
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Andrew Morton,
	Johannes Weiner, Mauro Carvalho Chehab, David Howells,
	Patrick Bellasi, Dan Williams, Adrian Reber, Richard Guy Briggs

If MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is enabled (default=n), the
requirement for modules to import all namespaces that are used by
the module is relaxed.

Enabling this option effectively allows (invalid) modules to be loaded
while only a warning is emitted.

Disabling this option keeps the enforcement at module loading time and
loading is denied if the module's imports are not satisfactory.

Reviewed-by: Martijn Coenen <maco@android.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 init/Kconfig    | 13 +++++++++++++
 kernel/module.c | 11 +++++++++--
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index bd7d650d4a99..cc28561288a7 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2119,6 +2119,19 @@ config MODULE_COMPRESS_XZ
 
 endchoice
 
+config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
+	bool "Allow loading of modules with missing namespace imports"
+	help
+	  Symbols exported with EXPORT_SYMBOL_NS*() are considered exported in
+	  a namespace. A module that makes use of a symbol exported with such a
+	  namespace is required to import the namespace via MODULE_IMPORT_NS().
+	  There is no technical reason to enforce correct namespace imports,
+	  but it creates consistency between symbols defining namespaces and
+	  users importing namespaces they make use of. This option relaxes this
+	  requirement and lifts the enforcement when loading a module.
+
+	  If unsure, say N.
+
 config TRIM_UNUSED_KSYMS
 	bool "Trim unused exported kernel symbols"
 	depends on MODULES && !UNUSED_SYMBOLS
diff --git a/kernel/module.c b/kernel/module.c
index 57e8253f2251..7c934aaae2d3 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -1408,9 +1408,16 @@ static int verify_namespace_is_imported(const struct load_info *info,
 			imported_namespace = get_next_modinfo(
 				info, "import_ns", imported_namespace);
 		}
-		pr_err("%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
-		       mod->name, kernel_symbol_name(sym), namespace);
+#ifdef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
+		pr_warn(
+#else
+		pr_err(
+#endif
+			"%s: module uses symbol (%s) from namespace %s, but does not import it.\n",
+			mod->name, kernel_symbol_name(sym), namespace);
+#ifndef CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
 		return -EINVAL;
+#endif
 	}
 	return 0;
 }
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
                       ` (4 preceding siblings ...)
  2019-08-21 11:49     ` [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
                       ` (3 subsequent siblings)
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Michael Ellerman, Ingo Molnar,
	Ard Biesheuvel

To avoid excessive usage of EXPORT_SYMBOL_NS(sym, MY_NAMESPACE), where
MY_NAMESPACE will always be the namespace we are exporting to, allow
exporting all definitions of EXPORT_SYMBOL() and friends by defining
DEFAULT_SYMBOL_NAMESPACE.

For example, to export all symbols defined in usb-common into the
namespace USB_COMMON, add a line like this to drivers/usb/common/Makefile:

  ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=USB_COMMON

That is equivalent to changing all EXPORT_SYMBOL(sym) definitions to
EXPORT_SYMBOL_NS(sym, USB_COMMON). Subsequently all symbol namespaces
functionality will apply.

Another way of making use of this feature is to define the namespace
within source or header files similar to how TRACE_SYSTEM defines are
used:
  #undef DEFAULT_SYMBOL_NAMESPACE
  #define DEFAULT_SYMBOL_NAMESPACE USB_COMMON

Please note that, as opposed to TRACE_SYSTEM, DEFAULT_SYMBOL_NAMESPACE
has to be defined before including include/linux/export.h.

If DEFAULT_SYMBOL_NAMESPACE is defined, a symbol can still be exported
to another namespace by using EXPORT_SYMBOL_NS() and friends with
explicitly specifying the namespace.

Suggested-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Martijn Coenen <maco@android.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 include/linux/export.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/export.h b/include/linux/export.h
index 8e12e05444d1..1fb243abdbc4 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -166,6 +166,12 @@ struct kernel_symbol {
 #define __EXPORT_SYMBOL ___EXPORT_SYMBOL
 #endif
 
+#ifdef DEFAULT_SYMBOL_NAMESPACE
+#undef __EXPORT_SYMBOL
+#define __EXPORT_SYMBOL(sym, sec)				\
+	__EXPORT_SYMBOL_NS(sym, sec, DEFAULT_SYMBOL_NAMESPACE)
+#endif
+
 #define EXPORT_SYMBOL(sym) __EXPORT_SYMBOL(sym, "")
 #define EXPORT_SYMBOL_GPL(sym) __EXPORT_SYMBOL(sym, "_gpl")
 #define EXPORT_SYMBOL_GPL_FUTURE(sym) __EXPORT_SYMBOL(sym, "_gpl_future")
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 07/11] modpost: add support for generating namespace dependencies
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
                       ` (5 preceding siblings ...)
  2019-08-21 11:49     ` [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 08/11] scripts: Coccinelle script for " Matthias Maennich
                       ` (2 subsequent siblings)
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Jani Nikula, Rob Herring,
	Toru Komatsu, Alexey Gladkov

This patch adds an option to modpost to generate a <module>.ns_deps file
per module, containing the namespace dependencies for that module.

E.g. if the linked module my-module.ko would depend on the symbol
myfunc.MY_NS in the namespace MY_NS, the my-module.ns_deps file created
by modpost would contain the entry MY_NS to express the namespace
dependency of my-module imposed by using the symbol myfunc.

These files can subsequently be used by static analysis tools (like
coccinelle scripts) to address issues with missing namespace imports. A
later patch of this series will introduce such a script 'nsdeps' and a
corresponding make target to automatically add missing
MODULE_IMPORT_NS() definitions to the module's sources. For that it uses
the information provided in the generated .ns_deps files.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 .gitignore            |  1 +
 Makefile              |  2 +-
 scripts/mod/modpost.c | 54 +++++++++++++++++++++++++++++++++++++++----
 scripts/mod/modpost.h |  2 ++
 4 files changed, 53 insertions(+), 6 deletions(-)

diff --git a/.gitignore b/.gitignore
index 2030c7a4d2f8..9ee63aa2a3fb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -32,6 +32,7 @@
 *.lzo
 *.mod
 *.mod.c
+*.ns_deps
 *.o
 *.o.*
 *.order
diff --git a/Makefile b/Makefile
index 9fa18613566f..a89870188c09 100644
--- a/Makefile
+++ b/Makefile
@@ -1669,7 +1669,7 @@ clean: $(clean-dirs)
 		-o -name '*.ko.*' \
 		-o -name '*.dtb' -o -name '*.dtb.S' -o -name '*.dt.yaml' \
 		-o -name '*.dwo' -o -name '*.lst' \
-		-o -name '*.su' -o -name '*.mod' \
+		-o -name '*.su' -o -name '*.mod' -o -name '*.ns_deps' \
 		-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
 		-o -name '*.lex.c' -o -name '*.tab.[ch]' \
 		-o -name '*.asn1.[ch]' \
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 538bb24ffee3..81eeec063709 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -38,6 +38,8 @@ static int sec_mismatch_count = 0;
 static int sec_mismatch_fatal = 0;
 /* ignore missing files */
 static int ignore_missing_files;
+/* write namespace dependencies */
+static int write_namespace_deps;
 
 enum export {
 	export_plain,      export_unused,     export_gpl,
@@ -2176,10 +2178,15 @@ static int check_exports(struct module *mod)
 		else
 			basename = mod->name;
 
-		if (exp->namespace &&
-		    !module_imports_namespace(mod, exp->namespace)) {
-			warn("module %s uses symbol %s from namespace %s, but does not import it.\n",
-			     basename, exp->name, exp->namespace);
+		if (exp->namespace) {
+			add_namespace(&mod->required_namespaces,
+				      exp->namespace);
+
+			if (!write_namespace_deps &&
+			    !module_imports_namespace(mod, exp->namespace)) {
+				warn("module %s uses symbol %s from namespace %s, but does not import it.\n",
+				     basename, exp->name, exp->namespace);
+			}
 		}
 
 		if (!mod->gpl_compatible)
@@ -2481,6 +2488,31 @@ static void write_dump(const char *fname)
 	free(buf.p);
 }
 
+static void write_namespace_deps_files(void)
+{
+	struct module *mod;
+	struct namespace_list *ns;
+	struct buffer ns_deps_buf = {};
+
+	for (mod = modules; mod; mod = mod->next) {
+		char fname[PATH_MAX];
+
+		if (mod->skip)
+			continue;
+
+		ns_deps_buf.pos = 0;
+
+		for (ns = mod->required_namespaces; ns; ns = ns->next)
+			buf_printf(&ns_deps_buf, "%s\n", ns->namespace);
+
+		if (ns_deps_buf.pos == 0)
+			continue;
+
+		sprintf(fname, "%s.ns_deps", mod->name);
+		write_if_changed(&ns_deps_buf, fname);
+	}
+}
+
 struct ext_sym_list {
 	struct ext_sym_list *next;
 	const char *file;
@@ -2497,7 +2529,7 @@ int main(int argc, char **argv)
 	struct ext_sym_list *extsym_iter;
 	struct ext_sym_list *extsym_start = NULL;
 
-	while ((opt = getopt(argc, argv, "i:I:e:mnsT:o:awE")) != -1) {
+	while ((opt = getopt(argc, argv, "i:I:e:mnsT:o:awEd")) != -1) {
 		switch (opt) {
 		case 'i':
 			kernel_read = optarg;
@@ -2538,6 +2570,9 @@ int main(int argc, char **argv)
 		case 'E':
 			sec_mismatch_fatal = 1;
 			break;
+		case 'd':
+			write_namespace_deps = 1;
+			break;
 		default:
 			exit(1);
 		}
@@ -2572,6 +2607,9 @@ int main(int argc, char **argv)
 
 		err |= check_modname_len(mod);
 		err |= check_exports(mod);
+		if (write_namespace_deps)
+			continue;
+
 		add_header(&buf, mod);
 		add_intree_flag(&buf, !external_module);
 		add_retpoline(&buf);
@@ -2584,6 +2622,12 @@ int main(int argc, char **argv)
 		sprintf(fname, "%s.mod.c", mod->name);
 		write_if_changed(&buf, fname);
 	}
+
+	if (write_namespace_deps) {
+		write_namespace_deps_files();
+		return 0;
+	}
+
 	if (dump_write)
 		write_dump(dump_write);
 	if (sec_mismatch_count && sec_mismatch_fatal)
diff --git a/scripts/mod/modpost.h b/scripts/mod/modpost.h
index 9626bf3e7424..92a926d375d2 100644
--- a/scripts/mod/modpost.h
+++ b/scripts/mod/modpost.h
@@ -126,6 +126,8 @@ struct module {
 	struct buffer dev_table_buf;
 	char	     srcversion[25];
 	int is_dot_o;
+	// Required namespace dependencies
+	struct namespace_list *required_namespaces;
 	// Actual imported namespaces
 	struct namespace_list *imported_namespaces;
 };
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 08/11] scripts: Coccinelle script for namespace dependencies.
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
                       ` (6 preceding siblings ...)
  2019-08-21 11:49     ` [PATCH v3 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
@ 2019-08-21 11:49     ` " Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
  2019-08-21 11:49     ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
  9 siblings, 0 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro, Julia Lawall, Julia Lawall,
	Gilles Muller, Nicolas Palix, Mauro Carvalho Chehab,
	David S. Miller, Nicolas Ferre, cocci

A script that uses the '<module>.ns_deps' files generated by modpost to
automatically add the required symbol namespace dependencies to each
module.

Usage:
1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
   DEFAULT_SYMBOL_NAMESPACE
2) Run 'make' (or 'make modules') and get warnings about modules not
   importing that namespace.
3) Run 'make nsdeps' to automatically add required import statements
   to said modules.

This makes it easer for subsystem maintainers to introduce and maintain
symbol namespaces into their codebase.

Co-developed-by: Martijn Coenen <maco@android.com>
Signed-off-by: Martijn Coenen <maco@android.com>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Matthias Maennich <maennich@google.com>
---
 MAINTAINERS                                 |  5 ++
 Makefile                                    | 12 +++++
 scripts/Makefile.modpost                    |  4 +-
 scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++
 scripts/nsdeps                              | 56 +++++++++++++++++++++
 5 files changed, 99 insertions(+), 1 deletion(-)
 create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
 create mode 100644 scripts/nsdeps

diff --git a/MAINTAINERS b/MAINTAINERS
index 08176d64eed5..dd5b37b49a07 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11428,6 +11428,11 @@ S:	Maintained
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
 F:	tools/include/nolibc/
 
+NSDEPS
+M:	Matthias Maennich <maennich@google.com>
+S:	Maintained
+F:	scripts/nsdeps
+
 NTB AMD DRIVER
 M:	Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
 L:	linux-ntb@googlegroups.com
diff --git a/Makefile b/Makefile
index a89870188c09..40311f583ee1 100644
--- a/Makefile
+++ b/Makefile
@@ -1500,6 +1500,9 @@ help:
 	@echo  '  headerdep       - Detect inclusion cycles in headers'
 	@echo  '  coccicheck      - Check with Coccinelle'
 	@echo  ''
+	@echo  'Tools:'
+	@echo  '  nsdeps          - Generate missing symbol namespace dependencies'
+	@echo  ''
 	@echo  'Kernel selftest:'
 	@echo  '  kselftest       - Build and run kernel selftest (run as root)'
 	@echo  '                    Build, install, and boot kernel before'
@@ -1687,6 +1690,15 @@ quiet_cmd_tags = GEN     $@
 tags TAGS cscope gtags: FORCE
 	$(call cmd,tags)
 
+# Script to generate missing namespace dependencies
+# ---------------------------------------------------------------------------
+
+PHONY += nsdeps
+
+nsdeps:
+	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost nsdeps
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/$@
+
 # Scripts to check various things for consistency
 # ---------------------------------------------------------------------------
 
diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
index 26e6574ecd08..743fe3a2e885 100644
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -56,7 +56,8 @@ MODPOST = scripts/mod/modpost						\
 	$(if $(KBUILD_EXTMOD),$(addprefix -e ,$(KBUILD_EXTRA_SYMBOLS)))	\
 	$(if $(KBUILD_EXTMOD),-o $(modulesymfile))			\
 	$(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E)			\
-	$(if $(KBUILD_MODPOST_WARN),-w)
+	$(if $(KBUILD_MODPOST_WARN),-w)					\
+	$(if $(filter nsdeps,$(MAKECMDGOALS)),-d)
 
 ifdef MODPOST_VMLINUX
 
@@ -134,6 +135,7 @@ $(modules): %.ko :%.o %.mod.o FORCE
 
 targets += $(modules)
 
+nsdeps: __modpost
 
 # Add FORCE to the prequisites of a target to force it to be always rebuilt.
 # ---------------------------------------------------------------------------
diff --git a/scripts/coccinelle/misc/add_namespace.cocci b/scripts/coccinelle/misc/add_namespace.cocci
new file mode 100644
index 000000000000..c832bb6445a8
--- /dev/null
+++ b/scripts/coccinelle/misc/add_namespace.cocci
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0-only
+//
+/// Adds missing MODULE_IMPORT_NS statements to source files
+///
+/// This script is usually called from scripts/nsdeps with -D ns=<namespace> to
+/// add a missing namespace tag to a module source file.
+///
+
+@has_ns_import@
+declarer name MODULE_IMPORT_NS;
+identifier virtual.ns;
+@@
+MODULE_IMPORT_NS(ns);
+
+// Add missing imports, but only adjacent to a MODULE_LICENSE statement.
+// That ensures we are adding it only to the main module source file.
+@do_import depends on !has_ns_import@
+declarer name MODULE_LICENSE;
+expression license;
+identifier virtual.ns;
+@@
+MODULE_LICENSE(license);
++ MODULE_IMPORT_NS(ns);
diff --git a/scripts/nsdeps b/scripts/nsdeps
new file mode 100644
index 000000000000..3b5995a61e65
--- /dev/null
+++ b/scripts/nsdeps
@@ -0,0 +1,56 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Linux kernel symbol namespace import generator
+#
+# This script requires a minimum spatch version.
+SPATCH_REQ_VERSION="1.0.4"
+
+DIR="$(dirname $(readlink -f $0))/.."
+SPATCH="`which ${SPATCH:=spatch}`"
+if [ ! -x "$SPATCH" ]; then
+	echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/'
+	exit 1
+fi
+
+SPATCH_REQ_VERSION_NUM=$(echo $SPATCH_REQ_VERSION | ${DIR}/scripts/ld-version.sh)
+SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}')
+SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh)
+
+if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then
+	echo "spatch needs to be version $SPATCH_REQ_VERSION or higher"
+	exit 1
+fi
+
+generate_deps_for_ns() {
+	$SPATCH --very-quiet --in-place --sp-file \
+		$srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2
+}
+
+generate_deps() {
+	local mod_name=`basename $@ .ko`
+	local mod_file=`echo $@ | sed -e 's/\.ko/\.mod/'`
+	local ns_deps_file=`echo $@ | sed -e 's/\.ko/\.ns_deps/'`
+	if [ ! -f "$ns_deps_file" ]; then return; fi
+	local mod_source_files=`cat $mod_file | sed -n 1p | sed -e 's/\.o/\.c/g'`
+	for ns in `cat $ns_deps_file`; do
+		echo "Adding namespace $ns to module $mod_name (if needed)."
+		generate_deps_for_ns $ns $mod_source_files
+		# sort the imports
+		for source_file in $mod_source_files; do
+			sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp
+			offset=$(wc -l ${source_file}.tmp | awk '{print $1;}')
+			cat $source_file | grep MODULE_IMPORT_NS | sort -u >> ${source_file}.tmp
+			tail -n +$((offset +1)) ${source_file} | grep -v MODULE_IMPORT_NS >> ${source_file}.tmp
+			if ! diff -q ${source_file} ${source_file}.tmp; then
+				mv ${source_file}.tmp ${source_file}
+			else
+				rm ${source_file}.tmp
+			fi
+		done
+	done
+}
+
+for f in `cat $srctree/modules.order`; do
+	generate_deps $f
+done
+
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 09/11] usb-storage: remove single-use define for debugging
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
                       ` (7 preceding siblings ...)
  2019-08-21 11:49     ` [PATCH v3 08/11] scripts: Coccinelle script for " Matthias Maennich
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 12:37       ` Greg KH
  2019-08-21 13:21       ` Thomas Gleixner
  2019-08-21 11:49     ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
  9 siblings, 2 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

USB_STORAGE was defined as "usb-storage: " and used in a single location
as argument to printk. In order to be able to use the name
'USB_STORAGE', drop the definition and use the string directly for the
printk call.

Signed-off-by: Matthias Maennich <maennich@google.com>
---
 drivers/usb/storage/debug.h    | 2 --
 drivers/usb/storage/scsiglue.c | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/usb/storage/debug.h b/drivers/usb/storage/debug.h
index 6d64f342f587..16ce06039a4d 100644
--- a/drivers/usb/storage/debug.h
+++ b/drivers/usb/storage/debug.h
@@ -29,8 +29,6 @@
 
 #include <linux/kernel.h>
 
-#define USB_STORAGE "usb-storage: "
-
 #ifdef CONFIG_USB_STORAGE_DEBUG
 void usb_stor_show_command(const struct us_data *us, struct scsi_cmnd *srb);
 void usb_stor_show_sense(const struct us_data *us, unsigned char key,
diff --git a/drivers/usb/storage/scsiglue.c b/drivers/usb/storage/scsiglue.c
index 05b80211290d..df4de8323eff 100644
--- a/drivers/usb/storage/scsiglue.c
+++ b/drivers/usb/storage/scsiglue.c
@@ -379,7 +379,7 @@ static int queuecommand_lck(struct scsi_cmnd *srb,
 
 	/* check for state-transition errors */
 	if (us->srb != NULL) {
-		printk(KERN_ERR USB_STORAGE "Error in %s: us->srb = %p\n",
+		printk(KERN_ERR "usb-storage: Error in %s: us->srb = %p\n",
 			__func__, us->srb);
 		return SCSI_MLQUEUE_HOST_BUSY;
 	}
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace
       [not found]   ` <20190821114955.12788-1-maennich@google.com>
                       ` (8 preceding siblings ...)
  2019-08-21 11:49     ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
@ 2019-08-21 11:49     ` Matthias Maennich
  2019-08-21 12:38       ` Greg KH
  2019-08-21 23:13       ` Christoph Hellwig
  9 siblings, 2 replies; 51+ messages in thread
From: Matthias Maennich @ 2019-08-21 11:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: kernel-team, maennich, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

Modules using these symbols are required to explicitly import the
namespace. This patch was generated with the following steps and serves
as a reference to use the symbol namespace feature:

 1) Define DEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
 2) make  (see warnings during modpost about missing imports)
 3) make nsdeps

Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
variants can be used to explicitly specify the namespace. The advantage
of the method used here is that newly added symbols are automatically
exported and existing ones are exported without touching their
respective EXPORT_SYMBOL macro expansion.

Signed-off-by: Matthias Maennich <maennich@google.com>
---
 drivers/usb/storage/Makefile        | 2 ++
 drivers/usb/storage/alauda.c        | 1 +
 drivers/usb/storage/cypress_atacb.c | 1 +
 drivers/usb/storage/datafab.c       | 1 +
 drivers/usb/storage/ene_ub6250.c    | 1 +
 drivers/usb/storage/freecom.c       | 1 +
 drivers/usb/storage/isd200.c        | 1 +
 drivers/usb/storage/jumpshot.c      | 1 +
 drivers/usb/storage/karma.c         | 1 +
 drivers/usb/storage/onetouch.c      | 1 +
 drivers/usb/storage/realtek_cr.c    | 1 +
 drivers/usb/storage/sddr09.c        | 1 +
 drivers/usb/storage/sddr55.c        | 1 +
 drivers/usb/storage/shuttle_usbat.c | 1 +
 drivers/usb/storage/uas.c           | 1 +
 15 files changed, 16 insertions(+)

diff --git a/drivers/usb/storage/Makefile b/drivers/usb/storage/Makefile
index a67ddcbb4e24..46635fa4a340 100644
--- a/drivers/usb/storage/Makefile
+++ b/drivers/usb/storage/Makefile
@@ -8,6 +8,8 @@
 
 ccflags-y := -I $(srctree)/drivers/scsi
 
+ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=USB_STORAGE
+
 obj-$(CONFIG_USB_UAS)		+= uas.o
 obj-$(CONFIG_USB_STORAGE)	+= usb-storage.o
 
diff --git a/drivers/usb/storage/alauda.c b/drivers/usb/storage/alauda.c
index 6b8edf6178df..ddab2cd3d2e7 100644
--- a/drivers/usb/storage/alauda.c
+++ b/drivers/usb/storage/alauda.c
@@ -36,6 +36,7 @@
 MODULE_DESCRIPTION("Driver for Alauda-based card readers");
 MODULE_AUTHOR("Daniel Drake <dsd@gentoo.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * Status bytes
diff --git a/drivers/usb/storage/cypress_atacb.c b/drivers/usb/storage/cypress_atacb.c
index 4825902377eb..a6f3267bbef6 100644
--- a/drivers/usb/storage/cypress_atacb.c
+++ b/drivers/usb/storage/cypress_atacb.c
@@ -22,6 +22,7 @@
 MODULE_DESCRIPTION("SAT support for Cypress USB/ATA bridges with ATACB");
 MODULE_AUTHOR("Matthieu Castet <castet.matthieu@free.fr>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * The table of devices
diff --git a/drivers/usb/storage/datafab.c b/drivers/usb/storage/datafab.c
index 09353be199be..588818483f4b 100644
--- a/drivers/usb/storage/datafab.c
+++ b/drivers/usb/storage/datafab.c
@@ -54,6 +54,7 @@
 MODULE_DESCRIPTION("Driver for Datafab USB Compact Flash reader");
 MODULE_AUTHOR("Jimmie Mayfield <mayfield+datafab@sackheads.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 struct datafab_info {
 	unsigned long   sectors;	/* total sector count */
diff --git a/drivers/usb/storage/ene_ub6250.c b/drivers/usb/storage/ene_ub6250.c
index c26129d5b943..8b1b73065421 100644
--- a/drivers/usb/storage/ene_ub6250.c
+++ b/drivers/usb/storage/ene_ub6250.c
@@ -26,6 +26,7 @@
 
 MODULE_DESCRIPTION("Driver for ENE UB6250 reader");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 MODULE_FIRMWARE(SD_INIT1_FIRMWARE);
 MODULE_FIRMWARE(SD_INIT2_FIRMWARE);
 MODULE_FIRMWARE(SD_RW_FIRMWARE);
diff --git a/drivers/usb/storage/freecom.c b/drivers/usb/storage/freecom.c
index 4f542df37a44..34e7eaff1174 100644
--- a/drivers/usb/storage/freecom.c
+++ b/drivers/usb/storage/freecom.c
@@ -29,6 +29,7 @@
 MODULE_DESCRIPTION("Driver for Freecom USB/IDE adaptor");
 MODULE_AUTHOR("David Brown <usb-storage@davidb.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 #ifdef CONFIG_USB_STORAGE_DEBUG
 static void pdump(struct us_data *us, void *ibuffer, int length);
diff --git a/drivers/usb/storage/isd200.c b/drivers/usb/storage/isd200.c
index 2b474d60b4db..c4da3fd6eff9 100644
--- a/drivers/usb/storage/isd200.c
+++ b/drivers/usb/storage/isd200.c
@@ -53,6 +53,7 @@
 MODULE_DESCRIPTION("Driver for In-System Design, Inc. ISD200 ASIC");
 MODULE_AUTHOR("Björn Stenberg <bjorn@haxx.se>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 static int isd200_Initialization(struct us_data *us);
 
diff --git a/drivers/usb/storage/jumpshot.c b/drivers/usb/storage/jumpshot.c
index 917f170c4124..229bf0c1afc9 100644
--- a/drivers/usb/storage/jumpshot.c
+++ b/drivers/usb/storage/jumpshot.c
@@ -51,6 +51,7 @@
 MODULE_DESCRIPTION("Driver for Lexar \"Jumpshot\" Compact Flash reader");
 MODULE_AUTHOR("Jimmie Mayfield <mayfield+usb@sackheads.org>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * The table of devices
diff --git a/drivers/usb/storage/karma.c b/drivers/usb/storage/karma.c
index 395cf8fb5870..05cec81dcd3f 100644
--- a/drivers/usb/storage/karma.c
+++ b/drivers/usb/storage/karma.c
@@ -23,6 +23,7 @@
 MODULE_DESCRIPTION("Driver for Rio Karma");
 MODULE_AUTHOR("Bob Copeland <me@bobcopeland.com>, Keith Bennett <keith@mcs.st-and.ac.uk>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 #define RIO_PREFIX "RIOP\x00"
 #define RIO_PREFIX_LEN 5
diff --git a/drivers/usb/storage/onetouch.c b/drivers/usb/storage/onetouch.c
index 39a5009a41a6..a989fe930e21 100644
--- a/drivers/usb/storage/onetouch.c
+++ b/drivers/usb/storage/onetouch.c
@@ -25,6 +25,7 @@
 MODULE_DESCRIPTION("Maxtor USB OneTouch hard drive button driver");
 MODULE_AUTHOR("Nick Sillik <n.sillik@temple.edu>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 #define ONETOUCH_PKT_LEN        0x02
 #define ONETOUCH_BUTTON         KEY_PROG1
diff --git a/drivers/usb/storage/realtek_cr.c b/drivers/usb/storage/realtek_cr.c
index cc794e25a0b6..edbe419053d6 100644
--- a/drivers/usb/storage/realtek_cr.c
+++ b/drivers/usb/storage/realtek_cr.c
@@ -35,6 +35,7 @@
 MODULE_DESCRIPTION("Driver for Realtek USB Card Reader");
 MODULE_AUTHOR("wwang <wei_wang@realsil.com.cn>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 static int auto_delink_en = 1;
 module_param(auto_delink_en, int, S_IRUGO | S_IWUSR);
diff --git a/drivers/usb/storage/sddr09.c b/drivers/usb/storage/sddr09.c
index bc9da736bdfc..51bcd4a43690 100644
--- a/drivers/usb/storage/sddr09.c
+++ b/drivers/usb/storage/sddr09.c
@@ -47,6 +47,7 @@
 MODULE_DESCRIPTION("Driver for SanDisk SDDR-09 SmartMedia reader");
 MODULE_AUTHOR("Andries Brouwer <aeb@cwi.nl>, Robert Baruch <autophile@starband.net>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 static int usb_stor_sddr09_dpcm_init(struct us_data *us);
 static int sddr09_transport(struct scsi_cmnd *srb, struct us_data *us);
diff --git a/drivers/usb/storage/sddr55.c b/drivers/usb/storage/sddr55.c
index b8527c55335b..ba955d65eb0e 100644
--- a/drivers/usb/storage/sddr55.c
+++ b/drivers/usb/storage/sddr55.c
@@ -29,6 +29,7 @@
 MODULE_DESCRIPTION("Driver for SanDisk SDDR-55 SmartMedia reader");
 MODULE_AUTHOR("Simon Munton");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /*
  * The table of devices
diff --git a/drivers/usb/storage/shuttle_usbat.c b/drivers/usb/storage/shuttle_usbat.c
index 854498e1012c..54aa1392c9ca 100644
--- a/drivers/usb/storage/shuttle_usbat.c
+++ b/drivers/usb/storage/shuttle_usbat.c
@@ -48,6 +48,7 @@
 MODULE_DESCRIPTION("Driver for SCM Microsystems (a.k.a. Shuttle) USB-ATAPI cable");
 MODULE_AUTHOR("Daniel Drake <dsd@gentoo.org>, Robert Baruch <autophile@starband.net>");
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 
 /* Supported device types */
 #define USBAT_DEV_HP8200	0x01
diff --git a/drivers/usb/storage/uas.c b/drivers/usb/storage/uas.c
index 047c5922618f..bf80d6f81f58 100644
--- a/drivers/usb/storage/uas.c
+++ b/drivers/usb/storage/uas.c
@@ -1219,5 +1219,6 @@ static struct usb_driver uas_driver = {
 module_usb_driver(uas_driver);
 
 MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS(USB_STORAGE);
 MODULE_AUTHOR(
 	"Hans de Goede <hdegoede@redhat.com>, Matthew Wilcox and Sarah Sharp");
-- 
2.23.0.rc1.153.gdeed80330f-goog


^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v3 09/11] usb-storage: remove single-use define for debugging
  2019-08-21 11:49     ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
@ 2019-08-21 12:37       ` Greg KH
  2019-08-21 13:21       ` Thomas Gleixner
  1 sibling, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-21 12:37 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Wed, Aug 21, 2019 at 12:49:24PM +0100, Matthias Maennich wrote:
> USB_STORAGE was defined as "usb-storage: " and used in a single location
> as argument to printk. In order to be able to use the name
> 'USB_STORAGE', drop the definition and use the string directly for the
> printk call.
> 
> Signed-off-by: Matthias Maennich <maennich@google.com>

As you know, this patch is already in the usb-next tree, and will be in
the 5.4-rc1 merge.

But, as this series will end up going through a different tree than the
usb tree, here's my reviewed-by so that it can be included with the rest
of these patches:

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace
  2019-08-21 11:49     ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
@ 2019-08-21 12:38       ` Greg KH
  2019-08-21 14:36         ` Jessica Yu
  2019-08-21 23:13       ` Christoph Hellwig
  1 sibling, 1 reply; 51+ messages in thread
From: Greg KH @ 2019-08-21 12:38 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, kernel-team, arnd, geert, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Wed, Aug 21, 2019 at 12:49:25PM +0100, Matthias Maennich wrote:
> Modules using these symbols are required to explicitly import the
> namespace. This patch was generated with the following steps and serves
> as a reference to use the symbol namespace feature:
> 
>  1) Define DEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
>  2) make  (see warnings during modpost about missing imports)
>  3) make nsdeps
> 
> Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
> variants can be used to explicitly specify the namespace. The advantage
> of the method used here is that newly added symbols are automatically
> exported and existing ones are exported without touching their
> respective EXPORT_SYMBOL macro expansion.
> 
> Signed-off-by: Matthias Maennich <maennich@google.com>

This looks good to me.  This can be included with the rest of this
series when/if it goes through the kbuild or module tree:

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Actually, which tree will this be going through?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v3 09/11] usb-storage: remove single-use define for debugging
  2019-08-21 11:49     ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
  2019-08-21 12:37       ` Greg KH
@ 2019-08-21 13:21       ` Thomas Gleixner
  2019-08-21 13:32         ` Greg KH
  1 sibling, 1 reply; 51+ messages in thread
From: Thomas Gleixner @ 2019-08-21 13:21 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, kernel-team, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, usb-storage,
	x86, yamada.masahiro

On Wed, 21 Aug 2019, Matthias Maennich wrote:

> USB_STORAGE was defined as "usb-storage: " and used in a single location
> as argument to printk. In order to be able to use the name
> 'USB_STORAGE', drop the definition and use the string directly for the
> printk call.
> 
> Signed-off-by: Matthias Maennich <maennich@google.com>
> ---
>  drivers/usb/storage/debug.h    | 2 --
>  drivers/usb/storage/scsiglue.c | 2 +-
>  2 files changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/drivers/usb/storage/debug.h b/drivers/usb/storage/debug.h
> index 6d64f342f587..16ce06039a4d 100644
> --- a/drivers/usb/storage/debug.h
> +++ b/drivers/usb/storage/debug.h
> @@ -29,8 +29,6 @@
>  
>  #include <linux/kernel.h>
>  
> -#define USB_STORAGE "usb-storage: "
> -
>  #ifdef CONFIG_USB_STORAGE_DEBUG
>  void usb_stor_show_command(const struct us_data *us, struct scsi_cmnd *srb);
>  void usb_stor_show_sense(const struct us_data *us, unsigned char key,
> diff --git a/drivers/usb/storage/scsiglue.c b/drivers/usb/storage/scsiglue.c
> index 05b80211290d..df4de8323eff 100644
> --- a/drivers/usb/storage/scsiglue.c
> +++ b/drivers/usb/storage/scsiglue.c
> @@ -379,7 +379,7 @@ static int queuecommand_lck(struct scsi_cmnd *srb,
>  
>  	/* check for state-transition errors */
>  	if (us->srb != NULL) {
> -		printk(KERN_ERR USB_STORAGE "Error in %s: us->srb = %p\n",
> +		printk(KERN_ERR "usb-storage: Error in %s: us->srb = %p\n",
>  			__func__, us->srb);

The proper fix for this is to use pr_fmt and convert the printk to pr_err().

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v3 09/11] usb-storage: remove single-use define for debugging
  2019-08-21 13:21       ` Thomas Gleixner
@ 2019-08-21 13:32         ` Greg KH
  0 siblings, 0 replies; 51+ messages in thread
From: Greg KH @ 2019-08-21 13:32 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Matthias Maennich, linux-kernel, kernel-team, arnd, geert, hpa,
	jeyu, joel, kstewart, linux-arch, linux-kbuild, linux-m68k,
	linux-modules, linux-scsi, linux-usb, lucas.de.marchi, maco,
	maco, michal.lkml, mingo, oneukum, pombredanne, sam, sspatil,
	stern, usb-storage, x86, yamada.masahiro

On Wed, Aug 21, 2019 at 03:21:22PM +0200, Thomas Gleixner wrote:
> On Wed, 21 Aug 2019, Matthias Maennich wrote:
> 
> > USB_STORAGE was defined as "usb-storage: " and used in a single location
> > as argument to printk. In order to be able to use the name
> > 'USB_STORAGE', drop the definition and use the string directly for the
> > printk call.
> > 
> > Signed-off-by: Matthias Maennich <maennich@google.com>
> > ---
> >  drivers/usb/storage/debug.h    | 2 --
> >  drivers/usb/storage/scsiglue.c | 2 +-
> >  2 files changed, 1 insertion(+), 3 deletions(-)
> > 
> > diff --git a/drivers/usb/storage/debug.h b/drivers/usb/storage/debug.h
> > index 6d64f342f587..16ce06039a4d 100644
> > --- a/drivers/usb/storage/debug.h
> > +++ b/drivers/usb/storage/debug.h
> > @@ -29,8 +29,6 @@
> >  
> >  #include <linux/kernel.h>
> >  
> > -#define USB_STORAGE "usb-storage: "
> > -
> >  #ifdef CONFIG_USB_STORAGE_DEBUG
> >  void usb_stor_show_command(const struct us_data *us, struct scsi_cmnd *srb);
> >  void usb_stor_show_sense(const struct us_data *us, unsigned char key,
> > diff --git a/drivers/usb/storage/scsiglue.c b/drivers/usb/storage/scsiglue.c
> > index 05b80211290d..df4de8323eff 100644
> > --- a/drivers/usb/storage/scsiglue.c
> > +++ b/drivers/usb/storage/scsiglue.c
> > @@ -379,7 +379,7 @@ static int queuecommand_lck(struct scsi_cmnd *srb,
> >  
> >  	/* check for state-transition errors */
> >  	if (us->srb != NULL) {
> > -		printk(KERN_ERR USB_STORAGE "Error in %s: us->srb = %p\n",
> > +		printk(KERN_ERR "usb-storage: Error in %s: us->srb = %p\n",
> >  			__func__, us->srb);
> 
> The proper fix for this is to use pr_fmt and convert the printk to pr_err().

Yeah, that's the correct long-term fix, I think someone already sent
that in for the usb tree, where I have taken this patch already.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace
  2019-08-21 12:38       ` Greg KH
@ 2019-08-21 14:36         ` Jessica Yu
  0 siblings, 0 replies; 51+ messages in thread
From: Jessica Yu @ 2019-08-21 14:36 UTC (permalink / raw)
  To: Greg KH
  Cc: Matthias Maennich, linux-kernel, kernel-team, arnd, geert, hpa,
	joel, kstewart, linux-arch, linux-kbuild, linux-m68k,
	linux-modules, linux-scsi, linux-usb, lucas.de.marchi, maco,
	maco, michal.lkml, mingo, oneukum, pombredanne, sam, sspatil,
	stern, tglx, usb-storage, x86, yamada.masahiro

+++ Greg KH [21/08/19 05:38 -0700]:
>On Wed, Aug 21, 2019 at 12:49:25PM +0100, Matthias Maennich wrote:
>> Modules using these symbols are required to explicitly import the
>> namespace. This patch was generated with the following steps and serves
>> as a reference to use the symbol namespace feature:
>>
>>  1) Define DEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
>>  2) make  (see warnings during modpost about missing imports)
>>  3) make nsdeps
>>
>> Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
>> variants can be used to explicitly specify the namespace. The advantage
>> of the method used here is that newly added symbols are automatically
>> exported and existing ones are exported without touching their
>> respective EXPORT_SYMBOL macro expansion.
>>
>> Signed-off-by: Matthias Maennich <maennich@google.com>
>
>This looks good to me.  This can be included with the rest of this
>series when/if it goes through the kbuild or module tree:
>
>Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>
>Actually, which tree will this be going through?

I would be happy to take the patchset through the modules tree once it
collects the appropriate ack/reviewed-by's and once I get a chance to
sit down and review/test it next week :)

Thanks!

Jessica

^ permalink raw reply	[flat|nested] 51+ messages in thread

* Re: [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace
  2019-08-21 11:49     ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
  2019-08-21 12:38       ` Greg KH
@ 2019-08-21 23:13       ` Christoph Hellwig
  1 sibling, 0 replies; 51+ messages in thread
From: Christoph Hellwig @ 2019-08-21 23:13 UTC (permalink / raw)
  To: Matthias Maennich
  Cc: linux-kernel, kernel-team, arnd, geert, gregkh, hpa, jeyu, joel,
	kstewart, linux-arch, linux-kbuild, linux-m68k, linux-modules,
	linux-scsi, linux-usb, lucas.de.marchi, maco, maco, michal.lkml,
	mingo, oneukum, pombredanne, sam, sspatil, stern, tglx,
	usb-storage, x86, yamada.masahiro

On Wed, Aug 21, 2019 at 12:49:25PM +0100, Matthias Maennich wrote:
> Modules using these symbols are required to explicitly import the
> namespace. This patch was generated with the following steps and serves
> as a reference to use the symbol namespace feature:
> 
>  1) Define DEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile
>  2) make  (see warnings during modpost about missing imports)
>  3) make nsdeps
> 
> Instead of a DEFAULT_SYMBOL_NAMESPACE definition, the EXPORT_SYMBOL_NS
> variants can be used to explicitly specify the namespace. The advantage
> of the method used here is that newly added symbols are automatically
> exported and existing ones are exported without touching their
> respective EXPORT_SYMBOL macro expansion.

So what is USB_STORAGE here?  It isn't a C string, so where does it
come from?  To me using a C string would seem like the nicer interface
vs a random cpp symbol that gets injected somewhere.

^ permalink raw reply	[flat|nested] 51+ messages in thread

end of thread, back to index

Thread overview: 51+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20180716122125.175792-1-maco@android.com>
     [not found] ` <20180716122125.175792-3-maco@android.com>
     [not found]   ` <CAB0TPYEOVHcFGFLTjVvk7R0VSgSnXZRi1PjSYXONJSjqd4NewQ@mail.gmail.com>
     [not found]     ` <20180725155507.umb5oatduquxwlmt@linux-8ccs>
2018-07-25 16:48       ` [PATCH 2/6] module: add support for symbol namespaces Lucas De Marchi
2018-07-26  7:44         ` Martijn Coenen
2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
2019-08-13 12:16   ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-13 12:40     ` Greg KH
2019-08-13 12:16   ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-13 12:41     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich
2019-08-13 15:26     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 04/10] modpost: " Matthias Maennich
2019-08-13 15:27     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-13 18:17     ` Greg KH
2019-08-13 20:15     ` Saravana Kannan
2019-08-14 12:54       ` Matthias Maennich
2019-08-14 17:34         ` Saravana Kannan
2019-08-13 12:17   ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-13 18:16     ` Greg KH
2019-08-13 18:16     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-13 18:21     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
2019-08-13 12:31     ` Julia Lawall
2019-08-13 12:44     ` Greg KH
2019-08-14  6:36     ` [Cocci] " Himanshu Jha
2019-08-14  8:03       ` Matthias Maennich
2019-08-14 12:00     ` [v2 " Markus Elfring
2019-08-15 13:50     ` Markus Elfring
2019-08-13 12:17   ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-13 12:42     ` Greg KH
2019-08-13 13:12       ` Greg KH
2019-08-13 12:17   ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-13 12:45     ` Greg KH
2019-08-13 12:47     ` Greg KH
2019-08-13 15:02       ` Matthias Maennich
     [not found]   ` <20190821114955.12788-1-maennich@google.com>
2019-08-21 11:49     ` [PATCH v3 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 03/11] module: add support for symbol namespaces Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 04/11] modpost: " Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 08/11] scripts: Coccinelle script for " Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-21 12:37       ` Greg KH
2019-08-21 13:21       ` Thomas Gleixner
2019-08-21 13:32         ` Greg KH
2019-08-21 11:49     ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-21 12:38       ` Greg KH
2019-08-21 14:36         ` Jessica Yu
2019-08-21 23:13       ` Christoph Hellwig

Linux-Modules Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-modules/0 linux-modules/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-modules linux-modules/ https://lore.kernel.org/linux-modules \
		linux-modules@vger.kernel.org linux-modules@archiver.kernel.org
	public-inbox-index linux-modules


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-modules


AGPL code for this site: git clone https://public-inbox.org/ public-inbox