From: Matthias Maennich <maennich@google.com> To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, maennich@google.com, arnd@arndb.de, gregkh@linuxfoundation.org, jeyu@kernel.org, joel@joelfernandes.org, lucas.de.marchi@gmail.com, maco@android.com, sspatil@google.com, will@kernel.org, yamada.masahiro@socionext.com, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-usb@vger.kernel.org, usb-storage@lists.one-eyed-alien.net Subject: [PATCH v5 00/11] Symbol Namespaces Date: Fri, 6 Sep 2019 11:32:24 +0100 Message-ID: <20190906103235.197072-1-maennich@google.com> (raw) In-Reply-To: <20180716122125.175792-1-maco@android.com> As of Linux 5.3-rc7, there are 31207 [1] exported symbols in the kernel. That is a growth of roughly 1000 symbols since 4.17 (30206 [2]). There seems to be some consensus amongst kernel devs that the export surface is too large, and hard to reason about. Generally, these symbols fall in one of these categories: 1) Symbols actually meant for drivers 2) Symbols that are only exported because functionality is split over multiple modules, yet they really shouldn't be used by modules outside of their own subsystem 3) Symbols really only meant for in-tree use When module developers try to upstream their code, it regularly turns out that they are using exported symbols that they really shouldn't be using. This problem is even bigger for drivers that are currently out-of-tree, which may be using many symbols that they shouldn't be using, and that break when those symbols are removed or modified. This patch allows subsystem maintainers to partition their exported symbols into separate namespaces, and module authors to import such namespaces only when needed. This allows subsystem maintainers to more easily limit availability of these namespaced symbols to other parts of the kernel. It can also be used to partition the set of exported symbols for documentation purposes; for example, a set of symbols that is really only used for debugging could be in a "SUBSYSTEM_DEBUG" namespace. I continued the work mainly done by Martijn Coenen. Changes in v2: - Rather than adding and evaluating separate sections __knsimport_NS, use modinfo tags to declare the namespaces a module introduces. Adjust modpost and the module loader accordingly. - Also add support for reading multiple modinfo values for the same tag to allow list-like access to modinfo tags. - The macros in export.h have been cleaned up to avoid redundancy in the macro parameters (ns, nspost, nspost2). - The introduction of relative references in the ksymtab entries caused a rework of the macros to accommodate that configuration as well. - Alignment of kernel_symbol in the ksymtab needed to be fixed to allow growing the kernel_symbol struct. - Modpost does now also append the namespace suffix to the symbol entries in Module.symvers. - The configuration option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS allows relaxing the enforcement of properly declared namespace imports at module loading time. - Symbols can be collectively exported into a namespace by defining DEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile. - The requirement for a very recent coccinelle spatch has been lifted by simplifying the script. - nsdeps does now ensures MODULE_IMPORT_NS statements are sorted when patching the module source files. - Some minor bugs have been addressed in nsdeps to allow it to work with modules that have more than one source file. - The RFC for the usb-storage symbols has been simplified by using DEFAULT_SYMBOL_NAMESPACE=USB_STORAGE rather than explicitly exporting each and every symbol into that new namespace. Changes in v3: - Reword the documentation for the MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS option for clarification. - Fix printed required version of spatch in coccinelle script. - Adopt kbuild changes for modpost: .mod files are no longer generated in .tmp_versions. Similarely, generate the .ns_deps files in the tree along with the .mod files. Also, nsdeps now uses modules.order as source for the list modules to consider. - Add an RFC patch to introduce the namespace WATCHDOG_CORE for symbols exported in watchdog_core.c. Changes in v4: - scripts/nsdeps: - exit on first error - support out-of-tree builds O=... - scripts/export_report.pl: update for new Module.symvers format - scripts/mod/modpost: make the namespace a separate field when exporting to Module.symvers (rather than symbol.NS) - include/linux/export.h: fixed style nits - kernel/module.c: ensure namespaces are imported before taking a reference to the owner module - Documentation: document the Symbol Namespace feature and update references to Module.symvers and EXPORT_SYMBOL* Changes in v5: - Makefile: let 'nsdeps' depend on 'modules' to allow `make clean; make nsdeps` to work - scripts/nsdeps: drop 'exit on first error' again as it just makes more problems than it solves - drop the watchdog RFC patch for now This patch series was developed against v5.3-rc7. [1] git grep "^EXPORT_SYMBOL\w*(" v5.3-rc7 | wc -l [2] git grep "^EXPORT_SYMBOL\w*(" v4.17 | wc -l Cc: arnd@arndb.de Cc: gregkh@linuxfoundation.org Cc: jeyu@kernel.org Cc: joel@joelfernandes.org Cc: lucas.de.marchi@gmail.com Cc: maco@android.com Cc: sspatil@google.com Cc: will@kernel.org Cc: yamada.masahiro@socionext.com Cc: linux-kbuild@vger.kernel.org Cc: linux-modules@vger.kernel.org Cc: linux-usb@vger.kernel.org Cc: usb-storage@lists.one-eyed-alien.net Cc: kernel-team@android.com Matthias Maennich (11): module: support reading multiple values per modinfo tag export: explicitly align struct kernel_symbol module: add support for symbol namespaces. modpost: add support for symbol namespaces module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS export: allow definition default namespaces in Makefiles or sources modpost: add support for generating namespace dependencies scripts: Coccinelle script for namespace dependencies. docs: Add documentation for Symbol Namespaces usb-storage: remove single-use define for debugging usb-storage: export symbols in USB_STORAGE namespace .gitignore | 1 + Documentation/kbuild/modules.rst | 7 +- Documentation/kbuild/namespaces.rst | 154 ++++++++++++++++++++ Documentation/kernel-hacking/hacking.rst | 18 +++ MAINTAINERS | 5 + Makefile | 14 +- arch/m68k/include/asm/export.h | 1 - drivers/usb/storage/Makefile | 2 + drivers/usb/storage/alauda.c | 1 + drivers/usb/storage/cypress_atacb.c | 1 + drivers/usb/storage/datafab.c | 1 + drivers/usb/storage/debug.h | 2 - drivers/usb/storage/ene_ub6250.c | 1 + drivers/usb/storage/freecom.c | 1 + drivers/usb/storage/isd200.c | 1 + drivers/usb/storage/jumpshot.c | 1 + drivers/usb/storage/karma.c | 1 + drivers/usb/storage/onetouch.c | 1 + drivers/usb/storage/realtek_cr.c | 1 + drivers/usb/storage/scsiglue.c | 2 +- drivers/usb/storage/sddr09.c | 1 + drivers/usb/storage/sddr55.c | 1 + drivers/usb/storage/shuttle_usbat.c | 1 + drivers/usb/storage/uas.c | 1 + include/asm-generic/export.h | 14 +- include/linux/export.h | 98 +++++++++++-- include/linux/module.h | 2 + init/Kconfig | 13 ++ kernel/module.c | 67 ++++++++- scripts/Makefile.modpost | 4 +- scripts/coccinelle/misc/add_namespace.cocci | 23 +++ scripts/export_report.pl | 2 +- scripts/mod/modpost.c | 150 ++++++++++++++++--- scripts/mod/modpost.h | 9 ++ scripts/nsdeps | 58 ++++++++ 35 files changed, 608 insertions(+), 52 deletions(-) create mode 100644 Documentation/kbuild/namespaces.rst create mode 100644 scripts/coccinelle/misc/add_namespace.cocci create mode 100644 scripts/nsdeps -- 2.23.0.187.g17f5b7556c-goog
next prev parent reply index Thread overview: 134+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-07-16 12:21 [PATCH 0/6] Symbol namespaces Martijn Coenen 2018-07-16 12:21 ` [PATCH 1/6] export: explicitly align struct kernel_symbol Martijn Coenen 2018-07-16 12:21 ` [PATCH 2/6] module: add support for symbol namespaces Martijn Coenen 2018-07-19 16:32 ` Jessica Yu 2018-07-20 7:54 ` Martijn Coenen 2018-07-20 14:49 ` Jessica Yu 2018-07-20 15:42 ` Martijn Coenen 2018-07-23 11:12 ` Jessica Yu 2018-07-24 7:44 ` Martijn Coenen 2018-07-24 7:56 ` Martijn Coenen 2018-07-25 15:55 ` Jessica Yu 2018-07-25 16:48 ` Lucas De Marchi 2018-07-26 7:44 ` Martijn Coenen 2018-07-16 12:21 ` [PATCH 3/6] modpost: add support for checking " Martijn Coenen 2018-07-16 12:21 ` [PATCH 4/6] modpost: add support for generating namespace dependencies Martijn Coenen 2018-07-23 6:49 ` Jessica Yu 2018-07-16 12:21 ` [PATCH 5/6] scripts: Coccinelle script for " Martijn Coenen 2018-07-16 12:21 ` [PATCH 6/6] RFC: USB: storage: move symbols into USB_STORAGE namespace Martijn Coenen 2018-07-17 15:04 ` Alan Stern 2018-07-18 13:28 ` Martijn Coenen 2018-07-16 15:33 ` [PATCH 0/6] Symbol namespaces Greg Kroah-Hartman 2018-07-23 14:28 ` Arnd Bergmann 2018-07-24 8:09 ` Martijn Coenen 2018-07-24 9:08 ` Arnd Bergmann 2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich 2019-08-13 12:16 ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich 2019-08-13 12:40 ` Greg KH 2019-08-13 12:16 ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich 2019-08-13 12:41 ` Greg KH 2019-08-13 12:17 ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich 2019-08-13 15:26 ` Greg KH 2019-08-13 12:17 ` [PATCH v2 04/10] modpost: " Matthias Maennich 2019-08-13 15:27 ` Greg KH 2019-08-13 12:17 ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich 2019-08-13 18:17 ` Greg KH 2019-08-13 20:15 ` Saravana Kannan 2019-08-14 12:54 ` Matthias Maennich 2019-08-14 17:34 ` Saravana Kannan 2019-08-13 12:17 ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich 2019-08-13 18:16 ` Greg KH 2019-08-13 18:16 ` Greg KH 2019-08-13 12:17 ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich 2019-08-13 18:21 ` Greg KH 2019-08-13 12:17 ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich 2019-08-13 12:31 ` Julia Lawall 2019-08-13 12:44 ` Greg KH 2019-08-14 6:36 ` [Cocci] " Himanshu Jha 2019-08-14 8:03 ` Matthias Maennich 2019-08-15 13:50 ` [v2 " Markus Elfring 2019-08-22 9:18 ` Matthias Maennich 2019-08-22 11:00 ` Markus Elfring 2019-08-13 12:17 ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich 2019-08-13 12:42 ` Greg KH 2019-08-13 13:12 ` Greg KH 2019-08-13 12:17 ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich 2019-08-13 12:45 ` Greg KH 2019-08-13 12:47 ` Greg KH 2019-08-13 15:02 ` Matthias Maennich [not found] ` <20190821114955.12788-1-maennich@google.com> 2019-08-21 11:49 ` [PATCH v3 01/11] module: support reading multiple values per modinfo tag Matthias Maennich 2019-08-21 11:49 ` [PATCH v3 02/11] export: explicitly align struct kernel_symbol Matthias Maennich 2019-08-21 11:49 ` [PATCH v3 03/11] module: add support for symbol namespaces Matthias Maennich 2019-08-27 15:37 ` Jessica Yu 2019-08-27 16:04 ` Matthias Maennich 2019-08-21 11:49 ` [PATCH v3 04/11] modpost: " Matthias Maennich 2019-08-26 16:21 ` Jessica Yu 2019-08-27 14:41 ` Matthias Maennich 2019-08-28 9:43 ` Jessica Yu 2019-08-28 9:55 ` Matthias Maennich 2019-08-28 10:16 ` Jessica Yu 2019-08-21 11:49 ` [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich 2019-08-21 11:49 ` [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich 2019-08-28 10:49 ` Jessica Yu 2019-08-28 10:56 ` Matthias Maennich 2019-08-21 11:49 ` [PATCH v3 07/11] modpost: add support for generating namespace dependencies Matthias Maennich 2019-08-21 11:49 ` [PATCH v3 08/11] scripts: Coccinelle script for " Matthias Maennich 2019-08-22 6:09 ` [v3 " Markus Elfring 2019-08-29 12:13 ` [PATCH v3 " Jessica Yu 2019-08-21 11:49 ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich 2019-08-21 12:37 ` Greg KH 2019-08-21 13:21 ` Thomas Gleixner 2019-08-21 13:32 ` Greg KH 2019-08-21 11:49 ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich 2019-08-21 12:38 ` Greg KH 2019-08-21 14:36 ` Jessica Yu 2019-08-21 23:13 ` Christoph Hellwig 2019-08-22 8:32 ` Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 00/12] Symbol Namespaces Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 01/12] module: support reading multiple values per modinfo tag Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 02/12] export: explicitly align struct kernel_symbol Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 03/12] module: add support for symbol namespaces Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 04/12] modpost: " Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 05/12] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 06/12] export: allow definition default namespaces in Makefiles or sources Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 07/12] modpost: add support for generating namespace dependencies Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 08/12] scripts: Coccinelle script for " Matthias Maennich 2019-09-04 9:53 ` Masahiro Yamada 2019-09-05 14:46 ` Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 09/12] docs: Add documentation for Symbol Namespaces Matthias Maennich 2019-09-04 7:16 ` Greg KH 2019-09-03 15:06 ` [PATCH v4 10/12] usb-storage: remove single-use define for debugging Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 11/12] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich 2019-09-03 15:06 ` [PATCH v4 12/12] RFC: watchdog: export core symbols in WATCHDOG_CORE namespace Matthias Maennich 2019-09-03 16:10 ` Guenter Roeck 2019-09-04 8:45 ` Masahiro Yamada 2019-09-04 12:12 ` Guenter Roeck 2019-09-04 16:16 ` [usb-storage] " Matthew Dharm 2019-09-05 10:41 ` Jessica Yu 2019-09-05 10:52 ` Arnd Bergmann 2019-09-05 11:16 ` Jessica Yu 2019-09-05 11:25 ` Masahiro Yamada 2019-09-05 12:00 ` Greg Kroah-Hartman 2019-09-05 11:25 ` Matthias Maennich 2019-09-04 9:37 ` [PATCH v4 00/12] Symbol Namespaces Masahiro Yamada 2019-09-06 10:32 ` Matthias Maennich [this message] 2019-09-06 10:32 ` [PATCH v5 01/11] module: support reading multiple values per modinfo tag Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 02/11] export: explicitly align struct kernel_symbol Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 03/11] module: add support for symbol namespaces Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 04/11] modpost: " Matthias Maennich 2019-09-26 22:24 ` [PATCH] modpost: Copy namespace string into 'struct symbol' Shaun Ruffell 2019-09-27 5:31 ` Greg Kroah-Hartman 2019-09-27 8:03 ` Matthias Maennich 2019-09-30 21:20 ` Shaun Ruffell 2019-10-01 16:19 ` Matthias Maennich 2019-10-01 19:37 ` Shaun Ruffell 2019-09-06 10:32 ` [PATCH v5 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 07/11] modpost: add support for generating namespace dependencies Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 08/11] scripts: Coccinelle script for " Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 09/11] docs: Add documentation for Symbol Namespaces Matthias Maennich 2019-09-06 10:32 ` [PATCH v5 10/11] usb-storage: remove single-use define for debugging Matthias Maennich 2019-09-06 12:59 ` Jessica Yu 2019-09-06 13:22 ` Greg KH 2019-09-06 10:32 ` [PATCH v5 11/11] usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich 2019-09-09 8:35 ` [PATCH v5 00/11] Symbol Namespaces Jessica Yu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190906103235.197072-1-maennich@google.com \ --to=maennich@google.com \ --cc=arnd@arndb.de \ --cc=gregkh@linuxfoundation.org \ --cc=jeyu@kernel.org \ --cc=joel@joelfernandes.org \ --cc=kernel-team@android.com \ --cc=linux-kbuild@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-modules@vger.kernel.org \ --cc=linux-usb@vger.kernel.org \ --cc=lucas.de.marchi@gmail.com \ --cc=maco@android.com \ --cc=sspatil@google.com \ --cc=usb-storage@lists.one-eyed-alien.net \ --cc=will@kernel.org \ --cc=yamada.masahiro@socionext.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
LKML Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \ linux-kernel@vger.kernel.org public-inbox-index lkml Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel AGPL code for this site: git clone https://public-inbox.org/public-inbox.git