Linux-USB Archive on lore.kernel.org
 help / color / Atom feed
From: Matthias Maennich <maennich@google.com>
To: linux-kernel@vger.kernel.org, maco@android.com
Cc: kernel-team@android.com, maennich@google.com, arnd@arndb.de,
	geert@linux-m68k.org, gregkh@linuxfoundation.org, hpa@zytor.com,
	jeyu@kernel.org, joel@joelfernandes.org,
	kstewart@linuxfoundation.org, linux-arch@vger.kernel.org,
	linux-kbuild@vger.kernel.org, linux-m68k@lists.linux-m68k.org,
	linux-modules@vger.kernel.org, linux-scsi@vger.kernel.org,
	linux-usb@vger.kernel.org, lucas.de.marchi@gmail.com,
	maco@google.com, michal.lkml@markovi.net, mingo@redhat.com,
	oneukum@suse.com, pombredanne@nexb.com, sam@ravnborg.org,
	sboyd@codeaurora.org, sspatil@google.com,
	stern@rowland.harvard.edu, tglx@linutronix.de,
	usb-storage@lists.one-eyed-alien.net, x86@kernel.org,
	yamada.masahiro@socionext.com, Adrian Reber <adrian@lisas.de>,
	Andrew Morton <akpm@linux-foundation.org>,
	Ard Biesheuvel <ard.biesheuvel@linaro.org>,
	cocci@systeme.lip6.fr, Dan Williams <dan.j.williams@intel.com>,
	David Howells <dhowells@redhat.com>,
	"David S. Miller" <davem@davemloft.net>,
	Ingo Molnar <mingo@kernel.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Jonathan Cameron <Jonathan.Cameron@huawei.com>,
	Mauro Carvalho Chehab <mchehab+samsung@kernel.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Nicolas Ferre <nicolas.ferre@microchip.com>,
	Patrick Bellasi <patrick.bellasi@arm.com>,
	Richard Guy Briggs <rgb@redhat.com>,
	Will Deacon <will@kernel.org>
Subject: [PATCH v2 0/10] Symbol namespaces - RFC
Date: Tue, 13 Aug 2019 13:16:57 +0100
Message-ID: <20190813121733.52480-1-maennich@google.com> (raw)
In-Reply-To: <20180716122125.175792-1-maco@android.com>

As of Linux 5.3-rc4, there are 31203 [1] exported symbols in the kernel.
That is a growth of almost 1000 symbols since 4.17 (30206 [2]).  There
seems to be some consensus amongst kernel devs that the export surface
is too large, and hard to reason about.

Generally, these symbols fall in one of these categories:
1) Symbols actually meant for drivers
2) Symbols that are only exported because functionality is split over
   multiple modules, yet they really shouldn't be used by modules outside
   of their own subsystem
3) Symbols really only meant for in-tree use

When module developers try to upstream their code, it regularly turns
out that they are using exported symbols that they really shouldn't be
using. This problem is even bigger for drivers that are currently
out-of-tree, which may be using many symbols that they shouldn't be
using, and that break when those symbols are removed or modified.

This patch allows subsystem maintainers to partition their exported
symbols into separate namespaces, and module authors to import such
namespaces only when needed.

This allows subsystem maintainers to more easily limit availability of
these namespaced symbols to other parts of the kernel. It can also be
used to partition the set of exported symbols for documentation
purposes; for example, a set of symbols that is really only used for
debugging could be in a "SUBSYSTEM_DEBUG" namespace.

I continued the work mainly done by Martijn Coenen. In this v2 the
following changes have been introduced compared to v1 of this series:

- Rather than adding and evaluating separate sections __knsimport_NS,
  use modinfo tags to declare the namespaces a module introduces.
  Adjust modpost and the module loader accordingly.
- Also add support for reading multiple modinfo values for the same tag
  to allow list-like access to modinfo tags.
- The macros in export.h have been cleaned up to avoid redundancy in the
  macro parameters (ns, nspost, nspost2).
- The introduction of relative references in the ksymtab entries caused
  a rework of the macros to accommodate that configuration as well.
- Alignment of kernel_symbol in the ksymtab needed to be fixed to allow
  growing the kernel_symbol struct.
- Modpost does now also append the namespace suffix to the symbol
  entries in Module.symvers.
- The configuration option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS allows
  relaxing the enforcement of properly declared namespace imports at module
  loading time.
- Symbols can be collectively exported into a namespace by defining
  DEFAULT_SYMBOL_NAMESPACE in the corresponding Makefile.
- The requirement for a very recent coccinelle spatch has been lifted by
  simplifying the script.
- nsdeps does now ensures MODULE_IMPORT_NS statements are sorted when
  patching the module source files.
- Some minor bugs have been addressed in nsdeps to allow it to work with
  modules that have more than one source file.
- The RFC for the usb-storage symbols has been simplified by using
  DEFAULT_SYMBOL_NAMESPACE=USB_STORAGE rather than explicitly exporting each
  and every symbol into that new namespace.

This patch series was developed against v5.3-rc4.

[1] git grep "^EXPORT_SYMBOL\w*(" v5.3-rc4 | wc -l
[2] git grep "^EXPORT_SYMBOL\w*(" v4.17    | wc -l

Matthias Maennich (10):
  module: support reading multiple values per modinfo tag
  export: explicitly align struct kernel_symbol
  module: add support for symbol namespaces.
  modpost: add support for symbol namespaces
  module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
  export: allow definition default namespaces in Makefiles or sources
  modpost: add support for generating namespace dependencies
  scripts: Coccinelle script for namespace dependencies.
  usb-storage: remove single-use define for debugging
  RFC: usb-storage: export symbols in USB_STORAGE namespace

 MAINTAINERS                                 |   5 +
 Makefile                                    |  12 ++
 arch/m68k/include/asm/export.h              |   1 -
 drivers/usb/storage/Makefile                |   2 +
 drivers/usb/storage/alauda.c                |   1 +
 drivers/usb/storage/cypress_atacb.c         |   1 +
 drivers/usb/storage/datafab.c               |   1 +
 drivers/usb/storage/debug.h                 |   2 -
 drivers/usb/storage/ene_ub6250.c            |   1 +
 drivers/usb/storage/freecom.c               |   1 +
 drivers/usb/storage/isd200.c                |   1 +
 drivers/usb/storage/jumpshot.c              |   1 +
 drivers/usb/storage/karma.c                 |   1 +
 drivers/usb/storage/onetouch.c              |   1 +
 drivers/usb/storage/realtek_cr.c            |   1 +
 drivers/usb/storage/scsiglue.c              |   2 +-
 drivers/usb/storage/sddr09.c                |   1 +
 drivers/usb/storage/sddr55.c                |   1 +
 drivers/usb/storage/shuttle_usbat.c         |   1 +
 drivers/usb/storage/uas.c                   |   1 +
 include/asm-generic/export.h                |  14 +-
 include/linux/export.h                      |  92 +++++++++++--
 include/linux/module.h                      |   2 +
 init/Kconfig                                |  14 ++
 kernel/module.c                             |  67 ++++++++-
 scripts/Makefile.modpost                    |   4 +-
 scripts/coccinelle/misc/add_namespace.cocci |  23 ++++
 scripts/mod/modpost.c                       | 144 ++++++++++++++++++--
 scripts/mod/modpost.h                       |   9 ++
 scripts/nsdeps                              |  54 ++++++++
 30 files changed, 421 insertions(+), 40 deletions(-)
 create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
 create mode 100644 scripts/nsdeps

-- 
2.23.0.rc1.153.gdeed80330f-goog


       reply index

Thread overview: 105+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20180716122125.175792-1-maco@android.com>
2019-08-13 12:16 ` Matthias Maennich [this message]
2019-08-13 12:16   ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-13 12:40     ` Greg KH
2019-08-13 12:16   ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-13 12:41     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich
2019-08-13 15:26     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 04/10] modpost: " Matthias Maennich
2019-08-13 15:27     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-13 18:17     ` Greg KH
2019-08-13 20:15     ` Saravana Kannan
2019-08-14 12:54       ` Matthias Maennich
2019-08-14 17:34         ` Saravana Kannan
2019-08-13 12:17   ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-13 18:16     ` Greg KH
2019-08-13 18:16     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-13 18:21     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
2019-08-13 12:31     ` Julia Lawall
2019-08-13 12:44     ` Greg KH
2019-08-14  6:36     ` [Cocci] " Himanshu Jha
2019-08-14  8:03       ` Matthias Maennich
2019-08-14 12:00     ` [v2 " Markus Elfring
2019-08-15 13:50     ` Markus Elfring
2019-08-22  9:18       ` Matthias Maennich
2019-08-22 11:00         ` Markus Elfring
2019-08-13 12:17   ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-13 12:42     ` Greg KH
2019-08-13 13:12       ` Greg KH
2019-08-13 12:17   ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-13 12:45     ` Greg KH
2019-08-13 12:47     ` Greg KH
2019-08-13 15:02       ` Matthias Maennich
     [not found]   ` <20190821114955.12788-1-maennich@google.com>
2019-08-21 11:49     ` [PATCH v3 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 03/11] module: add support for symbol namespaces Matthias Maennich
2019-08-27 15:37       ` Jessica Yu
2019-08-27 16:04         ` Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 04/11] modpost: " Matthias Maennich
2019-08-26 16:21       ` Jessica Yu
2019-08-27 14:41         ` Matthias Maennich
2019-08-28  9:43           ` Jessica Yu
2019-08-28  9:55             ` Matthias Maennich
2019-08-28 10:16               ` Jessica Yu
2019-08-21 11:49     ` [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-28 10:49       ` Jessica Yu
2019-08-28 10:56         ` Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 08/11] scripts: Coccinelle script for " Matthias Maennich
2019-08-22  6:09       ` [v3 " Markus Elfring
2019-08-29 12:13       ` [PATCH v3 " Jessica Yu
2019-08-21 11:49     ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-21 12:37       ` Greg KH
2019-08-21 13:21       ` Thomas Gleixner
2019-08-21 13:32         ` Greg KH
2019-08-21 11:49     ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-21 12:38       ` Greg KH
2019-08-21 14:36         ` Jessica Yu
2019-08-21 23:13       ` Christoph Hellwig
2019-08-22  8:32         ` Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 00/12] Symbol Namespaces Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 01/12] module: support reading multiple values per modinfo tag Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 02/12] export: explicitly align struct kernel_symbol Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 03/12] module: add support for symbol namespaces Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 04/12] modpost: " Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 05/12] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 06/12] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 07/12] modpost: add support for generating namespace dependencies Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 08/12] scripts: Coccinelle script for " Matthias Maennich
2019-09-04  9:53     ` Masahiro Yamada
2019-09-05 14:46       ` Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 09/12] docs: Add documentation for Symbol Namespaces Matthias Maennich
2019-09-04  7:16     ` Greg KH
2019-09-03 15:06   ` [PATCH v4 10/12] usb-storage: remove single-use define for debugging Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 11/12] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 12/12] RFC: watchdog: export core symbols in WATCHDOG_CORE namespace Matthias Maennich
2019-09-03 16:10     ` Guenter Roeck
2019-09-04  8:45       ` Masahiro Yamada
2019-09-04 12:12         ` Guenter Roeck
2019-09-04 16:16           ` [usb-storage] " Matthew Dharm
2019-09-05 10:41             ` Jessica Yu
2019-09-05 10:52               ` Arnd Bergmann
2019-09-05 11:16                 ` Jessica Yu
2019-09-05 11:25                   ` Masahiro Yamada
2019-09-05 12:00                     ` Greg Kroah-Hartman
2019-09-05 11:25               ` Matthias Maennich
2019-09-04  9:37   ` [PATCH v4 00/12] Symbol Namespaces Masahiro Yamada
2019-09-06 10:32 ` [PATCH v5 00/11] " Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 03/11] module: add support for symbol namespaces Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 04/11] modpost: " Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 08/11] scripts: Coccinelle script for " Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 09/11] docs: Add documentation for Symbol Namespaces Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 10/11] usb-storage: remove single-use define for debugging Matthias Maennich
2019-09-06 12:59     ` Jessica Yu
2019-09-06 13:22       ` Greg KH
2019-09-06 10:32   ` [PATCH v5 11/11] usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-09-09  8:35   ` [PATCH v5 00/11] Symbol Namespaces Jessica Yu

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190813121733.52480-1-maennich@google.com \
    --to=maennich@google.com \
    --cc=Jonathan.Cameron@huawei.com \
    --cc=adrian@lisas.de \
    --cc=akpm@linux-foundation.org \
    --cc=ard.biesheuvel@linaro.org \
    --cc=arnd@arndb.de \
    --cc=cocci@systeme.lip6.fr \
    --cc=dan.j.williams@intel.com \
    --cc=davem@davemloft.net \
    --cc=dhowells@redhat.com \
    --cc=geert@linux-m68k.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hannes@cmpxchg.org \
    --cc=hpa@zytor.com \
    --cc=jeyu@kernel.org \
    --cc=joel@joelfernandes.org \
    --cc=kernel-team@android.com \
    --cc=kstewart@linuxfoundation.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-m68k@lists.linux-m68k.org \
    --cc=linux-modules@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=lucas.de.marchi@gmail.com \
    --cc=maco@android.com \
    --cc=maco@google.com \
    --cc=mchehab+samsung@kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=mingo@kernel.org \
    --cc=mingo@redhat.com \
    --cc=mpe@ellerman.id.au \
    --cc=nicolas.ferre@microchip.com \
    --cc=oneukum@suse.com \
    --cc=patrick.bellasi@arm.com \
    --cc=pombredanne@nexb.com \
    --cc=rgb@redhat.com \
    --cc=sam@ravnborg.org \
    --cc=sboyd@codeaurora.org \
    --cc=sspatil@google.com \
    --cc=stern@rowland.harvard.edu \
    --cc=tglx@linutronix.de \
    --cc=usb-storage@lists.one-eyed-alien.net \
    --cc=will@kernel.org \
    --cc=x86@kernel.org \
    --cc=yamada.masahiro@socionext.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-USB Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-usb/0 linux-usb/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-usb linux-usb/ https://lore.kernel.org/linux-usb \
		linux-usb@vger.kernel.org
	public-inbox-index linux-usb

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-usb


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git