From: Jessica Yu <jeyu@kernel.org>
To: Matthias Maennich <maennich@google.com>
Cc: linux-kernel@vger.kernel.org, kernel-team@android.com,
arnd@arndb.de, geert@linux-m68k.org, gregkh@linuxfoundation.org,
hpa@zytor.com, joel@joelfernandes.org,
kstewart@linuxfoundation.org, linux-arch@vger.kernel.org,
linux-kbuild@vger.kernel.org, linux-m68k@lists.linux-m68k.org,
linux-modules@vger.kernel.org, linux-scsi@vger.kernel.org,
linux-usb@vger.kernel.org, lucas.de.marchi@gmail.com,
maco@android.com, maco@google.com, michal.lkml@markovi.net,
mingo@redhat.com, oneukum@suse.com, pombredanne@nexb.com,
sam@ravnborg.org, sspatil@google.com, stern@rowland.harvard.edu,
tglx@linutronix.de, usb-storage@lists.one-eyed-alien.net,
x86@kernel.org, yamada.masahiro@socionext.com,
Julia Lawall <julia.lawall@lip6.fr>,
Gilles Muller <Gilles.Muller@lip6.fr>,
Nicolas Palix <nicolas.palix@imag.fr>,
Mauro Carvalho Chehab <mchehab+samsung@kernel.org>,
"David S. Miller" <davem@davemloft.net>,
Nicolas Ferre <nicolas.ferre@microchip.com>,
cocci@systeme.lip6.fr
Subject: Re: [PATCH v3 08/11] scripts: Coccinelle script for namespace dependencies.
Date: Thu, 29 Aug 2019 14:13:17 +0200 [thread overview]
Message-ID: <20190829121317.GA22914@linux-8ccs> (raw)
In-Reply-To: <20190821114955.12788-9-maennich@google.com>
+++ Matthias Maennich [21/08/19 12:49 +0100]:
>A script that uses the '<module>.ns_deps' files generated by modpost to
>automatically add the required symbol namespace dependencies to each
>module.
>
>Usage:
>1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
> DEFAULT_SYMBOL_NAMESPACE
>2) Run 'make' (or 'make modules') and get warnings about modules not
> importing that namespace.
>3) Run 'make nsdeps' to automatically add required import statements
> to said modules.
>
>This makes it easer for subsystem maintainers to introduce and maintain
>symbol namespaces into their codebase.
>
>Co-developed-by: Martijn Coenen <maco@android.com>
>Signed-off-by: Martijn Coenen <maco@android.com>
>Acked-by: Julia Lawall <julia.lawall@lip6.fr>
>Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>Signed-off-by: Matthias Maennich <maennich@google.com>
>---
> MAINTAINERS | 5 ++
> Makefile | 12 +++++
> scripts/Makefile.modpost | 4 +-
> scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++
> scripts/nsdeps | 56 +++++++++++++++++++++
> 5 files changed, 99 insertions(+), 1 deletion(-)
> create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
> create mode 100644 scripts/nsdeps
>
>diff --git a/MAINTAINERS b/MAINTAINERS
>index 08176d64eed5..dd5b37b49a07 100644
>--- a/MAINTAINERS
>+++ b/MAINTAINERS
>@@ -11428,6 +11428,11 @@ S: Maintained
> T: git git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
> F: tools/include/nolibc/
>
>+NSDEPS
>+M: Matthias Maennich <maennich@google.com>
>+S: Maintained
>+F: scripts/nsdeps
>+
> NTB AMD DRIVER
> M: Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
> L: linux-ntb@googlegroups.com
>diff --git a/Makefile b/Makefile
>index a89870188c09..40311f583ee1 100644
>--- a/Makefile
>+++ b/Makefile
>@@ -1500,6 +1500,9 @@ help:
> @echo ' headerdep - Detect inclusion cycles in headers'
> @echo ' coccicheck - Check with Coccinelle'
> @echo ''
>+ @echo 'Tools:'
>+ @echo ' nsdeps - Generate missing symbol namespace dependencies'
>+ @echo ''
> @echo 'Kernel selftest:'
> @echo ' kselftest - Build and run kernel selftest (run as root)'
> @echo ' Build, install, and boot kernel before'
>@@ -1687,6 +1690,15 @@ quiet_cmd_tags = GEN $@
> tags TAGS cscope gtags: FORCE
> $(call cmd,tags)
>
>+# Script to generate missing namespace dependencies
>+# ---------------------------------------------------------------------------
>+
>+PHONY += nsdeps
>+
>+nsdeps:
>+ $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost nsdeps
>+ $(Q)$(CONFIG_SHELL) $(srctree)/scripts/$@
>+
> # Scripts to check various things for consistency
> # ---------------------------------------------------------------------------
>
>diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
>index 26e6574ecd08..743fe3a2e885 100644
>--- a/scripts/Makefile.modpost
>+++ b/scripts/Makefile.modpost
>@@ -56,7 +56,8 @@ MODPOST = scripts/mod/modpost \
> $(if $(KBUILD_EXTMOD),$(addprefix -e ,$(KBUILD_EXTRA_SYMBOLS))) \
> $(if $(KBUILD_EXTMOD),-o $(modulesymfile)) \
> $(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E) \
>- $(if $(KBUILD_MODPOST_WARN),-w)
>+ $(if $(KBUILD_MODPOST_WARN),-w) \
>+ $(if $(filter nsdeps,$(MAKECMDGOALS)),-d)
>
> ifdef MODPOST_VMLINUX
>
>@@ -134,6 +135,7 @@ $(modules): %.ko :%.o %.mod.o FORCE
>
> targets += $(modules)
>
>+nsdeps: __modpost
>
> # Add FORCE to the prequisites of a target to force it to be always rebuilt.
> # ---------------------------------------------------------------------------
>diff --git a/scripts/coccinelle/misc/add_namespace.cocci b/scripts/coccinelle/misc/add_namespace.cocci
>new file mode 100644
>index 000000000000..c832bb6445a8
>--- /dev/null
>+++ b/scripts/coccinelle/misc/add_namespace.cocci
>@@ -0,0 +1,23 @@
>+// SPDX-License-Identifier: GPL-2.0-only
>+//
>+/// Adds missing MODULE_IMPORT_NS statements to source files
>+///
>+/// This script is usually called from scripts/nsdeps with -D ns=<namespace> to
>+/// add a missing namespace tag to a module source file.
>+///
>+
>+@has_ns_import@
>+declarer name MODULE_IMPORT_NS;
>+identifier virtual.ns;
>+@@
>+MODULE_IMPORT_NS(ns);
>+
>+// Add missing imports, but only adjacent to a MODULE_LICENSE statement.
>+// That ensures we are adding it only to the main module source file.
>+@do_import depends on !has_ns_import@
>+declarer name MODULE_LICENSE;
>+expression license;
>+identifier virtual.ns;
>+@@
>+MODULE_LICENSE(license);
>++ MODULE_IMPORT_NS(ns);
>diff --git a/scripts/nsdeps b/scripts/nsdeps
>new file mode 100644
>index 000000000000..3b5995a61e65
>--- /dev/null
>+++ b/scripts/nsdeps
>@@ -0,0 +1,56 @@
>+#!/bin/bash
>+# SPDX-License-Identifier: GPL-2.0
>+# Linux kernel symbol namespace import generator
>+#
>+# This script requires a minimum spatch version.
>+SPATCH_REQ_VERSION="1.0.4"
>+
>+DIR="$(dirname $(readlink -f $0))/.."
>+SPATCH="`which ${SPATCH:=spatch}`"
>+if [ ! -x "$SPATCH" ]; then
>+ echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/'
>+ exit 1
>+fi
>+
>+SPATCH_REQ_VERSION_NUM=$(echo $SPATCH_REQ_VERSION | ${DIR}/scripts/ld-version.sh)
>+SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}')
>+SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh)
>+
>+if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then
>+ echo "spatch needs to be version $SPATCH_REQ_VERSION or higher"
>+ exit 1
>+fi
>+
>+generate_deps_for_ns() {
>+ $SPATCH --very-quiet --in-place --sp-file \
>+ $srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2
>+}
>+
>+generate_deps() {
>+ local mod_name=`basename $@ .ko`
>+ local mod_file=`echo $@ | sed -e 's/\.ko/\.mod/'`
>+ local ns_deps_file=`echo $@ | sed -e 's/\.ko/\.ns_deps/'`
>+ if [ ! -f "$ns_deps_file" ]; then return; fi
>+ local mod_source_files=`cat $mod_file | sed -n 1p | sed -e 's/\.o/\.c/g'`
>+ for ns in `cat $ns_deps_file`; do
>+ echo "Adding namespace $ns to module $mod_name (if needed)."
>+ generate_deps_for_ns $ns $mod_source_files
>+ # sort the imports
>+ for source_file in $mod_source_files; do
>+ sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp
>+ offset=$(wc -l ${source_file}.tmp | awk '{print $1;}')
>+ cat $source_file | grep MODULE_IMPORT_NS | sort -u >> ${source_file}.tmp
>+ tail -n +$((offset +1)) ${source_file} | grep -v MODULE_IMPORT_NS >> ${source_file}.tmp
>+ if ! diff -q ${source_file} ${source_file}.tmp; then
>+ mv ${source_file}.tmp ${source_file}
>+ else
>+ rm ${source_file}.tmp
>+ fi
>+ done
>+ done
>+}
>+
>+for f in `cat $srctree/modules.order`; do
>+ generate_deps $f
>+done
Hi Matthias!
I normally build outside of the source tree (make O=..) and I think
that choked up the nsdeps script a bit. For example when I run
'make nsdeps O=/tmp/linux' I get:
cat: /home/jeyu/linux/modules.order: No such file or directory
I just changed $srctree/modules.order to $objtree/modules.order and
that fixed it for me. Also, I had to prefix $source_file in the script
with $srctree so that it'd find the right file to modify.
Thanks!
Jessica
next prev parent reply other threads:[~2019-08-29 12:13 UTC|newest]
Thread overview: 134+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-16 12:21 [PATCH 0/6] Symbol namespaces Martijn Coenen
2018-07-16 12:21 ` [PATCH 1/6] export: explicitly align struct kernel_symbol Martijn Coenen
2018-07-16 12:21 ` [PATCH 2/6] module: add support for symbol namespaces Martijn Coenen
2018-07-19 16:32 ` Jessica Yu
2018-07-20 7:54 ` Martijn Coenen
2018-07-20 14:49 ` Jessica Yu
2018-07-20 15:42 ` Martijn Coenen
2018-07-23 11:12 ` Jessica Yu
2018-07-24 7:44 ` Martijn Coenen
2018-07-24 7:56 ` Martijn Coenen
2018-07-25 15:55 ` Jessica Yu
2018-07-25 16:48 ` Lucas De Marchi
2018-07-26 7:44 ` Martijn Coenen
2018-07-16 12:21 ` [PATCH 3/6] modpost: add support for checking " Martijn Coenen
2018-07-16 12:21 ` [PATCH 4/6] modpost: add support for generating namespace dependencies Martijn Coenen
2018-07-23 6:49 ` Jessica Yu
2018-07-16 12:21 ` [PATCH 5/6] scripts: Coccinelle script for " Martijn Coenen
2018-07-16 12:21 ` [PATCH 6/6] RFC: USB: storage: move symbols into USB_STORAGE namespace Martijn Coenen
2018-07-17 15:04 ` Alan Stern
2018-07-18 13:28 ` Martijn Coenen
2018-07-16 15:33 ` [PATCH 0/6] Symbol namespaces Greg Kroah-Hartman
2018-07-23 14:28 ` Arnd Bergmann
2018-07-24 8:09 ` Martijn Coenen
2018-07-24 9:08 ` Arnd Bergmann
2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
2019-08-13 12:16 ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-13 12:40 ` Greg KH
2019-08-13 12:16 ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-13 12:41 ` Greg KH
2019-08-13 12:17 ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich
2019-08-13 15:26 ` Greg KH
2019-08-13 12:17 ` [PATCH v2 04/10] modpost: " Matthias Maennich
2019-08-13 15:27 ` Greg KH
2019-08-13 12:17 ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-13 18:17 ` Greg KH
2019-08-13 20:15 ` Saravana Kannan
2019-08-14 12:54 ` Matthias Maennich
2019-08-14 17:34 ` Saravana Kannan
2019-08-13 12:17 ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-13 18:16 ` Greg KH
2019-08-13 18:16 ` Greg KH
2019-08-13 12:17 ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-13 18:21 ` Greg KH
2019-08-13 12:17 ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
2019-08-13 12:31 ` Julia Lawall
2019-08-13 12:44 ` Greg KH
2019-08-14 6:36 ` [Cocci] " Himanshu Jha
2019-08-14 8:03 ` Matthias Maennich
2019-08-15 13:50 ` [v2 " Markus Elfring
2019-08-22 9:18 ` Matthias Maennich
2019-08-22 11:00 ` Markus Elfring
2019-08-13 12:17 ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-13 12:42 ` Greg KH
2019-08-13 13:12 ` Greg KH
2019-08-13 12:17 ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-13 12:45 ` Greg KH
2019-08-13 12:47 ` Greg KH
2019-08-13 15:02 ` Matthias Maennich
[not found] ` <20190821114955.12788-1-maennich@google.com>
2019-08-21 11:49 ` [PATCH v3 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-21 11:49 ` [PATCH v3 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-21 11:49 ` [PATCH v3 03/11] module: add support for symbol namespaces Matthias Maennich
2019-08-27 15:37 ` Jessica Yu
2019-08-27 16:04 ` Matthias Maennich
2019-08-21 11:49 ` [PATCH v3 04/11] modpost: " Matthias Maennich
2019-08-26 16:21 ` Jessica Yu
2019-08-27 14:41 ` Matthias Maennich
2019-08-28 9:43 ` Jessica Yu
2019-08-28 9:55 ` Matthias Maennich
2019-08-28 10:16 ` Jessica Yu
2019-08-21 11:49 ` [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-21 11:49 ` [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-28 10:49 ` Jessica Yu
2019-08-28 10:56 ` Matthias Maennich
2019-08-21 11:49 ` [PATCH v3 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-21 11:49 ` [PATCH v3 08/11] scripts: Coccinelle script for " Matthias Maennich
2019-08-22 6:09 ` [v3 " Markus Elfring
2019-08-29 12:13 ` Jessica Yu [this message]
2019-08-21 11:49 ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-21 12:37 ` Greg KH
2019-08-21 13:21 ` Thomas Gleixner
2019-08-21 13:32 ` Greg KH
2019-08-21 11:49 ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-21 12:38 ` Greg KH
2019-08-21 14:36 ` Jessica Yu
2019-08-21 23:13 ` Christoph Hellwig
2019-08-22 8:32 ` Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 00/12] Symbol Namespaces Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 01/12] module: support reading multiple values per modinfo tag Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 02/12] export: explicitly align struct kernel_symbol Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 03/12] module: add support for symbol namespaces Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 04/12] modpost: " Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 05/12] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 06/12] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 07/12] modpost: add support for generating namespace dependencies Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 08/12] scripts: Coccinelle script for " Matthias Maennich
2019-09-04 9:53 ` Masahiro Yamada
2019-09-05 14:46 ` Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 09/12] docs: Add documentation for Symbol Namespaces Matthias Maennich
2019-09-04 7:16 ` Greg KH
2019-09-03 15:06 ` [PATCH v4 10/12] usb-storage: remove single-use define for debugging Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 11/12] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 12/12] RFC: watchdog: export core symbols in WATCHDOG_CORE namespace Matthias Maennich
2019-09-03 16:10 ` Guenter Roeck
2019-09-04 8:45 ` Masahiro Yamada
2019-09-04 12:12 ` Guenter Roeck
2019-09-04 16:16 ` [usb-storage] " Matthew Dharm
2019-09-05 10:41 ` Jessica Yu
2019-09-05 10:52 ` Arnd Bergmann
2019-09-05 11:16 ` Jessica Yu
2019-09-05 11:25 ` Masahiro Yamada
2019-09-05 12:00 ` Greg Kroah-Hartman
2019-09-05 11:25 ` Matthias Maennich
2019-09-04 9:37 ` [PATCH v4 00/12] Symbol Namespaces Masahiro Yamada
2019-09-06 10:32 ` [PATCH v5 00/11] " Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 03/11] module: add support for symbol namespaces Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 04/11] modpost: " Matthias Maennich
2019-09-26 22:24 ` [PATCH] modpost: Copy namespace string into 'struct symbol' Shaun Ruffell
2019-09-27 5:31 ` Greg Kroah-Hartman
2019-09-27 8:03 ` Matthias Maennich
2019-09-30 21:20 ` Shaun Ruffell
2019-10-01 16:19 ` Matthias Maennich
2019-10-01 19:37 ` Shaun Ruffell
2019-09-06 10:32 ` [PATCH v5 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 08/11] scripts: Coccinelle script for " Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 09/11] docs: Add documentation for Symbol Namespaces Matthias Maennich
2019-09-06 10:32 ` [PATCH v5 10/11] usb-storage: remove single-use define for debugging Matthias Maennich
2019-09-06 12:59 ` Jessica Yu
2019-09-06 13:22 ` Greg KH
2019-09-06 10:32 ` [PATCH v5 11/11] usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-09-09 8:35 ` [PATCH v5 00/11] Symbol Namespaces Jessica Yu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190829121317.GA22914@linux-8ccs \
--to=jeyu@kernel.org \
--cc=Gilles.Muller@lip6.fr \
--cc=arnd@arndb.de \
--cc=cocci@systeme.lip6.fr \
--cc=davem@davemloft.net \
--cc=geert@linux-m68k.org \
--cc=gregkh@linuxfoundation.org \
--cc=hpa@zytor.com \
--cc=joel@joelfernandes.org \
--cc=julia.lawall@lip6.fr \
--cc=kernel-team@android.com \
--cc=kstewart@linuxfoundation.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-m68k@lists.linux-m68k.org \
--cc=linux-modules@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=linux-usb@vger.kernel.org \
--cc=lucas.de.marchi@gmail.com \
--cc=maco@android.com \
--cc=maco@google.com \
--cc=maennich@google.com \
--cc=mchehab+samsung@kernel.org \
--cc=michal.lkml@markovi.net \
--cc=mingo@redhat.com \
--cc=nicolas.ferre@microchip.com \
--cc=nicolas.palix@imag.fr \
--cc=oneukum@suse.com \
--cc=pombredanne@nexb.com \
--cc=sam@ravnborg.org \
--cc=sspatil@google.com \
--cc=stern@rowland.harvard.edu \
--cc=tglx@linutronix.de \
--cc=usb-storage@lists.one-eyed-alien.net \
--cc=x86@kernel.org \
--cc=yamada.masahiro@socionext.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).