linux-modules.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Julia Lawall <julia.lawall@lip6.fr>
To: Matthias Maennich <maennich@google.com>
Cc: linux-kernel@vger.kernel.org, maco@android.com,
	kernel-team@android.com, arnd@arndb.de, geert@linux-m68k.org,
	gregkh@linuxfoundation.org, hpa@zytor.com, jeyu@kernel.org,
	joel@joelfernandes.org, kstewart@linuxfoundation.org,
	linux-arch@vger.kernel.org, linux-kbuild@vger.kernel.org,
	linux-m68k@vger.kernel.org, linux-modules@vger.kernel.org,
	linux-scsi@vger.kernel.org, linux-usb@vger.kernel.org,
	lucas.de.marchi@gmail.com, maco@google.com,
	michal.lkml@markovi.net, mingo@redhat.com, oneukum@suse.com,
	pombredanne@nexb.com, sam@ravnborg.org, sboyd@codeaurora.org,
	sspatil@google.com, stern@rowland.harvard.edu,
	tglx@linutronix.de, usb-storage@lists.one-eyed-alien.net,
	x86@kernel.org, yamada.masahiro@socionext.com,
	Julia Lawall <Julia.Lawall@lip6.fr>,
	Gilles Muller <Gilles.Muller@lip6.fr>,
	Nicolas Palix <nicolas.palix@imag.fr>,
	Mauro Carvalho Chehab <mchehab+samsung@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	Jonathan Cameron <Jonathan.Cameron@huawei.com>,
	Nicolas Ferre <nicolas.ferre@microchip.com>,
	cocci@systeme.lip6.fr
Subject: Re: [PATCH v2 08/10] scripts: Coccinelle script for namespace dependencies.
Date: Tue, 13 Aug 2019 14:31:18 +0200 (CEST)	[thread overview]
Message-ID: <alpine.DEB.2.21.1908131430530.4608@hadrien> (raw)
In-Reply-To: <20190813121733.52480-9-maennich@google.com>



On Tue, 13 Aug 2019, Matthias Maennich wrote:

> A script that uses the '<module>.ns_deps' file generated by modpost to
> automatically add the required symbol namespace dependencies to each
> module.
>
> Usage:
> 1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define
>    DEFAULT_SYMBOL_NAMESPACE
> 2) Run 'make' (or 'make modules') and get warnings about modules not
>    importing that namespace.
> 3) Run 'make nsdeps' to automatically add required import statements
>    to said modules.
>
> This makes it easer for subsystem maintainers to introduce and maintain
> symbol namespaces into their codebase.
>
> Co-developed-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Martijn Coenen <maco@android.com>
> Signed-off-by: Matthias Maennich <maennich@google.com>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>


> ---
>  MAINTAINERS                                 |  5 ++
>  Makefile                                    | 12 +++++
>  scripts/Makefile.modpost                    |  4 +-
>  scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++
>  scripts/nsdeps                              | 54 +++++++++++++++++++++
>  5 files changed, 97 insertions(+), 1 deletion(-)
>  create mode 100644 scripts/coccinelle/misc/add_namespace.cocci
>  create mode 100644 scripts/nsdeps
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e81e60bd7c26..aa169070a052 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -11414,6 +11414,11 @@ S:	Maintained
>  T:	git git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
>  F:	tools/include/nolibc/
>
> +NSDEPS
> +M:	Matthias Maennich <maennich@google.com>
> +S:	Maintained
> +F:	scripts/nsdeps
> +
>  NTB AMD DRIVER
>  M:	Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
>  L:	linux-ntb@googlegroups.com
> diff --git a/Makefile b/Makefile
> index 1b23f95db176..c5c3356e133c 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1500,6 +1500,9 @@ help:
>  	@echo  '  headerdep       - Detect inclusion cycles in headers'
>  	@echo  '  coccicheck      - Check with Coccinelle'
>  	@echo  ''
> +	@echo  'Tools:'
> +	@echo  '  nsdeps          - Generate missing symbol namespace dependencies'
> +	@echo  ''
>  	@echo  'Kernel selftest:'
>  	@echo  '  kselftest       - Build and run kernel selftest (run as root)'
>  	@echo  '                    Build, install, and boot kernel before'
> @@ -1687,6 +1690,15 @@ quiet_cmd_tags = GEN     $@
>  tags TAGS cscope gtags: FORCE
>  	$(call cmd,tags)
>
> +# Script to generate missing namespace dependencies
> +# ---------------------------------------------------------------------------
> +
> +PHONY += nsdeps
> +
> +nsdeps:
> +	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost nsdeps
> +	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/$@
> +
>  # Scripts to check various things for consistency
>  # ---------------------------------------------------------------------------
>
> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> index 26e6574ecd08..743fe3a2e885 100644
> --- a/scripts/Makefile.modpost
> +++ b/scripts/Makefile.modpost
> @@ -56,7 +56,8 @@ MODPOST = scripts/mod/modpost						\
>  	$(if $(KBUILD_EXTMOD),$(addprefix -e ,$(KBUILD_EXTRA_SYMBOLS)))	\
>  	$(if $(KBUILD_EXTMOD),-o $(modulesymfile))			\
>  	$(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E)			\
> -	$(if $(KBUILD_MODPOST_WARN),-w)
> +	$(if $(KBUILD_MODPOST_WARN),-w)					\
> +	$(if $(filter nsdeps,$(MAKECMDGOALS)),-d)
>
>  ifdef MODPOST_VMLINUX
>
> @@ -134,6 +135,7 @@ $(modules): %.ko :%.o %.mod.o FORCE
>
>  targets += $(modules)
>
> +nsdeps: __modpost
>
>  # Add FORCE to the prequisites of a target to force it to be always rebuilt.
>  # ---------------------------------------------------------------------------
> diff --git a/scripts/coccinelle/misc/add_namespace.cocci b/scripts/coccinelle/misc/add_namespace.cocci
> new file mode 100644
> index 000000000000..c832bb6445a8
> --- /dev/null
> +++ b/scripts/coccinelle/misc/add_namespace.cocci
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +//
> +/// Adds missing MODULE_IMPORT_NS statements to source files
> +///
> +/// This script is usually called from scripts/nsdeps with -D ns=<namespace> to
> +/// add a missing namespace tag to a module source file.
> +///
> +
> +@has_ns_import@
> +declarer name MODULE_IMPORT_NS;
> +identifier virtual.ns;
> +@@
> +MODULE_IMPORT_NS(ns);
> +
> +// Add missing imports, but only adjacent to a MODULE_LICENSE statement.
> +// That ensures we are adding it only to the main module source file.
> +@do_import depends on !has_ns_import@
> +declarer name MODULE_LICENSE;
> +expression license;
> +identifier virtual.ns;
> +@@
> +MODULE_LICENSE(license);
> ++ MODULE_IMPORT_NS(ns);
> diff --git a/scripts/nsdeps b/scripts/nsdeps
> new file mode 100644
> index 000000000000..148db65a830f
> --- /dev/null
> +++ b/scripts/nsdeps
> @@ -0,0 +1,54 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Linux kernel symbol namespace import generator
> +#
> +# This script requires at least spatch
> +# version 1.0.4.
> +SPATCH_REQ_VERSION="1.0.4"
> +
> +DIR="$(dirname $(readlink -f $0))/.."
> +SPATCH="`which ${SPATCH:=spatch}`"
> +if [ ! -x "$SPATCH" ]; then
> +    echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/'
> +    exit 1
> +fi
> +
> +SPATCH_REQ_VERSION_NUM=$(echo $SPATCH_REQ_VERSION | ${DIR}/scripts/ld-version.sh)
> +SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}')
> +SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh)
> +
> +if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then
> +    echo 'spatch needs to be version 1.06 or higher'
> +    exit 1
> +fi
> +
> +generate_deps_for_ns() {
> +    $SPATCH --very-quiet --in-place --sp-file \
> +	    $srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2
> +}
> +
> +generate_deps() {
> +    local mod_file=`echo $@ | sed -e 's/\.ns_deps/\.mod/'`
> +    local mod_name=`cat $mod_file | sed -n 1p | sed -e 's/\/[^.]*$//'`
> +    local mod_source_files=`cat $mod_file | sed -n 2p | sed -e 's/\.o/\.c/g'`
> +    for ns in `cat $@`; do
> +	echo "Adding namespace $ns to module $mod_name (if needed)."
> +        generate_deps_for_ns $ns $mod_source_files
> +	# sort the imports
> +        for source_file in $mod_source_files; do
> +            sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp
> +            offset=$(wc -l ${source_file}.tmp | awk '{print $1;}')
> +            cat $source_file | grep MODULE_IMPORT_NS | sort -u >> ${source_file}.tmp
> +            tail -n +$((offset +1)) ${source_file} | grep -v MODULE_IMPORT_NS >> ${source_file}.tmp
> +            if ! diff -q ${source_file} ${source_file}.tmp; then
> +                mv ${source_file}.tmp ${source_file}
> +            else
> +                rm ${source_file}.tmp
> +            fi
> +        done
> +    done
> +}
> +
> +for f in `find $srctree/.tmp_versions/ -name *.ns_deps`; do
> +    generate_deps $f
> +done
> --
> 2.23.0.rc1.153.gdeed80330f-goog
>
>

  reply	other threads:[~2019-08-13 12:31 UTC|newest]

Thread overview: 107+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20180716122125.175792-1-maco@android.com>
     [not found] ` <20180716122125.175792-3-maco@android.com>
     [not found]   ` <CAB0TPYEOVHcFGFLTjVvk7R0VSgSnXZRi1PjSYXONJSjqd4NewQ@mail.gmail.com>
     [not found]     ` <20180725155507.umb5oatduquxwlmt@linux-8ccs>
2018-07-25 16:48       ` [PATCH 2/6] module: add support for symbol namespaces Lucas De Marchi
2018-07-26  7:44         ` Martijn Coenen
2019-08-13 12:16 ` [PATCH v2 0/10] Symbol namespaces - RFC Matthias Maennich
2019-08-13 12:16   ` [PATCH v2 01/10] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-13 12:40     ` Greg KH
2019-08-13 12:16   ` [PATCH v2 02/10] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-13 12:41     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 03/10] module: add support for symbol namespaces Matthias Maennich
2019-08-13 15:26     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 04/10] modpost: " Matthias Maennich
2019-08-13 15:27     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 05/10] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-13 18:17     ` Greg KH
2019-08-13 20:15     ` Saravana Kannan
2019-08-14 12:54       ` Matthias Maennich
2019-08-14 17:34         ` Saravana Kannan
2019-08-13 12:17   ` [PATCH v2 06/10] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-13 18:16     ` Greg KH
2019-08-13 18:16     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 07/10] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-13 18:21     ` Greg KH
2019-08-13 12:17   ` [PATCH v2 08/10] scripts: Coccinelle script for " Matthias Maennich
2019-08-13 12:31     ` Julia Lawall [this message]
2019-08-13 12:44     ` Greg KH
2019-08-14  6:36     ` [Cocci] " Himanshu Jha
2019-08-14  8:03       ` Matthias Maennich
2019-08-14 12:00     ` [v2 " Markus Elfring
2019-08-15 13:50     ` Markus Elfring
2019-08-22  9:18       ` Matthias Maennich
2019-08-22 11:00         ` Markus Elfring
2019-08-13 12:17   ` [PATCH v2 09/10] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-13 12:42     ` Greg KH
2019-08-13 13:12       ` Greg KH
2019-08-13 12:17   ` [PATCH v2 10/10] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-13 12:45     ` Greg KH
2019-08-13 12:47     ` Greg KH
2019-08-13 15:02       ` Matthias Maennich
     [not found]   ` <20190821114955.12788-1-maennich@google.com>
2019-08-21 11:49     ` [PATCH v3 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 03/11] module: add support for symbol namespaces Matthias Maennich
2019-08-27 15:37       ` Jessica Yu
2019-08-27 16:04         ` Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 04/11] modpost: " Matthias Maennich
2019-08-26 16:21       ` Jessica Yu
2019-08-27 14:41         ` Matthias Maennich
2019-08-28  9:43           ` Jessica Yu
2019-08-28  9:55             ` Matthias Maennich
2019-08-28 10:16               ` Jessica Yu
2019-08-21 11:49     ` [PATCH v3 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-08-28 10:49       ` Jessica Yu
2019-08-28 10:56         ` Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
2019-08-21 11:49     ` [PATCH v3 08/11] scripts: Coccinelle script for " Matthias Maennich
2019-08-22  6:09       ` [v3 " Markus Elfring
2019-08-29 12:13       ` [PATCH v3 " Jessica Yu
2019-08-21 11:49     ` [PATCH v3 09/11] usb-storage: remove single-use define for debugging Matthias Maennich
2019-08-21 12:37       ` Greg KH
2019-08-21 13:21       ` Thomas Gleixner
2019-08-21 13:32         ` Greg KH
2019-08-21 11:49     ` [PATCH v3 10/11] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-08-21 12:38       ` Greg KH
2019-08-21 14:36         ` Jessica Yu
2019-08-21 23:13       ` Christoph Hellwig
2019-08-22  8:32         ` Matthias Maennich
2019-09-03 15:06 ` [PATCH v4 00/12] Symbol Namespaces Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 01/12] module: support reading multiple values per modinfo tag Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 02/12] export: explicitly align struct kernel_symbol Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 03/12] module: add support for symbol namespaces Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 04/12] modpost: " Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 05/12] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 06/12] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 07/12] modpost: add support for generating namespace dependencies Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 08/12] scripts: Coccinelle script for " Matthias Maennich
2019-09-04  9:53     ` Masahiro Yamada
2019-09-05 14:46       ` Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 09/12] docs: Add documentation for Symbol Namespaces Matthias Maennich
2019-09-04  7:16     ` Greg KH
2019-09-03 15:06   ` [PATCH v4 10/12] usb-storage: remove single-use define for debugging Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 11/12] RFC: usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-09-03 15:06   ` [PATCH v4 12/12] RFC: watchdog: export core symbols in WATCHDOG_CORE namespace Matthias Maennich
2019-09-03 16:10     ` Guenter Roeck
2019-09-04  8:45       ` Masahiro Yamada
2019-09-04 12:12         ` Guenter Roeck
2019-09-04 16:16           ` [usb-storage] " Matthew Dharm
2019-09-05 10:41             ` Jessica Yu
2019-09-05 10:52               ` Arnd Bergmann
2019-09-05 11:16                 ` Jessica Yu
2019-09-05 11:25                   ` Masahiro Yamada
2019-09-05 12:00                     ` Greg Kroah-Hartman
2019-09-05 11:25               ` Matthias Maennich
2019-09-04  9:37   ` [PATCH v4 00/12] Symbol Namespaces Masahiro Yamada
2019-09-06 10:32 ` [PATCH v5 00/11] " Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 01/11] module: support reading multiple values per modinfo tag Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 02/11] export: explicitly align struct kernel_symbol Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 03/11] module: add support for symbol namespaces Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 04/11] modpost: " Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 05/11] module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 06/11] export: allow definition default namespaces in Makefiles or sources Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 07/11] modpost: add support for generating namespace dependencies Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 08/11] scripts: Coccinelle script for " Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 09/11] docs: Add documentation for Symbol Namespaces Matthias Maennich
2019-09-06 10:32   ` [PATCH v5 10/11] usb-storage: remove single-use define for debugging Matthias Maennich
2019-09-06 12:59     ` Jessica Yu
2019-09-06 13:22       ` Greg KH
2019-09-06 10:32   ` [PATCH v5 11/11] usb-storage: export symbols in USB_STORAGE namespace Matthias Maennich
2019-09-09  8:35   ` [PATCH v5 00/11] Symbol Namespaces Jessica Yu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.21.1908131430530.4608@hadrien \
    --to=julia.lawall@lip6.fr \
    --cc=Gilles.Muller@lip6.fr \
    --cc=Jonathan.Cameron@huawei.com \
    --cc=arnd@arndb.de \
    --cc=cocci@systeme.lip6.fr \
    --cc=davem@davemloft.net \
    --cc=geert@linux-m68k.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hpa@zytor.com \
    --cc=jeyu@kernel.org \
    --cc=joel@joelfernandes.org \
    --cc=kernel-team@android.com \
    --cc=kstewart@linuxfoundation.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-m68k@vger.kernel.org \
    --cc=linux-modules@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=lucas.de.marchi@gmail.com \
    --cc=maco@android.com \
    --cc=maco@google.com \
    --cc=maennich@google.com \
    --cc=mchehab+samsung@kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=mingo@redhat.com \
    --cc=nicolas.ferre@microchip.com \
    --cc=nicolas.palix@imag.fr \
    --cc=oneukum@suse.com \
    --cc=pombredanne@nexb.com \
    --cc=sam@ravnborg.org \
    --cc=sboyd@codeaurora.org \
    --cc=sspatil@google.com \
    --cc=stern@rowland.harvard.edu \
    --cc=tglx@linutronix.de \
    --cc=usb-storage@lists.one-eyed-alien.net \
    --cc=x86@kernel.org \
    --cc=yamada.masahiro@socionext.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).