[patch 0/2] LICENSES: Fix fallout from recent SPDX conversions

[patch 0/2] LICENSES: Fix fallout from recent SPDX conversions

[Thomas Gleixner]

Recent SPDX conversions introduced new wreckage:

 arch/sh/lib/ashiftrt.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 ...
 arch/sh/include/mach-ecovec24/mach/partner-jet-setup.txt: 1:38 Invalid token: "
 ,,,

The first group is due to using an exception identifier without adding the
exception text to the LICENSES directory.

The second one is due to the magic conment format in the SuperH bootcode
files.

At least the first one could have been caught by checkpatch.pl:

WARNING: 'SPDX-License-Identifier: GPL-2.0+ WITH GCC-exception-2.0' is not supported in LICENSES/...
#38: FILE: arch/sh/lib/ashiftrt.S:1:
+/* SPDX-License-Identifier: GPL-2.0+ WITH GCC-exception-2.0

....

The following series addresses both issues.

Thanks,

	tglx

[patch 1/2] LICENSES: Add GCC runtime library exception text

[Thomas Gleixner]

A recent commit added SPDX identifiers to the SuperH low level library code
which originates from GCC. This code is licensed under the GPL 2.0 or later
with the GCC runtime library exception.

Unfortunately the authors did not bother to add the exception text to the
LICENSES directory so spdxcheck fails with:

 arch/sh/lib/ashiftrt.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/ashlsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/ashrsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/lshrsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/movmem.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/udiv_qrnnd.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/udivsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/udivsi3_i4i-Os.S: 1:42 Invalid Exception ID: GCC-exception-2.0
 arch/sh/lib/udivsi3_i4i.S: 1:42 Invalid Exception ID: GCC-exception-2.0

Add the exception text along with the required tags which allow automated
checking.

Fixes: 4494ce4fb4ff ("sh: lib: convert to SPDX identifiers")
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Cc: Simon Horman <horms+renesas@verge.net.au>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kate Stewart <kstewart@linuxfoundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jonathan Corbet <corbet@lwn.net>
---
 LICENSES/exceptions/GCC-exception-2.0 |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

--- /dev/null
+++ b/LICENSES/exceptions/GCC-exception-2.0
@@ -0,0 +1,18 @@
+SPDX-Exception-Identifier: GCC-exception-2.0
+SPDX-URL: https://spdx.org/licenses/GCC-exception-2.0.html
+SPDX-Licenses: GPL-2.0, GPL-2.0+, GPL-2.0-only, GPL-2.0-or-later
+Usage-Guide:
+  This exception is used together with one of the above SPDX-Licenses to
+  allow linking the compiled version of code to non GPL compliant code.
+  To use this exception add it with the keyword WITH to one of the
+  identifiers in the SPDX-Licenses tag:
+    SPDX-License-Identifier: <SPDX-License> WITH GCC-exception-2.0
+License-Text:
+
+In addition to the permissions in the GNU Library General Public License,
+the Free Software Foundation gives you unlimited permission to link the
+compiled version of this file into combinations with other programs, and to
+distribute those programs without any restriction coming from the use of
+this file. (The General Public License restrictions do apply in other
+respects; for example, they cover modification of the file, and
+distribution when not linked into another program.)

[patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Thomas Gleixner]

The SuperH boot code files use a magic format for the SPDX identifier
comment:

  LIST "SPDX-License-Identifier: .... "

The trailing quotation mark is not stripped before the token parser is
invoked and causes the scan to fail. Handle it gracefully.

Fixes: 6a0abce4c4cc ("sh: include: convert to SPDX identifiers")
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Cc: Simon Horman <horms+renesas@verge.net.au>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kate Stewart <kstewart@linuxfoundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jonathan Corbet <corbet@lwn.net>
---
 scripts/spdxcheck.py |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

--- a/scripts/spdxcheck.py
+++ b/scripts/spdxcheck.py
@@ -175,7 +175,13 @@ import os
                 self.lines_checked += 1
                 if line.find("SPDX-License-Identifier:") < 0:
                     continue
-                expr = line.split(':')[1].replace('*/', '').strip()
+                expr = line.split(':')[1].strip()
+                # Remove trailing comment closure
+                if line.startswith('/*'):
+                    expr = expr.rstrip('*/').strip()
+                # Special case for SH magic boot code files
+                if line.startswith('LIST \"'):
+                    expr = expr.rstrip('\"').strip()
                 self.parse(expr)
                 self.spdx_valid += 1
                 #

Re: [patch 1/2] LICENSES: Add GCC runtime library exception text

[Greg Kroah-Hartman]

On Wed, Jan 16, 2019 at 11:26:52AM +0100, Thomas Gleixner wrote:
> A recent commit added SPDX identifiers to the SuperH low level library code
> which originates from GCC. This code is licensed under the GPL 2.0 or later
> with the GCC runtime library exception.
> 
> Unfortunately the authors did not bother to add the exception text to the
> LICENSES directory so spdxcheck fails with:
> 
>  arch/sh/lib/ashiftrt.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/ashlsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/ashrsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/lshrsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/movmem.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/udiv_qrnnd.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/udivsi3.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/udivsi3_i4i-Os.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  arch/sh/lib/udivsi3_i4i.S: 1:42 Invalid Exception ID: GCC-exception-2.0
> 
> Add the exception text along with the required tags which allow automated
> checking.
> 
> Fixes: 4494ce4fb4ff ("sh: lib: convert to SPDX identifiers")
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
> Cc: Simon Horman <horms+renesas@verge.net.au>
> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> Cc: Rich Felker <dalias@libc.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Kate Stewart <kstewart@linuxfoundation.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Jonathan Corbet <corbet@lwn.net>
> ---
>  LICENSES/exceptions/GCC-exception-2.0 |   18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)

Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Greg Kroah-Hartman]

On Wed, Jan 16, 2019 at 11:26:53AM +0100, Thomas Gleixner wrote:
> The SuperH boot code files use a magic format for the SPDX identifier
> comment:
> 
>   LIST "SPDX-License-Identifier: .... "
> 
> The trailing quotation mark is not stripped before the token parser is
> invoked and causes the scan to fail. Handle it gracefully.
> 
> Fixes: 6a0abce4c4cc ("sh: include: convert to SPDX identifiers")
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
> Cc: Simon Horman <horms+renesas@verge.net.au>
> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> Cc: Rich Felker <dalias@libc.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Kate Stewart <kstewart@linuxfoundation.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Jonathan Corbet <corbet@lwn.net>
> ---
>  scripts/spdxcheck.py |    8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> --- a/scripts/spdxcheck.py
> +++ b/scripts/spdxcheck.py
> @@ -175,7 +175,13 @@ import os
>                  self.lines_checked += 1
>                  if line.find("SPDX-License-Identifier:") < 0:
>                      continue
> -                expr = line.split(':')[1].replace('*/', '').strip()
> +                expr = line.split(':')[1].strip()
> +                # Remove trailing comment closure
> +                if line.startswith('/*'):
> +                    expr = expr.rstrip('*/').strip()
> +                # Special case for SH magic boot code files
> +                if line.startswith('LIST \"'):
> +                    expr = expr.rstrip('\"').strip()
>                  self.parse(expr)
>                  self.spdx_valid += 1
>                  #
> 
> 

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [patch 0/2] LICENSES: Fix fallout from recent SPDX conversions

[Jonathan Corbet]

On Wed, 16 Jan 2019 11:26:51 +0100
Thomas Gleixner <tglx@linutronix.de> wrote:

> Recent SPDX conversions introduced new wreckage:
> 
>  arch/sh/lib/ashiftrt.S: 1:42 Invalid Exception ID: GCC-exception-2.0
>  ...
>  arch/sh/include/mach-ecovec24/mach/partner-jet-setup.txt: 1:38 Invalid token: "
>  ,,,
> 
> The first group is due to using an exception identifier without adding the
> exception text to the LICENSES directory.
> 
> The second one is due to the magic conment format in the SuperH bootcode
> files.
> 
> At least the first one could have been caught by checkpatch.pl:
> 
> WARNING: 'SPDX-License-Identifier: GPL-2.0+ WITH GCC-exception-2.0' is not supported in LICENSES/...
> #38: FILE: arch/sh/lib/ashiftrt.S:1:
> +/* SPDX-License-Identifier: GPL-2.0+ WITH GCC-exception-2.0
> 
> ....
> 
> The following series addresses both issues.

I've applied both to the docs tree, thanks.

jon

Re: [patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Sven Eckelmann]

[-- Attachment #1: Type: text/plain, Size: 1192 bytes --]

On Wednesday, 16 January 2019 11.26.53 CET Thomas Gleixner wrote:
> The SuperH boot code files use a magic format for the SPDX identifier
> comment:
> 
>   LIST "SPDX-License-Identifier: .... "
> 
> The trailing quotation mark is not stripped before the token parser is
> invoked and causes the scan to fail. Handle it gracefully.
[...]

This patch introduces a false positive when checking files with an ANSI-C 
style /* SPDX-License-Identifier: .... */ comment line.

    $ ./scripts/checkpatch.pl -q -f include/linux/bug.h
    WARNING: 'SPDX-License-Identifier: GPL-2.0 */' is not supported in LICENSES/...
    #1: FILE: include/linux/bug.h:1:
    +/* SPDX-License-Identifier: GPL-2.0 */
    
    total: 0 errors, 1 warnings, 79 lines checked

checkpatch.pl is already stripping the "/* " prefix and only sends the 
remaining "SPDX-License-Identifier: GPL-2.0 */" via stdin to
scripts/spdxcheck.py. Thus the newly introduced check

> +                # Remove trailing comment closure
> +                if line.startswith('/*'):
> +                    expr = expr.rstrip('*/').strip()

doesn't match and thus the code doesn't remove the " */" at the end of the 
line.

Kind regards,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [B.A.T.M.A.N.] [patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Sven Eckelmann]

[-- Attachment #1: Type: text/plain, Size: 1192 bytes --]

On Wednesday, 16 January 2019 11.26.53 CET Thomas Gleixner wrote:
> The SuperH boot code files use a magic format for the SPDX identifier
> comment:
> 
>   LIST "SPDX-License-Identifier: .... "
> 
> The trailing quotation mark is not stripped before the token parser is
> invoked and causes the scan to fail. Handle it gracefully.
[...]

This patch introduces a false positive when checking files with an ANSI-C 
style /* SPDX-License-Identifier: .... */ comment line.

    $ ./scripts/checkpatch.pl -q -f include/linux/bug.h
    WARNING: 'SPDX-License-Identifier: GPL-2.0 */' is not supported in LICENSES/...
    #1: FILE: include/linux/bug.h:1:
    +/* SPDX-License-Identifier: GPL-2.0 */
    
    total: 0 errors, 1 warnings, 79 lines checked

checkpatch.pl is already stripping the "/* " prefix and only sends the 
remaining "SPDX-License-Identifier: GPL-2.0 */" via stdin to
scripts/spdxcheck.py. Thus the newly introduced check

> +                # Remove trailing comment closure
> +                if line.startswith('/*'):
> +                    expr = expr.rstrip('*/').strip()

doesn't match and thus the code doesn't remove the " */" at the end of the 
line.

Kind regards,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Joe Perches]

On Sun, 2019-01-20 at 16:32 +0100, Sven Eckelmann wrote:
> On Wednesday, 16 January 2019 11.26.53 CET Thomas Gleixner wrote:
> > The SuperH boot code files use a magic format for the SPDX identifier
> > comment:
> > 
> >   LIST "SPDX-License-Identifier: .... "
> > 
> > The trailing quotation mark is not stripped before the token parser is
> > invoked and causes the scan to fail. Handle it gracefully.
> [...]
> 
> This patch introduces a false positive when checking files with an ANSI-C 
> style /* SPDX-License-Identifier: .... */ comment line.
> 
>     $ ./scripts/checkpatch.pl -q -f include/linux/bug.h
>     WARNING: 'SPDX-License-Identifier: GPL-2.0 */' is not supported in LICENSES/...
>     #1: FILE: include/linux/bug.h:1:
>     +/* SPDX-License-Identifier: GPL-2.0 */
>     
>     total: 0 errors, 1 warnings, 79 lines checked
> 
> checkpatch.pl is already stripping the "/* " prefix and only sends the 
> remaining "SPDX-License-Identifier: GPL-2.0 */" via stdin to
> scripts/spdxcheck.py. Thus the newly introduced check
> 
> > +                # Remove trailing comment closure
> > +                if line.startswith('/*'):
> > +                    expr = expr.rstrip('*/').strip()
> 
> doesn't match and thus the code doesn't remove the " */" at the end of the 
> line.

Well, maybe checkpatch should remove the comment trailer.
---
Miscellanea:

o Indent a block properly too

 scripts/checkpatch.pl | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 155fa9305166..e0b542008256 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -3029,8 +3029,10 @@ sub process {
 				$checklicenseline = 2;
 			} elsif ($rawline =~ /^\+/) {
 				my $comment = "";
+				my $trailer = "";
 				if ($realfile =~ /\.(h|s|S)$/) {
 					$comment = '/*';
+					$trailer = '*/';
 				} elsif ($realfile =~ /\.(c|dts|dtsi)$/) {
 					$comment = '//';
 				} elsif (($checklicenseline == 2) || $realfile =~ /\.(sh|pl|py|awk|tc)$/) {
@@ -3044,11 +3046,12 @@ sub process {
 					 WARN("SPDX_LICENSE_TAG",
 					      "Missing or malformed SPDX-License-Identifier tag in line $checklicenseline\n" . $herecurr);
 				} elsif ($rawline =~ /(SPDX-License-Identifier: .*)/) {
-					 my $spdx_license = $1;
-					 if (!is_SPDX_License_valid($spdx_license)) {
-						  WARN("SPDX_LICENSE_TAG",
-						       "'$spdx_license' is not supported in LICENSES/...\n" . $herecurr);
-					 }
+					my $spdx_license = rtrim($1);
+					$spdx_license =~ s/\s*\Q$trailer\E$//;
+					if (!is_SPDX_License_valid($spdx_license)) {
+						WARN("SPDX_LICENSE_TAG",
+						     "'$spdx_license' is not supported in LICENSES/...\n" . $herecurr);
+					}
 				}
 			}
 		}

Re: [B.A.T.M.A.N.] [patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Joe Perches]

On Sun, 2019-01-20 at 16:32 +0100, Sven Eckelmann wrote:
> On Wednesday, 16 January 2019 11.26.53 CET Thomas Gleixner wrote:
> > The SuperH boot code files use a magic format for the SPDX identifier
> > comment:
> > 
> >   LIST "SPDX-License-Identifier: .... "
> > 
> > The trailing quotation mark is not stripped before the token parser is
> > invoked and causes the scan to fail. Handle it gracefully.
> [...]
> 
> This patch introduces a false positive when checking files with an ANSI-C 
> style /* SPDX-License-Identifier: .... */ comment line.
> 
>     $ ./scripts/checkpatch.pl -q -f include/linux/bug.h
>     WARNING: 'SPDX-License-Identifier: GPL-2.0 */' is not supported in LICENSES/...
>     #1: FILE: include/linux/bug.h:1:
>     +/* SPDX-License-Identifier: GPL-2.0 */
>     
>     total: 0 errors, 1 warnings, 79 lines checked
> 
> checkpatch.pl is already stripping the "/* " prefix and only sends the 
> remaining "SPDX-License-Identifier: GPL-2.0 */" via stdin to
> scripts/spdxcheck.py. Thus the newly introduced check
> 
> > +                # Remove trailing comment closure
> > +                if line.startswith('/*'):
> > +                    expr = expr.rstrip('*/').strip()
> 
> doesn't match and thus the code doesn't remove the " */" at the end of the 
> line.

Well, maybe checkpatch should remove the comment trailer.
---
Miscellanea:

o Indent a block properly too

 scripts/checkpatch.pl | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 155fa9305166..e0b542008256 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -3029,8 +3029,10 @@ sub process {
 				$checklicenseline = 2;
 			} elsif ($rawline =~ /^\+/) {
 				my $comment = "";
+				my $trailer = "";
 				if ($realfile =~ /\.(h|s|S)$/) {
 					$comment = '/*';
+					$trailer = '*/';
 				} elsif ($realfile =~ /\.(c|dts|dtsi)$/) {
 					$comment = '//';
 				} elsif (($checklicenseline == 2) || $realfile =~ /\.(sh|pl|py|awk|tc)$/) {
@@ -3044,11 +3046,12 @@ sub process {
 					 WARN("SPDX_LICENSE_TAG",
 					      "Missing or malformed SPDX-License-Identifier tag in line $checklicenseline\n" . $herecurr);
 				} elsif ($rawline =~ /(SPDX-License-Identifier: .*)/) {
-					 my $spdx_license = $1;
-					 if (!is_SPDX_License_valid($spdx_license)) {
-						  WARN("SPDX_LICENSE_TAG",
-						       "'$spdx_license' is not supported in LICENSES/...\n" . $herecurr);
-					 }
+					my $spdx_license = rtrim($1);
+					$spdx_license =~ s/\s*\Q$trailer\E$//;
+					if (!is_SPDX_License_valid($spdx_license)) {
+						WARN("SPDX_LICENSE_TAG",
+						     "'$spdx_license' is not supported in LICENSES/...\n" . $herecurr);
+					}
 				}
 			}
 		}

Re: [patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Sven Eckelmann]

[-- Attachment #1: Type: text/plain, Size: 323 bytes --]

On Sunday, 20 January 2019 19.39.55 CET Joe Perches wrote:
> Well, maybe checkpatch should remove the comment trailer.
> ---
> Miscellanea:

In case you are waiting for me (haven't seen it popping up in linux-next): 
yes, this patch solves the problem for me.

Tested-by: Sven Eckelmann <sven@narfation.org>

Thanks,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [B.A.T.M.A.N.] [patch 2/2] scripts/spdxcheck.py: Handle special quotation mark comments

[Sven Eckelmann]

[-- Attachment #1: Type: text/plain, Size: 323 bytes --]

On Sunday, 20 January 2019 19.39.55 CET Joe Perches wrote:
> Well, maybe checkpatch should remove the comment trailer.
> ---
> Miscellanea:

In case you are waiting for me (haven't seen it popping up in linux-next): 
yes, this patch solves the problem for me.

Tested-by: Sven Eckelmann <sven@narfation.org>

Thanks,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]