[PATCH 0/3] diagnose unknown escapes after preprocessing

[PATCH 0/3] diagnose unknown escapes after preprocessing

[Luc Van Oostenryck]

This series aims to solve the wrong warning recently 
received when using sparse on Linux's kernel for arm64. 
It quite straightforwardly move the diagnostic of
unknown escape sequence together with escape conversion,
after preprocessing.

Patch 1 is just a test case (and could be folded with patch 2)
Patch 2 is the fix in itself
Patch 3 remove potential cruft left by patch 2.


Luc Van Oostenryck (3):
  add testcase for wrong early escape conversion
  warn on unknown escapes after preprocessing
  remove 'Escape' from token character class

 char.c                                 |  6 ++++++
 tokenize.c                             | 28 +++++++++++-----------------
 validation/escapes.c                   |  2 +-
 validation/preprocessor/early-escape.c | 22 ++++++++++++++++++++++
 4 files changed, 40 insertions(+), 18 deletions(-)
 create mode 100644 validation/preprocessor/early-escape.c

-- 
2.11.0

[PATCH 1/3] add testcase for wrong early escape conversion

[Luc Van Oostenryck]

Reported-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
 validation/preprocessor/early-escape.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 validation/preprocessor/early-escape.c

diff --git a/validation/preprocessor/early-escape.c b/validation/preprocessor/early-escape.c
new file mode 100644
index 000000000..c7beba5d8
--- /dev/null
+++ b/validation/preprocessor/early-escape.c
@@ -0,0 +1,23 @@
+#if 0
+"\l"
+#endif
+
+/*
+ * check-description:
+ *	Following the C standard, escape conversion must be
+ *	done in phase 5, just after preprocessing and just
+ *	before string concatenation. So we're not supposed
+ *	to receive a diagnostic for an unknown escape char
+ *	for a token which is excluded by the preprocessor.
+ * check-name: early-escape
+ * check-command: sparse -E $file
+ * check-known-to-fail
+ *
+ * check-output-start
+
+
+ * check-output-end
+ *
+ * check-error-start
+ * check-error-end
+ */
-- 
2.11.0

[PATCH 2/3] warn on unknown escapes after preprocessing

[Luc Van Oostenryck]

Following the C standards conversion of escaped characters
must be done after preprocessing, just before adjacent
string concatenation. This is what is done but escape
sequence were recognized already at tokenization phase
and a warning is given then when an unknonw escape
sequence was encountered.
But there is no reason to give such warning at this
earlier phase, manly because yhere is no reasons
to warn on things which will be discarded during
the preprocessing.

Fix this by moving the diagnostic of unknown escape
sequence together with the escape conversion.

Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
 char.c                                 | 6 ++++++
 tokenize.c                             | 3 ---
 validation/escapes.c                   | 2 +-
 validation/preprocessor/early-escape.c | 1 -
 4 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/char.c b/char.c
index 9bd3fc0db..e80fceec3 100644
--- a/char.c
+++ b/char.c
@@ -53,7 +53,13 @@ static const char *parse_escape(const char *p, unsigned *val, const char *end, i
 		break;
 	}
 	default:	/* everything else is left as is */
+		warning(pos, "unknown escape sequence: '\\%c'", c);
 		break;
+	case '\\':
+	case '\'':
+	case '"':
+	case '?':
+		break;	/* those are legal, so no warnings */
 	}
 	*val = c & ~((~0U << (bits - 1)) << 1);
 	return p;
diff --git a/tokenize.c b/tokenize.c
index f93ce46ff..632413f0c 100644
--- a/tokenize.c
+++ b/tokenize.c
@@ -615,9 +615,6 @@ static int eat_string(int next, stream_t *stream, enum token_type type)
 			want_hex = 0;
 			escape = next == '\\';
 		} else {
-			if (!(cclass[next + 1] & Escape))
-				warning(stream_pos(stream),
-					"Unknown escape '%c'", next);
 			escape = 0;
 			want_hex = next == 'x';
 		}
diff --git a/validation/escapes.c b/validation/escapes.c
index 5f526b9ca..604a37d33 100644
--- a/validation/escapes.c
+++ b/validation/escapes.c
@@ -18,10 +18,10 @@ static char d_oct[3] = "\141";
  * check-name: Character escape sequences
  *
  * check-error-start
-escapes.c:6:26: warning: Unknown escape 'c'
 escapes.c:3:34: warning: hex escape sequence out of range
 escapes.c:3:44: warning: hex escape sequence out of range
 escapes.c:4:18: warning: hex escape sequence out of range
+escapes.c:6:24: warning: unknown escape sequence: '\c'
 escapes.c:6:30: warning: multi-character character constant
 escapes.c:6:39: warning: multi-character character constant
 escapes.c:6:47: warning: hex escape sequence out of range
diff --git a/validation/preprocessor/early-escape.c b/validation/preprocessor/early-escape.c
index c7beba5d8..5ca5e8f42 100644
--- a/validation/preprocessor/early-escape.c
+++ b/validation/preprocessor/early-escape.c
@@ -11,7 +11,6 @@
  *	for a token which is excluded by the preprocessor.
  * check-name: early-escape
  * check-command: sparse -E $file
- * check-known-to-fail
  *
  * check-output-start
 
-- 
2.11.0

[PATCH 3/3] remove 'Escape' from token character class

[Luc Van Oostenryck]

Now that diagnostic on unknown escape sequence
have moved to post-preprocessing phase this is not more used.

Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@gmail.com>
---
 tokenize.c | 25 +++++++++++--------------
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/tokenize.c b/tokenize.c
index 632413f0c..bc97242b4 100644
--- a/tokenize.c
+++ b/tokenize.c
@@ -481,11 +481,10 @@ enum {
 	Dot = 16,
 	ValidSecond = 32,
 	Quote = 64,
-	Escape = 128,
 };
 
 static const long cclass[257] = {
-	['0' + 1 ... '7' + 1] = Digit | Hex | Escape,	/* \<octal> */
+	['0' + 1 ... '7' + 1] = Digit | Hex,	/* \<octal> */
 	['8' + 1 ... '9' + 1] = Digit | Hex,
 	['A' + 1 ... 'D' + 1] = Letter | Hex,
 	['E' + 1] = Letter | Hex | Exp,	/* E<exp> */
@@ -493,22 +492,22 @@ static const long cclass[257] = {
 	['G' + 1 ... 'O' + 1] = Letter,
 	['P' + 1] = Letter | Exp,	/* P<exp> */
 	['Q' + 1 ... 'Z' + 1] = Letter,
-	['a' + 1 ... 'b' + 1] = Letter | Hex | Escape, /* \a, \b */
+	['a' + 1 ... 'b' + 1] = Letter | Hex, /* \a, \b */
 	['c' + 1 ... 'd' + 1] = Letter | Hex,
-	['e' + 1] = Letter | Hex | Exp | Escape,/* \e, e<exp> */
-	['f' + 1] = Letter | Hex | Escape,	/* \f */
+	['e' + 1] = Letter | Hex | Exp,/* \e, e<exp> */
+	['f' + 1] = Letter | Hex,	/* \f */
 	['g' + 1 ... 'm' + 1] = Letter,
-	['n' + 1] = Letter | Escape,	/* \n */
+	['n' + 1] = Letter,	/* \n */
 	['o' + 1] = Letter,
 	['p' + 1] = Letter | Exp,	/* p<exp> */
 	['q' + 1] = Letter,
-	['r' + 1] = Letter | Escape,	/* \r */
+	['r' + 1] = Letter,	/* \r */
 	['s' + 1] = Letter,
-	['t' + 1] = Letter | Escape,	/* \t */
+	['t' + 1] = Letter,	/* \t */
 	['u' + 1] = Letter,
-	['v' + 1] = Letter | Escape,	/* \v */
+	['v' + 1] = Letter,	/* \v */
 	['w' + 1] = Letter,
-	['x' + 1] = Letter | Escape,	/* \x<hex> */
+	['x' + 1] = Letter,	/* \x<hex> */
 	['y' + 1 ... 'z' + 1] = Letter,
 	['_' + 1] = Letter,
 	['.' + 1] = Dot | ValidSecond,
@@ -520,10 +519,8 @@ static const long cclass[257] = {
 	['&' + 1] = ValidSecond,
 	['|' + 1] = ValidSecond,
 	['#' + 1] = ValidSecond,
-	['\'' + 1] = Quote | Escape,
-	['"' + 1] = Quote | Escape,
-	['\\' + 1] = Escape,
-	['?' + 1] = Escape,
+	['\'' + 1] = Quote,
+	['"' + 1] = Quote,
 };
 
 /*
-- 
2.11.0

Re: [PATCH 0/3] diagnose unknown escapes after preprocessing

[Christopher Li]

On Wed, Feb 8, 2017 at 1:45 AM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> This series aims to solve the wrong warning recently
> received when using sparse on Linux's kernel for arm64.
> It quite straightforwardly move the diagnostic of
> unknown escape sequence together with escape conversion,
> after preprocessing.
>
> Patch 1 is just a test case (and could be folded with patch 2)
> Patch 2 is the fix in itself
> Patch 3 remove potential cruft left by patch 2.

Looks great.

I already apply and push to sparse-next.

Chris

Re: [PATCH 0/3] diagnose unknown escapes after preprocessing

[Stephen Boyd]

On 02/07/2017 09:45 AM, Luc Van Oostenryck wrote:
> This series aims to solve the wrong warning recently 
> received when using sparse on Linux's kernel for arm64. 
> It quite straightforwardly move the diagnostic of
> unknown escape sequence together with escape conversion,
> after preprocessing.
>
> Patch 1 is just a test case (and could be folded with patch 2)
> Patch 2 is the fix in itself
> Patch 3 remove potential cruft left by patch 2.
>
>
> Luc Van Oostenryck (3):
>   add testcase for wrong early escape conversion
>   warn on unknown escapes after preprocessing
>   remove 'Escape' from token character class\

Thanks for the fix!

Tested-by: Stephen Boyd <sboyd@codeaurora.org>

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project