[0/3] Minor parser fixes/cleanups

[0/3] Minor parser fixes/cleanups

[Herbert Xu]

Hi:

The recent discussion on backslash newline led me to some small
fixes and cleanups in the parser which I'm posting now.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[PATCH 1/3] [PARSER] Removed unnecessary pungetc on EOF from parser

[Herbert Xu]

Doing a pungetc on an EOF is a noop and is only useful when we
don't know what character we're putting back.  This patch removes
an unnecessary pungetc when we know it's EOF.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---

 ChangeLog    |    4 ++++
 src/parser.c |    2 --
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 5212a9a..70ccfed 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,7 @@
+2014-10-28  Herbert Xu <herbert@gondor.apana.org.au>
+
+	* Removed unnecessary pungetc on EOF from parser.
+
 2014-10-27  Herbert Xu <herbert@gondor.apana.org.au>
 
 	* Add printf support for format string a, A, and F.
diff --git a/src/parser.c b/src/parser.c
index f6c43be..f0c919d 100644
--- a/src/parser.c
+++ b/src/parser.c
@@ -208,8 +208,6 @@ list(int nlflag)
 		case TEOF:
 			if (heredoclist)
 				parseheredoc();
-			else
-				pungetc();		/* push back EOF on input */
 			tokpushback++;
 			return n1;
 		default:

[PATCH 2/3] [PARSER] Simplify EOF/newline handling in list parser

[Herbert Xu]

This patch simplifies the EOF and new handling in the list parser.
In particular, it eliminates a case where we may leave here-documents
unfinished upon EOF.

It also removes special EOF/newline handling from parsecmd.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---

 ChangeLog    |    1 
 src/parser.c |   60 +++++++++++++++++++++++++----------------------------------
 2 files changed, 27 insertions(+), 34 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 70ccfed..8e0d276 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,6 +1,7 @@
 2014-10-28  Herbert Xu <herbert@gondor.apana.org.au>
 
 	* Removed unnecessary pungetc on EOF from parser.
+	* Simplify EOF/newline handling in list parser.
 
 2014-10-27  Herbert Xu <herbert@gondor.apana.org.au>
 
diff --git a/src/parser.c b/src/parser.c
index f0c919d..382ddf2 100644
--- a/src/parser.c
+++ b/src/parser.c
@@ -135,19 +135,13 @@ static inline int realeofmark(const char *eofmark)
 union node *
 parsecmd(int interact)
 {
-	int t;
-
 	tokpushback = 0;
+	checkkwd = 0;
+	heredoclist = 0;
 	doprompt = interact;
 	if (doprompt)
 		setprompt(doprompt);
 	needprompt = 0;
-	t = readtoken();
-	if (t == TEOF)
-		return NEOF;
-	if (t == TNL)
-		return NULL;
-	tokpushback++;
 	return list(1);
 }
 
@@ -158,11 +152,27 @@ list(int nlflag)
 	union node *n1, *n2, *n3;
 	int tok;
 
-	checkkwd = CHKNL | CHKKWD | CHKALIAS;
-	if (nlflag == 2 && tokendlist[peektoken()])
-		return NULL;
 	n1 = NULL;
 	for (;;) {
+		switch (peektoken()) {
+		case TNL:
+			if (!(nlflag & 1))
+				break;
+			parseheredoc();
+			return n1;
+
+		case TEOF:
+			if (!n1 && (nlflag & 1))
+				n1 = NEOF;
+			parseheredoc();
+			return n1;
+		}
+
+		checkkwd = CHKNL | CHKKWD | CHKALIAS;
+		if (nlflag == 2 && tokendlist[peektoken()])
+			return n1;
+		nlflag |= 2;
+
 		n2 = andor();
 		tok = readtoken();
 		if (tok == TBACKGND) {
@@ -189,29 +199,15 @@ list(int nlflag)
 			n1 = n3;
 		}
 		switch (tok) {
-		case TBACKGND:
-		case TSEMI:
-			tok = readtoken();
-			/* fall through */
 		case TNL:
-			if (tok == TNL) {
-				parseheredoc();
-				if (nlflag == 1)
-					return n1;
-			} else {
-				tokpushback++;
-			}
-			checkkwd = CHKNL | CHKKWD | CHKALIAS;
-			if (tokendlist[peektoken()])
-				return n1;
-			break;
 		case TEOF:
-			if (heredoclist)
-				parseheredoc();
 			tokpushback++;
-			return n1;
+			/* fall through */
+		case TBACKGND:
+		case TSEMI:
+			break;
 		default:
-			if (nlflag == 1)
+			if ((nlflag & 1))
 				synexpect(-1);
 			tokpushback++;
 			return n1;
@@ -1443,10 +1439,6 @@ parsearith: {
 
 #ifdef mkinit
 INCLUDE "parser.h"
-RESET {
-	tokpushback = 0;
-	checkkwd = 0;
-}
 #endif
 
 

[PATCH 3/3] [PARSER] Catch variable length expansions on non-existant specials

[Herbert Xu]

Currently we only check special variable names that follow directly
after $ or ${.  So errors such as ${#&} are not caught.  This patch
fixes that by moving the is_special check to just before we print out
the special variable name.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---

 ChangeLog    |    4 ++++
 src/parser.c |   11 +++++++----
 2 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 8e0d276..6a64f74 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,7 @@
+2014-10-30  Herbert Xu <herbert@gondor.apana.org.au>
+
+	* Catch variable length expansions on non-existant specials.
+
 2014-10-28  Herbert Xu <herbert@gondor.apana.org.au>
 
 	* Removed unnecessary pungetc on EOF from parser.
diff --git a/src/parser.c b/src/parser.c
index 382ddf2..382658e 100644
--- a/src/parser.c
+++ b/src/parser.c
@@ -1229,8 +1229,7 @@ varname:
 				STPUTC(c, out);
 				c = pgetc_eatbnl();
 			} while (is_digit(c));
-		}
-		else if (is_special(c)) {
+		} else {
 			int cc = c;
 
 			c = pgetc_eatbnl();
@@ -1251,10 +1250,14 @@ varname:
 				}
 			}
 
+			if (!is_special(cc)) {
+				if (subtype == VSLENGTH)
+					subtype = 0;
+				goto badsub;
+			}
+
 			USTPUTC(cc, out);
 		}
-		else
-			goto badsub;
 
 		if (subtype == 0) {
 			switch (c) {