From: Masahiro Yamada <yamada.masahiro@socionext.com>
To: linux-kbuild@vger.kernel.org
Cc: Ulf Magnusson <ulfalizer@gmail.com>,
linux-kernel@vger.kernel.org,
Masahiro Yamada <yamada.masahiro@socionext.com>
Subject: [PATCH 06/27] kconfig: fix ambiguous grammar in terms of new lines
Date: Tue, 11 Dec 2018 20:00:49 +0900 [thread overview]
Message-ID: <1544526070-16690-7-git-send-email-yamada.masahiro@socionext.com> (raw)
In-Reply-To: <1544526070-16690-1-git-send-email-yamada.masahiro@socionext.com>
This commit decreases 8 shift/reduce conflicts.
A certain amount of grammatical ambiguity comes from how to reduce
excessive T_EOL tokens.
Let's take a look at the example code below:
1 config A
2 bool "a"
3
4 depends on B
5
6 config B
7 def_bool y
The line 3 is melt into "config_option_list", but the line 5 can be
either a part of "config_option_list" or "common_stmt" by itself.
Currently, the lexer converts '\n' to T_EOL verbatim. In Kconfig,
a new line is critical as a statement terminator, but new lines
in empty lines are not important since empty lines (or lines that
contain only whitespaces/comments) are just no-op.
If the lexer simply discards no-op lines, the parser will not be
bothered by excessive T_EOL tokens.
Of course, this means we are shifting the complexity from the parser
to the lexer, but it is much easier than tackling on shift/reduce
conflicts.
I introduced the second stage lexer to tweak the lexer.
Discard T_EOL if the previous token is T_EOL or T_HELPTEXT.
Two T_EOL tokens in a row is meaningless. T_HELPTEXT is a special
token that is reduced without T_EOL.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
scripts/kconfig/zconf.l | 21 +++++++++++++++++++++
scripts/kconfig/zconf.y | 18 +++---------------
2 files changed, 24 insertions(+), 15 deletions(-)
diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l
index b7bc164..847ba42 100644
--- a/scripts/kconfig/zconf.l
+++ b/scripts/kconfig/zconf.l
@@ -16,6 +16,8 @@
#include "lkc.h"
+#define YY_DECL static int yylex1(void)
+
#define START_STRSIZE 16
static struct {
@@ -23,6 +25,7 @@ static struct {
int lineno;
} current_pos;
+static int prev_token = T_EOL;
static char *text;
static int text_size, text_asize;
@@ -268,6 +271,24 @@ n [A-Za-z0-9_-]
}
%%
+
+/* second stage lexer */
+int yylex(void)
+{
+ int token;
+
+repeat:
+ token = yylex1();
+
+ /* Do not pass unneeded T_EOL to the parser. */
+ if ((prev_token == T_EOL || prev_token == T_HELPTEXT) && token == T_EOL)
+ goto repeat;
+
+ prev_token = token;
+
+ return token;
+}
+
static char *expand_token(const char *in, size_t n)
{
char *out;
diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y
index c28f1a8..02bfc62 100644
--- a/scripts/kconfig/zconf.y
+++ b/scripts/kconfig/zconf.y
@@ -31,7 +31,7 @@ struct symbol *symbol_hash[SYMBOL_HASHSIZE];
static struct menu *current_menu, *current_entry;
%}
-%expect 29
+%expect 21
%union
{
@@ -111,9 +111,7 @@ static struct menu *current_menu, *current_entry;
%}
%%
-input: nl start | start;
-
-start: mainmenu_stmt stmt_list | stmt_list;
+input: mainmenu_stmt stmt_list | stmt_list;
/* mainmenu entry */
@@ -141,8 +139,7 @@ option_name:
;
common_stmt:
- T_EOL
- | if_stmt
+ if_stmt
| comment_stmt
| config_stmt
| menuconfig_stmt
@@ -193,7 +190,6 @@ config_option_list:
| config_option_list depends
| config_option_list help
| config_option_list option_error
- | config_option_list T_EOL
;
config_option: T_TYPE prompt_stmt_opt T_EOL
@@ -293,7 +289,6 @@ choice_option_list:
| choice_option_list choice_option
| choice_option_list depends
| choice_option_list help
- | choice_option_list T_EOL
| choice_option_list option_error
;
@@ -443,7 +438,6 @@ help: help_start T_HELPTEXT
depends_list:
/* empty */
| depends_list depends
- | depends_list T_EOL
| depends_list option_error
;
@@ -458,7 +452,6 @@ depends: T_DEPENDS T_ON expr T_EOL
visibility_list:
/* empty */
| visibility_list visible
- | visibility_list T_EOL
;
visible: T_VISIBLE if_expr T_EOL
@@ -484,11 +477,6 @@ end: T_ENDMENU T_EOL { $$ = $1; }
| T_ENDIF T_EOL { $$ = $1; }
;
-nl:
- T_EOL
- | nl T_EOL
-;
-
if_expr: /* empty */ { $$ = NULL; }
| T_IF expr { $$ = $2; }
;
--
2.7.4
next prev parent reply other threads:[~2018-12-11 11:03 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-11 11:00 [PATCH 00/27] kconfig: remove all reduce/shift conflicts, refactor lexer, fix various issues Masahiro Yamada
2018-12-11 11:00 ` [PATCH 01/27] kconfig: fix file name and line number of warn_ignored_character() Masahiro Yamada
2018-12-11 11:00 ` [PATCH 02/27] kconfig: fix memory leak when EOF is encountered in quotation Masahiro Yamada
2018-12-11 11:00 ` [PATCH 03/27] kconfig: require T_EOL to reduce visible statement Masahiro Yamada
2018-12-11 11:00 ` [PATCH 04/27] kconfig: remove unneeded pattern matching to whitespaces Masahiro Yamada
2018-12-11 11:00 ` [PATCH 05/27] kconfig: refactor pattern matching in STRING state Masahiro Yamada
2018-12-11 11:00 ` Masahiro Yamada [this message]
2018-12-11 11:00 ` [PATCH 07/27] kconfig: clean up EOF handling in the lexer Masahiro Yamada
2018-12-11 11:00 ` [PATCH 08/27] kconfig: warn no new line at end of file Masahiro Yamada
2018-12-11 11:00 ` [PATCH 09/27] kconfig: remove grammatically ambiguous "unexpected option" diagnostic Masahiro Yamada
2018-12-11 11:00 ` [PATCH 10/27] kconfig: remove grammatically ambiguous option_error Masahiro Yamada
2018-12-11 11:00 ` [PATCH 11/27] kconfig: remove redundant if_block rule Masahiro Yamada
2018-12-11 11:00 ` [PATCH 12/27] kconfig: remove redundant menu_block rule Masahiro Yamada
2018-12-11 11:00 ` [PATCH 13/27] kconfig: loosen the order of "visible" and "depends on" in menu entry Masahiro Yamada
2018-12-11 11:00 ` [PATCH 14/27] kconfig: rename depends_list to comment_option_list Masahiro Yamada
2018-12-11 11:00 ` [PATCH 15/27] kconfig: remove redundant token defines Masahiro Yamada
2018-12-11 11:00 ` [PATCH 16/27] kconfig: use distinct tokens for type and default properties Masahiro Yamada
2018-12-11 11:01 ` [PATCH 17/27] kconfig: refactor scanning and parsing "option" properties Masahiro Yamada
2018-12-11 11:01 ` [PATCH 18/27] kconfig: use specific tokens instead of T_ASSIGN for assignments Masahiro Yamada
2018-12-11 11:01 ` [PATCH 19/27] kconfig: use T_WORD instead of T_VARIABLE for variables Masahiro Yamada
2018-12-11 11:01 ` [PATCH 20/27] microblaze: surround string default in Kconfig with double quotes Masahiro Yamada
2018-12-12 8:28 ` Michal Simek
2018-12-11 11:01 ` [PATCH 21/27] treewide: surround file paths in Kconfig files " Masahiro Yamada
2018-12-11 11:19 ` Wolfram Sang
2018-12-11 11:25 ` Geert Uytterhoeven
2018-12-11 14:43 ` Ingo Molnar
2018-12-11 11:01 ` [PATCH 22/27] kconfig: ban the use of '.' and '/' in unquoted words Masahiro Yamada
2018-12-11 11:01 ` [PATCH 23/27] kconfig: refactor end token rules Masahiro Yamada
2018-12-11 11:01 ` [PATCH 24/27] kconfig: stop associating kconf_id with yylval Masahiro Yamada
2018-12-11 11:01 ` [PATCH 25/27] kconfig: switch to ASSIGN_VAL state in the second lexer Masahiro Yamada
2018-12-11 11:01 ` [PATCH 26/27] kconfig: update current_pos " Masahiro Yamada
2018-12-11 11:01 ` [PATCH 27/27] kconfig: remove keyword lookup table entirely Masahiro Yamada
2018-12-19 14:59 ` [PATCH 00/27] kconfig: remove all reduce/shift conflicts, refactor lexer, fix various issues Masahiro Yamada
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1544526070-16690-7-git-send-email-yamada.masahiro@socionext.com \
--to=yamada.masahiro@socionext.com \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ulfalizer@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).