From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C48AC5CFFE for ; Tue, 11 Dec 2018 11:03:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 082D12082F for ; Tue, 11 Dec 2018 11:03:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nifty.com header.i=@nifty.com header.b="Na+UKrL6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 082D12082F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=socionext.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726455AbeLKLCp (ORCPT ); Tue, 11 Dec 2018 06:02:45 -0500 Received: from conuserg-11.nifty.com ([210.131.2.78]:36702 "EHLO conuserg-11.nifty.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726606AbeLKLBo (ORCPT ); Tue, 11 Dec 2018 06:01:44 -0500 Received: from pug.e01.socionext.com (p14092-ipngnfx01kyoto.kyoto.ocn.ne.jp [153.142.97.92]) (authenticated) by conuserg-11.nifty.com with ESMTP id wBBB1C5h017210; Tue, 11 Dec 2018 20:01:17 +0900 DKIM-Filter: OpenDKIM Filter v2.10.3 conuserg-11.nifty.com wBBB1C5h017210 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nifty.com; s=dec2015msa; t=1544526077; bh=G/ycLbPhkj1QR+lVIupeFYkdZAVqycrRJHc5wDj08Ec=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Na+UKrL6DVf5o+zTz2mscnsaOfrJg/RpbNiehbQPgf1+O9/dl0AO8EB0bnm84MXzi UoxIcRd2bkcrQAMoap9eh9FgE0MA2Y1GgndVnQ0+fqG1FG/mZv0NrfT2JN/3XhDuBa /HYby/XiCXzsI2EpEXncyyyVqlCRwPV6Y7w+HTXRtTdwwUTZ79Tzb57T+2oZW0b4gf 4gXaVcl1OAh1eZYaDXRKtJjJFdRgTcsVdRtmRBawetptn5kFNvkqPb5j65mRbCNmZJ bV++p3kPVJvf1bLFkTZBnYDCDS3uG2vOkbDPiEL76ZJqWI4FnHxYBEx1hIDDE4OLW9 6As1Z/EdpfQUA== X-Nifty-SrcIP: [153.142.97.92] From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: Ulf Magnusson , linux-kernel@vger.kernel.org, Masahiro Yamada Subject: [PATCH 06/27] kconfig: fix ambiguous grammar in terms of new lines Date: Tue, 11 Dec 2018 20:00:49 +0900 Message-Id: <1544526070-16690-7-git-send-email-yamada.masahiro@socionext.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1544526070-16690-1-git-send-email-yamada.masahiro@socionext.com> References: <1544526070-16690-1-git-send-email-yamada.masahiro@socionext.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This commit decreases 8 shift/reduce conflicts. A certain amount of grammatical ambiguity comes from how to reduce excessive T_EOL tokens. Let's take a look at the example code below: 1 config A 2 bool "a" 3 4 depends on B 5 6 config B 7 def_bool y The line 3 is melt into "config_option_list", but the line 5 can be either a part of "config_option_list" or "common_stmt" by itself. Currently, the lexer converts '\n' to T_EOL verbatim. In Kconfig, a new line is critical as a statement terminator, but new lines in empty lines are not important since empty lines (or lines that contain only whitespaces/comments) are just no-op. If the lexer simply discards no-op lines, the parser will not be bothered by excessive T_EOL tokens. Of course, this means we are shifting the complexity from the parser to the lexer, but it is much easier than tackling on shift/reduce conflicts. I introduced the second stage lexer to tweak the lexer. Discard T_EOL if the previous token is T_EOL or T_HELPTEXT. Two T_EOL tokens in a row is meaningless. T_HELPTEXT is a special token that is reduced without T_EOL. Signed-off-by: Masahiro Yamada --- scripts/kconfig/zconf.l | 21 +++++++++++++++++++++ scripts/kconfig/zconf.y | 18 +++--------------- 2 files changed, 24 insertions(+), 15 deletions(-) diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l index b7bc164..847ba42 100644 --- a/scripts/kconfig/zconf.l +++ b/scripts/kconfig/zconf.l @@ -16,6 +16,8 @@ #include "lkc.h" +#define YY_DECL static int yylex1(void) + #define START_STRSIZE 16 static struct { @@ -23,6 +25,7 @@ static struct { int lineno; } current_pos; +static int prev_token = T_EOL; static char *text; static int text_size, text_asize; @@ -268,6 +271,24 @@ n [A-Za-z0-9_-] } %% + +/* second stage lexer */ +int yylex(void) +{ + int token; + +repeat: + token = yylex1(); + + /* Do not pass unneeded T_EOL to the parser. */ + if ((prev_token == T_EOL || prev_token == T_HELPTEXT) && token == T_EOL) + goto repeat; + + prev_token = token; + + return token; +} + static char *expand_token(const char *in, size_t n) { char *out; diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y index c28f1a8..02bfc62 100644 --- a/scripts/kconfig/zconf.y +++ b/scripts/kconfig/zconf.y @@ -31,7 +31,7 @@ struct symbol *symbol_hash[SYMBOL_HASHSIZE]; static struct menu *current_menu, *current_entry; %} -%expect 29 +%expect 21 %union { @@ -111,9 +111,7 @@ static struct menu *current_menu, *current_entry; %} %% -input: nl start | start; - -start: mainmenu_stmt stmt_list | stmt_list; +input: mainmenu_stmt stmt_list | stmt_list; /* mainmenu entry */ @@ -141,8 +139,7 @@ option_name: ; common_stmt: - T_EOL - | if_stmt + if_stmt | comment_stmt | config_stmt | menuconfig_stmt @@ -193,7 +190,6 @@ config_option_list: | config_option_list depends | config_option_list help | config_option_list option_error - | config_option_list T_EOL ; config_option: T_TYPE prompt_stmt_opt T_EOL @@ -293,7 +289,6 @@ choice_option_list: | choice_option_list choice_option | choice_option_list depends | choice_option_list help - | choice_option_list T_EOL | choice_option_list option_error ; @@ -443,7 +438,6 @@ help: help_start T_HELPTEXT depends_list: /* empty */ | depends_list depends - | depends_list T_EOL | depends_list option_error ; @@ -458,7 +452,6 @@ depends: T_DEPENDS T_ON expr T_EOL visibility_list: /* empty */ | visibility_list visible - | visibility_list T_EOL ; visible: T_VISIBLE if_expr T_EOL @@ -484,11 +477,6 @@ end: T_ENDMENU T_EOL { $$ = $1; } | T_ENDIF T_EOL { $$ = $1; } ; -nl: - T_EOL - | nl T_EOL -; - if_expr: /* empty */ { $$ = NULL; } | T_IF expr { $$ = $2; } ; -- 2.7.4