Coccinelle Archive on lore.kernel.org
 help / color / Atom feed
From: Jaskaran Singh <jaskaransingh7654321@gmail.com>
To: cocci@systeme.lip6.fr
Cc: linux-kernel-mentees@lists.linuxfoundation.org
Subject: [Cocci] [RFC PATCH 00/25] cocci: Improve C parsing of attributes
Date: Fri, 24 Apr 2020 14:47:36 +0530
Message-ID: <20200424091801.13871-1-jaskaransingh7654321@gmail.com> (raw)

This patch series aims to improve parsing of attributes in C by
Coccinelle's C parser.

These parsing errors were discovered by running a build of Coccinelle's
master branch on gitlab to parse the source code of Linux v5.5-rc4 and Linux
v5.6-rc7. Coccinelle currently manages attributes similar to comments, so
to explicity state what the attributes are to the C parser, a
MACROANNOTATION hint was used in Coccinelle's standard.h file. The hint
enables the parser to correctly identify attributes, and thus potentially
handle them differently in the AST.

Upon collection of these parsing errors, The GNU C Compiler's
grammar was taken as a reference for introducing attributes in correct
places in Coccinelle's C parser.

Upon applying the MACROANNOTATION hint to a majority of the attributes in
Coccinelle's standard.h file and parsing the source code of Linux
v5.6-rc7, the difference between the stats (of --parse-c) before and
after applying this patch series are as follows:

Before:

	NB total files = 28268; perfect = 25697; pbs = 2570; timeout = 0; =========> 90%
	nb good = 19160664,  nb passed = 135235 =========> 0.70% passed
	nb good = 19160664,  nb bad = 129685 =========> 99.33% good or passed

After:

	NB total files = 28268; perfect = 25699; pbs = 2568; timeout = 0; =========> 90%
	nb good = 19160856,  nb passed = 71716 =========> 0.37% passed
	nb good = 19160856,  nb bad = 129493 =========> 99.33% good or passed

Fixes for a few test case failures that arise from applying the new
productions are also included. The test cases that failed were:

	unstruct
	nocast

What this patch series does not do:
- Add attributes from all the new productions into the C AST. Some of
  these are dropped.
- Reflect changes from the C parser in the SmPL parser.

Separate patches will be sent for the above.

Jaskaran Singh (25):
      parsing_c: parse_c: Ignore TMacroAttr and TMacroEndAttr in passed tokens
      parsing_c: parsing_hacks: Label end attributes correctly
      parsing_c: parsing_hacks: Commentize attributes before qualif/type
      parsing_c: parser: Add rule for multiple end attributes
      parsing_c: parser: Add expression statement attributes
      parsing_c: parser: Add attribute production in spec_qualif_list
      parsing_c: parser: Add init_declarator_attrs rule
      parsing_c: parser: Add field declaration end attributes production
      parsing_c: parser: Handle struct/union/enum end attributes
      parsing_c: parser: Add MacroDecl end attributes production
      parsing_c: parser: cpp_other end attributes production
      parsing_c: cpp_token_c: Introduce MACROANNOTATION hint
      cocci: standard.h: Annotate attributes with MACROANNOTATION
      parsing_c: visitor_c: Add vk_attribute and vk_attribute_s
      parsing_c: lib_parsing_c: Add ii_of_attr
      engine: Add distrf_attr to PARAM functor
      engine: cocci_vs_c: Add allminus argument to attribute_list
      parsing_c: parser: Make abstract_declarator pass attributes
      cocci: Add parameter attributes to C AST
      parsing_c: parser: Place parameter attributes in C AST
      parsing_c: visitor_c: Visit parameter attributes
      parsing_c: unparse_hrule: Add parameter attributes in record
      parsing_c: type_c: Add parameter attributes to record
      engine: cocci_vs_c: "Match" parameter attributes
      engine: c_vs_c: Match parameter attributes

 engine/c_vs_c.ml            |   12 ++-
 engine/cocci_vs_c.ml        |   85 +++++++++++++++++-----
 engine/cocci_vs_c.mli       |    3 
 engine/pattern_c.ml         |    1 
 engine/transformation_c.ml  |    5 +
 ocaml/coccilib.mli          |    1 
 parsing_c/ast_c.ml          |    1 
 parsing_c/ast_c.mli         |    1 
 parsing_c/cpp_token_c.ml    |    2 
 parsing_c/lib_parsing_c.ml  |    1 
 parsing_c/lib_parsing_c.mli |    1 
 parsing_c/parse_c.ml        |    4 -
 parsing_c/parser_c.mly      |  165 +++++++++++++++++++++++++++++++++++++-------
 parsing_c/parsing_hacks.ml  |   37 +++++++++
 parsing_c/type_c.ml         |    1 
 parsing_c/unparse_hrule.ml  |    3 
 parsing_c/visitor_c.ml      |    7 +
 parsing_c/visitor_c.mli     |    3 
 standard.h                  |  116 +++++++++++++++---------------
 19 files changed, 342 insertions(+), 107 deletions(-)



_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

             reply index

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-24  9:17 Jaskaran Singh [this message]
2020-04-24  9:17 ` [Cocci] [RFC PATCH 01/25] parsing_c: parse_c: Ignore TMacroAttr and TMacroEndAttr in passed tokens Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 02/25] parsing_c: parsing_hacks: Label end attributes correctly Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 03/25] parsing_c: parsing_hacks: Commentize attributes before qualif/type Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 04/25] parsing_c: parser: Add rule for multiple end attributes Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 05/25] parsing_c: parser: Add expression statement attributes Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 06/25] parsing_c: parser: Add attribute production in spec_qualif_list Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 07/25] parsing_c: parser: Add init_declarator_attrs rule Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 08/25] parsing_c: parser: Add field declaration end attributes production Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 09/25] parsing_c: parser: Handle struct/union/enum end attributes Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 10/25] parsing_c: parser: Add MacroDecl end attributes production Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 11/25] parsing_c: parser: cpp_other " Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 12/25] parsing_c: cpp_token_c: Introduce MACROANNOTATION hint Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 13/25] cocci: standard.h: Annotate attributes with MACROANNOTATION Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 14/25] parsing_c: visitor_c: Add vk_attribute and vk_attribute_s Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 15/25] parsing_c: lib_parsing_c: Add ii_of_attr Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 16/25] engine: Add distrf_attr to PARAM functor Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 17/25] engine: cocci_vs_c: Add allminus argument to attribute_list Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 18/25] parsing_c: parser: Make abstract_declarator pass attributes Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 19/25] cocci: Add parameter attributes to C AST Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 20/25] parsing_c: parser: Place parameter attributes in " Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 21/25] parsing_c: visitor_c: Visit parameter attributes Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 22/25] parsing_c: unparse_hrule: Add parameter attributes in record Jaskaran Singh
2020-04-24  9:17 ` [Cocci] [RFC PATCH 23/25] parsing_c: type_c: Add parameter attributes to record Jaskaran Singh
2020-04-24  9:18 ` [Cocci] [RFC PATCH 24/25] engine: cocci_vs_c: "Match" parameter attributes Jaskaran Singh
2020-04-24  9:18 ` [Cocci] [RFC PATCH 25/25] engine: c_vs_c: Match " Jaskaran Singh
2020-04-24 12:02 [Cocci] [RFC PATCH 00/25] cocci: Improve C parsing of attributes Markus Elfring
2020-04-25 13:15 ` Jaskaran Singh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200424091801.13871-1-jaskaransingh7654321@gmail.com \
    --to=jaskaransingh7654321@gmail.com \
    --cc=cocci@systeme.lip6.fr \
    --cc=linux-kernel-mentees@lists.linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Coccinelle Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/cocci/0 cocci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 cocci cocci/ https://lore.kernel.org/cocci \
		cocci@systeme.lip6.fr
	public-inbox-index cocci

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/fr.lip6.systeme.cocci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git