From: Jaskaran Singh <jaskaransingh7654321@gmail.com> To: cocci@systeme.lip6.fr Cc: linux-kernel-mentees@lists.linuxfoundation.org Subject: [Cocci] [PATCH v2 00/25] cocci: Improve C parsing of attributes Date: Thu, 28 May 2020 17:54:02 +0530 [thread overview] Message-ID: <20200528122428.4212-1-jaskaransingh7654321@gmail.com> (raw) These parsing errors were discovered by running a build of Coccinelle's master branch on gitlab to parse the source code of Linux v5.5-rc4 and Linux v5.6-rc7. Coccinelle currently manages attributes similar to comments, so to explicity state what the attributes are to the C parser, a MACROANNOTATION hint was used in Coccinelle's standard.h file. The hint enables the parser to correctly identify attributes, and thus potentially handle them differently in the AST. Upon collection of these parsing errors, The GNU C Compiler's grammar was taken as a reference for introducing attributes in correct places in Coccinelle's C parser. Upon applying the MACROANNOTATION hint to a majority of the attributes in Coccinelle's standard.h file and parsing the source code of Linux v5.6-rc7, the difference between the stats (of --parse-c) before and after applying this patch series are as follows: Before: NB total files = 28268; perfect = 25697; pbs = 2570; timeout = 0; =========> 90% nb good = 19160664, nb passed = 135235 =========> 0.70% passed nb good = 19160664, nb bad = 129685 =========> 99.33% good or passed After: NB total files = 28268; perfect = 25699; pbs = 2568; timeout = 0; =========> 90% nb good = 19160856, nb passed = 71716 =========> 0.37% passed nb good = 19160856, nb bad = 129493 =========> 99.33% good or passed Fixes for a few test case failures that arise from applying the new productions are also included. The test cases that failed were: unstruct nocast What this patch series does not do: - Add attributes from all the new productions into the C AST. Some of these are dropped. - Reflect changes from the C parser in the SmPL parser. Separate patches will be sent for the above. Changes in v2: -------------- parsing_c: parser: Add attribute production in spec_qualif_list - This patch applied on its own causes a build error. Fix the error by removing the let binding for abstract_declaratort in the type_name rule parsing_c: parser: Make abstract_declarator pass attributes - Reflect changes from the above build error fix. Jaskaran Singh (25): parsing_c: parse_c: Ignore TMacroAttr and TMacroEndAttr in passed tokens parsing_c: parsing_hacks: Label end attributes correctly parsing_c: parsing_hacks: Commentize attributes before qualif/type parsing_c: parser: Add rule for multiple end attributes parsing_c: parser: Add expression statement attributes parsing_c: parser: Add attribute production in spec_qualif_list parsing_c: parser: Add init_declarator_attrs rule parsing_c: parser: Add field declaration end attributes production parsing_c: parser: Handle struct/union/enum end attributes parsing_c: parser: Add MacroDecl end attributes production parsing_c: parser: cpp_other end attributes production parsing_c: cpp_token_c: Introduce MACROANNOTATION hint cocci: standard.h: Annotate attributes with MACROANNOTATION parsing_c: visitor_c: Add vk_attribute and vk_attribute_s parsing_c: lib_parsing_c: Add ii_of_attr engine: Add distrf_attr to PARAM functor engine: cocci_vs_c: Add allminus argument to attribute_list parsing_c: parser: Make abstract_declarator pass attributes cocci: Add parameter attributes to C AST parsing_c: parser: Place parameter attributes in C AST parsing_c: visitor_c: Visit parameter attributes parsing_c: unparse_hrule: Add parameter attributes in record parsing_c: type_c: Add parameter attributes to record engine: cocci_vs_c: "Match" parameter attributes engine: c_vs_c: Match parameter attributes engine/c_vs_c.ml | 12 ++- engine/cocci_vs_c.ml | 85 +++++++++++++++++----- engine/cocci_vs_c.mli | 3 engine/pattern_c.ml | 1 engine/transformation_c.ml | 5 + ocaml/coccilib.mli | 1 parsing_c/ast_c.ml | 1 parsing_c/ast_c.mli | 1 parsing_c/cpp_token_c.ml | 2 parsing_c/lib_parsing_c.ml | 1 parsing_c/lib_parsing_c.mli | 1 parsing_c/parse_c.ml | 4 - parsing_c/parser_c.mly | 167 +++++++++++++++++++++++++++++++++++++------- parsing_c/parsing_hacks.ml | 37 +++++++++ parsing_c/type_c.ml | 1 parsing_c/unparse_hrule.ml | 3 parsing_c/visitor_c.ml | 7 + parsing_c/visitor_c.mli | 3 standard.h | 116 +++++++++++++++--------------- 19 files changed, 343 insertions(+), 108 deletions(-) _______________________________________________ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
WARNING: multiple messages have this Message-ID (diff)
From: Jaskaran Singh <jaskaransingh7654321@gmail.com> To: cocci@systeme.lip6.fr Cc: julia.lawall@inria.fr, jaskaransingh7654321@gmail.com, linux-kernel-mentees@lists.linuxfoundation.org Subject: [Linux-kernel-mentees] [PATCH v2 00/25] cocci: Improve C parsing of attributes Date: Thu, 28 May 2020 17:54:02 +0530 [thread overview] Message-ID: <20200528122428.4212-1-jaskaransingh7654321@gmail.com> (raw) These parsing errors were discovered by running a build of Coccinelle's master branch on gitlab to parse the source code of Linux v5.5-rc4 and Linux v5.6-rc7. Coccinelle currently manages attributes similar to comments, so to explicity state what the attributes are to the C parser, a MACROANNOTATION hint was used in Coccinelle's standard.h file. The hint enables the parser to correctly identify attributes, and thus potentially handle them differently in the AST. Upon collection of these parsing errors, The GNU C Compiler's grammar was taken as a reference for introducing attributes in correct places in Coccinelle's C parser. Upon applying the MACROANNOTATION hint to a majority of the attributes in Coccinelle's standard.h file and parsing the source code of Linux v5.6-rc7, the difference between the stats (of --parse-c) before and after applying this patch series are as follows: Before: NB total files = 28268; perfect = 25697; pbs = 2570; timeout = 0; =========> 90% nb good = 19160664, nb passed = 135235 =========> 0.70% passed nb good = 19160664, nb bad = 129685 =========> 99.33% good or passed After: NB total files = 28268; perfect = 25699; pbs = 2568; timeout = 0; =========> 90% nb good = 19160856, nb passed = 71716 =========> 0.37% passed nb good = 19160856, nb bad = 129493 =========> 99.33% good or passed Fixes for a few test case failures that arise from applying the new productions are also included. The test cases that failed were: unstruct nocast What this patch series does not do: - Add attributes from all the new productions into the C AST. Some of these are dropped. - Reflect changes from the C parser in the SmPL parser. Separate patches will be sent for the above. Changes in v2: -------------- parsing_c: parser: Add attribute production in spec_qualif_list - This patch applied on its own causes a build error. Fix the error by removing the let binding for abstract_declaratort in the type_name rule parsing_c: parser: Make abstract_declarator pass attributes - Reflect changes from the above build error fix. Jaskaran Singh (25): parsing_c: parse_c: Ignore TMacroAttr and TMacroEndAttr in passed tokens parsing_c: parsing_hacks: Label end attributes correctly parsing_c: parsing_hacks: Commentize attributes before qualif/type parsing_c: parser: Add rule for multiple end attributes parsing_c: parser: Add expression statement attributes parsing_c: parser: Add attribute production in spec_qualif_list parsing_c: parser: Add init_declarator_attrs rule parsing_c: parser: Add field declaration end attributes production parsing_c: parser: Handle struct/union/enum end attributes parsing_c: parser: Add MacroDecl end attributes production parsing_c: parser: cpp_other end attributes production parsing_c: cpp_token_c: Introduce MACROANNOTATION hint cocci: standard.h: Annotate attributes with MACROANNOTATION parsing_c: visitor_c: Add vk_attribute and vk_attribute_s parsing_c: lib_parsing_c: Add ii_of_attr engine: Add distrf_attr to PARAM functor engine: cocci_vs_c: Add allminus argument to attribute_list parsing_c: parser: Make abstract_declarator pass attributes cocci: Add parameter attributes to C AST parsing_c: parser: Place parameter attributes in C AST parsing_c: visitor_c: Visit parameter attributes parsing_c: unparse_hrule: Add parameter attributes in record parsing_c: type_c: Add parameter attributes to record engine: cocci_vs_c: "Match" parameter attributes engine: c_vs_c: Match parameter attributes engine/c_vs_c.ml | 12 ++- engine/cocci_vs_c.ml | 85 +++++++++++++++++----- engine/cocci_vs_c.mli | 3 engine/pattern_c.ml | 1 engine/transformation_c.ml | 5 + ocaml/coccilib.mli | 1 parsing_c/ast_c.ml | 1 parsing_c/ast_c.mli | 1 parsing_c/cpp_token_c.ml | 2 parsing_c/lib_parsing_c.ml | 1 parsing_c/lib_parsing_c.mli | 1 parsing_c/parse_c.ml | 4 - parsing_c/parser_c.mly | 167 +++++++++++++++++++++++++++++++++++++------- parsing_c/parsing_hacks.ml | 37 +++++++++ parsing_c/type_c.ml | 1 parsing_c/unparse_hrule.ml | 3 parsing_c/visitor_c.ml | 7 + parsing_c/visitor_c.mli | 3 standard.h | 116 +++++++++++++++--------------- 19 files changed, 343 insertions(+), 108 deletions(-) _______________________________________________ Linux-kernel-mentees mailing list Linux-kernel-mentees@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
next reply other threads:[~2020-05-28 12:24 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-05-28 12:24 Jaskaran Singh [this message] 2020-05-28 12:24 ` [Linux-kernel-mentees] [PATCH v2 00/25] cocci: Improve C parsing of attributes Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 01/25] parsing_c: parse_c: Ignore TMacroAttr and TMacroEndAttr in passed tokens Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 02/25] parsing_c: parsing_hacks: Label end attributes correctly Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 03/25] parsing_c: parsing_hacks: Commentize attributes before qualif/type Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 04/25] parsing_c: parser: Add rule for multiple end attributes Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 05/25] parsing_c: parser: Add expression statement attributes Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 06/25] parsing_c: parser: Add attribute production in spec_qualif_list Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 07/25] parsing_c: parser: Add init_declarator_attrs rule Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 08/25] parsing_c: parser: Add field declaration end attributes production Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 09/25] parsing_c: parser: Handle struct/union/enum end attributes Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 10/25] parsing_c: parser: Add MacroDecl end attributes production Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 11/25] parsing_c: parser: cpp_other " Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 12/25] parsing_c: cpp_token_c: Introduce MACROANNOTATION hint Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 13/25] cocci: standard.h: Annotate attributes with MACROANNOTATION Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 14/25] parsing_c: visitor_c: Add vk_attribute and vk_attribute_s Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 15/25] parsing_c: lib_parsing_c: Add ii_of_attr Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 16/25] engine: Add distrf_attr to PARAM functor Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 17/25] engine: cocci_vs_c: Add allminus argument to attribute_list Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 18/25] parsing_c: parser: Make abstract_declarator pass attributes Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 19/25] cocci: Add parameter attributes to C AST Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 20/25] parsing_c: parser: Place parameter attributes in " Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 21/25] parsing_c: visitor_c: Visit parameter attributes Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 22/25] parsing_c: unparse_hrule: Add parameter attributes in record Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 23/25] parsing_c: type_c: Add parameter attributes to record Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 24/25] engine: cocci_vs_c: "Match" parameter attributes Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 12:24 ` [Cocci] [PATCH v2 25/25] engine: c_vs_c: Match " Jaskaran Singh 2020-05-28 12:24 ` [Linux-kernel-mentees] " Jaskaran Singh 2020-05-28 13:29 ` [Cocci] [PATCH v3 20/25] parsing_c: parser: Place parameter attributes in C AST Jaskaran Singh 2020-05-28 13:29 ` [Linux-kernel-mentees] " Jaskaran Singh
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200528122428.4212-1-jaskaransingh7654321@gmail.com \ --to=jaskaransingh7654321@gmail.com \ --cc=cocci@systeme.lip6.fr \ --cc=linux-kernel-mentees@lists.linuxfoundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.