[Cocci] [PATCH v2 0/3] cocci: Improve management of macros before typedefs

[Cocci] [PATCH v2 0/3] cocci: Improve management of macros before typedefs

[Jaskaran Singh]

This patch series is for improving the management of the following
case:

<macro> <typedef> <ident>

If <typedef> is a known typedef (such as u8 or *_t) then label <macro>
as a CppMacro. Subsequent cases will label <typedef> correctly as a
typedef ident.

Also add bool to the list of known typedefs so that cases of

<macro> bool <ident>

are handled correctly just like any known typedef will be.

Changes in v2:
--------------
- Group these patches as a series for clarity
- Add a test case for <macro> bool <ident> (Patch 3/3)
- Explain what has been improved with Patch 2/3

 parsing_c/parsing_hacks.ml    |   37 ++++++++++++++++++++++++++++++++++++-
 tests/macro_before_bool.c     |    4 ++++
 tests/macro_before_bool.cocci |    9 +++++++++
 tests/macro_before_bool.res   |    5 +++++
 4 files changed, 54 insertions(+), 1 deletion(-)


_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] [PATCH v2 1/3] parsing_c: Handle case of macro before typedef

[Jaskaran Singh]

For the following case:

<macro> <typedef> <identifier>

A case in parsing_hacks.ml sometimes mislabels <macro> as a
typedef ident.

If typedef is a known typedef (such as u8 or *_t), then label
<macro> as a CppMacro. Subsequent cases will then label <typedef>
correctly.

Following are results of --parse-c on the Linux Kernel v5.5-rc4:

Before:

  nb good = 18956150,  nb passed = 134061 =========> 0.70% passed

After:

  nb good = 18956150,  nb passed = 134062 =========> 0.70% passed

Signed-off-by: Jaskaran Singh <jaskaransingh7654321@gmail.com>
---
 parsing_c/parsing_hacks.ml | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/parsing_c/parsing_hacks.ml b/parsing_c/parsing_hacks.ml
index 8374731b..23d675cf 100644
--- a/parsing_c/parsing_hacks.ml
+++ b/parsing_c/parsing_hacks.ml
@@ -2184,6 +2184,41 @@ let lookahead2 ~pass next before =
       && ok_typedef s && is_macro s2 && is_type type_
         ->
 	  TIdent (s, i1)
+
+  (* xx yy zz : xx is a macro *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::TIdent(_,_)::_ , _)
+    when not_struct_enum before
+	&& ok_typedef s2
+	&& is_known_typdef s2
+        ->
+	  TCommentCpp(Token_c.CppMacro, i1)
+
+  (* xx yy zz : xx is a typedef ident *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::TIdent(_,_)::_ , _)
+    when not_struct_enum before
+	&& ok_typedef s
+        ->
+      msg_typedef s i1 2; LP.add_typedef_root s i1;
+      TypedefIdent (s, i1)
+
+  (* xx yy * zz : xx is a macro *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::ptr , _)
+    when pointer ~followed_by:(function TIdent _ -> true | _ -> false) ptr
+	&& not_struct_enum before
+	&& ok_typedef s2
+	&& is_known_typdef s2
+        ->
+	  TCommentCpp(Token_c.CppMacro, i1)
+
+  (* xx yy * zz : xx is a typedef ident *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::ptr , _)
+    when pointer ~followed_by:(function TIdent _ -> true | _ -> false) ptr
+	&& not_struct_enum before
+	&& ok_typedef s
+        ->
+      msg_typedef s i1 2; LP.add_typedef_root s i1;
+      TypedefIdent (s, i1)
+
   (* xx yy *)
   | (TIdent (s, i1)::TIdent (s2, i2)::rest  , _) when not_struct_enum before
 	&& ok_typedef s && not (is_macro_paren s2 rest)
-- 
2.21.1

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] [PATCH v2 2/3] parsing_hacks: Add bool to list of known typedefs

[Jaskaran Singh]

bool is widely used in the Linux kernel. Certain cases of
parsing_hacks.ml would mislabel bool.

Add bool to the list of known typedefs.

Stats of --parse-c on Linux v5.5-rc4 are as follows:

Before:

  nb good = 18956150,  nb passed = 134062 =========> 0.70% passed

After:

  nb good = 18956150,  nb passed = 134073 =========> 0.70% passed

The increase in passed tokens is due to functions in
kernel/trace/trace_kprobe.c using nokprobe_inline. For instances of
nokprobe_inline bool, nokprobe_inline is labeled as a CppMacro.

Examples of this in the --parse-c diff are as follows:

Before:
  passed:bool
  passed:bool
  passed:bool
  passed:bool
  passed:bool

After:
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline

Signed-off-by: Jaskaran Singh <jaskaransingh7654321@gmail.com>
---
 parsing_c/parsing_hacks.ml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/parsing_c/parsing_hacks.ml b/parsing_c/parsing_hacks.ml
index 23d675cf..42ad9ccc 100644
--- a/parsing_c/parsing_hacks.ml
+++ b/parsing_c/parsing_hacks.ml
@@ -61,7 +61,7 @@ let is_known_typdef =
       | "u_char"   | "u_short"  | "u_int"  | "u_long"
       | "u8" | "u16" | "u32" | "u64"
       | "s8"  | "s16" | "s32" | "s64"
-      | "__u8" | "__u16" | "__u32"  | "__u64"
+      | "__u8" | "__u16" | "__u32"  | "__u64" | "bool"
         -> true
 
       | "acpi_handle"
-- 
2.21.1

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] [PATCH v2 3/3] tests: Add test case for <macro> bool <ident>

[Jaskaran Singh]

This would previously yield pretty printing errors,
i.e. bool would be printed on the next line, followed by ret on
the next to next line.

The metatype should be only bool and not <macro> bool.

Signed-off-by: Jaskaran Singh <jaskaransingh7654321@gmail.com>
---
 tests/macro_before_bool.c     | 4 ++++
 tests/macro_before_bool.cocci | 9 +++++++++
 tests/macro_before_bool.res   | 5 +++++
 3 files changed, 18 insertions(+)
 create mode 100644 tests/macro_before_bool.c
 create mode 100644 tests/macro_before_bool.cocci
 create mode 100644 tests/macro_before_bool.res

diff --git a/tests/macro_before_bool.c b/tests/macro_before_bool.c
new file mode 100644
index 00000000..a59cba5a
--- /dev/null
+++ b/tests/macro_before_bool.c
@@ -0,0 +1,4 @@
+static nokprobe_inline bool trace_kprobe_is_return(struct trace_kprobe *tk)
+{
+	return false;
+}
diff --git a/tests/macro_before_bool.cocci b/tests/macro_before_bool.cocci
new file mode 100644
index 00000000..53b2fd2e
--- /dev/null
+++ b/tests/macro_before_bool.cocci
@@ -0,0 +1,9 @@
+@@
+type t;
+identifier x;
+@@
+
+t x(...) {
++	t ret;
+	return false;
+}
diff --git a/tests/macro_before_bool.res b/tests/macro_before_bool.res
new file mode 100644
index 00000000..1b0ec319
--- /dev/null
+++ b/tests/macro_before_bool.res
@@ -0,0 +1,5 @@
+static nokprobe_inline bool trace_kprobe_is_return(struct trace_kprobe *tk)
+{
+	bool ret;
+	return false;
+}
-- 
2.21.1

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] [PATCH v3 0/3] cocci: Improve management of macros before typedefs

[Jaskaran Singh]

This patch series is for improving the management of the following
case:

<macro> <typedef> <ident>

If <typedef> is a known typedef (such as u8 or *_t) then label <macro> as a
CppMacro. Subsequent cases will continue to label <typedef> correctly as a
typedef identifier.

Also add bool to the list of known typedefs so that cases of

<macro> bool <ident>

are handled correctly just like any known typedef will be.

Changes in v3:
--------------
- Change "subsequent cases will label" to "subsequent cases will continue to
  label" in commit messages of Patch 0/3 and Patch 1/3.
- Change the term "typedef ident" to "typedef identifier" wherever
  suitable.
- Add metavariable symbol false to tests/macro_before_bool.cocci to
  suppress warning.

Changes in v2:
--------------
- Group these patches as a series for clarity
- Add a test case for <macro> bool <ident> (Patch 3/3)
- Explain what has been improved with Patch 2/3

 parsing_c/parsing_hacks.ml    |   37 ++++++++++++++++++++++++++++++++++++-
 tests/macro_before_bool.c     |    4 ++++
 tests/macro_before_bool.cocci |   10 ++++++++++
 tests/macro_before_bool.res   |    5 +++++
 4 files changed, 55 insertions(+), 1 deletion(-)


_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [PATCH v3 0/3] cocci: Improve management of macros before typedefs

[Julia Lawall]

On Tue, 21 Jan 2020, Jaskaran Singh wrote:

> This patch series is for improving the management of the following
> case:
>
> <macro> <typedef> <ident>
>
> If <typedef> is a known typedef (such as u8 or *_t) then label <macro> as a
> CppMacro. Subsequent cases will continue to label <typedef> correctly as a
> typedef identifier.
>
> Also add bool to the list of known typedefs so that cases of
>
> <macro> bool <ident>
>
> are handled correctly just like any known typedef will be.
>
> Changes in v3:
> --------------
> - Change "subsequent cases will label" to "subsequent cases will continue to
>   label" in commit messages of Patch 0/3 and Patch 1/3.
> - Change the term "typedef ident" to "typedef identifier" wherever
>   suitable.
> - Add metavariable symbol false to tests/macro_before_bool.cocci to
>   suppress warning.
>
> Changes in v2:
> --------------
> - Group these patches as a series for clarity
> - Add a test case for <macro> bool <ident> (Patch 3/3)
> - Explain what has been improved with Patch 2/3

All applied.  Thanks!

julia

>
>  parsing_c/parsing_hacks.ml    |   37 ++++++++++++++++++++++++++++++++++++-
>  tests/macro_before_bool.c     |    4 ++++
>  tests/macro_before_bool.cocci |   10 ++++++++++
>  tests/macro_before_bool.res   |    5 +++++
>  4 files changed, 55 insertions(+), 1 deletion(-)
>
>
>
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] [PATCH v3 1/3] parsing_c: Handle case of macro before typedef

[Jaskaran Singh]

For the following case:

<macro> <typedef> <identifier>

A case in parsing_hacks.ml sometimes mislabels <macro> as a
typedef identifier.

If typedef is a known typedef (such as u8 or *_t), then label <macro>
as a CppMacro. Subsequent cases will continue to label <typedef>
correctly as a typedef identifier.

Following are results of --parse-c on the Linux Kernel v5.5-rc4:

Before:

  nb good = 18956150,  nb passed = 134061 =========> 0.70% passed

After:

  nb good = 18956150,  nb passed = 134062 =========> 0.70% passed

Signed-off-by: Jaskaran Singh <jaskaransingh7654321@gmail.com>
---
 parsing_c/parsing_hacks.ml | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/parsing_c/parsing_hacks.ml b/parsing_c/parsing_hacks.ml
index 8374731b..23d675cf 100644
--- a/parsing_c/parsing_hacks.ml
+++ b/parsing_c/parsing_hacks.ml
@@ -2184,6 +2184,41 @@ let lookahead2 ~pass next before =
       && ok_typedef s && is_macro s2 && is_type type_
         ->
 	  TIdent (s, i1)
+
+  (* xx yy zz : xx is a macro *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::TIdent(_,_)::_ , _)
+    when not_struct_enum before
+	&& ok_typedef s2
+	&& is_known_typdef s2
+        ->
+	  TCommentCpp(Token_c.CppMacro, i1)
+
+  (* xx yy zz : xx is a typedef ident *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::TIdent(_,_)::_ , _)
+    when not_struct_enum before
+	&& ok_typedef s
+        ->
+      msg_typedef s i1 2; LP.add_typedef_root s i1;
+      TypedefIdent (s, i1)
+
+  (* xx yy * zz : xx is a macro *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::ptr , _)
+    when pointer ~followed_by:(function TIdent _ -> true | _ -> false) ptr
+	&& not_struct_enum before
+	&& ok_typedef s2
+	&& is_known_typdef s2
+        ->
+	  TCommentCpp(Token_c.CppMacro, i1)
+
+  (* xx yy * zz : xx is a typedef ident *)
+  | (TIdent (s, i1)::TIdent (s2, i2)::ptr , _)
+    when pointer ~followed_by:(function TIdent _ -> true | _ -> false) ptr
+	&& not_struct_enum before
+	&& ok_typedef s
+        ->
+      msg_typedef s i1 2; LP.add_typedef_root s i1;
+      TypedefIdent (s, i1)
+
   (* xx yy *)
   | (TIdent (s, i1)::TIdent (s2, i2)::rest  , _) when not_struct_enum before
 	&& ok_typedef s && not (is_macro_paren s2 rest)
-- 
2.21.1

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] [PATCH v3 2/3] parsing_hacks: Add bool to list of known typedefs

[Jaskaran Singh]

bool is widely used in the Linux kernel. Certain cases of
parsing_hacks.ml would mislabel bool.

Add bool to the list of known typedefs.

Stats of --parse-c on Linux v5.5-rc4 are as follows:

Before:

  nb good = 18956150,  nb passed = 134062 =========> 0.70% passed

After:

  nb good = 18956150,  nb passed = 134073 =========> 0.70% passed

The increase in passed tokens is due to functions in
kernel/trace/trace_kprobe.c using nokprobe_inline. For instances of
nokprobe_inline bool, nokprobe_inline is labeled as a CppMacro.

Examples of this in the --parse-c diff are as follows:

Before:
  passed:bool
  passed:bool
  passed:bool
  passed:bool
  passed:bool

After:
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline
  passed:nokprobe_inline

Signed-off-by: Jaskaran Singh <jaskaransingh7654321@gmail.com>
---
 parsing_c/parsing_hacks.ml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/parsing_c/parsing_hacks.ml b/parsing_c/parsing_hacks.ml
index 23d675cf..42ad9ccc 100644
--- a/parsing_c/parsing_hacks.ml
+++ b/parsing_c/parsing_hacks.ml
@@ -61,7 +61,7 @@ let is_known_typdef =
       | "u_char"   | "u_short"  | "u_int"  | "u_long"
       | "u8" | "u16" | "u32" | "u64"
       | "s8"  | "s16" | "s32" | "s64"
-      | "__u8" | "__u16" | "__u32"  | "__u64"
+      | "__u8" | "__u16" | "__u32"  | "__u64" | "bool"
         -> true
 
       | "acpi_handle"
-- 
2.21.1

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] [PATCH v3 3/3] tests: Add test case for <macro> bool <ident>

[Jaskaran Singh]

This would previously yield pretty printing errors,
i.e. bool would be printed on the next line, followed by ret on
the next to next line.

The metatype should be only bool and not <macro> bool.

Signed-off-by: Jaskaran Singh <jaskaransingh7654321@gmail.com>
---
 tests/macro_before_bool.c     |  4 ++++
 tests/macro_before_bool.cocci | 10 ++++++++++
 tests/macro_before_bool.res   |  5 +++++
 3 files changed, 19 insertions(+)
 create mode 100644 tests/macro_before_bool.c
 create mode 100644 tests/macro_before_bool.cocci
 create mode 100644 tests/macro_before_bool.res

diff --git a/tests/macro_before_bool.c b/tests/macro_before_bool.c
new file mode 100644
index 00000000..a59cba5a
--- /dev/null
+++ b/tests/macro_before_bool.c
@@ -0,0 +1,4 @@
+static nokprobe_inline bool trace_kprobe_is_return(struct trace_kprobe *tk)
+{
+	return false;
+}
diff --git a/tests/macro_before_bool.cocci b/tests/macro_before_bool.cocci
new file mode 100644
index 00000000..667e4662
--- /dev/null
+++ b/tests/macro_before_bool.cocci
@@ -0,0 +1,10 @@
+@@
+type t;
+identifier x;
+symbol false;
+@@
+
+t x(...) {
++	t ret;
+	return false;
+}
diff --git a/tests/macro_before_bool.res b/tests/macro_before_bool.res
new file mode 100644
index 00000000..1b0ec319
--- /dev/null
+++ b/tests/macro_before_bool.res
@@ -0,0 +1,5 @@
+static nokprobe_inline bool trace_kprobe_is_return(struct trace_kprobe *tk)
+{
+	bool ret;
+	return false;
+}
-- 
2.21.1

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [PATCH v2 3/3] tests: Add test case for <macro> bool <ident>

[Markus Elfring]

> …, followed by ret on
> the next to next line.

Would you like to improve this wording?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci