[Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Dwaipayan Ray]

checkpatch has limited support for parsing email comments. It only
support single name comments or single after address comments.
Whereas, RFC 5322 specifies that comments can be inserted in
between any tokens of the email fields.

On analyzing 50,000 commits from v5.4 it was found that there were
about 370 false positives resulting from wrong parsing of comments.

Improve comment parsing mechanism in checkpatch.

What is handled now:

- Multiple name/address comments
- Comments anywhere in between name/address
- Multi level comments like (John (Doe) )

Signed-off-by: Dwaipayan Ray <dwaipayanray1@gmail.com>
---
 scripts/checkpatch.pl | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index fab38b493cef..ae8436385fc1 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -1183,14 +1183,20 @@ sub parse_email {
 		}
 	}
 
-	$comment = trim($comment);
+	# Comments in between name like John(A nice chap) Doe
+	while ($name =~ s/\s*($balanced_parens)\s*/ /) {
+		$name_comment .= trim($1);
+	}
 	$name = trim($name);
 	$name =~ s/^\"|\"$//g;
-	if ($name =~ s/(\s*\([^\)]+\))\s*//) {
-		$name_comment = trim($1);
+
+	# Comments in between address like <john(his account)@doe.com>
+	while ($address =~ s/\s*($balanced_parens)\s*//) {
+		$comment .= trim($1);
 	}
 	$address = trim($address);
 	$address =~ s/^\<|\>$//g;
+	$comment = trim($comment);
 
 	if ($name =~ /[^\w \-]/i) { ##has "must quote" chars
 		$name =~ s/(?<!\\)"/\\"/g; ##escape quotes
@@ -1205,8 +1211,6 @@ sub format_email {
 
 	my $formatted_email;
 
-	$name_comment = trim($name_comment);
-	$comment = trim($comment);
 	$name = trim($name);
 	$name =~ s/^\"|\"$//g;
 	$address = trim($address);
@@ -1216,6 +1220,11 @@ sub format_email {
 		$name = "\"$name\"";
 	}
 
+	$name_comment = trim($name_comment);
+	$name_comment =~ s/(.+)/ $1/;
+	$comment = trim($comment);
+	$comment =~ s/(.+)/ $1/;
+
 	if ("$name" eq "") {
 		$formatted_email = "$address";
 	} else {
-- 
2.27.0

_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Dwaipayan Ray]

On Wed, Oct 28, 2020 at 8:55 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
>
> checkpatch has limited support for parsing email comments. It only
> support single name comments or single after address comments.
> Whereas, RFC 5322 specifies that comments can be inserted in
> between any tokens of the email fields.
>
> On analyzing 50,000 commits from v5.4 it was found that there were
> about 370 false positives resulting from wrong parsing of comments.
>
> Improve comment parsing mechanism in checkpatch.
>
> What is handled now:
>
> - Multiple name/address comments
> - Comments anywhere in between name/address
> - Multi level comments like (John (Doe) )
>
> Signed-off-by: Dwaipayan Ray <dwaipayanray1@gmail.com>
> ---
>  scripts/checkpatch.pl | 19 ++++++++++++++-----
>  1 file changed, 14 insertions(+), 5 deletions(-)
>
> diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
> index fab38b493cef..ae8436385fc1 100755
> --- a/scripts/checkpatch.pl
> +++ b/scripts/checkpatch.pl
> @@ -1183,14 +1183,20 @@ sub parse_email {
>                 }
>         }
>
> -       $comment = trim($comment);
> +       # Comments in between name like John(A nice chap) Doe
> +       while ($name =~ s/\s*($balanced_parens)\s*/ /) {
> +               $name_comment .= trim($1);
> +       }
>         $name = trim($name);
>         $name =~ s/^\"|\"$//g;
> -       if ($name =~ s/(\s*\([^\)]+\))\s*//) {
> -               $name_comment = trim($1);
> +
> +       # Comments in between address like <john(his account)@doe.com>
> +       while ($address =~ s/\s*($balanced_parens)\s*//) {
> +               $comment .= trim($1);
>         }
>         $address = trim($address);
>         $address =~ s/^\<|\>$//g;
> +       $comment = trim($comment);
>
>         if ($name =~ /[^\w \-]/i) { ##has "must quote" chars
>                 $name =~ s/(?<!\\)"/\\"/g; ##escape quotes
> @@ -1205,8 +1211,6 @@ sub format_email {
>
>         my $formatted_email;
>
> -       $name_comment = trim($name_comment);
> -       $comment = trim($comment);
>         $name = trim($name);
>         $name =~ s/^\"|\"$//g;
>         $address = trim($address);
> @@ -1216,6 +1220,11 @@ sub format_email {
>                 $name = "\"$name\"";
>         }
>
> +       $name_comment = trim($name_comment);
> +       $name_comment =~ s/(.+)/ $1/;
> +       $comment = trim($comment);
> +       $comment =~ s/(.+)/ $1/;
> +
>         if ("$name" eq "") {
>                 $formatted_email = "$address";
>         } else {
> --
> 2.27.0
>

Hi,
This patch is a follow up to our discussion earlier about improving the
comment handling at:
https://lore.kernel.org/linux-kernel-mentees/alpine.DEB.2.21.2010251022060.25172@felia/

I have only tested it on a few patterns.  But I will run an
evaluation again on those 50k commits to see what changes.

What do you think of this?

Thanks,
Dwaipayan.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Lukas Bulwahn]

On Wed, 28 Oct 2020, Dwaipayan Ray wrote:

> On Wed, Oct 28, 2020 at 8:55 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
> >
> > checkpatch has limited support for parsing email comments. It only
> > support single name comments or single after address comments.
> > Whereas, RFC 5322 specifies that comments can be inserted in
> > between any tokens of the email fields.
> >
> > On analyzing 50,000 commits from v5.4 it was found that there were
> > about 370 false positives resulting from wrong parsing of comments.
> >
> > Improve comment parsing mechanism in checkpatch.
> >
> > What is handled now:
> >
> > - Multiple name/address comments
> > - Comments anywhere in between name/address
> > - Multi level comments like (John (Doe) )
> >
> > Signed-off-by: Dwaipayan Ray <dwaipayanray1@gmail.com>
> > ---
> >  scripts/checkpatch.pl | 19 ++++++++++++++-----
> >  1 file changed, 14 insertions(+), 5 deletions(-)
> >
> > diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
> > index fab38b493cef..ae8436385fc1 100755
> > --- a/scripts/checkpatch.pl
> > +++ b/scripts/checkpatch.pl
> > @@ -1183,14 +1183,20 @@ sub parse_email {
> >                 }
> >         }
> >
> > -       $comment = trim($comment);
> > +       # Comments in between name like John(A nice chap) Doe
> > +       while ($name =~ s/\s*($balanced_parens)\s*/ /) {
> > +               $name_comment .= trim($1);
> > +       }
> >         $name = trim($name);
> >         $name =~ s/^\"|\"$//g;
> > -       if ($name =~ s/(\s*\([^\)]+\))\s*//) {
> > -               $name_comment = trim($1);
> > +
> > +       # Comments in between address like <john(his account)@doe.com>
> > +       while ($address =~ s/\s*($balanced_parens)\s*//) {
> > +               $comment .= trim($1);
> >         }
> >         $address = trim($address);
> >         $address =~ s/^\<|\>$//g;
> > +       $comment = trim($comment);
> >
> >         if ($name =~ /[^\w \-]/i) { ##has "must quote" chars
> >                 $name =~ s/(?<!\\)"/\\"/g; ##escape quotes
> > @@ -1205,8 +1211,6 @@ sub format_email {
> >
> >         my $formatted_email;
> >
> > -       $name_comment = trim($name_comment);
> > -       $comment = trim($comment);
> >         $name = trim($name);
> >         $name =~ s/^\"|\"$//g;
> >         $address = trim($address);
> > @@ -1216,6 +1220,11 @@ sub format_email {
> >                 $name = "\"$name\"";
> >         }
> >
> > +       $name_comment = trim($name_comment);
> > +       $name_comment =~ s/(.+)/ $1/;
> > +       $comment = trim($comment);
> > +       $comment =~ s/(.+)/ $1/;
> > +
> >         if ("$name" eq "") {
> >                 $formatted_email = "$address";
> >         } else {
> > --
> > 2.27.0
> >
> 
> Hi,
> This patch is a follow up to our discussion earlier about improving the
> comment handling at:
> https://lore.kernel.org/linux-kernel-mentees/alpine.DEB.2.21.2010251022060.25172@felia/
> 
> I have only tested it on a few patterns.  But I will run an
> evaluation again on those 50k commits to see what changes.
> 
> What do you think of this?
>

That was exactly the comment I wanted to make.

Commit message is clear; code addition also looks pretty clear.

We are just missing the evaluation to make sure how much we improve and 
that we did not add some stupid bug on the way.

Aditya, can you help us here with evaluation?

E.g., Dwaipayan takes from v5.4 into the future, e.g., until v5.8/v5.9 or 
so.

Aditya, you could then check e.g., v5.0..v5.4, so v5.4 down to the past as 
far as your computer and scripts handles within roughly half a day...

An evaluation on 100,000 commits is certainly a good basis.

Of course, every mentorship candidate can join here as well and show off 
their own evaluation script.

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Lukas Bulwahn]

On Wed, 28 Oct 2020, Dwaipayan Ray wrote:

> On Wed, Oct 28, 2020 at 8:55 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
> >
> > checkpatch has limited support for parsing email comments. It only
> > support single name comments or single after address comments.
> > Whereas, RFC 5322 specifies that comments can be inserted in
> > between any tokens of the email fields.
> >
> > On analyzing 50,000 commits from v5.4 it was found that there were
> > about 370 false positives resulting from wrong parsing of comments.
> >
> > Improve comment parsing mechanism in checkpatch.
> >
> > What is handled now:
> >
> > - Multiple name/address comments
> > - Comments anywhere in between name/address
> > - Multi level comments like (John (Doe) )
> >
> > Signed-off-by: Dwaipayan Ray <dwaipayanray1@gmail.com>
> > ---
> >  scripts/checkpatch.pl | 19 ++++++++++++++-----
> >  1 file changed, 14 insertions(+), 5 deletions(-)
> >
> > diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
> > index fab38b493cef..ae8436385fc1 100755
> > --- a/scripts/checkpatch.pl
> > +++ b/scripts/checkpatch.pl
> > @@ -1183,14 +1183,20 @@ sub parse_email {
> >                 }
> >         }
> >
> > -       $comment = trim($comment);
> > +       # Comments in between name like John(A nice chap) Doe
> > +       while ($name =~ s/\s*($balanced_parens)\s*/ /) {
> > +               $name_comment .= trim($1);
> > +       }
> >         $name = trim($name);
> >         $name =~ s/^\"|\"$//g;
> > -       if ($name =~ s/(\s*\([^\)]+\))\s*//) {
> > -               $name_comment = trim($1);
> > +
> > +       # Comments in between address like <john(his account)@doe.com>
> > +       while ($address =~ s/\s*($balanced_parens)\s*//) {
> > +               $comment .= trim($1);
> >         }
> >         $address = trim($address);
> >         $address =~ s/^\<|\>$//g;
> > +       $comment = trim($comment);
> >
> >         if ($name =~ /[^\w \-]/i) { ##has "must quote" chars
> >                 $name =~ s/(?<!\\)"/\\"/g; ##escape quotes
> > @@ -1205,8 +1211,6 @@ sub format_email {
> >
> >         my $formatted_email;
> >
> > -       $name_comment = trim($name_comment);
> > -       $comment = trim($comment);
> >         $name = trim($name);
> >         $name =~ s/^\"|\"$//g;
> >         $address = trim($address);
> > @@ -1216,6 +1220,11 @@ sub format_email {
> >                 $name = "\"$name\"";
> >         }
> >
> > +       $name_comment = trim($name_comment);
> > +       $name_comment =~ s/(.+)/ $1/;
> > +       $comment = trim($comment);
> > +       $comment =~ s/(.+)/ $1/;
> > +
> >         if ("$name" eq "") {
> >                 $formatted_email = "$address";
> >         } else {
> > --
> > 2.27.0
> >
> 
> Hi,
> This patch is a follow up to our discussion earlier about improving the
> comment handling at:
> https://lore.kernel.org/linux-kernel-mentees/alpine.DEB.2.21.2010251022060.25172@felia/
> 
> I have only tested it on a few patterns.  But I will run an
> evaluation again on those 50k commits to see what changes.
> 
> What do you think of this?
>

That was exactly the comment I wanted to make.

Commit message is clear; code addition also looks pretty clear.

We are just missing the evaluation to make sure how much we improve and 
that we did not add some stupid bug on the way.

Aditya, can you help us here with evaluation?

E.g., Dwaipayan takes from v5.4 into the future, e.g., until v5.8/v5.9 or 
so.

Aditya, you could then check e.g., v5.0..v5.4, so v5.4 down to the past as 
far as your computer and scripts handles within roughly half a day...

An evaluation on 100,000 commits is certainly a good basis.

Of course, every mentorship candidate can join here as well and show off 
their own evaluation script.

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Aditya]

On 28/10/20 9:29 pm, Lukas Bulwahn wrote:
> 
> 
> On Wed, 28 Oct 2020, Dwaipayan Ray wrote:
> 
>> On Wed, Oct 28, 2020 at 8:55 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
>>>
>>> checkpatch has limited support for parsing email comments. It only
>>> support single name comments or single after address comments.
>>> Whereas, RFC 5322 specifies that comments can be inserted in
>>> between any tokens of the email fields.
>>>
>>> On analyzing 50,000 commits from v5.4 it was found that there were
>>> about 370 false positives resulting from wrong parsing of comments.
>>>
>>> Improve comment parsing mechanism in checkpatch.
>>>
>>> What is handled now:
>>>
>>> - Multiple name/address comments
>>> - Comments anywhere in between name/address
>>> - Multi level comments like (John (Doe) )
>>>
>>> Signed-off-by: Dwaipayan Ray <dwaipayanray1@gmail.com>
>>> ---
>>>  scripts/checkpatch.pl | 19 ++++++++++++++-----
>>>  1 file changed, 14 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
>>> index fab38b493cef..ae8436385fc1 100755
>>> --- a/scripts/checkpatch.pl
>>> +++ b/scripts/checkpatch.pl
>>> @@ -1183,14 +1183,20 @@ sub parse_email {
>>>                 }
>>>         }
>>>
>>> -       $comment = trim($comment);
>>> +       # Comments in between name like John(A nice chap) Doe
>>> +       while ($name =~ s/\s*($balanced_parens)\s*/ /) {
>>> +               $name_comment .= trim($1);
>>> +       }
>>>         $name = trim($name);
>>>         $name =~ s/^\"|\"$//g;
>>> -       if ($name =~ s/(\s*\([^\)]+\))\s*//) {
>>> -               $name_comment = trim($1);
>>> +
>>> +       # Comments in between address like <john(his account)@doe.com>
>>> +       while ($address =~ s/\s*($balanced_parens)\s*//) {
>>> +               $comment .= trim($1);
>>>         }
>>>         $address = trim($address);
>>>         $address =~ s/^\<|\>$//g;
>>> +       $comment = trim($comment);
>>>
>>>         if ($name =~ /[^\w \-]/i) { ##has "must quote" chars
>>>                 $name =~ s/(?<!\\)"/\\"/g; ##escape quotes
>>> @@ -1205,8 +1211,6 @@ sub format_email {
>>>
>>>         my $formatted_email;
>>>
>>> -       $name_comment = trim($name_comment);
>>> -       $comment = trim($comment);
>>>         $name = trim($name);
>>>         $name =~ s/^\"|\"$//g;
>>>         $address = trim($address);
>>> @@ -1216,6 +1220,11 @@ sub format_email {
>>>                 $name = "\"$name\"";
>>>         }
>>>
>>> +       $name_comment = trim($name_comment);
>>> +       $name_comment =~ s/(.+)/ $1/;
>>> +       $comment = trim($comment);
>>> +       $comment =~ s/(.+)/ $1/;
>>> +
>>>         if ("$name" eq "") {
>>>                 $formatted_email = "$address";
>>>         } else {
>>> --
>>> 2.27.0
>>>
>>
>> Hi,
>> This patch is a follow up to our discussion earlier about improving the
>> comment handling at:
>> https://lore.kernel.org/linux-kernel-mentees/alpine.DEB.2.21.2010251022060.25172@felia/
>>
>> I have only tested it on a few patterns.  But I will run an
>> evaluation again on those 50k commits to see what changes.
>>
>> What do you think of this?
>>
> 
> That was exactly the comment I wanted to make.
> 
> Commit message is clear; code addition also looks pretty clear.
> 
> We are just missing the evaluation to make sure how much we improve and 
> that we did not add some stupid bug on the way.
> 
> Aditya, can you help us here with evaluation?
> 
> E.g., Dwaipayan takes from v5.4 into the future, e.g., until v5.8/v5.9 or 
> so.
> 
> Aditya, you could then check e.g., v5.0..v5.4, so v5.4 down to the past as 
> far as your computer and scripts handles within roughly half a day...
> 
> An evaluation on 100,000 commits is certainly a good basis.
> 
> Of course, every mentorship candidate can join here as well and show off 
> their own evaluation script.
> 
> Lukas
> 

Sure, would love to help.
Just to clear, I have to apply this patch over my last patch right? Or
do I need to apply any more patch before it?

Thanks
Aditya
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Dwaipayan Ray]

> > Aditya, can you help us here with evaluation?
> >
> > E.g., Dwaipayan takes from v5.4 into the future, e.g., until v5.8/v5.9 or
> > so.
> >
> > Aditya, you could then check e.g., v5.0..v5.4, so v5.4 down to the past as
> > far as your computer and scripts handles within roughly half a day...
> >
> > An evaluation on 100,000 commits is certainly a good basis.
> >
> > Of course, every mentorship candidate can join here as well and show off
> > their own evaluation script.
> >
> > Lukas
> >
>
> Sure, would love to help.
> Just to clear, I have to apply this patch over my last patch right? Or
> do I need to apply any more patch before it?
>

Thanks Aditya.

I think you can apply the patch directly on the latest official tree.

The patch is here:
https://lore.kernel.org/linux-kernel-mentees/20201028152501.106117-1-dwaipayanray1@gmail.com/

Regards,
Dwaipayan.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Aditya]

On 28/10/20 10:08 pm, Dwaipayan Ray wrote:
>>> Aditya, can you help us here with evaluation?
>>>
>>> E.g., Dwaipayan takes from v5.4 into the future, e.g., until v5.8/v5.9 or
>>> so.
>>>
>>> Aditya, you could then check e.g., v5.0..v5.4, so v5.4 down to the past as
>>> far as your computer and scripts handles within roughly half a day...
>>>
>>> An evaluation on 100,000 commits is certainly a good basis.
>>>
>>> Of course, every mentorship candidate can join here as well and show off
>>> their own evaluation script.
>>>
>>> Lukas
>>>
>>
>> Sure, would love to help.
>> Just to clear, I have to apply this patch over my last patch right? Or
>> do I need to apply any more patch before it?
>>
> 
> Thanks Aditya.
> 
> I think you can apply the patch directly on the latest official tree.
> 
> The patch is here:
> https://lore.kernel.org/linux-kernel-mentees/20201028152501.106117-1-dwaipayanray1@gmail.com/
> 
> Regards,
> Dwaipayan.
> 

Hi Dwaipayan,
I have generated reports on v5.0..v5.4.

Relative change in warnings/errors before/after your patch:
https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task4/relative_summary/summary_relative.txt

Dropped warnings:
https://github.com/AdityaSrivast/kernel-tasks/tree/master/Task4/relative_summary/dropped_warnings

There may be some discrepancies due to multi-threading. But it should
give us a rough idea.

Thanks
Aditya
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Dwaipayan Ray]

>
> Hi Dwaipayan,
> I have generated reports on v5.0..v5.4.
>
> Relative change in warnings/errors before/after your patch:
> https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task4/relative_summary/summary_relative.txt
>
> Dropped warnings:
> https://github.com/AdityaSrivast/kernel-tasks/tree/master/Task4/relative_summary/dropped_warnings
>
> There may be some discrepancies due to multi-threading. But it should
> give us a rough idea.
>

Thanks Aditya.

Seems like 763 false positives were resolved successfully.
So I guess the patch is working as intended.

My evaluation is currently running and might need a few
hours more. I will share it once it completes.

Thanks,
Dwaipayan.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Dwaipayan Ray]

On Thu, Oct 29, 2020 at 9:17 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
>
> >
> > Hi Dwaipayan,
> > I have generated reports on v5.0..v5.4.
> >
> > Relative change in warnings/errors before/after your patch:
> > https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task4/relative_summary/summary_relative.txt
> >
> > Dropped warnings:
> > https://github.com/AdityaSrivast/kernel-tasks/tree/master/Task4/relative_summary/dropped_warnings
> >
> > There may be some discrepancies due to multi-threading. But it should
> > give us a rough idea.
> >
>
> Thanks Aditya.
>
> Seems like 763 false positives were resolved successfully.
> So I guess the patch is working as intended.
>
> My evaluation is currently running and might need a few
> hours more. I will share it once it completes.
>

The summary of the analysis is as follows:

Commits checked: 47, 377 from v 5.4

Before the patch:
BAD_SIGN_OFF: 1516
FROM_SIGN_OFF_MISMATCH: 931
NO_AUTHOR_SIGN_OFF: 54

After the patch:
BAD_SIGN_OFF: 756
FROM_SIGN_OFF_MISMATCH: 898
NO_AUTHOR_SIGN_OFF: 54

A total of 793 false positives were resolved.

I think this patch does the job.
Lukas, what do you think?

Thanks,
Dwaipayan.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Lukas Bulwahn]

On Thu, Oct 29, 2020 at 8:43 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
>
> On Thu, Oct 29, 2020 at 9:17 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
> >
> > >
> > > Hi Dwaipayan,
> > > I have generated reports on v5.0..v5.4.
> > >
> > > Relative change in warnings/errors before/after your patch:
> > > https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task4/relative_summary/summary_relative.txt
> > >
> > > Dropped warnings:
> > > https://github.com/AdityaSrivast/kernel-tasks/tree/master/Task4/relative_summary/dropped_warnings
> > >
> > > There may be some discrepancies due to multi-threading. But it should
> > > give us a rough idea.
> > >
> >
> > Thanks Aditya.
> >
> > Seems like 763 false positives were resolved successfully.
> > So I guess the patch is working as intended.
> >
> > My evaluation is currently running and might need a few
> > hours more. I will share it once it completes.
> >
>
> The summary of the analysis is as follows:
>
> Commits checked: 47, 377 from v 5.4
>
> Before the patch:
> BAD_SIGN_OFF: 1516
> FROM_SIGN_OFF_MISMATCH: 931
> NO_AUTHOR_SIGN_OFF: 54
>
> After the patch:
> BAD_SIGN_OFF: 756
> FROM_SIGN_OFF_MISMATCH: 898
> NO_AUTHOR_SIGN_OFF: 54
>
> A total of 793 false positives were resolved.
>
> I think this patch does the job.
> Lukas, what do you think?
>

Agree. The numbers and my quick spot checks suggest it is a good fix
to get merged.

Update the commit message to provide the overall summary of the two
evaluations (by adding them up for the combined git range) and send
the patch to Joe for a quick check.

By the way, I guess it is a good time to share the checkpatch.pl
evaluation scripts with each other, compare them and settle among us
for one common script to use.

(That gives away the mentee intro coding challenge but I am sure we
can come up with a new one...)


Lukas
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Aditya]

On 30/10/20 11:00 am, Lukas Bulwahn wrote:
> On Thu, Oct 29, 2020 at 8:43 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
>>
>> On Thu, Oct 29, 2020 at 9:17 PM Dwaipayan Ray <dwaipayanray1@gmail.com> wrote:
>>>
>>>>
>>>> Hi Dwaipayan,
>>>> I have generated reports on v5.0..v5.4.
>>>>
>>>> Relative change in warnings/errors before/after your patch:
>>>> https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task4/relative_summary/summary_relative.txt
>>>>
>>>> Dropped warnings:
>>>> https://github.com/AdityaSrivast/kernel-tasks/tree/master/Task4/relative_summary/dropped_warnings
>>>>
>>>> There may be some discrepancies due to multi-threading. But it should
>>>> give us a rough idea.
>>>>
>>>
>>> Thanks Aditya.
>>>
>>> Seems like 763 false positives were resolved successfully.
>>> So I guess the patch is working as intended.
>>>
>>> My evaluation is currently running and might need a few
>>> hours more. I will share it once it completes.
>>>
>>
>> The summary of the analysis is as follows:
>>
>> Commits checked: 47, 377 from v 5.4
>>
>> Before the patch:
>> BAD_SIGN_OFF: 1516
>> FROM_SIGN_OFF_MISMATCH: 931
>> NO_AUTHOR_SIGN_OFF: 54
>>
>> After the patch:
>> BAD_SIGN_OFF: 756
>> FROM_SIGN_OFF_MISMATCH: 898
>> NO_AUTHOR_SIGN_OFF: 54
>>
>> A total of 793 false positives were resolved.
>>
>> I think this patch does the job.
>> Lukas, what do you think?
>>
> 
> Agree. The numbers and my quick spot checks suggest it is a good fix
> to get merged.
> 
> Update the commit message to provide the overall summary of the two
> evaluations (by adding them up for the combined git range) and send
> the patch to Joe for a quick check.
> 
> By the way, I guess it is a good time to share the checkpatch.pl
> evaluation scripts with each other, compare them and settle among us
> for one common script to use.
> 
> (That gives away the mentee intro coding challenge but I am sure we
> can come up with a new one...)
> 
> 
> Lukas
> 

Sure. The scripts I am using are:
For generating checkpatch reports over commit ids:
https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task2/reports/before_commit/script.pl

For finding difference in errors/warning messages with count:
https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task4/relative_summary/script.pl

For generating dropped warnings:
https://github.com/AdityaSrivast/kernel-tasks/blob/master/Task4/relative_summary/dropped_warnings/from_sign_off_mismatch/script.pl


_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] [PATCH RFC] checkpatch: improve handling of email comments

[Dwaipayan Ray]

>
> By the way, I guess it is a good time to share the checkpatch.pl
> evaluation scripts with each other, compare them and settle among us
> for one common script to use.
>
> (That gives away the mentee intro coding challenge but I am sure we
> can come up with a new one...)
>
>

Sure that sounds good to me. We could even combine the scripts to
create something more efficient probably.

Here is the one I use for generating the checkpatch output data over
the supplied commits:
https://gist.github.com/raydwaipayan/57cc003e74618fe8de8917bc343fd234

(Multithreaded and semaphores to eliminate races between threads
to write to file)

For generating summary:
https://gist.github.com/raydwaipayan/9e1f340e3e892bd48a2c972f0517698d

For comparisons and individual warnings I usually rely on diff and grep.

Thanks,
Dwaipayan.
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees