Re: [PATCH] checkpatch: improve handling of email comments

[Joe Perches <joe@perches.com>]

On Fri, 2020-10-30 at 12:58 +0100, Lukas Bulwahn wrote:
> 
> On Fri, 30 Oct 2020, Joe Perches wrote:
> 
> > On Fri, 2020-10-30 at 14:37 +0530, Dwaipayan Ray wrote:
> > > checkpatch has limited support for parsing email comments. It only
> > > support single name comments or single after address comments.
> > > Whereas, RFC 5322 specifies that comments can be inserted in
> > > between any tokens of the email fields.
> > > 
> > > Improve comment parsing mechanism in checkpatch.
> > > 
> > > What is handled now:
> > > 
> > > - Multiple name/address comments
> > > - Comments anywhere in between name/address
> > > - Nested comments like (John (Doe))
> > > 
> > > A brief analysis of checkpatch output on v5.0..v5.7 showed that
> > > after these modifications, the number of BAD_SIGN_OFF warnings
> > > came down from 2944 to 1424, and FROM_SIGN_OFF_MISMATCH came
> > > down from 2366 to 2330.
> > > 
> > > So, a total of 1556 false positives were resolved in total.
> > 
> > A mere reduction in messages emitted isn't necessarily good.
> > 
> 
> Agree. That is why I also went through the list of those warnings.
> 
> I could not spot any obvious true positive among the reduced ones.
>  
> 
> > Please send me privately a complete list of these nominally
> > false positive messages that are no longer emitted.
> > 
> > I believe one of the relatively common incorrect messages is
> > for the cc: <stable@vger.kernel.org> where a version number is
> > continued on the same line after a #.
> > 
> > CC: stable@vger.kernel.org # for versions x.y.z and above
> > 
> 
> That was one,

It's not just one, it's ~90% of the list that Dwaipayan sent me.

$ wc -l mismatches
831 mismatches

$ grep -v -i stable mismatches | wc -l
98

> another common pattern was just quotes put inconsistently at 
> different places.

Yes, there are some defects there.
But there are also now false negatives.

For instance, this is not appropriate to ignore:

WARNING:BAD_SIGN_OFF: email address 'jacek.anaszewski@gmail.com, linux-leds@vger.kernel.org, linux-kernel@vger.kernel.org, dmurphy@ti.com' might be better as 'jacek.anaszewski@gmail.com,linux-leds@vger.kernel.org, linux-kernel@vger.kernel.org, dmurphy@ti.com'

From the file that Dwaipayan sent me, all the rest not including the
stable variants, which IMO should be handled separately, are below.

Of these 98 in total, 60+% are unicode which IMO should always be quoted
and most are doubled with BAD_SIGN_OFF doubling FROM_SIGN_OFF_MISMATCH
(and I don't quite understand why it's "From:/" then "Signed-off-by:"

$ grep -v -i stable dwai | sort | uniq -c | sort -rn
     31 WARNING:BAD_SIGN_OFF: email address '周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>' might be better as '"周琰杰"(Zhou Yanjie) <zhouyanjie@wanyeetech.com>'
     30 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "周琰杰"(Zhou Yanjie) <zhouyanjie@wanyeetech.com>' != 'Signed-off-by: 周琰杰 (Zhou Yanjie) <zhouyanjie@wanyeetech.com>'
     
These 29 in total would be better stripping any bits in parentheses from
the name portion only when _not_ inside quotes.

     20 WARNING:BAD_SIGN_OFF: email address 'Thomas Hellström (VMware) <thomas_os@shipmail.org>' might be better as '"Thomas Hellström"(VMware) <thomas_os@shipmail.org>'
      5 WARNING:BAD_SIGN_OFF: email address 'H. Peter Anvin (Intel) <hpa@zytor.com>' might be better as '"H. Peter Anvin"(Intel) <hpa@zytor.com>'
      

      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Thomas Hellström"(VMware) <thomas_os@shipmail.org>' != 'Signed-off-by: Thomas Hellström (VMware) <thomas_os@shipmail.org>'
      
      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Srivatsa S. Bhat"(VMware) <srivatsa@csail.mit.edu>' != 'Signed-off-by: Srivatsa S. Bhat (VMware) <srivatsa@csail.mit.edu>'
      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: JanNieuwenhuizen(janneke) <janneke@gnu.org>' != 'Signed-off-by: Jan Nieuwenhuizen <janneke@gnu.org>'  
      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Frédéric Pierret"(fepitre) <frederic.pierret@qubes-os.org>' != 'Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>'

So these 8 others are ones where quotes are either oddly placed
or perhaps should always exist and the comment in parentheses
is suggested poorly.  7 of these should be fixed and one should
still be reported.

      1 WARNING:BAD_SIGN_OFF: email address '"Thomas Hellström (VMware)" <thomas_os@shipmail.org>' might be better as '"Thomas Hellström"(VMware) <thomas_os@shipmail.org>'
      1 WARNING:BAD_SIGN_OFF: email address 'Srivatsa S. Bhat (VMware) <srivatsa@csail.mit.edu>' might be better as '"Srivatsa S. Bhat"(VMware) <srivatsa@csail.mit.edu>'
      1 WARNING:BAD_SIGN_OFF: email address '"Rantala, Tommi T. (Nokia - FI/Espoo)" <tommi.t.rantala@nokia.com>' might be better as '"Rantala, Tommi T."(Nokia - FI/Espoo) <tommi.t.rantala@nokia.com>'
      1 WARNING:BAD_SIGN_OFF: email address '"Kai Mäkisara (Kolumbus)" <kai.makisara@kolumbus.fi>' might be better as '"Kai Mäkisara"(Kolumbus) <kai.makisara@kolumbus.fi>'
      1 WARNING:BAD_SIGN_OFF: email address 'jacek.anaszewski@gmail.com, linux-leds@vger.kernel.org, linux-kernel@vger.kernel.org, dmurphy@ti.com' might be better as 'jacek.anaszewski@gmail.com,linux-leds@vger.kernel.org, linux-kernel@vger.kernel.org, dmurphy@ti.com'
      1 WARNING:BAD_SIGN_OFF: email address 'Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org>' might be better as '"Frédéric Pierret"(fepitre) <frederic.pierret@qubes-os.org>'
      1 WARNING:BAD_SIGN_OFF: email address 'David.Laight@aculab.com (big endian system concerns)' might be better as 'David.Laight@aculab.com(big endian system concerns)'
      1 WARNING:BAD_SIGN_OFF: email address 'apenwarr@foxnet.net (Avery Pennarun)' might be better as 'apenwarr@foxnet.net(Avery Pennarun)'