LKML Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH v5] checkpatch: add new exception to repeated word check
@ 2020-10-17 16:27 Dwaipayan Ray
  2020-10-17 16:33 ` Joe Perches
  2020-10-18 18:04 ` Joe Perches
  0 siblings, 2 replies; 8+ messages in thread
From: Dwaipayan Ray @ 2020-10-17 16:27 UTC (permalink / raw)
  To: joe; +Cc: linux-kernel-mentees, dwaipayanray1, linux-kernel, lukas.bulwahn

Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test")
moved the repeated word test to check for more file types. But after
this, if checkpatch.pl is run on MAINTAINERS, it generates several
new warnings of the type:

WARNING: Possible repeated word: 'git'

For example:
WARNING: Possible repeated word: 'git'
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git

So, the pattern "git git://..." is a false positive in this case.

There are several other combinations which may produce a wrong
warning message, such as "@size size", ":Begin begin", etc.

Extend repeated word check to compare the characters before and
after the word matches.

If there is a non whitespace character before the first word or a
non whitespace character excluding punctuation characters after
the second word, then the check is skipped and the warning is avoided.

Also add case insensitive word matching to the repeated word check.

Link: https://lore.kernel.org/linux-kernel-mentees/81b6a0bb2c7b9256361573f7a13201ebcd4876f1.camel@perches.com/
Suggested-by: Joe Perches <joe@perches.com>
Suggested-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
Signed-off-by: Dwaipayan Ray <dwaipayanray1@gmail.com>
---
 scripts/checkpatch.pl | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index f1a4e61917eb..ec380626bebc 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -3052,19 +3052,30 @@ sub process {
 
 # check for repeated words separated by a single space
 		if ($rawline =~ /^\+/ || $in_commit_log) {
+			pos($rawline) = 1 if (!$in_commit_log);
 			while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {
 
 				my $first = $1;
 				my $second = $2;
-
+				my $start_pos = $-[1];
+				my $end_pos = $+[2];
 				if ($first =~ /(?:struct|union|enum)/) {
 					pos($rawline) += length($first) + length($second) + 1;
 					next;
 				}
 
-				next if ($first ne $second);
+				next if (lc($first) ne lc($second));
 				next if ($first eq 'long');
 
+				# check for character before and after the word matches
+				my $start_char = '';
+				my $end_char = '';
+				$start_char = substr($rawline, $start_pos - 1, 1) if ($start_pos > ($in_commit_log ? 0 : 1));
+				$end_char = substr($rawline, $end_pos, 1) if ($end_pos < length($rawline));
+
+				next if ($start_char =~ /^\S$/);
+				next if (index(" \t.,;?!", $end_char) == -1);
+
 				if (WARN("REPEATED_WORD",
 					 "Possible repeated word: '$first'\n" . $herecurr) &&
 				    $fix) {
-- 
2.27.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v5] checkpatch: add new exception to repeated word check
  2020-10-17 16:27 [PATCH v5] checkpatch: add new exception to repeated word check Dwaipayan Ray
@ 2020-10-17 16:33 ` Joe Perches
  2020-10-17 16:49   ` Dwaipayan Ray
  2020-10-18 18:04 ` Joe Perches
  1 sibling, 1 reply; 8+ messages in thread
From: Joe Perches @ 2020-10-17 16:33 UTC (permalink / raw)
  To: Dwaipayan Ray; +Cc: linux-kernel-mentees, linux-kernel, lukas.bulwahn

On Sat, 2020-10-17 at 21:57 +0530, Dwaipayan Ray wrote:
> Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test")
> moved the repeated word test to check for more file types. But after
> this, if checkpatch.pl is run on MAINTAINERS, it generates several
> new warnings of the type:

NAK.

Slow down and test before you send more patch versions.

> +				next if (index(" \t.,;?!", $end_char) == -1);

what does this do?



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v5] checkpatch: add new exception to repeated word check
  2020-10-17 16:33 ` Joe Perches
@ 2020-10-17 16:49   ` Dwaipayan Ray
  2020-10-18  6:20     ` Joe Perches
  0 siblings, 1 reply; 8+ messages in thread
From: Dwaipayan Ray @ 2020-10-17 16:49 UTC (permalink / raw)
  To: Joe Perches; +Cc: linux-kernel-mentees, linux-kernel, Lukas Bulwahn

On Sat, Oct 17, 2020 at 10:03 PM Joe Perches <joe@perches.com> wrote:
>
> On Sat, 2020-10-17 at 21:57 +0530, Dwaipayan Ray wrote:
> > Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test")
> > moved the repeated word test to check for more file types. But after
> > this, if checkpatch.pl is run on MAINTAINERS, it generates several
> > new warnings of the type:
>
> NAK.
>
> Slow down and test before you send more patch versions.
>
> > +                             next if (index(" \t.,;?!", $end_char) == -1);
>
> what does this do?

Um, it checks if end_char is not present in " \t.,;?!".
If end_char doesn't belong to this list, then the check shall
skip. That is the test will skip for "word word:", but will produce
a warning for "word word." or "word word?", etc.

Shouldn't this itself be the case or am I perhaps going wrong
somewhere?

Thanks,
Dwaipayan.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v5] checkpatch: add new exception to repeated word check
  2020-10-17 16:49   ` Dwaipayan Ray
@ 2020-10-18  6:20     ` Joe Perches
  2020-10-18  6:40       ` Dwaipayan Ray
  0 siblings, 1 reply; 8+ messages in thread
From: Joe Perches @ 2020-10-18  6:20 UTC (permalink / raw)
  To: Dwaipayan Ray; +Cc: linux-kernel-mentees, linux-kernel, Lukas Bulwahn

On Sat, 2020-10-17 at 22:19 +0530, Dwaipayan Ray wrote:
> On Sat, Oct 17, 2020 at 10:03 PM Joe Perches <joe@perches.com> wrote:
> > On Sat, 2020-10-17 at 21:57 +0530, Dwaipayan Ray wrote:
> > > Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test")
> > > moved the repeated word test to check for more file types. But after
> > > this, if checkpatch.pl is run on MAINTAINERS, it generates several
> > > new warnings of the type:
> > 
> > NAK.
> > 
> > Slow down and test before you send more patch versions.
> > 
> > > +                             next if (index(" \t.,;?!", $end_char) == -1);
> > 
> > what does this do?
> 
> Um, it checks if end_char is not present in " \t.,;?!".
> If end_char doesn't belong to this list, then the check shall
> skip. That is the test will skip for "word word:", but will produce
> a warning for "word word." or "word word?", etc.
> 
> Shouldn't this itself be the case or am I perhaps going wrong
> somewhere?

No, you were right, I was being a numbskull.

btw: I think this should set
	my $end_char = ' ';
not '' 

so that if the last word on a line is a
repeat the test still works.



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v5] checkpatch: add new exception to repeated word check
  2020-10-18  6:20     ` Joe Perches
@ 2020-10-18  6:40       ` Dwaipayan Ray
  2020-10-18  7:11         ` Joe Perches
  0 siblings, 1 reply; 8+ messages in thread
From: Dwaipayan Ray @ 2020-10-18  6:40 UTC (permalink / raw)
  To: Joe Perches; +Cc: linux-kernel-mentees, linux-kernel, Lukas Bulwahn

On Sun, Oct 18, 2020 at 11:50 AM Joe Perches <joe@perches.com> wrote:
>
> On Sat, 2020-10-17 at 22:19 +0530, Dwaipayan Ray wrote:
> > On Sat, Oct 17, 2020 at 10:03 PM Joe Perches <joe@perches.com> wrote:
> > > On Sat, 2020-10-17 at 21:57 +0530, Dwaipayan Ray wrote:
> > > > Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test")
> > > > moved the repeated word test to check for more file types. But after
> > > > this, if checkpatch.pl is run on MAINTAINERS, it generates several
> > > > new warnings of the type:
> > >
> > > NAK.
> > >
> > > Slow down and test before you send more patch versions.
> > >
> > > > +                             next if (index(" \t.,;?!", $end_char) == -1);
> > >
> > > what does this do?
> >
> > Um, it checks if end_char is not present in " \t.,;?!".
> > If end_char doesn't belong to this list, then the check shall
> > skip. That is the test will skip for "word word:", but will produce
> > a warning for "word word." or "word word?", etc.
> >
> > Shouldn't this itself be the case or am I perhaps going wrong
> > somewhere?
>
> No, you were right, I was being a numbskull.
>
> btw: I think this should set
>         my $end_char = ' ';
> not ''
>
> so that if the last word on a line is a
> repeat the test still works.
>
Hi,
Umm, index() function seems to return 0 when an empty string is
passed. I tried this:

print index(" \t.,;?!", '');

It output 0 in my case. So last words on a line seems to work.
I don't know if this changes with the perl version though.

So given this, will it be necessary to change end_char to ' ' ?
or perhaps change both start_char and end_char to a ' ' to maintain
uniformity?

Thanks,
Dwaipayan.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v5] checkpatch: add new exception to repeated word check
  2020-10-18  6:40       ` Dwaipayan Ray
@ 2020-10-18  7:11         ` Joe Perches
  2020-10-18 16:50           ` Dwaipayan Ray
  0 siblings, 1 reply; 8+ messages in thread
From: Joe Perches @ 2020-10-18  7:11 UTC (permalink / raw)
  To: Dwaipayan Ray; +Cc: linux-kernel-mentees, linux-kernel, Lukas Bulwahn

On Sun, 2020-10-18 at 12:10 +0530, Dwaipayan Ray wrote:
> print index(" \t.,;?!", '');
> 
> It output 0 in my case. So last words on a line seems to work.
> I don't know if this changes with the perl version though.
> 
> So given this, will it be necessary to change end_char to ' ' ?
> or perhaps change both start_char and end_char to a ' ' to maintain
> uniformity?

It seems it's fine even back to perl 5.6, so it's fine as is.

cheers, Joe



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v5] checkpatch: add new exception to repeated word check
  2020-10-18  7:11         ` Joe Perches
@ 2020-10-18 16:50           ` Dwaipayan Ray
  0 siblings, 0 replies; 8+ messages in thread
From: Dwaipayan Ray @ 2020-10-18 16:50 UTC (permalink / raw)
  To: Joe Perches; +Cc: linux-kernel-mentees, linux-kernel, Lukas Bulwahn

On Sun, Oct 18, 2020 at 12:41 PM Joe Perches <joe@perches.com> wrote:
>
> On Sun, 2020-10-18 at 12:10 +0530, Dwaipayan Ray wrote:
> > print index(" \t.,;?!", '');
> >
> > It output 0 in my case. So last words on a line seems to work.
> > I don't know if this changes with the perl version though.
> >
> > So given this, will it be necessary to change end_char to ' ' ?
> > or perhaps change both start_char and end_char to a ' ' to maintain
> > uniformity?
>
> It seems it's fine even back to perl 5.6, so it's fine as is.
>
> cheers, Joe
>
Okay thanks!
Please let me know if any other change is needed
or if it's good to go.

Regards,
Dwaipayan.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH v5] checkpatch: add new exception to repeated word check
  2020-10-17 16:27 [PATCH v5] checkpatch: add new exception to repeated word check Dwaipayan Ray
  2020-10-17 16:33 ` Joe Perches
@ 2020-10-18 18:04 ` Joe Perches
  1 sibling, 0 replies; 8+ messages in thread
From: Joe Perches @ 2020-10-18 18:04 UTC (permalink / raw)
  To: Dwaipayan Ray, Andrew Morton
  Cc: linux-kernel-mentees, linux-kernel, lukas.bulwahn

On Sat, 2020-10-17 at 21:57 +0530, Dwaipayan Ray wrote:
> Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test")
> moved the repeated word test to check for more file types. But after
> this, if checkpatch.pl is run on MAINTAINERS, it generates several
> new warnings of the type:

Andrew, can you pick this up please?

Acked-by: Joe Perches <joe@perches.com>

> WARNING: Possible repeated word: 'git'
> 
> For example:
> WARNING: Possible repeated word: 'git'
> +T:	git git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git
> 
> So, the pattern "git git://..." is a false positive in this case.
> 
> There are several other combinations which may produce a wrong
> warning message, such as "@size size", ":Begin begin", etc.
> 
> Extend repeated word check to compare the characters before and
> after the word matches.
> 
> If there is a non whitespace character before the first word or a
> non whitespace character excluding punctuation characters after
> the second word, then the check is skipped and the warning is avoided.
> 
> Also add case insensitive word matching to the repeated word check.
> 
> Link: https://lore.kernel.org/linux-kernel-mentees/81b6a0bb2c7b9256361573f7a13201ebcd4876f1.camel@perches.com/
> Suggested-by: Joe Perches <joe@perches.com>
> Suggested-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> Signed-off-by: Dwaipayan Ray <dwaipayanray1@gmail.com>
> ---
>  scripts/checkpatch.pl | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
> index f1a4e61917eb..ec380626bebc 100755
> --- a/scripts/checkpatch.pl
> +++ b/scripts/checkpatch.pl
> @@ -3052,19 +3052,30 @@ sub process {
>  
>  # check for repeated words separated by a single space
>  		if ($rawline =~ /^\+/ || $in_commit_log) {
> +			pos($rawline) = 1 if (!$in_commit_log);
>  			while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {
>  
>  				my $first = $1;
>  				my $second = $2;
> -
> +				my $start_pos = $-[1];
> +				my $end_pos = $+[2];
>  				if ($first =~ /(?:struct|union|enum)/) {
>  					pos($rawline) += length($first) + length($second) + 1;
>  					next;
>  				}
>  
> -				next if ($first ne $second);
> +				next if (lc($first) ne lc($second));
>  				next if ($first eq 'long');
>  
> +				# check for character before and after the word matches
> +				my $start_char = '';
> +				my $end_char = '';
> +				$start_char = substr($rawline, $start_pos - 1, 1) if ($start_pos > ($in_commit_log ? 0 : 1));
> +				$end_char = substr($rawline, $end_pos, 1) if ($end_pos < length($rawline));
> +
> +				next if ($start_char =~ /^\S$/);
> +				next if (index(" \t.,;?!", $end_char) == -1);
> +
>  				if (WARN("REPEATED_WORD",
>  					 "Possible repeated word: '$first'\n" . $herecurr) &&
>  				    $fix) {


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, back to index

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-17 16:27 [PATCH v5] checkpatch: add new exception to repeated word check Dwaipayan Ray
2020-10-17 16:33 ` Joe Perches
2020-10-17 16:49   ` Dwaipayan Ray
2020-10-18  6:20     ` Joe Perches
2020-10-18  6:40       ` Dwaipayan Ray
2020-10-18  7:11         ` Joe Perches
2020-10-18 16:50           ` Dwaipayan Ray
2020-10-18 18:04 ` Joe Perches

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git