[PATCH] userdiff: update Ada patterns

[PATCH] userdiff: update Ada patterns

[Adrian Johnson]

- Allow extra space in "is new" and "is separate"
- Fix bug in word regex for numbers

Signed-off-by: Adrian Johnson <ajohnson@redneon.com>
---
 t/t4034/ada/expect | 2 +-
 userdiff.c         | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/t4034/ada/expect b/t/t4034/ada/expect
index be2376e..a682d28 100644
--- a/t/t4034/ada/expect
+++ b/t/t4034/ada/expect
@@ -4,7 +4,7 @@
 <BOLD>+++ b/post<RESET>
 <CYAN>@@ -1,13 +1,13 @@<RESET>
 Ada.Text_IO.Put_Line("Hello World<RED>!<RESET><GREEN>?<RESET>");
-1 1e<RED>-<RESET>10 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
+1 <RED>1e-10<RESET><GREEN>1e10<RESET> 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
 <RED>a<RESET><GREEN>x<RESET>+<RED>b a<RESET><GREEN>y x<RESET>-<RED>b<RESET>
 <RED>a<RESET><GREEN>y<RESET>
 <GREEN>x<RESET>*<RED>b a<RESET><GREEN>y x<RESET>/<RED>b<RESET>
diff --git a/userdiff.c b/userdiff.c
index ea43a03..e8915bf 100644
--- a/userdiff.c
+++ b/userdiff.c
@@ -15,13 +15,13 @@ static int drivers_alloc;
 	  word_regex "|[^[:space:]]|[\xc0-\xff][\x80-\xbf]+" }
 static struct userdiff_driver builtin_drivers[] = {
 IPATTERN("ada",
-	 "!^(.*[ \t])?(is new|renames|is separate)([ \t].*)?$\n"
+	 "!^(.*[ \t])?(is[ \t]+new|renames|is[ \t]+separate)([ \t].*)?$\n"
 	 "!^[ \t]*with[ \t].*$\n"
 	 "^[ \t]*((procedure|function)[ \t]+.*)$\n"
 	 "^[ \t]*((package|protected|task)[ \t]+.*)$",
 	 /* -- */
 	 "[a-zA-Z][a-zA-Z0-9_]*"
-	 "|[0-9][-+0-9#_.eE]"
+	 "|[-+0-9#_.eE]+"
 	 "|=>|\\.\\.|\\*\\*|:=|/=|>=|<=|<<|>>|<>"),
 IPATTERN("fortran",
 	 "!^([C*]|[ \t]*!)\n"
-- 
1.8.3.2

Re: [PATCH] userdiff: update Ada patterns

[Jeff King]

On Sun, Feb 02, 2014 at 09:21:56PM +1030, Adrian Johnson wrote:

> - Allow extra space in "is new" and "is separate"
> [...]
> -	 "!^(.*[ \t])?(is new|renames|is separate)([ \t].*)?$\n"
> +	 "!^(.*[ \t])?(is[ \t]+new|renames|is[ \t]+separate)([ \t].*)?$\n"

I do not know anything about Ada syntax, but this change looks obviously
sensible.

> - Fix bug in word regex for numbers
> -	 "|[0-9][-+0-9#_.eE]"
> +	 "|[-+0-9#_.eE]+"

This makes "E" or "_" a number. Is that right?

I think the intent of the original was "starts with a digit, and then
has one or more of these other things after it". You do not describe the
bug, but I guess it would be that it does not match obvious things like
"5". Should it be "zero or more" instead, like:

  [0-9][-+0-9#_.eE]*

? Also, should -/+ be hoisted to the front?

  [-+]?[0-9][0-9#_.eE]*

Again, I am just guessing, as I am not familiar enough with Ada.

-Peff

Re: [PATCH] userdiff: update Ada patterns

[Adrian Johnson]

On 03/02/14 10:05, Jeff King wrote:
> On Sun, Feb 02, 2014 at 09:21:56PM +1030, Adrian Johnson wrote:
>> - Fix bug in word regex for numbers
>> -	 "|[0-9][-+0-9#_.eE]"
>> +	 "|[-+0-9#_.eE]+"
> 
> This makes "E" or "_" a number. Is that right?
> 
> I think the intent of the original was "starts with a digit, and then
> has one or more of these other things after it". You do not describe the
> bug, but I guess it would be that it does not match obvious things like
> "5". Should it be "zero or more" instead, like:
> 
>   [0-9][-+0-9#_.eE]*

Yes, the original was missing the '*' at the end. 

I changed it to be similar to the number regexes used by the other builtin
patterns which are of the form '[-+0-9#_.eE]+'.

> 
> ? Also, should -/+ be hoisted to the front?
> 
>   [-+]?[0-9][0-9#_.eE]*

The other builtins don't do this. But it is probably better to have
[-+]?[0-9] at the front.

> Again, I am just guessing, as I am not familiar enough with Ada.

Ada numbers have the form:

- integers and reals eg 123, 1.23, 1e-2 ('.' can not be first)
- a '_' may be used between digits to improve readability eg 123_456
- base n (2 <= n <= 16) is of the form n#digits# eg 16#FFEF#
- base n numbers can include a radix point and/or an exponent
  eg 16#FF12.8#e-2
- Ada is case insensitive

After having another look I noticed it was missing the hex characters.
The new number regex I am proposing is:

[-+]?[0-9][0-9#_.aAbBcCdDeEfF]*([eE][+-]?[0-9_]+)?

I kept exponents containing a +/- sign separate from the digits
to prevent things like '1+2' from matching. I'll send an updated patch.

> 
> -Peff
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH v2] userdiff: update Ada patterns

[Adrian Johnson]

- Allow extra space in "is new" and "is separate"
- Fix bug in word regex for numbers

Signed-off-by: Adrian Johnson <ajohnson@redneon.com>
---
 t/t4034/ada/expect | 2 +-
 userdiff.c         | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/t/t4034/ada/expect b/t/t4034/ada/expect
index be2376e..a682d28 100644
--- a/t/t4034/ada/expect
+++ b/t/t4034/ada/expect
@@ -4,7 +4,7 @@
 <BOLD>+++ b/post<RESET>
 <CYAN>@@ -1,13 +1,13 @@<RESET>
 Ada.Text_IO.Put_Line("Hello World<RED>!<RESET><GREEN>?<RESET>");
-1 1e<RED>-<RESET>10 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
+1 <RED>1e-10<RESET><GREEN>1e10<RESET> 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
 <RED>a<RESET><GREEN>x<RESET>+<RED>b a<RESET><GREEN>y x<RESET>-<RED>b<RESET>
 <RED>a<RESET><GREEN>y<RESET>
 <GREEN>x<RESET>*<RED>b a<RESET><GREEN>y x<RESET>/<RED>b<RESET>
diff --git a/userdiff.c b/userdiff.c
index ea43a03..10b61ec 100644
--- a/userdiff.c
+++ b/userdiff.c
@@ -15,13 +15,13 @@ static int drivers_alloc;
 	  word_regex "|[^[:space:]]|[\xc0-\xff][\x80-\xbf]+" }
 static struct userdiff_driver builtin_drivers[] = {
 IPATTERN("ada",
-	 "!^(.*[ \t])?(is new|renames|is separate)([ \t].*)?$\n"
+	 "!^(.*[ \t])?(is[ \t]+new|renames|is[ \t]+separate)([ \t].*)?$\n"
 	 "!^[ \t]*with[ \t].*$\n"
 	 "^[ \t]*((procedure|function)[ \t]+.*)$\n"
 	 "^[ \t]*((package|protected|task)[ \t]+.*)$",
 	 /* -- */
 	 "[a-zA-Z][a-zA-Z0-9_]*"
-	 "|[0-9][-+0-9#_.eE]"
+	 "|[-+]?[0-9][0-9#_.aAbBcCdDeEfF]*([eE][+-]?[0-9_]+)?"
 	 "|=>|\\.\\.|\\*\\*|:=|/=|>=|<=|<<|>>|<>"),
 IPATTERN("fortran",
 	 "!^([C*]|[ \t]*!)\n"
-- 
1.8.3.2

Re: [PATCH v2] userdiff: update Ada patterns

[Junio C Hamano]

Adrian Johnson <ajohnson@redneon.com> writes:

> - Allow extra space in "is new" and "is separate"
> - Fix bug in word regex for numbers
>
> Signed-off-by: Adrian Johnson <ajohnson@redneon.com>
> ---
>  t/t4034/ada/expect | 2 +-
>  userdiff.c         | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/t/t4034/ada/expect b/t/t4034/ada/expect
> index be2376e..a682d28 100644
> --- a/t/t4034/ada/expect
> +++ b/t/t4034/ada/expect
> @@ -4,7 +4,7 @@
>  <BOLD>+++ b/post<RESET>
>  <CYAN>@@ -1,13 +1,13 @@<RESET>
>  Ada.Text_IO.Put_Line("Hello World<RED>!<RESET><GREEN>?<RESET>");
> -1 1e<RED>-<RESET>10 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
> +1 <RED>1e-10<RESET><GREEN>1e10<RESET> 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
>  <RED>a<RESET><GREEN>x<RESET>+<RED>b a<RESET><GREEN>y x<RESET>-<RED>b<RESET>
>  <RED>a<RESET><GREEN>y<RESET>
>  <GREEN>x<RESET>*<RED>b a<RESET><GREEN>y x<RESET>/<RED>b<RESET>
> diff --git a/userdiff.c b/userdiff.c
> index ea43a03..10b61ec 100644
> --- a/userdiff.c
> +++ b/userdiff.c
> @@ -15,13 +15,13 @@ static int drivers_alloc;
>  	  word_regex "|[^[:space:]]|[\xc0-\xff][\x80-\xbf]+" }
>  static struct userdiff_driver builtin_drivers[] = {
>  IPATTERN("ada",
> -	 "!^(.*[ \t])?(is new|renames|is separate)([ \t].*)?$\n"
> +	 "!^(.*[ \t])?(is[ \t]+new|renames|is[ \t]+separate)([ \t].*)?$\n"
>  	 "!^[ \t]*with[ \t].*$\n"
>  	 "^[ \t]*((procedure|function)[ \t]+.*)$\n"
>  	 "^[ \t]*((package|protected|task)[ \t]+.*)$",
>  	 /* -- */
>  	 "[a-zA-Z][a-zA-Z0-9_]*"
> -	 "|[0-9][-+0-9#_.eE]"
> +	 "|[-+]?[0-9][0-9#_.aAbBcCdDeEfF]*([eE][+-]?[0-9_]+)?"

This would match a lot wider than what I read you said you wanted to
match in your previous message.  Does "-04##4_3_2Ee-9" count as a
number, for example, or can we just ignore such syntactically
incorrect sequence?

>  	 "|=>|\\.\\.|\\*\\*|:=|/=|>=|<=|<<|>>|<>"),
>  IPATTERN("fortran",
>  	 "!^([C*]|[ \t]*!)\n"

Re: [PATCH v2] userdiff: update Ada patterns

[Adrian Johnson]

On 04/02/14 06:30, Junio C Hamano wrote:
> Adrian Johnson <ajohnson@redneon.com> writes:
> 
>> - Allow extra space in "is new" and "is separate"
>> - Fix bug in word regex for numbers
>>
>> Signed-off-by: Adrian Johnson <ajohnson@redneon.com>
>> ---
>>  t/t4034/ada/expect | 2 +-
>>  userdiff.c         | 4 ++--
>>  2 files changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/t/t4034/ada/expect b/t/t4034/ada/expect
>> index be2376e..a682d28 100644
>> --- a/t/t4034/ada/expect
>> +++ b/t/t4034/ada/expect
>> @@ -4,7 +4,7 @@
>>  <BOLD>+++ b/post<RESET>
>>  <CYAN>@@ -1,13 +1,13 @@<RESET>
>>  Ada.Text_IO.Put_Line("Hello World<RED>!<RESET><GREEN>?<RESET>");
>> -1 1e<RED>-<RESET>10 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
>> +1 <RED>1e-10<RESET><GREEN>1e10<RESET> 16#FE12#E2 3.141_592 '<RED>x<RESET><GREEN>y<RESET>'
>>  <RED>a<RESET><GREEN>x<RESET>+<RED>b a<RESET><GREEN>y x<RESET>-<RED>b<RESET>
>>  <RED>a<RESET><GREEN>y<RESET>
>>  <GREEN>x<RESET>*<RED>b a<RESET><GREEN>y x<RESET>/<RED>b<RESET>
>> diff --git a/userdiff.c b/userdiff.c
>> index ea43a03..10b61ec 100644
>> --- a/userdiff.c
>> +++ b/userdiff.c
>> @@ -15,13 +15,13 @@ static int drivers_alloc;
>>  	  word_regex "|[^[:space:]]|[\xc0-\xff][\x80-\xbf]+" }
>>  static struct userdiff_driver builtin_drivers[] = {
>>  IPATTERN("ada",
>> -	 "!^(.*[ \t])?(is new|renames|is separate)([ \t].*)?$\n"
>> +	 "!^(.*[ \t])?(is[ \t]+new|renames|is[ \t]+separate)([ \t].*)?$\n"
>>  	 "!^[ \t]*with[ \t].*$\n"
>>  	 "^[ \t]*((procedure|function)[ \t]+.*)$\n"
>>  	 "^[ \t]*((package|protected|task)[ \t]+.*)$",
>>  	 /* -- */
>>  	 "[a-zA-Z][a-zA-Z0-9_]*"
>> -	 "|[0-9][-+0-9#_.eE]"
>> +	 "|[-+]?[0-9][0-9#_.aAbBcCdDeEfF]*([eE][+-]?[0-9_]+)?"
> 
> This would match a lot wider than what I read you said you wanted to
> match in your previous message.  Does "-04##4_3_2Ee-9" count as a
> number, for example, or can we just ignore such syntactically
> incorrect sequence?

Maybe I am misunderstanding the purpose of the word diff regexes. I
thought the purpose of the word regex is to split lines into words, not
determine what is syntactically correct.

For example decimal number regex for pascal is: [-+0-9.e]+
and for cpp: [-+0-9.e]+[fFlL]?

These will obviously match stuff that is not a number.

> 
>>  	 "|=>|\\.\\.|\\*\\*|:=|/=|>=|<=|<<|>>|<>"),
>>  IPATTERN("fortran",
>>  	 "!^([C*]|[ \t]*!)\n"
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH v2] userdiff: update Ada patterns

[Junio C Hamano]

Adrian Johnson <ajohnson@redneon.com> writes:

>>> -	 "|[0-9][-+0-9#_.eE]"
>>> +	 "|[-+]?[0-9][0-9#_.aAbBcCdDeEfF]*([eE][+-]?[0-9_]+)?"
>> 
>> This would match a lot wider than what I read you said you wanted to
>> match in your previous message.  Does "-04##4_3_2Ee-9" count as a
>> number, for example, or can we just ignore such syntactically
>> incorrect sequence?
>
> Maybe I am misunderstanding the purpose of the word diff regexes. I
> thought the purpose of the word regex is to split lines into words, not
> determine what is syntactically correct.

I agree that the purpose is former---So you could have just said
"the latter" ;-).

Any other nitpick, anybody?  Otherwise I'll queue this version.

Re: [PATCH v2] userdiff: update Ada patterns

[Jeff King]

On Wed, Feb 05, 2014 at 09:17:47AM -0800, Junio C Hamano wrote:

> Adrian Johnson <ajohnson@redneon.com> writes:
> 
> >>> -	 "|[0-9][-+0-9#_.eE]"
> >>> +	 "|[-+]?[0-9][0-9#_.aAbBcCdDeEfF]*([eE][+-]?[0-9_]+)?"
> >> 
> >> This would match a lot wider than what I read you said you wanted to
> >> match in your previous message.  Does "-04##4_3_2Ee-9" count as a
> >> number, for example, or can we just ignore such syntactically
> >> incorrect sequence?
> >
> > Maybe I am misunderstanding the purpose of the word diff regexes. I
> > thought the purpose of the word regex is to split lines into words, not
> > determine what is syntactically correct.
> 
> I agree that the purpose is former---So you could have just said
> "the latter" ;-).
> 
> Any other nitpick, anybody?  Otherwise I'll queue this version.

No nitpick here, I had the same thought as Adrian while reading the
thread (and if somebody comes up with a real case where the output looks
bad, we can iterate on it).

-Peff

Re: [PATCH v2] userdiff: update Ada patterns

[Junio C Hamano]

"George Spelvin" <linux@horizon.com> writes:

> Another point is that Ada doesn't actually include leading + or -
> signs in the syntax for "number", but rather makes them unary operators.
> This means that spaces are allowed, and whether you want to include them
> in the "number" pattern is a judgement call.

I tend to agree.  What do the patterns for other languages do?

Re: [PATCH v2] userdiff: update Ada patterns

[George Spelvin]

Looking at the grammar at http://www.adahome.com/rm95/rm9x-P.html
and http://www.adaic.org/resources/add_content/standards/05rm/html/RM-2-4.html
I see the following restrictions apply:

- A number must begin and end with a digit.  There must be at least one
  digit on either side of each underscore and decimal point.
- There may be at most one decimal point in a number.
- The base must be between 2 and 16, inclusive.

[-+]?(
        [0-9](_?[0-9])*([.][0-9](_?[0-9])*)?|
        ([2-9]|1[0-6])#[0-9a-fA-F](_?[0-9a-fA-F])*([.][0-9a-fA-F](_?[0-9a-fA-F])*)?#
)([eE][+-]?[0-9](_?[0-9])*)?

If you want a slightly smaller version, we can drop the "only one decimal
point" rule and get

[-+]?(
        [0-9]([._]?[0-9])*|
        ([2-9]|1[0-6])#[0-9a-fA-F]([._]?[0-9a-fA-F])*#
)([eE][+-]?[0-9](_?[0-9])*)?

Ideally, for a based number, the range of acceptable digits would
depend on the base.  Which is possible for a finite-state machine since
the number of bases is finite, but laeds to a bit of a state explosion.
Here's an approximation that separates bases 2-9 and 10-16:

[-+]?(
        [0-9](_?[0-9])*([.][0-9](_?[0-9])*)?|
        [2-9]#[0-8](_?[0-8])*([.][0-8](_?[0-8])*)?#|
        1[0-6]#[0-9a-fA-F](_?[0-9a-fA-F])*([.][0-9a-fA-F](_?[0-9a-fA-F])*)?#
)([eE][+-]?[0-9](_?[0-9])*)?

because I think that this is overdoing it:

[-+]?(
        [0-9](_?[0-9])*([.][0-9](_?[0-9])*)?|
        2#[0-1](_?[0-1])*([.][0-1](_?[0-1])*)?#|
        3#[0-2](_?[0-2])*([.][0-2](_?[0-2])*)?#|
        4#[0-3](_?[0-3])*([.][0-3](_?[0-3])*)?#|
        5#[0-4](_?[0-4])*([.][0-4](_?[0-4])*)?#|
        6#[0-5](_?[0-5])*([.][0-5](_?[0-5])*)?#|
        7#[0-5](_?[0-5])*([.][0-5](_?[0-5])*)?#|
        8#[0-7](_?[0-7])*([.][0-7](_?[0-7])*)?#|
        9#[0-8](_?[0-8])*([.][0-8](_?[0-8])*)?#|
        10#[0-9](_?[0-9])*([.][0-9](_?[0-9])*)?#|
        11#[0-9aA](_?[0-9aA])*([.][0-9aA](_?[0-9aA])*)?#|
        12#[0-9abAB](_?[0-9abAB])*([.][0-9abAB](_?[0-9abAB])*)?#|
        13#[0-9a-cA-C](_?[0-9a-cA-C])*([.][0-9a-cA-C](_?[0-9a-cA-C])*)?#|
        14#[0-9a-dA-D](_?[0-9a-dA-D])*([.][0-9a-dA-D](_?[0-9a-dA-D])*)?#|
        15#[0-9a-eA-E](_?[0-9a-eA-E])*([.][0-9a-eA-E](_?[0-9a-eA-E])*)?#|
        16#[0-9a-fA-F](_?[0-9a-fA-F])*([.][0-9a-fA-F](_?[0-9a-fA-F])*)?#|
)([eE][+-]?[0-9](_?[0-9])*)?

Another point is that Ada doesn't actually include leading + or -
signs in the syntax for "number", but rather makes them unary operators.
This means that spaces are allowed, and whether you want to include them
in the "number" pattern is a judgement call.