From: Jim Meyering <jim@meyering.net>
To: Paul Eggert <eggert@cs.ucla.edu>
Cc: 60690@debbugs.gnu.org, demerphq@gmail.com,
"mega lith01" <megalith01@gmail.com>,
"Carlo Marcelo Arenas Belón" <carenas@gmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
git@vger.kernel.org, gitster@pobox.com,
"Tukusej’s Sirs" <tukusejssirs@protonmail.com>,
pcre-dev@exim.org
Subject: Re: bug#60690: -P '\d' in GNU and git grep
Date: Tue, 4 Apr 2023 08:31:44 -0700 [thread overview]
Message-ID: <CA+8g5KGGnsf0xMCXO28R1m8-z76=kG_AiYRh6=OgRL+x5C1yqQ@mail.gmail.com> (raw)
In-Reply-To: <920dcc8d-9e45-a03e-af06-6b420c6e0f81@cs.ucla.edu>
On Mon, Apr 3, 2023 at 11:47 PM Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 2023-04-03 20:30, Jim Meyering wrote:
> > have you seen justification
> > (other than for compatibility with some other tool or language) for
> > allowing \d to match non-ASCII by default, in spite of the risks?
>
> In the example Ævar supplied in <https://bugs.gnu.org/60690>, my
> impression was that it was better when \d matched non-ASCII digits. That
> is, in a UTF-8 locale it's better when \d finds matches in these lines:
>
> >> > git-gui/po/ja.po:"- 第1行: 何をしたか、を1行で要約。\n"
> >> > git-gui/po/ja.po:"- 第2行: 空白\n"
>
> because they contain the Japanese digits "1" and "2". This was the only
> example I recall being given.
Before it was unintentionally enabled in grep-3.9, lines like that have
never been matched by grep -P's '\d'. By relaxing \d, we'd weaken
any application that uses say grep -P '^\d+$' to perform input
validation intending to ensure that some input is all ASCII digits.
It's not a big stretch to imagine that some downstream processor
of that "verified" data is not prepared to deal with multi-byte digits.
> Also, I find it odd that grep -P '^[\w\d]*$' matches lines containing
> any sort of Arabic word characters, but it rejects lines containing
> Arabic digits like "٣" that are perfectly reasonable in Arabic-language
> text. I also find it odd that [\d] and [[:digit:]] mean different things.
>
> There are arguments on the other side, otherwise we wouldn't be having
> this discussion. And it's true that grep -P '\d' formerly rejected
> Arabic digits (though it's also true that grep -P '\w' formerly rejected
> Arabic letters...). Still, the cure's oddness and incompatibility with
> Git, Perl, etc. appears to me to be worse than the disease of dealing
> with grep -P invocations that need to use [0-9] or LC_ALL="C" anyway if
> they want to be portable to any program other than GNU grep.
I'm primarily concerned about not introducing a persistent regression in
how GNU grep's -P '\d' works in multibyte locales. The corner cases you
mention do matter, of course, but are far less likely to matter in practice.
next prev parent reply other threads:[~2023-04-04 15:32 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-08 6:23 [PATCH] grep: correctly identify utf-8 characters with \{b,w} in -P Carlo Marcelo Arenas Belón
2023-01-08 6:39 ` Junio C Hamano
2023-01-08 15:52 ` [PATCH v2] " Carlo Marcelo Arenas Belón
2023-01-09 11:35 ` Ævar Arnfjörð Bjarmason
2023-01-09 18:40 ` bug#60690: [PATCH v2] grep: correctly identify utf-8 characters with \{b, w} " Paul Eggert
2023-01-09 19:51 ` Ævar Arnfjörð Bjarmason
2023-01-09 23:12 ` Paul Eggert
2023-01-10 4:49 ` [PATCH v2] grep: correctly identify utf-8 characters with \{b,w} " Carlo Arenas
2023-01-16 20:48 ` Junio C Hamano
2023-04-03 21:38 ` -P '\d' in GNU and git grep Paul Eggert
2023-04-04 3:30 ` bug#60690: " Jim Meyering
2023-04-04 6:46 ` Paul Eggert
2023-04-04 15:31 ` Jim Meyering [this message]
2023-04-04 6:56 ` Carlo Arenas
2023-04-04 18:25 ` bug#60690: " Paul Eggert
2023-04-04 19:31 ` Junio C Hamano
2023-04-05 18:32 ` Paul Eggert
2023-04-05 19:04 ` Paul Eggert
2023-04-05 19:37 ` Junio C Hamano
2023-04-05 19:40 ` Jim Meyering
2023-04-05 20:03 ` Paul Eggert
2023-04-05 21:20 ` Carlo Arenas
2023-04-06 15:45 ` demerphq
2023-04-07 16:48 ` Paul Eggert
2023-04-06 13:39 ` demerphq
2023-04-07 19:00 ` Paul Eggert
2023-04-08 5:01 ` Carlo Arenas
2023-04-08 22:45 ` Paul Eggert
2023-01-17 10:51 ` [PATCH v3] grep: correctly identify utf-8 characters with \{b,w} in -P Carlo Marcelo Arenas Belón
2023-01-17 12:38 ` Ævar Arnfjörð Bjarmason
2023-01-17 15:19 ` Junio C Hamano
2023-01-18 7:35 ` Carlo Arenas
2023-01-18 11:49 ` Ævar Arnfjörð Bjarmason
2023-01-18 16:20 ` Junio C Hamano
2023-01-18 23:06 ` Ævar Arnfjörð Bjarmason
2023-01-18 23:24 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CA+8g5KGGnsf0xMCXO28R1m8-z76=kG_AiYRh6=OgRL+x5C1yqQ@mail.gmail.com' \
--to=jim@meyering.net \
--cc=60690@debbugs.gnu.org \
--cc=avarab@gmail.com \
--cc=carenas@gmail.com \
--cc=demerphq@gmail.com \
--cc=eggert@cs.ucla.edu \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=megalith01@gmail.com \
--cc=pcre-dev@exim.org \
--cc=tukusejssirs@protonmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).