Is there any interest in localizing term delimiters in git messages?

Is there any interest in localizing term delimiters in git messages?

[Alexander Shopov]

Hello all,

Is there any interest in being able to change the delimiters of the
changeable terms in git messages?

Typical example:
ORIGINAL
msgid "  (use \"git rm --cached <file>...\" to unstage)"

TRANSLATION
msgstr ""
"  (използвайте „git rm --cached %s ФАЙЛ…“, за да извадите ФАЙЛа от индекса)"

The important part are the `<' and `>' delimiters of the term "file"

Instead of using them - I omit them and capitalize the term. As if `<'
and `>' are declared as localizable and then I translate them as `',
`'

This has the following benefits:
1. The translation gets shorter
2. We skip potentially dangerous shell characters (<> redirect IN/OUT)
3. Readability improves for some strings, ex:
- git pack-objects [<options>] <base-name> [< <ref-list> | < <object-list>]
- git mailinfo [<options>] <msg> <patch> < mail >info

On the other hand - this can increase the maintenance burden of
messages and tests and the shortening benefit is applicable to
languages using capitalization or some other form of letter changing
that preserves readability (I understand there are many languages with
lots of speakers that are not like that). They might decide to convey
`<' and `>' as `«', `»' to get benefits 2 and 3.

So I am asking - is there any interest from other localizers to have
such a feature? Would the additional maintenance be OK for the
developers?

It is possible that no one besides me is interested in this - in which
case I will rework the Bulgarian translation as:
- More and more messages containing only the term automatically add
the `<' and `>'
- I need to keep adding smudge rules to the git-po-helper tool
(https://github.com/git-l10n/git-po-helper).

Kind regards:
al_shopov

Re: Is there any interest in localizing term delimiters in git messages?

[Junio C Hamano]

Alexander Shopov <ash@kambanaria.org> writes:

> Typical example:
> ORIGINAL
> msgid "  (use \"git rm --cached <file>...\" to unstage)"
>
> TRANSLATION
> msgstr ""
> "  (използвайте „git rm --cached %s ФАЙЛ…“, за да извадите ФАЙЛа от индекса)"
>
> The important part are the `<' and `>' delimiters of the term "file"
>
> Instead of using them - I omit them and capitalize the term. As if `<'
> and `>' are declared as localizable and then I translate them as `',
> `'

Is it because it is more common in your target language to omit <>
around the placeholder word, or is it just your personal preference?

Whichever is the case, I am not sure how it affects ...

> So I am asking - is there any interest from other localizers to have
> such a feature? Would the additional maintenance be OK for the
> developers?

... the maintenance burden for developers.  Perhaps I am not getting
what you are proposing, but we are not going to change the message
in "C" locale (the original you see in msgid).  In untranslated Git,
we will keep the convention to highlight the placeholder word by
having <> around it, so the "(use \"git rm --cached <file>...\" to
unstage)" message will be spelled with "<file>".  You can translate
that to a msgstr without <> markings without asking anybody's
permission, and I do not think of a reason why it would burden
developers to do so.

As long as the target audience of your translation wants to see
<file> to be translated to ФАЙЛ without <> around the word, I do
not think there is any problem doing so.  I of course am assuming
that using capitalized placeholder is the norm for all users who use
Bulgarian translated Git---if it is not some users want to see <>
around the placeholder word just like "C" locale, then you'd need to
answer your users wish first, or course, but that would not need to
concern the developers who write the "C" locale messages.

Thanks for helping Git easier to use for users with your language.

Re: Is there any interest in localizing term delimiters in git messages?

[Jiang Xin]

On Wed, Oct 18, 2023 at 5:50 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Alexander Shopov <ash@kambanaria.org> writes:
>
> > Typical example:
> > ORIGINAL
> > msgid "  (use \"git rm --cached <file>...\" to unstage)"
> >
> > TRANSLATION
> > msgstr ""
> > "  (използвайте „git rm --cached %s ФАЙЛ…“, за да извадите ФАЙЛа от индекса)"
> >
> > The important part are the `<' and `>' delimiters of the term "file"
> >
> > Instead of using them - I omit them and capitalize the term. As if `<'
> > and `>' are declared as localizable and then I translate them as `',
> > `'
>
> Is it because it is more common in your target language to omit <>
> around the placeholder word, or is it just your personal preference?
>
> Whichever is the case, I am not sure how it affects ...
>
> > So I am asking - is there any interest from other localizers to have
> > such a feature? Would the additional maintenance be OK for the
> > developers?
>
> ... the maintenance burden for developers.  Perhaps I am not getting
> what you are proposing, but we are not going to change the message
> in "C" locale (the original you see in msgid).  In untranslated Git,
> we will keep the convention to highlight the placeholder word by
> having <> around it, so the "(use \"git rm --cached <file>...\" to
> unstage)" message will be spelled with "<file>".  You can translate
> that to a msgstr without <> markings without asking anybody's
> permission, and I do not think of a reason why it would burden
> developers to do so.

Starting with the release of git 2.34.0 two years ago, we had a new
l10n pipeline and the git-po-helper tool as part of our l10n workflow.
The first version of git-po-helper introduced a validator to protect
git command parameters and variable names in megid. E.g. In pull
request 541 (https://github.com/git-l10n/git-po/pull/541), a
mismatched variable name "new_index" was reported in bg.po as below:

    level=warning msg="mismatch variable names in msgstr: new_index"
    level=warning msg=">> msgid: unable to write new_index file"
    level=warning msg=">> msgstr: новият индекс не може да бъде записан"

And po/bg.po changed as below:

    msgid "unable to write new_index file"
    msgstr "новият индекс (new_index) не може да бъде записан"

Later, more validators were introduced into git-po-helper for checking
git config name, place holders, etc. "git-po-helper" used a list of
regular expressions to find git config names, placeholders, and there
are some false positive cases need to be ignored. So I added tweaks in
smarge tables in "dict/*.go" of git-po-helper. E.g. For German
translation, there are two exceptions that need to be ignored:

    "e.g." was translated to "z.B.",
    "you@example.com" was translated to "ihre@emailadresse.de"

In pull request 593 (https://github.com/git-l10n/git-po/pull/593), it
was the first time I know that in Bulgarian translations, markers
around <placeholder> were not suitable for Bulgarian. So I decided to
add more tweaks for Bulgarian by adding more exception rules in
"dict/smudge-bg.go".

I wonder if Bulgarian can use some unique characters to wrap the
placeholders (e.g. Chinese can use wrappers around placeholders
like「placeholder」，【placeholder】，etc). It will be much simpler to
define exception rules for Bulgarian. Otherwize, maybe I can add
filters for validators in "po-helper", and Bulgarian can bypass some
validators to suppress warnings in pull requests.

--
Jiang Xin

Re: Is there any interest in localizing term delimiters in git messages?

[Junio C Hamano]

Jiang Xin <worldhello.net@gmail.com> writes:

> Starting with the release of git 2.34.0 two years ago, we had a new
> l10n pipeline and the git-po-helper tool as part of our l10n workflow.
> The first version of git-po-helper introduced a validator to protect
> git command parameters and variable names in megid.

Ahh, that is the piece I was missing.  I didn't know you guys are
doing extra checks that could trigger false positives.

> E.g. In pull
> request 541 (https://github.com/git-l10n/git-po/pull/541), a
> mismatched variable name "new_index" was reported in bg.po as below:
>
>     level=warning msg="mismatch variable names in msgstr: new_index"
>     level=warning msg=">> msgid: unable to write new_index file"
>     level=warning msg=">> msgstr: новият индекс не може да бъде записан"
>
> And po/bg.po changed as below:
>
>     msgid "unable to write new_index file"
>     msgstr "новият индекс (new_index) не може да бъде записан"

Wait.  Is this supposed to be a good example of validator working
well?  We use this exact message three times in builtin/commit.c; is
the validator insisting on the translated message to have verbatim
string "new_index" in it so that the end-users will see it?

I may still be confused, but if that is what is going on, I think it
is a wrong validation in this particular case.  I can understand if
we were creating say .git/new_index file and it helps the end users
to diagnose a troubled repository by running "ls .git" to see if a
file called "new_index" exists and getting in the way, but I do not
think it is the case.  A new file ".git/index.lock" is created via
repo_hold_locked_index() and I do not think it helps the end-user to
know that we may be calling it "new_index" internally among the
developers' circle.  If the message were about "index.lock", it
might be a different story, but such an error would probably have
been issued long before write_locked_index() gets called.

I'd suggest doing s/new_index/new index/ to msgid string for these
anyway.

> Later, more validators were introduced into git-po-helper for checking
> git config name, place holders, etc. "git-po-helper" used a list of
> regular expressions to find git config names, placeholders, and there
> are some false positive cases need to be ignored.

OK, and "<file>" in msgid string, for example, will automatically
insist on the translated msgstr string to have a string that is
enclosed by a pair of such angle brackets, regardless of the target
language convention?  If so, I can now understand where Alexander
comes from (assuming that the common convention in Bulgarian language
is not to use a pair of angle brackets to highlight such a placeholder
word).

I can see that you have a lot better handle on the matter than I do,
so I trust you and Alexander can resolve what the best "validation"
(and possibly override per language) should be in the git-po-helper
tool.

Thanks for explaining.

[PATCH] commit: do not use cryptic "new_index" in end-user facing messages

[Junio C Hamano]

These error messages say "new_index" as if that spelling has some
significance to the end users (e.g. the file "$GIT_DIR/new_index"
has some issues), but that is not the case at all.  The i18n folks
were made to include the word literally in the translated messages,
which was not a good idea at all.  Spell it "new index", as we are
just telling the users that we failed to create a new index file.
The term is expected to be translated to the end-users' languages,
not left as if it were a literal file name.

This dates all the way back to the first re-implemenation of "git
commit" command in C (the scripted version did not have such wording
in its error messages), in f5bbc322 (Port git commit to C.,
2007-11-08).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

    Junio C Hamano <gitster@pobox.com> writes:

    > Jiang Xin <worldhello.net@gmail.com> writes:
    > ...
    > Wait.  Is this supposed to be a good example of validator working
    > well?  We use this exact message three times in builtin/commit.c; is
    > the validator insisting on the translated message to have verbatim
    > string "new_index" in it so that the end-users will see it?
    > ...
    > I'd suggest doing s/new_index/new index/ to msgid string for these
    > anyway.

    Just so that we do not forget, in case my "using new_index in
    the message is wrong" is not my misunderstanding, here is such a
    patch.  Note that "checkout", "clone", "read-tree", and "stash"
    all use the "new index" spelling so this is not introducing any
    new message.  "git merge" and "git pull" that fast-forward also
    use this same message when they cannot write a new index file.

 builtin/commit.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 985a0445b7..7abd566bc7 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -446,7 +446,7 @@ static const char *prepare_index(const char **argv, const char *prefix,
 		refresh_cache_or_die(refresh_flags);
 		cache_tree_update(&the_index, WRITE_TREE_SILENT);
 		if (write_locked_index(&the_index, &index_lock, 0))
-			die(_("unable to write new_index file"));
+			die(_("unable to write new index file"));
 		commit_style = COMMIT_NORMAL;
 		ret = get_lock_file_path(&index_lock);
 		goto out;
@@ -470,7 +470,7 @@ static const char *prepare_index(const char **argv, const char *prefix,
 			cache_tree_update(&the_index, WRITE_TREE_SILENT);
 		if (write_locked_index(&the_index, &index_lock,
 				       COMMIT_LOCK | SKIP_IF_UNCHANGED))
-			die(_("unable to write new_index file"));
+			die(_("unable to write new index file"));
 		commit_style = COMMIT_AS_IS;
 		ret = get_index_file();
 		goto out;
@@ -518,7 +518,7 @@ static const char *prepare_index(const char **argv, const char *prefix,
 	refresh_index(&the_index, REFRESH_QUIET, NULL, NULL, NULL);
 	cache_tree_update(&the_index, WRITE_TREE_SILENT);
 	if (write_locked_index(&the_index, &index_lock, 0))
-		die(_("unable to write new_index file"));
+		die(_("unable to write new index file"));
 
 	hold_lock_file_for_update(&false_lock,
 				  git_path("next-index-%"PRIuMAX,
@@ -1852,7 +1852,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 
 	if (commit_index_files())
 		die(_("repository has been updated, but unable to write\n"
-		      "new_index file. Check that disk is not full and quota is\n"
+		      "new index file. Check that disk is not full and quota is\n"
 		      "not exceeded, and then \"git restore --staged :/\" to recover."));
 
 	git_test_write_commit_graph_or_die();
-- 
2.42.0-398-ga9ecda2788

Re: Is there any interest in localizing term delimiters in git messages?

[Jiang Xin]

On Wed, Oct 18, 2023 at 10:47 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Jiang Xin <worldhello.net@gmail.com> writes:
>
> > Starting with the release of git 2.34.0 two years ago, we had a new
> > l10n pipeline and the git-po-helper tool as part of our l10n workflow.
> > The first version of git-po-helper introduced a validator to protect
> > git command parameters and variable names in megid.
>
> Ahh, that is the piece I was missing.  I didn't know you guys are
> doing extra checks that could trigger false positives.
>
> > E.g. In pull
> > request 541 (https://github.com/git-l10n/git-po/pull/541), a
> > mismatched variable name "new_index" was reported in bg.po as below:
> >
> >     level=warning msg="mismatch variable names in msgstr: new_index"
> >     level=warning msg=">> msgid: unable to write new_index file"
> >     level=warning msg=">> msgstr: новият индекс не може да бъде записан"
> >
> > And po/bg.po changed as below:
> >
> >     msgid "unable to write new_index file"
> >     msgstr "новият индекс (new_index) не може да бъде записан"
>
> Wait.  Is this supposed to be a good example of validator working
> well?  We use this exact message three times in builtin/commit.c; is
> the validator insisting on the translated message to have verbatim
> string "new_index" in it so that the end-users will see it?
>
> I may still be confused, but if that is what is going on, I think it
> is a wrong validation in this particular case.  I can understand if
> we were creating say .git/new_index file and it helps the end users
> to diagnose a troubled repository by running "ls .git" to see if a
> file called "new_index" exists and getting in the way, but I do not
> think it is the case.  A new file ".git/index.lock" is created via
> repo_hold_locked_index() and I do not think it helps the end-user to
> know that we may be calling it "new_index" internally among the
> developers' circle.  If the message were about "index.lock", it
> might be a different story, but such an error would probably have
> been issued long before write_locked_index() gets called.
>
> I'd suggest doing s/new_index/new index/ to msgid string for these
> anyway.

I tried to find similar patterns in `po/bg.po` using:

    $ git  grep -h -B5 '([a-zA-Z_\.]*_[a-zA-Z_\.]\+)' po/bg.po

And find other translated variable names in Bulgarian as follows:

 * cookie_result in builtin/fsmonitor--daemon.c:

   error(_("fsmonitor: cookie_result '%d' != SEEN"),

 * run_command in builtin/submodule--helper.c:

    die(_("run_command returned non-zero status for %s\n."),
    die(_("run_command returned non-zero status while "

 * crlf_action in convert.c:

    warning(_("illegal crlf_action %d"), (int)crlf_action);

 * lazy_dir in name-hash.c:

    die(_("unable to create lazy_dir thread: %s"),

 * lazy_name in name-hash.c:

    die(_("unable to create lazy_name thread: %s"),
    die(_("unable to join lazy_name thread: %s"),

 * load_cache_entries in read-cache.c:

    die(_("unable to create load_cache_entries thread: %s"),
    die(_("unable to join load_cache_entries thread: %s"),

 * load_index_extensions in read-cache.c:

    die(_("unable to create load_index_extensions thread: %s"),
    die(_("unable to join load_index_extensions thread: %s"),

Apart from "new_index", it seems that none of the above sentences can
be rewritten simply by removing the underscores in variable names
without breaking the grammar, and I suppose it would be better to keep
those variable names unchanged.

Re: Is there any interest in localizing term delimiters in git messages?

[Junio C Hamano]

Jiang Xin <worldhello.net@gmail.com> writes:

> I tried to find similar patterns in `po/bg.po` using:
>
>     $ git  grep -h -B5 '([a-zA-Z_\.]*_[a-zA-Z_\.]\+)' po/bg.po
>
> And find other translated variable names in Bulgarian as follows:
> ...
> I suppose it would be better to keep those variable names
> unchanged.

To me, all of them refer to names given to variables, functions, and
mechanisms used internally as implementation details, and they are
meant to help developers diagnose when end-users hit these errors.

I agree with you that translating these would be counter-productive
for that purpose.

Having said that, I have to wonder if in an ideal world these should
be written in terms that are more end-user facing.

>  * cookie_result in builtin/fsmonitor--daemon.c:
>
>    error(_("fsmonitor: cookie_result '%d' != SEEN"),

[jch: cc'ed JeffH for area expertise]

For example, what does it mean to the end user when the
cookie->result we retrieve is different from FCIR_SEEN?  We lost
sync with the fsmonitor daemon backend and to avoid yielding
incorrect data we will be giving the "trivial" response only?  It is
not obvious from the code and b05880d3 (fsmonitor--daemon: use a
cookie file to sync with file system, 2022-03-25) that added it why
the end-user might even want to be shown this message [*].  I wonder
if this should be an untranslated trace2_* message that are meant
for debugging.

	Side note: and isn't the significance of the event
	    "warning", not "error"?  As far as the end-user is
	    concerned, after emitting this message

Also some of them might better be a BUG(), instead of die(_()).

>  * crlf_action in convert.c:
>
>     warning(_("illegal crlf_action %d"), (int)crlf_action);

[jch: cc'ed Torsten for area expertise].

For example, can convert.c::output_eol() be called with an illegal
crlf_action that is not covered by the switch() statement due to
data error, not a programming error?  From my quick scan, it looks
like that the error should never happen no matter what end-user
mistakes (e.g., misspelt attribute and configuration variable names
in their files) are fed to convert_attrs(), and can come only from a
bug in that function (e.g., long and convoluted if/else cascade fails
to assign any value to ca->crlf_action and leaves an undefined and
"illegal" value there).

Thanks.

Re: Is there any interest in localizing term delimiters in git messages?

[Jeff Hostetler]

On 10/19/23 1:52 PM, Junio C Hamano wrote:
> Jiang Xin <worldhello.net@gmail.com> writes:
> 
>> I tried to find similar patterns in `po/bg.po` using:
>>
>>      $ git  grep -h -B5 '([a-zA-Z_\.]*_[a-zA-Z_\.]\+)' po/bg.po
>>
>> And find other translated variable names in Bulgarian as follows:
>> ...
>> I suppose it would be better to keep those variable names
>> unchanged.
> 
> To me, all of them refer to names given to variables, functions, and
> mechanisms used internally as implementation details, and they are
> meant to help developers diagnose when end-users hit these errors.
> 
> I agree with you that translating these would be counter-productive
> for that purpose.
> 
> Having said that, I have to wonder if in an ideal world these should
> be written in terms that are more end-user facing.
> 
>>   * cookie_result in builtin/fsmonitor--daemon.c:
>>
>>     error(_("fsmonitor: cookie_result '%d' != SEEN"),
> 
> [jch: cc'ed JeffH for area expertise]
> 
> For example, what does it mean to the end user when the
> cookie->result we retrieve is different from FCIR_SEEN?  We lost
> sync with the fsmonitor daemon backend and to avoid yielding
> incorrect data we will be giving the "trivial" response only?  It is
> not obvious from the code and b05880d3 (fsmonitor--daemon: use a
> cookie file to sync with file system, 2022-03-25) that added it why
> the end-user might even want to be shown this message [*].  I wonder
> if this should be an untranslated trace2_* message that are meant
> for debugging.
> 
> 	Side note: and isn't the significance of the event
> 	    "warning", not "error"?  As far as the end-user is
> 	    concerned, after emitting this message
> 
> Also some of them might better be a BUG(), instead of die(_()).
...


Yeah, I think it should be an untranslated trace2 message rather
than an error.  You're right, the user cannot do anything with
that information -- and by emitting a "trivial" result, we fall
back to the normal behavior and cause the client to a regular
scan. So there is no reason to scare the user.

Jeff

Re: Is there any interest in localizing term delimiters in git messages?

[Junio C Hamano]

Jeff Hostetler <git@jeffhostetler.com> writes:

> Yeah, I think it should be an untranslated trace2 message rather
> than an error.  You're right, the user cannot do anything with
> that information -- and by emitting a "trivial" result, we fall
> back to the normal behavior and cause the client to a regular
> scan. So there is no reason to scare the user.

Thanks for a quick response.  Note that this was something we
discovered while talking about i18n and no immediate action is
required---it is not like we saw a report that tells us that end
users are actively getting confused.

THanks.

Re: Is there any interest in localizing term delimiters in git messages?

[Torsten Bögershausen]

On Thu, Oct 19, 2023 at 10:52:11AM -0700, Junio C Hamano wrote:

>
> Also some of them might better be a BUG(), instead of die(_()).
>
> >  * crlf_action in convert.c:
> >
> >     warning(_("illegal crlf_action %d"), (int)crlf_action);
>
> [jch: cc'ed Torsten for area expertise].
>
> For example, can convert.c::output_eol() be called with an illegal
> crlf_action that is not covered by the switch() statement due to
> data error, not a programming error?  From my quick scan, it looks
> like that the error should never happen no matter what end-user
> mistakes (e.g., misspelt attribute and configuration variable names
> in their files) are fed to convert_attrs(), and can come only from a
> bug in that function (e.g., long and convoluted if/else cascade fails
> to assign any value to ca->crlf_action and leaves an undefined and
> "illegal" value there).

The switch case covers all 8 values of "enum crlf_action",
and removing these 2 lines
 -	warning("Illegal crlf_action %d\n", (int)crlf_action);
 -	return core_eol;
does still compile without a compiler warning.
So yes, a BUG is more appropriate here.
I hopefully find some time to send a patch the next days.

>
> Thanks.
>

Re: Is there any interest in localizing term delimiters in git messages?

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> The switch case covers all 8 values of "enum crlf_action",
> and removing these 2 lines
>  -	warning("Illegal crlf_action %d\n", (int)crlf_action);
>  -	return core_eol;
> does still compile without a compiler warning.
> So yes, a BUG is more appropriate here.

Yeah, and if our expectation is whenever we add a new value to enum
convert_crlf_action, we will handle in and return from the switch
statement, so I agree with you that BUG() is more appropriate.

Thanks for a quick response.  Note that this was something we
discovered while talking about i18n and no immediate action is
required---it is not like we saw a report that tells us that end
users are actively getting confused by this message.

Thanks.

Re: Is there any interest in localizing term delimiters in git messages?

[Peter Krefting]

Alexander Shopov:

> Hello all,
>
> Is there any interest in being able to change the delimiters of the
> changeable terms in git messages?
>
> Typical example:
> ORIGINAL
> msgid "  (use \"git rm --cached <file>...\" to unstage)"

I think there should be something indicating the variables, and with 
Unicode there are better choices than the ASCII 
less-than-greater-than, for instance U+2039/U+203A. In the same way, 
we could also fix the quotation marks:

   "  (use “git rm --cached ‹file›...” to unstage)"

The source should perhaps still be ASCII-only to be compatible with 
older systems, but we could create a en_US.UTF-8 localization file 
that does the above, and apply similar changes to other localizations 
(I have been thinking about doing it to the Swedish translation for a 
while, but so far have not come around to; of course quoting differs 
from language to language, with different styles for ‘English’, 
“American”, „German”, ”Swedish” and «Norwegian», for instance; it is 
all very confusing and difficult to get right).

-- 
\\// Peter - http://www.softwolves.pp.se/