l10n: the maintaince of git-po repo

l10n: the maintaince of git-po repo

[Jiang Xin]

I'm looking forward to the next git release with Chinese l10n, so that
I can start to rewrite my book on Git. ;-)

The maintaince of the git-po and git-po-zh_CN repositories so far:

git-po repository
-----------------
Location   : https://github.com/gotgit/git-po
Fork from  : https://github.com/git/git
Description: This is the Git l10n master repo, and is used to coordinate
             works of l10n teams. L10n team leaders send pull requests
             to this repo, while other l10n contributors should wok on
             the language specified repository created by the l10n team
             leader.
Typical commits:

- commit https://github.com/gotgit/git-po/commit/71eb878

      merge the pull request sent from the Chinese l10n team leader.

- commit https://github.com/gotgit/git-po/commit/1d4ac88

      l10n: Update git.pot (12 new messages)

- commit https://github.com/gotgit/git-po/commit/508d124

      Merge branch 'master' of 'git.git' into git-po,
      and will generate new po/git.pot in next commit.

- commit https://github.com/gotgit/git-po/commit/dce37b6

      l10n: initial git.pot for 1.7.10 upcoming release


git-po-zh_CN repository
-----------------------
Location   : https://github.com/jiangxin/git-po-zh_CN
Fork from  : https://github.com/gotgit/git-po
Description: Git l10n for Simplified Chinese.
Typical commits:

- commit https://github.com/jiangxin/git-po-zh_CN/commit/7ac1c0a

      l10n: update Chinese translation to the new git.po

- commit https://github.com/jiangxin/git-po-zh_CN/commit/0ad9e96

      fix of Chinese translation

- commit https://github.com/jiangxin/git-po-zh_CN/commit/d4a8e00

      fix of Chinese translation

- commit https://github.com/jiangxin/git-po-zh_CN/commit/c04f5ac

      l10n: initial of zh_CN l10n


-- 
Jiang Xin

Re: l10n: the maintaince of git-po repo

[Junio C Hamano]

Jiang Xin <worldhello.net@gmail.com> writes:

> I'm looking forward to the next git release with Chinese l10n, so that
> I can start to rewrite my book on Git. ;-)

For people who don't know, Jiang kindly sent me an autographed copy of his
book https://plus.google.com/u/0/108182106411180467879/posts/XX2CewgxSAb I
can only read the command examples, but the book seems to have a good
coverage of modern Git, judging from the table of contents.

> The maintaince of the git-po and git-po-zh_CN repositories so far:
>
> git-po repository
> -----------------
> Location   : https://github.com/gotgit/git-po
> Fork from  : https://github.com/git/git
> Description: This is the Git l10n master repo, and is used to coordinate
>              works of l10n teams. L10n team leaders send pull requests
>              to this repo, while other l10n contributors should wok on
>              the language specified repository created by the l10n team
>              leader.

Thanks.  I took a look.

I'll comment on the changes as if they came in the patch form here, to ask
for sanity checks from the list members.

>  Makefile      |    1 -
>  po/.gitignore |    1 -
>  po/git.pot    | 3494 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  po/zh_CN.po   | 3671 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 7165 insertions(+), 2 deletions(-)
>  create mode 100644 po/git.pot
>  create mode 100644 po/zh_CN.po
> 
> 
> diff --git a/Makefile b/Makefile
> index cf2c40b..be1957a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -2635,7 +2635,6 @@ dist-doc:
>  
>  distclean: clean
>  	$(RM) configure
> -	$(RM) po/git.pot
>  
>  profile-clean:
>  	$(RM) $(addsuffix *.gcda,$(addprefix $(PROFILE_DIR)/, $(object_dirs)))

The file is no longer a generated file but is tracked, and should not be
removed upon distclean.  Good.

> diff --git a/po/.gitignore b/po/.gitignore
> index 4caa631..796b96d 100644
> --- a/po/.gitignore
> +++ b/po/.gitignore
> @@ -1,2 +1 @@
> -/git.pot
>  /build

Likewise, it is not ignored anymore. Good.

> diff --git a/po/git.pot b/po/git.pot
> new file mode 100644
> index 0000000..efcda39
> --- /dev/null
> +++ b/po/git.pot
> @@ -0,0 +1,3494 @@
> +# SOME DESCRIPTIVE TITLE.
> +# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
> +# This file is distributed under the same license as the PACKAGE package.

Should the above placeholder lines left as they are, or should they be
updated to name "Git" as the "PACKAGE" with "SOME DESCRIPTIVE TITLE"?

If the answer is "latter", I think we would need to apply a similar update
to po/git-gui.pot for git-gui package.

On the other hand, I think the placeholder lines below should stay as they
are.

> +# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
> +#
> +#, fuzzy
> +msgid ""
> +msgstr ""
> +"Project-Id-Version: PACKAGE VERSION\n"
> +"Report-Msgid-Bugs-To: Git Mailing List <git@vger.kernel.org>\n"
> +"POT-Creation-Date: 2012-02-28 09:17+0800\n"
> +"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
> +"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
> +"Language-Team: LANGUAGE <LL@li.org>\n"
> +"Language: \n"
> +"MIME-Version: 1.0\n"
> +"Content-Type: text/plain; charset=CHARSET\n"
> +"Content-Transfer-Encoding: 8bit\n"
> +"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"
> +
> +#: advice.c:34
> +#, c-format
> +msgid "hint: %.*s\n"
> +msgstr ""
> ...

[PATCH] Update l10n guide

[Junio C Hamano]

Describe the high-level overview of the localization workflow, and update
the description on po/git.pot file, which no longer is an untracked file.

Clarify the roles and responsibilities among:

 - the core developers (responsible for marking the translatable strings
   and restructuring the code to avoid message-lego);

 - the l10n coordinator (maintaining po/git.pot, interfacing with language
   teams and feeding updated translations back to the core); and

 - the language teams (updating po/*.po for their language and feeding
   their work back to the l10n coordinator).

This is largely based on an earlier message $gmane/189584 from me.

  Date: Wed, 01 Feb 2012 20:56:18 -0800
  Message-ID: <7vmx91ygst.fsf@alter.siamese.dyndns.org>

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * I would like to see something like this before we start getting the
   second language team.  Comments are very much appreciated.

   I am not sure if the existing instruction to update the new XX.po
   file copied from git.pot is correct, but I left it as-is.

    @@ -1,6 +1,6 @@
    -# Icelandic translations for PACKAGE package.
    -# Copyright (C) 2010 THE PACKAGE'S COPYRIGHT HOLDER
    -# This file is distributed under the same license as the PACKAGE package.
    +# Icelandic translations for Git.
    +# Copyright (C) 2010 Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +# This file is distributed under the same license as the Git package.
     # Ævar Arnfjörð Bjarmason <avarab@gmail.com>, 2010.

   The second line of the template tells us to name the copyright holder
   of the package, not of the translation, but the above example makes it
   appear as if we are assigning the copyright to the translator.

 po/README |  106 ++++++++++++++++++++++++++++++++++++++++++++++++-------------
 po/TEAMS  |   10 ++++++
 2 files changed, 94 insertions(+), 22 deletions(-)
 create mode 100644 po/TEAMS

diff --git a/po/README b/po/README
index 10b0ad2..6480882 100644
--- a/po/README
+++ b/po/README
@@ -1,33 +1,78 @@
 Core GIT Translations
 =====================
 
-This directory holds the translations for the core of Git. This
-document describes how to add to and maintain these translations, and
-how to mark source strings for translation.
-
+This directory holds the translations for the core of Git. This document
+describes how you can contribute to the effort of enhancing the language
+coverage and maintaining the translation.
+
+The localization (l10n) coordinator, Jiang Xin <worldhello.net@gmail.com>,
+coordinates our localization effort in his repository:
+
+        https://github.com/gotgit/git-po/
+
+As a contributor for a language XX, you would fork this repository,
+prepare and/or update the translated message file po/XX.po (described
+later), and ask the l10n coordinator to pull your work.
+
+If there are multiple contributors for the same language, please first
+coordinate among yourselves and nominate the team leader for your
+language, so that the l10n coordinator only needs to interact with one
+person per language.
+
+For the list of exiting translations and language teams, see TEAMS file in
+this directory.
+
+The overall data-flow looks like this:
+
+    +-------------------+            +------------------+
+    | Git source code   | ---(1)---> | L10n coordinator |
+    | repository        | <---(4)--- | repository       |
+    +-------------------+            +------------------+
+                                          |      ^
+                                         (2)    (3)
+                                          V      |
+                                     +------------------+
+                                     | Language Team XX |
+                                     +------------------+
+
+ * Translatable strings are marked in the source file.
+ * L10n coordinator pulls from the source (1)
+ * L10n coordinator updates the message template po/git.pot
+ * Language team pulls from L10n coordinator (2)
+ * Language team updates the message file po/XX.po
+ * L10n coordinator pulls from Language team (3)
+ * L10n coordinator asks the result to be pulled (4).
+
+
+Maintaining the po/git.pot file
+-------------------------------
 
-Generating a .pot file
-----------------------
+(This is done by the l10n coordinator).
 
 The po/git.pot file contains a message catalog extracted from Git's
-sources. You need to generate it to add new translations with
-msginit(1), or update existing ones with msgmerge(1).
+sources. The l10n coordinator maintains it by adding new translations with
+msginit(1), or update existing ones with msgmerge(1).  In order to update
+the Git sources to extract the messages from, the l10n coordinator is
+expected to pull from the main git repository at strategic point in
+history (e.g. when a major release and release candidates are tagged),
+and then run "make pot" at the top-level directory.
 
-Since the file can be automatically generated it's not checked into
-git.git. To generate it do, at the top-level:
+Language contributors use this file to prepare translations for their
+language, but they are not expected to modify it.
 
-    make pot
 
+Initializing a XX.po file
+-------------------------
 
-Initializing a .po file
------------------------
+(This is done by the language teams).
 
-To add a new translation first generate git.pot (see above) and then
-in the po/ directory do:
+If your language XX does not have translated message file po/XX.po yet,
+you add a translation for the first time by running:
 
     msginit --locale=XX
 
-Where XX is your locale, e.g. "is", "de" or "pt_BR".
+in the po/ directory, where XX is the locale, e.g. "de", "is", "pt_BR",
+"zh_CN", etc.
 
 Then edit the automatically generated copyright info in your new XX.po
 to be correct, e.g. for Icelandic:
@@ -46,21 +91,36 @@ just "Git":
 
     perl -pi -e 's/(?<="Project-Id-Version: )PACKAGE VERSION/Git/' XX.po
 
+Once you are done testing the translation (see below), commit the result
+and ask the l10n coordinator to pull from you.
+
+
+Updating a XX.po file
+---------------------
 
-Updating a .po file
--------------------
+(This is done by the language teams).
 
-If there's an existing *.po file for your language but you need to
-update the translation you first need to generate git.pot (see above)
-and then in the po/ directory do:
+If you are replacing translation strings in an existing XX.po file to
+improve the translation, just edit the file.
+
+If there's an existing XX.po file for your language, but the repository
+of the l10n coordinator has newer po/git.pot file, you would need to first
+pull from the l10n coordinator (see the beginning of this document for its
+URL), and then update the existing translation by running:
 
     msgmerge --add-location --backup=off -U XX.po git.pot
 
-Where XX.po is the file you want to update.
+in the po/ directory, where XX.po is the file you want to update.
+
+Once you are done testing the translation (see below), commit the result
+and ask the l10n coordinator to pull from you.
+
 
 Testing your changes
 --------------------
 
+(This is done by the language teams, after creating or updating XX.po file).
+
 Before you submit your changes go back to the top-level and do:
 
     make
@@ -75,6 +135,8 @@ with a newline or not.
 Marking strings for translation
 -------------------------------
 
+(This is done by the core developers).
+
 Before strings can be translated they first have to be marked for
 translation.
 
diff --git a/po/TEAMS b/po/TEAMS
new file mode 100644
index 0000000..8ee6199
--- /dev/null
+++ b/po/TEAMS
@@ -0,0 +1,10 @@
+Core Git translation language teams
+(please keep the list sorted alphabetically on language field)
+
+Language:	is (Icelandic)
+Leader:		Ævar Arnfjörð Bjarmason <avarab@gmail.com>
+
+Language:	zh_CN (Simplified Chinese)
+Leader:		Jiang Xin <worldhello.net@gmail.com>
+Members:	Yichao Yu <yyc1992@gmail.com>
+		Riku <lu.riku@gmail.com>
-- 
1.7.9.2.344.g3e8c86

Re: [PATCH] Update l10n guide

[Jiang Xin]

2012/2/29 Junio C Hamano <gitster@pobox.com>:

> @@ -1,33 +1,78 @@
> +The localization (l10n) coordinator, Jiang Xin <worldhello.net@gmail.com>,
> +coordinates our localization effort in his repository:
> +
> +        https://github.com/gotgit/git-po/

I create a new dedicated team, named git-l10n, and transfer this repo to the
new place: https://github.com/git-l10n/git-po/ for the following reasons:

 - git-po is a long-term repo, so has a dedicated, stabilized URL is better.
   The repo URL may not change if l10n coordinator change.

 - git-l10n is a github organization account, and has a dedicated owners
   group, who can assign new l10n coordinator.

 - Owners group of git-l10n organization will not receive pull requests to
   git-po. I add gister, avar to the owners team.

 - Members of the git-l10n-coordinators team of git-l10n organization
   will receive pull requests.

 - Some Chinese guys used to send pull request to gotgit/git-po, so you
   can find some closed pull requests written in Chinese. It won't happen
   after the repo transfer to git-l10n/git-po.

> +As a contributor for a language XX, you would fork this repository,

As a contributor for a language XX, you should first check TEAMS file in
this directory to see whether a dedicated repository for your language XX
exists. Fork the dedicated repository and start to work if it exists.

If you are the first contributor for the language XX, please fork this
repository,

> +prepare and/or update the translated message file po/XX.po (described
> +later), and ask the l10n coordinator to pull your work.

Shall we remind the contributors for l10n to write commit log in English,
but not language XX? It's a common mistake for translators.

> diff --git a/po/TEAMS b/po/TEAMS
> new file mode 100644
> index 0000000..8ee6199
> --- /dev/null
> +++ b/po/TEAMS
> @@ -0,0 +1,10 @@
> +Core Git translation language teams
> +(please keep the list sorted alphabetically on language field)
> +
> +Language:      is (Icelandic)
> +Leader:                Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> +

I think we should add the URL of the dedicated repository for language XX here.
Example:

Language:      zh_CN (Simplified Chinese)
Leader:           Jiang Xin <worldhello.net@gmail.com>
Homepage:     https://github.com/jiangxin/git-po-zh_CN/


> +Members:       Yichao Yu <yyc1992@gmail.com>
> +               Riku <lu.riku@gmail.com>

Contributors and members can add to the header of the XX.po file.
Add here may cause the TEAMS file grows rapidly.


-- 
Jiang Xin

Re: [PATCH] Update l10n guide

[Junio C Hamano]

Jiang Xin <worldhello.net@gmail.com> writes:

> 2012/2/29 Junio C Hamano <gitster@pobox.com>:
>
>> @@ -1,33 +1,78 @@
>> +The localization (l10n) coordinator, Jiang Xin <worldhello.net@gmail.com>,
>> +coordinates our localization effort in his repository:
>> +
>> +        https://github.com/gotgit/git-po/
>
> I create a new dedicated team, named git-l10n, and transfer this repo to the
> new place: https://github.com/git-l10n/git-po/ for the following reasons:
>
>  - git-po is a long-term repo, so has a dedicated, stabilized URL is better.
>    The repo URL may not change if l10n coordinator change.
>
>  - git-l10n is a github organization account, and has a dedicated owners
>    group, who can assign new l10n coordinator.
>
>  - Owners group of git-l10n organization will not receive pull requests to
>    git-po. I add gister, avar to the owners team.
>
>  - Members of the git-l10n-coordinators team of git-l10n organization
>    will receive pull requests.
>
>  - Some Chinese guys used to send pull request to gotgit/git-po, so you
>    can find some closed pull requests written in Chinese. It won't happen
>    after the repo transfer to git-l10n/git-po.

It is unclear the distinction between gotgit and git-l10n but I am
guessing that (1) you feel "gotgit" is more for _your_ personal use, (2)
you happen to be the volunteer coordinator right now, but (3) the team
effort deserves more permanent home, detached from your identity.

And if that is the case, I agree 100% with the reasoning.

>> +As a contributor for a language XX, you would fork this repository,
>
> As a contributor for a language XX, you should first check TEAMS file in
> this directory to see whether a dedicated repository for your language XX
> exists. Fork the dedicated repository and start to work if it exists.
>
> If you are the first contributor for the language XX, please fork this
> repository,

That's better.

>> +prepare and/or update the translated message file po/XX.po (described
>> +later), and ask the l10n coordinator to pull your work.
>
> Shall we remind the contributors for l10n to write commit log in English,
> but not language XX? It's a common mistake for translators.

I would prefer to see the subject line (the first line summary) in English
to say something like "Update zh_CN translation for 1.7.12".  That much of
English shouldn't be too much burden for language teams.

On the other hand, the body of the log that describes the change _may_ be
hurt if you forbid the use of language XX.  The translators may want to
write something like:

	The earlier translation used word '[%#$@$#]' for original phrase
	'to distim' in English, but word '[**&*#$]' is a more appropriate
        language XX translation in the context the message for such and
	such reasons.

and I suspect that "for such and such reasons" part is far easier to
explain and understand if written in language XX for native speakers of
language XX.  Such a description often has to discuss connotations and
subtle nuances of the particular word that is hard to explain.

>> diff --git a/po/TEAMS b/po/TEAMS
>> new file mode 100644
>> index 0000000..8ee6199
>> --- /dev/null
>> +++ b/po/TEAMS
>> @@ -0,0 +1,10 @@
>> +Core Git translation language teams
>> +(please keep the list sorted alphabetically on language field)
>> +
>> +Language:      is (Icelandic)
>> +Leader:                Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> +
>
> I think we should add the URL of the dedicated repository for language XX here.
> Example:
>
> Language:      zh_CN (Simplified Chinese)
> Leader:           Jiang Xin <worldhello.net@gmail.com>
> Homepage:     https://github.com/jiangxin/git-po-zh_CN/

Good thinking; I don't know if Homepage or Repository is more appropriate,
but some URL would definitely help.
>
>> +Members:       Yichao Yu <yyc1992@gmail.com>
>> +               Riku <lu.riku@gmail.com>
>
> Contributors and members can add to the header of the XX.po file.
> Add here may cause the TEAMS file grows rapidly.

Sorry, I do not understand what you are worried about here.  Could you
clarify a bit?

In any case, the patch you are replying to is "I'd like to see *something*
*like* *this*...", so feel free to take the ownership and update it.  I'll
try to help refining it if you send an updated draft to back the list for
review.

Thanks.

Re: [PATCH] Update l10n guide

[Jiang Xin]

2012/3/1 Junio C Hamano <gitster@pobox.com>:
>>
>> Shall we remind the contributors for l10n to write commit log in English,
>> but not language XX? It's a common mistake for translators.
>
> I would prefer to see the subject line (the first line summary) in English
> to say something like "Update zh_CN translation for 1.7.12".  That much of
> English shouldn't be too much burden for language teams.

I noticed that all i18n related commit logs all have prefix "i18n:".
It would be nice to add a prefix "l10n:" to the l10n related commit log,
such as: "l10n: Update zh_CN translation for 1.7.12".

>
> On the other hand, the body of the log that describes the change _may_ be
> hurt if you forbid the use of language XX.  The translators may want to
> write something like:
>

Git does not handle multi-bytes character well. Multi-bytes characters
not convert to UTF-8 when git write tree objects and commit objects.
I think allow multi-bytes characters in commit log would hurt git.git until
git gives full support UTF-8 (are there any plans for this?).
I just tested several versions of msysgit:

 - Only Git-1.7.8-preview20111229-unicode.exe saves commit objects
   with utf-8 encode.

 - Other versions such as Git-1.7.9-preview20120201.exe and
   Git-1.7.8-preview20111206.exe will not convert multi-bytes characters
   in commit log into UTF-8.

-- 
Jiang Xin

[PATCH] Update l10n guide: change the repository URL, etc

[Jiang Xin]

Host the l10n coordinator repository in a dedicated github organization
account "git-l10n", so that the team may have a more permanent home.
Also add a hint about reference of TEAMS file for l10n contributors.

Update TEAMS file with new zh_CN l10n team members and a repository URL.

Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
---
 po/README |   17 +++++++++--------
 po/TEAMS  |    9 +++++++--
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/po/README b/po/README
index 64808..188ea 100644
--- a/po/README
+++ b/po/README
@@ -6,22 +6,23 @@ describes how you can contribute to the effort of enhancing the language
 coverage and maintaining the translation.
 
 The localization (l10n) coordinator, Jiang Xin <worldhello.net@gmail.com>,
-coordinates our localization effort in his repository:
+coordinates our localization effort in the l10 coordinator repository:
 
-        https://github.com/gotgit/git-po/
+        https://github.com/git-l10n/git-po/
 
-As a contributor for a language XX, you would fork this repository,
-prepare and/or update the translated message file po/XX.po (described
-later), and ask the l10n coordinator to pull your work.
+As a contributor for a language XX, you should first check TEAMS file in
+this directory to see whether a dedicated repository for your language XX
+exists. Fork the dedicated repository and start to work if it exists.
+
+If you are the first contributor for the language XX, please fork this
+repository, prepare and/or update the translated message file po/XX.po
+(described later), and ask the l10n coordinator to pull your work.
 
 If there are multiple contributors for the same language, please first
 coordinate among yourselves and nominate the team leader for your
 language, so that the l10n coordinator only needs to interact with one
 person per language.
 
-For the list of exiting translations and language teams, see TEAMS file in
-this directory.
-
 The overall data-flow looks like this:
 
     +-------------------+            +------------------+
diff --git a/po/TEAMS b/po/TEAMS
index 8ee61..1d173 100644
--- a/po/TEAMS
+++ b/po/TEAMS
@@ -5,6 +5,11 @@ Language:	is (Icelandic)
 Leader:		Ævar Arnfjörð Bjarmason <avarab@gmail.com>
 
 Language:	zh_CN (Simplified Chinese)
+Repository:	https://github.com/gotgit/git-po-zh_CN/
 Leader:		Jiang Xin <worldhello.net@gmail.com>
-Members:	Yichao Yu <yyc1992@gmail.com>
-		Riku <lu.riku@gmail.com>
+Members:	Riku <lu.riku AT gmail.com>
+		Zhuang Ya <zhuangya AT me.com>
+		Lian Cheng <rhythm.mail AT gmail.com>
+		Yichao Yu <yyc1992 AT gmail.com>
+		ws3389 <willsmith3389 AT gmail.com>
+		Thynson <lanxingcan AT gmail.com>
-- 
1.7.9.2.322.g7ac1c

Re: [PATCH] Update l10n guide

[Johannes Sixt]

Am 3/2/2012 8:30, schrieb Jiang Xin:
> Git does not handle multi-bytes character well. Multi-bytes characters
> not convert to UTF-8 when git write tree objects and commit objects.

It does not convert, but it records which encoding the text has. If you
don't specify anything, UTF-8 is assumed, and if your text is actually not
UTF-8, the result is necessarily garbage.

> I think allow multi-bytes characters in commit log would hurt git.git until
> git gives full support UTF-8 (are there any plans for this?).
> I just tested several versions of msysgit:
> 
>  - Only Git-1.7.8-preview20111229-unicode.exe saves commit objects
>    with utf-8 encode.
> 
>  - Other versions such as Git-1.7.9-preview20120201.exe and
>    Git-1.7.8-preview20111206.exe will not convert multi-bytes characters
>    in commit log into UTF-8.

Then you haven't set i18n.commitEncoding. Try this:

   git config i18n.commitEncoding CP936

(substitute whatever Windows codepage you are using).

-- Hannes

Re: [PATCH] Update l10n guide

[Jiang Xin]

2012/3/2 Johannes Sixt <j.sixt@viscovery.net>:
> It does not convert, but it records which encoding the text has. If you
> don't specify anything, UTF-8 is assumed, and if your text is actually not
> UTF-8, the result is necessarily garbage.
>
> Then you haven't set i18n.commitEncoding. Try this:
>
>   git config i18n.commitEncoding CP936

I know there are two config variables. i18n.commitEncoding will insert
a "encoding XX" line to the commit object, while i18n.logOutputEncoding
will set the default output encoding. But this implementation seems like
a workaround.

* Tree objects do not have such implementation, so multibyte characters
   can not be used as filenames.

* Commit object without "encoding" instruction will be used as it is. So
  people under the same non-utf8 locale may not notice that they
  have not set the proper i18n.commitEncoding, until one day they
  need accross platform development.

* As raw commit log alway used as fallback, sometimes it is hard to find
  commit objects with wrong encoding instructions.

I think save commit object, tree object, packed-refs in UTF-8 is
a better implementation.

-- 
Jiang Xin

Re: [PATCH] Update l10n guide

[Jakub Narebski]

Jiang Xin <worldhello.net@gmail.com> writes:
> 2012/3/2 Johannes Sixt <j.sixt@viscovery.net>:

> > It does not convert, but it records which encoding the text has. If you
> > don't specify anything, UTF-8 is assumed, and if your text is actually not
> > UTF-8, the result is necessarily garbage.
> >
> > Then you haven't set i18n.commitEncoding. Try this:
> >
> >   git config i18n.commitEncoding CP936
> 
> I know there are two config variables. i18n.commitEncoding will insert
> a "encoding XX" line to the commit object, while i18n.logOutputEncoding
> will set the default output encoding.

Note that according to documentation 'git commit' issues a warning if
the commit log message given to it does not look like a valid UTF-8
string, unless you explicitly say your project uses a legacy encoding.
Modern git would also warn if you have NUL ("\0") character in your
commit message, e.g. when using UCS-2 / UTF-16 encoding.

> But this implementation seems like a workaround.
> 
> * Tree objects do not have such implementation, so multibyte characters
>   can not be used as filenames.

And there is no place on pathnameEncoding in 'tree' object,
unfortunately.

One proposed solution was to convert filenames from filesystem
encoding to normal-form composed UTF-8 when creating tree objects, but
this would have to be optional.

Anyway at least for source code using characters outside US-ASCII is
really discouraged anyway.  Note also that sample 'pre-commit' hook
prevents adding files with non-ascii filenames.
 
> * Commit object without "encoding" instruction will be used as it is. So
>   people under the same non-utf8 locale may not notice that they
>   have not set the proper i18n.commitEncoding, until one day they
>   need accross platform development.

But see above.
 
> * As raw commit log alway used as fallback, sometimes it is hard to find
>   commit objects with wrong encoding instructions.
> 
> I think save commit object, tree object, packed-refs in UTF-8 is
> a better implementation.

Backward compatibiltiyt and performance.

-- 
Jakub Narebski

Re: [PATCH] Update l10n guide

[Junio C Hamano]

Jiang Xin <worldhello.net@gmail.com> writes:

> I know there are two config variables. i18n.commitEncoding will insert
> a "encoding XX" line to the commit object, while i18n.logOutputEncoding
> will set the default output encoding. But this implementation seems like
> a workaround.
>
> * Tree objects do not have such implementation, so multibyte characters
>    can not be used as filenames.

Even outside i18n/l10n context, projects that want to be usable by wide
population avoid filesystem-unsafe characters, which are not limited to
just multi-byte. Even though Git started on Linux, we do not have a file
whose name contains '\' or HT, which would have made it impossible to
check out on DOS.

It's just "common sense" or "common courtesy".

In any case, I do not expect we would need to have po/國語.po in git-l10n
repository, so this is a non issue in the context of this discussion
thread.

> * Commit object without "encoding" instruction will be used as it is. So
>   people under the same non-utf8 locale may not notice that they
>   have not set the proper i18n.commitEncoding, until one day they
>   need accross platform development.

That is why we have l10n coordinator who can vet such mistakes crawling
into our system, isn't it?

> I think save commit object, tree object, packed-refs in UTF-8 is
> a better implementation.

It is one of the better conventions to encourage.

It is entirely a different matter to forbid a closed local user community
from adopting GB, Big5, EUC or whatever character encoding as the encoding
of their choice, and using it throughout their project for pathnames, blob
payloads and log messages.