[PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

t0204 tries to use a ISO-8859-1 locale and expects that characters
outside the ISO-8859-1 code range are converted in "question marks".

The gettext under Mac OS X ignores the ISO-8859-1 locale and uses
UTF-8 anyway.

Fixed even whitespace in this very test case.

Let the test pass when UTF-8 is returned.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---
 This is a suggestion to make t0204--gettext-reencode-sanity.sh to pass under Mac OSX
 The tests checks if unicode characters outside ISO-8859-1 are converted into '?'.
 The Mac OS X version of gettext does not seem to support ISO-8859-1 at all.
 Even if we set the locale to ISO-8859-1, UTF-8 is used anyway.


 t/t0204-gettext-reencode-sanity.sh |   16 +++++++++++-----
 1 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/t/t0204-gettext-reencode-sanity.sh b/t/t0204-gettext-reencode-sanity.sh
index 189af90..78f0889 100755
--- a/t/t0204-gettext-reencode-sanity.sh
+++ b/t/t0204-gettext-reencode-sanity.sh
@@ -27,16 +27,22 @@ test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UT
 '
 
 test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
-    LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English Runes" >runes &&
+	LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English Runes" >runes &&
 
 	if grep "^TEST: Old English Runes$" runes
 	then
 		say "Your system can not handle this complexity and returns the string as-is"
 	else
-		# Both Solaris and GNU libintl will return this stream of
-		# question marks, so it is s probably portable enough
-		printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????" >runes-expect &&
-		test_cmp runes-expect runes
+		if grep "^TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ" runes
+		then
+			# Mac OS supports only UTF-8, even if we want ISO-8859-1 we get UTF-8
+			say "Your system handles only utf-8"
+		else
+			# Both Solaris and GNU libintl will return this stream of
+			# question marks, so it is s probably portable enough
+			printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????" >runes-expect &&
+			test_cmp runes-expect runes
+		fi
 	fi
 '
 
-- 
1.7.9.rc2

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

Swapping to: and cc: as Torsten is not the primary "guilty party" on
this one.

> t0204 tries to use a ISO-8859-1 locale and expects that characters
> outside the ISO-8859-1 code range are converted in "question marks".
>
> The gettext under Mac OS X ignores the ISO-8859-1 locale and uses
> UTF-8 anyway.
> ...
> Let the test pass when UTF-8 is returned.

This is insane.  Is passing this test *that* important to the point
that you would need to change the passing criteria, instead of
fixing the behaviour of the code being tested?

It looks like these tests prepare *.mo files based on UTF-8 *.po
files, and ask the system iconv machinery to show them in UTF-8 and
non UTF-8 locales.  And the conversion is an impossible one in this
case, as the messages won't fit in iso-8859-1.

Some platforms may be capable of noticing that and fill them with
question marks, and some others may notice the impossible request
but do something other than "question marks", and yet some others
may not even notice iso-8859-1 is requested and spit out the
original UTF-8 verbatim.

I do not see any reason to insist that their fallback behaviour must
be to fill them with question marks.  In other words, I simply do
not see the point of this test and the original "pass runs of
question" to begin with, and this patch makes things even worse.

Why not get rid of this single test that only tries to check the
output for an impossible request?

Ævar, am I missing something?

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Ævar Arnfjörð Bjarmason]

On Mon, Mar 5, 2012 at 21:39, Junio C Hamano <gitster@pobox.com> wrote:
> I do not see any reason to insist that their fallback behaviour must
> be to fill them with question marks.  In other words, I simply do
> not see the point of this test and the original "pass runs of
> question" to begin with, and this patch makes things even worse.
>
> Why not get rid of this single test that only tries to check the
> output for an impossible request?
>
> Ævar, am I missing something?

Sorry about the late reply, I think if anything it needs to be
expanded beyond its current scope.

Our gettext implementation is peculiar in two ways:

 * We support 3 different languages, one of which (Perl) is supported
   through a non-GNU package.

 * We have this fantastic hack in gettext.c's init_gettext_charset to
   deal with a glibc bug.

I think in particular re-encoding works completely differently under
the Perl i18n library than it does with GNU gettext (and we don't have
tests for this) and instead of just dropping unknown characters under
non-UTF-8 locales on some platforms everything may turn into complete
gibberish.

When I added this test this was what I was planning to smoke
out. Different behaviors on different platforms. In this case the OSX
behavior seems benign, but I'd much rather patch the test to detect
OSX rather than removing it entirely, maybe other obscure operating
systems (Aix, HP/UX etc.) will behave differently.

We still don't really have any use of the i18n feature, i.e. we
haven't had translations submitted yet, and thus don't have any
users. I think it's important to be pro-active about trying to spot
any issues that might affect end users before they happen.

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> I think in particular re-encoding works completely differently under
> the Perl i18n library than it does with GNU gettext (and we don't have
> tests for this) and instead of just dropping unknown characters under
> non-UTF-8 locales on some platforms everything may turn into complete
> gibberish.
>
> When I added this test this was what I was planning to smoke
> out. Different behaviors on different platforms. In this case the OSX
> behavior seems benign,...

I am not sure where you are drawing the line between "complete
gibberish" and "incorrect transliteration", but if you mean by
"gibberish" something that makes user's terminal go berserk,
spitting out UTF-8 when the reader has asked for and is expecting
Latin-1 is already throwing "gibberish", so Mac OS X should fail the
test, I would think.

But for this particular test case, it is even worse, as the test is
asking to do an impossible task to the transliteration machinery, so
it is not even "gibberish" test.  If the input caused the program to
terminate without an error but without producing an output, that
won't be "gibberish" case, but should that be treated as "success"?

> I think it's important to be pro-active about trying to spot
> any issues that might affect end users before they happen.

The goal is noble, but asking the platform to perform an impossible
task and subjectively judging if the failure mode is acceptable is
not the way to do so, I would have to say.

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Ævar Arnfjörð Bjarmason]

On Wed, Mar 7, 2012 at 22:34, Junio C Hamano <gitster@pobox.com> wrote:
>> I think it's important to be pro-active about trying to spot
>> any issues that might affect end users before they happen.
>
> The goal is noble, but asking the platform to perform an impossible
> task and subjectively judging if the failure mode is acceptable is
> not the way to do so, I would have to say.

I leave it up to you whether you want to accept the patch to remove
it, but with it included we at least *know* what the failure modes
are, since we get user reports about it.

That's the reason I put it in there to begin with. Because I have no
idea how all these pieces play together with systems in the wild, and
I'd like to pro-actively find out about that.

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Wed, Mar 7, 2012 at 22:34, Junio C Hamano <gitster@pobox.com> wrote:
>>> I think it's important to be pro-active about trying to spot
>>> any issues that might affect end users before they happen.
>>
>> The goal is noble, but asking the platform to perform an impossible
>> task and subjectively judging if the failure mode is acceptable is
>> not the way to do so, I would have to say.
>
> I leave it up to you whether you want to accept the patch to remove
> it, but with it included we at least *know* what the failure modes
> are, since we get user reports about it.
>
> That's the reason I put it in there to begin with. Because I have no
> idea how all these pieces play together with systems in the wild, and
> I'd like to pro-actively find out about that.

Are we talking about the same specific test?

What you said above all makes sense and I agreed that it is a noble
goal, *if* and only if the test is about the case we *expect* to
work.  This particular one prepares a message that cannot possibly
be transliterated to iso-8859-1 and asks the system to show it.

What scenario do you have in mind that we (or the end user for that
matter) might benefit by having this test?  What would be the next
step for us, when this test finds yet another mode of breakage that
spews out a different kind of garbage other than runs of "?", the
key used to query the message catalog ("TEST: Old English Runes"),
or the raw value stored in the message catalog (runes in UTF-8)?

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Ævar Arnfjörð Bjarmason]

2012/3/7 Junio C Hamano <gitster@pobox.com>:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Wed, Mar 7, 2012 at 22:34, Junio C Hamano <gitster@pobox.com> wrote:
>>>> I think it's important to be pro-active about trying to spot
>>>> any issues that might affect end users before they happen.
>>>
>>> The goal is noble, but asking the platform to perform an impossible
>>> task and subjectively judging if the failure mode is acceptable is
>>> not the way to do so, I would have to say.
>>
>> I leave it up to you whether you want to accept the patch to remove
>> it, but with it included we at least *know* what the failure modes
>> are, since we get user reports about it.
>>
>> That's the reason I put it in there to begin with. Because I have no
>> idea how all these pieces play together with systems in the wild, and
>> I'd like to pro-actively find out about that.
>
> Are we talking about the same specific test?
>
> What you said above all makes sense and I agreed that it is a noble
> goal, *if* and only if the test is about the case we *expect* to
> work.  This particular one prepares a message that cannot possibly
> be transliterated to iso-8859-1 and asks the system to show it.
>
> What scenario do you have in mind that we (or the end user for that
> matter) might benefit by having this test?

E.g. if we find out that their implementation of gettext panics
instead of showing question marks that means we'd have to pay special
attention to that platform.

Anyway, I admit that this is an obscure edge case. I really just like
the idea of having complete knowledge (which we'll have in time since
we get ported everywhere) about how the various aspects of our i18n
implementation work.

Having that information doesn't cost us a lot, in this case we can
just amend the test to assert that on OSX the output should be the
same as under the UTF-8 output.

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> Having that information doesn't cost us a lot, in this case we can
> just amend the test to assert that on OSX the output should be the
> same as under the UTF-8 output.

Known modes of error behaviour are to show:

 - runs of "?", saying "I dunno";

 - the key used to query the message catalog ("TEST: Old English
   Runes"), saying "I don't do i18n"; or

 - the raw value stored in the message catalog (runes in UTF-8).

and each makes sort-of sense in its own way.

I would be OK if the patch read like the attached, but I do not
think treating the "show key" and "stuff with ?" case differently
like the original code did makes any sense.

It may be better to clarify in the "say" comment that this test is
not about finding if the user's system is *broken* but about seeing
if the "undefined" behaviour upon user error is one that we
recognize, though.

 t/t0204-gettext-reencode-sanity.sh |   21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/t/t0204-gettext-reencode-sanity.sh b/t/t0204-gettext-reencode-sanity.sh
index 189af90..8911e93 100755
--- a/t/t0204-gettext-reencode-sanity.sh
+++ b/t/t0204-gettext-reencode-sanity.sh
@@ -7,6 +7,7 @@ test_description="Gettext reencoding of our *.po/*.mo files works"
 
 . ./lib-gettext.sh
 
+RUNES="TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ"
 
 test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic' '
     printf "TILRAUN: Halló Heimur!" >expect &&
@@ -15,7 +16,7 @@ test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo
 '
 
 test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes' '
-    printf "TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ" >expect &&
+    printf "%s" "$RUNES" >expect &&
     LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: Old English Runes" >actual &&
     test_cmp expect actual
 '
@@ -28,15 +29,19 @@ test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UT
 
 test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
     LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English Runes" >runes &&
-
-	if grep "^TEST: Old English Runes$" runes
+	runes=$(cat runes) &&
+	if test "z$runes" = "zTEST: Old English Runes"
+	then
+		say "Your system gives back the key to message catalog for an impossible request"
+	elif test "z$runes" = "zTILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????"
+	then
+		say "Your system replaces an impossible character with ?"
+ 	elif test "z$runes" = "z$RUNES"
 	then
-		say "Your system can not handle this complexity and returns the string as-is"
+		say "Your system gives back the raw message for an impossible request"
 	else
-		# Both Solaris and GNU libintl will return this stream of
-		# question marks, so it is s probably portable enough
-		printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????" >runes-expect &&
-		test_cmp runes-expect runes
+		say "We error behaviour your system shows"
+		false
 	fi
 '
 

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> +		say "We error behaviour your system shows"
> +		false

Typo--"We do not recognize the error behaviour your system shows"
or something.

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

On 08.03.12 00:36, Junio C Hamano wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> 
>> Having that information doesn't cost us a lot, in this case we can
>> just amend the test to assert that on OSX the output should be the
>> same as under the UTF-8 output.
> 
> Known modes of error behaviour are to show:
> 
>  - runs of "?", saying "I dunno";
> 
>  - the key used to query the message catalog ("TEST: Old English
>    Runes"), saying "I don't do i18n"; or
> 
>  - the raw value stored in the message catalog (runes in UTF-8).
> 
> and each makes sort-of sense in its own way.
> 
> I would be OK if the patch read like the attached, but I do not
> think treating the "show key" and "stuff with ?" case differently
> like the original code did makes any sense.
> 
> It may be better to clarify in the "say" comment that this test is
> not about finding if the user's system is *broken* but about seeing
> if the "undefined" behaviour upon user error is one that we
> recognize, though.
> 
>  t/t0204-gettext-reencode-sanity.sh |   21 +++++++++++++--------
>  1 file changed, 13 insertions(+), 8 deletions(-)
> 
> diff --git a/t/t0204-gettext-reencode-sanity.sh b/t/t0204-gettext-reencode-sanity.sh
> index 189af90..8911e93 100755
> --- a/t/t0204-gettext-reencode-sanity.sh
> +++ b/t/t0204-gettext-reencode-sanity.sh
> @@ -7,6 +7,7 @@ test_description="Gettext reencoding of our *.po/*.mo files works"
>  
>  . ./lib-gettext.sh
>  
> +RUNES="TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ"
>  
>  test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic' '
>      printf "TILRAUN: Halló Heimur!" >expect &&
> @@ -15,7 +16,7 @@ test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo
>  '
>  
>  test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes' '
> -    printf "TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ" >expect &&
> +    printf "%s" "$RUNES" >expect &&
>      LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: Old English Runes" >actual &&
>      test_cmp expect actual
>  '
> @@ -28,15 +29,19 @@ test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UT
>  
>  test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
>      LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English Runes" >runes &&
> -
> -	if grep "^TEST: Old English Runes$" runes
> +	runes=$(cat runes) &&
> +	if test "z$runes" = "zTEST: Old English Runes"
> +	then
> +		say "Your system gives back the key to message catalog for an impossible request"
> +	elif test "z$runes" = "zTILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????"
> +	then
> +		say "Your system replaces an impossible character with ?"
> + 	elif test "z$runes" = "z$RUNES"
>  	then
> -		say "Your system can not handle this complexity and returns the string as-is"
> +		say "Your system gives back the raw message for an impossible request"
>  	else
> -		# Both Solaris and GNU libintl will return this stream of
> -		# question marks, so it is s probably portable enough
> -		printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????" >runes-expect &&
> -		test_cmp runes-expect runes
> +		say "We error behaviour your system shows"
> +		false
>  	fi
>  '
>  
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

With that patch the t0204 test is passed.

(based on 1.7.10-rc0).

 And all the other tests passed as well.

/Torsten



Applying: Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX
/Users/tb/projects/git/git_git/.git/rebase-apply/patch:35: space before tab in indent.
        elif test "z$runes" = "z$RUNES"
warning: 1 line adds whitespace errors.
~/projects/git/git_git> cd t
~/projects/git/git_git/t> ./t0204-gettext-reencode-sanity.sh
# lib-gettext: No is_IS UTF-8 locale available
# lib-gettext: No is_IS ISO-8859-1 locale available
ok 1 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_LOCALE)
ok 2 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes (missing GETTEXT_LOCALE)
ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_ISO_LOCALE)
ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes (missing GETTEXT_ISO_LOCALE)
ok 5 # skip gettext: Fetching a UTF-8 msgid -> UTF-8 (missing GETTEXT_LOCALE)
ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
ok 7 # skip gettext.c: git init UTF-8 -> UTF-8 (missing GETTEXT_LOCALE)
ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
# passed all 8 test(s)
1..8

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Johannes Sixt]

Am 3/8/2012 6:54, schrieb Torsten Bögershausen:
> On 08.03.12 00:36, Junio C Hamano wrote:
>>  test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '

> ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes (missing GETTEXT_ISO_LOCALE)
         ^^^^
Huh? How can you have tripped over this test in the first place? The
prerequisite did not change in Junio's patch. Why was the test not skipped
before you started this thread with your own patch?

-- Hannes

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

Am 08.03.2012 um 08:23 schrieb Johannes Sixt:

> Am 3/8/2012 6:54, schrieb Torsten Bögershausen:
>> On 08.03.12 00:36, Junio C Hamano wrote:
>>> test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
> 
>> ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes (missing GETTEXT_ISO_LOCALE)
>         ^^^^
> Huh? How can you have tripped over this test in the first place? The
> prerequisite did not change in Junio's patch. Why was the test not skipped
> before you started this thread with your own patch?
> 
> -- Hannes
> 

Hm, 
things are getting confusing.

After re-booting the machine, cloning git ... I get the following on a clean 1.7.10-rc0 (without Junios patch):

tb@birne:~/projects/git/git_clean/t> ./t0204-gettext-reencode-sanity.sh 
# lib-gettext: No is_IS UTF-8 locale available
# lib-gettext: No is_IS ISO-8859-1 locale available
ok 1 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_LOCALE)
ok 2 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes (missing GETTEXT_LOCALE)
ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_ISO_LOCALE)
ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes (missing GETTEXT_ISO_LOCALE)
ok 5 # skip gettext: Fetching a UTF-8 msgid -> UTF-8 (missing GETTEXT_LOCALE)
ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
ok 7 # skip gettext.c: git init UTF-8 -> UTF-8 (missing GETTEXT_LOCALE)
ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
# passed all 8 test(s)
1..8

on 5e9637c629702e3d41ad01d95956d1835  the log looks like this:


tb@birne:~/projects/git/git_clean/t> ./t0204-gettext-reencode-sanity.sh 
# lib-gettext: Found 'is_IS.UTF-8' as an is_IS UTF-8 locale
# lib-gettext: Found 'is_IS.ISO8859-1' as an is_IS ISO-8859-1 locale
ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic
ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes
ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic
ok 4 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes
ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8
ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1
ok 7 - gettext.c: git init UTF-8 -> UTF-8
ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1
# passed all 8 test(s)
1..8
tb@birne:~/projects/git/git_clean/t> 


At the moment I would like to say sorry for the noise.
If somebody is interested. I can dig into details later this evening (european time) 
/Torsten

Re: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

On 08.03.12 10:21, Torsten Bögershausen wrote:
> 
> Am 08.03.2012 um 08:23 schrieb Johannes Sixt:
> 
>> Am 3/8/2012 6:54, schrieb Torsten Bögershausen:
>>> On 08.03.12 00:36, Junio C Hamano wrote:
>>>> test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
>>
>>> ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes (missing GETTEXT_ISO_LOCALE)
>>         ^^^^
>> Huh? How can you have tripped over this test in the first place? The
>> prerequisite did not change in Junio's patch. Why was the test not skipped
>> before you started this thread with your own patch?
>>
>> -- Hannes
>>
> 
> Hm, 
> things are getting confusing.
> 
> After re-booting the machine, cloning git ... I get the following on a clean 1.7.10-rc0 (without Junios patch):
> 
> tb@birne:~/projects/git/git_clean/t> ./t0204-gettext-reencode-sanity.sh 
> # lib-gettext: No is_IS UTF-8 locale available
> # lib-gettext: No is_IS ISO-8859-1 locale available
> ok 1 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_LOCALE)
> ok 2 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes (missing GETTEXT_LOCALE)
> ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_ISO_LOCALE)
> ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes (missing GETTEXT_ISO_LOCALE)
> ok 5 # skip gettext: Fetching a UTF-8 msgid -> UTF-8 (missing GETTEXT_LOCALE)
> ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
> ok 7 # skip gettext.c: git init UTF-8 -> UTF-8 (missing GETTEXT_LOCALE)
> ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
> # passed all 8 test(s)
> 1..8
> 
> on 5e9637c629702e3d41ad01d95956d1835  the log looks like this:
> 
> 
> tb@birne:~/projects/git/git_clean/t> ./t0204-gettext-reencode-sanity.sh 
> # lib-gettext: Found 'is_IS.UTF-8' as an is_IS UTF-8 locale
> # lib-gettext: Found 'is_IS.ISO8859-1' as an is_IS ISO-8859-1 locale
> ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic
> ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes
> ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic
> ok 4 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes
> ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8
> ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1
> ok 7 - gettext.c: git init UTF-8 -> UTF-8
> ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1
> # passed all 8 test(s)
> 1..8
> tb@birne:~/projects/git/git_clean/t> 
> 
> 
> At the moment I would like to say sorry for the noise.
> If somebody is interested. I can dig into details later this evening (european time) 
> /Torsten

Here we have the change from t0204-gettext-reencode-sanity.sh beeing passed 
into all tests being skipped under Mac OS X:

42f16113ee87ff667fef3a821e:
# lib-gettext: Found 'is_IS.UTF-8' as an is_IS UTF-8 locale
# lib-gettext: Found 'is_IS.ISO8859-1' as an is_IS ISO-8859-1 locale
ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic
ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes
ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic
ok 4 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes
ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8
ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1
ok 7 - gettext.c: git init UTF-8 -> UTF-8
ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1
# passed all 8 test(s)
1..8

ad17ea73477012bfe1027a8a999e9e2d9f62145:
# lib-gettext: No is_IS UTF-8 locale available
# lib-gettext: No is_IS ISO-8859-1 locale available
ok 1 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_LOCALE)
ok 2 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes (missing GETTEXT_LOCALE)
ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic (missing GETTEXT_ISO_LOCALE)
ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes (missing GETTEXT_ISO_LOCALE)
ok 5 # skip gettext: Fetching a UTF-8 msgid -> UTF-8 (missing GETTEXT_LOCALE)
ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
ok 7 # skip gettext.c: git init UTF-8 -> UTF-8 (missing GETTEXT_LOCALE)
ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing GETTEXT_ISO_LOCALE)
# passed all 8 test(s)


commit ad17ea734770126bfe1027a8a999e9e2d9f62145
Author: Alex Riesen <raa.lkml@gmail.com>
Date:   Mon Jan 23 14:04:29 2012 -0800

    add a Makefile switch to avoid gettext translation in shell scripts
    
    Some systems have gettext.sh (GNU gettext) installed, but it is either
    broken or misconfigured in such a way so its output is not usable.  In
    case the users of these systems are unable or not interested in fixing
    them, setting the new Makefile switch should help:
    
        make USE_GETTEXT_SCHEME=fallthrough
    
    This will replace the translation routines with fallthrough versions,
    that does not use gettext from the platform.
    
    Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>

:100644 100644 9470a1034396a5f3ee36c5d0e6ffc54e21bb3820 a0f2464a9b647fcd6b817e9bb86b7f79190acb01 M      Makefile
:100644 100644 6648bd3726f18c95de5737663fae652cadac0003 d5fae993b0c093bdd07079101df303c2c74deae3 M      git-sh-i18n.sh

Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> I would be OK if the patch read like the attached, but I do not
> think treating the "show key" and "stuff with ?" case differently
> like the original code did makes any sense.

Ok, here is a re-roll with proper-ish log message.  I've tested this
only in one of my VMs with is_IS locale (which happened to be Fedora
15).

-- >8 --
Subject: [PATCH] t0204: clarify the "observe undefined behaviour" test

This test asks for an impossible conversion to the system by
preparing an UTF-8 translation with characters that cannot be
expressed in ISO-8859-1, and then asking the message shown in
ISO-8859-1.  Even though the behaviour against such a request is
undefined, it may be interesting to see what the system does, and
the purpose of this test is to see if there are platforms that
exhibit behaviour that we haven't seen.

The original recognized two known modes of behaviour:

 - the key used to query the message catalog ("TEST: Old English
   Runes"), saying "I cannot do that i18n".
 - impossible characters replaced with ASCII "?", saying "I punt".

but they were treated totally differently.  The test simply issued
an informational message "Your system punts on this one" for the
first error mode, while it diagnosed the latter as "Your system is
good; you pass!".

It turns out that Mac OS X exhibits a third mode of error behaviour,
to spew out the raw value stored in the message catalog.  The test
diagnosed this behaviour as "broken", but it is merely trying to do
its best to respond to an impossible request by saying "I punt" in a
way that is slightly different from the second one.

Update the offending test to make it clear what is (and is not)
being tested, update the code structure so that newly discovered
error mode can easily be added to it later, and reword the message
that comes from a failing case to clarify that it is not the system
that is broken when it fails, but merely that the behaviour is not
something we have seen.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t0204-gettext-reencode-sanity.sh |   37 ++++++++++++++++++++++--------------
 1 file changed, 23 insertions(+), 14 deletions(-)

diff --git a/t/t0204-gettext-reencode-sanity.sh b/t/t0204-gettext-reencode-sanity.sh
index 189af90..8437e51 100755
--- a/t/t0204-gettext-reencode-sanity.sh
+++ b/t/t0204-gettext-reencode-sanity.sh
@@ -7,6 +7,10 @@ test_description="Gettext reencoding of our *.po/*.mo files works"
 
 . ./lib-gettext.sh
 
+# The constants used in a tricky observation for undefined behaviour
+RUNES="TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ"
+PUNTS="TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????"
+MSGKEY="TEST: Old English Runes"
 
 test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic' '
     printf "TILRAUN: Halló Heimur!" >expect &&
@@ -15,8 +19,8 @@ test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo
 '
 
 test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes' '
-    printf "TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ" >expect &&
-    LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: Old English Runes" >actual &&
+    printf "%s" "$RUNES" >expect &&
+    LANGUAGE=is LC_ALL="$is_IS_locale" gettext "$MSGKEY" >actual &&
     test_cmp expect actual
 '
 
@@ -26,18 +30,23 @@ test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UT
     test_cmp expect actual
 '
 
-test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
-    LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English Runes" >runes &&
-
-	if grep "^TEST: Old English Runes$" runes
-	then
-		say "Your system can not handle this complexity and returns the string as-is"
-	else
-		# Both Solaris and GNU libintl will return this stream of
-		# question marks, so it is s probably portable enough
-		printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????" >runes-expect &&
-		test_cmp runes-expect runes
-	fi
+test_expect_success GETTEXT_ISO_LOCALE 'gettext: impossible ISO-8859-1 output' '
+	LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "$MSGKEY" >runes &&
+	case "$(cat runes)" in
+	"$MSGKEY")
+		say "Your system gives back the key to message catalog"
+		;;
+	"$PUNTS")
+		say "Your system replaces an impossible character with ?"
+		;;
+	"$RUNES")
+		say "Your system gives back the raw message for an impossible request"
+		;;
+	*)
+		say "We never saw the error behaviour your system exhibits"
+		false
+		;;
+	esac
 '
 
 test_expect_success GETTEXT_LOCALE 'gettext: Fetching a UTF-8 msgid -> UTF-8' '
-- 
1.7.10.rc0.32.g00187

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

On 03/09/2012 11:30 PM, Junio C Hamano wrote:
> Junio C Hamano<gitster@pobox.com>  writes:
>
>> I would be OK if the patch read like the attached, but I do not
>> think treating the "show key" and "stuff with ?" case differently
>> like the original code did makes any sense.
> Ok, here is a re-roll with proper-ish log message.  I've tested this
> only in one of my VMs with is_IS locale (which happened to be Fedora
> 15).
>
> -- >8 --
> Subject: [PATCH] t0204: clarify the "observe undefined behaviour" test
>
> This test asks for an impossible conversion to the system by
> preparing an UTF-8 translation with characters that cannot be
> expressed in ISO-8859-1, and then asking the message shown in
> ISO-8859-1.  Even though the behaviour against such a request is
> undefined, it may be interesting to see what the system does, and
> the purpose of this test is to see if there are platforms that
> exhibit behaviour that we haven't seen.
>
> The original recognized two known modes of behaviour:
>
>   - the key used to query the message catalog ("TEST: Old English
>     Runes"), saying "I cannot do that i18n".
>   - impossible characters replaced with ASCII "?", saying "I punt".
>
> but they were treated totally differently.  The test simply issued
> an informational message "Your system punts on this one" for the
> first error mode, while it diagnosed the latter as "Your system is
> good; you pass!".
>
> It turns out that Mac OS X exhibits a third mode of error behaviour,
> to spew out the raw value stored in the message catalog.  The test
> diagnosed this behaviour as "broken", but it is merely trying to do
> its best to respond to an impossible request by saying "I punt" in a
> way that is slightly different from the second one.
>
> Update the offending test to make it clear what is (and is not)
> being tested, update the code structure so that newly discovered
> error mode can easily be added to it later, and reword the message
> that comes from a failing case to clarify that it is not the system
> that is broken when it fails, but merely that the behaviour is not
> something we have seen.
>
> Signed-off-by: Junio C Hamano<gitster@pobox.com>
> ---
>   t/t0204-gettext-reencode-sanity.sh |   37 ++++++++++++++++++++++--------------
>   1 file changed, 23 insertions(+), 14 deletions(-)
>
> diff --git a/t/t0204-gettext-reencode-sanity.sh b/t/t0204-gettext-reencode-sanity.sh
> index 189af90..8437e51 100755
> --- a/t/t0204-gettext-reencode-sanity.sh
> +++ b/t/t0204-gettext-reencode-sanity.sh
> @@ -7,6 +7,10 @@ test_description="Gettext reencoding of our *.po/*.mo files works"
>
>   . ./lib-gettext.sh
>
> +# The constants used in a tricky observation for undefined behaviour
> +RUNES="TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ"
> +PUNTS="TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????"
> +MSGKEY="TEST: Old English Runes"
>
>   test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic' '
>       printf "TILRAUN: Halló Heimur!">expect&&
> @@ -15,8 +19,8 @@ test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo
>   '
>
>   test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes' '
> -    printf "TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ">expect&&
> -    LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: Old English Runes">actual&&
> +    printf "%s" "$RUNES">expect&&
> +    LANGUAGE=is LC_ALL="$is_IS_locale" gettext "$MSGKEY">actual&&
>       test_cmp expect actual
>   '
>
> @@ -26,18 +30,23 @@ test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UT
>       test_cmp expect actual
>   '
>
> -test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes' '
> -    LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English Runes">runes&&
> -
> -	if grep "^TEST: Old English Runes$" runes
> -	then
> -		say "Your system can not handle this complexity and returns the string as-is"
> -	else
> -		# Both Solaris and GNU libintl will return this stream of
> -		# question marks, so it is s probably portable enough
> -		printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ?? ????">runes-expect&&
> -		test_cmp runes-expect runes
> -	fi
> +test_expect_success GETTEXT_ISO_LOCALE 'gettext: impossible ISO-8859-1 output' '
> +	LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "$MSGKEY">runes&&
> +	case "$(cat runes)" in
> +	"$MSGKEY")
> +		say "Your system gives back the key to message catalog"
> +		;;
> +	"$PUNTS")
> +		say "Your system replaces an impossible character with ?"
> +		;;
> +	"$RUNES")
> +		say "Your system gives back the raw message for an impossible request"
> +		;;
> +	*)
> +		say "We never saw the error behaviour your system exhibits"
> +		false
> +		;;
> +	esac
>   '
>
>   test_expect_success GETTEXT_LOCALE 'gettext: Fetching a UTF-8 msgid ->  UTF-8' '
Hi Junio and all,
Thanks, the suggestion looks good to me.

I want be able to test it, but there is one problem:
I can not reproduce the original problem any more.
I went through all commit, all my branches, run git bisect, restore old 
backups:
The problem is gone.

Then I diged into the test code. As Hannes pointed out, all test cases 
in t0204 are
skipped.
This is not because Mac OS X does not have the island locale (it has),
but because GIT_INTERNAL_GETTEXT_SH_SCHEME" = fallthrough.

Side note: I will send an improved lib-gettext.sh as a suggested patch.

Back to my setup:
I realized, that there is i18n stuff installed by git under $HOME/share.
Could that be a reason why I got that non-reproducable failure in t0204?

And the skipped test cases seem to indicate that the gettext feature is
not working under Mac OS X?

Does anybody else have t0204 working and not skipped?
Any hints are welcome.

/Torsten

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> This is not because Mac OS X does not have the island locale (it has),
> but because GIT_INTERNAL_GETTEXT_SH_SCHEME" = fallthrough.

You need to trace and see what happens in git-sh-i18n.sh then, I
would think.

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

On 03/10/2012 11:28 PM, Torsten Bögershausen wrote:
> On 03/09/2012 11:30 PM, Junio C Hamano wrote:
>> Junio C Hamano<gitster@pobox.com> writes:
>>
>>> I would be OK if the patch read like the attached, but I do not
>>> think treating the "show key" and "stuff with ?" case differently
>>> like the original code did makes any sense.
>> Ok, here is a re-roll with proper-ish log message. I've tested this
>> only in one of my VMs with is_IS locale (which happened to be Fedora
>> 15).
>>
>> -- >8 --
>> Subject: [PATCH] t0204: clarify the "observe undefined behaviour" test
>>
>> This test asks for an impossible conversion to the system by
>> preparing an UTF-8 translation with characters that cannot be
>> expressed in ISO-8859-1, and then asking the message shown in
>> ISO-8859-1. Even though the behaviour against such a request is
>> undefined, it may be interesting to see what the system does, and
>> the purpose of this test is to see if there are platforms that
>> exhibit behaviour that we haven't seen.
>>
>> The original recognized two known modes of behaviour:
>>
>> - the key used to query the message catalog ("TEST: Old English
>> Runes"), saying "I cannot do that i18n".
>> - impossible characters replaced with ASCII "?", saying "I punt".
>>
>> but they were treated totally differently. The test simply issued
>> an informational message "Your system punts on this one" for the
>> first error mode, while it diagnosed the latter as "Your system is
>> good; you pass!".
>>
>> It turns out that Mac OS X exhibits a third mode of error behaviour,
>> to spew out the raw value stored in the message catalog. The test
>> diagnosed this behaviour as "broken", but it is merely trying to do
>> its best to respond to an impossible request by saying "I punt" in a
>> way that is slightly different from the second one.
>>
>> Update the offending test to make it clear what is (and is not)
>> being tested, update the code structure so that newly discovered
>> error mode can easily be added to it later, and reword the message
>> that comes from a failing case to clarify that it is not the system
>> that is broken when it fails, but merely that the behaviour is not
>> something we have seen.
>>
>> Signed-off-by: Junio C Hamano<gitster@pobox.com>
>> ---
>> t/t0204-gettext-reencode-sanity.sh | 37
>> ++++++++++++++++++++++--------------
>> 1 file changed, 23 insertions(+), 14 deletions(-)
>>
>> diff --git a/t/t0204-gettext-reencode-sanity.sh
>> b/t/t0204-gettext-reencode-sanity.sh
>> index 189af90..8437e51 100755
>> --- a/t/t0204-gettext-reencode-sanity.sh
>> +++ b/t/t0204-gettext-reencode-sanity.sh
>> @@ -7,6 +7,10 @@ test_description="Gettext reencoding of our *.po/*.mo
>> files works"
>>
>> . ./lib-gettext.sh
>>
>> +# The constants used in a tricky observation for undefined behaviour
>> +RUNES="TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ
>> ᚹᛖᛥᚫ"
>> +PUNTS="TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ??
>> ????"
>> +MSGKEY="TEST: Old English Runes"
>>
>> test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our
>> UTF-8 *.mo files / Icelandic' '
>> printf "TILRAUN: Halló Heimur!">expect&&
>> @@ -15,8 +19,8 @@ test_expect_success GETTEXT_LOCALE 'gettext:
>> Emitting UTF-8 from our UTF-8 *.mo
>> '
>>
>> test_expect_success GETTEXT_LOCALE 'gettext: Emitting UTF-8 from our
>> UTF-8 *.mo files / Runes' '
>> - printf "TILRAUN: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ
>> ᚹᛖᛥᚫ">expect&&
>> - LANGUAGE=is LC_ALL="$is_IS_locale" gettext "TEST: Old English
>> Runes">actual&&
>> + printf "%s" "$RUNES">expect&&
>> + LANGUAGE=is LC_ALL="$is_IS_locale" gettext "$MSGKEY">actual&&
>> test_cmp expect actual
>> '
>>
>> @@ -26,18 +30,23 @@ test_expect_success GETTEXT_ISO_LOCALE 'gettext:
>> Emitting ISO-8859-1 from our UT
>> test_cmp expect actual
>> '
>>
>> -test_expect_success GETTEXT_ISO_LOCALE 'gettext: Emitting ISO-8859-1
>> from our UTF-8 *.mo files / Runes' '
>> - LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old English
>> Runes">runes&&
>> -
>> - if grep "^TEST: Old English Runes$" runes
>> - then
>> - say "Your system can not handle this complexity and returns the
>> string as-is"
>> - else
>> - # Both Solaris and GNU libintl will return this stream of
>> - # question marks, so it is s probably portable enough
>> - printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? ????? ??????????? ??? ??
>> ????">runes-expect&&
>> - test_cmp runes-expect runes
>> - fi
>> +test_expect_success GETTEXT_ISO_LOCALE 'gettext: impossible
>> ISO-8859-1 output' '
>> + LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "$MSGKEY">runes&&
>> + case "$(cat runes)" in
>> + "$MSGKEY")
>> + say "Your system gives back the key to message catalog"
>> + ;;
>> + "$PUNTS")
>> + say "Your system replaces an impossible character with ?"
>> + ;;
>> + "$RUNES")
>> + say "Your system gives back the raw message for an impossible request"
>> + ;;
>> + *)
>> + say "We never saw the error behaviour your system exhibits"
>> + false
>> + ;;
>> + esac
>> '
>>
>> test_expect_success GETTEXT_LOCALE 'gettext: Fetching a UTF-8 msgid ->
>> UTF-8' '
> Hi Junio and all,
> Thanks, the suggestion looks good to me.
>
> I want be able to test it, but there is one problem:

Here the results of my investigations:
a) git-sh-i18n is used by the test suite, not git-sh-i18n.sh
    git-sh-i18n is generated from git-sh-i18n.sh when running make

b) When running
    make clean && make USE_GETTEXT_SCHEME=gnu && (cd t && make)
   the log of t0204 looks like this:

# lib-gettext: Found 'is_IS.UTF-8' as an is_IS UTF-8 locale
# lib-gettext: Found 'is_IS.ISO8859-1' as an is_IS ISO-8859-1 locale
ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic
ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes
ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic
ok 4 - gettext: impossible ISO-8859-1 output
ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8
ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1
ok 7 - gettext.c: git init UTF-8 -> UTF-8
ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1
# passed all 8 test(s)
1..8
===========
When running
./t0204-gettext-reencode-sanity.sh --verbose
we find the line
"Your system gives back the raw message for an impossible request"
in the log.


Summary: test OK,

May I say
"Thanks for looking into it,
please go ahead with your suggested re-roll" ?

/Torsten

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> Here the results of my investigations:
> a) git-sh-i18n is used by the test suite, not git-sh-i18n.sh
>    git-sh-i18n is generated from git-sh-i18n.sh when running make

Yeah, that is the correct behaviour.

> b) When running
>    make clean && make USE_GETTEXT_SCHEME=gnu && (cd t && make)
>   the log of t0204 looks like this:
> ...
> Summary: test OK,

Now, the last remaining issues appear to be these:

 * Is USE_GETTEXT_SCHEME=gnu the right thing for Mac OS X?  It may
   make this test pass, but does it not break other things?  Does
   Mac OS X come with a working gettext.sh to help i18n of shell
   scripts?  Does it also give eval_gettext?

 * If so, Mac OS X users shouldn't have to say that from the command
   line of "make". Do we need a Makefile update?

 * If not, what is the appropriate value for USE_GETTEXT_SCHEME on
   Mac OS X?  If the answer is not "fallthrough", do we need a
   Makefile update?

Thanks.

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

On 03/11/2012 08:22 PM, Junio C Hamano wrote:
> Torsten Bögershausen<tboegi@web.de>  writes:
>
>> Here the results of my investigations:
>> a) git-sh-i18n is used by the test suite, not git-sh-i18n.sh
>>     git-sh-i18n is generated from git-sh-i18n.sh when running make
>
> Yeah, that is the correct behaviour.
>
>> b) When running
>>     make clean&&  make USE_GETTEXT_SCHEME=gnu&&  (cd t&&  make)
>>    the log of t0204 looks like this:
>> ...
>> Summary: test OK,
>
> Now, the last remaining issues appear to be these:
>
>   * Is USE_GETTEXT_SCHEME=gnu the right thing for Mac OS X?  It may
>     make this test pass, but does it not break other things?  Does
>     Mac OS X come with a working gettext.sh to help i18n of shell
>     scripts?  Does it also give eval_gettext?
When I remember it right, the following comes from the Fink installation 
on my machine:
which gettext
/sw/bin/gettext

which gettext.sh
/sw/bin/gettext.sh

None of the machines here (Mac OS X, Ubuntu 10.04.4 LTS, OpenSuse 12.1)
has a eval_gettext

>   * If so, Mac OS X users shouldn't have to say that from the command
>     line of "make". Do we need a Makefile update?
Good question.
I couldn't find any setup of USE_GETTEXT_SCHEME in Makefile
(except when NO_GETTEXT is set), do I need to run configure?
So far the Makefile worked for all my systems here.

>
>   * If not, what is the appropriate value for USE_GETTEXT_SCHEME on=
>     Mac OS X?  If the answer is not "fallthrough", do we need a
>     Makefile update?
My guess is that "gettext_without_eval_gettext" would be right
(if we have Fink installed)

But again, what did I overlook in the Makefile?
Should there be some hard coded default for e.g. Darwin?
With a switch when Fink is installed?

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> On 03/11/2012 08:22 PM, Junio C Hamano wrote:
> ...
>>   * Is USE_GETTEXT_SCHEME=gnu the right thing for Mac OS X?  It may
>>     make this test pass, but does it not break other things?  Does
>>     Mac OS X come with a working gettext.sh to help i18n of shell
>>     scripts?  Does it also give eval_gettext?
> When I remember it right, the following comes from the Fink
> installation on my machine:
> which gettext
> /sw/bin/gettext
>
> which gettext.sh
> /sw/bin/gettext.sh
>
> None of the machines here (Mac OS X, Ubuntu 10.04.4 LTS, OpenSuse 12.1)
> has a eval_gettext

No, we do not expect a standalone eval_gettext program.  If you
dot-source gettext.sh, some systems give you the eval_gettext shell
function, while there are other systems that don't.  Gnu systems
that ship with /usr/bin/gettext.sh do give you eval_gettext while
Solaris is known not to. I do not know what your system does, and
that was what the question was about.  What does this say there?

	$ (. gettext.sh; eval_gettext frotz; echo $?)

>>   * If so, Mac OS X users shouldn't have to say that from the command
>>     line of "make". Do we need a Makefile update?
> Good question.
> I couldn't find any setup of USE_GETTEXT_SCHEME in Makefile
> (except when NO_GETTEXT is set), do I need to run configure?
> So far the Makefile worked for all my systems here.

You shouldn't have to muck with autoconf around here. It is one of
the design goals of our Makefile to work out of the box on major
platforms as long as the prerequisites like POSIX compliant tools
and use of GNU make are met. An optional configure support comes
into the picture only once we know what we want to have in the
Makefile to be tweaked.

Also, as you noticed, even though we mention USE_GETTEXT_SCHEME in
the Makefile, no platforms with explicit support needs to tweak it;
if git-sh-i18n.sh can figure out what to do without having to rely
on any tweak in the Makefile, we would prefer to keep it that way.

What is curious about your system is that git-sh-i18n.sh should have
figured out that you want INTERNAL_GETTEXT_SH_SCHEME to be gnu by
itself, without any need for "make USE_GETTEXT_SCHEME=gnu", with
these lines:

                ...
        elif type gettext.sh >/dev/null 2>&1
        then
                # GNU libintl's gettext.sh
                GIT_INTERNAL_GETTEXT_SH_SCHEME=gnu
        elif test ...

as long as you have /sw/bin/gettext.sh on your PATH, that is.

So I am not sure what's broken.

>>   * If not, what is the appropriate value for USE_GETTEXT_SCHEME on=
>>     Mac OS X?  If the answer is not "fallthrough", do we need a
>>     Makefile update?
> My guess is that "gettext_without_eval_gettext" would be right
> (if we have Fink installed)

Are you sure that after dot-sourcing /sw/bin/gettext.sh, you do not
have eval_gettext?  If you do, then I think it can be set to gnu.

Ævar, I am wondering if the auto-detection part of git-sh-i18n
should read more like this to make sure when we declare gnu we know
eval_gettext is available.  What do you think?

 git-sh-i18n.sh |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-sh-i18n.sh b/git-sh-i18n.sh
index d5fae99..a17688b 100644
--- a/git-sh-i18n.sh
+++ b/git-sh-i18n.sh
@@ -27,7 +27,7 @@ then
 elif test -n "$GIT_GETTEXT_POISON"
 then
 	GIT_INTERNAL_GETTEXT_SH_SCHEME=poison
-elif type gettext.sh >/dev/null 2>&1
+elif (. gettext.sh && eval_gettext frotz) >/dev/null 2>&1
 then
 	# GNU libintl's gettext.sh
 	GIT_INTERNAL_GETTEXT_SH_SCHEME=gnu

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> May I say
> "Thanks for looking into it,
> please go ahead with your suggested re-roll" ?

Although I am not sure if we got everything right for your
particular platform, I think the patch I sent out to categorize
three-known plausible error behaviours is a much saner approach
than the current code, so I am fine with committing it.

I am sort-of waiting for comments from Ævar, though.

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

[snip]
 (. gettext.sh; eval_gettext frotz; echo $?)
frotz0

> What is curious about your system is that git-sh-i18n.sh should have
> figured out that you want INTERNAL_GETTEXT_SH_SCHEME to be gnu by
> itself, without any need for "make USE_GETTEXT_SCHEME=gnu", with
> these lines:
> 
>                 ...
>         elif type gettext.sh >/dev/null 2>&1
>         then
>                 # GNU libintl's gettext.sh
>                 GIT_INTERNAL_GETTEXT_SH_SCHEME=gnu
>         elif test ...
> 
> as long as you have /sw/bin/gettext.sh on your PATH, that is.
> 
> So I am not sure what's broken.
=====================
When I run the following scriptlet, 
extracted from git-sh-i18n
=====================
#!/bin/sh
GIT_INTERNAL_GETTEXT_SH_SCHEME=fallthrough
if test -n ""
then
	GIT_INTERNAL_GETTEXT_SH_SCHEME=""
elif test -n "@@USE_FALLTHROUGH_GETTEXT_SCHEME@@$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
then
		echo ": no probing necessary"
elif test -n "$GIT_GETTEXT_POISON"
then
	echo GIT_INTERNAL_GETTEXT_SH_SCHEME=poison
elif type gettext.sh >/dev/null 2>&1
then
	# GNU libintl's gettext.sh
	echo GIT_INTERNAL_GETTEXT_SH_SCHEME=gnu
elif test "$(gettext -h 2>&1)" = "-h"
then
	# gettext binary exists but no gettext.sh. likely to be a gettext
	# binary on a Solaris or something that is not GNU libintl and
	# lack eval_gettext.
	echo GIT_INTERNAL_GETTEXT_SH_SCHEME=gettext_without_eval_gettext
fi
======================
I get
: no probing necessary
=====================

The generated git-sh-i18n is identical under both Ubuntu and Mac OS X.
(Except that Mac OS uses /Users instead of /home)

And may be the diff between git-sh-i18n.sh and the generated git-sh-i18n can give a hint:

 diff git-sh-i18n.sh git-sh-i18n
13c13
<       TEXTDOMAINDIR="@@LOCALEDIR@@"
---
>       TEXTDOMAINDIR="/Users/tb/share/locale"
21c21
< if test -n "@@USE_GETTEXT_SCHEME@@"
---
> if test -n ""
23c23
<       GIT_INTERNAL_GETTEXT_SH_SCHEME="@@USE_GETTEXT_SCHEME@@"
---
>       GIT_INTERNAL_GETTEXT_SH_SCHEME=""

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> [snip]
>  (. gettext.sh; eval_gettext frotz; echo $?)
> frotz0

So your system can claim "gnu".  The patch I sent earlier that
replaced "type gettext.sh" with something like the above should work
along that line.

> I get
> : no probing necessary

> When I run the following scriptlet, 
> ...
> elif test -n "@@USE_FALLTHROUGH_GETTEXT_SCHEME@@$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
> then
> 		echo ": no probing necessary"

Gaah.  That should read as @@USE_GETTEXT_SCHEME@@, which would be
replaced by the Makefile to become either empty (if you run "make"
without forcing USE_GETTEXT_SCHEME to any particular value).

It comes from ad17ea7 (add a Makefile switch to avoid gettext
translation in shell scripts, 2012-01-23) but nobody replaces the
token anywhere.

http://thread.gmane.org/gmane.comp.version-control.git/188706/focus=189016
and comparing it with the original (focus=188827 in the same thread),
we can see that this was my mistake while re-rolling the patch.

-- >8 --
Subject: i18n: fix auto detection of gettext scheme for shell scripts

A new code added by ad17ea7 (add a Makefile switch to avoid gettext
translation in shell scripts, 2012-01-23) tried to optionally force
a gettext scheme to "fallthrough", but ended up forcing it to everybody.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 git-sh-i18n.sh |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-sh-i18n.sh b/git-sh-i18n.sh
index d5fae99..6a27f68 100644
--- a/git-sh-i18n.sh
+++ b/git-sh-i18n.sh
@@ -21,7 +21,7 @@ GIT_INTERNAL_GETTEXT_SH_SCHEME=fallthrough
 if test -n "@@USE_GETTEXT_SCHEME@@"
 then
 	GIT_INTERNAL_GETTEXT_SH_SCHEME="@@USE_GETTEXT_SCHEME@@"
-elif test -n "@@USE_FALLTHROUGH_GETTEXT_SCHEME@@$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
+elif test -n "$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
 then
 	: no probing necessary
 elif test -n "$GIT_GETTEXT_POISON"

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Torsten Bögershausen]

> -- >8 --
> Subject: i18n: fix auto detection of gettext scheme for shell scripts
>
> A new code added by ad17ea7 (add a Makefile switch to avoid gettext
> translation in shell scripts, 2012-01-23) tried to optionally force
> a gettext scheme to "fallthrough", but ended up forcing it to everybody.
>
> Signed-off-by: Junio C Hamano<gitster@pobox.com>
> ---
>   git-sh-i18n.sh |    2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/git-sh-i18n.sh b/git-sh-i18n.sh
> index d5fae99..6a27f68 100644
> --- a/git-sh-i18n.sh
> +++ b/git-sh-i18n.sh
> @@ -21,7 +21,7 @@ GIT_INTERNAL_GETTEXT_SH_SCHEME=fallthrough
>   if test -n "@@USE_GETTEXT_SCHEME@@"
>   then
>   	GIT_INTERNAL_GETTEXT_SH_SCHEME="@@USE_GETTEXT_SCHEME@@"
> -elif test -n "@@USE_FALLTHROUGH_GETTEXT_SCHEME@@$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
> +elif test -n "$GIT_INTERNAL_GETTEXT_TEST_FALLBACKS"
>   then
>   	: no probing necessary
>   elif test -n "$GIT_GETTEXT_POISON"
> --
Thanks for the patch. Here are the results of t0204:
a) Suse 12.1
# lib-gettext: Found 'is_IS.utf8' as an is_IS UTF-8 locale
# lib-gettext: No is_IS ISO-8859-1 locale available
ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic
ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes
ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / 
Icelandic (missing GETTEXT_ISO_LOCALE)
ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / 
Runes (missing GETTEXT_ISO_LOCALE)
ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8
ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing 
GETTEXT_ISO_LOCALE)
ok 7 - gettext.c: git init UTF-8 -> UTF-8
ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing 
GETTEXT_ISO_LOCALE)
# passed all 8 test(s)
1..8
====================
b) Ubuntu 10.4:
tb@maxi:~/projects/git/tb/t$ ./t0204-gettext-reencode-sanity.sh
# lib-gettext: No is_IS UTF-8 locale available
# lib-gettext: No is_IS ISO-8859-1 locale available
ok 1 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / 
Icelandic (missing GETTEXT_LOCALE)
ok 2 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes 
(missing GETTEXT_LOCALE)
ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / 
Icelandic (missing GETTEXT_ISO_LOCALE)
ok 4 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / 
Runes (missing GETTEXT_ISO_LOCALE)
ok 5 # skip gettext: Fetching a UTF-8 msgid -> UTF-8 (missing 
GETTEXT_LOCALE)
ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing 
GETTEXT_ISO_LOCALE)
ok 7 # skip gettext.c: git init UTF-8 -> UTF-8 (missing GETTEXT_LOCALE)
ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing 
GETTEXT_ISO_LOCALE)
# passed all 8 test(s)
1..8
==============
c) Mac OS
# lib-gettext: Found 'is_IS.UTF-8' as an is_IS UTF-8 locale
# lib-gettext: Found 'is_IS.ISO8859-1' as an is_IS ISO-8859-1 locale
ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic
ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes
ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic
not ok - 4 gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Runes
#
#           LANGUAGE=is LC_ALL="$is_IS_iso_locale" gettext "TEST: Old 
English Runes" >runes &&
#
#               if grep "^TEST: Old English Runes$" runes
#               then
#                       say "Your system can not handle this complexity 
and returns the string as-is"
#               else
#                       # Both Solaris and GNU libintl will return this 
stream of
#                       # question marks, so it is s probably portable 
enough
#                       printf "TILRAUN: ?? ???? ??? ?? ???? ?? ??? 
????? ??????????? ??? ?? ????" >runes-expect &&
#                       test_cmp runes-expect runes
#               fi
#
ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8
ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1
ok 7 - gettext.c: git init UTF-8 -> UTF-8
ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1
# failed 1 among 8 test(s)
1..8
==============
(But that was expected, wasn't it? I added your original t0204 patch on 
top of that, and here we go:
==============
# lib-gettext: Found 'is_IS.UTF-8' as an is_IS UTF-8 locale
# lib-gettext: Found 'is_IS.ISO8859-1' as an is_IS ISO-8859-1 locale
ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic
ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes
ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic
ok 4 - gettext: impossible ISO-8859-1 output
ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8
ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1
ok 7 - gettext.c: git init UTF-8 -> UTF-8
ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1
# passed all 8 test(s)
1..8
/Torsten

Re: Re*: [PATCH/RFC] Change t0204-gettext-reencode-sanity.sh to pass under Mac OSX

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

>> -- >8 --
>> Subject: i18n: fix auto detection of gettext scheme for shell scripts
>> ...
> Thanks for the patch. Here are the results of t0204:

So in short, "everything looks peachy with the patch" seems to be the
conclusion.

Thanks.