msgmft segfaulting on tiger

msgmft segfaulting on tiger

[Martin Langhoff]

Today I've debugged a compile of git 1.5.4.2 failing on OSX Tiger (on
Basilio's machine). The problem is during the git-gui build, in the PO
file handling. Apparently fmtmsg (from fink) segfaults on de.po so the
fix is to say

   make NO_MSGFMT=NoneThere

I don't think we can do much about it -- my guess is that fink has
been built and tested in a different OSX release, and happens to go
bellyup on Tiger. Here's a transcript in case it helps anyone.

Here's the tail of the build message, with make called with -d:

GITGUI_VERSION = 0.9.2
     * new locations or Tcl/Tk interpreter
     GEN git-gui
     INDEX lib/
     MSGFMT    po/de.msg make[1]: *** [po/de.msg] Error 1
 make: *** [all] Error 2

Various versions of things involved:
 $ dpkg -S /sw/bin/msgfmt
 gettext-tools: /sw/bin/msgfmt
 $ dpkg -l gettext-tools
(...)
 ii  gettext-tools                     0.10.40-19
 $ uname -a
 Darwin developer 8.11.1 Darwin Kernel Version 8.11.1: Wed Oct 10
 18:23:28 PDT 2007; root:xnu-792.25.20~1/RELEASE_I386 i386 i386
 $ msgfmt -V
 msgfmt (GNU gettext) 0.10.40
 Copyright (C) 1995-1998, 2000, 2001 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 Written by Ulrich Drepper.

Has anyone got any ideas on how to debug this further? I couldn't find
any core files.

cheers,



martin

Re: msgmft segfaulting on tiger

[Martin Langhoff]

On Fri, Feb 22, 2008 at 12:53 PM, Martin Langhoff
<martin.langhoff@gmail.com> wrote:
> Today I've debugged a compile of git 1.5.4.2 failing on OSX Tiger (on
>  Basilio's machine). The problem is during the git-gui build, in the PO
>  file handling. Apparently fmtmsg (from fink) segfaults on de.po so the
>  fix is to say

Grumble. It seems like msgfmt that fink provides actually does not
support the --tcl flag. So perhaps the right thing is to always say
NO_MSGFMT on OSX?

cheers,


m

Re: msgmft segfaulting on tiger

[Shawn O. Pearce]

Martin Langhoff <martin.langhoff@gmail.com> wrote:
> On Fri, Feb 22, 2008 at 12:53 PM, Martin Langhoff
> <martin.langhoff@gmail.com> wrote:
> > Today I've debugged a compile of git 1.5.4.2 failing on OSX Tiger (on
> >  Basilio's machine). The problem is during the git-gui build, in the PO
> >  file handling. Apparently fmtmsg (from fink) segfaults on de.po so the
> >  fix is to say
> 
> Grumble. It seems like msgfmt that fink provides actually does not
> support the --tcl flag. So perhaps the right thing is to always say
> NO_MSGFMT on OSX?

I'd hate to do this.  GNU msgfmt does better checking on the
messages then po2msg.sh does, and its a tiny bit faster.  So
I'd rather stick to the standard tool that is widely used.

Currently we are looking to see if msgfmt generates an exit status
of 127, and if so we fallback to po2msg.sh.  Apparently however
that test is passing and thus you are using the fink msgfmt.

Did you try building with `make V=1` to look at the full output and
not the prettied up simplified output the Makefile uses by default?
It won't fix your issue but it may be interesting to see if msgfmt
is failing with a particular message or not.

-- 
Shawn.

Re: msgmft segfaulting on tiger

[Martin Langhoff]

On Fri, Feb 22, 2008 at 7:58 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
>  Did you try building with `make V=1` to look at the full output and
>  not the prettied up simplified output the Makefile uses by default?
>  It won't fix your issue but it may be interesting to see if msgfmt
>  is failing with a particular message or not.

I'll track down Basilio (it's his laptop) today and try, I think he's
still at the conference. I also suspect that OSX's own build tools may
include a different msgfmt, but the fink one is earlier in the path.

cheers,



m

Re: msgmft segfaulting on tiger

[Gabriel Saldaña]

We tried to build with make V=1 and this was the output:

msgfmt --statistics --tcl -l de -d po/ po/de.po 
msgfmt: unrecognized option `--tcl'
Try `msgfmt --help' for more information.
make[1]: *** [po/de.msg] Error 1
make: *** [all] Error 2

Still the error is in the po/de.po file

Hope this info helps

Gabriel Saldana

Martin Langhoff wrote:
> On Fri, Feb 22, 2008 at 7:58 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
>>  Did you try building with `make V=1` to look at the full output and
>>  not the prettied up simplified output the Makefile uses by default?
>>  It won't fix your issue but it may be interesting to see if msgfmt
>>  is failing with a particular message or not.
> 
> I'll track down Basilio (it's his laptop) today and try, I think he's
> still at the conference. I also suspect that OSX's own build tools may
> include a different msgfmt, but the fink one is earlier in the path.
> 
> cheers,
> 
> 
> 
> m
> 

Re: msgmft segfaulting on tiger

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 291 bytes --]

Hi,

On Wed, 27 Feb 2008, Gabriel Saldaña wrote:

> msgfmt --statistics --tcl -l de -d po/ po/de.po msgfmt: unrecognized option
> `--tcl'

Could you try this, please:

$ msgfmt --tcl; echo $?

Maybe we can have a clever autodetection of msgfmt which does not 
understand --tcl?

Ciao,
Dscho

Re: msgmft segfaulting on tiger

[Gabriel Saldaña]

Here it is:

$ msgfmt --tcl; echo $?
msgfmt: unrecognized option `--tcl'
Try `msgfmt --help' for more information.
1

Gabriel

Johannes Schindelin wrote:
> Hi,
> 
> On Wed, 27 Feb 2008, Gabriel Saldaña wrote:
> 
>> msgfmt --statistics --tcl -l de -d po/ po/de.po msgfmt: unrecognized option
>> `--tcl'
> 
> Could you try this, please:
> 
> $ msgfmt --tcl; echo $?
> 
> Maybe we can have a clever autodetection of msgfmt which does not 
> understand --tcl?
> 
> Ciao,
> Dscho

Re: msgmft segfaulting on tiger

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 365 bytes --]

Hi,

On Wed, 27 Feb 2008, Gabriel Saldaña wrote:

> Here it is:
> 
> $ msgfmt --tcl; echo $?
> msgfmt: unrecognized option `--tcl'
> Try `msgfmt --help' for more information.
> 1

Darn.  I think that's the same exit code as for any other invocation 
without filename.

So it seems that there is no easy way to tell a --tcl aware msgfmt from 
the other.

Ciao,
Dscho

Re: msgmft segfaulting on tiger

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> $ msgfmt --tcl; echo $?
>> msgfmt: unrecognized option `--tcl'
>> Try `msgfmt --help' for more information.
>> 1
>
> Darn.  I think that's the same exit code as for any other invocation 
> without filename.
>
> So it seems that there is no easy way to tell a --tcl aware msgfmt from 
> the other.

How about...

    $ msgfmt --tcl -l C -d . /dev/null; echo $?

Re: msgmft segfaulting on tiger

[Shawn O. Pearce]

Junio C Hamano <gitster@pobox.com> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> $ msgfmt --tcl; echo $?
> >> msgfmt: unrecognized option `--tcl'
> >> Try `msgfmt --help' for more information.
> >> 1
> >
> > Darn.  I think that's the same exit code as for any other invocation 
> > without filename.
> 
> How about...
> 
>     $ msgfmt --tcl -l C -d . /dev/null; echo $?


I can't test this, my msgfmt --tcl works properly.  Can someone
who has a msgfmt lacking tcl support confirm this will work?

--8<--
[PATCH] git-gui: Gracefully fall back to po2msg.sh if msgfmt --tcl fails

Mac OS X Tiger may have a msgfmt available but it doesn't understand
how to implement --tcl.  Falling back to po2msg.sh on such systems
is a reasonable behavior.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 Makefile |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index 01e0a46..1bd11eb 100644
--- a/Makefile
+++ b/Makefile
@@ -224,6 +224,11 @@ else
 	ifeq ($(shell $(MSGFMT) >/dev/null 2>&1 || echo $$?),127)
 		MSGFMT := $(TCL_PATH) po/po2msg.sh
 	endif
+	ifeq (msgfmt,$(MSGFMT))
+	ifeq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null || echo $?),1)
+		MSGFMT := $(TCL_PATH) po/po2msg.sh
+	endif
+	endif
 endif
 
 msgsdir     = $(gg_libdir)/msgs
-- 
1.5.4.3.295.g6b554


-- 
Shawn.

Re: msgmft segfaulting on tiger

[Gabriel Saldaña]

Here's the output, hope it helps:

$ msgfmt --tcl -l C -d . /dev/null; echo $?
msgfmt: unrecognized option `--tcl'
Try `msgfmt --help' for more information.
1

somehow msgfmt doesn't recognize the --tcl flag.

Gabriel Saldana

Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
>>> $ msgfmt --tcl; echo $?
>>> msgfmt: unrecognized option `--tcl'
>>> Try `msgfmt --help' for more information.
>>> 1
>> Darn.  I think that's the same exit code as for any other invocation 
>> without filename.
>>
>> So it seems that there is no easy way to tell a --tcl aware msgfmt from 
>> the other.
> 
> How about...
> 
>     $ msgfmt --tcl -l C -d . /dev/null; echo $?
> 

msgmft segfaulting on ubuntu gutsy (clean)

[Gabriel Saldaña]

I'm getting the msgmft error message when compiling git 1.5.4.3 on a 
clean install of Ubuntu Gutsy. Installed dependencies using "apt-get 
build-dep git-core".

Here's make output:

GITGUI_VERSION = 0.9.2.7.g3bae
     * new locations or Tcl/Tk interpreter
     GEN git-gui
     INDEX lib/
     MSGFMT    po/de.msg 371 translated.
     MSGFMT    po/fr.msg 371 translated.
     MSGFMT    po/hu.msg 371 translated.
     MSGFMT    po/it.msg 370 translated, 1 untranslated.
     MSGFMT    po/ja.msg 371 translated.
     MSGFMT    po/ru.msg 367 translated, 4 untranslated.
     MSGFMT    po/sv.msg 371 translated.
     MSGFMT po/zh_cn.msg 53 translated, 57 fuzzy, 253 untranslated.
     SUBDIR gitk-git
     GEN gitk-wish
Generating catalog po/de.msg
msgfmt --statistics --tcl po/de.po -l de -d po/
make[1]: msgfmt: Command not found
make[1]: *** [po/de.msg] Error 127
make: *** [all] Error 2

It is very similar to Tiger's error message.

Gabriel Saldana

Re: msgmft segfaulting on ubuntu gutsy (clean)

[Shawn O. Pearce]

Gabriel Saldaa <gsaldana@gmail.com> wrote:
> I'm getting the msgmft error message when compiling git 1.5.4.3 on a 
> clean install of Ubuntu Gutsy. Installed dependencies using "apt-get 
> build-dep git-core".
> 
> Here's make output:
> 
> GITGUI_VERSION = 0.9.2.7.g3bae
>     * new locations or Tcl/Tk interpreter
>     GEN git-gui
>     INDEX lib/
>     MSGFMT    po/de.msg 371 translated.
>     MSGFMT    po/fr.msg 371 translated.
>     MSGFMT    po/hu.msg 371 translated.
>     MSGFMT    po/it.msg 370 translated, 1 untranslated.
>     MSGFMT    po/ja.msg 371 translated.
>     MSGFMT    po/ru.msg 367 translated, 4 untranslated.
>     MSGFMT    po/sv.msg 371 translated.
>     MSGFMT po/zh_cn.msg 53 translated, 57 fuzzy, 253 untranslated.
>     SUBDIR gitk-git
>     GEN gitk-wish
> Generating catalog po/de.msg
> msgfmt --statistics --tcl po/de.po -l de -d po/
> make[1]: msgfmt: Command not found
> make[1]: *** [po/de.msg] Error 127
> make: *** [all] Error 2
> 
> It is very similar to Tiger's error message.

Rebuild with "NO_MSGFMT=1", or install GNU msgfmt first.

The issue is gitk's Makefile is missing the fallback rule that
git-gui has, which is use our Tcl based msgfmt work-a-like if msgfmt
is not found.  That's exactly what you see above; git-gui fell back
automatically to its work-a-like but gitk did not.

-- 
Shawn.

Re: msgmft segfaulting on tiger

[Shawn O. Pearce]

Gabriel Saldaa <gsaldana@gmail.com> wrote:
> Here's the output, hope it helps:
> 
> $ msgfmt --tcl -l C -d . /dev/null; echo $?
> msgfmt: unrecognized option `--tcl'
> Try `msgfmt --help' for more information.
> 1
> 
> somehow msgfmt doesn't recognize the --tcl flag.

Just to revive a nearly dead thead...  I have a workaround now in
git-gui maint that looks for this case and falls into our po2msg
script when msgfmt doesn't see --tcl.

I also set Paul Mackerras a patch for gitk, so it can do the
same thing.  Maybe Git 1.5.4.4 will include both patches.
 
> Junio C Hamano wrote:
> >Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> >>>$ msgfmt --tcl; echo $?
> >>>msgfmt: unrecognized option `--tcl'
> >>>Try `msgfmt --help' for more information.
> >>>1
> >>Darn.  I think that's the same exit code as for any other invocation 
> >>without filename.
> >>
> >>So it seems that there is no easy way to tell a --tcl aware msgfmt from 
> >>the other.
> >
> >How about...
> >
> >    $ msgfmt --tcl -l C -d . /dev/null; echo $?

-- 
Shawn.

Re: msgmft segfaulting on tiger

[Junio C Hamano]

"Shawn O. Pearce" <spearce@spearce.org> writes:

> Just to revive a nearly dead thead...  I have a workaround now in
> git-gui maint that looks for this case and falls into our po2msg
> script when msgfmt doesn't see --tcl.
>
> I also set Paul Mackerras a patch for gitk, so it can do the
> same thing.  Maybe Git 1.5.4.4 will include both patches.

I looked at what you have in your Makefile.  

It makes me wonder, however, if it makes more sense to squash the two
invocations of msgfmt you make into one, like the attached patch.

The point is that msgfmt you guys want to use _must_ understand --tcl,
and a msgfmt new enough to be usable would exit with 0 on the above
command line.

-- >8 --
gitk i18n: Fall back to po2msg.sh if msgfmt --tcl fails

Some platforms may have a msgfmt that doesn't understand --tcl.  Mac OS X
Tiger is an example.

Falling back to po2msg.sh on such systems is a reasonable behavior.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Makefile |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index ae2b80b..c72e7f7 100644
--- a/Makefile
+++ b/Makefile
@@ -22,6 +22,9 @@ ifdef NO_MSGFMT
 	MSGFMT ?= $(TCL_PATH) po/po2msg.sh
 else
 	MSGFMT ?= msgfmt
+	ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $?),0)
+		MSGFMT = $(TCL_PATH) po/po2msg.sh 
+	endif
 endif
 
 PO_TEMPLATE = po/gitk.pot

[PATCH] Simplify MSGFMT setting in Makefile

[Junio C Hamano]

To prepare msg files for Tcl scripts, the command that is set to MSGFMT
make variable needs to be able to grok "--tcl -l <lang> -d <here>" options
correctly.  This patch simplifies the tests done in git-gui's Makefile to
directly test this condition.  If the test run does not exit properly with
zero status (either because you do not have "msgfmt" itself, or your
"msgfmt" is too old to grok --tcl option --- the reason does not matter),
have it fall back to po/po2msg.sh

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---

 * I am planning to close 'master' for -rc1 by the end of this week, so
   please plan to tell me to pull if you have accumulated changes,
   thanks.

 Makefile |    7 +------
 1 files changed, 1 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 4e32174..95e99f2 100644
--- a/Makefile
+++ b/Makefile
@@ -221,14 +221,9 @@ ifdef NO_MSGFMT
 	MSGFMT ?= $(TCL_PATH) po/po2msg.sh
 else
 	MSGFMT ?= msgfmt
-	ifeq ($(shell $(MSGFMT) >/dev/null 2>&1 || echo $$?),127)
+	ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $?),0)
 		MSGFMT := $(TCL_PATH) po/po2msg.sh
 	endif
-	ifeq (msgfmt,$(MSGFMT))
-	ifeq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null || echo $?),1)
-		MSGFMT := $(TCL_PATH) po/po2msg.sh
-	endif
-	endif
 endif
 
 msgsdir     = $(gg_libdir)/msgs

Re: [PATCH] Simplify MSGFMT setting in Makefile

[Shawn O. Pearce]

Junio C Hamano <gitster@pobox.com> wrote:
> To prepare msg files for Tcl scripts, the command that is set to MSGFMT
...

Thanks.

>  * I am planning to close 'master' for -rc1 by the end of this week, so
>    please plan to tell me to pull if you have accumulated changes,
>    thanks.

Please pull.  What is in my master right now (7276607 which is
this patch) will likely be gitgui-0.10.0.  I'm unlikely to have
any further changes (beyond perhaps updated po files) between now
and 1.5.5 final.

>  Makefile |    7 +------
>  1 files changed, 1 insertions(+), 6 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 4e32174..95e99f2 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -221,14 +221,9 @@ ifdef NO_MSGFMT
>  	MSGFMT ?= $(TCL_PATH) po/po2msg.sh
>  else
>  	MSGFMT ?= msgfmt
> -	ifeq ($(shell $(MSGFMT) >/dev/null 2>&1 || echo $$?),127)
> +	ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $?),0)

Missing a $ here, it should be $$? not $?.  Fixed up when I applied.

-- 
Shawn.

Re: [PATCH] Simplify MSGFMT setting in Makefile

[Charles Bailey]

On Tue, Mar 11, 2008 at 10:29:52PM -0700, Junio C Hamano wrote:
> To prepare msg files for Tcl scripts, the command that is set to MSGFMT
> make variable needs to be able to grok "--tcl -l <lang> -d <here>" options
> correctly.  This patch simplifies the tests done in git-gui's Makefile to
> directly test this condition.  If the test run does not exit properly with
> zero status (either because you do not have "msgfmt" itself, or your
> "msgfmt" is too old to grok --tcl option --- the reason does not matter),
> have it fall back to po/po2msg.sh
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Tested-by: Charles Bailey <charles@hashpling.org>

This looks good to me on Mac OS X 10.5.2, would anyone like me to
prepare a patch for gitk as without it we still need to manually
specifiy NO_MSGFMT to the top level git make in any case?

Charles.

Re: [PATCH] Simplify MSGFMT setting in Makefile

[Paul Mackerras]

Junio C Hamano writes:

>  * I am planning to close 'master' for -rc1 by the end of this week, so
>    please plan to tell me to pull if you have accumulated changes,
>    thanks.

I have a few commits for you to pull, but I am about to push a patch
for gitk like the one you sent for git-gui, so I'll send you a pull
request when I've done that.  However, I found that in this line:

>  else
>  	MSGFMT ?= msgfmt
> -	ifeq ($(shell $(MSGFMT) >/dev/null 2>&1 || echo $$?),127)
> +	ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $?),0)

I had to put "echo $$?" for it to work rather than "echo $?".

Paul.

Re: [PATCH] Simplify MSGFMT setting in Makefile

[Junio C Hamano]

Paul Mackerras <paulus@samba.org> writes:

> I have a few commits for you to pull, but I am about to push a patch
> for gitk like the one you sent for git-gui, so I'll send you a pull
> request when I've done that.

Thanks.

>>  else
>>  	MSGFMT ?= msgfmt
>> -	ifeq ($(shell $(MSGFMT) >/dev/null 2>&1 || echo $$?),127)
>> +	ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $?),0)
>
> I had to put "echo $$?" for it to work rather than "echo $?".

Yeah, very sorry about this; Shawn already has pointed it out to me.

Re: [PATCH] Simplify MSGFMT setting in Makefile

[Andreas Ericsson]

Paul Mackerras wrote:
> Junio C Hamano writes:
> 
>>  * I am planning to close 'master' for -rc1 by the end of this week, so
>>    please plan to tell me to pull if you have accumulated changes,
>>    thanks.
> 
> I have a few commits for you to pull, but I am about to push a patch
> for gitk like the one you sent for git-gui, so I'll send you a pull
> request when I've done that.  However, I found that in this line:
> 
>>  else
>>  	MSGFMT ?= msgfmt
>> -	ifeq ($(shell $(MSGFMT) >/dev/null 2>&1 || echo $$?),127)
>> +	ifneq ($(shell $(MSGFMT) --tcl -l C -d . /dev/null 2>/dev/null; echo $?),0)
> 
> I had to put "echo $$?" for it to work rather than "echo $?".
> 

$$ is the way to pass a literal $ to the shell in Makefiles, so that
seems in order.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231