bug: "fatal: cannot pread pack file", version 1.7.5.4

bug: "fatal: cannot pread pack file", version 1.7.5.4

[Lars Winterfeld]

Hi,

I have used git for some time and regularly pushed changes to another 
server (as some kind of backup), but today I got this error during `git 
push`:

Counting objects: 1293, done.
Compressing objects: 100% (1027/1027), done.
Writing objects: 100% (1229/1229), 70.83 MiB | 364 KiB/s, done.
Total 1229 (delta 343), reused 0 (delta 0)
fatal: cannot pread pack file: No such file or directory
error: unpack failed: index-pack abnormal exit
To user@host:~/path/
 ! [remote rejected] master -> master (n/a (unpacker error))
error: failed to push some refs to 'user@host:~/path/'

I found similar bug reports suggesting to upgrade too version 1.7.x, 
but I am already using git version 1.7.5.4. Can somebody explain to me 
what this error is about? What can I do to resolve it / is there a 
workaround?

(There is enough disk space on the other server.)

Lars

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Junio C Hamano]

Lars Winterfeld <lars.winterfeld@tu-ilmenau.de> writes:

> Hi,
>
> I have used git for some time and regularly pushed changes to another
> server (as some kind of backup), but today I got this error during
> git push`:
>
> Counting objects: 1293, done.
> Compressing objects: 100% (1027/1027), done.
> Writing objects: 100% (1229/1229), 70.83 MiB | 364 KiB/s, done.
> Total 1229 (delta 343), reused 0 (delta 0)
> fatal: cannot pread pack file: No such file or directory

Curious.  That is typically given for ENOENT but pread(2) would not
give that error according to

  http://pubs.opengroup.org/onlinepubs/9699919799/functions/pread.html

You didn't say what platform you are seeing the issue (the receiving
end of the "push"), but are you using Git on a pread-deficient
platform with emulation perhaps?

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Jeff King]

On Mon, Jul 02, 2012 at 12:03:13PM -0700, Lars Winterfeld wrote:

> I have used git for some time and regularly pushed changes to another
> server (as some kind of backup), but today I got this error during
> `git push`:
> 
> Counting objects: 1293, done.
> Compressing objects: 100% (1027/1027), done.
> Writing objects: 100% (1229/1229), 70.83 MiB | 364 KiB/s, done.
> Total 1229 (delta 343), reused 0 (delta 0)
> fatal: cannot pread pack file: No such file or directory
> error: unpack failed: index-pack abnormal exit
> To user@host:~/path/
> ! [remote rejected] master -> master (n/a (unpacker error))
> error: failed to push some refs to 'user@host:~/path/'
> 
> I found similar bug reports suggesting to upgrade too version 1.7.x,
> but I am already using git version 1.7.5.4. Can somebody explain to
> me what this error is about? What can I do to resolve it / is there a
> workaround?

It's very odd for pread to report ENOENT (since it is always operating
on an already-opened file descriptor). Older versions of git would
sometimes sometimes report errno even when we get a return of 0 (when
errno is bogus, and they should say "truncated file" or similar), but
that was long ago fixed in fb74243 (Do not use errno when pread()
returns 0, 2008-10-06). There was also an actual bug fixed by a91d49c
(index-pack: Loop over pread until data loading is complete.,
2007-02-27) that I think could also generate this error.

What OS is the server running? Is it using NFS? Are you sure that the
server version is also v1.7.5.4?

-Peff

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Andreas Schwab]

Jeff King <peff@peff.net> writes:

> It's very odd for pread to report ENOENT (since it is always operating
> on an already-opened file descriptor).

It doesn't, but gettext will clobber errno:

		n = pread(pack_fd, inbuf, n, from);
		if (n < 0)
			die_errno(_("cannot pread pack file"));

There is nothing that saves errno.  This isn't limited to i18n though,
any function call in the arguments may potentially clobber errno.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Junio C Hamano]

Andreas Schwab <schwab@linux-m68k.org> writes:

> Jeff King <peff@peff.net> writes:
>
>> It's very odd for pread to report ENOENT (since it is always operating
>> on an already-opened file descriptor).
>
> It doesn't, but gettext will clobber errno:
>
> 		n = pread(pack_fd, inbuf, n, from);
> 		if (n < 0)
> 			die_errno(_("cannot pread pack file"));
>
> There is nothing that saves errno.  This isn't limited to i18n though,
> any function call in the arguments may potentially clobber errno.

Argh...

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Jeff King]

On Tue, Jul 03, 2012 at 12:43:42AM +0200, Andreas Schwab wrote:

> Jeff King <peff@peff.net> writes:
> 
> > It's very odd for pread to report ENOENT (since it is always operating
> > on an already-opened file descriptor).
> 
> It doesn't, but gettext will clobber errno:
> 
> 		n = pread(pack_fd, inbuf, n, from);
> 		if (n < 0)
> 			die_errno(_("cannot pread pack file"));
> 
> There is nothing that saves errno.  This isn't limited to i18n though,
> any function call in the arguments may potentially clobber errno.

That's horribly lame of gettext. I don't expect arbitrary functions to
save errno, but when the entire purpose of a function is to be a
non-intrusive wrapper to massage messages to the user, it seems kind of
evil to overwrite errno. Isn't the whole point of calling it "_" that
you don't want to or have to notice it?

Can we do something like this to get around it?

diff --git a/gettext.h b/gettext.h
index 57ba8bb..b7c3ae5 100644
--- a/gettext.h
+++ b/gettext.h
@@ -44,7 +44,10 @@ extern int use_gettext_poison(void);
 
 static inline FORMAT_PRESERVING(1) const char *_(const char *msgid)
 {
-	return use_gettext_poison() ? "# GETTEXT POISON #" : gettext(msgid);
+	int saved_errno = errno;
+	const char *r = use_gettext_poison() ? "# GETTEXT POISON #" : gettext(msgid);
+	errno = saved_errno;
+	return r;
 }
 
 static inline FORMAT_PRESERVING(1) FORMAT_PRESERVING(2)

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Nguyen Thai Ngoc Duy]

On Tue, Jul 3, 2012 at 10:45 AM, Jeff King <peff@peff.net> wrote:
> On Tue, Jul 03, 2012 at 12:43:42AM +0200, Andreas Schwab wrote:
>
>> Jeff King <peff@peff.net> writes:
>>
>> > It's very odd for pread to report ENOENT (since it is always operating
>> > on an already-opened file descriptor).
>>
>> It doesn't, but gettext will clobber errno:
>>
>>               n = pread(pack_fd, inbuf, n, from);
>>               if (n < 0)
>>                       die_errno(_("cannot pread pack file"));
>>
>> There is nothing that saves errno.  This isn't limited to i18n though,
>> any function call in the arguments may potentially clobber errno.
>
> That's horribly lame of gettext. I don't expect arbitrary functions to
> save errno, but when the entire purpose of a function is to be a
> non-intrusive wrapper to massage messages to the user, it seems kind of
> evil to overwrite errno. Isn't the whole point of calling it "_" that
> you don't want to or have to notice it?

Agreed.

> Can we do something like this to get around it?
>
> diff --git a/gettext.h b/gettext.h
> index 57ba8bb..b7c3ae5 100644
> --- a/gettext.h
> +++ b/gettext.h
> @@ -44,7 +44,10 @@ extern int use_gettext_poison(void);
>
>  static inline FORMAT_PRESERVING(1) const char *_(const char *msgid)
>  {
> -       return use_gettext_poison() ? "# GETTEXT POISON #" : gettext(msgid);
> +       int saved_errno = errno;
> +       const char *r = use_gettext_poison() ? "# GETTEXT POISON #" : gettext(msgid);
> +       errno = saved_errno;
> +       return r;
>  }
>
>  static inline FORMAT_PRESERVING(1) FORMAT_PRESERVING(2)

The last line belongs to Q_(), which needs the same treatment.
-- 
Duy

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Jeff King]

On Tue, Jul 03, 2012 at 11:25:16AM +0700, Nguyen Thai Ngoc Duy wrote:

> On Tue, Jul 3, 2012 at 10:45 AM, Jeff King <peff@peff.net> wrote:
> > On Tue, Jul 03, 2012 at 12:43:42AM +0200, Andreas Schwab wrote:
> >
> >> Jeff King <peff@peff.net> writes:
> >>
> >> > It's very odd for pread to report ENOENT (since it is always operating
> >> > on an already-opened file descriptor).
> >>
> >> It doesn't, but gettext will clobber errno:
> >>
> >>               n = pread(pack_fd, inbuf, n, from);
> >>               if (n < 0)
> >>                       die_errno(_("cannot pread pack file"));
> >>
> >> There is nothing that saves errno.  This isn't limited to i18n though,
> >> any function call in the arguments may potentially clobber errno.
> >
> > That's horribly lame of gettext. I don't expect arbitrary functions to
> > save errno, but when the entire purpose of a function is to be a
> > non-intrusive wrapper to massage messages to the user, it seems kind of
> > evil to overwrite errno. Isn't the whole point of calling it "_" that
> > you don't want to or have to notice it?
> 
> Agreed.

Hmm. According to gettext(3):

  ERRORS
         errno is not modified.

And googling for "gettext" and "errno" reveals several bugfixes in GNU
gettext to make sure that gettext preserves errno. I wonder if there are
systems where that is not the case, though; I don't know what non-GNU
gettext implementations are in common use these days. I'd still be
curious to hear what platform the server is for this bug report.

-Peff

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Hmm. According to gettext(3):
>
>   ERRORS
>          errno is not modified.
>
> And googling for "gettext" and "errno" reveals several bugfixes in GNU
> gettext to make sure that gettext preserves errno. I wonder if there are
> systems where that is not the case, though; I don't know what non-GNU
> gettext implementations are in common use these days. I'd still be
> curious to hear what platform the server is for this bug report.

Yes, I am also curious.

Regardless, we may need your "save_errno" patch if the bulk of it
may have to be made conditional on NEED_SAVE_ERRNO_IN_GETTEXT=YesPlease
or something.

Thanks for digging.

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Tue, Jul 03, 2012 at 11:25:16AM +0700, Nguyen Thai Ngoc Duy wrote:
>
>> On Tue, Jul 3, 2012 at 10:45 AM, Jeff King <peff@peff.net> wrote:
>> > On Tue, Jul 03, 2012 at 12:43:42AM +0200, Andreas Schwab wrote:
>> >
>> >> Jeff King <peff@peff.net> writes:
>> >>
>> >> > It's very odd for pread to report ENOENT (since it is always operating
>> >> > on an already-opened file descriptor).
>> >>
>> >> It doesn't, but gettext will clobber errno:
>> >>
>> >>               n = pread(pack_fd, inbuf, n, from);
>> >>               if (n < 0)
>> >>                       die_errno(_("cannot pread pack file"));
>> >>
>> >> There is nothing that saves errno.  This isn't limited to i18n though,
>> >> any function call in the arguments may potentially clobber errno.
>> >
>> > That's horribly lame of gettext. I don't expect arbitrary functions to
>> > save errno, but when the entire purpose of a function is to be a
>> > non-intrusive wrapper to massage messages to the user, it seems kind of
>> > evil to overwrite errno. Isn't the whole point of calling it "_" that
>> > you don't want to or have to notice it?
>> 
>> Agreed.
>
> Hmm. According to gettext(3):
>
>   ERRORS
>          errno is not modified.
>
> And googling for "gettext" and "errno" reveals several bugfixes in GNU
> gettext to make sure that gettext preserves errno. I wonder if there are
> systems where that is not the case, though; I don't know what non-GNU
> gettext implementations are in common use these days. I'd still be
> curious to hear what platform the server is for this bug report.

Hrm, has this ever been resolved?

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Lars Winterfeld]

Junio C Hamano:
> Jeff King <peff@peff.net> writes:
> 
>> On Tue, Jul 03, 2012 at 11:25:16AM +0700, Nguyen Thai Ngoc Duy wrote:
>>
>>> On Tue, Jul 3, 2012 at 10:45 AM, Jeff King <peff@peff.net> wrote:
>>> > On Tue, Jul 03, 2012 at 12:43:42AM +0200, Andreas Schwab wrote:
>>> >
>>> >> Jeff King <peff@peff.net> writes:
>>> >>
>>> >> > It's very odd for pread to report ENOENT (since it is always operating
>>> >> > on an already-opened file descriptor).
>>> >>
>>> >> It doesn't, but gettext will clobber errno:
>>> >>
>>> >>               n = pread(pack_fd, inbuf, n, from);
>>> >>               if (n < 0)
>>> >>                       die_errno(_("cannot pread pack file"));
>>> >>
>>> >> There is nothing that saves errno.  This isn't limited to i18n though,
>>> >> any function call in the arguments may potentially clobber errno.
>>> >
>>> > That's horribly lame of gettext. I don't expect arbitrary functions to
>>> > save errno, but when the entire purpose of a function is to be a
>>> > non-intrusive wrapper to massage messages to the user, it seems kind of
>>> > evil to overwrite errno. Isn't the whole point of calling it "_" that
>>> > you don't want to or have to notice it?
>>>
>>> Agreed.
>>
>> Hmm. According to gettext(3):
>>
>>   ERRORS
>>          errno is not modified.
>>
>> And googling for "gettext" and "errno" reveals several bugfixes in GNU
>> gettext to make sure that gettext preserves errno. I wonder if there are
>> systems where that is not the case, though; I don't know what non-GNU
>> gettext implementations are in common use these days. I'd still be
>> curious to hear what platform the server is for this bug report.
> 
> Hrm, has this ever been resolved?

Thank you for asking.
My local git version is 1.7.5.4, the server version that I actually
ended up pushing the files to was however still some 1.6.x.
So it was a false alarm. Sorry about that.

As far as I can follow, there is no non-GNU gettext involved. Thanks
for digging that deep into the problem, but it was my fault, probably
not something about lost errno.

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Junio C Hamano]

Lars Winterfeld <lars.winterfeld@tu-ilmenau.de> writes:

>> Hrm, has this ever been resolved?
>
> Thank you for asking.
> My local git version is 1.7.5.4, the server version that I actually
> ended up pushing the files to was however still some 1.6.x.
> So it was a false alarm. Sorry about that.

Not really.  Pushing or fetching by a new version of Git against old
version of Git *should* work; otherwise it is a bug.

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Lars Winterfeld]

Junio C Hamano schrieb:
> Lars Winterfeld <lars.winterfeld@tu-ilmenau.de> writes:
> 
>>> Hrm, has this ever been resolved?
>>
>> Thank you for asking.
>> My local git version is 1.7.5.4, the server version that I actually
>> ended up pushing the files to was however still some 1.6.x.
>> So it was a false alarm. Sorry about that.
> 
> Not really.  Pushing or fetching by a new version of Git against old
> version of Git *should* work; otherwise it is a bug.

Well, okay. The (outdated) server
runs on openSUSE 11.1,
has kernel 2.6.27.39-0.2-default x86_64 GNU/Linux, 
uses git version 1.6.0.2
and gettext (GNU gettext-runtime) 0.17 .

As I said earlier, `git push` worked in the beginning and just stopped
working one day.
I can still reproduce this error, so let me know if you want more info.

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Jeff King]

On Thu, Aug 16, 2012 at 04:05:22PM -0700, Junio C Hamano wrote:

> Lars Winterfeld <lars.winterfeld@tu-ilmenau.de> writes:
> 
> >> Hrm, has this ever been resolved?
> >
> > Thank you for asking.
> > My local git version is 1.7.5.4, the server version that I actually
> > ended up pushing the files to was however still some 1.6.x.
> > So it was a false alarm. Sorry about that.
> 
> Not really.  Pushing or fetching by a new version of Git against old
> version of Git *should* work; otherwise it is a bug.

Yes, but there may be old bugs in the server version that have since
been fixed. For example, the bug fixed by 838cd346 might be the cause of
this (that fix went into 1.6.0.3).

-Peff

Re: bug: "fatal: cannot pread pack file", version 1.7.5.4

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Thu, Aug 16, 2012 at 04:05:22PM -0700, Junio C Hamano wrote:
>
>> Lars Winterfeld <lars.winterfeld@tu-ilmenau.de> writes:
>> 
>> >> Hrm, has this ever been resolved?
>> >
>> > Thank you for asking.
>> > My local git version is 1.7.5.4, the server version that I actually
>> > ended up pushing the files to was however still some 1.6.x.
>> > So it was a false alarm. Sorry about that.
>> 
>> Not really.  Pushing or fetching by a new version of Git against old
>> version of Git *should* work; otherwise it is a bug.
>
> Yes, but there may be old bugs in the server version that have since
> been fixed. For example, the bug fixed by 838cd346 might be the cause of
> this (that fix went into 1.6.0.3).

Yeah, Lars said 1.6.x above, but in the response it turns out the
version used is 1.6.0.2, so lack of that commit is a very likely
reason.

Thanks.