RE: git failure on HP-UX - more data

["Kibler, Bill" <bill.kibler@hp.com>]

I spent some time thinking about this problem and realized that guessing the
pipe size was the issue probably was wrong, when it is likely something more 
generic in nature. I did some more testing, and remembered a similar problem
when trying to do a git clone of the arm kernel for my $99 netbook article
(see www.kiblerelectronics./corner/ccii.shtml), where it failed much
the same as on HP-UX. My thinking is that this is a variable that is controlled
by the OS/Libc settings and not something that can be selected as a
good enough value. My real problem and reason for putting in a bug report,
is wanting to know how this value was intended to be used.

I just ran several tests on hp-UX using various values for the
"LARGE_PACKET_MAX", all trying to see what the actual failure value is.
I thought it might be related to ssize_t, which in HP-UX is 32K, but
values around 32K all worked fine. 65000 failed, while 48000 worked.
I went through the normal trial and error process and found the value
to vary over a number of tests - in short - not fixed. That suggested
it is partly controlled by some amount of free space(?).

I will test out my arm netbook and see if changing this value helps git
on my system, but for now I feel this value was selected somewhat
arbitrarily when it should be selected as the smallest value that can
work for your OS of choice. I thought of using 32K on HP-UX, but decided
that if I don't really know what the mechanism is controlling this value
then the smallest working value is the safest to use (8208).

Can anyone in this group explain what is going on for me? Why was 64K
selected in the first place?
Thanks.
Bill. 

-----Original Message-----
From: Junio C Hamano [mailto:gitster@pobox.com] 
Sent: Thursday, May 05, 2011 12:07 PM
To: Kibler, Bill
Cc: git@vger.kernel.org; Richard Lloyd
Subject: Re: git failure on HP-UX

"Kibler, Bill" <bill.kibler@hp.com> writes:

> In looking at the code, "sideband.h" defines "LARGE_PACKET_MAX 65520"
> and is related to the passed flag "side-band-64k" as discussed in git
> document pack-protocol.txt. The current default usage seems to be 64K
> transfers, yet if we check the "include/limits.h" of HP-UX we see a
> "PIPE_BUF" set to 8192. Along with the tusc indication of 8K pipe size,
> I suspect that HP-UX is coring due to git trying to use a 64K pipe when
> 8K is max.
>
> I solved the probem for now, by changing the file sideband.h to use
> "LARGE_PACKET_MAX 8208".

This does not make any sense.  We may make write(2) and read(2) system
calls with 64k (or maybe bit more) chunk, but that does not mean the
implementation of these system calls must take that as a whole.  Your
write(2) is allowed to write only whatever fits your pipe buffer, and tell
the caller "I wrote only 8192 bytes", and the code is supposed to loop,
advancing the write pointer by 8k and calling write(2) again, until you
write everything to whoever is reading the other end of the pipe.  The
same thing for the read(2).

If you can find a place where we make write(2)/read(2) and blindly assumes
that a non-negative return means everything was written/read successfully,
then you have found a bug.

If the symptom _were_ a deadlock where the writer of one pipe expected to
be able to send 64k to the other end of the pipe and then hear back from
the other side with a separate read, I would understand that could happen
(actually we know a local pipe transfer without ssh has that kind of
potential deadlock but I think the size we assume that can fit in the pipe
buffer is far smaller than 8k).  But I do not understand where a SIGBUS
can come from.