Re: git failure on HP-UX - more data

From: Junio C Hamano <gitster@pobox.com>
To: "Kibler\, Bill" <bill.kibler@hp.com>
Cc: "git\@vger.kernel.org" <git@vger.kernel.org>,
	Richard Lloyd <richard.lloyd@connectinternetsolutions.com>,
	"kibler\@psyber.com" <kibler@psyber.com>
Subject: Re: git failure on HP-UX - more data
Date: Fri, 06 May 2011 13:11:36 -0700	[thread overview]
Message-ID: <7vliyjzj0n.fsf@alter.siamese.dyndns.org> (raw)
In-Reply-To: <88C5107F36BD7E4BBFDFDB140E3D81077FA78CF67E@GVW1362EXC.americas.hpqcorp.net> (Bill Kibler's message of "Fri, 6 May 2011 19:02:50 +0000")

"Kibler, Bill" <bill.kibler@hp.com> writes:

> I just ran several tests on hp-UX using various values for the
> "LARGE_PACKET_MAX", ...

The original sideband protocol had a fixed receiver side buffer that was
only 1000 bytes long.  When we updated the protocol so that we can carry
more payload with a single logical pkt-line format, which has the maximum
packet length of a bit less than 64k (it has a fixed 4 hexadecimal digits
field at the front that indicates its size, so the maximum payload size is
64k minus 4 or something like that), we added a protocol extension that is
negotiated between the server and the client for both sides to make sure
that they have the updated implementation in which the receiver is
prepared to accept a 64k packet, not just a small 1000-byte static buffer.

But all of that is at the logical protocol level.

Even if the transfer goes over the Ethernet, this size is in no way
limited by its MTU of 1500 bytes, because the kernel will take care of
buffering and reassembling for us.

It is the same deal for the pkt-line protocol, where we issue a write(2)
and expect that the system may write less than what we asked it to write,
and return us how many bytes it has actually written. As long as write(2)
correctly returns the number of bytes it wrote, and our code that calls
write(2) correctly expects a short-write and loops until writing
everything out, there is no need to worry about LARGE_PACKET_MAX.

At least, that is the theory.

I think already said this in my previous message to you, but it is
possible that we have a bug in our code that fails to expect write(2) to
result in short-write and loop until we write everything out.  My gut
feeling is that it is slightly more plausible that we have such a bug than
that your libc has a buggy implementation of write(2) that returns a bogus
value (say 64k) when in fact it wrote only what would fit in your pipe
buffer (you said 8k, I think) when asked to write 64k.

And the right thing to do is to find and fix such a bug. I am afraid you
are wasting your time by futzing LARGE_PACKET_MAX. Even if you find a good
small value that happens to work on _your_ machine, it would not be a real
fix for the problem.