Re: [PATCH 2/2] index-pack: prefetch missing REF_DELTA bases

From: Jeff King <peff@peff.net>
To: Jonathan Tan <jonathantanmy@google.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 2/2] index-pack: prefetch missing REF_DELTA bases
Date: Thu, 16 May 2019 21:22:34 -0400	[thread overview]
Message-ID: <20190517012234.GA31027@sigill.intra.peff.net> (raw)
In-Reply-To: <20190517010950.GA30146@sigill.intra.peff.net>

On Thu, May 16, 2019 at 09:09:50PM -0400, Jeff King wrote:

>   - will we ever append a presumed-thin base to the pack, only to later
>     realize that we already have that object, creating a duplicate
>     object in the pack? If so, do we handle this correctly when
>     generating the index (I know we've had issues in the past and have
>     expressly forbidden duplicates from appearing in the index; even
>     having a duplicate in the pack stream itself is non-ideal, though,
>     as it screws up things like on-disk size calculations).
> 
>     Because of the sorting in fix_unresolved_deltas(), I think this
>     could easily be prevented if the non-thin delta is OFS_DELTA (by
>     just checking for the base in our already-found list of objects
>     before we call read_object_file(). But for REF_DELTA, I think we
>     have no way of knowing that appending is the wrong thing (and no
>     good way of backing it out afterwards).

Actually, I think even for REF_DELTA our pack-objects would never
produce such a pack, because IIRC we _always_ put bases in the pack
before their deltas. But that's a pretty subtle thing to depend on. I'm
fine with it if violating it just means things are slightly less
optimal. I'm more worried if it means that index-pack silently produces
a bogus pack.

I think to trigger it you'd have to manually assemble an evil pack as I
described (e.g., using the routines in t/lib-pack.sh). I'm going offline
for a bit, but I may have a go at it later tonight or tomorrow.

-Peff