Re: [RFC PATCH 00/13] Add bundle-uri: resumably clones, static "dumb" CDN etc.

From: Jonathan Nieder <jrnieder@gmail.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: git@vger.kernel.org, Jeff King <peff@peff.net>,
	Patrick Steinhardt <ps@pks.im>,
	Christian Couder <christian.couder@gmail.com>,
	Albert Cui <albertqcui@gmail.com>,
	Jonathan Tan <jonathantanmy@google.com>
Subject: Re: [RFC PATCH 00/13] Add bundle-uri: resumably clones, static "dumb" CDN etc.
Date: Fri, 6 Aug 2021 07:38:19 -0700	[thread overview]
Message-ID: <YQ1JW8hHmG8B/oE3@google.com> (raw)
In-Reply-To: <RFC-cover-00.13-0000000000-20210805T150534Z-avarab@gmail.com>

Hi,

Ævar Arnfjörð Bjarmason wrote:

> We're in the 2.33.0 rc cycle, and I'd hoped to have some more prep
> work for this integrated already, but for now here's something
> interesting I've been working on for early commentary/feedback.
>
> This adds the the ability to protocol v2 for servers to optimistically
> pre-seed supporting clients with one or more bundles via a new
> "bundle-uri" protocol extension.

My initial thought here is that even though this includes a comparison
to packfile URIs, I suspect you're underestimating them. :)

Would it be possible to do the same pre-seeding using the packfile
URIs protocol?  Nothing stops a server from sending more objects than
the client asked for.  Is the issue that you want the client to be
able to list "have"s based on that pack?  Can't the server obtain that
same information at the same time as it obtains the bundle URL?

The reason I ask is that this contains a number of differences
relative to packfile URIs, most noticeably the use of bundles instead
of packfiles as the format for the static content.  If we were
starting from scratch and chose this design _instead_ of packfile URIs
then that could make sense (though there are issues with the bundle
format that we can also go into), but in a world where people are also
using packfile URIs it makes for a kind of confusing UX.  Is a server
operator expected to put both kinds of files on CDN and double their
storage bill?  Is this meant as an alternative, a replacement, or
something that combines well together with the packfile URIs feature?
What does the intended end state look like?

Projects like chromium have been using packfile URIs in production for
about 11 months now and it's been working well.  Because of that, I'd
be interested in understanding its shortcomings and improving it in
place --- or in other words, I want _you_ to benefit from them instead
of having to create an alternative to them.  Alternatively, if the
packfile URIs protocol is fundamentally flawed, then I'd like us to
understand that early and act on it instead of creating a parallel
alternative and waiting for it to bitrot.

I'll try to find time to look more closely at the patches to
understand the use case in more detail, but it will take some time
since I'm currently focused on the -rc.

Thanks,
Jonathan