Re: [RFC PATCH 13/13] bundle-uri docs: add design notes

From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: "brian m. carlson" <sandals@crustytoothpaste.net>
Cc: git@vger.kernel.org, Jeff King <peff@peff.net>,
	Patrick Steinhardt <ps@pks.im>,
	Christian Couder <christian.couder@gmail.com>,
	Albert Cui <albertqcui@gmail.com>,
	Jonathan Tan <jonathantanmy@google.com>
Subject: Re: [RFC PATCH 13/13] bundle-uri docs: add design notes
Date: Wed, 25 Aug 2021 00:33:04 +0200	[thread overview]
Message-ID: <87y28qcvrp.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <YSVpLjAhhS8v2dR7@camp.crustytoothpaste.net>

On Tue, Aug 24 2021, brian m. carlson wrote:

> [[PGP Signed Part:Undecided]]
> On 2021-08-05 at 15:07:29, Ævar Arnfjörð Bjarmason wrote:
>> Add a design doc for the bundle-uri protocol extension to go along
>> with the packfile-uri extension added in cd8402e0fd8 (Documentation:
>> add Packfile URIs design doc, 2020-06-10).
>> 
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>>  Documentation/technical/bundle-uri.txt  | 119 ++++++++++++++++++++++++
>>  Documentation/technical/protocol-v2.txt |   5 +
>>  2 files changed, 124 insertions(+)
>>  create mode 100644 Documentation/technical/bundle-uri.txt
>> 
>> diff --git a/Documentation/technical/bundle-uri.txt b/Documentation/technical/bundle-uri.txt
>> new file mode 100644
>> index 0000000000..5ae9a15eaf
>> --- /dev/null
>> +++ b/Documentation/technical/bundle-uri.txt
>> @@ -0,0 +1,119 @@
>> +Bundle URI Design Notes
>> +=======================
>> +
>> +Protocol
>> +--------
>> +
>> +See `bundle-uri` in the link:protocol-v2.html[protocol-v2]
>> +documentation for a discussion of the bundle-uri command, and the
>> +expectations of clients and servers.
>> +
>> +This document is a a more general discussion of how the `bundle-uri`
>> +command fits in with the rest of the git ecosystem, its design goals
>> +and non-goals, comparison to alternatives etc.
>> +
>> +Comparison with Packfile URIs
>> +-----------------------------
>> +
>> +There is a similar "Packfile URIs" facility, see the
>> +link:packfile-uri.html[packfile-uri] documentation for details.
>> +
>> +The Packfile URIs facility requires a much closer cooperation between
>> +CDN and server than the bundle URI facility.
>> +
>> +I.e. the server MUST know what objects exist in the packfile URI it's
>> +pointing to, as well as its pack checksum. Failure to do so will not
>> +only result in a client error (the packfile hash won't match), but
>> +even if it got past that would likely result in a corrupt repository
>> +with tips pointing to unreachable objects.
>> +
>> +By comparison the bundle URIs are meant to be a "dumb" solution
>> +friendly to e.g. having a weekly cronjob take a snapshot of a git
>> +repository, that snapshot being uploaded to a network of FTP mirrors
>> +(which may be inconsistent or out of date).
>> +
>> +The server does not need to know what state the side-channel download
>> +is at, because the client will first validate it, and then optionally
>> +negotiate with the server using what it discovers there.
>> +
>> +Using the local `transfer.injectBundleURI` configuration variable (see
>> +linkgit:git-config[1]) the `bundle-uri` mechanism doesn't even need
>> +the server to support it.
>
> One thing I'm not seeing with this doc that I brought up during the
> packfile URI discussion is that HTTPS is broken for a decent number of
> Git users, and for them SSH is the only viable option.  This is true for
> users of certain antivirus programs on Windows, as well as people who
> have certain corporate proxies in their workplace.  For those people, as
> soon as the server offers a bundle URI, their connection will stop
> working.
>
> I know that you're probably thinking, "Gee, how often does that happen?"
> but judging by the number of people on StackOverflow, this is actually
> very common.  The antivirus programs that break Git are actually not
> uncommon and they are widely deployed on corporate machines, plus the
> fact that lots of companies sell TLS intercepting proxies, which are
> almost always broken in this way.  Many of these users don't even know
> what's going on, so they simply lack the knowledge to take any action or
> ask their network administrator for a fix.  For them, HTTPS just doesn't
> work with Git, while it does for a web browser.
>
> So we will probably want to make this behavior opt-in with a config
> option for SSH, or just not available for SSH at all, so that we don't
> magically break users on upgrade who are relying on the SSH protocol not
> using HTTPS under the hood[0], especially the users who won't even know
> what's wrong.

Good point, I think this sort of thing will be a non-issue with
bundle-uri, because in general it handles any sort of network / fetching
/ validation failures gracefully. I.e. with these patches you can point
at a bad URI, broken non-bundle etc. We'll just move on to a full clone.

Whereas with packfile-uri the inline PACK and the URI are things you
MUST both get, as the provided packfile-uri completes the incomplete
inline PACK. So once you say that you're willing to accept things over
https, you MUST be able to get that thing.

We'll still waste a bit of time trying though with bundle-uri. But I
think for the common case of bundle-uri helping more than not (which
presumably, the server operator has tested), it's a better default to
try https:// even if the main dialog is over ssh://.