Re: Fetching too many tags?

[Jeff King <peff@peff.net>]

On Fri, Aug 11, 2023 at 10:06:43PM +0000, Ronan Pigott wrote:

> > When you use "--no-tags", that explicitly says "do not bother with tags
> > at all". Recent versions of Git have a protocol extension where the
> > client can say "I am only interested in refs/heads/master; don't bother
> > telling me about other stuff". Since the client knows we do not care
> > about tags, it can use that extension to get a much smaller ref
> > advertisement from the server.
> 
> Do you mean the --negotiation-tip fetch option? In my experience, it doesn't
> appear to have much of an effect in this case.

No, the "negotiation" phase only happens when there are objects to
fetch, and the client and server have to agree on which ones. That's not
happening at all in your case (so --negotiation-tip won't have any
effect).

The feature I was thinking of is that in Git's "v2" protocol, the client
gets to speak first, and so it can say "btw, I am only interested in
these refs". v2 became the default in git v2.29 (of course both client
and server have to support it, but kernel.org is definitely up to date
there).

You can see it in action with something like this:

  GIT_TRACE_PACKET=1 git fetch --no-tags origin master

The "ref-prefix" lines are the client telling the server which prefixes
it's interested in (we have to ask for several variants because "master"
from the command line gets fully qualified based on what the other side
offers). Try it without --no-tags and you'll see a wider ref-prefix
request. If you try:

  GIT_TRACE_PACKET=1 git -c protocol.version=0 fetch --no-tags origin master

you'll see the full advertisement, even with --no-tags. In v0, the
server speaks first and just dumps its complete list of refs.

> > By default, Git will report, to the server, commits reachable from all local
> > refs to find common commits in an attempt to reduce the size of the
> > to-be-received packfile. If specified, Git will only report commits reachable
> > from the given tips. This is useful to speed up fetches when the user knows
> > which local ref is likely to have commits in common with the upstream ref being
> > fetched.
> 
> Now, if I understand correctly, the report does not include the tags that we
> already have? 

So there's no negotiation here at all, as I explained above. But when it
does happen, Git should use all refs, including tags and branches, to
try to reach a common point in the history graph. If you run with
GIT_TRACE_PACKET on a request that actually fetches objects, you'll see
"have" and "want" lines from the client.

For a vanilla fetch from a server you regularly fetch from, the
negotiation is pretty boring and fast (the client tells the server about
the old commit at the tip of the branch, and the server immediately says
"OK, I know about that").

A more interesting one is if you fetch the kernel from Linus's repo, and
then fetch from the stable kernel repo after that. Or maybe vice versa.
There you have two histories that share significant chunks, but also
have each diverged. So you should see the client and server dumping
sha1's at each other until they reach a common point. That's a case
where --negotiation-tip can sometimes speed things up.

-Peff