Re: git-grep in sparse checkout

[Elijah Newren <newren@gmail.com>]

On Tue, Oct 1, 2019 at 11:33 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > * other commands (archive, bisect, clean?, gitk, shortlog, blame,
> > fsck?, etc.) likely need to pay attention to sparsity patterns as
> > well, but there are some special cases:
>
> "git archive" falls into the same class as fast-(im|ex)port; it
> should ignore the sparse cone by default.  I suspect you threw
> "fsck" as a joke, but I do not think it should pay attention to the
> sparse cone, either (besides, most of the time in fsck the objects
> subject to checking do not know all the paths that reach them).

archive in the same category as fast-(im|ex)port makes sense.  I'm not
sure if "ignore the sparse cone" by default makes sense or if it
should be a case where we error out if --ignore-sparsity-patterns
isn't specified, especially if history is also sparse.

In terms of fsck, I agree that if history is dense and the worktree is
sparse that you want to walk all history.  I was thinking further
along the lines when partial clones and sparse checkouts are combined
so that history is also sparse.  In cases where a partial clone is in
use, rather than download everything in order to walk it, wouldn't it
make more sense to have fsck walk over the bits that are already
downloaded?  I don't really know how that'd all work, but it seems
that if fsck walked over all history it'd be treated as a
useless/dangerous command by those who are doing partial clones
because the repo is just too big.

> > * merge, cherry-pick, and rebase (anything touching the merge
> > machinery) will need to expand the size of the non-sparse worktree if
> > there are files outside the sparsity patterns with conflicts.  (Though
> > merge should do a better job of not expanding the non-sparse worktree
> > when files can cleanly be resolved.)
>
> I think the important point is what is done to the result of
> operation.  Result of these operations that create new commits are
> meant to be consumed by other people, who may not share your
> definition of sparse cone.  And such a command (i.e. those whose
> results are consumed by others who may have different sparse cone)
> must be full-tree by default.
>
> > * fast-export and format-patch are not about viewing history but about
> > exporting it, and limiting to sparsity patterns would result in the
> > creation of an incompatible history.
>
> I agree with the conclusion; see above.
>
> > * New worktrees, by default, should copy the sparsity-patterns of the
> > worktree they were created from (much like a new shell inherits the
> > current working directory of it's parent process)
>
> Sorry, but I do not share this view at all.
>
> In my mental model, "worktree new" is to attach a brand-new worktree
> to a bare repository that underlies the existing worktree I happen
> to be in, and that existing worktree that I happen to type "worktree
> new" in is no more or no less special than other worktrees.
>
> The above isn't to say that I'd veto your "a new worktree inherits
> traits from an existing worktree that 'git worktree add' was invoked
> in" idea.  I am just saying that I have a problem with that mode of
> operation and mental model being the default.

If worktrees are the only area we disagree on, then I'll happily take
the stuff we agree on and can overlook this piece.

But, perhaps some further explaining on worktrees might help us reach
some middle ground. If worktrees are dense by default and folks have
not only sparse checkouts but sparse history, then creating a new
worktree would suddenly mandate downloading a lot more of history --
which could be prohibitively expensive, forcing people to instead have
N clones without any shared history.  That may be fine (I tend to not
be a heavy worktree user, I just support some users who are), but is
it the route we want to push people with big repos towards?


Thanks for the feedback on the ideas,
Elijah