Re: [Possible Bug] sparse-checkout disable followed by re-init

[Elijah Newren <newren@gmail.com>]

Hi Randall,

On Tue, 7 Feb 2022 04:57:26 -0500, Randall Becker wrote:

> Hi All,
> 
> I noticed a change at 2.35.1 regarding sparse-checkout and am wondering
> whether I am just missing something:
> 
> 1. Create upstream.
> 2. Clone with --no-checkout --depth=1 --filter=blob:none

A quick aside:

You may want to use --sparse instead of --no-checkout.  The fact that
there is no checkout and no index, has caused problems in combination
with sparse checkouts before.  In particular, a --no-checkout state is
hard to distinguish from the user intentionally deleted all files and
staging that deletion (i.e. a `git rm -rf .`).  Obviously,
sparse-checkout shouldn't be undeleting files when the user has staged
those deletions, so this is just a confusing state.

The --sparse flag exists for jumping straight into a sparse-checkout after
the clone.

That said, you suggested that things were fine before your first
disable, and my testing of your description seemed to suggest that
there wasn't a problem here.  I'd still recommend using --sparse over
--no-checkout when you want sparse checkouts, though.

> 3. sparse-checkout init
> 4. sparse-checkout set, with a pattern
> 5. sparse-checkout list
> 6. checkout master
> 7. sparse-checkout reapply
> 8. sparse-checkout add (up to here, everything is fine, the working area is
> consistent with the sparsity patterns)
> 9. sparse-checkout disable (the disable works - I think)

You can verify with `git ls-files -t`, and looking in the working directory.
Both to make sure that the appropriate files are marked with `H` or `S`, and
that the working directory matches that specification.

> 10. sparse-checkout list reports fatal: this worktree is not sparse, which
> is new in 2.35.1. In 2.34.1, the sparse patterns are reported without error.
> I think this is incorrect behaviour because the sparse patterns are still
> configured.
> Then I try to recover

Yeah, I'd agree that's incorrect behavior from 2.34.1; there are no
active sparsity patterns and we shouldn't mislead by listing them.
See 45c5e47048 ("sparse-checkout: add sanity-checks on initial
sparsity state", 2021-12-14) for where the fix was made.

> 11. sparse-checkout init (seems to work - no errors)
> 12. sparse-checkout list (correctly reports what was previously set/add)
> 13. sparse-checkout reapply (does nothing apparent - the objects that added
> to the working area during the disable are still there. My assumption was
> that the files not matching the sparsity patterns would be pruned).

I would presume that'd be done at the `init` step.  Indeed that's what I
observed creating a script that did:

    mkdir testing
    cd testing
    $GIT clone --no-checkout --depth=1 --filter=blob:none file:///home/newren/floss/git
    cd git
    $GIT sparse-checkout init
    $GIT sparse-checkout set Documentation
    $GIT sparse-checkout list
    $GIT checkout replay
    $GIT sparse-checkout reapply
    $GIT sparse-checkout add ci
    $GIT status
    $GIT sparse-checkout disable
    $GIT sparse-checkout list
    $GIT sparse-checkout init
    $GIT status
    $GIT sparse-checkout reapply
    $GIT status
    $GIT --version

And, in fact the `$GIT status` line at the end reported for me (with
either git version)

   You are in a sparse checkout with 21% of tracked files present.

And a `ls` showed only `Documentation` and `ci` directories.

The only differences in output for that script for me with git-2.34.1
and git-2.35.1 was the error message when calling `$GIT
sparse-checkout list` after the disable step, and the version number
printed at the end.

> The .git/config is:
> [core]
>         repositoryformatversion = 1
>         filemode = true
>         bare = false
>         logallrefupdates = true
> [remote "origin"]
>         url = file:///home/randall/stuff/src.git
>         fetch = +refs/heads/master:refs/remotes/origin/master
>         promisor = true
>         partialclonefilter = blob:none
> [branch "master"]
>         remote = origin
>         merge = refs/heads/master
> [extensions]
>         worktreeConfig = true
> 
> My question is, from this point, is there a way to recover the sparsity of
> the working index (which I expect), or do I have to clone again (not
> desirable).

I'm guessing from your wording that `git ls-files -t` will show a 'S' status
for the files you don't want to be present in the working tree, but they are
present in the working tree anyway?  I can't duplicate with your instructions
here; it seems like there's some other step involved.  However, there are
certainly a variety of ways to get into this problematic state.  And, sadly,
not a lot of good ways to discover that you have gotten into this state or
tools for getting out of it.

Fixing this class of problems was the purpose of:
  https://lore.kernel.org/git/pull.1114.v2.git.1642175983.gitgitgadget@gmail.com/
which is still waiting to merge down.

> This seems like a functional change.

The only functional change I could duplicate from your description was
the `git sparse-checkout list` behavior, which was an intentional
change.  I suspect there's some other command you ran or maybe an
ill-timed Ctrl-C that got you into the weird state; can you find a
consistent way to reproduce?

> What is weird to me is that
> I did not expect a worktreeConfig=true in this situation.

We've been setting that with sparse-checkout since the command was
introduced (see commit bab3c35908 ("sparse-checkout: create 'init'
subcommand", 2019-11-21)); and is documented in the sparse-checkout
manual.  That's not new at all.


Hope that helps,
Elijah