[Possible Bug] sparse-checkout disable followed by re-init

[Possible Bug] sparse-checkout disable followed by re-init

[rsbecker]

Hi All,

I noticed a change at 2.35.1 regarding sparse-checkout and am wondering
whether I am just missing something:

1. Create upstream.
2. Clone with --no-checkout --depth=1 --filter=blob:none
3. sparse-checkout init
4. sparse-checkout set, with a pattern
5. sparse-checkout list
6. checkout master
7. sparse-checkout reapply
8. sparse-checkout add (up to here, everything is fine, the working area is
consistent with the sparsity patterns)
9. sparse-checkout disable (the disable works - I think)
10. sparse-checkout list reports fatal: this worktree is not sparse, which
is new in 2.35.1. In 2.34.1, the sparse patterns are reported without error.
I think this is incorrect behaviour because the sparse patterns are still
configured.
Then I try to recover
11. sparse-checkout init (seems to work - no errors)
12. sparse-checkout list (correctly reports what was previously set/add)
13. sparse-checkout reapply (does nothing apparent - the objects that added
to the working area during the disable are still there. My assumption was
that the files not matching the sparsity patterns would be pruned).

The .git/config is:
[core]
        repositoryformatversion = 1
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = file:///home/randall/stuff/src.git
        fetch = +refs/heads/master:refs/remotes/origin/master
        promisor = true
        partialclonefilter = blob:none
[branch "master"]
        remote = origin
        merge = refs/heads/master
[extensions]
        worktreeConfig = true

My question is, from this point, is there a way to recover the sparsity of
the working index (which I expect), or do I have to clone again (not
desirable). This seems like a functional change. What is weird to me is that
I did not expect a worktreeConfig=true in this situation.

Thanks,
Randall

Re: [Possible Bug] sparse-checkout disable followed by re-init

[Elijah Newren]

Hi Randall,

On Tue, 7 Feb 2022 04:57:26 -0500, Randall Becker wrote:

> Hi All,
> 
> I noticed a change at 2.35.1 regarding sparse-checkout and am wondering
> whether I am just missing something:
> 
> 1. Create upstream.
> 2. Clone with --no-checkout --depth=1 --filter=blob:none

A quick aside:

You may want to use --sparse instead of --no-checkout.  The fact that
there is no checkout and no index, has caused problems in combination
with sparse checkouts before.  In particular, a --no-checkout state is
hard to distinguish from the user intentionally deleted all files and
staging that deletion (i.e. a `git rm -rf .`).  Obviously,
sparse-checkout shouldn't be undeleting files when the user has staged
those deletions, so this is just a confusing state.

The --sparse flag exists for jumping straight into a sparse-checkout after
the clone.

That said, you suggested that things were fine before your first
disable, and my testing of your description seemed to suggest that
there wasn't a problem here.  I'd still recommend using --sparse over
--no-checkout when you want sparse checkouts, though.

> 3. sparse-checkout init
> 4. sparse-checkout set, with a pattern
> 5. sparse-checkout list
> 6. checkout master
> 7. sparse-checkout reapply
> 8. sparse-checkout add (up to here, everything is fine, the working area is
> consistent with the sparsity patterns)
> 9. sparse-checkout disable (the disable works - I think)

You can verify with `git ls-files -t`, and looking in the working directory.
Both to make sure that the appropriate files are marked with `H` or `S`, and
that the working directory matches that specification.

> 10. sparse-checkout list reports fatal: this worktree is not sparse, which
> is new in 2.35.1. In 2.34.1, the sparse patterns are reported without error.
> I think this is incorrect behaviour because the sparse patterns are still
> configured.
> Then I try to recover

Yeah, I'd agree that's incorrect behavior from 2.34.1; there are no
active sparsity patterns and we shouldn't mislead by listing them.
See 45c5e47048 ("sparse-checkout: add sanity-checks on initial
sparsity state", 2021-12-14) for where the fix was made.

> 11. sparse-checkout init (seems to work - no errors)
> 12. sparse-checkout list (correctly reports what was previously set/add)
> 13. sparse-checkout reapply (does nothing apparent - the objects that added
> to the working area during the disable are still there. My assumption was
> that the files not matching the sparsity patterns would be pruned).

I would presume that'd be done at the `init` step.  Indeed that's what I
observed creating a script that did:

    mkdir testing
    cd testing
    $GIT clone --no-checkout --depth=1 --filter=blob:none file:///home/newren/floss/git
    cd git
    $GIT sparse-checkout init
    $GIT sparse-checkout set Documentation
    $GIT sparse-checkout list
    $GIT checkout replay
    $GIT sparse-checkout reapply
    $GIT sparse-checkout add ci
    $GIT status
    $GIT sparse-checkout disable
    $GIT sparse-checkout list
    $GIT sparse-checkout init
    $GIT status
    $GIT sparse-checkout reapply
    $GIT status
    $GIT --version

And, in fact the `$GIT status` line at the end reported for me (with
either git version)

   You are in a sparse checkout with 21% of tracked files present.

And a `ls` showed only `Documentation` and `ci` directories.

The only differences in output for that script for me with git-2.34.1
and git-2.35.1 was the error message when calling `$GIT
sparse-checkout list` after the disable step, and the version number
printed at the end.

> The .git/config is:
> [core]
>         repositoryformatversion = 1
>         filemode = true
>         bare = false
>         logallrefupdates = true
> [remote "origin"]
>         url = file:///home/randall/stuff/src.git
>         fetch = +refs/heads/master:refs/remotes/origin/master
>         promisor = true
>         partialclonefilter = blob:none
> [branch "master"]
>         remote = origin
>         merge = refs/heads/master
> [extensions]
>         worktreeConfig = true
> 
> My question is, from this point, is there a way to recover the sparsity of
> the working index (which I expect), or do I have to clone again (not
> desirable).

I'm guessing from your wording that `git ls-files -t` will show a 'S' status
for the files you don't want to be present in the working tree, but they are
present in the working tree anyway?  I can't duplicate with your instructions
here; it seems like there's some other step involved.  However, there are
certainly a variety of ways to get into this problematic state.  And, sadly,
not a lot of good ways to discover that you have gotten into this state or
tools for getting out of it.

Fixing this class of problems was the purpose of:
  https://lore.kernel.org/git/pull.1114.v2.git.1642175983.gitgitgadget@gmail.com/
which is still waiting to merge down.

> This seems like a functional change.

The only functional change I could duplicate from your description was
the `git sparse-checkout list` behavior, which was an intentional
change.  I suspect there's some other command you ran or maybe an
ill-timed Ctrl-C that got you into the weird state; can you find a
consistent way to reproduce?

> What is weird to me is that
> I did not expect a worktreeConfig=true in this situation.

We've been setting that with sparse-checkout since the command was
introduced (see commit bab3c35908 ("sparse-checkout: create 'init'
subcommand", 2019-11-21)); and is documented in the sparse-checkout
manual.  That's not new at all.


Hope that helps,
Elijah

RE: [Possible Bug] sparse-checkout disable followed by re-init

[rsbecker]

> -----Original Message-----
> From: rsbecker@nexbridge.com <rsbecker@nexbridge.com>
> Sent: February 7, 2022 4:57 AM
> To: git@vger.kernel.org
> Subject: [Possible Bug] sparse-checkout disable followed by re-init
> 
> Hi All,
> 
> I noticed a change at 2.35.1 regarding sparse-checkout and am wondering
> whether I am just missing something:
> 
> 1. Create upstream.
> 2. Clone with --no-checkout --depth=1 --filter=blob:none 3.
sparse-checkout init
> 4. sparse-checkout set, with a pattern 5. sparse-checkout list 6. checkout
master
> 7. sparse-checkout reapply 8. sparse-checkout add (up to here, everything
is fine,
> the working area is consistent with the sparsity patterns) 9.
sparse-checkout
> disable (the disable works - I think) 10. sparse-checkout list reports
fatal: this
> worktree is not sparse, which is new in 2.35.1. In 2.34.1, the sparse
patterns are
> reported without error. I think this is incorrect behaviour because the
sparse
> patterns are still configured.
> Then I try to recover
> 11. sparse-checkout init (seems to work - no errors) 12. sparse-checkout
list
> (correctly reports what was previously set/add) 13. sparse-checkout
reapply
> (does nothing apparent - the objects that added to the working area during
the
> disable are still there. My assumption was that the files not matching the
sparsity
> patterns would be pruned).
> 
> The .git/config is:
> [core]
>         repositoryformatversion = 1
>         filemode = true
>         bare = false
>         logallrefupdates = true
> [remote "origin"]
>         url = file:///home/randall/stuff/src.git
>         fetch = +refs/heads/master:refs/remotes/origin/master
>         promisor = true
>         partialclonefilter = blob:none
> [branch "master"]
>         remote = origin
>         merge = refs/heads/master
> [extensions]
>         worktreeConfig = true
> 
> My question is, from this point, is there a way to recover the sparsity of
the
> working index (which I expect), or do I have to clone again (not
desirable). This
> seems like a functional change. What is weird to me is that I did not
expect a
> worktreeConfig=true in this situation.
> 
> Thanks,
> Randall

Apologies for the wrapping. Outlook. Argh.