git checkout under 1.7.6 does not properly list untracked files and aborts

git checkout under 1.7.6 does not properly list untracked files and aborts

[Joshua Jensen]

This may be an msysGit 1.7.6 issue, as that is what I am using.  It also 
occurs in msysGit 1.7.5, but I am almost certain it did not happen in 
msysGit 1.7.2.

Given an untracked file in the working directory that has been added to 
an alternate branch, when switching to that alternate branch, 'git 
checkout' exits with an error code but does not print anything to the 
console.  It should print an untracked file error.

I have been trying to track this down in code.  The point where the 
error messages are printed, display_error_msgs, is never hit.

I don't have a debugger to trace this on Windows, so I've just been 
adding lots of debug output.  So far, I have not found the catalyst.

I am out on vacation now for the next week and a half.  However, I 
thought I would send this along to see if someone else can confirm the 
the problem.  I'll continue investigation when I'm back.

Thanks.

Josh

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Joshua Jensen]

----- Original Message -----
From: Joshua Jensen
Date: 9/9/2011 2:04 PM
> This may be an msysGit 1.7.6 issue, as that is what I am using.  It 
> also occurs in msysGit 1.7.5, but I am almost certain it did not 
> happen in msysGit 1.7.2.
>
> Given an untracked file in the working directory that has been added 
> to an alternate branch, when switching to that alternate branch, 'git 
> checkout' exits with an error code but does not print anything to the 
> console.  It should print an untracked file error.
>
> I have been trying to track this down in code.  The point where the 
> error messages are printed, display_error_msgs, is never hit.
Okay, so I've tracked this down, but I am unsure what the correct fix is.

In unpack-trees.c's unpack_trees() function, there are some lines that read:

     if (ce->ce_flags & CE_ADDED &&
         verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
             return -1;

If the 'return -1' is changed to 'goto return_failed', then a proper 
error message appears:

     error: The following untracked working tree files would be 
overwritten by checkout:
         one/file/listed/here.txt
     Please move or remove them before you can switch branches.
     Aborting

The thing is, there are multiple files that would be overwritten by 
checkout, and I believe an older version of Git showed me the entire 
list before aborting.

What would be the proper fix here?  What am I doing wrong?

Thanks.

Josh

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Joshua Jensen]

----- Original Message -----
From: Joshua Jensen
Date: 9/19/2011 1:44 PM
> ----- Original Message -----
> From: Joshua Jensen
> Date: 9/9/2011 2:04 PM
>> This may be an msysGit 1.7.6 issue, as that is what I am using.  It 
>> also occurs in msysGit 1.7.5, but I am almost certain it did not 
>> happen in msysGit 1.7.2.
>>
>> Given an untracked file in the working directory that has been added 
>> to an alternate branch, when switching to that alternate branch, 'git 
>> checkout' exits with an error code but does not print anything to the 
>> console.  It should print an untracked file error.
>>
>> I have been trying to track this down in code.  The point where the 
>> error messages are printed, display_error_msgs, is never hit.
> Okay, so I've tracked this down, but I am unsure what the correct fix is.
>
> In unpack-trees.c's unpack_trees() function, there are some lines that 
> read:
>
>     if (ce->ce_flags & CE_ADDED &&
>         verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
>             return -1;
>
> If the 'return -1' is changed to 'goto return_failed', then a proper 
> error message appears:
>
>     error: The following untracked working tree files would be 
> overwritten by checkout:
>         one/file/listed/here.txt
>     Please move or remove them before you can switch branches.
>     Aborting
>
> The thing is, there are multiple files that would be overwritten by 
> checkout, and I believe an older version of Git showed me the entire 
> list before aborting.
While probably not correct, the following patch collects all files that 
would be overwritten and logs them to the output:

---
  unpack-trees.c |   11 +++++++----
  1 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index cc616c3..e1f7263 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -993,7 +993,7 @@ static int verify_absent(struct cache_entry *, enum 
unpack_trees_error_types, st
   */
  int unpack_trees(unsigned len, struct tree_desc *t, struct 
unpack_trees_options *o)
  {
-    int i, ret;
+    int i, ret = 0;
      static struct cache_entry *dfc;
      struct exclude_list el;

@@ -1100,9 +1100,10 @@ int unpack_trees(unsigned len, struct tree_desc 
*t, struct unpack_trees_options
               * Do the real check now because we have had
               * correct CE_NEW_SKIP_WORKTREE
               */
-            if (ce->ce_flags & CE_ADDED &&
-                verify_absent(ce, 
ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
-                    return -1;
+            if (ce->ce_flags & CE_ADDED) {
+                int result = verify_absent(ce, 
ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o);
+                ret = min(ret, result);
+            }

              if (apply_sparse_checkout(ce, o)) {
                  ret = -1;
@@ -1112,6 +1113,8 @@ int unpack_trees(unsigned len, struct tree_desc 
*t, struct unpack_trees_options
                  empty_worktree = 0;

          }
+        if (ret == -1)
+            goto return_failed;
          if (o->result.cache_nr && empty_worktree) {
              /* dubious---why should this fail??? */
              ret = unpack_failed(o, "Sparse checkout leaves no entry on 
working directory");
-- 
1.7.6.msysgit.1

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Michael J Gruber]

Joshua Jensen venit, vidit, dixit 19.09.2011 21:44:
> ----- Original Message -----
> From: Joshua Jensen
> Date: 9/9/2011 2:04 PM
>> This may be an msysGit 1.7.6 issue, as that is what I am using.  It 
>> also occurs in msysGit 1.7.5, but I am almost certain it did not 
>> happen in msysGit 1.7.2.
>>
>> Given an untracked file in the working directory that has been added 
>> to an alternate branch, when switching to that alternate branch, 'git 
>> checkout' exits with an error code but does not print anything to the 
>> console.  It should print an untracked file error.
>>
>> I have been trying to track this down in code.  The point where the 
>> error messages are printed, display_error_msgs, is never hit.
> Okay, so I've tracked this down, but I am unsure what the correct fix is.
> 
> In unpack-trees.c's unpack_trees() function, there are some lines that read:
> 
>      if (ce->ce_flags & CE_ADDED &&
>          verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
>              return -1;
> 
> If the 'return -1' is changed to 'goto return_failed', then a proper 
> error message appears:
> 
>      error: The following untracked working tree files would be 
> overwritten by checkout:
>          one/file/listed/here.txt
>      Please move or remove them before you can switch branches.
>      Aborting
> 
> The thing is, there are multiple files that would be overwritten by 
> checkout, and I believe an older version of Git showed me the entire 
> list before aborting.
> 
> What would be the proper fix here?  What am I doing wrong?

Can you provide a simple test case, such as this one:

---%<---
#!/bin/sh

rm -Rf utest || exit 1
mkdir utest || exit 1
cd utest || exit 1
git init
echo tracked >a
git add a
git commit -m a a
git branch side
echo tracked >b
git add b
git commit -m b
cat b
git checkout side
cat b
echo untracked >b
cat b
git checkout master
cat b
---%<---

With 1.7.6 and current next this gives (Linux):

Initialized empty Git repository in /tmp/t/utest/.git/
[master (root-commit) b462c80] a
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 a
[master 22d8f2f] b
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 b
tracked
Switched to branch 'side'
cat: b: No such file or directory
untracked
error: The following untracked working tree files would be overwritten
by checkout:
        b
Please move or remove them before you can switch branches.
Aborting
untracked

Does this test reproduce your problem on msysgit?

Michael

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Joshua Jensen]

----- Original Message -----
From: Michael J Gruber
Date: 9/20/2011 9:10 AM
> Can you provide a simple test case, such as this one:
>
> ---%<---
> #!/bin/sh
>
> rm -Rf utest || exit 1
> mkdir utest || exit 1
> cd utest || exit 1
> git init
> echo tracked>a
> git add a
> git commit -m a a
> git branch side
> echo tracked>b
> git add b
> git commit -m b
> cat b
> git checkout side
> cat b
> echo untracked>b
> cat b
> git checkout master
> cat b
> ---%<---
>
> Does this test reproduce your problem on msysgit?
As listed, your script produces the same results on msysGit.

It appears the issue is related to sparse checkouts:

---%<---
#!/bin/sh

rm -Rf utest || exit 1
mkdir utest || exit 1
cd utest || exit 1
git init
git config core.sparseCheckout true
echo *>.git/info/sparse-checkout
echo tracked>a
git add a
git commit -m a a
git branch side
echo tracked>b
git add b
git commit -m b
cat b
git checkout side
cat b
echo untracked>b
cat b
git checkout master
cat b
---%<---

-Josh

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Michael J Gruber]

Joshua Jensen venit, vidit, dixit 20.09.2011 20:26:
> ----- Original Message -----
> From: Michael J Gruber
> Date: 9/20/2011 9:10 AM
>> Can you provide a simple test case, such as this one:
>>
>> ---%<---
>> #!/bin/sh
>>
>> rm -Rf utest || exit 1
>> mkdir utest || exit 1
>> cd utest || exit 1
>> git init
>> echo tracked>a
>> git add a
>> git commit -m a a
>> git branch side
>> echo tracked>b
>> git add b
>> git commit -m b
>> cat b
>> git checkout side
>> cat b
>> echo untracked>b
>> cat b
>> git checkout master
>> cat b
>> ---%<---
>>
>> Does this test reproduce your problem on msysgit?
> As listed, your script produces the same results on msysGit.
> 
> It appears the issue is related to sparse checkouts:
> 
> ---%<---
> #!/bin/sh
> 
> rm -Rf utest || exit 1
> mkdir utest || exit 1
> cd utest || exit 1
> git init
> git config core.sparseCheckout true
> echo *>.git/info/sparse-checkout
> echo tracked>a
> git add a
> git commit -m a a
> git branch side
> echo tracked>b
> git add b
> git commit -m b
> cat b
> git checkout side
> cat b
> echo untracked>b
> cat b
> git checkout master
> cat b
> ---%<---
> 
> -Josh

Interesting. So, it is related to sparse, it seems, and there is also a
mistake in the documentation (which contradicts itself, btw). With your
script, since all files are requested to be checked out ('*'), none
should have skip-worktree set, and the result should be the same as with
my version. But I get this:

Initialized empty Git repository in /tmp/t/utest/.git/
[master (root-commit) 5929c8b] a
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 a
[master 60ad69e] b
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 b
tracked
Switched to branch 'side'
cat: b: No such file or directory
untracked
untracked

That is probably the same output as for you. The final "git checkout
master" errored out without doing anything and without message. I'll
send a few cleanups now, and do some bisecting and thinking later;
though there are people whose sparse-knowledge is way less sparse than
mine ;)

Michael

[PATCH 0/3] read-tree cleanups

[Michael J Gruber]

These are a few cleanups I noticed while checking Joshua's sparse checkout
problem when swithcing branches with a dirty sparse tree.
They are independent of a possible fix.

Michael J Gruber (3):
  unpack-trees: print "Aborting" to stderr
  git-read-tree.txt: language and typography fixes
  git-read-tree.txt: correct sparse-checkout and skip-worktree
    description

 Documentation/git-read-tree.txt |   48 +++++++++++++++++++-------------------
 unpack-trees.c                  |    2 +-
 2 files changed, 25 insertions(+), 25 deletions(-)

-- 
1.7.7.rc0.469.g9eb94

[PATCH 1/3] unpack-trees: print "Aborting" to stderr

[Michael J Gruber]

display_error_msgs() prints all the errors to stderr already (if any),
followed by "Aborting" (if any) to stdout. Make the latter go to stderr
instead.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 unpack-trees.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index 670b464..237aed8 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -159,7 +159,7 @@ static void display_error_msgs(struct unpack_trees_options *o)
 		string_list_clear(rejects, 0);
 	}
 	if (something_displayed)
-		printf("Aborting\n");
+		fprintf(stderr, "Aborting\n");
 }
 
 /*
-- 
1.7.7.rc0.469.g9eb94

[PATCH 2/3] git-read-tree.txt: language and typography fixes

[Michael J Gruber]

Fix a few missing articles and such, and mark-up 'commands' and `files`
appropriately.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 Documentation/git-read-tree.txt |   40 +++++++++++++++++++-------------------
 1 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index c45d53c..0004f4b 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -47,7 +47,7 @@ OPTIONS
 
 -i::
 	Usually a merge requires the index file as well as the
-	files in the working tree are up to date with the
+	files in the working tree to be up to date with the
 	current head commit, in order not to lose local
 	changes.  This flag disables the check with the working
 	tree and is meant to be used when creating a merge of
@@ -71,21 +71,21 @@ OPTIONS
 --aggressive::
 	Usually a three-way merge by 'git read-tree' resolves
 	the merge for really trivial cases and leaves other
-	cases unresolved in the index, so that Porcelains can
+	cases unresolved in the index, so that porcelains can
 	implement different merge policies.  This flag makes the
-	command to resolve a few more cases internally:
+	command resolve a few more cases internally:
 +
 * when one side removes a path and the other side leaves the path
   unmodified.  The resolution is to remove that path.
 * when both sides remove a path.  The resolution is to remove that path.
-* when both sides adds a path identically.  The resolution
+* when both sides add a path identically.  The resolution
   is to add that path.
 
 --prefix=<prefix>/::
 	Keep the current index contents, and read the contents
-	of named tree-ish under directory at `<prefix>`.  The
+	of the named tree-ish under the directory at `<prefix>`. The
 	original index file cannot have anything at the path
-	`<prefix>` itself, and have nothing in `<prefix>/`
+	`<prefix>` itself, nor anything in the `<prefix>/`
 	directory.  Note that the `<prefix>/` value must end
 	with a slash.
 
@@ -379,15 +379,15 @@ have finished your work-in-progress), attempt the merge again.
 Sparse checkout
 ---------------
 
-"Sparse checkout" allows to sparsely populate working directory.
-It uses skip-worktree bit (see linkgit:git-update-index[1]) to tell
-Git whether a file on working directory is worth looking at.
+"Sparse checkout" allows populating the working directory sparsely.
+It uses the skip-worktree bit (see linkgit:git-update-index[1]) to tell
+Git whether a file in the working directory is worth looking at.
 
-"git read-tree" and other merge-based commands ("git merge", "git
-checkout"...) can help maintaining skip-worktree bitmap and working
+'git read-tree' and other merge-based commands ('git merge', 'git
+checkout'...) can help maintaining the skip-worktree bitmap and working
 directory update. `$GIT_DIR/info/sparse-checkout` is used to
-define the skip-worktree reference bitmap. When "git read-tree" needs
-to update working directory, it will reset skip-worktree bit in index
+define the skip-worktree reference bitmap. When 'git read-tree' needs
+to update the working directory, it resets the skip-worktree bit in the index
 based on this file, which uses the same syntax as .gitignore files.
 If an entry matches a pattern in this file, skip-worktree will be
 set on that entry. Otherwise, skip-worktree will be unset.
@@ -397,18 +397,18 @@ skip-worktree turns from unset to set, it will add the corresponding
 file back. If it turns from set to unset, that file will be removed.
 
 While `$GIT_DIR/info/sparse-checkout` is usually used to specify what
-files are in. You can also specify what files are _not_ in, using
-negate patterns. For example, to remove file "unwanted":
+files are in, you can also specify what files are _not_ in, using
+negate patterns. For example, to remove the file `unwanted`:
 
 ----------------
 *
 !unwanted
 ----------------
 
-Another tricky thing is fully repopulating working directory when you
+Another tricky thing is fully repopulating the working directory when you
 no longer want sparse checkout. You cannot just disable "sparse
-checkout" because skip-worktree are still in the index and you working
-directory is still sparsely populated. You should re-populate working
+checkout" because skip-worktree bits are still in the index and your working
+directory is still sparsely populated. You should re-populate the working
 directory with the `$GIT_DIR/info/sparse-checkout` file content as
 follows:
 
@@ -416,8 +416,8 @@ follows:
 *
 ----------------
 
-Then you can disable sparse checkout. Sparse checkout support in "git
-read-tree" and similar commands is disabled by default. You need to
+Then you can disable sparse checkout. Sparse checkout support in 'git
+read-tree' and similar commands is disabled by default. You need to
 turn `core.sparseCheckout` on in order to have sparse checkout
 support.
 
-- 
1.7.7.rc0.469.g9eb94

[PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

[Michael J Gruber]

The description of .git/info/sparse-checkout and
skip-worktree is exactly the opposite of what is true, which is:

If a file matches a pattern in sparse-checkout, then (it is to be
checked out and therefore) skip-worktree is unset for that file;
otherwise, it is set (so that it is not checked out).

Currently, the opposite is documented, and (consistently) read-tree's
behavior with respect to bit flips is descibed incorrectly.

Fix it.

In hindsight, it would have been much better to have a "sparse-ignore"
or "sparse-skip" file so that an empty file would mean a full checkout,
and the file logic would be analogous to that of .gitignore, excludes
and skip-worktree.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
 Documentation/git-read-tree.txt |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index 0004f4b..1bd0317 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -389,12 +389,12 @@ directory update. `$GIT_DIR/info/sparse-checkout` is used to
 define the skip-worktree reference bitmap. When 'git read-tree' needs
 to update the working directory, it resets the skip-worktree bit in the index
 based on this file, which uses the same syntax as .gitignore files.
-If an entry matches a pattern in this file, skip-worktree will be
-set on that entry. Otherwise, skip-worktree will be unset.
+If an entry matches a pattern in this file, skip-worktree will not be
+set on that entry. Otherwise, skip-worktree will be set.
 
 Then it compares the new skip-worktree value with the previous one. If
-skip-worktree turns from unset to set, it will add the corresponding
-file back. If it turns from set to unset, that file will be removed.
+skip-worktree turns from set to unset, it will add the corresponding
+file back. If it turns from unset to set, that file will be removed.
 
 While `$GIT_DIR/info/sparse-checkout` is usually used to specify what
 files are in, you can also specify what files are _not_ in, using
-- 
1.7.7.rc0.469.g9eb94

Re: [PATCH 0/3] read-tree cleanups

[Michael J Gruber]

Michael J Gruber venit, vidit, dixit 21.09.2011 09:48:
> These are a few cleanups I noticed while checking Joshua's sparse checkout
> problem when swithcing branches with a dirty sparse tree.
> They are independent of a possible fix.
> 
> Michael J Gruber (3):
>   unpack-trees: print "Aborting" to stderr
>   git-read-tree.txt: language and typography fixes
>   git-read-tree.txt: correct sparse-checkout and skip-worktree
>     description
> 
>  Documentation/git-read-tree.txt |   48 +++++++++++++++++++-------------------
>  unpack-trees.c                  |    2 +-
>  2 files changed, 25 insertions(+), 25 deletions(-)
> 

Uhm, I forgot about the dont-cc-jc-in-rc policy, sorry. This is not a
late-minute regression fix and can wait.

Michael

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Michael J Gruber]

Michael J Gruber venit, vidit, dixit 21.09.2011 09:47:

> Interesting. So, it is related to sparse, it seems, and there is also a
> mistake in the documentation (which contradicts itself, btw). With your
> script, since all files are requested to be checked out ('*'), none
> should have skip-worktree set, and the result should be the same as with
> my version. But I get this:
> 
> Initialized empty Git repository in /tmp/t/utest/.git/
> [master (root-commit) 5929c8b] a
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 a
> [master 60ad69e] b
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 b
> tracked
> Switched to branch 'side'
> cat: b: No such file or directory
> untracked
> untracked
> 
> That is probably the same output as for you. The final "git checkout
> master" errored out without doing anything and without message. I'll
> send a few cleanups now, and do some bisecting and thinking later;
> though there are people whose sparse-knowledge is way less sparse than
> mine ;)

So, I bisected it. The first bad commit is

9037026 (unpack-trees: fix sparse checkout's "unable to match
directories", 2010-11-27)

although the real culprit may be its predecessor

2431afb (unpack-trees: move all skip-worktree checks back to
unpack_trees(), 2010-11-27)

which does not compile:

    CC unpack-trees.o
unpack-trees.c: In function 'mark_new_skip_worktree':
unpack-trees.c:852:75: error: 'o' undeclared (first use in this function)
unpack-trees.c:852:75: note: each undeclared identifier is reported only
once for each function it appears in
make: *** [unpack-trees.o] Error 1

So, this problem was introduced after v1.7.3.2. Given the extent of
those two commits, I can't help further (but your proposed fix may be fine).

Duy, sorry for prodding you again. My bisect script was (based on
Joshua's, based on mine):

#!/bin/sh
GIT_EXEC_DIR=~/src/git
PATH=$GIT_EXEC_DIR:$PATH
export GIT_EXEC_DIR
d=/tmp/t/utest
rm -Rf $d || exit 1
mkdir $d || exit 1
cd $d || exit 1
git init
git config core.sparseCheckout true
echo *>.git/info/sparse-checkout
echo tracked>a
git add a
git commit -m a a
git branch side
echo tracked>b
git add b
git commit -m b
git checkout side
echo untracked>b
git checkout master 2>&1 | grep "would be"

Hope this helps.

Michael

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Nguyen Thai Ngoc Duy]

On Wed, Sep 21, 2011 at 10:28:59AM +0200, Michael J Gruber wrote:
> So, I bisected it. The first bad commit is
> 
> 9037026 (unpack-trees: fix sparse checkout's "unable to match
> directories", 2010-11-27)
> 
> although the real culprit may be its predecessor
> 
> 2431afb (unpack-trees: move all skip-worktree checks back to
> unpack_trees(), 2010-11-27)
> 
> which does not compile:
> 
>     CC unpack-trees.o
> unpack-trees.c: In function 'mark_new_skip_worktree':
> unpack-trees.c:852:75: error: 'o' undeclared (first use in this function)
> unpack-trees.c:852:75: note: each undeclared identifier is reported only
> once for each function it appears in
> make: *** [unpack-trees.o] Error 1

This may help

--8<--
diff --git a/unpack-trees.c b/unpack-trees.c
index a6518db..a239af7 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -245,13 +245,13 @@ static int check_updates(struct unpack_trees_options *o)
 static int verify_uptodate_sparse(struct cache_entry *ce, struct unpack_trees_options *o);
 static int verify_absent_sparse(struct cache_entry *ce, enum unpack_trees_error_types, struct unpack_trees_options *o);
 
-static int will_have_skip_worktree(const struct cache_entry *ce, struct unpack_trees_options *o)
+static int will_have_skip_worktree(const struct cache_entry *ce, struct exclude_list *el)
 {
 	const char *basename;
 
 	basename = strrchr(ce->name, '/');
 	basename = basename ? basename+1 : ce->name;
-	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, o->el) <= 0;
+	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, el) <= 0;
 }

 static int apply_sparse_checkout(struct cache_entry *ce, struct unpack_trees_options *o)
@@ -849,7 +849,7 @@ static void mark_new_skip_worktree(struct exclude_list *el,
 		if (select_flag && !(ce->ce_flags & select_flag))
 			continue;
 
-		if (!ce_stage(ce) && will_have_skip_worktree(ce, o))
+		if (!ce_stage(ce) && will_have_skip_worktree(ce, el))
 			ce->ce_flags |= skip_wt_flag;
 		else
 			ce->ce_flags &= ~skip_wt_flag;
--8<--

> Duy, sorry for prodding you again.

No problem (and sorry for breaking the build). I'll also have a look
at this problem.

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Michael J Gruber]

Nguyen Thai Ngoc Duy venit, vidit, dixit 21.09.2011 10:58:
> On Wed, Sep 21, 2011 at 10:28:59AM +0200, Michael J Gruber wrote:
>> So, I bisected it. The first bad commit is
>>
>> 9037026 (unpack-trees: fix sparse checkout's "unable to match
>> directories", 2010-11-27)
>>
>> although the real culprit may be its predecessor
>>
>> 2431afb (unpack-trees: move all skip-worktree checks back to
>> unpack_trees(), 2010-11-27)
>>
>> which does not compile:
>>
>>     CC unpack-trees.o
>> unpack-trees.c: In function 'mark_new_skip_worktree':
>> unpack-trees.c:852:75: error: 'o' undeclared (first use in this function)
>> unpack-trees.c:852:75: note: each undeclared identifier is reported only
>> once for each function it appears in
>> make: *** [unpack-trees.o] Error 1
> 
> This may help
> 
> --8<--
> diff --git a/unpack-trees.c b/unpack-trees.c
> index a6518db..a239af7 100644
> --- a/unpack-trees.c
> +++ b/unpack-trees.c
> @@ -245,13 +245,13 @@ static int check_updates(struct unpack_trees_options *o)
>  static int verify_uptodate_sparse(struct cache_entry *ce, struct unpack_trees_options *o);
>  static int verify_absent_sparse(struct cache_entry *ce, enum unpack_trees_error_types, struct unpack_trees_options *o);
>  
> -static int will_have_skip_worktree(const struct cache_entry *ce, struct unpack_trees_options *o)
> +static int will_have_skip_worktree(const struct cache_entry *ce, struct exclude_list *el)
>  {
>  	const char *basename;
>  
>  	basename = strrchr(ce->name, '/');
>  	basename = basename ? basename+1 : ce->name;
> -	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, o->el) <= 0;
> +	return excluded_from_list(ce->name, ce_namelen(ce), basename, NULL, el) <= 0;
>  }
> 
>  static int apply_sparse_checkout(struct cache_entry *ce, struct unpack_trees_options *o)
> @@ -849,7 +849,7 @@ static void mark_new_skip_worktree(struct exclude_list *el,
>  		if (select_flag && !(ce->ce_flags & select_flag))
>  			continue;
>  
> -		if (!ce_stage(ce) && will_have_skip_worktree(ce, o))
> +		if (!ce_stage(ce) && will_have_skip_worktree(ce, el))
>  			ce->ce_flags |= skip_wt_flag;
>  		else
>  			ce->ce_flags &= ~skip_wt_flag;
> --8<--
> 
>> Duy, sorry for prodding you again.
> 
> No problem (and sorry for breaking the build). I'll also have a look
> at this problem.

Thanks. I can confirm that with the above patch, the code compiles and
fails my test. So it's the earlier of the two commits which introduces this.

Michael

Re: [PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

[Nguyen Thai Ngoc Duy]

On Wed, Sep 21, 2011 at 5:48 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> The description of .git/info/sparse-checkout and
> skip-worktree is exactly the opposite of what is true, which is:
>
> If a file matches a pattern in sparse-checkout, then (it is to be
> checked out and therefore) skip-worktree is unset for that file;
> otherwise, it is set (so that it is not checked out).
>
> Currently, the opposite is documented, and (consistently) read-tree's
> behavior with respect to bit flips is descibed incorrectly.
>
> Fix it.

Ack.

> In hindsight, it would have been much better to have a "sparse-ignore"
> or "sparse-skip" file so that an empty file would mean a full checkout,
> and the file logic would be analogous to that of .gitignore, excludes
> and skip-worktree.

.gitignore works towards excluding files. No rule means no excluding.
sparse-checkout file works towards including files, no rule means no
inclusion.
-- 
Duy

Re: [PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

[Michael J Gruber]

Nguyen Thai Ngoc Duy venit, vidit, dixit 21.09.2011 11:26:
> On Wed, Sep 21, 2011 at 5:48 PM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> The description of .git/info/sparse-checkout and
>> skip-worktree is exactly the opposite of what is true, which is:
>>
>> If a file matches a pattern in sparse-checkout, then (it is to be
>> checked out and therefore) skip-worktree is unset for that file;
>> otherwise, it is set (so that it is not checked out).
>>
>> Currently, the opposite is documented, and (consistently) read-tree's
>> behavior with respect to bit flips is descibed incorrectly.
>>
>> Fix it.
> 
> Ack.
> 
>> In hindsight, it would have been much better to have a "sparse-ignore"
>> or "sparse-skip" file so that an empty file would mean a full checkout,
>> and the file logic would be analogous to that of .gitignore, excludes
>> and skip-worktree.
> 
> .gitignore works towards excluding files. No rule means no excluding.
> sparse-checkout file works towards including files, no rule means no
> inclusion.

Sure, but with a "sparse-skip" rather than "sparse-checkout", we would
not even need an additional config variable, and the skip-worktree
centered explanations would follow the same logic (no need for the
additional negation) as the ignore files and the new sparse-skip file.
Also, I'm not sure whether more sparse users say "I want only that
subdir." than "I don't want that subdir."

But it's there to stay, of course.

Cheers,
Michael

Re: [PATCH 3/3] git-read-tree.txt: correct sparse-checkout and skip-worktree description

[Nguyen Thai Ngoc Duy]

On Wed, Sep 21, 2011 at 7:33 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
>>> In hindsight, it would have been much better to have a "sparse-ignore"
>>> or "sparse-skip" file so that an empty file would mean a full checkout,
>>> and the file logic would be analogous to that of .gitignore, excludes
>>> and skip-worktree.
>>
>> .gitignore works towards excluding files. No rule means no excluding.
>> sparse-checkout file works towards including files, no rule means no
>> inclusion.
>
> Sure, but with a "sparse-skip" rather than "sparse-checkout", we would
> not even need an additional config variable, and the skip-worktree
> centered explanations would follow the same logic (no need for the
> additional negation) as the ignore files and the new sparse-skip file.

I'll kill that config variable some day when sparse checkout code has
no overhead over normal case ("include all").
-- 
Duy

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Nguyen Thai Ngoc Duy]

On Wed, Sep 21, 2011 at 11:16:15AM +0200, Michael J Gruber wrote:
> Thanks. I can confirm that with the above patch, the code compiles and
> fails my test. So it's the earlier of the two commits which introduces this.

I was afraid some logic had gone horribly wrong. Turns out I did not
catch up with unpack_trees() coding style and drop the error messages
so "git checkout" in this case becomes "git checkout -q".

This patch should fix it. Need another look before I submit a real
patch though.

diff --git a/unpack-trees.c b/unpack-trees.c
index cc616c3..79e9e88 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1102,12 +1102,10 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
 			 */
 			if (ce->ce_flags & CE_ADDED &&
 			    verify_absent(ce, ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN, o))
-					return -1;
+				goto return_failed;
 
-			if (apply_sparse_checkout(ce, o)) {
-				ret = -1;
-				goto done;
-			}
+			if (apply_sparse_checkout(ce, o))
+				goto return_failed;
 			if (!ce_skip_worktree(ce))
 				empty_worktree = 0;
 

Re: git checkout under 1.7.6 does not properly list untracked files and aborts

[Nguyen Thai Ngoc Duy]

On Wed, Sep 21, 2011 at 8:28 PM, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> I was afraid some logic had gone horribly wrong. Turns out I did not
> catch up with unpack_trees() coding style and drop the error messages
> so "git checkout" in this case becomes "git checkout -q".

Gaah.. if I did read Joshua's problem description, it would not need
to worry about the logic. He said it returned error code correctly.
Silly me.
-- 
Duy

Re: [PATCH 0/3] read-tree cleanups

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> Uhm, I forgot about the dont-cc-jc-in-rc policy, sorry. This is not a
> late-minute regression fix and can wait.

Thanks but that is not a policy; merely my preference.  It also is a bit
unnecessarily stronger than my actual preferences, which is

 (1) I do not want to see a patch sent "To:" me as the maintainer the
     first round of a new, undiscussed patch, as if it is ready for
     inclusion, at any time, not just during rc.

 (2) I do appreciate a patch sent "Cc:" me as a member of the development
     community patches for discussion, especially to areas in which I am
     an area expert, at any time, including during rc.

Re: [PATCH 1/3] unpack-trees: print "Aborting" to stderr

[Junio C Hamano]

Forgot to update a test or two that this breaks?

Re: [PATCH 1/3] unpack-trees: print "Aborting" to stderr

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Forgot to update a test or two that this breaks?

In the meantime I've squashed this in.

 t/t7607-merge-overwrite.sh     |    1 +
 t/t7609-merge-co-error-msgs.sh |    5 +++++
 2 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/t/t7607-merge-overwrite.sh b/t/t7607-merge-overwrite.sh
index 72a8731..aa74184 100755
--- a/t/t7607-merge-overwrite.sh
+++ b/t/t7607-merge-overwrite.sh
@@ -107,6 +107,7 @@ error: The following untracked working tree files would be overwritten by merge:
 	sub
 	sub2
 Please move or remove them before you can merge.
+Aborting
 EOF
 
 test_expect_success 'will not overwrite untracked file in leading path' '
diff --git a/t/t7609-merge-co-error-msgs.sh b/t/t7609-merge-co-error-msgs.sh
index c994836..0e4a682 100755
--- a/t/t7609-merge-co-error-msgs.sh
+++ b/t/t7609-merge-co-error-msgs.sh
@@ -32,6 +32,7 @@ error: The following untracked working tree files would be overwritten by merge:
 	three
 	two
 Please move or remove them before you can merge.
+Aborting
 EOF
 
 test_expect_success 'untracked files overwritten by merge (fast and non-fast forward)' '
@@ -56,6 +57,7 @@ Please, commit your changes or stash them before you can merge.
 error: The following untracked working tree files would be overwritten by merge:
 	five
 Please move or remove them before you can merge.
+Aborting
 EOF
 
 test_expect_success 'untracked files or local changes ovewritten by merge' '
@@ -71,6 +73,7 @@ error: Your local changes to the following files would be overwritten by checkou
 	rep/one
 	rep/two
 Please, commit your changes or stash them before you can switch branches.
+Aborting
 EOF
 
 test_expect_success 'cannot switch branches because of local changes' '
@@ -92,6 +95,7 @@ error: Your local changes to the following files would be overwritten by checkou
 	rep/one
 	rep/two
 Please, commit your changes or stash them before you can switch branches.
+Aborting
 EOF
 
 test_expect_success 'not uptodate file porcelain checkout error' '
@@ -105,6 +109,7 @@ error: Updating the following directories would lose untracked files in it:
 	rep
 	rep2
 
+Aborting
 EOF
 
 test_expect_success 'not_uptodate_dir porcelain checkout error' '

Re: [PATCH 1/3] unpack-trees: print "Aborting" to stderr

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 22.09.2011 00:40:
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> Forgot to update a test or two that this breaks?
> 
> In the meantime I've squashed this in.

Yikes. Sorry and thanks!

> 
>  t/t7607-merge-overwrite.sh     |    1 +
>  t/t7609-merge-co-error-msgs.sh |    5 +++++
>  2 files changed, 6 insertions(+), 0 deletions(-)
> 
> diff --git a/t/t7607-merge-overwrite.sh b/t/t7607-merge-overwrite.sh
> index 72a8731..aa74184 100755
> --- a/t/t7607-merge-overwrite.sh
> +++ b/t/t7607-merge-overwrite.sh
> @@ -107,6 +107,7 @@ error: The following untracked working tree files would be overwritten by merge:
>  	sub
>  	sub2
>  Please move or remove them before you can merge.
> +Aborting
>  EOF
>  
>  test_expect_success 'will not overwrite untracked file in leading path' '
> diff --git a/t/t7609-merge-co-error-msgs.sh b/t/t7609-merge-co-error-msgs.sh
> index c994836..0e4a682 100755
> --- a/t/t7609-merge-co-error-msgs.sh
> +++ b/t/t7609-merge-co-error-msgs.sh
> @@ -32,6 +32,7 @@ error: The following untracked working tree files would be overwritten by merge:
>  	three
>  	two
>  Please move or remove them before you can merge.
> +Aborting
>  EOF
>  
>  test_expect_success 'untracked files overwritten by merge (fast and non-fast forward)' '
> @@ -56,6 +57,7 @@ Please, commit your changes or stash them before you can merge.
>  error: The following untracked working tree files would be overwritten by merge:
>  	five
>  Please move or remove them before you can merge.
> +Aborting
>  EOF
>  
>  test_expect_success 'untracked files or local changes ovewritten by merge' '
> @@ -71,6 +73,7 @@ error: Your local changes to the following files would be overwritten by checkou
>  	rep/one
>  	rep/two
>  Please, commit your changes or stash them before you can switch branches.
> +Aborting
>  EOF
>  
>  test_expect_success 'cannot switch branches because of local changes' '
> @@ -92,6 +95,7 @@ error: Your local changes to the following files would be overwritten by checkou
>  	rep/one
>  	rep/two
>  Please, commit your changes or stash them before you can switch branches.
> +Aborting
>  EOF
>  
>  test_expect_success 'not uptodate file porcelain checkout error' '
> @@ -105,6 +109,7 @@ error: Updating the following directories would lose untracked files in it:
>  	rep
>  	rep2
>  
> +Aborting
>  EOF
>  
>  test_expect_success 'not_uptodate_dir porcelain checkout error' '