What's cooking in git.git (topics)

What's cooking in git.git (topics)

[Junio C Hamano]

Being in the pre-release freeze, nothing should be cooking in
'next' nor 'pu', and you should not be reading this ;-)

In any case, on the 'master' front there is only one fix since
the last -rc.  Will tag and declare 1.5.1 this Wednesday.  And
after that happens, here is a list of what will come.

Commits prefixed with '-' are only in 'pu' while commits
prefixed with '+' are in 'next'.  The topics list the commits in
reverse chronological order.

=======================================
=== To merge immediately post 1.5.1 ===
=======================================

* lt/dirwalk (Fri Mar 30 20:39:30 2007 -0700) 1 commit
 + Optimize directory listing with pathspec limiter.

This is to help "git add single-path" in a huge directory that
does not have .gitignore.

* post1.5.1/tcltk (Fri Mar 30 00:59:43 2007 -0700) 5 commits
 + Optional Tck/Tk: ignore generated files.
 + Eliminate checks of user-specified Tcl/Tk interpreter.
 + Rewrite Tcl/Tk interpreter path for the GUI tools.
 + Add --with-tcltk and --without-tcltk to configure.
 + NO_TCLTK

* post1.5.1/p4 (Thu Mar 29 14:07:47 2007 +0400) 4 commits
 + Added correct Python path to the RPM specfile.
 + Remove unused WITH_OWN_SUBPROCESS_PY from RPM spec
 + Added git-p4 package to the list of git RPMs.
 + Add the WITH_P4IMPORT knob to the Makefile.

Build tweaks.

* post1.5.1/blame.el (Wed Mar 28 18:44:34 2007 +0200) 2 commits
 + git-blame.el: pick a set of random colors for each git-blame turn
 + git-blame.el: separate git-blame-mode to ease maintenance

Emacs.

* fl/doc (Mon Mar 26 23:45:23 2007 -0700) 3 commits
 + Documentation: unbreak user-manual.
 + Documentation: Add version information to man pages
 + Documentation: Replace @@GIT_VERSION@@ in documentation

Add autogenerated version stamp in documentation.

* jc/bisect (Fri Mar 23 17:54:03 2007 -0700) 6 commits
 + make the previous optimization work also on path-limited rev-list
   --bisect
 + rev-list --bisect: Fix "halfway" optimization.
 + t6004: add a bit more path optimization test.
 + git-rev-list --bisect: optimization
 + git-rev-list: add --bisect-vars option.
 + t6002: minor spelling fix.

Speeding up bisection between v2.6.18..v2.6.20 in the kernel
repository from 22 seconds down to 4 seconds.

===============================
=== Will cook a bit further ===
===============================

* jc/index-output (Sat Mar 31 23:27:41 2007 -0700) 2 commits
 - git-read-tree --index-output=<file>
 - _GIT_INDEX_OUTPUT: allow plumbing to output to an alternative
   index file.

This is to avoid an extra copy during index-jumping commit.
Will merge to 'next' post 1.5.1, and hopefully 'master' soon.

* fl/cvsserver (Sat Mar 31 15:57:47 2007 +0200) 6 commits
 + cvsserver: Use DBI->table_info instead of DBI->tables
 + cvsserver: Abort if connect to database fails
 + cvsserver: Make the database backend configurable
 + cvsserver: Allow to override the configuration per access method
 + cvsserver: Handle three part keys in git config correctly
 + cvsserver: Introduce new state variable 'method'

I do think there is no need to rush this without positive
reports by people who tested with backends other than SQLite.

* jc/checkout (Thu Mar 29 01:23:12 2007 -0700) 4 commits
 - Use BASE index extension in git-commit and git-merge.
 - update-index --{set,get}-base
 - Add BASE index extension.
 - checkout -d: explicitly detach HEAD even when switching to the tip
   of a branch

This lays the foundation to detect and warn cases where the tip
of the current branch is modified underneath you.  Will merge to
'next' post 1.5.1, but this has to wait for motivated people to 
the cases the series currently does not record and/or check the
base where it should, before graduating to 'master'.

* jc/read-tree-df (Thu Mar 15 23:25:22 2007 -0700) 6 commits
 - Fix switching to a branch with D/F when current branch has file D.
 - Fix twoway_merge that passed d/f conflict marker to
   merged_entry().
 - Fix read-tree --prefix=dir/.
 - unpack-trees: get rid of *indpos parameter.
 - unpack_trees.c: pass unpack_trees_options structure to
   keep_entry() as well.
 - add_cache_entry(): removal of file foo does not conflict with
   foo/bar

This series is almost re-done since I sent the initial patch.
The code is much nicer, and I think it is safer.

* jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 7 commits
 - Make read-cache.c "the_index" free.
 - Move index-related variables into a structure.
 - Rename add_file_to_index() to add_file_to_cache()
 - Rename static variable write_index to update_index in builtin-
   apply.c
 - Rename internal function "add_file_to_cache" in builtin-update-
   index.c
 - Propagate cache error internal to refresh_cache() via parameter.
 - Fix bogus error message from merge-recursive error path

This defines a structure that bundles active_cache, active_nr,
active_cache_tree and friends, defines a single instance of such
structure "the_index".  All cache access/manipulation functions
in read-cache.c are updated to take a pointer to this structure
to specify which index to operate on.  The traditional functions
are redefined as macros, e.g.

    #define add_cache_entry(ce,opt) add_index_entry(&the_index, (ce), (opt))

This fell out by accident while I was working on jc/read-tree-df
topic.  The largest problem there was that we lose sight of what
was originally in the index, after replacing a set of paths
(e.g. path/a path/b path/c) in the current index with a single
parent path (e.g. "path" becomes a blob) from the other tree.
To solve this, I initially planned to modify unpack_trees() to
read from the current index and build into a separate index, and
that is why I needed this conversion.

But it turns out that I can use a single index to solve that, so
this series is not needed.  But people into libification may
find it interesting.

==========================
=== not ready for next ===
==========================

* jc/blame (Tue Mar 27 01:58:01 2007 -0700) 3 commits
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up

I've been trying to optimize this on and off but haven't made
much progress.

* jc/pathattr (Thu Mar 1 01:20:21 2007 -0800) 5 commits
 - pathattr: allow piping to external program.
 - pathattr: read from git_config().
 - git-show: use pathattr to run "display"
 - pathattr: path based configuration of various attributes.
 + convert: add scaffolding for path based selection of conversion
   routines.
* js/fetch-progress (Sun Feb 25 13:13:17 2007 -0800) 1 commit
 + git-fetch: add --quiet
* jc/merge-subtree (Thu Feb 15 16:32:45 2007 -0800) 1 commit
 - A new merge stragety 'subtree'.
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

Stalled.

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

* cc/bisect (Thu Apr 5 05:33:53 2007 +0200) 2 commits
 + Documentation: bisect: "start" accepts one bad and many good
   commits
 + Bisect: teach "bisect start" to optionally use one bad and many
   good revs.

This allows "git bisect start <bad> <good1> <good2>... -- <paths>".

Being able to give more than one good commits upfront may not
happen very often, but when you are, this makes it unnecessary
to avoid an extra whole-tree checkout that happens when you give
a new good commit every time.

Being able to bisect without any good commit is outside of the
scope of this series, but I suspect that it probably is a simple
matter of making "git bisect next", after getting a <bad> commit
but before gettingt a <good> commit, not to barf but warn
(because it would pick an ancient commit and wastes one checkout
if done by mistake), ask confirmation if interactive, and drive
"git-rev-list --bisect".

* fp/make-j (Wed Apr 4 22:42:33 2007 +0200) 1 commit
 + Makefile: Add '+' to QUIET_SUBDIR0 to fix parallel make.

This makes our Makefile more dependent on GNU make, but I think
it already is.

* jc/index-output (Sat Mar 31 23:27:41 2007 -0700) 2 commits
 + git-read-tree --index-output=<file>
 + _GIT_INDEX_OUTPUT: allow plumbing to output to an alternative
   index file.

This is primarily to make index manipulation more efficient and
safer in the codepath to do "git commit <paths>".  Although I
tested the results, I do not know if it breaks things for other
people in real life projects, as I almost never do this "index
jumping" commit myself.  Testing and feedback is appreciated.

I think the above three series are ready for 'master'.

* fl/cvsserver (Sat Mar 31 15:57:47 2007 +0200) 6 commits
 + cvsserver: Use DBI->table_info instead of DBI->tables
 + cvsserver: Abort if connect to database fails
 + cvsserver: Make the database backend configurable
 + cvsserver: Allow to override the configuration per access method
 + cvsserver: Handle three part keys in git config correctly
 + cvsserver: Introduce new state variable 'method'

Although the code looked sane, I do not interact with git
repositories over CVS protocol in real life, so I haven't
personally tested it.  I cannot push this out to 'master'
without positive feedbacks.  Testing needed.

* jc/checkout (Thu Mar 29 01:23:12 2007 -0700) 4 commits
 + Use BASE index extension in git-commit and git-merge.
 + update-index --{set,get}-base
 + Add BASE index extension.
 + checkout: allow detaching to HEAD even when switching to the tip
   of a branch

I've rewritten the bottom commit not to require an explicit -d
option when detaching.  You can say "git checkout master^0"
instead to get a detached head that is at the tip of master.  I
may make that one commit graduate to 'master' earlier than
others.

This series is primarily to make it safer when somebody else
updates the tip of the branch you have currently checked out.
As I said in previous messages, I think the series covers basic
operations fine but there probably are gaps in the coverage.
Motivated volunteers are needed to fill them.

* jc/read-tree-df (Thu Mar 15 23:25:22 2007 -0700) 6 commits
 + Fix switching to a branch with D/F when current branch has file D.
 + Fix twoway_merge that passed d/f conflict marker to
   merged_entry().
 + Fix read-tree --prefix=dir/.
 + unpack-trees: get rid of *indpos parameter.
 + unpack_trees.c: pass unpack_trees_options structure to
   keep_entry() as well.
 + add_cache_entry(): removal of file foo does not conflict with
   foo/bar

I think this fixes the "my branch has foo/bar and I cannot
switch to another branch that has foo as a file" issue better
than the previous attempts.  Heavy testing and reporting is
needed until it graduates to 'master'.

* jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 2 commits
 - Make read-cache.c "the_index" free.
 - Move index-related variables into a structure.

This libifies the "cache" part of the system.  Parked in 'pu' as
there is no immediate need.

* jc/blame (Tue Mar 27 01:58:01 2007 -0700) 3 commits
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
* jc/pathattr (Thu Mar 1 01:20:21 2007 -0800) 5 commits
 - pathattr: allow piping to external program.
 - pathattr: read from git_config().
 - git-show: use pathattr to run "display"
 - pathattr: path based configuration of various attributes.
 + convert: add scaffolding for path based selection of conversion
   routines.
* js/fetch-progress (Sun Feb 25 13:13:17 2007 -0800) 1 commit
 + git-fetch: add --quiet
* jc/merge-subtree (Thu Feb 15 16:32:45 2007 -0800) 1 commit
 - A new merge stragety 'subtree'.
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

These are stalled.

The index is based on blah, however, the HEAD points at different commit.

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> * jc/checkout (Thu Mar 29 01:23:12 2007 -0700) 4 commits
>  + Use BASE index extension in git-commit and git-merge.
>  + update-index --{set,get}-base
>  + Add BASE index extension.
>  + checkout: allow detaching to HEAD even when switching to the tip
>    of a branch
>
> I've rewritten the bottom commit not to require an explicit -d
> option when detaching.  You can say "git checkout master^0"
> instead to get a detached head that is at the tip of master.  I
> may make that one commit graduate to 'master' earlier than
> others.
>
> This series is primarily to make it safer when somebody else
> updates the tip of the branch you have currently checked out.
> As I said in previous messages, I think the series covers basic
> operations fine but there probably are gaps in the coverage.
> Motivated volunteers are needed to fill them.

Regarding this series, if you find a command sequence that does
this:

 (1) A command that is BASE aware (e.g. "git commit") is used to
     point HEAD at a commit and records it in the index; and then,

 (2) A command that is not yet BASE aware moves the HEAD
     (perhaps creating a commit) but does not cause the
     previously recorded BASE in the index to be updated to
     point at the new HEAD (e.g. "git am");

your next invocation of a command that is BASE aware would barf,
saying something like:

  * The index is based on 'c053f05... My earlier commit made in (1).', however, the HEAD
    points at different commit '4fc2da4... The last of commits made with (2).'

When this happens, please first run "git-runstatus" [*1*] and
"git-diff HEAD" to see if this is a false positive.  If the
output shows the differences from the HEAD you expect to see and
you are sure your index is derived from the current HEAD, and
not the error message indicates (in the above example, c053f05
is such a wrong commit), that means you ran some command that
needs to be taught about the BASE.

A work-around until that command is fixed is to run this:

	$ git update-index --set-base `git-rev-parse HEAD`

This corrects the base commit recorded in the index to match
your current HEAD commit to allows you to keep going.

Motivated volunteers can help us further by doing two more
things.  One is obviously to report such a case and identify
such a BASE unaware command, and even better, make it BASE
aware and send in patches.

Another thing is to make a habit of running 'git update-index
--get-base' after doing any git operation to see if the index
correctly records the base commit.  

Currently, when git-read-tree is used to update the index, the
base information is discarded, and 'git update-index --get-base'
will return empty.  Commands that are already BASE aware will
not trigger when the index does not record any base.  This is to
prevent too many false positives while this safety feature is
still in development, but that means more false negatives.  We
should minimize this base clobbering so that we can use multiple
work trees tied to the same repository more safely.

I'll follow-up this message with four patch series to make "git am"
and "git reset" BASE aware.


[Footnote]

*1* We need to fix "git status" to be usable for this as well,
but currently it shares much of its code with "git commit" and
triggers the same false positive)

[PATCH 1/4] Move check_base() shell function to git-sh-setup

[Junio C Hamano]

The scripts that attempt to make a commit on top of the current
HEAD need to use the same logic in check_base() to make sure the
current index is based on the HEAD commit we are going to build
on top of.  Move this function from git-commit to git-sh-setup,
so that it is available to everybody else.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 git-commit.sh   |   16 ----------------
 git-sh-setup.sh |   16 ++++++++++++++++
 2 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/git-commit.sh b/git-commit.sh
index 42f1c93..6b4c784 100755
--- a/git-commit.sh
+++ b/git-commit.sh
@@ -25,22 +25,6 @@ refuse_partial () {
 	exit 1
 }
 
-check_base () {
-	quiet="$1"
-	if HEAD=`git rev-parse --verify HEAD 2>/dev/null` &&
-	   BASE=`git update-index --get-base` &&
-	   test -n "$BASE" &&
-	   test "$BASE" != "$HEAD"
-	then
-		test -z "$quiet" || exit 1
-		ours=`git show -s --pretty=oneline --abbrev-commit $BASE`
-		theirs=`git show -s --pretty=oneline --abbrev-commit $HEAD`
-		echo >&2 "* The index is based on '$ours', however, the HEAD"
-		echo >&2 "  points at different commit '$theirs'"
-		exit 1
-	fi
-}
-
 THIS_INDEX="$GIT_DIR/index"
 NEXT_INDEX="$GIT_DIR/next-index$$"
 rm -f "$NEXT_INDEX"
diff --git a/git-sh-setup.sh b/git-sh-setup.sh
index f24c7f2..0797acd 100755
--- a/git-sh-setup.sh
+++ b/git-sh-setup.sh
@@ -28,6 +28,22 @@ set_reflog_action() {
 	fi
 }
 
+check_base () {
+	quiet="$1"
+	if HEAD=`git rev-parse --verify HEAD 2>/dev/null` &&
+	   BASE=`git update-index --get-base` &&
+	   test -n "$BASE" &&
+	   test "$BASE" != "$HEAD"
+	then
+		test -z "$quiet" || exit 1
+		ours=`git show -s --pretty=oneline --abbrev-commit $BASE`
+		theirs=`git show -s --pretty=oneline --abbrev-commit $HEAD`
+		echo >&2 "* The index is based on '$ours', however, the HEAD"
+		echo >&2 "  points at different commit '$theirs'"
+		exit 1
+	fi
+}
+
 is_bare_repository () {
 	git-config --bool --get core.bare ||
 	case "$GIT_DIR" in
-- 
1.5.1.730.g0d43be

[PATCH 2/4] Use BASE index extension in git-am.

[Junio C Hamano]

This makes git-am to record the expected HEAD location after
finishing its operation in the index, so that subsequent BASE
check would notice when somebody else updated your branch head
while you are looking the other way.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 git-am.sh |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/git-am.sh b/git-am.sh
index e69ecbf..27912ce 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -13,6 +13,7 @@ git var GIT_COMMITTER_IDENT >/dev/null || exit
 
 stop_here () {
     echo "$1" >"$dotest/next"
+    git update-index --set-base $(git rev-parse --verify HEAD)
     exit 1
 }
 
@@ -157,6 +158,8 @@ do
 	esac
 done
 
+check_base || exit
+
 # If the dotest directory exists, but we have finished applying all the
 # patches in them, clear it out.
 if test -d "$dotest" &&
@@ -468,5 +471,6 @@ do
 
 	go_next
 done
+git update-index --set-base $(git rev-parse --verify HEAD)
 
 rm -fr "$dotest"
-- 
1.5.1.730.g0d43be

[PATCH 3/4] git-read-tree --set-base=<commit>

[Junio C Hamano]

This allows the caller of read-tree to record the base commit
object in the index, so that later operations that build a new
commit based on the contents of the index can verify that the
HEAD is still at the expected place, and notice cases where
somebody else updated the tip of the current branch while we are
looking the other way.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 Documentation/git-read-tree.txt |    6 +++++-
 builtin-read-tree.c             |   19 +++++++++++++++++++
 2 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index 0ff2890..ef731e2 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -8,7 +8,7 @@ git-read-tree - Reads tree information into the index
 
 SYNOPSIS
 --------
-'git-read-tree' (<tree-ish> | [[-m [--aggressive] | --reset | --prefix=<prefix>] [-u | -i]] [--exclude-per-directory=<gitignore>] <tree-ish1> [<tree-ish2> [<tree-ish3>]])
+'git-read-tree' (<tree-ish> | [[-m [--aggressive] | --reset | --prefix=<prefix>] [-u | -i]] [--exclude-per-directory=<gitignore>] [--set-base=<commit>] <tree-ish1> [<tree-ish2> [<tree-ish3>]])
 
 
 DESCRIPTION
@@ -86,6 +86,10 @@ OPTIONS
 	file (usually '.gitignore') and allows such an untracked
 	but explicitly ignored file to be overwritten.
 
+--set-base=<commit>::
+	This option records the commit object name in the index,
+	later to be retrieved with the `git-update-index --get-base`.
+
 <tree-ish#>::
 	The id of the tree object(s) to be read/merged.
 
diff --git a/builtin-read-tree.c b/builtin-read-tree.c
index 06c2912..d1c4489 100644
--- a/builtin-read-tree.c
+++ b/builtin-read-tree.c
@@ -93,6 +93,8 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 {
 	int i, newfd, stage = 0;
 	unsigned char sha1[20];
+	unsigned char newbase[20];
+	int newbase_set = 0;
 	struct unpack_trees_options opts;
 
 	memset(&opts, 0, sizeof(opts));
@@ -158,6 +160,15 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 			continue;
 		}
 
+		if (!prefixcmp(arg, "--set-base=")) {
+			if (get_sha1(arg+11, newbase) ||
+			    sha1_object_info(newbase, NULL) != OBJ_COMMIT)
+				die("Specified base is not a valid commit object name '%s'",
+				    arg);
+			newbase_set = 1;
+			continue;
+		}
+
 		if (!strcmp(arg, "--trivial")) {
 			opts.trivial_merges_only = 1;
 			continue;
@@ -261,12 +272,20 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 	 * "-m ent" or "--reset ent" form), we can obtain a fully
 	 * valid cache-tree because the index must match exactly
 	 * what came from the tree.
+	 *
+	 * Also when we read from a single commit with --reset, the
+	 * index will be used to build a commit on top of it.
 	 */
 	if (trees && trees->item && !opts.prefix && (!opts.merge || (stage == 2))) {
 		cache_tree_free(&active_cache_tree);
 		prime_cache_tree();
 	}
 
+	if (newbase_set) {
+		active_cache_base_valid = 1;
+		hashcpy(active_cache_base, newbase);
+	}
+
 	if (write_cache(newfd, active_cache, active_nr) ||
 	    close(newfd) || commit_lock_file(&lock_file))
 		die("unable to write new index file");
-- 
1.5.1.730.g0d43be

[PATCH 4/4] Teach git-reset to use index BASE extension.

[Junio C Hamano]

After resetting the HEAD to point at a different commit, the
user obviously intends to make the next commit a child of the
updated HEAD commit.  Record it in the index so that we can
detect the case where somebody else updates the tip of the
current branch while we are looking the other way.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 git-reset.sh |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/git-reset.sh b/git-reset.sh
index fee6d98..982c726 100755
--- a/git-reset.sh
+++ b/git-reset.sh
@@ -71,7 +71,7 @@ then
 		die "Cannot do a soft reset in the middle of a merge."
 	fi
 else
-	git-read-tree --reset $update "$rev" || exit
+	git-read-tree --reset $update --set-base="$rev" "$rev" || exit
 fi
 
 # Any resets update HEAD to the head being switched to.
@@ -93,10 +93,11 @@ case "$reset_type" in
 	}
 	;;
 --soft )
-	;; # Nothing else to do
+	git-update-index --set-base "$rev"
+	;;
 --mixed )
 	# Report what has not been updated.
-	git-update-index --refresh
+	git-update-index --set-base "$rev" --refresh
 	;;
 esac
 
-- 
1.5.1.730.g0d43be