What's cooking in git.git (topics)

What's cooking in git.git (topics)

[Junio C Hamano]

Things that I feel should be done need to be done to complete
v1.5.0 are:

 - 'git-rm' needs to be fixed up as Linus outlined; remove
   working tree file and index entry but have a sanity check to
   make sure the working tree file match the index and HEAD.

 - tutorials and other Porcelain documentation pages need to be
   updated to match the updated 'git-add' and 'git-rm' (to be
   updated), and their description should be made much less
   about implementation; they should talk in terms of end-user
   workflows.

   We need a full sweep on Porcelain-ish documentation.

 - now reflog is enabled by default for user repositories, I
   have two worries about its effect, fortunately can be killed
   with a single stone.

   * the reflog grows unbounded;

   * revisions recorded in the reflog can be pruned out,
     rendering some entries in reflog useless.

   I am thinking about teaching fsck-objects and prune to keep
   revisions recorded in the reflog; we would need an end-user
   way to prune older reflog entries and I would appreciate
   somebody codes it up, but even without it, people can always
   use "vi" or "ed" on reflog files ;-).

 - 'git-add' might want to do 'update-index --replace'; probably
   needs a sanity-checking discussion before implementing it.

 - 'git-svn' users should speak out about two issues:

   * use of svn command line client as the backend is being
     removed;

   * 'git-svn commit' command is being renamed to avoid
     confusion, and potentially 'dcommit' will be renamed to 
     'commit'.

   Please discuss these with Eric.

 - we might want to address the issue that 'git-status' output
   does not make it easy to tell mode changes from content
   changes.  Personally I do not like cryptic "M+" output format
   proposed by Lars and find the current output more readable.

----------------------------------------------------------------
As usual, '+' are both in 'next' and 'pu', '-' are in 'pu' only.

*  jn/web (Sat Dec 16 17:12:55 2006 +0100) 9 commits
 - gitweb: Add some mod_perl specific support
 + gitweb: Add "next" link to commit view
 + gitweb: Add title attribute to ref marker with full ref name
 + gitweb: Do not show difftree for merges in "commit" view
 + gitweb: SHA-1 in commit log message links to "object" view
 + gitweb: Hyperlink target of symbolic link in "tree" view (if
   possible)
 + gitweb: Add generic git_object subroutine to display object of any
   type
 + gitweb: Show target of symbolic link in "tree" view
 + gitweb: Don't use Content-Encoding: header in git_snapshot

All except the tip (mod_perl) looked good and should be in
v1.5.0; I haven't formed an opinion on mod_perl change yet.

*  js/branch-config (Sat Dec 16 15:15:02 2006 +0100) 2 commits
 + git-branch: rename config vars branch.<branch>.*, too
 + add a function to rename sections in the config

This moves branch.$foo.* variables to branch.$bar.* when $foo
branch is renamed to $bar.  Because we already have branch -m
(stands for "mv"), this series is a must-have in v1.5.0.

*  jc/clone (Sat Dec 16 01:53:10 2006 -0800) 4 commits
 + git-clone: lose the traditional 'no-separate-remote' layout
 + git-clone: lose the artificial "first" fetch refspec
 + git-pull: refuse default merge without branch.*.merge
 + git-clone: use wildcard specification for tracking branches

Fixes the workflow wart and removes the traditional layout that
maps remote 'master' to refs/heads/origin.  Also 'git pull' and
'git pull origin' would not merge the ref on the first Pull:
line (nor remote.*.fetch item) anymore.  You either have to give
an explicit command line parameter or branch.$currbranch.merge
item.  With blessing from Linus, this will be in v1.5.0.

*  ew/svn-pm (Fri Dec 15 23:58:08 2006 -0800) 3 commits
 + git-svn: rename 'commit' command to 'set-tree'
 + git-svn: remove support for the svn command-line client
 + git-svn: convert to using Git.pm

It is in 'next' because I agree it is in the right direction in
the longer term, but not in 'master' because this might be
controvertial for shorter term.  The users should decide.

** jc/reflog (Thu Dec 14 15:58:56 2006 -0800) 1 commit
 - Teach show-branch how to show ref-log data.

A strawman to make reflog data a bit more browsable; it would be
useful while recovering from a mistake you made recently.  Not
essential and can wait or be dropped if people do not find it
useful.

** jc/git-add--interactive (Mon Dec 11 17:09:26 2006 -0800) 2 commits
 - git-add --interactive: hunk splitting
 - git-add --interactive

I've thought about further allowing to edit the patches in the
'patch' subcommand, but the more I think about it, the less it
makes sense from the workflow point of view.  Will be the topic
for a separate message.

** sv/git-svn (Tue Dec 5 16:17:38 2006 +1100) 5 commits
 . git-svn: re-map repository URLs and UUIDs on SVK mirror paths
 . git-svn: collect revision properties when fetching
 . git-svn: collect SVK source URL on mirror paths
 . git-svn: let libsvn_ls_fullurl return properties too
 . git-svn: make test for SVK mirror path import

Still held but dropped from 'pu' for now (depends on "sub sys"
that was removed from git-svn).

** jc/explain (Mon Dec 4 19:35:04 2006 -0800) 1 commit
 - git-explain

Backburnered.

*  jc/blame-boundary (Fri Dec 1 20:45:45 2006 -0800) 1 commit
 + git-blame: show lines attributed to boundary commits differently.

Will merge.

*  jc/3way (Wed Nov 29 18:53:13 2006 -0800) 1 commit
 + git-merge: preserve and merge local changes when doing fast
   forward

Will merge, if only to see what breaks.

*  js/shallow (Fri Nov 24 16:00:13 2006 +0100) 15 commits
 + fetch-pack: Do not fetch tags for shallow clones.
 + get_shallow_commits: Avoid memory leak if a commit has been
   reached already.
 + git-fetch: Reset shallow_depth before auto-following tags.
 + upload-pack: Check for NOT_SHALLOW flag before sending a shallow
   to the client.
 + fetch-pack: Properly remove the shallow file when it becomes
   empty.
 + shallow clone: unparse and reparse an unshallowed commit
 + Why didn't we mark want_obj as ~UNINTERESTING in the old code?
 + Why does it mean we do not have to register shallow if we have
   one?
 + We should make sure that the protocol is still extensible.
 + add tests for shallow stuff
 + Shallow clone: do not ignore shallowness when following tags
 + allow deepening of a shallow repository
 + allow cloning a repository "shallowly"
 + support fetching into a shallow repository
 + upload-pack: no longer call rev-list

Undecided but not likely to be in v1.5.0.  Needs more real
project testing.

** jc/web (Wed Nov 8 14:54:09 2006 -0800) 1 commit
 - gitweb: steal loadavg throttle from kernel.org

Undecided.

** jc/pickaxe (Sun Nov 5 11:52:43 2006 -0800) 1 commit
 - blame: --show-stats for easier optimization work.

Developer only.

** jc/leftright (Sun Oct 22 17:32:47 2006 -0700) 1 commit
 - rev-list --left-right

Backburnered.

** jc/diff (Mon Sep 25 23:03:34 2006 -0700) 1 commit
 - para-walk: walk n trees, index and working tree in parallel

Backburnered.

*  jc/diff-apply-patch (Fri Sep 22 16:17:58 2006 -0700) 1 commit
 + git-diff/git-apply: make diff output a bit friendlier to GNU patch
   (part 2)

Not in v1.5.0.

Re: What's cooking in git.git (topics)

[Jakub Narebski]

Junio C Hamano wrote:

> Things that I feel should be done need to be done to complete
> v1.5.0 are:
[...]
>  - now reflog is enabled by default for user repositories, I
>    have two worries about its effect, fortunately can be killed
>    with a single stone.
> 
>    * the reflog grows unbounded;
> 
>    * revisions recorded in the reflog can be pruned out,
>      rendering some entries in reflog useless.
> 
>    I am thinking about teaching fsck-objects and prune to keep
>    revisions recorded in the reflog; we would need an end-user
>    way to prune older reflog entries and I would appreciate
>    somebody codes it up, but even without it, people can always
>    use "vi" or "ed" on reflog files ;-).

I'd rather not have prune keep revisions recorded in reflog. Reflog
keeps also amended commits, and blobs from incrementally staged
commits. Or perhaps make it an configuration option, default to
true for new users (or have an option to git-prune to ignore reflog).

As of "reflog grows unbounded"... perhaps we should encourage to use
logrotate for that (well, perhaps git-prune and porcelains which deal
with reflog should be able to uncompress reflog if needed).

>  - 'git-svn' users should speak out about two issues:
> 
>    * use of svn command line client as the backend is being
>      removed;
> 
>    * 'git-svn commit' command is being renamed to avoid
>      confusion, and potentially 'dcommit' will be renamed to 
>      'commit'.
> 
>    Please discuss these with Eric.

What about remote.<repo>.url = svn://ser.ver/repo/ idea?
 
> ** jc/reflog (Thu Dec 14 15:58:56 2006 -0800) 1 commit
>  - Teach show-branch how to show ref-log data.
> 
> A strawman to make reflog data a bit more browsable; it would be
> useful while recovering from a mistake you made recently.  Not
> essential and can wait or be dropped if people do not find it
> useful.

Looks useful.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: What's cooking in git.git (topics)

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> Junio C Hamano wrote:
>
>> Things that I feel should be done need to be done to complete
>> v1.5.0 are:
> [...]
>>  - now reflog is enabled by default for user repositories, I
>>    have two worries about its effect, fortunately can be killed
>>    with a single stone.
>> 
>>    * the reflog grows unbounded;
>> 
>>    * revisions recorded in the reflog can be pruned out,
>>      rendering some entries in reflog useless.
>> 
>>    I am thinking about teaching fsck-objects and prune to keep
>>    revisions recorded in the reflog; we would need an end-user
>>    way to prune older reflog entries and I would appreciate
>>    somebody codes it up, but even without it, people can always
>>    use "vi" or "ed" on reflog files ;-).
>
> I'd rather not have prune keep revisions recorded in reflog. Reflog
> keeps also amended commits, and blobs from incrementally staged
> commits.

That is exactly why I would want to protect them from pruning;
the current alternative, git-lost-found, is usable but not so
nice from the point of view of end users .  The user should be
able to use show-branch --reflog, or use a different mode of
operation in the log family [*1*] to inspect the otherwise lost
commits, and as you prune unneeded reflog entries you could
prune those objects.

You could certainly arrange things the other way around.  As a
part of prune, you can prune the reflog entries that refer to
commits that were pruned away and no longer available.

HOWEVER, that is very unfriendly to the end users, because they
do not have much control in what 'prune' removes.  If you want
to keep a dozen or so recent states just in case you may have to
salvage rewound states (e.g. you may realize your amended commit
was faulty), how would you tell 'prune' to keep only those
objects from getting pruned?

That's right -- by holding references to them.  And that is
exactly how making prune to take reflog into account would help
the user.  If the user has unwanted commits (say, everything
that are more than 7 days old, plus this and that commits you
rewound an hour ago), they can be pruned away from reflog and
prune will take care of the rest.

So doing it in the way I described makes a lot more sense than
doing the other way around.

[Footnote]

*1* I have to warn you that this would require quite different
code to walk the commits, but certainly a lot simpler than the
ancestry traversal.  If you are interested in learning the
internals, this may be a good project to start with.

Re: What's cooking in git.git (topics)

[Brian Gernhardt]

> ** jc/reflog (Thu Dec 14 15:58:56 2006 -0800) 1 commit
>  - Teach show-branch how to show ref-log data.
>
> A strawman to make reflog data a bit more browsable; it would be
> useful while recovering from a mistake you made recently.  Not
> essential and can wait or be dropped if people do not find it
> useful.

I'd prefer not to add clutter into show-branch.  I use it on a  
regular basis to see what I've added to what topic branch recently,  
and to look at branches before rebasing.  It also just seems like the  
wrong place to have that kind of data, although I guess it's more  
useful for people who do merges more often than I do.

What about a "git reflog [<branch>]" command instead?  Would show  
output similar to "git log" (or "git show-branch" for brevity).

Re: What's cooking in git.git (topics)

[Shawn Pearce]

Brian Gernhardt <benji@silverinsanity.com> wrote:
> >** jc/reflog (Thu Dec 14 15:58:56 2006 -0800) 1 commit
> > - Teach show-branch how to show ref-log data.
> 
> What about a "git reflog [<branch>]" command instead?  Would show  
> output similar to "git log" (or "git show-branch" for brevity).

It has been proposed on the list.  I even sketched out what such
a command would look like, how it should present the data, etc.
But I never wrote it.  Nobody else has taken up the task either.  :-)

-- 

[PATCH] revision: introduce ref@{N..M} syntax.

[Junio C Hamano]

Brian Gernhardt <benji@silverinsanity.com> writes:

>> ** jc/reflog (Thu Dec 14 15:58:56 2006 -0800) 1 commit
>>  - Teach show-branch how to show ref-log data.
>>
>> A strawman to make reflog data a bit more browsable; it would be
>> useful while recovering from a mistake you made recently.  Not
>> essential and can wait or be dropped if people do not find it
>> useful.
>
> I'd prefer not to add clutter into show-branch.  I use it on a regular
> basis to see what I've added to what topic branch recently,  and to
> look at branches before rebasing.  It also just seems like the  wrong
> place to have that kind of data, although I guess it's more  useful
> for people who do merges more often than I do.

I happen to find it a reasonable way to present how a topic was
rewind and rebuilt, but I think that is I rewind my internal
topic (i.e. the parts that have not been merged into 'next' yet)
all the time.

> What about a "git reflog [<branch>]" command instead?  Would show
> output similar to "git log" (or "git show-branch" for brevity).

This is not a 'git reflog' command, but is another way to show
the tips of reflog entries (use it with 'git show').  I find the
show-branch variant easier to see, but you can judge for
yourself.

This happens to show why pruning while not culling reflog
entries is a bad idea.  If your master@{2} is gone due to rewind
and prune, and master@{0}, master@{1} and master@{3} are still
available, the pattern master@{0..3} would not give you a usable
result.

-- >8 --
This allows you to add between Nth and Mth (inclusive) reflog entries.
"git show master@{1} master@{2} master@{3}" is equivalent to
"git show master@{1..3}".

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 revision.c |   59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 59 insertions(+), 0 deletions(-)

diff --git a/revision.c b/revision.c
index 993bb66..3f1d65e 100644
--- a/revision.c
+++ b/revision.c
@@ -592,6 +592,59 @@ static void prepare_show_merge(struct rev_info *revs)
 	revs->prune_data = prune;
 }
 
+static int parse_digits(const char *p, int len)
+{
+	int accum = 0;
+	while (0 < len--) {
+		int ch = *p++;
+		if ('0' <= ch && ch <= '9')
+			accum = accum * 10 + ch - '0';
+		else
+			return -1;
+	}
+	return accum;
+}
+
+static int list_of_reflog_entries(const char *arg, const char *dotdot,
+				  struct rev_info *revs, int flags)
+{
+	/*
+	 * "ref@{N..M}" -- push them for "git show".
+	 * dotdot points at the first dot
+	 */
+	int n, m, i, added;
+	int len;
+	char *at_brace = strstr(arg, "@{");
+
+
+	if (!at_brace)
+		return 0;
+	len = strlen(arg);
+	if (arg[len-1] != '}')
+		return 0;
+	if ((m = parse_digits(dotdot + 2, len - (dotdot - arg) - 3)) < 0)
+		return 0;
+	if ((n = parse_digits(at_brace + 2, dotdot - at_brace - 2)) < 0)
+		return 0;
+	if (n > m)
+		return 0;
+
+	added = 0;
+	for (i = n; i <= m; i++) {
+		char buf[1024];
+		unsigned char sha1[20];
+
+		snprintf(buf, sizeof(buf), "%.*s@{%d}",
+			 (int) (at_brace - arg), arg, i);
+		if (!get_sha1(buf, sha1)) {
+			/* this is synthetic -- do not check filename */
+			handle_revision_arg(buf, revs, flags, 1);
+			added = 1;
+		}
+	}
+	return added;
+}
+
 int handle_revision_arg(const char *arg, struct rev_info *revs,
 			int flags,
 			int cant_be_filename)
@@ -648,7 +701,13 @@ int handle_revision_arg(const char *arg, struct rev_info *revs,
 			add_pending_object(revs, &b->object, next);
 			return 0;
 		}
+
 		*dotdot = '.';
+
+		/* It could be ref@{N..M} */
+		if (list_of_reflog_entries(arg, dotdot, revs, flags))
+			return 0;
+
 	}
 	dotdot = strstr(arg, "^@");
 	if (dotdot && !dotdot[2]) {
-- 
1.4.4.2.g83c5

Re: What's cooking in git.git (topics)

[Yann Dirson]

On Sun, Dec 17, 2006 at 12:29:25AM +0100, Jakub Narebski wrote:
> >    I am thinking about teaching fsck-objects and prune to keep
> >    revisions recorded in the reflog; we would need an end-user
> >    way to prune older reflog entries and I would appreciate
> >    somebody codes it up, but even without it, people can always
> >    use "vi" or "ed" on reflog files ;-).
> 
> I'd rather not have prune keep revisions recorded in reflog. Reflog
> keeps also amended commits, and blobs from incrementally staged
> commits. Or perhaps make it an configuration option, default to
> true for new users (or have an option to git-prune to ignore reflog).

I think that is quite near to other issues: we already have other pieces
of information that we would like to sometimes have ignored and
sometimes not, when running fsck-objects/prune.  Namely, revisions
hidden by grafts, as already discussed on this list.

An idea I had to handle that case, and which could be useful with the
current problem, as well as others, like dealing with stgit's patch
logging, would be to define "reachable commits" using a modular
architecture.  That way we would be able to select what we want
fsck-object/prune to consider reachable, in objects reachable from:

- raw "parents" field of commit objects
- the latter as modified by info/grafts
- reflogs
- stgit patchlogs

The set of rules to consider could be declared in repo-config, thus
stgit would be able to declare that its patchlogs should not be ignored,
and people wishing to prune commits hidden by grafts in one repo could
just remove the "raw-parents" rule from their repo's config.

Obviously, mentionning stgit-specific rules here immediately suggests a
plugin-based architecture.

Does that make any sense ?

-- 

Re: [PATCH] revision: introduce ref@{N..M} syntax.

[Linus Torvalds]

On Sat, 16 Dec 2006, Junio C Hamano wrote:
>
> This allows you to add between Nth and Mth (inclusive) reflog entries.
> "git show master@{1} master@{2} master@{3}" is equivalent to
> "git show master@{1..3}".

Well, logically, if you do that, then you should also allow

	git log master@{one.week.ago..yesterday}

as a reflog expression.

"Because It Only Makes Sense(tm)".

		Linus

PS. Yeah, I'm only half serious. I like our revision parsing, and the 
above _would_ actually be consistent with the "master@{1..3}" kind of 
specification, but at the same time, it's also obviously more complex, and 
maybe it's not THAT usable.

But I think the "master@{date..date}" syntax would actually fall out 
automatically if you did the {x..y} parsing at a higher level and didn't 

Re: [PATCH] revision: introduce ref@{N..M} syntax.

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> Well, logically, if you do that, then you should also allow
>
> 	git log master@{one.week.ago..yesterday}
>
> as a reflog expression.
> ...
> PS. Yeah, I'm only half serious. I like our revision parsing, and the 
> above _would_ actually be consistent with the "master@{1..3}" kind of 
> specification, but at the same time, it's also obviously more complex, and 
> maybe it's not THAT usable.
>
> But I think the "master@{date..date}" syntax would actually fall out 
> automatically if you did the {x..y} parsing at a higher level and didn't 
> force "x" and "y" to be digits only.

Syntax, yes, usage of it in "git show" yes,

But giving it to "git log" would not work as a naive user would
expect, which your example suggested ;-).

Re: What's cooking in git.git (topics)

[Josef Weidendorfer]

On Sunday 17 December 2006 18:35, Yann Dirson wrote:
> An idea I had to handle that case, and which could be useful with the
> current problem, as well as others, like dealing with stgit's patch
> logging, would be to define "reachable commits" using a modular
> architecture.  That way we would be able to select what we want
> fsck-object/prune to consider reachable, in objects reachable from:
> 
> - raw "parents" field of commit objects
> - the latter as modified by info/grafts
> - reflogs
> - stgit patchlogs

Could be interesting for pruning support in submodules, too.

> The set of rules to consider could be declared in repo-config, thus
> stgit would be able to declare that its patchlogs should not be ignored,
> and people wishing to prune commits hidden by grafts in one repo could
> just remove the "raw-parents" rule from their repo's config.
> 
> Obviously, mentionning stgit-specific rules here immediately suggests a
> plugin-based architecture.

Isn't it enough to specify further subdirectories of .git/, holding
references which should be taken into account while pruning?
Hmm.. Grafting would be a special case.

Re: What's cooking in git.git (topics)

[Andy Parkins]

On Saturday 2006, December 16 23:10, Junio C Hamano wrote:

>    * revisions recorded in the reflog can be pruned out,
>      rendering some entries in reflog useless.

Can I suggest that it should be fine to prune reflog entries but that the act 
of pruning be a log entry itself?

Andy
-- 
Dr Andrew Parkins, M Eng (Hons), AMIEE

Re: What's cooking in git.git (topics)

[Junio C Hamano]

Andy Parkins <andyparkins@gmail.com> writes:

> On Saturday 2006, December 16 23:10, Junio C Hamano wrote:
>
>>    * revisions recorded in the reflog can be pruned out,
>>      rendering some entries in reflog useless.
>
> Can I suggest that it should be fine to prune reflog entries but that the act 
> of pruning be a log entry itself?

I do not understand.  What would that "pruning event" log entry
would say?

By definition each reflog entry says "it was pointing at this
object before, and it was changed by this user to point at that
object at this time and the reason for the change was this".

I personally do not think recording "at this point these things
were pruned" makes _any_ sense whatsoever --- if you care about
the pruned objects that simply means you pruned them before you
are ready to lose them.  But even if for whatever reason you
choose to log that anyway, I have a feeling that that record
does not belong to the reflog itself.

Re: What's cooking in git.git (topics)

[Andy Parkins]

On Monday 2006 December 18 08:09, Junio C Hamano wrote:

> By definition each reflog entry says "it was pointing at this
> object before, and it was changed by this user to point at that
> object at this time and the reason for the change was this".

I'm daft.  I've realised, pruning doesn't remove the ref, it removes one of 
the hashes in the reflog.  I withdraw my comment.

I'd imagined it' as the opposite of a creation.

000000 abcdef  branch created
abcdef 000000  branch deleted

Which is fine, except that isn't what prune is doing at all.

> I personally do not think recording "at this point these things
> were pruned" makes _any_ sense whatsoever --- if you care about

I think you're right. :-)


Andy

-- 
Dr Andy Parkins, M Eng (hons), MIEE

Re: What's cooking in git.git (topics)

[Shawn Pearce]

Andy Parkins <andyparkins@gmail.com> wrote:
> I'm daft.  I've realised, pruning doesn't remove the ref, it removes one of 
> the hashes in the reflog.  I withdraw my comment.
> 
> I'd imagined it' as the opposite of a creation.
> 
> 000000 abcdef  branch created
> abcdef 000000  branch deleted

Except that branch deletion also deletes the reflog.  So yes, we
could log it as you show above, but right after we appended it to
the log we'd delete the log anyway.  :-)

The immediate log deletion is necessary to support something like:

	git branch foo
	git branch -d foo
	git branch foo/bar

as foo switches from being a file to a directory, which means that
.git/logs/refs/heads/foo also needs to switch from being a file to
being a directory.


The only way to fix the above situation is to make the reflog a
single log for the entire repository, rather than one log per ref.
This may cause locking problems for us as we need to lock not only
the ref but also the repository-wide reflog lock.  Note that right
now the reflog is also protected by the ref lock itself, killing
two birds with one stone.  :-)

-- 

[PATCH 1/2] add for_each_reflog_ent() iterator

[Junio C Hamano]

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 refs.c |   25 +++++++++++++++++++++++++
 refs.h |    4 ++++
 2 files changed, 29 insertions(+), 0 deletions(-)

diff --git a/refs.c b/refs.c
index d911b9e..db8fdd4 100644
--- a/refs.c
+++ b/refs.c
@@ -1093,3 +1093,28 @@ int read_ref_at(const char *ref, unsigned long at_time, int cnt, unsigned char *
 		logfile, show_rfc2822_date(date, tz));
 	return 0;
 }
+
+void for_each_reflog_ent(const char *ref, each_reflog_ent_fn fn, void *cb_data)
+{
+	const char *logfile;
+	FILE *logfp;
+	char buf[1024];
+
+	logfile = git_path("logs/%s", ref);
+	logfp = fopen(logfile, "r");
+	if (!logfp)
+		return;
+	while (fgets(buf, sizeof(buf), logfp)) {
+		unsigned char osha1[20], nsha1[20];
+		int len;
+
+		/* old SP new SP name <email> SP time TAB msg LF */
+		len = strlen(buf);
+		if (len < 83 || buf[len-1] != '\n' ||
+		    get_sha1_hex(buf, osha1) || buf[40] != ' ' ||
+		    get_sha1_hex(buf + 41, nsha1) || buf[81] != ' ')
+			continue; /* corrupt? */
+		fn(osha1, nsha1, buf+82, cb_data);
+	}
+	fclose(logfp);
+}
diff --git a/refs.h b/refs.h
index 51aab1e..de43cc7 100644
--- a/refs.h
+++ b/refs.h
@@ -44,6 +44,10 @@ extern int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1, cons
 /** Reads log for the value of ref during at_time. **/
 extern int read_ref_at(const char *ref, unsigned long at_time, int cnt, unsigned char *sha1);
 
+/* iterate over reflog entries */
+typedef int each_reflog_ent_fn(unsigned char *osha1, unsigned char *nsha1, char *, void *);
+void for_each_reflog_ent(const char *ref, each_reflog_ent_fn fn, void *cb_data);
+
 /** Returns 0 if target has the right format for a ref. **/
 extern int check_ref_format(const char *target);
 
-- 
1.4.4.2.gc30f

[PATCH 2/2] Protect commits recorded in reflog from pruning.

[Junio C Hamano]

This teaches fsck-objects and prune to protect objects referred
to by reflog entries.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---

 >>  - now reflog is enabled by default for user repositories, I
 >>    have two worries about its effect, fortunately can be killed
 >>    with a single stone.
 >> 
 >>    * the reflog grows unbounded;
 >> 
 >>    * revisions recorded in the reflog can be pruned out,
 >>      rendering some entries in reflog useless.
 >> 
 >>    I am thinking about teaching fsck-objects and prune to keep
 >>    revisions recorded in the reflog; we would need an end-user
 >>    way to prune older reflog entries and I would appreciate
 >>    somebody codes it up, but even without it, people can always
 >>    use "vi" or "ed" on reflog files ;-).

 Rewinding would not lose any objects from the object database
 with this; somebody needs to code up a garbage collector and/or
 expiration mechanism for reflogs.

 builtin-prune.c |   16 ++++++++++++++++
 fsck-objects.c  |   22 ++++++++++++++++++++++
 2 files changed, 38 insertions(+), 0 deletions(-)

diff --git a/builtin-prune.c b/builtin-prune.c
index 8591d28..00a53b3 100644
--- a/builtin-prune.c
+++ b/builtin-prune.c
@@ -181,12 +181,28 @@ static void walk_commit_list(struct rev_info *revs)
 	}
 }
 
+static int add_one_reflog_ent(unsigned char *osha1, unsigned char *nsha1, char *datail, void *cb_data)
+{
+	struct object *object;
+
+	object = parse_object(osha1);
+	if (object)
+		add_pending_object(&revs, object, "");
+	object = parse_object(nsha1);
+	if (object)
+		add_pending_object(&revs, object, "");
+	return 0;
+}
+
 static int add_one_ref(const char *path, const unsigned char *sha1, int flag, void *cb_data)
 {
 	struct object *object = parse_object(sha1);
 	if (!object)
 		die("bad object ref: %s:%s", path, sha1_to_hex(sha1));
 	add_pending_object(&revs, object, "");
+
+	for_each_reflog_ent(path, add_one_reflog_ent, NULL);
+
 	return 0;
 }
 
diff --git a/fsck-objects.c b/fsck-objects.c
index 46b628c..e3746e8 100644
--- a/fsck-objects.c
+++ b/fsck-objects.c
@@ -402,6 +402,25 @@ static void fsck_dir(int i, char *path)
 
 static int default_refs;
 
+static int fsck_handle_reflog_ent(unsigned char *osha1, unsigned char *nsha1, char *datail, void *cb_data)
+{
+	struct object *obj;
+
+	if (!is_null_sha1(osha1)) {
+		obj = lookup_object(osha1);
+		if (obj) {
+			obj->used = 1;
+			mark_reachable(obj, REACHABLE);
+		}
+	}
+	obj = lookup_object(nsha1);
+	if (obj) {
+		obj->used = 1;
+		mark_reachable(obj, REACHABLE);
+	}
+	return 0;
+}
+
 static int fsck_handle_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)
 {
 	struct object *obj;
@@ -419,6 +438,9 @@ static int fsck_handle_ref(const char *refname, const unsigned char *sha1, int f
 	default_refs++;
 	obj->used = 1;
 	mark_reachable(obj, REACHABLE);
+
+	for_each_reflog_ent(refname, fsck_handle_reflog_ent, NULL);
+
 	return 0;
 }
 
-- 
1.4.4.2.gc30f

Re: [PATCH 2/2] Protect commits recorded in reflog from pruning.

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> This teaches fsck-objects and prune to protect objects referred
> to by reflog entries.

Nice!

But its not enough.

  $ git-repack -a -d
  $ git reset --hard HEAD^
  $ git-repack -a -d
  $ git reset --hard HEAD@{1}

that last reset would fail now, wouldn't it?  pack-objects needs
to know it should be pulling in the objects stuff reachable from
reflogs too.

-- 

Re: [PATCH 2/2] Protect commits recorded in reflog from pruning.

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> Junio C Hamano <junkio@cox.net> wrote:
>> This teaches fsck-objects and prune to protect objects referred
>> to by reflog entries.
>
> Nice!
>
> But its not enough.
>
>   $ git-repack -a -d
>   $ git reset --hard HEAD^
>   $ git-repack -a -d
>   $ git reset --hard HEAD@{1}
>
> that last reset would fail now, wouldn't it?  pack-objects needs
> to know it should be pulling in the objects stuff reachable from
> reflogs too.

Sure.

-- >8 --
Teach git-repack to preserve objects referred to by reflog entries.

This adds a new option --reflog to pack-objects and revision
machinery; do not bother documenting it for now, since this is
only useful for local repacking.

When the option is passed, objects reachable from reflog entries
are marked as interesting while computing the set of objects to
pack.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---
 builtin-pack-objects.c |    3 +-
 git-repack.sh          |    2 +-
 revision.c             |   56 ++++++++++++++++++++++++++++++++++++++++++------
 3 files changed, 52 insertions(+), 9 deletions(-)

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index a2dc7d1..928684b 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -19,7 +19,7 @@ static const char pack_usage[] = "\
 git-pack-objects [{ -q | --progress | --all-progress }] \n\
 	[--local] [--incremental] [--window=N] [--depth=N] \n\
 	[--no-reuse-delta] [--delta-base-offset] [--non-empty] \n\
-	[--revs [--unpacked | --all]*] [--stdout | base-name] \n\
+	[--revs [--unpacked | --all]*] [--reflog] [--stdout | base-name] \n\
 	[<ref-list | <object-list]";
 
 struct object_entry {
@@ -1577,6 +1577,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		}
 		if (!strcmp("--unpacked", arg) ||
 		    !strncmp("--unpacked=", arg, 11) ||
+		    !strcmp("--reflog", arg) ||
 		    !strcmp("--all", arg)) {
 			use_internal_rev_list = 1;
 			if (ARRAY_SIZE(rp_av) - 1 <= rp_ac)
diff --git a/git-repack.sh b/git-repack.sh
index 067898f..375434b 100755
--- a/git-repack.sh
+++ b/git-repack.sh
@@ -62,7 +62,7 @@ case ",$all_into_one," in
 esac
 
 args="$args $local $quiet $no_reuse_delta$extra"
-name=$(git-pack-objects --non-empty --all $args </dev/null "$PACKTMP") ||
+name=$(git-pack-objects --non-empty --all --reflog $args </dev/null "$PACKTMP") ||
 	exit 1
 if [ -z "$name" ]; then
 	echo Nothing new to pack.
diff --git a/revision.c b/revision.c
index 993bb66..cbf1045 100644
--- a/revision.c
+++ b/revision.c
@@ -462,21 +462,59 @@ static void limit_list(struct rev_info *revs)
 	revs->commits = newlist;
 }
 
-static int all_flags;
-static struct rev_info *all_revs;
+struct all_refs_cb {
+	int all_flags;
+	struct rev_info *all_revs;
+	const char *name_for_errormsg;
+};
 
 static int handle_one_ref(const char *path, const unsigned char *sha1, int flag, void *cb_data)
 {
-	struct object *object = get_reference(all_revs, path, sha1, all_flags);
-	add_pending_object(all_revs, object, "");
+	struct all_refs_cb *cb = cb_data;
+	struct object *object = get_reference(cb->all_revs, path, sha1,
+					      cb->all_flags);
+	add_pending_object(cb->all_revs, object, "");
 	return 0;
 }
 
 static void handle_all(struct rev_info *revs, unsigned flags)
 {
-	all_revs = revs;
-	all_flags = flags;
-	for_each_ref(handle_one_ref, NULL);
+	struct all_refs_cb cb;
+	cb.all_revs = revs;
+	cb.all_flags = flags;
+	for_each_ref(handle_one_ref, &cb);
+}
+
+static int handle_one_reflog_ent(unsigned char *osha1, unsigned char *nsha1, char *detail, void *cb_data)
+{
+	struct all_refs_cb *cb = cb_data;
+	struct object *object;
+
+	if (!is_null_sha1(osha1)) {
+		object = get_reference(cb->all_revs, cb->name_for_errormsg,
+				       osha1, cb->all_flags);
+		add_pending_object(cb->all_revs, object, "");
+	}
+	object = get_reference(cb->all_revs, cb->name_for_errormsg,
+			       nsha1, cb->all_flags);
+	add_pending_object(cb->all_revs, object, "");
+	return 0;
+}
+
+static int handle_one_reflog(const char *path, const unsigned char *sha1, int flag, void *cb_data)
+{
+	struct all_refs_cb *cb = cb_data;
+	cb->name_for_errormsg = path;
+	for_each_reflog_ent(path, handle_one_reflog_ent, cb_data);
+	return 0;
+}
+
+static void handle_reflog(struct rev_info *revs, unsigned flags)
+{
+	struct all_refs_cb cb;
+	cb.all_revs = revs;
+	cb.all_flags = flags;
+	for_each_ref(handle_one_reflog, &cb);
 }
 
 static int add_parents_only(struct rev_info *revs, const char *arg, int flags)
@@ -803,6 +841,10 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, const ch
 				handle_all(revs, flags);
 				continue;
 			}
+			if (!strcmp(arg, "--reflog")) {
+				handle_reflog(revs, flags);
+				continue;
+			}
 			if (!strcmp(arg, "--not")) {
 				flags ^= UNINTERESTING;
 				continue;

[PATCH 1/2] Move in_merge_bases() to commit.c

[Junio C Hamano]

This reasonably useful function was hidden inside builtin-branch.c
---

 * This is used by the next one, which is why this is part of
   the 'reflog entry and pruning' series.

 builtin-branch.c |   21 +--------------------
 commit.c         |   17 +++++++++++++++++
 commit.h         |    1 +
 3 files changed, 19 insertions(+), 20 deletions(-)

diff --git a/builtin-branch.c b/builtin-branch.c
index 560309c..12eebc0 100644
--- a/builtin-branch.c
+++ b/builtin-branch.c
@@ -70,25 +70,6 @@ const char *branch_get_color(enum color_branch ix)
 	return "";
 }
 
-static int in_merge_bases(const unsigned char *sha1,
-			  struct commit *rev1,
-			  struct commit *rev2)
-{
-	struct commit_list *bases, *b;
-	int ret = 0;
-
-	bases = get_merge_bases(rev1, rev2, 1);
-	for (b = bases; b; b = b->next) {
-		if (!hashcmp(sha1, b->item->object.sha1)) {
-			ret = 1;
-			break;
-		}
-	}
-
-	free_commit_list(bases);
-	return ret;
-}
-
 static void delete_branches(int argc, const char **argv, int force)
 {
 	struct commit *rev, *head_rev = head_rev;
@@ -119,7 +100,7 @@ static void delete_branches(int argc, const char **argv, int force)
 		 */
 
 		if (!force &&
-		    !in_merge_bases(sha1, rev, head_rev)) {
+		    !in_merge_bases(rev, head_rev)) {
 			fprintf(stderr,
 				"The branch '%s' is not a strict subset of your current HEAD.\n"
 				"If you are sure you want to delete it, run 'git branch -D %s'.\n",
diff --git a/commit.c b/commit.c
index a6d543e..4bddcbe 100644
--- a/commit.c
+++ b/commit.c
@@ -1009,3 +1009,20 @@ struct commit_list *get_merge_bases(struct commit *one,
 	free(rslt);
 	return result;
 }
+
+int in_merge_bases(struct commit *rev1, struct commit *rev2)
+{
+	struct commit_list *bases, *b;
+	int ret = 0;
+
+	bases = get_merge_bases(rev1, rev2, 1);
+	for (b = bases; b; b = b->next) {
+		if (!hashcmp(rev1->object.sha1, b->item->object.sha1)) {
+			ret = 1;
+			break;
+		}
+	}
+
+	free_commit_list(bases);
+	return ret;
+}
diff --git a/commit.h b/commit.h
index fc13de9..10eea9f 100644
--- a/commit.h
+++ b/commit.h
@@ -107,4 +107,5 @@ int read_graft_file(const char *graft_file);
 
 extern struct commit_list *get_merge_bases(struct commit *rev1, struct commit *rev2, int cleanup);
 
+int in_merge_bases(struct commit *rev1, struct commit *rev2);
 #endif /* COMMIT_H */
-- 
1.4.4.2.g688739

[PATCH 2/2] git reflog expire

[Junio C Hamano]

This prepares a place to collect reflog management subcommands,
and implements "expire" action.

	$ git reflog expire --dry-run \
		--expire=4.weeks \
		--expire-lost=1.week \
		refs/heads/master

The expiration uses two timestamps: --expire and --expire-lost.
Entries older than expire time (defaults to 90 days), and
entries older than expire-lost time (defaults to 30 days) and
talk about a commit that has been rewound and made unreachable
from the current tip of the ref are removed from the reflog.

The parameter handling is still rough, but I think the
core logic for expiration is already sound.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 Makefile         |    1 +
 builtin-reflog.c |  175 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 builtin.h        |    1 +
 git.c            |    1 +
 4 files changed, 178 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index 8919dab..17dabde 100644
--- a/Makefile
+++ b/Makefile
@@ -292,6 +292,7 @@ BUILTIN_OBJS = \
 	builtin-prune-packed.o \
 	builtin-push.o \
 	builtin-read-tree.o \
+	builtin-reflog.o \
 	builtin-repo-config.o \
 	builtin-rev-list.o \
 	builtin-rev-parse.o \
diff --git a/builtin-reflog.c b/builtin-reflog.c
new file mode 100644
index 0000000..aef2fc2
--- /dev/null
+++ b/builtin-reflog.c
@@ -0,0 +1,175 @@
+#include "cache.h"
+#include "builtin.h"
+#include "commit.h"
+#include "refs.h"
+#include "dir.h"
+#include <time.h>
+
+struct expire_reflog_cb {
+	FILE *newlog;
+	const char *ref;
+	struct commit *ref_commit;
+	unsigned long expire_total;
+	unsigned long expire_lost;
+};
+
+static int keep_entry(struct commit **it, unsigned char *sha1)
+{
+	*it = NULL;
+	if (is_null_sha1(sha1))
+		return 1;
+	*it = lookup_commit_reference_gently(sha1, 1);
+	return (*it != NULL);
+}
+
+static int expire_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
+			     char *data, void *cb_data)
+{
+	struct expire_reflog_cb *cb = cb_data;
+	unsigned long timestamp;
+	char *cp, *ep;
+	struct commit *old, *new;
+
+	cp = strchr(data, '>');
+	if (!cp || *++cp != ' ')
+		goto prune;
+	timestamp = strtoul(cp, &ep, 10);
+	if (*ep != ' ')
+		goto prune;
+	if (timestamp < cb->expire_total)
+		goto prune;
+
+	if (!keep_entry(&old, osha1) || !keep_entry(&new, nsha1))
+		goto prune;
+
+	if ((timestamp < cb->expire_lost) &&
+	    ((old && !in_merge_bases(old, cb->ref_commit)) ||
+	     (new && !in_merge_bases(new, cb->ref_commit))))
+		goto prune;
+
+	if (cb->newlog)
+		fprintf(cb->newlog, "%s %s %s",
+			sha1_to_hex(osha1), sha1_to_hex(nsha1), data);
+	return 0;
+ prune:
+	if (!cb->newlog)
+		fprintf(stderr, "would prune %s", data);
+	return 0;
+}
+
+struct cmd_reflog_expire_cb {
+	int dry_run;
+	unsigned long expire_total;
+	unsigned long expire_lost;
+};
+
+static int expire_reflog(const char *ref, const unsigned char *sha1, int unused, void *cb_data)
+{
+	struct cmd_reflog_expire_cb *cmd = cb_data;
+	struct expire_reflog_cb cb;
+	struct ref_lock *lock;
+	char *newlog_path = NULL;
+	int status = 0;
+
+	if (strncmp(ref, "refs/", 5))
+		return error("not a ref '%s'", ref);
+
+	memset(&cb, 0, sizeof(cb));
+	/* we take the lock for the ref itself to prevent it from
+	 * getting updated.
+	 */
+	lock = lock_ref_sha1(ref + 5, sha1);
+	if (!lock)
+		return error("cannot lock ref '%s'", ref);
+	if (!file_exists(lock->log_file))
+		goto finish;
+	if (!cmd->dry_run) {
+		newlog_path = xstrdup(git_path("logs/%s.lock", ref));
+		cb.newlog = fopen(newlog_path, "w");
+	}
+
+	cb.ref_commit = lookup_commit_reference_gently(sha1, 1);
+	if (!cb.ref_commit) {
+		status = error("ref '%s' does not point at a commit", ref);
+		goto finish;
+	}
+	cb.ref = ref;
+	cb.expire_total = cmd->expire_total;
+	cb.expire_lost = cmd->expire_lost;
+	for_each_reflog_ent(ref, expire_reflog_ent, &cb);
+ finish:
+	if (cb.newlog) {
+		if (fclose(cb.newlog))
+			status |= error("%s: %s", strerror(errno),
+					newlog_path);
+		if (rename(newlog_path, lock->log_file)) {
+			status |= error("cannot rename %s to %s",
+					newlog_path, lock->log_file);
+			unlink(newlog_path);
+		}
+	}
+	free(newlog_path);
+	unlock_ref(lock);
+	return status;
+}
+
+static const char reflog_expire_usage[] =
+"git-reflog expire [--dry-run] [--expire=<time>] [--expire-lost=<time>] [--all] <refs>...";
+
+static int cmd_reflog_expire(int argc, const char **argv, const char *prefix)
+{
+	struct cmd_reflog_expire_cb cb;
+	unsigned long now = time(NULL);
+	int i, status, do_all;
+
+	save_commit_buffer = 0;
+	do_all = status = 0;
+	memset(&cb, 0, sizeof(cb));
+	cb.expire_total = now - 90 * 24 * 3600;
+	cb.expire_lost = now - 30 * 24 * 3600;
+
+	for (i = 1; i < argc; i++) {
+		const char *arg = argv[i];
+		if (!strcmp(arg, "--dry-run") || !strcmp(arg, "-n"))
+			cb.dry_run = 1;
+		else if (!strncmp(arg, "--expire=", 9))
+			cb.expire_total = approxidate(arg + 9);
+		else if (!strncmp(arg, "--expire-lost=", 14))
+			cb.expire_lost = approxidate(arg + 14);
+		else if (!strcmp(arg, "--all"))
+			do_all = 1;
+		else if (!strcmp(arg, "--")) {
+			i++;
+			break;
+		}
+		else if (arg[0] == '-')
+			usage(reflog_expire_usage);
+		else
+			break;
+	}
+	if (do_all)
+		status |= for_each_ref(expire_reflog, &cb);
+	while (i < argc) {
+		const char *ref = argv[i++];
+		unsigned char sha1[20];
+		if (!resolve_ref(ref, sha1, 1, NULL)) {
+			status |= error("%s points nowhere!", ref);
+			continue;
+		}
+		status |= expire_reflog(ref, sha1, 0, &cb);
+	}
+	return status;
+}
+
+static const char reflog_usage[] =
+"git-reflog (expire | ...)";
+
+int cmd_reflog(int argc, const char **argv, const char *prefix)
+{
+	if (argc < 2)
+		usage(reflog_usage);
+	else if (!strcmp(argv[1], "expire"))
+		return cmd_reflog_expire(argc - 1, argv + 1, prefix);
+	else
+		usage(reflog_usage);
+}
diff --git a/builtin.h b/builtin.h
index 08519e7..fdc0907 100644
--- a/builtin.h
+++ b/builtin.h
@@ -51,6 +51,7 @@ extern int cmd_prune(int argc, const char **argv, const char *prefix);
 extern int cmd_prune_packed(int argc, const char **argv, const char *prefix);
 extern int cmd_push(int argc, const char **argv, const char *prefix);
 extern int cmd_read_tree(int argc, const char **argv, const char *prefix);
+extern int cmd_reflog(int argc, const char **argv, const char *prefix);
 extern int cmd_repo_config(int argc, const char **argv, const char *prefix);
 extern int cmd_rev_list(int argc, const char **argv, const char *prefix);
 extern int cmd_rev_parse(int argc, const char **argv, const char *prefix);
diff --git a/git.c b/git.c
index 016ee8a..ae4c99f 100644
--- a/git.c
+++ b/git.c
@@ -256,6 +256,7 @@ static void handle_internal_command(int argc, const char **argv, char **envp)
 		{ "prune-packed", cmd_prune_packed, RUN_SETUP },
 		{ "push", cmd_push, RUN_SETUP },
 		{ "read-tree", cmd_read_tree, RUN_SETUP },
+		{ "reflog", cmd_reflog, RUN_SETUP },
 		{ "repo-config", cmd_repo_config },
 		{ "rev-list", cmd_rev_list, RUN_SETUP },
 		{ "rev-parse", cmd_rev_parse, RUN_SETUP },
-- 
1.4.4.2.g688739

Re: [PATCH 2/2] git reflog expire

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> This prepares a place to collect reflog management subcommands,
> and implements "expire" action.
> 
> 	$ git reflog expire --dry-run \
> 		--expire=4.weeks \
> 		--expire-lost=1.week \
> 		refs/heads/master
> 
> The expiration uses two timestamps: --expire and --expire-lost.
> Entries older than expire time (defaults to 90 days), and
> entries older than expire-lost time (defaults to 30 days) and
> talk about a commit that has been rewound and made unreachable
> from the current tip of the ref are removed from the reflog.
> 
> The parameter handling is still rough, but I think the
> core logic for expiration is already sound.

I agree, this looked pretty good to me.

I disagree with the option name '--expire-lost'.  The way I
initially read that was:

    --expire-lost: ok, if the object was already pruned out of
    the ODB and the reflog entry is older than 1 week, remove it;
    otherwise keep it in case the user could manually recover the
    object from another ODB.

Of course that's not what the code does, because if either the
old or the new object is no longer in the ODB you are pruning away
the log entry.  I cannot however come up with a better name than
--expire-lost.  :-(

Perhaps --expire-lost should default to just be 1/3 of the time of
--expire, whatever --expire is: 90 day default or command line value?


I'm thinking that we may want the 'expire' subcommand to simply be
implied by '--expire' instead.  Basically my rational here is I want
to be able to do 'git reflog HEAD' to view the reflog associated with
my current branch, effectively seeing the Git operational history
of this branch.  Or 'git reflog a b' to see the operational history
of two branches with their reflogs interleaved based on entry time.

Needing a subcommand like 'git reflog show HEAD' is just a lot
of typing[*1*].

I would also say maybe we want to make --dry-run the default, with
a final message which tells the user that if they really want to
make it possible to throw away the commits printed above then
restart the expire operation, e.g.:

  $ git reflog --expire=4.weeks --expire-lost=1.week master
  would expire ...
  would expire ...
  would expire ...

  Restart with '--prune' to expire the above log entries and commits.
  $ git reflog --expire=4.weeks --expire-lost=1.week master --prune

?


I'd like to take a stab at the log display code for the reflog
command, but I'd also really like to port forward (aka rewrite)
that mmap window code I keep saying I'll work on, but never quite
seem to do...


*1* Yes, I know, I'll do bash completion for this command too.  ;-)

-- 

Re: [PATCH 2/2] git reflog expire

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> Of course that's not what the code does, because if either the
> old or the new object is no longer in the ODB you are pruning away
> the log entry.  I cannot however come up with a better name than
> --expire-lost.  :-(

How about --expire-unreachable?

> I'm thinking that we may want the 'expire' subcommand to simply be
> implied by '--expire' instead.

After coding this, my conclusion is the same as yours, but
reasoning behind it is slightly different.

To have 'expire' action as a subcommand to 'git-reflog' is from
implementor's point of view, and is a horrible organization from
the UI standpoint.  To the end users, it may be easier to have a
single 'git-gc' command that runs these commands with reasonable
set of defaults:

	- rerere gc
        - reflog expire --all
	- (possibly) repack -a -d
        - prune

If we go that route, it probably is not even necessary to
advertise that 'expire' is a subcommand of reflog.  The users
would not run it from the command line; it is an implementation
detail of 'git-gc' command.

> Needing a subcommand like 'git reflog show HEAD' is just a lot
> of typing[*1*].

I am very interested in seeing how 'git reflog show HEAD' would
show the reflog entries.  I've tried showing it just like log
family shows (it is reasonably easy; you build the list of revs
out of reflog entries and feed them to 'git-show' machinery),
and while it works, it is unusable for the purpose of seeing
which ones are the lost ones (amended commits and rebased branch
remnants).

The best I came up with is still my "show-branch --reflog" so
far.  You really need to show not just the commit title but how
they topologically relate to the commits on the surviving
branch, and I think having something graphical or semi-graphical
is a must.

> I would also say maybe we want to make --dry-run the default, with
> a final message which tells the user that if they really want to
> make it possible to throw away the commits printed above then
> restart the expire operation.

I am moderately negative on that.  Nobody does it like that;
prune, branch -d, tag -d,...

> I'd like to take a stab at the log display code for the reflog
> command, but I'd also really like to port forward (aka rewrite)
> that mmap window code I keep saying I'll work on, but never quite
> seem to do...

After today's pread() thing, I was also wondering about that too
;-).

Re: [PATCH 2/2] git reflog expire

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
> > I'd like to take a stab at the log display code for the reflog
> > command, but I'd also really like to port forward (aka rewrite)
> > that mmap window code I keep saying I'll work on, but never quite
> > seem to do...
> 
> After today's pread() thing, I was also wondering about that too
> ;-).

I'm working on it now.  I have rather long days on Monday and
Tuesday which keep me otherwise occupied and away from Git work.
Hopefully I'll have something sometime on Wednesday, which so far
I have all day free for hacking.

Smaller mmap units may help on some larger projects. For example
'git whatchanged' shouldn't need to setup page table entries for
the entire pack file when only the front and middle are needed to
obtain the data for the most recent 25 commits, before the user
kills the pager.

But that's just me pulling words out of the air.  I'm tired too,
time to catch a few hours of sleep.  Probably by the time I get back
to my email our local kernel hackers will tell me how wrong I am.

-- 

Re: [PATCH 2/2] git reflog expire

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
> 
> > Of course that's not what the code does, because if either the
> > old or the new object is no longer in the ODB you are pruning away
> > the log entry.  I cannot however come up with a better name than
> > --expire-lost.  :-(
> 
> How about --expire-unreachable?

That actually does make more sense.  Those commits are what we would
traditionally have called unreachable commits, or dangling commits.
Hmm.  Flip a coin.  I'm probably as bad at UI as you are.  :-)
 
> To have 'expire' action as a subcommand to 'git-reflog' is from
> implementor's point of view, and is a horrible organization from
> the UI standpoint.  To the end users, it may be easier to have a
> single 'git-gc' command that runs these commands with reasonable
> set of defaults:
> 
>   - rerere gc
>   - reflog expire --all
>   - (possibly) repack -a -d
>   - prune
> 
> If we go that route, it probably is not even necessary to
> advertise that 'expire' is a subcommand of reflog.  The users
> would not run it from the command line; it is an implementation
> detail of 'git-gc' command.

Yes!  I agree with you completely here.  I'd also include 'pack-refs
--prune' to your list above, at least for refs/tags.  Those suckers
can eat up a lot of disk, slow down viualizers, and well, just
don't change.  Pack 'em!

> > Needing a subcommand like 'git reflog show HEAD' is just a lot
> > of typing[*1*].
> 
> I am very interested in seeing how 'git reflog show HEAD' would
> show the reflog entries.  I've tried showing it just like log
> family shows (it is reasonably easy; you build the list of revs
> out of reflog entries and feed them to 'git-show' machinery),
> and while it works, it is unusable for the purpose of seeing
> which ones are the lost ones (amended commits and rebased branch
> remnants).

At this point I'm going to wind up getting less than 4 hours of
sleep tonight (I'll be lucky if I get 3).  So I'm just going to skip
trying to draft out what I might want this to look like right now.
I'll try to sketch out a dump by hand from a couple of my own reflogs
and post 'em later tonight.  Ugh.  Tonight.  Its Tuesday already.
My body still thinks its Monday.

> The best I came up with is still my "show-branch --reflog" so
> far.  You really need to show not just the commit title but how
> they topologically relate to the commits on the surviving
> branch, and I think having something graphical or semi-graphical
> is a must.

Probably right.  But I can't make heads or tails out of that
output for just one topic branch whose reflog consists of only
the following:

  $ cut -f2 .git/logs/refs/heads/sp/mapwin 
  checkout: Created from origin^0
  am: Create read_or_die utility routine.
  reset --hard 106d710b
  am: Create read_or_die utility routine.
  reset --hard 
  rebase: Create read_or_die utility routine.

`show-branch --reflog --topics sp/mapwin` prints much shorter output
and almost says something useful, but not really.

-- 

Re: [PATCH 2/2] git reflog expire

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

>> The best I came up with is still my "show-branch --reflog" so
>> far.  You really need to show not just the commit title but how
>> they topologically relate to the commits on the surviving
>> branch, and I think having something graphical or semi-graphical
>> is a must.
>
> Probably right.  But I can't make heads or tails out of that
> output for just one topic branch...

Here is what my private 'pu' look like:

! [pu@{0}] Merge branch 'jc/fsck-reflog' into pu
 ! [pu@{1}] Merge branch 'jc/pickaxe' into pu
  ! [pu@{2}] Merge branches 'jn/web', 'jc/explain', 'jc/web' and
   ! [pu@{3}] Merge branch 'jc/leftright' into next
    ! [pu@{4}] Merge branch 'jc/fsck-reflog' into pu
     ! [pu@{5}] Merge branch 'jc/pickaxe' into pu
      ! [pu@{6}] Merge branches 'jc/fsck-reflog', 'jc/clone', 'j
       ! [pu@{7}] Merge branch 'jc/blame' into next
--------
-        [pu@{0}] Merge branch 'jc/fsck-reflog' into pu
+        [pu@{0}^2] git reflog expire
+        [pu@{0}^2^] Move in_merge_bases() to commit.c
+        [pu@{0}^2~2] reflog: fix warning message.
--       [pu@{1}] Merge branch 'jc/pickaxe' into pu
---      [pu@{2}] Merge branches 'jn/web', 'jc/explain', 'jc/web
----     [pu@{3}] Merge branch 'jc/leftright' into next
++++     [pu@{3}^2] Revert "Make left-right automatic."
      ...
++++     [pu@{3}~2^2~3^2^] Revert "fix testsuite: make sure they
    -    [pu@{4}] Merge branch 'jc/fsck-reflog' into pu
+   +    [pu@{0}^2~3] Teach git-repack to preserve objects refer
    --   [pu@{5}] Merge branch 'jc/pickaxe' into pu
    ---  [pu@{6}] Merge branches 'jc/fsck-reflog', 'jc/clone', '
+++ +++  [pu@{2}^5] para-walk: walk n trees, index and working t
      ...
+   +++  [pu@{0}^2~5] add for_each_reflog_ent() iterator
-------- [pu@{7}] Merge branch 'jc/blame' into next

What matters most is the leftmost column.  Most of the time when
you are looking at reflog, you are looking for what's rewound
and lost.  Stretches of lines with symbols (e.g. pu@{3}^2 to
pu@{3}~2^2~3^2^, and pu@{0}^2 to pu@{0}^2~2) are not so
interesting [*1*].

The entry whose column have whitespace there are the lost ones
(rewound or amended).  pu@{4} used to be the tip of 'pu', but
after I whipped 'next' into a reasonable shape, I did "git
branch -f pu next" to bundle the tips of topics that are not
ready for 'next' into it (and all the rest pu@{3}, pu@{2},
pu@{1} and pu@{0} are sequence of merges into 'pu').


[Footnote]

*1* We might want to add an output filter that lets the command
omit such lines with the same set of reachable tips when showing
the reflog entries.

Re: [PATCH 2/2] git reflog expire

[Linus Torvalds]

On Tue, 19 Dec 2006, Shawn Pearce wrote:
> 
> Smaller mmap units may help on some larger projects. For example
> 'git whatchanged' shouldn't need to setup page table entries for
> the entire pack file when only the front and middle are needed to
> obtain the data for the most recent 25 commits, before the user
> kills the pager.

At least on Linux, we do NOT set up page table entries unnecessarily. The 
cost of a mmap() is largely independent of the size of the mmap, and the 
costs will be on page fault time (which is obviously O(n) in number of 
pages you need).

The same is _largely_ true of munmap() too. There's a small cost to large 
mmaps that were only sparsely accessed being unmapped (we have to look at 
the page tables closely, even if they end up being empty), but quite 
frankly, it's not going to be all that noticeable.

So mmap() costs end up largely being one (fairly small) fixed cost, plus a 
(fairly small) cost for each page beign demand-mapped in. 

Of course, Linux is just _better_ than the competition, so things that 
don't help Linux might still help other systems. Linux does both page 
faults and the mmap system call well, other systems will generally be an 
order of magnitude worse in both. And as we've seen, it can be more.

(Things that are really fast on Linux: VM manipulation, and filename path 
lookup. Those are both ops that Linux _really_ shines on. It may not sound 
like much, but when you do millions of them, it's the difference between 
seconds and hours).

Re: [PATCH 2/2] git reflog expire

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> (Things that are really fast on Linux: VM manipulation, and filename path 
> lookup. Those are both ops that Linux _really_ shines on. It may not sound 
> like much, but when you do millions of them, it's the difference between 
> seconds and hours).
>
> 		Linus

... and both have been _heavily_ used in/assumed to be fast by
git implementation.

Linus, your userland programming skills are _spoiled_ by Linux
;-).

Re: [PATCH 2/2] git reflog expire

[Linus Torvalds]

On Tue, 19 Dec 2006, Junio C Hamano wrote:
>
> Linus, your userland programming skills are _spoiled_ by Linux
> ;-).

Maybe. On the other hand, there's definitely a bit of just my programming 
style involved: the things I care most about are the kinds of things I'd 
personally use in programming. So I claim that one of the reasons Linux 
does pathname lookup well is simply that every single program I've ever 
done that cared about performance of either open() or stat() or both. And 
yes, I tend to have a tendency to mmap() rather than read. So I optimize 
stuff I do.

That said, Linux does a lot of _other_ things well too, so it's not like 
it's just the things I do.