getting rid of extra directories

getting rid of extra directories

[Zack Brown]

Hi folks,

I've been using the latest Cogito. When I seek back and forth in a git repo,
directories I've created tend to persist backward in time, cluttering up
earlier versions.

By the discussions on this list, I'm pretty sure there's a way to stop this,
but I don't know clearly what it is, and I'm worried about trying things
that might corrupt my repository. Could someone spell it out for me please?

Be well,
Zack

-- 
Zack Brown

Re: getting rid of extra directories

[Junio C Hamano]

Zack Brown <zbrown@tumblerings.org> writes:

> I've been using the latest Cogito. When I seek back and forth in a git repo,
> directories I've created tend to persist backward in time, cluttering up
> earlier versions.
>
> By the discussions on this list, I'm pretty sure there's a way to stop this,
> but I don't know clearly what it is, and I'm worried about trying things
> that might corrupt my repository. Could someone spell it out for me please?

I do not know exactly what Cogito "seek back and forth" does,
but updating to git-core "master" branch may help.

Specifically, this commit intends to remove empty directory from
the working tree, where previously checked out tree object had
files/directories inside while newly checked out tree does not.

    commit 340e4f88c083b0692e6554b1c2c27fd43c7cc8d3
    Author: Junio C Hamano <junkio@cox.net>
    Date:   Mon Oct 10 17:34:08 2005 -0700

        Remove empty directories after read-tree -u.

        This fixes everybody's favorite gripe that switching branch
        with 'git checkout' leaves empty directories.

        Signed-off-by: Junio C Hamano <junkio@cox.net>

Re: getting rid of extra directories

[Zack Brown]

On Sat, Oct 15, 2005 at 11:00:52AM -0700, Junio C Hamano wrote:
> Zack Brown <zbrown@tumblerings.org> writes:
> 
> > I've been using the latest Cogito. When I seek back and forth in a git repo,
> > directories I've created tend to persist backward in time, cluttering up
> > earlier versions.
> >
> > By the discussions on this list, I'm pretty sure there's a way to stop this,
> > but I don't know clearly what it is, and I'm worried about trying things
> > that might corrupt my repository. Could someone spell it out for me please?
> 
> I do not know exactly what Cogito "seek back and forth" does,

It means to "cg-seek" back to an earlier state of the repo, and forward to a
more recent state of the repo. "seeking back and forth" means doing a lot of
"cg-seek" commands. Sorry that wasn't clear.

> but updating to git-core "master" branch may help.

I already use that branch.

Here's some commands that reproduce the problem:

$ mkdir tmpproj
$ cd tmpproj
$ cg-init

give the initial commit message and save

$ ls > a
$ cg-add a
$ cg-commit

give the commit message and save

$ cg-tag nodirs
$ mkdir d
$ mv a d/a
$ cg-rm a
$ cg-add d/a
$ cg-commit

give the commit message and save

$ cg-seek nodirs

now I would expect to see the original directory, with no 'd' subdirectory, and
just the 'a' file sitting there.

$ ls -F
a  d/
$

See the problem? The 'd' directory did not exist in this version of the
repository, but it's there anyway.

Be well,
Zack



> 
> Specifically, this commit intends to remove empty directory from
> the working tree, where previously checked out tree object had
> files/directories inside while newly checked out tree does not.
> 
>     commit 340e4f88c083b0692e6554b1c2c27fd43c7cc8d3
>     Author: Junio C Hamano <junkio@cox.net>
>     Date:   Mon Oct 10 17:34:08 2005 -0700
> 
>         Remove empty directories after read-tree -u.
> 
>         This fixes everybody's favorite gripe that switching branch
>         with 'git checkout' leaves empty directories.
> 
>         Signed-off-by: Junio C Hamano <junkio@cox.net>
> 
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Zack Brown

Re: getting rid of extra directories

[Junio C Hamano]

Zack Brown <zbrown@tumblerings.org> writes:

>> I do not know exactly what Cogito "seek back and forth" does,
>
> It means to "cg-seek" back to an earlier state of the repo, and forward to a
> more recent state of the repo. "seeking back and forth" means doing a lot of
> "cg-seek" commands. Sorry that wasn't clear.
>
>> but updating to git-core "master" branch may help.
>
> I already use that branch.

I usually do not use Cogito, so I updated to the latest and
tried it myself.  No wonder.

cg-seek does not use "git-read-tree -m -u $old $new", which was
updated by that commit I mentioned to remove the empty
directories. It does "git-read-tree -m $new" and does the
removal part manually by reading from diff-tree $old $new.  I
suspect the code was there before the two-tree read-tree was
invented.

A similar sequence as you did, in pure git, would remove empty
directory d/ when switching branches between.

    $ mkdir junkproj
    $ cd junkproj
    $ git-init-db
    $ ls >a
    $ git-add a
    $ git-commit -a -m 'Add a'
    $ git-checkout -b adddir
    $ mkdir d
    $ mv a d/a
    $ git-add d
    $ git-commit -a -m 'Move a to d/a'

    $ git-checkout master

I suspect cg-Xlib::tree_timewarp, which currently does
"git-read-tree -m $branch" followed by diff-tree piped to xargs,
can be taught to use "git-read-tree -m -u $base $branch" (and
lose the git-checkout-index -f -a immediately after that while
we are at it), but I do not do Porcelain, so...

Three-way merge with the index as one way

[Petr Baudis]

Dear diary, on Sun, Oct 16, 2005 at 12:34:29AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> I suspect cg-Xlib::tree_timewarp, which currently does
> "git-read-tree -m $branch" followed by diff-tree piped to xargs,
> can be taught to use "git-read-tree -m -u $base $branch" (and
> lose the git-checkout-index -f -a immediately after that while
> we are at it), but I do not do Porcelain, so...

This is problematic, since with our current way of tree timewarping,
restoring local changes to files which got deleted will make patch error
out, etc.

I wanted to make tree_timewarp do three-way merge, but didn't figure a
good way to do it. First, what do I want - a two-way merge between two
trees, which will however respect (not die on) local changes in the
working tree.

One approach is to take the working tree, construct a tree object from
it and then run the regular three-way merge on that. But that is a waste
of time and pollutes the database with nonsensical objects. The
advantage is that I can reuse the per-file merge resolution script
without any changes.

However, this seems to be the only way to do this right now. What do you
think about the disadvantages - should we care? A possible alternative I
can imagine is to make a two-way merge mode which somehow records the
conflicts with index back in the index file instead of committing
suicide. Possibly the hex(-1) sha1 might be used for the relevant index
items, meaning "the current working copy". The per-file merge resolution
scripts would have to be tweaked for this then, but it shouldn't be that
hard; what I found hard was actually getting my head 'round the weird
git-read-tree code recording the stages in the index file. ;-)

Opinions?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: Three-way merge with the index as one way

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> One approach is to take the working tree, construct a tree object from
> it and then run the regular three-way merge on that. But that is a waste
> of time and pollutes the database with nonsensical objects. The
> advantage is that I can reuse the per-file merge resolution script
> without any changes.

For a large tree, constructing a full tree just to merge a
couple of potentially conflicting paths might turn out be a
waste of time, but I suspect that is something you could
optimize later.  git-am 3-way fallback logic constructs a sparse
temporary tree and uses it for running regular 3-way, and you
may be able to do something similar.  Right now git-am takes an
e-mail patch and when it runs all the way it creates a commit,
but the core logic that applies patch and falls back to 3-way
could be separated out for your application.  Then you could:
(1) 'git diff HEAD' before tree-warp to preserve the user
changes, (2) clean up the working tree to match HEAD, (3) warp
to the other tree, and (4) fed the user change perserved to
git-am core logic.  That might give you something near optimum.

About "nonsensical" objects, I am not so sure how nonsensical
those objects you would record from the working tree
(intermediate state) are.  If you find two trees match on a path
that has changed in the working tree, there won't be any
conflict on that paths so it is like the user did an extra
git-update-index on that path when no merge or warp is involved,
from object database pollution POV; I do not personally think
that is such a bad thing.

Re: Three-way merge with the index as one way

[Petr Baudis]

Dear diary, on Thu, Nov 03, 2005 at 09:02:10AM CET, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> git-am 3-way fallback logic constructs a sparse
> temporary tree and uses it for running regular 3-way, and you
> may be able to do something similar.

I'm confused now - if the tree is sparse, how does git-read-tree -m know
that the missing files from the other tree are not deletes?

> Right now git-am takes an
> e-mail patch and when it runs all the way it creates a commit,
> but the core logic that applies patch and falls back to 3-way
> could be separated out for your application.  Then you could:
> (1) 'git diff HEAD' before tree-warp to preserve the user
> changes, (2) clean up the working tree to match HEAD,

At this point I get nervous since there is a potential for data loss
when something goes wrong at this point (e.g. system crash). I'm not
saying there are no places like that in Cogito now, but I generally
try to avoid them whenever possible.

> (3) warp to the other tree, and (4) fed the user change perserved
> to git-am core logic.  That might give you something near optimum.
> 
> About "nonsensical" objects, I am not so sure how nonsensical
> those objects you would record from the working tree
> (intermediate state) are.  If you find two trees match on a path
> that has changed in the working tree, there won't be any
> conflict on that paths so it is like the user did an extra
> git-update-index on that path when no merge or warp is involved,
> from object database pollution POV; I do not personally think
> that is such a bad thing.

But if they don't match, you'll get superfluous blobs for the original
index versions which are almost certain to go away.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: Three-way merge with the index as one way

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

>> git-am 3-way fallback logic constructs a sparse
>> temporary tree and uses it for running regular 3-way, and you
>> may be able to do something similar.
>
> I'm confused now - if the tree is sparse, how does git-read-tree -m know
> that the missing files from the other tree are not deletes?

The git-am 3-way fallback works like this:

    - the patch did not apply to our current head cleanly.

    - however, we have all the original blobs, recorded on
      'index xxxxxxx..xxxxxxx' line of the patch.

    - construct a temporary tree T0 that has only paths affected
      by that patch, with the original blobs.

    - apply the patch to T0 to construct another temporary tree
      T1.

    - 3-way merge our current head and T1, pretending as if T0
      is our common ancestor.  I.e. "read-tree -m -u T0 HEAD T1"

The paths that did not change by the patch (in your case, the
paths without local modification before tree-warp) are not
present in either T0 or T1, so the 3-way merge in the last step
would see that our current head added those paths while T0->T1
transition kept their absense intact --- which results in
added-in-one-side-only.

I think the problem you are solving would be solved by what I
suggested in the earlier message, but probably always doing that
may not be the optimum solution.

    - the working tree (and index) is derived from HEAD, but may
      not match HEAD (local modifications).

    - you want to tree-warp to T1, doing an equivalent of
      "read-tree -m -u HEAD T1" but preserving the local
      modifications.

You would want a merge between INDEX and T1 using HEAD as the
common ancestor (INDEX is a tree that your user would get if all
the local modification in the working tree were registered with
update-index and then git-write-tree were run).

	 INDEX
        /
    HEAD
        \
         T1

Two-tree form of read-tree -m -u should work as long as
differences between HEAD and INDEX do not interfere with
HEAD->T1 transition.  So you probably would first want to try
"read-tree -m -u HEAD T1", and use the 3-way fallback I
described earlier when it fails?

> But if they don't match, you'll get superfluous blobs for the original
> index versions which are almost certain to go away.

Another way of putting it is that I would not particularly worry
about those dangling blobs and trees -- that's what we have 'git
prune' for.

Re: Three-way merge with the index as one way

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> I wanted to make tree_timewarp do three-way merge, but didn't figure a
> good way to do it. First, what do I want - a two-way merge between two
> trees, which will however respect (not die on) local changes in the
> working tree.

I remember that we used to have --emu23 flag to read-tree.  It
was yanked out when we did multi-merge-base conversion, because
nobody was using it, but I wonder if that is close to what you
are looking for.

$ git whatchanged -S--emu23 read-tree.c

tells me that 32192e6622d78347448cfc0572827d6e64e0de28 commit
completed it and ee6566e8d70da682ac4926dd8a67ac821b2c1743 commit
removed it.