Git drawbacks?

Git drawbacks?

[Dmitry Smirnov]

Hi,

Sorry if I selected the wrong place to discuss the drawbacks of the Git. Just 
point to the proper one...

I'm just trying to select the best VCS for me personally.
I have a very small experience with Git but I see is also not very sutable for 
me.

First, it seems to be very hard to setup some really big project (like Android, 
for example). Otherwise, why do they need to invent 'repo'? What purpose it 
solves? It looks like it
1. Integrates few subcomponents (projects) and checkout the code in the proper 
configuration. The question is why this is not the Git task? For me, it looks 
like the ClearCase client spec.
2.? What others (except integration with review tool)? 

The next issue with git is its clone. Why do I need the whole set of revisions? 
Why do I need to get 1GB of Android? You could say this should happen once. I 
would agree but when I tried to resync the Android tree after 2 months, I was 
struggled with many errors (both git and repo). Finally, I had decided to sync 
again. :-)
There is one point against clone. The typical situation in my office is to have 
few Perforce clients with the same or slightly  different code. This is just 
wasting a space since you need them all but versions of many files are the same. 
I'm trying to imagine the same situation with Git. Are there any benefits? It 
seems, no. Moreover, I will have not only few working trees but few repository 
clones!

It is obvious that configuration management with Git is very difficult (for ex, 
http://groups.google.com/group/repo-
discuss/browse_thread/thread/2fa368ed7cac5d79/64ced51656240ddc?
lnk=gst&q=create+android+bare+repository#64ced51656240ddc)

Let's consider the foolwing use case. Suppose I'm intending to create a new 
product that consists of specific versions (or branches) of some subcomponents 
(or directories). How can I do this with Git? Subsequent changes could either be 
submitted to the appropriate component branch or branched to the new one (this 
way is possible with Git, of course, if I will branch the code I need to this 
new branch).

So, I'm wondering, why Git (or any other VCS) is not trying to solve these 
problems? Perhaps, there is a simple solution with Git I'm not aware of?

Here is the wish list for the VCS I would prefer:
1. Atomit commits
2. The possibility to get any slice of the code repository with the possibility 
to commit my changes on tip or on separate branch.
3. The minimum footprint of the same code on my local machine.
4. No code/history on my machine untill I really need it.
5. Easy mirroring and replication

I would say, ClearCase might be my favorite if it is not commercial. :-)

Dmitry

Re: Git drawbacks?

[Avery Pennarun]

On Fri, Nov 6, 2009 at 11:17 AM, Dmitry Smirnov <divis1969@gmail.com> wrote:
> Here is the wish list for the VCS I would prefer:
> 1. Atomit commits
> 2. The possibility to get any slice of the code repository with the possibility
> to commit my changes on tip or on separate branch.
> 3. The minimum footprint of the same code on my local machine.
> 4. No code/history on my machine untill I really need it.
> 5. Easy mirroring and replication
>
> I would say, ClearCase might be my favorite if it is not commercial. :-)

#1 and #5 are features of any DVCS, so git already has them.  #2, 3,
and 4 are all just saying the same thing: "I can't afford the disk
space to store the entire repo."  Are you sure this is true, or is it
a preconception?  Even a 1GB repository is tiny by modern disk
standards.

My (limited) experience with ClearCase is that it's so slow that you'd
do *anything* to track fewer files in your working copy, so they put a
lot of work into exactly that, and no work into performance.  This
lousy performance isn't the case in git (except in Windows).  Are you
using Windows, by chance?

Have fun,

Avery

Re: Git drawbacks?

[Dmitry Smirnov]

> > Here is the wish list for the VCS I would prefer:
> > 1. Atomit commits
> > 2. The possibility to get any slice of the code repository with the 
possibility
> > to commit my changes on tip or on separate branch.
> > 3. The minimum footprint of the same code on my local machine.
> > 4. No code/history on my machine untill I really need it.
> > 5. Easy mirroring and replication
> >
> > I would say, ClearCase might be my favorite if it is not commercial. 
> 
> #1 and #5 are features of any DVCS, so git already has them.  #2, 3,
> and 4 are all just saying the same thing:

No, #2 is about the repository slicing, branching, merging (SCM in other words). 
Let's suppose I have the product that have 2 directories: component1 and 
component2. They were developing together for  previous product (on the same 
branch, for example). Now, I would like to have component1 and replace 
component2 with some 3rd party component. What should I do with Git to get this? 
Or maybe I wish to stick with some version of component2 and provide only bug 
fixes for this product...
Or let's take a look at GDB. They are using binutils which are in separate 
repository (they use CVS, but let's imagine they use Git). How many effors they 
will need for SCM? For example, they would prefer to stick to some stable 
version/branch of the binutils but should be able to commit bug fixes.

Once again, perhaps there is some way to do this with Git? I did not yet find 
it.

> "I can't afford the disk
> space to store the entire repo."  Are you sure this is true, or is it
> a preconception?  Even a 1GB repository is tiny by modern disk
> standards.

oh, yes, since we have big drives and fast internet, we do not have to worry 
about space and download time... :-)

> My (limited) experience with ClearCase is that it's so slow that you'd
> do *anything* to track fewer files in your working copy, so they put a
> lot of work into exactly that, and no work into performance.

This probably true. Thought I did not have a lot of problems with it unless I 
use GUI.

>  This
> lousy performance isn't the case in git (except in Windows).  Are you
> using Windows, by chance?

yes. I did not yet noticed any performance problems with Git on windows, except 
a sync/download time (for android, mostly)

Re: Git drawbacks?

[Jacob Helwig]

On Fri, Nov 6, 2009 at 09:35, Dmitry Smirnov <divis1969@gmail.com> wrote:
> No, #2 is about the repository slicing, branching, merging (SCM in other words).
> Let's suppose I have the product that have 2 directories: component1 and
> component2. They were developing together for  previous product (on the same
> branch, for example). Now, I would like to have component1 and replace
> component2 with some 3rd party component. What should I do with Git to get this?
> Or maybe I wish to stick with some version of component2 and provide only bug
> fixes for this product...
> Or let's take a look at GDB. They are using binutils which are in separate
> repository (they use CVS, but let's imagine they use Git). How many effors they
> will need for SCM? For example, they would prefer to stick to some stable
> version/branch of the binutils but should be able to commit bug fixes.
>
> Once again, perhaps there is some way to do this with Git? I did not yet find
> it.
>

Sounds like you want submodules.  Check out the git-submodule(1) manpage.

Re: Git drawbacks?

[Avery Pennarun]

On Fri, Nov 6, 2009 at 12:35 PM, Dmitry Smirnov <divis1969@gmail.com> wrote:
> No, #2 is about the repository slicing, branching, merging (SCM in other words).
> Let's suppose I have the product that have 2 directories: component1 and
> component2. They were developing together for  previous product (on the same
> branch, for example). Now, I would like to have component1 and replace
> component2 with some 3rd party component. What should I do with Git to get this?
> Or maybe I wish to stick with some version of component2 and provide only bug
> fixes for this product...

There are three methods I know of to manage this:

1) Just commit whatever version of a subproject you want as a subtree
of your current project, and if you want to replace/delete/upgrade it,
just do that.  (You rarely want to track the actual *history* of the
third party tool, just the history of versions *you* used, which is
easy to do.)

2) Use git-submodule to link repositories together.  (Arguably, one
major reason 'repo' was written is that git-submodule is too
complicated, though.)

3) Try my git-subtree tool, which basically makes it easier to
split/join repositories (similar to #1) without losing the history
(similar to #2).

>> This
>> lousy performance isn't the case in git (except in Windows).  Are you
>> using Windows, by chance?
>
> yes. I did not yet noticed any performance problems with Git on windows, except
> a sync/download time (for android, mostly)

Basically, performance is linear with the number of files in your
repo.  If you can check out just a "slice" of your repo (say 10% of
the whole), you'll have faster performance (eg. 10x) from any VCS.

git on Linux is so fast that this isn't very necessary most of the
time.  But git on Windows isn't really any faster than other VCSes on
Windows, so the time-per-file is much greater, and thus the penalty
for huge repositories is much worse.  Doing things like switching
branches, which is near-instantaneous on Linux even with tens of
thousands of files, really crawls on Windows.

So I can see an argument that Windows users would want arbitrary
"slices" much more often than Linux+git users, but I think this is
largely due to performance, not because people really *want* to be
stuck with a restricted view of the repo.

Have fun,

Avery

Re: Git drawbacks?

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2206 bytes --]

On Fri, 6 Nov 2009, Avery Pennarun wrote:

>>> This
>>> lousy performance isn't the case in git (except in Windows).  Are you
>>> using Windows, by chance?
>>
>> yes. I did not yet noticed any performance problems with Git on windows, except
>> a sync/download time (for android, mostly)
>
> Basically, performance is linear with the number of files in your
> repo.  If you can check out just a "slice" of your repo (say 10% of
> the whole), you'll have faster performance (eg. 10x) from any VCS.
>
> git on Linux is so fast that this isn't very necessary most of the
> time.  But git on Windows isn't really any faster than other VCSes on
> Windows, so the time-per-file is much greater, and thus the penalty
> for huge repositories is much worse.  Doing things like switching
> branches, which is near-instantaneous on Linux even with tens of
> thousands of files, really crawls on Windows.

but is that scale based on the number of files you are tracking, or the 
number of revisions that exist in the repository.

i.e.  10,000 files in the source code with 10 revisions each vs 1,000 
files with 100 revisions each.

my understanding of git is that it's the number of files, with very little 
impact due to having lots of revisions. so eliminating 90 revisions of 
each file would not significantly speed up git in the second case.

going back to the initial poster's comments. if the android repo is 1G, 
eliminating the history will probably have significantly less impact than 
you expect it to. for source code the compression factor that git is able 
to get is spectacular. I've seen several cases posted with large projects 
where the full repo with ALL history is <2x the size of a tar.gz of the 
latest release.

David Lang

> So I can see an argument that Windows users would want arbitrary
> "slices" much more often than Linux+git users, but I think this is
> largely due to performance, not because people really *want* to be
> stuck with a restricted view of the repo.
>
> Have fun,
>
> Avery
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Git drawbacks?

[Dmitry Smirnov]

Avery Pennarun <apenwarr <at> gmail.com> writes:

> There are three methods I know of to manage this:
> 
> 1) Just commit whatever version of a subproject you want as a subtree
> of your current project, and if you want to replace/delete/upgrade it,
> just do that.  (You rarely want to track the actual *history* of the
> third party tool, just the history of versions *you* used, which is
> easy to do.)

Ok. Does that mean that new component2 and common component1 should leave on the 
new branch (having in mind that old component2 and component1 are still living 
on previous branch)? So, how many efforts will I need to get both component1 
versions in sync (it is supposed that most of the changes in this component are 
common for both)? Is is supposed that having 2 branches for this component is 
cheaper (from development cycle POW)?

> 2) Use git-submodule to link repositories together.  (Arguably, one
> major reason 'repo' was written is that git-submodule is too
> complicated, though.)

> 3) Try my git-subtree tool, which basically makes it easier to
> split/join repositories (similar to #1) without losing the history
> (similar to #2).

I'll try to learn it.

I suppose, both these tools (repo and git-subtree) are the indication of some 
contradiction between  the tool and SCM practice (especially, for big projects).

> 
> Basically, performance is linear with the number of files in your
> repo.  If you can check out just a "slice" of your repo (say 10% of
> the whole), you'll have faster performance (eg. 10x) from any VCS.

Yes, I just wish to see this feature in some VCS. Why not Git? ;-)

> So I can see an argument that Windows users would want arbitrary
> "slices" much more often than Linux+git users, but I think this is
> largely due to performance, not because people really *want* to be
> stuck with a restricted view of the repo.

How offten do you use this info? I mean the whole stuff?

Re: Git drawbacks?

[Dmitry Smirnov]

 <david <at> lang.hm> writes:

> going back to the initial poster's comments. if the android repo is 1G, 
> eliminating the history will probably have significantly less impact than 
> you expect it to. 

Do you have 2 or more copies of the same repository at the same time?
If yes, can I skip cloning new copy from network? 
Or even skip cloning it at all? 
Is it possible with Git to chekout into two (few) working trees?

Re: Git drawbacks?

[Jacob Helwig]

On Sun, Nov 8, 2009 at 23:53, Dmitry Smirnov <divis1969@gmail.com> wrote:
>  <david <at> lang.hm> writes:
>
>> going back to the initial poster's comments. if the android repo is 1G,
>> eliminating the history will probably have significantly less impact than
>> you expect it to.
>
> Do you have 2 or more copies of the same repository at the same time?
> If yes, can I skip cloning new copy from network?
> Or even skip cloning it at all?
> Is it possible with Git to chekout into two (few) working trees?
>

Take a look at git-new-workdir.  It's in the contrib directory of
git.git.  This lets you skip re-cloning the same repository over
again, if you want a new working copy of it.  It'll also give you some
space savings, by sharing certain key things in the .git directory
between working copies, by using symlinks.

It does have a few caveats, however.  If you have the same branch
checked out in two different working copies created using
git-new-workdir, and update the branch in one of them, then the other
will appear to have a bunch of staged changes, even though you haven't
done anything in it. The branch pointer will have been updated out
from underneath it, and it will be "confused".  As long as you
remember not to update a branch that is checked out in more than one
place on your machine, you'll never notice, thuogh.

Re: Git drawbacks?

[Dmitry Potapov]

On Mon, Nov 09, 2009 at 07:53:24AM +0000, Dmitry Smirnov wrote:
>  <david <at> lang.hm> writes:
> 
> > going back to the initial poster's comments. if the android repo is 1G, 
> > eliminating the history will probably have significantly less impact than 
> > you expect it to. 
> 
> Do you have 2 or more copies of the same repository at the same time?
> If yes, can I skip cloning new copy from network? 
> Or even skip cloning it at all? 
> Is it possible with Git to chekout into two (few) working trees?

Jacob has already mentioned git-new-workdir from Git contrib, but
there are other ways to do the same or almost the same....

First of all, you can always copy your directory and thus creating
another clone. It is very simple and straightforward solution, but
it takes extra space due an extra copy of the repository. Usually,
it is not a big issue in practice, because your working tree tends
to be larger than the repository itself...

However, if you want to save disk space, you can use local clone. When
you clone your (like: git clone old_dir new_dir), git tries to use hard
links if it is possible. So, it may save disk space. However, if you
repack your original repo then a new pack will be created, and saving
from using the hard link will be lost.  To prevent that from happening,
you can tell to the garbage collector to keep the main existing pack by
create a file that has the same name as the pack file plus the .keep
suffix:

   touch .git/objects/pack-<SHA-1>.keep

then all changes will be put into a separate pack.

There is one more way to save disk space is to use git clone --shared.
It does not require hard links, but it has some caveats. If you want to
use it, then read the documentation carefully and make sure you understand
all implications.


Actually, in most use cases, there is no reason to have more than one
working tree. Git is designed to work well with plenty branches and one
working tree. So, switching between two branches and recompiling a few
changed files is much faster then going to another directory and try to
work there, because when you go to another directory, you may hit cold
cache and disk is *slow*... Another thing is that you can do a lot of
things without checking out some branch. You can grep any revision in
your repository, you can insect any file from it, etc and you do not
have to checkout this revision in your working tree.


Dmitry

Re: Git drawbacks?

[Dmitry Smirnov]

Jacob Helwig <jacob.helwig <at> gmail.com> writes:

> It does have a few caveats, however.  If you have the same branch
> checked out in two different working copies created using
>...
> place on your machine, you'll never notice, thuogh.


So, i cannot recover from this situation by upmerging to the new head?

Re: Git drawbacks?

[Dmitry Smirnov]

Dmitry Potapov <dpotapov <at> gmail.com> writes:


> Actually, in most use cases, there is no reason to have more than one
> working tree. Git is designed to work well with plenty branches and one
> working tree. So, switching between two branches and recompiling a few
> changed files is much faster then going to another directory and try to
> work there, because when you go to another directory, you may hit cold
> cache and disk is *slow*... Another thing is that you can do a lot of
> things without checking out some branch. You can grep any revision in
> your repository, you can insect any file from it, etc and you do not
> have to checkout this revision in your working tree.

Shouldn't I even worry about my not yet commited changes before switching the 
branch?

I would say that this approach does not work if the build and test could take
significant time. While in CR fix I don't want to wait for a build to complete
before I countinue with another bug/fix. That is why I'm curious about 
few working trees...

Re: Git drawbacks?

[Jacob Helwig]

On Mon, Nov 9, 2009 at 07:59, Dmitry Smirnov <divis1969@gmail.com> wrote:
> Jacob Helwig <jacob.helwig <at> gmail.com> writes:
>
>> It does have a few caveats, however.  If you have the same branch
>> checked out in two different working copies created using
>>...
>> place on your machine, you'll never notice, thuogh.
>
>
> So, i cannot recover from this situation by upmerging to the new head?
>

If you haven't actually made any changes in the second checkout of the
branch, then recovery is a simple reset --hard.  If you have made any
changes in that working copy, then recovery is a bit more complicated,
and it's probably best if I leave that explanation up to someone else,
since I'm very careful to avoid this situation in the first place, and
have never actually had to recover from it.

Re: Git drawbacks?

[Dmitry Potapov]

On Mon, Nov 09, 2009 at 04:11:48PM +0000, Dmitry Smirnov wrote:
> Dmitry Potapov <dpotapov <at> gmail.com> writes:
> 
> 
> > Actually, in most use cases, there is no reason to have more than one
> > working tree. Git is designed to work well with plenty branches and one
> > working tree. So, switching between two branches and recompiling a few
> > changed files is much faster then going to another directory and try to
> > work there, because when you go to another directory, you may hit cold
> > cache and disk is *slow*... Another thing is that you can do a lot of
> > things without checking out some branch. You can grep any revision in
> > your repository, you can insect any file from it, etc and you do not
> > have to checkout this revision in your working tree.
> 
> Shouldn't I even worry about my not yet commited changes before switching the 
> branch?

You probably want to use 'git stash save' and when you return back you
just do 'git stash pop'. Also, keep in mind that you can amend any
previous commit using 'git commit --amend'.

> 
> I would say that this approach does not work if the build and test could take
> significant time.

Yes, but then I do not see any reason to do any time consuming building
and testing in the working tree. I create a snapshot of the interesting
version using 'git archive' and then run build&test on it... In this
way, I can make sure that the archive I deliver is tested properly. If
you do your testing in the working tree, sometimes uncommitted or some
other files that are left over from previous builds may affect result.
So, if it takes considerable time anyhow, why do not do clean build and
test? And if you worry about compilation time, you can use ccache.


Dmitry

Re: Git drawbacks?

[B Smith-Mannschott]

On Mon, Nov 9, 2009 at 17:11, Dmitry Smirnov <divis1969@gmail.com> wrote:
> Dmitry Potapov <dpotapov <at> gmail.com> writes:
>
>
>> Actually, in most use cases, there is no reason to have more than one
>> working tree. Git is designed to work well with plenty branches and one
>> working tree. So, switching between two branches and recompiling a few
>> changed files is much faster then going to another directory and try to
>> work there, because when you go to another directory, you may hit cold
>> cache and disk is *slow*... Another thing is that you can do a lot of
>> things without checking out some branch. You can grep any revision in
>> your repository, you can insect any file from it, etc and you do not
>> have to checkout this revision in your working tree.
>
> Shouldn't I even worry about my not yet commited changes before switching the
> branch?

commit them before you switch. you could:

- commit them to the current branch before you switch the branch.
- commit them to a new branch before you switch
- use git stash to move your changes aside.

> I would say that this approach does not work if the build and test could take
> significant time. While in CR fix I don't want to wait for a build to complete
> before I countinue with another bug/fix. That is why I'm curious about
> few working trees...

You don't have to wait to comitting to your own branches, but do make
sure to run your usual builds and tests before pushing or asking
another to pull changes from you.

// Ben

Re: Git drawbacks?

[Dmitry Potapov]

On Mon, Nov 09, 2009 at 07:47:08PM +0100, B Smith-Mannschott wrote:
> 
> You don't have to wait to comitting to your own branches, but do make
> sure to run your usual builds and tests before pushing or asking
> another to pull changes from you.

Yes, it is one most useful feature of Git that you can commit (save)
your changes immediately but amend them later. This helps a lot to
make changes smaller, cleaner and easier to review.

With many other VCS, a typical policy is that you do not commit your
changes unless you have finished and tested them. But it means that
your changes are not committed and stored only in the work tree for
a long time. Moreover, when you eventually decide that they are good
enough to commit, you will produce a huge patch, which will be difficult
to review or to bisect history later.

With Git you do not have to worry about testing when you can commit your
changes. Typically I would commit some my changes as I progress to my
goal, but later I will review all commits. Probably, squash some changes
with fixes, clean up some other, add better explanations of what is done
and why, etc... But I do not have to worry about all those trifles as
I write code to see if some feature is worth or not, if this solution
works or I should try something else...

So, you can always commit your changes as your progress to your goal and
review amend them later before publishing. This means that you can have
as many work-in-progress branches as you wish, and you do not need a
separate work tree for each of them -- everything can be stored in the
repository, and you can go to another computer, issue 'git fetch' and
continue your work at the exact point where you left it. So, it is very
flexible.


Dmitry

Re: Git drawbacks?

[Dmitry Smirnov]

Dmitry Potapov <dpotapov <at> gmail.com> writes:
 
> Yes, but then I do not see any reason to do any time consuming building
> and testing in the working tree. I create a snapshot of the interesting
> version using 'git archive' and then run build&test on it... In this
> way, I can make sure that the archive I deliver is tested properly. If
> you do your testing in the working tree, sometimes uncommitted or some
> other files that are left over from previous builds may affect result.
> So, if it takes considerable time anyhow, why do not do clean build and
> test? And if you worry about compilation time, you can use ccache.

It is not clear for me. Yes, I have to get some fixed version to reproduce
the bug reported by someone. Then I need to fix it and commit the change 
back (on the head). Also, it is obvious to reproduce the issue and 
test the fix on the tip. Can do this with 'git archive'?
BTW, doesn't 'git archive' sync to some version that I probably already 
have in other clone? ;-)

Re: Git drawbacks?

[Dmitry Smirnov]

Dmitry Potapov <dpotapov <at> gmail.com> writes:

> With many other VCS, a typical policy is that you do not commit your
> changes unless you have finished and tested them. But it means that
> your changes are not committed and stored only in the work tree for
> a long time. Moreover, when you eventually decide that they are good
> enough to commit, you will produce a huge patch, which will be difficult
> to review or to bisect history later.

oh, yes. but this is just a policy. You can make your changes on your 
branch and commit them (for example, for review). Later someone just 
need to integrate it on original branch. The same as with Git, 
isn't it? The problem is just a price to branch.
BTW, once I started to talk about review, we can see that most 
"benefits" of DVCS go away... Just because you still need some 
central storage to save the record of this review that should 
be available for SQA later...
 
> So, you can always commit your changes as your progress to your goal and
> review amend them later before publishing. This means that you can have
> as many work-in-progress branches as you wish, and you do not need a
> separate work tree for each of them -- everything can be stored in the
> repository, and you can go to another computer, issue 'git fetch' and
> continue your work at the exact point where you left it. So, it is very
> flexible.

As for me, I would not to have more than 4-5 such deferred changes in the same 
repository. Otherwise, I will be entangled finally :-)

Re: Git drawbacks?

[Dmitry Smirnov]

B Smith-Mannschott <bsmith.occs <at> gmail.com> writes:
 
> You don't have to wait to comitting to your own branches, but do make
> sure to run your usual builds and tests before pushing or asking
> another to pull changes from you.

Perhaps, I was not clear in my questions. 
I do not want to build and test to complete before I get back to my stashed
or commited changes. I.e. I need to have 2 working trees (perhaps different, but 
from the same repository): one for build/test and one for another task.

Dmitry

Re: Git drawbacks?

[Dmitry Potapov]

On Tue, Nov 10, 2009 at 08:31:48AM +0000, Dmitry Smirnov wrote:
> Dmitry Potapov <dpotapov <at> gmail.com> writes:
>  
> > Yes, but then I do not see any reason to do any time consuming building
> > and testing in the working tree. I create a snapshot of the interesting
> > version using 'git archive' and then run build&test on it... In this
> > way, I can make sure that the archive I deliver is tested properly. If
> > you do your testing in the working tree, sometimes uncommitted or some
> > other files that are left over from previous builds may affect result.
> > So, if it takes considerable time anyhow, why do not do clean build and
> > test? And if you worry about compilation time, you can use ccache.
> 
> It is not clear for me. Yes, I have to get some fixed version to reproduce
> the bug reported by someone. Then I need to fix it and commit the change 
> back (on the head). Also, it is obvious to reproduce the issue and 
> test the fix on the tip. Can do this with 'git archive'?
> BTW, doesn't 'git archive' sync to some version that I probably already 
> have in other clone? ;-)

I am not sure I understood your question. What 'git archive' does is to
create a tar or zip archive or any version that you specify. So, you can
use it to export the snapshot of any version to a temporary directory
for testing if this testing takes a noticeable time and you want to be
able to work on something else in meanwhile. If necessary, you can
create a temporary branch starting at the intesting point and to add
some commits there (such as a new test case to reproduce the problem)
and then run test on it. Later, you can either rebase or merge these
commits to any branch that needs them.

Actually, when someone reports a problem, I do not use 'git archive'
because I write a test case for this bug and run it alone, and it does
not take much time to run it. But periodically (a few times a day), I
run the full test suit, which takes considerable amount of time, so I
run this test outside of my working directory, using 'git archive'.


Dmitry

Re: Git drawbacks?

[Dmitry Potapov]

On Tue, Nov 10, 2009 at 08:51:53AM +0000, Dmitry Smirnov wrote:
> Dmitry Potapov <dpotapov <at> gmail.com> writes:
> 
> > With many other VCS, a typical policy is that you do not commit your
> > changes unless you have finished and tested them. But it means that
> > your changes are not committed and stored only in the work tree for
> > a long time. Moreover, when you eventually decide that they are good
> > enough to commit, you will produce a huge patch, which will be difficult
> > to review or to bisect history later.
> 
> oh, yes. but this is just a policy. You can make your changes on your 
> branch and commit them (for example, for review). Later someone just 
> need to integrate it on original branch. The same as with Git, 
> isn't it? The problem is just a price to branch.

With some VCSes, it is not just the cost of branch, but it is also name
pollution that happens when many branches created. But even the cost of
branch creation alone may be high enough to make creating small feature
branches impractical. You can try to force that on developers, but they
will be frustrated with that pretty soon... And then if you really want
to have good and clean history, you need more than just a branch. You
should be able to amend your previous commits while you work on some
feature. With Git, it is trivial, you just run 'git rebase -i' and may
edit some previous commits, correct comments, squash fix-ups, etc...
How can you model that? By creating another branch and moving patches
to it by hands... Well, it is not very productive time spending, and
the cost of branch becomes even more prominent.

> BTW, once I started to talk about review, we can see that most 
> "benefits" of DVCS go away... Just because you still need some 
> central storage to save the record of this review that should 
> be available for SQA later...

I do not see how any benefit go away because of having some central
storage. Most benefits are not due to that you do not have it, but due
to cheap branching, easy mirroring, and flexibility in what you want to
store and where...

>  
> > So, you can always commit your changes as your progress to your goal and
> > review amend them later before publishing. This means that you can have
> > as many work-in-progress branches as you wish, and you do not need a
> > separate work tree for each of them -- everything can be stored in the
> > repository, and you can go to another computer, issue 'git fetch' and
> > continue your work at the exact point where you left it. So, it is very
> > flexible.
> 
> As for me, I would not to have more than 4-5 such deferred changes in the same 
> repository. Otherwise, I will be entangled finally :-)

A quick look at "What's cooking in git.git" that Junio posted a few days
ago reveals that there are at least 43 branches that are cooking now and
the total number of branches that have been merged to 'master' over all
Git history is 3290. And Git is not the largest project out there...


Dmitry

Re: Git drawbacks?

[Dmitry Smirnov]

Dmitry Potapov <dpotapov <at> gmail.com> writes:

> I am not sure I understood your question. What 'git archive' does is to
> create a tar or zip archive or any version that you specify. So, you can
> use it to export the snapshot of any version to a temporary directory
> for testing if this testing takes a noticeable time and you want to be
> able to work on something else in meanwhile. If necessary, you can
> create a temporary branch starting at the intesting point and to add
> some commits there (such as a new test case to reproduce the problem)
> and then run test on it. Later, you can either rebase or merge these
> commits to any branch that needs them.

Ok, I got it. So the scenario is something like that:
1. Stash or commit my current changes
2. Sync (reset or checkout) to some other version/branch
3. Create temporary branch 
4. Edit and Commit
5. "git archive" to get snapshot
6. Get back to original change

This looks reasonable. Until you need to correct the fix and test it again.

Perhaps, I have to explain why fix/test cycle is so long. I'm working on mobile 
platform development. It is not so modular, so I need to build the whole image, 
then flash the mobile and test it in a real network. This takes much time and 
need much disk space (about 2GB of source files. In fact, significant number of 
these files are not needed for the build, but it is too hard to force CM 
engineers to fix this :-) )

Re: Git drawbacks?

[Dmitry Smirnov]

Dmitry Potapov <dpotapov <at> gmail.com> writes:

> And then if you really want
> to have good and clean history, you need more than just a branch. You
> should be able to amend your previous commits while you work on some
> feature. With Git, it is trivial, you just run 'git rebase -i' and may
> edit some previous commits, correct comments, squash fix-ups, etc...
> How can you model that? By creating another branch and moving patches
> to it by hands... Well, it is not very productive time spending, and
> the cost of branch becomes even more prominent.

This is a cool feature, but it contradicts to my understanding of VCS. 
It is some kind of re-writing the history of WWII :-) 
BTW, as I undestood it, it is just a feature that can be implemented 
in any VCS (if you have access to its internals).

> A quick look at "What's cooking in git.git" that Junio posted a few days
> ago reveals that there are at least 43 branches that are cooking now and
> the total number of branches that have been merged to 'master' over all
> Git history is 3290. And Git is not the largest project out there...

I meant 4-5 per person (me, precisely speaking)

Re: Git drawbacks?

[Paolo Bonzini]

> Perhaps, I have to explain why fix/test cycle is so long. I'm working on mobile
> platform development. It is not so modular, so I need to build the whole image,
> then flash the mobile and test it in a real network. This takes much time and
> need much disk space (about 2GB of source files. In fact, significant number of
> these files are not needed for the build, but it is too hard to force CM
> engineers to fix this :-) )

In that case I think using multiple working trees, but sharing the pack 
files is a good idea.

Paolo

Re: Git drawbacks?

[Paolo Bonzini]

On 11/10/2009 03:54 PM, Dmitry Smirnov wrote:
> This is a cool feature, but it contradicts to my understanding of VCS.

Not really.  As long as the history is not public, nobody will notice.

It is basically the same as using a centralized svn repository 
(corresponding to public branches) and quilt for private patch queues. 
You can push and pop patches as much as you want in quilt, and you can 
rewrite history as much as you want in git private branches.

Now I'm not very proficient with quilt, but if I understood it right, 
git is much easier to do because you use only one tool and thus you can 
rely on the same algorithms for merging and conflict reporting also in 
the case of private branches.  I've certainly done some insane 
reordering of patches and it worked like a charm; also, git keeps a log 
of the states so it takes a single "git diff" invocation to check that 
the end result of a reorganization is the same as what you started from.

Paolo

Re: Git drawbacks?

[Dmitry Potapov]

On Tue, Nov 10, 2009 at 02:54:43PM +0000, Dmitry Smirnov wrote:
> Dmitry Potapov <dpotapov <at> gmail.com> writes:
> 
> > And then if you really want
> > to have good and clean history, you need more than just a branch. You
> > should be able to amend your previous commits while you work on some
> > feature. With Git, it is trivial, you just run 'git rebase -i' and may
> > edit some previous commits, correct comments, squash fix-ups, etc...
> > How can you model that? By creating another branch and moving patches
> > to it by hands... Well, it is not very productive time spending, and
> > the cost of branch becomes even more prominent.
> 
> This is a cool feature, but it contradicts to my understanding of VCS. 

Yes, it may *appear* to contradict one of basic axioms of any VCS --
history is immutable, but here is the rule -- you never ever rebase or
change in any other way the public history of the official repository.
So once your commits hit the master branch, you should never try to
change them. This may be enforced through Git hooks.

However, when we speak about history of commits in your private feature
branch then no one cares about how you have arrived to these series of
patches more than in what order you have edited your files. What really
matters is the end result -- our patches are clean and easy to review.

Now, what is 'public' history or what is not. If no one has seen your
commits then it is clearly not public. If your commits are integrated
to the main development branch ('master') then they are clearly public.
However, many Git users have an international branch (often called as
proposed updates or 'pu' for short). The 'pu' branch is regularly
rewound, so no one should based their work on it. It is more for review
and additional testing of things that may not ready yet. When your
branch is merged to 'pu', it considered to be a fair play to re-write
your patches. However, you should do that in the way that will not
cause problems for people who review your changes.

Finally, when you re-write some branch using interactive rebase or
something else. Your old changes do not disappear immediately from your
repository. Git has 'reflog', which keeps _all_ commits in your local
repository for 30 days (or whatever time your choose). So, if you did
something wrong during rebase, you can always go back to the original
version, though you will hardly ever need that feature unless you do
something really stupid...


> BTW, as I undestood it, it is just a feature that can be implemented 
> in any VCS (if you have access to its internals).

At least, not so simple. Have you heard about Mercurial? It is another
DVCS, which in many aspects similar to Git, but the underlying backend
is very different. I do not follow it very closely, but for a long time
it did not have 'rebase' and users were advised to use Mercurial Queues
instead, which is patch management system on top of Mercurial. While Git
has its own implementations of patch management systems on top of Git
such as StGit and recently TopGit, I do not think they come even close
when it comes to easy to use in everyday work, especially if you do not
want to be bothered with thinking about patch management.

Much power of Git and its flexibility comes from clean separation of the
underlying storage format and repository history.  This makes 'rebase'
almost trivial in Git (unless you have merges in rebased history and you
try to preserve them), while with many other VCSes, those changes will
require significant changes to the underlying storage, which may be
difficult to implement safely and efficiently. The price that Git has to
pay for its flexibility is the need to run the garbage collection to
purge loose objects and compact all objects in one compressed pack...


Dmitry

Re: Git drawbacks?

[Dmitry Smirnov]

Dmitry Smirnov <divis1969 <at> gmail.com> writes:


Ok, I have heard a lot of perfect words about Git. I'm almost on your side :-)

Just need some advice on (small for Git, I hope) problem. 
I'm trying to import P4 depot into Git (for mirroring purpose).
It seems a non-trivial task with the current git-p4 script.
Perhaps, I had selected a wrong way: i'm trying to import some client.

In fact, as I said in previous mails, typically I have few similar clients.
So, maybe it is simpler to import some set of branches (which I suppose a little 
bit simpler with git-p4).

Unfortunaley, the directory structure of the depot differs from client's (i.e. 
working tree differs from repository tree).
For example,
//depot/component/version could be mapped to the 
<root_of_working_tree>/component.

Thus if I import //depot/component/version1 and //depot/component/version2
as is, I should be able to checkout either version1 or version2.
Note that there could be few components in the same working tree:

//depot/component1/version1 (mapped to <root>/component1)
//depot/component2/versionX (mapped to <root>/component2)
//depot/component3/versionY (mapped to <root>/component3)

With Perforce, there could also be a more complex mappings, but maybe we will 
discuss it later).

Is there any way to make this mapping with Git? Should I invent some kind of 
tool like 'repo' or there is a simpler way?


Dmitry