Branches & directories

Branches & directories

[Hilco Wijbenga]

Hi all,

I have been noticing strange behaviour that I would like to be able to
explain or report as a bug as the case may be.

What happens is that I create and commit a new directory in branch
'next' and then when I checkout 'master' this new directory is still
there. I think this is wrong as this new directory does not exist yet
in 'master'. Is my understanding correct?

I tried recreating this scenario in a clean Git repo with a simple
mkdir and commit but when I did a checkout of 'master' the new
directory was removed. So the basic scenario seems to work the way I
expect it to.

Assuming I ran into a bug, I would like some suggestions to properly
investigate this. Clearly, I'm doing something else that triggers the
behaviour I'm seeing but I'm not sure what it is. What might trigger
Git "remembering" a directory? Or what would prevent it from removing
a directory when checking out a different branch?

Extra information: "git status" (in 'master') yields nothing. But
after adding a new file in the directory-that-should-not-be-there, Git
treats the entire directory as untracked and new (as one would
expect). I can also safely remove the directory with no (obvious) ill
effects.

Cheers,
Hilco

Re: Branches & directories

[Evan Shelhamer]

Hey Hilco,

I'm not sure exactly what you did because you didn't give a list of
git commands, but I'm guessing you ran into the fact that git doesn't
track empty directories.
For instance if I do:

(in repo on your branch)
mkdir test_dir
touch test_file
git add *
git commit -m "test commit"

test_file will be tracked but test_dir will not (because it is empty).
To avoid this problem, a convention is to add an empty ".gitkeep" file
to directories. For example:

(in repo on your branch)
mkdir test_dir
touch test_dir/.gitkeep
touch test_file
git add *
git commit -m "test commit (with directory)"

Will commit the directory as expected in your branch, and when you go
to checkout another branch it will not exist.

Hope that helps. Sorry if you already know this and I misunderstood
your question.
Evan Shelhamer

On Wed, Aug 17, 2011 at 2:35 PM, Hilco Wijbenga
<hilco.wijbenga@gmail.com> wrote:
> Hi all,
>
> I have been noticing strange behaviour that I would like to be able to
> explain or report as a bug as the case may be.
>
> What happens is that I create and commit a new directory in branch
> 'next' and then when I checkout 'master' this new directory is still
> there. I think this is wrong as this new directory does not exist yet
> in 'master'. Is my understanding correct?
>
> I tried recreating this scenario in a clean Git repo with a simple
> mkdir and commit but when I did a checkout of 'master' the new
> directory was removed. So the basic scenario seems to work the way I
> expect it to.
>
> Assuming I ran into a bug, I would like some suggestions to properly
> investigate this. Clearly, I'm doing something else that triggers the
> behaviour I'm seeing but I'm not sure what it is. What might trigger
> Git "remembering" a directory? Or what would prevent it from removing
> a directory when checking out a different branch?
>
> Extra information: "git status" (in 'master') yields nothing. But
> after adding a new file in the directory-that-should-not-be-there, Git
> treats the entire directory as untracked and new (as one would
> expect). I can also safely remove the directory with no (obvious) ill
> effects.
>
> Cheers,
> Hilco
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Branches & directories

[Junio C Hamano]

Evan Shelhamer <shelhamer@imaginarynumber.net> writes:

> (in repo on your branch)
> mkdir test_dir
> touch test_dir/.gitkeep
> touch test_file
> git add *
> git commit -m "test commit (with directory)"
>
> Will commit the directory as expected in your branch, and when you go
> to checkout another branch it will not exist.

... unless you have untracked files in test_dir/, that is.

If the branch you are switching to does not have anything at test_dir, you
will not lose the untracked file because git does not remove test_dir/ nor
its untracked contents.

Depending on what you have at the path "test_dir" in the branch you are
switching to, the actual rules are a bit more subtle than that, though.

Re: Branches & directories

[Hilco Wijbenga]

On 17 August 2011 12:14, Junio C Hamano <gitster@pobox.com> wrote:
> Evan Shelhamer <shelhamer@imaginarynumber.net> writes:
>
>> (in repo on your branch)
>> mkdir test_dir
>> touch test_dir/.gitkeep
>> touch test_file
>> git add *
>> git commit -m "test commit (with directory)"
>>
>> Will commit the directory as expected in your branch, and when you go
>> to checkout another branch it will not exist.
>
> ... unless you have untracked files in test_dir/, that is.
>
> If the branch you are switching to does not have anything at test_dir, you
> will not lose the untracked file because git does not remove test_dir/ nor
> its untracked contents.
>
> Depending on what you have at the path "test_dir" in the branch you are
> switching to, the actual rules are a bit more subtle than that, though.

Yes, I should have thought of that. The problem is with .gitignore. I
have a number of files ignored that are important but can't be shared.
So when I switch to 'master' the directory is not removed because it
contains untracked, ignored files. And, of course, git status does not
complain about them either because they are in .gitignore. :-)

This is still very annoying but hardly Git's fault (Git is working as
intended). It would be really nice, though, if Git could somehow
"stash" such files when checking out a different branch. In general, I
would prefer if uncommitted changes and untracked and/or ignored files
stuck to the branch where they were created. If I want to take them
with me when I switch branches that should be possible but not the
default behaviour. Is anything like that possible? Are there
configuration options for this?

Re: Branches & directories

[Jonathan Nieder]

Hi Hilco,

Hilco Wijbenga wrote:

> It would be really nice, though, if Git could somehow
> "stash" such files when checking out a different branch. In general, I
> would prefer if uncommitted changes and untracked and/or ignored files
> stuck to the branch where they were created.

This is just a random guess, but: wouldn't it be convenient in this
workflow to have a separate worktree for each branch you are working
on?  That way, switching branches would not carry over unwanted state
or throw away valuable state, clobber timestamps that "make" pays
attention to, etc.

If I am understanding correctly, then the git-new-workdir script
from contrib/workdir might help.  (Note, though, that it comes with
some caveats.  A quick mailing list search should find them.)

Re: Branches & directories

[Hilco Wijbenga]

Hi Jonathan,

On 17 August 2011 21:45, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi Hilco,
>
> Hilco Wijbenga wrote:
>
>> It would be really nice, though, if Git could somehow
>> "stash" such files when checking out a different branch. In general, I
>> would prefer if uncommitted changes and untracked and/or ignored files
>> stuck to the branch where they were created.
>
> This is just a random guess, but: wouldn't it be convenient in this
> workflow to have a separate worktree for each branch you are working
> on?  That way, switching branches would not carry over unwanted state
> or throw away valuable state, clobber timestamps that "make" pays
> attention to, etc.
>
> If I am understanding correctly, then the git-new-workdir script
> from contrib/workdir might help.  (Note, though, that it comes with
> some caveats.  A quick mailing list search should find them.)

Yes, both a separate clone and git-new-workdir (which I've just
started using) would work.

I'm not entirely happy with this solution, though. It means having to
create an Eclipse workspace per branch, this is a *lot* of work. (This
is mostly Eclipse's fault but still.) One of the exceedingly
attractive features of Git is the easy branching and merging.
Branching is now no longer easy. :-(

I assume it would be possible for Git to move untracked/ignored files
out of the way when checking out a different branch? (And moving them
back in when going back to that branch.) Are there any objections to
doing this? I.e. would it be a bad idea for some reason?

So, in essence, would it be possible (and desirable) to associate
untracked/ignored files with a particular branch? And only show them
when that branch is checked out?

Re: Branches & directories

[Michael Witten]

On Wed, Aug 17, 2011 at 21:23, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> It would be really nice, though, if Git could somehow
> "stash" such files when checking out a different branch. In general, I
> would prefer if uncommitted changes and untracked and/or ignored files
> stuck to the branch where they were created.

As an aside, the problem here is likely a manifestation of the fact
that nobody understands what a branch is; the word 'branch' is
TERRIBLE, as everyone has a different idea for what that should mean.
In my opinion, `git branch' should become `git ref' or the like.

One of git's worst faults is that a complicated and imprecise
interface has been draped over a very simple and precise underlying
structure.

Re: Branches & directories

[Michael J Gruber]

Michael Witten venit, vidit, dixit 18.08.2011 07:52:
> On Wed, Aug 17, 2011 at 21:23, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> It would be really nice, though, if Git could somehow
>> "stash" such files when checking out a different branch. In general, I
>> would prefer if uncommitted changes and untracked and/or ignored files
>> stuck to the branch where they were created.
> 
> As an aside, the problem here is likely a manifestation of the fact
> that nobody understands what a branch is; the word 'branch' is

I would reject "nobody"...

> TERRIBLE, as everyone has a different idea for what that should mean.

... and insist that this statement is true either trivially true for all
words, or for none, depending on your understanding of "everyone has a
different".

> In my opinion, `git branch' should become `git ref' or the like.

"branch" and "tag" are boths refs. Their only essential difference is
that one "moves" and the other doesn't.

> One of git's worst faults is that a complicated and imprecise
> interface has been draped over a very simple and precise underlying
> structure.

A name is a name and just that. The use of any existing word may clash
with someone's expectations.

I find the concepts "file created on a branch", "commit created on a
branch" silly, it's part of what drove me from hg to git early on. A git
"branch" is an hg "bookmark" these days (a named "head"), and if that
name triggers the right associations for some people, its best used in
explanations for those. git's branches do exactly what I (and many
others) expect branches to do and what I need daily, even coming from a
svn and hg background.

Michael

Re: Branches & directories

[Michael Witten]

On Thu, Aug 18, 2011 at 10:56, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> Michael Witten venit, vidit, dixit 18.08.2011 07:52:
>> On Wed, Aug 17, 2011 at 21:23, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>> It would be really nice, though, if Git could somehow
>>> "stash" such files when checking out a different branch. In general, I
>>> would prefer if uncommitted changes and untracked and/or ignored files
>>> stuck to the branch where they were created.
>>
>> As an aside, the problem here is likely a manifestation of the fact
>> that nobody understands what a branch is; the word 'branch' is
>
> I would reject "nobody"...

Is it really necessary to attack something that is obviously
hyperbolic rhetoric? The point is that a lot of people struggle with
git's concepts, and I think that the reason is largely two fold:

  * Terminology that is inaccurate by virtue of terminological baggage.
  * Terminology that is imprecisely used across the documentation.

>> TERRIBLE, as everyone has a different idea for what that should mean.
>
> ... and insist that this statement is true either trivially true for all
> words, or for none, depending on your understanding of "everyone has a
> different".

So, precision is impossible?

The word 'branch' is clearly a sticking point, and you know it. You
can't deny that people have a hard time with what a branch really is
in git.

>> In my opinion, `git branch' should become `git ref' or the like.
>
> "branch" and "tag" are boths refs. Their only essential difference is
> that one "moves" and the other doesn't.

Indeed, and it seems like there's room for abstraction; a tool like
`git ref' or `git pointer' or `git ptr' could probably handle both
quite well.

Terminology for references is a problem that was solved by the
Ancients, so it's a wonder we didn't use their work (pointers,
constant pointers, dereferencing, etc.).

>> One of git's worst faults is that a complicated and imprecise
>> interface has been draped over a very simple and precise underlying
>> structure.
>
> A name is a name and just that. The use of any existing word may clash
> with someone's expectations.

Yeah? So why wasn't the term 'foobar' used instead of 'branch'? Most
terms are chosen for their associations (and only really novel
concepts receive an entirely new label), and I proffer that 'branch'
has too many associations to be considered accurate enough.

The idea was that one line of development can 'branch' off from
another, so it was called a 'branch'. But what is a line of
development? (merges certainly make the concept fuzzy; tree branches
usually don't join with the tree trunk again). The thing is, though,
when you're working with what git calls a 'branch', you're really just
working with a pointer to a commit object; the term 'pointer' would
have been much more accurate.

Now, I'm not saying we should be using 'pointer', but what I am saying
is that it's necessary to choose words carefully, preferably in a way
that carries just enough associations to aid the user in remembering
what a term describes.

> I find the concepts "file created on a branch", "commit created on a
> branch" silly, it's part of what drove me from hg to git early on.

There's no shortage of that kind of thinking in the git community,
which is my point. The reason is a failure of both accurate
terminology and precise usage.

> git's branches do exactly what I (and many others) expect branches
> to do and what I need daily, even coming from a svn and hg background.

Yes, well, many figure it out EVENTUALLY; it just requires them to
ignore most of the associations of 'branch', thereby tacitly
translating the word 'branch' into 'pointer'.

Re: Branches & directories

[Jonathan Nieder]

Hi Michael,

Michael Witten wrote:

> As an aside, the problem here is likely a manifestation of the fact
> that nobody understands what a branch is

I agree with Michael J Gruber that this is not the useful kind of
hyperbole.

In olden times, there was more talk of the refs in the refs/heads/
hierarchy as being branch heads which point to the tip of a particular
branch of history.  That is still exactly what they are.  By abuse of
language, people sometimes use the term "branch" as a synonym when
there is little risk of confusion.

If you have found tutorial documentation that is confusing on this
point, by all means, fix it.  You are talking as though a
documentation bug is a fundamental design flaw that cannot be fixed by
the likes of you and me.

Re: Branches & directories

[Hilco Wijbenga]

On 17 August 2011 22:52, Michael Witten <mfwitten@gmail.com> wrote:
> On Wed, Aug 17, 2011 at 21:23, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> It would be really nice, though, if Git could somehow
>> "stash" such files when checking out a different branch. In general, I
>> would prefer if uncommitted changes and untracked and/or ignored files
>> stuck to the branch where they were created.
>
> As an aside, the problem here is likely a manifestation of the fact
> that nobody understands what a branch is; the word 'branch' is
> TERRIBLE, as everyone has a different idea for what that should mean.
> In my opinion, `git branch' should become `git ref' or the like.

Are you saying that Git's branches are different from branches in
other SCMs? How does my understanding of "branch" hinder me in the
above scenario? Am I doing something wrong?

Isn't a branch simply a way to track changes separately?

Re: Branches & directories

[Michael Witten]

On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
<hilco.wijbenga@gmail.com> wrote:
> Isn't a branch simply a way to track changes separately?

Well, what does that mean, really? You can certainly use branches to
help you achieve that goal.

In git usage, a `branch' is just a human-readable name for any given
commit object; it points to a commit object, and you can change to
which commit it points. Furthermore, to help you work with commit
lineages, some of the git machinery updates these branches (or
`pointers', if you like) automatically (for instance, when you make a
new commit object with `git commit', then the `current branch' is
updated to point to the newly created commit object).

Of course, 2 different branches may be used to point to the same commit object.

You should really think of your repository as a giant web of commit
objects (or, more technically, as a directed acyclic graph where each
node is a commit object); a commit object can point 'backwards'
towards its parent commit objects. A branch (like `master') just
points to one of these commit objects at any given time (that is, a
branch just gives a nice human-readable label by which to reference
one of these commit objects at any given time).

See here too:

  http://slashdot.org/comments.pl?sid=2350536&cid=36903136

Re: Branches & directories

[Hilco Wijbenga]

On 21 August 2011 13:53, Michael Witten <mfwitten@gmail.com> wrote:
> On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
> <hilco.wijbenga@gmail.com> wrote:
>> Isn't a branch simply a way to track changes separately?
>
> Well, what does that mean, really? You can certainly use branches to
> help you achieve that goal.

It means my commits are chained together separate from, say, master.

> In git usage, a `branch' is just a human-readable name for any given
> commit object; it points to a commit object, and you can change to
> which commit it points. Furthermore, to help you work with commit
> lineages, some of the git machinery updates these branches (or
> `pointers', if you like) automatically (for instance, when you make a
> new commit object with `git commit', then the `current branch' is
> updated to point to the newly created commit object).
>
> Of course, 2 different branches may be used to point to the same commit object.
>
> You should really think of your repository as a giant web of commit
> objects (or, more technically, as a directed acyclic graph where each
> node is a commit object); a commit object can point 'backwards'
> towards its parent commit objects. A branch (like `master') just
> points to one of these commit objects at any given time (that is, a
> branch just gives a nice human-readable label by which to reference
> one of these commit objects at any given time).

Yes, I agree with all of that (especially the last sentence). I don't
see how it changes the concept of a branch is, though. You are, I
think, simply listing the technical implementation.

> See here too:
>
>  http://slashdot.org/comments.pl?sid=2350536&cid=36903136

Because of the revision number you have to go "the wrong way". Any
single commit can have many children but a branch only points to a
single, unique commit (or at least that's how I understand it).

I feel like we're talking in circles. I get (and even agree with) what
you're saying but I don't see how it changes the concept of a branch.

In any case, what I'm more interested in is knowing whether we can
(optionally) add state (i.e. untracked/ignored files and unstaged
changes) to a branch.

Re: Branches & directories

[Michael Witten]

On Sun, Aug 21, 2011 at 21:37, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> On 21 August 2011 13:53, Michael Witten <mfwitten@gmail.com> wrote:
>> On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
>> <hilco.wijbenga@gmail.com> wrote:
>>> Isn't a branch simply a way to track changes separately?
>>
>> Well, what does that mean, really? You can certainly use branches to
>> help you achieve that goal.
>
> It means my commits are chained together separate from, say, master.

Well, that's not what a git branch provides in general.

>> In git usage, a `branch' is just a human-readable name for any given
>> commit object; it points to a commit object, and you can change to
>> which commit it points. Furthermore, to help you work with commit
>> lineages, some of the git machinery updates these branches (or
>> `pointers', if you like) automatically (for instance, when you make a
>> new commit object with `git commit', then the `current branch' is
>> updated to point to the newly created commit object).
>>
>> Of course, 2 different branches may be used to point to the same commit object.
>>
>> You should really think of your repository as a giant web of commit
>> objects (or, more technically, as a directed acyclic graph where each
>> node is a commit object); a commit object can point 'backwards'
>> towards its parent commit objects. A branch (like `master') just
>> points to one of these commit objects at any given time (that is, a
>> branch just gives a nice human-readable label by which to reference
>> one of these commit objects at any given time).
>
> Yes, I agree with all of that (especially the last sentence). I don't
> see how it changes the concept of a branch is, though. You are, I
> think, simply listing the technical implementation.

No, I'm simply listing the reality of what a branch is.

> ...
>
> I feel like we're talking in circles. I get (and even agree with) what
> you're saying but I don't see how it changes the concept of a branch.
>
> In any case, what I'm more interested in is knowing whether we can
> (optionally) add state (i.e. untracked/ignored files and unstaged
> changes) to a branch.

No, because a branch doesn't IN ANY WAY provide the structure for that
kind of thing. Of course, you could use what git calls a 'branch' in
order to implement what you imply is a 'branch', but git's concept of
a branch and your concept of a branch are not at all the same concept
(which is why the term 'branch' is so unfortunate).

Re: Branches & directories

[Hilco Wijbenga]

On 21 August 2011 16:06, Michael Witten <mfwitten@gmail.com> wrote:
> On Sun, Aug 21, 2011 at 21:37, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> On 21 August 2011 13:53, Michael Witten <mfwitten@gmail.com> wrote:
>>> On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
>>> <hilco.wijbenga@gmail.com> wrote:
>>>> Isn't a branch simply a way to track changes separately?
>>>
>>> Well, what does that mean, really? You can certainly use branches to
>>> help you achieve that goal.
>>
>> It means my commits are chained together separate from, say, master.
>
> Well, that's not what a git branch provides in general.

Er, so what *does* a Git branch provide then?

>> I feel like we're talking in circles. I get (and even agree with) what
>> you're saying but I don't see how it changes the concept of a branch.
>>
>> In any case, what I'm more interested in is knowing whether we can
>> (optionally) add state (i.e. untracked/ignored files and unstaged
>> changes) to a branch.
>
> No, because a branch doesn't IN ANY WAY provide the structure for that
> kind of thing.

Obviously, we'd need to expand that structure.

I tried (ab)using git stash to get what I want but it ignores
untracked/ignored files (not a big surprise, of course). It seems the
functionality is almost there. If I could just combine git checkout
with git stash (and have it work with untracked/ignored files) in a
script or alias, I'd be a happy camper. I'll have to give it some more
thought.

> Of course, you could use what git calls a 'branch' in
> order to implement what you imply is a 'branch', but git's concept of
> a branch and your concept of a branch are not at all the same concept
> (which is why the term 'branch' is so unfortunate).

You've completely lost me. You may very well be right but all I see is
that you're pointing out how branches are implemented in Git.

Re: Branches & directories

[Michael Witten]

On Sun, Aug 21, 2011 at 23:35, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> On 21 August 2011 16:06, Michael Witten <mfwitten@gmail.com> wrote:
>> On Sun, Aug 21, 2011 at 21:37, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>> On 21 August 2011 13:53, Michael Witten <mfwitten@gmail.com> wrote:
>>>> On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
>>>> <hilco.wijbenga@gmail.com> wrote:
>>>>> Isn't a branch simply a way to track changes separately?
>>>>
>>>> Well, what does that mean, really? You can certainly use branches to
>>>> help you achieve that goal.
>>>
>>> It means my commits are chained together separate from, say, master.
>>
>> Well, that's not what a git branch provides in general.
>
> Er, so what *does* a Git branch provide then?

I think my other replies (including the link) repeat myself quite enough.

A branch is just a pointer. That's it.

Quit saying `branch' to yourself. Start saying `pointer' or
`reference' or `commit label' or even `special tag'.

>>> I feel like we're talking in circles. I get (and even agree with) what
>>> you're saying but I don't see how it changes the concept of a branch.
>>>
>>> In any case, what I'm more interested in is knowing whether we can
>>> (optionally) add state (i.e. untracked/ignored files and unstaged
>>> changes) to a branch.
>>
>> No, because a branch doesn't IN ANY WAY provide the structure for that
>> kind of thing.
>
> Obviously, we'd need to expand that structure.
>
> I tried (ab)using git stash to get what I want but it ignores
> untracked/ignored files (not a big surprise, of course). It seems the
> functionality is almost there. If I could just combine git checkout
> with git stash (and have it work with untracked/ignored files) in a
> script or alias, I'd be a happy camper. I'll have to give it some more
> thought.

This cobbling together of git's components for this purpose is
actually a fairly frequent story on this list. Either git does indeed
need something more substantial as a `branch', or people (meaning you)
need to change the way they think (and I'm not sure which solution
would be best, honestly).

If there is a change, then what is currently called a `branch' should
be renamed explicitly to `pointer' or a `reference' or something like
that.

>> Of course, you could use what git calls a 'branch' in
>> order to implement what you imply is a 'branch', but git's concept of
>> a branch and your concept of a branch are not at all the same concept
>> (which is why the term 'branch' is so unfortunate).
>
> You've completely lost me. You may very well be right but all I see is
> that you're pointing out how branches are implemented in Git.

That last sentence and your earlier sentence:

> Obviously, we'd need to expand that structure.

vindicate everything I've said about the choice of nomenclature. The
term `branch' is a TERRIBLE choice.

Re: Branches & directories

[Hilco Wijbenga]

On 21 August 2011 17:07, Michael Witten <mfwitten@gmail.com> wrote:
> On Sun, Aug 21, 2011 at 23:35, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> On 21 August 2011 16:06, Michael Witten <mfwitten@gmail.com> wrote:
>>> On Sun, Aug 21, 2011 at 21:37, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>>> On 21 August 2011 13:53, Michael Witten <mfwitten@gmail.com> wrote:
>>>>> On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
>>>>> <hilco.wijbenga@gmail.com> wrote:
>>>>>> Isn't a branch simply a way to track changes separately?
>>>>>
>>>>> Well, what does that mean, really? You can certainly use branches to
>>>>> help you achieve that goal.
>>>>
>>>> It means my commits are chained together separate from, say, master.
>>>
>>> Well, that's not what a git branch provides in general.
>>
>> Er, so what *does* a Git branch provide then?
>
> I think my other replies (including the link) repeat myself quite enough.
>
> A branch is just a pointer. That's it.
>
> Quit saying `branch' to yourself. Start saying `pointer' or
> `reference' or `commit label' or even `special tag'.

:-) Again, we are going in circles. I *know* a branch is just a
pointer. So what? To me, that's just the implementation. Why is that
relevant? What am I missing?

>>>> I feel like we're talking in circles. I get (and even agree with) what
>>>> you're saying but I don't see how it changes the concept of a branch.
>>>>
>>>> In any case, what I'm more interested in is knowing whether we can
>>>> (optionally) add state (i.e. untracked/ignored files and unstaged
>>>> changes) to a branch.
>>>
>>> No, because a branch doesn't IN ANY WAY provide the structure for that
>>> kind of thing.
>>
>> Obviously, we'd need to expand that structure.
>>
>> I tried (ab)using git stash to get what I want but it ignores
>> untracked/ignored files (not a big surprise, of course). It seems the
>> functionality is almost there. If I could just combine git checkout
>> with git stash (and have it work with untracked/ignored files) in a
>> script or alias, I'd be a happy camper. I'll have to give it some more
>> thought.
>
> This cobbling together of git's components for this purpose is
> actually a fairly frequent story on this list. Either git does indeed
> need something more substantial as a `branch', or people (meaning you)
> need to change the way they think (and I'm not sure which solution
> would be best, honestly).

I don't think that changing the way I think would be very effective. I
still have to get my work done. And I don't want build artifacts from
one "pointer" ;-) leak into the working directory of another "pointer"
just because I changed "pointers". (I'm sorry, "pointer" just doesn't
work for me.)

How is this normally resolved? What do the Linux kernel developers do
when changing or creating a branch? Do they do some sort of "clean"
first?

And I'm getting quite close with "git ls-files -io --exclude-standard
--directory". :-) The cobbling-together-of-components approach looks
promising. ;-)

> If there is a change, then what is currently called a `branch' should
> be renamed explicitly to `pointer' or a `reference' or something like
> that.

Quite possibly but it seems to me that this is too low level. I think
there are already too many places where Git's implementation leaks
into its API. E.g., anything explicitly related to the index.

>>> Of course, you could use what git calls a 'branch' in
>>> order to implement what you imply is a 'branch', but git's concept of
>>> a branch and your concept of a branch are not at all the same concept
>>> (which is why the term 'branch' is so unfortunate).
>>
>> You've completely lost me. You may very well be right but all I see is
>> that you're pointing out how branches are implemented in Git.
>
> That last sentence and your earlier sentence:
>
>> Obviously, we'd need to expand that structure.
>
> vindicate everything I've said about the choice of nomenclature. The
> term `branch' is a TERRIBLE choice.

Re: Branches & directories

[Hilco Wijbenga]

On 21 August 2011 17:47, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> And I'm getting quite close with "git ls-files -io --exclude-standard
> --directory". :-) The cobbling-together-of-components approach looks
> promising. ;-)

Wow, it looks like this sort functionality is already available in
Git. I just cloned the Git repo and git stash now has (in master) two
new options: -u|--include-untracked and -a|--all. It looks like my
wish is about to come true. :-)

Re: Branches & directories

[Michael Witten]

On Mon, Aug 22, 2011 at 00:47, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> On 21 August 2011 17:07, Michael Witten <mfwitten@gmail.com> wrote:
>> On Sun, Aug 21, 2011 at 23:35, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>> On 21 August 2011 16:06, Michael Witten <mfwitten@gmail.com> wrote:
>>>> On Sun, Aug 21, 2011 at 21:37, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>>>> On 21 August 2011 13:53, Michael Witten <mfwitten@gmail.com> wrote:
>>>>>> On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
>>>>>> <hilco.wijbenga@gmail.com> wrote:
>>>>>>> Isn't a branch simply a way to track changes separately?
>>>>>>
>>>>>> Well, what does that mean, really? You can certainly use branches to
>>>>>> help you achieve that goal.
>>>>>
>>>>> It means my commits are chained together separate from, say, master.
>>>>
>>>> Well, that's not what a git branch provides in general.
>>>
>>> Er, so what *does* a Git branch provide then?
>>
>> I think my other replies (including the link) repeat myself quite enough.
>>
>> A branch is just a pointer. That's it.
>>
>> Quit saying `branch' to yourself. Start saying `pointer' or
>> `reference' or `commit label' or even `special tag'.
>
> :-) Again, we are going in circles. I *know* a branch is just a
> pointer. So what? To me, that's just the implementation. Why is that
> relevant? What am I missing?

You're missing (or, rather, ignoring) the fact that what is called a
`branch' in git is exactly what is intended to be called a `branch' in
git; it is not merely an incomplete or weak implementation of
something.

You're probably also missing the fact that I'm simply arguing the
point that the term `branch' was a TERRIBLE choice, and your remarks
and confusion have confirmed that. Other than that, most of what I'm
saying has very little relevance to what you're trying to do; I was
following a philosophical tangent inspired by this thread.

>>>>> I feel like we're talking in circles. I get (and even agree with) what
>>>>> you're saying but I don't see how it changes the concept of a branch.
>>>>>
>>>>> In any case, what I'm more interested in is knowing whether we can
>>>>> (optionally) add state (i.e. untracked/ignored files and unstaged
>>>>> changes) to a branch.
>>>>
>>>> No, because a branch doesn't IN ANY WAY provide the structure for that
>>>> kind of thing.
>>>
>>> Obviously, we'd need to expand that structure.
>>>
>>> I tried (ab)using git stash to get what I want but it ignores
>>> untracked/ignored files (not a big surprise, of course). It seems the
>>> functionality is almost there. If I could just combine git checkout
>>> with git stash (and have it work with untracked/ignored files) in a
>>> script or alias, I'd be a happy camper. I'll have to give it some more
>>> thought.
>>
>> This cobbling together of git's components for this purpose is
>> actually a fairly frequent story on this list. Either git does indeed
>> need something more substantial as a `branch', or people (meaning you)
>> need to change the way they think (and I'm not sure which solution
>> would be best, honestly).
>
> I don't think that changing the way I think would be very effective. I
> still have to get my work done. And I don't want build artifacts from
> one "pointer" ;-) leak into the working directory of another "pointer"
> just because I changed "pointers". (I'm sorry, "pointer" just doesn't
> work for me.)

It doesn't make any sense to say `artifacts from one "pointer"'. You
are now, unfortunately, just using the term `pointer' like the term
`branch'; you are refusing to let go of the concept that the word
`branch' alone has seared into your brain.

As has been my entire point: You have been corrupted by the wrong
choice of words.

> How is this normally resolved? What do the Linux kernel developers do
> when changing or creating a branch? Do they do some sort of "clean"
> first?

Perhaps `git clean' would be of use to you.

> And I'm getting quite close with "git ls-files -io --exclude-standard
> --directory". :-) The cobbling-together-of-components approach looks
> promising. ;-)

Hence what I said earlier:

  Of course, you could use what git calls a 'branch' in order
  to implement what you imply is a 'branch', but git's concept
  of a branch and your concept of a branch are not at all the
  same concept (which is why the term 'branch' is so
  unfortunate).

However, please note that your solution is not completing the
implementation of branches in git, but rather using git's branches to
implement something entirely different (and which you think should be
called `branches'). The terminology is the key point in the confusion.
I dare say, had git's branches been instead called `pointers' or the
like, then you would have just gone about the business of implementing
your `branches' on top of them without thinking twice.

>> If there is a change, then what is currently called a `branch' should
>> be renamed explicitly to `pointer' or a `reference' or something like
>> that.
>
> Quite possibly but it seems to me that this is too low level. I think
> there are already too many places where Git's implementation leaks
> into its API. E.g., anything explicitly related to the index.

Git's power is its simple conceptual basis, and what you are calling
low-level is in fact just the ability to deal within the language of
those simple concepts.

Git's weakness is that the most public interface shoehorns that simple
conceptual basis into not just a more automated system, but into a
weaker model that works well most of the time, yet fails and even
corrupts the mind of the user when it comes to more interesting
scenarios.

What you call a leaky abstraction is not the `low-level' conceptual
basis peeking through innappropriately. Rather, it is the falling
apart of the shoddy `higher-level' facade.

>>>> Of course, you could use what git calls a 'branch' in
>>>> order to implement what you imply is a 'branch', but git's concept of
>>>> a branch and your concept of a branch are not at all the same concept
>>>> (which is why the term 'branch' is so unfortunate).
>>>
>>> You've completely lost me. You may very well be right but all I see is
>>> that you're pointing out how branches are implemented in Git.
>>
>> That last sentence and your earlier sentence:
>>
>>> Obviously, we'd need to expand that structure.
>>
>> vindicate everything I've said about the choice of nomenclature. The
>> term `branch' is a TERRIBLE choice.

Re: Branches & directories

[Hilco Wijbenga]

On 21 August 2011 19:05, Michael Witten <mfwitten@gmail.com> wrote:
> On Mon, Aug 22, 2011 at 00:47, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> On 21 August 2011 17:07, Michael Witten <mfwitten@gmail.com> wrote:
>>> On Sun, Aug 21, 2011 at 23:35, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>>> On 21 August 2011 16:06, Michael Witten <mfwitten@gmail.com> wrote:
>>>>> On Sun, Aug 21, 2011 at 21:37, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>>>>> On 21 August 2011 13:53, Michael Witten <mfwitten@gmail.com> wrote:
>>>>>>> On Sun, Aug 21, 2011 at 13:42 -0700, Hilco Wijbenga
>>>>>>> <hilco.wijbenga@gmail.com> wrote:
>>>>>>>> Isn't a branch simply a way to track changes separately?
>>>>>>>
>>>>>>> Well, what does that mean, really? You can certainly use branches to
>>>>>>> help you achieve that goal.
>>>>>>
>>>>>> It means my commits are chained together separate from, say, master.
>>>>>
>>>>> Well, that's not what a git branch provides in general.
>>>>
>>>> Er, so what *does* a Git branch provide then?
>>>
>>> I think my other replies (including the link) repeat myself quite enough.
>>>
>>> A branch is just a pointer. That's it.
>>>
>>> Quit saying `branch' to yourself. Start saying `pointer' or
>>> `reference' or `commit label' or even `special tag'.
>>
>> :-) Again, we are going in circles. I *know* a branch is just a
>> pointer. So what? To me, that's just the implementation. Why is that
>> relevant? What am I missing?
>
> You're missing (or, rather, ignoring) the fact that what is called a
> `branch' in git is exactly what is intended to be called a `branch' in
> git; it is not merely an incomplete or weak implementation of
> something.
>
> You're probably also missing the fact that I'm simply arguing the
> point that the term `branch' was a TERRIBLE choice, and your remarks
> and confusion have confirmed that. Other than that, most of what I'm
> saying has very little relevance to what you're trying to do; I was
> following a philosophical tangent inspired by this thread.
>
>>>>>> I feel like we're talking in circles. I get (and even agree with) what
>>>>>> you're saying but I don't see how it changes the concept of a branch.
>>>>>>
>>>>>> In any case, what I'm more interested in is knowing whether we can
>>>>>> (optionally) add state (i.e. untracked/ignored files and unstaged
>>>>>> changes) to a branch.
>>>>>
>>>>> No, because a branch doesn't IN ANY WAY provide the structure for that
>>>>> kind of thing.
>>>>
>>>> Obviously, we'd need to expand that structure.
>>>>
>>>> I tried (ab)using git stash to get what I want but it ignores
>>>> untracked/ignored files (not a big surprise, of course). It seems the
>>>> functionality is almost there. If I could just combine git checkout
>>>> with git stash (and have it work with untracked/ignored files) in a
>>>> script or alias, I'd be a happy camper. I'll have to give it some more
>>>> thought.
>>>
>>> This cobbling together of git's components for this purpose is
>>> actually a fairly frequent story on this list. Either git does indeed
>>> need something more substantial as a `branch', or people (meaning you)
>>> need to change the way they think (and I'm not sure which solution
>>> would be best, honestly).
>>
>> I don't think that changing the way I think would be very effective. I
>> still have to get my work done. And I don't want build artifacts from
>> one "pointer" ;-) leak into the working directory of another "pointer"
>> just because I changed "pointers". (I'm sorry, "pointer" just doesn't
>> work for me.)
>
> It doesn't make any sense to say `artifacts from one "pointer"'. You
> are now, unfortunately, just using the term `pointer' like the term
> `branch'; you are refusing to let go of the concept that the word
> `branch' alone has seared into your brain.
>
> As has been my entire point: You have been corrupted by the wrong
> choice of words.
>
>> How is this normally resolved? What do the Linux kernel developers do
>> when changing or creating a branch? Do they do some sort of "clean"
>> first?
>
> Perhaps `git clean' would be of use to you.
>
>> And I'm getting quite close with "git ls-files -io --exclude-standard
>> --directory". :-) The cobbling-together-of-components approach looks
>> promising. ;-)
>
> Hence what I said earlier:
>
>  Of course, you could use what git calls a 'branch' in order
>  to implement what you imply is a 'branch', but git's concept
>  of a branch and your concept of a branch are not at all the
>  same concept (which is why the term 'branch' is so
>  unfortunate).
>
> However, please note that your solution is not completing the
> implementation of branches in git, but rather using git's branches to
> implement something entirely different (and which you think should be
> called `branches'). The terminology is the key point in the confusion.
> I dare say, had git's branches been instead called `pointers' or the
> like, then you would have just gone about the business of implementing
> your `branches' on top of them without thinking twice.
>
>>> If there is a change, then what is currently called a `branch' should
>>> be renamed explicitly to `pointer' or a `reference' or something like
>>> that.
>>
>> Quite possibly but it seems to me that this is too low level. I think
>> there are already too many places where Git's implementation leaks
>> into its API. E.g., anything explicitly related to the index.
>
> Git's power is its simple conceptual basis, and what you are calling
> low-level is in fact just the ability to deal within the language of
> those simple concepts.
>
> Git's weakness is that the most public interface shoehorns that simple
> conceptual basis into not just a more automated system, but into a
> weaker model that works well most of the time, yet fails and even
> corrupts the mind of the user when it comes to more interesting
> scenarios.
>
> What you call a leaky abstraction is not the `low-level' conceptual
> basis peeking through innappropriately. Rather, it is the falling
> apart of the shoddy `higher-level' facade.
>
>>>>> Of course, you could use what git calls a 'branch' in
>>>>> order to implement what you imply is a 'branch', but git's concept of
>>>>> a branch and your concept of a branch are not at all the same concept
>>>>> (which is why the term 'branch' is so unfortunate).
>>>>
>>>> You've completely lost me. You may very well be right but all I see is
>>>> that you're pointing out how branches are implemented in Git.
>>>
>>> That last sentence and your earlier sentence:
>>>
>>>> Obviously, we'd need to expand that structure.
>>>
>>> vindicate everything I've said about the choice of nomenclature. The
>>> term `branch' is a TERRIBLE choice.

Thanks for sticking it out. :-) That all makes sense. I think it's all
pretty clear now.

Re: Branches & directories

[Kyle Moffett]

On Sun, Aug 21, 2011 at 22:13, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> Thanks for sticking it out. :-) That all makes sense. I think it's all
> pretty clear now.

It's worth mentioning that in most cases you DON'T want to delete untracked and
ignored files when switching branches.

For example, when I'm doing kernel development, I will frequently do this:

$ git checkout -b my-new-feature origin/master
[...hack hack hack...]
$ make -j16 && make install && reboot
[...find unrelated bug...]
$ git checkout -b my-new-bugfix origin/master
[...fix the bug...]
$ make -j16 && make install && reboot
$ git commit -m 'I fixed this bug'
$ git checkout my-new-feature
$ git rebase my-new-bugfix
$ make -j16 && make install && reboot

To avoid wasting time I don't want to completely rebuild the kernel each
and every time I switch branches, I just want to rebuild the files that
changed when I switched.  The way GIT works lets me do that quite
easily, and the kernel Makefiles detect the changed files and rebuild
the minimum amount necessary.

GIT's behavior when you switch between branches is effectively the
same as applying a patch generated by diffing between the two
branches.  Any files which would not be changed are left alone, their
timestamps completely unchanged.

It sounds like Eclipse is simply not detecting changes to your working
tree by outside programs, and as a result it's not rebuilding files and
indexes the way that it should.

Obviously the easiest way to work around that issue is "git clean",
which has options to select all untracked files or just ignored ones.

Cheers,
Kyle Moffett

Re: Branches & directories

[Hilco Wijbenga]

On 21 August 2011 20:01, Kyle Moffett <kyle@moffetthome.net> wrote:
> On Sun, Aug 21, 2011 at 22:13, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> Thanks for sticking it out. :-) That all makes sense. I think it's all
>> pretty clear now.
>
> It's worth mentioning that in most cases you DON'T want to delete untracked and
> ignored files when switching branches.

For the record, I don't want them deleted but replaced with the
artifacts in the other branch. A bit wasteful but it saves a lot of
build time.

> For example, when I'm doing kernel development, I will frequently do this:
>
> $ git checkout -b my-new-feature origin/master
> [...hack hack hack...]
> $ make -j16 && make install && reboot
> [...find unrelated bug...]
> $ git checkout -b my-new-bugfix origin/master
> [...fix the bug...]
> $ make -j16 && make install && reboot
> $ git commit -m 'I fixed this bug'
> $ git checkout my-new-feature
> $ git rebase my-new-bugfix
> $ make -j16 && make install && reboot
>
> To avoid wasting time I don't want to completely rebuild the kernel each
> and every time I switch branches, I just want to rebuild the files that
> changed when I switched.  The way GIT works lets me do that quite
> easily, and the kernel Makefiles detect the changed files and rebuild
> the minimum amount necessary.
>
> GIT's behavior when you switch between branches is effectively the
> same as applying a patch generated by diffing between the two
> branches.  Any files which would not be changed are left alone, their
> timestamps completely unchanged.

For small changes that makes perfect sense. I'm at a stage where APIs
are still evolving and changing an API means rebuilding lots of code.
I'd like to avoid that.

> It sounds like Eclipse is simply not detecting changes to your working
> tree by outside programs, and as a result it's not rebuilding files and
> indexes the way that it should.

While Eclipse isn't great at detecting such changes, this isn't really
an Eclipse problem. It's just that lots of things are still changing
and that leads to lots of building.

> Obviously the easiest way to work around that issue is "git clean",
> which has options to select all untracked files or just ignored ones.

As I mentioned above, I don't want to *delete* untracked/ignored
files, I just want them to stick to the branch I was working on. So if
I change to a different branch I get the appropriate build artifacts.

Something like: git stash --everything "All artifacts for
this-branch." && git checkout other-branch && git stash apply
stash-for-other-branch.

Re: Branches & directories

[Kyle Moffett]

On Mon, Aug 22, 2011 at 01:36, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> On 21 August 2011 20:01, Kyle Moffett <kyle@moffetthome.net> wrote:
>> On Sun, Aug 21, 2011 at 22:13, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>> Thanks for sticking it out. :-) That all makes sense. I think it's all
>>> pretty clear now.
>>
>> It's worth mentioning that in most cases you DON'T want to delete untracked and
>> ignored files when switching branches.
>
> For the record, I don't want them deleted but replaced with the
> artifacts in the other branch. A bit wasteful but it saves a lot of
> build time.
>
>> For example, when I'm doing kernel development, I will frequently do this:
>>
>> $ git checkout -b my-new-feature origin/master
>> [...hack hack hack...]
>> $ make -j16 && make install && reboot
>> [...find unrelated bug...]
>> $ git checkout -b my-new-bugfix origin/master
>> [...fix the bug...]
>> $ make -j16 && make install && reboot
>> $ git commit -m 'I fixed this bug'
>> $ git checkout my-new-feature
>> $ git rebase my-new-bugfix
>> $ make -j16 && make install && reboot
>>
>> To avoid wasting time I don't want to completely rebuild the kernel each
>> and every time I switch branches, I just want to rebuild the files that
>> changed when I switched.  The way GIT works lets me do that quite
>> easily, and the kernel Makefiles detect the changed files and rebuild
>> the minimum amount necessary.
>>
>> GIT's behavior when you switch between branches is effectively the
>> same as applying a patch generated by diffing between the two
>> branches.  Any files which would not be changed are left alone, their
>> timestamps completely unchanged.
>
> For small changes that makes perfect sense. I'm at a stage where APIs
> are still evolving and changing an API means rebuilding lots of code.
> I'd like to avoid that.
>
>> It sounds like Eclipse is simply not detecting changes to your working
>> tree by outside programs, and as a result it's not rebuilding files and
>> indexes the way that it should.
>
> While Eclipse isn't great at detecting such changes, this isn't really
> an Eclipse problem. It's just that lots of things are still changing
> and that leads to lots of building.
>
>> Obviously the easiest way to work around that issue is "git clean",
>> which has options to select all untracked files or just ignored ones.
>
> As I mentioned above, I don't want to *delete* untracked/ignored
> files, I just want them to stick to the branch I was working on. So if
> I change to a different branch I get the appropriate build artifacts.
>
> Something like: git stash --everything "All artifacts for
> this-branch." && git checkout other-branch && git stash apply
> stash-for-other-branch.

When I am in those sorts of situations I generally just use separate
working directories or separate checkouts entirely; if you really prefer
to have everything in one directory you would be better served by
integrating "ccache" into your workflow.

In particular, even "git stash" intentionally does not preserve file times,
so you would end up rebuilding everything anyways because all of your
source files would be as new as your object files.

If you use "ccache" then it does not matter what branches you are
on; identical source files and compiler options will be looked up in the
cache and the result (if found) copied to the output file.

Cheers,
Kyle Moffett

Re: Branches & directories

[Hilco Wijbenga]

On 22 August 2011 05:46, Kyle Moffett <kyle@moffetthome.net> wrote:
> On Mon, Aug 22, 2011 at 01:36, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> On 21 August 2011 20:01, Kyle Moffett <kyle@moffetthome.net> wrote:
>>> On Sun, Aug 21, 2011 at 22:13, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>>> Thanks for sticking it out. :-) That all makes sense. I think it's all
>>>> pretty clear now.
>>>
>>> It's worth mentioning that in most cases you DON'T want to delete untracked and
>>> ignored files when switching branches.
>>
>> For the record, I don't want them deleted but replaced with the
>> artifacts in the other branch. A bit wasteful but it saves a lot of
>> build time.
>>
>>> For example, when I'm doing kernel development, I will frequently do this:
>>>
>>> $ git checkout -b my-new-feature origin/master
>>> [...hack hack hack...]
>>> $ make -j16 && make install && reboot
>>> [...find unrelated bug...]
>>> $ git checkout -b my-new-bugfix origin/master
>>> [...fix the bug...]
>>> $ make -j16 && make install && reboot
>>> $ git commit -m 'I fixed this bug'
>>> $ git checkout my-new-feature
>>> $ git rebase my-new-bugfix
>>> $ make -j16 && make install && reboot
>>>
>>> To avoid wasting time I don't want to completely rebuild the kernel each
>>> and every time I switch branches, I just want to rebuild the files that
>>> changed when I switched.  The way GIT works lets me do that quite
>>> easily, and the kernel Makefiles detect the changed files and rebuild
>>> the minimum amount necessary.
>>>
>>> GIT's behavior when you switch between branches is effectively the
>>> same as applying a patch generated by diffing between the two
>>> branches.  Any files which would not be changed are left alone, their
>>> timestamps completely unchanged.
>>
>> For small changes that makes perfect sense. I'm at a stage where APIs
>> are still evolving and changing an API means rebuilding lots of code.
>> I'd like to avoid that.
>>
>>> It sounds like Eclipse is simply not detecting changes to your working
>>> tree by outside programs, and as a result it's not rebuilding files and
>>> indexes the way that it should.
>>
>> While Eclipse isn't great at detecting such changes, this isn't really
>> an Eclipse problem. It's just that lots of things are still changing
>> and that leads to lots of building.
>>
>>> Obviously the easiest way to work around that issue is "git clean",
>>> which has options to select all untracked files or just ignored ones.
>>
>> As I mentioned above, I don't want to *delete* untracked/ignored
>> files, I just want them to stick to the branch I was working on. So if
>> I change to a different branch I get the appropriate build artifacts.
>>
>> Something like: git stash --everything "All artifacts for
>> this-branch." && git checkout other-branch && git stash apply
>> stash-for-other-branch.
>
> When I am in those sorts of situations I generally just use separate
> working directories or separate checkouts entirely; if you really prefer
> to have everything in one directory you would be better served by
> integrating "ccache" into your workflow.

Unfortunately, that seems for C/C++ code only. I use Java. Besides,
it's not the Java compilation that takes most of the time.

> In particular, even "git stash" intentionally does not preserve file times,
> so you would end up rebuilding everything anyways because all of your
> source files would be as new as your object files.

Yes, I just noticed that. Why do you say "intentionally"? Is extra
work being done to make it so? If yes, then shouldn't that be
configurable?

Cheers,
Hilco

Re: Branches & directories

[Kyle Moffett]

On Mon, Aug 22, 2011 at 14:49, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> On 22 August 2011 05:46, Kyle Moffett <kyle@moffetthome.net> wrote:
>> On Mon, Aug 22, 2011 at 01:36, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>> On 21 August 2011 20:01, Kyle Moffett <kyle@moffetthome.net> wrote:
>>>> Obviously the easiest way to work around that issue is "git clean",
>>>> which has options to select all untracked files or just ignored ones.
>>>
>>> As I mentioned above, I don't want to *delete* untracked/ignored
>>> files, I just want them to stick to the branch I was working on. So if
>>> I change to a different branch I get the appropriate build artifacts.
>>>
>>> Something like: git stash --everything "All artifacts for
>>> this-branch." && git checkout other-branch && git stash apply
>>> stash-for-other-branch.
>>
>> When I am in those sorts of situations I generally just use separate
>> working directories or separate checkouts entirely; if you really prefer
>> to have everything in one directory you would be better served by
>> integrating "ccache" into your workflow.
>
> Unfortunately, that seems for C/C++ code only. I use Java. Besides,
> it's not the Java compilation that takes most of the time.

Hm, that sounds like a very serious Eclipse limitation with almost all forms
of source control.  What is done with other VCSes (IE: Subversion, etc)?

>From this I believe the best option is to just make use of multiple separate
checkouts or worktrees.

>> In particular, even "git stash" intentionally does not preserve file times,
>> so you would end up rebuilding everything anyways because all of your
>> source files would be as new as your object files.
>
> Yes, I just noticed that. Why do you say "intentionally"? Is extra
> work being done to make it so? If yes, then shouldn't that be
> configurable?

Well, there's 2 reasons:

(1) The GIT data-structures simply have no place for file timestamps, and
"git stash" is simply a special way of dumping files into a temporary commit.
This is exactly the same as the

(2) You almost always *don't* want to preserve timestamps.  For example:

$ git checkout -b my-feature origin/master
$ vim some-file.c
$ make
$ git add some-file.c && git commit -m "A new feature"
$ git checkout -b my-bugfix origin/master
$ vim other-file.c
$ make

If GIT preserved timestamps, the second "make" would fail to rebuild the
product "some-file.o" because "some-file.c" is still older than it, even
though "some-file.c" has changed since the last build!!!

Really, GIT is only intended for storing source code.  If you want to store
other kinds of things (like timestamps, permissions, etc), then you need to
turn them into source code (IE: a text file and a "make install" target) and
then store them that way.

Cheers,
Kyle Moffett

Re: Branches & directories

[Hilco Wijbenga]

On 22 August 2011 12:31, Kyle Moffett <kyle@moffetthome.net> wrote:
> On Mon, Aug 22, 2011 at 14:49, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>> On 22 August 2011 05:46, Kyle Moffett <kyle@moffetthome.net> wrote:
>>> On Mon, Aug 22, 2011 at 01:36, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>>> On 21 August 2011 20:01, Kyle Moffett <kyle@moffetthome.net> wrote:
>>>>> Obviously the easiest way to work around that issue is "git clean",
>>>>> which has options to select all untracked files or just ignored ones.
>>>>
>>>> As I mentioned above, I don't want to *delete* untracked/ignored
>>>> files, I just want them to stick to the branch I was working on. So if
>>>> I change to a different branch I get the appropriate build artifacts.
>>>>
>>>> Something like: git stash --everything "All artifacts for
>>>> this-branch." && git checkout other-branch && git stash apply
>>>> stash-for-other-branch.
>>>
>>> When I am in those sorts of situations I generally just use separate
>>> working directories or separate checkouts entirely; if you really prefer
>>> to have everything in one directory you would be better served by
>>> integrating "ccache" into your workflow.
>>
>> Unfortunately, that seems for C/C++ code only. I use Java. Besides,
>> it's not the Java compilation that takes most of the time.
>
> Hm, that sounds like a very serious Eclipse limitation with almost all forms
> of source control.  What is done with other VCSes (IE: Subversion, etc)?

It has nothing to do with Eclipse. There is just a lot going on in the build.

> From this I believe the best option is to just make use of multiple separate
> checkouts or worktrees.

Sounds like it.

>>> In particular, even "git stash" intentionally does not preserve file times,
>>> so you would end up rebuilding everything anyways because all of your
>>> source files would be as new as your object files.
>>
>> Yes, I just noticed that. Why do you say "intentionally"? Is extra
>> work being done to make it so? If yes, then shouldn't that be
>> configurable?
>
> Well, there's 2 reasons:
>
> (1) The GIT data-structures simply have no place for file timestamps, and
> "git stash" is simply a special way of dumping files into a temporary commit.

That's what I thought. The "intentionally" threw me off. It's not
intentional, it's just a side effect.

> This is exactly the same as the

There seems to be some text missing here?

> (2) You almost always *don't* want to preserve timestamps.  For example:
>
> $ git checkout -b my-feature origin/master
> $ vim some-file.c
> $ make
> $ git add some-file.c && git commit -m "A new feature"
> $ git checkout -b my-bugfix origin/master
> $ vim other-file.c
> $ make
>
> If GIT preserved timestamps, the second "make" would fail to rebuild the
> product "some-file.o" because "some-file.c" is still older than it, even
> though "some-file.c" has changed since the last build!!!
>
> Really, GIT is only intended for storing source code.  If you want to store
> other kinds of things (like timestamps, permissions, etc), then you need to
> turn them into source code (IE: a text file and a "make install" target) and
> then store them that way.

Yep, that all makes sense. I just wish there was at least an option to
keep the timestamp (and possibly other such things). Even Subversion
can do that... ;-) After all, not everybody uses C & make.

Cheers,
Hilco

Restoring timestamps (Re: Branches & directories)

[Jonathan Nieder]

Hilco Wijbenga wrote:
> On 22 August 2011 12:31, Kyle Moffett <kyle@moffetthome.net> wrote:

>> (1) The GIT data-structures simply have no place for file timestamps, and
>> "git stash" is simply a special way of dumping files into a temporary commit.
>
> That's what I thought. The "intentionally" threw me off. It's not
> intentional, it's just a side effect.

For what it's worth: no, it's intentional.  See, for example:

 http://thread.gmane.org/gmane.comp.version-control.git/1564/focus=1680
 https://git.wiki.kernel.org/index.php/GitFaq#Why_isn.27t_Git_preserving_modification_time_on_files.3F

That said, something being intentional does not necessarily mean it is
always _right_.  So, for example, patches to allow a commit to store
some timestamps, with documentation explaining when this is
appropriate, would probably be welcome.  Maybe a good place to store
such information would be a dotfile alongside the file (so old,
unaware git versions could extract the same information without fuss).

Even if this feature were implemented just for "git stash", personally
I would turn it off so "make" could continue to behave as I expect it
to.  But in principle, I don't mind the idea of it existing. :)

Hope that helps,
Jonathan

Re: Restoring timestamps (Re: Branches & directories)

[Hilco Wijbenga]

On 22 August 2011 14:01, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hilco Wijbenga wrote:
>> On 22 August 2011 12:31, Kyle Moffett <kyle@moffetthome.net> wrote:
>
>>> (1) The GIT data-structures simply have no place for file timestamps, and
>>> "git stash" is simply a special way of dumping files into a temporary commit.
>>
>> That's what I thought. The "intentionally" threw me off. It's not
>> intentional, it's just a side effect.
>
> For what it's worth: no, it's intentional.  See, for example:
>
>  http://thread.gmane.org/gmane.comp.version-control.git/1564/focus=1680
>  https://git.wiki.kernel.org/index.php/GitFaq#Why_isn.27t_Git_preserving_modification_time_on_files.3F

After I had sent the above I realized it was worded a bit too strong.

> That said, something being intentional does not necessarily mean it is
> always _right_.  So, for example, patches to allow a commit to store
> some timestamps, with documentation explaining when this is
> appropriate, would probably be welcome.  Maybe a good place to store
> such information would be a dotfile alongside the file (so old,
> unaware git versions could extract the same information without fuss).

You mean an extra dotfile per file in the commit?

> Even if this feature were implemented just for "git stash", personally
> I would turn it off so "make" could continue to behave as I expect it
> to.  But in principle, I don't mind the idea of it existing. :)

Given that apparently the majority of Git users don't want/need this,
it should probably be off by default. So you would not even need to
turn it off. :-)

In fact, I would not want it on by default either. It's really only
useful when switching branches when I want to keep the exact state of
the branch.

Re: Restoring timestamps (Re: Branches & directories)

[Jonathan Nieder]

Hilco Wijbenga wrote:

> You mean an extra dotfile per file in the commit?

Let me step back a moment.

When you mentioned mtime, a switch went off and reminded me of the
problem of metadata in general, especially owner and permissions. That
problem is important for people keeping track of /etc or $HOME in
version control (always a little dangerous because of the effect of a
stray "git reset --hard", but never mind). And the last time it came
up, I convinced myself that a hook script setting up entries in
.gitattributes, .gitmetadata, or .etckeepr with information like "all
files are owned by root:root unless otherwise specified" could be a
good and not too invasive way to deal with that.

Now that you remind me that the mtime of every file is likely to
differ from every other file, it is harder to imagine what situation
would make this meaningful information that should be stored in the
repository and shared with other people. It seems more like something
associated to the worktree, which fits more closely with what you are
trying to do, anyway.

Regarding the problem "eclipse metadata is not carried over from one
worktree to another", isn't that going to be a problem regardless? In
your proposed setup, each time you stash everything and start work on
a different branch, the eclipse metadata before would be stashed along
with everything else, which doesn't make anything any easier (unless
disk space is very scarce or metadata stores the absolute path to the
cwd).

Sorry for the lack of clarity.
Jonathan

Re: Restoring timestamps (Re: Branches & directories)

[Hilco Wijbenga]

On 22 August 2011 16:21, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hilco Wijbenga wrote:
>
>> You mean an extra dotfile per file in the commit?
>
> Let me step back a moment.
>
> When you mentioned mtime, a switch went off and reminded me of the
> problem of metadata in general, especially owner and permissions. That
> problem is important for people keeping track of /etc or $HOME in
> version control (always a little dangerous because of the effect of a
> stray "git reset --hard", but never mind). And the last time it came
> up, I convinced myself that a hook script setting up entries in
> .gitattributes, .gitmetadata, or .etckeepr with information like "all
> files are owned by root:root unless otherwise specified" could be a
> good and not too invasive way to deal with that.
>
> Now that you remind me that the mtime of every file is likely to
> differ from every other file, it is harder to imagine what situation
> would make this meaningful information that should be stored in the
> repository and shared with other people. It seems more like something
> associated to the worktree, which fits more closely with what you are
> trying to do, anyway.

Yes, indeed.

> Regarding the problem "eclipse metadata is not carried over from one
> worktree to another", isn't that going to be a problem regardless? In
> your proposed setup, each time you stash everything and start work on
> a different branch, the eclipse metadata before would be stashed along
> with everything else, which doesn't make anything any easier (unless
> disk space is very scarce or metadata stores the absolute path to the
> cwd).

Eclipse is a wonderful IDE except for how it makes sharing workspaces
practically impossible (where "share" means "put in SCM", not "used my
several developers at the same time"). It's mostly (AFAICT) binary
data with lots of absolute paths. So it's a pain but it doesn't make
my scenario all that much more complicated.

The hard part is creating a new branch. Somehow I would need to
duplicate the state in the parent branch in the child branch. It needs
to be duplicated because I need to be able to do git stash in the
child branch *and* the parent branch. Is it possible to do git stash
pop without losing the stash? Or to "copy" a stash? Otherwise, it's
probably easier to simply write a separate script to do it. That
should not be very hard. I would not use git stash at all.

For a new branch, the script would
1. in "parent", move workspace (i.e. the root working dir, where .git
is)  into .git/branches/parent/
2. create and jump to "child"
3. in "child", copy .git/branches/parent into workspace (so creating a
new branch is no longer a cheap operation)

For an existing branch, the script would
1. in "current-branch", move workspace into .git/branches/current-branch
2. jump to "other-branch"
3. in "other-branch", move .git/branches/other-branch into workspace

The moves (assuming everything is on the same partition) should make
switching branches relatively cheap. Not very elegant (quite brute
force in fact) but it's simple and I think it would work. Or did I
overlook something? Better/other ideas are certainly welcome. :-)

Re: Restoring timestamps (Re: Branches & directories)

[Hilco Wijbenga]

On 22 August 2011 20:06, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> On 22 August 2011 16:21, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> Hilco Wijbenga wrote:
>>
>>> You mean an extra dotfile per file in the commit?
>>
>> Let me step back a moment.
>>
>> When you mentioned mtime, a switch went off and reminded me of the
>> problem of metadata in general, especially owner and permissions. That
>> problem is important for people keeping track of /etc or $HOME in
>> version control (always a little dangerous because of the effect of a
>> stray "git reset --hard", but never mind). And the last time it came
>> up, I convinced myself that a hook script setting up entries in
>> .gitattributes, .gitmetadata, or .etckeepr with information like "all
>> files are owned by root:root unless otherwise specified" could be a
>> good and not too invasive way to deal with that.
>>
>> Now that you remind me that the mtime of every file is likely to
>> differ from every other file, it is harder to imagine what situation
>> would make this meaningful information that should be stored in the
>> repository and shared with other people. It seems more like something
>> associated to the worktree, which fits more closely with what you are
>> trying to do, anyway.
>
> Yes, indeed.
>
>> Regarding the problem "eclipse metadata is not carried over from one
>> worktree to another", isn't that going to be a problem regardless? In
>> your proposed setup, each time you stash everything and start work on
>> a different branch, the eclipse metadata before would be stashed along
>> with everything else, which doesn't make anything any easier (unless
>> disk space is very scarce or metadata stores the absolute path to the
>> cwd).
>
> Eclipse is a wonderful IDE except for how it makes sharing workspaces
> practically impossible (where "share" means "put in SCM", not "used my
> several developers at the same time"). It's mostly (AFAICT) binary
> data with lots of absolute paths. So it's a pain but it doesn't make
> my scenario all that much more complicated.
>
> The hard part is creating a new branch. Somehow I would need to
> duplicate the state in the parent branch in the child branch. It needs
> to be duplicated because I need to be able to do git stash in the
> child branch *and* the parent branch. Is it possible to do git stash
> pop without losing the stash? Or to "copy" a stash? Otherwise, it's
> probably easier to simply write a separate script to do it. That
> should not be very hard. I would not use git stash at all.
>
> For a new branch, the script would
> 1. in "parent", move workspace (i.e. the root working dir, where .git
> is)  into .git/branches/parent/
> 2. create and jump to "child"
> 3. in "child", copy .git/branches/parent into workspace (so creating a
> new branch is no longer a cheap operation)
>
> For an existing branch, the script would
> 1. in "current-branch", move workspace into .git/branches/current-branch
> 2. jump to "other-branch"
> 3. in "other-branch", move .git/branches/other-branch into workspace
>
> The moves (assuming everything is on the same partition) should make
> switching branches relatively cheap. Not very elegant (quite brute
> force in fact) but it's simple and I think it would work. Or did I
> overlook something? Better/other ideas are certainly welcome. :-)

P.S. I do not mean to imply that I have discounted simply using
different clones or git-new-workdir. Those are still valid options.
The script is probably a bit faster, though.

Re: Restoring timestamps (Re: Branches & directories)

[Enrico Weigelt]

* Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:

> Eclipse is a wonderful IDE except for how it makes sharing workspaces
> practically impossible (where "share" means "put in SCM", not "used my
> several developers at the same time"). 

This is one the major points which render it rather useless for me ;-o

> Is it possible to do git stash pop without losing the stash? 

git cherry-pick stash{0}


Apropos IDEs:

I've been thinking about how to properly integrate IDEs with VCS'es
like git. My conclusion is that it should be directly built ontop
of it. Project metadata itself belongs into git, and the project
management tool should automatically create local working copies
on-demand. I would even delegate *all* the VCS handling to git
(even when using other VCS'es in the back)

Maybe I'll find some time to do some initial concepts.
Perhaps anybody likes to join in ?


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: Restoring timestamps (Re: Branches & directories)

[Hilco Wijbenga]

On 2 October 2011 08:06, Enrico Weigelt <weigelt@metux.de> wrote:
> * Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>
>> Eclipse is a wonderful IDE except for how it makes sharing workspaces
>> practically impossible (where "share" means "put in SCM", not "used by
>> several developers at the same time").
>
> This is one the major points which render it rather useless for me ;-o

As long as Eclipse is the only IDE (AFAIK, of course) that has
workspaces (allowing you to easily group projects) and "Clean Up"
(which helps clean up Java code), I'll stick with it despite its
deficiencies.

Re: Restoring timestamps (Re: Branches & directories)

[Hilco Wijbenga]

On 2 October 2011 08:47, Brandon Casey <drafnel@gmail.com> wrote:
> On Aug 22, 2011 10:06 PM, "Hilco Wijbenga" <hilco.wijbenga@gmail.com> wrote:
>> Is it possible to do git stash
>> pop without losing the stash?
>
> That's called 'git stash apply'.

:-) Yeah, I had actually discovered that myself. I had been using
"pop" and "apply" interchangeably until I noticed I had a number of
stashes stored that I was not aware of. That's when I discovered they
did in fact *not* have exactly the same meaning. :-)

Re: Branches & directories

[Michael Witten]

On Mon, Aug 22, 2011 at 20:10, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
> On 22 August 2011 12:31, Kyle Moffett <kyle@moffetthome.net> wrote:
>> On Mon, Aug 22, 2011 at 14:49, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>> On 22 August 2011 05:46, Kyle Moffett <kyle@moffetthome.net> wrote:
>>>> On Mon, Aug 22, 2011 at 01:36, Hilco Wijbenga <hilco.wijbenga@gmail.com> wrote:
>>>>> On 21 August 2011 20:01, Kyle Moffett <kyle@moffetthome.net> wrote:
>>>> In particular, even "git stash" intentionally does not preserve file times,
>>>> so you would end up rebuilding everything anyways because all of your
>>>> source files would be as new as your object files.
>>>
>>> Yes, I just noticed that. Why do you say "intentionally"? Is extra
>>> work being done to make it so? If yes, then shouldn't that be
>>> configurable?
>>
>> Well, there's 2 reasons:
>>
>> (1) The GIT data-structures simply have no place for file timestamps, and
>> "git stash" is simply a special way of dumping files into a temporary commit.
>
> That's what I thought. The "intentionally" threw me off. It's not
> intentional, it's just a side effect.

No, it's intentional.

This discussion about timestamps is not new; the mailing list is
littered with people wondering about it. The data structures are the
way they are for intentional reasons, as suggested by Kyle's example.

>> (2) You almost always *don't* want to preserve timestamps.  For example:
>>
>> $ git checkout -b my-feature origin/master
>> $ vim some-file.c
>> $ make
>> $ git add some-file.c && git commit -m "A new feature"
>> $ git checkout -b my-bugfix origin/master
>> $ vim other-file.c
>> $ make
>>
>> If GIT preserved timestamps, the second "make" would fail to rebuild the
>> product "some-file.o" because "some-file.c" is still older than it, even
>> though "some-file.c" has changed since the last build!!!
>>
>> Really, GIT is only intended for storing source code.  If you want to store
>> other kinds of things (like timestamps, permissions, etc), then you need to
>> turn them into source code (IE: a text file and a "make install" target) and
>> then store them that way.
>
> Yep, that all makes sense. I just wish there was at least an option to
> keep the timestamp (and possibly other such things). Even Subversion
> can do that... ;-) After all, not everybody uses C & make.

Yes, well, the crux there is `(and possibly other such things)'. That
is an open-ended specification because nobody can agree on what such
things should include.

One of git's major design goals has been efficiency, so it's natural
that git only handles what is necessary for its prime purpose of
managing a history of line-oriented textual content.

However, other tools have been built on top of git's infrastructure in
order to store all manner of metadata (usually for the purposes of
backing up files with all of their special metadata).

Essentially, only *you* know what `other such things' *you* need, so
it's up to you to design a way of storing such information and
retrieving it yourself. After all, not everybody (indeed, nearly
nobody, I'd wager) has your specific difficulties.

Re: Branches & directories

[Robin Rosenberg]

Hilco Wijbenga skrev 2011-08-22 22.10:
> [...] I just wish there was at least an option to
> keep the timestamp (and possibly other such things). Even Subversion
> can do that... ;-) After all, not everybody uses C&  make.
>
What tools do you use that need the benefits from retaining timestamps? 
The only
one I can think of is clearmake, but then that tool goes with another 
SCM. Eclipse,
for example, will be just as confused by timestamps that travel 
backwards in time, as
make is.

-- robin

Re: Branches & directories

[Ronan Keryell]

>>>>> On Sun, 02 Oct 2011 18:57:55 +0200, Robin Rosenberg <robin.rosenberg@gmail.com> said:

    Robin> Hilco Wijbenga skrev 2011-08-22 22.10:
    >> [...] I just wish there was at least an option to keep the
    >> timestamp (and possibly other such things). Even Subversion can
    >> do that... ;-) After all, not everybody uses C& make.

    Robin> What tools do you use that need the benefits from retaining
    Robin> timestamps?  The only one I can think of is clearmake, but
    Robin> then that tool goes with another SCM. Eclipse, for example,
    Robin> will be just as confused by timestamps that travel backwards
    Robin> in time, as make is.

I think of tools called "humans", very common indeed on Earth. :-)

The reward of git success is that it is not only used to develop the
Linux kernel. :-)

We use also git as a very smart repositories to store administrative
documents. It is very convenient to look at the real modification or
creation dates to figure out some historical aspects for example.

metastore is a nice tool providing a begin of this on top of git (or
whatever) but :
- this is not very convenient, needing to deal manually with these
  aspects ;
- the metadata is binary and not textual (à la YAML ?) so we loose the
  classical textual merging candies when conflict arises on metadata
  (ouch).

It is one of my future project to do a more textual version of
metastore, but I'm afraid it is an unbound future... :-/
-- 
  Ronan KERYELL                      |\/  Phone:  +1 408 844 HPC0
  HPC Project, Inc.                  |/)  Cell:   +33 613 143 766
  5201 Great America Parkway #3241   K    Ronan.Keryell@hpc-project.com
  Santa Clara, CA 95054              |\   skype:keryell
  USA                                | \  http://hpc-project.com

Re: Branches & directories

[Matthieu Moy]

Ronan Keryell <Ronan.Keryell@hpc-project.com> writes:

>>>>>> On Sun, 02 Oct 2011 18:57:55 +0200, Robin Rosenberg <robin.rosenberg@gmail.com> said:
>
>     Robin> Hilco Wijbenga skrev 2011-08-22 22.10:
>     >> [...] I just wish there was at least an option to keep the
>     >> timestamp (and possibly other such things). Even Subversion can
>     >> do that... ;-) After all, not everybody uses C& make.

AFAIK, Subversion doesn't version timestamps. What it can do is to set
the timestamp to the commit date at the time the file is checked-out.

>     Robin> What tools do you use that need the benefits from retaining
>     Robin> timestamps?  The only one I can think of is clearmake, but
>     Robin> then that tool goes with another SCM. Eclipse, for example,
>     Robin> will be just as confused by timestamps that travel backwards
>     Robin> in time, as make is.
>
> I think of tools called "humans", very common indeed on Earth. :-)

For human beings, it's not really harder to run

  git log -1 file

than to look at the on-disk timestamp. And it continues working after
you start modifying the file, so it's much less fragile than the
filesystem timestamp.

But if you insist in reproducing SVN's "use-commit-times = yes" setting,
it should be doable with a post-checkout hook.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Branches & directories

[Hilco Wijbenga]

On 2 October 2011 12:09, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
> Ronan Keryell <Ronan.Keryell@hpc-project.com> writes:
>
>>>>>>> On Sun, 02 Oct 2011 18:57:55 +0200, Robin Rosenberg <robin.rosenberg@gmail.com> said:
>>
>>     Robin> Hilco Wijbenga skrev 2011-08-22 22.10:
>>     >> [...] I just wish there was at least an option to keep the
>>     >> timestamp (and possibly other such things). Even Subversion can
>>     >> do that... ;-) After all, not everybody uses C& make.
>
> AFAIK, Subversion doesn't version timestamps. What it can do is to set
> the timestamp to the commit date at the time the file is checked-out.

Correct and this would fix my problem, I believe.

>>     Robin> What tools do you use that need the benefits from retaining
>>     Robin> timestamps?  The only one I can think of is clearmake, but
>>     Robin> then that tool goes with another SCM. Eclipse, for example,
>>     Robin> will be just as confused by timestamps that travel backwards
>>     Robin> in time, as make is.
>>
>> I think of tools called "humans", very common indeed on Earth. :-)
>
> For human beings, it's not really harder to run
>
>  git log -1 file

I think the idea is that computers do the work, not humans... :-)

> than to look at the on-disk timestamp. And it continues working after
> you start modifying the file, so it's much less fragile than the
> filesystem timestamp.
>
> But if you insist in reproducing SVN's "use-commit-times = yes" setting,
> it should be doable with a post-checkout hook.

Mmm, this post-checkout hook would get the commit time from the commit
and "touch" all relevant files with that timestamp? Is that easily
done?

Re: Branches & directories

[Hilco Wijbenga]

On 2 October 2011 09:57, Robin Rosenberg <robin.rosenberg@gmail.com> wrote:
> Hilco Wijbenga skrev 2011-08-22 22.10:
>>
>> [...] I just wish there was at least an option to
>> keep the timestamp (and possibly other such things). Even Subversion
>> can do that... ;-) After all, not everybody uses C&  make.
>>
> What tools do you use that need the benefits from retaining timestamps? The
> only
> one I can think of is clearmake, but then that tool goes with another SCM.
> Eclipse,
> for example, will be just as confused by timestamps that travel backwards in
> time, as
> make is.

Why would timestamps travel back in time? They simply would not change.

Anyway, we're not really talking about the same thing. If there's an
update (i.e. git pull or something similar) then changing the
timestamp to something (anything) newer is the right thing to do. In
fact, it would be painful (as you already alluded to) if this were not
the case.

That's however not the scenario that I'm talking about. I'm talking about doing

git checkout branch
git checkout master

or

git stash
git stash pop

In both cases all files (or at least all affected files, in case of
git stash) get the current time as their timestamp instead of the
timestamp they had before. This is forcing (completely unnecessary)
rebuilds. *Nothing* has changed but I have to do a complete rebuild
(well, I suppose I could "touch" all build artifacts and such but I'm
sure you get the idea).

I understand *why* it's happening (it's simply reusing the existing
Git functionality) but in the scenarios above nothing has really
changed, I should be able to pick up from where I left off, shouldn't
I?

(Obviously, I moved the discussion off track when I started talking
about Subversion, commits, and commit times. That's really just an
implementation detail and I wish I had never brought it up.)

P.S. I'm quite happy with git-new-workdir so I do believe I have a
good workaround.

Re: Branches & directories

[Jeff King]

On Sun, Oct 02, 2011 at 04:40:14PM -0700, Hilco Wijbenga wrote:

> That's however not the scenario that I'm talking about. I'm talking about doing
> 
> git checkout branch
> git checkout master
> 
> or
> 
> git stash
> git stash pop
> 
> In both cases all files (or at least all affected files, in case of
> git stash) get the current time as their timestamp instead of the
> timestamp they had before. This is forcing (completely unnecessary)
> rebuilds. *Nothing* has changed but I have to do a complete rebuild
> (well, I suppose I could "touch" all build artifacts and such but I'm
> sure you get the idea).
> 
> I understand *why* it's happening (it's simply reusing the existing
> Git functionality) but in the scenarios above nothing has really
> changed, I should be able to pick up from where I left off, shouldn't
> I?

No. There are cases where that will fool timestamp-based tools. The
problem is that the build products are not tracked by git, and so they
are not changed when you switch branches. But the timestamps of build
products and branches are compared.

So let's imagine you have two branches, with two different versions of
foo.c, both of which use "make" to build them into foo.o. Their
timestamps are from an hour ago and two hours ago. And that git restores
those old timestamps. You do:

  git checkout master
  make

Now foo.c is one hour old (from master). But foo.o is only a few seconds
old (it was just created by make. Now you do:

  git checkout branch
  make

Now foo.c is two hours old (from branch). But foo.o is still new, so
make doesn't rebuild it, which is an error.

Or did you really mean your example literally, as in you run two
checkouts back to back, without running anything in between, and the
second checkout restores the state before the first one. In that case,
yes, it would be correct to keep the old timestamps. But this is an
optimization that can only apply in a few very specific cases. And
moreoever, how can git know when it is OK to apply that optimization? It
has no idea what commands you might have run since the last time we were
at "master".

-Peff

Re: Branches & directories

[Hilco Wijbenga]

On 2 October 2011 20:07, Jeff King <peff@peff.net> wrote:
<snip/>
> Or did you really mean your example literally, as in you run two
> checkouts back to back, without running anything in between, and the
> second checkout restores the state before the first one. In that case,
> yes, it would be correct to keep the old timestamps. But this is an
> optimization that can only apply in a few very specific cases. And
> moreoever, how can git know when it is OK to apply that optimization? It
> has no idea what commands you might have run since the last time we were
> at "master".

Yes, I meant it literally. And, no, Git could not possibly know so it
would have to be optional behaviour. But it's probably a lot of work
for (for most people) little gain.

Re: Branches & directories

[Jeff King]

On Mon, Oct 03, 2011 at 12:15:33AM -0700, Hilco Wijbenga wrote:

> On 2 October 2011 20:07, Jeff King <peff@peff.net> wrote:
> <snip/>
> > Or did you really mean your example literally, as in you run two
> > checkouts back to back, without running anything in between, and the
> > second checkout restores the state before the first one. In that case,
> > yes, it would be correct to keep the old timestamps. But this is an
> > optimization that can only apply in a few very specific cases. And
> > moreoever, how can git know when it is OK to apply that optimization? It
> > has no idea what commands you might have run since the last time we were
> > at "master".
> 
> Yes, I meant it literally. And, no, Git could not possibly know so it
> would have to be optional behaviour. But it's probably a lot of work
> for (for most people) little gain.

If you really want the human to trigger it, then you can do something
like this:

  cat >git-checkout-timestamp <<\EOF
  #!/bin/sh

  old=`git rev-parse HEAD`
  git checkout "$@" || exit 1
  time=`git log -1 --format=%at`
  git diff-tree --name-only -z "$old" HEAD |
    perl -0ne "utime($time, $time, \$_)";
  EOF

Drop that somewhere in your $PATH, and use it instead of regular
checkout. It restores the timestamps on any changed files, but not on
those that were not touched. So your:

  git checkout branch
  git checkout master

example would end up with timestamps set for "master" on the changed
files. Two caveats:

  1. This can still break makefiles! For example, like this:

       make foo.o ;# now foo.o is recent
       vi foo.c   ;# but foo.c is _more_ recent
       git checkout branch ;# now it's even newer
       git checkout-timestamp master ;# now we've restored it to some
                                     ;# old timestamp, and make will
                                     ;# think it's older than foo.o

  2. In general, I'm not sure it makes any sense if there are local
     worktree modifications to the files in question. But I didn't think
     about it too hard. That ways madness lies.

-Peff

Re: Branches & directories

[Matthieu Moy]

Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:

> Yes, I meant it literally. And, no, Git could not possibly know so it
> would have to be optional behaviour. But it's probably a lot of work
> for (for most people) little gain.

Not only little gain, but also important risk: users of this feature
would be likely to spend hours debugging something just because some
files weren't recompiled at the right time.

If you want to optimize the number of files compiled by "make", then
ccache is your friend. This one is safe.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Branches & directories

[Jeff King]

On Mon, Oct 03, 2011 at 09:32:07AM +0200, Matthieu Moy wrote:

> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
> 
> > Yes, I meant it literally. And, no, Git could not possibly know so it
> > would have to be optional behaviour. But it's probably a lot of work
> > for (for most people) little gain.
> 
> Not only little gain, but also important risk: users of this feature
> would be likely to spend hours debugging something just because some
> files weren't recompiled at the right time.
> 
> If you want to optimize the number of files compiled by "make", then
> ccache is your friend. This one is safe.

Yes. Despite my previous message showing what _could_ be done, I do
think it's crazy. You should just use ccache.

Speaking of which; does anybody know of a git-aware ccache-like tool?
We already have a nice index of the sha1 of each file in the repository
(along with a stat cache showing us whether it's up-to-date or not).
Something like ccache could avoid even looking in the C files at all if
it relied on git's index.

I don't know how much speedup it would yield in practice, though.

-Peff

Re: Branches & directories

[Matthieu Moy]

Jeff King <peff@peff.net> writes:

> Speaking of which; does anybody know of a git-aware ccache-like tool?
> We already have a nice index of the sha1 of each file in the repository
> (along with a stat cache showing us whether it's up-to-date or not).
> Something like ccache could avoid even looking in the C files at all if
> it relied on git's index.

It would be a bit harder than that I think. IIRC, ccache hashes the
preprocessed file, hence it will notice if a .h file changed, even if
it's outside the project.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: Branches & directories

[Jeff King]

On Mon, Oct 03, 2011 at 09:41:29AM +0200, Matthieu Moy wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Speaking of which; does anybody know of a git-aware ccache-like tool?
> > We already have a nice index of the sha1 of each file in the repository
> > (along with a stat cache showing us whether it's up-to-date or not).
> > Something like ccache could avoid even looking in the C files at all if
> > it relied on git's index.
> 
> It would be a bit harder than that I think. IIRC, ccache hashes the
> preprocessed file, hence it will notice if a .h file changed, even if
> it's outside the project.

Yeah, you'd have to maintain your own dependency tree, then. Which is
nasty (aside from the work involved), because I don't think you can
portably get the header dependencies out of the C compiler.

Oh well.

-Peff

Re: Branches & directories

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Yeah, you'd have to maintain your own dependency tree, then. Which is
> nasty (aside from the work involved), because I don't think you can
> portably get the header dependencies out of the C compiler.

Heh, but doesn't your Makefile know the header dependencies anyway?

Re: Branches & directories

[Jeff King]

On Mon, Oct 03, 2011 at 12:48:27AM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Yeah, you'd have to maintain your own dependency tree, then. Which is
> > nasty (aside from the work involved), because I don't think you can
> > portably get the header dependencies out of the C compiler.
> 
> Heh, but doesn't your Makefile know the header dependencies anyway?

Sort of. It's often wrong (e.g., we are way over-inclusive in the git
Makefile; one of the things I like about ccache is that even when our
Makefile is overly conservative, ccache is fast. Or at least faster than
actually recompiling).

And of course it doesn't generally involve headers outside of your
project, whereas ccache does recompile if those change.

-Peff

Re: Branches & directories

[Hilco Wijbenga]

On 3 October 2011 00:32, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
> Hilco Wijbenga <hilco.wijbenga@gmail.com> writes:
>
>> Yes, I meant it literally. And, no, Git could not possibly know so it
>> would have to be optional behaviour. But it's probably a lot of work
>> for (for most people) little gain.
>
> Not only little gain, but also important risk: users of this feature
> would be likely to spend hours debugging something just because some
> files weren't recompiled at the right time.

Possibly. When I do a git pull or similar I do a full build
regardless. I don't know of any build tool that triggers a build
because of a deleted source file (that's an actual problem I ran into
only a couple of weeks ago). Of course, in that scenario the build was
succeeding where it should have been failing. :-)

> If you want to optimize the number of files compiled by "make", then
> ccache is your friend. This one is safe.

This is all C, right? I'm in Java land so I would assume ccache is of
no use to me. And we certainly don't use make.

Re: Branches & directories

[Robin Rosenberg]

Hilco Wijbenga skrev 2011-10-03 09.15:
> On 2 October 2011 20:07, Jeff King<peff@peff.net>  wrote:
> <snip/>
>> Or did you really mean your example literally, as in you run two
>> checkouts back to back, without running anything in between, and the
>> second checkout restores the state before the first one. In that case,
>> yes, it would be correct to keep the old timestamps. But this is an
>> optimization that can only apply in a few very specific cases. And
>> moreoever, how can git know when it is OK to apply that optimization? It
>> has no idea what commands you might have run since the last time we were
>> at "master".
> Yes, I meant it literally. And, no, Git could not possibly know so it
> would have to be optional behaviour. But it's probably a lot of work
> for (for most people) little gain.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
I wouldn't use stash for that. Just regular commit/amend and your
timestamps should be fine. Alternative submit a patch for either
the save or create subcommands of stash. That would not be very
hard (technically)  and no one needs to mess with the timestamps;
they will just survive.

-- robin

Re: Branches & directories

[Hilco Wijbenga]

On 3 October 2011 07:59, Robin Rosenberg <robin.rosenberg@gmail.com> wrote:
> Hilco Wijbenga skrev 2011-10-03 09.15:
>>
>> On 2 October 2011 20:07, Jeff King<peff@peff.net>  wrote:
>> <snip/>
>>>
>>> Or did you really mean your example literally, as in you run two
>>> checkouts back to back, without running anything in between, and the
>>> second checkout restores the state before the first one. In that case,
>>> yes, it would be correct to keep the old timestamps. But this is an
>>> optimization that can only apply in a few very specific cases. And
>>> moreoever, how can git know when it is OK to apply that optimization? It
>>> has no idea what commands you might have run since the last time we were
>>> at "master".
>>
>> Yes, I meant it literally. And, no, Git could not possibly know so it
>> would have to be optional behaviour. But it's probably a lot of work
>> for (for most people) little gain.
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> I wouldn't use stash for that. Just regular commit/amend and your
> timestamps should be fine. Alternative submit a patch for either
> the save or create subcommands of stash. That would not be very
> hard (technically)  and no one needs to mess with the timestamps;
> they will just survive.

By "that" you mean jump to another branch? I don't see how doing an
explicit commit changes anything. Stashing is essentially committing
(i.e. a "dummy" commit is created to store the stash, IIUC), isn't it?

As I mentioned before, I'm quite happy with git-new-workdir. It allows
me to work exactly the way I want. I may have some philosophical
reservations about Git's timestamp handling but practically speaking
I'm a happy little camper. :-)