What's cooking in git.git (topics)

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

* jc/checkout (Sun Apr 8 22:08:31 2007 -0700) 7 commits
 + Teach git-reset to use index BASE extension.
 + git-read-tree --set-base=<sha1>
 + Use BASE index extension in git-am.
 + Move check_base() shell function to git-sh-setup
 + Use BASE index extension in git-commit and git-merge.
 + update-index --{set,get}-base
 + Add BASE index extension.

This series is primarily to make it safer when somebody else
updates the tip of the branch you have currently checked out.
As I said in previous messages, I think the series covers basic
operations fine but there probably are gaps in the coverage.
Motivated volunteers are needed to fill them.

* fl/cvsserver (Sat Apr 7 16:58:10 2007 +0200) 8 commits
 - cvsserver: Add asciidoc documentation for new database backend
   configuration
 + cvsserver: Corrections to the database backend configuration
 + cvsserver: Use DBI->table_info instead of DBI->tables
 + cvsserver: Abort if connect to database fails
 + cvsserver: Make the database backend configurable
 + cvsserver: Allow to override the configuration per access method
 + cvsserver: Handle three part keys in git config correctly
 + cvsserver: Introduce new state variable 'method'

Although the code looked sane, I do not interact with git
repositories over CVS protocol in real life, so I haven't
personally tested it.  I cannot push this out to 'master'
without positive feedbacks.  Testing needed.

* jc/read-tree-df (Sat Apr 7 07:17:35 2007 -0700) 5 commits
 - t3030: merge-recursive backend test.
 - merge-recursive: handle D/F conflict case more carefully.
 - merge-recursive: do not barf on "to be removed" entries.
 - Treat D/F conflict entry more carefully in unpack-
   trees.c::threeway_merge()
 - t1000: fix case table.

This is a follow-up to "my branch has foo/bar and I cannot
switch to another branch that has foo as a file" series.  It
deals with three-way merges that involve D/F conflicts.  Again,
heavy testing and reporting is needed until it graduates to
'master'.

* jc/quickfetch (Thu Apr 5 03:22:55 2007 -0700) 2 commits
 - git-fetch: use fetch--tool pick-rref to avoid local fetch from
   alternate
 - git-fetch--tool pick-rref

This is to alleviate Linus's worries about "git fetch" from a
repository whose object store is your alternate consuming too
much cycles in the new "SHA-1 collision detection" code, at the
same time avoiding to create a pack that has duplicate objects
in the repository due to the recent "keep" behaviour of
fetch-pack.

* js/wrap-log (Sun Apr 8 01:28:00 2007 -0700) 2 commits
 - shortlog -w: make wrap-line behaviour optional.
 - Use print_wrapped_text() in shortlog

This teaches "git shortlog" to wrap lines that are too long (by
default, 76 columns) when "-w" option is given.

This was resurrected from my held-box.  I do not have an
immediate need for it but somebody might.  Cheering-on and
documentation updates are needed for it to go anywhere.

* jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 2 commits
 - Make read-cache.c "the_index" free.
 - Move index-related variables into a structure.

This libifies the "cache" part of the system.  Parked in 'pu' as
there is no immediate need, at least for me.

* jc/blame (Tue Mar 27 01:58:01 2007 -0700) 3 commits
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
* jc/pathattr (Thu Mar 1 01:20:21 2007 -0800) 5 commits
 - pathattr: allow piping to external program.
 - pathattr: read from git_config().
 - git-show: use pathattr to run "display"
 - pathattr: path based configuration of various attributes.
 + convert: add scaffolding for path based selection of conversion
   routines.
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

These are stalled...

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

* lt/gitlink (Sun Apr 15 11:14:28 2007 -0700) 17 commits
 + Expose subprojects as special files to "git diff" machinery
 + Fix some "git ls-files -o" fallout from gitlinks
 + Teach "git-read-tree -u" to check out submodules as a directory
 + Teach git list-objects logic to not follow gitlinks
 + Fix gitlink index entry filesystem matching
 + Teach "git-read-tree -u" to check out submodules as a directory
 + Teach git list-objects logic not to follow gitlinks
 + Don't show gitlink directories when we want "other" files
 + Teach git-update-index about gitlinks
 + Teach directory traversal about subprojects
 + Fix thinko in subproject entry sorting
 + Teach core object handling functions about gitlinks
 + Teach "fsck" not to follow subproject links
 + Add "S_IFDIRLNK" file mode infrastructure for git links
 + Add 'resolve_gitlink_ref()' helper function
 + Avoid overflowing name buffer in deep directory structures
 + diff-lib: use ce_mode_from_stat() rather than messing with modes
   manually

Everybody loves when Linus jumps in and gets the ball rolling
for a topic that has been only an idle-talk wishlist with a
minimum set of patches.  Let's see where this goes.

* jc/attr (Sat Apr 14 21:27:20 2007 -0400) 10 commits
 - Document git-check-attr
 - Change attribute negation marker from '!' to '-'.
 - Define a built-in attribute macro "binary".
 - attribute macro support
 + Makefile: add patch-ids.h back in.
 + Fix 'diff' attribute semantics.
 + Fix 'crlf' attribute semantics.
 + Teach 'diff' about 'diff' attribute.
 + Define 'crlf' attribute.
 + Add basic infrastructure to assign attributes to paths

... and I tried to learn from that.  I do not know how
successful I was, though.

But I earlier said that one of the focus of 1.5.2 should be the
gitattributes support.

* fl/cvsserver (Fri Apr 13 18:13:42 2007 +0200) 12 commits
 + config.txt: Add gitcvs.db* variables
 + cvsserver: Document the GIT branches -> CVS modules mapping more
   prominently
 + cvsserver: Reword documentation on necessity of write access
 + cvsserver: Allow to "add" a removed file
 + cvsserver: Add asciidoc documentation for new database backend
   configuration
 + cvsserver: Corrections to the database backend configuration
 + cvsserver: Use DBI->table_info instead of DBI->tables
 + cvsserver: Abort if connect to database fails
 + cvsserver: Make the database backend configurable
 + cvsserver: Allow to override the configuration per access method
 + cvsserver: Handle three part keys in git config correctly
 + cvsserver: Introduce new state variable 'method'

Waiting for Ack's from the field, which unfortunately I haven't
seen any yet.

* np/pack (Tue Apr 10 22:54:36 2007 -0400) 16 commits
 + clean up add_object_entry()
 + tests for various pack index features
 + use test-genrandom in tests instead of /dev/urandom
 + simple random data generator for tests
 + validate reused pack data with CRC when possible
 + allow forcing index v2 and 64-bit offset treshold
 + pack-redundant.c: learn about index v2
 + show-index.c: learn about index v2
 + sha1_file.c: learn about index version 2
 + index-pack: learn about pack index version 2
 + pack-objects: learn about pack index version 2
 + compute object CRC32 with index-pack
 + compute a CRC32 for each object as stored in a pack
 + add overflow tests on pack offset variables
 + make overflow test on delta base offset work regardless of
   variable size
 + get rid of num_packed_objects()

Haven't seen any breakage report so far.  After giving them a
final look, let's push this out to 'master' soonish.

* js/wrap-log (Sun Apr 8 01:28:00 2007 -0700) 2 commits
 + shortlog -w: make wrap-line behaviour optional.
 + Use print_wrapped_text() in shortlog

I do not think it breaks anything but I do not think we are in a
hurry, either.

* jc/read-tree-df (Sat Apr 7 07:17:35 2007 -0700) 5 commits
 + t3030: merge-recursive backend test.
 + merge-recursive: handle D/F conflict case more carefully.
 + merge-recursive: do not barf on "to be removed" entries.
 + Treat D/F conflict entry more carefully in unpack-
   trees.c::threeway_merge()
 + t1000: fix case table.

This series should not matter in practice as I do not think any
project that changes between directory and file is sane, but
people are known to do insane things, and this would help them.

Any comments for or against their graduation to 'master'?

* jc/quickfetch (Thu Apr 5 03:22:55 2007 -0700) 2 commits
 + git-fetch: use fetch--tool pick-rref to avoid local fetch from
   alternate
 + git-fetch--tool pick-rref

This would make fetching from your alternate more efficient by
not fetching any objects (because by definition it is not
necessary).  It doubly matters in this case performance-wise as
the recent code verifies fetched objects that were already in
the repository, which tends to be expensive.

* jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 2 commits
 - Make read-cache.c "the_index" free.
 - Move index-related variables into a structure.

Sort of "libification", which nobody seems to need right now,
but I did it already and there is no reason to throw away.

* jc/blame (Tue Mar 27 01:58:01 2007 -0700) 4 commits
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
 - blame -s: suppress author name and time.
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

Stalled.

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

* np/pack (Mon Apr 16 12:32:13 2007 -0400) 25 commits
 + pack-objects: better check_object() performances
 + add get_size_from_delta()
 + pack-objects: make in_pack_header_size a variable of its own
 + pack-objects: get rid of create_final_object_list()
 + pack-objects: get rid of reuse_cached_pack
 + pack-objects: clean up list sorting
 + pack-objects: rework check_delta_limit usage
 + pack-objects: equal objects in size should delta against newer
   objects
 + pack-objects: optimize preferred base handling a bit
 + clean up add_object_entry()
 + tests for various pack index features
 + use test-genrandom in tests instead of /dev/urandom
 + simple random data generator for tests
 + validate reused pack data with CRC when possible
 + allow forcing index v2 and 64-bit offset treshold
 + pack-redundant.c: learn about index v2
 + show-index.c: learn about index v2
 + sha1_file.c: learn about index version 2
 + index-pack: learn about pack index version 2
 + pack-objects: learn about pack index version 2
 + compute object CRC32 with index-pack
 + compute a CRC32 for each object as stored in a pack
 + add overflow tests on pack offset variables
 + make overflow test on delta base offset work regardless of
   variable size
 + get rid of num_packed_objects()

Nico's "optionally 64-bit pack idx" aka idx format version 2.
I've taken another pass at them and am planning to push them out
on 'master' hopefully this weekend, but a documentation update
that mention the new --index-version option to git-pack-objects
would be nice to have before that happens.

* lt/objalloc (Mon Apr 16 22:13:09 2007 -0700) 3 commits
 - Make the object lookup hash use a "object index" instead of a
   pointer
 + Clean up object creation to use more common code
 + Use proper object allocators for unknown object nodes too

The bottom 2 are genuine clean-ups, and I'd like to push them
out to 'master' this weekend after giving them one last look.

* jc/attr (Wed Apr 18 16:16:37 2007 -0700) 19 commits
 + Fix funny types used in attribute value representation
 + Allow low-level driver to specify different behaviour during
   internal merge.
 + Custom low-level merge driver: change the configuration scheme.
 + Allow the default low-level merge driver to be configured.
 + Custom low-level merge driver support.
 + Add a demonstration/test of customized merge.
 + Allow specifying specialized merge-backend per path.
 + merge-recursive: separate out xdl_merge() interface.
 + Allow more than true/false to attributes.
 + Document git-check-attr
 + Change attribute negation marker from '!' to '-'.
 + Define a built-in attribute macro "binary".
 + attribute macro support
 + Makefile: add patch-ids.h back in.
 + Fix 'diff' attribute semantics.
 + Fix 'crlf' attribute semantics.
 + Teach 'diff' about 'diff' attribute.
 + Define 'crlf' attribute.
 + Add basic infrastructure to assign attributes to paths

You've heard a lot of noises between me and Linus on this one
for the past few days.  Remaining tasks before it can graduate
to 'master' are:

 (1) I haven't decided how to allow the merge driver override
     the decision to punt on symlinks and some other minor
     policy decision merge-recursive hardcodes yet;

 (2) Not enough test coverage;

 (3) No documentation other than my e-mail messages to the list
     and commit log messages as to how you use this stuff;

 (4) Example "low level merge driver" scripts; covering the same
     set of tools git-mergetool knows about would be good.

Help from the list, probably starting from (3) and (4), would
help polish the series and squash any remaining bugs.

We could do $blobid$ only keywords by hooking into this, but
that would involve philosophical discussion and I'd rather
postpone that and have the infrastructure in 'master' first.

* jp/refs (Tue Apr 17 02:42:50 2007 +0100) 1 commit
 + refs.c: add a function to sort a ref list, rather then sorting on
   add
* jc/quickfetch (Mon Apr 16 00:42:29 2007 -0700) 3 commits
 + Make sure quickfetch is not fooled with a previous, incomplete
   fetch.
 + git-fetch: use fetch--tool pick-rref to avoid local fetch from
   alternate
 + git-fetch--tool pick-rref

I think these two topics should graduate to 'master' this
weekend, as I haven't heard any breakage from anybody, and they
do seem to help.

* lt/gitlink (Sun Apr 15 11:14:28 2007 -0700) 17 commits
 + Expose subprojects as special files to "git diff" machinery
 + Fix some "git ls-files -o" fallout from gitlinks
 + Teach "git-read-tree -u" to check out submodules as a directory
 + Teach git list-objects logic to not follow gitlinks
 + Fix gitlink index entry filesystem matching
 + Teach "git-read-tree -u" to check out submodules as a directory
 + Teach git list-objects logic not to follow gitlinks
 + Don't show gitlink directories when we want "other" files
 + Teach git-update-index about gitlinks
 + Teach directory traversal about subprojects
 + Fix thinko in subproject entry sorting
 + Teach core object handling functions about gitlinks
 + Teach "fsck" not to follow subproject links
 + Add "S_IFDIRLNK" file mode infrastructure for git links
 + Add 'resolve_gitlink_ref()' helper function
 + Avoid overflowing name buffer in deep directory structures
 + diff-lib: use ce_mode_from_stat() rather than messing with modes
   manually

Stalled; Alex has a set of tests that should go on top of this
series but I haven't taken a look at it yet.  I think we should
have enough for interested people to start futzing with, and I
am wondering why nobody has sent a note saying "Hey, I did this
using tree objects with commits in it, it works nicely for these
operations but these things are still cumbersome to do and I
need to polish it more".

* jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 2 commits
 - Make read-cache.c "the_index" free.
 - Move index-related variables into a structure.

A small part of libification; nobody seems to want it.

* jc/blame (Tue Mar 27 01:58:01 2007 -0700) 4 commits
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
 - blame -s: suppress author name and time.

* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

Stalled.

Re: What's cooking in git.git (topics)

[Alex Riesen]

Junio C Hamano, Thu, Apr 19, 2007 02:04:13 +0200:
> 
> Stalled; Alex has a set of tests that should go on top of this
> series but I haven't taken a look at it yet.  I think we should
> have enough for interested people to start futzing with, and I
> am wondering why nobody has sent a note saying "Hey, I did this
> using tree objects with commits in it, it works nicely for these
> operations but these things are still cumbersome to do and I
> need to polish it more".
> 

I am setting up a super-repo for my own very private use (small home
server setup). Still working on what _recursive_ tools do I really
need (and fsck is not the most interesting one: git-diff-files is. Am
afraid of releasing a system I wont ever be able to get to the source
of).

It is, as predicted, becoming mostly work on build infrastructure and
integrity checks in the super-project. Being the sole user of this
project I'll definitely miss all the issues of really big modularized
projects, though.

> 
> * jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 2 commits
>  - Make read-cache.c "the_index" free.
>  - Move index-related variables into a structure.
> 
> A small part of libification; nobody seems to want it.
> 

No user _can_ want it. We need to make the future less a nightmare (it
may not even become one).

Re: What's cooking in git.git (topics)

[Nicolas Pitre]

On Wed, 18 Apr 2007, Junio C Hamano wrote:

> * np/pack (Mon Apr 16 12:32:13 2007 -0400) 25 commits
> 
> Nico's "optionally 64-bit pack idx" aka idx format version 2.
> I've taken another pass at them and am planning to push them out
> on 'master' hopefully this weekend, but a documentation update
> that mention the new --index-version option to git-pack-objects
> would be nice to have before that happens.

Well... In fact I didn't intend for that option to ever be used by 
anything but test scripts in order to trigger some code paths without 
generating jumbo packs (don't know what people would think of me if I 
created a 8GB pack in my test ;-)).  Therefore I didn't think that 
option was worth documenting.

But if you insist I'll send you a patch.


Nicolas

Re: What's cooking in git.git (topics)

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 1457 bytes --]

hoi :)

On Wed, Apr 18, 2007 at 05:04:13PM -0700, Junio C Hamano wrote:
> * lt/gitlink (Sun Apr 15 11:14:28 2007 -0700) 17 commits
> 
> Stalled; Alex has a set of tests that should go on top of this
> series but I haven't taken a look at it yet.  I think we should
> have enough for interested people to start futzing with, and I
> am wondering why nobody has sent a note saying "Hey, I did this
> using tree objects with commits in it, it works nicely for these
> operations but these things are still cumbersome to do and I
> need to polish it more".

You know that I am interested, but sadly I don't have as much time to
really have a look / work on it as I'd liked.  But Linus' series
looks very good from what I've seen now.
Conceptually Linus approach is very much identical my prototyping work
and I am convinced that it is the right way to go.  Only his code is
much better (hey, it's Linus after all ;-) and about our branch-vs-HEAD
discussion, well we'll see how it works out.

Now, how to go on?
The next thing we need is a real checkout & merge support -- but that
is not that hard.
Then we need to think about how to handle the submodule object database,
e.g. when fetching.

I'd also like to be able to support bare supermodule repositories which
include all neccessary submodule objects.  But I guess we need some
more experience with submodules before we can solve that in a scalable
way.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: What's cooking in git.git (topics)

[Junio C Hamano]

Martin Waitz <tali@admingilde.org> writes:

> Now, how to go on?
> The next thing we need is a real checkout & merge support -- but that
> is not that hard.

As git.git is the project that everybody who is interested in
making the feature to materialize fetches, looks at and works on
anyway, once the support at the plumbing level is complete, an
obvious thing to do is to use it in git.git tree itself.

For example, I would like to eventually be able to remove
git-gui/ subdirectory and bind git-gui.git as a subproject.
Another possibility that is probably of a smaller impact is to
bind what is known as 'todo' branch at Meta/ directory, as that
is where I have the branch checked out in my worktree.  People
who are not interested in what are in 'todo' would not mind
having an empty directory there in their checkout, and
interested ones can use the same layout as I do.

Making git.git the first guinea pig has a unique bootstrapping
problem involved, however.  These kind of changes in git.git
itself has to wait at least until what we have in 'next' today
is in everybody's hands.  Otherwise, people who want to use git
for their real work need to first grab a tarball snapshot that
has the plumbing subproject support, and then update to
'master', because we are still too fast moving for any distro
binary packaged version to be satisfactory solution for people
who want to have all the bells and whistles.  Also, I cannot
have subproject in git.git until kernel.org starts running git
with subproject support -- otherwise nobody can clone or pull
from git.git X-<.

If there was a project of lessor importance that can afford to
say "if you want to track this project, you have to use git from
'next', which has not yet been officially released, but we are a
small closely knit group and we can live with this limitation",
it would be easier, but that would not be as effective guinea
pig as git.git itself would be.

Eating our own dog food is how git has evolved since its early
days.  There was no Porcelain to speak of back then; Linus gave
a recipe for keeping track of your work using 'update-index',
'write-tree', 'commit-tree' and 'echo' (we did not even have
'update-ref' to advance the tip of the branch; instead we did
"commit=$(commit-tree) && echo $commit >.git/HEAD"), and people
first followed that recipe, and later wrote a set of thin shell
wrappers around that recipe.

> Then we need to think about how to handle the submodule object
> database, e.g. when fetching.

With the clear separation of connectivity rules between modules,
I do not think this is an issue at all.

Re: What's cooking in git.git (topics)

[Alex Riesen]

On 4/20/07, Junio C Hamano <junkio@cox.net> wrote:
> Making git.git the first guinea pig has a unique bootstrapping
> problem involved, however.  These kind of changes in git.git
> itself has to wait at least until what we have in 'next' today
> is in everybody's hands.

Have you any plans as to when that should begin to happen?
We can warn user if he tries to add a subproject until
porcelain support can be considered usable. It certainly
wont be a problem for early adopters, they know what they're
doing, and an accidental git add of a directory (which by accident
is a git repo all by itself) does not go unnoticed.
Or even disallow it by default (unset dir_struct:dir_links), and
give git add/update-index an option to allow them. We can
reconsider the default later.

Re: What's cooking in git.git (topics)

[Sam Ravnborg]

On Fri, Apr 20, 2007 at 04:14:01AM -0700, Junio C Hamano wrote:
> 
> Making git.git the first guinea pig has a unique bootstrapping
> problem involved, however.  These kind of changes in git.git
> itself has to wait at least until what we have in 'next' today
> is in everybody's hands.  Otherwise, people who want to use git
> for their real work need to first grab a tarball snapshot that
> has the plumbing subproject support, and then update to
> 'master', because we are still too fast moving for any distro
> binary packaged version to be satisfactory solution for people
> who want to have all the bells and whistles.  Also, I cannot
> have subproject in git.git until kernel.org starts running git
> with subproject support -- otherwise nobody can clone or pull
> from git.git X-<.

The bootstrapping issue could be fixed by having a separate
git-subproject.git on kernel.org.

But I see no easy solution for the requireent for kernel.org to
a new git (and I doubt kernel.org sysadmin is too keen to
update to a next-based git).

	Sam

Re: What's cooking in git.git (topics)

[Martin Waitz]

[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]

hoi :)

On Fri, Apr 20, 2007 at 09:31:42PM +0200, Sam Ravnborg wrote:
> On Fri, Apr 20, 2007 at 04:14:01AM -0700, Junio C Hamano wrote:
> > 
> > Making git.git the first guinea pig has a unique bootstrapping
> > problem involved, however.  These kind of changes in git.git
> > itself has to wait at least until what we have in 'next' today
> > is in everybody's hands.  Otherwise, people who want to use git
> > for their real work need to first grab a tarball snapshot that
> > has the plumbing subproject support, and then update to
> > 'master', because we are still too fast moving for any distro
> > binary packaged version to be satisfactory solution for people
> > who want to have all the bells and whistles.  Also, I cannot
> > have subproject in git.git until kernel.org starts running git
> > with subproject support -- otherwise nobody can clone or pull
> > from git.git X-<.
> 
> The bootstrapping issue could be fixed by having a separate
> git-subproject.git on kernel.org.
> 
> But I see no easy solution for the requireent for kernel.org to
> a new git (and I doubt kernel.org sysadmin is too keen to
> update to a next-based git).

Well, it only needs to be new enough to understand enough of
submodules so that it can play the server part.
So once we are in that part to be stable we can merge it to master,
so that kernel.org can use it.
Full submodule support should then mature until the next major version
after which git.git could use it itself.

-- 
Martin Waitz

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: What's cooking in git.git (topics)

[Linus Torvalds]

On Sat, 21 Apr 2007, Martin Waitz wrote:
> On Fri, Apr 20, 2007 at 09:31:42PM +0200, Sam Ravnborg wrote:
> > 
> > But I see no easy solution for the requireent for kernel.org to
> > a new git (and I doubt kernel.org sysadmin is too keen to
> > update to a next-based git).
> 
> Well, it only needs to be new enough to understand enough of
> submodules so that it can play the server part.

Yes. I don't think kernel.org itself really needs more than already exists 
in 'next': it needs the ability to *serve* projects (and that means doing 
the tree traversal properly and know to stop traversing at gitlink 
entries), but kernel.org itself wouldn't actually need any of the 
porcelain at all. The porcelain would all be used on the client sides.

> So once we are in that part to be stable we can merge it to master,
> so that kernel.org can use it.
> Full submodule support should then mature until the next major version
> after which git.git could use it itself.

Yes. I *think* that the gitlink stuff in 'next' is ready to be merged, if 
only because (a) there really hasn't been any disagreement about it (yeah, 
partly probably simply because it was me writing the patches, but I think 
largely because the patches simply were pretty clean!) and (b) there 
aren't any real downsides either, since it won't actually affect any 
non-gitlink use.

So there's certainly the *possible* downside that the whole approach is 
broken and won't work, and merging something broken is pointless. However, 
we've had people thinking about this for quite so time, and I don't think 
anybody seriously believes that it's not a fairly straightforward 
(although probably time-consuming and painful) thing to do all the 
porcelain stuff and it will "just work". So it's _possible_ that there is 
some roadblock that everybody has just ignored, but that just doesn't seem 
very likely.

So it could stay in 'next' until we have everything else in place too, and
the argument for getting it into master literally boils down to the fact 
that it's probably already in a good enough shape for the server side 
(even if the client side is obviously totally missing, and we may find 
*bugs* that are just hiding because it's not used very actively as a 
result). 

I don't really have a huge strong personal feeling either way. I've not 
thought about the patches lately, partly because I'm just fairly happy 
with the core, and partly because I'm just waiting for somebody else to 
start working on it, and then I'll happily jump in and fix any issues that 
come up.

So I would kind of prefer to get it merged sooner rather than later, but 
it's not a huge deal for me - what's more important is probably that 
somebody else rolls up his sleeves and gets dirty with it too ;)

			Linus

[RFR] gitattributes(5) documentation

[Junio C Hamano]

Here is my current draft.  I have build-infrastructure updates
to deal with a new manual section "man5/" (file formats) already
but that is boring stuff, so I am sending out only the new file
to be added for review.

We probably would want to have gitconfig(5) to describe its file
format, and include::config.txt[] in there as well.

Although I do not think it is particularly necessary, we _might_
want to also have gitdata(5) that describes the file format of
$GIT_DIR/index, loose objects, .pack, .idx, packed-refs.

-- >8 -- Documentation/gitattributes.txt -- >8 --

gitattributes(5)
================

NAME
----
gitattributes - defining attributes per path

SYNOPSIS
--------
.gitattributes


DESCRIPTION
-----------

A `gitattributes` file is a simple text file that gives
`attributes` to pathnames.

Each line in `gitattributes` file is of form:

	glob	attr1 attr2 ...

That is, a glob pattern followed by an attributes list,
separated by whitespaces.  When the glob pattern matches the
path in question, the list of attributes are given to the path.

Each attribute can be in one of these states for a given path:

Set::

	The path has the attribute with special value "true";
	this is specified by listing only the name of the
	attribute in the attribute list.

Unset::

	The path has the attribute with special value "false";
	this is specified by listing the name of the attribute
	prefixed with a dash `-` in the attribute list.

Set to a value::

	The path has the attribute with specified string value;
	this is specified by listing the name of the attribute
	followed by an equal sign `=` and its value in the
	attribute list.

Unspecified::

	No glob pattern matches the path, and nothing says if
	the path has or does not have the attribute.

When more than one glob pattern matches the path, a later line
that matches overrides an earlier line.

When deciding what attributes are assigned to a path, git
consults `.gitattributes` file in the same directory as the path
in question, and its parent directories, and then finally
`$GIT_DIR/info/attributes` file, in this order.

Sometimes you would need to override an setting of an attribute
for a path to `unspecified` state.  This can be done by listing
the name of the attribute prefixed with an exclamation point `!`.


EFFECTS
-------

Certain operations by git can be influenced by assigning
particular attributes to a path.  Currently, three operations
are attributes-aware.

Checking-out and checking-in
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The attribute `crlf` affects how the contents stored in the
repository are copied to the working tree files when commands
such as `git checkout` and `git merge` is run.  It also affects
how the contents you prepare in the working tree is stored back
in the repository when you do `git add` and `git commit`.

Set::
	A path to which the `crlf` attribute is set is converted
	to have CRLF line endings in the working tree upon
	checkout, and converted back to strip CRLF line endings
	to LF line endings upon checkin.

Unset::
	A path to which the `crlf` attribute is unset (do not
	confuse this with 'unspecified') does not go through
	line endings conversion upon checkin/checkout.

Unspecified::
	If the configuration variable `core.autocrlf` is false, no
	conversion is done for paths with `crlf` attribute
	unspecified.
+
Othewise, the contents of the path is inspected, and if it does
not look like a text file, no conversion is done.
+
If `core.autocrlf` is true, and the contents of the path does
look like a text file, line endings are converted to CRLF upon
checkout and LF upon checkin.
+
If `core.autocrlf` is set to "input", and the contents of the
path does look like a text file, line endings are converted to
LF upon checkin, but there is no conversion done upon checkout.

Any other value set to `crlf` attribute is ignored and git acts
as if the attribute is left unspecified.


Generating diff text
~~~~~~~~~~~~~~~~~~~~

The attribute `diff` affects if `git diff` generates textual
patch for the path or just says `Binary files differ`.

Set::
	A path to which the `crlf` attribute is set is treated
	as text, even when they contain funny bytes such as NUL.

Unset::
	A path to which the `crlf` attribute is unset will
	generate `Binary files differ`.

Unspecified::
	A path to which the `crlf` attribute is unspecified
	first gets its contents inspected, and if it looks like
	text, it is treated as text.  Otherwise it would
	generate `Binary files differ`.

Any other value set to `diff` attribute is ignored and git acts
as if the attribute is left unspecified.


Performing a three-way merge
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The attribute `merge` affects how three versions of a file is
merged when a file-level merge is necessary during `git merge`,
and other programs such as `git revert` and `git cherry-pick`.

Set::
	Built-in 3-way merge driver is used to merge the
	contents in a way similar to `merge` command of `RCS`
	suite.  This is suitable for ordinary text files.

Unset::
	Take the version from the current branch as the
	tentative merge result, and declare that the merge has
	conflicts.  This is suitable for binary files that does
	not have a well-defined merge semantics.

Unspecified::
	By default, this uses the same built-in 3-way merge
	driver as is the case the `merge` attribute is set.
	However, `merge.default` configuration variable can name
	different merge driver to be used for paths to which the
	`merge` attribute is unspecified.

Other string value::
	3-way merge is performed using the specified custom
	merge driver.  The built-in 3-way merge driver can be
	explicitly specified by asking for "text" driver; the
	built-in "take the current branch" driver can be
	requested by "binary".

Defining a custom merge driver
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The definition of a merge driver is done in `gitconfig` not
`gitattributes` file, so strictly speaking this manual page is a
wrong place to talk about it.  However...

To define a custom merge driver `filfre`, add a section to your
`$GIT_DIR/config` file (or `$HOME/.gitconfig` file) like this:

----------------------------------------------------------------
[merge "filfre"]
	name = feel-free merge driver
	driver = filfre %O %A %B
	recursive = binary
----------------------------------------------------------------

The `merge.*.name` variable gives the driver a human-readable
name.

The `merge.*.driver` variable's value is used to construct a
command to run to merge ancestor's version (`%O`), current
version (`%A`) and the other branches' version (`%B`).  These
three tokens are replaced with the names of temporary files that
hold the contents of these versions when the command line is
built.

The merge driver is expected to leave the result of the merge in
the file named with `%A` by overwriting it, and exit with zero
status if it managed to merge them cleanly, or non-zero if there
were conflicts.

The `merge.*.recursive` variable specifies what other merge
driver to use when the merge driver is called for an internal
merge between common ancestors, when there are more than one.
When left unspecified, the driver itself is used for both
internal merge and the final merge.


EXAMPLE
-------

If you have these three `gitattributes` file:

----------------------------------------------------------------
(in $GIT_DIR/info/attributes)

a*	foo !bar -baz

(in .gitattributes)
abc	foo bar baz

(in t/.gitattributes)
ab*	merge=filfre
abc	-foo -bar
*.c	frotz
----------------------------------------------------------------

the attributes given to path `t/abc` are computed as follows:

1. By examining `t/.gitattributes` (which is in the same
   diretory as the path in question), git finds that the first
   line matches.  `merge` attribute is set.  It also finds that
   the second line matches, and attributes `foo` and `bar`
   are unset.

2. Then it examines `.gitattributes` (which is in the parent
   directory), and finds that the first line matches, but
   `t/.gitattributes` file already decided how `merge`, `foo`
   and `bar` attributes should be given to this path, so it
   leaves `foo` and `bar` unset.  Attribute `baz` is set.

3. Finally it examines `$GIT_DIR/info/gitattributes`.  This file
   is used to override the in-tree settings.  The first line is
   a match, and `foo` is set, `bar` is reverted to unspecified
   state, and `baz` is unset.

As the result, the attributes assignement to `t/abc` becomes:

----------------------------------------------------------------
foo	set to true
bar	unspecified
baz	set to false
merge	set to string value "filfre"
frotz	unspecified
----------------------------------------------------------------


GIT
---
Part of the gitlink:git[7] suite

Re: [RFR] gitattributes(5) documentation

[Linus Torvalds]

Great. And reading the documentation, something struck me: wonderful docs 
about crlf, but it became clear that either the docs are wrong, or the 
behaviour is less than optimal: you cannot specify "crlf=input" any way?

So I would sugegst that
 - if crlf is set, we still honor the value of "core.autocrlf", we just 
   don't care about the *content*.

Maybe that's what the code is doing (I thought it did, but I'm too lazy to 
check), but the docs don't say that:

On Thu, 19 Apr 2007, Junio C Hamano wrote:
> 
> Set::
> 	A path to which the `crlf` attribute is set is converted
> 	to have CRLF line endings in the working tree upon
> 	checkout, and converted back to strip CRLF line endings
> 	to LF line endings upon checkin.

This documented behaviour is non-optimal for a few reasons:
 - it makes it impossible to say "this is text", and have it work on UNIX 
   platforms ;)
 - it makes it impossible to have "autocrlf=input", and then correct one 
   single file that was incorrectly guessed to be binary, and have that 
   file behave like other files.

So I _think_ the right rules are:

 - unspecified: use autocrlf *and* content detection logic
 - unset: never do crlf<->lf ("binary")
 - set: use autocrlf without content detection logic ("text")

with possibly an added rule:

 - set to value: "true" or "input" force that particular setting 
   *regardless* of autocrlf, ie we'd always get CRLF even on UNIX.

Hmm?

			Linus

Re: [RFR] gitattributes(5) documentation

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> This documented behaviour is non-optimal for a few reasons:
>  - it makes it impossible to say "this is text", and have it work on UNIX 
>    platforms ;)
>  - it makes it impossible to have "autocrlf=input", and then correct one 
>    single file that was incorrectly guessed to be binary, and have that 
>    file behave like other files.
>
> So I _think_ the right rules are:
>
>  - unspecified: use autocrlf *and* content detection logic
>  - unset: never do crlf<->lf ("binary")
>  - set: use autocrlf without content detection logic ("text")
>
> with possibly an added rule:
>
>  - set to value: "true" or "input" force that particular setting 
>    *regardless* of autocrlf, ie we'd always get CRLF even on UNIX.

A patch (only compile and testsuite tested but otherwise not
tested) is attached; loses more lines than it adds.

Here is a rewrite of the `crlf` section.

-- >8 --
Checking-out and checking-in
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The attribute `crlf` affects how the contents stored in the
repository are copied to the working tree files when commands
such as `git checkout` and `git merge` run.  It also affects how
git stores the contents you prepare in the working tree in the
repository upon `git add` and `git commit`.

Set::

	Setting the `crlf` attribute on a path is meant to mark
	the path as a "text" file.  'core.autocrlf' conversion
	takes place without guessing the content type by
	inspection.

Unset::

	Unsetting the `crlf` attribute on a path is meant to
	mark the path as a "binary" file.  The path never goes
	through line endings conversion upon checkin/checkout.

Unspecified::

	Unspecified `crlf` attribute tells git to apply the
	`core.autocrlf` conversion when the file content looks
	like text.

Set to string value "input"::

	This is similar to setting the attribute to `true`, but
	also forces git to act as if `core.autocrlf` is set to
	`input` for the path.

Any other value set to `crlf` attribute is ignored and git acts
as if the attribute is left unspecified.


The `core.autocrlf` conversion
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If the configuration variable `core.autocrlf` is false, no
conversion is done.

When `core.autocrlf` is true, it means that the platform wants
CRLF line endings for files in the working tree, and you want to
convert them back to the normal LF line endings when checking
in to the repository.

When `core.autocrlf` is set to "input", line endings are
converted to LF upon checkin, but there is no conversion done
upon checkout.
-- 8< --

 convert.c |   75 +++++++++++++++++++++---------------------------------------
 1 files changed, 26 insertions(+), 49 deletions(-)

diff --git a/convert.c b/convert.c
index a5f60c7..da64253 100644
--- a/convert.c
+++ b/convert.c
@@ -10,6 +10,11 @@
  * translation when the "auto_crlf" option is set.
  */
 
+#define CRLF_GUESS	(-1)
+#define CRLF_BINARY	0
+#define CRLF_TEXT	1
+#define CRLF_INPUT	2
+
 struct text_stat {
 	/* CR, LF and CRLF counts */
 	unsigned cr, lf, crlf;
@@ -74,13 +79,13 @@ static int is_binary(unsigned long size, struct text_stat *stats)
 	return 0;
 }
 
-static int crlf_to_git(const char *path, char **bufp, unsigned long *sizep, int guess)
+static int crlf_to_git(const char *path, char **bufp, unsigned long *sizep, int action)
 {
 	char *buffer, *nbuf;
 	unsigned long size, nsize;
 	struct text_stat stats;
 
-	if (guess && !auto_crlf)
+	if ((action == CRLF_BINARY) || (action == CRLF_GUESS && !auto_crlf))
 		return 0;
 
 	size = *sizep;
@@ -94,7 +99,7 @@ static int crlf_to_git(const char *path, char **bufp, unsigned long *sizep, int
 	if (!stats.cr)
 		return 0;
 
-	if (guess) {
+	if (action == CRLF_GUESS) {
 		/*
 		 * We're currently not going to even try to convert stuff
 		 * that has bare CR characters. Does anybody do that crazy
@@ -119,7 +124,12 @@ static int crlf_to_git(const char *path, char **bufp, unsigned long *sizep, int
 	*bufp = nbuf;
 	*sizep = nsize;
 
-	if (guess) {
+	if (action == CRLF_GUESS) {
+		/*
+		 * If we guessed, we already know we rejected a file with
+		 * lone CR, and we can strip a CR without looking at what
+		 * follow it.
+		 */
 		do {
 			unsigned char c = *buffer++;
 			if (c != '\r')
@@ -136,24 +146,15 @@ static int crlf_to_git(const char *path, char **bufp, unsigned long *sizep, int
 	return 1;
 }
 
-static int autocrlf_to_git(const char *path, char **bufp, unsigned long *sizep)
-{
-	return crlf_to_git(path, bufp, sizep, 1);
-}
-
-static int forcecrlf_to_git(const char *path, char **bufp, unsigned long *sizep)
-{
-	return crlf_to_git(path, bufp, sizep, 0);
-}
-
-static int crlf_to_working_tree(const char *path, char **bufp, unsigned long *sizep, int guess)
+static int crlf_to_worktree(const char *path, char **bufp, unsigned long *sizep, int action)
 {
 	char *buffer, *nbuf;
 	unsigned long size, nsize;
 	struct text_stat stats;
 	unsigned char last;
 
-	if (guess && auto_crlf <= 0)
+	if ((action == CRLF_BINARY) || (action == CRLF_INPUT) ||
+	    (action == CRLF_GUESS && auto_crlf <= 0))
 		return 0;
 
 	size = *sizep;
@@ -171,7 +172,7 @@ static int crlf_to_working_tree(const char *path, char **bufp, unsigned long *si
 	if (stats.lf == stats.crlf)
 		return 0;
 
-	if (guess) {
+	if (action == CRLF_GUESS) {
 		/* If we have any bare CR characters, we're not going to touch it */
 		if (stats.cr != stats.crlf)
 			return 0;
@@ -200,16 +201,6 @@ static int crlf_to_working_tree(const char *path, char **bufp, unsigned long *si
 	return 1;
 }
 
-static int autocrlf_to_working_tree(const char *path, char **bufp, unsigned long *sizep)
-{
-	return crlf_to_working_tree(path, bufp, sizep, 1);
-}
-
-static int forcecrlf_to_working_tree(const char *path, char **bufp, unsigned long *sizep)
-{
-	return crlf_to_working_tree(path, bufp, sizep, 0);
-}
-
 static void setup_crlf_check(struct git_attr_check *check)
 {
 	static struct git_attr *attr_crlf;
@@ -228,38 +219,24 @@ static int git_path_check_crlf(const char *path)
 	if (!git_checkattr(path, 1, &attr_crlf_check)) {
 		const char *value = attr_crlf_check.value;
 		if (ATTR_TRUE(value))
-			return 1;
+			return CRLF_TEXT;
 		else if (ATTR_FALSE(value))
-			return 0;
+			return CRLF_BINARY;
 		else if (ATTR_UNSET(value))
 			;
-		else
-			die("unknown value %s given to 'crlf' attribute",
-			    (char *)value);
+		else if (!strcmp(value, "input"))
+			return CRLF_INPUT;
+		/* fallthru */
 	}
-	return -1;
+	return CRLF_GUESS;
 }
 
 int convert_to_git(const char *path, char **bufp, unsigned long *sizep)
 {
-	switch (git_path_check_crlf(path)) {
-	case 0:
-		return 0;
-	case 1:
-		return forcecrlf_to_git(path, bufp, sizep);
-	default:
-		return autocrlf_to_git(path, bufp, sizep);
-	}
+	return crlf_to_git(path, bufp, sizep, git_path_check_crlf(path));
 }
 
 int convert_to_working_tree(const char *path, char **bufp, unsigned long *sizep)
 {
-	switch (git_path_check_crlf(path)) {
-	case 0:
-		return 0;
-	case 1:
-		return forcecrlf_to_working_tree(path, bufp, sizep);
-	default:
-		return autocrlf_to_working_tree(path, bufp, sizep);
-	}
+	return crlf_to_worktree(path, bufp, sizep, git_path_check_crlf(path));
 }

Re: [RFR] gitattributes(5) documentation

[David Lang]

On Thu, 19 Apr 2007, Junio C Hamano wrote:

> Set to string value "input"::
>
> 	This is similar to setting the attribute to `true`, but
> 	also forces git to act as if `core.autocrlf` is set to
> 	`input` for the path.
>
> Any other value set to `crlf` attribute is ignored and git acts
> as if the attribute is left unspecified.

I think that a better option would be that if it's set to a string value, that 
string value is treated as if core.autocrlf was set to that value.

in the long run this would let you phase out the core.autocrlf option entirely, 
letting the bahavior be specified in gitattributes.

David Lang

Re: [RFR] gitattributes(5) documentation

[Junio C Hamano]

David Lang <david.lang@digitalinsight.com> writes:

> in the long run this would let you phase out the core.autocrlf option
> entirely, letting the bahavior be specified in gitattributes.

You _could_, but that is quite against what we want.  These
should stay separate, and the gitattributes mechanism is
designed specifically to allow them cleanly separated.

The configuration "core.autcrlf" describes a particular
repository.  If the platform the repository is on expects text
files to be line-terminated with CRLF, you would have
core.autocrlf set; otherwise you don't.

On the other hand, gitattributes' 'crlf' describes if the path
is text, and that is the reason it can and should be "in-tree",
i.e. not just $GIT_DIR/info/attributes (which is private to the
repository) but in .gitattributes (and subdirectories'), which
is given to everybody who has a copy of the project.

How text files are handled is a local matter, and stays in the
config.  Which ones are text is the same for everybody who has a
copy of the project, and is in-tree information.

Re: [RFR] gitattributes(5) documentation

[David Lang]

On Sun, 22 Apr 2007, Junio C Hamano wrote:

> David Lang <david.lang@digitalinsight.com> writes:
>
>> in the long run this would let you phase out the core.autocrlf option
>> entirely, letting the bahavior be specified in gitattributes.
>
> You _could_, but that is quite against what we want.  These
> should stay separate, and the gitattributes mechanism is
> designed specifically to allow them cleanly separated.
>
> The configuration "core.autcrlf" describes a particular
> repository.  If the platform the repository is on expects text
> files to be line-terminated with CRLF, you would have
> core.autocrlf set; otherwise you don't.
>
> On the other hand, gitattributes' 'crlf' describes if the path
> is text, and that is the reason it can and should be "in-tree",
> i.e. not just $GIT_DIR/info/attributes (which is private to the
> repository) but in .gitattributes (and subdirectories'), which
> is given to everybody who has a copy of the project.

I was thinking that it should be in $GIT_DIR/info/gitattributes along with the 
rest of the crlf defintitions.

> How text files are handled is a local matter, and stays in the
> config.  Which ones are text is the same for everybody who has a
> copy of the project, and is in-tree information.

I understand what you are aiming for, but you are depending on people doing the 
defining of what files are text in the .gitattributes files instead of 
$GIT_DIR/info/gitattributes, which is also valid to do with things as currently 
defined (at least if I'm understanding them correctly)

what you really do want for crlf is one variable that, if set, uses the value of 
another variable.

i wonder if this is useful enough to define formally. something along the lines 
of
[default] crlf=input
in the $GIT_DIR/info/gitattributes file

at the moment we have crlf (set by core.autocrlf) and merge (set by the 
environment variable), I think I saw that it may also be possible to define a 
different diff to use by default as well (possibly by useing a third method to 
define the default, possibly by an environment variable, I don't remember)

if you could set the defaults in the $GIT_DIR/info/gitattributes file and then 
have the flag set/unset/set-to-value in the whatever gitattributes file(s) are 
appropriate, it would consolodate the configurations into one related place 
rather then spreading them around with different ways to set the defaults for 
different things.

David Lang

Re: [RFR] gitattributes(5) documentation

[Nicolas Pitre]

On Thu, 19 Apr 2007, Junio C Hamano wrote:

> Generating diff text
> ~~~~~~~~~~~~~~~~~~~~
> 
> The attribute `diff` affects if `git diff` generates textual
> patch for the path or just says `Binary files differ`.
> 
> Set::
> 	A path to which the `crlf` attribute is set is treated
                             ^^^^

> 	as text, even when they contain funny bytes such as NUL.
> 
> Unset::
> 	A path to which the `crlf` attribute is unset will
                             ^^^^

> 	generate `Binary files differ`.
> 
> Unspecified::
> 	A path to which the `crlf` attribute is unspecified
                             ^^^^

Remnants of a cut'n paste?


Nicolas

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

* jc/attr (Sat Apr 21 19:09:02 2007 -0700) 2 commits
 - Add 'ident' conversion.
 - Add 'filter' attribute and external filter driver definition.

This is the remaining "controversial" bits.

* lt/objalloc (Mon Apr 16 22:13:09 2007 -0700) 1 commit
 . Make the object lookup hash use a "object index" instead of a
   pointer
* jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 2 commits
 - Make read-cache.c "the_index" free.
 - Move index-related variables into a structure.
* jc/blame (Tue Mar 27 01:58:01 2007 -0700) 4 commits
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
 - blame -s: suppress author name and time.
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

The rest are stalled.

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

* jc/the-index (Sun Apr 1 23:26:07 2007 -0700) 2 commits
 + Make read-cache.c "the_index" free.
 + Move index-related variables into a structure.

I gave a brief look at the beginning of libification in
lcapitulino's repository at repo.or.cz, and I think this is
related to his topic, so instead of leaving this in limbo, I'm
planning to merge this in v1.5.2-rc1, hopefully to make the
later merge easier.

* mk/diff (Sun Apr 22 23:56:22 2007 -0700) 6 commits
 - Diff between two blobs should take mode changes into account now.
 - use mode of the tree in git-diff, if <tree>:<file> syntax is used
 - store mode in rev_list, if <tree>:<filename> syntax is used
 - add add_object_array_with_mode
 - add get_sha1_with_mode
 - Add S_IFINVALID mode

This attempts to do something we wanted to do for a long time
(the comment removed from the top of builtin-diff.c with this
series has been there for almost a year).  I haven't tried it
yet myself; it needs a few test.  This may help some parts of
gitweb so it would be desirable if we can fast-track this by
v1.5.2-rc1.

* jc/attr (Sat Apr 21 03:14:13 2007 -0700) 2 commits
 - Add 'filter' attribute and external filter driver definition.
 - Add 'ident' conversion.

As 'ident' conversion is stateless, I do not mind too much
including it in v1.5.2-rc1.  On the other hand, the arbitrary
'filter' is quite contentious, although the character-code
conversion example I gave myself might be a good enough reason
for people to want it.  Undecided.

* lt/objalloc (Mon Apr 16 22:13:09 2007 -0700) 1 commit
 - Make the object lookup hash use a "object index" instead of a
   pointer
* jc/blame (Tue Mar 27 01:58:01 2007 -0700) 4 commits
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
 - blame -s: suppress author name and time.
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

These are not considered for v1.5.2.

Re: What's cooking in git.git (topics)

[Nicolas Pitre]

On Mon, 23 Apr 2007, Junio C Hamano wrote:

> As 'ident' conversion is stateless, I do not mind too much
> including it in v1.5.2-rc1.  On the other hand, the arbitrary
> 'filter' is quite contentious, although the character-code
> conversion example I gave myself might be a good enough reason
> for people to want it.  Undecided.

Like I said there are certainly plenty of good (and bad) reasons for 
using this facility, and many of them we might not imagine now.  Since 
the code is already written I think you should include it.


Nicolas

Re: What's cooking in git.git (topics)

[Alex Riesen]

On 4/23/07, Junio C Hamano <junkio@cox.net> wrote:
> * jc/attr (Sat Apr 21 03:14:13 2007 -0700) 2 commits
>  - Add 'filter' attribute and external filter driver definition.
>  - Add 'ident' conversion.
>
> As 'ident' conversion is stateless, I do not mind too much
> including it in v1.5.2-rc1.  On the other hand, the arbitrary
> 'filter' is quite contentious, although the character-code
> conversion example I gave myself might be a good enough reason
> for people to want it.  Undecided.

Can I suggest a config option to completely disable content
munging code? So that people who really care about the
real content, or just don't have the tools for the filters still
can checkout the repos depending on the filters.

Re: What's cooking in git.git (topics)

[Junio C Hamano]

"Alex Riesen" <raa.lkml@gmail.com> writes:

> On 4/23/07, Junio C Hamano <junkio@cox.net> wrote:
>> * jc/attr (Sat Apr 21 03:14:13 2007 -0700) 2 commits
>>  - Add 'filter' attribute and external filter driver definition.
>>  - Add 'ident' conversion.
>>
>> As 'ident' conversion is stateless, I do not mind too much
>> including it in v1.5.2-rc1.  On the other hand, the arbitrary
>> 'filter' is quite contentious, although the character-code
>> conversion example I gave myself might be a good enough reason
>> for people to want it.  Undecided.
>
> Can I suggest a config option to completely disable content
> munging code? So that people who really care about the
> real content, or just don't have the tools for the filters still
> can checkout the repos depending on the filters.

The code may have bugs, but the intent is that you can have this
line in your $GIT_DIR/info/attributes to override whatever
attribute settings used in .gitattributes files that are
in-tree:

	*	!ident !filter

Re: What's cooking in git.git (topics)

[Alex Riesen]

Junio C Hamano, Mon, Apr 23, 2007 19:15:16 +0200:
> >> As 'ident' conversion is stateless, I do not mind too much
> >> including it in v1.5.2-rc1.  On the other hand, the arbitrary
> >> 'filter' is quite contentious, although the character-code
> >> conversion example I gave myself might be a good enough reason
> >> for people to want it.  Undecided.
> >
> > Can I suggest a config option to completely disable content
> > munging code? So that people who really care about the
> > real content, or just don't have the tools for the filters still
> > can checkout the repos depending on the filters.
> 
> The code may have bugs, but the intent is that you can have this
> line in your $GIT_DIR/info/attributes to override whatever
> attribute settings used in .gitattributes files that are
> in-tree:
> 
> 	*	!ident !filter
> 

Imagine a project which started using the attributes at some point of
time. And imagine developers whose repos suddenly start breaking
because of clueless integrator created a filter which does not work
anywere but his system (typical, really) and didn't tell anyone to
update their configuration (whereas .gitattribute files are in working
trees already).

How do you suggest to distribute filter configurations, BTW?
They are not cloned (can they?)
How about checkout performance impact? (in case they are not active,
of course. You're hosed anyway if the filters used. Especially if you
happen to have real big files).

Re: What's cooking in git.git (topics)

[Junio C Hamano]

Alex Riesen <raa.lkml@gmail.com> writes:

> Imagine a project which started using the attributes at some point of
> time. And imagine developers whose repos suddenly start breaking
> because of clueless integrator created a filter which does not work
> anywere but his system (typical, really) and didn't tell anyone to
> update their configuration (whereas .gitattribute files are in working
> trees already).

That's one of the reasons why only the filter names are assigned
to paths using gitattributes mechanism and what action to take
when a specific filter name is attached to a path is determined
by the config.  Missing filter driver definition in the config
is not an error but makes the filter a no-op passthru.

The content filtering is to massage the content into a shape
that is more convenient for the platform/filesystem/the user to
use.  The keyword here is "more convenient" and not "usable"; in
other words, it is "hanging yourself because we gave you a long
rope" if your project tries to do something with the filtering
mechanism to make your project unusable unless the checkout is
done with specific filter in effect.  So defaulting to passthru
is meant to fall-back on the plain-old inconvenient checkout,
which is not a bad thing.

> How do you suggest to distribute filter configurations, BTW?

The same project description message the participant learn about
the project that says the public repository locations and such,
and perhaps in-tree READ.ME file.

The earlier example I gave would fit this pattern rather well.
If somebody (me) cannot deal with UTF-8 encoded Japanese text
very well, that user personally can mark such a file in
$GIT_DIR/info/attributes as 'filter=utf8-japanese-text' and
define the iconv based filtering driver in $GIT_DIR/config in
the repository that he (me) uses for editing.

In addition, I would most likely have another repository that
does not have the filtering driver defined, and that would be
where I would run the build tools for documentation part, since
the project documentation is supposed to be in UTF-8.  

This is a "purely personal" setting that does not have to be
known to the outside world.  But the filter=utf8-japanese-text
attribute could be shared in-tree if the project has more then
one person with difficulty dealing with UTF-8 encoded Japanese
text.  I may personally edit the file after having iconv convert
to EUC-JP and convert it back to UTF-8 when checking in, but the
other person may use local encoding different from EUC-JP for
editing.  In such a case, only the definition in our config
files are different, and in-tree Documentation/.gitattributes
file would have

	git-lost-found.txt	filter=utf8-japanese-text

which is distributed project-wide.

Repositories used by people who do not have trouble handling
UTF-8 encoded Japanese text would not have any filtering driver
defined for utf8-japanese-text in their $GIT_DIR/config, and
their checkout would be in UTF-8, because of this passthru
behaviour.

> How about checkout performance impact?

Measurement would be interesting; I haven't done it, and that is
one of the smaller reasons I am not particularly keen on pushing
the 'filter' attribute.  Hawk-eyed people might have noticed
that I swapped the order of the series in 'pu' to have 'ident'
first and then 'filter'.

Re: What's cooking in git.git (topics)

[Alex Riesen]

On 4/23/07, Junio C Hamano <junkio@cox.net> wrote:
> Alex Riesen <raa.lkml@gmail.com> writes:
>
> > Imagine a project which started using the attributes at some point of
> > time. And imagine developers whose repos suddenly start breaking
> > because of clueless integrator created a filter which does not work
> > anywere but his system (typical, really) and didn't tell anyone to
> > update their configuration (whereas .gitattribute files are in working
> > trees already).
>
> That's one of the reasons why only the filter names are assigned
> to paths using gitattributes mechanism and what action to take
> when a specific filter name is attached to a path is determined
> by the config.  Missing filter driver definition in the config
> is not an error but makes the filter a no-op passthru.

Fragile. What if content is useless without filter? How does
the user know about the fact so he can work the problem
around?

What if you have multiple filters matching the same path?
(does not seem to be possible. Someone will ask you why)

> The content filtering is to massage the content into a shape
> that is more convenient for the platform/filesystem/the user to
> use.  The keyword here is "more convenient" and not "usable"; in

how can "not usable" be "more convenient"?

> > How do you suggest to distribute filter configurations, BTW?
>
> The same project description message the participant learn about
> the project that says the public repository locations and such,
> and perhaps in-tree READ.ME file.

But there seem to be no way to notice that the READ.ME should
be reread by project participants downstream.

> The earlier example I gave would fit this pattern rather well.
> If somebody (me) cannot deal with UTF-8 encoded Japanese text
> very well, that user personally can mark such a file in
> $GIT_DIR/info/attributes as 'filter=utf8-japanese-text' and
> define the iconv based filtering driver in $GIT_DIR/config in
> the repository that he (me) uses for editing.

which will be a PITA to setup in each and every clone of the
repository, unless it is cloned with the repo.

Re: What's cooking in git.git (topics)

[Johannes Schindelin]

Hi,

On Tue, 24 Apr 2007, Alex Riesen wrote:

> On 4/23/07, Junio C Hamano <junkio@cox.net> wrote:
>
> > The earlier example I gave would fit this pattern rather well.
> > If somebody (me) cannot deal with UTF-8 encoded Japanese text
> > very well, that user personally can mark such a file in
> > $GIT_DIR/info/attributes as 'filter=utf8-japanese-text' and
> > define the iconv based filtering driver in $GIT_DIR/config in
> > the repository that he (me) uses for editing.
> 
> which will be a PITA to setup in each and every clone of the
> repository, unless it is cloned with the repo.

Not if you do it with templates. If it is such a special case that you 
absolutely _need_ filters, and cannot use it without filters, it is 
probably in a very small group. And there, you just setup the templates, 
and voila: you have your filters without much ado.

Ciao,
Dscho

Re: What's cooking in git.git (topics)

[Alex Riesen]

On 4/24/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> >
> > which will be a PITA to setup in each and every clone of the
> > repository, unless it is cloned with the repo.
>
> Not if you do it with templates. If it is such a special case that you
> absolutely _need_ filters, and cannot use it without filters, it is
> probably in a very small group. And there, you just setup the templates,
> and voila: you have your filters without much ado.

It can be a very big group. Than, even if it is the only group in the world,
it can complain loud and long enough to become a major annoyance.

Re: What's cooking in git.git (topics)

[Johannes Schindelin]

Hi,

On Tue, 24 Apr 2007, Alex Riesen wrote:

> On 4/24/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > which will be a PITA to setup in each and every clone of the
> > > repository, unless it is cloned with the repo.
> >
> > Not if you do it with templates. If it is such a special case that you 
> > absolutely _need_ filters, and cannot use it without filters, it is 
> > probably in a very small group. And there, you just setup the 
> > templates, and voila: you have your filters without much ado.
> 
> It can be a very big group. Than, even if it is the only group in the 
> world, it can complain loud and long enough to become a major annoyance.

Yes, they can.

And if we can prove that it would have been cleaner and better and more 
stable to do the same without attributes, they will look like a big group 
of total morons.

Ciao,
Dscho

Re: What's cooking in git.git (topics)

[Junio C Hamano]

"Alex Riesen" <raa.lkml@gmail.com> writes:

> On 4/23/07, Junio C Hamano <junkio@cox.net> wrote:
>> ...
>> That's one of the reasons why only the filter names are assigned
>> to paths using gitattributes mechanism and what action to take
>> when a specific filter name is attached to a path is determined
>> by the config.  Missing filter driver definition in the config
>> is not an error but makes the filter a no-op passthru.
>
> Fragile. What if content is useless without filter?

In that case, the project screwed itself and it is not our
problem anymore ;-).

>> The content filtering is to massage the content into a shape
>> that is more convenient for the platform/filesystem/the user to
>> use.  The keyword here is "more convenient" and not "usable"; in
>
> how can "not usable" be "more convenient"?

I think I worded it incorrectly to be misunderstood, but I
couldn't word them better then, I do not know I can word them
better now.

Something could be 1. unusable, or 2. usable.  Among usable
shapes, there are 2-a. inconvenient but usable and 2-b. very
convenient to use.

What I tried to say was that if you use filtering mechanism to
massage contents that is unusable into usable (i.e. crossing
from 1 to 2), you are already misusing the mechanism (but we do
not prevent you because we are only "giving you a long rope").
The filter is meant to be used to cross from 2-a to 2-b.

Re: What's cooking in git.git (topics)

[Alex Riesen]

On 4/24/07, Junio C Hamano <junkio@cox.net> wrote:
> >> The content filtering is to massage the content into a shape
> >> that is more convenient for the platform/filesystem/the user to
> >> use.  The keyword here is "more convenient" and not "usable"; in
> >
> > how can "not usable" be "more convenient"?
>
> I think I worded it incorrectly to be misunderstood, but I
> couldn't word them better then, I do not know I can word them
> better now.
>

You don't have to. I just can't force myself to believe it can be
made useful. I'll shut up for now, and wait until I or someone else
proves the code has negligible negative impact on the normal
usage scenarios.

Re: What's cooking in git.git (topics)

[Johannes Schindelin]

Hi,

On Mon, 23 Apr 2007, Alex Riesen wrote:

> On 4/23/07, Junio C Hamano <junkio@cox.net> wrote:
> > * jc/attr (Sat Apr 21 03:14:13 2007 -0700) 2 commits
> >  - Add 'filter' attribute and external filter driver definition.
> >  - Add 'ident' conversion.
> >
> > As 'ident' conversion is stateless, I do not mind too much
> > including it in v1.5.2-rc1.  On the other hand, the arbitrary
> > 'filter' is quite contentious, although the character-code
> > conversion example I gave myself might be a good enough reason
> > for people to want it.  Undecided.
> 
> Can I suggest a config option to completely disable content
> munging code? So that people who really care about the
> real content, or just don't have the tools for the filters still
> can checkout the repos depending on the filters.

In my worldview, these filters are a local thing. Exactly like crlf. So, 
no need for a config option.

Ciao,
Dscho

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

* jc/attr (Sat Apr 21 03:14:13 2007 -0700) 2 commits
 + Add 'filter' attribute and external filter driver definition.
 + Add 'ident' conversion.

As two people on the list whose judgement on design issues I
trust both say "give them rope is Ok", perhaps I should push
this out to 'master' before v1.5.2-rc1.  I am still worried
about the rope being too long, though, and tried to describe the
intent and limitation in the documentation to prevent users from
hurting themselves, but I do not think the descriptions I have
are good enough yet.

* jc/blame (Fri Apr 27 00:42:15 2007 -0700) 7 commits
 - Apply mailmap in git-blame output.
 - Split out mailmap handling out of shortlog
 - blame: show log as it goes
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
 - blame -s: suppress author name and time.

In addition to the update to use .mailmap, this has the "log"
output that uses the blame machinery Linus suggested.  I think I
know what more are needed to make it more pleasant to use, but
the necessary changes seem a bit too involved.  I might advance
the topic a bit more during the stabilization period for v1.5.2,
but I am planning to leave the actual merge until v1.5.3 cycle.

* lt/objalloc (Mon Apr 16 22:13:09 2007 -0700) 1 commit
 - Make the object lookup hash use a "object index" instead of a
   pointer
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

These are stalled.

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

Well, everything meant for v1.5.2 is in 'master' now.  There is
nothing to see here.

* jc/blame (Fri Apr 20 16:25:50 2007 -0700) 4 commits
 - blame: show log as it goes
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up
* lt/objalloc (Mon Apr 16 22:13:09 2007 -0700) 1 commit
 - Make the object lookup hash use a "object index" instead of a
   pointer
* jc/diff (Mon Dec 25 01:08:50 2006 -0800) 2 commits
 - test-para: combined diff between HEAD, index and working tree.
 - para-walk: walk n trees, index and working tree in parallel

Re: What's cooking in git.git (topics)

[Linus Torvalds]

On Sun, 29 Apr 2007, Junio C Hamano wrote:
>
> * lt/objalloc (Mon Apr 16 22:13:09 2007 -0700) 1 commit
>  - Make the object lookup hash use a "object index" instead of a
>    pointer

I think you should just drop this. 

You merged the two patches that made this possible, and the third in the 
series isn't really worth it. I can re-create it at will (maybe better) 
now that the core object allocations are all cleaned up, and that patch 
simply didn't give enough of an advantage to be worth it.

Maybe inlining the object index -> ptr conversion would have solved the 
performance regression, but it migth also be something more fundamental, 
like the potential extra cache miss in the converstion part (when looking 
up the allocation block).

So it was interesting to try, but I don't think it's worth carrying around 
considering the results.

		Linus

Re: What's cooking in git.git (topics)

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Sun, 29 Apr 2007, Junio C Hamano wrote:
>>
>> * lt/objalloc (Mon Apr 16 22:13:09 2007 -0700) 1 commit
>>  - Make the object lookup hash use a "object index" instead of a
>>    pointer
>
> I think you should just drop this. 

Yeah.  I was mostly concentrating on maint/master for the past
several days, and blindly carrying it around was cheaper than
deciding to drop it in my workflow.

What's cooking in git.git (topics)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.  The topics list the commits in reverse chronological
order.

As I have been sick and am mostly concentrating on fixes on
'master' anyway, there aren't much to see here.  Indeed,
'master' and 'next' still have identical trees.

* fl/cvsserver (Wed May 2 02:45:22 2007 +0200) 1 commit
 - cvsserver: Add test cases for git-cvsserver

I need to ask k.org people to install sqlite and
libdvd-sqlite-perl before I can advance this to 'next'.

* jc/blame (Fri Apr 20 16:25:50 2007 -0700) 4 commits
 - blame: show log as it goes
 - git-blame: optimize get_origin() from linear search to hash-
   lookup.
 - git-blame: pass "struct scoreboard *" pointers around.
 - blame: lift structure definitions up