[RFH] hackday and GSoC topic suggestions

[RFH] hackday and GSoC topic suggestions

[Jeff King]

This Saturday I'm going to be attending a Git hackday held by Bloomberg
in New York. The participants will be eager C coders who have experience
using git, but not contributing to it. As somebody who has read The
Mythical Man Month, I don't expect huge productivity, but I'm hoping to
do some bug triage and fixes, and maybe get some people involved who
might join the community. I'm hoping some of the folks will have
features _they_ want to work on, and I can help get them started. But
I'd like to have a list of potential projects to direct people towards
as a backup.

On a similar note, the GSoC application deadline is Feb 14th. I am
happy to be admin again and put together the application, but we will
need an idea page. I'll set up a page to collect them, but in the
meantime, please dump any ideas/discussion in this thread.

Below is a list of features / bugs that I am taking to the hackday. The
bug list was collected by grepping the mailing list for items without
responses. They haven't been triaged at all, so I'm sure some of them
are "not a bug" or "won't fix".  But the triage process is part of what
we'll be doing on Saturday.

I don't think the list below and the potential GSoC list really have any
overlap, as the project scales are completely different. But I'd be
happy to take suggestions for either.

---
features:

 - negative refspecs
   http://thread.gmane.org/gmane.comp.version-control.git/240997/focus=241019

 - optionally remove tempfiles on failed pack-objects (especially when
   we get ENOSPC)
   http://article.gmane.org/gmane.comp.version-control.git/241466

 - previewing "git pull"
   http://article.gmane.org/gmane.comp.version-control.git/236732

bugs:

 - branch.*.merge interpreted too strictly by tracking logic
   http://article.gmane.org/gmane.comp.version-control.git/241582

 - relative core.worktree is resolved from symlink and not its target
   http://article.gmane.org/gmane.comp.version-control.git/241519

 - Branch rename breaks local downstream branches
   http://article.gmane.org/gmane.comp.version-control.git/241228

 - git clone on out-of-space device causes incorrect errors
   http://article.gmane.org/gmane.comp.version-control.git/241206

 - commit-msg hook and merges
   http://article.gmane.org/gmane.comp.version-control.git/241203

 - inconsistent include behaviour for core.sharedRepository
   http://article.gmane.org/gmane.comp.version-control.git/241277

 - Rebase options '--whitespace=fix' and '--keep-empty' are incompatible
   http://article.gmane.org/gmane.comp.version-control.git/238055

 - git stash doesn't use --index as default
   http://article.gmane.org/gmane.comp.version-control.git/235892

 - git describe --contains --abbrev=0 <sha1> doesn't work as expected
   http://article.gmane.org/gmane.comp.version-control.git/236707

 - using git commit-tree with "-F -" adds trailing newlines
   http://article.gmane.org/gmane.comp.version-control.git/236583

 - Pull and fetch don't honor `--progress` flag
   http://thread.gmane.org/gmane.comp.version-control.git/236257/focus=236262

 - Unexpected outputs of git pull on stdout v.s. stderr
   http://article.gmane.org/gmane.comp.version-control.git/235716

 - MERGE_HEAD lost with git checkout master
   http://article.gmane.org/gmane.comp.version-control.git/233806

 - git stash doesn't always save work dir as-is: bug?
   http://article.gmane.org/gmane.comp.version-control.git/234153

 - Well-past commit dates unsupported
   http://article.gmane.org/gmane.comp.version-control.git/236827

 - we do not handle integer overflow in commit/author timestamps
   [no reference, but something I have noticed; we should probably
   return a sentinel "0" rather than a random overflow value, and
   we should probably detect and warn in git-fsck]

 - git stash does not work when directory is replaced by a symlink to itself
   http://article.gmane.org/gmane.comp.version-control.git/236798

 - rebase not recovering gracefully from repack error
   http://article.gmane.org/gmane.comp.version-control.git/234261

 - (broken ?) output of "git diff --color-word"
   http://article.gmane.org/gmane.comp.version-control.git/237370

 - git filter-branch does not make tree replacements permanent
   http://article.gmane.org/gmane.comp.version-control.git/220931

 - 'git show' gives duplicate errors for ambiguous args
   http://article.gmane.org/gmane.comp.version-control.git/205023

 - (git commit --patch --message $MESSAGE) disallows hunk editing
   http://article.gmane.org/gmane.comp.version-control.git/208213

 - filter-branch --parent-filter in bare repository
   http://article.gmane.org/gmane.comp.version-control.git/203619

insanely hard bugs:

 - shallow clones over http
   http://article.gmane.org/gmane.comp.version-control.git/223682

Re: [RFH] hackday and GSoC topic suggestions

[Christian Couder]

On Wed, Feb 5, 2014 at 11:57 PM, Jeff King <peff@peff.net> wrote:
>
> On a similar note, the GSoC application deadline is Feb 14th. I am
> happy to be admin again and put together the application, but we will
> need an idea page. I'll set up a page to collect them, but in the
> meantime, please dump any ideas/discussion in this thread.

Matthieu Moy created this nice page some time ago in the wiki:

https://git.wiki.kernel.org/index.php/SmallProjectsIdeas

and I think there is good stuff in it.

I would be interested in mentoring a GSoC student working on the "git
bisect fix/unfixed" feature.
Some of Matthieu's students worked on it a few years ago but didn't finish.

Thanks,
Christian.

Re: [RFH] hackday and GSoC topic suggestions

[Matthieu Moy]

Christian Couder <christian.couder@gmail.com> writes:

> I would be interested in mentoring a GSoC student working on the "git
> bisect fix/unfixed" feature.

Interestingly, I used the feature in real-life last week, and had to
think upside-down to type the right "good"/"bad" keywords ;-).

> Some of Matthieu's students worked on it a few years ago but didn't finish.

Right. There was still quite some work to do, but this is most likely
too small for a GSoC project. But that could be a part of it. I'm not
sure how google welcomes GSoC projects made of multiple small tasks, but
my experience with students is that it's much better than a single (too)
big task, and I think that was the general feeling on this list when we
discussed it last year.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [RFH] hackday and GSoC topic suggestions

[Jeff King]

On Thu, Feb 06, 2014 at 10:51:54AM +0100, Matthieu Moy wrote:

> > Some of Matthieu's students worked on it a few years ago but didn't finish.
> 
> Right. There was still quite some work to do, but this is most likely
> too small for a GSoC project. But that could be a part of it. I'm not
> sure how google welcomes GSoC projects made of multiple small tasks, but
> my experience with students is that it's much better than a single (too)
> big task, and I think that was the general feeling on this list when we
> discussed it last year.

I think Google leaves it up to us to decide. I'd be OK with a project
made of multiple small tasks, as I think it would be an interesting
experiment.  I'd rather not do all of them like that, though. And bonus
points if they are on a theme that will let the student use the ramp-up
time from one for another.

-Peff

Re: [RFH] hackday and GSoC topic suggestions

[Christian Couder]

On Thu, Feb 13, 2014 at 9:50 AM, Jeff King <peff@peff.net> wrote:
> On Thu, Feb 06, 2014 at 10:51:54AM +0100, Matthieu Moy wrote:
>
>> > Some of Matthieu's students worked on it a few years ago but didn't finish.
>>
>> Right. There was still quite some work to do, but this is most likely
>> too small for a GSoC project. But that could be a part of it. I'm not
>> sure how google welcomes GSoC projects made of multiple small tasks, but
>> my experience with students is that it's much better than a single (too)
>> big task, and I think that was the general feeling on this list when we
>> discussed it last year.
>
> I think Google leaves it up to us to decide. I'd be OK with a project
> made of multiple small tasks, as I think it would be an interesting
> experiment.  I'd rather not do all of them like that, though. And bonus
> points if they are on a theme that will let the student use the ramp-up
> time from one for another.

Yeah, a student working on the "git bisect fix/unfixed" feature, could
fix git bisect testing too many merge bases, and if there is still
time work on moving more code from shell to C.

Re: [RFH] hackday and GSoC topic suggestions

[David Kastrup]

Christian Couder <christian.couder@gmail.com> writes:

> On Thu, Feb 13, 2014 at 9:50 AM, Jeff King <peff@peff.net> wrote:
>
>> I think Google leaves it up to us to decide. I'd be OK with a project
>> made of multiple small tasks, as I think it would be an interesting
>> experiment.  I'd rather not do all of them like that, though. And
>> bonus points if they are on a theme that will let the student use the
>> ramp-up time from one for another.
>
> Yeah, a student working on the "git bisect fix/unfixed" feature, could
> fix git bisect testing too many merge bases, and if there is still
> time work on moving more code from shell to C.

In the context of programming tasks, "if there is still time" is a prime
candidate for the successful application of branch prediction.

-- 
David Kastrup

Re: [RFH] hackday and GSoC topic suggestions

[Thomas Rast]

Jeff King <peff@peff.net> writes:

> On a similar note, the GSoC application deadline is Feb 14th. I am
> happy to be admin again and put together the application, but we will
> need an idea page. I'll set up a page to collect them, but in the
> meantime, please dump any ideas/discussion in this thread.

Sorry for being so procrastinative :-(

I suggest we make it a rule that old projects cannot be proposed from
year to year.

-- 
Thomas Rast
tr@thomasrast.ch

Re: [RFH] hackday and GSoC topic suggestions

[Jeff King]

On Sat, Feb 08, 2014 at 07:43:40PM +0100, Thomas Rast wrote:

> Jeff King <peff@peff.net> writes:
> 
> > On a similar note, the GSoC application deadline is Feb 14th. I am
> > happy to be admin again and put together the application, but we will
> > need an idea page. I'll set up a page to collect them, but in the
> > meantime, please dump any ideas/discussion in this thread.
> 
> Sorry for being so procrastinative :-(
> 
> I suggest we make it a rule that old projects cannot be proposed from
> year to year.

I would very much agree with that. There are projects that I have posted
multiple years, and nobody ever selects them. I should take that as a
hint that they are not appealing to students for whatever reason, and
are probably just clogging up the ideas list.

I'd make an exception for somebody who wants to substantially rework the
proposal in an attempt to make it more appealing, or who has a reason
why it is being re-proposed (e.g., we had student interest, but we ended
up not selecting it due to running out of slots or something). But those
should be the exception.

-Peff

Re: [RFH] hackday and GSoC topic suggestions

[Thomas Rast]

Jeff King <peff@peff.net> writes:

> Below is a list of features / bugs that I am taking to the hackday.

I have the list below at

  https://github.com/trast/git/wiki/Todo-items

(started at git-merge last year).  I did a quick triage, but don't take
my word for it.  Perhaps it's not too late for your hackathon yet ;-)

Some items may have been fixed by other people when I wasn't looking, so
first check if the problem still exists.  The ones I knew are done I
moved to the bottom.


Easy:

* Add -p 'e' when it fails to apply should offer an obvious way of
  starting from the original hunk (not the broken one) or both

* filter-branch should apply {tree,index}-filter to HEAD and complain if
  it's not invariant (cf. BFG)

* git-config should follow symlinks when rewriting the file, so as to
  avoid replacing a symlink at ~/.gitconfig with the actual contents

* git-send-email should complain when given an option --cc= (i.e., --cc
  with empty argument) instead of silently not using any address
  there. It's easy to accidentally cause this in combination with
  $()-expanding a script that searches for the address by name when
  there are no matches. Note that the cccmd feature should not error if
  it doesn't come up with anything!


Medium difficulty:  
  
* LC_CTYPE=C to work around an old issue in glibc that has since been
  fixed (ask jrn). We should autodetect if we still need the fix, and
  otherwise avoid it; it causes trouble with localized messages

* rebase -i --exec should be able to work elsewhere so you can continue
  doing useful work

* git-send-email could build smarter reference lists if it had
  strategies to fetch the in-reply-to email. One obvious, and very
  useful for git and linux, strategy to do so would be to append the
  References header of the mail obtained from
  http://mid.gmane.org/$in_reply_to (follow the 403 and then append /raw
  to get the headers).

* git-svn should have a diff subcommand, to match svn's output for
  feeding into tools that need the precise format with 'Index:' lines

* Resurrect the move of config var descriptions onto the separate pages
  for commands

  
Hard or unknown difficulty:
 
* Add -p 's' followed by 'e' fails to apply even a no-op edit if the
  context is small

* git-bisect appears to test a linear number of merge bases, thus
  ruining its usually log(n) complexity, in cases involving 'git bisect
  good origin/next', 'git bisect bad origin/pu', in git.git

* I have a script git-pie which runs 'perl -i -pe' as follows:

        #!/bin/sh
        
        cmd=$1
        shift
        
        git ls-files -z "$@" |
        xargs -0 perl -i -pe "$cmd"

  The prompt display (__git_ps1) in the prompt immediately after using
  'git pie ...' always shows that there are no uncommitted changes, even
  if that is not correct. Hitting RET again yields a correct status.


Already underway or done (only here so people don't say "but you dropped
this from the wiki list"):

* In-core merge: a clean merge should be able to work fully in memory,
  esp. without any worktree

* For the issue of "what did the evil merge do", should try to flatten
  out the conflicted merge (i.e. mechanically re-merge, and scrap the
  conflicted state in favor of conflict hunks in stage 0) and diff that
  against the merge result. if that works out, store the tree as a note
  somewhere for caching; git-notes should be able to handle that

  * For the last point: need a way of storing trees as
    notes. Technically the notes infrastructure does not require things
    to be blobs, but in practice all the frontends we have error out, or
    do worse things, when handed a tree as a note.

* ./t0000-foo.sh --valgrind should be able to: a) run only a single
  specific test under valgrind, using the normal git for the rest; and
  b) parallelize over (a) so as to speed up a complete ./t0000-foo.sh
  --valgrind run

-- 
Thomas Rast
tr@thomasrast.ch

Re: [RFH] hackday and GSoC topic suggestions

[Duy Nguyen]

On Sun, Feb 9, 2014 at 2:03 AM, Thomas Rast <tr@thomasrast.ch> wrote:
> Easy:
>
> * Add -p 'e' when it fails to apply should offer an obvious way of
>   starting from the original hunk (not the broken one) or both

If it's too easy, you can add a command to change diff display
settings (--color-words, context size...)
-- 
Duy

Re: [RFH] hackday and GSoC topic suggestions

[Thomas Rast]

Duy Nguyen <pclouds@gmail.com> writes:

> On Sun, Feb 9, 2014 at 2:03 AM, Thomas Rast <tr@thomasrast.ch> wrote:
>> Easy:
>>
>> * Add -p 'e' when it fails to apply should offer an obvious way of
>>   starting from the original hunk (not the broken one) or both
>
> If it's too easy, you can add a command to change diff display
> settings (--color-words, context size...)

If you mean for GSoC, that violates the "previously suggested" rule -- I
had it as a project proposal for at least two years and nobody was ever
interested.

-- 
Thomas Rast
tr@thomasrast.ch

Re: [RFH] hackday and GSoC topic suggestions

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>  - Branch rename breaks local downstream branches
>    http://article.gmane.org/gmane.comp.version-control.git/241228

If you have a branch B that builds on A, if you are renaming A to C,
you may want B to automatically set to build on C in some cases, and
in other cases your renaming A is done explicitly in order to severe
the tie between A and B and setting the latter to build on C can be
a bad thing---after all, the user's intention may be to create a
branch A starting at some commit immediately after this rename so
that B will keep building on that updated A.

So I am not sure if this is a bug.

>  - git stash doesn't use --index as default
>    http://article.gmane.org/gmane.comp.version-control.git/235892

I tend to think "git stash" was designed to work this way from day
one.

>  - using git commit-tree with "-F -" adds trailing newlines
>    http://article.gmane.org/gmane.comp.version-control.git/236583

According to the initial commit that introduced this option, it
deliberately calls strbuf_complete_line() to make sure we do not
leave an incomplete line at the end.


Many of the other items in your "bugs" section seem to be worth
fixing.

Thanks.

Re: [RFH] hackday and GSoC topic suggestions

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Jeff King <peff@peff.net> writes:
>
>>  - Branch rename breaks local downstream branches
>>    http://article.gmane.org/gmane.comp.version-control.git/241228
>
> If you have a branch B that builds on A, if you are renaming A to C,
> you may want B to automatically set to build on C in some cases, and
> in other cases your renaming A is done explicitly in order to severe
> the tie between A and B and setting the latter to build on C can be
> a bad thing---after all, the user's intention may be to create a
> branch A starting at some commit immediately after this rename so
> that B will keep building on that updated A.
>
> So I am not sure if this is a bug.

Having said that, the current behaviour of leaving B half-configured
to build on a missing branch is undesirable. If we were to change
this so that any branch B that used to build on branch A being
renamed to build on the branch under the new name C, the user may
have to do an extra "--set-upstream-to A" on B after recreating A if
this was done to save away the current state of A to C and then keep
building B on an updated A, so we may have to give _some_ clue what
we are doing behind their back when we rename, e.g.

	$ git branch -m A C
        warning: branch B is set to build on C now.

or something.

Re: [RFH] hackday and GSoC topic suggestions

[Jeff King]

On Tue, Feb 11, 2014 at 11:38:09AM -0800, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Jeff King <peff@peff.net> writes:
> >
> >>  - Branch rename breaks local downstream branches
> >>    http://article.gmane.org/gmane.comp.version-control.git/241228
> >
> > If you have a branch B that builds on A, if you are renaming A to C,
> > you may want B to automatically set to build on C in some cases, and
> > in other cases your renaming A is done explicitly in order to severe
> > the tie between A and B and setting the latter to build on C can be
> > a bad thing---after all, the user's intention may be to create a
> > branch A starting at some commit immediately after this rename so
> > that B will keep building on that updated A.
> >
> > So I am not sure if this is a bug.
> 
> Having said that, the current behaviour of leaving B half-configured
> to build on a missing branch is undesirable. If we were to change
> this so that any branch B that used to build on branch A being
> renamed to build on the branch under the new name C, the user may
> have to do an extra "--set-upstream-to A" on B after recreating A if
> this was done to save away the current state of A to C and then keep
> building B on an updated A, so we may have to give _some_ clue what
> we are doing behind their back when we rename, e.g.
> 
> 	$ git branch -m A C
>         warning: branch B is set to build on C now.
> 
> or something.

Yeah, I agree. I think most of the time people would want the dependency
to move with the branch, and the key is being clear about it so the user
can override. I don't think there is a problem with overriding with
`--set-upstream-to` after the fact, rather than giving a special option
to the move operation.

There was a team working on this at the hackday, and they seemed to make
reasonable progress, but I haven't seen the final output yet. Most teams
did not finish their projects, or if they did, I didn't get a chance to
coach them through the patch submission process. I'll hopefully be
revisiting those in the next week or two as they finish them up offline.

My goal is to get them all posting to the list themselves, but I fear I
may have to pick up the pieces in some cases.

-Peff

Re: [RFH] hackday and GSoC topic suggestions

[Jeff King]

On Tue, Feb 11, 2014 at 10:57:22AM -0800, Junio C Hamano wrote:

> >  - git stash doesn't use --index as default
> >    http://article.gmane.org/gmane.comp.version-control.git/235892
> 
> I tend to think "git stash" was designed to work this way from day
> one.

Right. The thing that bothers me is the data loss of:

  echo 1 >foo
  git add foo
  echo 2 >foo
  git stash
  git stash pop

The content in "1" is stashed, but gets lost on the apply, and our pop
drops the stash.

> >  - using git commit-tree with "-F -" adds trailing newlines
> >    http://article.gmane.org/gmane.comp.version-control.git/236583
> 
> According to the initial commit that introduced this option, it
> deliberately calls strbuf_complete_line() to make sure we do not
> leave an incomplete line at the end.

Yeah, I think this one is a non-bug. I hadn't read it carefully but the
"trailing newlines" made me think it was adding multiple extra blank
lines. But it looks like it is just the normal strbuf_complete_line, and
that is the right thing to be doing (it has _never_ been encouraged to
have a commit message that does not have a newline at all, and
I think stripspace has enforced that from day one).

> Many of the other items in your "bugs" section seem to be worth
> fixing.

As I mentioned elsewhere, I've yet to see the full results of the
hackday. Hopefully soon. :)

-Peff