Idea: "git format-patch" should get more information out of git

Idea: "git format-patch" should get more information out of git

[Michael Haggerty]

It is a nuisance that the emails for a patch series have to be
hand-edited after they are generated by "git format-patch".  Some of the
problems with this workflow are

1. The extra information typed into the email series is not archived in
the git repository, and therefore the emails have to be kept around.

2. If it is discovered between "git format-patch" and "git send-email"
that one of the patches needs a tweak, then the email series has to be
generated anew and all of the hand-edits need to be redone.

3. If a "v2" of the patch series has to be created, all of the
hand-edits have to be done again.

4. There is no place to store the "additional information" (the part
that comes in patch emails between the "---" and the diffstat) while
working on the patch series; it all has to be remembered or noted
somewhere else until just before sending off the patch emails.

Wouldn't it be nice if more of the information needed by "git
format-patch" could be extracted from the git repository?  For example:

* The subject and body of the cover letter could be written to the log
message of an annotated tag at the tip of the patch series.  "git
format-patch" could retrieve it if the "--cover-letter" option is used.

* The CC and ACK lists could be written as specially-formatted lines in
the annotated tag's log message, or perhaps in a git note.

* The "additional information" could be written to git notes.

I don't know whether git notes are the right vessel for such
information.  For example, are they carried along when a commit is rebased?

I don't have time to implement any of this, but perhaps somebody thinks
this is a good idea and is inspired to work on it.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: Idea: "git format-patch" should get more information out of git

[Junio C Hamano]

Michael Haggerty <mhagger@alum.mit.edu> writes:

> 1. The extra information typed into the email series is not archived in
> the git repository, and therefore the emails have to be kept around.

The convention is to have information that is not meant to be kept around
after '---', so...

> 2. If it is discovered between "git format-patch" and "git send-email"
> that one of the patches needs a tweak, then the email series has to be
> generated anew and all of the hand-edits need to be redone.

I actually often work this backwards. Tweak the patch files, apply them to
commit that they are meant to be applied, retest and then finally send.
There is no "generating anew" involved.

> 4. There is no place to store the "additional information" (the part
> that comes in patch emails between the "---" and the diffstat) while
> working on the patch series;...

I thought there was a RFC floating around to do this using notes and also
teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
one of the J's were involved, so I am CC'ing them.

Re: Idea: "git format-patch" should get more information out of git

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 27.08.2011 20:46:
> Michael Haggerty <mhagger@alum.mit.edu> writes:
> 
>> 1. The extra information typed into the email series is not archived in
>> the git repository, and therefore the emails have to be kept around.
> 
> The convention is to have information that is not meant to be kept around
> after '---', so...
> 
>> 2. If it is discovered between "git format-patch" and "git send-email"
>> that one of the patches needs a tweak, then the email series has to be
>> generated anew and all of the hand-edits need to be redone.
> 
> I actually often work this backwards. Tweak the patch files, apply them to
> commit that they are meant to be applied, retest and then finally send.
> There is no "generating anew" involved.
> 
>> 4. There is no place to store the "additional information" (the part
>> that comes in patch emails between the "---" and the diffstat) while
>> working on the patch series;...
> 
> I thought there was a RFC floating around to do this using notes and also
> teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
> one of the J's were involved, so I am CC'ing them.

Don't forget the MJ ;)

My current setup is:

A. I put the additional per commit info in notes. "git format-patch
--notes" takes it from there and adds it where it belongs.

B. I put series title and cover letter text into a ref note. "git branch
--notes" displays them as a branch note, and "git format-patch --cover
--notes" fills in the cover letter for me.

A patch for A has been on the list, but Junio didn't like the additional
"---" which I had for technical reasons, and I haven't had time to
revisit that patch yet. (The code would have to store a flag about notes
having been inserted or not.) Nobody has complained about the patches
I've created with that form, though ;)

The stuff for B is part of my "ref notes" stuff which I haven't
submitted so far. I've been using it for quite a while now, but am still
unsure about some design aspects.

A is simply my branch "format-patch-notes".

B uses these:
vob/virtual-objects
vob/refrev-hash
vob/branch-notes
vob/format-patch-branch-note
(Merge in this order. Last one is the most immature.)

Note that I rebase frequently these days because of overlapping work in
these areas.

I think Jeff had an interesting patch allowing to edit the note along
side (OK: at the bottom of) the commit message.

Cheers,
Michael

Oh, repo, yes... Any of these:

git://github.com/gitigit/git.git
git://gitorious.org/~mjg/git/mjg.git
git://repo.or.cz/git/mjg.git

Re: Idea: "git format-patch" should get more information out of git

[Michael Haggerty]

On 08/27/2011 10:35 PM, Michael J Gruber wrote:
> My current setup is:
> 
> A. I put the additional per commit info in notes. "git format-patch
> --notes" takes it from there and adds it where it belongs.
> 
> B. I put series title and cover letter text into a ref note. "git branch
> --notes" displays them as a branch note, and "git format-patch --cover
> --notes" fills in the cover letter for me.

Very cool.  I will check this out.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: Idea: "git format-patch" should get more information out of git

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Michael Haggerty <mhagger@alum.mit.edu> writes:
>
>> 4. There is no place to store the "additional information" (the part
>> that comes in patch emails between the "---" and the diffstat) while
>> working on the patch series;...
>
> I thought there was a RFC floating around to do this using notes and also
> teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
> one of the J's were involved, so I am CC'ing them.

Also, when I prepare a commit to be sent with an additional piece of
information, I often write "---" and the additional message after my
S-o-b: line while preparing the commit log message. Unlike format-patch
that strips that off, commit keeps it, which is handy.

Re: Idea: "git format-patch" should get more information out of git

[Jeff King]

On Sat, Aug 27, 2011 at 11:34:13PM -0700, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Michael Haggerty <mhagger@alum.mit.edu> writes:
> >
> >> 4. There is no place to store the "additional information" (the part
> >> that comes in patch emails between the "---" and the diffstat) while
> >> working on the patch series;...
> >
> > I thought there was a RFC floating around to do this using notes and also
> > teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
> > one of the J's were involved, so I am CC'ing them.
> 
> Also, when I prepare a commit to be sent with an additional piece of
> information, I often write "---" and the additional message after my
> S-o-b: line while preparing the commit log message. Unlike format-patch
> that strips that off, commit keeps it, which is handy.

After playing around a bit with my earlier series, I made the
realization (perhaps obvious to others :) ), that if you are in a
pure-patch workflow, keeping the "---" in your commit message locally is
much simpler. It follows the commit around through rebases
automatically, it gets put into format-patch output automatically, and
so forth.

The only real downside is that you can never tell git "don't show me the
cover letter cruft". Which is probably OK for your own local patches.
But the point of the "---" is that information should never make it into
a repo, which means in any workflow that involves pulling actual git
commits, it won't work (after reading Michael's response in another
thread, though, I think he would be interested in a hybrid
pull-or-apply-via-mail system).

-Peff

Re: Idea: "git format-patch" should get more information out of git

[Michael J Gruber]

Jeff King venit, vidit, dixit 29.08.2011 20:55:
> On Sat, Aug 27, 2011 at 11:34:13PM -0700, Junio C Hamano wrote:
> 
>> Junio C Hamano <gitster@pobox.com> writes:
>>
>>> Michael Haggerty <mhagger@alum.mit.edu> writes:
>>>
>>>> 4. There is no place to store the "additional information" (the part
>>>> that comes in patch emails between the "---" and the diffstat) while
>>>> working on the patch series;...
>>>
>>> I thought there was a RFC floating around to do this using notes and also
>>> teach it to "commit -e" a few months ago? I vaguelly recall that Peff and
>>> one of the J's were involved, so I am CC'ing them.
>>
>> Also, when I prepare a commit to be sent with an additional piece of
>> information, I often write "---" and the additional message after my
>> S-o-b: line while preparing the commit log message. Unlike format-patch
>> that strips that off, commit keeps it, which is handy.
> 
> After playing around a bit with my earlier series, I made the
> realization (perhaps obvious to others :) ), that if you are in a
> pure-patch workflow, keeping the "---" in your commit message locally is
> much simpler. It follows the commit around through rebases
> automatically, it gets put into format-patch output automatically, and
> so forth.

I'm confused: Does format-patch keep it or strip it as you say above?

Anyways, notes survive rebase etc., and at format-patch time you can
decide whether you want to include them or not (with my patch).

> The only real downside is that you can never tell git "don't show me the
> cover letter cruft". Which is probably OK for your own local patches.

You can with notes...

> But the point of the "---" is that information should never make it into
> a repo, which means in any workflow that involves pulling actual git
> commits, it won't work (after reading Michael's response in another
> thread, though, I think he would be interested in a hybrid
> pull-or-apply-via-mail system).

The only difficulty would be sharing notes in a push-pull workflow (for
lack of tool support, though we do have merge support).

Cheers,
Michael

Re: Idea: "git format-patch" should get more information out of git

[Jeff King]

On Tue, Aug 30, 2011 at 02:21:13PM +0200, Michael J Gruber wrote:

> > After playing around a bit with my earlier series, I made the
> > realization (perhaps obvious to others :) ), that if you are in a
> > pure-patch workflow, keeping the "---" in your commit message locally is
> > much simpler. It follows the commit around through rebases
> > automatically, it gets put into format-patch output automatically, and
> > so forth.
> 
> I'm confused: Does format-patch keep it or strip it as you say above?

Format-patch will include the whole commit message, including "---" and
what comes after. But an "am" recipient will strip it out of the commit
message. IOW, you locally treat your "---" notes as part of the commit
message, but in a pure-patch workflow, upstream treats it as something
to be stripped out.

So there is asymmetry in what is in your commit versus what is in
upstream's commit. But that's OK, because how you treat the commit
(something to be tweaked and rebased, and then mailed out) and how
upstream treats it (something to go into the long-term project history)
are different.

But that breaks down if you ever want to share actual commit objects
with upstream.

> Anyways, notes survive rebase etc.

You still have to manually configure:

  git config notes.rewriteRef refs/notes/commits

Perhaps that is something that should be changed.

> and at format-patch time you can decide whether you want to include
> them or not (with my patch).

Yeah. Don't get me wrong, I think putting "---" notes into git-notes is
way more flexible. It's just that it's also more complex, and
unnecessarily so for many use cases.

We have some patches to hide that complexity, but I was at one point
wondering if all of the complexity was worthwhile. But as I mentioned
above, if you want to build anything more complex than a pure-patch
workflow, the simple solution breaks down. So it probably is worth
pursuing.

-Peff

Re: Idea: "git format-patch" should get more information out of git

[Michael J Gruber]

Jeff King venit, vidit, dixit 30.08.2011 17:22:
> On Tue, Aug 30, 2011 at 02:21:13PM +0200, Michael J Gruber wrote:
...
>> Anyways, notes survive rebase etc.
> 
> You still have to manually configure:
> 
>   git config notes.rewriteRef refs/notes/commits
> 
> Perhaps that is something that should be changed.

Right, I've done that quite a while ago and forgot.

>> and at format-patch time you can decide whether you want to include
>> them or not (with my patch).
> 
> Yeah. Don't get me wrong, I think putting "---" notes into git-notes is
> way more flexible. It's just that it's also more complex, and
> unnecessarily so for many use cases.
> 
> We have some patches to hide that complexity, but I was at one point
> wondering if all of the complexity was worthwhile. But as I mentioned
> above, if you want to build anything more complex than a pure-patch
> workflow, the simple solution breaks down. So it probably is worth
> pursuing.

Reminds me of the ref namespace restructuring which could help sharing
notes... Oh, lots to do before git 3.0!

Michael

Re: Idea: "git format-patch" should get more information out of git

[Johan Herland]

On Tuesday 30. August 2011, Michael J Gruber wrote:
> Reminds me of the ref namespace restructuring which could help
> sharing notes... Oh, lots to do before git 3.0!

Indeed. I am very sorry to not have been able to properly follow up on 
that proposal (or a couple of other patches for that matter), but I'm 
currently in the middle of changing jobs and moving to a new apartment, 
which unfortunately leaves little if any spare time for git work. I 
don't believe I'll find the time to pick this up in the next month or 
two, so if someone wants to run with it, you are very welcome to do so.

I will try to participate in discussions where I'm CCed, but the rest of 
the mailing list is mostly a blur, these days.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: Idea: "git format-patch" should get more information out of git

[Michael Haggerty]

On 08/30/2011 07:39 PM, Johan Herland wrote:
> On Tuesday 30. August 2011, Michael J Gruber wrote:
>> Reminds me of the ref namespace restructuring which could help
>> sharing notes... Oh, lots to do before git 3.0!
> 
> Indeed. I am very sorry to not have been able to properly follow up on 
> that proposal [...]

What "ref rename restructuring" are you talking about?  Can you give a
mailing list link?

(I've been working on refs lately and want to know whether your proposal
is relevant to my work.)

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: Idea: "git format-patch" should get more information out of git

[Michael J Gruber]

Michael Haggerty venit, vidit, dixit 01.09.2011 06:32:
> On 08/30/2011 07:39 PM, Johan Herland wrote:
>> On Tuesday 30. August 2011, Michael J Gruber wrote:
>>> Reminds me of the ref namespace restructuring which could help
>>> sharing notes... Oh, lots to do before git 3.0!
>>
>> Indeed. I am very sorry to not have been able to properly follow up on 
>> that proposal [...]
> 
> What "ref rename restructuring" are you talking about?  Can you give a
> mailing list link?
> 
> (I've been working on refs lately and want to know whether your proposal
> is relevant to my work.)

Searching for "namespace" in the subject should give you all relevant
proposals plus the new "namespaces" which might affect you also:

http://permalink.gmane.org/gmane.comp.version-control.git/140681
http://permalink.gmane.org/gmane.comp.version-control.git/165813
http://permalink.gmane.org/gmane.comp.version-control.git/165799

http://permalink.gmane.org/gmane.comp.version-control.git/176808

Michael