Problem with "From:" line on "git format-patch" generated patches

Problem with "From:" line on "git format-patch" generated patches

[André Goddard Rosa]

Hi, everybody!

    When I generate patches using "git format-patch" it always makes
my name garbled in the "From:" line
as shown below. I'm using openSUSE 11.2 RC 1 (x86_64) with the
following settings:

        # locale charmap
        UTF-8
        # echo $LANG
        en_US.UTF-8

    I've tried changing my environment to use another encoding like
ISO-8859-1, but it didn't work as well.

    Does someone can explain why does this happens? Any suggestions?

P.s.: the problem never occurs on the commit message (Signed-off-by)


>> >From 584d9bfc7c1d41b76a05655b4562b98fcbef6ee4 Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Andr=C3=A9=20Goddard=20Rosa?= <andre.goddard@gmail.com>
>> Date: Sun, 1 Nov 2009 14:09:06 -0200
>> Subject: [PATCH v2 7/7] vsprintf: factor out skip_space code in a
>> separate function
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=UTF-8
>> Content-Transfer-Encoding: 8bit
>
> btw., for reviewability's sake, could you please fix your patch
> submission method to not include such garbled headers in the mail body?
>
> (Also, it would be nice to reply-thread the 7 patches on the 0/7 mail,
> as git-send-email does.)


Thank you,
André

Re: Problem with "From:" line on "git format-patch" generated patches

[Santi Béjar]

On Tue, Nov 3, 2009 at 5:30 PM, André Goddard Rosa
<andre.goddard@gmail.com> wrote:
> Hi, everybody!
>
>    When I generate patches using "git format-patch" it always makes
> my name garbled in the "From:" line
> as shown below. I'm using openSUSE 11.2 RC 1 (x86_64) with the
> following settings:
>
>        # locale charmap
>        UTF-8
>        # echo $LANG
>        en_US.UTF-8
>
>    I've tried changing my environment to use another encoding like
> ISO-8859-1, but it didn't work as well.
>
>    Does someone can explain why does this happens? Any suggestions?
>
> P.s.: the problem never occurs on the commit message (Signed-off-by)
>
>
>>> >From 584d9bfc7c1d41b76a05655b4562b98fcbef6ee4 Mon Sep 17 00:00:00 2001
>>> From: =?UTF-8?q?Andr=C3=A9=20Goddard=20Rosa?= <andre.goddard@gmail.com>
>>> Date: Sun, 1 Nov 2009 14:09:06 -0200
>>> Subject: [PATCH v2 7/7] vsprintf: factor out skip_space code in a
>>> separate function
>>> MIME-Version: 1.0
>>> Content-Type: text/plain; charset=UTF-8
>>> Content-Transfer-Encoding: 8bit

This is the normal encoding for email headers where you cannot use
8bit characters. You have to use a 7bit characters with this
=?UTF-8... encoding.

You can check the From: line in your mail, the mail I'm replying:

From:	=?ISO-8859-1?Q?Andr=E9_Goddard_Rosa?= <andre.goddard@gmail.com>

At the other hand the tools using the output of git-format-patch must
deal with this all, as they do. git-am handles it well, if not it's a
bug that should be reported.

HTH,
Santi

Re: Problem with "From:" line on "git format-patch" generated patches

[André Goddard Rosa]

On 11/3/09, Santi Béjar <santi@agolina.net> wrote:
> On Tue, Nov 3, 2009 at 5:30 PM, André Goddard Rosa
> <andre.goddard@gmail.com> wrote:
>> Hi, everybody!
>>
>>    When I generate patches using "git format-patch" it always makes
>> my name garbled in the "From:" line
>> as shown below. I'm using openSUSE 11.2 RC 1 (x86_64) with the
>> following settings:
>>
>>        # locale charmap
>>        UTF-8
>>        # echo $LANG
>>        en_US.UTF-8
>>
>>    I've tried changing my environment to use another encoding like
>> ISO-8859-1, but it didn't work as well.
>>
>>    Does someone can explain why does this happens? Any suggestions?
>>
>> P.s.: the problem never occurs on the commit message (Signed-off-by)
>>
>>>> >From 584d9bfc7c1d41b76a05655b4562b98fcbef6ee4 Mon Sep 17 00:00:00 2001
>>>> From: =?UTF-8?q?Andr=C3=A9=20Goddard=20Rosa?= <andre.goddard@gmail.com>
>>>> Date: Sun, 1 Nov 2009 14:09:06 -0200
>>>> Subject: [PATCH v2 7/7] vsprintf: factor out skip_space code in a
>>>> separate function
>>>> MIME-Version: 1.0
>>>> Content-Type: text/plain; charset=UTF-8
>>>> Content-Transfer-Encoding: 8bit
>
> This is the normal encoding for email headers where you cannot use
> 8bit characters. You have to use a 7bit characters with this
> =?UTF-8... encoding.
>
> You can check the From: line in your mail, the mail I'm replying:
>
> From:	=?ISO-8859-1?Q?Andr=E9_Goddard_Rosa?= <andre.goddard@gmail.com>
>
> At the other hand the tools using the output of git-format-patch must
> deal with this all, as they do. git-am handles it well, if not it's a
> bug that should be reported.
>

Great, Santi!

     I really appreciate your reply!!!

    I was just in the process of debugging this issue when I landed
into function pp_user_info(), which calls add_rfc2047(). So I started
looking into http://www.faqs.org/rfcs/rfc2047.html , which specifies:

"Generally, an "encoded-word" is a sequence of printable ASCII
characters that begins with "=?", ends with "?=", and has two "?"s in
between."

Ok... I got it; it's necessary for proper signaling of the email
header when it detects the presence of certain characters outside the
ASCII range 0..127 (7 bits). It's the case for latin "é" letter in my
name.

So, let me explain what happened to me:

I'm not using any specific tool for inputting the git-format-patch,
but instead I'm sending the files generated by it through gmail as an
inlined patch in the email body.

I like the convenience of format-patch for generating the patch files,
but in this case, formatting the header as rfc2047 is not necessary
and makes a funny/garbled output in my patch submission.

Do you have a suggestion for my workflow?

Thanks a lot,
André

Re: Problem with "From:" line on "git format-patch" generated patches

[André Goddard Rosa]

> I'm not using any specific tool for inputting the git-format-patch,
> but instead I'm sending the files generated by it through gmail as an
> inlined patch in the email body.
>
> I like the convenience of format-patch for generating the patch files,
> but in this case, formatting the header as rfc2047 is not necessary
> and makes a funny/garbled output in my patch submission.
>
> Do you have a suggestion for my workflow?
>

I really would like continuing having the convenience of using a web
access to my gmail for sending the patches, so I just need a way to
format the patches which makes it easy submitting them later. I'd like
to avoid using any other email client for that, if possible.

Re: Problem with "From:" line on "git format-patch" generated patches

[Jonathan Nieder]

Hi André,

André Goddard Rosa wrote:

>> I'm not using any specific tool for inputting the git-format-patch,
>> but instead I'm sending the files generated by it through gmail as an
>> inlined patch in the email body.
>>
>> I like the convenience of format-patch for generating the patch files,
>> but in this case, formatting the header as rfc2047 is not necessary
>> and makes a funny/garbled output in my patch submission.

The header fields git format-patch outputs are just intended as a
starting point for the header of your mailing.  It is more convenient
to receive an e-mail with

	Delivered-to: maintainer@example.com
	Received: [...]
	Message-ID: <patch.sender.0001@example.com>
	Date: Tue, 03 Nov 2009 16:33:54 -0600
	From: Patch Sender <patch.sender@example.com>
	Subject: [PATCH] Fix one bug, add another
	Content-Type: text/plain; charset=us-ascii

	Blah blah blah

than one in which the content includes some useless metadata that was
already in the header.  So you should just strip the header out from
the body before sending.

There are three common exceptions: 1) you might want to send a patch
written by someone else, 2) you might want to mark a patch as written
before it was sent, and 3) some people like to receive patches as
attachments rather than inlined in messages.  For the first two cases,
the solution is to include the header fields to change in the body:

	From: Patch Writer <patch.writer@example.com>
	Date: Wed, 01 Apr 1970 01:23:45 +0100

	Blah blah blah
	---
	Hi,

	Patch Writer wrote this patch a while ago that might be
	relevant.  It needed a straightforward one-line change to
	apply and is otherwise unchanged.

	What do you think?
[...]

For the last case, I think it is most common to send unchanged 'git
format-patch' output.  But only the From, Date, and Subject fields
are actually needed.

I am not sure how well 'git am' copes with non-ascii characters in
the pseudo-header lines: I would have guessed it could handle them
both rfc2047-encoded and not, but I have not tried.

> I really would like continuing having the convenience of using a web
> access to my gmail for sending the patches, so I just need a way to
> format the patches which makes it easy submitting them later. I'd like
> to avoid using any other email client for that, if possible.

Here, there is another danger: the Gmail web interface does not
consider your whitespace precious, so it is very prone to mangling
patches (especially with long lines).

Documentation/SubmittingPatches [1] has some advice:

| Gmail
| -----
| 
| GMail does not appear to have any way to turn off line wrapping in the web
| interface, so this will mangle any emails that you send.  You can however
| use any IMAP email client to connect to the google imap server, and forward
| the emails through that.  Just make sure to disable line wrapping in that
| email client.  Alternatively, use "git send-email" instead.
| 
| Submitting properly formatted patches via Gmail is simple now that
| IMAP support is available. First, edit your ~/.gitconfig to specify your
| account settings:
| 
| [imap]
| 	folder = "[Gmail]/Drafts"
| 	host = imaps://imap.gmail.com
| 	user = user@gmail.com
| 	pass = p4ssw0rd
| 	port = 993
| 	sslverify = false
| 
| You might need to instead use: folder = "[Google Mail]/Drafts" if you get an error
| that the "Folder doesn't exist".
| 
| Next, ensure that your Gmail settings are correct. In "Settings" the
| "Use Unicode (UTF-8) encoding for outgoing messages" should be checked.
| 
| Once your commits are ready to send to the mailing list, run the following
| command to send the patch emails to your Gmail Drafts folder.
| 
| 	$ git format-patch -M --stdout origin/master | git imap-send
| 
| Go to your Gmail account, open the Drafts folder, find the patch email, fill
| in the To: and CC: fields and send away!

Good luck.

Hope that helps,
Jonathan

[1] <http://git.kernel.org/?p=git/git.git;a=blob_plain;f=Documentation/SubmittingPatches>
converting tabs to spaces.  

Re: Problem with "From:" line on "git format-patch" generated patches

[Jeff King]

On Tue, Nov 03, 2009 at 04:06:39PM -0200, André Goddard Rosa wrote:

> I'm not using any specific tool for inputting the git-format-patch,
> but instead I'm sending the files generated by it through gmail as an
> inlined patch in the email body.
> 
> I like the convenience of format-patch for generating the patch files,
> but in this case, formatting the header as rfc2047 is not necessary
> and makes a funny/garbled output in my patch submission.
> 
> Do you have a suggestion for my workflow?

I don't think there's currently a way to turn off the rfc2047 from
within format-patch. You can generate a single patch with the same
format using:

  git log -1 -p --stat --summary \
    --pretty=tformat:'From: %an <%ae>%nDate: %aD%nSubject: [PATCH] %s%n%n%b'

but it won't do nice things like putting one patch in each file.

Probably it would make sense for format-patch to have an option to
indicate that you are going to inline these patches into a different
MUA. So drop the 'From' mbox header line, don't rfc2047 encode, and
maybe some other behaviors. I do the same thing (including inline in
mutt), but I just delete the unwanted lines manually, and fortunately my
name doesn't contain any non-ascii characters. ;)

-Peff

Re: Problem with "From:" line on "git format-patch" generated patches

[André Goddard Rosa]

On 11/3/09, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi André,
>
> André Goddard Rosa wrote:
>
>>> I'm not using any specific tool for inputting the git-format-patch,
>>> but instead I'm sending the files generated by it through gmail as an
>>> inlined patch in the email body.
>>>
>>> I like the convenience of format-patch for generating the patch files,
>>> but in this case, formatting the header as rfc2047 is not necessary
>>> and makes a funny/garbled output in my patch submission.
>
> The header fields git format-patch outputs are just intended as a
> starting point for the header of your mailing.  It is more convenient
> to receive an e-mail with
>
> 	Delivered-to: maintainer@example.com
> 	Received: [...]
> 	Message-ID: <patch.sender.0001@example.com>
> 	Date: Tue, 03 Nov 2009 16:33:54 -0600
> 	From: Patch Sender <patch.sender@example.com>
> 	Subject: [PATCH] Fix one bug, add another
> 	Content-Type: text/plain; charset=us-ascii
>
> 	Blah blah blah
>
> than one in which the content includes some useless metadata that was
> already in the header.  So you should just strip the header out from
> the body before sending.
>
> There are three common exceptions: 1) you might want to send a patch
> written by someone else, 2) you might want to mark a patch as written
> before it was sent, and 3) some people like to receive patches as
> attachments rather than inlined in messages.  For the first two cases,
> the solution is to include the header fields to change in the body:
>
> 	From: Patch Writer <patch.writer@example.com>
> 	Date: Wed, 01 Apr 1970 01:23:45 +0100
>
> 	Blah blah blah
> 	---
> 	Hi,
>
> 	Patch Writer wrote this patch a while ago that might be
> 	relevant.  It needed a straightforward one-line change to
> 	apply and is otherwise unchanged.
>
> 	What do you think?
> [...]
>
> For the last case, I think it is most common to send unchanged 'git
> format-patch' output.  But only the From, Date, and Subject fields
> are actually needed.
>
> I am not sure how well 'git am' copes with non-ascii characters in
> the pseudo-header lines: I would have guessed it could handle them
> both rfc2047-encoded and not, but I have not tried.
>
>> I really would like continuing having the convenience of using a web
>> access to my gmail for sending the patches, so I just need a way to
>> format the patches which makes it easy submitting them later. I'd like
>> to avoid using any other email client for that, if possible.
>
> Here, there is another danger: the Gmail web interface does not
> consider your whitespace precious, so it is very prone to mangling
> patches (especially with long lines).
>
> Documentation/SubmittingPatches [1] has some advice:
>
> | Gmail
> | -----
> |
> | GMail does not appear to have any way to turn off line wrapping in the web
> | interface, so this will mangle any emails that you send.  You can however
> | use any IMAP email client to connect to the google imap server, and
> forward
> | the emails through that.  Just make sure to disable line wrapping in that
> | email client.  Alternatively, use "git send-email" instead.
> |
> | Submitting properly formatted patches via Gmail is simple now that
> | IMAP support is available. First, edit your ~/.gitconfig to specify your
> | account settings:
> |
> | [imap]
> | 	folder = "[Gmail]/Drafts"
> | 	host = imaps://imap.gmail.com
> | 	user = user@gmail.com
> | 	pass = p4ssw0rd
> | 	port = 993
> | 	sslverify = false
> |
> | You might need to instead use: folder = "[Google Mail]/Drafts" if you get
> an error
> | that the "Folder doesn't exist".
> |
> | Next, ensure that your Gmail settings are correct. In "Settings" the
> | "Use Unicode (UTF-8) encoding for outgoing messages" should be checked.
> |
> | Once your commits are ready to send to the mailing list, run the following
> | command to send the patch emails to your Gmail Drafts folder.
> |
> | 	$ git format-patch -M --stdout origin/master | git imap-send
> |
> | Go to your Gmail account, open the Drafts folder, find the patch email,
> fill
> | in the To: and CC: fields and send away!
>
> Good luck.
>
> Hope that helps,

Hello, Jonatan!

     Thanks for your insights, surely I understand what is reasoning behind it.
     I expect this thread will be useful to others in the future. Here
goes another relevant reference about sending patches using web GUI of
gmail :
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/email-clients.txt;h=a618efab7b156658be70b29c1a6a9b9c4093e0f5;hb=HEAD

Thanks a lot,
André

Re: Problem with "From:" line on "git format-patch" generated patches

[André Goddard Rosa]

On 11/4/09, Jeff King <peff@peff.net> wrote:
> On Tue, Nov 03, 2009 at 04:06:39PM -0200, André Goddard Rosa wrote:
>
>> I'm not using any specific tool for inputting the git-format-patch,
>> but instead I'm sending the files generated by it through gmail as an
>> inlined patch in the email body.
>>
>> I like the convenience of format-patch for generating the patch files,
>> but in this case, formatting the header as rfc2047 is not necessary
>> and makes a funny/garbled output in my patch submission.
>>
>> Do you have a suggestion for my workflow?
>
> I don't think there's currently a way to turn off the rfc2047 from
> within format-patch. You can generate a single patch with the same
> format using:
>
>   git log -1 -p --stat --summary \
>     --pretty=tformat:'From: %an <%ae>%nDate: %aD%nSubject: [PATCH] %s%n%n%b'
>
> but it won't do nice things like putting one patch in each file.
>
> Probably it would make sense for format-patch to have an option to
> indicate that you are going to inline these patches into a different
> MUA. So drop the 'From' mbox header line, don't rfc2047 encode, and
> maybe some other behaviors. I do the same thing (including inline in
> mutt), but I just delete the unwanted lines manually, and fortunately my
> name doesn't contain any non-ascii characters. ;)
>
> -Peff

Hello, Peff!

     It's good to know that I'm not alone on this. I think that should
be fairly easy, yes.
     Thanks for helping!

Thank you all,
André