[PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

[PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

[Lars Wendler]

From: Stefan Agner <stefan@agner.ch>

Sometimes sending huge patches/commits fail with

[Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email
line 1320.

Running the command with --smtp-debug=1 yields to

Net::SMTP::SSL: Net::Cmd::datasend(): unexpected EOF on command channel:
at /usr/lib/git-core/git-send-email line 1320.
[Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email
line 1320.

Stefan described it in his mail like this:

It seems to me that there is a size limit, after cutting down the patch
to ~16K, sending started to work. I cut it twice, once by removing lines
from the head and once from the bottom, in both cases at the size of
around 16K I could send the patch.

See also original report:
http://permalink.gmane.org/gmane.comp.version-control.git/274569

Reported-by: Juston Li <juston.h.li@gmail.com>
Tested-by: Markos Chandras <hwoarang@gentoo.org>
Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
---
 git-send-email.perl | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index e3ff44b..e907e0ea 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1365,7 +1365,11 @@ Message-Id: $message_id
 		$smtp->mail( $raw_from ) or die $smtp->message;
 		$smtp->to( @recipients ) or die $smtp->message;
 		$smtp->data or die $smtp->message;
-		$smtp->datasend("$header\n$message") or die $smtp->message;
+		$smtp->datasend("$header\n") or die $smtp->message;
+		my @lines = split /^/, $message;
+		foreach my $line (@lines) {
+			$smtp->datasend("$line") or die $smtp->message;
+		}
 		$smtp->dataend() or die $smtp->message;
 		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
 	}
-- 
2.6.0

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

[Johannes Schindelin]

Hi Lars,

On 2015-09-30 09:26, Lars Wendler wrote:
> From: Stefan Agner <stefan@agner.ch>
> 
> Sometimes sending huge patches/commits fail with
> 
> [Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email
> line 1320.
> 
> Running the command with --smtp-debug=1 yields to
> 
> Net::SMTP::SSL: Net::Cmd::datasend(): unexpected EOF on command channel:
> at /usr/lib/git-core/git-send-email line 1320.
> [Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email
> line 1320.
> 
> Stefan described it in his mail like this:
> 
> It seems to me that there is a size limit, after cutting down the patch
> to ~16K, sending started to work. I cut it twice, once by removing lines
> from the head and once from the bottom, in both cases at the size of
> around 16K I could send the patch.
> 
> See also original report:
> http://permalink.gmane.org/gmane.comp.version-control.git/274569
> 
> Reported-by: Juston Li <juston.h.li@gmail.com>
> Tested-by: Markos Chandras <hwoarang@gentoo.org>
> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
> ---

Very nice! Thank you,
Dscho

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

[Junio C Hamano]

Lars Wendler <polynomial-c@gentoo.org> writes:

> It seems to me that there is a size limit, after cutting down the patch
> to ~16K, sending started to work. I cut it twice, once by removing lines
> from the head and once from the bottom, in both cases at the size of
> around 16K I could send the patch.
>
> See also original report:
> http://permalink.gmane.org/gmane.comp.version-control.git/274569
>
> Reported-by: Juston Li <juston.h.li@gmail.com>
> Tested-by: Markos Chandras <hwoarang@gentoo.org>
> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
> ---
>  git-send-email.perl | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/git-send-email.perl b/git-send-email.perl
> index e3ff44b..e907e0ea 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -1365,7 +1365,11 @@ Message-Id: $message_id
>  		$smtp->mail( $raw_from ) or die $smtp->message;
>  		$smtp->to( @recipients ) or die $smtp->message;
>  		$smtp->data or die $smtp->message;
> -		$smtp->datasend("$header\n$message") or die $smtp->message;
> +		$smtp->datasend("$header\n") or die $smtp->message;
> +		my @lines = split /^/, $message;
> +		foreach my $line (@lines) {
> +			$smtp->datasend("$line") or die $smtp->message;
> +		}

Thanks.  One and a half comments.

 * If 16k is the limit, and smtp payload line limit is much much
   shorter than that, is it sensible to send data line by line?

 * Has this been reported to Net::Cmd::datasend() upstream?

>  		$smtp->dataend() or die $smtp->message;
>  		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
>  	}

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

[Stefan Agner]

On 2015-09-30 10:51, Junio C Hamano wrote:
> Lars Wendler <polynomial-c@gentoo.org> writes:
> 
>> It seems to me that there is a size limit, after cutting down the patch
>> to ~16K, sending started to work. I cut it twice, once by removing lines
>> from the head and once from the bottom, in both cases at the size of
>> around 16K I could send the patch.
>>
>> See also original report:
>> http://permalink.gmane.org/gmane.comp.version-control.git/274569
>>
>> Reported-by: Juston Li <juston.h.li@gmail.com>
>> Tested-by: Markos Chandras <hwoarang@gentoo.org>
>> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
>> ---
>>  git-send-email.perl | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/git-send-email.perl b/git-send-email.perl
>> index e3ff44b..e907e0ea 100755
>> --- a/git-send-email.perl
>> +++ b/git-send-email.perl
>> @@ -1365,7 +1365,11 @@ Message-Id: $message_id
>>  		$smtp->mail( $raw_from ) or die $smtp->message;
>>  		$smtp->to( @recipients ) or die $smtp->message;
>>  		$smtp->data or die $smtp->message;
>> -		$smtp->datasend("$header\n$message") or die $smtp->message;
>> +		$smtp->datasend("$header\n") or die $smtp->message;
>> +		my @lines = split /^/, $message;
>> +		foreach my $line (@lines) {
>> +			$smtp->datasend("$line") or die $smtp->message;
>> +		}
> 

Hi,

> Thanks.  One and a half comments.
> 
>  * If 16k is the limit, and smtp payload line limit is much much
>    shorter than that, is it sensible to send data line by line?

It was a easy and quick way which popped into my mind. The 16k limit has
been evaluated just by trying out some patch sizes, not sure whether
that is really right and where that really comes from. I guess before
using this as split size one would need to understand what causes that
limit actually...

>  * Has this been reported to Net::Cmd::datasend() upstream?

Not by me. I also have to admit that I am not a perl hacker at all...

--
Stefan

> 
>>  		$smtp->dataend() or die $smtp->message;
>>  		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
>>  	}

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

[Stefan Agner]

On 2015-09-30 10:51, Junio C Hamano wrote:
> Lars Wendler <polynomial-c@gentoo.org> writes:
> 
>> It seems to me that there is a size limit, after cutting down the patch
>> to ~16K, sending started to work. I cut it twice, once by removing lines
>> from the head and once from the bottom, in both cases at the size of
>> around 16K I could send the patch.
>>
>> See also original report:
>> http://permalink.gmane.org/gmane.comp.version-control.git/274569
>>
>> Reported-by: Juston Li <juston.h.li@gmail.com>
>> Tested-by: Markos Chandras <hwoarang@gentoo.org>
>> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
>> ---
>>  git-send-email.perl | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/git-send-email.perl b/git-send-email.perl
>> index e3ff44b..e907e0ea 100755
>> --- a/git-send-email.perl
>> +++ b/git-send-email.perl
>> @@ -1365,7 +1365,11 @@ Message-Id: $message_id
>>  		$smtp->mail( $raw_from ) or die $smtp->message;
>>  		$smtp->to( @recipients ) or die $smtp->message;
>>  		$smtp->data or die $smtp->message;
>> -		$smtp->datasend("$header\n$message") or die $smtp->message;
>> +		$smtp->datasend("$header\n") or die $smtp->message;
>> +		my @lines = split /^/, $message;
>> +		foreach my $line (@lines) {
>> +			$smtp->datasend("$line") or die $smtp->message;
>> +		}
> 
> Thanks.  One and a half comments.
> 
>  * If 16k is the limit, and smtp payload line limit is much much
>    shorter than that, is it sensible to send data line by line?
> 
>  * Has this been reported to Net::Cmd::datasend() upstream?

I still constantly run in to this issue. Fixing it locally, and next
time git gets updated and I send a larger patch, it happens again.

Just dig a bit more into that, it seems that this is a documented Perl
limitation:
http://search.cpan.org/~sullr/IO-Socket-SSL-2.020/lib/IO/Socket/SSL.pod#syswrite

All examples use datasend line wise, so I guess it is not a all to bad
choice...

--
Stefan


> 
>>  		$smtp->dataend() or die $smtp->message;
>>  		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
>>  	}