[PATCH] git-send-email: add ~/.authinfo parsing

[PATCH] git-send-email: add ~/.authinfo parsing

[Michal Nazarewicz]

From: Michal Nazarewicz <mina86@mina86.com>

Make git-send-email read password from a ~/.authinfo file instead of
requiring it to be stored in git configuration, passed as command line
argument or typed in.

There are various other applications that use this file for
authentication information so letting users use it for git-send-email
is convinient.  Furthermore, some users store their ~/.gitconfig file
in a public repository and having to store password there makes it
easy to publish the password.

Not to introduce any new dependencies, ~/.authinfo file is parsed only
if Text::CSV Perl module is installed.  If it's not, a notification is
printed and the file is ignored.

Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
---
 Documentation/git-send-email.txt | 15 +++++++--
 git-send-email.perl              | 69 +++++++++++++++++++++++++++++++++-------
 2 files changed, 70 insertions(+), 14 deletions(-)

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index eeb561c..b83576e 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -158,14 +158,25 @@ Sending
 --smtp-pass[=<password>]::
 	Password for SMTP-AUTH. The argument is optional: If no
 	argument is specified, then the empty string is used as
-	the password. Default is the value of 'sendemail.smtppass',
-	however '--smtp-pass' always overrides this value.
+	the password. Default is the value of 'sendemail.smtppass'
+	or value read from '~/.authinfo' file, however '--smtp-pass'
+	always overrides this value.
 +
 Furthermore, passwords need not be specified in configuration files
 or on the command line. If a username has been specified (with
 '--smtp-user' or a 'sendemail.smtpuser'), but no password has been
 specified (with '--smtp-pass' or 'sendemail.smtppass'), then the
 user is prompted for a password while the input is masked for privacy.
++
+The '~/.authinfo' file is read if Text::CSV Perl module is installed
+on the system; if it's missing, a notification message will be printed
+and the file ignored altogether.  The file should contain a line with
+the following format:
++
+  machine <domain> port <port> login <user> password <pass>
++
+Contrary to other tools, 'git-send-email' does not support symbolic
+port names like 'imap' thus `<port>` must be a number.
 
 --smtp-server=<host>::
 	If set, specifies the outgoing SMTP server to use (e.g.
diff --git a/git-send-email.perl b/git-send-email.perl
index be809e5..d824098 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1045,6 +1045,62 @@ sub maildomain {
 	return maildomain_net() || maildomain_mta() || 'localhost.localdomain';
 }
 
+
+sub read_password_from_stdin {
+	my $line;
+
+	system "stty -echo";
+
+	do {
+		print "Password: ";
+		$line = <STDIN>;
+		print "\n";
+	} while (!defined $line);
+
+	system "stty echo";
+
+	chomp $line;
+	return $line;
+}
+
+sub read_password_from_authinfo {
+	my $fd;
+	if (!open $fd, '<', $ENV{'HOME'} . '/.authinfo') {
+		return;
+	}
+
+	if (!eval { require Text::CSV; 1 }) {
+		print STDERR "Text::CSV missing, won't read ~/.authinfo\n";
+		close $fd;
+		return;
+	}
+
+	my $csv = Text::CSV->new( { sep_char => ' ' } );
+	my $password;
+	while (my $line = <$fd>) {
+		chomp $line;
+		$csv->parse($line);
+		my %row = $csv->fields();
+		if (defined $row{'machine'} &&
+		    defined $row{'login'} &&
+		    defined $row{'port'} &&
+		    defined $row{'password'} &&
+		    $row{'machine'} eq $smtp_server &&
+		    $row{'login'} eq $smtp_authuser &&
+		    $row{'port'} eq $smtp_server_port) {
+			$password = $row{'password'};
+			last;
+		}
+	}
+
+	close $fd;
+	return $password;
+}
+
+sub read_password {
+	return read_password_from_authinfo || read_password_from_stdin;
+}
+
 # Returns 1 if the message was sent, and 0 otherwise.
 # In actuality, the whole program dies when there
 # is an error sending a message.
@@ -1194,18 +1250,7 @@ X-Mailer: git-send-email $gitversion
 			};
 
 			if (!defined $smtp_authpass) {
-
-				system "stty -echo";
-
-				do {
-					print "Password: ";
-					$_ = <STDIN>;
-					print "\n";
-				} while (!defined $_);
-
-				chomp($smtp_authpass = $_);
-
-				system "stty echo";
+				$smtp_authpass = read_password
 			}
 
 			$auth ||= $smtp->auth( $smtp_authuser, $smtp_authpass ) or die $smtp->message;
-- 
1.8.1

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Junio C Hamano]

Michal Nazarewicz <mpn@google.com> writes:

> From: Michal Nazarewicz <mina86@mina86.com>
>
> Make git-send-email read password from a ~/.authinfo file instead of
> requiring it to be stored in git configuration, passed as command line
> argument or typed in.

Makes one wonder why .authinfo and not .netrc; 

http://www.gnu.org/software/emacs/manual/html_node/auth/Help-for-users.html

phrases it amusingly:

        “Netrc” files are usually called .authinfo or .netr
        nowadays .authinfo seems to be more popular and the
        auth-source library encourages this confusion by accepting
        both

Either way it still encourages a plaintext password to be on disk,
which may not be what we want, even though it may be slight if not
really much of an improvement.  Again the Help-for-users has this
amusing bit:

	You could just say (but we don't recommend it, we're just
	showing that it's possible)

	     password mypassword

	to use the same password everywhere. Again, DO NOT DO THIS
	or you will be pwned as the kids say.

> +The '~/.authinfo' file is read if Text::CSV Perl module is installed
> +on the system; if it's missing, a notification message will be printed
> +and the file ignored altogether.  The file should contain a line with
> +the following format:
> ++
> +  machine <domain> port <port> login <user> password <pass>

It is rather strange to require a comma-separated-values parser to
read a file format this simple, isn't it?

> ++
> +Contrary to other tools, 'git-send-email' does not support symbolic
> +port names like 'imap' thus `<port>` must be a number.

Perhaps you can convert at least some popular ones yourself?  After
all, the user may be using an _existing_ .authinfo/.netrc that she
has been using with other programs that do understand symbolic port
names.  Rather than forcing all such users to update their files,
the patch can work a bit harder for them and the world will be a
better place, no?

[PATCHv2] git-send-email: add ~/.authinfo parsing

[Michal Nazarewicz]

From: Michal Nazarewicz <mina86@mina86.com>

Make git-send-email read password from a ~/.authinfo or a ~/.netrc
file instead of requiring it to be stored in git configuration, passed
as command line argument or typed in.

There are various other applications that use this file for
authentication information so letting users use it for git-send-email
is convinient.  Furthermore, some users store their ~/.gitconfig file
in a public repository and having to store password there makes it
easy to publish the password.

Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
---
 Documentation/git-send-email.txt |  34 +++++++++--
 git-send-email.perl              | 124 +++++++++++++++++++++++++++++++++++----
 2 files changed, 140 insertions(+), 18 deletions(-)

On Tue, Jan 29 2013, Junio C Hamano wrote:
> Makes one wonder why .authinfo and not .netrc; 

Fine… Let's parse both. ;)

> Either way it still encourages a plaintext password to be on disk,
> which may not be what we want, even though it may be slight if not
> really much of an improvement.

Well… Users store passwords on disks in a lot of places.  I wager that
most have mail clients configured not to ask for password but instead
store it on hard drive.  I don't see that changing any time soon, so
at least we can try and minimise number of places where a password is
stored.

> It is rather strange to require a comma-separated-values parser to
> read a file format this simple, isn't it?

I was worried about spaces in password.  CVS should handle such case
nicely, whereas simple split won't.  Nonetheless, I guess that in the
end this is not likely enough to add the dependency.

> Perhaps you can convert at least some popular ones yourself?  After
> all, the user may be using an _existing_ .authinfo/.netrc that she
> has been using with other programs that do understand symbolic port
> names.  Rather than forcing all such users to update their files,
> the patch can work a bit harder for them and the world will be a
> better place, no?

Parsing /etc/services added.

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index eeb561c..ee20714 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -158,14 +158,36 @@ Sending
 --smtp-pass[=<password>]::
 	Password for SMTP-AUTH. The argument is optional: If no
 	argument is specified, then the empty string is used as
-	the password. Default is the value of 'sendemail.smtppass',
-	however '--smtp-pass' always overrides this value.
+	the password. Default is the value of 'sendemail.smtppass'
+	or value read from ~/.authinfo file, however '--smtp-pass'
+	always overrides this value.
 +
-Furthermore, passwords need not be specified in configuration files
-or on the command line. If a username has been specified (with
+Furthermore, passwords need not be specified in configuration files or
+on the command line. If a username has been specified (with
 '--smtp-user' or a 'sendemail.smtpuser'), but no password has been
-specified (with '--smtp-pass' or 'sendemail.smtppass'), then the
-user is prompted for a password while the input is masked for privacy.
+specified (with '--smtp-pass', 'sendemail.smtppass' or via
+~/.authinfo file), then the user is prompted for a password while
+the input is masked for privacy.
++
+The ~/.authinfo file should contain a line with the following
+format:
++
+  machine <domain> port <port> login <user> password <pass>
++
+Each pair (expect for `password <pass>`) can be omitted which will
+skip matching of the given value.  Lines are interpreted in order and
+password from the first line that matches will be used.  `<port>` can
+be either an integer or a symbolic name.  In the latter case, it is
+looked up in `/etc/services` file (if it exists).  For instance, you
+can put
++
+  machine example.com login testuser port ssmtp password smtppassword
+  machine example.com login testuser            password testpassword
++
+if you want to use `smtppassword` for authenticating to a service at
+port 465 (SSMTP) and `testpassword` for all other services.  As shown
+in the example, `<port>` can use   If ~/.authinfo file is
+missing, 'git-send-email' will also try ~/.netrc file.
 
 --smtp-server=<host>::
 	If set, specifies the outgoing SMTP server to use (e.g.
diff --git a/git-send-email.perl b/git-send-email.perl
index be809e5..2d8fd1b 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1045,6 +1045,117 @@ sub maildomain {
 	return maildomain_net() || maildomain_mta() || 'localhost.localdomain';
 }
 
+
+sub read_password_from_stdin {
+	my $line;
+
+	system "stty -echo";
+
+	do {
+		print "Password: ";
+		$line = <STDIN>;
+		print "\n";
+	} while (!defined $line);
+
+	system "stty echo";
+
+	chomp $line;
+	return $line;
+}
+
+sub read_etc_services {
+	my $fd;
+	if (!open $fd, '<', '/etc/services') {
+		return {};
+	}
+
+	my $ret = {};
+	while (my $line = <$fd>) {
+		$line =~ s/^\s+|\s*(?:#.*)?$//g;
+		my @line = split /\s+/, $line;
+		if (@line < 2 || $line[1] !~ m~^(\d+)/tcp$~) {
+			next;
+		}
+
+		my $num = int $1;
+		undef $line[1];
+		for my $service (@line) {
+			if (defined $service && !defined $ret->{$service}) {
+				$ret->{$service} = $num;
+			}
+		}
+	}
+
+	close $fd;
+	return $ret;
+}
+
+my $authinfo_parse_port;
+
+sub authinfo_is_eq_port {
+	my ($from_file, $value, $filename) = @_;
+
+	if (!defined $from_file) {
+		return 1;
+	} elsif ($from_file =~ /^\d+$/) {
+		return $from_file == $value;
+	}
+
+	if (!defined $authinfo_parse_port) {
+		$authinfo_parse_port = read_etc_services;
+	}
+
+	my $port = $authinfo_parse_port->{$from_file};
+	if (!defined $port) {
+		print STDERR "$filename: invalid port name: $from_file\n";
+		return;
+	}
+
+	return $port == $value;
+}
+
+sub authinfo_is_eq {
+	my ($from_file, $value) = @_;
+	return defined $from_file || $from_file eq $value;
+}
+
+sub read_password_from_authinfo {
+	my $filename = join '/', $ENV{'HOME'}, $_[0] // '.authinfo';
+	my $fd;
+	if (!open $fd, '<', $filename) {
+		return;
+	}
+
+	my $password;
+	while (my $line = <$fd>) {
+		$line =~ s/^\s+|\s+$//g;
+		my @line = split /\s+/, $line;
+		if (@line % 2) {
+			next;
+		}
+
+		my %line = @line;
+		if (defined $line{'password'} &&
+		    authinfo_is_eq $line{'machine'}, $smtp_server &&
+		    authinfo_is_eq $line{'login'}, $smtp_authuser &&
+		    authinfo_is_eq_port $line{'port'}, $smtp_server_port, $filename) {
+			$password = $line{'password'};
+			last;
+		}
+	}
+
+	close $fd;
+	return $password;
+}
+
+sub read_password {
+	return
+	  read_password_from_authinfo '.authinfo' ||
+	  read_password_from_authinfo '.netrc' ||
+	  read_password_from_stdin;
+}
+
+
 # Returns 1 if the message was sent, and 0 otherwise.
 # In actuality, the whole program dies when there
 # is an error sending a message.
@@ -1194,18 +1305,7 @@ X-Mailer: git-send-email $gitversion
 			};
 
 			if (!defined $smtp_authpass) {
-
-				system "stty -echo";
-
-				do {
-					print "Password: ";
-					$_ = <STDIN>;
-					print "\n";
-				} while (!defined $_);
-
-				chomp($smtp_authpass = $_);
-
-				system "stty echo";
+				$smtp_authpass = read_password
 			}
 
 			$auth ||= $smtp->auth( $smtp_authuser, $smtp_authpass ) or die $smtp->message;
-- 
1.8.1

Re: [PATCHv2] git-send-email: add ~/.authinfo parsing

[Junio C Hamano]

Michal Nazarewicz <mpn@google.com> writes:

>> It is rather strange to require a comma-separated-values parser to
>> read a file format this simple, isn't it?
>
> I was worried about spaces in password.  CVS should handle such case
> nicely, whereas simple split won't.  Nonetheless, I guess that in the
> end this is not likely enough to add the dependency.

But .netrc/.authinfo format separates its entries with SP, HT, or
LF.  An entry begins with "machine <remote-hostname>" token pair.

split(/\s+/) will not work for an entry that span multiple lines but
CSV will not help, either.

Is it bad to use Net::Netrc instead?  This looks like exactly the
use case that module was written for, no?

>> Perhaps you can convert at least some popular ones yourself?  After
>> all, the user may be using an _existing_ .authinfo/.netrc that she
>> has been using with other programs that do understand symbolic port
>> names.  Rather than forcing all such users to update their files,
>> the patch can work a bit harder for them and the world will be a
>> better place, no?
>
> Parsing /etc/services added.

Hmph.  I would have expected to see getservbyname.

[PATCHv3] git-send-email: add ~/.authinfo parsing

[Michal Nazarewicz]

From: Michal Nazarewicz <mina86@mina86.com>

Make git-send-email read password from a ~/.authinfo or ~/.netrc file
instead of requiring it to be stored in git configuration, passed as
command line argument or typed in.

There are various other applications that use this file for
authentication information so letting users use it for git-send-email
is convinient.  Furthermore, some users store their ~/.gitconfig file
in a public repository and having to store password there makes it
easy to publish the password.

Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
---
 Documentation/git-send-email.txt | 47 +++++++++++++++++---
 git-send-email.perl              | 93 ++++++++++++++++++++++++++++++++++------
 2 files changed, 122 insertions(+), 18 deletions(-)

On Tue, Jan 29 2013, Junio C Hamano wrote:
> But .netrc/.authinfo format separates its entries with SP, HT, or
> LF.  An entry begins with "machine <remote-hostname>" token pair.
>
> split(/\s+/) will not work for an entry that span multiple lines but
> CSV will not help, either.
>
> Is it bad to use Net::Netrc instead?  This looks like exactly the
> use case that module was written for, no?

I don't think that's the case.  For one, Net::Netrc does not seem to
process port number.

There is a Text::Authinfo module but it just uses Text::CSV.

I can change the code to use Net::Netrc, but I dunno if that's really
the best option, since I feel people would expect parsing to be
somehow compatible with
<http://www.gnu.org/software/emacs/manual/html_node/gnus/NNTP.html>
rather than the original .netrc file format.

> Hmph.  I would have expected to see getservbyname.

Ha!  Even better. :]

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index eeb561c..ac020d1 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -158,14 +158,49 @@ Sending
 --smtp-pass[=<password>]::
 	Password for SMTP-AUTH. The argument is optional: If no
 	argument is specified, then the empty string is used as
-	the password. Default is the value of 'sendemail.smtppass',
-	however '--smtp-pass' always overrides this value.
+	the password. Default is the value of 'sendemail.smtppass'
+	or value read from ~/.authinfo file, however '--smtp-pass'
+	always overrides this value.
 +
-Furthermore, passwords need not be specified in configuration files
-or on the command line. If a username has been specified (with
+Furthermore, passwords need not be specified in configuration files or
+on the command line. If a username has been specified (with
 '--smtp-user' or a 'sendemail.smtpuser'), but no password has been
-specified (with '--smtp-pass' or 'sendemail.smtppass'), then the
-user is prompted for a password while the input is masked for privacy.
+specified (with '--smtp-pass', 'sendemail.smtppass' or via
+~/.authinfo file), then the user is prompted for a password while
+the input is masked for privacy.
++
+The ~/.authinfo file should contain a line with the following
+format:
++
+  machine <domain> port <port> login <user> password <pass>
++
+Instead of `machine <domain>` pair a `default` token can be used
+instead in which case all domains will match.  Similarly, `port
+<port>` and `login <user>` pairs can be omitted in which case matching
+of the given value will be skipped.  `<port>` can be either an integer
+or a symbolic name.  Lines are interpreted in order and password from
+the first line that matches will be used.  For instance, one may end
+up with:
++
+  machine example.com login jane port ssmtp password smtppassword
+  machine example.com login jane            password janepassword
+  default             login janedoe         password doepassword
++
+if she wants to use `smtppassword` for authenticating as `jane` to
+a service at example.com:465 (SSMTP), `janepassword` for all other
+services at example.com; and `doepassword` when authonticating as
+`janedoe` to any service.  If ~/.authinfo file is missing,
+'git-send-email' will also try ~/.netrc file (even though parsing is
+not fully compatible with ftp's .netrc file format).
++
+Note that you should never make ~/.authinfo file world-readable.  To
+help guarantee that, you might want to create the file with the
+following command:
++
+  ( umask 077; cat >~/.authinfo <<EOF
+  ... file contents ...
+  EOF
+  )
 
 --smtp-server=<host>::
 	If set, specifies the outgoing SMTP server to use (e.g.
diff --git a/git-send-email.perl b/git-send-email.perl
index be809e5..a62dfa4 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1045,6 +1045,86 @@ sub maildomain {
 	return maildomain_net() || maildomain_mta() || 'localhost.localdomain';
 }
 
+
+sub read_password_from_stdin {
+	my $line;
+
+	system "stty -echo";
+
+	do {
+		print "Password: ";
+		$line = <STDIN>;
+		print "\n";
+	} while (!defined $line);
+
+	system "stty echo";
+
+	chomp $line;
+	return $line;
+}
+
+sub authinfo_is_port_eq {
+	my ($from_file, $value, $filename) = @_;
+
+	if (!defined $from_file) {
+		return 1;
+	} elsif ($from_file =~ /^\d+$/) {
+		return $from_file == $value;
+	}
+
+	my $port = getservbyname $from_file, 'tcp';
+	if (!defined $port) {
+		print STDERR "$filename: invalid port name: $from_file\n";
+		return;
+	}
+
+	return $port == $value;
+}
+
+sub read_password_from_authinfo {
+	my $filename = join '/', $ENV{'HOME'}, $_[0] // '.authinfo';
+	my $fd;
+	if (!open $fd, '<', $filename) {
+		return;
+	}
+
+	my $password;
+	while (my $line = <$fd>) {
+		$line =~ s/^\s+|\s+$//g;
+		my @line = split /\s+/, $line;
+		my %line;
+		while (@line) {
+			my $token = shift @line;
+			if ($token eq 'default') {
+				$line{'machine'} = $smtp_server;
+			} elsif (@line) {
+				$line{$token} = shift @line;
+			}
+		}
+
+		if (defined $line{'password'} &&
+		    defined $line{'machine'} &&
+		    $line{'machine'} eq $smtp_server &&
+		    (!defined $line{'login'} ||
+		     $line{'login'} eq $smtp_authuser) &&
+		    authinfo_is_port_eq($line{'port'}, $smtp_server_port, $filename)) {
+			$password = $line{'password'};
+			last;
+		}
+	}
+
+	close $fd;
+	return $password;
+}
+
+sub read_password {
+	return
+	  read_password_from_authinfo '.authinfo' ||
+	  read_password_from_authinfo '.netrc' ||
+	  read_password_from_stdin;
+}
+
+
 # Returns 1 if the message was sent, and 0 otherwise.
 # In actuality, the whole program dies when there
 # is an error sending a message.
@@ -1194,18 +1274,7 @@ X-Mailer: git-send-email $gitversion
 			};
 
 			if (!defined $smtp_authpass) {
-
-				system "stty -echo";
-
-				do {
-					print "Password: ";
-					$_ = <STDIN>;
-					print "\n";
-				} while (!defined $_);
-
-				chomp($smtp_authpass = $_);
-
-				system "stty echo";
+				$smtp_authpass = read_password
 			}
 
 			$auth ||= $smtp->auth( $smtp_authuser, $smtp_authpass ) or die $smtp->message;
-- 
1.8.1

Re: [PATCHv3] git-send-email: add ~/.authinfo parsing

[Junio C Hamano]

Michal Nazarewicz <mpn@google.com> writes:

>> Is it bad to use Net::Netrc instead?  This looks like exactly the
>> use case that module was written for, no?
>
> I don't think that's the case.  For one, Net::Netrc does not seem to
> process port number.
>
> There is a Text::Authinfo module but it just uses Text::CSV.
>
> I can change the code to use Net::Netrc, but I dunno if that's really
> the best option, since I feel people would expect parsing to be
> somehow compatible with
> <http://www.gnu.org/software/emacs/manual/html_node/gnus/NNTP.html>
> rather than the original .netrc file format.

Thanks for pushing back (I wish more contributors did so when I
suggest nonsense ;-)); you are right that both canned modules are
lacking.

Will queue.  Thanks.

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Jeff King]

On Tue, Jan 29, 2013 at 11:53:19AM -0800, Junio C Hamano wrote:

> Either way it still encourages a plaintext password to be on disk,
> which may not be what we want, even though it may be slight if not
> really much of an improvement.  Again the Help-for-users has this
> amusing bit:

I do not mind a .netrc or .authinfo parser, because while those formats
do have security problems, they are standard files that may already be
in use. So as long as we are not encouraging their use, I do not see a
problem in supporting them (and we already do the same with curl's netrc
support).

But it would probably make sense for send-email to support the existing
git-credential subsystem, so that it can take advantage of secure
system-specific storage. And that is where we should be pointing new
users. I think contrib/mw-to-git even has credential support written in
perl, so it would just need to be factored out to Git.pm.

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> But it would probably make sense for send-email to support the existing
> git-credential subsystem, so that it can take advantage of secure
> system-specific storage. And that is where we should be pointing new
> users. I think contrib/mw-to-git even has credential support written in
> perl, so it would just need to be factored out to Git.pm.

Yeah, that sounds like a neat idea.

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Wed, 30 Jan 2013 07:57:29 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> Jeff King <peff@peff.net> writes:
>> But it would probably make sense for send-email to support the existing
>> git-credential subsystem, so that it can take advantage of secure
>> system-specific storage. And that is where we should be pointing new
>> users. I think contrib/mw-to-git even has credential support written in
>> perl, so it would just need to be factored out to Git.pm.

JCH> Yeah, that sounds like a neat idea.

Jeff, is there a way for git-credential to currently support
authinfo/netrc parsing?  I assume that's the right way, instead of using
Michal's proposal to parse internally?

I'd like to add that, plus support for the 'string' and "string"
formats, and authinfo.gpg decoding through GPG.  I'd write it in Perl,
if there's a choice.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Jeff King]

On Thu, Jan 31, 2013 at 10:23:51AM -0500, Ted Zlatanov wrote:

> Jeff, is there a way for git-credential to currently support
> authinfo/netrc parsing?  I assume that's the right way, instead of using
> Michal's proposal to parse internally?
> 
> I'd like to add that, plus support for the 'string' and "string"
> formats, and authinfo.gpg decoding through GPG.  I'd write it in Perl,
> if there's a choice.

Yes, you could write a credential helper that understands netrc and
friends; git talks to the helpers over a socket, so there is no problem
with writing it in Perl. See Documentation/technical/api-credentials.txt
for an overview, or the sample implementation in credential-store.c for a
simple example.

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

On Thu, 31 Jan 2013 14:38:45 -0500 Jeff King <peff@peff.net> wrote: 

JK> On Thu, Jan 31, 2013 at 10:23:51AM -0500, Ted Zlatanov wrote:
>> Jeff, is there a way for git-credential to currently support
>> authinfo/netrc parsing?  I assume that's the right way, instead of using
>> Michal's proposal to parse internally?
>> 
>> I'd like to add that, plus support for the 'string' and "string"
>> formats, and authinfo.gpg decoding through GPG.  I'd write it in Perl,
>> if there's a choice.

JK> Yes, you could write a credential helper that understands netrc and
JK> friends; git talks to the helpers over a socket, so there is no problem
JK> with writing it in Perl. See Documentation/technical/api-credentials.txt
JK> for an overview, or the sample implementation in credential-store.c for a
JK> simple example.

I wrote a Perl credential helper for netrc parsing which is pretty
robust, has built-in docs with -h, and doesn't depend on external
modules.  The netrc parser regex was stolen from Net::Netrc.

It will by default use ~/.authinfo.gpg, ~/.netrc.gpg, ~/.authinfo, and
~/.netrc (whichever is found first) and this can be overridden with -f.

If the file name ends with ".gpg", it will run "gpg --decrypt FILE" and
use the output.  So non-interactively, that could hang if GPG was
waiting for input.  Does Git handle that, or should I check for a TTY?

Take a look at the proposed patch and let me know if it's usable, if you
need a formal copyright assignment, etc.

Thanks
Ted


[-- Attachment #2: Add git-credential-netrc --]
[-- Type: text/x-patch, Size: 6016 bytes --]

commit 3d28bc2a610ebcc988eba5443d82d0ded92c24bc
Author: Ted Zlatanov <tzz@lifelogs.com>
Date:   Sat Feb 2 06:42:13 2013 -0500

    Add contrib/credentials/netrc with GPG support

diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
new file mode 100755
index 0000000..92fc306
--- /dev/null
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -0,0 +1,242 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Data::Dumper;
+
+use Getopt::Long;
+use File::Basename;
+
+my $VERSION = "0.1";
+
+my %options = (
+               help => 0,
+               debug => 0,
+
+               # identical token maps, e.g. host -> host, will be inserted later
+               tmap => {
+                        port => 'protocol',
+                        machine => 'host',
+                        path => 'path',
+                        login => 'username',
+                        user => 'username',
+                        password => 'password',
+                       }
+              );
+
+foreach my $v (values %{$options{tmap}})
+{
+ $options{tmap}->{$v} = $v;
+}
+
+foreach my $suffix ('.gpg', '')
+{
+ foreach my $base (qw/authinfo netrc/)
+ {
+  my $file = glob("~/.$base$suffix");
+  next unless (defined $file && -f $file);
+  $options{file} = $file ;
+ }
+}
+
+Getopt::Long::Configure("bundling");
+
+# TODO: maybe allow the token map $options{tmap} to be configurable.
+GetOptions(\%options,
+           "help|h",
+           "debug|d",
+           "file|f=s",
+          );
+
+if ($options{help})
+{
+ my $shortname = basename($0);
+ $shortname =~ s/git-credential-//;
+
+ print <<EOHIPPUS;
+
+$0 [-f AUTHFILE] [-d] get
+
+Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.
+
+Options:
+  -f AUTHFILE: specify a netrc-style file
+  -d: turn on debugging
+
+To enable (note that Git will prepend "git-credential-" to the helper
+name and look for it in the path):
+
+  git config credential.helper '$shortname -f AUTHFILE'
+
+And if you want lots of debugging info:
+
+  git config credential.helper '$shortname -f AUTHFILE -d'
+
+Only "get" mode is supported by this credential helper.  It opens
+AUTHFILE and looks for entries that match the requested search
+criteria:
+
+ 'port|protocol':
+   The protocol that will be used (e.g., https). (protocol=X)
+
+ 'machine|host':
+   The remote hostname for a network credential. (host=X)
+
+ 'path':
+   The path with which the credential will be used. (path=X)
+
+ 'login|user|username':
+   The credential’s username, if we already have one. (username=X)
+
+Thus, when we get "protocol=https\nusername=tzz", this credential
+helper will look for lines in AUTHFILE that match
+
+port https login tzz
+
+OR
+
+protocol https login tzz
+
+OR... etc. acceptable tokens as listed above.  Any unknown tokens are
+simply ignored.
+
+Then, the helper will print out whatever tokens it got from the line,
+including "password" tokens, mapping e.g. "port" back to "protocol".
+
+The first matching line is used.  Tokens can be quoted as 'STRING' or
+"STRING".
+
+No caching is performed by this credential helper.
+
+EOHIPPUS
+
+ exit;
+}
+
+my $mode = shift @ARGV;
+
+# credentials may get 'get', 'store', or 'erase' as parameters but
+# only acknowledge 'get'
+die "Syntax: $0 [-f AUTHFILE] [-d] get" unless defined $mode;
+
+# only support 'get' mode
+exit unless $mode eq 'get';
+
+my $debug = $options{debug};
+my $file = $options{file};
+
+die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
+ unless defined $file;
+
+die "Sorry, the specified netrc $file is not accessible"
+ unless -f $file;
+
+if ($file =~ m/\.gpg$/)
+{
+ $file = "gpg --decrypt $file|";
+}
+
+my @data = load($file);
+chomp @data;
+
+die "Sorry, we could not load data from [$file]"
+ unless (scalar @data);
+
+# the query
+my %q;
+
+foreach my $v (values %{$options{tmap}})
+{
+ undef $q{$v};
+}
+
+while (<STDIN>)
+{
+ next unless m/([a-z]+)=(.+)/;
+
+ my ($token, $value) = ($1, $2);
+ die "Unknown search token $1" unless exists $q{$token};
+ $q{$token} = $value;
+}
+
+# build reverse token map
+my %rmap;
+foreach my $k (keys %{$options{tmap}})
+{
+ push @{$rmap{$options{tmap}->{$k}}}, $k;
+}
+
+# there are CPAN modules to do this better, but we want to avoid
+# dependencies and generally, complex netrc-style files are rare
+
+if ($debug)
+{
+ foreach (sort keys %q)
+ {
+  printf STDERR "searching for %s = %s\n",
+   $_, $q{$_} || '(any value)';
+ }
+}
+
+LINE: foreach my $line (@data)
+{
+
+ print STDERR "line [$line]\n" if $debug;
+ my @tok;
+ # gratefully stolen from Net::Netrc
+ while (length $line &&
+        $line =~ s/^("((?:[^"]+|\\.)*)"|((?:[^\\\s]+|\\.)*))\s*//)
+ {
+  (my $tok = $+) =~ s/\\(.)/$1/g;
+  push(@tok, $tok);
+ }
+
+ my %tokens;
+ while (@tok)
+ {
+  my ($k, $v) = (shift @tok, shift @tok);
+  next unless defined $v;
+  next unless exists $options{tmap}->{$k};
+  $tokens{$options{tmap}->{$k}} = $v;
+ }
+
+ foreach my $check (sort keys %q)
+ {
+  if (exists $tokens{$check} && defined $q{$check})
+  {
+   print STDERR "comparing [$tokens{$check}] to [$q{$check}] in line [$line]\n" if $debug;
+   next LINE unless $tokens{$check} eq $q{$check};
+  }
+  else
+  {
+   print STDERR "we could not find [$check] but it's OK\n" if $debug;
+  }
+ }
+
+ print STDERR "line has passed all the search checks\n" if $debug;
+ foreach my $token (sort keys %rmap)
+ {
+  print STDERR "looking for useful token $token\n" if $debug;
+  next unless exists $tokens{$token}; # did we match?
+
+  foreach my $rctoken (@{$rmap{$token}})
+  {
+   next if defined $q{$rctoken};           # don't re-print given tokens
+  }
+
+  print STDERR "FOUND: $token=$tokens{$token}\n" if $debug;
+  printf "%s=%s\n", $token, $tokens{$token};
+ }
+
+ last;
+}
+
+sub load
+{
+ my $file = shift;
+ # this supports pipes too
+ my $io = new IO::File($file) or die "Could not open $file: $!\n";
+
+ return <$io>;                          # whole file
+}

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Jeff King]

On Sat, Feb 02, 2013 at 06:57:29AM -0500, Ted Zlatanov wrote:

> I wrote a Perl credential helper for netrc parsing which is pretty
> robust, has built-in docs with -h, and doesn't depend on external
> modules.  The netrc parser regex was stolen from Net::Netrc.
>
> It will by default use ~/.authinfo.gpg, ~/.netrc.gpg, ~/.authinfo, and
> ~/.netrc (whichever is found first) and this can be overridden with -f.

Cool, thanks for working on this.

> If the file name ends with ".gpg", it will run "gpg --decrypt FILE" and
> use the output.  So non-interactively, that could hang if GPG was
> waiting for input.  Does Git handle that, or should I check for a TTY?

No, git does not do anything special with respect to credential helpers
and ttys (nor should it, since one use of helpers is to get credentials
when there is no tty). I think it is GPG's problem to deal with, though.
We will invoke it, and it is up to it to decide whether it can acquire
the passphrase or not (either through the tty, or possibly from
gpg-agent). So it would be wrong to do the tty check yourself.

I haven't tested GPG, but I assume it properly tries to read from
/dev/tty and not stdin. Your helper's stdio is connected to git and
speaking the credential-helper protocol, so GPG reading from stdin would
either steal your input (if run before you read it), or just get EOF (if
you have read all of the pipe content already). If GPG isn't well
behaved, it may be worth redirecting its stdin from /dev/null as a
safety measure.

> Take a look at the proposed patch and let me know if it's usable, if you
> need a formal copyright assignment, etc.

Overall looks sane to me, though my knowledge of .netrc is not
especially good. Usually we try to send patches inline in the email
(i.e., as generated by git-format-patch), and include a "Signed-off-by"
line indicating that content is released to the project; see
Documentation/SubmittingPatches.

> +use Data::Dumper;

I don't see it used here. Leftover from debugging?

> + print <<EOHIPPUS;

Cute, I haven't seen that one before.

> +$0 [-f AUTHFILE] [-d] get
> +
> +Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.

I don't know if we have a particular policy for items in contrib/, but
this license may be too vague. In particular, it does not explicitly
allow redistribution, which would make Junio shipping a release with it
a copyright violation.

Any objection to just putting it under some well-known simple license
(GPL, BSD, or whatever)?

> +if ($file =~ m/\.gpg$/)
> +{
> + $file = "gpg --decrypt $file|";
> +}

Does this need to quote $file, since the result will get passed to the
shell? It might be easier to just use the list form of open(), like:

  my @data = $file =~ /\.gpg$/ ?
             load('-|', qw(gpg --decrypt), $file) :
             load('<', $file);

(and then obviously update load to just dump all of @_ to open()).

> +die "Sorry, we could not load data from [$file]"
> + unless (scalar @data);

Probably not that interesting a corner case, but this means we die on an
empty .netrc, whereas it might be more sensible for it to behave as "no
match".

For the same reason, it might be worth silently exiting when we don't
find a .netrc (or any of its variants). That lets people who share their
dot-files across machines configure git globally, even if they don't
necessarily have a netrc on every machine.

> +# the query
> +my %q;
> +
> +foreach my $v (values %{$options{tmap}})
> +{
> + undef $q{$v};
> +}

Just my personal style, but I find the intent more obvious with "map" (I
know some people find it unreadable, though):

  my %q = map { $_ => undef } values(%{$options{tmap}});

> +while (<STDIN>)
> +{
> + next unless m/([a-z]+)=(.+)/;

We don't currently have any exotic tokens that this would not match, nor
do I plan to add them, but the credential documentation defines a valid
line as /^([^=]+)=(.+)/.

It's also possible for the value to be empty, but I do not think
off-hand that current git will ever send such an empty value.

> [...]

The rest of it looks fine to me. I don't think any of my comments are
show-stoppers. Tests would be nice, but integrating contrib/ stuff with
the test harness is kind of a pain.

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Sun, 3 Feb 2013 14:41:49 -0500 Jeff King <peff@peff.net> wrote: 

JK> On Sat, Feb 02, 2013 at 06:57:29AM -0500, Ted Zlatanov wrote:
>> If the file name ends with ".gpg", it will run "gpg --decrypt FILE" and
>> use the output.  So non-interactively, that could hang if GPG was
>> waiting for input.  Does Git handle that, or should I check for a TTY?

JK> No, git does not do anything special with respect to credential helpers
JK> and ttys (nor should it, since one use of helpers is to get credentials
JK> when there is no tty). I think it is GPG's problem to deal with, though.
JK> We will invoke it, and it is up to it to decide whether it can acquire
JK> the passphrase or not (either through the tty, or possibly from
JK> gpg-agent). So it would be wrong to do the tty check yourself.

JK> I haven't tested GPG, but I assume it properly tries to read from
JK> /dev/tty and not stdin. Your helper's stdio is connected to git and
JK> speaking the credential-helper protocol, so GPG reading from stdin would
JK> either steal your input (if run before you read it), or just get EOF (if
JK> you have read all of the pipe content already). If GPG isn't well
JK> behaved, it may be worth redirecting its stdin from /dev/null as a
JK> safety measure.

In my testing GPG did the right thing, so I think this is OK.

>> Take a look at the proposed patch and let me know if it's usable, if you
>> need a formal copyright assignment, etc.

JK> Overall looks sane to me, though my knowledge of .netrc is not
JK> especially good. Usually we try to send patches inline in the email
JK> (i.e., as generated by git-format-patch), and include a "Signed-off-by"
JK> line indicating that content is released to the project; see
JK> Documentation/SubmittingPatches.

OK, thanks.  I will fire that off.

>> +use Data::Dumper;

JK> I don't see it used here. Leftover from debugging?

It's part of my Perl new script skeleton, sorry.

>> + print <<EOHIPPUS;

JK> Cute, I haven't seen that one before.

Heh heh.  I've had to explain that one in code review many times.  "See,
it's the precursor to the modern horse..."

>> +$0 [-f AUTHFILE] [-d] get
>> +
>> +Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.

JK> I don't know if we have a particular policy for items in contrib/, but
JK> this license may be too vague. In particular, it does not explicitly
JK> allow redistribution, which would make Junio shipping a release with it
JK> a copyright violation.

JK> Any objection to just putting it under some well-known simple license
JK> (GPL, BSD, or whatever)?

No, I didn't know what Git requires, and I'd like it to be the least
restrictive, so BSD is OK.  Stated in -h now.

>> +if ($file =~ m/\.gpg$/)
>> +{
>> + $file = "gpg --decrypt $file|";
>> +}

JK> Does this need to quote $file, since the result will get passed to the
JK> shell? It might be easier to just use the list form of open(), like:

JK>   my @data = $file =~ /\.gpg$/ ?
JK>              load('-|', qw(gpg --decrypt), $file) :
JK>              load('<', $file);

JK> (and then obviously update load to just dump all of @_ to open()).

Yes, thanks.  Done.

>> +die "Sorry, we could not load data from [$file]"
>> + unless (scalar @data);

JK> Probably not that interesting a corner case, but this means we die on an
JK> empty .netrc, whereas it might be more sensible for it to behave as "no
JK> match".

JK> For the same reason, it might be worth silently exiting when we don't
JK> find a .netrc (or any of its variants). That lets people who share their
JK> dot-files across machines configure git globally, even if they don't
JK> necessarily have a netrc on every machine.

OK; done.

>> +# the query
>> +my %q;
>> +
>> +foreach my $v (values %{$options{tmap}})
>> +{
>> + undef $q{$v};
>> +}

JK> Just my personal style, but I find the intent more obvious with "map" (I
JK> know some people find it unreadable, though):

JK>   my %q = map { $_ => undef } values(%{$options{tmap}});

Yes, changed.

>> +while (<STDIN>)
>> +{
>> + next unless m/([a-z]+)=(.+)/;

JK> We don't currently have any exotic tokens that this would not match, nor
JK> do I plan to add them, but the credential documentation defines a valid
JK> line as /^([^=]+)=(.+)/.

JK> It's also possible for the value to be empty, but I do not think
JK> off-hand that current git will ever send such an empty value.

Yes, changed.

JK> The rest of it looks fine to me. I don't think any of my comments are
JK> show-stoppers. Tests would be nice, but integrating contrib/ stuff with
JK> the test harness is kind of a pain.

"I tested it on AIX, it works great!" :)

It's pretty easy to write a local Makefile with a test target, if you
think it worthwhile.

Ted

[PATCH 1/3] Add contrib/credentials/netrc with GPG support

[Ted Zlatanov]

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 contrib/credential/netrc/git-credential-netrc |  242 +++++++++++++++++++++++++
 1 files changed, 242 insertions(+), 0 deletions(-)
 create mode 100755 contrib/credential/netrc/git-credential-netrc

diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
new file mode 100755
index 0000000..92fc306
--- /dev/null
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -0,0 +1,242 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Data::Dumper;
+
+use Getopt::Long;
+use File::Basename;
+
+my $VERSION = "0.1";
+
+my %options = (
+               help => 0,
+               debug => 0,
+
+               # identical token maps, e.g. host -> host, will be inserted later
+               tmap => {
+                        port => 'protocol',
+                        machine => 'host',
+                        path => 'path',
+                        login => 'username',
+                        user => 'username',
+                        password => 'password',
+                       }
+              );
+
+foreach my $v (values %{$options{tmap}})
+{
+ $options{tmap}->{$v} = $v;
+}
+
+foreach my $suffix ('.gpg', '')
+{
+ foreach my $base (qw/authinfo netrc/)
+ {
+  my $file = glob("~/.$base$suffix");
+  next unless (defined $file && -f $file);
+  $options{file} = $file ;
+ }
+}
+
+Getopt::Long::Configure("bundling");
+
+# TODO: maybe allow the token map $options{tmap} to be configurable.
+GetOptions(\%options,
+           "help|h",
+           "debug|d",
+           "file|f=s",
+          );
+
+if ($options{help})
+{
+ my $shortname = basename($0);
+ $shortname =~ s/git-credential-//;
+
+ print <<EOHIPPUS;
+
+$0 [-f AUTHFILE] [-d] get
+
+Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.
+
+Options:
+  -f AUTHFILE: specify a netrc-style file
+  -d: turn on debugging
+
+To enable (note that Git will prepend "git-credential-" to the helper
+name and look for it in the path):
+
+  git config credential.helper '$shortname -f AUTHFILE'
+
+And if you want lots of debugging info:
+
+  git config credential.helper '$shortname -f AUTHFILE -d'
+
+Only "get" mode is supported by this credential helper.  It opens
+AUTHFILE and looks for entries that match the requested search
+criteria:
+
+ 'port|protocol':
+   The protocol that will be used (e.g., https). (protocol=X)
+
+ 'machine|host':
+   The remote hostname for a network credential. (host=X)
+
+ 'path':
+   The path with which the credential will be used. (path=X)
+
+ 'login|user|username':
+   The credential’s username, if we already have one. (username=X)
+
+Thus, when we get "protocol=https\nusername=tzz", this credential
+helper will look for lines in AUTHFILE that match
+
+port https login tzz
+
+OR
+
+protocol https login tzz
+
+OR... etc. acceptable tokens as listed above.  Any unknown tokens are
+simply ignored.
+
+Then, the helper will print out whatever tokens it got from the line,
+including "password" tokens, mapping e.g. "port" back to "protocol".
+
+The first matching line is used.  Tokens can be quoted as 'STRING' or
+"STRING".
+
+No caching is performed by this credential helper.
+
+EOHIPPUS
+
+ exit;
+}
+
+my $mode = shift @ARGV;
+
+# credentials may get 'get', 'store', or 'erase' as parameters but
+# only acknowledge 'get'
+die "Syntax: $0 [-f AUTHFILE] [-d] get" unless defined $mode;
+
+# only support 'get' mode
+exit unless $mode eq 'get';
+
+my $debug = $options{debug};
+my $file = $options{file};
+
+die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
+ unless defined $file;
+
+die "Sorry, the specified netrc $file is not accessible"
+ unless -f $file;
+
+if ($file =~ m/\.gpg$/)
+{
+ $file = "gpg --decrypt $file|";
+}
+
+my @data = load($file);
+chomp @data;
+
+die "Sorry, we could not load data from [$file]"
+ unless (scalar @data);
+
+# the query
+my %q;
+
+foreach my $v (values %{$options{tmap}})
+{
+ undef $q{$v};
+}
+
+while (<STDIN>)
+{
+ next unless m/([a-z]+)=(.+)/;
+
+ my ($token, $value) = ($1, $2);
+ die "Unknown search token $1" unless exists $q{$token};
+ $q{$token} = $value;
+}
+
+# build reverse token map
+my %rmap;
+foreach my $k (keys %{$options{tmap}})
+{
+ push @{$rmap{$options{tmap}->{$k}}}, $k;
+}
+
+# there are CPAN modules to do this better, but we want to avoid
+# dependencies and generally, complex netrc-style files are rare
+
+if ($debug)
+{
+ foreach (sort keys %q)
+ {
+  printf STDERR "searching for %s = %s\n",
+   $_, $q{$_} || '(any value)';
+ }
+}
+
+LINE: foreach my $line (@data)
+{
+
+ print STDERR "line [$line]\n" if $debug;
+ my @tok;
+ # gratefully stolen from Net::Netrc
+ while (length $line &&
+        $line =~ s/^("((?:[^"]+|\\.)*)"|((?:[^\\\s]+|\\.)*))\s*//)
+ {
+  (my $tok = $+) =~ s/\\(.)/$1/g;
+  push(@tok, $tok);
+ }
+
+ my %tokens;
+ while (@tok)
+ {
+  my ($k, $v) = (shift @tok, shift @tok);
+  next unless defined $v;
+  next unless exists $options{tmap}->{$k};
+  $tokens{$options{tmap}->{$k}} = $v;
+ }
+
+ foreach my $check (sort keys %q)
+ {
+  if (exists $tokens{$check} && defined $q{$check})
+  {
+   print STDERR "comparing [$tokens{$check}] to [$q{$check}] in line [$line]\n" if $debug;
+   next LINE unless $tokens{$check} eq $q{$check};
+  }
+  else
+  {
+   print STDERR "we could not find [$check] but it's OK\n" if $debug;
+  }
+ }
+
+ print STDERR "line has passed all the search checks\n" if $debug;
+ foreach my $token (sort keys %rmap)
+ {
+  print STDERR "looking for useful token $token\n" if $debug;
+  next unless exists $tokens{$token}; # did we match?
+
+  foreach my $rctoken (@{$rmap{$token}})
+  {
+   next if defined $q{$rctoken};           # don't re-print given tokens
+  }
+
+  print STDERR "FOUND: $token=$tokens{$token}\n" if $debug;
+  printf "%s=%s\n", $token, $tokens{$token};
+ }
+
+ last;
+}
+
+sub load
+{
+ my $file = shift;
+ # this supports pipes too
+ my $io = new IO::File($file) or die "Could not open $file: $!\n";
+
+ return <$io>;                          # whole file
+}
-- 
1.7.9.rc2

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

[Ted Zlatanov]

Grr, sorry for the bad formatting.  First time doing format-patch.

Ted

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

[Junio C Hamano]

[administrivia: I would really wish you didn't put "Mail-copies-to:
never" above].

Ted Zlatanov <tzz@lifelogs.com> writes:

> +foreach my $v (values %{$options{tmap}})
> +{
> + $options{tmap}->{$v} = $v;
> +}

Please follow the styles of existing Perl scripts, e.g. indent with
tab, etc.  Style requests are not optional; it is a prerequisite to
make the patch readable and reviewable.

> + print <<EOHIPPUS;
> + ...
> +EOHIPPUS

Do we really need to refer readers to Wikipedia or something to
learn about extinct equid ungulates ;-)?

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 1969 bytes --]

On Mon, 04 Feb 2013 09:24:03 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> [administrivia: I would really wish you didn't put "Mail-copies-to:
JCH> never" above].

I normally post through GMane and don't need the extra CC on any list I
read.  I'll make an effort to remove that header here, and apologize for
the inconvenience.

JCH> Ted Zlatanov <tzz@lifelogs.com> writes:

>> +foreach my $v (values %{$options{tmap}})
>> +{
>> + $options{tmap}->{$v} = $v;
>> +}

JCH> Please follow the styles of existing Perl scripts, e.g. indent with
JCH> tab, etc.  Style requests are not optional; it is a prerequisite to
JCH> make the patch readable and reviewable.

Sorry, I didn't realize contrib/ stuff was under the same rules.  I will
attempt to make my contributions fit the project's requirements.

It would help if the requirements were codified as the fairly standard
Emacs file-local variables, so I can just put them in the Perl code or
in .dir-locals.el in the source tree.  At least for Perl I'd like that,
and it could be nice for the Emacs users who write C too.

Would you like me to propose that as a patch?

Either way, I guessed that these settings are what you want as far as
tabs and indentation (I use cperl-mode but perl-mode is the same):

# -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-

...plus hanging braces and avoiding one-line blocks.  I hope that's right.

>> + print <<EOHIPPUS;
>> + ...
>> +EOHIPPUS

JCH> Do we really need to refer readers to Wikipedia or something to
JCH> learn about extinct equid ungulates ;-)?

I think the marker's name is irrelevant, and hope you are OK with
leaving it.

Since the change is a pretty big reformatting, should I squash my 3
commits plus the reformatting commit into one patch, or keep them as a
series?

I am appending the script in its current form so you can review it and
tell me if there's anything else I should add or change in the
formatting.

Thanks
Ted


[-- Attachment #2: git-credential-netrc --]
[-- Type: text/plain, Size: 5958 bytes --]

#!/usr/bin/perl
# -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-

use strict;
use warnings;

use Getopt::Long;
use File::Basename;

my $VERSION = "0.1";

my %options = (
               help => 0,
               debug => 0,

               # identical token maps, e.g. host -> host, will be inserted later
               tmap => {
                        port => 'protocol',
                        machine => 'host',
                        path => 'path',
                        login => 'username',
                        user => 'username',
                        password => 'password',
                       }
              );

# map each credential protocol token to itself on the netrc side
$options{tmap}->{$_} = $_ foreach my $v (values %{$options{tmap}});

foreach my $suffix ('.gpg', '') {
    foreach my $base (qw/authinfo netrc/) {
	my $file = glob("~/.$base$suffix");
	next unless (defined $file && -f $file);
	$options{file} = $file ;
    }
}

Getopt::Long::Configure("bundling");

# TODO: maybe allow the token map $options{tmap} to be configurable.
GetOptions(\%options,
           "help|h",
           "debug|d",
           "file|f=s",
          );

if ($options{help}) {
    my $shortname = basename($0);
    $shortname =~ s/git-credential-//;

    print <<EOHIPPUS;

$0 [-f AUTHFILE] [-d] get

Version $VERSION by tzz\@lifelogs.com.  License: BSD.

Options:
  -f AUTHFILE: specify a netrc-style file
  -d: turn on debugging

To enable (note that Git will prepend "git-credential-" to the helper
name and look for it in the path):

  git config credential.helper '$shortname -f AUTHFILE'

And if you want lots of debugging info:

  git config credential.helper '$shortname -f AUTHFILE -d'

Only "get" mode is supported by this credential helper.  It opens
AUTHFILE and looks for entries that match the requested search
criteria:

 'port|protocol':
   The protocol that will be used (e.g., https). (protocol=X)

 'machine|host':
   The remote hostname for a network credential. (host=X)

 'path':
   The path with which the credential will be used. (path=X)

 'login|user|username':
   The credential’s username, if we already have one. (username=X)

Thus, when we get "protocol=https\nusername=tzz", this credential
helper will look for lines in AUTHFILE that match

port https login tzz

OR

protocol https login tzz

OR... etc. acceptable tokens as listed above.  Any unknown tokens are
simply ignored.

Then, the helper will print out whatever tokens it got from the line,
including "password" tokens, mapping e.g. "port" back to "protocol".

The first matching line is used.  Tokens can be quoted as 'STRING' or
"STRING".

No caching is performed by this credential helper.

EOHIPPUS

    exit;
}

my $mode = shift @ARGV;

# credentials may get 'get', 'store', or 'erase' as parameters but
# only acknowledge 'get'
die "Syntax: $0 [-f AUTHFILE] [-d] get" unless defined $mode;

# only support 'get' mode
exit unless $mode eq 'get';

my $debug = $options{debug};
my $file = $options{file};

die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
 unless defined $file;

unless (-f $file) {
    print STDERR "Sorry, the specified netrc $file is not accessible\n" if $debug;
    exit 0;
}

my @data;
if ($file =~ m/\.gpg$/) {
    @data = load('-|', qw(gpg --decrypt), $file)
}
else {
    @data = load('<', $file);
}

chomp @data;

unless (scalar @data) {
    print STDERR "Sorry, we could not load data from [$file]\n" if $debug;
    exit;
}

# the query: start with every token with no value
my %q = map { $_ => undef } values(%{$options{tmap}});

while (<STDIN>) {
    next unless m/([^=]+)=(.+)/;

    my ($token, $value) = ($1, $2);
    die "Unknown search token $1" unless exists $q{$token};
    $q{$token} = $value;
}

# build reverse token map
my %rmap;
foreach my $k (keys %{$options{tmap}}) {
    push @{$rmap{$options{tmap}->{$k}}}, $k;
}

# there are CPAN modules to do this better, but we want to avoid
# dependencies and generally, complex netrc-style files are rare

if ($debug) {
    printf STDERR "searching for %s = %s\n", $_, $q{$_} || '(any value)'
     foreach sort keys %q;
}

LINE: foreach my $line (@data) {

    print STDERR "line [$line]\n" if $debug;
    my @tok;
    # gratefully stolen from Net::Netrc
    while (length $line &&
	   $line =~ s/^("((?:[^"]+|\\.)*)"|((?:[^\\\s]+|\\.)*))\s*//) {
	(my $tok = $+) =~ s/\\(.)/$1/g;
	push(@tok, $tok);
    }

    # skip blank lines, comments, etc.
    next LINE unless scalar @tok;

    my %tokens;
    while (@tok) {
	my ($k, $v) = (shift @tok, shift @tok);
	next unless defined $v;
	next unless exists $options{tmap}->{$k};
	$tokens{$options{tmap}->{$k}} = $v;
    }

    foreach my $check (sort keys %q) {
	if (exists $tokens{$check} && defined $q{$check}) {
	    print STDERR "comparing [$tokens{$check}] to [$q{$check}] in line [$line]\n" if $debug;
	    next LINE unless $tokens{$check} eq $q{$check};
	}
	else {
	    print STDERR "we could not find [$check] but it's OK\n" if $debug;
	}
    }

    print STDERR "line has passed all the search checks\n" if $debug;
 TOKEN:
    foreach my $token (sort keys %rmap) {
	print STDERR "looking for useful token $token\n" if $debug;
	next unless exists $tokens{$token}; # did we match?

	foreach my $rctoken (@{$rmap{$token}}) {
	    next TOKEN if defined $q{$rctoken};           # don't re-print given tokens
	}

	print STDERR "FOUND: $token=$tokens{$token}\n" if $debug;
	printf "%s=%s\n", $token, $tokens{$token};
    }

    last;
}

sub load {
    # this supports pipes too
    my $io = new IO::File(@_) or die "Could not open [@_]: $!\n";
    return <$io>;                          # whole file
}

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

[Junio C Hamano]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Sorry, I didn't realize contrib/ stuff was under the same rules.

I had a feeling that this may start out from contrib/ but will soon
prove to be fairly important to be part of the Git proper.

> It would help if the requirements were codified as the fairly standard
> Emacs file-local variables, so I can just put them in the Perl code or
> in .dir-locals.el in the source tree.  At least for Perl I'd like that,
> and it could be nice for the Emacs users who write C too.
>
> Would you like me to propose that as a patch?

I thought that we tend to avoid Emacs/Vim formatting cruft left in
the file.  Do we have any in existing file outside contrib/?

> Either way, I guessed that these settings are what you want as far as
> tabs and indentation (I use cperl-mode but perl-mode is the same):
>
> # -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-

Indent is done with a Tab and indent level is 8 places (check add--interactive.perl
and imitate it, perhaps?).

Thanks.

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

[Ted Zlatanov]

On Mon, 04 Feb 2013 11:06:16 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Sorry, I didn't realize contrib/ stuff was under the same rules.

JCH> I had a feeling that this may start out from contrib/ but will soon
JCH> prove to be fairly important to be part of the Git proper.

Cool!

>> It would help if the requirements were codified as the fairly standard
>> Emacs file-local variables, so I can just put them in the Perl code or
>> in .dir-locals.el in the source tree.  At least for Perl I'd like that,
>> and it could be nice for the Emacs users who write C too.
>> 
>> Would you like me to propose that as a patch?

JCH> I thought that we tend to avoid Emacs/Vim formatting cruft left in
JCH> the file.  Do we have any in existing file outside contrib/?

No, but it's a nice way to express the settings so no one is guessing
what the project prefers.  At least for me it's not an issue anymore,
since I understand your criteria better now, so let me know if you want
me to express it in the CodingGuidelines, in a dir-locals.el file, or
somewhere else.

>> Either way, I guessed that these settings are what you want as far as
>> tabs and indentation (I use cperl-mode but perl-mode is the same):
>> 
>> # -*- mode: cperl; tab-width: 8; cperl-indent-level: 4; indent-tabs-mode: t; -*-

JCH> Indent is done with a Tab and indent level is 8 places (check add--interactive.perl
JCH> and imitate it, perhaps?).

Yup, got it.  My mistake on the size-4 indents.

I found this helpful, at least while I was indenting, for anyone else
who might want to indent Perl appropriately to imitate existing Perl code:

# -*- mode: cperl; tab-width: 8; cperl-indent-level: 8; indent-tabs-mode: t; -*-

I'll resubmit now.

Thanks
Ted

Re: [PATCH 1/3] Add contrib/credentials/netrc with GPG support

[Junio C Hamano]

Ted Zlatanov <tzz@lifelogs.com> writes:

> JCH> I thought that we tend to avoid Emacs/Vim formatting cruft left in
> JCH> the file.  Do we have any in existing file outside contrib/?
>
> No, but it's a nice way to express the settings so no one is guessing
> what the project prefers.  At least for me it's not an issue anymore,
> since I understand your criteria better now, so let me know if you want
> me to express it in the CodingGuidelines, in a dir-locals.el file, or
> somewhere else.

Historically we treated this from CodingGuidelines a sufficient
clue:

    As for more concrete guidelines, just imitate the existing code
    (this is a good guideline, no matter which project you are
    contributing to). It is always preferable to match the _local_
    convention. New code added to git suite is expected to match
    the overall style of existing code. Modifications to existing
    code is expected to match the style the surrounding code already
    uses (even if it doesn't match the overall style of existing code).

but over time people wanted more specific guidelines and added
language specific style guides there.  We have sections that cover
C, shell and Python, and I do not think adding Perl would not hurt.

CodingGuidelines Perl amendment (was: [PATCH 1/3] Add contrib/credentials/netrc with GPG support)

[Ted Zlatanov]

On Mon, 04 Feb 2013 15:10:45 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> Ted Zlatanov <tzz@lifelogs.com> writes:
JCH> I thought that we tend to avoid Emacs/Vim formatting cruft left in
JCH> the file.  Do we have any in existing file outside contrib/?
>> 
>> No, but it's a nice way to express the settings so no one is guessing
>> what the project prefers.  At least for me it's not an issue anymore,
>> since I understand your criteria better now, so let me know if you want
>> me to express it in the CodingGuidelines, in a dir-locals.el file, or
>> somewhere else.

JCH> Historically we treated this from CodingGuidelines a sufficient
JCH> clue:

JCH>     As for more concrete guidelines, just imitate the existing code
JCH>     (this is a good guideline, no matter which project you are
JCH>     contributing to). It is always preferable to match the _local_
JCH>     convention. New code added to git suite is expected to match
JCH>     the overall style of existing code. Modifications to existing
JCH>     code is expected to match the style the surrounding code already
JCH>     uses (even if it doesn't match the overall style of existing code).

JCH> but over time people wanted more specific guidelines and added
JCH> language specific style guides there.  We have sections that cover
JCH> C, shell and Python, and I do not think adding Perl would not hurt.

The following is how I have interpreted the Perl guidelines.  I hope
it's OK to include Emacs-specific settings; they make it much easier to
reindent code to be acceptable.

I will submit as a patch if you think this is reasonable at all.

The org-mode markers around the code are just a suggestion.

For Perl 5 programs:

 - Most of the C guidelines above apply.

 - We try to support Perl 5.8 and later ("use Perl 5.008").

 - use strict and use warnings are strongly preferred.

 - As in C (see above), we avoid using braces unnecessarily (but Perl
   forces braces around if/unless/else/foreach blocks, so this is not
   always possible).

 - Don't abuse statement modifiers (unless $youmust).

 - We try to avoid assignments inside if().

 - Learn and use Git.pm if you need that functionality.

 - For Emacs, it's useful to put the following in
   GIT_CHECKOUT/.dir-locals.el, assuming you use cperl-mode:

#+begin_src lisp
((nil . ((indent-tabs-mode . t)
              (tab-width . 8)
              (fill-column . 80)))
 (cperl-mode . ((cperl-indent-level . 8)
                (cperl-extra-newline-before-brace . nil)
                (cperl-merge-trailing-else . t))))
#+end_src

Re: CodingGuidelines Perl amendment

[Junio C Hamano]

Ted Zlatanov <tzz@lifelogs.com> writes:

>  - As in C (see above), we avoid using braces unnecessarily (but Perl
>    forces braces around if/unless/else/foreach blocks, so this is not
>    always possible).

Is it ever (as opposed to "not always") possible to omit braces?

It sounds as if we encourage the use of statement modifiers, which
certainly is not what I want to see.

You probably would want to mention that opening braces for
"if/else/elsif" do not sit on their own line, and closing braces for
them will be followed the next "else/elseif" on the same line
instead, but that is part of "most of the C guidelines above apply"
so it may be redundant.

>  - Don't abuse statement modifiers (unless $youmust).

It does not make a useful guidance to leave $youmust part
unspecified.

Incidentally, your sentence is a good example of where use of
statement modifiers is appropriate: $youmust is rarely true.

In general:

	... do something ...
	do_this() unless (condition);
        ... do something else ...

is easier to follow the flow of the logic than

	... do something ...
	unless (condition) {
		do_this();
	}
        ... do something else ...

*only* when condition is extremely rare, iow, when do_this() is
expected to be almost always called.

Re: CodingGuidelines Perl amendment

[demerphq]

On 6 February 2013 17:29, Junio C Hamano <gitster@pobox.com> wrote:
> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>>  - As in C (see above), we avoid using braces unnecessarily (but Perl
>>    forces braces around if/unless/else/foreach blocks, so this is not
>>    always possible).
>
> Is it ever (as opposed to "not always") possible to omit braces?

Only in a statement modifier.

> It sounds as if we encourage the use of statement modifiers, which
> certainly is not what I want to see.

As you mention below statement modifiers have their place. For instance

  next if $whatever;

Is considered preferable to

if ($whatever) {
  next;
}

Similarly

open my $fh, ">", $filename
   or die "Failed to open '$filename': $!";

Is considered preferable by most Perl programmers to:

my $fh;
if ( not open $fh, ">", $filename ) {
  die "Failed to open '$filename': $!";
}

> You probably would want to mention that opening braces for
> "if/else/elsif" do not sit on their own line,
> and closing braces for
> them will be followed the next "else/elseif" on the same line
> instead, but that is part of "most of the C guidelines above apply"
> so it may be redundant.
>
>>  - Don't abuse statement modifiers (unless $youmust).
>
> It does not make a useful guidance to leave $youmust part
> unspecified.
>
> Incidentally, your sentence is a good example of where use of
> statement modifiers is appropriate: $youmust is rarely true.

"unless" often leads to maintenance errors as the expression gets more
complicated over time, more branches need to be added to the
statement, etc. Basically people are bad at doing De Morgans law in
their head.

> In general:
>
>         ... do something ...
>         do_this() unless (condition);
>         ... do something else ...
>
> is easier to follow the flow of the logic than
>
>         ... do something ...
>         unless (condition) {
>                 do_this();
>         }
>         ... do something else ...
>
> *only* when condition is extremely rare, iow, when do_this() is
> expected to be almost always called.

if (not $condition) {
  do_this();
}

Is much less error prone in terms of maintenance than

unless ($condition) {
  do_this();
}

Similarly

do_this() if not $condition;

leads to less maintenance errors than

do_this() unless $condition;

So if you objective is maintainability I would just ban "unless" outright.

Cheers,
Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: CodingGuidelines Perl amendment

[Ted Zlatanov]

On Wed, 6 Feb 2013 18:45:56 +0100 demerphq <demerphq@gmail.com> wrote: 

d> So if you objective is maintainability I would just ban "unless" outright.

Please consider me opposed to such a ban.

Ted

Re: CodingGuidelines Perl amendment

[Junio C Hamano]

demerphq <demerphq@gmail.com> writes:

> As you mention below statement modifiers have their place. For instance
>
>   next if $whatever;
>
> Is considered preferable to
>
> if ($whatever) {
>   next;
> }
>
> Similarly
>
> open my $fh, ">", $filename
>    or die "Failed to open '$filename': $!";
>
> Is considered preferable by most Perl programmers to:
>
> my $fh;
> if ( not open $fh, ">", $filename ) {
>   die "Failed to open '$filename': $!";
> }

Yeah, and that is for the same reason.  When you are trying to get a
birds-eye view of the codeflow, the former makes it clear that "we
do something, and then we open, and then we ...", without letting
the error handling (which also is rare case) distract us.

> "unless" often leads to maintenance errors as the expression gets more
> complicated over time,...

That might also be true, but my comment was not an endorsement for
(or suggestion against) use of unless.  I was commenting on
statement modifiers, which some people tend to overuse (or abuse)
and make the resulting code harder to follow.

Re: CodingGuidelines Perl amendment

[demerphq]

On 6 February 2013 19:14, Junio C Hamano <gitster@pobox.com> wrote:
> demerphq <demerphq@gmail.com> writes:
>
>> As you mention below statement modifiers have their place. For instance
>>
>>   next if $whatever;
>>
>> Is considered preferable to
>>
>> if ($whatever) {
>>   next;
>> }
>>
>> Similarly
>>
>> open my $fh, ">", $filename
>>    or die "Failed to open '$filename': $!";
>>
>> Is considered preferable by most Perl programmers to:
>>
>> my $fh;
>> if ( not open $fh, ">", $filename ) {
>>   die "Failed to open '$filename': $!";
>> }
>
> Yeah, and that is for the same reason.  When you are trying to get a
> birds-eye view of the codeflow, the former makes it clear that "we
> do something, and then we open, and then we ...", without letting
> the error handling (which also is rare case) distract us.

perldoc perlstyle has language which explains this well if you want to
crib a description from somewhere.

>> "unless" often leads to maintenance errors as the expression gets more
>> complicated over time,...
>
> That might also be true, but my comment was not an endorsement for
> (or suggestion against) use of unless.  I was commenting on
> statement modifiers, which some people tend to overuse (or abuse)
> and make the resulting code harder to follow.

That's also my point about unless. They tend to get abused and then
lead to maint devs making errors, and people misunderstanding the
code. The only time that unless IMO is "ok" (ish) is when it really is
a very simple statement. As soon as it mentions more than one var it
should be converted to an if. This applies even more so to the
modifier form.

Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

[PATCH] Update CodingGuidelines for Perl 5

[Ted Zlatanov]

Update the coding guidelines for Perl 5.

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 Documentation/CodingGuidelines |   44 ++++++++++++++++++++++++++++++++++++++++
 1 files changed, 44 insertions(+), 0 deletions(-)

diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 1d7de5f..951d74c 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -179,6 +179,50 @@ For C programs:
  - Use Git's gettext wrappers to make the user interface
    translatable. See "Marking strings for translation" in po/README.
 
+For Perl 5 programs:
+
+ - Most of the C guidelines above apply.
+
+ - We try to support Perl 5.8 and later ("use Perl 5.008").
+
+ - use strict and use warnings are strongly preferred.
+
+ - As in C (see above), we avoid using braces unnecessarily (but Perl forces
+   braces around if/unless/else/foreach blocks, so this is not always possible).
+   At least make sure braces do not sit on their own line, like with C.
+
+ - Don't abuse statement modifiers--they are discouraged.  But in general:
+
+	... do something ...
+	do_this() unless (condition);
+        ... do something else ...
+
+   should be used instead of
+
+	... do something ...
+	unless (condition) {
+		do_this();
+	}
+        ... do something else ...
+
+   *only* when when the condition is so rare that do_this() will be called
+   almost always.
+
+ - We try to avoid assignments inside if().
+
+ - Learn and use Git.pm if you need that functionality.
+
+ - For Emacs, it's useful to put the following in
+   GIT_CHECKOUT/.dir-locals.el, assuming you use cperl-mode:
+
+    ;; note the first part is useful for C editing, too
+    ((nil . ((indent-tabs-mode . t)
+                  (tab-width . 8)
+                  (fill-column . 80)))
+     (cperl-mode . ((cperl-indent-level . 8)
+                    (cperl-extra-newline-before-brace . nil)
+                    (cperl-merge-trailing-else . t))))
+
 Writing Documentation:
 
  Every user-visible change should be reflected in the documentation.
-- 
1.7.9.rc2

Re: CodingGuidelines Perl amendment

[Ted Zlatanov]

On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> Is it ever (as opposed to "not always") possible to omit braces?

Oh yes!  Not that I recommend it, and I'm not even going to touch on
Perl Golf :)

JCH> It sounds as if we encourage the use of statement modifiers, which
JCH> certainly is not what I want to see.

Yup.  I think I captured that in the patch, but please feel free to
revise it after applying or throw it back to me.

JCH> You probably would want to mention that opening braces for
JCH> "if/else/elsif" do not sit on their own line, and closing braces for
JCH> them will be followed the next "else/elseif" on the same line
JCH> instead, but that is part of "most of the C guidelines above apply"
JCH> so it may be redundant.

OK; done.

>> - Don't abuse statement modifiers (unless $youmust).

JCH> It does not make a useful guidance to leave $youmust part
JCH> unspecified.

JCH> Incidentally, your sentence is a good example of where use of
JCH> statement modifiers is appropriate: $youmust is rarely true.

I was trying to be funny, honestly.  But OK; reworded.

Ted

Re: CodingGuidelines Perl amendment

[Junio C Hamano]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote: 
>
> JCH> Is it ever (as opposed to "not always") possible to omit braces?
>
> Oh yes!  Not that I recommend it, and I'm not even going to touch on
> Perl Golf :)
>
> JCH> It sounds as if we encourage the use of statement modifiers, which
> JCH> certainly is not what I want to see.
>
> Yup.  I think I captured that in the patch, but please feel free to
> revise it after applying or throw it back to me.

I'd suggest to just drop that "try to write without braces" entirely.

> JCH> Incidentally, your sentence is a good example of where use of
> JCH> statement modifiers is appropriate: $youmust is rarely true.
>
> I was trying to be funny, honestly.  But OK; reworded.

It wasn't a useful guidance, but it _was_ funny.  

Re: CodingGuidelines Perl amendment

[Ted Zlatanov]

On Wed, 06 Feb 2013 10:16:21 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> I'd suggest to just drop that "try to write without braces" entirely.

OK, I'll do it on the reroll, or you can just make the change directly.

I agree it was not going anywhere :)

Ted

diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 951d74c..857f4e2 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -187,10 +187,6 @@ For Perl 5 programs:
 
  - use strict and use warnings are strongly preferred.
 
- - As in C (see above), we avoid using braces unnecessarily (but Perl forces
-   braces around if/unless/else/foreach blocks, so this is not always possible).
-   At least make sure braces do not sit on their own line, like with C.
-
  - Don't abuse statement modifiers--they are discouraged.  But in general:
 
        ... do something ...

Re: CodingGuidelines Perl amendment

[demerphq]

On 6 February 2013 19:05, Ted Zlatanov <tzz@lifelogs.com> wrote:
> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote:
>
> JCH> Is it ever (as opposed to "not always") possible to omit braces?
>
> Oh yes!  Not that I recommend it, and I'm not even going to touch on
> Perl Golf :)

I think you are wrong. Can you provide an example?

Larry specifically wanted to avoid the "dangling else" problem that C
suffers from, and made it so that blocks are mandatory. The only
exception is statement modifiers, which are not only allowed to omit
the braces but also the parens on the condition.

Yves


-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: CodingGuidelines Perl amendment

[Ted Zlatanov]

On Wed, 6 Feb 2013 19:25:43 +0100 demerphq <demerphq@gmail.com> wrote: 

d> On 6 February 2013 19:05, Ted Zlatanov <tzz@lifelogs.com> wrote:
>> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote:
>> 
JCH> Is it ever (as opposed to "not always") possible to omit braces?
>> 
>> Oh yes!  Not that I recommend it, and I'm not even going to touch on
>> Perl Golf :)

d> I think you are wrong. Can you provide an example?

d> Larry specifically wanted to avoid the "dangling else" problem that C
d> suffers from, and made it so that blocks are mandatory. The only
d> exception is statement modifiers, which are not only allowed to omit
d> the braces but also the parens on the condition.

Oh, perhaps I didn't state it correctly.  You can avoid braces, but not
if you want to use if/elsif/else/unless/etc. which require them:

condition && do_this();
condition || do_this();
condition ? do_this() : do_that();

(and others I can't recall right now)

But my point was only that it's always possible to get around these
artificial restrictions; it's more important to ask for legible sensible
code.  Sorry if that was unclear!

Ted

Re: CodingGuidelines Perl amendment

[demerphq]

On 6 February 2013 19:35, Ted Zlatanov <tzz@lifelogs.com> wrote:
> On Wed, 6 Feb 2013 19:25:43 +0100 demerphq <demerphq@gmail.com> wrote:
>
> d> On 6 February 2013 19:05, Ted Zlatanov <tzz@lifelogs.com> wrote:
>>> On Wed, 06 Feb 2013 08:29:30 -0800 Junio C Hamano <gitster@pobox.com> wrote:
>>>
> JCH> Is it ever (as opposed to "not always") possible to omit braces?
>>>
>>> Oh yes!  Not that I recommend it, and I'm not even going to touch on
>>> Perl Golf :)
>
> d> I think you are wrong. Can you provide an example?
>
> d> Larry specifically wanted to avoid the "dangling else" problem that C
> d> suffers from, and made it so that blocks are mandatory. The only
> d> exception is statement modifiers, which are not only allowed to omit
> d> the braces but also the parens on the condition.
>
> Oh, perhaps I didn't state it correctly.  You can avoid braces, but not
> if you want to use if/elsif/else/unless/etc. which require them:
>
> condition && do_this();
> condition || do_this();
> condition ? do_this() : do_that();
>
> (and others I can't recall right now)
>
> But my point was only that it's always possible to get around these
> artificial restrictions; it's more important to ask for legible sensible
> code.  Sorry if that was unclear!

Ah ok. Right, at a low level:

if (condition) { do_this() }

is identical to

condition && do_this();

IOW, Perl allows logical operators to act as control flow statements.

I hope your document include something that says that using logical
operators as control flow statements should be used sparingly, and
generally should be restricted to low precedence operators and should
never involve more than one operator.

Yves




-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: CodingGuidelines Perl amendment

[Ted Zlatanov]

On Wed, 6 Feb 2013 19:44:16 +0100 demerphq <demerphq@gmail.com> wrote: 

d> Ah ok. Right, at a low level:

d> if (condition) { do_this() }

d> is identical to

d> condition && do_this();

d> IOW, Perl allows logical operators to act as control flow statements.

d> I hope your document include something that says that using logical
d> operators as control flow statements should be used sparingly, and
d> generally should be restricted to low precedence operators and should
d> never involve more than one operator.

I'd stay away from wording it so tightly, but instead just say

"Make your code readable and sensible, and don't try to be clever."

But this is good C and shell advice too, so I'd put it under "General
Guidelines" and leave it for Junio to decide if it's appropriate.

Ted

Re: CodingGuidelines Perl amendment

[Junio C Hamano]

Ted Zlatanov <tzz@lifelogs.com> writes:

> "Make your code readable and sensible, and don't try to be clever."
>
> But this is good C and shell advice too,...

Sounds sensible.

[PATCH v2] Update CodingGuidelines for Perl 5

[Ted Zlatanov]

Update the coding guidelines for Perl 5.

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
Changes since PATCHv1:
- removed brace guidelines
- add "don't try to be clever" at beginning

 Documentation/CodingGuidelines |   42 ++++++++++++++++++++++++++++++++++++++++
 1 files changed, 42 insertions(+), 0 deletions(-)

diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index 1d7de5f..166c141 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -18,6 +18,8 @@ code.  For Git in general, three rough rules are:
    judgement call, the decision based more on real world
    constraints people face than what the paper standard says.
 
+For any programming language below, make your code readable and sensible, and
+don't try to be clever.
 
 As for more concrete guidelines, just imitate the existing code
 (this is a good guideline, no matter which project you are
@@ -179,6 +181,46 @@ For C programs:
  - Use Git's gettext wrappers to make the user interface
    translatable. See "Marking strings for translation" in po/README.
 
+For Perl 5 programs:
+
+ - Most of the C guidelines above apply.
+
+ - We try to support Perl 5.8 and later ("use Perl 5.008").
+
+ - use strict and use warnings are strongly preferred.
+
+ - Don't abuse statement modifiers--they are discouraged.  But in general:
+
+	... do something ...
+	do_this() unless (condition);
+        ... do something else ...
+
+   should be used instead of
+
+	... do something ...
+	unless (condition) {
+		do_this();
+	}
+        ... do something else ...
+
+   *only* when when the condition is so rare that do_this() will be called
+   almost always.
+
+ - We try to avoid assignments inside if().
+
+ - Learn and use Git.pm if you need that functionality.
+
+ - For Emacs, it's useful to put the following in
+   GIT_CHECKOUT/.dir-locals.el, assuming you use cperl-mode:
+
+    ;; note the first part is useful for C editing, too
+    ((nil . ((indent-tabs-mode . t)
+                  (tab-width . 8)
+                  (fill-column . 80)))
+     (cperl-mode . ((cperl-indent-level . 8)
+                    (cperl-extra-newline-before-brace . nil)
+                    (cperl-merge-trailing-else . t))))
+
 Writing Documentation:
 
  Every user-visible change should be reflected in the documentation.
-- 
1.7.9.rc2

[PATCH 2/3] Skip blank and commented lines in contrib/credentials/netrc

[Ted Zlatanov]

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 contrib/credential/netrc/git-credential-netrc |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
index 92fc306..a47a223 100755
--- a/contrib/credential/netrc/git-credential-netrc
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -192,6 +192,9 @@ LINE: foreach my $line (@data)
   push(@tok, $tok);
  }
 
+ # skip blank lines, comments, etc.
+ next LINE unless scalar @tok;
+
  my %tokens;
  while (@tok)
  {
-- 
1.7.9.rc2

[PATCH 3/3] Fix contrib/credentials/netrc minor issues: exit quietly; use 3-parameter open; etc.

[Ted Zlatanov]

Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
---
 contrib/credential/netrc/git-credential-netrc |   38 +++++++++++++------------
 1 files changed, 20 insertions(+), 18 deletions(-)

diff --git a/contrib/credential/netrc/git-credential-netrc b/contrib/credential/netrc/git-credential-netrc
index a47a223..0e35918 100755
--- a/contrib/credential/netrc/git-credential-netrc
+++ b/contrib/credential/netrc/git-credential-netrc
@@ -3,8 +3,6 @@
 use strict;
 use warnings;
 
-use Data::Dumper;
-
 use Getopt::Long;
 use File::Basename;
 
@@ -58,7 +56,7 @@ if ($options{help})
 
 $0 [-f AUTHFILE] [-d] get
 
-Version $VERSION by tzz\@lifelogs.com.  License: any use is OK.
+Version $VERSION by tzz\@lifelogs.com.  License: BSD.
 
 Options:
   -f AUTHFILE: specify a netrc-style file
@@ -129,31 +127,36 @@ my $file = $options{file};
 die "Sorry, you need to specify an existing netrc file (with or without a .gpg extension) with -f AUTHFILE"
  unless defined $file;
 
-die "Sorry, the specified netrc $file is not accessible"
- unless -f $file;
+unless (-f $file)
+{
+ print STDERR "Sorry, the specified netrc $file is not accessible\n" if $debug;
+ exit 0;
+}
 
+my @data;
 if ($file =~ m/\.gpg$/)
 {
- $file = "gpg --decrypt $file|";
+ @data = load('-|', qw(gpg --decrypt), $file)
+}
+else
+{
+ @data = load('<', $file);
 }
 
-my @data = load($file);
 chomp @data;
 
-die "Sorry, we could not load data from [$file]"
- unless (scalar @data);
-
-# the query
-my %q;
-
-foreach my $v (values %{$options{tmap}})
+unless (scalar @data)
 {
- undef $q{$v};
+ print STDERR "Sorry, we could not load data from [$file]\n" if $debug;
+ exit;
 }
 
+# the query: start with every token with no value
+my %q = map { $_ => undef } values(%{$options{tmap}});
+
 while (<STDIN>)
 {
- next unless m/([a-z]+)=(.+)/;
+ next unless m/([^=]+)=(.+)/;
 
  my ($token, $value) = ($1, $2);
  die "Unknown search token $1" unless exists $q{$token};
@@ -237,9 +240,8 @@ LINE: foreach my $line (@data)
 
 sub load
 {
- my $file = shift;
  # this supports pipes too
- my $io = new IO::File($file) or die "Could not open $file: $!\n";
+ my $io = new IO::File(@_) or die "Could not open [@_]: $!\n";
 
  return <$io>;                          # whole file
 }
-- 
1.7.9.rc2

Re: [PATCH 3/3] Fix contrib/credentials/netrc minor issues: exit quietly; use 3-parameter open; etc.

[Junio C Hamano]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
> ---
>  contrib/credential/netrc/git-credential-netrc |   38 +++++++++++++------------
>  1 files changed, 20 insertions(+), 18 deletions(-)

Especially because this is an initial submission, please equash
three patches into one, instead of sending three "here is my first
attempt with many problems I know I do not want to be there", "one
small improvement", "another one to fix remaining issues".

Otherwise you will waste reviewers' time, getting distracted by
undesirable details they find in an earlier patch while reviewing,
without realizing that some of them are fixed in a later one.

Thanks.

Re: [PATCH 3/3] Fix contrib/credentials/netrc minor issues: exit quietly; use 3-parameter open; etc.

[Ted Zlatanov]

On Mon, 04 Feb 2013 09:27:46 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Signed-off-by: Ted Zlatanov <tzz@lifelogs.com>
>> ---
>> contrib/credential/netrc/git-credential-netrc |   38 +++++++++++++------------
>> 1 files changed, 20 insertions(+), 18 deletions(-)

JCH> Especially because this is an initial submission, please equash
JCH> three patches into one, instead of sending three "here is my first
JCH> attempt with many problems I know I do not want to be there", "one
JCH> small improvement", "another one to fix remaining issues".

JCH> Otherwise you will waste reviewers' time, getting distracted by
JCH> undesirable details they find in an earlier patch while reviewing,
JCH> without realizing that some of them are fixed in a later one.

OK, thanks.  I wasn't sure, since Jeff already reviewed it, if it was
better to squash or not.  Ignore this same question in my other reply to
you, and thanks for your patience.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Michal Nazarewicz]

[-- Attachment #1: Type: text/plain, Size: 1271 bytes --]

On Wed, Jan 30 2013, Jeff King wrote:
> I do not mind a .netrc or .authinfo parser, because while those formats
> do have security problems, they are standard files that may already be
> in use. So as long as we are not encouraging their use, I do not see a
> problem in supporting them (and we already do the same with curl's netrc
> support).
>
> But it would probably make sense for send-email to support the existing
> git-credential subsystem, so that it can take advantage of secure
> system-specific storage. And that is where we should be pointing new
> users. I think contrib/mw-to-git even has credential support written in
> perl, so it would just need to be factored out to Git.pm.

As far as I understand, there could be a git-credential helper that
reads ~/.authinfo and than git-send-email would just call “git
credential fill”, right?

I've noticed though, that git-credential does not support port argument,
which makes it slightly incompatible with ~/.authinfo.

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +----<email/xmpp: mpn@google.com>--------------ooO--(_)--Ooo--

[-- Attachment #2.1: Type: text/plain, Size: 0 bytes --]



[-- Attachment #2.2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Mon, 04 Feb 2013 17:33:58 +0100 Michal Nazarewicz <mina86@mina86.com> wrote: 

MN> As far as I understand, there could be a git-credential helper that
MN> reads ~/.authinfo and than git-send-email would just call “git
MN> credential fill”, right?

MN> I've noticed though, that git-credential does not support port argument,
MN> which makes it slightly incompatible with ~/.authinfo.

My proposed netrc credential helper does this :)

The token mapping I use:

port, protocol        => protocol
machine, host         => host
path                  => path
login, username, user => username
password              => password

I think that's sensible.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Jeff King]

On Mon, Feb 04, 2013 at 12:00:34PM -0500, Ted Zlatanov wrote:

> On Mon, 04 Feb 2013 17:33:58 +0100 Michal Nazarewicz <mina86@mina86.com> wrote: 
> 
> MN> As far as I understand, there could be a git-credential helper that
> MN> reads ~/.authinfo and than git-send-email would just call “git
> MN> credential fill”, right?
> 
> MN> I've noticed though, that git-credential does not support port argument,
> MN> which makes it slightly incompatible with ~/.authinfo.
> 
> My proposed netrc credential helper does this :)
> 
> The token mapping I use:
> 
> port, protocol        => protocol
> machine, host         => host
> path                  => path
> login, username, user => username
> password              => password
> 
> I think that's sensible.

Technically you can speak a particular protocol on an alternate port:

  https://example.com:31337/repo.git

In this case, git will send you the host as:

  example.com:31337

You might want to map this to "port" in .autoinfo separately if it's
available.

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Mon, 4 Feb 2013 15:10:40 -0500 Jeff King <peff@peff.net> wrote: 

JK> Technically you can speak a particular protocol on an alternate port:

JK>   https://example.com:31337/repo.git

JK> In this case, git will send you the host as:

JK>   example.com:31337

JK> You might want to map this to "port" in .autoinfo separately if it's
JK> available.

That would create the following possibilities:

* host example.com:31337, protocol https
* host example.com:31337, protocol unspecified
* host example.com, protocol https
* host example.com, protocol unspecified

How would you like each one to be handled?  My preference would be to
make the user say "host example.com:31337" in the netrc file (the
current situation); that's what we do in Emacs and it lets applications
request credentials for a logical service no matter what the port is.

It means that example.com credentials won't be used for
example.com:31337.  In practice, that has not been a problem for us.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Jeff King]

On Mon, Feb 04, 2013 at 03:28:52PM -0500, Ted Zlatanov wrote:

> JK> You might want to map this to "port" in .autoinfo separately if it's
> JK> available.
> 
> That would create the following possibilities:
> 
> * host example.com:31337, protocol https
> * host example.com:31337, protocol unspecified
> * host example.com, protocol https
> * host example.com, protocol unspecified

Possibilities for .netrc, or for git? Git will always specify the
protocol.

> How would you like each one to be handled?  My preference would be to
> make the user say "host example.com:31337" in the netrc file (the
> current situation); that's what we do in Emacs and it lets applications
> request credentials for a logical service no matter what the port is.
> 
> It means that example.com credentials won't be used for
> example.com:31337.  In practice, that has not been a problem for us.

Yeah, I think that is a good thing. The credentials used for
example.com:31337 are not necessarily the same as for the main site.
It's less convenient, but a more secure default.

What I was more wondering (and I know very little about .netrc, so this
might not be a possibility at all) is a line like:

  host example.com port 5001 protocol https username foo password bar

To match git's representation on a token-by-token basis, you would have
to either split out git's "host:port" pair, or combine the .netrc's
representation to "example.com:5001".

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Mon, 4 Feb 2013 15:59:11 -0500 Jeff King <peff@peff.net> wrote: 

JK> On Mon, Feb 04, 2013 at 03:28:52PM -0500, Ted Zlatanov wrote:
JK> You might want to map this to "port" in .autoinfo separately if it's
JK> available.
>> 
>> That would create the following possibilities:
>> 
>> * host example.com:31337, protocol https
>> * host example.com:31337, protocol unspecified
>> * host example.com, protocol https
>> * host example.com, protocol unspecified

JK> Possibilities for .netrc, or for git? Git will always specify the
JK> protocol.

Possibilities for the netrc data.  How clever do we want to be with
taking 31337 and mapping it to the "protocol"?  My preference is to be
very simple here.

JK> What I was more wondering (and I know very little about .netrc, so this
JK> might not be a possibility at all) is a line like:

JK>   host example.com port 5001 protocol https username foo password bar

JK> To match git's representation on a token-by-token basis, you would have
JK> to either split out git's "host:port" pair, or combine the .netrc's
JK> representation to "example.com:5001".

Currently, we map both the "port" and "protocol" netrc tokens to the
credential helper protocol's "protocol".  So this will have undefined
results.  To do what you specify could be pretty simple: we could do a
preliminary scan of the tokens, looking for "host X port Y" where Y is
an integer, and rewriting the host to be "X:Y".  That would be clean and
simple, unless the user breaks it with "host x:23 port 22".  Let me know
if you agree and I'll do.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Jeff King]

On Mon, Feb 04, 2013 at 04:08:52PM -0500, Ted Zlatanov wrote:

> >> That would create the following possibilities:
> >> 
> >> * host example.com:31337, protocol https
> >> * host example.com:31337, protocol unspecified
> >> * host example.com, protocol https
> >> * host example.com, protocol unspecified
> 
> JK> Possibilities for .netrc, or for git? Git will always specify the
> JK> protocol.
> 
> Possibilities for the netrc data.  How clever do we want to be with
> taking 31337 and mapping it to the "protocol"?  My preference is to be
> very simple here.

I think simple is OK, as we can iterate on it as specific use-cases come
up. The important thing is to make sure we err on the side of "does not
match" and not "oops, we accidentally sent your plaintext credentials to
the wrong server".

> Currently, we map both the "port" and "protocol" netrc tokens to the
> credential helper protocol's "protocol".  So this will have undefined
> results.  To do what you specify could be pretty simple: we could do a
> preliminary scan of the tokens, looking for "host X port Y" where Y is
> an integer, and rewriting the host to be "X:Y".  That would be clean and
> simple, unless the user breaks it with "host x:23 port 22".  Let me know
> if you agree and I'll do.

Yeah, I think that is simple and obvious. If the user is saying "host
x:23 port 22", that is nonsensical.

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Mon, 4 Feb 2013 16:22:03 -0500 Jeff King <peff@peff.net> wrote: 

>> Currently, we map both the "port" and "protocol" netrc tokens to the
>> credential helper protocol's "protocol".  So this will have undefined
>> results.  To do what you specify could be pretty simple: we could do a
>> preliminary scan of the tokens, looking for "host X port Y" where Y is
>> an integer, and rewriting the host to be "X:Y".  That would be clean and
>> simple, unless the user breaks it with "host x:23 port 22".  Let me know
>> if you agree and I'll do.

JK> Yeah, I think that is simple and obvious. If the user is saying "host
JK> x:23 port 22", that is nonsensical.

OK; added.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Tue, Jan 29, 2013 at 11:53:19AM -0800, Junio C Hamano wrote:
>
>> Either way it still encourages a plaintext password to be on disk,
>> which may not be what we want, even though it may be slight if not
>> really much of an improvement.  Again the Help-for-users has this
>> amusing bit:
>
> I do not mind a .netrc or .authinfo parser, because while those formats
> do have security problems, they are standard files that may already be
> in use. So as long as we are not encouraging their use, I do not see a
> problem in supporting them (and we already do the same with curl's netrc
> support).
>
> But it would probably make sense for send-email to support the existing
> git-credential subsystem, so that it can take advantage of secure
> system-specific storage. And that is where we should be pointing new
> users. I think contrib/mw-to-git even has credential support written in
> perl, so it would just need to be factored out to Git.pm.

I see a lot of rerolls on the credential helper front, but is there
anybody working on hooking send-email to the credential framework?

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Matthieu Moy]

Junio C Hamano <gitster@pobox.com> writes:

> I see a lot of rerolls on the credential helper front, but is there
> anybody working on hooking send-email to the credential framework?

Not answering the question, but git-remote-mediawiki supports the
credential framework. It is written in perl, and the credential support
is rather cleanly written and doesn't have dependencies on the wiki
part, so the way to go for send-email is probably to libify the
credential support in git-remote-mediawiki, and to use it in send-email.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Wed, 06 Feb 2013 09:11:17 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 

MM> Junio C Hamano <gitster@pobox.com> writes:
>> I see a lot of rerolls on the credential helper front, but is there
>> anybody working on hooking send-email to the credential framework?

MM> Not answering the question, but git-remote-mediawiki supports the
MM> credential framework. It is written in perl, and the credential support
MM> is rather cleanly written and doesn't have dependencies on the wiki
MM> part, so the way to go for send-email is probably to libify the
MM> credential support in git-remote-mediawiki, and to use it in send-email.

I looked and that's indeed very useful.  If it's put in a library, I'd
use credential_read() and credential_write() in my netrc credential
helper.  But I would formalize it a little more about the token names
and output, and I wouldn't necessarily die() on error.  Maybe this can
be merged with the netrc credential helper's
read_credential_data_from_stdin() and print_credential_data()?

Let me know if you'd like me to libify this...  I'm happy to leave it to
Matthieu or Michal, or anyone else interested.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Matthieu Moy]

Ted Zlatanov <tzz@lifelogs.com> writes:

> MM> [...] so the way to go for send-email is probably to libify the
> MM> credential support in git-remote-mediawiki, and to use it in send-email.
>
> I looked and that's indeed very useful.  If it's put in a library, I'd
> use credential_read() and credential_write() in my netrc credential
> helper.  But I would formalize it a little more about the token names
> and output,

Can you elaborate on this? The idea of the Perl code was to mimick a
call to the C API, keeping essentially the same names.

> and I wouldn't necessarily die() on error. 

Sure, die()ing in a library is bad.

> Maybe this can be merged with the netrc credential helper's
> read_credential_data_from_stdin() and print_credential_data()?

I don't know about the netrc credential helper, but I guess that's
another layer. The git-remote-mediawiki code is the code to call the
credential C API, that in turn may (or may not) call a credential
helper.

> Let me know if you'd like me to libify this...  I'm happy to leave it to
> Matthieu or Michal, or anyone else interested.

I'd happily let you do the job, but I can help if needed. One thing to
be careful about: git-remote-mediawiki is currently a standalone script,
so it can be installed with a plain "cp git-remote-mediawiki $somewhere/".
One consequence of libification is that it adds a dependency on the
library (e.g. Git.pm). We should be carefull to keep it easy for the
user to install it (e.g. some kind of "make install", or update the doc).

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Wed, 06 Feb 2013 16:10:12 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 

MM> Ted Zlatanov <tzz@lifelogs.com> writes:
MM> [...] so the way to go for send-email is probably to libify the
MM> credential support in git-remote-mediawiki, and to use it in send-email.
>> 
>> I looked and that's indeed very useful.  If it's put in a library, I'd
>> use credential_read() and credential_write() in my netrc credential
>> helper.  But I would formalize it a little more about the token names
>> and output,

MM> Can you elaborate on this? The idea of the Perl code was to mimick a
MM> call to the C API, keeping essentially the same names.

None of these are a big deal, and Michal said he's working on libifying
this anyhow:

- making 'fill' a special operation is weird
- anchor the key regex to beginning of line (not strictly necessary)
- sort the output tokens (after 'url' is extracted) so the output is consistent and testable

>> and I wouldn't necessarily die() on error. 

MM> Sure, die()ing in a library is bad.

>> Maybe this can be merged with the netrc credential helper's
>> read_credential_data_from_stdin() and print_credential_data()?

MM> I don't know about the netrc credential helper, but I guess that's
MM> another layer. The git-remote-mediawiki code is the code to call the
MM> credential C API, that in turn may (or may not) call a credential
MM> helper.

Yup.  But what you call "read" and "write" are, to the credential
helper, "write" and "read" but it's the same protocol :)  So maybe the
names should be changed to reflect that, e.g. "query" and "response."

MM> One thing to be careful about: git-remote-mediawiki is currently a
MM> standalone script, so it can be installed with a plain "cp
MM> git-remote-mediawiki $somewhere/".  One consequence of libification
MM> is that it adds a dependency on the library (e.g. Git.pm). We should
MM> be carefull to keep it easy for the user to install it (e.g. some
MM> kind of "make install", or update the doc).

I don't know--it's up to the `git-remote-mediawiki' maintainers...  But
I think anywhere you have Git, you also have Git.pm, right?  Maybe?  But
then you also have to look at whether Git.pm has the functionality you
need... so I better go quiet :)

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Matthieu Moy]

Ted Zlatanov <tzz@lifelogs.com> writes:

> None of these are a big deal, and Michal said he's working on libifying
> this anyhow:
>
> - making 'fill' a special operation is weird

Well, 'fill' is the only operation that mutates the credential structure
(i.e. the only one for which "git credential" emits an output to be
parsed), so you don't have much choice.

> - anchor the key regex to beginning of line (not strictly necessary)

Right. The greedyness of * ensures correction, but I like explicit
anchors ^...$ too.

> - sort the output tokens (after 'url' is extracted) so the output is consistent and testable

Why not, if you want to use the output of credential_write in tests. But
credential_write is essentially used to talk to "git credential", so the
important information is the content of the hash before credential_write
and after credential_read. They are unordered, but consistent and
testable.

>>> Maybe this can be merged with the netrc credential helper's
>>> read_credential_data_from_stdin() and print_credential_data()?
>
> MM> I don't know about the netrc credential helper, but I guess that's
> MM> another layer. The git-remote-mediawiki code is the code to call the
> MM> credential C API, that in turn may (or may not) call a credential
> MM> helper.
>
> Yup.  But what you call "read" and "write" are, to the credential
> helper, "write" and "read" but it's the same protocol :)  So maybe the
> names should be changed to reflect that, e.g. "query" and "response."

I don't think that would be a better naming. Maybe "serialize" and
"parse" would be better, but "query" would sound like it establishes the
connection and possibly reads the response to me.

> MM> One thing to be careful about: git-remote-mediawiki is currently a
> MM> standalone script, so it can be installed with a plain "cp
> MM> git-remote-mediawiki $somewhere/".  One consequence of libification
> MM> is that it adds a dependency on the library (e.g. Git.pm). We should
> MM> be carefull to keep it easy for the user to install it (e.g. some
> MM> kind of "make install", or update the doc).
>
> I don't know--it's up to the `git-remote-mediawiki' maintainers...

That is, me ;-).

> But I think anywhere you have Git, you also have Git.pm, right?

Yes, but you have to find out where it is installed. Git's Makefile
hardcodes the path to Git.pm at build time, inserting one line in the
perl script:

use lib (split(/:/, $ENV{GITPERLLIB} || "$INSTLIBDIR"));

The same needs to be done for git-remote-mediawiki. As much as possible,
I'd rather avoid copy-pasting from Git's Makefile, so this means
extracting the perl part of Git's Makefile and make it available in
contrib/.

I'll try a patch in this direction.

> Maybe? But then you also have to look at whether Git.pm has the
> functionality you need...

If git-remote-mediawiki is installed from Git's source, I think it's OK
to assume that Git.pm will be up to date, but that would be even better
if we can issue a clean error message when the functions to be called do
not exist.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Wed, 06 Feb 2013 17:41:01 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 

MM> Ted Zlatanov <tzz@lifelogs.com> writes:
>> - sort the output tokens (after 'url' is extracted) so the output is consistent and testable

MM> Why not, if you want to use the output of credential_write in tests. But
MM> credential_write is essentially used to talk to "git credential", so the
MM> important information is the content of the hash before credential_write
MM> and after credential_read. They are unordered, but consistent and
MM> testable.

I like testing output (especially when it's part of an API), so we
should make the externally observable output consistent and testable.

The change is tiny, just sort the keys instead of calling each(), so I
hope it makes it in the final version.

>> Yup.  But what you call "read" and "write" are, to the credential
>> helper, "write" and "read" but it's the same protocol :)  So maybe the
>> names should be changed to reflect that, e.g. "query" and "response."

MM> I don't think that would be a better naming. Maybe "serialize" and
MM> "parse" would be better, but "query" would sound like it establishes the
MM> connection and possibly reads the response to me.

I'm OK with anything unambiguous.

Thanks!
Ted

[PATCH 0/4] Allow contrib/ to use Git's Makefile for perl code

[Matthieu Moy]

The very final goal is to be able to move painlessly (credential) code
from git-remote-mediawiki to Git.pm, but then it's nice for the user
to be able to say just "cd contrib/mw-to-git && make install" and let
the Makefile set perl's library path just like other Git commands
written in perl.

This series does this while trying to minimize code duplication, and
to make it easy for future other tools in contrib to do the same.

Matthieu Moy (4):
  Makefile: extract perl-related rules to make them available from other
    dirs
  perl.mak: introduce $(GIT_ROOT_DIR) to allow inclusion from other
    directories
  Makefile: factor common configuration in git-default-config.mak
  git-remote-mediawiki: use Git's Makefile to build the script

 Makefile                                           | 108 +--------------------
 contrib/mw-to-git/.gitignore                       |   1 +
 contrib/mw-to-git/Makefile                         |  45 ++++++---
 ...-remote-mediawiki => git-remote-mediawiki.perl} |   0
 default-config.mak                                 |  61 ++++++++++++
 perl.mak                                           |  52 ++++++++++
 6 files changed, 145 insertions(+), 122 deletions(-)
 create mode 100644 contrib/mw-to-git/.gitignore
 rename contrib/mw-to-git/{git-remote-mediawiki => git-remote-mediawiki.perl} (100%)
 create mode 100644 default-config.mak
 create mode 100644 perl.mak

-- 
1.8.1.2.526.gf51a757

[PATCH 1/4] Makefile: extract perl-related rules to make them available from other dirs

[Matthieu Moy]

The final goal is to make it easy to write Git commands in perl in the
contrib/ directory. It is currently possible to do so, but without the
benefits of Git's Makefile: adapt first line with $(PERL_PATH),
hardcode the path to Git.pm, ...

We make the perl-related part of the Makefile available from directories
other than the toplevel so that:

* Developers can include it, to avoid code duplication

* Users can get a consistent behavior of "make install"

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 Makefile | 46 +---------------------------------------------
 perl.mak | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 45 deletions(-)
 create mode 100644 perl.mak

diff --git a/Makefile b/Makefile
index 731b6a8..f39d4a9 100644
--- a/Makefile
+++ b/Makefile
@@ -573,14 +573,10 @@ BINDIR_PROGRAMS_NO_X += git-cvsserver
 ifndef SHELL_PATH
 	SHELL_PATH = /bin/sh
 endif
-ifndef PERL_PATH
-	PERL_PATH = /usr/bin/perl
-endif
 ifndef PYTHON_PATH
 	PYTHON_PATH = /usr/bin/python
 endif
 
-export PERL_PATH
 export PYTHON_PATH
 
 LIB_FILE = libgit.a
@@ -1441,10 +1437,6 @@ ifeq ($(TCLTK_PATH),)
 NO_TCLTK = NoThanks
 endif
 
-ifeq ($(PERL_PATH),)
-NO_PERL = NoThanks
-endif
-
 ifeq ($(PYTHON_PATH),)
 NO_PYTHON = NoThanks
 endif
@@ -1522,7 +1514,6 @@ prefix_SQ = $(subst ','\'',$(prefix))
 gitwebdir_SQ = $(subst ','\'',$(gitwebdir))
 
 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
-PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
 PYTHON_PATH_SQ = $(subst ','\'',$(PYTHON_PATH))
 TCLTK_PATH_SQ = $(subst ','\'',$(TCLTK_PATH))
 DIFF_SQ = $(subst ','\'',$(DIFF))
@@ -1715,9 +1706,6 @@ $(SCRIPT_LIB) : % : %.sh GIT-SCRIPT-DEFINES
 	$(QUIET_GEN)$(cmd_munge_script) && \
 	mv $@+ $@
 
-ifndef NO_PERL
-$(patsubst %.perl,%,$(SCRIPT_PERL)): perl/perl.mak
-
 perl/perl.mak: perl/PM.stamp
 
 perl/PM.stamp: FORCE
@@ -1728,39 +1716,7 @@ perl/PM.stamp: FORCE
 perl/perl.mak: GIT-CFLAGS GIT-PREFIX perl/Makefile perl/Makefile.PL
 	$(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' $(@F)
 
-$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl GIT-VERSION-FILE
-	$(QUIET_GEN)$(RM) $@ $@+ && \
-	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
-	sed -e '1{' \
-	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
-	    -e '	h' \
-	    -e '	s=.*=use lib (split(/$(pathsep)/, $$ENV{GITPERLLIB} || "'"$$INSTLIBDIR"'"));=' \
-	    -e '	H' \
-	    -e '	x' \
-	    -e '}' \
-	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
-	    $@.perl >$@+ && \
-	chmod +x $@+ && \
-	mv $@+ $@
-
-
-.PHONY: gitweb
-gitweb:
-	$(QUIET_SUBDIR0)gitweb $(QUIET_SUBDIR1) all
-
-git-instaweb: git-instaweb.sh gitweb GIT-SCRIPT-DEFINES
-	$(QUIET_GEN)$(cmd_munge_script) && \
-	chmod +x $@+ && \
-	mv $@+ $@
-else # NO_PERL
-$(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh
-	$(QUIET_GEN)$(RM) $@ $@+ && \
-	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
-	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
-	    unimplemented.sh >$@+ && \
-	chmod +x $@+ && \
-	mv $@+ $@
-endif # NO_PERL
+include perl.mak
 
 ifndef NO_PYTHON
 $(patsubst %.py,%,$(SCRIPT_PYTHON)): GIT-CFLAGS GIT-PREFIX GIT-PYTHON-VARS
diff --git a/perl.mak b/perl.mak
new file mode 100644
index 0000000..8bbeef3
--- /dev/null
+++ b/perl.mak
@@ -0,0 +1,49 @@
+# Rules to build Git commands written in perl
+
+ifndef PERL_PATH
+	PERL_PATH = /usr/bin/perl
+endif
+export PERL_PATH
+PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
+
+ifeq ($(PERL_PATH),)
+NO_PERL = NoThanks
+endif
+
+ifndef NO_PERL
+$(patsubst %.perl,%,$(SCRIPT_PERL)): perl/perl.mak
+
+
+$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl GIT-VERSION-FILE
+	$(QUIET_GEN)$(RM) $@ $@+ && \
+	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
+	sed -e '1{' \
+	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
+	    -e '	h' \
+	    -e '	s=.*=use lib (split(/$(pathsep)/, $$ENV{GITPERLLIB} || "'"$$INSTLIBDIR"'"));=' \
+	    -e '	H' \
+	    -e '	x' \
+	    -e '}' \
+	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
+	    $@.perl >$@+ && \
+	chmod +x $@+ && \
+	mv $@+ $@
+
+
+.PHONY: gitweb
+gitweb:
+	$(QUIET_SUBDIR0)gitweb $(QUIET_SUBDIR1) all
+
+git-instaweb: git-instaweb.sh gitweb GIT-SCRIPT-DEFINES
+	$(QUIET_GEN)$(cmd_munge_script) && \
+	chmod +x $@+ && \
+	mv $@+ $@
+else # NO_PERL
+$(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh
+	$(QUIET_GEN)$(RM) $@ $@+ && \
+	sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
+	    -e 's|@@REASON@@|NO_PERL=$(NO_PERL)|g' \
+	    unimplemented.sh >$@+ && \
+	chmod +x $@+ && \
+	mv $@+ $@
+endif # NO_PERL
-- 
1.8.1.2.526.gf51a757

Re: [PATCH 1/4] Makefile: extract perl-related rules to make them available from other dirs

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> The final goal is to make it easy to write Git commands in perl in the
> contrib/ directory. It is currently possible to do so, but without the
> benefits of Git's Makefile: adapt first line with $(PERL_PATH),
> hardcode the path to Git.pm, ...
>
> We make the perl-related part of the Makefile available from directories
> other than the toplevel so that:
>
> * Developers can include it, to avoid code duplication
>
> * Users can get a consistent behavior of "make install"
>
> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>

The goal may be worthy, but the split does not look quite right.

What business do contrib/ scripts have knowing how gitweb and
git-instaweb are built and what they depend on, for example?

[PATCH 2/4] perl.mak: introduce $(GIT_ROOT_DIR) to allow inclusion from other directories

[Matthieu Moy]

perl.mak uses relative path, which is OK when called from the toplevel,
but won't be anymore if one includes it from elsewhere. It is now
possible to include the file using:

GIT_ROOT_DIR=<whatever>
include $(GIT_ROOT_DIR)/perl.mak

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 perl.mak | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/perl.mak b/perl.mak
index 8bbeef3..a2b8717 100644
--- a/perl.mak
+++ b/perl.mak
@@ -1,5 +1,9 @@
 # Rules to build Git commands written in perl
 
+ifndef GIT_ROOT_DIR
+	GIT_ROOT_DIR = .
+endif
+
 ifndef PERL_PATH
 	PERL_PATH = /usr/bin/perl
 endif
@@ -11,12 +15,11 @@ NO_PERL = NoThanks
 endif
 
 ifndef NO_PERL
-$(patsubst %.perl,%,$(SCRIPT_PERL)): perl/perl.mak
-
+$(patsubst %.perl,%,$(SCRIPT_PERL)): $(GIT_ROOT_DIR)/perl/perl.mak
 
-$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl GIT-VERSION-FILE
+$(patsubst %.perl,%,$(SCRIPT_PERL)): % : %.perl $(GIT_ROOT_DIR)/GIT-VERSION-FILE
 	$(QUIET_GEN)$(RM) $@ $@+ && \
-	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
+	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C $(GIT_ROOT_DIR)/perl -s --no-print-directory instlibdir` && \
 	sed -e '1{' \
 	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
 	    -e '	h' \
-- 
1.8.1.2.526.gf51a757

[PATCH 3/4] Makefile: factor common configuration in git-default-config.mak

[Matthieu Moy]

Similarly to the extraction of perl-related code in perl.mak, we extract
general default configuration from the Makefile to make it available from
directories other than the toplevel.

This is required to make perl.mak usable because it requires $(pathsep)
to be set.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 Makefile           | 62 +-----------------------------------------------------
 default-config.mak | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+), 61 deletions(-)
 create mode 100644 default-config.mak

diff --git a/Makefile b/Makefile
index f39d4a9..9649a41 100644
--- a/Makefile
+++ b/Makefile
@@ -346,67 +346,7 @@ GIT-VERSION-FILE: FORCE
 	@$(SHELL_PATH) ./GIT-VERSION-GEN
 -include GIT-VERSION-FILE
 
-# CFLAGS and LDFLAGS are for the users to override from the command line.
-
-CFLAGS = -g -O2 -Wall
-LDFLAGS =
-ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS)
-ALL_LDFLAGS = $(LDFLAGS)
-STRIP ?= strip
-
-# Among the variables below, these:
-#   gitexecdir
-#   template_dir
-#   mandir
-#   infodir
-#   htmldir
-#   sysconfdir
-# can be specified as a relative path some/where/else;
-# this is interpreted as relative to $(prefix) and "git" at
-# runtime figures out where they are based on the path to the executable.
-# This can help installing the suite in a relocatable way.
-
-prefix = $(HOME)
-bindir_relative = bin
-bindir = $(prefix)/$(bindir_relative)
-mandir = share/man
-infodir = share/info
-gitexecdir = libexec/git-core
-mergetoolsdir = $(gitexecdir)/mergetools
-sharedir = $(prefix)/share
-gitwebdir = $(sharedir)/gitweb
-localedir = $(sharedir)/locale
-template_dir = share/git-core/templates
-htmldir = share/doc/git-doc
-ETC_GITCONFIG = $(sysconfdir)/gitconfig
-ETC_GITATTRIBUTES = $(sysconfdir)/gitattributes
-lib = lib
-# DESTDIR =
-pathsep = :
-
-export prefix bindir sharedir sysconfdir gitwebdir localedir
-
-CC = cc
-AR = ar
-RM = rm -f
-DIFF = diff
-TAR = tar
-FIND = find
-INSTALL = install
-RPMBUILD = rpmbuild
-TCL_PATH = tclsh
-TCLTK_PATH = wish
-XGETTEXT = xgettext
-MSGFMT = msgfmt
-PTHREAD_LIBS = -lpthread
-PTHREAD_CFLAGS =
-GCOV = gcov
-
-export TCL_PATH TCLTK_PATH
-
-SPARSE_FLAGS =
-
-
+include default-config.mak
 
 ### --- END CONFIGURATION SECTION ---
 
diff --git a/default-config.mak b/default-config.mak
new file mode 100644
index 0000000..b2aab3d
--- /dev/null
+++ b/default-config.mak
@@ -0,0 +1,61 @@
+# CFLAGS and LDFLAGS are for the users to override from the command line.
+
+CFLAGS = -g -O2 -Wall
+LDFLAGS =
+ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS)
+ALL_LDFLAGS = $(LDFLAGS)
+STRIP ?= strip
+
+# Among the variables below, these:
+#   gitexecdir
+#   template_dir
+#   mandir
+#   infodir
+#   htmldir
+#   sysconfdir
+# can be specified as a relative path some/where/else;
+# this is interpreted as relative to $(prefix) and "git" at
+# runtime figures out where they are based on the path to the executable.
+# This can help installing the suite in a relocatable way.
+
+prefix = $(HOME)
+bindir_relative = bin
+bindir = $(prefix)/$(bindir_relative)
+mandir = share/man
+infodir = share/info
+gitexecdir = libexec/git-core
+mergetoolsdir = $(gitexecdir)/mergetools
+sharedir = $(prefix)/share
+gitwebdir = $(sharedir)/gitweb
+localedir = $(sharedir)/locale
+template_dir = share/git-core/templates
+htmldir = share/doc/git-doc
+ETC_GITCONFIG = $(sysconfdir)/gitconfig
+ETC_GITATTRIBUTES = $(sysconfdir)/gitattributes
+lib = lib
+# DESTDIR =
+pathsep = :
+
+export prefix bindir sharedir sysconfdir gitwebdir localedir
+
+CC = cc
+AR = ar
+RM = rm -f
+DIFF = diff
+TAR = tar
+FIND = find
+INSTALL = install
+RPMBUILD = rpmbuild
+TCL_PATH = tclsh
+TCLTK_PATH = wish
+XGETTEXT = xgettext
+MSGFMT = msgfmt
+PTHREAD_LIBS = -lpthread
+PTHREAD_CFLAGS =
+GCOV = gcov
+
+export TCL_PATH TCLTK_PATH
+
+SPARSE_FLAGS =
+
+
-- 
1.8.1.2.526.gf51a757

Re: [PATCH 3/4] Makefile: factor common configuration in git-default-config.mak

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> Similarly to the extraction of perl-related code in perl.mak, we extract
> general default configuration from the Makefile to make it available from
> directories other than the toplevel.
>
> This is required to make perl.mak usable because it requires $(pathsep)
> to be set.
>
> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
> ---

I really think this is going in a wrong direction.  Whatever you
happen to have chosen in this patch will be available to others,
while whatever are left out will not be.  When adding new things,
people need to ask if it needs to be sharable or not, and the right
answer to that question will even change over time.

Re: [PATCH 3/4] Makefile: factor common configuration in git-default-config.mak

[Matthieu Moy]

Junio C Hamano <gitster@pobox.com> writes:

> I really think this is going in a wrong direction.  Whatever you
> happen to have chosen in this patch will be available to others,
> while whatever are left out will not be.  When adding new things,
> people need to ask if it needs to be sharable or not, and the right
> answer to that question will even change over time.

My feeling is that Git's toplevel Makefile has become too large to
remain completely monolithic, and splitting is good to organize it (and
yes, splitting code into several files imply that future added code will
have to be added in the right file, but that's not very different from
splitting C code into several .c files to me). But that is another
matter, and ...

Junio C Hamano <gitster@pobox.com> writes:

> Then do something like
>
> 	all::
> 		$(MAKE) -C ../.. \
> 			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
>                         build-perl-script

This ended up being very simple to implement (essentially, the Makefile
already knows how to do this, so this just means adding convenience
build-perl-script target to the toplevel), so 2 new patches follow doing
this, with a ridiculously small new version of mw-to-git/Makefile.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

[PATCH 1/2] Makefile: make script-related rules usable from subdirectories

[Matthieu Moy]

Git's Makefile provides a few nice features for script build and
installation (substitute the first line with the right path, hardcode the
path to Git library, ...).

The Makefile already knows how to process files outside the toplevel
directory with e.g.

  make SCRIPT_PERL=path/to/file.perl path/to/file

but we can make it simpler for callers by exposing build, install and
clean rules as .PHONY targets.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
The goal of this series is to use perl, but it is as easy to do it
with sh and python too, so I did it for them too. I tested a manual
"make -C ../../" in contrib/subtree and contrib/hg-to-git/ to check that
it actually works.

 Makefile | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 5a2e02d..b4af30d 100644
--- a/Makefile
+++ b/Makefile
@@ -480,9 +480,38 @@ SCRIPT_PERL += git-svn.perl
 SCRIPT_PYTHON += git-remote-testpy.py
 SCRIPT_PYTHON += git-p4.py
 
-SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) \
-	  $(patsubst %.perl,%,$(SCRIPT_PERL)) \
-	  $(patsubst %.py,%,$(SCRIPT_PYTHON)) \
+# Generated files for scripts
+SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
+SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
+SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
+
+# Individual rules to allow e.g.
+# "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
+# from subdirectories like contrib/*/
+.PHONY: build-perl-script build-sh-script build-python-script
+build-perl-script: $(SCRIPT_PERL_GEN)
+build-sh-script: $(SCRIPT_SH_GEN)
+build-python-script: $(SCRIPT_PYTHON_GEN)
+
+.PHONY: install-perl-script install-sh-script install-python-script
+install-sh-script: $(SCRIPT_SH_GEN)
+	$(INSTALL) $(SCRIPT_SH_GEN) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
+install-perl-script: $(SCRIPT_PERL_GEN)
+	$(INSTALL) $(SCRIPT_PERL_GEN) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
+install-python-script: $(SCRIPT_PYTHON_GEN)
+	$(INSTALL) $(SCRIPT_PYTHON_GEN) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
+
+.PHONY: clean-perl-script clean-sh-script clean-python-script
+clean-sh-script:
+	$(RM) $(SCRIPT_SH_GEN)
+clean-perl-script:
+	$(RM) $(SCRIPT_PERL_GEN)
+clean-python-script:
+	$(RM) $(SCRIPT_PYTHON_GEN)
+
+SCRIPTS = $(SCRIPT_SH_GEN) \
+	  $(SCRIPT_PERL_GEN) \
+	  $(SCRIPT_PYTHON_GEN) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
-- 
1.8.1.2.530.g3cc16e4.dirty

[PATCH 2/2] git-remote-mediawiki: use toplevel's Makefile

[Matthieu Moy]

This makes the Makefile simpler, while providing more features, and more
consistency (the exact same rules with the exact same configuration as
Git official commands are applied with the new version).

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 contrib/mw-to-git/.gitignore                       |  1 +
 contrib/mw-to-git/Makefile                         | 64 ++++++----------------
 ...-remote-mediawiki => git-remote-mediawiki.perl} |  0
 3 files changed, 18 insertions(+), 47 deletions(-)
 create mode 100644 contrib/mw-to-git/.gitignore
 rewrite contrib/mw-to-git/Makefile (96%)
 rename contrib/mw-to-git/{git-remote-mediawiki => git-remote-mediawiki.perl} (100%)

diff --git a/contrib/mw-to-git/.gitignore b/contrib/mw-to-git/.gitignore
new file mode 100644
index 0000000..b919655
--- /dev/null
+++ b/contrib/mw-to-git/.gitignore
@@ -0,0 +1 @@
+git-remote-mediawiki
diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile
dissimilarity index 96%
index 3ed728b..f149719 100644
--- a/contrib/mw-to-git/Makefile
+++ b/contrib/mw-to-git/Makefile
@@ -1,47 +1,17 @@
-#
-# Copyright (C) 2012
-#     Charles Roussel <charles.roussel@ensimag.imag.fr>
-#     Simon Cathebras <simon.cathebras@ensimag.imag.fr>
-#     Julien Khayat <julien.khayat@ensimag.imag.fr>
-#     Guillaume Sasdy <guillaume.sasdy@ensimag.imag.fr>
-#     Simon Perrat <simon.perrat@ensimag.imag.fr>
-#
-## Build git-remote-mediawiki
-
--include ../../config.mak.autogen
--include ../../config.mak
-
-ifndef PERL_PATH
-	PERL_PATH = /usr/bin/perl
-endif
-ifndef gitexecdir
-	gitexecdir = $(shell git --exec-path)
-endif
-
-PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
-gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
-SCRIPT = git-remote-mediawiki
-
-.PHONY: install help doc test clean
-
-help:
-	@echo 'This is the help target of the Makefile. Current configuration:'
-	@echo '  gitexecdir = $(gitexecdir_SQ)'
-	@echo '  PERL_PATH = $(PERL_PATH_SQ)'
-	@echo 'Run "$(MAKE) install" to install $(SCRIPT) in gitexecdir'
-	@echo 'Run "$(MAKE) test" to run the testsuite'
-
-install:
-	sed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \
-		> '$(gitexecdir_SQ)/$(SCRIPT)'
-	chmod +x '$(gitexecdir)/$(SCRIPT)'
-
-doc:
-	@echo 'Sorry, "make doc" is not implemented yet for $(SCRIPT)'
-
-test:
-	$(MAKE) -C t/ test
-
-clean:
-	$(RM) '$(gitexecdir)/$(SCRIPT)'
-	$(MAKE) -C t/ clean
+#
+# Copyright (C) 2013
+#     Matthieu Moy <Matthieu.Moy@imag.fr>
+#
+## Build git-remote-mediawiki
+
+SCRIPT_PERL=git-remote-mediawiki.perl
+GIT_ROOT_DIR=../..
+HERE=contrib/mw-to-git/
+
+SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL))
+
+all: build
+
+build install clean:
+	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
+                $@-perl-script
diff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki.perl
similarity index 100%
rename from contrib/mw-to-git/git-remote-mediawiki
rename to contrib/mw-to-git/git-remote-mediawiki.perl
-- 
1.8.1.2.530.g3cc16e4.dirty

[PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

[Matthieu Moy]

The configuration of the install directory is not reused from the
toplevel Makefile: we assume Git is already built, hence just call
"git --exec-path". This avoids too much surgery in the toplevel Makefile.

git-remote-mediawiki.perl can now "use Git;".

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
---
 contrib/mw-to-git/.gitignore                       |  1 +
 contrib/mw-to-git/Makefile                         | 45 ++++++++++++++--------
 ...-remote-mediawiki => git-remote-mediawiki.perl} |  0
 3 files changed, 30 insertions(+), 16 deletions(-)
 create mode 100644 contrib/mw-to-git/.gitignore
 rename contrib/mw-to-git/{git-remote-mediawiki => git-remote-mediawiki.perl} (100%)

diff --git a/contrib/mw-to-git/.gitignore b/contrib/mw-to-git/.gitignore
new file mode 100644
index 0000000..b919655
--- /dev/null
+++ b/contrib/mw-to-git/.gitignore
@@ -0,0 +1 @@
+git-remote-mediawiki
diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile
index 3ed728b..ed8073b 100644
--- a/contrib/mw-to-git/Makefile
+++ b/contrib/mw-to-git/Makefile
@@ -8,40 +8,53 @@
 #
 ## Build git-remote-mediawiki
 
--include ../../config.mak.autogen
--include ../../config.mak
+all:
+
+GIT_ROOT_DIR=../../
+include $(GIT_ROOT_DIR)/default-config.mak
+-include $(GIT_ROOT_DIR)/config.mak.autogen
+-include $(GIT_ROOT_DIR)/config.mak
+-include $(GIT_ROOT_DIR)/GIT-VERSION-FILE
+
+
+SCRIPT_PERL = git-remote-mediawiki.perl
+ALL_PROGRAMS = $(patsubst %.perl,%,$(SCRIPT_PERL))
+
+include $(GIT_ROOT_DIR)/perl.mak
 
-ifndef PERL_PATH
-	PERL_PATH = /usr/bin/perl
-endif
 ifndef gitexecdir
 	gitexecdir = $(shell git --exec-path)
 endif
 
-PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
-gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
-SCRIPT = git-remote-mediawiki
+ifneq ($(filter /%,$(firstword $(gitexecdir))),)
+gitexec_instdir = $(gitexecdir)
+else
+gitexec_instdir = $(prefix)/$(gitexecdir)
+endif
+gitexec_instdir_SQ = $(subst ','\'',$(gitexec_instdir))
 
 .PHONY: install help doc test clean
 
 help:
 	@echo 'This is the help target of the Makefile. Current configuration:'
-	@echo '  gitexecdir = $(gitexecdir_SQ)'
+	@echo '  gitexec_instdir = $(gitexec_instdir_SQ)'
 	@echo '  PERL_PATH = $(PERL_PATH_SQ)'
-	@echo 'Run "$(MAKE) install" to install $(SCRIPT) in gitexecdir'
+	@echo 'Run "$(MAKE) all" to build the script'
+	@echo 'Run "$(MAKE) install" to install $(ALL_PROGRAMS) in gitexec_instdir'
 	@echo 'Run "$(MAKE) test" to run the testsuite'
 
-install:
-	sed -e '1s|#!.*/perl|#!$(PERL_PATH_SQ)|' $(SCRIPT) \
-		> '$(gitexecdir_SQ)/$(SCRIPT)'
-	chmod +x '$(gitexecdir)/$(SCRIPT)'
+all: $(ALL_PROGRAMS)
+
+install: $(ALL_PROGRAMS)
+	$(INSTALL) $(ALL_PROGRAMS) '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
 doc:
-	@echo 'Sorry, "make doc" is not implemented yet for $(SCRIPT)'
+	@echo 'Sorry, "make doc" is not implemented yet for $(ALL_PROGRAMS)'
 
 test:
 	$(MAKE) -C t/ test
 
 clean:
-	$(RM) '$(gitexecdir)/$(SCRIPT)'
+	$(RM) $(ALL_PROGRAMS)
+	$(RM) $(patsubst %,$(gitexec_instdir)/%,/$(ALL_PROGRAMS))
 	$(MAKE) -C t/ clean
diff --git a/contrib/mw-to-git/git-remote-mediawiki b/contrib/mw-to-git/git-remote-mediawiki.perl
similarity index 100%
rename from contrib/mw-to-git/git-remote-mediawiki
rename to contrib/mw-to-git/git-remote-mediawiki.perl
-- 
1.8.1.2.526.gf51a757

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

[Junio C Hamano]

Matthieu Moy <Matthieu.Moy@imag.fr> writes:

> The configuration of the install directory is not reused from the
> toplevel Makefile: we assume Git is already built, hence just call
> "git --exec-path". This avoids too much surgery in the toplevel Makefile.
>
> git-remote-mediawiki.perl can now "use Git;".
>
> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
> ---

Continuing to the comment on 3/4, I wonder if it would be a lot
simpler and more maintainable if you replaced 1/4 to 3/4 with a
smaller patch to the top-level Makefile to teach it to munge
arbitrary path/to/foo.perl to path/to/foo the same way as we do to
other path/tool.perl that are known to the top-level Makefile
(similarly, another target to install the resulting path/to/foo at
an arbitrary place).  Then do something like

	all::
		$(MAKE) -C ../.. \
			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
                        build-perl-script
	install::
		$(MAKE) -C ../.. \
			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
                        install-perl-script

in this step.

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

[Jeff King]

On Thu, Feb 07, 2013 at 11:28:31AM -0800, Junio C Hamano wrote:

> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
> 
> > The configuration of the install directory is not reused from the
> > toplevel Makefile: we assume Git is already built, hence just call
> > "git --exec-path". This avoids too much surgery in the toplevel Makefile.
> >
> > git-remote-mediawiki.perl can now "use Git;".
> >
> > Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
> > ---
> 
> Continuing to the comment on 3/4, I wonder if it would be a lot
> simpler and more maintainable if you replaced 1/4 to 3/4 with a
> smaller patch to the top-level Makefile to teach it to munge
> arbitrary path/to/foo.perl to path/to/foo the same way as we do to
> other path/tool.perl that are known to the top-level Makefile
> (similarly, another target to install the resulting path/to/foo at
> an arbitrary place).  Then do something like
> 
> 	all::
> 		$(MAKE) -C ../.. \
> 			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
>                         build-perl-script
> 	install::
> 		$(MAKE) -C ../.. \
> 			PERL_SCRIPT=contrib/mw-to-git/git-remote-mediawiki.perl \
>                         install-perl-script
> 
> in this step.

That seems much cleaner to me. If done right, it could also let people
put:

  CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki

or similar into their config.mak, and just get specific contrib bits
built and installed along with the rest of git.

-Peff

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

[Matthieu Moy]

Jeff King <peff@peff.net> writes:

> That seems much cleaner to me. If done right, it could also let people
> put:
>
>   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki

Actually, you can already do this:

  SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl

probably not by design, but it works!

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

[Jeff King]

On Fri, Feb 08, 2013 at 06:34:37PM +0100, Matthieu Moy wrote:

> Jeff King <peff@peff.net> writes:
> 
> > That seems much cleaner to me. If done right, it could also let people
> > put:
> >
> >   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki
> 
> Actually, you can already do this:
> 
>   SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl
> 
> probably not by design, but it works!

So putting:

  ROOT=contrib/mw-to-git
  git-remote-mediawiki: FORCE
          @make -C ../.. SCRIPT_PERL=$(ROOT)/$@.perl $(ROOT)/$@

in contrib/mw-to-git/Makefile would already work? Neat.

-Peff

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Fri, Feb 08, 2013 at 06:34:37PM +0100, Matthieu Moy wrote:
>
>> Jeff King <peff@peff.net> writes:
>> 
>> > That seems much cleaner to me. If done right, it could also let people
>> > put:
>> >
>> >   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki
>> 
>> Actually, you can already do this:
>> 
>>   SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl
>> 
>> probably not by design, but it works!
>
> So putting:
>
>   ROOT=contrib/mw-to-git
>   git-remote-mediawiki: FORCE
>           @make -C ../.. SCRIPT_PERL=$(ROOT)/$@.perl $(ROOT)/$@
>
> in contrib/mw-to-git/Makefile would already work? Neat.

That essentially is what [v2 2/2] does, no?

Re: [PATCH 4/4] git-remote-mediawiki: use Git's Makefile to build the script

[Jeff King]

On Fri, Feb 08, 2013 at 10:13:23AM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > On Fri, Feb 08, 2013 at 06:34:37PM +0100, Matthieu Moy wrote:
> >
> >> Jeff King <peff@peff.net> writes:
> >> 
> >> > That seems much cleaner to me. If done right, it could also let people
> >> > put:
> >> >
> >> >   CONTRIB_PERL += contrib/mw-to-git/git-remote-mediawiki
> >> 
> >> Actually, you can already do this:
> >> 
> >>   SCRIPT_PERL += contrib/mw-to-git/git-remote-mediawiki.perl
> >> 
> >> probably not by design, but it works!
> >
> > So putting:
> >
> >   ROOT=contrib/mw-to-git
> >   git-remote-mediawiki: FORCE
> >           @make -C ../.. SCRIPT_PERL=$(ROOT)/$@.perl $(ROOT)/$@
> >
> > in contrib/mw-to-git/Makefile would already work? Neat.
> 
> That essentially is what [v2 2/2] does, no?

Yes (this one was cc'd to me, but the others were not, so I read it in
isolation). I think Matthieu's series is nicer than just that, though,
because it handles the single-file case installation, too, which
requires more support from the parent Makefile.

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Jeff King]

On Wed, Feb 06, 2013 at 10:58:13AM -0500, Ted Zlatanov wrote:

> MM> I don't know about the netrc credential helper, but I guess that's
> MM> another layer. The git-remote-mediawiki code is the code to call the
> MM> credential C API, that in turn may (or may not) call a credential
> MM> helper.
> 
> Yup.  But what you call "read" and "write" are, to the credential
> helper, "write" and "read" but it's the same protocol :)  So maybe the
> names should be changed to reflect that, e.g. "query" and "response."

Is that true? As a user of the credential system, git-remote-mediawiki
would want to "write" to git-credential, then "read" the response. As a
helper, git-credential-netrc would want to "read" the query then
"write" the response. The order is different, but the operations should
be the same in both cases.

The big difference is that mediawiki would want an additional function
to open a pipe to "git credential" and operate on that, whereas the
helper will be reading/writing stdio.

-Peff

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Wed, 6 Feb 2013 16:57:24 -0500 Jeff King <peff@peff.net> wrote: 

JK> On Wed, Feb 06, 2013 at 10:58:13AM -0500, Ted Zlatanov wrote:
MM> I don't know about the netrc credential helper, but I guess that's
MM> another layer. The git-remote-mediawiki code is the code to call the
MM> credential C API, that in turn may (or may not) call a credential
MM> helper.
>> 
>> Yup.  But what you call "read" and "write" are, to the credential
>> helper, "write" and "read" but it's the same protocol :)  So maybe the
>> names should be changed to reflect that, e.g. "query" and "response."

JK> Is that true? As a user of the credential system, git-remote-mediawiki
JK> would want to "write" to git-credential, then "read" the response. As a
JK> helper, git-credential-netrc would want to "read" the query then
JK> "write" the response. The order is different, but the operations should
JK> be the same in both cases.

Logically they are different steps (query and response), even though the
data protocol is the same.  But it's really not a big deal, I know what
it means either way.

JK> The big difference is that mediawiki would want an additional function
JK> to open a pipe to "git credential" and operate on that, whereas the
JK> helper will be reading/writing stdio.

Yup.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Matthieu Moy]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Logically they are different steps (query and response), even though the
> data protocol is the same.  But it's really not a big deal, I know what
> it means either way.

Yes, but if you rename write() to query(), then on the helper side,
you'll have to call query() to send the response, and response() to read
the query. Much worse than keeping read/write.

Plus, read/write has already been used for a while in the C API, so I'd
rather keep the same names for the Perl equivalent.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Thu, 07 Feb 2013 08:08:59 +0100 Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote: 

MM> Plus, read/write has already been used for a while in the C API, so I'd
MM> rather keep the same names for the Perl equivalent.

That makes perfect sense.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Michal Nazarewicz]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

On Wed, Feb 06 2013, Junio C Hamano <gitster@pobox.com> wrote:
> I see a lot of rerolls on the credential helper front, but is there
> anybody working on hooking send-email to the credential framework?

I assumed someone had, but if not I can take a stab at it.  I'm not sure
however how should I map server, server-port, and user to credential
key-value pairs.  I'm leaning towards

	protocol=smtp
	host=<smtp-server>:<smtp-port>
	user=<user>

and than netrc/authinfo helper splitting host to host name and port
number, unless port is not in host in which case protocol is assumed as
port.

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +----<email/xmpp: mpn@google.com>--------------ooO--(_)--Ooo--

[-- Attachment #2.1: Type: text/plain, Size: 0 bytes --]



[-- Attachment #2.2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Wed, 06 Feb 2013 14:26:46 +0100 Michal Nazarewicz <mina86@mina86.com> wrote: 

MN> On Wed, Feb 06 2013, Junio C Hamano <gitster@pobox.com> wrote:
>> I see a lot of rerolls on the credential helper front, but is there
>> anybody working on hooking send-email to the credential framework?

MN> I assumed someone had, but if not I can take a stab at it.  I'm not sure
MN> however how should I map server, server-port, and user to credential
MN> key-value pairs.  I'm leaning towards

MN> 	protocol=smtp
MN> 	host=<smtp-server>:<smtp-port>
MN> 	user=<user>

MN> and than netrc/authinfo helper splitting host to host name and port
MN> number, unless port is not in host in which case protocol is assumed as
MN> port.

That would work (with my PATCHv6 of the netrc credential helper) as
follows:

1) just host

host=H

maps to

machine H login Y password Z

2) host + protocol smtp

host=H
protocol=smtp

maps to any of:

machine H port smtp login Y password Z
machine H protocol smtp login Y password Z

3) host:port + protocol smtp

host=H:25
protocol=smtp

maps to any of:

machine H port 25 protocol smtp login Y password Z
machine H:25 port smtp login Y password Z
machine H:25 protocol smtp login Y password Z

That's my understanding of what we discussed with Peff and Junio about
token mapping.  Note we don't split the input host, but instead say "if
token 'port' is numeric, append it to the host token" on the netrc side.

Does that sound reasonable?  If yes, I can add it to the testing
Makefile for the netrc credential helper, to make sure it's clearly
stated and tested.

Ted

Re: [PATCH] git-send-email: add ~/.authinfo parsing

[Ted Zlatanov]

On Tue, 29 Jan 2013 11:53:19 -0800 Junio C Hamano <gitster@pobox.com> wrote: 

JCH> Makes one wonder why .authinfo and not .netrc; 

JCH> http://www.gnu.org/software/emacs/manual/html_node/auth/Help-for-users.html

JCH> phrases it amusingly:

JCH>         “Netrc” files are usually called .authinfo or .netr
JCH>         nowadays .authinfo seems to be more popular and the
JCH>         auth-source library encourages this confusion by accepting
JCH>         both

I wrote this and the auth-source.el library in Emacs (I'm glad it was
amusing :).  The confusion is further perpetuated by our (in Emacs)
encouragement to use a .authinfo.gpg file, which is then decrypted on
the fly by Emacs through GPG.  The format is the same; by the time
auth-source.el sees the contents, they are plain text since the decoding
happens at the file handler level.

I think it makes sense to write the code to support both
`git-send-email' and credentials.  I have had it in my TODO list for
almost 2 years now to work on credential support, and to support the
~/.authinfo.gpg decoding specifically.  Ideally this would also support
the other formats... Michal, would you be interested in that feature?  I
promise to get off my rear and help out.

>> +The '~/.authinfo' file is read if Text::CSV Perl module is installed
>> +on the system; if it's missing, a notification message will be printed
>> +and the file ignored altogether.  The file should contain a line with
>> +the following format:
>> ++
>> +  machine <domain> port <port> login <user> password <pass>

JCH> It is rather strange to require a comma-separated-values parser to
JCH> read a file format this simple, isn't it?

I'd recommend a hand-crafted parser.  Among other things, you should
accept both "strings" and 'strings' if possible (I've seen both formats
in the wild), and the format is simple enough to avoid the module
dependency.

>> ++
>> +Contrary to other tools, 'git-send-email' does not support symbolic
>> +port names like 'imap' thus `<port>` must be a number.

JCH> Perhaps you can convert at least some popular ones yourself?  After
JCH> all, the user may be using an _existing_ .authinfo/.netrc that she
JCH> has been using with other programs that do understand symbolic port
JCH> names.  Rather than forcing all such users to update their files,
JCH> the patch can work a bit harder for them and the world will be a
JCH> better place, no?

I agree, "port imap" is a nice self-documenting token.  Maybe it can be
interpreted by the program that requests the token with a services
lookup, where supported.

Ted