[Wishlist] could git tell which password it is asking when asking a password.

[Wishlist] could git tell which password it is asking when asking a password.

[Rémi Vanicat]

Hello,

When git is asking for a password (for example for pushing over https)
it call the $GIT_ASKPASS script with only "Password: " as a an argument,
so when one have several remote, it might not know which one is asking
the password. 

It would be interesting also to plug some sort of password-safe unto
git, or some "git-agent". 
-- 
Rémi Vanicat

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Junio C Hamano]

Rémi Vanicat <vanicat@debian.org> writes:

> When git is asking for a password (for example for pushing over https)
> it call the $GIT_ASKPASS script with only "Password: " as a an argument,
> so when one have several remote, it might not know which one is asking
> the password. 

On the C side, git_getpass() is the function to touch. It has only three
callers.

In init_curl_http_auth() in http.c, we know "user_name" but we do not give
it as a hint when coming up with a prompt.  A possible update to the
git_getpass() API would be to make the call look like this:

diff --git a/http.c b/http.c
index a1ea3db..ee3e821 100644
--- a/http.c
+++ b/http.c
@@ -214,7 +214,7 @@ static void init_curl_http_auth(CURL *result)
 	if (user_name) {
 		struct strbuf up = STRBUF_INIT;
 		if (!user_pass)
-			user_pass = xstrdup(git_getpass("Password: "));
+			user_pass = git_getpass(_("Password for %s: "), user_name);
 		strbuf_addf(&up, "%s:%s", user_name, user_pass);
 		curl_easy_setopt(result, CURLOPT_USERPWD,
 				 strbuf_detach(&up, NULL));

The points are

 (1) to show identity for which the password is being asked for;
 (2) to give a fresh and stable memory, making it unnecessary to
     xstrdup() while at it.

The same thing for the call in has_cert_password(), which may want to use ssl_cert
as the identifier.

There is an abuse of git_getpass() in http_request() to ask for the
username. It inherits the "noecho"-ness of git_getpass() which gives a
bad user experience. We _may_ want to give another parameter to git_getpass()
to specify if we want noecho.

The other call is from imap-send.c that knows srvc->user and srvc->host
and formulates the prompt including the identity. So an alternative route
may be to keep git_getpass() as-is, and update the init_curl_http_auth()
callsite to include the username (but imap-send assumes that user and host
are relatively short without verifying that assumption, and should not be
used as a model of good existing code).

> It would be interesting also to plug some sort of password-safe unto
> git, or some "git-agent". 

I am not particularly interested in seeing git specific agent. Something
that can be called as an external process that talks with existing
practices (gpg agent and friends) would be nice.

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Ted Zlatanov]

On Fri, 01 Jul 2011 15:59:09 +0200 Rémi Vanicat <vanicat@debian.org> wrote: 

RV> When git is asking for a password (for example for pushing over https)
RV> it call the $GIT_ASKPASS script with only "Password: " as a an argument,
RV> so when one have several remote, it might not know which one is asking
RV> the password. 

Seconded, I run into this all the time.  A configurable prompt with %h
for the host, etc. would be really nice.

RV> It would be interesting also to plug some sort of password-safe unto
RV> git, or some "git-agent". 

This would also be really nice.  ~/.netrc is not a great place to put
passwords for the HTTP transport.  In GNU Emacs we have ~/.authinfo.gpg
with the same content as ~/.netrc but encrypted by GPG and thus more
secure (the user is either prompted for the password, if the file is
encrypted symmetrically, or the user simply loads their private key into
the GPG agent).  I believe all this can be done with the GPGME library.
There's also the Secrets API on newer Gnome and KDE installs, which has
a pretty nice D-Bus interface.

But is this a libcurl feature request?  Or can a Git plugin (an
alternate HTTPS transport maybe?) handle it?

Thanks
Ted

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> ... So an alternative route
> may be to keep git_getpass() as-is, and update the init_curl_http_auth()
> callsite to include the username (but imap-send assumes that user and host
> are relatively short without verifying that assumption, and should not be
> used as a model of good existing code).

And here is such a lazy patch, completely untested, of course ;-).

 http.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/http.c b/http.c
index b2ae8de..44948a7 100644
--- a/http.c
+++ b/http.c
@@ -209,8 +209,11 @@ static void init_curl_http_auth(CURL *result)
 {
 	if (user_name) {
 		struct strbuf up = STRBUF_INIT;
-		if (!user_pass)
-			user_pass = xstrdup(git_getpass("Password: "));
+		if (!user_pass) {
+			strbuf_addf(&up, "Password for %s: ", user_name);
+			user_pass = xstrdup(git_getpass(up.buf));
+			strbuf_reset(&up);
+		}
 		strbuf_addf(&up, "%s:%s", user_name, user_pass);
 		curl_easy_setopt(result, CURLOPT_USERPWD,
 				 strbuf_detach(&up, NULL));

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Shawn Pearce]

On Fri, Jul 1, 2011 at 10:16, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> diff --git a/http.c b/http.c
> index b2ae8de..44948a7 100644
> --- a/http.c
> +++ b/http.c
> @@ -209,8 +209,11 @@ static void init_curl_http_auth(CURL *result)
>  {
>        if (user_name) {
>                struct strbuf up = STRBUF_INIT;
> -               if (!user_pass)
> -                       user_pass = xstrdup(git_getpass("Password: "));
> +               if (!user_pass) {
> +                       strbuf_addf(&up, "Password for %s: ", user_name);

The user_name by itself may not be sufficient. I may also need the URL
to correctly answer the question. I don't always use the same password
on every website. :-)

As a human sure, I know what URL I asked Git to poke for me. But if I
want to write a script that looks this up in a password manager for
me, that script needs the URL.

Food for thought.

-- 
Shawn.

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> On Fri, Jul 1, 2011 at 10:16, Junio C Hamano <gitster@pobox.com> wrote:
>> Junio C Hamano <gitster@pobox.com> writes:
>> diff --git a/http.c b/http.c
>> index b2ae8de..44948a7 100644
>> --- a/http.c
>> +++ b/http.c
>> @@ -209,8 +209,11 @@ static void init_curl_http_auth(CURL *result)
>>  {
>>        if (user_name) {
>>                struct strbuf up = STRBUF_INIT;
>> -               if (!user_pass)
>> -                       user_pass = xstrdup(git_getpass("Password: "));
>> +               if (!user_pass) {
>> +                       strbuf_addf(&up, "Password for %s: ", user_name);
>
> The user_name by itself may not be sufficient. I may also need the URL
> to correctly answer the question. I don't always use the same password
> on every website. :-)
>
> As a human sure, I know what URL I asked Git to poke for me.

I was wondering about that when I gave that quick patch.  And "as a human"
may not necessarily apply when you are letting submodule fetch to recurse.

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Rémi Vanicat]

Junio C Hamano <gitster@pobox.com> writes:

> Shawn Pearce <spearce@spearce.org> writes:
>
>> On Fri, Jul 1, 2011 at 10:16, Junio C Hamano <gitster@pobox.com> wrote:
>>> Junio C Hamano <gitster@pobox.com> writes:
>>> diff --git a/http.c b/http.c
>>> index b2ae8de..44948a7 100644
>>> --- a/http.c
>>> +++ b/http.c
>>> @@ -209,8 +209,11 @@ static void init_curl_http_auth(CURL *result)
>>>  {
>>>        if (user_name) {
>>>                struct strbuf up = STRBUF_INIT;
>>> -               if (!user_pass)
>>> -                       user_pass = xstrdup(git_getpass("Password: "));
>>> +               if (!user_pass) {
>>> +                       strbuf_addf(&up, "Password for %s: ", user_name);
>>
>> The user_name by itself may not be sufficient. I may also need the URL
>> to correctly answer the question. I don't always use the same password
>> on every website. :-)
>>
>> As a human sure, I know what URL I asked Git to poke for me.
>
> I was wondering about that when I gave that quick patch.  And "as a human"
> may not necessarily apply when you are letting submodule fetch to recurse.

I also believe that having the host name would be useful, both for human
(another example would be git remote update when there are several
remote) and script.  

-- 
Rémi Vanicat

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Ted Zlatanov]

On Fri, 01 Jul 2011 21:25:59 +0200 Rémi Vanicat <vanicat@debian.org> wrote: 

RV> Junio C Hamano <gitster@pobox.com> writes:
>> I was wondering about that when I gave that quick patch.  And "as a human"
>> may not necessarily apply when you are letting submodule fetch to recurse.

RV> I also believe that having the host name would be useful, both for human
RV> (another example would be git remote update when there are several
RV> remote) and script.  

Maybe (using %U for the URL, %h for the host, %u for the user name):

"User name for %h: " OR "User name for %h (%U): "

"Password for %u@%h: " OR "Password for %u@%h (%U): "

with some way to switch between the two styles?

Ted

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Junio C Hamano]

Rémi Vanicat <vanicat@debian.org> writes:

> I also believe that having the host name would be useful, both for human
> (another example would be git remote update when there are several
> remote) and script.  

Patches welcome, but I have to warn you that the code may _not_ know the
URL at that point in the callchain.

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Jeff King]

On Fri, Jul 01, 2011 at 10:00:27AM -0700, Junio C Hamano wrote:

> > It would be interesting also to plug some sort of password-safe unto
> > git, or some "git-agent". 
> 
> I am not particularly interested in seeing git specific agent. Something
> that can be called as an external process that talks with existing
> practices (gpg agent and friends) would be nice.

Coincidentally, I am working on a big patch series for exactly this.

It is still lacking some docs and tests, but if you want to take a peek,
it's at:

  https://github.com/peff/git.git jk/http-auth

It runs external credential helpers, giving them a unique context (like
the protocol and hostname for http connections), as well as a username,
if we already have one. The goal is two-fold:

  1. Plug into existing password wallets that are going to be
     user- and OS- specific.

  2. Provide a few stock helpers to implement simple policies in a
     pluggable way.

Right now I have two helpers: "store", and "cache". Both will check
internal storage for a password; if we don't have one, they will prompt
and put the result in internal storage. In either case, the password is
sent back to git.  They differ in what "internal storage" means.

In the case of "store", it is a mode 600 ~/.git-credentials file.  Yes,
this is horribly insecure. But it's what some people want, it's no worse
than .netrc, and it's what svn does (and what gitter wouldn't be swayed
by that last argument? :) ). It's a little more friendly than netrc
because the storage happens transparently. I think the docs for this
should discourage it because of the security implications, and it should
definitely never become the default.

For "cache", we fork off a storage daemon which keeps the password in
memory for a specified period of time. The password never touches the
disk. It doesn't mlock() right now, but it could on platforms that
support it.

Both are obviously mediocre solutions in comparison to a real password
wallet[1]. But they're really simple to implement and give us a better
baseline to ship with git. I'm hoping users of individual keychains will
implement helpers to use them. Somebody from GitHub is going to work on
an OS X keychain helper. I personally use a home-grown password safe;
writing a read-only helper was about 10 lines of shell code.

Anyway, if people want to try it out, build the branch I mentioned above
and configure it like:

  # horribly insecure
  git config http.credentialhelper store

or

  # a little better
  git config http.credentialhelper cache

  # we will now cache results for 15 minutes, but there's no reason not
  # to store the username all the time, to avoid having to type it on a
  # cache miss
  git config credential.https:github.com.username peff

I've been using it for the past week or so, but I'm sure there are
lurking bugs. If you run into any, let me know.

-Peff

[1] Obviously this entire exercise is an attempt to make https
authentication as nice a user-experience as ssh with keys and ssh-agent.
So it's tempting to say "just use ssh with keys". But I think the
reality is that ssh and keys are just too challenging for a subset of
the user population (especially ones on operating systems without good
support). Apparently GitHub's most common tech support issues are people
unable to figure out how to make ssh keys, set up an agent, etc.

Re: [Wishlist] could git tell which password it is asking when asking a password.

[Jeff King]

On Fri, Jul 01, 2011 at 01:30:49PM -0700, Junio C Hamano wrote:

> Rémi Vanicat <vanicat@debian.org> writes:
> 
> > I also believe that having the host name would be useful, both for human
> > (another example would be git remote update when there are several
> > remote) and script.  
> 
> Patches welcome, but I have to warn you that the code may _not_ know the
> URL at that point in the callchain.

Yeah, you have to save it; the simplest time is when we parse the
username+password bits out of the URL during http_auth_init, and then
pass it through to git_getpass.

But see the patch series I just mentioned elsewhere in the thread. Right
now it still uses "Password:" for http authentication, but it would be a
one-liner to have it use "Password for 'user@example.com':".

-Peff

encrypted netrc for Git (was: [Wishlist] could git tell which password it is asking when asking a password.)

[Ted Zlatanov]

On Fri, 01 Jul 2011 12:04:02 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Fri, 01 Jul 2011 15:59:09 +0200 Rémi Vanicat <vanicat@debian.org> wrote: 

RV> It would be interesting also to plug some sort of password-safe unto
RV> git, or some "git-agent". 

TZ> This would also be really nice.  ~/.netrc is not a great place to put
TZ> passwords for the HTTP transport.  In GNU Emacs we have ~/.authinfo.gpg
TZ> with the same content as ~/.netrc but encrypted by GPG and thus more
TZ> secure (the user is either prompted for the password, if the file is
TZ> encrypted symmetrically, or the user simply loads their private key into
TZ> the GPG agent).  I believe all this can be done with the GPGME library.
TZ> There's also the Secrets API on newer Gnome and KDE installs, which has
TZ> a pretty nice D-Bus interface.

TZ> But is this a libcurl feature request?  Or can a Git plugin (an
TZ> alternate HTTPS transport maybe?) handle it?

Ping?  I'd like to work on this if it seems like a feasible feature.

Thanks
Ted

Re: encrypted netrc for Git (was: [Wishlist] could git tell which password it is asking when asking a password.)

[Jeff King]

On Thu, Jul 14, 2011 at 09:05:50AM -0500, Ted Zlatanov wrote:

> On Fri, 01 Jul 2011 12:04:02 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 
> 
> TZ> On Fri, 01 Jul 2011 15:59:09 +0200 Rémi Vanicat <vanicat@debian.org> wrote: 
> 
> RV> It would be interesting also to plug some sort of password-safe unto
> RV> git, or some "git-agent". 
> 
> TZ> This would also be really nice.  ~/.netrc is not a great place to put
> TZ> passwords for the HTTP transport.  In GNU Emacs we have ~/.authinfo.gpg
> TZ> with the same content as ~/.netrc but encrypted by GPG and thus more
> TZ> secure (the user is either prompted for the password, if the file is
> TZ> encrypted symmetrically, or the user simply loads their private key into
> TZ> the GPG agent).  I believe all this can be done with the GPGME library.
> TZ> There's also the Secrets API on newer Gnome and KDE installs, which has
> TZ> a pretty nice D-Bus interface.
> 
> TZ> But is this a libcurl feature request?  Or can a Git plugin (an
> TZ> alternate HTTPS transport maybe?) handle it?
> 
> Ping?  I'd like to work on this if it seems like a feasible feature.

Check out:

  https://github.com/peff/git/commits/jk/http-auth

which provides an interface for getting credentials from external
helpers.

I need to write docs for a few of the top commits before posting the
patches to the list, but other than that, it should be fairly solid and
usable. And I'd love to get feedback from somebody trying to write a new
helper for it (i.e., to tell if the interface to the helpers is good
enough).

-Peff

Re: encrypted netrc for Git

[Ted Zlatanov]

On Thu, 14 Jul 2011 11:00:33 -0400 Jeff King <peff@peff.net> wrote: 

JK> On Thu, Jul 14, 2011 at 09:05:50AM -0500, Ted Zlatanov wrote:

TZ> This would also be really nice.  ~/.netrc is not a great place to put
TZ> passwords for the HTTP transport.  In GNU Emacs we have ~/.authinfo.gpg
TZ> with the same content as ~/.netrc but encrypted by GPG and thus more
TZ> secure (the user is either prompted for the password, if the file is
TZ> encrypted symmetrically, or the user simply loads their private key into
TZ> the GPG agent).  I believe all this can be done with the GPGME library.
TZ> There's also the Secrets API on newer Gnome and KDE installs, which has
TZ> a pretty nice D-Bus interface.

JK> Check out:

JK>   https://github.com/peff/git/commits/jk/http-auth

JK> which provides an interface for getting credentials from external
JK> helpers.

The API is good, but it's not clear from the docs how to configure
credential helpers from the user side.  From the tests it looks like you
set GIT_ASKPASS to them, is that right?  And you can also set
credential.helper?

Where do those helpers fit with the .netrc file?  Are they called before
or after or instead of the .netrc parse?

Linking these with external libraries like GPGME and the Secrets API
will be pretty easy and improve the user experience.  So I'll be glad to
work on it and provide you with feedback.  Would you be interested in
pushing your patches further after the testing?  They seem pretty
complete.

I'm off-line for the next 10 days or so; I'll start testing when I get
back.

Thanks for your help
Ted

Re: encrypted netrc for Git

[Jeff King]

On Fri, Jul 15, 2011 at 12:08:49PM -0500, Ted Zlatanov wrote:

> JK> Check out:
> 
> JK>   https://github.com/peff/git/commits/jk/http-auth
> 
> JK> which provides an interface for getting credentials from external
> JK> helpers.
> 
> The API is good, but it's not clear from the docs how to configure
> credential helpers from the user side.  From the tests it looks like you
> set GIT_ASKPASS to them, is that right?  And you can also set
> credential.helper?

Yes, that is the documentation I need to write before I can send in the
patches. :)

The answer is that you use "credential.helper". For example:

  $ git config credential.helper cache

  $ git push https://your.server/repo.git
  Username: <input your username>
  Password: <input your password>
  ... push happens ...

  [five minutes pass]
  $ git push https://your.server/repo.git
  ... push happens, no auth required ...

> Where do those helpers fit with the .netrc file?  Are they called before
> or after or instead of the .netrc parse?

They are what git provides to curl, either because we have "user@" in
the URL, or because we tried curl once and got an HTTP 401. Curl uses
netrc automagically behind the scenes.

So for a URL without "user@" I believe the order would be:

  1. Curl tries the request with what's in your netrc (or maybe it
     transparently requests and uses the netrc after getting a 401; I'm
     not sure).

  2. Curl gives us a 401, and we ask for credentials via getpass(). Or a
     credential helper, if defined. Any username given in netrc will not
     be considered a partial credential (i.e., you will be prompted for
     username and password as if netrc didn't exist).

  3. If those credentials fail (i.e., we get a 401 again), we quit.

> Linking these with external libraries like GPGME and the Secrets API
> will be pretty easy and improve the user experience.  So I'll be glad to
> work on it and provide you with feedback.

Yes, exactly. I think somebody at GitHub will probably work on OS X
Keychain integration, too.

I personally use a home-grown password safe that is a searchable
gpg-encrypted file (which then gets unlocked by gpg-agent). My helper is
more or less:

-- >8 --
#!/bin/sh

unique=
for i in "$@"; do
  case "$i" in
    --unique=*) unique=${i#--unique=} ;;
  esac
done

# find lines of the form
# example.com.username=me
# example.com.password=mypass
gpg -qd --no-tty $HOME/.pass.gpg |
sed -n 's/^$unique.//p
-- >8 --

(actually, my file format is quite a bit more complex and robust than
that, and I use a perl script to parse it instead of sed, but this was
meant to be illustrative of how simple it could be).

Obviously something integrated with the secrets API would be way nicer,
if you are running GNOME Keyring (that's part of why I pushed it out to
an external helper; there are nearly as many password wallet solutions
as there are users, and everybody will have their favorite).

> Would you be interested in pushing your patches further after the
> testing?  They seem pretty complete.

Absolutely. I'm planning on finishing up the docs and posting the
patches in the next couple days, so hopefully they will get more
feedback and testing there, too.

-Peff