* [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF
2020-09-28 11:40 ` [PATCH v2 0/3] Prepare git credential to read input with DOS line endings Johannes Schindelin via GitGitGadget
@ 2020-09-28 11:40 ` Nikita Leonov via GitGitGadget
2020-09-29 0:42 ` Jeff King
2020-09-28 11:40 ` [PATCH v2 2/3] credentials: make line reading Windows compatible Nikita Leonov via GitGitGadget
` (3 subsequent siblings)
4 siblings, 1 reply; 32+ messages in thread
From: Nikita Leonov via GitGitGadget @ 2020-09-28 11:40 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Nikita Leonov
From: Nikita Leonov <nykyta.leonov@gmail.com>
This fix makes using Git credentials more friendly to Windows users. In
previous version it was unable to finish input correctly without
configuration changes (tested in PowerShell, CMD, Cygwin).
We know credential filling should be finished by empty input, but the
current implementation does not take into account CR/LF ending, and
hence instead of the empty string we get '\r', which is interpreted as
an incorrect string.
So this commit changes default reading function to a more Windows
compatible reading function.
Signed-off-by: Nikita Leonov <nykyta.leonov@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
credential.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/credential.c b/credential.c
index efc29dc5e1..e5202fbef2 100644
--- a/credential.c
+++ b/credential.c
@@ -202,7 +202,7 @@ int credential_read(struct credential *c, FILE *fp)
{
struct strbuf line = STRBUF_INIT;
- while (strbuf_getline_lf(&line, fp) != EOF) {
+ while (strbuf_getline(&line, fp) != EOF) {
char *key = line.buf;
char *value = strchr(key, '=');
--
gitgitgadget
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF
2020-09-28 11:40 ` [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF Nikita Leonov via GitGitGadget
@ 2020-09-29 0:42 ` Jeff King
2020-10-02 11:37 ` Johannes Schindelin
0 siblings, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-09-29 0:42 UTC (permalink / raw)
To: Nikita Leonov via GitGitGadget; +Cc: git, Johannes Schindelin, Nikita Leonov
On Mon, Sep 28, 2020 at 11:40:22AM +0000, Nikita Leonov via GitGitGadget wrote:
> From: Nikita Leonov <nykyta.leonov@gmail.com>
>
> This fix makes using Git credentials more friendly to Windows users. In
> previous version it was unable to finish input correctly without
> configuration changes (tested in PowerShell, CMD, Cygwin).
>
> We know credential filling should be finished by empty input, but the
> current implementation does not take into account CR/LF ending, and
> hence instead of the empty string we get '\r', which is interpreted as
> an incorrect string.
>
> So this commit changes default reading function to a more Windows
> compatible reading function.
Unlike the credential-store file case, where we expect the data to be
URL-encoded anyway (and so any true "\r" in the data would not be found
in raw form), this means that the credential protocol can no longer
represent "\r" at the end of a value.
And we'd match "example.com\r" and "example.com" as the same (unlikely,
since carriage returns aren't allowed in hostnames, and curl will
complain about this). We'd also match "cert://some/path\r" and
"cert://some/path". Or "https://example.com/path\r" and its match, if
you have credential.useHTTPPath set.
That may be acceptable if it makes things more convenient. Those are all
pretty obscure cases, and I find it hard to believe an attacker could
hijack credentials using this (it implies that the only difference
between their malicious url and a known-good one is a trailing CR).
This part of the commit message confused me a little:
> We know credential filling should be finished by empty input, but the
> current implementation does not take into account CR/LF ending, and
> hence instead of the empty string we get '\r', which is interpreted as
> an incorrect string.
If all we care about is the empty line, and not data lines, then we
could do this:
diff --git a/credential.c b/credential.c
index efc29dc5e1..73143c5ed0 100644
--- a/credential.c
+++ b/credential.c
@@ -206,7 +206,7 @@ int credential_read(struct credential *c, FILE *fp)
char *key = line.buf;
char *value = strchr(key, '=');
- if (!line.len)
+ if (!line.len || (line.len == 1 && line.buf[0] == '\r'))
break;
if (!value) {
without impacting the ability to send raw CR in the lines with actual
data. But I imagine that a trailing CR in all of the data would also
cause problems.
-Peff
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF
2020-09-29 0:42 ` Jeff King
@ 2020-10-02 11:37 ` Johannes Schindelin
2020-10-02 12:01 ` Jeff King
2020-10-02 16:32 ` Junio C Hamano
0 siblings, 2 replies; 32+ messages in thread
From: Johannes Schindelin @ 2020-10-02 11:37 UTC (permalink / raw)
To: Jeff King; +Cc: Nikita Leonov via GitGitGadget, git, Nikita Leonov
Hi Peff,
On Mon, 28 Sep 2020, Jeff King wrote:
> On Mon, Sep 28, 2020 at 11:40:22AM +0000, Nikita Leonov via GitGitGadget wrote:
>
> > From: Nikita Leonov <nykyta.leonov@gmail.com>
> >
> > This fix makes using Git credentials more friendly to Windows users. In
> > previous version it was unable to finish input correctly without
> > configuration changes (tested in PowerShell, CMD, Cygwin).
> >
> > We know credential filling should be finished by empty input, but the
> > current implementation does not take into account CR/LF ending, and
> > hence instead of the empty string we get '\r', which is interpreted as
> > an incorrect string.
> >
> > So this commit changes default reading function to a more Windows
> > compatible reading function.
>
> Unlike the credential-store file case, where we expect the data to be
> URL-encoded anyway (and so any true "\r" in the data would not be found
> in raw form), this means that the credential protocol can no longer
> represent "\r" at the end of a value.
Indeed.
> And we'd match "example.com\r" and "example.com" as the same (unlikely,
> since carriage returns aren't allowed in hostnames, and curl will
> complain about this). We'd also match "cert://some/path\r" and
> "cert://some/path". Or "https://example.com/path\r" and its match, if
> you have credential.useHTTPPath set.
True. It's a problem with all of those URLs that end in Carriage Returns
;-)
> That may be acceptable if it makes things more convenient. Those are all
> pretty obscure cases, and I find it hard to believe an attacker could
> hijack credentials using this (it implies that the only difference
> between their malicious url and a known-good one is a trailing CR).
Indeed.
> This part of the commit message confused me a little:
>
> > We know credential filling should be finished by empty input, but the
> > current implementation does not take into account CR/LF ending, and
> > hence instead of the empty string we get '\r', which is interpreted as
> > an incorrect string.
>
> If all we care about is the empty line, and not data lines, then we
> could do this:
>
> diff --git a/credential.c b/credential.c
> index efc29dc5e1..73143c5ed0 100644
> --- a/credential.c
> +++ b/credential.c
> @@ -206,7 +206,7 @@ int credential_read(struct credential *c, FILE *fp)
> char *key = line.buf;
> char *value = strchr(key, '=');
>
> - if (!line.len)
> + if (!line.len || (line.len == 1 && line.buf[0] == '\r'))
> break;
>
> if (!value) {
>
> without impacting the ability to send raw CR in the lines with actual
> data. But I imagine that a trailing CR in all of the data would also
> cause problems.
I checked again with github.com/git-for-windows/git/pull/2516, where the
patch originally entered the public eye, but could not find any background
information.
But I would highly doubt that the empty lines were the biggest problem:
Sure, we would fail to recognize an empty line with CR/LF line endings
when reading with `strbuf_getline_lf()`, but we would totally
misunderstand the entire rest of the lines, too. For example, we would
mistake `quit\r` for an unknown command, and hence simply ignore it.
I do agree, however, that your confusion validly points out a flaw in the
commit message: the "empty line" comment is a red herring.
Therefore, I spent some time pouring over the commit message. This is my
current version:
credential: treat CR/LF as line endings in the credential protocol
This fix makes using Git credentials more friendly to Windows users: it
allows a credential helper to communicate using CR/LF line endings ("DOS
line endings" commonly found on Windows) instead of LF-only line endings
("Unix line endings").
Note that this changes the behavior a bit: if a credential helper
produces, say, a password with a trailing Carriage Return character,
that will now be culled even when the rest of the lines end only in Line
Feed characters, indicating that the Carriage Return was not meant to be
part of the line ending.
In practice, it seems _very_ unlikely that something like this happens.
Passwords usually need to consist of non-control characters, URLs need
to have special characters URL-encoded, and user names, well, are names.
So let's change the credential machinery to accept both CR/LF and LF
line endings.
While we do this for the credential helper protocol, we do _not_ do
adjust `git credential-cache--daemon` (which won't work on Windows,
anyway, because it requires Unix sockets) nor `git credential-store`
(which writes the file `~/.git-credentials` which we consider an
implementation detail that should be opaque to the user, read: we do
expect users _not_ to edit this file manually).
What do you think?
Ciao,
Dscho
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF
2020-10-02 11:37 ` Johannes Schindelin
@ 2020-10-02 12:01 ` Jeff King
2020-10-02 12:27 ` Johannes Schindelin
2020-10-02 16:32 ` Junio C Hamano
1 sibling, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-10-02 12:01 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Nikita Leonov via GitGitGadget, git, Nikita Leonov
On Fri, Oct 02, 2020 at 01:37:23PM +0200, Johannes Schindelin wrote:
> But I would highly doubt that the empty lines were the biggest problem:
> Sure, we would fail to recognize an empty line with CR/LF line endings
> when reading with `strbuf_getline_lf()`, but we would totally
> misunderstand the entire rest of the lines, too. For example, we would
> mistake `quit\r` for an unknown command, and hence simply ignore it.
>
> I do agree, however, that your confusion validly points out a flaw in the
> commit message: the "empty line" comment is a red herring.
>
> Therefore, I spent some time pouring over the commit message. This is my
> current version:
> [...]
> What do you think?
I think we are on the same page, and this revision does a good job of
fixing my complaint about the commit message. Thanks. One minor typo:
> While we do this for the credential helper protocol, we do _not_ do
> adjust `git credential-cache--daemon` (which won't work on Windows,
s/do$//
-Peff
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF
2020-10-02 12:01 ` Jeff King
@ 2020-10-02 12:27 ` Johannes Schindelin
0 siblings, 0 replies; 32+ messages in thread
From: Johannes Schindelin @ 2020-10-02 12:27 UTC (permalink / raw)
To: Jeff King; +Cc: Nikita Leonov via GitGitGadget, git, Nikita Leonov
Hi Peff,
On Fri, 2 Oct 2020, Jeff King wrote:
> On Fri, Oct 02, 2020 at 01:37:23PM +0200, Johannes Schindelin wrote:
>
> > But I would highly doubt that the empty lines were the biggest problem:
> > Sure, we would fail to recognize an empty line with CR/LF line endings
> > when reading with `strbuf_getline_lf()`, but we would totally
> > misunderstand the entire rest of the lines, too. For example, we would
> > mistake `quit\r` for an unknown command, and hence simply ignore it.
> >
> > I do agree, however, that your confusion validly points out a flaw in the
> > commit message: the "empty line" comment is a red herring.
> >
> > Therefore, I spent some time pouring over the commit message. This is my
> > current version:
> > [...]
> > What do you think?
>
> I think we are on the same page, and this revision does a good job of
> fixing my complaint about the commit message. Thanks. One minor typo:
>
> > While we do this for the credential helper protocol, we do _not_ do
> > adjust `git credential-cache--daemon` (which won't work on Windows,
>
> s/do$//
Thanks, fixed!
Ciao,
Dscho
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF
2020-10-02 11:37 ` Johannes Schindelin
2020-10-02 12:01 ` Jeff King
@ 2020-10-02 16:32 ` Junio C Hamano
2020-10-03 13:28 ` Johannes Schindelin
1 sibling, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-10-02 16:32 UTC (permalink / raw)
To: Johannes Schindelin
Cc: Jeff King, Nikita Leonov via GitGitGadget, git, Nikita Leonov
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Therefore, I spent some time pouring over the commit message. This is my
> current version:
>
> credential: treat CR/LF as line endings in the credential protocol
>
> This fix makes using Git credentials more friendly to Windows users: it
> allows a credential helper to communicate using CR/LF line endings ("DOS
> line endings" commonly found on Windows) instead of LF-only line endings
> ("Unix line endings").
>
> Note that this changes the behavior a bit: if a credential helper
> produces, say, a password with a trailing Carriage Return character,
> that will now be culled even when the rest of the lines end only in Line
> Feed characters, indicating that the Carriage Return was not meant to be
> part of the line ending.
>
> In practice, it seems _very_ unlikely that something like this happens.
> Passwords usually need to consist of non-control characters, URLs need
> to have special characters URL-encoded, and user names, well, are names.
>
> So let's change the credential machinery to accept both CR/LF and LF
> line endings.
>
> While we do this for the credential helper protocol, we do _not_ do
> adjust `git credential-cache--daemon` (which won't work on Windows,
> anyway, because it requires Unix sockets) nor `git credential-store`
> (which writes the file `~/.git-credentials` which we consider an
> implementation detail that should be opaque to the user, read: we do
> expect users _not_ to edit this file manually).
>
> What do you think?
I am not Peff, but I was also drawn into the same confusion by the
"we never see an empty line" red herring.
There are some micronits, but the above made a lot easier to
understand (I think you could even add "quit\r" bit to make it even
easier to understand) than the original description.
Thanks.
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF
2020-10-02 16:32 ` Junio C Hamano
@ 2020-10-03 13:28 ` Johannes Schindelin
0 siblings, 0 replies; 32+ messages in thread
From: Johannes Schindelin @ 2020-10-03 13:28 UTC (permalink / raw)
To: Junio C Hamano
Cc: Jeff King, Nikita Leonov via GitGitGadget, git, Nikita Leonov
Hi Junio,
On Fri, 2 Oct 2020, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > Therefore, I spent some time pouring over the commit message. This is my
> > current version:
> >
> > credential: treat CR/LF as line endings in the credential protocol
> >
> > This fix makes using Git credentials more friendly to Windows users: it
> > allows a credential helper to communicate using CR/LF line endings ("DOS
> > line endings" commonly found on Windows) instead of LF-only line endings
> > ("Unix line endings").
> >
> > Note that this changes the behavior a bit: if a credential helper
> > produces, say, a password with a trailing Carriage Return character,
> > that will now be culled even when the rest of the lines end only in Line
> > Feed characters, indicating that the Carriage Return was not meant to be
> > part of the line ending.
> >
> > In practice, it seems _very_ unlikely that something like this happens.
> > Passwords usually need to consist of non-control characters, URLs need
> > to have special characters URL-encoded, and user names, well, are names.
> >
> > So let's change the credential machinery to accept both CR/LF and LF
> > line endings.
> >
> > While we do this for the credential helper protocol, we do _not_ do
> > adjust `git credential-cache--daemon` (which won't work on Windows,
> > anyway, because it requires Unix sockets) nor `git credential-store`
> > (which writes the file `~/.git-credentials` which we consider an
> > implementation detail that should be opaque to the user, read: we do
> > expect users _not_ to edit this file manually).
> >
> > What do you think?
>
> I am not Peff, but I was also drawn into the same confusion by the
> "we never see an empty line" red herring.
:-)
> There are some micronits, but the above made a lot easier to
> understand (I think you could even add "quit\r" bit to make it even
> easier to understand) than the original description.
Okay, I incorporated a comment talking about `quit\r` and will submit
a new iteration right now.
Thanks,
Dscho
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-28 11:40 ` [PATCH v2 0/3] Prepare git credential to read input with DOS line endings Johannes Schindelin via GitGitGadget
2020-09-28 11:40 ` [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF Nikita Leonov via GitGitGadget
@ 2020-09-28 11:40 ` Nikita Leonov via GitGitGadget
2020-09-28 20:58 ` Junio C Hamano
2020-09-28 23:26 ` Carlo Arenas
2020-09-28 11:40 ` [PATCH v2 3/3] docs: make notes regarding credential line reading Nikita Leonov via GitGitGadget
` (2 subsequent siblings)
4 siblings, 2 replies; 32+ messages in thread
From: Nikita Leonov via GitGitGadget @ 2020-09-28 11:40 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Nikita Leonov
From: Nikita Leonov <nykyta.leonov@gmail.com>
This commit makes reading process regarding credentials compatible with
'CR/LF' line ending. It makes using git more convenient on systems like
Windows.
Signed-off-by: Nikita Leonov <nykyta.leonov@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
builtin/credential-cache--daemon.c | 4 ++--
builtin/credential-store.c | 2 +-
t/t0302-credential-store.sh | 16 ++++++----------
3 files changed, 9 insertions(+), 13 deletions(-)
diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c
index c61f123a3b..17664bb0d5 100644
--- a/builtin/credential-cache--daemon.c
+++ b/builtin/credential-cache--daemon.c
@@ -99,12 +99,12 @@ static int read_request(FILE *fh, struct credential *c,
static struct strbuf item = STRBUF_INIT;
const char *p;
- strbuf_getline_lf(&item, fh);
+ strbuf_getline(&item, fh);
if (!skip_prefix(item.buf, "action=", &p))
return error("client sent bogus action line: %s", item.buf);
strbuf_addstr(action, p);
- strbuf_getline_lf(&item, fh);
+ strbuf_getline(&item, fh);
if (!skip_prefix(item.buf, "timeout=", &p))
return error("client sent bogus timeout line: %s", item.buf);
*timeout = atoi(p);
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index 5331ab151a..d4e90b68df 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -23,7 +23,7 @@ static int parse_credential_file(const char *fn,
return found_credential;
}
- while (strbuf_getline_lf(&line, fh) != EOF) {
+ while (strbuf_getline(&line, fh) != EOF) {
if (!credential_from_url_gently(&entry, line.buf, 1) &&
entry.username && entry.password &&
credential_match(c, &entry)) {
diff --git a/t/t0302-credential-store.sh b/t/t0302-credential-store.sh
index 716bf1af9f..f2c672e4b6 100755
--- a/t/t0302-credential-store.sh
+++ b/t/t0302-credential-store.sh
@@ -142,7 +142,7 @@ invalid_credential_test "scheme" ://user:pass@example.com
invalid_credential_test "valid host/path" https://user:pass@
invalid_credential_test "username/password" https://pass@example.com
-test_expect_success 'get: credentials with DOS line endings are invalid' '
+test_expect_success 'get: credentials with DOS line endings are valid' '
printf "https://user:pass@example.com\r\n" >"$HOME/.git-credentials" &&
check fill store <<-\EOF
protocol=https
@@ -150,11 +150,9 @@ test_expect_success 'get: credentials with DOS line endings are invalid' '
--
protocol=https
host=example.com
- username=askpass-username
- password=askpass-password
+ username=user
+ password=pass
--
- askpass: Username for '\''https://example.com'\'':
- askpass: Password for '\''https://askpass-username@example.com'\'':
--
EOF
'
@@ -172,7 +170,7 @@ test_expect_success 'get: credentials with path and DOS line endings are valid'
EOF
'
-test_expect_success 'get: credentials with DOS line endings are invalid if path is relevant' '
+test_expect_success 'get: credentials with DOS line endings are valid if path is relevant' '
printf "https://user:pass@example.com/repo.git\r\n" >"$HOME/.git-credentials" &&
test_config credential.useHttpPath true &&
check fill store <<-\EOF
@@ -181,11 +179,9 @@ test_expect_success 'get: credentials with DOS line endings are invalid if path
protocol=https
host=example.com
path=repo.git
- username=askpass-username
- password=askpass-password
+ username=user
+ password=pass
--
- askpass: Username for '\''https://example.com/repo.git'\'':
- askpass: Password for '\''https://askpass-username@example.com/repo.git'\'':
--
EOF
'
--
gitgitgadget
^ permalink raw reply related [flat|nested] 32+ messages in thread* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-28 11:40 ` [PATCH v2 2/3] credentials: make line reading Windows compatible Nikita Leonov via GitGitGadget
@ 2020-09-28 20:58 ` Junio C Hamano
2020-09-29 0:35 ` Jeff King
2020-09-28 23:26 ` Carlo Arenas
1 sibling, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-09-28 20:58 UTC (permalink / raw)
To: Nikita Leonov via GitGitGadget; +Cc: git, Johannes Schindelin, Nikita Leonov
"Nikita Leonov via GitGitGadget" <gitgitgadget@gmail.com> writes:
> From: Nikita Leonov <nykyta.leonov@gmail.com>
>
> This commit makes reading process regarding credentials compatible with
> 'CR/LF' line ending. It makes using git more convenient on systems like
> Windows.
I can see why this is a good thing for "store" and the two updated
pieces of the test script demonstrate it very well.
But it is unclear why and how cache-daemon benefits from this change.
That needs to be justified.
As to the log message (this is 70% style, but there are
consequences), we tend not to say "This commit does X"---because
such a statement is often insufficient to illustrate if the commit
indeed does X, and explain why it is a good thing to do X in the
first place.
Instead, we first explain that the current system does not do X (in
present tense, so we do NOT say "previously we did not do X"), then
explain why doing X would be a good thing, and finally give an order
to the codebase to start doing X. For this change, it might look
like this:
The credential subsystem has assumed that lines always end with
LF. On a system whose text file ends with CRLF (e.g. Windows),
this assumption causes the CR at the end of the line as an extra
byte appended to the data, and worse, there is no way to send an
"empty line" to signal an end of a group of lines, because such
a line would be taken as a line with a lone CR on it.
Because credential-store writes to and reads from a text file on
disk, which users may manually edit with their editor and turn
the line-ending to CRLF.
Accept lines that end with CR/LF, as well as those that end with
LF.
Note that the credential-cache--daemon is not touched, because
it is only about interacting with in-core cache, and there is
nowhere CRLF comes into the picture.
Note: I didn't know why we need to touch the daemon, and that is why
I wrote the paragraph on it as such, but I expect that the real log
message to justify the change would be quite different and explain
why the daemon code needs the same change well---such a description
would not be "Note that", but would come before the "Accept lines
that end with..." and after the paragraph about credential-store.
Thanks.
> Signed-off-by: Nikita Leonov <nykyta.leonov@gmail.com>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> builtin/credential-cache--daemon.c | 4 ++--
> builtin/credential-store.c | 2 +-
> t/t0302-credential-store.sh | 16 ++++++----------
> 3 files changed, 9 insertions(+), 13 deletions(-)
>
> diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c
> index c61f123a3b..17664bb0d5 100644
> --- a/builtin/credential-cache--daemon.c
> +++ b/builtin/credential-cache--daemon.c
> @@ -99,12 +99,12 @@ static int read_request(FILE *fh, struct credential *c,
> static struct strbuf item = STRBUF_INIT;
> const char *p;
>
> - strbuf_getline_lf(&item, fh);
> + strbuf_getline(&item, fh);
> if (!skip_prefix(item.buf, "action=", &p))
> return error("client sent bogus action line: %s", item.buf);
> strbuf_addstr(action, p);
>
> - strbuf_getline_lf(&item, fh);
> + strbuf_getline(&item, fh);
> if (!skip_prefix(item.buf, "timeout=", &p))
> return error("client sent bogus timeout line: %s", item.buf);
> *timeout = atoi(p);
> diff --git a/builtin/credential-store.c b/builtin/credential-store.c
> index 5331ab151a..d4e90b68df 100644
> --- a/builtin/credential-store.c
> +++ b/builtin/credential-store.c
> @@ -23,7 +23,7 @@ static int parse_credential_file(const char *fn,
> return found_credential;
> }
>
> - while (strbuf_getline_lf(&line, fh) != EOF) {
> + while (strbuf_getline(&line, fh) != EOF) {
> if (!credential_from_url_gently(&entry, line.buf, 1) &&
> entry.username && entry.password &&
> credential_match(c, &entry)) {
> diff --git a/t/t0302-credential-store.sh b/t/t0302-credential-store.sh
> index 716bf1af9f..f2c672e4b6 100755
> --- a/t/t0302-credential-store.sh
> +++ b/t/t0302-credential-store.sh
> @@ -142,7 +142,7 @@ invalid_credential_test "scheme" ://user:pass@example.com
> invalid_credential_test "valid host/path" https://user:pass@
> invalid_credential_test "username/password" https://pass@example.com
>
> -test_expect_success 'get: credentials with DOS line endings are invalid' '
> +test_expect_success 'get: credentials with DOS line endings are valid' '
> printf "https://user:pass@example.com\r\n" >"$HOME/.git-credentials" &&
> check fill store <<-\EOF
> protocol=https
> @@ -150,11 +150,9 @@ test_expect_success 'get: credentials with DOS line endings are invalid' '
> --
> protocol=https
> host=example.com
> - username=askpass-username
> - password=askpass-password
> + username=user
> + password=pass
> --
> - askpass: Username for '\''https://example.com'\'':
> - askpass: Password for '\''https://askpass-username@example.com'\'':
> --
> EOF
> '
> @@ -172,7 +170,7 @@ test_expect_success 'get: credentials with path and DOS line endings are valid'
> EOF
> '
>
> -test_expect_success 'get: credentials with DOS line endings are invalid if path is relevant' '
> +test_expect_success 'get: credentials with DOS line endings are valid if path is relevant' '
> printf "https://user:pass@example.com/repo.git\r\n" >"$HOME/.git-credentials" &&
> test_config credential.useHttpPath true &&
> check fill store <<-\EOF
> @@ -181,11 +179,9 @@ test_expect_success 'get: credentials with DOS line endings are invalid if path
> protocol=https
> host=example.com
> path=repo.git
> - username=askpass-username
> - password=askpass-password
> + username=user
> + password=pass
> --
> - askpass: Username for '\''https://example.com/repo.git'\'':
> - askpass: Password for '\''https://askpass-username@example.com/repo.git'\'':
> --
> EOF
> '
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-28 20:58 ` Junio C Hamano
@ 2020-09-29 0:35 ` Jeff King
2020-09-30 22:33 ` Junio C Hamano
0 siblings, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-09-29 0:35 UTC (permalink / raw)
To: Junio C Hamano
Cc: Nikita Leonov via GitGitGadget, git, Johannes Schindelin, Nikita Leonov
On Mon, Sep 28, 2020 at 01:58:03PM -0700, Junio C Hamano wrote:
> "Nikita Leonov via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Nikita Leonov <nykyta.leonov@gmail.com>
> >
> > This commit makes reading process regarding credentials compatible with
> > 'CR/LF' line ending. It makes using git more convenient on systems like
> > Windows.
>
> I can see why this is a good thing for "store" and the two updated
> pieces of the test script demonstrate it very well.
>
> But it is unclear why and how cache-daemon benefits from this change.
> That needs to be justified.
I suspect it doesn't need touched, because it is internal to git-daemon.
But it does handle CRLF for some lines, because the first patch
modified credential_read(), which the daemon builds on (and which _is_
user-facing via git-credential). So there's perhaps an argument that
these calls should just be made consistent, even though the only one who
would write them is our matching client. If that is the argument to be
made, I think it would make sense to do so in a separate patch, since
there's no functional change.
(I'm also slightly puzzled that anybody on Windows would care about
credential-cache, since it require unix sockets. But maybe in a world of
WSL people are actually able to mix the two. I confess I haven't kept up
with the state of things in Windows).
-Peff
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-29 0:35 ` Jeff King
@ 2020-09-30 22:33 ` Junio C Hamano
2020-10-02 7:53 ` Johannes Schindelin
0 siblings, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-09-30 22:33 UTC (permalink / raw)
To: Jeff King
Cc: Nikita Leonov via GitGitGadget, git, Johannes Schindelin, Nikita Leonov
Jeff King <peff@peff.net> writes:
> On Mon, Sep 28, 2020 at 01:58:03PM -0700, Junio C Hamano wrote:
>
>> "Nikita Leonov via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>> > From: Nikita Leonov <nykyta.leonov@gmail.com>
>> >
>> > This commit makes reading process regarding credentials compatible with
>> > 'CR/LF' line ending. It makes using git more convenient on systems like
>> > Windows.
>>
>> I can see why this is a good thing for "store" and the two updated
>> pieces of the test script demonstrate it very well.
>>
>> But it is unclear why and how cache-daemon benefits from this change.
>> That needs to be justified.
>
> I suspect it doesn't need touched, because it is internal to git-daemon.
> But it does handle CRLF for some lines, because the first patch
> modified credential_read(), which the daemon builds on (and which _is_
> user-facing via git-credential). So there's perhaps an argument that
> these calls should just be made consistent, even though the only one who
> would write them is our matching client. If that is the argument to be
> made, I think it would make sense to do so in a separate patch, since
> there's no functional change.
>
> (I'm also slightly puzzled that anybody on Windows would care about
> credential-cache, since it require unix sockets. But maybe in a world of
> WSL people are actually able to mix the two. I confess I haven't kept up
> with the state of things in Windows).
True, so some, if not all, parts of these changes start to look more
and more like "I change LF to CRLF purely for political correctness,
even I know nobody sends CRLF in these cases (or "even I do not know
if anybody sends CRLF in these cases", which essentially amounts to
the same thing but may be worse)", needless code churns.
Sigh.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-30 22:33 ` Junio C Hamano
@ 2020-10-02 7:53 ` Johannes Schindelin
0 siblings, 0 replies; 32+ messages in thread
From: Johannes Schindelin @ 2020-10-02 7:53 UTC (permalink / raw)
To: Junio C Hamano
Cc: Jeff King, Nikita Leonov via GitGitGadget, git, Nikita Leonov
Hi Junio & Peff,
On Wed, 30 Sep 2020, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > On Mon, Sep 28, 2020 at 01:58:03PM -0700, Junio C Hamano wrote:
> >
> >> "Nikita Leonov via GitGitGadget" <gitgitgadget@gmail.com> writes:
> >>
> >> > From: Nikita Leonov <nykyta.leonov@gmail.com>
> >> >
> >> > This commit makes reading process regarding credentials compatible with
> >> > 'CR/LF' line ending. It makes using git more convenient on systems like
> >> > Windows.
> >>
> >> I can see why this is a good thing for "store" and the two updated
> >> pieces of the test script demonstrate it very well.
> >>
> >> But it is unclear why and how cache-daemon benefits from this change.
> >> That needs to be justified.
> >
> > I suspect it doesn't need touched, because it is internal to git-daemon.
> > But it does handle CRLF for some lines, because the first patch
> > modified credential_read(), which the daemon builds on (and which _is_
> > user-facing via git-credential). So there's perhaps an argument that
> > these calls should just be made consistent, even though the only one who
> > would write them is our matching client. If that is the argument to be
> > made, I think it would make sense to do so in a separate patch, since
> > there's no functional change.
> >
> > (I'm also slightly puzzled that anybody on Windows would care about
> > credential-cache, since it require unix sockets. But maybe in a world of
> > WSL people are actually able to mix the two. I confess I haven't kept up
> > with the state of things in Windows).
>
> True, so some, if not all, parts of these changes start to look more
> and more like "I change LF to CRLF purely for political correctness,
> even I know nobody sends CRLF in these cases (or "even I do not know
> if anybody sends CRLF in these cases", which essentially amounts to
> the same thing but may be worse)", needless code churns.
Indeed. I did not make the connection that `credential-cache--daemon` uses
Unix sockets internally (and hence if you try to use it on Windows, you
are greeted with the message "credential-cache--daemon unavailable; no
unix socket support").
Will drop this part from the next iteration.
Ciao,
Dscho
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-28 11:40 ` [PATCH v2 2/3] credentials: make line reading Windows compatible Nikita Leonov via GitGitGadget
2020-09-28 20:58 ` Junio C Hamano
@ 2020-09-28 23:26 ` Carlo Arenas
2020-09-28 23:41 ` Junio C Hamano
2020-09-29 0:30 ` Jeff King
1 sibling, 2 replies; 32+ messages in thread
From: Carlo Arenas @ 2020-09-28 23:26 UTC (permalink / raw)
To: Nikita Leonov via GitGitGadget
Cc: git, Johannes Schindelin, Nikita Leonov, peff
On Mon, Sep 28, 2020 at 4:41 AM Nikita Leonov via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> diff --git a/t/t0302-credential-store.sh b/t/t0302-credential-store.sh
> index 716bf1af9f..f2c672e4b6 100755
> --- a/t/t0302-credential-store.sh
> +++ b/t/t0302-credential-store.sh
> @@ -142,7 +142,7 @@ invalid_credential_test "scheme" ://user:pass@example.com
> invalid_credential_test "valid host/path" https://user:pass@
> invalid_credential_test "username/password" https://pass@example.com
>
> -test_expect_success 'get: credentials with DOS line endings are invalid' '
> +test_expect_success 'get: credentials with DOS line endings are valid' '
> printf "https://user:pass@example.com\r\n" >"$HOME/.git-credentials" &&
> check fill store <<-\EOF
> protocol=https
> @@ -150,11 +150,9 @@ test_expect_success 'get: credentials with DOS line endings are invalid' '
> --
> protocol=https
> host=example.com
> - username=askpass-username
> - password=askpass-password
> + username=user
> + password=pass
> --
> - askpass: Username for '\''https://example.com'\'':
> - askpass: Password for '\''https://askpass-username@example.com'\'':
> --
> EOF
> '
> @@ -172,7 +170,7 @@ test_expect_success 'get: credentials with path and DOS line endings are valid'
> EOF
> '
>
> -test_expect_success 'get: credentials with DOS line endings are invalid if path is relevant' '
> +test_expect_success 'get: credentials with DOS line endings are valid if path is relevant' '
note that this test was put in place to protect users from regressions
like the one we got after the release of 2.26.1 where users that had
'\r' as part of their credentials were getting an error[1]
while I am sympathetic to the change (indeed I proposed something
similar, but was reminded by Peff that while it looks like a text file
it was designed to be considered opaque and therefore should use UNIX
LF as record terminator by specification), I am concerned this could
result in a similar regression since we know they are still users out
there that had modified this file manually (something that was not
recommended) and are currently relying on the fact that these lines
are invalid and therefore silently ignored.
Carlo
[1] https://lore.kernel.org/git/ad80aa0d-3a35-6d7e-7958-b3520e16c855@ed4u.de/
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-28 23:26 ` Carlo Arenas
@ 2020-09-28 23:41 ` Junio C Hamano
2020-09-29 0:30 ` Jeff King
1 sibling, 0 replies; 32+ messages in thread
From: Junio C Hamano @ 2020-09-28 23:41 UTC (permalink / raw)
To: Carlo Arenas
Cc: Nikita Leonov via GitGitGadget, git, Johannes Schindelin,
Nikita Leonov, peff
Carlo Arenas <carenas@gmail.com> writes:
>> -test_expect_success 'get: credentials with DOS line endings are invalid if path is relevant' '
>> +test_expect_success 'get: credentials with DOS line endings are valid if path is relevant' '
>
> note that this test was put in place to protect users from regressions
> like the one we got after the release of 2.26.1 where users that had
> '\r' as part of their credentials were getting an error[1]
>
> while I am sympathetic to the change (indeed I proposed something
> similar, but was reminded by Peff that while it looks like a text file
> it was designed to be considered opaque and therefore should use UNIX
> LF as record terminator by specification), I am concerned this could
> result in a similar regression since we know they are still users out
> there that had modified this file manually (something that was not
> recommended) and are currently relying on the fact that these lines
> are invalid and therefore silently ignored.
>
> Carlo
>
> [1] https://lore.kernel.org/git/ad80aa0d-3a35-6d7e-7958-b3520e16c855@ed4u.de/
I think you meant to remind us and this thread of the earlier review
and discussion thread, which begins at
https://lore.kernel.org/git/20200426234750.40418-1-carenas@gmail.com/
And thanks for doing so---I totally forgot about it.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-28 23:26 ` Carlo Arenas
2020-09-28 23:41 ` Junio C Hamano
@ 2020-09-29 0:30 ` Jeff King
2020-09-29 0:41 ` Junio C Hamano
1 sibling, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-09-29 0:30 UTC (permalink / raw)
To: Carlo Arenas
Cc: Nikita Leonov via GitGitGadget, git, Johannes Schindelin, Nikita Leonov
On Mon, Sep 28, 2020 at 04:26:38PM -0700, Carlo Arenas wrote:
> > -test_expect_success 'get: credentials with DOS line endings are invalid if path is relevant' '
> > +test_expect_success 'get: credentials with DOS line endings are valid if path is relevant' '
>
> note that this test was put in place to protect users from regressions
> like the one we got after the release of 2.26.1 where users that had
> '\r' as part of their credentials were getting an error[1]
>
> while I am sympathetic to the change (indeed I proposed something
> similar, but was reminded by Peff that while it looks like a text file
> it was designed to be considered opaque and therefore should use UNIX
> LF as record terminator by specification), I am concerned this could
> result in a similar regression since we know they are still users out
> there that had modified this file manually (something that was not
> recommended) and are currently relying on the fact that these lines
> are invalid and therefore silently ignored.
Going over that old thread, I'm not sure that anybody raised a real use
case where somebody expected a CR at the end of a line. The discussion
was mostly about whether proposed changes would or would not be
compatible with existing behavior.
I think that:
- we'd never write a raw CR ourselves, as we'd urlencode the character
- if somebody did put in a raw CR manually like:
https://example.com\r\n
then we'd currently fail to match "example.com". Which is probably
not what they wanted. I suspect that \r in a hostname is bogus
anyway (certainly curl will complain about it).
- you could do the same in a path:
https://example.com/foo\r\n
which _is_ legal, but I think we'd generally ignore it completely
unless credential.usehttppath is set (for network-accessible
requests, that is; you could probably point your local cert file at
something bogus, but I think the main areas of interest here are
people manipulating the credential protocol via malicious urls).
So I'm OK loosening the format in the name of convenience, as long as
we're confident that it's not opening up any security problems. I can't
think of any such problems, though.
It sounds from your email like you may have found me arguing the
opposite. That's entirely possible. ;) But I couldn't find it in the
thread Junio linked.
-Peff
^ permalink raw reply [flat|nested] 32+ messages in thread* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-29 0:30 ` Jeff King
@ 2020-09-29 0:41 ` Junio C Hamano
2020-09-29 0:44 ` Jeff King
0 siblings, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-09-29 0:41 UTC (permalink / raw)
To: Jeff King
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
Jeff King <peff@peff.net> writes:
> I think that:
>
> - we'd never write a raw CR ourselves, as we'd urlencode the character
>
> - if somebody did put in a raw CR manually like:
>
> https://example.com\r\n
>
> then we'd currently fail to match "example.com". Which is probably
> not what they wanted. I suspect that \r in a hostname is bogus
> anyway (certainly curl will complain about it).
I may be misremembering, but an argument I recall against the kind
of change we are dicussing now was that we ignore such an entry
right now, and the user may have added an entry for the host anew,
possibly with a more recent password. Changing the parsing to
ignore CR would silently resurrect such a stale entry that the user
has written off as unused, and depending on the order of entries in
the file, a site that used to work may start failing suddenly.
I don't think I'd care too heavily either way. As long as other
people will deal with possible backward-incompatibility fallout,
I do not mind seeing the change ;-).
I still don't see why we need to touch the cache-daemon, though.
Thanks.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-29 0:41 ` Junio C Hamano
@ 2020-09-29 0:44 ` Jeff King
2020-09-29 0:54 ` Junio C Hamano
0 siblings, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-09-29 0:44 UTC (permalink / raw)
To: Junio C Hamano
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
On Mon, Sep 28, 2020 at 05:41:14PM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > I think that:
> >
> > - we'd never write a raw CR ourselves, as we'd urlencode the character
> >
> > - if somebody did put in a raw CR manually like:
> >
> > https://example.com\r\n
> >
> > then we'd currently fail to match "example.com". Which is probably
> > not what they wanted. I suspect that \r in a hostname is bogus
> > anyway (certainly curl will complain about it).
>
> I may be misremembering, but an argument I recall against the kind
> of change we are dicussing now was that we ignore such an entry
> right now, and the user may have added an entry for the host anew,
> possibly with a more recent password. Changing the parsing to
> ignore CR would silently resurrect such a stale entry that the user
> has written off as unused, and depending on the order of entries in
> the file, a site that used to work may start failing suddenly.
Yeah, that is probably what would happen. I have to admit that it's such
an obscure case that I'm not sure I really care. It's unlikely in
practice, and if somebody did report such a case, I think my first
response would be "well, why did you have a broken entry stuck in your
file?".
> I still don't see why we need to touch the cache-daemon, though.
Yeah, I touched on that more in another response.
-Peff
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-29 0:44 ` Jeff King
@ 2020-09-29 0:54 ` Junio C Hamano
2020-09-29 3:00 ` Jeff King
0 siblings, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-09-29 0:54 UTC (permalink / raw)
To: Jeff King
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
Jeff King <peff@peff.net> writes:
> Yeah, that is probably what would happen. I have to admit that it's such
> an obscure case that I'm not sure I really care. It's unlikely in
> practice, and if somebody did report such a case, I think my first
> response would be "well, why did you have a broken entry stuck in your
> file?".
I think we know the likely answer. "I once used Windows to edit the
file manually".
After which the file looks broken, so the user may have re-added via
the credential API (with LF line ending) a new entry for the host
and have been happily using the result.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-29 0:54 ` Junio C Hamano
@ 2020-09-29 3:00 ` Jeff King
2020-09-30 22:25 ` Junio C Hamano
0 siblings, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-09-29 3:00 UTC (permalink / raw)
To: Junio C Hamano
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
On Mon, Sep 28, 2020 at 05:54:37PM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Yeah, that is probably what would happen. I have to admit that it's such
> > an obscure case that I'm not sure I really care. It's unlikely in
> > practice, and if somebody did report such a case, I think my first
> > response would be "well, why did you have a broken entry stuck in your
> > file?".
>
> I think we know the likely answer. "I once used Windows to edit the
> file manually".
>
> After which the file looks broken, so the user may have re-added via
> the credential API (with LF line ending) a new entry for the host
> and have been happily using the result.
I wrote something less charitable towards the user at first, and then
toned it down. But I think I toned it down too much. Maybe my response
would be "garbage in, garbage out; it was lucky that it worked before".
-Peff
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-29 3:00 ` Jeff King
@ 2020-09-30 22:25 ` Junio C Hamano
2020-09-30 22:39 ` Jeff King
0 siblings, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-09-30 22:25 UTC (permalink / raw)
To: Jeff King
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
Jeff King <peff@peff.net> writes:
> On Mon, Sep 28, 2020 at 05:54:37PM -0700, Junio C Hamano wrote:
>
>> Jeff King <peff@peff.net> writes:
>>
>> > Yeah, that is probably what would happen. I have to admit that it's such
>> > an obscure case that I'm not sure I really care. It's unlikely in
>> > practice, and if somebody did report such a case, I think my first
>> > response would be "well, why did you have a broken entry stuck in your
>> > file?".
>>
>> I think we know the likely answer. "I once used Windows to edit the
>> file manually".
>>
>> After which the file looks broken, so the user may have re-added via
>> the credential API (with LF line ending) a new entry for the host
>> and have been happily using the result.
>
> I wrote something less charitable towards the user at first, and then
> toned it down. But I think I toned it down too much. Maybe my response
> would be "garbage in, garbage out; it was lucky that it worked before".
OK, so what's the final verdict from us? This is good enough to
move forward as-is?
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-30 22:25 ` Junio C Hamano
@ 2020-09-30 22:39 ` Jeff King
2020-09-30 22:56 ` Junio C Hamano
0 siblings, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-09-30 22:39 UTC (permalink / raw)
To: Junio C Hamano
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
On Wed, Sep 30, 2020 at 03:25:09PM -0700, Junio C Hamano wrote:
> OK, so what's the final verdict from us? This is good enough to
> move forward as-is?
I'd prefer to see the credential-cache--daemon changes either dropped,
or split out into a separate patch with the justification of: this
probably doesn't matter in practice, but it makes the whole protocol
between client and server treat CRLF consistently.
I had also raised a question in:
https://lore.kernel.org/git/20200929004220.GC898702@coredump.intra.peff.net/
about whether they just care about the empty line, or about CRLF on data
lines. If the former (which is what the commit message claims), then I
think we can do something much simpler. But I suspect it is the latter,
and it is simply the commit message that is a bit misleading.
Other than those nits, I think the series is OK.
-Peff
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-30 22:39 ` Jeff King
@ 2020-09-30 22:56 ` Junio C Hamano
2020-10-01 13:54 ` Jeff King
0 siblings, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-09-30 22:56 UTC (permalink / raw)
To: Jeff King
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
Jeff King <peff@peff.net> writes:
> On Wed, Sep 30, 2020 at 03:25:09PM -0700, Junio C Hamano wrote:
>
>> OK, so what's the final verdict from us? This is good enough to
>> move forward as-is?
>
> I'd prefer to see the credential-cache--daemon changes either dropped,
> or split out into a separate patch with the justification of: this
> probably doesn't matter in practice, but it makes the whole protocol
> between client and server treat CRLF consistently.
>
> I had also raised a question in:
>
> https://lore.kernel.org/git/20200929004220.GC898702@coredump.intra.peff.net/
>
> about whether they just care about the empty line, or about CRLF on data
> lines. If the former (which is what the commit message claims), then I
> think we can do something much simpler. But I suspect it is the latter,
> and it is simply the commit message that is a bit misleading.
>
> Other than those nits, I think the series is OK.
Sure. But credential-store side is also iffy; it is not like they
want CRLF on data lines (if they want CR in data, that needs to be
encoded). The only reason I can think of that the change to
"-store" makes any difference is when people edit it, but the file
is not designed to be manually edited, so even that part of the
series needs a better justification. It's not like "We want to be
compatible" without "why it is better to be compatible" is a good
rationale, when we define the file format not to be manually edited
in the first place.
Thanks.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-09-30 22:56 ` Junio C Hamano
@ 2020-10-01 13:54 ` Jeff King
2020-10-01 15:42 ` Junio C Hamano
0 siblings, 1 reply; 32+ messages in thread
From: Jeff King @ 2020-10-01 13:54 UTC (permalink / raw)
To: Junio C Hamano
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
On Wed, Sep 30, 2020 at 03:56:27PM -0700, Junio C Hamano wrote:
> > Other than those nits, I think the series is OK.
>
> Sure. But credential-store side is also iffy; it is not like they
> want CRLF on data lines (if they want CR in data, that needs to be
> encoded). The only reason I can think of that the change to
> "-store" makes any difference is when people edit it, but the file
> is not designed to be manually edited, so even that part of the
> series needs a better justification. It's not like "We want to be
> compatible" without "why it is better to be compatible" is a good
> rationale, when we define the file format not to be manually edited
> in the first place.
Yeah, I agree that just teaching git-credential's stdin to handle CR
would be an OK stopping point. I don't have a strong opinion on
credential-store's on disk format. At least allowing CRLF there is
_plausibly_ useful, unlike credential-cache--daemon's pipe. And I doubt
that making the change would hurt anybody.
-Peff
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-10-01 13:54 ` Jeff King
@ 2020-10-01 15:42 ` Junio C Hamano
2020-10-02 8:07 ` Johannes Schindelin
0 siblings, 1 reply; 32+ messages in thread
From: Junio C Hamano @ 2020-10-01 15:42 UTC (permalink / raw)
To: Jeff King
Cc: Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Johannes Schindelin, Nikita Leonov
Jeff King <peff@peff.net> writes:
> Yeah, I agree that just teaching git-credential's stdin to handle CR
> would be an OK stopping point. I don't have a strong opinion on
> credential-store's on disk format. At least allowing CRLF there is
> _plausibly_ useful, unlike credential-cache--daemon's pipe. And I doubt
> that making the change would hurt anybody.
I agree 100% with all of the above, including the part that says it
is also OK to let credential-store read from its files with line
ending converted to CRLF.
Thanks.
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v2 2/3] credentials: make line reading Windows compatible
2020-10-01 15:42 ` Junio C Hamano
@ 2020-10-02 8:07 ` Johannes Schindelin
0 siblings, 0 replies; 32+ messages in thread
From: Johannes Schindelin @ 2020-10-02 8:07 UTC (permalink / raw)
To: Junio C Hamano
Cc: Jeff King, Carlo Arenas, Nikita Leonov via GitGitGadget, git,
Nikita Leonov
Hi,
On Thu, 1 Oct 2020, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Yeah, I agree that just teaching git-credential's stdin to handle CR
> > would be an OK stopping point. I don't have a strong opinion on
> > credential-store's on disk format. At least allowing CRLF there is
> > _plausibly_ useful, unlike credential-cache--daemon's pipe. And I doubt
> > that making the change would hurt anybody.
>
> I agree 100% with all of the above, including the part that says it
> is also OK to let credential-store read from its files with line
> ending converted to CRLF.
After mulling over this, I am more inclined to drop the credential-store
changes, too. That file _is_ an implementation detail, and not intended
for interactive editing. And as Carlo pointed out so correctly, the
regression tests were not introduced "just for fun", they really want to
make sure that we keep handling CR/LF the way we currently do.
So I will drop both credential-cache--daemon and credential-store changes
from the next iteration.
Ciao,
Dscho
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v2 3/3] docs: make notes regarding credential line reading
2020-09-28 11:40 ` [PATCH v2 0/3] Prepare git credential to read input with DOS line endings Johannes Schindelin via GitGitGadget
2020-09-28 11:40 ` [PATCH v2 1/3] credential.c: fix credential reading with regards to CR/LF Nikita Leonov via GitGitGadget
2020-09-28 11:40 ` [PATCH v2 2/3] credentials: make line reading Windows compatible Nikita Leonov via GitGitGadget
@ 2020-09-28 11:40 ` Nikita Leonov via GitGitGadget
2020-09-28 20:31 ` [PATCH v2 0/3] Prepare git credential to read input with DOS line endings Junio C Hamano
2020-10-03 13:29 ` [PATCH v3] credential: treat CR/LF as line endings in the credential protocol Johannes Schindelin via GitGitGadget
4 siblings, 0 replies; 32+ messages in thread
From: Nikita Leonov via GitGitGadget @ 2020-09-28 11:40 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Nikita Leonov
From: Nikita Leonov <nykyta.leonov@gmail.com>
This commit adds notes to git-credential.txt and to gitcredentials.txt
specifying that 'LF' and 'CR/LF' endings are treated the same.
Signed-off-by: Nikita Leonov <nykyta.leonov@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Documentation/git-credential.txt | 4 +++-
Documentation/gitcredentials.txt | 2 ++
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt
index 31c81c4c02..9cf25126f2 100644
--- a/Documentation/git-credential.txt
+++ b/Documentation/git-credential.txt
@@ -114,7 +114,9 @@ The key may contain any bytes except `=`, newline, or NUL. The value may
contain any bytes except newline or NUL.
In both cases, all bytes are treated as-is (i.e., there is no quoting,
-and one cannot transmit a value with newline or NUL in it). The list of
+and one cannot transmit a value with newline or NUL in it). Note that
+Git will treat a carriage return before the final newline as part of
+line ending, and not part of the data. The list of
attributes is terminated by a blank line or end-of-file.
Git understands the following attributes:
diff --git a/Documentation/gitcredentials.txt b/Documentation/gitcredentials.txt
index 758bf39ba3..184079eaa4 100644
--- a/Documentation/gitcredentials.txt
+++ b/Documentation/gitcredentials.txt
@@ -141,6 +141,8 @@ entry for `https://example.com/bar/baz.git` (in addition to matching the config
entry for `https://example.com`) but will not match a config entry for
`https://example.com/bar`.
+Note that Git will treat a carriage return before the final newline as part
+of line ending, and not part of the data.
CONFIGURATION OPTIONS
---------------------
--
gitgitgadget
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH v2 0/3] Prepare git credential to read input with DOS line endings
2020-09-28 11:40 ` [PATCH v2 0/3] Prepare git credential to read input with DOS line endings Johannes Schindelin via GitGitGadget
` (2 preceding siblings ...)
2020-09-28 11:40 ` [PATCH v2 3/3] docs: make notes regarding credential line reading Nikita Leonov via GitGitGadget
@ 2020-09-28 20:31 ` Junio C Hamano
2020-10-03 13:29 ` [PATCH v3] credential: treat CR/LF as line endings in the credential protocol Johannes Schindelin via GitGitGadget
4 siblings, 0 replies; 32+ messages in thread
From: Junio C Hamano @ 2020-09-28 20:31 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:
> This contribution came in via Git for Windows
> [https://github.com/git-for-windows/git/pull/2516].
>
> Sadly, I did not find the time to go through all the changes of 8f309aeb
> ("strbuf: introduce strbuf_getline_{lf,nul}()", 2016-01-13) (as Junio asked
> [https://public-inbox.org/git/xmqqmu9lnjdh.fsf@gitster-ct.c.googlers.com]).
> Rather than delaying this patch indefinitely, I admit defeat on that angle.
Oh, that's OK. Thanks for resurrecting the stalled patch. I didn't
mean the suggested action while I was waiting to be an exhaustive
audit---I merely wanted to avoid having to queue a few more separate
ones soon after applying a patch for just a single place.
> Changes since v1:
>
> * Added a commit to adjust credential-daemon and credential-store in the
> same manner.
I am kind of surprised to see that these need to be touched at all.
The change to credential-store is sort-of understandable, as the
file it uses could be hand-modified in an editor and its line ending
could have been changed to CRLF, hence it needs to be prepared.
But does inter-process communication with the daemon need to care
about CRLF line endings? Do parts of the protocol within the
credential system use platform specific line ending?
^ permalink raw reply [flat|nested] 32+ messages in thread* [PATCH v3] credential: treat CR/LF as line endings in the credential protocol
2020-09-28 11:40 ` [PATCH v2 0/3] Prepare git credential to read input with DOS line endings Johannes Schindelin via GitGitGadget
` (3 preceding siblings ...)
2020-09-28 20:31 ` [PATCH v2 0/3] Prepare git credential to read input with DOS line endings Junio C Hamano
@ 2020-10-03 13:29 ` Johannes Schindelin via GitGitGadget
4 siblings, 0 replies; 32+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2020-10-03 13:29 UTC (permalink / raw)
To: git
Cc: Carlo Arenas, Jeff King, Johannes Schindelin,
Johannes Schindelin, Nikita Leonov
From: Nikita Leonov <nykyta.leonov@gmail.com>
This fix makes using Git credentials more friendly to Windows users: it
allows a credential helper to communicate using CR/LF line endings ("DOS
line endings" commonly found on Windows) instead of LF-only line endings
("Unix line endings").
Note that this changes the behavior a bit: if a credential helper
produces, say, a password with a trailing Carriage Return character,
that will now be culled even when the rest of the lines end only in Line
Feed characters, indicating that the Carriage Return was not meant to be
part of the line ending.
In practice, it seems _very_ unlikely that something like this happens.
Passwords usually need to consist of non-control characters, URLs need
to have special characters URL-encoded, and user names, well, are names.
However, it _does_ help on Windows, where CR/LF line endings are common:
as unrecognized commands are simply ignored by the credential machinery,
even a command like `quit\r` (which is clearly intended to abort) would
simply be ignored (silently) by Git.
So let's change the credential machinery to accept both CR/LF and LF
line endings.
While we do this for the credential helper protocol, we do _not_ adjust
`git credential-cache--daemon` (which won't work on Windows, anyway,
because it requires Unix sockets) nor `git credential-store` (which
writes the file `~/.git-credentials` which we consider an implementation
detail that should be opaque to the user, read: we do expect users _not_
to edit this file manually).
Signed-off-by: Nikita Leonov <nykyta.leonov@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Prepare git credential to read input with DOS line endings
This contribution came in via Git for Windows
[https://github.com/git-for-windows/git/pull/2516].
Sadly, I did not find the time to go through all the changes of 8f309aeb
("strbuf: introduce strbuf_getline_{lf,nul}()", 2016-01-13) (as Junio
asked
[https://public-inbox.org/git/xmqqmu9lnjdh.fsf@gitster-ct.c.googlers.com]
). Rather than delaying this patch indefinitely, I admit defeat on that
angle.
Changes since v2:
* Dropped the credential-cache--daemon and credential-store changes
again.
* Enhanced the commit message (also explaining why we don't touch the
daemon and the store).
Changes since v1:
* Added a commit to adjust credential-daemon and credential-store in
the same manner.
* Adjusted the documentation accordingly.
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-710%2Fdscho%2Fcrlf-aware-git-credential-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-710/dscho/crlf-aware-git-credential-v3
Pull-Request: https://github.com/git/git/pull/710
Range-diff vs v2:
1: 27f6400a21 ! 1: f6eeb18d3a credential.c: fix credential reading with regards to CR/LF
@@ Metadata
Author: Nikita Leonov <nykyta.leonov@gmail.com>
## Commit message ##
- credential.c: fix credential reading with regards to CR/LF
+ credential: treat CR/LF as line endings in the credential protocol
- This fix makes using Git credentials more friendly to Windows users. In
- previous version it was unable to finish input correctly without
- configuration changes (tested in PowerShell, CMD, Cygwin).
+ This fix makes using Git credentials more friendly to Windows users: it
+ allows a credential helper to communicate using CR/LF line endings ("DOS
+ line endings" commonly found on Windows) instead of LF-only line endings
+ ("Unix line endings").
- We know credential filling should be finished by empty input, but the
- current implementation does not take into account CR/LF ending, and
- hence instead of the empty string we get '\r', which is interpreted as
- an incorrect string.
+ Note that this changes the behavior a bit: if a credential helper
+ produces, say, a password with a trailing Carriage Return character,
+ that will now be culled even when the rest of the lines end only in Line
+ Feed characters, indicating that the Carriage Return was not meant to be
+ part of the line ending.
- So this commit changes default reading function to a more Windows
- compatible reading function.
+ In practice, it seems _very_ unlikely that something like this happens.
+ Passwords usually need to consist of non-control characters, URLs need
+ to have special characters URL-encoded, and user names, well, are names.
+
+ However, it _does_ help on Windows, where CR/LF line endings are common:
+ as unrecognized commands are simply ignored by the credential machinery,
+ even a command like `quit\r` (which is clearly intended to abort) would
+ simply be ignored (silently) by Git.
+
+ So let's change the credential machinery to accept both CR/LF and LF
+ line endings.
+
+ While we do this for the credential helper protocol, we do _not_ adjust
+ `git credential-cache--daemon` (which won't work on Windows, anyway,
+ because it requires Unix sockets) nor `git credential-store` (which
+ writes the file `~/.git-credentials` which we consider an implementation
+ detail that should be opaque to the user, read: we do expect users _not_
+ to edit this file manually).
Signed-off-by: Nikita Leonov <nykyta.leonov@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2: f69076036f < -: ---------- credentials: make line reading Windows compatible
3: 61baea1061 < -: ---------- docs: make notes regarding credential line reading
credential.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/credential.c b/credential.c
index efc29dc5e1..e5202fbef2 100644
--- a/credential.c
+++ b/credential.c
@@ -202,7 +202,7 @@ int credential_read(struct credential *c, FILE *fp)
{
struct strbuf line = STRBUF_INIT;
- while (strbuf_getline_lf(&line, fp) != EOF) {
+ while (strbuf_getline(&line, fp) != EOF) {
char *key = line.buf;
char *value = strchr(key, '=');
base-commit: 9bc233ae1cf19a49e51842c7959d80a675dbd1c0
--
gitgitgadget
^ permalink raw reply related [flat|nested] 32+ messages in thread