Bug using `fetch` with blank `-c` arguments to git

Bug using `fetch` with blank `-c` arguments to git

[Adam Dinwoodie]

While investigating some issues with a different project, I discovered
the command `git -c config.helper= fetch` was working with the Debian
stable version of Git (v2.30.2) but not with my local build
(v2.34.1.428.gdcc0cd074f).

Specifically, I see the following output:

$ ./git -c credential.helper= fetch
error: bogus format in GIT_CONFIG_PARAMETERS
fatal: unable to parse command-line config

Investigating with `git bisect`, the change in behaviour seems to have
been introduced in 1ff21c05ba ("config: store "git -c" variables using
more robust format", 2021-01-12).

I see the same behaviour with `-c config.helper=`, `-c
core.autocrlf=`, `-c core.autocrlf` and `-c core.autocrlf=true`..
Notably the behaviour does not affect all other git commands; `git -c
core.autocrlf= log -1` works as expected.

I think this is a regression; I can't see any reason why these
commands shouldn't work.

Curiously, I'm seeing this behaviour on both my Raspberry Pi OS and
Debian Bullseye systems, but not my Cygwin systems. I've not yet tried
to work out what the difference is there. In all cases, I was testing
with my own build, built with `make -j<num> configure && ./configure
--prefix=$HOME/.local && make -j<num>`.

Re: Bug using `fetch` with blank `-c` arguments to git

[Erik Cervin Edin]

Minor clarifications

On Tue, Jan 4, 2022 at 3:48 PM Adam Dinwoodie <adam@dinwoodie.org> wrote:
>
> While investigating some issues with a different project, I discovered
> the command `git -c config.helper= fetch` was working with the Debian
> ...
> I see the same behaviour with `-c config.helper=`, `-c
typos for "credential.helper" ?

> Notably the behaviour does not affect all other git commands; `git -c
> core.autocrlf= log -1` works as expected.
You've only seen this with "git fetch"?

Re: Bug using `fetch` with blank `-c` arguments to git

[Adam Dinwoodie]

On Tue, 4 Jan 2022 at 15:35, Erik Cervin Edin <erik@cervined.in> wrote:
>
> Minor clarifications
>
> On Tue, Jan 4, 2022 at 3:48 PM Adam Dinwoodie <adam@dinwoodie.org> wrote:
> >
> > While investigating some issues with a different project, I discovered
> > the command `git -c config.helper= fetch` was working with the Debian
> > ...
> > I see the same behaviour with `-c config.helper=`, `-c
> typos for "credential.helper" ?

"credential.helper" was certainly what I'd intended, but I see the
same behaviour with both options.

(I'd initially thought this was specifically due to the blank
argument, hence the email subject which I clearly failed to update,
but it looks like the issue occurs when using `-c` to set a config
option regardless of the option being set.)

> > Notably the behaviour does not affect all other git commands; `git -c
> > core.autocrlf= log -1` works as expected.
> You've only seen this with "git fetch"?

I've not tried to do any exhaustive testing, but I see this behaviour
with "git fetch", "git pull", "git bisect" and "git submodule"; I see
things apparently working as expected "git blame", "git commit", "git
push", "git reset", "git switch" and "git log".

Re: Bug using `fetch` with blank `-c` arguments to git

[Erik Cervin Edin]

On Tue, Jan 4, 2022 at 3:48 PM Adam Dinwoodie <adam@dinwoodie.org> wrote:
> Investigating with `git bisect`, the change in behaviour seems to have
> been introduced in 1ff21c05ba ("config: store "git -c" variables using
> more robust format", 2021-01-12).
If that's the case it should be present since v2.31.0

I can't replicate in git version 2.34.1.windows.1
But that's MinGW based so if you don't see it in cygwin it's not a big surprise

On Tue, Jan 4, 2022 at 5:16 PM Adam Dinwoodie <adam@dinwoodie.org> wrote:
> but it looks like the issue occurs when using `-c` to set a config
> option regardless of the option being set.)
:(

Re: Bug using `fetch` with blank `-c` arguments to git

[Bryan Turner]

On Tue, Jan 4, 2022 at 4:37 AM Adam Dinwoodie <adam@dinwoodie.org> wrote:
>
> While investigating some issues with a different project, I discovered
> the command `git -c config.helper= fetch` was working with the Debian
> stable version of Git (v2.30.2) but not with my local build
> (v2.34.1.428.gdcc0cd074f).

Since you're working with a locally-built Git, have you, by chance,
actually _installed_ that build, or is it simply in the Git repository
itself after running make?

If you haven't _installed_ your build, my guess is you might be
getting a mismatch wherein your _built_ Git, when it forks out
subprocesses, is triggering your _installed_ Git (which I assume you
have, and which I assume is not 2.34.1). Git compiles paths into
itself to know where to find certain binaries, and if you run a
compiled-but-not-installed Git then those paths are "wrong". (I see
administrators do this fairly often when building Git from source to
set up Bitbucket Server.)

What does `./git --exec-path` print, when you run your 2.34.1 binary?
And is that where, for example, the compiled 2.34.1 versions of things
like `git-remote-https` are?

Hope this helps,
Bryan

>
> Specifically, I see the following output:
>
> $ ./git -c credential.helper= fetch
> error: bogus format in GIT_CONFIG_PARAMETERS
> fatal: unable to parse command-line config
>
> Investigating with `git bisect`, the change in behaviour seems to have
> been introduced in 1ff21c05ba ("config: store "git -c" variables using
> more robust format", 2021-01-12).
>
> I see the same behaviour with `-c config.helper=`, `-c
> core.autocrlf=`, `-c core.autocrlf` and `-c core.autocrlf=true`..
> Notably the behaviour does not affect all other git commands; `git -c
> core.autocrlf= log -1` works as expected.
>
> I think this is a regression; I can't see any reason why these
> commands shouldn't work.
>
> Curiously, I'm seeing this behaviour on both my Raspberry Pi OS and
> Debian Bullseye systems, but not my Cygwin systems. I've not yet tried
> to work out what the difference is there. In all cases, I was testing
> with my own build, built with `make -j<num> configure && ./configure
> --prefix=$HOME/.local && make -j<num>`.

Re: Bug using `fetch` with blank `-c` arguments to git

[Adam Dinwoodie]

On Tue, 4 Jan 2022 at 20:04, Bryan Turner <bturner@atlassian.com> wrote:
>
> On Tue, Jan 4, 2022 at 4:37 AM Adam Dinwoodie <adam@dinwoodie.org> wrote:
> >
> > While investigating some issues with a different project, I discovered
> > the command `git -c config.helper= fetch` was working with the Debian
> > stable version of Git (v2.30.2) but not with my local build
> > (v2.34.1.428.gdcc0cd074f).
>
> Since you're working with a locally-built Git, have you, by chance,
> actually _installed_ that build, or is it simply in the Git repository
> itself after running make?
>
> If you haven't _installed_ your build, my guess is you might be
> getting a mismatch wherein your _built_ Git, when it forks out
> subprocesses, is triggering your _installed_ Git (which I assume you
> have, and which I assume is not 2.34.1). Git compiles paths into
> itself to know where to find certain binaries, and if you run a
> compiled-but-not-installed Git then those paths are "wrong". (I see
> administrators do this fairly often when building Git from source to
> set up Bitbucket Server.)
>
> What does `./git --exec-path` print, when you run your 2.34.1 binary?
> And is that where, for example, the compiled 2.34.1 versions of things
> like `git-remote-https` are?

Good thoughts, but I initially hit this problem after having installed
it; I reproduced it running Git from the working copy for ease of
bisecting, but the problem definitely occurs using the compiled
version after installation. The below was collected after running
`make install` (plus all the previously noted build commands,
including running the configure script to specify the installation
path) with the commit I identified as introducing the problem:

```
$ type git
git is hashed (/home/adam/.local/bin/git)

$ which git
/home/adam/.local/bin/git

$ git --version
git version 2.29.2.372.g1ff21c05ba

$ git --exec-path
/home/adam/.local/libexec/git-core

$ ls $(git --exec-path)/git $(git --exec-path)/git-remote-https
/home/adam/.local/libexec/git-core/git
/home/adam/.local/libexec/git-core/git-remote-https

$ $(git --exec-path)/git --version
git version 2.29.2.372.g1ff21c05ba

$ rm -rf tmp && git -c core.autocrlf=true clone git://github.com/git/git tmp
Cloning into 'tmp'...
error: bogus format in GIT_CONFIG_PARAMETERS
fatal: unable to parse command-line config
```

For the sake of double-checking, though, I just uninstalled the
version of Git in /usr/bin (after spending some time working out how
to do that with apt, without also uninstalling dependencies I wanted
to leave alone) and repeated the above commands, and got exactly the
same output.

Re: Bug using `fetch` with blank `-c` arguments to git

[Adam Dinwoodie]

On Tue, 4 Jan 2022 at 21:00, Adam Dinwoodie <adam@dinwoodie.org> wrote:
> <snip>
> For the sake of double-checking, though, I just uninstalled the
> version of Git in /usr/bin (after spending some time working out how
> to do that with apt, without also uninstalling dependencies I wanted
> to leave alone) and repeated the above commands, and got exactly the
> same output.

On the off-chance anyone is following along at home: I've just
attempted to reproduce this problem with a fresh Debian installation,
and the problem does not reproduce. So there's clearly something odd
about my environments. I'm baffled about what it might be, but for now
I'll keep investigating on my side.

Re: Bug using `fetch` with blank `-c` arguments to git

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3965 bytes --]

On Tue, Jan 04, 2022 at 09:00:45PM +0000, Adam Dinwoodie wrote:
> On Tue, 4 Jan 2022 at 20:04, Bryan Turner <bturner@atlassian.com> wrote:
> >
> > On Tue, Jan 4, 2022 at 4:37 AM Adam Dinwoodie <adam@dinwoodie.org> wrote:
> > >
> > > While investigating some issues with a different project, I discovered
> > > the command `git -c config.helper= fetch` was working with the Debian
> > > stable version of Git (v2.30.2) but not with my local build
> > > (v2.34.1.428.gdcc0cd074f).
> >
> > Since you're working with a locally-built Git, have you, by chance,
> > actually _installed_ that build, or is it simply in the Git repository
> > itself after running make?
> >
> > If you haven't _installed_ your build, my guess is you might be
> > getting a mismatch wherein your _built_ Git, when it forks out
> > subprocesses, is triggering your _installed_ Git (which I assume you
> > have, and which I assume is not 2.34.1). Git compiles paths into
> > itself to know where to find certain binaries, and if you run a
> > compiled-but-not-installed Git then those paths are "wrong". (I see
> > administrators do this fairly often when building Git from source to
> > set up Bitbucket Server.)
> >
> > What does `./git --exec-path` print, when you run your 2.34.1 binary?
> > And is that where, for example, the compiled 2.34.1 versions of things
> > like `git-remote-https` are?
> 
> Good thoughts, but I initially hit this problem after having installed
> it; I reproduced it running Git from the working copy for ease of
> bisecting, but the problem definitely occurs using the compiled
> version after installation. The below was collected after running
> `make install` (plus all the previously noted build commands,
> including running the configure script to specify the installation
> path) with the commit I identified as introducing the problem:
> 
> ```
> $ type git
> git is hashed (/home/adam/.local/bin/git)
> 
> $ which git
> /home/adam/.local/bin/git
> 
> $ git --version
> git version 2.29.2.372.g1ff21c05ba
> 
> $ git --exec-path
> /home/adam/.local/libexec/git-core
> 
> $ ls $(git --exec-path)/git $(git --exec-path)/git-remote-https
> /home/adam/.local/libexec/git-core/git
> /home/adam/.local/libexec/git-core/git-remote-https
> 
> $ $(git --exec-path)/git --version
> git version 2.29.2.372.g1ff21c05ba
> 
> $ rm -rf tmp && git -c core.autocrlf=true clone git://github.com/git/git tmp
> Cloning into 'tmp'...
> error: bogus format in GIT_CONFIG_PARAMETERS
> fatal: unable to parse command-line config
> ```
> 
> For the sake of double-checking, though, I just uninstalled the
> version of Git in /usr/bin (after spending some time working out how
> to do that with apt, without also uninstalling dependencies I wanted
> to leave alone) and repeated the above commands, and got exactly the
> same output.

I cannot really reproduce this locally. But given that it happens on
some installations while it works alright on others my best guess is
that you're effectively running a "mixed" setup, where Git binaries of
one version execute Git binaries of a different version. The result
would be that they have different ways to encode GIT_CONFIG_PARAMETERS,
where new versions use the more robust, quoted format, which older
versions don't understand, or the other way round.

If my theory is correct, then I'm a bit on the edge to call this a bug.
Git expects its binaries to all be of the same version, so running such
a mixed setup is only going to cause problems. This isn't only true for
internal variables like the one we have here, but we also introduce new
command line switches and expect Git helpers to understand them without
any fallback if an old binary was executed.

You may want to verify that the Git executables you have are indeed able
to execute the correct auxiliary binaries as expected and that they all
stem from the same Git version.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bug using `fetch` with blank `-c` arguments to git

[Adam Dinwoodie]

On Fri, 7 Jan 2022 at 12:52, Patrick Steinhardt <ps@pks.im> wrote:
>
> On Tue, Jan 04, 2022 at 09:00:45PM +0000, Adam Dinwoodie wrote:
> > On Tue, 4 Jan 2022 at 20:04, Bryan Turner <bturner@atlassian.com> wrote:
> > >
> > > On Tue, Jan 4, 2022 at 4:37 AM Adam Dinwoodie <adam@dinwoodie.org> wrote:
> > > >
> > > > While investigating some issues with a different project, I discovered
> > > > the command `git -c config.helper= fetch` was working with the Debian
> > > > stable version of Git (v2.30.2) but not with my local build
> > > > (v2.34.1.428.gdcc0cd074f).
> > >
> > > Since you're working with a locally-built Git, have you, by chance,
> > > actually _installed_ that build, or is it simply in the Git repository
> > > itself after running make?
> > >
> > > If you haven't _installed_ your build, my guess is you might be
> > > getting a mismatch wherein your _built_ Git, when it forks out
> > > subprocesses, is triggering your _installed_ Git (which I assume you
> > > have, and which I assume is not 2.34.1). Git compiles paths into
> > > itself to know where to find certain binaries, and if you run a
> > > compiled-but-not-installed Git then those paths are "wrong". (I see
> > > administrators do this fairly often when building Git from source to
> > > set up Bitbucket Server.)
> > >
> > > What does `./git --exec-path` print, when you run your 2.34.1 binary?
> > > And is that where, for example, the compiled 2.34.1 versions of things
> > > like `git-remote-https` are?
> >
> > Good thoughts, but I initially hit this problem after having installed
> > it; I reproduced it running Git from the working copy for ease of
> > bisecting, but the problem definitely occurs using the compiled
> > version after installation. The below was collected after running
> > `make install` (plus all the previously noted build commands,
> > including running the configure script to specify the installation
> > path) with the commit I identified as introducing the problem:
> >
> > ```
> > $ type git
> > git is hashed (/home/adam/.local/bin/git)
> >
> > $ which git
> > /home/adam/.local/bin/git
> >
> > $ git --version
> > git version 2.29.2.372.g1ff21c05ba
> >
> > $ git --exec-path
> > /home/adam/.local/libexec/git-core
> >
> > $ ls $(git --exec-path)/git $(git --exec-path)/git-remote-https
> > /home/adam/.local/libexec/git-core/git
> > /home/adam/.local/libexec/git-core/git-remote-https
> >
> > $ $(git --exec-path)/git --version
> > git version 2.29.2.372.g1ff21c05ba
> >
> > $ rm -rf tmp && git -c core.autocrlf=true clone git://github.com/git/git tmp
> > Cloning into 'tmp'...
> > error: bogus format in GIT_CONFIG_PARAMETERS
> > fatal: unable to parse command-line config
> > ```
> >
> > For the sake of double-checking, though, I just uninstalled the
> > version of Git in /usr/bin (after spending some time working out how
> > to do that with apt, without also uninstalling dependencies I wanted
> > to leave alone) and repeated the above commands, and got exactly the
> > same output.
>
> I cannot really reproduce this locally. But given that it happens on
> some installations while it works alright on others my best guess is
> that you're effectively running a "mixed" setup, where Git binaries of
> one version execute Git binaries of a different version. The result
> would be that they have different ways to encode GIT_CONFIG_PARAMETERS,
> where new versions use the more robust, quoted format, which older
> versions don't understand, or the other way round.

Ah, that makes a lot of sense, thank you! In particular, I hadn't
quite put together that if different Git binaries compiled with code
both before and after this change attempted to communicate, they'd be
doing so using a mutually incompatible interface, and that would
produce this error.

> If my theory is correct, then I'm a bit on the edge to call this a bug.
> Git expects its binaries to all be of the same version, so running such
> a mixed setup is only going to cause problems. This isn't only true for
> internal variables like the one we have here, but we also introduce new
> command line switches and expect Git helpers to understand them without
> any fallback if an old binary was executed.
>
> You may want to verify that the Git executables you have are indeed able
> to execute the correct auxiliary binaries as expected and that they all
> stem from the same Git version.

Yes, that's very much the conclusion I'm coming to as well. As I
understand things, it should be possible to have multiple Git
installations at different versions on the same system, provided
they're all configured to find their libraries and executables in the
right place, so I've not yet ruled out there being a bug here. But if
there is a bug, it'll be something related to how Git finds its
libraries and executables at runtime, rather than anything directly to
do with this change. And at this point I very strongly suspect I've
just screwed up somewhere and there's no bug whatsoever.