[PATCH] git-sh: Avoid sourcing scripts with git --exec-path

[PATCH] git-sh: Avoid sourcing scripts with git --exec-path

[Dridi Boukelmoune]

For end users making use of a custom exec path many commands will simply
fail. Adding git's exec path to the PATH also allows overriding git-sh-*
scripts, not just adding commands. One can then patch a script without
tainting their system installation of git for example.

Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>
---

Hi,

I've been bothered by this problem ever since I upgraded my system to
Fedora 26, and I grew tired of locally patching git-sh-setup after every
git-core update. So I decided to look at the instructions on how to send
patches to the Git project, and here is my first patch.

I hope you will find it appropriate, I didn't study the test suite to
have it enforce that files shouldn't be sourced from the "system"
installation. I couldn't reproduce it after a quick look, and I'm not
familiar enough to tinker with it yet, so I reached my trial&error quota
before I could figure things out.

Best Regards,
Dridi

 Documentation/CodingGuidelines            | 22 ++++++++++++++++++++++
 contrib/convert-grafts-to-replace-refs.sh |  3 ++-
 contrib/rerere-train.sh                   |  3 ++-
 contrib/subtree/git-subtree.sh            |  1 -
 git-sh-setup.sh                           |  2 +-
 5 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines
index c4cb5ff0d..fdc0d3318 100644
--- a/Documentation/CodingGuidelines
+++ b/Documentation/CodingGuidelines
@@ -151,6 +151,28 @@ For shell scripts specifically (not exhaustive):
    interface translatable. See "Marking strings for translation" in
    po/README.
 
+ - When sourcing a "git-sh-*" file using "git --exec-path" make sure
+   to only use its base name.
+
+	(incorrect)
+	. "$(git --exec-path)/git-sh-setup"
+
+	(correct)
+	. git-sh-setup
+
+   Otherwise for a user with a custom "GIT_EXEC_PATH=/foo" the shell
+   expects "/foo:/usr/libexec/git-core/git-sh-setup" or something
+   similar depending on the installation. The git command already
+   adds the git exec path to the regular path before executing a
+   command.
+
+   For scripts that aren't run via the git command, add the git exec
+   path to the path instead.
+
+	(correct)
+	PATH="$(git --exec-path):$PATH"
+	. git-sh-setup
+
  - We do not write our "test" command with "-a" and "-o" and use "&&"
    or "||" to concatenate multiple "test" commands instead, because
    the use of "-a/-o" is often error-prone.  E.g.
diff --git a/contrib/convert-grafts-to-replace-refs.sh b/contrib/convert-grafts-to-replace-refs.sh
index 0cbc917b8..f7d2fab03 100755
--- a/contrib/convert-grafts-to-replace-refs.sh
+++ b/contrib/convert-grafts-to-replace-refs.sh
@@ -5,7 +5,8 @@
 
 GRAFTS_FILE="${GIT_DIR:-.git}/info/grafts"
 
-. $(git --exec-path)/git-sh-setup
+PATH="$(git --exec-path):$PATH"
+. git-sh-setup
 
 test -f "$GRAFTS_FILE" || die "Could not find graft file: '$GRAFTS_FILE'"
 
diff --git a/contrib/rerere-train.sh b/contrib/rerere-train.sh
index 52ad9e41f..07bad67e6 100755
--- a/contrib/rerere-train.sh
+++ b/contrib/rerere-train.sh
@@ -7,7 +7,8 @@ USAGE="$me rev-list-args"
 
 SUBDIRECTORY_OK=Yes
 OPTIONS_SPEC=
-. "$(git --exec-path)/git-sh-setup"
+PATH="$(git --exec-path):$PATH"
+. git-sh-setup
 require_work_tree
 cd_to_toplevel
 
diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
index dec085a23..1d61f75d0 100755
--- a/contrib/subtree/git-subtree.sh
+++ b/contrib/subtree/git-subtree.sh
@@ -32,7 +32,6 @@ squash        merge subtree changes as a single commit
 "
 eval "$(echo "$OPTS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?)"
 
-PATH=$PATH:$(git --exec-path)
 . git-sh-setup
 
 require_work_tree
diff --git a/git-sh-setup.sh b/git-sh-setup.sh
index 378928518..12e1220f8 100644
--- a/git-sh-setup.sh
+++ b/git-sh-setup.sh
@@ -44,7 +44,7 @@ git_broken_path_fix () {
 # @@BROKEN_PATH_FIX@@
 
 # Source git-sh-i18n for gettext support.
-. "$(git --exec-path)/git-sh-i18n"
+. git-sh-i18n
 
 die () {
 	die_with_status 1 "$@"
-- 
2.13.5

Re: [PATCH] git-sh: Avoid sourcing scripts with git --exec-path

[Jonathan Nieder]

Hi,

Dridi Boukelmoune wrote:

> For end users making use of a custom exec path many commands will simply
> fail. Adding git's exec path to the PATH also allows overriding git-sh-*
> scripts, not just adding commands. One can then patch a script without
> tainting their system installation of git for example.
>
> Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>

This has been broken for a while, but better late than never to
address it.

[...]
> --- a/Documentation/CodingGuidelines
> +++ b/Documentation/CodingGuidelines
> @@ -151,6 +151,28 @@ For shell scripts specifically (not exhaustive):
>     interface translatable. See "Marking strings for translation" in
>     po/README.
>  
> + - When sourcing a "git-sh-*" file using "git --exec-path" make sure
> +   to only use its base name.
> +
> +	(incorrect)
> +	. "$(git --exec-path)/git-sh-setup"
> +
> +	(correct)
> +	. git-sh-setup

I wonder if we can make this so intuitive that it doesn't need
mentioning in CodingGuidelines.  What if the test harness
t/test-lib.sh were to set a GIT_EXEC_PATH with multiple directories in
it?  That way, authors of new commands would not have to be careful to
look out for this issue proactively because the testsuite would catch
it.

[...]
> +++ b/contrib/convert-grafts-to-replace-refs.sh
> @@ -5,7 +5,8 @@
>  
>  GRAFTS_FILE="${GIT_DIR:-.git}/info/grafts"
>  
> -. $(git --exec-path)/git-sh-setup
> +PATH="$(git --exec-path):$PATH"
> +. git-sh-setup

I wish there were a simple way to do this that doesn't involve
polluting $PATH with the dashed commands.  E.g. we can do something
like

	OIFS=$IFS
	IFS=:
	set -f
	for d in $(git --exec-path)
	do
		if test -e "$d/git-sh-setup"
		then
			. "$d/git-sh-setup"
			break
		fi
	done
	set +f
	IFS=$OIFS

but that is not very simple.  Something like

	old_PATH=$PATH
	PATH=$(git --exec-path):$PATH
	. git-sh-setup
	PATH=$old_PATH

looks like it could work, but it would undo the work git-sh-setup
does to set a sane $PATH on platforms like Solaris.

> --- a/contrib/rerere-train.sh
> +++ b/contrib/rerere-train.sh
> @@ -7,7 +7,8 @@ USAGE="$me rev-list-args"
>  
>  SUBDIRECTORY_OK=Yes
>  OPTIONS_SPEC=
> -. "$(git --exec-path)/git-sh-setup"
> +PATH="$(git --exec-path):$PATH"
> +. git-sh-setup

This makes me similarly unhappy about PATH pollution, but it may be
that there's nothing to be done about that.

[...]
> --- a/contrib/subtree/git-subtree.sh
> +++ b/contrib/subtree/git-subtree.sh
> @@ -32,7 +32,6 @@ squash        merge subtree changes as a single commit
>  "
>  eval "$(echo "$OPTS_SPEC" | git rev-parse --parseopt -- "$@" || echo exit $?)"
>  
> -PATH=$PATH:$(git --exec-path)
>  . git-sh-setup

This looks like a change that could be separated into a separate
patch, both because it is to contrib/subtree (which is maintained
separately) and because it is not necessary for the goal described in
this patch's commit message.  I do like this change, since it improves
consistency with other commands.

> --- a/git-sh-setup.sh
> +++ b/git-sh-setup.sh
> @@ -44,7 +44,7 @@ git_broken_path_fix () {
>  # @@BROKEN_PATH_FIX@@
>  
>  # Source git-sh-i18n for gettext support.
> -. "$(git --exec-path)/git-sh-i18n"
> +. git-sh-i18n

Do git-mergetool--lib.txt, git-parse-remote.txt, git-sh-i18n.txt,
and git-sh-setup.txt in Documentation/ need the same treatment?

Summary: I like the goal of this patch but I am nervous about the
side-effect it introduces of PATH pollution.  Is there a way around
that?  If there isn't, then this needs a few tweaks and then it should
be ready to go.

Thanks and hope that helps,
Jonathan

Re: [PATCH] git-sh: Avoid sourcing scripts with git --exec-path

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> This has been broken for a while, but better late than never to
> address it.

I am not sure if this is broken in the first place.  We do want to
make sure that the scripted porcelains will source the shell helper
library from matching Git release.  The proposed patch goes directly
against that and I do not see how it could be an improvement.

>> --- a/contrib/rerere-train.sh
>> +++ b/contrib/rerere-train.sh
>> @@ -7,7 +7,8 @@ USAGE="$me rev-list-args"
>>  
>>  SUBDIRECTORY_OK=Yes
>>  OPTIONS_SPEC=
>> -. "$(git --exec-path)/git-sh-setup"
>> +PATH="$(git --exec-path):$PATH"
>> +. git-sh-setup
>
> This makes me similarly unhappy about PATH pollution, but it may be
> that there's nothing to be done about that.

To me, all the instances of the unhappiness in your review seem to
be caused by the unjustified declaration that it is bad to source
from the directory "gti --exec-path" reports.  If a user wants to
tweak things Git does, why should the user futz with his own copy
of sh-setup and force scripted Porcelains to read from it, which
would only affect the scripted Porcelains and have no chance of
affecting compiled commands?  Is building from the source so bad for
an open source tool?

Re: [PATCH] git-sh: Avoid sourcing scripts with git --exec-path

[Jonathan Nieder]

Junio C Hamano wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:

>> This has been broken for a while, but better late than never to
>> address it.
>
> I am not sure if this is broken in the first place.  We do want to
> make sure that the scripted porcelains will source the shell helper
> library from matching Git release.  The proposed patch goes directly
> against that and I do not see how it could be an improvement.

It used to be that git allowed setting a colon-separated list of paths
in GIT_EXEC_PATH.  (Very long ago, I relied on that feature.)  If we
can restore that functionality without too much cost, then I think
it's worthwhile.

But the cost in this patch for that is pretty high.  And

	$ git log -S'$(git --exec-path)/'

tells me that colon-separated paths in GIT_EXEC_PATH did not work for
some use cases as far back as 2006.  Since 2008 the documentation has
encouraged using "git --exec-path" in a way that does not work with
colon-separated paths, too.  I hadn't realized it was so long.  Given
that it hasn't been supported for more than ten years, I was wrong to
read this as a bug report --- instead, it's a feature request.

In that context, this cost is likely not worth paying.

I wonder if there's another way to achieve this patch's goal.  E.g.
what if there were a way to specify some paths git could search for
custom commands, separate from "git --exec-path"?

Putting the custom commands on the $PATH seems nicer unless you're
trying to override the implementation of an existing git command.
And we already discourage overriding the implementation of an existing
git command (as it's open source, you can patch them instead), so...

Another possible motivation (the one that led me to use this mechnism
long ago) is avoiding providing the dashed form git-$cmd in $PATH.  I
think time has shown that having git-$cmd in $PATH is not too painful.

Thanks and hope that helps,
Jonathan

Re: [PATCH] git-sh: Avoid sourcing scripts with git --exec-path

[Junio C Hamano]

Dridi Boukelmoune <dridi.boukelmoune@gmail.com> writes:

> For end users making use of a custom exec path many commands will simply
> fail. Adding git's exec path to the PATH also allows overriding git-sh-*
> scripts, not just adding commands. One can then patch a script without
> tainting their system installation of git for example.

I think the first sentence is where you went wrong.  It seems that
you think this ought to work:

    rm -fr $HOME/random-stuff
    mkdir $HOME/random-stuff
    echo "echo happy" >$HOME/random-stuff/git-happy
    chmod +x $HOME/random-stuff/git-happy
    GIT_EXEC_PATH=$HOME/random-stuff
    export GIT_EXEC_PATH
    # then...
    git happy

But that is not the right/officially sanctioned/kosher way to add
custom git commands (adding your directory that has git-happy in it
to $PATH is).  GIT_EXEC_PATH is for the git-cmd binaries and scripts
we ship; it always is used to find non built-in commands, and even
for built-in commands, the command found via alias look-up is invoked
that way.

By insisting on overriding GIT_EXEC_PATH and not populating with
the stuff we ship, you'd need a workaround like your patch just to
make the scripts "work" again.  I have a feeling that even with your
patch you wouldn't be able to make non built-in commands, unless you
copy them (or write a thin wrapper that exec's the real thing).

So, instead of the two GIT_EXEC_PATH steps in the above example,
you can do

	PATH=$HOME/random-stuff:$PATH

and you'll see "git happy" to work, I would think, without breaking
other things.

Re: [PATCH] git-sh: Avoid sourcing scripts with git --exec-path

[Dridi Boukelmoune]

Hi all,

Thanks for the fast feedback, I'll answer everyone in a single email
if you don't mind.

On Fri, Sep 29, 2017 at 5:48 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
snip
> I wonder if we can make this so intuitive that it doesn't need
> mentioning in CodingGuidelines.  What if the test harness
> t/test-lib.sh were to set a GIT_EXEC_PATH with multiple directories in
> it?  That way, authors of new commands would not have to be careful to
> look out for this issue proactively because the testsuite would catch
> it.

Now that you pointed out that I missed the relevant documentations, I
don't think this belongs in the guidelines at all.

snip
> Do git-mergetool--lib.txt, git-parse-remote.txt, git-sh-i18n.txt,
> and git-sh-setup.txt in Documentation/ need the same treatment?

That is embarrassing, I thought I had done my research properly...

> Summary: I like the goal of this patch but I am nervous about the
> side-effect it introduces of PATH pollution.  Is there a way around
> that?  If there isn't, then this needs a few tweaks and then it should
> be ready to go.

The PATH is already "polluted" when git-* commands are run via git,
and in the context of a script using git-sh-setup I wouldn't consider
that completely irrelevant.

On Fri, Sep 29, 2017 at 5:58 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>> This has been broken for a while, but better late than never to
>> address it.
>
> I am not sure if this is broken in the first place.  We do want to
> make sure that the scripted porcelains will source the shell helper
> library from matching Git release.  The proposed patch goes directly
> against that and I do not see how it could be an improvement.

But the problem is that just by having a GIT_EXEC_PATH you will source
an incorrect file name. If there was something like --exec-dir that
wouldn't take the PATH into account. Before I tried to contribute a
fix, my local patching of git-sh-setup after git-core upgrades was
actually this:

    -. "$(git --exec-path)/git-sh-i18n"
    +. "$(GIT_EXEC_PATH= git --exec-path)/git-sh-i18n"

That's not pretty, but it gives the guarantee to source from matching
Git release. Considering the PATH semantics, this is how I would fix
it after reading your feedback.

On Fri, Sep 29, 2017 at 6:21 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Junio C Hamano wrote:
>> Jonathan Nieder <jrnieder@gmail.com> writes:
>
>>> This has been broken for a while, but better late than never to
>>> address it.
>>
>> I am not sure if this is broken in the first place.  We do want to
>> make sure that the scripted porcelains will source the shell helper
>> library from matching Git release.  The proposed patch goes directly
>> against that and I do not see how it could be an improvement.
>
> It used to be that git allowed setting a colon-separated list of paths
> in GIT_EXEC_PATH.  (Very long ago, I relied on that feature.)  If we
> can restore that functionality without too much cost, then I think
> it's worthwhile.
>
> But the cost in this patch for that is pretty high.  And
>
>         $ git log -S'$(git --exec-path)/'
>
> tells me that colon-separated paths in GIT_EXEC_PATH did not work for
> some use cases as far back as 2006.  Since 2008 the documentation has
> encouraged using "git --exec-path" in a way that does not work with
> colon-separated paths, too.  I hadn't realized it was so long.  Given
> that it hasn't been supported for more than ten years, I was wrong to
> read this as a bug report --- instead, it's a feature request.

Well, from my perspective it's a bug report, upgrading git caused a
regression in my setup. I didn't know I was doing it wrong ;)

snip
> Another possible motivation (the one that led me to use this mechnism
> long ago) is avoiding providing the dashed form git-$cmd in $PATH.  I
> think time has shown that having git-$cmd in $PATH is not too painful.

In my case, yes, I'm maintaining commands but don't really want to
pollute my general-purpose PATH. But I can live with that and use PATH
instead.

On Fri, Sep 29, 2017 at 7:00 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Dridi Boukelmoune <dridi.boukelmoune@gmail.com> writes:
>
>> For end users making use of a custom exec path many commands will simply
>> fail. Adding git's exec path to the PATH also allows overriding git-sh-*
>> scripts, not just adding commands. One can then patch a script without
>> tainting their system installation of git for example.
>
> I think the first sentence is where you went wrong.  It seems that
> you think this ought to work:
>
>     rm -fr $HOME/random-stuff
>     mkdir $HOME/random-stuff
>     echo "echo happy" >$HOME/random-stuff/git-happy
>     chmod +x $HOME/random-stuff/git-happy
>     GIT_EXEC_PATH=$HOME/random-stuff
>     export GIT_EXEC_PATH
>     # then...
>     git happy

Exactly!

> But that is not the right/officially sanctioned/kosher way to add
> custom git commands (adding your directory that has git-happy in it
> to $PATH is).  GIT_EXEC_PATH is for the git-cmd binaries and scripts
> we ship; it always is used to find non built-in commands, and even
> for built-in commands, the command found via alias look-up is invoked
> that way.

The git(1) manual says:

> This can also be controlled by setting the GIT_EXEC_PATH environment
> variable.

This is why I set up my system like this years ago. The manual could
say when the envvar usage is appropriate.

> By insisting on overriding GIT_EXEC_PATH and not populating with
> the stuff we ship, you'd need a workaround like your patch just to
> make the scripts "work" again.  I have a feeling that even with your

I forgot about that, and I actually had to look at my bashrc to realize that
GIT_EXEC_PATH completely overrides the exec path. I did this, and
forgot about it once I was done:

    export GIT_EXEC_PATH=<my-path>:"$(GIT_EXEC_PATH= git --exec-path)"

So for so while I managed to convince myself that git --exec-path
always includes the local installation path. Today I re-learn.

> patch you wouldn't be able to make non built-in commands, unless you
> copy them (or write a thin wrapper that exec's the real thing).
>
> So, instead of the two GIT_EXEC_PATH steps in the above example,
> you can do
>
>         PATH=$HOME/random-stuff:$PATH
>
> and you'll see "git happy" to work, I would think, without breaking
> other things.

That creates a gap between git-core commands being in libexec outside
of the PATH and my git-* commands having to be in the PATH even though
they aren't meant to be executed directly.

Should I attempt a new patch? I'd make sure to source by neutering
GIT_EXEC_PATH and update the relevant documentation this time.

Thanks for the review, it has been very instructive, even for a simple shell
problem!

Dridi