% in $PATH

% in $PATH

[Stephane Chazelas]

Hello,

ash/dash have a nice feature that allows to have:

PATH=/bin:%builtin:/usr/bin:/some/dir%func:/sbin

To have commands in /bin take precedence over builtins and
files in /some/dir being looked up for autoloaded functions (a
bit like FPATH in ksh/zsh).

That's nice but the way it is implemented, that means that %
characters in $PATH cause problems. See for instance:

http://unix.stackexchange.com/questions/126955/percent-in-path-environment-variable

Where someone ran into it when they had a:

/home/torbjorr/deployed/vector/x86_64-GNU%2fLinux directory in
there.

Given than only "builtin" and "func" are recognised after "%", I
think it would be better if the PATH handling accepted any other
string like "foo%bar", "foo%2f", "foo%functions" as-is as a
normal $PATH component.

And foo%2fbar%func meaning "foo%2fbar" treated as a directory
with autoloadable functions.

-- 
Stephane

Re: % in $PATH

[Herbert Xu]

On Wed, Nov 05, 2014 at 11:59:47AM +0000, Stephane Chazelas wrote:
> Hello,
> 
> ash/dash have a nice feature that allows to have:
> 
> PATH=/bin:%builtin:/usr/bin:/some/dir%func:/sbin
> 
> To have commands in /bin take precedence over builtins and
> files in /some/dir being looked up for autoloaded functions (a
> bit like FPATH in ksh/zsh).
> 
> That's nice but the way it is implemented, that means that %
> characters in $PATH cause problems. See for instance:
> 
> http://unix.stackexchange.com/questions/126955/percent-in-path-environment-variable

I'm inclined to just kill this feature, or at least make it a
configuration option that's disabled by default.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: % in $PATH

[Stephane Chazelas]

2014-11-10 21:20:02 +0800, Herbert Xu:
> On Wed, Nov 05, 2014 at 11:59:47AM +0000, Stephane Chazelas wrote:
> > Hello,
> > 
> > ash/dash have a nice feature that allows to have:
> > 
> > PATH=/bin:%builtin:/usr/bin:/some/dir%func:/sbin
> > 
> > To have commands in /bin take precedence over builtins and
> > files in /some/dir being looked up for autoloaded functions (a
> > bit like FPATH in ksh/zsh).
> > 
> > That's nice but the way it is implemented, that means that %
> > characters in $PATH cause problems. See for instance:
> > 
> > http://unix.stackexchange.com/questions/126955/percent-in-path-environment-variable
> 
> I'm inclined to just kill this feature, or at least make it a
> configuration option that's disabled by default.
[...]

Though I'd agree there's little chance of many people using it as
the documentation about it has been removed in dash, I don't
think there's any harm in leaving it in but implemented the way
I suggest.

It's useful as an equivalent to bash's exported functions (and
is a better/safer approach IMO) as an instrumentation tool.

Example: redefine "echo" as a Unix conformant one before running
something that expects a Unix conformant "echo":

$ printf '%s\n' 'echo() { local IFS=" "; printf "%b\n" "$*"; }' > echo
$ PATH=$PWD%func:%builtins:$PATH dash -c 'echo "-n\c"; echo x'
-nx

I don't see the point in keeping it if it's to make it disabled
by default though (unless we add an equivalent of BASHOPTS which
can be used to turn it on via the environment)

-- 
Stephane

Re: % in $PATH

[Guido Berhoerster]

* Stephane Chazelas <stephane.chazelas@gmail.com> [2014-11-10 22:35]:
> 2014-11-10 21:20:02 +0800, Herbert Xu:
> > On Wed, Nov 05, 2014 at 11:59:47AM +0000, Stephane Chazelas wrote:
> > > Hello,
> > > 
> > > ash/dash have a nice feature that allows to have:
> > > 
> > > PATH=/bin:%builtin:/usr/bin:/some/dir%func:/sbin
> > > 
> > > To have commands in /bin take precedence over builtins and
> > > files in /some/dir being looked up for autoloaded functions (a
> > > bit like FPATH in ksh/zsh).
> > > 
> > > That's nice but the way it is implemented, that means that %
> > > characters in $PATH cause problems. See for instance:
> > > 
> > > http://unix.stackexchange.com/questions/126955/percent-in-path-environment-variable
> > 
> > I'm inclined to just kill this feature, or at least make it a
> > configuration option that's disabled by default.
> [...]
> 
> Though I'd agree there's little chance of many people using it as
> the documentation about it has been removed in dash, I don't
> think there's any harm in leaving it in but implemented the way
> I suggest.
> 
> It's useful as an equivalent to bash's exported functions (and
> is a better/safer approach IMO) as an instrumentation tool.
> 
> Example: redefine "echo" as a Unix conformant one before running
> something that expects a Unix conformant "echo":
> 
> $ printf '%s\n' 'echo() { local IFS=" "; printf "%b\n" "$*"; }' > echo
> $ PATH=$PWD%func:%builtins:$PATH dash -c 'echo "-n\c"; echo x'
> -nx
> 
> I don't see the point in keeping it if it's to make it disabled
> by default though (unless we add an equivalent of BASHOPTS which
> can be used to turn it on via the environment)

A much nicer solution would be to do something similar to the
original Korn shell and assign additional builtins a virtual
path which can be freely assigned in PATH and with which they can
be explicitly called. No more "%" in PATH and the feature can
be retained.
-- 
Guido Berhoerster

Re: % in $PATH

[Stephane Chazelas]

2014-11-10 22:59:38 +0100, Guido Berhoerster:
[...]
> A much nicer solution would be to do something similar to the
> original Korn shell and assign additional builtins a virtual
> path which can be freely assigned in PATH and with which they can
> be explicitly called. No more "%" in PATH and the feature can
> be retained.
[...]

Sounds to me like it's what dash does. It has %builtin (or
/%builtin) where ksh93 has /opt/ast/bin.

With ksh93, that's only for a few extra builtins that are only
enabled if you add /opt/ast/bin to $PATH (or you can call them
with "command /opt/ast/bin/cat").

I wouldn't say it's much better, it's still bad as being the
hijacking of one namespace for something else.

dash's %builtin (as opposed to /path/to/functions%func) is
potentially dangerous though because if you have that in your
$PATH, you're vulnerable to attackers planting %builtin
directories (in /tmp for instance) which will be considered by
other things than dash/ash when looking up $PATH.

That's worked around by using /%builtin instead of %builtin
though.

-- 
Stephane

Re: % in $PATH

[Guido Berhoerster]

* Stephane Chazelas <stephane.chazelas@gmail.com> [2014-11-10 23:23]:
> 2014-11-10 22:59:38 +0100, Guido Berhoerster:
> [...]
> > A much nicer solution would be to do something similar to the
> > original Korn shell and assign additional builtins a virtual
> > path which can be freely assigned in PATH and with which they can
> > be explicitly called. No more "%" in PATH and the feature can
> > be retained.
> [...]
> 
> Sounds to me like it's what dash does. It has %builtin (or
> /%builtin) where ksh93 has /opt/ast/bin.

That's kind of the point, this thread started about the trouble
that the "%" causes, using a virtual path like /usr/libexec/dash
or so instead should avoid the issue. That leaves the /path%func
case but there is precendence with FPATH as a solution if "%" is
to be completely eliminated from PATH.

> With ksh93, that's only for a few extra builtins that are only
> enabled if you add /opt/ast/bin to $PATH (or you can call them
> with "command /opt/ast/bin/cat").
> 
> I wouldn't say it's much better, it's still bad as being the
> hijacking of one namespace for something else.
> 
> dash's %builtin (as opposed to /path/to/functions%func) is
> potentially dangerous though because if you have that in your
> $PATH, you're vulnerable to attackers planting %builtin
> directories (in /tmp for instance) which will be considered by
> other things than dash/ash when looking up $PATH.
> 
> That's worked around by using /%builtin instead of %builtin
> though.
> 
> -- 
> Stephane
> --
> To unsubscribe from this list: send the line "unsubscribe dash" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Guido Berhoerster

Re: % in $PATH

[Stephane Chazelas]

2014-11-10 23:39:51 +0100, Guido Berhoerster:
[...]
> That leaves the /path%func
> case but there is precendence with FPATH as a solution if "%" is
> to be completely eliminated from PATH.
[...]

Where ash's $PATH handling is superior to $FPATH is that you can
specify which fpath component has precedence over which path
component and over builtins.

My problem as reported is that the namespace reserved by ash
(any component with % in it) was too wide (and
unnecessarily so). If you make it any component ending in %func
or %builtin, that's far less of an issue (as long as it's
documented).

I won't shed a tear if that feature is removed. I'm just feeling
it's not necessary to remove it, it could just be made more
innocuous.

-- 
Stephane