script relative shebang

script relative shebang

[Boris Rybalkin]

Hi All,

I would like to know if any changes to parsing '#!' script header line
are accepted in particular having ability to run interpreter from
relative to the script path?

Something like:

#!{dirname}/python/bin/python

Where {dirname} is a special keyword replaced with dirname of a script.

This is useful for apps which bring dependencies with them and do not
want to hardcode absolute paths as target fs layout is not known or
has to be flexible.

As I understand the implementation is here and I could probably provide a patch:
fs/binfmt_script.c

Sorry if this is not the right place to discuss these things.

Thank you.


--
Boris Rybalkin
ribalkin@gmail.com

Re: script relative shebang

[Nicolai Stange]

Hi Boris,

Boris Rybalkin <ribalkin@gmail.com> writes:

> I would like to know if any changes to parsing '#!' script header line
> are accepted in particular having ability to run interpreter from
> relative to the script path?
>
> Something like:
>
> #!{dirname}/python/bin/python
>
> Where {dirname} is a special keyword replaced with dirname of a script.

Just for the record, this can already be done without any help from the
kernel:

Assuming the following demonstration directory layout

  <some_test_dir>/subdir/catself
  <some_test_dir>/relshebang

where catself.sh is your "interpreter":

  #!/bin/sh
  tail -n +2 $1

and relshebang is your script file invoking the toy interpreter from its
shebang as follows:

  #!/usr/bin/gawk {exit system("/bin/sh -c 'exec \"$(dirname \"$0\")\"/subdir/catself \"$0\"' " FILENAME);}
  Hello world.


You don't necessarily need to use gawk here, anything being able to do
system() and taking some code snippet from its first argument will
certainly work.

If this is too ugly, you could also write your own wrapper a la
/usr/bin/env and install that at some central location.


Best,

Nicolai

Re: script relative shebang

[Boris]

Hi Nicolai,

Yes, I think this is too ugly:

#!/usr/bin/gawk {exit system("/bin/sh -c 'exec \"$(dirname \"$0\")\"/subdir/catself \"$0\"' " FILENAME);}

Imagine you have that feature in your kernel would you rather use:

#!{dirname}/subdir/catself

You second advice involves changing root fs which is not desirable in copy-deployment apps (bring all the dependencies)

This more about making kernel "user" friendly.

Thank you.

On 1 June 2016 00:02:05 BST, Nicolai Stange <nicstange@gmail.com> wrote:
>Hi Boris,
>
>Boris Rybalkin <ribalkin@gmail.com> writes:
>
>> I would like to know if any changes to parsing '#!' script header
>line
>> are accepted in particular having ability to run interpreter from
>> relative to the script path?
>>
>> Something like:
>>
>> #!{dirname}/python/bin/python
>>
>> Where {dirname} is a special keyword replaced with dirname of a
>script.
>
>Just for the record, this can already be done without any help from the
>kernel:
>
>Assuming the following demonstration directory layout
>
>  <some_test_dir>/subdir/catself
>  <some_test_dir>/relshebang
>
>where catself.sh is your "interpreter":
>
>  #!/bin/sh
>  tail -n +2 $1
>
>and relshebang is your script file invoking the toy interpreter from
>its
>shebang as follows:
>
>#!/usr/bin/gawk {exit system("/bin/sh -c 'exec \"$(dirname
>\"$0\")\"/subdir/catself \"$0\"' " FILENAME);}
>  Hello world.
>
>
>You don't necessarily need to use gawk here, anything being able to do
>system() and taking some code snippet from its first argument will
>certainly work.
>
>If this is too ugly, you could also write your own wrapper a la
>/usr/bin/env and install that at some central location.
>
>
>Best,
>
>Nicolai

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: script relative shebang

[Bernd Petrovitsch]

Hi all!

On Wed, 2016-06-01 at 09:00 +0100, Boris wrote:
[...]
> Yes, I think this is too ugly:
> 
> #!/usr/bin/gawk {exit system("/bin/sh -c 'exec \"$(dirname
> \"$0\")\"/subdir/catself \"$0\"' " FILENAME);}

The simplest solution for an application today is to provide a wrapper
script/program (to be placed in /usr/bin or /usr/local/bin) which knows
where to search for the applications executables - if adding some
directories to $PATH is not enough.

> Imagine you have that feature in your kernel would you rather use:
> 
> #!{dirname}/subdir/catself

And what if "{dirname}" (or any other character sequence) happens to be
a valid pathname?

> You second advice involves changing root fs which is not desirable in
> copy-deployment apps (bring all the dependencies)
> 
> This more about making kernel "user" friendly.

User-friendlyness is not the job of the kernel ...

[ Fullquote deleted as it's a bad habit ]

MfG,
	Bernd
-- 
Bernd Petrovitsch                  Email : bernd@petrovitsch.priv.at
                     LUGA : http://www.luga.at

Re: script relative shebang

[Austin S. Hemmelgarn]

On 2016-06-01 04:00, Boris wrote:
> Hi Nicolai,
>
> Yes, I think this is too ugly:
>
> #!/usr/bin/gawk {exit system("/bin/sh -c 'exec \"$(dirname \"$0\")\"/subdir/catself \"$0\"' " FILENAME);}
>
> Imagine you have that feature in your kernel would you rather use:
>
> #!{dirname}/subdir/catself
The problem with using a keyword is that current VFS standards on Linux 
mean that most filesystems that are native to Linux support _any_ 
character in a path name component except for a null byte.  Because of 
that, you can't from a practical perspective choose a keyword that is 
guaranteed to not clash with any path name, unless you want a 256 
character long keyword (which would be worse than the current option).
>
> You second advice involves changing root fs which is not desirable in copy-deployment apps (bring all the dependencies)
Not necessarily, include the wrapper in the app itself.  It's not hard 
even in shell script to figure out where you're being run from, so it's 
really not all that hard to handle this.

In bash, you can do the following in a script to get the canonical path 
to you're script (in an interactive shell, it will just point you at the 
bash executable):
SELF=$_
SELF=$(realpath ${SELF})

This only works if it's done at the top level of a script (not within a 
function).

If you go with the wrapper option, you can find your canonical path name 
in /proc/self/exe (note that this won't work right when doing something 
like `readlink /proc/self/exe` from an interpreter, as that will return 
the path to readlink), or just look up your PID and check /proc/PID/exe.

Also, the convention for such things on most UNIX like systems is to 
install wrappers, often using symlinks, in some location that is listed 
in $PATH.  For example, on many Linux systems, Dropbox gets installed 
into /opt/dropbox, and then a link for it is created in /opt/bin to 
point at the core program itself.  If it's a GUI only app, you could 
also just use .desktop files to point at the tools instead of symlinks, 
Google Chrome does this for example.

Using the bash example above, the following snippet could be used as a 
wrapper by placing it the top level directory of the above example and 
then symlinking to it from somewhere in $PATH:

#!/bin/bash
SELF=$_
SOURCE=$(dirname $(realpath ${SELF}))
exec ${SOURCE}/subdir/catself $@

You could then use that script as your interpreter.

Re: script relative shebang

[Boris Rybalkin]

Sorry for insisting, but I would like to explore potential solutions
for fixing the root problem (missing relative shebang),
I know there are ways to workaround that, but I would like to make
sure the proper fix is not possible.
I understood that it is too late to introduce additional keywords
after #! as existing systems expect fs path there, OK.
But what about changing #! itself, is it possible to introduce another
special sequence like #? to denote a relative mode:

#?python/bin/python



-- 
Boris Rybalkin
ribalkin@gmail.com

Re: script relative shebang

[Ken Moffat]

On Thu, Jun 02, 2016 at 01:04:46AM +0100, Boris Rybalkin wrote:
> Sorry for insisting, but I would like to explore potential solutions
> for fixing the root problem (missing relative shebang),
> I know there are ways to workaround that, but I would like to make
> sure the proper fix is not possible.
> I understood that it is too late to introduce additional keywords
> after #! as existing systems expect fs path there, OK.
> But what about changing #! itself, is it possible to introduce another
> special sequence like #? to denote a relative mode:
> 
> #?python/bin/python
> 
If you are able to get that accepted, it will only work on linux
systems running such recent kernels.  For your own systems, you can
of course do whatever you wish.  But for public availability you
will then need to wait several years until your target linux users
can be expected to have moved to a suitable kernel (presumably the
*next* long-term stable kernel after the change is accepted : I
guess that version is perhaps the best part of a year away even if
your change got accepted into 4.8, and then you need your users'
distros to move to it).

To me, that doesn't seem worth the trouble (to you) of coding it,
getting it reviewed and eventually accepted, and then fixing up
whatever problems arise after it gets into linux-next [ problems
will always appear, even if the new code turns out not to be the
cause ].

And first, you have to persuade somebody influential that this is a
good thing to do, particularly when people have suggested
alternative approaches.  I don't count, but at the moment I've not
seen any good reasons why the kernel should be changed to support
this.

But it's your time, and your itch to scratch.

ĸen
-- 
I had to walk fifteen miles to school, barefoot in the snow.  Uphill both ways.

Re: script relative shebang

[Boris]

I still think it is a good thing to do.
I will try to implement it on github and may be some day someone influential will help me with that :)

On 2 June 2016 05:19:34 BST, Ken Moffat <zarniwhoop@ntlworld.com> wrote:
>On Thu, Jun 02, 2016 at 01:04:46AM +0100, Boris Rybalkin wrote:
>> Sorry for insisting, but I would like to explore potential solutions
>> for fixing the root problem (missing relative shebang),
>> I know there are ways to workaround that, but I would like to make
>> sure the proper fix is not possible.
>> I understood that it is too late to introduce additional keywords
>> after #! as existing systems expect fs path there, OK.
>> But what about changing #! itself, is it possible to introduce
>another
>> special sequence like #? to denote a relative mode:
>> 
>> #?python/bin/python
>> 
>If you are able to get that accepted, it will only work on linux
>systems running such recent kernels.  For your own systems, you can
>of course do whatever you wish.  But for public availability you
>will then need to wait several years until your target linux users
>can be expected to have moved to a suitable kernel (presumably the
>*next* long-term stable kernel after the change is accepted : I
>guess that version is perhaps the best part of a year away even if
>your change got accepted into 4.8, and then you need your users'
>distros to move to it).
>
>To me, that doesn't seem worth the trouble (to you) of coding it,
>getting it reviewed and eventually accepted, and then fixing up
>whatever problems arise after it gets into linux-next [ problems
>will always appear, even if the new code turns out not to be the
>cause ].
>
>And first, you have to persuade somebody influential that this is a
>good thing to do, particularly when people have suggested
>alternative approaches.  I don't count, but at the moment I've not
>seen any good reasons why the kernel should be changed to support
>this.
>
>But it's your time, and your itch to scratch.
>
>ĸen

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.