Feature request: exec self for NOMMU.

Feature request: exec self for NOMMU.

[Rob Landley]

I'm trying to make some nommu-friendly busybox-like tools, which means using 
vfork() instead of fork().  This means that after I fork I have to exec in 
the child to unblock the parent, and if I want to exec my current executable 
I have to find out where it lives so I can feed the path to exec().  This is 
nontrivial.

Worse, it's not always possible.  If chroot() has happened since the program 
started, there may not _be_ a path to my current executable available from 
this process's current or root directories.

What would be really nice is if I could feed a NULL path to exec on NOMMU 
systems, and have that mean "re-exec the current executable".  I can't think 
of a way to do this without kernel support.  Any opinions on whether this is 
worthwhile?

A nommu-friendly daemonize() is another use for this, by the way...

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: Feature request: exec self for NOMMU.

[David Lang]

On Tue, 26 Dec 2006, Rob Landley wrote:

> I'm trying to make some nommu-friendly busybox-like tools, which means using
> vfork() instead of fork().  This means that after I fork I have to exec in
> the child to unblock the parent, and if I want to exec my current executable
> I have to find out where it lives so I can feed the path to exec().  This is
> nontrivial.
>
> Worse, it's not always possible.  If chroot() has happened since the program
> started, there may not _be_ a path to my current executable available from
> this process's current or root directories.

does this even make sense (as a general purpose function)? if the executable 
isn't available in your path it's likly that any config files it needs are not 
available either.

> What would be really nice is if I could feed a NULL path to exec on NOMMU
> systems, and have that mean "re-exec the current executable".  I can't think
> of a way to do this without kernel support.  Any opinions on whether this is
> worthwhile?

for something like busybox/toolbox where you have different functions based on 
the name used to execute the program, which name would you use?

David Lang

> A nommu-friendly daemonize() is another use for this, by the way...
>
> Rob
>

Re: Feature request: exec self for NOMMU.

[Rob Landley]

On Tuesday 26 December 2006 6:55 pm, David Lang wrote:
> > Worse, it's not always possible.  If chroot() has happened since the 
program
> > started, there may not _be_ a path to my current executable available from
> > this process's current or root directories.
> 
> does this even make sense (as a general purpose function)? if the executable 
> isn't available in your path it's likly that any config files it needs are
> not available either.

For a statically linked busybox it makes sense, but you're right that a 
dynamically linked one is going to suck however you do it...

I guess what I really want to do is promote a vfork() to a real fork() after 
the fact.  Which sounds painful.

For the daemonize() case I want to unblock the parent and have it exit.  Too 
bad I can't call clone() with some kind of CLONE_VFORK_BACKWARDS so it blocks 
the child until the parent exits.  (Hmmm...  CLONE_STOPPED doesn't help, just 
moves the race...)

For the "new process runs code out of this executable" case what I really want 
is an actual fork, hideously expensive as that is on nommu systems.  (There 
are things I can do to minimize that.)  Maybe I can call clone() directly on 
a nommu system to get it to fork, even when the C library hasn't got it?  
Hmmm...

> for something like busybox/toolbox where you have different functions based
> on the name used to execute the program, which name would you use?

The one from argv[0].  (Notice how exec has a "path to executable" and then an 
argv array?  Nobody said that the path to executable and argv[0] had to point 
to the same string...)

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: Feature request: exec self for NOMMU.

[Denis Vlasenko]

On Wednesday 27 December 2006 00:55, David Lang wrote:
> On Tue, 26 Dec 2006, Rob Landley wrote:
> 
> > I'm trying to make some nommu-friendly busybox-like tools, which means using
> > vfork() instead of fork().  This means that after I fork I have to exec in
> > the child to unblock the parent, and if I want to exec my current executable
> > I have to find out where it lives so I can feed the path to exec().  This is
> > nontrivial.
> >
> > Worse, it's not always possible.  If chroot() has happened since the program
> > started, there may not _be_ a path to my current executable available from
> > this process's current or root directories.
> 
> does this even make sense (as a general purpose function)? if the executable 
> isn't available in your path it's likly that any config files it needs are not 
> available either.

busybox needs it in order to spawn, for example, gzip/bzip2 helper
for tar. We know that our own executable has this function.
How to execute _our own executable_? exec("/proc/self/exe")
works only if /proc is mounted. I can imagine that some embedded
people want to be able to not rely on that.
--
vda

Re: Feature request: exec self for NOMMU.

[Rob Landley]

On Tuesday 26 December 2006 11:24 pm, Denis Vlasenko wrote:
> busybox needs it in order to spawn, for example, gzip/bzip2 helper
> for tar. We know that our own executable has this function.
> How to execute _our own executable_? exec("/proc/self/exe")
> works only if /proc is mounted. I can imagine that some embedded
> people want to be able to not rely on that.

Actually, we added CONFIG_BUSYBOX_EXEC_PATH so you could feed it a different 
path to the busybox executable if you haven't got proc.  It's still a hack, 
and it still breaks when you chroot, but you can use the standalone shell 
without /proc.

Why do people chroot?  To do system recovery using busybox with the standalone 
shell and built-in commands.  They chroot into the damaged root partition to 
run some of the commands in there (shared library paths get a bit twitchy 
without the chroot), but they want to use the built-in busybox commands for 
most of it because PAM and selinux can get screwed up by passing birds, 
brightly colored wallpaper or large flowers, and when they do they interfere 
with everything.

*shrug*  This was a bigger deal a few years ago, before the invention of the 
knoppix CD...

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: Feature request: exec self for NOMMU.

[Ray Lee]

On 12/26/06, Rob Landley <rob@landley.net> wrote:
> I'm trying to make some nommu-friendly busybox-like tools, which means using
> vfork() instead of fork().  This means that after I fork I have to exec in
> the child to unblock the parent, and if I want to exec my current executable
> I have to find out where it lives so I can feed the path to exec().  This is
> nontrivial.
>
> Worse, it's not always possible.  If chroot() has happened since the program
> started, there may not _be_ a path to my current executable available from
> this process's current or root directories.

How about openning an fd to yourself at the beginning of execution, then calling
fexecve later?

Re: Feature request: exec self for NOMMU.

[Rob Landley]

On Wednesday 27 December 2006 12:13 am, Ray Lee wrote:
> How about openning an fd to yourself at the beginning of execution, then
> calling fexecve later?

I haven't got a man page for fexecve.  Does libc have it?

In the 2.6.19 kernel: "find . | xargs grep fexecve" produces no hits.

Are you sure there _is_ one?

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: Feature request: exec self for NOMMU.

[Vadim Lobanov]

On Wed, 2006-12-27 at 00:51 -0500, Rob Landley wrote:
> On Wednesday 27 December 2006 12:13 am, Ray Lee wrote:
> > How about openning an fd to yourself at the beginning of execution, then
> > calling fexecve later?
> 
> I haven't got a man page for fexecve.  Does libc have it?

It's implemented inside glibc, and uses /proc to execve() the file that
the fd points to. Here's the code from
sysdeps/unix/sysv/linux/fexecve.c:

fexecve (fd, argv, envp)
     int fd;
     char *const argv[];
     char *const envp[];
{
  ...
  /* We use the /proc filesystem to get the information.  If it is not
     mounted we fail.  */
  char buf[sizeof "/proc/self/fd/" + sizeof (int) * 3];
  __snprintf (buf, sizeof (buf), "/proc/self/fd/%d", fd);

  /* We do not need the return value.  */
  __execve (buf, argv, envp);
  ...
}

Re: Feature request: exec self for NOMMU.

[Rob Landley]

On Wednesday 27 December 2006 1:08 am, Vadim Lobanov wrote:
> On Wed, 2006-12-27 at 00:51 -0500, Rob Landley wrote:
> > On Wednesday 27 December 2006 12:13 am, Ray Lee wrote:
> > > How about openning an fd to yourself at the beginning of execution, then
> > > calling fexecve later?
> > 
> > I haven't got a man page for fexecve.  Does libc have it?
> 
> It's implemented inside glibc, and uses /proc to execve() the file that
> the fd points to.

Cute, and I can do that.  Assuming /proc is mounted in the chroot 
environment...

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: Feature request: exec self for NOMMU.

[Ray Lee]

Rob Landley wrote:
> On Wednesday 27 December 2006 1:08 am, Vadim Lobanov wrote:
>> On Wed, 2006-12-27 at 00:51 -0500, Rob Landley wrote:
>>> On Wednesday 27 December 2006 12:13 am, Ray Lee wrote:
>>>> How about openning an fd to yourself at the beginning of execution, then
>>>> calling fexecve later?
>>> I haven't got a man page for fexecve.  Does libc have it?
>> It's implemented inside glibc, and uses /proc to execve() the file that
>> the fd points to.

Oh, hmm. Then I think it won't work, will it? I'd assumed fexecve was
implemented in kernel.

> Cute, and I can do that.  Assuming /proc is mounted in the chroot 
> environment...

Maybe I'm just confused -- wouldn't be the first time -- but if it's
implemented inside userspace, then once you chroot() you won't be able
to execute the path you find via /proc, will you?

Re: Feature request: exec self for NOMMU.

[Rob Landley]

On Wednesday 27 December 2006 1:49 pm, Ray Lee wrote:
> >>> I haven't got a man page for fexecve.  Does libc have it?
> >> It's implemented inside glibc, and uses /proc to execve() the file that
> >> the fd points to.
> 
> Oh, hmm. Then I think it won't work, will it? I'd assumed fexecve was
> implemented in kernel.

It sort of is.  Through the /proc filesystem, the kernel provides a path 
through which to open any arbitrary file descriptor, thus providing 
the "path" argument that the exec syscall requires.

> > Cute, and I can do that.  Assuming /proc is mounted in the chroot 
> > environment...
> 
> Maybe I'm just confused -- wouldn't be the first time -- but if it's
> implemented inside userspace, then once you chroot() you won't be able
> to execute the path you find via /proc, will you?

You need /proc mounted in the new chroot for it to work.  It's not a complete 
solution, but an incremental improvement over the previous hack.

Of course today what you can do is copy busybox into the top directory of the 
new chroot directory, execute that via the standard chroot command, 
run "/busybox mount -n -t procfs /proc /proc", and then let the 
current /proc/self/exe logic handle things from there.  (Or configure busybox 
to use /busybox instead of /proc/self/exe as the re-exec-self path.)

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: Feature request: exec self for NOMMU.

[Denis Vlasenko]

On Wednesday 27 December 2006 06:13, Ray Lee wrote:
> On 12/26/06, Rob Landley <rob@landley.net> wrote:
> > I'm trying to make some nommu-friendly busybox-like tools, which means using
> > vfork() instead of fork().  This means that after I fork I have to exec in
> > the child to unblock the parent, and if I want to exec my current executable
> > I have to find out where it lives so I can feed the path to exec().  This is
> > nontrivial.
> >
> > Worse, it's not always possible.  If chroot() has happened since the program
> > started, there may not _be_ a path to my current executable available from
> > this process's current or root directories.
> 
> How about openning an fd to yourself at the beginning of execution, then calling
> fexecve later?

This solves chroot problem. How to find path-to-yourself reliably
(for one, without using /proc/self/exe) is not obvious to me.
--
vda

Re: Feature request: exec self for NOMMU.

[Rob Landley]

On Wednesday 27 December 2006 1:35 pm, Denis Vlasenko wrote:
> This solves chroot problem. How to find path-to-yourself reliably
> (for one, without using /proc/self/exe) is not obvious to me.

Been there, done that.  Both my toybox and Firmware Linux projects do this.  
In FWL it's line 115 of this file:
http://landley.net/hg/firmware?f=937346748ff4;file=sources/toys/gcc-uClibc.c

It's essentially the logic of the command line "which" utility applied to 
argv[0].  If argv[0] has a relative or absolute path, then it's vs cwd (this 
has to happen when you first run the program, before you cd).  If argv[0] has 
no path then look at $PATH.

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: Feature request: exec self for NOMMU.

[Denis Vlasenko]

On Wednesday 27 December 2006 22:03, Rob Landley wrote:
> On Wednesday 27 December 2006 1:35 pm, Denis Vlasenko wrote:
> > This solves chroot problem. How to find path-to-yourself reliably
> > (for one, without using /proc/self/exe) is not obvious to me.
> 
> Been there, done that.  Both my toybox and Firmware Linux projects do this.  
> In FWL it's line 115 of this file:
> http://landley.net/hg/firmware?f=937346748ff4;file=sources/toys/gcc-uClibc.c
> 
> It's essentially the logic of the command line "which" utility applied to 
> argv[0].  If argv[0] has a relative or absolute path, then it's vs cwd (this 
> has to happen when you first run the program, before you cd).  If argv[0] has 
> no path then look at $PATH.

Yes Rob, I know it can be done like this. But we don't want this.
In the tar example, we want :

'Run my own binary again, with parameters: "zcat" "a.tar.gz",
even if there is no [/usr][/local]/bin/zcat -> busybox link anywhere'

We do not want to _search for_ zcat. We want to reexec our own binary.
--
vda

Re: Feature request: exec self for NOMMU.

[Rob Landley]

On Wednesday 27 December 2006 9:48 pm, Denis Vlasenko wrote:
> Yes Rob, I know it can be done like this. But we don't want this.
> In the tar example, we want :
> 
> 'Run my own binary again, with parameters: "zcat" "a.tar.gz",
> even if there is no [/usr][/local]/bin/zcat -> busybox link anywhere'
> 
> We do not want to _search for_ zcat. We want to reexec our own binary.

If we find our own binary, we can reexec it.  What we search for isn't zcat, 
it's argv[0], and the search needs to be done in main() before any logic that 
can chdir or set $PATH gets called.  Then save that path until we need it.

The kernel does not currently provide an easy way to do exec ourselves, but we 
can do it ourself.  (And this is a way to do it _without_ proc.)

The problem is, there's no guarante that argv[0] is actually the first 
argument to exec(), it can be any arbitrary string.  (In fact, if tar wants 
to re-exec itself as zcat, we can take advantage of this with 
execv("/blah/tar", ["zcat", "-"]);)  So it's still a hack.  It should work if 
we're called from a shell, but not necessarily from elsewhere.

*shrug*  Kernel support for re-execing ourself would be nice, especially in 
combination with vfork().  If not, I'll figure something out and make it work 
in toybox.  There are a half-dozen non-kernel approaches, all varying degrees 
of hackish.  (And daemonize() can probably be done with clone().)

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery