Re: [uml-devel] SYSCALL, ptrace and syscall restart breakages (Re: [RFC] weird crap with vdso on uml/i386)

[Al Viro <viro@ZenIV.linux.org.uk>]

On Tue, Aug 23, 2011 at 03:17:18AM +0100, Al Viro wrote:

> I have a very strong suspicion that I know what will turn out to be involved
> into that - the page eviction done by sys_brk().  Note that dirtying this
> sucker is really necessary - without *s = 0 it won't segfault at all.  With
> it we get a segfault described above.
> 
> And page eviction on uml is nasty and convoluted as hell.  It has to do
> munmap() on process' VM.  Which is done in a rather sick way - we have a
> stub present in address space of all processes, with a function that
> does a given series of mmap/munmap/mprotect and traps itself.  Guest
> kernel puts arguments for that sucker into a shared data page and continues
> the process into that function.  Once it's done, we get the damn thing
> stopped again, nice and ready for us to continue dealing with it.
> 
> Something in that shitstorm of ptrace() calls ends up doing SETREGS
> when victim sits on the way out of (host) syscall.  Boom...

Almost, but not quite.  What happens is:
* process hits syscall insn
* it's stopped and tracer (guest kernel) does GETREGS
	+ looks at the registers (mapped to the normal layout)
	+ decides to call sys_brk()
	+ notices pages to kick out
	+ queues munmap request for stub
* tracer does SETREGS, pointing the child's eip to stub and sp to stub stack
* tracer does CONT, letting the child run
* child finishes with syscall insn, carefully preserving ebp.  It returns to
  userland, in the beginning of the stub.
* child does munmap() and hits int 3 in the end of stub.
* the damn thing is stopped again.  The tracer had been waiting for it.
* tracer finishes with sys_brk() and returns success.
* it does SETREGS, setting eax to return value, eip to original return
address of syscall insn... and ebp to what it had in regs.bp.  I.e. the
damn arg6 value.

And we are fucked.  It doesn't happen in syscall handler.  It's int3().
Having no idea that this request to set ebp should be interpreted in
a really different way - "put the value I asked to put into ecx here,
please, and ignore this one".

Sigh...  The really ugly part is that ebp can be changed by the stuff
done in stub - it's not just munmap, it can do mmap as well.  We can,
in principle, save ebp on its stack and restore it before trapping.
Then uml kernel could, in theory, replace that SETREGS with a bunch of
POKEUSER, leaving ebp alone.  Ho-hum...  In principle, that might even
be not too horrible - we need eax/eip/esp, of course, but the rest
could be dealt with by the same trick - have it pushed/popped in the
stub and to hell with wasting syscalls on setting them...

Anyway, bedtime for me.  I'll look into that again in the morning...