Re: [Qemu-devel] [Bug 902148] Re: qemu-img V1.0 hangs on creating Image (0.15.1 runs)

[Stefan Hajnoczi <stefanha@gmail.com>]

On Tue, Dec 20, 2011 at 10:49 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Tue, Dec 20, 2011 at 10:25 AM, Michael Niehren
> <902148@bugs.launchpad.net> wrote:
>> here the compiled binary with your patch and the --enable-debug option
>> This one works
>
> Thanks, I'm able to reproduce the problem here with your original
> binary.  Will send an update once I've figured out what the problem
> is.

This looks like an interesting bug, perhaps a bug in gcc.  I have to
preface that with what the QEMU code is doing here is pretty tricky,
so it might be a regular bug in QEMU... :)

Here is the C function that triggers the bug:

static void coroutine_trampoline(int i0, int i1)
{
    union cc_arg arg;
    CoroutineUContext *self;
    Coroutine *co;

    arg.i[0] = i0;  <-- workaround for makecontext() limitation,
    arg.i[1] = i1;      need to pass Coroutine *co argument in as
    self = arg.p;       two ints and use a union to extract it.
    co = &self->base;

    /* Initialize longjmp environment and switch back the caller */
    if (!setjmp(self->env)) {
        longjmp(*(jmp_buf *)co->entry_arg, 1);
    }  <-- I think this part is not relevant to the bug

    while (true) {  <-- here is the loop that miscompiles
        co->entry(co->entry_arg);
        qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE);
    }
}

Here is the loop output of my compiler - this works fine:

   31bd0:       48 8b 44 24 08          mov    0x8(%rsp),%rax  <-- co->entry
   31bd5:       48 8b 78 08             mov    0x8(%rax),%rdi  <-- co->entry_arg
   31bd9:       ff 10                   callq  *(%rax)
   31bdb:       48 8b 44 24 08          mov    0x8(%rsp),%rax
   31be0:       48 8b 7c 24 10          mov    0x10(%rsp),%rdi
   31be5:       ba 02 00 00 00          mov    $0x2,%edx
   31bea:       48 8b 70 10             mov    0x10(%rax),%rsi
   31bee:       e8 2d ff ff ff          callq  31b20 <qemu_coroutine_switch>
   31bf3:       eb db                   jmp    31bd0 <coroutine_trampoline+0x40>

Here is your compiler output - this is broken:

   31c40:       4c 89 e7                mov    %r12,%rdi  <--
co->entry_args from %r12
   31c43:       ff d5                   callq  *%rbp      <--
co->entry already in %rbp
   31c45:       48 8b 3c 24             mov    (%rsp),%rdi
   31c49:       ba 02 00 00 00          mov    $0x2,%edx
   31c4e:       48 89 de                mov    %rbx,%rsi
   31c51:       e8 2a ff ff ff          callq  31b80 <qemu_coroutine_switch>
   31c56:       eb e8                   jmp    31c40 <coroutine_trampoline+0x50>

Your binary does not reload the coroutine entry function pointer or
entry_args argument fields.  As a result coroutines are not working in
your QEMU build.

It seems your compiler is disregarding the co->entry() function
pointer call and assuming co's fields will not change.  The
setjmp()/longjmp() and cc_args union make this an ugly function -
perhaps they play a part in making the compiler think it's okay to
keep stale values of co's fields around.

Richard Sandiford has suggested a way to capture what gcc is doing:

$ cd qemu/
$ rm coroutine-ucontext.o
$ make V=1 coroutine-ucontext.o

Now copy the gcc command-line that was displayed but add
-fdump-tree-all-details:

$ gcc ... -fdump-tree-all-details coroutine-ucontext.c

Collect all the gcc internals output:

$ tar czf coroutine-ucontext.tgz coroutine-ucontext.c.*

Please upload the tarball and also post your Linux distro and gcc
--version details.

Thanks,
Stefan