Re: Thread safety of coroutine-sigaltstack

[Laszlo Ersek <lersek@redhat.com>]

On 01/21/21 18:24, Paolo Bonzini wrote:
> On 21/01/21 17:44, Peter Maydell wrote:
>> On Thu, 21 Jan 2021 at 16:10, Daniel P. Berrangé <berrange@redhat.com>
>> wrote:
>>> FWIW The libucontext impl is all ASM based and has coverage for all the
>>> arches we care about:
>>>
>>>     https://github.com/kaniini/libucontext
>>>
>>> so doesn't seem like there's a need for  coroutine-asm if we can rely
>>> on libucontext for portability where neede.
>>
>> The README for that notes a couple of pretty major omissions:
>>   * it doesn't handle floating point registers
>>   * it doesn't do anything about the signal mask
>> I'm pretty sure that not handling the fp regs could cause breakage
>> for Arm at least (depending on what the compiler chooses to do
>> in the functions that work with the ucontext functions). The
>> signal mask stuff might be OK for us because of the carefully
>> limited use we make of the ucontext functions, but it would be
>> a bit of a pain to have to analyse that code for both sets of semantics.
> 
> The lack of signal mask handling is an improvement for us. :)  We want
> the signal mask to be per thread, not per coroutine.

I didn't quite get this when I first read it, but now that I'm digging
through the code, I have a follow-up comment.

According to POSIX, passing savemask=0 to sigsetjmp() may or may not
save the current signal mask, into "env". A nonzero savemask is required
to save the signal mask, but a zero savemask is not forbidden to -- it
is only not required to:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/sigsetjmp.html#tag_16_554_07

    Note that since this function is defined in terms of setjmp(), if
    savemask is zero, it is unspecified whether the signal mask is
    saved.

And I feel that's a bit of a problem, because when we first exit the
trampoline -- executed as a signal handler -- via sigsetjmp(), *all
signals* are masked, and sigsetjmp might actually stash that mask in
"tr_reenter", because savemask=0 does not suffice for forbidding that.

When we reenter the trampoline via siglongjmp(tr_reenter), and
subsequently call coroutine_bootstrap(), it's possible (per POSIX, see
above) that all signals are masked again. And then that could further be
remembered in "self->env", in coroutine_bootstrap(). Which would be
wrong IMO; co-routines in general should receive synchronous signals if
they mess up somewhere (terminating the process).

IOW, just before the call to coroutine_bootstrap(),
coroutine_trampoline() should explicitly restore the signal mask that
was in effect when qemu_coroutine_new() was entered.

Has this been a problem in practice, or should we ignore it?

IOW, should we assume "savemask=0" for *never* saving the signal mask?

The behavior of "savemask=0" is a platform trait that platforms are not
required to document (the behavior is unspecified, not
implementation-defined), so it really boils down to where this code
actually runs...

NB Linux is more specific:

https://man7.org/linux/man-pages/man3/setjmp.3.html

   sigsetjmp() and siglongjmp()
       sigsetjmp() and siglongjmp() also perform nonlocal gotos, but
       provide predictable handling of the process signal mask.

       If, and only if, the savesigs argument provided to sigsetjmp() is
       nonzero, the process's current signal mask is saved in env and
       will be restored if a siglongjmp() is later performed with this
       env.

Cue "and only if".

Thanks
Laszlo

> 
> Floating point however is an issue if they don't save-restore v8-v15
> (for aarch64, I don't remember what the AAPCS32 says).
> 
> Paolo
> 
>