[Qemu-devel] Handling signal of Qemu thread

[Qemu-devel] Handling signal of Qemu thread

[Probir Roy]

I am registering a signal handler per Qemu thread (per VCPU) and
expecting to handle it in that thread context. But I never receive the
signal on the Qemu thread that is causing the event, rather the signal
is sent to parent thread context. Can you please explain the reason
behind this? I also see that Qemu has a function called
"kvm_eat_signals". Does it have to do anything with my issue?

Regards,
Probir

Re: [Qemu-devel] [Qemu-discuss] Handling signal of Qemu thread

[Peter Maydell]

On 16 August 2018 at 21:22, Probir Roy <roy@probir.info> wrote:
> I am registering a signal handler per Qemu thread (per VCPU) and
> expecting to handle it in that thread context. But I never receive the
> signal on the Qemu thread that is causing the event, rather the signal
> is sent to parent thread context. Can you please explain the reason
> behind this? I also see that Qemu has a function called
> "kvm_eat_signals". Does it have to do anything with my issue?

Signal handling is complicated, especially when the process
has multiple threads. QEMU's strategy for it is:

 * only the iothread deals with signal handling, except for
   one or two signals which are specifically directed to a
   particular CPU thread (notably SIG_IPI)
 * other threads block all signals, so that the iothread
   will handle them (this is done as part of qemu_thread_create())
 * the iothread handles most signals synchronously, using signalfd():
   this is done in qemu_signal_init(), and is how we handle
   SIGIO, SIGALRM and SIGBUS
 * SIGINT, SIGHUP, SIGTERM are handled by the iothread, asynchronously
   (their handlers are set in os_setup_signal_handling())
 * SIGPIPE is set to SIG_IGN, so we handle pipe-closed via the
   EPIPE errno rather than via a signal
 * SIG_IPI is one of the signals for specific CPU threads; so
   it is blocked in the iothread, and enabled in CPU threads
 * kvm_eat_signals() is specifically to handle SIG_IPI, and
   affects no other signal -- if the kernel returned control
   to QEMU because of a pending signal on this CPU thread,
   we must make sure we've processed all the SIG_IPIs before
   we continue

Adding support for a new usage of signals to this design is
complicated: depending on what is going on, it might be best
handled asynchronously in the iothread, synchronously in the
iothread, or per CPU thread. What exactly are you trying to do
with your new signal ?

thanks
-- PMM

Re: [Qemu-devel] [Qemu-discuss] Handling signal of Qemu thread

[Probir Roy]

> What exactly are you trying to do
> with your new signal ?

I am implementing PEBS (Intel's Precise-Event Based Sampling)
virtualization, so that I can sample guest OS from host machine using
Linux Perf. The PEBS device is configured from host's user space as
perf event. I am registering a perf-event using "perf_event_open" in
kvm_cpu_exec:

+++ accel/kvm/kvm-all.c    2018-08-07 11:01:05.326280431 -0500
   @@ -1903,6 +1904,12 @@
     qemu_mutex_unlock_iothread();
     cpu_exec_start(cpu);

+    /* Probir: vcpu thread starting. Should call the tool thread handler*/
+    kvm__VCPU_start(); // calling perf_event_open


"kvm__VCPU_start" calling perf_event_open and a signal handler
"generic_dev_signal_handler" is set via sigaction:

// Set a signal handler for SIGUSR1
        struct sigaction sa;
        sa.sa_sigaction = generic_dev_signal_handler;
        sa.sa_mask = block_mask_dev;
        sa.sa_flags = SA_SIGINFO | SA_RESTART | SA_NODEFER | SA_ONSTACK;

I am expecting to perform some task in "generic_dev_signal_handler" at
PEBS event when the VCPU is halted. This task can be reading/writing
hardware registers at that event point. How can I achieve this in Qemu
environment?

Regards,
Probir

Re: [Qemu-devel] [Qemu-discuss] Handling signal of Qemu thread

[Paolo Bonzini]

On 20/08/2018 15:06, Peter Maydell wrote:
>  * SIG_IPI is one of the signals for specific CPU threads; so
>    it is blocked in the iothread, and enabled in CPU threads
>  * kvm_eat_signals() is specifically to handle SIG_IPI, and
>    affects no other signal -- if the kernel returned control
>    to QEMU because of a pending signal on this CPU thread,
>    we must make sure we've processed all the SIG_IPIs before
>    we continue

Also, recent kernel versions do not need to "eat" the SIG_IPI anymore.

More precisely, on older versions, SIG_IPI was temporarily unblocked
during the KVM_RUN ioctl but never delivered to the CPU thread;
therefore, kvm_eat_signal() had to remove the signal from the queue via
the sigtimedwait system call.  This was slow, especially for very large
virtual machines, due to spinlock contention in the kernel.

On newer versions, QEMU can process the SIG_IPI synchronously in the CPU
thread and set a flag that KVM_RUN reads (cpu->kvm_run->immediate_exit).
 On those new versions, the signal is never blocked and kvm_eat_signal()
does not do sigtimedwait anymore, it just resets
cpu->kvm_run->immediate_exit back to zero.

Thanks,

Paolo

Re: [Qemu-devel] [Qemu-discuss] Handling signal of Qemu thread

[Peter Maydell]

On 20 August 2018 at 16:07, Probir Roy <roy@probir.info> wrote:
>> What exactly are you trying to do
>> with your new signal ?
>
> I am implementing PEBS (Intel's Precise-Event Based Sampling)
> virtualization, so that I can sample guest OS from host machine using
> Linux Perf. The PEBS device is configured from host's user space as
> perf event. I am registering a perf-event using "perf_event_open" in
> kvm_cpu_exec:
>
> +++ accel/kvm/kvm-all.c    2018-08-07 11:01:05.326280431 -0500
>    @@ -1903,6 +1904,12 @@
>      qemu_mutex_unlock_iothread();
>      cpu_exec_start(cpu);
>
> +    /* Probir: vcpu thread starting. Should call the tool thread handler*/
> +    kvm__VCPU_start(); // calling perf_event_open
>
>
> "kvm__VCPU_start" calling perf_event_open and a signal handler
> "generic_dev_signal_handler" is set via sigaction:
>
> // Set a signal handler for SIGUSR1
>         struct sigaction sa;
>         sa.sa_sigaction = generic_dev_signal_handler;
>         sa.sa_mask = block_mask_dev;
>         sa.sa_flags = SA_SIGINFO | SA_RESTART | SA_NODEFER | SA_ONSTACK;

You cannot use SIGUSR1 for this. QEMU uses it internally as
its SIG_IPI (inter-vcpu-communication). You'll need to get
the perf code to use a different signal. (And you'll need to
manually unblock that signal on the CPU thread when you install
your handler.)

> I am expecting to perform some task in "generic_dev_signal_handler" at
> PEBS event when the VCPU is halted. This task can be reading/writing
> hardware registers at that event point.

Guest or host hardware registers? Either way, this is tricky
stuff to be trying to do in a signal handler.

thanks
-- PMM