* [PATCH] replay: don't wait in run_on_cpu
@ 2021-03-11 7:14 Pavel Dovgalyuk
2021-03-15 5:59 ` Pavel Dovgalyuk
0 siblings, 1 reply; 6+ messages in thread
From: Pavel Dovgalyuk @ 2021-03-11 7:14 UTC (permalink / raw)
To: qemu-devel; +Cc: pbonzini, richard.henderson, pavel.dovgalyuk
In record/replay mode waiting for vCPU to execute
the task scheduled by run_on_cpu may lead to deadlock,
because when run_on_cpu is executed in main_loop
(e.g., in loadvm processing) it holds replay mutex.
This patch allows running scheduled task in iothread
when it holds the replay mutex.
Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
---
cpus-common.c | 9 ++++++++-
include/sysemu/replay.h | 1 +
replay/replay-internal.h | 1 -
stubs/replay-tools.c | 5 +++++
4 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/cpus-common.c b/cpus-common.c
index 6e73d3e58d..38ff510175 100644
--- a/cpus-common.c
+++ b/cpus-common.c
@@ -23,6 +23,7 @@
#include "hw/core/cpu.h"
#include "sysemu/cpus.h"
#include "qemu/lockable.h"
+#include "sysemu/replay.h"
static QemuMutex qemu_cpu_list_lock;
static QemuCond exclusive_cond;
@@ -136,7 +137,13 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data,
{
struct qemu_work_item wi;
- if (qemu_cpu_is_self(cpu)) {
+ if (qemu_cpu_is_self(cpu)
+ /*
+ * vCPU thread is waiting when replay mutex is locked
+ * and the task is not exclusive, the function may be called
+ * without other synchronization.
+ */
+ || (replay_mode != REPLAY_MODE_NONE && replay_mutex_locked())) {
func(cpu, data);
return;
}
diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
index 0f3b0f7eac..032256533b 100644
--- a/include/sysemu/replay.h
+++ b/include/sysemu/replay.h
@@ -62,6 +62,7 @@ extern char *replay_snapshot;
void replay_mutex_lock(void);
void replay_mutex_unlock(void);
+bool replay_mutex_locked(void);
/* Replay process control functions */
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 97649ed8d7..dada623527 100644
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -117,7 +117,6 @@ void replay_get_array_alloc(uint8_t **buf, size_t *size);
* synchronisation between vCPU and main-loop threads. */
void replay_mutex_init(void);
-bool replay_mutex_locked(void);
/*! Checks error status of the file. */
void replay_check_error(void);
diff --git a/stubs/replay-tools.c b/stubs/replay-tools.c
index 43296b3d4e..a42f2483d5 100644
--- a/stubs/replay-tools.c
+++ b/stubs/replay-tools.c
@@ -48,6 +48,11 @@ void replay_mutex_unlock(void)
{
}
+bool replay_mutex_locked(void)
+{
+ return false;
+}
+
void replay_register_char_driver(Chardev *chr)
{
}
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH] replay: don't wait in run_on_cpu
2021-03-11 7:14 [PATCH] replay: don't wait in run_on_cpu Pavel Dovgalyuk
@ 2021-03-15 5:59 ` Pavel Dovgalyuk
0 siblings, 0 replies; 6+ messages in thread
From: Pavel Dovgalyuk @ 2021-03-15 5:59 UTC (permalink / raw)
To: qemu-devel; +Cc: pbonzini, richard.henderson
ping
On 11.03.2021 10:14, Pavel Dovgalyuk wrote:
> In record/replay mode waiting for vCPU to execute
> the task scheduled by run_on_cpu may lead to deadlock,
> because when run_on_cpu is executed in main_loop
> (e.g., in loadvm processing) it holds replay mutex.
> This patch allows running scheduled task in iothread
> when it holds the replay mutex.
>
> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
> ---
> cpus-common.c | 9 ++++++++-
> include/sysemu/replay.h | 1 +
> replay/replay-internal.h | 1 -
> stubs/replay-tools.c | 5 +++++
> 4 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/cpus-common.c b/cpus-common.c
> index 6e73d3e58d..38ff510175 100644
> --- a/cpus-common.c
> +++ b/cpus-common.c
> @@ -23,6 +23,7 @@
> #include "hw/core/cpu.h"
> #include "sysemu/cpus.h"
> #include "qemu/lockable.h"
> +#include "sysemu/replay.h"
>
> static QemuMutex qemu_cpu_list_lock;
> static QemuCond exclusive_cond;
> @@ -136,7 +137,13 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data,
> {
> struct qemu_work_item wi;
>
> - if (qemu_cpu_is_self(cpu)) {
> + if (qemu_cpu_is_self(cpu)
> + /*
> + * vCPU thread is waiting when replay mutex is locked
> + * and the task is not exclusive, the function may be called
> + * without other synchronization.
> + */
> + || (replay_mode != REPLAY_MODE_NONE && replay_mutex_locked())) {
> func(cpu, data);
> return;
> }
> diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
> index 0f3b0f7eac..032256533b 100644
> --- a/include/sysemu/replay.h
> +++ b/include/sysemu/replay.h
> @@ -62,6 +62,7 @@ extern char *replay_snapshot;
>
> void replay_mutex_lock(void);
> void replay_mutex_unlock(void);
> +bool replay_mutex_locked(void);
>
> /* Replay process control functions */
>
> diff --git a/replay/replay-internal.h b/replay/replay-internal.h
> index 97649ed8d7..dada623527 100644
> --- a/replay/replay-internal.h
> +++ b/replay/replay-internal.h
> @@ -117,7 +117,6 @@ void replay_get_array_alloc(uint8_t **buf, size_t *size);
> * synchronisation between vCPU and main-loop threads. */
>
> void replay_mutex_init(void);
> -bool replay_mutex_locked(void);
>
> /*! Checks error status of the file. */
> void replay_check_error(void);
> diff --git a/stubs/replay-tools.c b/stubs/replay-tools.c
> index 43296b3d4e..a42f2483d5 100644
> --- a/stubs/replay-tools.c
> +++ b/stubs/replay-tools.c
> @@ -48,6 +48,11 @@ void replay_mutex_unlock(void)
> {
> }
>
> +bool replay_mutex_locked(void)
> +{
> + return false;
> +}
> +
> void replay_register_char_driver(Chardev *chr)
> {
> }
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH] replay: don't wait in run_on_cpu
@ 2021-03-29 8:05 Pavel Dovgalyuk
2021-03-29 9:42 ` Paolo Bonzini
0 siblings, 1 reply; 6+ messages in thread
From: Pavel Dovgalyuk @ 2021-03-29 8:05 UTC (permalink / raw)
To: qemu-devel; +Cc: pbonzini, richard.henderson, pavel.dovgalyuk
In record/replay mode waiting for vCPU to execute
the task scheduled by run_on_cpu may lead to deadlock,
because when run_on_cpu is executed in main_loop
(e.g., in loadvm processing) it holds replay mutex.
This patch allows running scheduled task in iothread
when it holds the replay mutex.
Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
---
cpus-common.c | 9 ++++++++-
include/sysemu/replay.h | 1 +
replay/replay-internal.h | 1 -
stubs/replay-tools.c | 5 +++++
4 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/cpus-common.c b/cpus-common.c
index 6e73d3e58d..38ff510175 100644
--- a/cpus-common.c
+++ b/cpus-common.c
@@ -23,6 +23,7 @@
#include "hw/core/cpu.h"
#include "sysemu/cpus.h"
#include "qemu/lockable.h"
+#include "sysemu/replay.h"
static QemuMutex qemu_cpu_list_lock;
static QemuCond exclusive_cond;
@@ -136,7 +137,13 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data,
{
struct qemu_work_item wi;
- if (qemu_cpu_is_self(cpu)) {
+ if (qemu_cpu_is_self(cpu)
+ /*
+ * vCPU thread is waiting when replay mutex is locked
+ * and the task is not exclusive, the function may be called
+ * without other synchronization.
+ */
+ || (replay_mode != REPLAY_MODE_NONE && replay_mutex_locked())) {
func(cpu, data);
return;
}
diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
index 0f3b0f7eac..032256533b 100644
--- a/include/sysemu/replay.h
+++ b/include/sysemu/replay.h
@@ -62,6 +62,7 @@ extern char *replay_snapshot;
void replay_mutex_lock(void);
void replay_mutex_unlock(void);
+bool replay_mutex_locked(void);
/* Replay process control functions */
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 97649ed8d7..dada623527 100644
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -117,7 +117,6 @@ void replay_get_array_alloc(uint8_t **buf, size_t *size);
* synchronisation between vCPU and main-loop threads. */
void replay_mutex_init(void);
-bool replay_mutex_locked(void);
/*! Checks error status of the file. */
void replay_check_error(void);
diff --git a/stubs/replay-tools.c b/stubs/replay-tools.c
index 43296b3d4e..a42f2483d5 100644
--- a/stubs/replay-tools.c
+++ b/stubs/replay-tools.c
@@ -48,6 +48,11 @@ void replay_mutex_unlock(void)
{
}
+bool replay_mutex_locked(void)
+{
+ return false;
+}
+
void replay_register_char_driver(Chardev *chr)
{
}
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH] replay: don't wait in run_on_cpu
2021-03-29 8:05 Pavel Dovgalyuk
@ 2021-03-29 9:42 ` Paolo Bonzini
2021-03-29 10:55 ` Pavel Dovgalyuk
0 siblings, 1 reply; 6+ messages in thread
From: Paolo Bonzini @ 2021-03-29 9:42 UTC (permalink / raw)
To: Pavel Dovgalyuk, qemu-devel; +Cc: richard.henderson
On 29/03/21 10:05, Pavel Dovgalyuk wrote:
> @@ -136,7 +137,13 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data,
> {
> struct qemu_work_item wi;
>
> - if (qemu_cpu_is_self(cpu)) {
> + if (qemu_cpu_is_self(cpu)
> + /*
> + * vCPU thread is waiting when replay mutex is locked
> + * and the task is not exclusive, the function may be called
> + * without other synchronization.
> + */
> + || (replay_mode != REPLAY_MODE_NONE && replay_mutex_locked())) {
> func(cpu, data);
> return;
> }
Is the "or" saying that the execution is using the lockstep mode? If
so, can you put it in a separate function so that it's more
self-explanatory and check if it should be used elsewhere?
Thanks,
Paolo
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] replay: don't wait in run_on_cpu
2021-03-29 9:42 ` Paolo Bonzini
@ 2021-03-29 10:55 ` Pavel Dovgalyuk
2021-03-29 12:11 ` Paolo Bonzini
0 siblings, 1 reply; 6+ messages in thread
From: Pavel Dovgalyuk @ 2021-03-29 10:55 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: richard.henderson
On 29.03.2021 12:42, Paolo Bonzini wrote:
> On 29/03/21 10:05, Pavel Dovgalyuk wrote:
>> @@ -136,7 +137,13 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func
>> func, run_on_cpu_data data,
>> {
>> struct qemu_work_item wi;
>> - if (qemu_cpu_is_self(cpu)) {
>> + if (qemu_cpu_is_self(cpu)
>> + /*
>> + * vCPU thread is waiting when replay mutex is locked
>> + * and the task is not exclusive, the function may be called
>> + * without other synchronization.
>> + */
>> + || (replay_mode != REPLAY_MODE_NONE && replay_mutex_locked())) {
>> func(cpu, data);
>> return;
>> }
>
> Is the "or" saying that the execution is using the lockstep mode? If
> so, can you put it in a separate function so that it's more
> self-explanatory and check if it should be used elsewhere?
It was replay (is that lockstep that you mentioned?).
I check that the mutex is already locked, which means, that vCPU
does nothing at this moment.
Pavel Dovgalyuk
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] replay: don't wait in run_on_cpu
2021-03-29 10:55 ` Pavel Dovgalyuk
@ 2021-03-29 12:11 ` Paolo Bonzini
0 siblings, 0 replies; 6+ messages in thread
From: Paolo Bonzini @ 2021-03-29 12:11 UTC (permalink / raw)
To: Pavel Dovgalyuk, qemu-devel; +Cc: richard.henderson
On 29/03/21 12:55, Pavel Dovgalyuk wrote:
>>>
>>
>> Is the "or" saying that the execution is using the lockstep mode? If
>> so, can you put it in a separate function so that it's more
>> self-explanatory and check if it should be used elsewhere?
>
> It was replay (is that lockstep that you mentioned?).
Lockstep in the sense that (as is the case in record/replay mode) the
I/O thread and vCPU thread execute in turns.
Paolo
> I check that the mutex is already locked, which means, that vCPU
> does nothing at this moment.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-03-29 12:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-11 7:14 [PATCH] replay: don't wait in run_on_cpu Pavel Dovgalyuk
2021-03-15 5:59 ` Pavel Dovgalyuk
2021-03-29 8:05 Pavel Dovgalyuk
2021-03-29 9:42 ` Paolo Bonzini
2021-03-29 10:55 ` Pavel Dovgalyuk
2021-03-29 12:11 ` Paolo Bonzini
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.