[Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[Paolo Bonzini]

The following changes since commit 6c5f30cad290c745f910481d0e890b3f4fad1f00:

  Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20151104' into staging (2015-11-05 10:10:57 +0000)

are available in the git repository at:

  git://github.com/bonzini/qemu.git tags/for-upstream-replay

for you to fetch changes up to ee312992a323530ea2cda8680f3a34746c72db8f:

  replay: recording of the user input (2015-11-06 10:16:03 +0100)

----------------------------------------------------------------
So here it is, let's see what happens.

----------------------------------------------------------------
Pavel Dovgalyuk (18):
      replay: global variables and function stubs
      replay: internal functions for replay log
      replay: introduce mutex to protect the replay log
      replay: introduce icount event
      cpu-exec: allow temporary disabling icount
      cpu: replay instructions sequence
      replay: interrupts and exceptions
      replay: asynchronous events infrastructure
      replay: recording and replaying clock ticks
      replay: shutdown event
      icount: improve counting for record/replay
      replay: checkpoints
      bottom halves: introduce bh call function
      replay: ptimer
      replay: initialization and deinitialization
      replay: replay blockers for devices
      replay: command line options
      replay: recording of the user input

 Makefile.objs             |   2 +
 async.c                   |   7 +-
 cpu-exec.c                |  55 ++++++--
 cpus.c                    |  64 ++++++---
 docs/replay.txt           | 168 +++++++++++++++++++++++
 exec.c                    |   2 +
 hw/bt/hci.c               |   7 +
 hw/core/ptimer.c          |   3 +-
 include/block/aio.h       |   5 +
 include/exec/exec-all.h   |   1 +
 include/qapi/qmp/qerror.h |   3 +
 include/sysemu/replay.h   | 120 ++++++++++++++++
 include/ui/input.h        |   2 +
 main-loop.c               |   3 +
 qapi-schema.json          |  18 +++
 qemu-options.hx           |   8 +-
 qemu-timer.c              |  43 +++++-
 replay/Makefile.objs      |   5 +
 replay/replay-events.c    | 279 +++++++++++++++++++++++++++++++++++++
 replay/replay-input.c     | 160 ++++++++++++++++++++++
 replay/replay-internal.c  | 206 ++++++++++++++++++++++++++++
 replay/replay-internal.h  | 140 +++++++++++++++++++
 replay/replay-time.c      |  64 +++++++++
 replay/replay.c           | 342 ++++++++++++++++++++++++++++++++++++++++++++++
 stubs/Makefile.objs       |   2 +
 stubs/replay-user.c       |  32 +++++
 stubs/replay.c            |  31 +++++
 translate-all.c           |   2 +-
 ui/input.c                |  27 ++--
 vl.c                      |  60 ++++++--
 30 files changed, 1803 insertions(+), 58 deletions(-)
 create mode 100644 docs/replay.txt
 create mode 100644 include/sysemu/replay.h
 create mode 100644 replay/Makefile.objs
 create mode 100644 replay/replay-events.c
 create mode 100644 replay/replay-input.c
 create mode 100644 replay/replay-internal.c
 create mode 100644 replay/replay-internal.h
 create mode 100644 replay/replay-time.c
 create mode 100644 replay/replay.c
 create mode 100644 stubs/replay-user.c
 create mode 100644 stubs/replay.c
-- 
2.5.0

[Qemu-devel] [PULL 07/18] replay: interrupts and exceptions

[Paolo Bonzini]

From: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>

This patch includes modifications of common cpu files. All interrupts and
exceptions occured during recording are written into the replay log.
These events allow correct replaying the execution by kicking cpu thread
when one of these events is found in the log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
Message-Id: <20150917162416.8676.57647.stgit@PASHA-ISP.def.inno>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 cpu-exec.c               | 48 ++++++++++++++++++++++++++--------
 include/sysemu/replay.h  | 17 ++++++++++++
 replay/replay-internal.h |  4 +++
 replay/replay.c          | 67 ++++++++++++++++++++++++++++++++++++++++++++++++
 stubs/replay-user.c      | 20 +++++++++++++++
 5 files changed, 146 insertions(+), 10 deletions(-)

diff --git a/cpu-exec.c b/cpu-exec.c
index a1e3d58..a57b388 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -30,6 +30,7 @@
 #if defined(TARGET_I386) && !defined(CONFIG_USER_ONLY)
 #include "hw/i386/apic.h"
 #endif
+#include "sysemu/replay.h"
 
 /* -icount align implementation. */
 
@@ -346,21 +347,25 @@ int cpu_exec(CPUState *cpu)
     uintptr_t next_tb;
     SyncClocks sc;
 
+    /* replay_interrupt may need current_cpu */
+    current_cpu = cpu;
+
     if (cpu->halted) {
 #if defined(TARGET_I386) && !defined(CONFIG_USER_ONLY)
-        if (cpu->interrupt_request & CPU_INTERRUPT_POLL) {
+        if ((cpu->interrupt_request & CPU_INTERRUPT_POLL)
+            && replay_interrupt()) {
             apic_poll_irq(x86_cpu->apic_state);
             cpu_reset_interrupt(cpu, CPU_INTERRUPT_POLL);
         }
 #endif
         if (!cpu_has_work(cpu)) {
+            current_cpu = NULL;
             return EXCP_HALTED;
         }
 
         cpu->halted = 0;
     }
 
-    current_cpu = cpu;
     atomic_mb_set(&tcg_current_cpu, cpu);
     rcu_read_lock();
 
@@ -402,10 +407,22 @@ int cpu_exec(CPUState *cpu)
                     cpu->exception_index = -1;
                     break;
 #else
-                    cc->do_interrupt(cpu);
-                    cpu->exception_index = -1;
+                    if (replay_exception()) {
+                        cc->do_interrupt(cpu);
+                        cpu->exception_index = -1;
+                    } else if (!replay_has_interrupt()) {
+                        /* give a chance to iothread in replay mode */
+                        ret = EXCP_INTERRUPT;
+                        break;
+                    }
 #endif
                 }
+            } else if (replay_has_exception()
+                       && cpu->icount_decr.u16.low + cpu->icount_extra == 0) {
+                /* try to cause an exception pending in the log */
+                cpu_exec_nocache(cpu, 1, tb_find_fast(cpu), true);
+                ret = -1;
+                break;
             }
 
             next_tb = 0; /* force lookup of first TB */
@@ -421,30 +438,40 @@ int cpu_exec(CPUState *cpu)
                         cpu->exception_index = EXCP_DEBUG;
                         cpu_loop_exit(cpu);
                     }
-                    if (interrupt_request & CPU_INTERRUPT_HALT) {
+                    if (replay_mode == REPLAY_MODE_PLAY
+                        && !replay_has_interrupt()) {
+                        /* Do nothing */
+                    } else if (interrupt_request & CPU_INTERRUPT_HALT) {
+                        replay_interrupt();
                         cpu->interrupt_request &= ~CPU_INTERRUPT_HALT;
                         cpu->halted = 1;
                         cpu->exception_index = EXCP_HLT;
                         cpu_loop_exit(cpu);
                     }
 #if defined(TARGET_I386)
-                    if (interrupt_request & CPU_INTERRUPT_INIT) {
+                    else if (interrupt_request & CPU_INTERRUPT_INIT) {
+                        replay_interrupt();
                         cpu_svm_check_intercept_param(env, SVM_EXIT_INIT, 0);
                         do_cpu_init(x86_cpu);
                         cpu->exception_index = EXCP_HALTED;
                         cpu_loop_exit(cpu);
                     }
 #else
-                    if (interrupt_request & CPU_INTERRUPT_RESET) {
+                    else if (interrupt_request & CPU_INTERRUPT_RESET) {
+                        replay_interrupt();
                         cpu_reset(cpu);
+                        cpu_loop_exit(cpu);
                     }
 #endif
                     /* The target hook has 3 exit conditions:
                        False when the interrupt isn't processed,
                        True when it is, and we should restart on a new TB,
                        and via longjmp via cpu_loop_exit.  */
-                    if (cc->cpu_exec_interrupt(cpu, interrupt_request)) {
-                        next_tb = 0;
+                    else {
+                        replay_interrupt();
+                        if (cc->cpu_exec_interrupt(cpu, interrupt_request)) {
+                            next_tb = 0;
+                        }
                     }
                     /* Don't use the cached interrupt_request value,
                        do_interrupt may have updated the EXITTB flag. */
@@ -455,7 +482,8 @@ int cpu_exec(CPUState *cpu)
                         next_tb = 0;
                     }
                 }
-                if (unlikely(cpu->exit_request)) {
+                if (unlikely(cpu->exit_request
+                             || replay_has_interrupt())) {
                     cpu->exit_request = 0;
                     cpu->exception_index = EXCP_INTERRUPT;
                     cpu_loop_exit(cpu);
diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h
index d19715f..8915523 100644
--- a/include/sysemu/replay.h
+++ b/include/sysemu/replay.h
@@ -27,4 +27,21 @@ int replay_get_instructions(void);
 /*! Updates instructions counter in replay mode. */
 void replay_account_executed_instructions(void);
 
+/* Interrupts and exceptions */
+
+/*! Called by exception handler to write or read
+    exception processing events. */
+bool replay_exception(void);
+/*! Used to determine that exception is pending.
+    Does not proceed to the next event in the log. */
+bool replay_has_exception(void);
+/*! Called by interrupt handlers to write or read
+    interrupt processing events.
+    \return true if interrupt should be processed */
+bool replay_interrupt(void);
+/*! Tries to read interrupt event from the file.
+    Returns true, when interrupt request is pending */
+bool replay_has_interrupt(void);
+
+
 #endif
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index ff4fabc..5ff1c14 100644
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -17,6 +17,10 @@
 enum ReplayEvents {
     /* for instruction event */
     EVENT_INSTRUCTION,
+    /* for software interrupt */
+    EVENT_INTERRUPT,
+    /* for emulated exceptions */
+    EVENT_EXCEPTION,
     EVENT_COUNT
 };
 
diff --git a/replay/replay.c b/replay/replay.c
index b2c6750..b4fc64a 100644
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -79,3 +79,70 @@ void replay_account_executed_instructions(void)
         replay_mutex_unlock();
     }
 }
+
+bool replay_exception(void)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_instructions();
+        replay_mutex_lock();
+        replay_put_event(EVENT_EXCEPTION);
+        replay_mutex_unlock();
+        return true;
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        bool res = replay_has_exception();
+        if (res) {
+            replay_mutex_lock();
+            replay_finish_event();
+            replay_mutex_unlock();
+        }
+        return res;
+    }
+
+    return true;
+}
+
+bool replay_has_exception(void)
+{
+    bool res = false;
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        replay_account_executed_instructions();
+        replay_mutex_lock();
+        res = replay_next_event_is(EVENT_EXCEPTION);
+        replay_mutex_unlock();
+    }
+
+    return res;
+}
+
+bool replay_interrupt(void)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_instructions();
+        replay_mutex_lock();
+        replay_put_event(EVENT_INTERRUPT);
+        replay_mutex_unlock();
+        return true;
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        bool res = replay_has_interrupt();
+        if (res) {
+            replay_mutex_lock();
+            replay_finish_event();
+            replay_mutex_unlock();
+        }
+        return res;
+    }
+
+    return true;
+}
+
+bool replay_has_interrupt(void)
+{
+    bool res = false;
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        replay_account_executed_instructions();
+        replay_mutex_lock();
+        res = replay_next_event_is(EVENT_INTERRUPT);
+        replay_mutex_unlock();
+    }
+    return res;
+}
diff --git a/stubs/replay-user.c b/stubs/replay-user.c
index 0c90bfc..cf33072 100644
--- a/stubs/replay-user.c
+++ b/stubs/replay-user.c
@@ -10,3 +10,23 @@
  */
 
 #include "sysemu/replay.h"
+
+bool replay_exception(void)
+{
+    return true;
+}
+
+bool replay_has_exception(void)
+{
+    return false;
+}
+
+bool replay_interrupt(void)
+{
+    return true;
+}
+
+bool replay_has_interrupt(void)
+{
+    return false;
+}
-- 
2.5.0

Re: [Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[Peter Maydell]

On 6 November 2015 at 09:37, Paolo Bonzini <pbonzini@redhat.com> wrote:
> The following changes since commit 6c5f30cad290c745f910481d0e890b3f4fad1f00:
>
>   Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20151104' into staging (2015-11-05 10:10:57 +0000)
>
> are available in the git repository at:
>
>   git://github.com/bonzini/qemu.git tags/for-upstream-replay
>
> for you to fetch changes up to ee312992a323530ea2cda8680f3a34746c72db8f:
>
>   replay: recording of the user input (2015-11-06 10:16:03 +0100)
>
> ----------------------------------------------------------------
> So here it is, let's see what happens.

Applied, thanks. Let's hope it doesn't break too much :-)

(Seriously, it's good to see this getting into the tree finally.
Do we have some documentation we can point to in the
release notes about how to use it?)

thanks
-- PMM

Re: [Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[Paolo Bonzini]

On 06/11/2015 13:26, Peter Maydell wrote:
> Applied, thanks. Let's hope it doesn't break too much :-)
> 
> (Seriously, it's good to see this getting into the tree finally.
> Do we have some documentation we can point to in the
> release notes about how to use it?)

Unfortunately, I don't think it's very usable yet.  It needs a
synchronous block device (such as -sd) and a framebuffer to give
instructions (since neither character devices nor network are supported
yet).

Pavel, can you prepare a page on the wiki?

Paolo

Re: [Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[Pavel Dovgalyuk]

> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo Bonzini
> On 06/11/2015 13:26, Peter Maydell wrote:
> > Applied, thanks. Let's hope it doesn't break too much :-)
> >
> > (Seriously, it's good to see this getting into the tree finally.
> > Do we have some documentation we can point to in the
> > release notes about how to use it?)
> 
> Unfortunately, I don't think it's very usable yet.  It needs a
> synchronous block device (such as -sd) and a framebuffer to give
> instructions (since neither character devices nor network are supported
> yet).
> 
> Pavel, can you prepare a page on the wiki?

Yes, how can I make it?

Pavel Dovgalyuk

Re: [Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[dovgaluk]

Paolo Bonzini писал 2015-11-06 15:59:
> On 06/11/2015 13:26, Peter Maydell wrote:
>> Applied, thanks. Let's hope it doesn't break too much :-)
>> 
>> (Seriously, it's good to see this getting into the tree finally.
>> Do we have some documentation we can point to in the
>> release notes about how to use it?)
> 
> Unfortunately, I don't think it's very usable yet.  It needs a
> synchronous block device (such as -sd) and a framebuffer to give
> instructions (since neither character devices nor network are supported
> yet).

Actually, it is already usable in some cases.
You can specify kernel and ramdisk in command line.
There shoudn't be any block devices, network and serial adapters
connected to simulator.

> Pavel, can you prepare a page on the wiki?

Some details are presented on page 
http://wiki.qemu.org/Features/record-replay

Pavel Dovgalyuk

Re: [Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[Markus Armbruster]

Peter Maydell <peter.maydell@linaro.org> writes:

> On 6 November 2015 at 09:37, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> The following changes since commit 6c5f30cad290c745f910481d0e890b3f4fad1f00:
>>
>>   Merge remote-tracking branch
>> 'remotes/juanquintela/tags/migration/20151104' into staging
>> (2015-11-05 10:10:57 +0000)
>>
>> are available in the git repository at:
>>
>>   git://github.com/bonzini/qemu.git tags/for-upstream-replay
>>
>> for you to fetch changes up to ee312992a323530ea2cda8680f3a34746c72db8f:
>>
>>   replay: recording of the user input (2015-11-06 10:16:03 +0100)
>>
>> ----------------------------------------------------------------
>> So here it is, let's see what happens.
>
> Applied, thanks. Let's hope it doesn't break too much :-)
>
> (Seriously, it's good to see this getting into the tree finally.
> Do we have some documentation we can point to in the
> release notes about how to use it?)

Can we have an entry in MAINTAINERS, please?

Re: [Qemu-devel] [PULL v3 00/18] Record/replay core for 2.5-rc1

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 390 bytes --]

On 11/11/2015 03:53 AM, Markus Armbruster wrote:

> 
> Can we have an entry in MAINTAINERS, please?
> 

A while ago, someone suggested patching checkpatch.pl to flag any new
files without a maintainer; I'm not the best perl expert to try and
write such a patch, though.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]