[Qemu-devel] [RFC PATCH v3 00/49] Deterministic replay and reverse execution

[Qemu-devel] [RFC PATCH v3 00/49] Deterministic replay and reverse execution

[Pavel Dovgalyuk]

This set of patches is related to the reverse execution and deterministic 
replay of qemu execution  Our implementation of deterministic replay can 
be used for deterministic and reverse debugging of guest code through gdb 
remote interface.

Execution recording writes non-deterministic events log, which can be later 
used for replaying the execution anywhere and for unlimited number of times. 
It also supports checkpointing for faster rewinding during reverse debugging. 
Execution replaying reads the log and replays all non-deterministic events 
including external input, hardware clocks, and interrupts.

Reverse execution has the following features:
 * Deterministically replays whole system execution and all contents of the memory,
   state of the hadrware devices, clocks, and screen of the VM.
 * Writes execution log into the file for latter replaying for multiple times 
   on different machines.
 * Supports i386, x86_64, and ARM hardware platforms.
 * Performs deterministic replay of all operations with keyboard, mouse, network adapters,
   audio devices, serial interfaces, and physical USB devices connected to the emulator.
 * Provides support for gdb reverse debugging commands like reverse-step and reverse-continue.
 * Supports auto-checkpointing for convenient reverse debugging.

Usage of the record/replay:
 * First, record the execution, by adding '-record fname=replay.bin' to the
   command line.
 * Then you can replay it for the multiple times by using another command
   line option: '-replay fname=replay.bin'
 * Virtual machine should have at least one virtual disk, which is used to
   store checkpoints. If you want to enable automatic checkpointing, simply
   add ',period=XX' to record options, where XX is the checkpointing period
   in seconds.
 * Using of the network adapters in record/replay mode is possible with 
   the following command-line options:
   - '-net user' (or another host adapter) in record mode
   - '-net replay' in replay mode. Every host network adapter should be
     replaced by 'replay' when replaying the execution.
 * Reverse debugging can be used through gdb remote interface.
   reverse-stepi and reverse-continue commands are supported. Other reverse
   commands should also work, because they reuse these ones.
 * Monitor is extended by the following commands:
   - replay_info - prints information about replay mode and current step
     (number of instructions executed)
   - replay_break - sets "breakpoint" at the specified instructions count.
   - replay_seek - rewinds (using the checkpoints, if possible) to the
     specified step of replay log.

Paper with short description of deterministic replay implementation:
http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html

Modifications of qemu include:
 * adding missed fields of the virtual devices' states to the vmstate 
   structures to allow deterministic saving and restoring the VM state
 * adding virtual clock-based timers to vmstate structures, because virtual 
   clock is the part of the virtual machine state
 * modification of block layer to support automatic creation of the overlay
   files to store the changes and snapshots while recording
 * disabling of system reset while loading VM state to avoid generating of
   interrupts by reset handlers
 * adding warpers for clock and time functions to save their return
   values in the log
 * saving different asynchronous events (e.g. system shutdown) into the log
 * synchronization of the bottom halves execution
 * synchronization of the threads from thread pool
 * recording/replaying user input (mouse and keyboard), input from virtual
   serial ports, incoming network packets, input from connected USB devices
 * adding HMP/QMP commands to monitor for controlling replay execution

v3 changes:
 * Fixed bug with replay of the aio write operations.
 * Added virtual clock based on replay icount.
 * Removed duplicated saving of interrupt_request CPU field.
 * Fixed some coding style issues.
 * Renamed QMP commands for controlling reverse execution (as suggested by Eric Blake)
 * Replay mode and submode implemented as QAPI enumerations (as suggested by Eric Blake)
 * Added description and example for replay-info command (as suggested by Eric Blake)
 * Added information about the current breakpoint to the output of replay-info (as suggested by Eric Blake)
 * Updated version id for HPET vmstate (as suggested by Paolo Bonzini)
 * Removed static fields from parallel vmstate (as suggested by Paolo Bonzini)
 * New vmstate fields for mc146818rtc, pckbd, kvmapic, serial, fdc, rtl8139 moved to subsection (as suggested by Paolo Bonzini)
 * Disabled textmode cursor blinking, when virtual machine is stopped (as suggested by Paolo Bonzini)
 * Extracted saving of exception_index to separate patch (as suggested by Paolo Bonzini)

v2 changes:
 * Patches are split to be reviewable and bisectable (as suggested by Kirill Batuzov)
 * Added QMP versions of replay commands (as suggested by Eric Blake)
 * Removed some optional features of replay to make patches cleaner
 * Minor changes and code cleanup were made

---

Pavel Dovgalyuk (49):
      acpi: accurate overflow check
      integratorcp: adding vmstate for save/restore
      pcspk: adding vmstate for save/restore
      fdc: adding vmstate for save/restore
      parallel: adding vmstate for save/restore
      serial: fixing vmstate for save/restore
      kvmapic: fixing loading vmstate
      hpet: fixing saving and loading process
      pckbd: adding new fields to vmstate
      rtl8139: adding new fields to vmstate
      piix: do not raise irq while loading vmstate
      mc146818rtc: add missed field to vmstate
      pl031: add missed field to vmstate
      ide pci: reset status field before loading the vmstate
      softmmu: fixing usage of cpu_st/ld* from helpers
      target-i386: update fp status fix
      migration: add vmstate for int8 and char arrays
      replay: global variables and function stubs
      block: add suffix parameter to bdrv_open functions
      sysemu: system functions for replay
      replay: internal functions for replay log
      cpu: invent instruction count for accurate replay
      target-arm: instructions counting code for replay
      target-i386: instructions counting code for replay
      replay: interrupts and exceptions
      vga: do not use virtual clock for blinking cursor
      replay: asynchronous events infrastructure
      replay: recording and replaying clock ticks
      replay: recording and replaying different timers
      replay: shutdown event
      replay: checkpoints
      vmclock: add virtual clock based on replay icount
      replay: bottom halves
      replay: replay aio requests
      replay: thread pool
      pl031: vmstate in replay mode
      replay: initialization and deinitialization
      replay: command line options
      replay: snapshotting the virtual machine
      replay: recording of the user input
      tap-win32: destroy the thread at exit
      replay: network packets record/replay
      replay: audio data record/replay
      replay: serial port
      replay: USB passthrough
      replay: replay_info command
      replay: replay_break command
      replay: replay_seek_step command
      gdbstub: reverse debugging


 Makefile.target                  |    1 
 arch_init.c                      |    8 
 async.c                          |   46 ++
 audio/audio.c                    |   14 +
 audio/audio_win_int.h            |    3 
 audio/winwaveaudio.c             |  167 +++++++--
 block.c                          |  140 ++++++-
 block/blkdebug.c                 |    2 
 block/blkverify.c                |    5 
 block/cow.c                      |    2 
 block/qcow.c                     |    2 
 block/qcow2.c                    |   10 -
 block/qed.c                      |    2 
 block/raw-posix.c                |    6 
 block/raw-win32.c                |    4 
 block/sheepdog.c                 |    4 
 block/vmdk.c                     |    8 
 block/vvfat.c                    |    2 
 blockdev.c                       |   14 -
 cpu-exec.c                       |   34 +-
 cpus.c                           |  101 +++++
 dma-helpers.c                    |   10 -
 exec.c                           |   12 +
 gdbstub.c                        |   79 +++-
 hmp-commands.hx                  |   41 ++
 hw/acpi/core.c                   |    7 
 hw/arm/integratorcp.c            |   38 ++
 hw/audio/pcspk.c                 |   18 +
 hw/block/fdc.c                   |   85 ++++
 hw/block/virtio-blk.c            |   10 -
 hw/char/parallel.c               |   20 +
 hw/char/serial.c                 |  264 ++++++++++++--
 hw/display/vga.c                 |    6 
 hw/i386/kvmvapic.c               |   22 +
 hw/ide/ahci.c                    |    4 
 hw/ide/atapi.c                   |    9 
 hw/ide/core.c                    |   18 +
 hw/ide/pci.c                     |   11 +
 hw/input/pckbd.c                 |   53 +++
 hw/intc/apic_common.c            |   56 +++
 hw/net/rtl8139.c                 |   50 +++
 hw/pci-host/piix.c               |   22 +
 hw/timer/arm_timer.c             |    2 
 hw/timer/hpet.c                  |   15 -
 hw/timer/mc146818rtc.c           |   37 ++
 hw/timer/pl031.c                 |   38 ++
 hw/usb/hcd-uhci.c                |    2 
 hw/usb/host-libusb.c             |  551 +++++++++++++++++++----------
 include/block/aio.h              |   18 +
 include/block/block.h            |   22 +
 include/block/thread-pool.h      |    4 
 include/exec/cpu-defs.h          |    2 
 include/exec/cpu_ldst_template.h |   31 +-
 include/exec/exec-all.h          |   31 ++
 include/hw/host-libusb.h         |  105 ++++++
 include/hw/i386/apic_internal.h  |    2 
 include/migration/vmstate.h      |   13 +
 include/qemu-common.h            |    3 
 include/qemu/main-loop.h         |    1 
 include/qemu/timer.h             |   48 ++-
 include/qom/cpu.h                |    3 
 include/sysemu/char.h            |   25 +
 include/sysemu/cpus.h            |    1 
 include/sysemu/sysemu.h          |    2 
 include/ui/input.h               |    2 
 main-loop.c                      |    5 
 monitor.c                        |   58 +++
 net/Makefile.objs                |    1 
 net/clients.h                    |    3 
 net/dump.c                       |    6 
 net/hub.c                        |    1 
 net/net-replay.c                 |   68 ++++
 net/net.c                        |    7 
 net/slirp.c                      |   14 +
 net/socket.c                     |   35 ++
 net/tap-win32.c                  |   25 +
 net/tap.c                        |   24 +
 net/vde.c                        |   14 +
 qapi-schema.json                 |  102 +++++
 qemu-char.c                      |   56 +++
 qemu-img.c                       |    6 
 qemu-io-cmds.c                   |    2 
 qemu-io.c                        |    7 
 qemu-nbd.c                       |    2 
 qemu-options.hx                  |   30 ++
 qemu-timer.c                     |   52 ++-
 qmp-commands.hx                  |   75 ++++
 replay/Makefile.objs             |   12 +
 replay/replay-audio.c            |  228 ++++++++++++
 replay/replay-char.c             |  100 +++++
 replay/replay-debug.c            |  154 ++++++++
 replay/replay-events.c           |  382 ++++++++++++++++++++
 replay/replay-icount.c           |  130 +++++++
 replay/replay-input.c            |  107 ++++++
 replay/replay-internal.c         |  159 ++++++++
 replay/replay-internal.h         |  224 ++++++++++++
 replay/replay-net.c              |  191 ++++++++++
 replay/replay-qmp.c              |   58 +++
 replay/replay-time.c             |  183 ++++++++++
 replay/replay-usb.c              |  188 ++++++++++
 replay/replay.c                  |  718 ++++++++++++++++++++++++++++++++++++++
 replay/replay.h                  |  214 +++++++++++
 savevm.c                         |   33 +-
 slirp/slirp.c                    |    9 
 softmmu_template.h               |   18 +
 stubs/Makefile.objs              |    1 
 stubs/replay.c                   |   47 ++
 target-arm/Makefile.objs         |    1 
 target-arm/helper.h              |    3 
 target-arm/replay_helper.c       |   38 ++
 target-arm/translate.c           |   62 +++
 target-i386/Makefile.objs        |    1 
 target-i386/cpu.c                |    1 
 target-i386/cpu.h                |    8 
 target-i386/fpu_helper.c         |    5 
 target-i386/helper.h             |    3 
 target-i386/machine.c            |    2 
 target-i386/replay_helper.c      |   38 ++
 target-i386/translate.c          |   93 ++++-
 tests/test-thread-pool.c         |    7 
 thread-pool.c                    |   53 ++-
 trace-events                     |    2 
 translate-all.c                  |   15 +
 ui/input.c                       |   80 +++-
 util/iov.c                       |    4 
 vl.c                             |  122 ++++++
 vmstate.c                        |    6 
 127 files changed, 6102 insertions(+), 559 deletions(-)
 create mode 100755 include/hw/host-libusb.h
 create mode 100755 net/net-replay.c
 create mode 100755 replay/Makefile.objs
 create mode 100755 replay/replay-audio.c
 create mode 100755 replay/replay-char.c
 create mode 100755 replay/replay-debug.c
 create mode 100755 replay/replay-events.c
 create mode 100755 replay/replay-icount.c
 create mode 100755 replay/replay-input.c
 create mode 100755 replay/replay-internal.c
 create mode 100755 replay/replay-internal.h
 create mode 100755 replay/replay-net.c
 create mode 100755 replay/replay-qmp.c
 create mode 100755 replay/replay-time.c
 create mode 100755 replay/replay-usb.c
 create mode 100755 replay/replay.c
 create mode 100755 replay/replay.h
 create mode 100755 stubs/replay.c
 create mode 100755 target-arm/replay_helper.c
 create mode 100755 target-i386/replay_helper.c

-- 
Pavel Dovgalyuk

[Qemu-devel] [RFC PATCH v3 01/49] acpi: accurate overflow check

[Pavel Dovgalyuk]

Compare clock in ns, because acpi_pm_tmr_update uses rounded
to ns value instead of ticks.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/acpi/core.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/hw/acpi/core.c b/hw/acpi/core.c
index a7368fb..51913d6 100644
--- a/hw/acpi/core.c
+++ b/hw/acpi/core.c
@@ -376,8 +376,11 @@ static void acpi_notify_wakeup(Notifier *notifier, void *data)
 /* ACPI PM1a EVT */
 uint16_t acpi_pm1_evt_get_sts(ACPIREGS *ar)
 {
-    int64_t d = acpi_pm_tmr_get_clock();
-    if (d >= ar->tmr.overflow_time) {
+    /* Compare ns-clock, not PM timer ticks, because
+       acpi_pm_tmr_update function uses ns for setting the timer. */
+    int64_t d = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+    if (d >= muldiv64(ar->tmr.overflow_time,
+                      get_ticks_per_sec(), PM_TIMER_FREQUENCY)) {
         ar->pm1.evt.sts |= ACPI_BITMASK_TIMER_STATUS;
     }
     return ar->pm1.evt.sts;

[Qemu-devel] [RFC PATCH v3 02/49] integratorcp: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the integratorcp device state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/arm/integratorcp.c |   38 +++++++++++++++++++++++++++++++++++++-
 1 files changed, 37 insertions(+), 1 deletions(-)

diff --git a/hw/arm/integratorcp.c b/hw/arm/integratorcp.c
index 0e476c3..e5d5751 100644
--- a/hw/arm/integratorcp.c
+++ b/hw/arm/integratorcp.c
@@ -42,6 +42,27 @@ typedef struct IntegratorCMState {
     uint32_t fiq_enabled;
 } IntegratorCMState;
 
+static const VMStateDescription vmstate_integratorcm = {
+    .name = "integratorcm",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_UINT32(cm_osc, IntegratorCMState),
+        VMSTATE_UINT32(cm_ctrl, IntegratorCMState),
+        VMSTATE_UINT32(cm_lock, IntegratorCMState),
+        VMSTATE_UINT32(cm_auxosc, IntegratorCMState),
+        VMSTATE_UINT32(cm_sdram, IntegratorCMState),
+        VMSTATE_UINT32(cm_init, IntegratorCMState),
+        VMSTATE_UINT32(cm_flags, IntegratorCMState),
+        VMSTATE_UINT32(cm_nvflags, IntegratorCMState),
+        VMSTATE_UINT32(int_level, IntegratorCMState),
+        VMSTATE_UINT32(irq_enabled, IntegratorCMState),
+        VMSTATE_UINT32(fiq_enabled, IntegratorCMState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static uint8_t integrator_spd[128] = {
    128, 8, 4, 11, 9, 1, 64, 0,  2, 0xa0, 0xa0, 0, 0, 8, 0, 1,
    0xe, 4, 0x1c, 1, 2, 0x20, 0xc0, 0, 0, 0, 0, 0x30, 0x28, 0x30, 0x28, 0x40
@@ -272,7 +293,7 @@ static int integratorcm_init(SysBusDevice *dev)
     sysbus_init_mmio(dev, &s->iomem);
 
     integratorcm_do_remap(s);
-    /* ??? Save/restore.  */
+    vmstate_register(NULL, -1, &vmstate_integratorcm, s);
     return 0;
 }
 
@@ -296,6 +317,20 @@ typedef struct icp_pic_state {
     qemu_irq parent_fiq;
 } icp_pic_state;
 
+
+static const VMStateDescription vmstate_icp_pic = {
+    .name = "icp_pic_state",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_UINT32(level, icp_pic_state),
+        VMSTATE_UINT32(irq_enabled, icp_pic_state),
+        VMSTATE_UINT32(fiq_enabled, icp_pic_state),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static void icp_pic_update(icp_pic_state *s)
 {
     uint32_t flags;
@@ -399,6 +434,7 @@ static int icp_pic_init(SysBusDevice *sbd)
     memory_region_init_io(&s->iomem, OBJECT(s), &icp_pic_ops, s,
                           "icp-pic", 0x00800000);
     sysbus_init_mmio(sbd, &s->iomem);
+    vmstate_register(NULL, -1, &vmstate_icp_pic, s);
     return 0;
 }
 

[Qemu-devel] [RFC PATCH v3 03/49] pcspk: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the PC speaker device state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/audio/pcspk.c |   18 ++++++++++++++++--
 1 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/hw/audio/pcspk.c b/hw/audio/pcspk.c
index 1d81bbe..8b22dbf 100644
--- a/hw/audio/pcspk.c
+++ b/hw/audio/pcspk.c
@@ -50,8 +50,8 @@ typedef struct {
     unsigned int pit_count;
     unsigned int samples;
     unsigned int play_pos;
-    int data_on;
-    int dummy_refresh_clock;
+    uint8_t data_on;
+    uint8_t dummy_refresh_clock;
 } PCSpkState;
 
 static const char *s_spk = "pcspk";
@@ -163,6 +163,18 @@ static const MemoryRegionOps pcspk_io_ops = {
     },
 };
 
+static const VMStateDescription vmstate_spk = {
+    .name = "pcspk",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_UINT8(data_on, PCSpkState),
+        VMSTATE_UINT8(dummy_refresh_clock, PCSpkState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static void pcspk_initfn(Object *obj)
 {
     PCSpkState *s = PC_SPEAKER(obj);
@@ -175,6 +187,8 @@ static void pcspk_realizefn(DeviceState *dev, Error **errp)
     ISADevice *isadev = ISA_DEVICE(dev);
     PCSpkState *s = PC_SPEAKER(dev);
 
+    vmstate_register(NULL, 0, &vmstate_spk, s);
+
     isa_register_ioport(isadev, &s->ioport, s->iobase);
 
     pcspk_state = s;

[Qemu-devel] [RFC PATCH v3 04/49] fdc: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the FDC device state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/block/fdc.c |   85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 83 insertions(+), 2 deletions(-)

diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index 490d127..7305f75 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -695,10 +695,34 @@ static const VMStateDescription vmstate_fdrive_media_rate = {
     }
 };
 
+static bool fdrive_perpendicular_needed(void *opaque)
+{
+    FDrive *drive = opaque;
+
+    return drive->perpendicular != 0;
+}
+
+static const VMStateDescription vmstate_fdrive_perpendicular = {
+    .name = "fdrive/perpendicular",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT8(perpendicular, FDrive),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static int fdrive_post_load(void *opaque, int version_id)
+{
+    fd_revalidate(opaque);
+    return 0;
+}
+
 static const VMStateDescription vmstate_fdrive = {
     .name = "fdrive",
-    .version_id = 1,
+    .version_id = 2,
     .minimum_version_id = 1,
+    .post_load = fdrive_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT8(head, FDrive),
         VMSTATE_UINT8(track, FDrive),
@@ -713,6 +737,9 @@ static const VMStateDescription vmstate_fdrive = {
             .vmsd = &vmstate_fdrive_media_rate,
             .needed = &fdrive_media_rate_needed,
         } , {
+            .vmsd = &vmstate_fdrive_perpendicular,
+            .needed = &fdrive_perpendicular_needed,
+        } , {
             /* empty */
         }
     }
@@ -725,6 +752,14 @@ static void fdc_pre_save(void *opaque)
     s->dor_vmstate = s->dor | GET_CUR_DRV(s);
 }
 
+static int fdc_pre_load(void *opaque)
+{
+    FDCtrl *s = opaque;
+    s->reset_sensei = 0;
+    timer_del(s->result_timer);
+    return 0;
+}
+
 static int fdc_post_load(void *opaque, int version_id)
 {
     FDCtrl *s = opaque;
@@ -734,11 +769,46 @@ static int fdc_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static bool fdc_reset_sensei_needed(void *opaque)
+{
+    FDCtrl *s = opaque;
+
+    return s->reset_sensei != 0;
+}
+
+static const VMStateDescription vmstate_fdc_reset_sensei = {
+    .name = "fdc/reset_sensei",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT32(reset_sensei, FDCtrl),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool fdc_result_timer_needed(void *opaque)
+{
+    FDCtrl *s = opaque;
+
+    return timer_pending(s->result_timer);
+}
+
+static const VMStateDescription vmstate_fdc_result_timer = {
+    .name = "fdc/result_timer",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_TIMER(result_timer, FDCtrl),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static const VMStateDescription vmstate_fdc = {
     .name = "fdc",
-    .version_id = 2,
+    .version_id = 3,
     .minimum_version_id = 2,
     .pre_save = fdc_pre_save,
+    .pre_load = fdc_pre_load,
     .post_load = fdc_post_load,
     .fields = (VMStateField[]) {
         /* Controller State */
@@ -770,6 +840,17 @@ static const VMStateDescription vmstate_fdc = {
         VMSTATE_STRUCT_ARRAY(drives, FDCtrl, MAX_FD, 1,
                              vmstate_fdrive, FDrive),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_fdc_reset_sensei,
+            .needed = fdc_reset_sensei_needed,
+        } , {
+            .vmsd = &vmstate_fdc_result_timer,
+            .needed = fdc_result_timer_needed,
+        } , {
+            /* empty */
+        }
     }
 };
 

[Qemu-devel] [RFC PATCH v3 05/49] parallel: adding vmstate for save/restore

[Pavel Dovgalyuk]

VMState added by this patch preserves correct
loading of the parallel port controller state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/char/parallel.c |   20 ++++++++++++++++++++
 1 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/hw/char/parallel.c b/hw/char/parallel.c
index 7ac90a5..26c03d7 100644
--- a/hw/char/parallel.c
+++ b/hw/char/parallel.c
@@ -477,6 +477,24 @@ static const MemoryRegionPortio isa_parallel_portio_sw_list[] = {
     PORTIO_END_OF_LIST(),
 };
 
+
+static const VMStateDescription vmstate_parallel_isa = {
+    .name = "parallel_isa",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_UINT8(state.dataw, ISAParallelState),
+        VMSTATE_UINT8(state.datar, ISAParallelState),
+        VMSTATE_UINT8(state.status, ISAParallelState),
+        VMSTATE_UINT8(state.control, ISAParallelState),
+        VMSTATE_INT32(state.irq_pending, ISAParallelState),
+        VMSTATE_INT32(state.epp_timeout, ISAParallelState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+
 static void parallel_isa_realizefn(DeviceState *dev, Error **errp)
 {
     static int index;
@@ -518,6 +536,8 @@ static void parallel_isa_realizefn(DeviceState *dev, Error **errp)
                               ? &isa_parallel_portio_hw_list[0]
                               : &isa_parallel_portio_sw_list[0]),
                              s, "parallel");
+
+    vmstate_register(NULL, -1, &vmstate_parallel_isa, isa);
 }
 
 /* Memory mapped interface */

[Qemu-devel] [RFC PATCH v3 06/49] serial: fixing vmstate for save/restore

[Pavel Dovgalyuk]

Some fields were added to VMState by this patch to preserve correct
loading of the serial port controller state.
Updating FCR value while loading was also modified to disable generating
an interrupt by loadvm.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/char/serial.c |  264 +++++++++++++++++++++++++++++++++++++++++++++---------
 1 files changed, 218 insertions(+), 46 deletions(-)

diff --git a/hw/char/serial.c b/hw/char/serial.c
index 764e184..f5727dc 100644
--- a/hw/char/serial.c
+++ b/hw/char/serial.c
@@ -272,6 +272,64 @@ static gboolean serial_xmit(GIOChannel *chan, GIOCondition cond, void *opaque)
 }
 
 
+/* Setter for FCR.
+   is_load flag means, that value is set while loading VM state
+   and interrupt should not be invoked */
+static void serial_write_fcr(void *opaque, uint32_t val, int is_load)
+{
+    SerialState *s = opaque;
+    val = val & 0xFF;
+
+    if (s->fcr == val) {
+        return;
+    }
+
+    /* Did the enable/disable flag change? If so, make sure FIFOs get flushed */
+    if ((val ^ s->fcr) & UART_FCR_FE) {
+        val |= UART_FCR_XFR | UART_FCR_RFR;
+    }
+
+    /* FIFO clear */
+
+    if (val & UART_FCR_RFR) {
+        timer_del(s->fifo_timeout_timer);
+        s->timeout_ipending = 0;
+        fifo8_reset(&s->recv_fifo);
+    }
+
+    if (val & UART_FCR_XFR) {
+        fifo8_reset(&s->xmit_fifo);
+    }
+
+    if (val & UART_FCR_FE) {
+        s->iir |= UART_IIR_FE;
+        /* Set recv_fifo trigger Level */
+        switch (val & 0xC0) {
+        case UART_FCR_ITL_1:
+            s->recv_fifo_itl = 1;
+            break;
+        case UART_FCR_ITL_2:
+            s->recv_fifo_itl = 4;
+            break;
+        case UART_FCR_ITL_3:
+            s->recv_fifo_itl = 8;
+            break;
+        case UART_FCR_ITL_4:
+            s->recv_fifo_itl = 14;
+            break;
+        }
+    } else {
+        s->iir &= ~UART_IIR_FE;
+    }
+
+    /* Set fcr - or at least the bits in it that are supposed to "stick" */
+    s->fcr = val & 0xC9;
+
+    if (!is_load) {
+        serial_update_irq(s);
+    }
+}
+
 static void serial_ioport_write(void *opaque, hwaddr addr, uint64_t val,
                                 unsigned size)
 {
@@ -327,50 +385,7 @@ static void serial_ioport_write(void *opaque, hwaddr addr, uint64_t val,
         }
         break;
     case 2:
-        val = val & 0xFF;
-
-        if (s->fcr == val)
-            break;
-
-        /* Did the enable/disable flag change? If so, make sure FIFOs get flushed */
-        if ((val ^ s->fcr) & UART_FCR_FE)
-            val |= UART_FCR_XFR | UART_FCR_RFR;
-
-        /* FIFO clear */
-
-        if (val & UART_FCR_RFR) {
-            timer_del(s->fifo_timeout_timer);
-            s->timeout_ipending=0;
-            fifo8_reset(&s->recv_fifo);
-        }
-
-        if (val & UART_FCR_XFR) {
-            fifo8_reset(&s->xmit_fifo);
-        }
-
-        if (val & UART_FCR_FE) {
-            s->iir |= UART_IIR_FE;
-            /* Set recv_fifo trigger Level */
-            switch (val & 0xC0) {
-            case UART_FCR_ITL_1:
-                s->recv_fifo_itl = 1;
-                break;
-            case UART_FCR_ITL_2:
-                s->recv_fifo_itl = 4;
-                break;
-            case UART_FCR_ITL_3:
-                s->recv_fifo_itl = 8;
-                break;
-            case UART_FCR_ITL_4:
-                s->recv_fifo_itl = 14;
-                break;
-            }
-        } else
-            s->iir &= ~UART_IIR_FE;
-
-        /* Set fcr - or at least the bits in it that are supposed to "stick" */
-        s->fcr = val & 0xC9;
-        serial_update_irq(s);
+        serial_write_fcr(s, val, 0);
         break;
     case 3:
         {
@@ -590,6 +605,19 @@ static void serial_pre_save(void *opaque)
     s->fcr_vmstate = s->fcr;
 }
 
+static int serial_pre_load(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    s->tsr_retry = 0;
+    fifo8_reset(&s->recv_fifo);
+    fifo8_reset(&s->xmit_fifo);
+    timer_del(s->fifo_timeout_timer);
+    s->timeout_ipending = 0;
+    s->poll_msl = -1;
+    timer_del(s->modem_status_poll);
+    return 0;
+}
+
 static int serial_post_load(void *opaque, int version_id)
 {
     SerialState *s = opaque;
@@ -597,17 +625,135 @@ static int serial_post_load(void *opaque, int version_id)
     if (version_id < 3) {
         s->fcr_vmstate = 0;
     }
+    s->last_break_enable = (s->lcr >> 6) & 1;
     /* Initialize fcr via setter to perform essential side-effects */
-    serial_ioport_write(s, 0x02, s->fcr_vmstate, 1);
+    serial_write_fcr(s, s->fcr_vmstate, 1);
     serial_update_parameters(s);
     return 0;
 }
 
+static bool serial_thr_ipending_needed(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    return (s->iir & UART_IIR_ID) != UART_IIR_THRI;
+}
+
+const VMStateDescription vmstate_serial_thr_ipending = {
+    .name = "serial/thr_ipending",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT32(thr_ipending, SerialState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool serial_tsr_needed(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    return s->tsr_retry != 0;
+}
+
+const VMStateDescription vmstate_serial_tsr = {
+    .name = "serial/tsr",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT32(tsr_retry, SerialState),
+        VMSTATE_UINT8(thr, SerialState),
+        VMSTATE_UINT8(tsr, SerialState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool serial_recv_fifo_needed(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    return !fifo8_is_empty(&s->recv_fifo);
+
+}
+
+const VMStateDescription vmstate_serial_recv_fifo = {
+    .name = "serial/recv_fifo",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_STRUCT(recv_fifo, SerialState, 1, vmstate_fifo8, Fifo8),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool serial_xmit_fifo_needed(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    return !fifo8_is_empty(&s->xmit_fifo);
+}
+
+const VMStateDescription vmstate_serial_xmit_fifo = {
+    .name = "serial/xmit_fifo",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_STRUCT(xmit_fifo, SerialState, 1, vmstate_fifo8, Fifo8),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool serial_fifo_timeout_timer_needed(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    return timer_pending(s->fifo_timeout_timer);
+}
+
+const VMStateDescription vmstate_serial_fifo_timeout_timer = {
+    .name = "serial/fifo_timeout_timer",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_TIMER(fifo_timeout_timer, SerialState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool serial_timeout_ipending_needed(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    return s->timeout_ipending != 0;
+}
+
+const VMStateDescription vmstate_serial_timeout_ipending = {
+    .name = "serial/timeout_ipending",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT32(timeout_ipending, SerialState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool serial_poll_needed(void *opaque)
+{
+    SerialState *s = (SerialState *)opaque;
+    return s->poll_msl >= 0;
+}
+
+const VMStateDescription vmstate_serial_poll = {
+    .name = "serial/poll",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT32(poll_msl, SerialState),
+        VMSTATE_TIMER(modem_status_poll, SerialState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 const VMStateDescription vmstate_serial = {
     .name = "serial",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 2,
     .pre_save = serial_pre_save,
+    .pre_load = serial_pre_load,
     .post_load = serial_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT16_V(divider, SerialState, 2),
@@ -621,6 +767,32 @@ const VMStateDescription vmstate_serial = {
         VMSTATE_UINT8(scr, SerialState),
         VMSTATE_UINT8_V(fcr_vmstate, SerialState, 3),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_serial_thr_ipending,
+            .needed = &serial_thr_ipending_needed,
+        } , {
+            .vmsd = &vmstate_serial_tsr,
+            .needed = &serial_tsr_needed,
+        } , {
+            .vmsd = &vmstate_serial_recv_fifo,
+            .needed = &serial_recv_fifo_needed,
+        } , {
+            .vmsd = &vmstate_serial_xmit_fifo,
+            .needed = &serial_xmit_fifo_needed,
+        } , {
+            .vmsd = &vmstate_serial_fifo_timeout_timer,
+            .needed = &serial_fifo_timeout_timer_needed,
+        } , {
+            .vmsd = &vmstate_serial_timeout_ipending,
+            .needed = &serial_timeout_ipending_needed,
+        } , {
+            .vmsd = &vmstate_serial_poll,
+            .needed = &serial_poll_needed,
+        } , {
+            /* empty */
+        }
     }
 };
 

[Qemu-devel] [RFC PATCH v3 07/49] kvmapic: fixing loading vmstate

[Pavel Dovgalyuk]

vapic state should not be synchronized with APIC while loading,
because APIC state could be not loaded yet at that moment.
We just save vapic_paddr in APIC VMState instead of synchronization.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/i386/kvmvapic.c              |   22 +++++++++++++++
 hw/intc/apic_common.c           |   56 ++++++++++++++++++++++++++++++++++++++-
 include/hw/i386/apic_internal.h |    2 +
 3 files changed, 77 insertions(+), 3 deletions(-)

diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index cb855c7..417ab6a 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -351,6 +351,24 @@ static int get_kpcr_number(X86CPU *cpu)
     return kpcr.number;
 }
 
+static int vapic_enable_post_load(VAPICROMState *s, X86CPU *cpu)
+{
+    int cpu_number = get_kpcr_number(cpu);
+    hwaddr vapic_paddr;
+    static const uint8_t enabled = 1;
+
+    if (cpu_number < 0) {
+        return -1;
+    }
+    vapic_paddr = s->vapic_paddr +
+        (((hwaddr)cpu_number) << VAPIC_CPU_SHIFT);
+    cpu_physical_memory_rw(vapic_paddr + offsetof(VAPICState, enabled),
+                           (void *)&enabled, sizeof(enabled), 1);
+    s->state = VAPIC_ACTIVE;
+
+    return 0;
+}
+
 static int vapic_enable(VAPICROMState *s, X86CPU *cpu)
 {
     int cpu_number = get_kpcr_number(cpu);
@@ -731,7 +749,9 @@ static void do_vapic_enable(void *data)
     VAPICROMState *s = data;
     X86CPU *cpu = X86_CPU(first_cpu);
 
-    vapic_enable(s, cpu);
+    /* Do not synchronize with APIC, because it was not loaded yet.
+       Just call the enable function which does not have synchronization. */
+    vapic_enable_post_load(s, cpu);
 }
 
 static int vapic_post_load(void *opaque, int version_id)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index ce3d903..5afd5b2 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -324,6 +324,15 @@ static void apic_common_realize(DeviceState *dev, Error **errp)
 
 }
 
+static int apic_pre_load(void *opaque)
+{
+    APICCommonState *s = APIC_COMMON(opaque);
+    s->sipi_vector = 0;
+    s->wait_for_sipi = 0;
+    s->vapic_paddr = 0;
+    return 0;
+}
+
 static void apic_dispatch_pre_save(void *opaque)
 {
     APICCommonState *s = APIC_COMMON(opaque);
@@ -345,12 +354,46 @@ static int apic_dispatch_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static bool apic_common_sipi_needed(void *opaque)
+{
+    APICCommonState *s = APIC_COMMON(opaque);
+    return s->wait_for_sipi != 0;
+}
+
+static const VMStateDescription vmstate_apic_common_sipi = {
+    .name = "apic_sipi",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT32(sipi_vector, APICCommonState),
+        VMSTATE_INT32(wait_for_sipi, APICCommonState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool apic_common_vapic_paddr_needed(void *opaque)
+{
+    APICCommonState *s = APIC_COMMON(opaque);
+    return s->vapic_paddr != 0;
+}
+
+static const VMStateDescription vmstate_apic_common_vapic_paddr = {
+    .name = "apic_vapic_paddr",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(vapic_paddr, APICCommonState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static const VMStateDescription vmstate_apic_common = {
     .name = "apic",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = apic_load_old,
+    .pre_load = apic_pre_load,
     .pre_save = apic_dispatch_pre_save,
     .post_load = apic_dispatch_post_load,
     .fields = (VMStateField[]) {
@@ -375,6 +418,17 @@ static const VMStateDescription vmstate_apic_common = {
         VMSTATE_INT64(timer_expiry,
                       APICCommonState), /* open-coded timer state */
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_apic_common_sipi,
+            .needed = apic_common_sipi_needed,
+        },
+        {
+            .vmsd = &vmstate_apic_common_vapic_paddr,
+            .needed = apic_common_vapic_paddr_needed,
+        },
+        VMSTATE_END_OF_LIST()
     }
 };
 
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 83e2a42..df4381c 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -124,7 +124,7 @@ struct APICCommonState {
 
     uint32_t vapic_control;
     DeviceState *vapic;
-    hwaddr vapic_paddr; /* note: persistence via kvmvapic */
+    hwaddr vapic_paddr;
 };
 
 typedef struct VAPICState {

[Qemu-devel] [RFC PATCH v3 08/49] hpet: fixing saving and loading process

[Pavel Dovgalyuk]

VM clock does not run while saving, so there is no need for saving the ticks
in HPET. Also added saving of hpet_offset field.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/hpet.c |   15 ++-------------
 1 files changed, 2 insertions(+), 13 deletions(-)

diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c
index e160e8f..4cdda64 100644
--- a/hw/timer/hpet.c
+++ b/hw/timer/hpet.c
@@ -222,14 +222,6 @@ static void update_irq(struct HPETTimer *timer, int set)
     }
 }
 
-static void hpet_pre_save(void *opaque)
-{
-    HPETState *s = opaque;
-
-    /* save current counter value */
-    s->hpet_counter = hpet_get_ticks(s);
-}
-
 static int hpet_pre_load(void *opaque)
 {
     HPETState *s = opaque;
@@ -255,9 +247,6 @@ static int hpet_post_load(void *opaque, int version_id)
 {
     HPETState *s = opaque;
 
-    /* Recalculate the offset between the main counter and guest time */
-    s->hpet_offset = ticks_to_ns(s->hpet_counter) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-
     /* Push number of timers into capability returned via HPET_ID */
     s->capability &= ~HPET_ID_NUM_TIM_MASK;
     s->capability |= (s->num_timers - 1) << HPET_ID_NUM_TIM_SHIFT;
@@ -306,15 +295,15 @@ static const VMStateDescription vmstate_hpet_timer = {
 
 static const VMStateDescription vmstate_hpet = {
     .name = "hpet",
-    .version_id = 2,
+    .version_id = 3,
     .minimum_version_id = 1,
-    .pre_save = hpet_pre_save,
     .pre_load = hpet_pre_load,
     .post_load = hpet_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT64(config, HPETState),
         VMSTATE_UINT64(isr, HPETState),
         VMSTATE_UINT64(hpet_counter, HPETState),
+        VMSTATE_UINT64_V(hpet_offset, HPETState, 3),
         VMSTATE_UINT8_V(num_timers, HPETState, 2),
         VMSTATE_VALIDATE("num_timers in range", hpet_validate_num_timers),
         VMSTATE_STRUCT_VARRAY_UINT8(timer, HPETState, num_timers, 0,

[Qemu-devel] [RFC PATCH v3 09/49] pckbd: adding new fields to vmstate

[Pavel Dovgalyuk]

This patch adds outport to VMState to allow correct saving and restoring
the state of PC keyboard controller.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/input/pckbd.c |   53 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 52 insertions(+), 1 deletions(-)

diff --git a/hw/input/pckbd.c b/hw/input/pckbd.c
index ca1cffc..821f6f9 100644
--- a/hw/input/pckbd.c
+++ b/hw/input/pckbd.c
@@ -131,6 +131,7 @@ typedef struct KBDState {
     uint8_t status;
     uint8_t mode;
     uint8_t outport;
+    bool outport_present;
     /* Bitmask of devices with data available.  */
     uint8_t pending;
     void *kbd;
@@ -367,18 +368,68 @@ static void kbd_reset(void *opaque)
     s->mode = KBD_MODE_KBD_INT | KBD_MODE_MOUSE_INT;
     s->status = KBD_STAT_CMD | KBD_STAT_UNLOCKED;
     s->outport = KBD_OUT_RESET | KBD_OUT_A20;
+    s->outport_present = false;
+}
+
+static uint8_t kbd_outport_default(KBDState *s)
+{
+    return KBD_OUT_RESET | KBD_OUT_A20
+           | (s->status & KBD_STAT_OBF ? KBD_OUT_OBF : 0)
+           | (s->status & KBD_STAT_MOUSE_OBF ? KBD_OUT_MOUSE_OBF : 0);
+}
+
+static int kbd_outport_post_load(void *opaque, int version_id)
+{
+    KBDState *s = opaque;
+    s->outport_present = true;
+    return 0;
+}
+
+static const VMStateDescription vmstate_kbd_outport = {
+    .name = "pckbd_outport",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .post_load = kbd_outport_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT8(outport, KBDState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool kbd_outport_needed(void *opaque)
+{
+    KBDState *s = opaque;
+    return s->outport != kbd_outport_default(s);
+}
+
+static int kbd_post_load(void *opaque, int version_id)
+{
+    KBDState *s = opaque;
+    if (!s->outport_present) {
+        s->outport = kbd_outport_default(s);
+    }
+    s->outport_present = false;
+    return 0;
 }
 
 static const VMStateDescription vmstate_kbd = {
     .name = "pckbd",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
+    .post_load = kbd_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT8(write_cmd, KBDState),
         VMSTATE_UINT8(status, KBDState),
         VMSTATE_UINT8(mode, KBDState),
         VMSTATE_UINT8(pending, KBDState),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_kbd_outport,
+            .needed = kbd_outport_needed,
+        },
+        VMSTATE_END_OF_LIST()
     }
 };
 

[Qemu-devel] [RFC PATCH v3 10/49] rtl8139: adding new fields to vmstate

[Pavel Dovgalyuk]

This patch adds virtual clock-dependent timers to VMState to allow correct
saving and restoring the state of RTL8139 network controller.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/net/rtl8139.c |   50 ++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 48 insertions(+), 2 deletions(-)

diff --git a/hw/net/rtl8139.c b/hw/net/rtl8139.c
index 90bc5ec..b560c81 100644
--- a/hw/net/rtl8139.c
+++ b/hw/net/rtl8139.c
@@ -3246,10 +3246,17 @@ static uint32_t rtl8139_mmio_readl(void *opaque, hwaddr addr)
     return val;
 }
 
+static int rtl8139_pre_load(void *opaque)
+{
+    RTL8139State *s = opaque;
+    s->TimerExpire = 0;
+    timer_del(s->timer);
+    return 0;
+}
+
 static int rtl8139_post_load(void *opaque, int version_id)
 {
     RTL8139State* s = opaque;
-    rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
     if (version_id < 4) {
         s->cplus_enabled = s->CpCmd != 0;
     }
@@ -3275,6 +3282,38 @@ static const VMStateDescription vmstate_rtl8139_hotplug_ready ={
     }
 };
 
+static bool rtl8139_TimerExpire_needed(void *opaque)
+{
+    RTL8139State *s = (RTL8139State *)opaque;
+    return s->TimerExpire != 0;
+}
+
+static const VMStateDescription vmstate_rtl8139_TimerExpire = {
+    .name = "rtl8139/TimerExpire",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_INT64(TimerExpire, RTL8139State),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool rtl8139_timer_needed(void *opaque)
+{
+    RTL8139State *s = (RTL8139State *)opaque;
+    return timer_pending(s->timer);
+}
+
+static const VMStateDescription vmstate_rtl8139_timer = {
+    .name = "rtl8139/timer",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_TIMER(timer, RTL8139State),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static void rtl8139_pre_save(void *opaque)
 {
     RTL8139State* s = opaque;
@@ -3289,8 +3328,9 @@ static void rtl8139_pre_save(void *opaque)
 
 static const VMStateDescription vmstate_rtl8139 = {
     .name = "rtl8139",
-    .version_id = 4,
+    .version_id = 5,
     .minimum_version_id = 3,
+    .pre_load = rtl8139_pre_load,
     .post_load = rtl8139_post_load,
     .pre_save  = rtl8139_pre_save,
     .fields = (VMStateField[]) {
@@ -3371,6 +3411,12 @@ static const VMStateDescription vmstate_rtl8139 = {
             .vmsd = &vmstate_rtl8139_hotplug_ready,
             .needed = rtl8139_hotplug_ready_needed,
         }, {
+            .vmsd = &vmstate_rtl8139_TimerExpire,
+            .needed = rtl8139_TimerExpire_needed,
+        }, {
+            .vmsd = &vmstate_rtl8139_timer,
+            .needed = rtl8139_timer_needed,
+        }, {
             /* empty */
         }
     }

[Qemu-devel] [RFC PATCH v3 11/49] piix: do not raise irq while loading vmstate

[Pavel Dovgalyuk]

This patch disables raising an irq while loading the state of PCI bridge.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/pci-host/piix.c |   22 ++++++++++++++++++++--
 1 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c
index e0e0946..86d6d20 100644
--- a/hw/pci-host/piix.c
+++ b/hw/pci-host/piix.c
@@ -409,7 +409,7 @@ static void piix3_set_irq_pic(PIIX3State *piix3, int pic_irq)
                      (pic_irq * PIIX_NUM_PIRQS))));
 }
 
-static void piix3_set_irq_level(PIIX3State *piix3, int pirq, int level)
+static void piix3_set_irq_level_internal(PIIX3State *piix3, int pirq, int level)
 {
     int pic_irq;
     uint64_t mask;
@@ -422,6 +422,18 @@ static void piix3_set_irq_level(PIIX3State *piix3, int pirq, int level)
     mask = 1ULL << ((pic_irq * PIIX_NUM_PIRQS) + pirq);
     piix3->pic_levels &= ~mask;
     piix3->pic_levels |= mask * !!level;
+}
+
+static void piix3_set_irq_level(PIIX3State *piix3, int pirq, int level)
+{
+    int pic_irq;
+
+    pic_irq = piix3->dev.config[PIIX_PIRQC + pirq];
+    if (pic_irq >= PIIX_NUM_PIC_IRQS) {
+        return;
+    }
+
+    piix3_set_irq_level_internal(piix3, pirq, level);
 
     piix3_set_irq_pic(piix3, pic_irq);
 }
@@ -527,7 +539,13 @@ static void piix3_reset(void *opaque)
 static int piix3_post_load(void *opaque, int version_id)
 {
     PIIX3State *piix3 = opaque;
-    piix3_update_irq_levels(piix3);
+    int pirq;
+
+    piix3->pic_levels = 0;
+    for (pirq = 0; pirq < PIIX_NUM_PIRQS; pirq++) {
+        piix3_set_irq_level_internal(piix3, pirq,
+                            pci_bus_get_irq_level(piix3->dev.bus, pirq));
+    }
     return 0;
 }
 

[Qemu-devel] [RFC PATCH v3 12/49] mc146818rtc: add missed field to vmstate

[Pavel Dovgalyuk]

This patch adds irq_reinject_on_ack_count field to VMState to allow correct
saving/loading the state of MC146818 RTC.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/mc146818rtc.c |   34 +++++++++++++++++++++++++++++++++-
 1 files changed, 33 insertions(+), 1 deletions(-)

diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
index 233fc70..0b78d88 100644
--- a/hw/timer/mc146818rtc.c
+++ b/hw/timer/mc146818rtc.c
@@ -698,6 +698,13 @@ int rtc_get_memory(ISADevice *dev, int addr)
     return s->cmos_data[addr];
 }
 
+static int rtc_pre_load(void *opaque)
+{
+    RTCState *s = (RTCState *)opaque;
+    s->irq_reinject_on_ack_count = 0;
+    return 0;
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = MC146818_RTC(dev);
@@ -733,10 +740,27 @@ static int rtc_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static const VMStateDescription vmstate_rtc_irq_reinject_on_ack_count = {
+    .name = "irq_reinject_on_ack_count",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT16(irq_reinject_on_ack_count, RTCState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static bool rtc_irq_reinject_on_ack_count_needed(void *opaque)
+{
+    RTCState *s = (RTCState *)opaque;
+    return s->irq_reinject_on_ack_count != 0;
+}
+
 static const VMStateDescription vmstate_rtc = {
     .name = "mc146818rtc",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 1,
+    .pre_load = rtc_pre_load,
     .post_load = rtc_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_BUFFER(cmos_data, RTCState),
@@ -753,6 +777,14 @@ static const VMStateDescription vmstate_rtc = {
         VMSTATE_TIMER_V(update_timer, RTCState, 3),
         VMSTATE_UINT64_V(next_alarm_time, RTCState, 3),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_rtc_irq_reinject_on_ack_count,
+            .needed = rtc_irq_reinject_on_ack_count_needed,
+        }, {
+            /* empty */
+        }
     }
 };
 

[Qemu-devel] [RFC PATCH v3 13/49] pl031: add missed field to vmstate

[Pavel Dovgalyuk]

This patch adds timer which uses virtual clock to the VMState.
Such timers are required for saving because virtual clock is the part
of the virtual machine state.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/pl031.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/hw/timer/pl031.c b/hw/timer/pl031.c
index 34d9b44..f8e5abc 100644
--- a/hw/timer/pl031.c
+++ b/hw/timer/pl031.c
@@ -230,12 +230,13 @@ static int pl031_post_load(void *opaque, int version_id)
 
 static const VMStateDescription vmstate_pl031 = {
     .name = "pl031",
-    .version_id = 1,
+    .version_id = 2,
     .minimum_version_id = 1,
     .pre_save = pl031_pre_save,
     .post_load = pl031_post_load,
     .fields = (VMStateField[]) {
         VMSTATE_UINT32(tick_offset_vmstate, PL031State),
+        VMSTATE_TIMER_V(timer, PL031State, 2),
         VMSTATE_UINT32(mr, PL031State),
         VMSTATE_UINT32(lr, PL031State),
         VMSTATE_UINT32(cr, PL031State),

[Qemu-devel] [RFC PATCH v3 14/49] ide pci: reset status field before loading the vmstate

[Pavel Dovgalyuk]

This patch resets status field before loading the state of IDE BMDMA device.
Resetting status is needed for replay, because it does not reset whole virtual
machine before loading state of the VM.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/ide/pci.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/hw/ide/pci.c b/hw/ide/pci.c
index 6257a21..67cb751 100644
--- a/hw/ide/pci.c
+++ b/hw/ide/pci.c
@@ -382,6 +382,16 @@ static bool ide_bmdma_status_needed(void *opaque)
     return ((bm->status & abused_bits) != 0);
 }
 
+static int ide_bmdma_pre_load(void *opaque)
+{
+    BMDMAState *bm = opaque;
+    /* Reset is not performed in replay mode,
+       so reset status manually to allow ide_bmdma_post_load function
+       initialize it. */
+    bm->status = 0;
+    return 0;
+}
+
 static void ide_bmdma_pre_save(void *opaque)
 {
     BMDMAState *bm = opaque;
@@ -434,6 +444,7 @@ static const VMStateDescription vmstate_bmdma = {
     .name = "ide bmdma",
     .version_id = 3,
     .minimum_version_id = 0,
+    .pre_load  = ide_bmdma_pre_load,
     .pre_save  = ide_bmdma_pre_save,
     .fields = (VMStateField[]) {
         VMSTATE_UINT8(cmd, BMDMAState),

[Qemu-devel] [RFC PATCH v3 15/49] softmmu: fixing usage of cpu_st/ld* from helpers

[Pavel Dovgalyuk]

MMU helper functions are called from generated code and other helper
functions. In both cases they try to get function's return address for
using it while restoring virtual CPU state.

When MMU helper is called from some other helper function
(like helper_maskmov_xmm) through cpu_st* function, the return address
will point to that helper. That is why CPU state cannot be restored in
the case of MMU fault.

This patch introduces several inline helpers to load return address
which points to the right place.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/exec/cpu_ldst_template.h |   31 +++++++++++++++++++++++++++----
 include/exec/exec-all.h          |   27 +++++++++++++++++++++++++++
 softmmu_template.h               |   18 ++++++++++++++++++
 3 files changed, 72 insertions(+), 4 deletions(-)

diff --git a/include/exec/cpu_ldst_template.h b/include/exec/cpu_ldst_template.h
index 006093a..2658955 100644
--- a/include/exec/cpu_ldst_template.h
+++ b/include/exec/cpu_ldst_template.h
@@ -61,6 +61,17 @@
 #define MMUSUFFIX _mmu
 #endif
 
+/* inline helper ld function */
+
+static inline DATA_TYPE
+glue(glue(helper_inline_ld, SUFFIX), MEMSUFFIX)(CPUArchState *env,
+                                                target_ulong addr,
+                                                int mmu_idx)
+{
+    return glue(glue(helper_call_ld, SUFFIX), MMUSUFFIX)(env, addr, mmu_idx,
+                                                         GETRA());
+}
+
 /* generic load/store macros */
 
 static inline RES_TYPE
@@ -76,7 +87,8 @@ glue(glue(cpu_ld, USUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr)
     mmu_idx = CPU_MMU_INDEX;
     if (unlikely(env->tlb_table[mmu_idx][page_index].ADDR_READ !=
                  (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) {
-        res = glue(glue(helper_ld, SUFFIX), MMUSUFFIX)(env, addr, mmu_idx);
+        res = glue(glue(helper_inline_ld, SUFFIX), MEMSUFFIX)(env, addr,
+                                                              mmu_idx);
     } else {
         uintptr_t hostaddr = addr + env->tlb_table[mmu_idx][page_index].addend;
         res = glue(glue(ld, USUFFIX), _raw)(hostaddr);
@@ -97,8 +109,8 @@ glue(glue(cpu_lds, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr)
     mmu_idx = CPU_MMU_INDEX;
     if (unlikely(env->tlb_table[mmu_idx][page_index].ADDR_READ !=
                  (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) {
-        res = (DATA_STYPE)glue(glue(helper_ld, SUFFIX),
-                               MMUSUFFIX)(env, addr, mmu_idx);
+        res = (DATA_STYPE)glue(glue(helper_inline_ld, SUFFIX),
+                               MEMSUFFIX)(env, addr, mmu_idx);
     } else {
         uintptr_t hostaddr = addr + env->tlb_table[mmu_idx][page_index].addend;
         res = glue(glue(lds, SUFFIX), _raw)(hostaddr);
@@ -109,6 +121,17 @@ glue(glue(cpu_lds, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr)
 
 #ifndef SOFTMMU_CODE_ACCESS
 
+/* inline helper st function */
+
+static inline void
+glue(glue(helper_inline_st, SUFFIX), MEMSUFFIX)(CPUArchState *env,
+                                                target_ulong addr,
+                                                DATA_TYPE val, int mmu_idx)
+{
+    glue(glue(helper_call_st, SUFFIX), MMUSUFFIX)(env, addr, val, mmu_idx,
+                                                  GETRA());
+}
+
 /* generic store macro */
 
 static inline void
@@ -124,7 +147,7 @@ glue(glue(cpu_st, SUFFIX), MEMSUFFIX)(CPUArchState *env, target_ulong ptr,
     mmu_idx = CPU_MMU_INDEX;
     if (unlikely(env->tlb_table[mmu_idx][page_index].addr_write !=
                  (addr & (TARGET_PAGE_MASK | (DATA_SIZE - 1))))) {
-        glue(glue(helper_st, SUFFIX), MMUSUFFIX)(env, addr, v, mmu_idx);
+        glue(glue(helper_inline_st, SUFFIX), MEMSUFFIX)(env, addr, v, mmu_idx);
     } else {
         uintptr_t hostaddr = addr + env->tlb_table[mmu_idx][page_index].addend;
         glue(glue(st, SUFFIX), _raw)(hostaddr, v);
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 5e5d86e..528928f 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -344,6 +344,33 @@ bool io_mem_write(struct MemoryRegion *mr, hwaddr addr,
 void tlb_fill(CPUState *cpu, target_ulong addr, int is_write, int mmu_idx,
               uintptr_t retaddr);
 
+uint8_t helper_call_ldb_cmmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+uint16_t helper_call_ldw_cmmu(CPUArchState *env, target_ulong addr,
+                              int mmu_idx, uintptr_t retaddr);
+uint32_t helper_call_ldl_cmmu(CPUArchState *env, target_ulong addr,
+                              int mmu_idx, uintptr_t retaddr);
+uint64_t helper_call_ldq_cmmu(CPUArchState *env, target_ulong addr,
+                              int mmu_idx, uintptr_t retaddr);
+
+uint8_t helper_call_ldb_mmu(CPUArchState *env, target_ulong addr,
+                            int mmu_idx, uintptr_t retaddr);
+uint16_t helper_call_ldw_mmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+uint32_t helper_call_ldl_mmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+uint64_t helper_call_ldq_mmu(CPUArchState *env, target_ulong addr,
+                             int mmu_idx, uintptr_t retaddr);
+
+void helper_call_stb_mmu(CPUArchState *env, target_ulong addr,
+                         uint8_t val, int mmu_idx, uintptr_t retaddr);
+void helper_call_stw_mmu(CPUArchState *env, target_ulong addr,
+                         uint16_t val, int mmu_idx, uintptr_t retaddr);
+void helper_call_stl_mmu(CPUArchState *env, target_ulong addr,
+                         uint32_t val, int mmu_idx, uintptr_t retaddr);
+void helper_call_stq_mmu(CPUArchState *env, target_ulong addr,
+                         uint64_t val, int mmu_idx, uintptr_t retaddr);
+
 #endif
 
 #if defined(CONFIG_USER_ONLY)
diff --git a/softmmu_template.h b/softmmu_template.h
index 5a07f99..1053cf3 100644
--- a/softmmu_template.h
+++ b/softmmu_template.h
@@ -311,6 +311,15 @@ glue(glue(helper_ld, SUFFIX), MMUSUFFIX)(CPUArchState *env, target_ulong addr,
     return helper_te_ld_name (env, addr, mmu_idx, GETRA());
 }
 
+DATA_TYPE
+glue(glue(helper_call_ld, SUFFIX), MMUSUFFIX)(CPUArchState *env,
+                                              target_ulong addr,
+                                              int mmu_idx,
+                                              uintptr_t retaddr)
+{
+    return helper_te_ld_name(env, addr, mmu_idx, retaddr);
+}
+
 #ifndef SOFTMMU_CODE_ACCESS
 
 /* Provide signed versions of the load routines as well.  We can of course
@@ -505,6 +514,15 @@ glue(glue(helper_st, SUFFIX), MMUSUFFIX)(CPUArchState *env, target_ulong addr,
     helper_te_st_name(env, addr, val, mmu_idx, GETRA());
 }
 
+void
+glue(glue(helper_call_st, SUFFIX), MMUSUFFIX)(CPUArchState *env,
+                                              target_ulong addr,
+                                              DATA_TYPE val, int mmu_idx,
+                                              uintptr_t retaddr)
+{
+    helper_te_st_name(env, addr, val, mmu_idx, retaddr);
+}
+
 #endif /* !defined(SOFTMMU_CODE_ACCESS) */
 
 #undef READ_ACCESS_TYPE

[Qemu-devel] [RFC PATCH v3 16/49] target-i386: update fp status fix

[Pavel Dovgalyuk]

This patch adds calls to update_fp_status() function from several
places where FPU state is changed.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 target-i386/cpu.c        |    1 +
 target-i386/cpu.h        |    1 +
 target-i386/fpu_helper.c |    5 ++++-
 target-i386/machine.c    |    2 +-
 4 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 6d008ab..906d7d7 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2573,6 +2573,7 @@ static void x86_cpu_reset(CPUState *s)
         env->fptags[i] = 1;
     }
     env->fpuc = 0x37f;
+    update_fp_status(env);
 
     env->mxcsr = 0x1f80;
     env->xstate_bv = XSTATE_FP | XSTATE_SSE;
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index e634d83..42bda46 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -1249,6 +1249,7 @@ void QEMU_NORETURN raise_interrupt(CPUX86State *nenv, int intno, int is_int,
 /* cc_helper.c */
 extern const uint8_t parity_table[256];
 uint32_t cpu_cc_compute_all(CPUX86State *env1, int op);
+void update_fp_status(CPUX86State *env);
 
 static inline uint32_t cpu_compute_eflags(CPUX86State *env)
 {
diff --git a/target-i386/fpu_helper.c b/target-i386/fpu_helper.c
index 1b2900d..a8ffba9 100644
--- a/target-i386/fpu_helper.c
+++ b/target-i386/fpu_helper.c
@@ -537,7 +537,7 @@ uint32_t helper_fnstcw(CPUX86State *env)
     return env->fpuc;
 }
 
-static void update_fp_status(CPUX86State *env)
+void update_fp_status(CPUX86State *env)
 {
     int rnd_type;
 
@@ -1006,6 +1006,7 @@ void helper_fstenv(CPUX86State *env, target_ulong ptr, int data32)
         cpu_stw_data(env, ptr + 10, 0);
         cpu_stw_data(env, ptr + 12, 0);
     }
+    update_fp_status(env);
 }
 
 void helper_fldenv(CPUX86State *env, target_ulong ptr, int data32)
@@ -1055,6 +1056,7 @@ void helper_fsave(CPUX86State *env, target_ulong ptr, int data32)
     env->fptags[5] = 1;
     env->fptags[6] = 1;
     env->fptags[7] = 1;
+    update_fp_status(env);
 }
 
 void helper_frstor(CPUX86State *env, target_ulong ptr, int data32)
@@ -1158,6 +1160,7 @@ void helper_fxrstor(CPUX86State *env, target_ulong ptr, int data64)
     }
 
     env->fpuc = cpu_lduw_data(env, ptr);
+    update_fp_status(env);
     fpus = cpu_lduw_data(env, ptr + 2);
     fptag = cpu_lduw_data(env, ptr + 4);
     env->fpstt = (fpus >> 11) & 7;
diff --git a/target-i386/machine.c b/target-i386/machine.c
index 16d2f6a..2922c97 100644
--- a/target-i386/machine.c
+++ b/target-i386/machine.c
@@ -315,13 +315,13 @@ static int cpu_post_load(void *opaque, int version_id)
     env->hflags &= ~HF_CPL_MASK;
     env->hflags |= (env->segs[R_SS].flags >> DESC_DPL_SHIFT) & HF_CPL_MASK;
 
-    /* XXX: restore FPU round state */
     env->fpstt = (env->fpus_vmstate >> 11) & 7;
     env->fpus = env->fpus_vmstate & ~0x3800;
     env->fptag_vmstate ^= 0xff;
     for(i = 0; i < 8; i++) {
         env->fptags[i] = (env->fptag_vmstate >> i) & 1;
     }
+    update_fp_status(env);
 
     cpu_breakpoint_remove_all(cs, BP_CPU);
     cpu_watchpoint_remove_all(cs, BP_CPU);

[Qemu-devel] [RFC PATCH v3 17/49] migration: add vmstate for int8 and char arrays

[Pavel Dovgalyuk]

This patch adds macros for vmstate int8 and char arrays.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/migration/vmstate.h |   13 +++++++++++++
 vmstate.c                   |    6 ++++++
 2 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 9a001bd..2f7207b 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -143,6 +143,7 @@ extern const VMStateDescription vmstate_dummy;
 #endif
 
 extern const VMStateInfo vmstate_info_bool;
+extern const VMStateInfo vmstate_info_char;
 
 extern const VMStateInfo vmstate_info_int8;
 extern const VMStateInfo vmstate_info_int16;
@@ -694,6 +695,18 @@ extern const VMStateInfo vmstate_info_bitmap;
 #define VMSTATE_UINT64_ARRAY(_f, _s, _n)                              \
     VMSTATE_UINT64_ARRAY_V(_f, _s, _n, 0)
 
+#define VMSTATE_INT8_ARRAY_V(_f, _s, _n, _v)                          \
+    VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_int8, int8_t)
+
+#define VMSTATE_INT8_ARRAY(_f, _s, _n)                                \
+    VMSTATE_INT8_ARRAY_V(_f, _s, _n, 0)
+
+#define VMSTATE_CHAR_ARRAY_V(_f, _s, _n, _v)                          \
+    VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_char, char)
+
+#define VMSTATE_CHAR_ARRAY(_f, _s, _n)                                \
+    VMSTATE_CHAR_ARRAY_V(_f, _s, _n, 0)
+
 #define VMSTATE_INT16_ARRAY_V(_f, _s, _n, _v)                         \
     VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_int16, int16_t)
 
diff --git a/vmstate.c b/vmstate.c
index ef2f87b..6a7f7c2 100644
--- a/vmstate.c
+++ b/vmstate.c
@@ -286,6 +286,12 @@ const VMStateInfo vmstate_info_int8 = {
     .put  = put_int8,
 };
 
+const VMStateInfo vmstate_info_char = {
+    .name = "char",
+    .get  = get_int8,
+    .put  = put_int8,
+};
+
 /* 16 bit int */
 
 static int get_int16(QEMUFile *f, void *pv, size_t size)

[Qemu-devel] [RFC PATCH v3 18/49] replay: global variables and function stubs

[Pavel Dovgalyuk]

This patch adds global variables, defines, functions declarations,
and function stubs for deterministic VM replay used by external modules.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 Makefile.target      |    1 +
 qapi-schema.json     |   32 ++++++++++++++++++++++++++++++++
 replay/Makefile.objs |    1 +
 replay/replay.c      |   25 +++++++++++++++++++++++++
 replay/replay.h      |   23 +++++++++++++++++++++++
 stubs/Makefile.objs  |    1 +
 stubs/replay.c       |    8 ++++++++
 7 files changed, 91 insertions(+), 0 deletions(-)
 create mode 100755 replay/Makefile.objs
 create mode 100755 replay/replay.c
 create mode 100755 replay/replay.h
 create mode 100755 stubs/replay.c

diff --git a/Makefile.target b/Makefile.target
index 137d0b0..9830313 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -76,6 +76,7 @@ all: $(PROGS) stap
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
+obj-y += replay/
 obj-y += tcg/tcg.o tcg/optimize.o
 obj-$(CONFIG_TCG_INTERPRETER) += tci.o
 obj-$(CONFIG_TCG_INTERPRETER) += disas/tci.o
diff --git a/qapi-schema.json b/qapi-schema.json
index b11aad2..2de14c4 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3480,3 +3480,35 @@
 # Since: 2.1
 ##
 { 'command': 'rtc-reset-reinjection' }
+
+##
+# ReplayMode:
+#
+# Mode of the replay subsystem.
+#
+# @none: normal execution mode. Replay or record are not enabled.
+#
+# @record: record mode. All non-deterministic data is written into the
+#          replay log.
+#
+# @play: replay mode. Non-deterministic data required for system execution
+#        is read from the log.
+#
+# Since: 2.2
+##
+{ 'enum': 'ReplayMode',
+  'data': [ 'none', 'record', 'play' ] }
+
+##
+# ReplaySubmode:
+#
+# Submode of the replay subsystem.
+#
+# @unknown: used for modes different from play.
+#
+# @normal: normal replay mode.
+#
+# Since: 2.2
+##
+{ 'enum': 'ReplaySubmode',
+  'data': [ 'unknown', 'normal' ] }
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
new file mode 100755
index 0000000..7ea860f
--- /dev/null
+++ b/replay/Makefile.objs
@@ -0,0 +1 @@
+obj-y += replay.o
diff --git a/replay/replay.c b/replay/replay.c
new file mode 100755
index 0000000..ac976b2
--- /dev/null
+++ b/replay/replay.c
@@ -0,0 +1,25 @@
+/*
+ * replay.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "replay.h"
+
+ReplayMode replay_mode = REPLAY_MODE_NONE;
+/*! Stores current submode for PLAY mode */
+ReplaySubmode play_submode = REPLAY_SUBMODE_UNKNOWN;
+
+/* Suffix for the disk images filenames */
+char *replay_image_suffix;
+
+
+ReplaySubmode replay_get_play_submode(void)
+{
+    return play_submode;
+}
diff --git a/replay/replay.h b/replay/replay.h
new file mode 100755
index 0000000..51a18fe
--- /dev/null
+++ b/replay/replay.h
@@ -0,0 +1,23 @@
+#ifndef REPLAY_H
+#define REPLAY_H
+
+/*
+ * replay.h
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qapi-types.h"
+
+extern ReplayMode replay_mode;
+extern char *replay_image_suffix;
+
+/*! Returns replay play submode */
+ReplaySubmode replay_get_play_submode(void);
+
+#endif
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index 528e161..b90c60a 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -26,6 +26,7 @@ stub-obj-y += notify-event.o
 stub-obj-y += pci-drive-hot-add.o
 stub-obj-$(CONFIG_SPICE) += qemu-chr-open-spice.o
 stub-obj-y += qtest.o
+stub-obj-y += replay.o
 stub-obj-y += reset.o
 stub-obj-y += runstate-check.o
 stub-obj-y += set-fd-handler.o
diff --git a/stubs/replay.c b/stubs/replay.c
new file mode 100755
index 0000000..b146d55
--- /dev/null
+++ b/stubs/replay.c
@@ -0,0 +1,8 @@
+#include "replay/replay.h"
+
+ReplayMode replay_mode;
+
+ReplaySubmode replay_get_play_submode(void)
+{
+    return 0;
+}

[Qemu-devel] [RFC PATCH v3 19/49] block: add suffix parameter to bdrv_open functions

[Pavel Dovgalyuk]

This patch adds parameter with suffix for overlay filenames. This parameter
will be used by replay to automatically create overlay files based on
drive images supplied to VM.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block.c               |   49 +++++++++++++++++++++++++++++++++++--------------
 block/blkdebug.c      |    2 +-
 block/blkverify.c     |    5 +++--
 block/cow.c           |    2 +-
 block/qcow.c          |    2 +-
 block/qcow2.c         |    6 +++---
 block/qed.c           |    2 +-
 block/sheepdog.c      |    4 ++--
 block/vmdk.c          |    8 ++++----
 block/vvfat.c         |    2 +-
 blockdev.c            |   14 +++++++++-----
 include/block/block.h |    7 ++++---
 qemu-img.c            |    6 +++---
 qemu-io.c             |    7 +++----
 qemu-nbd.c            |    2 +-
 15 files changed, 72 insertions(+), 46 deletions(-)

diff --git a/block.c b/block.c
index 8cf519b..a7633fe 100644
--- a/block.c
+++ b/block.c
@@ -35,6 +35,7 @@
 #include "qmp-commands.h"
 #include "qemu/timer.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 #ifdef CONFIG_BSD
 #include <sys/types.h>
@@ -1222,7 +1223,8 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp)
     assert(bs->backing_hd == NULL);
     ret = bdrv_open(&backing_hd,
                     *backing_filename ? backing_filename : NULL, NULL, options,
-                    bdrv_backing_flags(bs->open_flags), back_drv, &local_err);
+                    bdrv_backing_flags(bs->open_flags), back_drv,
+                    &local_err, NULL);
     if (ret < 0) {
         bdrv_unref(backing_hd);
         backing_hd = NULL;
@@ -1257,7 +1259,7 @@ free_exit:
  */
 int bdrv_open_image(BlockDriverState **pbs, const char *filename,
                     QDict *options, const char *bdref_key, int flags,
-                    bool allow_none, Error **errp)
+                    bool allow_none, Error **errp, const char *fname_suffix)
 {
     QDict *image_options;
     int ret;
@@ -1284,14 +1286,16 @@ int bdrv_open_image(BlockDriverState **pbs, const char *filename,
         goto done;
     }
 
-    ret = bdrv_open(pbs, filename, reference, image_options, flags, NULL, errp);
+    ret = bdrv_open(pbs, filename, reference, image_options, flags,
+                    NULL, errp, NULL);
 
 done:
     qdict_del(options, bdref_key);
     return ret;
 }
 
-int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp)
+int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp,
+                              const char *fname_suffix)
 {
     /* TODO: extra byte is a hack to ensure MAX_PATH space on Windows. */
     char *tmp_filename = g_malloc0(PATH_MAX + 1);
@@ -1316,17 +1320,30 @@ int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp)
     total_size &= BDRV_SECTOR_MASK;
 
     /* Create the temporary image */
-    ret = get_tmp_filename(tmp_filename, PATH_MAX + 1);
-    if (ret < 0) {
-        error_setg_errno(errp, -ret, "Could not get temporary filename");
-        goto out;
+    /* We will not delete created file in record/replay mode */
+    if (replay_mode != REPLAY_MODE_NONE && fname_suffix) {
+        ret = snprintf(tmp_filename, PATH_MAX + 1, "%s.%s", bs->filename,
+                       fname_suffix);
+        if (ret < 0) {
+            goto out;
+        }
+    } else {
+        ret = get_tmp_filename(tmp_filename, PATH_MAX + 1);
+        if (ret < 0) {
+            error_setg_errno(errp, -ret, "Could not get temporary filename");
+            goto out;
+        }
     }
 
     bdrv_qcow2 = bdrv_find_format("qcow2");
     opts = qemu_opts_create(bdrv_qcow2->create_opts, NULL, 0,
                             &error_abort);
     qemu_opt_set_number(opts, BLOCK_OPT_SIZE, total_size);
-    ret = bdrv_create(bdrv_qcow2, tmp_filename, opts, &local_err);
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        ret = 0;
+    } else {
+        ret = bdrv_create(bdrv_qcow2, tmp_filename, opts, &local_err);
+    }
     qemu_opts_del(opts);
     if (ret < 0) {
         error_setg_errno(errp, -ret, "Could not create temporary overlay "
@@ -1346,7 +1363,7 @@ int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp)
     bs_snapshot = bdrv_new("", &error_abort);
 
     ret = bdrv_open(&bs_snapshot, NULL, NULL, snapshot_options,
-                    flags, bdrv_qcow2, &local_err);
+                    flags, bdrv_qcow2, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto out;
@@ -1376,7 +1393,7 @@ out:
  */
 int bdrv_open(BlockDriverState **pbs, const char *filename,
               const char *reference, QDict *options, int flags,
-              BlockDriver *drv, Error **errp)
+              BlockDriver *drv, Error **errp, const char *fname_suffix)
 {
     int ret;
     BlockDriverState *file = NULL, *bs;
@@ -1457,13 +1474,16 @@ int bdrv_open(BlockDriverState **pbs, const char *filename,
         }
         if (flags & BDRV_O_SNAPSHOT) {
             snapshot_flags = bdrv_temp_snapshot_flags(flags);
+            if (replay_mode != REPLAY_MODE_NONE) {
+                snapshot_flags &= ~BDRV_O_TEMPORARY;
+            }
             flags = bdrv_backing_flags(flags);
         }
 
         assert(file == NULL);
         ret = bdrv_open_image(&file, filename, options, "file",
                               bdrv_inherited_flags(flags),
-                              true, &local_err);
+                              true, &local_err, NULL);
         if (ret < 0) {
             goto fail;
         }
@@ -1506,7 +1526,8 @@ int bdrv_open(BlockDriverState **pbs, const char *filename,
     /* For snapshot=on, create a temporary qcow2 overlay. bs points to the
      * temporary snapshot afterwards. */
     if (snapshot_flags) {
-        ret = bdrv_append_temp_snapshot(bs, snapshot_flags, &local_err);
+        ret = bdrv_append_temp_snapshot(bs, snapshot_flags, &local_err,
+                                        fname_suffix);
         if (local_err) {
             goto close_and_fail;
         }
@@ -5600,7 +5621,7 @@ void bdrv_img_create(const char *filename, const char *fmt,
 
             bs = NULL;
             ret = bdrv_open(&bs, backing_file, NULL, NULL, back_flags,
-                            backing_drv, &local_err);
+                            backing_drv, &local_err, NULL);
             if (ret < 0) {
                 error_setg_errno(errp, -ret, "Could not open '%s': %s",
                                  backing_file,
diff --git a/block/blkdebug.c b/block/blkdebug.c
index f51407d..151538a 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -412,7 +412,7 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
     /* Open the backing file */
     assert(bs->file == NULL);
     ret = bdrv_open_image(&bs->file, qemu_opt_get(opts, "x-image"), options, "image",
-                          flags | BDRV_O_PROTOCOL, false, &local_err);
+                          flags | BDRV_O_PROTOCOL, false, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto out;
diff --git a/block/blkverify.c b/block/blkverify.c
index 621b785..6530835 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -138,7 +138,8 @@ static int blkverify_open(BlockDriverState *bs, QDict *options, int flags,
     /* Open the raw file */
     assert(bs->file == NULL);
     ret = bdrv_open_image(&bs->file, qemu_opt_get(opts, "x-raw"), options,
-                          "raw", flags | BDRV_O_PROTOCOL, false, &local_err);
+                          "raw", flags | BDRV_O_PROTOCOL, false, &local_err,
+                          NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto fail;
@@ -147,7 +148,7 @@ static int blkverify_open(BlockDriverState *bs, QDict *options, int flags,
     /* Open the test file */
     assert(s->test_file == NULL);
     ret = bdrv_open_image(&s->test_file, qemu_opt_get(opts, "x-image"), options,
-                          "test", flags, false, &local_err);
+                          "test", flags, false, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         s->test_file = NULL;
diff --git a/block/cow.c b/block/cow.c
index 6ee4833..d99c4d7 100644
--- a/block/cow.c
+++ b/block/cow.c
@@ -345,7 +345,7 @@ static int cow_create(const char *filename, QemuOpts *opts, Error **errp)
     }
 
     ret = bdrv_open(&cow_bs, filename, NULL, NULL,
-                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto exit;
diff --git a/block/qcow.c b/block/qcow.c
index a874056..6e21cea 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -720,7 +720,7 @@ static int qcow_create(const char *filename, QemuOpts *opts, Error **errp)
 
     qcow_bs = NULL;
     ret = bdrv_open(&qcow_bs, filename, NULL, NULL,
-                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto cleanup;
diff --git a/block/qcow2.c b/block/qcow2.c
index 1e3ab6b..43ceb34 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1653,7 +1653,7 @@ static int qcow2_create2(const char *filename, int64_t total_size,
 
     bs = NULL;
     ret = bdrv_open(&bs, filename, NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL,
-                    NULL, &local_err);
+                    NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return ret;
@@ -1715,7 +1715,7 @@ static int qcow2_create2(const char *filename, int64_t total_size,
     BlockDriver* drv = bdrv_find_format("qcow2");
     assert(drv != NULL);
     ret = bdrv_open(&bs, filename, NULL, NULL,
-        BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH, drv, &local_err);
+        BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH, drv, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto out;
@@ -1767,7 +1767,7 @@ static int qcow2_create2(const char *filename, int64_t total_size,
     /* Reopen the image without BDRV_O_NO_FLUSH to flush it before returning */
     ret = bdrv_open(&bs, filename, NULL, NULL,
                     BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_BACKING,
-                    drv, &local_err);
+                    drv, &local_err, NULL);
     if (local_err) {
         error_propagate(errp, local_err);
         goto out;
diff --git a/block/qed.c b/block/qed.c
index 7944832..e41c422 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -593,7 +593,7 @@ static int qed_create(const char *filename, uint32_t cluster_size,
     bs = NULL;
     ret = bdrv_open(&bs, filename, NULL, NULL,
                     BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_PROTOCOL, NULL,
-                    &local_err);
+                    &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return ret;
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 8d9350c..bb3dfef 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -1576,7 +1576,7 @@ static int sd_prealloc(const char *filename, Error **errp)
     int ret;
 
     ret = bdrv_open(&bs, filename, NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL,
-                    NULL, errp);
+                    NULL, errp, NULL);
     if (ret < 0) {
         goto out_with_err_set;
     }
@@ -1739,7 +1739,7 @@ static int sd_create(const char *filename, QemuOpts *opts,
 
         bs = NULL;
         ret = bdrv_open(&bs, backing_file, NULL, NULL, BDRV_O_PROTOCOL, NULL,
-                        errp);
+                        errp, NULL);
         if (ret < 0) {
             goto out;
         }
diff --git a/block/vmdk.c b/block/vmdk.c
index 0517bba..09587cb 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -809,7 +809,7 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
                 desc_file_path, fname);
         extent_file = NULL;
         ret = bdrv_open(&extent_file, extent_path, NULL, NULL,
-                        bs->open_flags | BDRV_O_PROTOCOL, NULL, errp);
+                        bs->open_flags | BDRV_O_PROTOCOL, NULL, errp, NULL);
         if (ret) {
             return ret;
         }
@@ -1545,7 +1545,7 @@ static int vmdk_create_extent(const char *filename, int64_t filesize,
 
     assert(bs == NULL);
     ret = bdrv_open(&bs, filename, NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL,
-                    NULL, &local_err);
+                    NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto exit;
@@ -1803,7 +1803,7 @@ static int vmdk_create(const char *filename, QemuOpts *opts, Error **errp)
     if (backing_file) {
         BlockDriverState *bs = NULL;
         ret = bdrv_open(&bs, backing_file, NULL, NULL, BDRV_O_NO_BACKING, NULL,
-                        errp);
+                        errp, NULL);
         if (ret != 0) {
             goto exit;
         }
@@ -1879,7 +1879,7 @@ static int vmdk_create(const char *filename, QemuOpts *opts, Error **errp)
     }
     assert(new_bs == NULL);
     ret = bdrv_open(&new_bs, filename, NULL, NULL,
-                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+                    BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         goto exit;
diff --git a/block/vvfat.c b/block/vvfat.c
index 70176b1..f86f890 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -2939,7 +2939,7 @@ static int enable_write_target(BDRVVVFATState *s, Error **errp)
     s->qcow = NULL;
     ret = bdrv_open(&s->qcow, s->qcow_filename, NULL, NULL,
                     BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH,
-                    bdrv_qcow, errp);
+                    bdrv_qcow, errp, NULL);
     if (ret < 0) {
         goto err;
     }
diff --git a/blockdev.c b/blockdev.c
index 48bd9a3..8173401 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -44,6 +44,7 @@
 #include "qmp-commands.h"
 #include "trace.h"
 #include "sysemu/arch_init.h"
+#include "replay/replay.h"
 
 static QTAILQ_HEAD(drivelist, DriveInfo) drives = QTAILQ_HEAD_INITIALIZER(drives);
 
@@ -520,7 +521,8 @@ static DriveInfo *blockdev_init(const char *file, QDict *bs_opts,
     bdrv_flags |= ro ? 0 : BDRV_O_RDWR;
 
     QINCREF(bs_opts);
-    ret = bdrv_open(&dinfo->bdrv, file, NULL, bs_opts, bdrv_flags, drv, &error);
+    ret = bdrv_open(&dinfo->bdrv, file, NULL, bs_opts, bdrv_flags, drv,
+                    &error, replay_image_suffix);
 
     if (ret < 0) {
         error_setg(errp, "could not open disk image %s: %s",
@@ -1367,7 +1369,7 @@ static void external_snapshot_prepare(BlkTransactionState *common,
      * extended QMP command? */
     assert(state->new_bs == NULL);
     ret = bdrv_open(&state->new_bs, new_image_file, NULL, options,
-                    flags | BDRV_O_NO_BACKING, drv, &local_err);
+                    flags | BDRV_O_NO_BACKING, drv, &local_err, NULL);
     /* We will manually add the backing_hd field to the bs later */
     if (ret != 0) {
         error_propagate(errp, local_err);
@@ -1616,7 +1618,8 @@ static void qmp_bdrv_open_encrypted(BlockDriverState *bs, const char *filename,
     Error *local_err = NULL;
     int ret;
 
-    ret = bdrv_open(&bs, filename, NULL, NULL, bdrv_flags, drv, &local_err);
+    ret = bdrv_open(&bs, filename, NULL, NULL, bdrv_flags, drv, &local_err,
+                    NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return;
@@ -2108,7 +2111,8 @@ void qmp_drive_backup(const char *device, const char *target,
     }
 
     target_bs = NULL;
-    ret = bdrv_open(&target_bs, target, NULL, NULL, flags, drv, &local_err);
+    ret = bdrv_open(&target_bs, target, NULL, NULL, flags, drv, &local_err,
+                    NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return;
@@ -2282,7 +2286,7 @@ void qmp_drive_mirror(const char *device, const char *target,
      */
     target_bs = NULL;
     ret = bdrv_open(&target_bs, target, NULL, options,
-                    flags | BDRV_O_NO_BACKING, drv, &local_err);
+                    flags | BDRV_O_NO_BACKING, drv, &local_err, NULL);
     if (ret < 0) {
         error_propagate(errp, local_err);
         return;
diff --git a/include/block/block.h b/include/block/block.h
index f08471d..95e6d1c 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -211,13 +211,14 @@ int bdrv_parse_cache_flags(const char *mode, int *flags);
 int bdrv_parse_discard_flags(const char *mode, int *flags);
 int bdrv_open_image(BlockDriverState **pbs, const char *filename,
                     QDict *options, const char *bdref_key, int flags,
-                    bool allow_none, Error **errp);
+                    bool allow_none, Error **errp, const char *fname_suffix);
 void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd);
 int bdrv_open_backing_file(BlockDriverState *bs, QDict *options, Error **errp);
-int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp);
+int bdrv_append_temp_snapshot(BlockDriverState *bs, int flags, Error **errp,
+                              const char *fname_suffix);
 int bdrv_open(BlockDriverState **pbs, const char *filename,
               const char *reference, QDict *options, int flags,
-              BlockDriver *drv, Error **errp);
+              BlockDriver *drv, Error **errp, const char *fname_suffix);
 BlockReopenQueue *bdrv_reopen_queue(BlockReopenQueue *bs_queue,
                                     BlockDriverState *bs, int flags);
 int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp);
diff --git a/qemu-img.c b/qemu-img.c
index d4518e7..30fd609 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -320,7 +320,7 @@ static BlockDriverState *bdrv_new_open(const char *id,
         drv = NULL;
     }
 
-    ret = bdrv_open(&bs, filename, NULL, NULL, flags, drv, &local_err);
+    ret = bdrv_open(&bs, filename, NULL, NULL, flags, drv, &local_err, NULL);
     if (ret < 0) {
         error_report("Could not open '%s': %s", filename,
                      error_get_pretty(local_err));
@@ -2383,7 +2383,7 @@ static int img_rebase(int argc, char **argv)
         bs_old_backing = bdrv_new("old_backing", &error_abort);
         bdrv_get_backing_filename(bs, backing_name, sizeof(backing_name));
         ret = bdrv_open(&bs_old_backing, backing_name, NULL, NULL, BDRV_O_FLAGS,
-                        old_backing_drv, &local_err);
+                        old_backing_drv, &local_err, NULL);
         if (ret) {
             error_report("Could not open old backing file '%s': %s",
                          backing_name, error_get_pretty(local_err));
@@ -2393,7 +2393,7 @@ static int img_rebase(int argc, char **argv)
         if (out_baseimg[0]) {
             bs_new_backing = bdrv_new("new_backing", &error_abort);
             ret = bdrv_open(&bs_new_backing, out_baseimg, NULL, NULL,
-                            BDRV_O_FLAGS, new_backing_drv, &local_err);
+                            BDRV_O_FLAGS, new_backing_drv, &local_err, NULL);
             if (ret) {
                 error_report("Could not open new backing file '%s': %s",
                              out_baseimg, error_get_pretty(local_err));
diff --git a/qemu-io.c b/qemu-io.c
index b55a550..2c5029e 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -60,7 +60,7 @@ static int openfile(char *name, int flags, int growable, QDict *opts)
 
     if (growable) {
         if (bdrv_open(&qemuio_bs, name, NULL, opts, flags | BDRV_O_PROTOCOL,
-                      NULL, &local_err))
+                      NULL, &local_err, NULL))
         {
             fprintf(stderr, "%s: can't open%s%s: %s\n", progname,
                     name ? " device " : "", name ?: "",
@@ -71,9 +71,8 @@ static int openfile(char *name, int flags, int growable, QDict *opts)
     } else {
         qemuio_bs = bdrv_new("hda", &error_abort);
 
-        if (bdrv_open(&qemuio_bs, name, NULL, opts, flags, NULL, &local_err)
-            < 0)
-        {
+        if (bdrv_open(&qemuio_bs, name, NULL, opts, flags, NULL,
+                      &local_err, NULL) < 0) {
             fprintf(stderr, "%s: can't open%s%s: %s\n", progname,
                     name ? " device " : "", name ?: "",
                     error_get_pretty(local_err));
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 626e584..e780d1c 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -663,7 +663,7 @@ int main(int argc, char **argv)
     bs = bdrv_new("hda", &error_abort);
 
     srcpath = argv[optind];
-    ret = bdrv_open(&bs, srcpath, NULL, NULL, flags, drv, &local_err);
+    ret = bdrv_open(&bs, srcpath, NULL, NULL, flags, drv, &local_err, NULL);
     if (ret < 0) {
         errno = -ret;
         err(EXIT_FAILURE, "Failed to bdrv_open '%s': %s", argv[optind],

[Qemu-devel] [RFC PATCH v3 20/49] sysemu: system functions for replay

[Pavel Dovgalyuk]

This patch removes "static" specifier from several qemu function to make
them visible to the replay module. It also invents several system functions
that will be used by replay.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 arch_init.c             |    8 ++++++++
 cpu-exec.c              |    6 ++++--
 cpus.c                  |    4 ++--
 include/exec/exec-all.h |    4 +++-
 include/sysemu/cpus.h   |    1 +
 include/sysemu/sysemu.h |    1 +
 savevm.c                |   33 +++++++++++++++++++++++++++------
 translate-all.c         |    8 ++++++++
 8 files changed, 54 insertions(+), 11 deletions(-)

diff --git a/arch_init.c b/arch_init.c
index 8ddaf35..9c9bae6 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -52,6 +52,7 @@
 #include "exec/ram_addr.h"
 #include "hw/acpi/acpi.h"
 #include "qemu/host-utils.h"
+#include "replay/replay.h"
 
 #ifdef DEBUG_ARCH_INIT
 #define DPRINTF(fmt, ...) \
@@ -1092,6 +1093,13 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
 
                 total_ram_bytes -= length;
             }
+            if (replay_mode == REPLAY_MODE_PLAY) {
+                RAMBlock *block;
+                /* Clear the blocks' memory instead of resetting the machine */
+                QTAILQ_FOREACH(block, &ram_list.blocks, next) {
+                    memset(block->host, 0, block->length);
+                }
+            }
         } else if (flags & RAM_SAVE_FLAG_COMPRESS) {
             void *host;
             uint8_t ch;
diff --git a/cpu-exec.c b/cpu-exec.c
index 38e5f02..66a693c 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -189,12 +189,14 @@ static inline TranslationBlock *tb_find_fast(CPUArchState *env)
 
 static CPUDebugExcpHandler *debug_excp_handler;
 
-void cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler)
+CPUDebugExcpHandler *cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler)
 {
+    CPUDebugExcpHandler *old = debug_excp_handler;
     debug_excp_handler = handler;
+    return old;
 }
 
-static void cpu_handle_debug_exception(CPUArchState *env)
+void cpu_handle_debug_exception(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
     CPUWatchpoint *wp;
diff --git a/cpus.c b/cpus.c
index 5e7f2cf..bbad529 100644
--- a/cpus.c
+++ b/cpus.c
@@ -85,7 +85,7 @@ static bool cpu_thread_is_idle(CPUState *cpu)
     return true;
 }
 
-static bool all_cpu_threads_idle(void)
+bool all_cpu_threads_idle(void)
 {
     CPUState *cpu;
 
@@ -1033,7 +1033,7 @@ bool qemu_cpu_is_self(CPUState *cpu)
     return qemu_thread_is_self(cpu->thread);
 }
 
-static bool qemu_in_vcpu_thread(void)
+bool qemu_in_vcpu_thread(void)
 {
     return current_cpu && qemu_cpu_is_self(current_cpu);
 }
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 528928f..4b7a241 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -212,6 +212,7 @@ static inline unsigned int tb_phys_hash_func(tb_page_addr_t pc)
 
 void tb_free(TranslationBlock *tb);
 void tb_flush(CPUArchState *env);
+void tb_flush_all(void);
 void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr);
 
 #if defined(USE_DIRECT_JUMP)
@@ -385,7 +386,8 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, target_ulong addr);
 
 typedef void (CPUDebugExcpHandler)(CPUArchState *env);
 
-void cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler);
+CPUDebugExcpHandler *cpu_set_debug_excp_handler(CPUDebugExcpHandler *handler);
+void cpu_handle_debug_exception(CPUArchState *env);
 
 /* vl.c */
 extern int singlestep;
diff --git a/include/sysemu/cpus.h b/include/sysemu/cpus.h
index 4f79081..8455e0e 100644
--- a/include/sysemu/cpus.h
+++ b/include/sysemu/cpus.h
@@ -6,6 +6,7 @@ void qemu_init_cpu_loop(void);
 void resume_all_vcpus(void);
 void pause_all_vcpus(void);
 void cpu_stop_current(void);
+bool all_cpu_threads_idle(void);
 
 void cpu_synchronize_all_states(void);
 void cpu_synchronize_all_post_reset(void);
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index d8539fd..1ebfef9 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -76,6 +76,7 @@ void qemu_add_machine_init_done_notifier(Notifier *notify);
 
 void do_savevm(Monitor *mon, const QDict *qdict);
 int load_vmstate(const char *name);
+int save_vmstate(Monitor *mon, const char *name);
 void do_delvm(Monitor *mon, const QDict *qdict);
 void do_info_snapshots(Monitor *mon, const QDict *qdict);
 
diff --git a/savevm.c b/savevm.c
index e19ae0a..b4079ad 100644
--- a/savevm.c
+++ b/savevm.c
@@ -42,7 +42,7 @@
 #include "qemu/iov.h"
 #include "block/snapshot.h"
 #include "block/qapi.h"
-
+#include "replay/replay.h"
 
 #ifndef ETH_P_RARP
 #define ETH_P_RARP 0x8035
@@ -1039,7 +1039,7 @@ static int del_existing_snapshots(Monitor *mon, const char *name)
     return 0;
 }
 
-void do_savevm(Monitor *mon, const QDict *qdict)
+int save_vmstate(Monitor *mon, const char *name)
 {
     BlockDriverState *bs, *bs1;
     QEMUSnapshotInfo sn1, *sn = &sn1, old_sn1, *old_sn = &old_sn1;
@@ -1049,7 +1049,7 @@ void do_savevm(Monitor *mon, const QDict *qdict)
     uint64_t vm_state_size;
     qemu_timeval tv;
     struct tm tm;
-    const char *name = qdict_get_try_str(qdict, "name");
+    int success = 0;
 
     /* Verify if there is a device that doesn't support snapshots and is writable */
     bs = NULL;
@@ -1062,14 +1062,19 @@ void do_savevm(Monitor *mon, const QDict *qdict)
         if (!bdrv_can_snapshot(bs)) {
             monitor_printf(mon, "Device '%s' is writable but does not support snapshots.\n",
                                bdrv_get_device_name(bs));
-            return;
+            return success;
         }
     }
 
     bs = find_vmstate_bs();
     if (!bs) {
         monitor_printf(mon, "No block device can accept snapshots\n");
-        return;
+        if (replay_mode != REPLAY_MODE_NONE) {
+            fprintf(stderr,
+                    "At least one hdd should be attached to QEMU for replay\n");
+            exit(1);
+        }
+        return success;
     }
 
     saved_vm_running = runstate_is_running();
@@ -1118,6 +1123,7 @@ void do_savevm(Monitor *mon, const QDict *qdict)
 
     /* create the snapshots */
 
+    success = 1;
     bs1 = NULL;
     while ((bs1 = bdrv_next(bs1))) {
         if (bdrv_can_snapshot(bs1)) {
@@ -1127,6 +1133,7 @@ void do_savevm(Monitor *mon, const QDict *qdict)
             if (ret < 0) {
                 monitor_printf(mon, "Error while creating snapshot on '%s'\n",
                                bdrv_get_device_name(bs1));
+                success = 0;
             }
         }
     }
@@ -1135,6 +1142,14 @@ void do_savevm(Monitor *mon, const QDict *qdict)
     if (saved_vm_running) {
         vm_start();
     }
+
+    return success;
+}
+
+void do_savevm(Monitor *mon, const QDict *qdict)
+{
+    const char *name = qdict_get_try_str(qdict, "name");
+    save_vmstate(mon, name);
 }
 
 void qmp_xen_save_devices_state(const char *filename, Error **errp)
@@ -1231,7 +1246,13 @@ int load_vmstate(const char *name)
         return -EINVAL;
     }
 
-    qemu_system_reset(VMRESET_SILENT);
+    /* Do not reset in replay mode.
+       1. Reset will alter the behavior in play mode compared to save one
+       2. Timers read by reset handlers are not correct, because
+          replay reads them from the unknown part of the log */
+    if (replay_mode == REPLAY_MODE_NONE) {
+        qemu_system_reset(VMRESET_SILENT);
+    }
     ret = qemu_loadvm_state(f);
 
     qemu_fclose(f);
diff --git a/translate-all.c b/translate-all.c
index 8f7e11b..fcb0697 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -803,6 +803,14 @@ void tb_flush(CPUArchState *env1)
     tcg_ctx.tb_ctx.tb_flush_count++;
 }
 
+void tb_flush_all(void)
+{
+    CPUState *cpu;
+    for (cpu = first_cpu ; cpu != NULL ; cpu = CPU_NEXT(cpu)) {
+        tb_flush(cpu->env_ptr);
+    }
+}
+
 #ifdef DEBUG_TB_CHECK
 
 static void tb_invalidate_check(target_ulong address)

[Qemu-devel] [RFC PATCH v3 21/49] replay: internal functions for replay log

[Pavel Dovgalyuk]

This patch adds functions to perform read and write operations
with replay log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 replay/Makefile.objs     |    1 
 replay/replay-internal.c |  141 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h |   50 ++++++++++++++++
 3 files changed, 192 insertions(+), 0 deletions(-)
 create mode 100755 replay/replay-internal.c
 create mode 100755 replay/replay-internal.h

diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 7ea860f..1148f45 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -1 +1,2 @@
 obj-y += replay.o
+obj-y += replay-internal.o
diff --git a/replay/replay-internal.c b/replay/replay-internal.c
new file mode 100755
index 0000000..29f995d
--- /dev/null
+++ b/replay/replay-internal.c
@@ -0,0 +1,141 @@
+/*
+ * replay-internal.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "replay-internal.h"
+
+volatile unsigned int replay_data_kind = -1;
+volatile unsigned int replay_has_unread_data;
+
+/* File for replay writing */
+FILE *replay_file;
+
+void replay_put_byte(unsigned char byte)
+{
+    if (replay_file) {
+        fwrite(&byte, sizeof(byte), 1, replay_file);
+    }
+}
+
+void replay_put_event(unsigned char event)
+{
+    replay_put_byte(event);
+}
+
+
+void replay_put_word(uint16_t word)
+{
+    if (replay_file) {
+        fwrite(&word, sizeof(word), 1, replay_file);
+    }
+}
+
+void replay_put_dword(unsigned int dword)
+{
+    if (replay_file) {
+        fwrite(&dword, sizeof(dword), 1, replay_file);
+    }
+}
+
+void replay_put_qword(int64_t qword)
+{
+    if (replay_file) {
+        fwrite(&qword, sizeof(qword), 1, replay_file);
+    }
+}
+
+void replay_put_array(const uint8_t *buf, size_t size)
+{
+    if (replay_file) {
+        fwrite(&size, sizeof(size), 1, replay_file);
+        fwrite(buf, 1, size, replay_file);
+    }
+}
+
+unsigned char replay_get_byte(void)
+{
+    unsigned char byte;
+    if (replay_file) {
+        fread(&byte, sizeof(byte), 1, replay_file);
+    }
+    return byte;
+}
+
+uint16_t replay_get_word(void)
+{
+    uint16_t word;
+    if (replay_file) {
+        fread(&word, sizeof(word), 1, replay_file);
+    }
+
+    return word;
+}
+
+unsigned int replay_get_dword(void)
+{
+    unsigned int dword;
+    if (replay_file) {
+        fread(&dword, sizeof(dword), 1, replay_file);
+    }
+
+    return dword;
+}
+
+int64_t replay_get_qword(void)
+{
+    int64_t qword;
+    if (replay_file) {
+        fread(&qword, sizeof(qword), 1, replay_file);
+    }
+
+    return qword;
+}
+
+void replay_get_array(uint8_t *buf, size_t *size)
+{
+    if (replay_file) {
+        fread(size, sizeof(*size), 1, replay_file);
+        fread(buf, 1, *size, replay_file);
+    }
+}
+
+void replay_get_array_alloc(uint8_t **buf, size_t *size)
+{
+    if (replay_file) {
+        fread(size, sizeof(*size), 1, replay_file);
+        *buf = g_malloc(*size);
+        fread(*buf, 1, *size, replay_file);
+    }
+}
+
+void replay_check_error(void)
+{
+    if (replay_file) {
+        if (feof(replay_file)) {
+            fprintf(stderr, "replay file is over\n");
+            exit(1);
+        } else if (ferror(replay_file)) {
+            fprintf(stderr, "replay file is over or something goes wrong\n");
+            exit(1);
+        }
+    }
+}
+
+void replay_fetch_data_kind(void)
+{
+    if (replay_file) {
+        if (!replay_has_unread_data) {
+            replay_data_kind = replay_get_byte();
+            replay_check_error();
+            replay_has_unread_data = 1;
+        }
+    }
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
new file mode 100755
index 0000000..45bb344
--- /dev/null
+++ b/replay/replay-internal.h
@@ -0,0 +1,50 @@
+#ifndef REPLAY_INTERNAL_H
+#define REPLAY_INTERNAL_H
+
+/*
+ * replay-internal.h
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include <stdio.h>
+
+extern volatile unsigned int replay_data_kind;
+extern volatile unsigned int replay_has_unread_data;
+
+/* File for replay writing */
+extern FILE *replay_file;
+
+void replay_put_byte(unsigned char byte);
+void replay_put_event(unsigned char event);
+void replay_put_word(uint16_t word);
+void replay_put_dword(unsigned int dword);
+void replay_put_qword(int64_t qword);
+void replay_put_array(const uint8_t *buf, size_t size);
+
+unsigned char replay_get_byte(void);
+uint16_t replay_get_word(void);
+unsigned int replay_get_dword(void);
+int64_t replay_get_qword(void);
+void replay_get_array(uint8_t *buf, size_t *size);
+void replay_get_array_alloc(uint8_t **buf, size_t *size);
+
+/*! Checks error status of the file. */
+void replay_check_error(void);
+
+/*! Reads data type from the file and stores it in the
+    replay_data_kind variable. */
+void replay_fetch_data_kind(void);
+
+/*! Saves queued events (like instructions and sound). */
+void replay_save_instructions(void);
+/*! Checks that the next data is corresponding to the desired kind.
+    Terminates the program in case of error. */
+void validate_data_kind(int kind);
+
+#endif

[Qemu-devel] [RFC PATCH v3 22/49] cpu: invent instruction count for accurate replay

[Pavel Dovgalyuk]

This patch adds instructions count fields to cpu structure and
invents several functions for increasing this counter while executing
translation blocks.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpu-exec.c               |   14 ++++-
 cpus.c                   |    5 ++
 exec.c                   |    4 +
 include/exec/cpu-defs.h  |    2 +
 include/qom/cpu.h        |    3 +
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   26 +++++++++
 replay/replay-internal.c |   14 +++++
 replay/replay-internal.h |   37 ++++++++++++-
 replay/replay.c          |  135 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   15 +++++
 translate-all.c          |    7 ++
 12 files changed, 258 insertions(+), 5 deletions(-)
 create mode 100755 replay/replay-events.c

diff --git a/cpu-exec.c b/cpu-exec.c
index 66a693c..13c0ec6 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -22,6 +22,7 @@
 #include "tcg.h"
 #include "qemu/atomic.h"
 #include "sysemu/qtest.h"
+#include "qemu/main-loop.h"
 
 void cpu_loop_exit(CPUState *cpu)
 {
@@ -283,19 +284,24 @@ int cpu_exec(CPUArchState *env)
 #else
 #error unsupported target CPU
 #endif
-    cpu->exception_index = -1;
-
     /* prepare setjmp context for exception handling */
     for(;;) {
         if (sigsetjmp(cpu->jmp_env, 0) == 0) {
             /* if an exception is pending, we execute it here */
             if (cpu->exception_index >= 0) {
+                if (cpu->exception_index == EXCP_REPLAY) {
+                    ret = cpu->exception_index;
+                    cpu->exception_index = -1;
+                    qemu_notify_event();
+                    break;
+                }
                 if (cpu->exception_index >= EXCP_INTERRUPT) {
                     /* exit request from the cpu execution loop */
                     ret = cpu->exception_index;
                     if (ret == EXCP_DEBUG) {
                         cpu_handle_debug_exception(env);
                     }
+                    cpu->exception_index = -1;
                     break;
                 } else {
 #if defined(CONFIG_USER_ONLY)
@@ -601,6 +607,10 @@ int cpu_exec(CPUArchState *env)
                         next_tb = 0;
                     }
                 }
+                if (cpu->exception_index == EXCP_REPLAY) {
+                    /* go to exception_index checking */
+                    break;
+                }
                 if (unlikely(cpu->exit_request)) {
                     cpu->exit_request = 0;
                     cpu->exception_index = EXCP_INTERRUPT;
diff --git a/cpus.c b/cpus.c
index bbad529..eb2a795 100644
--- a/cpus.c
+++ b/cpus.c
@@ -937,6 +937,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
     CPU_FOREACH(cpu) {
         cpu->thread_id = qemu_get_thread_id();
         cpu->created = true;
+        /* init exception index here */
+        cpu->exception_index = -1;
     }
     qemu_cond_signal(&qemu_cpu_cond);
 
@@ -1186,6 +1188,7 @@ void qemu_init_vcpu(CPUState *cpu)
     cpu->nr_cores = smp_cores;
     cpu->nr_threads = smp_threads;
     cpu->stopped = true;
+    cpu->instructions_count = 0;
     if (kvm_enabled()) {
         qemu_kvm_start_vcpu(cpu);
     } else if (tcg_enabled()) {
@@ -1307,6 +1310,8 @@ static void tcg_exec_all(void)
             if (r == EXCP_DEBUG) {
                 cpu_handle_guest_debug(cpu);
                 break;
+            } else if (r == EXCP_REPLAY) {
+                break;
             }
         } else if (cpu->stop || cpu->stopped) {
             break;
diff --git a/exec.c b/exec.c
index 307bc24..643e564 100644
--- a/exec.c
+++ b/exec.c
@@ -52,6 +52,7 @@
 #include "exec/ram_addr.h"
 
 #include "qemu/range.h"
+#include "replay/replay.h"
 
 //#define DEBUG_SUBPAGE
 
@@ -1649,6 +1650,9 @@ static void check_watchpoint(int offset, int len_mask, int flags)
             if (!cpu->watchpoint_hit) {
                 cpu->watchpoint_hit = wp;
                 tb_check_watchpoint(cpu);
+                /* Current instruction is already processed by replay.
+                   Set flags that allow skpping this events */
+                replay_undo_last_instruction();
                 if (wp->flags & BP_STOP_BEFORE_ACCESS) {
                     cpu->exception_index = EXCP_DEBUG;
                     cpu_loop_exit(cpu);
diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index 2dd6206..8fb8335 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -59,6 +59,8 @@ typedef uint64_t target_ulong;
 #define EXCP_DEBUG      0x10002 /* cpu stopped after a breakpoint or singlestep */
 #define EXCP_HALTED     0x10003 /* cpu is halted (waiting for external event) */
 #define EXCP_YIELD      0x10004 /* cpu wants to yield timeslice to another */
+#define EXCP_REPLAY     0x10005 /* for breaking execution loop to make
+                                   correct order of events */
 
 /* Only the bottom TB_JMP_PAGE_BITS of the jump cache hash bits vary for
    addresses on the same page.  The top bits are the same.  This allows
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 1aafbf5..26cea68 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -287,6 +287,7 @@ struct CPUState {
        (absolute value) offset as small as possible.  This reduces code
        size, especially for hosts without large memory offsets.  */
     volatile sig_atomic_t tcg_exit_req;
+    uint32_t instructions_count;
 };
 
 QTAILQ_HEAD(CPUTailQ, CPUState);
@@ -466,6 +467,8 @@ static inline bool cpu_has_work(CPUState *cpu)
  */
 bool qemu_cpu_is_self(CPUState *cpu);
 
+bool qemu_in_vcpu_thread(void);
+
 /**
  * qemu_cpu_kick:
  * @cpu: The vCPU to kick.
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 1148f45..56da09c 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -1,2 +1,3 @@
 obj-y += replay.o
 obj-y += replay-internal.o
+obj-y += replay-events.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
new file mode 100755
index 0000000..ffa2ccf
--- /dev/null
+++ b/replay/replay-events.c
@@ -0,0 +1,26 @@
+/*
+ * replay-events.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "replay.h"
+#include "replay-internal.h"
+
+bool replay_has_events(void)
+{
+    return false;
+}
+
+void replay_save_events(int opt)
+{
+}
+
+void replay_read_events(int opt)
+{
+}
diff --git a/replay/replay-internal.c b/replay/replay-internal.c
index 29f995d..0c303d6 100755
--- a/replay/replay-internal.c
+++ b/replay/replay-internal.c
@@ -10,6 +10,7 @@
  */
 
 #include "qemu-common.h"
+#include "replay.h"
 #include "replay-internal.h"
 
 volatile unsigned int replay_data_kind = -1;
@@ -139,3 +140,16 @@ void replay_fetch_data_kind(void)
         }
     }
 }
+
+/*! Saves cached instructions. */
+void replay_save_instructions(void)
+{
+    if (replay_file && replay_mode == REPLAY_MODE_RECORD) {
+        if (first_cpu != NULL && first_cpu->instructions_count > 0) {
+            replay_put_event(EVENT_INSTRUCTION);
+            replay_put_dword(first_cpu->instructions_count);
+            replay_state.current_step += first_cpu->instructions_count;
+            first_cpu->instructions_count = 0;
+        }
+    }
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 45bb344..6e7d96f 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -13,6 +13,21 @@
  */
 
 #include <stdio.h>
+#include "sysemu/sysemu.h"
+
+/* for async events */
+#define EVENT_ASYNC                 24
+/* for instruction event */
+#define EVENT_INSTRUCTION           32
+
+typedef struct ReplayState {
+    /*! Nonzero, when next instruction is repeated one and was already
+        processed. */
+    int skipping_instruction;
+    /*! Current step - number of processed instructions and timer events. */
+    uint64_t current_step;
+} ReplayState;
+extern ReplayState replay_state;
 
 extern volatile unsigned int replay_data_kind;
 extern volatile unsigned int replay_has_unread_data;
@@ -40,11 +55,27 @@ void replay_check_error(void);
 /*! Reads data type from the file and stores it in the
     replay_data_kind variable. */
 void replay_fetch_data_kind(void);
-
-/*! Saves queued events (like instructions and sound). */
-void replay_save_instructions(void);
 /*! Checks that the next data is corresponding to the desired kind.
     Terminates the program in case of error. */
 void validate_data_kind(int kind);
 
+/*! Saves queued events (like instructions and sound). */
+void replay_save_instructions(void);
+
+/*! Skips async events until some sync event will be found. */
+bool skip_async_events(int stop_event);
+/*! Skips async events invocations from the input,
+    until required data kind is found. If the requested data is not found
+    reports an error and stops the execution. */
+void skip_async_events_until(unsigned int kind);
+
+/* Asynchronous events queue */
+
+/*! Returns true if there are any unsaved events in the queue */
+bool replay_has_events(void);
+/*! Saves events from queue into the file */
+void replay_save_events(int opt);
+/*! Read events from the file into the input queue */
+void replay_read_events(int opt);
+
 #endif
diff --git a/replay/replay.c b/replay/replay.c
index ac976b2..10c67f7 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -9,7 +9,9 @@
  *
  */
 
+#include "qemu-common.h"
 #include "replay.h"
+#include "replay-internal.h"
 
 ReplayMode replay_mode = REPLAY_MODE_NONE;
 /*! Stores current submode for PLAY mode */
@@ -18,8 +20,141 @@ ReplaySubmode play_submode = REPLAY_SUBMODE_UNKNOWN;
 /* Suffix for the disk images filenames */
 char *replay_image_suffix;
 
+ReplayState replay_state;
+
 
 ReplaySubmode replay_get_play_submode(void)
 {
     return play_submode;
 }
+
+bool skip_async_events(int stop_event)
+{
+    /* nothing to skip - not all instructions used */
+    if (first_cpu != NULL && first_cpu->instructions_count != 0
+        && replay_has_unread_data) {
+        return stop_event == EVENT_INSTRUCTION;
+    }
+
+    bool res = false;
+    while (true) {
+        replay_fetch_data_kind();
+        if (stop_event == replay_data_kind) {
+            res = true;
+        }
+        switch (replay_data_kind) {
+        case EVENT_INSTRUCTION:
+            first_cpu->instructions_count = replay_get_dword();
+            return res;
+        default:
+            /* clock, time_t, checkpoint and other events */
+            return res;
+        }
+    }
+
+    return res;
+}
+
+void skip_async_events_until(unsigned int kind)
+{
+    if (!skip_async_events(kind)) {
+        if (replay_data_kind == EVENT_ASYNC && kind == EVENT_INSTRUCTION) {
+            return;
+        }
+
+        fprintf(stderr, "%"PRId64": Read data kind %d instead of expected %d\n",
+            replay_get_current_step(), replay_data_kind, kind);
+        exit(1);
+    }
+}
+
+void replay_instruction(int process_events)
+{
+    if (replay_state.skipping_instruction) {
+        replay_state.skipping_instruction = 0;
+        return;
+    }
+
+    if (replay_file) {
+        if (replay_mode == REPLAY_MODE_RECORD) {
+            if (process_events && replay_has_events()) {
+                replay_save_instructions();
+                /* events will be after the last instruction */
+                replay_save_events(-1);
+            } else {
+                /* instruction - increase the step counter */
+                ++first_cpu->instructions_count;
+            }
+        } else if (replay_mode == REPLAY_MODE_PLAY) {
+            skip_async_events_until(EVENT_INSTRUCTION);
+            if (first_cpu->instructions_count >= 1) {
+                ++replay_state.current_step;
+                --first_cpu->instructions_count;
+                if (first_cpu->instructions_count == 0) {
+                    replay_has_unread_data = 0;
+                }
+            } else {
+                replay_read_events(-1);
+            }
+        }
+    }
+}
+
+void replay_undo_last_instruction(void)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        first_cpu->instructions_count--;
+    } else {
+        replay_state.skipping_instruction = 1;
+    }
+}
+
+bool replay_has_async_request(void)
+{
+    if (replay_state.skipping_instruction) {
+        return false;
+    }
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        if (skip_async_events(EVENT_ASYNC)) {
+            return true;
+        }
+
+        return false;
+    } else if (replay_mode == REPLAY_MODE_RECORD) {
+        if (replay_has_events()) {
+            return true;
+        }
+    }
+
+    return false;
+}
+
+bool replay_has_instruction(void)
+{
+    if (replay_state.skipping_instruction) {
+        return true;
+    }
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        skip_async_events(EVENT_INSTRUCTION);
+        if (replay_data_kind != EVENT_INSTRUCTION
+            && replay_data_kind != EVENT_ASYNC) {
+            return false;
+        }
+    }
+    return true;
+}
+
+uint64_t replay_get_current_step(void)
+{
+    if (first_cpu == NULL) {
+        return 0;
+    }
+    if (replay_file) {
+        if (replay_mode == REPLAY_MODE_RECORD) {
+            return replay_state.current_step + first_cpu->instructions_count;
+        }
+    }
+    return replay_state.current_step;
+}
diff --git a/replay/replay.h b/replay/replay.h
index 51a18fe..b764df7 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -12,6 +12,8 @@
  *
  */
 
+#include <stdbool.h>
+#include <stdint.h>
 #include "qapi-types.h"
 
 extern ReplayMode replay_mode;
@@ -20,4 +22,17 @@ extern char *replay_image_suffix;
 /*! Returns replay play submode */
 ReplaySubmode replay_get_play_submode(void);
 
+/* Processing the instructions */
+
+/*! Returns number of executed instructions. */
+uint64_t replay_get_current_step(void);
+/*! Called before instruction execution */
+void replay_instruction(int process_events);
+/*! Undo last instruction count, when exception occurs */
+void replay_undo_last_instruction(void);
+/*! Returns true if asynchronous event is pending */
+bool replay_has_async_request(void);
+/*! Returns non-zero if next event is instruction. */
+bool replay_has_instruction(void);
+
 #endif
diff --git a/translate-all.c b/translate-all.c
index fcb0697..37582a2 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -59,6 +59,7 @@
 #include "exec/cputlb.h"
 #include "translate-all.h"
 #include "qemu/timer.h"
+#include "replay/replay.h"
 
 //#define DEBUG_TB_INVALIDATE
 //#define DEBUG_FLUSH
@@ -1171,6 +1172,9 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end,
                 cpu_restore_state_from_tb(cpu, current_tb, cpu->mem_io_pc);
                 cpu_get_tb_cpu_state(env, &current_pc, &current_cs_base,
                                      &current_flags);
+                /* Current instruction is already processed by replay.
+                   Set flags that allow skpping this event */
+                replay_undo_last_instruction();
             }
 #endif /* TARGET_HAS_PRECISE_SMC */
             /* we need to do that to handle the case where a signal
@@ -1290,6 +1294,9 @@ static void tb_invalidate_phys_page(tb_page_addr_t addr,
             cpu_restore_state_from_tb(cpu, current_tb, pc);
             cpu_get_tb_cpu_state(env, &current_pc, &current_cs_base,
                                  &current_flags);
+            /* Current instruction is already processed by replay.
+               Set flags that allow skpping this event */
+            replay_undo_last_instruction();
         }
 #endif /* TARGET_HAS_PRECISE_SMC */
         tb_phys_invalidate(tb, addr);

[Qemu-devel] [RFC PATCH v3 23/49] target-arm: instructions counting code for replay

[Pavel Dovgalyuk]

This patch adds instructions counting into the target-specific part
of arm simulator. In record/replay mode it inserts replay functions
calls and instructions counter increment into the translated code.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 target-arm/Makefile.objs   |    1 +
 target-arm/helper.h        |    2 ++
 target-arm/replay_helper.c |   33 ++++++++++++++++++++++++++++++
 target-arm/translate.c     |   48 +++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 81 insertions(+), 3 deletions(-)
 create mode 100755 target-arm/replay_helper.c

diff --git a/target-arm/Makefile.objs b/target-arm/Makefile.objs
index dcd167e..b8f61df 100644
--- a/target-arm/Makefile.objs
+++ b/target-arm/Makefile.objs
@@ -9,3 +9,4 @@ obj-y += neon_helper.o iwmmxt_helper.o
 obj-y += gdbstub.o
 obj-$(TARGET_AARCH64) += cpu64.o translate-a64.o helper-a64.o gdbstub64.o
 obj-y += crypto_helper.o
+obj-y += replay_helper.o
diff --git a/target-arm/helper.h b/target-arm/helper.h
index facfcd2..0233b64 100644
--- a/target-arm/helper.h
+++ b/target-arm/helper.h
@@ -529,3 +529,5 @@ DEF_HELPER_FLAGS_2(neon_pmull_64_hi, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 #ifdef TARGET_AARCH64
 #include "helper-a64.h"
 #endif
+
+DEF_HELPER_1(replay_instruction, i32, env)
diff --git a/target-arm/replay_helper.c b/target-arm/replay_helper.c
new file mode 100755
index 0000000..418d548
--- /dev/null
+++ b/target-arm/replay_helper.c
@@ -0,0 +1,33 @@
+/*
+ * replay_helper.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "cpu.h"
+#include "exec/helper-proto.h"
+#include "replay/replay.h"
+
+uint32_t helper_replay_instruction(CPUARMState *env)
+{
+    CPUState *cpu = ENV_GET_CPU(env);
+    if (replay_mode == REPLAY_MODE_PLAY
+        && !replay_has_instruction()) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    if (cpu->exit_request) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    int timer = replay_has_async_request();
+    replay_instruction(timer);
+    return timer;
+}
diff --git a/target-arm/translate.c b/target-arm/translate.c
index cf4e767..3407ced 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -31,6 +31,7 @@
 #include "qemu/log.h"
 #include "qemu/bitops.h"
 #include "arm_ldst.h"
+#include "replay/replay.h"
 
 #include "exec/helper-proto.h"
 #include "exec/helper-gen.h"
@@ -10858,6 +10859,32 @@ undef:
     gen_exception_insn(s, 2, EXCP_UDEF, syn_uncategorized());
 }
 
+static void gen_instructions_count(void)
+{
+    TCGv_i32 cpu_tmp_i32 = tcg_temp_new();
+    tcg_gen_ld_i32(cpu_tmp_i32, cpu_env,
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+    tcg_gen_addi_i32(cpu_tmp_i32, cpu_tmp_i32, 1);
+    tcg_gen_st_i32(cpu_tmp_i32, cpu_env,
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+    tcg_temp_free(cpu_tmp_i32);
+}
+
+static void gen_instr_replay(DisasContext *s, target_ulong pc_ptr)
+{
+    int l1 = gen_new_label();
+    TCGv_i32 cpu_tmp2_i32 = tcg_temp_new();
+
+    gen_helper_replay_instruction(cpu_tmp2_i32, cpu_env);
+    tcg_gen_brcondi_i32(TCG_COND_EQ, cpu_tmp2_i32, 0, l1);
+
+    gen_set_condexec(s);
+    gen_set_pc_im(s, pc_ptr);
+    tcg_gen_exit_tb(0);
+    gen_set_label(l1);
+    tcg_temp_free(cpu_tmp2_i32);
+}
+
 /* generate intermediate code in gen_opc_buf and gen_opparam_buf for
    basic block 'tb'. If search_pc is TRUE, also generate PC
    information for each intermediate instruction. */
@@ -11015,13 +11042,27 @@ static inline void gen_intermediate_code_internal(ARMCPU *cpu,
             tcg_ctx.gen_opc_icount[lj] = num_insns;
         }
 
-        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO))
+        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO)
+            && replay_mode == REPLAY_MODE_NONE) {
             gen_io_start();
+        }
 
         if (unlikely(qemu_loglevel_mask(CPU_LOG_TB_OP | CPU_LOG_TB_OP_OPT))) {
             tcg_gen_debug_insn_start(dc->pc);
         }
 
+        if (replay_mode != REPLAY_MODE_NONE) {
+            /* In PLAY mode check events at every instruction, not only at
+               the beginning of the block. This is needed, when replaying
+               has changed the bounds of translation blocks.
+             */
+            if (dc->pc == pc_start || replay_mode == REPLAY_MODE_PLAY) {
+                gen_instr_replay(dc, dc->pc);
+            } else {
+                gen_instructions_count();
+            }
+        }
+
         if (dc->thumb) {
             disas_thumb_insn(env, dc);
             if (dc->condexec_mask) {
@@ -11054,10 +11095,11 @@ static inline void gen_intermediate_code_internal(ARMCPU *cpu,
     } while (!dc->is_jmp && tcg_ctx.gen_opc_ptr < gen_opc_end &&
              !cs->singlestep_enabled &&
              !singlestep &&
-             dc->pc < next_page_start &&
+             /* +3 is for unaligned Thumb2 instructions */
+             dc->pc + 3 < next_page_start &&
              num_insns < max_insns);
 
-    if (tb->cflags & CF_LAST_IO) {
+    if ((tb->cflags & CF_LAST_IO) && replay_mode == REPLAY_MODE_NONE) {
         if (dc->condjmp) {
             /* FIXME:  This can theoretically happen with self-modifying
                code.  */

[Qemu-devel] [RFC PATCH v3 24/49] target-i386: instructions counting code for replay

[Pavel Dovgalyuk]

This patch adds instructions counting into the target-specific part
of i386/x86_64 simulator. In record/replay mode it inserts replay functions
calls and instructions counter increment into the translated code.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 target-i386/Makefile.objs   |    1 +
 target-i386/cpu.h           |    7 ++++
 target-i386/helper.h        |    2 +
 target-i386/replay_helper.c |   33 ++++++++++++++++++
 target-i386/translate.c     |   81 +++++++++++++++++++++++++++++++++++++++++--
 5 files changed, 120 insertions(+), 4 deletions(-)
 create mode 100755 target-i386/replay_helper.c

diff --git a/target-i386/Makefile.objs b/target-i386/Makefile.objs
index 027b94e..09a6e8a 100644
--- a/target-i386/Makefile.objs
+++ b/target-i386/Makefile.objs
@@ -2,6 +2,7 @@ obj-y += translate.o helper.o cpu.o
 obj-y += excp_helper.o fpu_helper.o cc_helper.o int_helper.o svm_helper.o
 obj-y += smm_helper.o misc_helper.o mem_helper.o seg_helper.o
 obj-y += gdbstub.o
+obj-y += replay_helper.o
 obj-$(CONFIG_SOFTMMU) += machine.o arch_memory_mapping.o arch_dump.o
 obj-$(CONFIG_KVM) += kvm.o
 obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 42bda46..c9b92af 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -28,6 +28,13 @@
 #define TARGET_LONG_BITS 32
 #endif
 
+/* Maximum instruction code size */
+#ifdef TARGET_X86_64
+#define TARGET_MAX_INSN_SIZE 16
+#else
+#define TARGET_MAX_INSN_SIZE 16
+#endif
+
 /* target supports implicit self modifying code */
 #define TARGET_HAS_SMC
 /* support for self modifying code even if the modified instruction is
diff --git a/target-i386/helper.h b/target-i386/helper.h
index 8eb0145..058302b 100644
--- a/target-i386/helper.h
+++ b/target-i386/helper.h
@@ -217,3 +217,5 @@ DEF_HELPER_3(rcrl, tl, env, tl, tl)
 DEF_HELPER_3(rclq, tl, env, tl, tl)
 DEF_HELPER_3(rcrq, tl, env, tl, tl)
 #endif
+
+DEF_HELPER_1(replay_instruction, i32, env)
diff --git a/target-i386/replay_helper.c b/target-i386/replay_helper.c
new file mode 100755
index 0000000..7e70c78
--- /dev/null
+++ b/target-i386/replay_helper.c
@@ -0,0 +1,33 @@
+/*
+ * replay_helper.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "cpu.h"
+#include "exec/helper-proto.h"
+#include "replay/replay.h"
+
+uint32_t helper_replay_instruction(CPUX86State *env)
+{
+    CPUState *cpu = ENV_GET_CPU(env);
+    if (replay_mode == REPLAY_MODE_PLAY
+        && !replay_has_instruction()) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    if (cpu->exit_request) {
+        cpu->exception_index = EXCP_REPLAY;
+        return 1;
+    }
+
+    int timer = replay_has_async_request();
+    replay_instruction(timer);
+    return timer;
+}
diff --git a/target-i386/translate.c b/target-i386/translate.c
index 6fcd824..1843b46 100644
--- a/target-i386/translate.c
+++ b/target-i386/translate.c
@@ -28,6 +28,7 @@
 #include "disas/disas.h"
 #include "tcg-op.h"
 #include "exec/cpu_ldst.h"
+#include "replay/replay.h"
 
 #include "exec/helper-proto.h"
 #include "exec/helper-gen.h"
@@ -112,6 +113,7 @@ typedef struct DisasContext {
     int tf;     /* TF cpu flag */
     int singlestep_enabled; /* "hardware" single step enabled */
     int jmp_opt; /* use direct block chaining for direct jumps */
+    int repz_opt; /* optimize jumps within repz instructions */
     int mem_index; /* select memory access functions */
     uint64_t flags; /* all execution flags */
     struct TranslationBlock *tb;
@@ -1212,8 +1214,9 @@ static inline void gen_repz_ ## op(DisasContext *s, TCGMemOp ot,              \
     gen_op_add_reg_im(s->aflag, R_ECX, -1);                                   \
     /* a loop would cause two single step exceptions if ECX = 1               \
        before rep string_insn */                                              \
-    if (!s->jmp_opt)                                                          \
+    if (!s->repz_opt) {                                                       \
         gen_op_jz_ecx(s->aflag, l2);                                          \
+    }                                                                         \
     gen_jmp(s, cur_eip);                                                      \
 }
 
@@ -1230,8 +1233,9 @@ static inline void gen_repz_ ## op(DisasContext *s, TCGMemOp ot,              \
     gen_op_add_reg_im(s->aflag, R_ECX, -1);                                   \
     gen_update_cc_op(s);                                                      \
     gen_jcc1(s, (JCC_Z << 1) | (nz ^ 1), l2);                                 \
-    if (!s->jmp_opt)                                                          \
+    if (!s->repz_opt) {                                                       \
         gen_op_jz_ecx(s->aflag, l2);                                          \
+    }                                                                         \
     gen_jmp(s, cur_eip);                                                      \
 }
 
@@ -7887,6 +7891,32 @@ void optimize_flags_init(void)
     }
 }
 
+static void gen_instr_replay(DisasContext *s, target_ulong pc_ptr)
+{
+    int l1 = gen_new_label();
+
+    gen_helper_replay_instruction(cpu_tmp2_i32, cpu_env);
+    tcg_gen_brcondi_i32(TCG_COND_EQ, cpu_tmp2_i32, 0, l1);
+
+    /* Don't reset dirty flag */
+    if (s->cc_op_dirty) {
+        tcg_gen_movi_i32(cpu_cc_op, s->cc_op);
+    }
+    gen_jmp_im(pc_ptr - s->cs_base);
+    tcg_gen_exit_tb(0);
+
+    gen_set_label(l1);
+}
+
+static void gen_instructions_count(void)
+{
+    tcg_gen_ld_i32(cpu_tmp2_i32, cpu_env,
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+    tcg_gen_addi_i32(cpu_tmp2_i32, cpu_tmp2_i32, 1);
+    tcg_gen_st_i32(cpu_tmp2_i32, cpu_env,
+                   offsetof(CPUState, instructions_count) - ENV_OFFSET);
+}
+
 /* generate intermediate code in gen_opc_buf and gen_opparam_buf for
    basic block 'tb'. If search_pc is TRUE, also generate PC
    information for each intermediate instruction. */
@@ -7948,6 +7978,19 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
                     || (flags & HF_SOFTMMU_MASK)
 #endif
                     );
+    dc->repz_opt = dc->jmp_opt
+                    /* Do not optimize repz jumps at all in replay mode, because
+                       rep movsS instructions are execured with different paths
+                       in repz_opt and !repz_opt modes. The first one was used
+                       always except single step mode. And this setting
+                       disables jumps optimization and control paths become
+                       equivalent in run and single step modes.
+                       Now there will be no jump optimization for repz in
+                       trace and replay modes and there will always be an
+                       additional step for ecx=0.
+                     */
+                    || replay_mode != REPLAY_MODE_NONE
+                    ;
 #if 0
     /* check addseg logic */
     if (!dc->addseg && (dc->vm86 || !dc->pe || !dc->code32))
@@ -8000,8 +8043,22 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
             tcg_ctx.gen_opc_instr_start[lj] = 1;
             tcg_ctx.gen_opc_icount[lj] = num_insns;
         }
-        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO))
+        if (num_insns + 1 == max_insns && (tb->cflags & CF_LAST_IO)
+            && replay_mode == REPLAY_MODE_NONE) {
             gen_io_start();
+        }
+        /* generate instruction counter code for replay */
+        if (replay_mode != REPLAY_MODE_NONE) {
+            /* In PLAY mode check timer event at every instruction,
+               not only at the beginning of the block. This is needed,
+               when replaying has changed the bounds of translation blocks.
+             */
+            if (pc_ptr == pc_start || replay_mode == REPLAY_MODE_PLAY) {
+                gen_instr_replay(dc, pc_ptr);
+            } else {
+                gen_instructions_count();
+            }
+        }
 
         pc_ptr = disas_insn(env, dc, pc_ptr);
         num_insns++;
@@ -8019,6 +8076,20 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
             gen_eob(dc);
             break;
         }
+        /* In replay mode do not cross the boundary of the pages,
+           it can cause an exception. Do it only when boundary is
+           crossed by the first instruction in the block.
+           If current instruction already crossed the bound - it's ok,
+           because an exception hasn't stopped this code.
+         */
+        if (replay_mode != REPLAY_MODE_NONE
+            && ((pc_ptr & TARGET_PAGE_MASK)
+                != ((pc_ptr + TARGET_MAX_INSN_SIZE - 1) & TARGET_PAGE_MASK)
+                || (pc_ptr & ~TARGET_PAGE_MASK) == 0)) {
+            gen_jmp_im(pc_ptr - dc->cs_base);
+            gen_eob(dc);
+            break;
+        }
         /* if too long translation, stop generation too */
         if (tcg_ctx.gen_opc_ptr >= gen_opc_end ||
             (pc_ptr - pc_start) >= (TARGET_PAGE_SIZE - 32) ||
@@ -8033,8 +8104,10 @@ static inline void gen_intermediate_code_internal(X86CPU *cpu,
             break;
         }
     }
-    if (tb->cflags & CF_LAST_IO)
+    if ((tb->cflags & CF_LAST_IO)
+        && replay_mode == REPLAY_MODE_NONE) {
         gen_io_end();
+    }
     gen_tb_end(tb, num_insns);
     *tcg_ctx.gen_opc_ptr = INDEX_op_end;
     /* we don't forget to fill the last values */

[Qemu-devel] [RFC PATCH v3 25/49] replay: interrupts and exceptions

[Pavel Dovgalyuk]

This patch includes modifications of common cpu files. All interrupts and
exceptions occured during recording are written into the replay log.
These events allow correct replaying the execution by kicking cpu thread
when one of these events is found in the log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpu-exec.c               |   14 +++++++++++---
 replay/replay-internal.h |    4 ++++
 replay/replay.c          |   42 ++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   14 ++++++++++++++
 4 files changed, 71 insertions(+), 3 deletions(-)

diff --git a/cpu-exec.c b/cpu-exec.c
index 13c0ec6..fcde7ce 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -23,6 +23,7 @@
 #include "qemu/atomic.h"
 #include "sysemu/qtest.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 void cpu_loop_exit(CPUState *cpu)
 {
@@ -314,8 +315,14 @@ int cpu_exec(CPUArchState *env)
                     ret = cpu->exception_index;
                     break;
 #else
-                    cc->do_interrupt(cpu);
-                    cpu->exception_index = -1;
+                    if (replay_exception()) {
+                        cc->do_interrupt(cpu);
+                        cpu->exception_index = -1;
+                    } else if (!replay_has_interrupt()) {
+                        /* give a chance to iothread in replay mode */
+                        ret = EXCP_REPLAY;
+                        break;
+                    }
 #endif
                 }
             }
@@ -323,7 +330,8 @@ int cpu_exec(CPUArchState *env)
             next_tb = 0; /* force lookup of first TB */
             for(;;) {
                 interrupt_request = cpu->interrupt_request;
-                if (unlikely(interrupt_request)) {
+                if (unlikely((interrupt_request & CPU_INTERRUPT_DEBUG)
+                             || (interrupt_request && replay_interrupt()))) {
                     if (unlikely(cpu->singlestep_enabled & SSTEP_NOIRQ)) {
                         /* Mask out external interrupts for this step. */
                         interrupt_request &= ~CPU_INTERRUPT_SSTEP_MASK;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 6e7d96f..4c8c785 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -15,6 +15,10 @@
 #include <stdio.h>
 #include "sysemu/sysemu.h"
 
+/* for software interrupt */
+#define EVENT_INTERRUPT             15
+/* for emulated exceptions */
+#define EVENT_EXCEPTION             23
 /* for async events */
 #define EVENT_ASYNC                 24
 /* for instruction event */
diff --git a/replay/replay.c b/replay/replay.c
index 10c67f7..6988994 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -158,3 +158,45 @@ uint64_t replay_get_current_step(void)
     }
     return replay_state.current_step;
 }
+
+bool replay_exception(void)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_instructions();
+        replay_put_event(EVENT_EXCEPTION);
+        return true;
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        if (skip_async_events(EVENT_EXCEPTION)) {
+            replay_has_unread_data = 0;
+            return true;
+        }
+        return false;
+    }
+
+    return true;
+}
+
+bool replay_interrupt(void)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_instructions();
+        replay_put_event(EVENT_INTERRUPT);
+        return true;
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        if (skip_async_events(EVENT_INTERRUPT)) {
+            replay_has_unread_data = 0;
+            return true;
+        }
+        return false;
+    }
+
+    return true;
+}
+
+bool replay_has_interrupt(void)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        return skip_async_events(EVENT_INTERRUPT);
+    }
+    return false;
+}
diff --git a/replay/replay.h b/replay/replay.h
index b764df7..37edf54 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -35,4 +35,18 @@ bool replay_has_async_request(void);
 /*! Returns non-zero if next event is instruction. */
 bool replay_has_instruction(void);
 
+/* Interrupts and exceptions */
+
+/*! Called by exception handler to write or read
+    exception processing events. */
+bool replay_exception(void);
+/*! Called by interrupt handlers to write or read
+    interrupt processing events.
+    \return true if interrupt should be processed */
+bool replay_interrupt(void);
+/*! Tries to read interrupt event from the file.
+    Returns true, when interrupt request is pending */
+bool replay_has_interrupt(void);
+
+
 #endif

[Qemu-devel] [RFC PATCH v3 26/49] vga: do not use virtual clock for blinking cursor

[Pavel Dovgalyuk]

This patch changes virtual clock to realtime one in vga_draw_text and
vga_update_display functions. Display update process depends on realtime
clock. If we read virtual clock there, virtual machine becomes
non-deterministic, because every read to virtual clock is written to the
replay log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/display/vga.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/display/vga.c b/hw/display/vga.c
index 4b089a3..0439586 100644
--- a/hw/display/vga.c
+++ b/hw/display/vga.c
@@ -1304,7 +1304,7 @@ static void vga_draw_text(VGACommonState *s, int full_update)
     uint32_t *ch_attr_ptr;
     vga_draw_glyph8_func *vga_draw_glyph8;
     vga_draw_glyph9_func *vga_draw_glyph9;
-    int64_t now = qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL);
+    int64_t now = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
 
     /* compute font data address (in plane 2) */
     v = s->sr[VGA_SEQ_CHARACTER_MAP];
@@ -1385,7 +1385,7 @@ static void vga_draw_text(VGACommonState *s, int full_update)
         s->cursor_end = s->cr[VGA_CRTC_CURSOR_END];
     }
     cursor_ptr = s->vram_ptr + (s->start_addr + cursor_offset) * 4;
-    if (now >= s->cursor_blink_time) {
+    if (now >= s->cursor_blink_time && runstate_is_running()) {
         s->cursor_blink_time = now + VGA_TEXT_CURSOR_PERIOD_MS / 2;
         s->cursor_visible_phase = !s->cursor_visible_phase;
     }
@@ -1906,7 +1906,7 @@ static void vga_update_display(void *opaque)
         }
         if (graphic_mode != s->graphic_mode) {
             s->graphic_mode = graphic_mode;
-            s->cursor_blink_time = qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL);
+            s->cursor_blink_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
             full_update = 1;
         }
         switch(graphic_mode) {

[Qemu-devel] [RFC PATCH v3 27/49] replay: asynchronous events infrastructure

[Pavel Dovgalyuk]

This patch adds module for saving and replaying asynchronous events.
These events include network packets, keyboard and mouse input,
USB packets, thread pool and bottom halves callbacks.
All events are stored in the queue to be processed at synchronization points
such as beginning of TB execution, or checkpoint in the iothread.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 replay/replay-events.c   |  200 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h |   17 ++++
 replay/replay.h          |    4 +
 3 files changed, 220 insertions(+), 1 deletions(-)

diff --git a/replay/replay-events.c b/replay/replay-events.c
index ffa2ccf..a6a0831 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -12,15 +12,213 @@
 #include "replay.h"
 #include "replay-internal.h"
 
+typedef struct Event {
+    int event_kind;
+    void *opaque;
+    void *opaque2;
+    uint64_t id;
+
+    QTAILQ_ENTRY(Event) events;
+} Event;
+
+static QTAILQ_HEAD(, Event) events_list = QTAILQ_HEAD_INITIALIZER(events_list);
+
+static QemuMutex lock;
+static unsigned int read_event_kind = -1;
+static uint64_t read_id = -1;
+static int read_opt = -1;
+
+static bool replay_events_enabled = false;
+
+/* Functions */
+
+static void replay_run_event(Event *event)
+{
+    switch (event->event_kind) {
+    default:
+        fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n",
+                event->event_kind);
+        exit(1);
+        break;
+    }
+}
+
+void replay_enable_events(void)
+{
+    replay_events_enabled = true;
+}
+
 bool replay_has_events(void)
 {
-    return false;
+    return !QTAILQ_EMPTY(&events_list);
+}
+
+void replay_flush_events(void)
+{
+    qemu_mutex_lock(&lock);
+    while (!QTAILQ_EMPTY(&events_list)) {
+        Event *event = QTAILQ_FIRST(&events_list);
+        replay_run_event(event);
+        QTAILQ_REMOVE(&events_list, event, events);
+        g_free(event);
+    }
+    qemu_mutex_unlock(&lock);
+}
+
+void replay_disable_events(void)
+{
+    replay_events_enabled = false;
+    /* Flush events queue before waiting of completion */
+    replay_flush_events();
+}
+
+void replay_clear_events(void)
+{
+    qemu_mutex_lock(&lock);
+    while (!QTAILQ_EMPTY(&events_list)) {
+        Event *event = QTAILQ_FIRST(&events_list);
+        QTAILQ_REMOVE(&events_list, event, events);
+
+        g_free(event);
+    }
+    qemu_mutex_unlock(&lock);
+}
+
+static void replay_add_event_internal(int event_kind, void *opaque,
+                                      void *opaque2, uint64_t id)
+{
+    if (event_kind >= REPLAY_ASYNC_COUNT) {
+        fprintf(stderr, "Replay: invalid async event ID (%d)\n", event_kind);
+        exit(1);
+    }
+    if (!replay_file || replay_mode == REPLAY_MODE_NONE
+        || !replay_events_enabled) {
+        Event e;
+        e.event_kind = event_kind;
+        e.opaque = opaque;
+        e.opaque2 = opaque2;
+        e.id = id;
+        replay_run_event(&e);
+        return;
+    }
+
+    Event *event = g_malloc0(sizeof(Event));
+    event->event_kind = event_kind;
+    event->opaque = opaque;
+    event->opaque2 = opaque2;
+    event->id = id;
+
+    qemu_mutex_lock(&lock);
+    QTAILQ_INSERT_TAIL(&events_list, event, events);
+    qemu_mutex_unlock(&lock);
+}
+
+void replay_add_event(int event_kind, void *opaque)
+{
+    replay_add_event_internal(event_kind, opaque, NULL, 0);
 }
 
 void replay_save_events(int opt)
 {
+    qemu_mutex_lock(&lock);
+    while (!QTAILQ_EMPTY(&events_list)) {
+        Event *event = QTAILQ_FIRST(&events_list);
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            /* put the event into the file */
+            if (opt == -1) {
+                replay_put_event(EVENT_ASYNC);
+            } else {
+                replay_put_event(EVENT_ASYNC_OPT);
+                replay_put_byte(opt);
+            }
+            replay_put_byte(event->event_kind);
+
+            /* save event-specific data */
+            switch (event->event_kind) {
+            }
+        }
+
+        replay_run_event(event);
+        QTAILQ_REMOVE(&events_list, event, events);
+        g_free(event);
+    }
+    qemu_mutex_unlock(&lock);
 }
 
 void replay_read_events(int opt)
 {
+    replay_fetch_data_kind();
+    while ((opt == -1 && replay_data_kind == EVENT_ASYNC)
+        || (opt != -1 && replay_data_kind == EVENT_ASYNC_OPT)) {
+
+        if (read_event_kind == -1) {
+            if (opt != -1) {
+                read_opt = replay_get_byte();
+            }
+            read_event_kind = replay_get_byte();
+            read_id = -1;
+            replay_check_error();
+        }
+
+        if (opt != read_opt) {
+            break;
+        }
+        /* Execute some events without searching them in the queue */
+        switch (read_event_kind) {
+        default:
+            fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
+            exit(1);
+            break;
+        }
+
+        qemu_mutex_lock(&lock);
+
+        Event *event = NULL;
+        Event *curr = NULL;
+        QTAILQ_FOREACH(curr, &events_list, events) {
+            if (curr->event_kind == read_event_kind
+                && (read_id == -1 || read_id == curr->id)) {
+                event = curr;
+                break;
+            }
+        }
+
+        if (event) {
+            /* read event-specific reading data */
+
+            QTAILQ_REMOVE(&events_list, event, events);
+
+            qemu_mutex_unlock(&lock);
+
+            /* reset unread data and other parameters to allow
+               reading other data from the log while
+               running the event */
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_id = -1;
+            read_opt = -1;
+
+            replay_run_event(event);
+            g_free(event);
+
+            replay_fetch_data_kind();
+        } else {
+            qemu_mutex_unlock(&lock);
+            /* No such event found in the queue */
+            break;
+        }
+    }
+}
+
+void replay_init_events(void)
+{
+    read_event_kind = -1;
+    qemu_mutex_init(&lock);
+}
+
+void replay_finish_events(void)
+{
+    replay_events_enabled = false;
+    replay_clear_events();
+    qemu_mutex_destroy(&lock);
 }
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 4c8c785..3042d42 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -21,9 +21,14 @@
 #define EVENT_EXCEPTION             23
 /* for async events */
 #define EVENT_ASYNC                 24
+#define EVENT_ASYNC_OPT             25
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
 
+/* Asynchronous events IDs */
+
+#define REPLAY_ASYNC_COUNT             0
+
 typedef struct ReplayState {
     /*! Nonzero, when next instruction is repeated one and was already
         processed. */
@@ -75,11 +80,23 @@ void skip_async_events_until(unsigned int kind);
 
 /* Asynchronous events queue */
 
+/*! Initializes events' processing internals */
+void replay_init_events(void);
+/*! Clears internal data structures for events handling */
+void replay_finish_events(void);
+/*! Enables storing events in the queue */
+void replay_enable_events(void);
+/*! Flushes events queue */
+void replay_flush_events(void);
+/*! Clears events list before loading new VM state */
+void replay_clear_events(void);
 /*! Returns true if there are any unsaved events in the queue */
 bool replay_has_events(void);
 /*! Saves events from queue into the file */
 void replay_save_events(int opt);
 /*! Read events from the file into the input queue */
 void replay_read_events(int opt);
+/*! Adds specified async event to the queue */
+void replay_add_event(int event_id, void *opaque);
 
 #endif
diff --git a/replay/replay.h b/replay/replay.h
index 37edf54..b14b88e 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -48,5 +48,9 @@ bool replay_interrupt(void);
     Returns true, when interrupt request is pending */
 bool replay_has_interrupt(void);
 
+/* Asynchronous events queue */
+
+/*! Disables storing events in the queue */
+void replay_disable_events(void);
 
 #endif

[Qemu-devel] [RFC PATCH v3 28/49] replay: recording and replaying clock ticks

[Pavel Dovgalyuk]

Clock ticks are considered as the sources of non-deterministic data for
virtual machine. This patch implements saving the clock values when they
are acquired (virtual, host clock, rdtsc, and some other timers).
When replaying the execution corresponding values are read from log and
transfered to the module, which wants to read the values.
Such a design required the clock polling to be synchronized. Sometimes
it is not true - e.g. when timeouts for timer lists are checked. In this case
we use a cached value of the clock, passing it to the client code.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpus.c                   |    9 ++++++
 include/qemu/timer.h     |   42 +++++++++++++++++++++++++-
 qemu-timer.c             |    3 +-
 replay/Makefile.objs     |    1 +
 replay/replay-internal.h |   11 +++++++
 replay/replay-time.c     |   75 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   17 ++++++++++
 stubs/replay.c           |    9 ++++++
 8 files changed, 164 insertions(+), 3 deletions(-)
 create mode 100755 replay/replay-time.c

diff --git a/cpus.c b/cpus.c
index eb2a795..905cd4f 100644
--- a/cpus.c
+++ b/cpus.c
@@ -40,6 +40,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/seqlock.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 #ifndef _WIN32
 #include "qemu/compatfd.h"
@@ -206,11 +207,19 @@ int64_t cpu_get_clock(void)
     int64_t ti;
     unsigned start;
 
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        return replay_read_clock(REPLAY_CLOCK_VIRTUAL);
+    }
+
     do {
         start = seqlock_read_begin(&timers_state.vm_clock_seqlock);
         ti = cpu_get_clock_locked();
     } while (seqlock_read_retry(&timers_state.vm_clock_seqlock, start));
 
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_clock(REPLAY_CLOCK_VIRTUAL, ti);
+    }
+
     return ti;
 }
 
diff --git a/include/qemu/timer.h b/include/qemu/timer.h
index 7f9a074..16b36c5 100644
--- a/include/qemu/timer.h
+++ b/include/qemu/timer.h
@@ -4,6 +4,7 @@
 #include "qemu/typedefs.h"
 #include "qemu-common.h"
 #include "qemu/notify.h"
+#include "replay/replay.h"
 
 /* timers */
 
@@ -699,8 +700,8 @@ static inline int64_t get_ticks_per_sec(void)
  * Low level clock functions
  */
 
-/* real time host monotonic timer */
-static inline int64_t get_clock_realtime(void)
+/* real time host monotonic timer implementation */
+static inline int64_t get_clock_realtime_impl(void)
 {
     struct timeval tv;
 
@@ -708,6 +709,23 @@ static inline int64_t get_clock_realtime(void)
     return tv.tv_sec * 1000000000LL + (tv.tv_usec * 1000);
 }
 
+/* real time host monotonic timer interface */
+static inline int64_t get_clock_realtime(void)
+{
+    int64_t result;
+
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        result = get_clock_realtime_impl();
+        replay_save_clock(REPLAY_CLOCK_REALTIME, result);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        result = replay_read_clock(REPLAY_CLOCK_REALTIME);
+    } else {
+        result = get_clock_realtime_impl();
+    }
+
+    return result;
+}
+
 /* Warning: don't insert tracepoints into these functions, they are
    also used by simpletrace backend and tracepoints would cause
    an infinite recursion! */
@@ -749,6 +767,8 @@ int64_t cpu_get_clock(void);
 /*******************************************/
 /* host CPU ticks (if available) */
 
+#define cpu_get_real_ticks cpu_get_real_ticks_impl
+
 #if defined(_ARCH_PPC)
 
 static inline int64_t cpu_get_real_ticks(void)
@@ -902,6 +922,24 @@ static inline int64_t cpu_get_real_ticks (void)
 }
 #endif
 
+#undef cpu_get_real_ticks
+
+static inline int64_t cpu_get_real_ticks(void)
+{
+    int64_t val;
+
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        val = cpu_get_real_ticks_impl();
+        replay_save_clock(REPLAY_CLOCK_REAL_TICKS, val);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        val = replay_read_clock(REPLAY_CLOCK_REAL_TICKS);
+    } else {
+        val = cpu_get_real_ticks_impl();
+    }
+
+    return val;
+}
+
 #ifdef CONFIG_PROFILER
 static inline int64_t profile_getclock(void)
 {
diff --git a/qemu-timer.c b/qemu-timer.c
index 00a5d35..8307913 100644
--- a/qemu-timer.c
+++ b/qemu-timer.c
@@ -25,6 +25,7 @@
 #include "sysemu/sysemu.h"
 #include "monitor/monitor.h"
 #include "ui/console.h"
+#include "replay/replay.h"
 
 #include "hw/hw.h"
 
@@ -562,7 +563,7 @@ int64_t qemu_clock_get_ns(QEMUClockType type)
         now = get_clock_realtime();
         last = clock->last;
         clock->last = now;
-        if (now < last) {
+        if (now < last && replay_mode == REPLAY_MODE_NONE) {
             notifier_list_notify(&clock->reset_notifiers, &now);
         }
         return now;
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 56da09c..257c320 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -1,3 +1,4 @@
 obj-y += replay.o
 obj-y += replay-internal.o
 obj-y += replay-events.o
+obj-y += replay-time.o
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 3042d42..14cb9d5 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -24,12 +24,17 @@
 #define EVENT_ASYNC_OPT             25
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
+/* for clock read/writes */
+#define EVENT_CLOCK                 64
+/* some of grteater codes are reserved for clocks */
 
 /* Asynchronous events IDs */
 
 #define REPLAY_ASYNC_COUNT             0
 
 typedef struct ReplayState {
+    /*! Cached clock values. */
+    int64_t cached_clock[REPLAY_CLOCK_COUNT];
     /*! Nonzero, when next instruction is repeated one and was already
         processed. */
     int skipping_instruction;
@@ -78,6 +83,12 @@ bool skip_async_events(int stop_event);
     reports an error and stops the execution. */
 void skip_async_events_until(unsigned int kind);
 
+/*! Reads next clock value from the file.
+    If clock kind read from the file is different from the parameter,
+    the value is not used.
+    If the parameter is -1, the clock value is read to the cache anyway. */
+void replay_read_next_clock(unsigned int kind);
+
 /* Asynchronous events queue */
 
 /*! Initializes events' processing internals */
diff --git a/replay/replay-time.c b/replay/replay-time.c
new file mode 100755
index 0000000..07c1599
--- /dev/null
+++ b/replay/replay-time.c
@@ -0,0 +1,75 @@
+/*
+ * replay-time.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "replay.h"
+#include "replay-internal.h"
+
+
+void replay_save_clock(unsigned int kind, int64_t clock)
+{
+    replay_save_instructions();
+
+    if (kind >= REPLAY_CLOCK_COUNT) {
+        fprintf(stderr, "invalid clock ID %d for replay\n", kind);
+        exit(1);
+    }
+
+    if (replay_file) {
+        replay_put_event(EVENT_CLOCK + kind);
+        replay_put_qword(clock);
+    }
+}
+
+void replay_read_next_clock(unsigned int kind)
+{
+    replay_fetch_data_kind();
+    if (replay_file) {
+        unsigned int read_kind = replay_data_kind - EVENT_CLOCK;
+
+        if (kind != -1 && read_kind != kind) {
+            return;
+        }
+        if (read_kind >= REPLAY_CLOCK_COUNT) {
+            fprintf(stderr,
+                    "invalid clock ID %d was read from replay\n", read_kind);
+            exit(1);
+        }
+
+        int64_t clock = replay_get_qword();
+
+        replay_check_error();
+        replay_has_unread_data = 0;
+
+        replay_state.cached_clock[read_kind] = clock;
+    }
+}
+
+/*! Reads next clock event from the input. */
+int64_t replay_read_clock(unsigned int kind)
+{
+    if (kind >= REPLAY_CLOCK_COUNT) {
+        fprintf(stderr, "invalid clock ID %d for replay\n", kind);
+        exit(1);
+    }
+
+    if (replay_file) {
+        if (skip_async_events(EVENT_CLOCK + kind)) {
+            replay_read_next_clock(kind);
+        }
+        int64_t ret = replay_state.cached_clock[kind];
+
+        return ret;
+    }
+
+    fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
+    exit(1);
+}
diff --git a/replay/replay.h b/replay/replay.h
index b14b88e..938d395 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -16,6 +16,16 @@
 #include <stdint.h>
 #include "qapi-types.h"
 
+/* replay clock kinds */
+/* rdtsc */
+#define REPLAY_CLOCK_REAL_TICKS 0
+/* host_clock */
+#define REPLAY_CLOCK_REALTIME   1
+/* vm_clock */
+#define REPLAY_CLOCK_VIRTUAL    2
+
+#define REPLAY_CLOCK_COUNT      3
+
 extern ReplayMode replay_mode;
 extern char *replay_image_suffix;
 
@@ -48,6 +58,13 @@ bool replay_interrupt(void);
     Returns true, when interrupt request is pending */
 bool replay_has_interrupt(void);
 
+/* Processing clocks and other time sources */
+
+/*! Save the specified clock */
+void replay_save_clock(unsigned int kind, int64_t clock);
+/*! Read the specified clock from the log or return cached data */
+int64_t replay_read_clock(unsigned int kind);
+
 /* Asynchronous events queue */
 
 /*! Disables storing events in the queue */
diff --git a/stubs/replay.c b/stubs/replay.c
index b146d55..2d581fa 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -6,3 +6,12 @@ ReplaySubmode replay_get_play_submode(void)
 {
     return 0;
 }
+
+void replay_save_clock(unsigned int kind, int64_t clock)
+{
+}
+
+int64_t replay_read_clock(unsigned int kind)
+{
+    return 0;
+}

[Qemu-devel] [RFC PATCH v3 29/49] replay: recording and replaying different timers

[Pavel Dovgalyuk]

This patch introduces functions for recording and replaying realtime sources,
that do not use qemu-clock interface. These include return value of time()
function in time_t and struct tm forms. Patch also adds warning to
get_timedate function to prevent its usage in recording mode, because it may
lead to non-determinism.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/mc146818rtc.c   |    3 +
 hw/timer/pl031.c         |   10 ++++
 include/qemu-common.h    |    1 
 replay/replay-internal.h |    4 ++
 replay/replay-time.c     |  108 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |    8 +++
 vl.c                     |   17 ++++++-
 7 files changed, 146 insertions(+), 5 deletions(-)

diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
index 0b78d88..b901851 100644
--- a/hw/timer/mc146818rtc.c
+++ b/hw/timer/mc146818rtc.c
@@ -28,6 +28,7 @@
 #include "qapi/visitor.h"
 #include "qapi-event.h"
 #include "qmp-commands.h"
+#include "replay/replay.h"
 
 #ifdef TARGET_I386
 #include "hw/i386/apic.h"
@@ -710,7 +711,7 @@ static void rtc_set_date_from_host(ISADevice *dev)
     RTCState *s = MC146818_RTC(dev);
     struct tm tm;
 
-    qemu_get_timedate(&tm, 0);
+    qemu_get_timedate_no_warning(&tm, 0);
 
     s->base_rtc = mktimegm(&tm);
     s->last_update = qemu_clock_get_ns(rtc_clock);
diff --git a/hw/timer/pl031.c b/hw/timer/pl031.c
index f8e5abc..40e1700 100644
--- a/hw/timer/pl031.c
+++ b/hw/timer/pl031.c
@@ -14,6 +14,7 @@
 #include "hw/sysbus.h"
 #include "qemu/timer.h"
 #include "sysemu/sysemu.h"
+#include "replay/replay.h"
 
 //#define DEBUG_PL031
 
@@ -200,7 +201,14 @@ static int pl031_init(SysBusDevice *dev)
     sysbus_init_mmio(dev, &s->iomem);
 
     sysbus_init_irq(dev, &s->irq);
-    qemu_get_timedate(&tm, 0);
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        qemu_get_timedate_no_warning(&tm, 0);
+        replay_save_tm(&tm);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        replay_read_tm(&tm);
+    } else {
+        qemu_get_timedate_no_warning(&tm, 0);
+    }
     s->tick_offset = mktimegm(&tm) -
         qemu_clock_get_ns(rtc_clock) / get_ticks_per_sec();
 
diff --git a/include/qemu-common.h b/include/qemu-common.h
index 6ef8282..5b9d358 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -121,6 +121,7 @@ extern int use_icount;
 int qemu_main(int argc, char **argv, char **envp);
 #endif
 
+void qemu_get_timedate_no_warning(struct tm *tm, int offset);
 void qemu_get_timedate(struct tm *tm, int offset);
 int qemu_timedate_diff(struct tm *tm);
 
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 14cb9d5..b21f76b 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -15,6 +15,10 @@
 #include <stdio.h>
 #include "sysemu/sysemu.h"
 
+/* for time_t event */
+#define EVENT_TIME_T                1
+/* for tm event */
+#define EVENT_TM                    2
 /* for software interrupt */
 #define EVENT_INTERRUPT             15
 /* for emulated exceptions */
diff --git a/replay/replay-time.c b/replay/replay-time.c
index 07c1599..e906481 100755
--- a/replay/replay-time.c
+++ b/replay/replay-time.c
@@ -73,3 +73,111 @@ int64_t replay_read_clock(unsigned int kind)
     fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
     exit(1);
 }
+
+/*! Saves time_t value to the log */
+static void replay_save_time_t(time_t tm)
+{
+    replay_save_instructions();
+
+    if (replay_file) {
+        replay_put_event(EVENT_TIME_T);
+        if (sizeof(tm) == 4) {
+            replay_put_dword(tm);
+        } else if (sizeof(tm) == 8) {
+            replay_put_qword(tm);
+        } else {
+            fprintf(stderr, "invalid time_t sizeof: %u\n",
+                    (unsigned)sizeof(tm));
+            exit(1);
+        }
+    }
+}
+
+/*! Reads time_t value from the log. Stops execution in case of error */
+static time_t replay_read_time_t(void)
+{
+    if (replay_file) {
+        time_t tm;
+
+        skip_async_events_until(EVENT_TIME_T);
+
+        if (sizeof(tm) == 4) {
+            tm = replay_get_dword();
+        } else if (sizeof(tm) == 8) {
+            tm = replay_get_qword();
+        } else {
+            fprintf(stderr, "invalid time_t sizeof: %u\n",
+                    (unsigned)sizeof(tm));
+            exit(1);
+        }
+
+        replay_check_error();
+
+        replay_has_unread_data = 0;
+
+        return tm;
+    }
+
+    fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
+    exit(1);
+}
+
+void replay_save_tm(struct tm *tm)
+{
+    replay_save_instructions();
+
+    if (replay_file) {
+        replay_put_event(EVENT_TM);
+
+        replay_put_dword(tm->tm_sec);
+        replay_put_dword(tm->tm_min);
+        replay_put_dword(tm->tm_hour);
+        replay_put_dword(tm->tm_mday);
+        replay_put_dword(tm->tm_mon);
+        replay_put_dword(tm->tm_year);
+        replay_put_dword(tm->tm_wday);
+        replay_put_dword(tm->tm_yday);
+        replay_put_dword(tm->tm_isdst);
+    }
+}
+
+void replay_read_tm(struct tm *tm)
+{
+    if (replay_file) {
+        skip_async_events_until(EVENT_TM);
+
+        tm->tm_sec = replay_get_dword();
+        tm->tm_min = replay_get_dword();
+        tm->tm_hour = replay_get_dword();
+        tm->tm_mday = replay_get_dword();
+        tm->tm_mon = replay_get_dword();
+        tm->tm_year = replay_get_dword();
+        tm->tm_wday = replay_get_dword();
+        tm->tm_yday = replay_get_dword();
+        tm->tm_isdst = replay_get_dword();
+
+        replay_check_error();
+        replay_has_unread_data = 0;
+
+        return;
+    }
+
+    fprintf(stderr, "REPLAY INTERNAL ERROR %d\n", __LINE__);
+    exit(1);
+}
+
+time_t replay_time(void)
+{
+    time_t systime;
+
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        systime = time(NULL);
+        replay_save_time_t(systime);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        systime = replay_read_time_t();
+    } else {
+        systime = time(NULL);
+    }
+
+    return systime;
+}
diff --git a/replay/replay.h b/replay/replay.h
index 938d395..e19ae08 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -14,6 +14,7 @@
 
 #include <stdbool.h>
 #include <stdint.h>
+#include <time.h>
 #include "qapi-types.h"
 
 /* replay clock kinds */
@@ -64,6 +65,13 @@ bool replay_has_interrupt(void);
 void replay_save_clock(unsigned int kind, int64_t clock);
 /*! Read the specified clock from the log or return cached data */
 int64_t replay_read_clock(unsigned int kind);
+/*! Returns result of time() function execution in normal and record modes.
+    In play mode returns value read from the log. */
+time_t replay_time(void);
+/*! Saves struct tm value to the log */
+void replay_save_tm(struct tm *tm);
+/*! Reads struct tm value from the log. Stops execution in case of error */
+void replay_read_tm(struct tm *tm);
 
 /* Asynchronous events queue */
 
diff --git a/vl.c b/vl.c
index fe451aa..4932fa8 100644
--- a/vl.c
+++ b/vl.c
@@ -118,6 +118,7 @@ int main(int argc, char **argv)
 #include "qapi/opts-visitor.h"
 #include "qom/object_interfaces.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 #define DEFAULT_RAM_SIZE 128
 
@@ -757,7 +758,7 @@ void vm_start(void)
 
 /***********************************************************/
 /* host time/date access */
-void qemu_get_timedate(struct tm *tm, int offset)
+void qemu_get_timedate_no_warning(struct tm *tm, int offset)
 {
     time_t ti;
 
@@ -774,6 +775,16 @@ void qemu_get_timedate(struct tm *tm, int offset)
     }
 }
 
+/* host time/date access with replay warning */
+void qemu_get_timedate(struct tm *tm, int offset)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        fprintf(stderr, "REPLAY WARNING! qemu_get_timedate "
+                        "function may lead to non-determinism\n");
+    }
+    qemu_get_timedate_no_warning(tm, offset);
+}
+
 int qemu_timedate_diff(struct tm *tm)
 {
     time_t seconds;
@@ -789,7 +800,7 @@ int qemu_timedate_diff(struct tm *tm)
     else
         seconds = mktimegm(tm) + rtc_date_offset;
 
-    return seconds - time(NULL);
+    return seconds - replay_time();
 }
 
 static void configure_rtc_date_offset(const char *startdate, int legacy)
@@ -827,7 +838,7 @@ static void configure_rtc_date_offset(const char *startdate, int legacy)
                             "'2006-06-17T16:01:21' or '2006-06-17'\n");
             exit(1);
         }
-        rtc_date_offset = time(NULL) - rtc_start_date;
+        rtc_date_offset = replay_time() - rtc_start_date;
     }
 }
 

[Qemu-devel] [RFC PATCH v3 30/49] replay: shutdown event

[Pavel Dovgalyuk]

This patch records and replays simulator shutdown event.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/sysemu/sysemu.h  |    1 +
 replay/replay-internal.h |    2 ++
 replay/replay.c          |   11 +++++++++++
 replay/replay.h          |    5 +++++
 vl.c                     |    8 +++++++-
 5 files changed, 26 insertions(+), 1 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 1ebfef9..9000feb 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -57,6 +57,7 @@ void qemu_register_suspend_notifier(Notifier *notifier);
 void qemu_system_wakeup_request(WakeupReason reason);
 void qemu_system_wakeup_enable(WakeupReason reason, bool enabled);
 void qemu_register_wakeup_notifier(Notifier *notifier);
+void qemu_system_shutdown_request_impl(void);
 void qemu_system_shutdown_request(void);
 void qemu_system_powerdown_request(void);
 void qemu_register_powerdown_notifier(Notifier *notifier);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index b21f76b..30b14ca 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -21,6 +21,8 @@
 #define EVENT_TM                    2
 /* for software interrupt */
 #define EVENT_INTERRUPT             15
+/* for shutdown request */
+#define EVENT_SHUTDOWN              20
 /* for emulated exceptions */
 #define EVENT_EXCEPTION             23
 /* for async events */
diff --git a/replay/replay.c b/replay/replay.c
index 6988994..b6b1f83 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -43,6 +43,10 @@ bool skip_async_events(int stop_event)
             res = true;
         }
         switch (replay_data_kind) {
+        case EVENT_SHUTDOWN:
+            replay_has_unread_data = 0;
+            qemu_system_shutdown_request_impl();
+            break;
         case EVENT_INSTRUCTION:
             first_cpu->instructions_count = replay_get_dword();
             return res;
@@ -200,3 +204,10 @@ bool replay_has_interrupt(void)
     }
     return false;
 }
+
+void replay_shutdown_request(void)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_put_event(EVENT_SHUTDOWN);
+    }
+}
diff --git a/replay/replay.h b/replay/replay.h
index e19ae08..532177b 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -73,6 +73,11 @@ void replay_save_tm(struct tm *tm);
 /*! Reads struct tm value from the log. Stops execution in case of error */
 void replay_read_tm(struct tm *tm);
 
+/* Events */
+
+/*! Called when qemu shutdown is requested. */
+void replay_shutdown_request(void);
+
 /* Asynchronous events queue */
 
 /*! Disables storing events in the queue */
diff --git a/vl.c b/vl.c
index 4932fa8..4b8d59f 100644
--- a/vl.c
+++ b/vl.c
@@ -1930,13 +1930,19 @@ void qemu_system_killed(int signal, pid_t pid)
     qemu_system_shutdown_request();
 }
 
-void qemu_system_shutdown_request(void)
+void qemu_system_shutdown_request_impl(void)
 {
     trace_qemu_system_shutdown_request();
     shutdown_requested = 1;
     qemu_notify_event();
 }
 
+void qemu_system_shutdown_request(void)
+{
+    replay_shutdown_request();
+    qemu_system_shutdown_request_impl();
+}
+
 static void qemu_system_powerdown(void)
 {
     qapi_event_send_powerdown(&error_abort);

[Qemu-devel] [RFC PATCH v3 31/49] replay: checkpoints

[Pavel Dovgalyuk]

This patch introduces checkpoints that synchronize cpu thread and iothread.
When checkpoint is met in the code all asynchronous events from the queue
are executed.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block.c                  |   11 +++++++++++
 cpus.c                   |    2 +-
 include/qemu/timer.h     |    6 ++++--
 qemu-timer.c             |   41 +++++++++++++++++++++++++++++++++--------
 replay/replay-internal.h |    3 +++
 replay/replay.c          |   28 ++++++++++++++++++++++++++++
 replay/replay.h          |    6 ++++++
 stubs/replay.c           |   11 +++++++++++
 vl.c                     |    3 ++-
 9 files changed, 99 insertions(+), 12 deletions(-)

diff --git a/block.c b/block.c
index a7633fe..3548be7 100644
--- a/block.c
+++ b/block.c
@@ -1932,6 +1932,11 @@ void bdrv_drain_all(void)
     BlockDriverState *bs;
 
     while (busy) {
+        if (!replay_checkpoint(8)) {
+            /* Do not wait anymore, we stopped at some place in
+               the middle of execution during replay */
+            return;
+        }
         busy = false;
 
         QTAILQ_FOREACH(bs, &bdrv_states, device_list) {
@@ -1948,6 +1953,12 @@ void bdrv_drain_all(void)
             busy |= bs_busy;
         }
     }
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        /* Skip checkpoints from the log */
+        while (replay_checkpoint(8)) {
+            /* Nothing */
+        }
+    }
 }
 
 /* make a BlockDriverState anonymous by removing from bdrv_state and
diff --git a/cpus.c b/cpus.c
index 905cd4f..fce5ebc 100644
--- a/cpus.c
+++ b/cpus.c
@@ -363,7 +363,7 @@ void qtest_clock_warp(int64_t dest)
         qemu_icount_bias += warp;
         seqlock_write_unlock(&timers_state.vm_clock_seqlock);
 
-        qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL);
+        qemu_clock_run_timers(QEMU_CLOCK_VIRTUAL, false);
         clock = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
     }
     qemu_clock_notify(QEMU_CLOCK_VIRTUAL);
diff --git a/include/qemu/timer.h b/include/qemu/timer.h
index 16b36c5..4f5b3f2 100644
--- a/include/qemu/timer.h
+++ b/include/qemu/timer.h
@@ -232,13 +232,14 @@ void qemu_clock_unregister_reset_notifier(QEMUClockType type,
 /**
  * qemu_clock_run_timers:
  * @type: clock on which to operate
+ * @run_all: true, when called from qemu_clock_run_all_timers
  *
  * Run all the timers associated with the default timer list
  * of a clock.
  *
  * Returns: true if any timer ran.
  */
-bool qemu_clock_run_timers(QEMUClockType type);
+bool qemu_clock_run_timers(QEMUClockType type, bool run_all);
 
 /**
  * qemu_clock_run_all_timers:
@@ -329,12 +330,13 @@ QEMUClockType timerlist_get_clock(QEMUTimerList *timer_list);
 /**
  * timerlist_run_timers:
  * @timer_list: the timer list to use
+ * @run_all: true, when called from qemu_clock_run_all_timers
  *
  * Call all expired timers associated with the timer list.
  *
  * Returns: true if any timer expired
  */
-bool timerlist_run_timers(QEMUTimerList *timer_list);
+bool timerlist_run_timers(QEMUTimerList *timer_list, bool run_all);
 
 /**
  * timerlist_notify:
diff --git a/qemu-timer.c b/qemu-timer.c
index 8307913..b06aa4a 100644
--- a/qemu-timer.c
+++ b/qemu-timer.c
@@ -458,7 +458,7 @@ bool timer_expired(QEMUTimer *timer_head, int64_t current_time)
     return timer_expired_ns(timer_head, current_time * timer_head->scale);
 }
 
-bool timerlist_run_timers(QEMUTimerList *timer_list)
+bool timerlist_run_timers(QEMUTimerList *timer_list, bool run_all)
 {
     QEMUTimer *ts;
     int64_t current_time;
@@ -466,6 +466,24 @@ bool timerlist_run_timers(QEMUTimerList *timer_list)
     QEMUTimerCB *cb;
     void *opaque;
 
+    switch (timer_list->clock->type) {
+    case QEMU_CLOCK_REALTIME:
+        break;
+    default:
+    case QEMU_CLOCK_VIRTUAL:
+        if ((replay_mode != REPLAY_MODE_NONE && !runstate_is_running())
+            || !replay_checkpoint(run_all ? 2 : 3)) {
+            return false;
+        }
+        break;
+    case QEMU_CLOCK_HOST:
+        if ((replay_mode != REPLAY_MODE_NONE && !runstate_is_running())
+            || !replay_checkpoint(run_all ? 5 : 6)) {
+            return false;
+        }
+        break;
+    }
+
     qemu_event_reset(&timer_list->timers_done_ev);
     if (!timer_list->clock->enabled) {
         goto out;
@@ -498,9 +516,9 @@ out:
     return progress;
 }
 
-bool qemu_clock_run_timers(QEMUClockType type)
+bool qemu_clock_run_timers(QEMUClockType type, bool run_all)
 {
-    return timerlist_run_timers(main_loop_tlg.tl[type]);
+    return timerlist_run_timers(main_loop_tlg.tl[type], run_all);
 }
 
 void timerlistgroup_init(QEMUTimerListGroup *tlg,
@@ -525,7 +543,7 @@ bool timerlistgroup_run_timers(QEMUTimerListGroup *tlg)
     QEMUClockType type;
     bool progress = false;
     for (type = 0; type < QEMU_CLOCK_MAX; type++) {
-        progress |= timerlist_run_timers(tlg->tl[type]);
+        progress |= timerlist_run_timers(tlg->tl[type], false);
     }
     return progress;
 }
@@ -534,11 +552,18 @@ int64_t timerlistgroup_deadline_ns(QEMUTimerListGroup *tlg)
 {
     int64_t deadline = -1;
     QEMUClockType type;
+    bool play = replay_mode == REPLAY_MODE_PLAY;
     for (type = 0; type < QEMU_CLOCK_MAX; type++) {
         if (qemu_clock_use_for_deadline(tlg->tl[type]->clock->type)) {
-            deadline = qemu_soonest_timeout(deadline,
-                                            timerlist_deadline_ns(
-                                                tlg->tl[type]));
+            if (!play || tlg->tl[type]->clock->type == QEMU_CLOCK_REALTIME) {
+                deadline = qemu_soonest_timeout(deadline,
+                                                timerlist_deadline_ns(
+                                                    tlg->tl[type]));
+            } else {
+                /* Read clock from the replay file and
+                   do not calculate the deadline, based on virtual clock. */
+                qemu_clock_get_ns(tlg->tl[type]->clock->type);
+            }
         }
     }
     return deadline;
@@ -606,7 +631,7 @@ bool qemu_clock_run_all_timers(void)
     QEMUClockType type;
 
     for (type = 0; type < QEMU_CLOCK_MAX; type++) {
-        progress |= qemu_clock_run_timers(type);
+        progress |= qemu_clock_run_timers(type, true);
     }
 
     return progress;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 30b14ca..d43dbb8 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -34,6 +34,9 @@
 #define EVENT_CLOCK                 64
 /* some of grteater codes are reserved for clocks */
 
+/* for checkpoint event */
+#define EVENT_CHECKPOINT            96
+
 /* Asynchronous events IDs */
 
 #define REPLAY_ASYNC_COUNT             0
diff --git a/replay/replay.c b/replay/replay.c
index b6b1f83..6567a97 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -211,3 +211,31 @@ void replay_shutdown_request(void)
         replay_put_event(EVENT_SHUTDOWN);
     }
 }
+
+/* Used checkpoints: 2 3 5 6 7 8 9 */
+int replay_checkpoint(unsigned int checkpoint)
+{
+    replay_save_instructions();
+
+    if (replay_file) {
+        if (replay_mode == REPLAY_MODE_PLAY) {
+            if (!skip_async_events(EVENT_CHECKPOINT + checkpoint)) {
+                if (replay_data_kind == EVENT_ASYNC_OPT) {
+                    replay_read_events(checkpoint);
+                    replay_fetch_data_kind();
+                    return replay_data_kind != EVENT_ASYNC_OPT;
+                }
+                return 0;
+            }
+            replay_has_unread_data = 0;
+            replay_read_events(checkpoint);
+            replay_fetch_data_kind();
+            return replay_data_kind != EVENT_ASYNC_OPT;
+        } else if (replay_mode == REPLAY_MODE_RECORD) {
+            replay_put_event(EVENT_CHECKPOINT + checkpoint);
+            replay_save_events(checkpoint);
+        }
+    }
+
+    return 1;
+}
diff --git a/replay/replay.h b/replay/replay.h
index 532177b..f659f54 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -77,6 +77,12 @@ void replay_read_tm(struct tm *tm);
 
 /*! Called when qemu shutdown is requested. */
 void replay_shutdown_request(void);
+/*! Should be called at check points in the execution.
+    These check points are skipped, if they were not met.
+    Saves checkpoint in the SAVE mode and validates in the PLAY mode.
+    Returns 0 in PLAY mode if checkpoint was not found.
+    Returns 1 in all other cases. */
+int replay_checkpoint(unsigned int checkpoint);
 
 /* Asynchronous events queue */
 
diff --git a/stubs/replay.c b/stubs/replay.c
index 2d581fa..8f0b3b5 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -1,4 +1,5 @@
 #include "replay/replay.h"
+#include "sysemu/sysemu.h"
 
 ReplayMode replay_mode;
 
@@ -15,3 +16,13 @@ int64_t replay_read_clock(unsigned int kind)
 {
     return 0;
 }
+
+int replay_checkpoint(unsigned int checkpoint)
+{
+    return 0;
+}
+
+int runstate_is_running(void)
+{
+    return 0;
+}
diff --git a/vl.c b/vl.c
index 4b8d59f..5768b34 100644
--- a/vl.c
+++ b/vl.c
@@ -1985,7 +1985,8 @@ static bool main_loop_should_exit(void)
             return true;
         }
     }
-    if (qemu_reset_requested()) {
+    if (qemu_reset_requested_get() && replay_checkpoint(7)) {
+        qemu_reset_requested();
         pause_all_vcpus();
         cpu_synchronize_all_states();
         qemu_system_reset(VMRESET_REPORT);

[Qemu-devel] [RFC PATCH v3 32/49] vmclock: add virtual clock based on replay icount

[Pavel Dovgalyuk]

This patch introduces virtual clock which values are calculated using
number of executed instructions. Instruction counter is taken from replay
module.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpus.c                 |   22 ++++++--
 qemu-timer.c           |    4 +
 replay/Makefile.objs   |    1 
 replay/replay-icount.c |  130 ++++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h        |   10 ++++
 stubs/replay.c         |    6 ++
 6 files changed, 167 insertions(+), 6 deletions(-)
 create mode 100755 replay/replay-icount.c

diff --git a/cpus.c b/cpus.c
index fce5ebc..d5fa5d1 100644
--- a/cpus.c
+++ b/cpus.c
@@ -172,6 +172,9 @@ int64_t cpu_get_ticks(void)
     if (use_icount) {
         return cpu_get_icount();
     }
+    if (replay_icount) {
+        return replay_get_icount();
+    }
 
     ticks = timers_state.cpu_ticks_offset;
     if (timers_state.cpu_ticks_enabled) {
@@ -231,8 +234,10 @@ void cpu_enable_ticks(void)
     /* Here, the really thing protected by seqlock is cpu_clock_offset. */
     seqlock_write_lock(&timers_state.vm_clock_seqlock);
     if (!timers_state.cpu_ticks_enabled) {
-        timers_state.cpu_ticks_offset -= cpu_get_real_ticks();
-        timers_state.cpu_clock_offset -= get_clock();
+        if (!replay_icount) {
+            timers_state.cpu_ticks_offset -= cpu_get_real_ticks();
+            timers_state.cpu_clock_offset -= get_clock();
+        }
         timers_state.cpu_ticks_enabled = 1;
     }
     seqlock_write_unlock(&timers_state.vm_clock_seqlock);
@@ -247,8 +252,10 @@ void cpu_disable_ticks(void)
     /* Here, the really thing protected by seqlock is cpu_clock_offset. */
     seqlock_write_lock(&timers_state.vm_clock_seqlock);
     if (timers_state.cpu_ticks_enabled) {
-        timers_state.cpu_ticks_offset += cpu_get_real_ticks();
-        timers_state.cpu_clock_offset = cpu_get_clock_locked();
+        if (!replay_icount) {
+            timers_state.cpu_ticks_offset += cpu_get_real_ticks();
+            timers_state.cpu_clock_offset = cpu_get_clock_locked();
+        }
         timers_state.cpu_ticks_enabled = 0;
     }
     seqlock_write_unlock(&timers_state.vm_clock_seqlock);
@@ -379,7 +386,12 @@ void qemu_clock_warp(QEMUClockType type)
      * applicable to other clocks.  But a clock argument removes the
      * need for if statements all over the place.
      */
-    if (type != QEMU_CLOCK_VIRTUAL || !use_icount) {
+    if (type != QEMU_CLOCK_VIRTUAL || (!use_icount || !replay_icount)) {
+        return;
+    }
+
+    if (replay_icount) {
+        replay_clock_warp();
         return;
     }
 
diff --git a/qemu-timer.c b/qemu-timer.c
index b06aa4a..f8bf060 100644
--- a/qemu-timer.c
+++ b/qemu-timer.c
@@ -579,7 +579,9 @@ int64_t qemu_clock_get_ns(QEMUClockType type)
         return get_clock();
     default:
     case QEMU_CLOCK_VIRTUAL:
-        if (use_icount) {
+        if (replay_icount) {
+            return replay_get_icount();
+        } else if (use_icount) {
             return cpu_get_icount();
         } else {
             return cpu_get_clock();
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 257c320..7dec93f 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -2,3 +2,4 @@ obj-y += replay.o
 obj-y += replay-internal.o
 obj-y += replay-events.o
 obj-y += replay-time.o
+obj-y += replay-icount.o
diff --git a/replay/replay-icount.c b/replay/replay-icount.c
new file mode 100755
index 0000000..3c9dad1
--- /dev/null
+++ b/replay/replay-icount.c
@@ -0,0 +1,130 @@
+/*
+ * replay-icount.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "qemu-common.h"
+#include "sysemu/cpus.h"
+#include "sysemu/sysemu.h"
+#include "qemu/timer.h"
+#include "migration/vmstate.h"
+#include "replay.h"
+#include "replay-internal.h"
+
+int replay_icount;
+
+typedef struct {
+    /* Compensate for varying guest execution speed.  */
+    int64_t bias;
+    /* Timer for advancing VM clock, when all CPUs are sleeping */
+    QEMUTimer *icount_warp_timer;
+    int64_t vm_clock_warp_start;
+} ReplayIcount;
+static ReplayIcount icount_data;
+
+
+/* Return the virtual CPU time, based on the instruction counter.  */
+int64_t replay_get_icount(void)
+{
+    int64_t icount = replay_get_current_step();
+    return icount_data.bias + (icount << replay_icount);
+}
+
+static void replay_icount_warp_rt(void *opaque)
+{
+    if (icount_data.vm_clock_warp_start == -1) {
+        return;
+    }
+
+    if (runstate_is_running()) {
+        int64_t clock = qemu_clock_get_ns(QEMU_CLOCK_HOST);
+        int64_t warp_delta = clock - icount_data.vm_clock_warp_start;
+        icount_data.bias += warp_delta;
+        if (qemu_clock_expired(QEMU_CLOCK_VIRTUAL)) {
+            qemu_notify_event();
+        }
+    }
+    icount_data.vm_clock_warp_start = -1;
+}
+
+void replay_clock_warp(void)
+{
+    int64_t deadline;
+    if (!replay_checkpoint(9)) {
+        return;
+    }
+    /*
+     * If the CPUs have been sleeping, advance the vm_clock timer now.  This
+     * ensures that the deadline for the timer is computed correctly below.
+     * This also makes sure that the insn counter is synchronized before the
+     * CPU starts running, in case the CPU is woken by an event other than
+     * the earliest vm_clock timer.
+     */
+    if (icount_data.vm_clock_warp_start != -1) {
+        replay_icount_warp_rt(NULL);
+    }
+    if (!all_cpu_threads_idle() || !qemu_clock_has_timers(QEMU_CLOCK_VIRTUAL)) {
+        timer_del(icount_data.icount_warp_timer);
+        return;
+    }
+
+    icount_data.vm_clock_warp_start = qemu_clock_get_ns(QEMU_CLOCK_HOST);
+    deadline = qemu_clock_deadline_ns_all(QEMU_CLOCK_VIRTUAL);
+    if (deadline > 0) {
+        /*
+         * Ensure the vm_clock proceeds even when the virtual CPU goes to
+         * sleep.  Otherwise, the CPU might be waiting for a future timer
+         * interrupt to wake it up, but the interrupt never comes because
+         * the vCPU isn't running any insns and thus doesn't advance the
+         * vm_clock.
+         *
+         * An extreme solution for this problem would be to never let VCPUs
+         * sleep in icount mode if there is a pending vm_clock timer; rather
+         * time could just advance to the next vm_clock event.  Instead, we
+         * do stop VCPUs and only advance vm_clock after some "real" time,
+         * (related to the time left until the next event) has passed.  This
+         * rt_clock timer will do this.  This avoids that the warps are too
+         * visible externally---for example, you will not be sending network
+         * packets continuously instead of every 100ms.
+         */
+        timer_mod_ns(icount_data.icount_warp_timer,
+                     icount_data.vm_clock_warp_start + deadline);
+    } else {
+        qemu_notify_event();
+    }
+}
+
+static const VMStateDescription vmstate_icount = {
+    .name = "icount",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_INT64(bias, ReplayIcount),
+        VMSTATE_TIMER(icount_warp_timer, ReplayIcount),
+        VMSTATE_INT64(vm_clock_warp_start, ReplayIcount),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+void replay_init_icount(void)
+{
+    if (!replay_icount) {
+        return;
+    }
+
+    vmstate_register(NULL, 0, &vmstate_icount, &icount_data);
+    icount_data.icount_warp_timer = timer_new_ns(QEMU_CLOCK_HOST,
+                                                 replay_icount_warp_rt, NULL);
+}
diff --git a/replay/replay.h b/replay/replay.h
index f659f54..ae76f23 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -29,6 +29,8 @@
 
 extern ReplayMode replay_mode;
 extern char *replay_image_suffix;
+/*! Shift value for icount based on replay or zero, if it is disabled. */
+extern int replay_icount;
 
 /*! Returns replay play submode */
 ReplaySubmode replay_get_play_submode(void);
@@ -89,4 +91,12 @@ int replay_checkpoint(unsigned int checkpoint);
 /*! Disables storing events in the queue */
 void replay_disable_events(void);
 
+/* icount-based virtual clock */
+
+/* Initializes icount-based virtual clock */
+void replay_init_icount(void);
+/* Returns the virtual CPU time, based on the instruction counter.  */
+int64_t replay_get_icount(void);
+void replay_clock_warp(void);
+
 #endif
diff --git a/stubs/replay.c b/stubs/replay.c
index 8f0b3b5..e0d8d12 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -2,6 +2,7 @@
 #include "sysemu/sysemu.h"
 
 ReplayMode replay_mode;
+int replay_icount;
 
 ReplaySubmode replay_get_play_submode(void)
 {
@@ -26,3 +27,8 @@ int runstate_is_running(void)
 {
     return 0;
 }
+
+int64_t replay_get_icount(void)
+{
+    return 0;
+}

[Qemu-devel] [RFC PATCH v3 33/49] replay: bottom halves

[Pavel Dovgalyuk]

This patch introduces bottom half event for replay queue. It saves the events
into the queue and process them at the checkpoints and instructions execution.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 async.c                  |   46 ++++++++++++++++++++++++++++++++++++++++------
 dma-helpers.c            |    4 +++-
 hw/ide/ahci.c            |    4 +++-
 hw/ide/core.c            |    4 +++-
 hw/timer/arm_timer.c     |    2 +-
 hw/usb/hcd-uhci.c        |    2 +-
 include/block/aio.h      |   18 ++++++++++++++++++
 include/qemu/main-loop.h |    1 +
 main-loop.c              |    5 +++++
 replay/replay-events.c   |   16 ++++++++++++++++
 replay/replay-internal.h |    3 ++-
 replay/replay.h          |    2 ++
 stubs/replay.c           |    4 ++++
 13 files changed, 99 insertions(+), 12 deletions(-)

diff --git a/async.c b/async.c
index 34af0b2..8bcf5ad 100644
--- a/async.c
+++ b/async.c
@@ -27,6 +27,7 @@
 #include "block/thread-pool.h"
 #include "qemu/main-loop.h"
 #include "qemu/atomic.h"
+#include "replay/replay.h"
 
 /***********************************************************/
 /* bottom halves (can be seen as timers which expire ASAP) */
@@ -39,24 +40,53 @@ struct QEMUBH {
     bool scheduled;
     bool idle;
     bool deleted;
+    bool replay;
+    uint64_t id;
 };
 
 QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
 {
-    QEMUBH *bh;
+    QEMUBH *bh, **last;
     bh = g_malloc0(sizeof(QEMUBH));
     bh->ctx = ctx;
     bh->cb = cb;
     bh->opaque = opaque;
     qemu_mutex_lock(&ctx->bh_lock);
-    bh->next = ctx->first_bh;
-    /* Make sure that the members are ready before putting bh into list */
-    smp_wmb();
-    ctx->first_bh = bh;
+    if (replay_mode != REPLAY_MODE_NONE) {
+        /* Slower way, but this is a queue and not a stack.
+           Replay will process the BH in the same order they
+           came into the queue. */
+        last = &ctx->first_bh;
+        while (*last) {
+            last = &(*last)->next;
+        }
+        smp_wmb();
+        *last = bh;
+    } else {
+        bh->next = ctx->first_bh;
+        /* Make sure that the members are ready before putting bh into list */
+        smp_wmb();
+        ctx->first_bh = bh;
+    }
     qemu_mutex_unlock(&ctx->bh_lock);
     return bh;
 }
 
+QEMUBH *aio_bh_new_replay(AioContext *ctx, QEMUBHFunc *cb, void *opaque,
+                          uint64_t id)
+{
+    QEMUBH *bh = aio_bh_new(ctx, cb, opaque);
+    bh->replay = true;
+    bh->id = id;
+    return bh;
+}
+
+void aio_bh_call(void *opaque)
+{
+    QEMUBH *bh = (QEMUBH *)opaque;
+    bh->cb(bh->opaque);
+}
+
 /* Multiple occurrences of aio_bh_poll cannot be called concurrently */
 int aio_bh_poll(AioContext *ctx)
 {
@@ -79,7 +109,11 @@ int aio_bh_poll(AioContext *ctx)
             if (!bh->idle)
                 ret = 1;
             bh->idle = 0;
-            bh->cb(bh->opaque);
+            if (!bh->replay) {
+                aio_bh_call(bh);
+            } else {
+                replay_add_bh_event(bh, bh->id);
+            }
         }
     }
 
diff --git a/dma-helpers.c b/dma-helpers.c
index 499b52b..8faf53d 100644
--- a/dma-helpers.c
+++ b/dma-helpers.c
@@ -12,6 +12,7 @@
 #include "qemu/range.h"
 #include "qemu/thread.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 /* #define DEBUG_IOMMU */
 
@@ -96,7 +97,8 @@ static void continue_after_map_failure(void *opaque)
 {
     DMAAIOCB *dbs = (DMAAIOCB *)opaque;
 
-    dbs->bh = qemu_bh_new(reschedule_dma, dbs);
+    dbs->bh = qemu_bh_new_replay(reschedule_dma, dbs,
+                                 replay_get_current_step());
     qemu_bh_schedule(dbs->bh);
 }
 
diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c
index 604152a..0762743 100644
--- a/hw/ide/ahci.c
+++ b/hw/ide/ahci.c
@@ -32,6 +32,7 @@
 #include "internal.h"
 #include <hw/ide/pci.h>
 #include <hw/ide/ahci.h>
+#include "replay/replay.h"
 
 /* #define DEBUG_AHCI */
 
@@ -1132,7 +1133,8 @@ static int ahci_async_cmd_done(IDEDMA *dma)
 
     if (!ad->check_bh) {
         /* maybe we still have something to process, check later */
-        ad->check_bh = qemu_bh_new(ahci_check_cmd_bh, ad);
+        ad->check_bh = qemu_bh_new_replay(ahci_check_cmd_bh, ad,
+                                          replay_get_current_step());
         qemu_bh_schedule(ad->check_bh);
     }
 
diff --git a/hw/ide/core.c b/hw/ide/core.c
index db191a6..ed2c769 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -32,6 +32,7 @@
 #include "sysemu/dma.h"
 #include "hw/block/block.h"
 #include "sysemu/blockdev.h"
+#include "replay/replay.h"
 
 #include <hw/ide/internal.h>
 
@@ -409,7 +410,8 @@ BlockDriverAIOCB *ide_issue_trim(BlockDriverState *bs,
     TrimAIOCB *iocb;
 
     iocb = qemu_aio_get(&trim_aiocb_info, bs, cb, opaque);
-    iocb->bh = qemu_bh_new(ide_trim_bh_cb, iocb);
+    iocb->bh = qemu_bh_new_replay(ide_trim_bh_cb, iocb,
+                                  replay_get_current_step());
     iocb->ret = 0;
     iocb->qiov = qiov;
     iocb->i = -1;
diff --git a/hw/timer/arm_timer.c b/hw/timer/arm_timer.c
index 1452910..97784a0 100644
--- a/hw/timer/arm_timer.c
+++ b/hw/timer/arm_timer.c
@@ -168,7 +168,7 @@ static arm_timer_state *arm_timer_init(uint32_t freq)
     s->freq = freq;
     s->control = TIMER_CTRL_IE;
 
-    bh = qemu_bh_new(arm_timer_tick, s);
+    bh = qemu_bh_new_replay(arm_timer_tick, s, 0);
     s->timer = ptimer_init(bh);
     vmstate_register(NULL, -1, &vmstate_arm_timer, s);
     return s;
diff --git a/hw/usb/hcd-uhci.c b/hw/usb/hcd-uhci.c
index c3bf72c..3aaf506 100644
--- a/hw/usb/hcd-uhci.c
+++ b/hw/usb/hcd-uhci.c
@@ -1224,7 +1224,7 @@ static int usb_uhci_common_initfn(PCIDevice *dev)
                               USB_SPEED_MASK_LOW | USB_SPEED_MASK_FULL);
         }
     }
-    s->bh = qemu_bh_new(uhci_bh, s);
+    s->bh = qemu_bh_new_replay(uhci_bh, s, 0);
     s->frame_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, uhci_frame_timer, s);
     s->num_ports_vmstate = NB_PORTS;
     QTAILQ_INIT(&s->queues);
diff --git a/include/block/aio.h b/include/block/aio.h
index c23de3c..c6cf7fe 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -34,6 +34,8 @@ struct BlockDriverAIOCB {
     const AIOCBInfo *aiocb_info;
     BlockDriverState *bs;
     BlockDriverCompletionFunc *cb;
+    bool replay;
+    uint64_t replay_step;
     void *opaque;
 };
 
@@ -141,6 +143,17 @@ void aio_context_release(AioContext *ctx);
 QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque);
 
 /**
+ * aio_bh_new_replay: Allocate a new bottom half structure for replay.
+ *
+ * This function calls aio_bh_new function and also fills replay parameters
+ * of the BH structure. BH created with this function in record/replay mode
+ * are executed through the replay queue only at checkpoints and instructions
+ * executions.
+ */
+QEMUBH *aio_bh_new_replay(AioContext *ctx, QEMUBHFunc *cb, void *opaque,
+                          uint64_t id);
+
+/**
  * aio_notify: Force processing of pending events.
  *
  * Similar to signaling a condition variable, aio_notify forces
@@ -156,6 +169,11 @@ QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque);
 void aio_notify(AioContext *ctx);
 
 /**
+ * aio_bh_call: Executes callback function of the specified BH.
+ */
+void aio_bh_call(void *opaque);
+
+/**
  * aio_bh_poll: Poll bottom halves for an AioContext.
  *
  * These are internal functions used by the QEMU main loop.
diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index 6f0200a..c221ddd 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -306,6 +306,7 @@ void qemu_iohandler_fill(GArray *pollfds);
 void qemu_iohandler_poll(GArray *pollfds, int rc);
 
 QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque);
+QEMUBH *qemu_bh_new_replay(QEMUBHFunc *cb, void *opaque, uint64_t id);
 void qemu_bh_schedule_idle(QEMUBH *bh);
 
 #endif
diff --git a/main-loop.c b/main-loop.c
index 3cc79f8..7ce4c92 100644
--- a/main-loop.c
+++ b/main-loop.c
@@ -498,3 +498,8 @@ QEMUBH *qemu_bh_new(QEMUBHFunc *cb, void *opaque)
 {
     return aio_bh_new(qemu_aio_context, cb, opaque);
 }
+
+QEMUBH *qemu_bh_new_replay(QEMUBHFunc *cb, void *opaque, uint64_t id)
+{
+    return aio_bh_new_replay(qemu_aio_context, cb, opaque, id);
+}
diff --git a/replay/replay-events.c b/replay/replay-events.c
index a6a0831..58fd63a 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -35,6 +35,9 @@ static bool replay_events_enabled = false;
 static void replay_run_event(Event *event)
 {
     switch (event->event_kind) {
+    case REPLAY_ASYNC_EVENT_BH:
+        aio_bh_call(event->opaque);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n",
                 event->event_kind);
@@ -118,6 +121,11 @@ void replay_add_event(int event_kind, void *opaque)
     replay_add_event_internal(event_kind, opaque, NULL, 0);
 }
 
+void replay_add_bh_event(void *bh, uint64_t id)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_BH, bh, NULL, id);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -135,6 +143,9 @@ void replay_save_events(int opt)
 
             /* save event-specific data */
             switch (event->event_kind) {
+            case REPLAY_ASYNC_EVENT_BH:
+                replay_put_qword(event->id);
+                break;
             }
         }
 
@@ -165,6 +176,11 @@ void replay_read_events(int opt)
         }
         /* Execute some events without searching them in the queue */
         switch (read_event_kind) {
+        case REPLAY_ASYNC_EVENT_BH:
+            if (read_id == -1) {
+                read_id = replay_get_qword();
+            }
+            break;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index d43dbb8..670a3cf 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -39,7 +39,8 @@
 
 /* Asynchronous events IDs */
 
-#define REPLAY_ASYNC_COUNT             0
+#define REPLAY_ASYNC_EVENT_BH          0
+#define REPLAY_ASYNC_COUNT             1
 
 typedef struct ReplayState {
     /*! Cached clock values. */
diff --git a/replay/replay.h b/replay/replay.h
index ae76f23..9c22993 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -90,6 +90,8 @@ int replay_checkpoint(unsigned int checkpoint);
 
 /*! Disables storing events in the queue */
 void replay_disable_events(void);
+/*! Adds BH event to the queue */
+void replay_add_bh_event(void *bh, uint64_t id);
 
 /* icount-based virtual clock */
 
diff --git a/stubs/replay.c b/stubs/replay.c
index e0d8d12..dfea8d8 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -32,3 +32,7 @@ int64_t replay_get_icount(void)
 {
     return 0;
 }
+
+void replay_add_bh_event(void *bh, uint64_t id)
+{
+}

[Qemu-devel] [RFC PATCH v3 34/49] replay: replay aio requests

[Pavel Dovgalyuk]

This patch adds identifier to aio requests. ID is used for creating bottom
halves and identifying them while replaying.
The patch also introduces several functions that make possible replaying
of the aio requests.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block.c               |   80 +++++++++++++++++++++++++++++++++++++++++++------
 block/qcow2.c         |    4 ++
 dma-helpers.c         |    6 ++--
 hw/block/virtio-blk.c |   10 +++---
 hw/ide/atapi.c        |    9 ++++--
 hw/ide/core.c         |   14 ++++++---
 include/block/block.h |   15 +++++++++
 include/qemu-common.h |    2 +
 qemu-io-cmds.c        |    2 +
 stubs/replay.c        |    5 +++
 trace-events          |    2 +
 util/iov.c            |    4 ++
 12 files changed, 127 insertions(+), 26 deletions(-)

diff --git a/block.c b/block.c
index 3548be7..1335259 100644
--- a/block.c
+++ b/block.c
@@ -84,7 +84,8 @@ static BlockDriverAIOCB *bdrv_co_aio_rw_vector(BlockDriverState *bs,
                                                BdrvRequestFlags flags,
                                                BlockDriverCompletionFunc *cb,
                                                void *opaque,
-                                               bool is_write);
+                                               bool is_write,
+                                               bool aio_replay);
 static void coroutine_fn bdrv_co_do_rw(void *opaque);
 static int coroutine_fn bdrv_co_do_write_zeroes(BlockDriverState *bs,
     int64_t sector_num, int nb_sectors, BdrvRequestFlags flags);
@@ -4402,7 +4403,19 @@ BlockDriverAIOCB *bdrv_aio_readv(BlockDriverState *bs, int64_t sector_num,
     trace_bdrv_aio_readv(bs, sector_num, nb_sectors, opaque);
 
     return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
-                                 cb, opaque, false);
+                                 cb, opaque, false, false);
+}
+
+BlockDriverAIOCB *bdrv_aio_readv_replay(BlockDriverState *bs,
+                                        int64_t sector_num,
+                                        QEMUIOVector *qiov, int nb_sectors,
+                                        BlockDriverCompletionFunc *cb,
+                                        void *opaque)
+{
+    trace_bdrv_aio_readv_replay(bs, sector_num, nb_sectors, opaque);
+
+    return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
+                                 cb, opaque, false, true);
 }
 
 BlockDriverAIOCB *bdrv_aio_writev(BlockDriverState *bs, int64_t sector_num,
@@ -4412,7 +4425,19 @@ BlockDriverAIOCB *bdrv_aio_writev(BlockDriverState *bs, int64_t sector_num,
     trace_bdrv_aio_writev(bs, sector_num, nb_sectors, opaque);
 
     return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
-                                 cb, opaque, true);
+                                 cb, opaque, true, false);
+}
+
+BlockDriverAIOCB *bdrv_aio_writev_replay(BlockDriverState *bs,
+                                         int64_t sector_num,
+                                         QEMUIOVector *qiov, int nb_sectors,
+                                         BlockDriverCompletionFunc *cb,
+                                         void *opaque)
+{
+    trace_bdrv_aio_writev_replay(bs, sector_num, nb_sectors, opaque);
+
+    return bdrv_co_aio_rw_vector(bs, sector_num, qiov, nb_sectors, 0,
+                                 cb, opaque, true, true);
 }
 
 BlockDriverAIOCB *bdrv_aio_write_zeroes(BlockDriverState *bs,
@@ -4423,7 +4448,7 @@ BlockDriverAIOCB *bdrv_aio_write_zeroes(BlockDriverState *bs,
 
     return bdrv_co_aio_rw_vector(bs, sector_num, NULL, nb_sectors,
                                  BDRV_REQ_ZERO_WRITE | flags,
-                                 cb, opaque, true);
+                                 cb, opaque, true, true);
 }
 
 
@@ -4559,7 +4584,8 @@ static int multiwrite_merge(BlockDriverState *bs, BlockRequest *reqs,
  * requests. However, the fields opaque and error are left unmodified as they
  * are used to signal failure for a single request to the caller.
  */
-int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs, int num_reqs)
+int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs, int num_reqs,
+                        bool replay)
 {
     MultiwriteCB *mcb;
     int i;
@@ -4597,7 +4623,7 @@ int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs, int num_reqs)
         bdrv_co_aio_rw_vector(bs, reqs[i].sector, reqs[i].qiov,
                               reqs[i].nb_sectors, reqs[i].flags,
                               multiwrite_cb, mcb,
-                              true);
+                              true, replay);
     }
 
     return 0;
@@ -4746,7 +4772,12 @@ static void coroutine_fn bdrv_co_do_rw(void *opaque)
             acb->req.nb_sectors, acb->req.qiov, acb->req.flags);
     }
 
-    acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    if (acb->common.replay) {
+        acb->bh = aio_bh_new_replay(bdrv_get_aio_context(bs), bdrv_co_em_bh,
+                                    acb, acb->common.replay_step);
+    } else {
+        acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    }
     qemu_bh_schedule(acb->bh);
 }
 
@@ -4757,7 +4788,8 @@ static BlockDriverAIOCB *bdrv_co_aio_rw_vector(BlockDriverState *bs,
                                                BdrvRequestFlags flags,
                                                BlockDriverCompletionFunc *cb,
                                                void *opaque,
-                                               bool is_write)
+                                               bool is_write,
+                                               bool aio_replay)
 {
     Coroutine *co;
     BlockDriverAIOCBCoroutine *acb;
@@ -4769,6 +4801,10 @@ static BlockDriverAIOCB *bdrv_co_aio_rw_vector(BlockDriverState *bs,
     acb->req.flags = flags;
     acb->is_write = is_write;
     acb->done = NULL;
+    acb->common.replay = aio_replay;
+    if (aio_replay) {
+        acb->common.replay_step = replay_get_current_step();
+    }
 
     co = qemu_coroutine_create(bdrv_co_do_rw);
     qemu_coroutine_enter(co, acb);
@@ -4782,7 +4818,12 @@ static void coroutine_fn bdrv_aio_flush_co_entry(void *opaque)
     BlockDriverState *bs = acb->common.bs;
 
     acb->req.error = bdrv_co_flush(bs);
-    acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    if (acb->common.replay) {
+        acb->bh = aio_bh_new_replay(bdrv_get_aio_context(bs), bdrv_co_em_bh,
+                                    acb, acb->common.replay_step);
+    } else {
+        acb->bh = aio_bh_new(bdrv_get_aio_context(bs), bdrv_co_em_bh, acb);
+    }
     qemu_bh_schedule(acb->bh);
 }
 
@@ -4803,6 +4844,25 @@ BlockDriverAIOCB *bdrv_aio_flush(BlockDriverState *bs,
     return &acb->common;
 }
 
+BlockDriverAIOCB *bdrv_aio_flush_replay(BlockDriverState *bs,
+        BlockDriverCompletionFunc *cb, void *opaque)
+{
+    trace_bdrv_aio_flush(bs, opaque);
+
+    Coroutine *co;
+    BlockDriverAIOCBCoroutine *acb;
+
+    acb = qemu_aio_get(&bdrv_em_co_aiocb_info, bs, cb, opaque);
+    acb->done = NULL;
+    acb->common.replay = true;
+    acb->common.replay_step = replay_get_current_step();
+
+    co = qemu_coroutine_create(bdrv_aio_flush_co_entry);
+    qemu_coroutine_enter(co, acb);
+
+    return &acb->common;
+}
+
 static void coroutine_fn bdrv_aio_discard_co_entry(void *opaque)
 {
     BlockDriverAIOCBCoroutine *acb = opaque;
@@ -4853,6 +4913,8 @@ void *qemu_aio_get(const AIOCBInfo *aiocb_info, BlockDriverState *bs,
     acb->bs = bs;
     acb->cb = cb;
     acb->opaque = opaque;
+    acb->replay_step = 0;
+    acb->replay = false;
     return acb;
 }
 
diff --git a/block/qcow2.c b/block/qcow2.c
index 43ceb34..c8169bb 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -997,6 +997,8 @@ static coroutine_fn int qcow2_co_readv(BlockDriverState *bs, int64_t sector_num,
     uint8_t *cluster_data = NULL;
 
     qemu_iovec_init(&hd_qiov, qiov->niov);
+    hd_qiov.replay = qiov->replay;
+    hd_qiov.replay_step = qiov->replay_step;
 
     qemu_co_mutex_lock(&s->lock);
 
@@ -1149,6 +1151,8 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState *bs,
                                  remaining_sectors);
 
     qemu_iovec_init(&hd_qiov, qiov->niov);
+    hd_qiov.replay = qiov->replay;
+    hd_qiov.replay_step = qiov->replay_step;
 
     s->cluster_cache_offset = -1; /* disable compressed cache */
 
diff --git a/dma-helpers.c b/dma-helpers.c
index 8faf53d..a020dc5 100644
--- a/dma-helpers.c
+++ b/dma-helpers.c
@@ -223,6 +223,8 @@ BlockDriverAIOCB *dma_bdrv_io(
     dbs->io_func = io_func;
     dbs->bh = NULL;
     qemu_iovec_init(&dbs->iov, sg->nsg);
+    dbs->iov.replay = true;
+    dbs->iov.replay_step = replay_get_current_step();
     dma_bdrv_cb(dbs, 0);
     return &dbs->common;
 }
@@ -232,7 +234,7 @@ BlockDriverAIOCB *dma_bdrv_read(BlockDriverState *bs,
                                 QEMUSGList *sg, uint64_t sector,
                                 void (*cb)(void *opaque, int ret), void *opaque)
 {
-    return dma_bdrv_io(bs, sg, sector, bdrv_aio_readv, cb, opaque,
+    return dma_bdrv_io(bs, sg, sector, bdrv_aio_readv_replay, cb, opaque,
                        DMA_DIRECTION_FROM_DEVICE);
 }
 
@@ -240,7 +242,7 @@ BlockDriverAIOCB *dma_bdrv_write(BlockDriverState *bs,
                                  QEMUSGList *sg, uint64_t sector,
                                  void (*cb)(void *opaque, int ret), void *opaque)
 {
-    return dma_bdrv_io(bs, sg, sector, bdrv_aio_writev, cb, opaque,
+    return dma_bdrv_io(bs, sg, sector, bdrv_aio_writev_replay, cb, opaque,
                        DMA_DIRECTION_TO_DEVICE);
 }
 
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index c241c50..5d238af 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -265,7 +265,7 @@ void virtio_submit_multiwrite(BlockDriverState *bs, MultiReqBuffer *mrb)
         return;
     }
 
-    ret = bdrv_aio_multiwrite(bs, mrb->blkreq, mrb->num_writes);
+    ret = bdrv_aio_multiwrite(bs, mrb->blkreq, mrb->num_writes, true);
     if (ret != 0) {
         for (i = 0; i < mrb->num_writes; i++) {
             if (mrb->blkreq[i].error) {
@@ -285,7 +285,7 @@ static void virtio_blk_handle_flush(VirtIOBlockReq *req, MultiReqBuffer *mrb)
      * Make sure all outstanding writes are posted to the backing device.
      */
     virtio_submit_multiwrite(req->dev->bs, mrb);
-    bdrv_aio_flush(req->dev->bs, virtio_blk_flush_complete, req);
+    bdrv_aio_flush_replay(req->dev->bs, virtio_blk_flush_complete, req);
 }
 
 static bool virtio_blk_sect_range_ok(VirtIOBlock *dev,
@@ -354,9 +354,9 @@ static void virtio_blk_handle_read(VirtIOBlockReq *req)
     }
 
     bdrv_acct_start(req->dev->bs, &req->acct, req->qiov.size, BDRV_ACCT_READ);
-    bdrv_aio_readv(req->dev->bs, sector, &req->qiov,
-                   req->qiov.size / BDRV_SECTOR_SIZE,
-                   virtio_blk_rw_complete, req);
+    bdrv_aio_readv_replay(req->dev->bs, sector, &req->qiov,
+                          req->qiov.size / BDRV_SECTOR_SIZE,
+                          virtio_blk_rw_complete, req);
 }
 
 void virtio_blk_handle_request(VirtIOBlockReq *req, MultiReqBuffer *mrb)
diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index f7d2009..e94cd93 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -25,6 +25,7 @@
 
 #include "hw/ide/internal.h"
 #include "hw/scsi/scsi.h"
+#include "replay/replay.h"
 
 static void ide_atapi_cmd_read_dma_cb(void *opaque, int ret);
 
@@ -348,10 +349,12 @@ static void ide_atapi_cmd_read_dma_cb(void *opaque, int ret)
     s->bus->dma->iov.iov_base = (void *)(s->io_buffer + data_offset);
     s->bus->dma->iov.iov_len = n * 4 * 512;
     qemu_iovec_init_external(&s->bus->dma->qiov, &s->bus->dma->iov, 1);
+    s->bus->dma->qiov.replay = true;
+    s->bus->dma->qiov.replay_step = replay_get_current_step();
 
-    s->bus->dma->aiocb = bdrv_aio_readv(s->bs, (int64_t)s->lba << 2,
-                                       &s->bus->dma->qiov, n * 4,
-                                       ide_atapi_cmd_read_dma_cb, s);
+    s->bus->dma->aiocb = bdrv_aio_readv_replay(s->bs, (int64_t)s->lba << 2,
+                                               &s->bus->dma->qiov, n * 4,
+                                               ide_atapi_cmd_read_dma_cb, s);
     return;
 
 eot:
diff --git a/hw/ide/core.c b/hw/ide/core.c
index ed2c769..8d77969 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -576,10 +576,12 @@ void ide_sector_read(IDEState *s)
     s->iov.iov_base = s->io_buffer;
     s->iov.iov_len  = n * BDRV_SECTOR_SIZE;
     qemu_iovec_init_external(&s->qiov, &s->iov, 1);
+    s->qiov.replay = true;
+    s->qiov.replay_step = replay_get_current_step();
 
     bdrv_acct_start(s->bs, &s->acct, n * BDRV_SECTOR_SIZE, BDRV_ACCT_READ);
-    s->pio_aiocb = bdrv_aio_readv(s->bs, sector_num, &s->qiov, n,
-                                  ide_sector_read_cb, s);
+    s->pio_aiocb = bdrv_aio_readv_replay(s->bs, sector_num, &s->qiov, n,
+                                         ide_sector_read_cb, s);
 }
 
 static void dma_buf_commit(IDEState *s)
@@ -823,10 +825,12 @@ void ide_sector_write(IDEState *s)
     s->iov.iov_base = s->io_buffer;
     s->iov.iov_len  = n * BDRV_SECTOR_SIZE;
     qemu_iovec_init_external(&s->qiov, &s->iov, 1);
+    s->qiov.replay = true;
+    s->qiov.replay_step = replay_get_current_step();
 
     bdrv_acct_start(s->bs, &s->acct, n * BDRV_SECTOR_SIZE, BDRV_ACCT_READ);
-    s->pio_aiocb = bdrv_aio_writev(s->bs, sector_num, &s->qiov, n,
-                                   ide_sector_write_cb, s);
+    s->pio_aiocb = bdrv_aio_writev_replay(s->bs, sector_num, &s->qiov, n,
+                                          ide_sector_write_cb, s);
 }
 
 static void ide_flush_cb(void *opaque, int ret)
@@ -855,7 +859,7 @@ void ide_flush_cache(IDEState *s)
 
     s->status |= BUSY_STAT;
     bdrv_acct_start(s->bs, &s->acct, 0, BDRV_ACCT_FLUSH);
-    bdrv_aio_flush(s->bs, ide_flush_cb, s);
+    bdrv_aio_flush_replay(s->bs, ide_flush_cb, s);
 }
 
 static void ide_cfata_metadata_inquiry(IDEState *s)
diff --git a/include/block/block.h b/include/block/block.h
index 95e6d1c..93b8e22 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -326,11 +326,24 @@ typedef void BlockDriverDirtyHandler(BlockDriverState *bs, int64_t sector,
 BlockDriverAIOCB *bdrv_aio_readv(BlockDriverState *bs, int64_t sector_num,
                                  QEMUIOVector *iov, int nb_sectors,
                                  BlockDriverCompletionFunc *cb, void *opaque);
+BlockDriverAIOCB *bdrv_aio_readv_replay(BlockDriverState *bs,
+                                        int64_t sector_num,
+                                        QEMUIOVector *iov, int nb_sectors,
+                                        BlockDriverCompletionFunc *cb,
+                                        void *opaque);
 BlockDriverAIOCB *bdrv_aio_writev(BlockDriverState *bs, int64_t sector_num,
                                   QEMUIOVector *iov, int nb_sectors,
                                   BlockDriverCompletionFunc *cb, void *opaque);
+BlockDriverAIOCB *bdrv_aio_writev_replay(BlockDriverState *bs,
+                                         int64_t sector_num,
+                                         QEMUIOVector *iov, int nb_sectors,
+                                         BlockDriverCompletionFunc *cb,
+                                         void *opaque);
 BlockDriverAIOCB *bdrv_aio_flush(BlockDriverState *bs,
                                  BlockDriverCompletionFunc *cb, void *opaque);
+BlockDriverAIOCB *bdrv_aio_flush_replay(BlockDriverState *bs,
+                                        BlockDriverCompletionFunc *cb,
+                                        void *opaque);
 BlockDriverAIOCB *bdrv_aio_discard(BlockDriverState *bs,
                                    int64_t sector_num, int nb_sectors,
                                    BlockDriverCompletionFunc *cb, void *opaque);
@@ -350,7 +363,7 @@ typedef struct BlockRequest {
 } BlockRequest;
 
 int bdrv_aio_multiwrite(BlockDriverState *bs, BlockRequest *reqs,
-    int num_reqs);
+                        int num_reqs, bool replay);
 
 /* sg packet commands */
 int bdrv_ioctl(BlockDriverState *bs, unsigned long int req, void *buf);
diff --git a/include/qemu-common.h b/include/qemu-common.h
index 5b9d358..126e138 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -309,6 +309,8 @@ typedef struct QEMUIOVector {
     int niov;
     int nalloc;
     size_t size;
+    bool replay;
+    uint64_t replay_step;
 } QEMUIOVector;
 
 void qemu_iovec_init(QEMUIOVector *qiov, int alloc_hint);
diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index c503fc6..fd54eed 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -592,7 +592,7 @@ static int do_aio_multiwrite(BlockDriverState *bs, BlockRequest* reqs,
         *total += reqs[i].qiov->size;
     }
 
-    ret = bdrv_aio_multiwrite(bs, reqs, num_reqs);
+    ret = bdrv_aio_multiwrite(bs, reqs, num_reqs, false);
     if (ret < 0) {
         return ret;
     }
diff --git a/stubs/replay.c b/stubs/replay.c
index dfea8d8..e6fb20a 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -36,3 +36,8 @@ int64_t replay_get_icount(void)
 void replay_add_bh_event(void *bh, uint64_t id)
 {
 }
+
+uint64_t replay_get_current_step(void)
+{
+    return 0;
+}
diff --git a/trace-events b/trace-events
index 11a17a8..8382348 100644
--- a/trace-events
+++ b/trace-events
@@ -59,7 +59,9 @@ bdrv_aio_multiwrite(void *mcb, int num_callbacks, int num_reqs) "mcb %p num_call
 bdrv_aio_discard(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
 bdrv_aio_flush(void *bs, void *opaque) "bs %p opaque %p"
 bdrv_aio_readv(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
+bdrv_aio_readv_replay(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
 bdrv_aio_writev(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
+bdrv_aio_writev_replay(void *bs, int64_t sector_num, int nb_sectors, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d opaque %p"
 bdrv_aio_write_zeroes(void *bs, int64_t sector_num, int nb_sectors, int flags, void *opaque) "bs %p sector_num %"PRId64" nb_sectors %d flags %#x opaque %p"
 bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d"
 bdrv_co_readv(void *bs, int64_t sector_num, int nb_sector) "bs %p sector_num %"PRId64" nb_sectors %d"
diff --git a/util/iov.c b/util/iov.c
index 24566c8..b03f83f 100644
--- a/util/iov.c
+++ b/util/iov.c
@@ -257,6 +257,8 @@ void qemu_iovec_init(QEMUIOVector *qiov, int alloc_hint)
     qiov->niov = 0;
     qiov->nalloc = alloc_hint;
     qiov->size = 0;
+    qiov->replay = false;
+    qiov->replay_step = 0;
 }
 
 void qemu_iovec_init_external(QEMUIOVector *qiov, struct iovec *iov, int niov)
@@ -267,6 +269,8 @@ void qemu_iovec_init_external(QEMUIOVector *qiov, struct iovec *iov, int niov)
     qiov->niov = niov;
     qiov->nalloc = -1;
     qiov->size = 0;
+    qiov->replay = false;
+    qiov->replay_step = 0;
     for (i = 0; i < niov; i++)
         qiov->size += iov[i].iov_len;
 }

[Qemu-devel] [RFC PATCH v3 35/49] replay: thread pool

[Pavel Dovgalyuk]

This patch modifies thread pool to allow replaying asynchronous thread tasks
synchronously in replay mode.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 block/raw-posix.c           |    6 +++--
 block/raw-win32.c           |    4 ++-
 include/block/thread-pool.h |    4 ++-
 replay/replay-events.c      |   11 +++++++++
 replay/replay-internal.h    |    3 ++
 replay/replay.h             |    2 ++
 stubs/replay.c              |    4 +++
 tests/test-thread-pool.c    |    7 +++---
 thread-pool.c               |   53 +++++++++++++++++++++++++++++--------------
 9 files changed, 69 insertions(+), 25 deletions(-)

diff --git a/block/raw-posix.c b/block/raw-posix.c
index 8e9758e..32d63d9 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -1049,7 +1049,9 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs, int fd,
 
     trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
     pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque,
+                                  qiov ? qiov->replay : false,
+                                  qiov ? qiov->replay_step : 0);
 }
 
 static BlockDriverAIOCB *raw_aio_submit(BlockDriverState *bs,
@@ -1877,7 +1879,7 @@ static BlockDriverAIOCB *hdev_aio_ioctl(BlockDriverState *bs,
     acb->aio_ioctl_buf = buf;
     acb->aio_ioctl_cmd = req;
     pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque, false, 0);
 }
 
 #elif defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
diff --git a/block/raw-win32.c b/block/raw-win32.c
index 902eab6..212307c 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -158,7 +158,9 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs, HANDLE hfile,
 
     trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
     pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+    return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque,
+                                  qiov ? qiov->replay : false,
+                                  qiov ? qiov->replay_step : 0);
 }
 
 int qemu_ftruncate64(int fd, int64_t length)
diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
index 32afcdd..df74f8d 100644
--- a/include/block/thread-pool.h
+++ b/include/block/thread-pool.h
@@ -33,9 +33,11 @@ void thread_pool_free(ThreadPool *pool);
 
 BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg,
-        BlockDriverCompletionFunc *cb, void *opaque);
+        BlockDriverCompletionFunc *cb, void *opaque,
+        bool replay, uint64_t replay_step);
 int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg);
 void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
+void thread_pool_work(ThreadPool *pool, void *r);
 
 #endif
diff --git a/replay/replay-events.c b/replay/replay-events.c
index 58fd63a..fa24726 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -11,6 +11,7 @@
 
 #include "replay.h"
 #include "replay-internal.h"
+#include "block/thread-pool.h"
 
 typedef struct Event {
     int event_kind;
@@ -38,6 +39,9 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_BH:
         aio_bh_call(event->opaque);
         break;
+    case REPLAY_ASYNC_EVENT_THREAD:
+        thread_pool_work((ThreadPool *)event->opaque, event->opaque2);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n",
                 event->event_kind);
@@ -126,6 +130,11 @@ void replay_add_bh_event(void *bh, uint64_t id)
     replay_add_event_internal(REPLAY_ASYNC_EVENT_BH, bh, NULL, id);
 }
 
+void replay_add_thread_event(void *opaque, void *opaque2, uint64_t id)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_THREAD, opaque, opaque2, id);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -144,6 +153,7 @@ void replay_save_events(int opt)
             /* save event-specific data */
             switch (event->event_kind) {
             case REPLAY_ASYNC_EVENT_BH:
+            case REPLAY_ASYNC_EVENT_THREAD:
                 replay_put_qword(event->id);
                 break;
             }
@@ -177,6 +187,7 @@ void replay_read_events(int opt)
         /* Execute some events without searching them in the queue */
         switch (read_event_kind) {
         case REPLAY_ASYNC_EVENT_BH:
+        case REPLAY_ASYNC_EVENT_THREAD:
             if (read_id == -1) {
                 read_id = replay_get_qword();
             }
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 670a3cf..fbc087f 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -40,7 +40,8 @@
 /* Asynchronous events IDs */
 
 #define REPLAY_ASYNC_EVENT_BH          0
-#define REPLAY_ASYNC_COUNT             1
+#define REPLAY_ASYNC_EVENT_THREAD      1
+#define REPLAY_ASYNC_COUNT             2
 
 typedef struct ReplayState {
     /*! Cached clock values. */
diff --git a/replay/replay.h b/replay/replay.h
index 9c22993..2ad15b2 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -92,6 +92,8 @@ int replay_checkpoint(unsigned int checkpoint);
 void replay_disable_events(void);
 /*! Adds BH event to the queue */
 void replay_add_bh_event(void *bh, uint64_t id);
+/*! Adds thread event to the queue */
+void replay_add_thread_event(void *pool, void *req, uint64_t id);
 
 /* icount-based virtual clock */
 
diff --git a/stubs/replay.c b/stubs/replay.c
index e6fb20a..ad555d8 100755
--- a/stubs/replay.c
+++ b/stubs/replay.c
@@ -41,3 +41,7 @@ uint64_t replay_get_current_step(void)
 {
     return 0;
 }
+
+void replay_add_thread_event(void *opaque, void *opaque2, uint64_t id)
+{
+}
diff --git a/tests/test-thread-pool.c b/tests/test-thread-pool.c
index f40b7fc..6b1cc18 100644
--- a/tests/test-thread-pool.c
+++ b/tests/test-thread-pool.c
@@ -55,7 +55,7 @@ static void test_submit_aio(void)
 {
     WorkerTestData data = { .n = 0, .ret = -EINPROGRESS };
     data.aiocb = thread_pool_submit_aio(pool, worker_cb, &data,
-                                        done_cb, &data);
+                                        done_cb, &data, false, 0);
 
     /* The callbacks are not called until after the first wait.  */
     active = 1;
@@ -119,7 +119,8 @@ static void test_submit_many(void)
     for (i = 0; i < 100; i++) {
         data[i].n = 0;
         data[i].ret = -EINPROGRESS;
-        thread_pool_submit_aio(pool, worker_cb, &data[i], done_cb, &data[i]);
+        thread_pool_submit_aio(pool, worker_cb, &data[i], done_cb, &data[i],
+                               false, 0);
     }
 
     active = 100;
@@ -148,7 +149,7 @@ static void test_cancel(void)
         data[i].n = 0;
         data[i].ret = -EINPROGRESS;
         data[i].aiocb = thread_pool_submit_aio(pool, long_cb, &data[i],
-                                               done_cb, &data[i]);
+                                               done_cb, &data[i], false, 0);
     }
 
     /* Starting the threads may be left to a bottom half.  Let it
diff --git a/thread-pool.c b/thread-pool.c
index dfb699d..a1d0785 100644
--- a/thread-pool.c
+++ b/thread-pool.c
@@ -24,6 +24,7 @@
 #include "qemu/event_notifier.h"
 #include "block/thread-pool.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 static void do_spawn_thread(ThreadPool *pool);
 
@@ -79,6 +80,30 @@ struct ThreadPool {
     bool stopping;
 };
 
+void thread_pool_work(ThreadPool *pool, void *r)
+{
+    ThreadPoolElement *req = (ThreadPoolElement *)r;
+    int ret;
+    if (replay_mode == REPLAY_MODE_NONE) {
+        qemu_mutex_unlock(&pool->lock);
+    }
+
+    ret = req->func(req->arg);
+    req->ret = ret;
+    /* Write ret before state.  */
+    smp_wmb();
+    req->state = THREAD_DONE;
+
+    if (replay_mode == REPLAY_MODE_NONE) {
+        qemu_mutex_lock(&pool->lock);
+    }
+    if (pool->pending_cancellations) {
+        qemu_cond_broadcast(&pool->check_cancel);
+    }
+
+    event_notifier_set(&pool->notifier);
+}
+
 static void *worker_thread(void *opaque)
 {
     ThreadPool *pool = opaque;
@@ -105,21 +130,12 @@ static void *worker_thread(void *opaque)
         req = QTAILQ_FIRST(&pool->request_list);
         QTAILQ_REMOVE(&pool->request_list, req, reqs);
         req->state = THREAD_ACTIVE;
-        qemu_mutex_unlock(&pool->lock);
-
-        ret = req->func(req->arg);
 
-        req->ret = ret;
-        /* Write ret before state.  */
-        smp_wmb();
-        req->state = THREAD_DONE;
-
-        qemu_mutex_lock(&pool->lock);
-        if (pool->pending_cancellations) {
-            qemu_cond_broadcast(&pool->check_cancel);
+        if (replay_mode != REPLAY_MODE_NONE && req->common.replay) {
+            replay_add_thread_event(pool, req, req->common.replay_step);
+        } else {
+            thread_pool_work(pool, req);
         }
-
-        event_notifier_set(&pool->notifier);
     }
 
     pool->cur_threads--;
@@ -234,7 +250,8 @@ static const AIOCBInfo thread_pool_aiocb_info = {
 
 BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg,
-        BlockDriverCompletionFunc *cb, void *opaque)
+        BlockDriverCompletionFunc *cb, void *opaque,
+        bool replay, uint64_t replay_step)
 {
     ThreadPoolElement *req;
 
@@ -243,6 +260,8 @@ BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
     req->arg = arg;
     req->state = THREAD_QUEUED;
     req->pool = pool;
+    req->common.replay = replay;
+    req->common.replay_step = replay_step;
 
     QLIST_INSERT_HEAD(&pool->head, req, all);
 
@@ -253,8 +272,8 @@ BlockDriverAIOCB *thread_pool_submit_aio(ThreadPool *pool,
         spawn_thread(pool);
     }
     QTAILQ_INSERT_TAIL(&pool->request_list, req, reqs);
-    qemu_mutex_unlock(&pool->lock);
     qemu_sem_post(&pool->sem);
+    qemu_mutex_unlock(&pool->lock);
     return &req->common;
 }
 
@@ -276,14 +295,14 @@ int coroutine_fn thread_pool_submit_co(ThreadPool *pool, ThreadPoolFunc *func,
 {
     ThreadPoolCo tpc = { .co = qemu_coroutine_self(), .ret = -EINPROGRESS };
     assert(qemu_in_coroutine());
-    thread_pool_submit_aio(pool, func, arg, thread_pool_co_cb, &tpc);
+    thread_pool_submit_aio(pool, func, arg, thread_pool_co_cb, &tpc, false, 0);
     qemu_coroutine_yield();
     return tpc.ret;
 }
 
 void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg)
 {
-    thread_pool_submit_aio(pool, func, arg, NULL, NULL);
+    thread_pool_submit_aio(pool, func, arg, NULL, NULL, false, 0);
 }
 
 static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)

[Qemu-devel] [RFC PATCH v3 36/49] pl031: vmstate in replay mode

[Pavel Dovgalyuk]

This patch modifies vmstate for PL031 RTC. It removes querying of the rtc
and virtual clocks while saving and restoring VM state, because in replay
mode these clocks are stopped while saving. And reading the clock while
restoring the VM state may lead to read of the incorrect values, because
clocks cache in replay module could not be loaded yet.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/timer/pl031.c |   25 ++++++++++++++++++-------
 1 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/hw/timer/pl031.c b/hw/timer/pl031.c
index 40e1700..600fa49 100644
--- a/hw/timer/pl031.c
+++ b/hw/timer/pl031.c
@@ -220,19 +220,30 @@ static void pl031_pre_save(void *opaque)
 {
     PL031State *s = opaque;
 
-    /* tick_offset is base_time - rtc_clock base time.  Instead, we want to
-     * store the base time relative to the QEMU_CLOCK_VIRTUAL for backwards-compatibility.  */
-    int64_t delta = qemu_clock_get_ns(rtc_clock) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-    s->tick_offset_vmstate = s->tick_offset + delta / get_ticks_per_sec();
+    if (replay_mode == REPLAY_MODE_NONE) {
+        /* tick_offset is base_time - rtc_clock base time.  Instead, we want to
+         * store the base time relative to the QEMU_CLOCK_VIRTUAL
+         * for backwards-compatibility. */
+        int64_t delta = qemu_clock_get_ns(rtc_clock)
+                        - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        s->tick_offset_vmstate = s->tick_offset + delta / get_ticks_per_sec();
+    } else {
+        s->tick_offset_vmstate = s->tick_offset;
+    }
 }
 
 static int pl031_post_load(void *opaque, int version_id)
 {
     PL031State *s = opaque;
 
-    int64_t delta = qemu_clock_get_ns(rtc_clock) - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-    s->tick_offset = s->tick_offset_vmstate - delta / get_ticks_per_sec();
-    pl031_set_alarm(s);
+    if (replay_mode == REPLAY_MODE_NONE) {
+        int64_t delta = qemu_clock_get_ns(rtc_clock)
+                        - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+        s->tick_offset = s->tick_offset_vmstate - delta / get_ticks_per_sec();
+        pl031_set_alarm(s);
+    } else {
+        s->tick_offset = s->tick_offset_vmstate;
+    }
     return 0;
 }
 

[Qemu-devel] [RFC PATCH v3 37/49] replay: initialization and deinitialization

[Pavel Dovgalyuk]

This patch introduces the functions for enabling the record/replay and for
freeing the resources when simulator closes.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 exec.c                   |    1 
 replay/replay-internal.h |    2 +
 replay/replay.c          |  128 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |   11 ++++
 vl.c                     |    8 +++
 5 files changed, 150 insertions(+), 0 deletions(-)

diff --git a/exec.c b/exec.c
index 643e564..8403cc5 100644
--- a/exec.c
+++ b/exec.c
@@ -767,6 +767,7 @@ void cpu_abort(CPUState *cpu, const char *fmt, ...)
     }
     va_end(ap2);
     va_end(ap);
+    replay_finish();
 #if defined(CONFIG_USER_ONLY)
     {
         struct sigaction act;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index fbc087f..25b133f 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -36,6 +36,8 @@
 
 /* for checkpoint event */
 #define EVENT_CHECKPOINT            96
+/* end of log event */
+#define EVENT_END                   127
 
 /* Asynchronous events IDs */
 
diff --git a/replay/replay.c b/replay/replay.c
index 6567a97..f3e7c09 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -13,10 +13,18 @@
 #include "replay.h"
 #include "replay-internal.h"
 
+/* Current version of the replay mechanism.
+   Increase it when file format changes. */
+#define REPLAY_VERSION              0xe02001
+/* Size of replay log header */
+#define HEADER_SIZE                 (sizeof(uint32_t) + sizeof(uint64_t))
+
 ReplayMode replay_mode = REPLAY_MODE_NONE;
 /*! Stores current submode for PLAY mode */
 ReplaySubmode play_submode = REPLAY_SUBMODE_UNKNOWN;
 
+/* Name of replay file  */
+static char *replay_filename;
 /* Suffix for the disk images filenames */
 char *replay_image_suffix;
 
@@ -239,3 +247,123 @@ int replay_checkpoint(unsigned int checkpoint)
 
     return 1;
 }
+
+static void replay_enable(const char *fname, int mode)
+{
+    const char *fmode = NULL;
+    if (replay_file) {
+        fprintf(stderr,
+                "Replay: some record/replay operation is already started\n");
+        return;
+    }
+
+    switch (mode) {
+    case REPLAY_MODE_RECORD:
+        fmode = "wb";
+        break;
+    case REPLAY_MODE_PLAY:
+        fmode = "rb";
+        play_submode = REPLAY_SUBMODE_NORMAL;
+        break;
+    default:
+        fprintf(stderr, "Replay: internal error: invalid replay mode\n");
+        exit(1);
+    }
+
+    atexit(replay_finish);
+
+    replay_file = fopen(fname, fmode);
+    if (replay_file == NULL) {
+        fprintf(stderr, "Replay: open %s: %s\n", fname, strerror(errno));
+        exit(1);
+    }
+
+    replay_filename = g_strdup(fname);
+
+    replay_mode = mode;
+    replay_has_unread_data = 0;
+    replay_data_kind = -1;
+    replay_state.skipping_instruction = 0;
+    replay_state.current_step = 0;
+
+    /* skip file header for RECORD and check it for PLAY */
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        fseek(replay_file, HEADER_SIZE, SEEK_SET);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        unsigned int version = replay_get_dword();
+        uint64_t offset = replay_get_qword();
+        if (version != REPLAY_VERSION) {
+            fprintf(stderr, "Replay: invalid input log file version\n");
+            exit(1);
+        }
+        /* go to the beginning */
+        fseek(replay_file, 12, SEEK_SET);
+    }
+
+    replay_init_events();
+}
+
+void replay_configure(QemuOpts *opts, int mode)
+{
+    const char *fname;
+
+    fname = qemu_opt_get(opts, "fname");
+    if (!fname) {
+        fprintf(stderr, "File name not specified for replay\n");
+        exit(1);
+    }
+
+    const char *suffix = qemu_opt_get(opts, "suffix");
+    if (suffix) {
+        replay_image_suffix = g_strdup(suffix);
+    } else {
+        replay_image_suffix = g_strdup("replay_qcow");
+    }
+
+    replay_enable(fname, mode);
+}
+
+void replay_init_timer(void)
+{
+    if (replay_mode == REPLAY_MODE_NONE) {
+        return;
+    }
+
+    replay_enable_events();
+}
+
+void replay_finish(void)
+{
+    if (replay_mode == REPLAY_MODE_NONE) {
+        return;
+    }
+
+    replay_save_instructions();
+
+    /* finalize the file */
+    if (replay_file) {
+        if (replay_mode == REPLAY_MODE_RECORD) {
+            uint64_t offset = 0;
+            /* write end event */
+            replay_put_event(EVENT_END);
+
+            /* write header */
+            fseek(replay_file, 0, SEEK_SET);
+            replay_put_dword(REPLAY_VERSION);
+            replay_put_qword(offset);
+        }
+
+        fclose(replay_file);
+        replay_file = NULL;
+    }
+    if (replay_filename) {
+        g_free(replay_filename);
+        replay_filename = NULL;
+    }
+    if (replay_image_suffix) {
+        g_free(replay_image_suffix);
+        replay_image_suffix = NULL;
+    }
+
+    replay_finish_events();
+}
diff --git a/replay/replay.h b/replay/replay.h
index 2ad15b2..4eac46f 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -17,6 +17,8 @@
 #include <time.h>
 #include "qapi-types.h"
 
+struct QemuOpts;
+
 /* replay clock kinds */
 /* rdtsc */
 #define REPLAY_CLOCK_REAL_TICKS 0
@@ -35,6 +37,15 @@ extern int replay_icount;
 /*! Returns replay play submode */
 ReplaySubmode replay_get_play_submode(void);
 
+/* Replay process control functions */
+
+/*! Enables recording or saving event log with specified parameters */
+void replay_configure(struct QemuOpts *opts, int mode);
+/*! Initializes timers used for snapshotting and enables events recording */
+void replay_init_timer(void);
+/*! Closes replay log file and frees other resources. */
+void replay_finish(void);
+
 /* Processing the instructions */
 
 /*! Returns number of executed instructions. */
diff --git a/vl.c b/vl.c
index 5768b34..bddea69 100644
--- a/vl.c
+++ b/vl.c
@@ -4521,6 +4521,9 @@ int main(int argc, char **argv, char **envp)
     /* Done notifiers can load ROMs */
     rom_load_done();
 
+    /* init replay-based icount virtual timer */
+    replay_init_icount();
+
     qemu_system_reset(VMRESET_SILENT);
     if (loadvm) {
         if (load_vmstate(loadvm) < 0) {
@@ -4556,7 +4559,12 @@ int main(int argc, char **argv, char **envp)
         }
     }
 
+    replay_init_timer();
+
     main_loop();
+    if (replay_mode != REPLAY_MODE_NONE) {
+        replay_disable_events();
+    }
     bdrv_close_all();
     pause_all_vcpus();
     res_free();

[Qemu-devel] [RFC PATCH v3 38/49] replay: command line options

[Pavel Dovgalyuk]

This patch introduces command line options for enabling recording or replaying
virtual machine behavior. "-record" option starts recording of the execution
and saves it into the log, specified with "fname" parameter. "-replay" option
is intended for replaying previously saved log.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpus.c          |   15 ++++++++--
 qemu-options.hx |   29 ++++++++++++++++++++
 replay/replay.c |    2 +
 vl.c            |   79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 4 files changed, 120 insertions(+), 5 deletions(-)

diff --git a/cpus.c b/cpus.c
index d5fa5d1..70df028 100644
--- a/cpus.c
+++ b/cpus.c
@@ -841,12 +841,21 @@ static void qemu_wait_io_event_common(CPUState *cpu)
 static void qemu_tcg_wait_io_event(void)
 {
     CPUState *cpu;
+    GMainContext *context = g_main_context_default();
 
-    while (all_cpu_threads_idle()) {
-       /* Start accounting real time to the virtual clock if the CPUs
-          are idle.  */
+    if (replay_mode == REPLAY_MODE_PLAY
+        && all_cpu_threads_idle() && first_cpu->halted) {
+        /* wakeup iothread when there is no code to execute in replay mode */
         qemu_clock_warp(QEMU_CLOCK_VIRTUAL);
+        g_main_context_wakeup(context);
         qemu_cond_wait(tcg_halt_cond, &qemu_global_mutex);
+    } else {
+        while (all_cpu_threads_idle()) {
+            /* Start accounting real time to the virtual clock if the CPUs
+              are idle.  */
+            qemu_clock_warp(QEMU_CLOCK_VIRTUAL);
+            qemu_cond_wait(tcg_halt_cond, &qemu_global_mutex);
+        }
     }
 
     while (iothread_requesting_mutex) {
diff --git a/qemu-options.hx b/qemu-options.hx
index 1549625..7dcdf68 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3348,6 +3348,35 @@ Dump json-encoded vmstate information for current machine type to file
 in @var{file}
 ETEXI
 
+DEF("record", HAS_ARG, QEMU_OPTION_record,
+    "-record fname=<filename>[,suffix=<suffix>,snapshot=<on/off>,icount=<icount>]\n"
+    "                writes replay file for latter replaying\n",
+    QEMU_ARCH_ALL)
+STEXI
+@item -record fname=@var{file}[,suffix=@var{suffix},snapshot=@var{snapshot},icount=@var{icount}]
+Writes compact execution trace into @var{file}.
+Changes for disk images are written
+into separate files with @var{suffix} added. If no @var{suffix} is
+specified, "replay_qcow" is used as suffix.
+If @var{snapshot} parameter is set as off, then original disk image will be
+modified. Default value is on.
+@var{icount} parameter is used for vm clock emulation.
+ETEXI
+
+DEF("replay", HAS_ARG, QEMU_OPTION_replay,
+    "-replay fname=<filename>[,suffix=<suffix>,snapshot=<on/off>,icount=<icount>]\n"
+    "                plays saved replay file\n", QEMU_ARCH_ALL)
+STEXI
+@item -replay fname=@var{filename}[,suffix=@var{suffix},snapshot=@var{snapshot},icount=@var{icount}]
+Plays compact execution trace from @var{filename}.
+Changes for disk images and VM states are read
+from separate files with @var{suffix} added. If no @var{suffix} is
+specified, "replay_qcow" is used as suffix.
+If @var{snapshot} parameter is set as off, then original disk image will be
+modified. Default value is on.
+@var{icount} parameter is used for vm clock emulation.
+ETEXI
+
 HXCOMM This is the last statement. Insert new options before this line!
 STEXI
 @end table
diff --git a/replay/replay.c b/replay/replay.c
index f3e7c09..ea4de5c 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -320,6 +320,8 @@ void replay_configure(QemuOpts *opts, int mode)
         replay_image_suffix = g_strdup("replay_qcow");
     }
 
+    replay_icount = (int)qemu_opt_get_number(opts, "icount", 0);
+
     replay_enable(fname, mode);
 }
 
diff --git a/vl.c b/vl.c
index bddea69..628aca6 100644
--- a/vl.c
+++ b/vl.c
@@ -538,6 +538,48 @@ static QemuOptsList qemu_mem_opts = {
     },
 };
 
+static QemuOptsList qemu_record_opts = {
+    .name = "record",
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_record_opts.head),
+    .desc = {
+        {
+            .name = "fname",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "suffix",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "snapshot",
+            .type = QEMU_OPT_BOOL,
+        },{
+            .name = "icount",
+            .type = QEMU_OPT_NUMBER,
+        },
+        { /* end of list */ }
+    },
+};
+
+static QemuOptsList qemu_replay_opts = {
+    .name = "replay",
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_replay_opts.head),
+    .desc = {
+        {
+            .name = "fname",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "suffix",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "snapshot",
+            .type = QEMU_OPT_BOOL,
+        },{
+            .name = "icount",
+            .type = QEMU_OPT_NUMBER,
+        },
+        { /* end of list */ }
+    },
+};
+
 /**
  * Get machine options
  *
@@ -2925,7 +2967,8 @@ out:
 int main(int argc, char **argv, char **envp)
 {
     int i;
-    int snapshot, linux_boot;
+    int snapshot, linux_boot, replay_snapshot;
+    int not_compatible_replay_param = 0;
     const char *icount_option = NULL;
     const char *initrd_filename;
     const char *kernel_filename, *kernel_cmdline;
@@ -2997,6 +3040,8 @@ int main(int argc, char **argv, char **envp)
     qemu_add_opts(&qemu_msg_opts);
     qemu_add_opts(&qemu_name_opts);
     qemu_add_opts(&qemu_numa_opts);
+    qemu_add_opts(&qemu_replay_opts);
+    qemu_add_opts(&qemu_record_opts);
 
     runstate_init();
 
@@ -3010,6 +3055,7 @@ int main(int argc, char **argv, char **envp)
     cpu_model = NULL;
     ram_size = default_ram_size;
     snapshot = 0;
+    replay_snapshot = 1;
     cyls = heads = secs = 0;
     translation = BIOS_ATA_TRANSLATION_AUTO;
 
@@ -3127,6 +3173,7 @@ int main(int argc, char **argv, char **envp)
                 break;
             case QEMU_OPTION_pflash:
                 drive_add(IF_PFLASH, -1, optarg, PFLASH_OPTS);
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_snapshot:
                 snapshot = 1;
@@ -3283,6 +3330,7 @@ int main(int argc, char **argv, char **envp)
 #endif
             case QEMU_OPTION_bt:
                 add_device_config(DEV_BT, optarg);
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_audio_help:
                 AUD_help ();
@@ -3497,6 +3545,7 @@ int main(int argc, char **argv, char **envp)
                 if (!opts) {
                     exit(1);
                 }
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_fsdev:
                 olist = qemu_find_opts("fsdev");
@@ -3625,6 +3674,7 @@ int main(int argc, char **argv, char **envp)
                 if (strncmp(optarg, "mon:", 4) == 0) {
                     default_monitor = 0;
                 }
+                not_compatible_replay_param++;
                 break;
             case QEMU_OPTION_debugcon:
                 add_device_config(DEV_DEBUGCON, optarg);
@@ -3745,6 +3795,7 @@ int main(int argc, char **argv, char **envp)
                 if (!qemu_opts_parse(qemu_find_opts("smp-opts"), optarg, 1)) {
                     exit(1);
                 }
+                not_compatible_replay_param++;
                 break;
 	    case QEMU_OPTION_vnc:
 #ifdef CONFIG_VNC
@@ -3979,6 +4030,24 @@ int main(int argc, char **argv, char **envp)
                     exit(1);
                 }
                 break;
+            case QEMU_OPTION_record:
+                opts = qemu_opts_parse(qemu_find_opts("record"), optarg, 0);
+                if (!opts) {
+                    fprintf(stderr, "Invalid record options: %s\n", optarg);
+                    exit(1);
+                }
+                replay_configure(opts, REPLAY_MODE_RECORD);
+                replay_snapshot = qemu_opt_get_bool(opts, "snapshot", 1);
+                break;
+            case QEMU_OPTION_replay:
+                opts = qemu_opts_parse(qemu_find_opts("replay"), optarg, 0);
+                if (!opts) {
+                    fprintf(stderr, "Invalid replay options: %s\n", optarg);
+                    exit(1);
+                }
+                replay_configure(opts, REPLAY_MODE_PLAY);
+                replay_snapshot = qemu_opt_get_bool(opts, "snapshot", 1);
+                break;
             default:
                 os_parse_cmd_args(popt->index, optarg);
             }
@@ -3993,6 +4062,12 @@ int main(int argc, char **argv, char **envp)
         exit(1);
     }
 
+    if (not_compatible_replay_param && (replay_mode != REPLAY_MODE_NONE)) {
+        fprintf(stderr, "options -smp, -pflash, -chardev, -bt, -parallel "
+                        "are not compatible with record/replay\n");
+        exit(1);
+    }
+
     if (qemu_opts_foreach(qemu_find_opts("sandbox"), parse_sandbox, NULL, 0)) {
         exit(1);
     }
@@ -4359,7 +4434,7 @@ int main(int argc, char **argv, char **envp)
     ram_mig_init();
 
     /* open the virtual block devices */
-    if (snapshot)
+    if (snapshot || (replay_mode != REPLAY_MODE_NONE && replay_snapshot))
         qemu_opts_foreach(qemu_find_opts("drive"), drive_enable_snapshot, NULL, 0);
     if (qemu_opts_foreach(qemu_find_opts("drive"), drive_init_func,
                           &machine_class->block_default_type, 1) != 0) {

[Qemu-devel] [RFC PATCH v3 39/49] replay: snapshotting the virtual machine

[Pavel Dovgalyuk]

This patch adds 'period' parameter to the 'record' command line option. This
parameters turns on periodic snapshotting of the VM which could be used by
replay to move forward and backward in time.
If 'period' parameter is not specified, only one snapshot is made at the start
of the virtual machine.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 cpus.c                   |   46 ++++++++-
 qemu-options.hx          |    5 +
 qemu-timer.c             |    4 +
 replay/replay-internal.h |   18 ++++
 replay/replay.c          |  226 ++++++++++++++++++++++++++++++++++++++++++++++
 vl.c                     |    3 +
 6 files changed, 294 insertions(+), 8 deletions(-)

diff --git a/cpus.c b/cpus.c
index 70df028..cd739d2 100644
--- a/cpus.c
+++ b/cpus.c
@@ -231,12 +231,23 @@ int64_t cpu_get_clock(void)
  */
 void cpu_enable_ticks(void)
 {
+    int64_t ti;
     /* Here, the really thing protected by seqlock is cpu_clock_offset. */
     seqlock_write_lock(&timers_state.vm_clock_seqlock);
     if (!timers_state.cpu_ticks_enabled) {
         if (!replay_icount) {
             timers_state.cpu_ticks_offset -= cpu_get_real_ticks();
-            timers_state.cpu_clock_offset -= get_clock();
+
+            if (replay_mode == REPLAY_MODE_RECORD) {
+                ti = get_clock();
+                replay_save_clock(REPLAY_CLOCK_VIRTUAL, ti);
+            } else if (replay_mode == REPLAY_MODE_PLAY) {
+                ti = replay_read_clock(REPLAY_CLOCK_VIRTUAL);
+            } else {
+                ti = get_clock();
+            }
+
+            timers_state.cpu_clock_offset -= ti;
         }
         timers_state.cpu_ticks_enabled = 1;
     }
@@ -449,6 +460,22 @@ void qemu_clock_warp(QEMUClockType type)
     }
 }
 
+static bool is_replay_enabled(void *opaque)
+{
+    return replay_mode != REPLAY_MODE_NONE;
+}
+
+static const VMStateDescription vmstate_timers_for_replay = {
+    .name = "timer for replay",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_INT64(cpu_ticks_prev, TimersState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static const VMStateDescription vmstate_timers = {
     .name = "timer",
     .version_id = 2,
@@ -458,6 +485,14 @@ static const VMStateDescription vmstate_timers = {
         VMSTATE_INT64(dummy, TimersState),
         VMSTATE_INT64_V(cpu_clock_offset, TimersState, 2),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_timers_for_replay,
+            .needed = is_replay_enabled,
+        }, {
+            /* empty */
+        }
     }
 };
 
@@ -549,9 +584,11 @@ static int do_vm_stop(RunState state)
     int ret = 0;
 
     if (runstate_is_running()) {
+        runstate_set(state);
+        /* Disable ticks can cause recursive call of vm_stop.
+           Stopping before calling functions prevents infinite recursion. */
         cpu_disable_ticks();
         pause_all_vcpus();
-        runstate_set(state);
         vm_state_notify(0, state);
         qapi_event_send_stop(&error_abort);
     }
@@ -1332,10 +1369,9 @@ static void tcg_exec_all(void)
         CPUState *cpu = next_cpu;
         CPUArchState *env = cpu->env_ptr;
 
-        qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
-                          (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
-
         if (cpu_can_run(cpu)) {
+            qemu_clock_enable(QEMU_CLOCK_VIRTUAL,
+                              (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0);
             r = tcg_cpu_exec(env);
             if (r == EXCP_DEBUG) {
                 cpu_handle_guest_debug(cpu);
diff --git a/qemu-options.hx b/qemu-options.hx
index 7dcdf68..00315fa 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3349,12 +3349,13 @@ in @var{file}
 ETEXI
 
 DEF("record", HAS_ARG, QEMU_OPTION_record,
-    "-record fname=<filename>[,suffix=<suffix>,snapshot=<on/off>,icount=<icount>]\n"
+    "-record fname=<filename>[,period=<period>,suffix=<suffix>,snapshot=<on/off>,icount=<icount>]\n"
     "                writes replay file for latter replaying\n",
     QEMU_ARCH_ALL)
 STEXI
-@item -record fname=@var{file}[,suffix=@var{suffix},snapshot=@var{snapshot},icount=@var{icount}]
+@item -record fname=@var{file}[,period=@var{period},suffix=@var{suffix},snapshot=@var{snapshot},icount=@var{icount}]
 Writes compact execution trace into @var{file}.
+VM state is auto saved every @var{period} second, if this parameter is specified.
 Changes for disk images are written
 into separate files with @var{suffix} added. If no @var{suffix} is
 specified, "replay_qcow" is used as suffix.
diff --git a/qemu-timer.c b/qemu-timer.c
index f8bf060..c33680f 100644
--- a/qemu-timer.c
+++ b/qemu-timer.c
@@ -104,7 +104,9 @@ QEMUTimerList *timerlist_new(QEMUClockType type,
     QEMUClock *clock = qemu_clock_ptr(type);
 
     timer_list = g_malloc0(sizeof(QEMUTimerList));
-    qemu_event_init(&timer_list->timers_done_ev, false);
+    /* Create signaled event, because they should be signaled
+       outside the timerlist_run_timers function */
+    qemu_event_init(&timer_list->timers_done_ev, true);
     timer_list->clock = clock;
     timer_list->notify_cb = cb;
     timer_list->notify_opaque = opaque;
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 25b133f..74ff59b 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -15,6 +15,8 @@
 #include <stdio.h>
 #include "sysemu/sysemu.h"
 
+/* internal data for savevm */
+#define EVENT_END_STARTUP           0
 /* for time_t event */
 #define EVENT_TIME_T                1
 /* for tm event */
@@ -23,6 +25,10 @@
 #define EVENT_INTERRUPT             15
 /* for shutdown request */
 #define EVENT_SHUTDOWN              20
+/* for save VM event */
+#define EVENT_SAVE_VM_BEGIN         21
+/* for save VM event */
+#define EVENT_SAVE_VM_END           22
 /* for emulated exceptions */
 #define EVENT_EXCEPTION             23
 /* for async events */
@@ -53,9 +59,21 @@ typedef struct ReplayState {
     int skipping_instruction;
     /*! Current step - number of processed instructions and timer events. */
     uint64_t current_step;
+    /*! Temporary data for saving/loading replay file position. */
+    uint64_t file_offset;
 } ReplayState;
 extern ReplayState replay_state;
 
+/*! Information about saved VM state */
+struct SavedStateInfo {
+    /* Offset in the replay log file where state is saved. */
+    uint64_t file_offset;
+    /* Step number, corresponding to the saved state. */
+    uint64_t step;
+};
+/*! Reference to the saved state */
+typedef struct SavedStateInfo SavedStateInfo;
+
 extern volatile unsigned int replay_data_kind;
 extern volatile unsigned int replay_has_unread_data;
 
diff --git a/replay/replay.c b/replay/replay.c
index ea4de5c..a9268db 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -12,6 +12,8 @@
 #include "qemu-common.h"
 #include "replay.h"
 #include "replay-internal.h"
+#include "migration/vmstate.h"
+#include "monitor/monitor.h"
 
 /* Current version of the replay mechanism.
    Increase it when file format changes. */
@@ -30,12 +32,178 @@ char *replay_image_suffix;
 
 ReplayState replay_state;
 
+/*
+    Auto-saving for VM states data
+*/
+
+/* Minimum capacity of saved states information array */
+#define SAVED_STATES_MIN_CAPACITY   128
+/* Format of the name for the saved state */
+#define SAVED_STATE_NAME_FORMAT     "replay_%" PRId64
+
+/* Timer for auto-save VM states */
+static QEMUTimer *save_timer;
+/* Save state period in seconds */
+static uint64_t save_state_period;
+/* List of the saved states information */
+SavedStateInfo *saved_states;
+/* Number of saved states */
+static size_t saved_states_count;
+/* Capacity of the buffer for saved states */
+static size_t saved_states_capacity;
+/* Number of last loaded/saved state */
+static uint64_t current_saved_state;
+
+/*
+   Replay functions
+ */
 
 ReplaySubmode replay_get_play_submode(void)
 {
     return play_submode;
 }
 
+static void replay_pre_save(void *opaque)
+{
+    ReplayState *state = opaque;
+    state->file_offset = ftello64(replay_file);
+}
+
+static int replay_post_load(void *opaque, int version_id)
+{
+    first_cpu->instructions_count = 0;
+
+    ReplayState *state = opaque;
+    fseeko64(replay_file, state->file_offset, SEEK_SET);
+    replay_has_unread_data = 0;
+
+    return 0;
+}
+
+static const VMStateDescription vmstate_replay = {
+    .name = "replay",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .pre_save = replay_pre_save,
+    .post_load = replay_post_load,
+    .fields      = (VMStateField[]) {
+        VMSTATE_INT64_ARRAY(cached_clock, ReplayState, REPLAY_CLOCK_COUNT),
+        VMSTATE_INT32(skipping_instruction, ReplayState),
+        VMSTATE_UINT64(current_step, ReplayState),
+        VMSTATE_UINT64(file_offset, ReplayState),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static void replay_savevm(void *opaque)
+{
+    char name[128];
+    uint64_t offset;
+
+    offset = ftello64(replay_file);
+
+    replay_save_instructions();
+
+    replay_put_event(EVENT_SAVE_VM_BEGIN);
+
+    vm_stop(RUN_STATE_SAVE_VM);
+
+    /* save VM state */
+    sprintf(name, SAVED_STATE_NAME_FORMAT, current_saved_state);
+    if (save_vmstate(default_mon, name) > 0) {
+        /* if period is 0, save only once */
+        if (save_state_period != 0) {
+            timer_mod(save_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME)
+                                  + save_state_period);
+        }
+
+        /* add more memory to buffer */
+        if (saved_states_count >= saved_states_capacity) {
+            saved_states_capacity += SAVED_STATES_MIN_CAPACITY;
+            saved_states = g_realloc(saved_states, saved_states_capacity
+                                                   * sizeof(SavedStateInfo));
+            if (!saved_states) {
+                saved_states_count = 0;
+                fprintf(stderr,
+                        "Replay: Saved states memory reallocation failed.\n");
+                exit(1);
+            }
+        }
+        /* save state ID into the buffer */
+        saved_states[saved_states_count].file_offset = offset;
+        saved_states[saved_states_count].step = replay_get_current_step();
+        ++saved_states_count;
+        ++current_saved_state;
+    } else {
+        fprintf(stderr, "Cannot save simulator states for replay.\n");
+    }
+
+    replay_put_event(EVENT_SAVE_VM_END);
+
+    tb_flush_all();
+
+    vm_start();
+}
+
+/*! Checks SAVEVM event while reading event log. */
+static void check_savevm(void)
+{
+    replay_fetch_data_kind();
+    if (replay_data_kind != EVENT_SAVE_VM_BEGIN
+        && replay_data_kind != EVENT_SAVE_VM_END) {
+        fprintf(stderr, "Replay: read wrong data kind %d within savevm\n",
+                replay_data_kind);
+        exit(1);
+    }
+    replay_has_unread_data = 0;
+}
+
+/*! Loads specified VM state. */
+static void replay_loadvm(int64_t state)
+{
+    char name[128];
+    bool running = runstate_is_running();
+    if (running && !qemu_in_vcpu_thread()) {
+        vm_stop(RUN_STATE_RESTORE_VM);
+    } else {
+        cpu_disable_ticks();
+    }
+
+    replay_clear_events();
+
+    sprintf(name, SAVED_STATE_NAME_FORMAT, state);
+    if (load_vmstate(name) < 0) {
+        fprintf(stderr, "Replay: cannot load VM state\n");
+        exit(1);
+    }
+    /* check end event */
+    check_savevm();
+
+    tb_flush_all();
+
+    current_saved_state = state;
+
+    cpu_enable_ticks();
+    if (running && !qemu_in_vcpu_thread()) {
+        vm_start();
+    }
+
+    replay_fetch_data_kind();
+    while (replay_data_kind >= EVENT_CLOCK
+           && replay_data_kind < EVENT_CLOCK + REPLAY_CLOCK_COUNT) {
+        replay_read_next_clock(-1);
+        replay_fetch_data_kind();
+    }
+}
+
+/*! Skips clock events saved to file while saving the VM state. */
+static void replay_skip_savevm(void)
+{
+    replay_has_unread_data = 0;
+    replay_loadvm(current_saved_state + 1);
+}
+
 bool skip_async_events(int stop_event)
 {
     /* nothing to skip - not all instructions used */
@@ -55,6 +223,13 @@ bool skip_async_events(int stop_event)
             replay_has_unread_data = 0;
             qemu_system_shutdown_request_impl();
             break;
+        case EVENT_SAVE_VM_BEGIN:
+            /* cannot correctly load VM while in CPU thread */
+            if (qemu_in_vcpu_thread()) {
+                return res;
+            }
+            replay_skip_savevm();
+            break;
         case EVENT_INSTRUCTION:
             first_cpu->instructions_count = replay_get_dword();
             return res;
@@ -285,6 +460,7 @@ static void replay_enable(const char *fname, int mode)
     replay_data_kind = -1;
     replay_state.skipping_instruction = 0;
     replay_state.current_step = 0;
+    current_saved_state = 0;
 
     /* skip file header for RECORD and check it for PLAY */
     if (replay_mode == REPLAY_MODE_RECORD) {
@@ -296,11 +472,23 @@ static void replay_enable(const char *fname, int mode)
             fprintf(stderr, "Replay: invalid input log file version\n");
             exit(1);
         }
+        /* read states table */
+        fseeko64(replay_file, offset, SEEK_SET);
+        saved_states_count = replay_get_qword();
+        saved_states_capacity = saved_states_count;
+        if (saved_states_count) {
+            saved_states = g_malloc(sizeof(SavedStateInfo)
+                                    * saved_states_count);
+            fread(saved_states, sizeof(SavedStateInfo), saved_states_count,
+                  replay_file);
+        }
         /* go to the beginning */
         fseek(replay_file, 12, SEEK_SET);
     }
 
     replay_init_events();
+
+    vmstate_register(NULL, 0, &vmstate_replay, &replay_state);
 }
 
 void replay_configure(QemuOpts *opts, int mode)
@@ -321,6 +509,7 @@ void replay_configure(QemuOpts *opts, int mode)
     }
 
     replay_icount = (int)qemu_opt_get_number(opts, "icount", 0);
+    save_state_period = 1000LL * qemu_opt_get_number(opts, "period", 0);
 
     replay_enable(fname, mode);
 }
@@ -332,6 +521,26 @@ void replay_init_timer(void)
     }
 
     replay_enable_events();
+
+    /* create timer for states auto-saving */
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        saved_states_count = 0;
+        if (!saved_states) {
+            saved_states = g_malloc(sizeof(SavedStateInfo)
+                                    * SAVED_STATES_MIN_CAPACITY);
+            saved_states_capacity = SAVED_STATES_MIN_CAPACITY;
+        }
+        if (save_state_period) {
+            save_timer = timer_new_ms(QEMU_CLOCK_REALTIME, replay_savevm, NULL);
+            timer_mod(save_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME));
+        }
+        replay_put_event(EVENT_END_STARTUP);
+        /* Save it right now without waiting for timer */
+        replay_savevm(NULL);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        /* load starting VM state */
+        replay_loadvm(0);
+    }
 }
 
 void replay_finish(void)
@@ -349,6 +558,14 @@ void replay_finish(void)
             /* write end event */
             replay_put_event(EVENT_END);
 
+            /* write states table */
+            offset = ftello64(replay_file);
+            replay_put_qword(saved_states_count);
+            if (saved_states && saved_states_count) {
+                fwrite(saved_states, sizeof(SavedStateInfo),
+                       saved_states_count, replay_file);
+            }
+
             /* write header */
             fseek(replay_file, 0, SEEK_SET);
             replay_put_dword(REPLAY_VERSION);
@@ -358,6 +575,15 @@ void replay_finish(void)
         fclose(replay_file);
         replay_file = NULL;
     }
+    if (save_timer) {
+        timer_del(save_timer);
+        timer_free(save_timer);
+        save_timer = NULL;
+    }
+    if (saved_states) {
+        g_free(saved_states);
+        saved_states = NULL;
+    }
     if (replay_filename) {
         g_free(replay_filename);
         replay_filename = NULL;
diff --git a/vl.c b/vl.c
index 628aca6..b0127a2 100644
--- a/vl.c
+++ b/vl.c
@@ -575,6 +575,9 @@ static QemuOptsList qemu_replay_opts = {
         },{
             .name = "icount",
             .type = QEMU_OPT_NUMBER,
+        },{
+            .name = "period",
+            .type = QEMU_OPT_NUMBER,
         },
         { /* end of list */ }
     },

[Qemu-devel] [RFC PATCH v3 40/49] replay: recording of the user input

[Pavel Dovgalyuk]

This records user input (keyboard and mouse events) in record mode and replays
these input events in replay mode.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/ui/input.h       |    2 +
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   48 +++++++++++++++++++++
 replay/replay-input.c    |  107 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h |   11 ++++-
 replay/replay.h          |    5 ++
 ui/input.c               |   80 ++++++++++++++++++++++++++--------
 7 files changed, 234 insertions(+), 20 deletions(-)
 create mode 100755 replay/replay-input.c

diff --git a/include/ui/input.h b/include/ui/input.h
index 5d5ac00..d06a12d 100644
--- a/include/ui/input.h
+++ b/include/ui/input.h
@@ -33,7 +33,9 @@ void qemu_input_handler_bind(QemuInputHandlerState *s,
                              const char *device_id, int head,
                              Error **errp);
 void qemu_input_event_send(QemuConsole *src, InputEvent *evt);
+void qemu_input_event_send_impl(QemuConsole *src, InputEvent *evt);
 void qemu_input_event_sync(void);
+void qemu_input_event_sync_impl(void);
 
 InputEvent *qemu_input_event_new_key(KeyValue *key, bool down);
 void qemu_input_event_send_key(QemuConsole *src, KeyValue *key, bool down);
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 7dec93f..59c3602 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -3,3 +3,4 @@ obj-y += replay-internal.o
 obj-y += replay-events.o
 obj-y += replay-time.o
 obj-y += replay-icount.o
+obj-y += replay-input.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
index fa24726..f8e58ea 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -12,6 +12,7 @@
 #include "replay.h"
 #include "replay-internal.h"
 #include "block/thread-pool.h"
+#include "ui/input.h"
 
 typedef struct Event {
     int event_kind;
@@ -42,6 +43,16 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_THREAD:
         thread_pool_work((ThreadPool *)event->opaque, event->opaque2);
         break;
+    case REPLAY_ASYNC_EVENT_INPUT:
+        qemu_input_event_send_impl(NULL, (InputEvent *)event->opaque);
+        /* Using local variables, when replaying. Do not free them. */
+        if (replay_mode == REPLAY_MODE_RECORD) {
+            qapi_free_InputEvent((InputEvent *)event->opaque);
+        }
+        break;
+    case REPLAY_ASYNC_EVENT_INPUT_SYNC:
+        qemu_input_event_sync_impl();
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n",
                 event->event_kind);
@@ -135,6 +146,16 @@ void replay_add_thread_event(void *opaque, void *opaque2, uint64_t id)
     replay_add_event_internal(REPLAY_ASYNC_EVENT_THREAD, opaque, opaque2, id);
 }
 
+void replay_add_input_event(struct InputEvent *event)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_INPUT, event, NULL, 0);
+}
+
+void replay_add_input_sync_event(void)
+{
+    replay_add_event_internal(REPLAY_ASYNC_EVENT_INPUT_SYNC, NULL, NULL, 0);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -156,6 +177,9 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_THREAD:
                 replay_put_qword(event->id);
                 break;
+            case REPLAY_ASYNC_EVENT_INPUT:
+                replay_save_input_event(event->opaque);
+                break;
             }
         }
 
@@ -185,6 +209,7 @@ void replay_read_events(int opt)
             break;
         }
         /* Execute some events without searching them in the queue */
+        Event e;
         switch (read_event_kind) {
         case REPLAY_ASYNC_EVENT_BH:
         case REPLAY_ASYNC_EVENT_THREAD:
@@ -192,6 +217,29 @@ void replay_read_events(int opt)
                 read_id = replay_get_qword();
             }
             break;
+        case REPLAY_ASYNC_EVENT_INPUT:
+            e.event_kind = read_event_kind;
+            e.opaque = replay_read_input_event();
+
+            replay_run_event(&e);
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+            /* continue with the next event */
+            continue;
+        case REPLAY_ASYNC_EVENT_INPUT_SYNC:
+            e.event_kind = read_event_kind;
+            e.opaque = 0;
+            replay_run_event(&e);
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+            /* continue with the next event */
+            continue;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-input.c b/replay/replay-input.c
new file mode 100755
index 0000000..93eb42d
--- /dev/null
+++ b/replay/replay-input.c
@@ -0,0 +1,107 @@
+/*
+ * replay-input.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "replay.h"
+#include "replay-internal.h"
+#include "ui/input.h"
+
+void replay_save_input_event(InputEvent *evt)
+{
+    replay_put_dword(evt->kind);
+
+    switch (evt->kind) {
+    case INPUT_EVENT_KIND_KEY:
+        replay_put_dword(evt->key->key->kind);
+
+        switch (evt->key->key->kind) {
+        case KEY_VALUE_KIND_NUMBER:
+            replay_put_qword(evt->key->key->number);
+            replay_put_byte(evt->key->down);
+            break;
+        case KEY_VALUE_KIND_QCODE:
+            replay_put_dword(evt->key->key->qcode);
+            replay_put_byte(evt->key->down);
+            break;
+        case KEY_VALUE_KIND_MAX:
+            /* keep gcc happy */
+            break;
+        }
+        break;
+    case INPUT_EVENT_KIND_BTN:
+        replay_put_dword(evt->btn->button);
+        replay_put_byte(evt->btn->down);
+        break;
+    case INPUT_EVENT_KIND_REL:
+        replay_put_dword(evt->rel->axis);
+        replay_put_qword(evt->rel->value);
+        break;
+    case INPUT_EVENT_KIND_ABS:
+        replay_put_dword(evt->abs->axis);
+        replay_put_qword(evt->abs->value);
+        break;
+    case INPUT_EVENT_KIND_MAX:
+        /* keep gcc happy */
+        break;
+    }
+}
+
+InputEvent *replay_read_input_event(void)
+{
+    static InputEvent evt;
+    static KeyValue keyValue;
+    static InputKeyEvent key;
+    key.key = &keyValue;
+    static InputBtnEvent btn;
+    static InputMoveEvent rel;
+    static InputMoveEvent abs;
+
+    evt.kind = replay_get_dword();
+    switch (evt.kind) {
+    case INPUT_EVENT_KIND_KEY:
+        evt.key = &key;
+        evt.key->key->kind = replay_get_dword();
+
+        switch (evt.key->key->kind) {
+        case KEY_VALUE_KIND_NUMBER:
+            evt.key->key->number = replay_get_qword();
+            evt.key->down = replay_get_byte();
+            break;
+        case KEY_VALUE_KIND_QCODE:
+            evt.key->key->qcode = (QKeyCode)replay_get_dword();
+            evt.key->down = replay_get_byte();
+            break;
+        case KEY_VALUE_KIND_MAX:
+            /* keep gcc happy */
+            break;
+        }
+        break;
+    case INPUT_EVENT_KIND_BTN:
+        evt.btn = &btn;
+        evt.btn->button = (InputButton)replay_get_dword();
+        evt.btn->down = replay_get_byte();
+        break;
+    case INPUT_EVENT_KIND_REL:
+        evt.rel = &rel;
+        evt.rel->axis = (InputAxis)replay_get_dword();
+        evt.rel->value = replay_get_qword();
+        break;
+    case INPUT_EVENT_KIND_ABS:
+        evt.abs = &abs;
+        evt.abs->axis = (InputAxis)replay_get_dword();
+        evt.abs->value = replay_get_qword();
+        break;
+    case INPUT_EVENT_KIND_MAX:
+        /* keep gcc happy */
+        break;
+    }
+
+    return &evt;
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 74ff59b..5d0aa71 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -49,7 +49,9 @@
 
 #define REPLAY_ASYNC_EVENT_BH          0
 #define REPLAY_ASYNC_EVENT_THREAD      1
-#define REPLAY_ASYNC_COUNT             2
+#define REPLAY_ASYNC_EVENT_INPUT       2
+#define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
+#define REPLAY_ASYNC_COUNT             4
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -141,4 +143,11 @@ void replay_read_events(int opt);
 /*! Adds specified async event to the queue */
 void replay_add_event(int event_id, void *opaque);
 
+/* Input events */
+
+/*! Saves input event to the log */
+void replay_save_input_event(InputEvent *evt);
+/*! Reads input event from the log */
+InputEvent *replay_read_input_event(void);
+
 #endif
diff --git a/replay/replay.h b/replay/replay.h
index 4eac46f..31c02ba 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -18,6 +18,7 @@
 #include "qapi-types.h"
 
 struct QemuOpts;
+struct InputEvent;
 
 /* replay clock kinds */
 /* rdtsc */
@@ -105,6 +106,10 @@ void replay_disable_events(void);
 void replay_add_bh_event(void *bh, uint64_t id);
 /*! Adds thread event to the queue */
 void replay_add_thread_event(void *pool, void *req, uint64_t id);
+/*! Adds input event to the queue */
+void replay_add_input_event(struct InputEvent *event);
+/*! Adds input sync event to the queue */
+void replay_add_input_sync_event(void);
 
 /* icount-based virtual clock */
 
diff --git a/ui/input.c b/ui/input.c
index 89d9db7..fb7b94a 100644
--- a/ui/input.c
+++ b/ui/input.c
@@ -5,6 +5,7 @@
 #include "trace.h"
 #include "ui/input.h"
 #include "ui/console.h"
+#include "replay/replay.h"
 
 struct QemuInputHandlerState {
     DeviceState       *dev;
@@ -258,14 +259,10 @@ static void qemu_input_queue_sync(struct QemuInputEventQueueHead *queue)
     QTAILQ_INSERT_TAIL(queue, item, node);
 }
 
-void qemu_input_event_send(QemuConsole *src, InputEvent *evt)
+void qemu_input_event_send_impl(QemuConsole *src, InputEvent *evt)
 {
     QemuInputHandlerState *s;
 
-    if (!runstate_is_running() && !runstate_check(RUN_STATE_SUSPENDED)) {
-        return;
-    }
-
     qemu_input_event_trace(src, evt);
 
     /* pre processing */
@@ -282,14 +279,25 @@ void qemu_input_event_send(QemuConsole *src, InputEvent *evt)
     s->events++;
 }
 
-void qemu_input_event_sync(void)
+void qemu_input_event_send(QemuConsole *src, InputEvent *evt)
 {
-    QemuInputHandlerState *s;
-
     if (!runstate_is_running() && !runstate_check(RUN_STATE_SUSPENDED)) {
         return;
     }
 
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        /* Nothing */
+    } else if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_add_input_event(evt);
+    } else {
+        qemu_input_event_send_impl(src, evt);
+    }
+}
+
+void qemu_input_event_sync_impl(void)
+{
+    QemuInputHandlerState *s;
+
     trace_input_event_sync();
 
     QTAILQ_FOREACH(s, &handlers, node) {
@@ -303,6 +311,21 @@ void qemu_input_event_sync(void)
     }
 }
 
+void qemu_input_event_sync(void)
+{
+    if (!runstate_is_running() && !runstate_check(RUN_STATE_SUSPENDED)) {
+        return;
+    }
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        /* Nothing */
+    } else if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_add_input_sync_event();
+    } else {
+        qemu_input_event_sync_impl();
+    }
+}
+
 InputEvent *qemu_input_event_new_key(KeyValue *key, bool down)
 {
     InputEvent *evt = g_new0(InputEvent, 1);
@@ -316,14 +339,23 @@ InputEvent *qemu_input_event_new_key(KeyValue *key, bool down)
 void qemu_input_event_send_key(QemuConsole *src, KeyValue *key, bool down)
 {
     InputEvent *evt;
-    evt = qemu_input_event_new_key(key, down);
-    if (QTAILQ_EMPTY(&kbd_queue)) {
-        qemu_input_event_send(src, evt);
-        qemu_input_event_sync();
-        qapi_free_InputEvent(evt);
-    } else {
-        qemu_input_queue_event(&kbd_queue, src, evt);
-        qemu_input_queue_sync(&kbd_queue);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        evt = qemu_input_event_new_key(key, down);
+        if (QTAILQ_EMPTY(&kbd_queue)) {
+            qemu_input_event_send(src, evt);
+            qemu_input_event_sync();
+            if (replay_mode != REPLAY_MODE_RECORD) {
+                qapi_free_InputEvent(evt);
+            }
+        } else {
+            if (replay_mode != REPLAY_MODE_NONE) {
+                fprintf(stderr, "Input queue is not supported "
+                                "in record/replay mode\n");
+                exit(1);
+            }
+            qemu_input_queue_event(&kbd_queue, src, evt);
+            qemu_input_queue_sync(&kbd_queue);
+        }
     }
 }
 
@@ -349,6 +381,10 @@ void qemu_input_event_send_key_delay(uint32_t delay_ms)
         kbd_timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, qemu_input_queue_process,
                                  &kbd_queue);
     }
+    if (replay_mode != REPLAY_MODE_NONE) {
+        fprintf(stderr, "Input queue is not supported in record/replay mode\n");
+        exit(1);
+    }
     qemu_input_queue_delay(&kbd_queue, kbd_timer,
                            delay_ms ? delay_ms : kbd_default_delay_ms);
 }
@@ -368,7 +404,9 @@ void qemu_input_queue_btn(QemuConsole *src, InputButton btn, bool down)
     InputEvent *evt;
     evt = qemu_input_event_new_btn(btn, down);
     qemu_input_event_send(src, evt);
-    qapi_free_InputEvent(evt);
+    if (replay_mode != REPLAY_MODE_RECORD) {
+        qapi_free_InputEvent(evt);
+    }
 }
 
 void qemu_input_update_buttons(QemuConsole *src, uint32_t *button_map,
@@ -421,7 +459,9 @@ void qemu_input_queue_rel(QemuConsole *src, InputAxis axis, int value)
     InputEvent *evt;
     evt = qemu_input_event_new_move(INPUT_EVENT_KIND_REL, axis, value);
     qemu_input_event_send(src, evt);
-    qapi_free_InputEvent(evt);
+    if (replay_mode != REPLAY_MODE_RECORD) {
+        qapi_free_InputEvent(evt);
+    }
 }
 
 void qemu_input_queue_abs(QemuConsole *src, InputAxis axis, int value, int size)
@@ -430,7 +470,9 @@ void qemu_input_queue_abs(QemuConsole *src, InputAxis axis, int value, int size)
     int scaled = qemu_input_scale_axis(value, size, INPUT_EVENT_ABS_SIZE);
     evt = qemu_input_event_new_move(INPUT_EVENT_KIND_ABS, axis, scaled);
     qemu_input_event_send(src, evt);
-    qapi_free_InputEvent(evt);
+    if (replay_mode != REPLAY_MODE_RECORD) {
+        qapi_free_InputEvent(evt);
+    }
 }
 
 void qemu_input_check_mode_change(void)

[Qemu-devel] [RFC PATCH v3 41/49] tap-win32: destroy the thread at exit

[Pavel Dovgalyuk]

This patch fixes resource leak caused by created thread which is not destroyed
at exit.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 net/tap-win32.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/net/tap-win32.c b/net/tap-win32.c
index 8aee611..efd1c75 100644
--- a/net/tap-win32.c
+++ b/net/tap-win32.c
@@ -99,6 +99,7 @@ typedef struct tap_win32_overlapped {
     HANDLE output_queue_semaphore;
     HANDLE free_list_semaphore;
     HANDLE tap_semaphore;
+    HANDLE hThread;
     CRITICAL_SECTION output_queue_cs;
     CRITICAL_SECTION free_list_cs;
     OVERLAPPED read_overlapped;
@@ -625,8 +626,9 @@ static int tap_win32_open(tap_win32_overlapped_t **phandle,
 
     *phandle = &tap_overlapped;
 
-    CreateThread(NULL, 0, tap_win32_thread_entry,
-                 (LPVOID)&tap_overlapped, 0, &idThread);
+    tap_overlapped.hThread = CreateThread(NULL, 0, tap_win32_thread_entry,
+                                          (LPVOID)&tap_overlapped,
+                                          0, &idThread);
     return 0;
 }
 
@@ -643,9 +645,8 @@ static void tap_cleanup(NetClientState *nc)
 
     qemu_del_wait_object(s->handle->tap_semaphore, NULL, NULL);
 
-    /* FIXME: need to kill thread and close file handle:
-       tap_win32_close(s);
-    */
+    TerminateThread(s->handle->hThread, 0);
+    CloseHandle(s->handle->handle);
 }
 
 static ssize_t tap_receive(NetClientState *nc, const uint8_t *buf, size_t size)

[Qemu-devel] [RFC PATCH v3 42/49] replay: network packets record/replay

[Pavel Dovgalyuk]

This patch implements passing network packets to replay module in
record mode. New virtual network adapter is implemented to replay the
packets when they are read from the log file.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 net/Makefile.objs        |    1 
 net/clients.h            |    3 +
 net/dump.c               |    6 +
 net/hub.c                |    1 
 net/net-replay.c         |   68 ++++++++++++++++
 net/net.c                |    7 +-
 net/slirp.c              |   14 +++
 net/socket.c             |   35 ++++++++
 net/tap-win32.c          |   14 +++
 net/tap.c                |   24 +++++-
 net/vde.c                |   14 +++
 qapi-schema.json         |   13 +++
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   17 ++++
 replay/replay-internal.h |   21 +++++
 replay/replay-net.c      |  191 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay.c          |    5 +
 replay/replay.h          |    9 ++
 slirp/slirp.c            |    9 ++
 19 files changed, 439 insertions(+), 14 deletions(-)
 create mode 100755 net/net-replay.c
 create mode 100755 replay/replay-net.c

diff --git a/net/Makefile.objs b/net/Makefile.objs
index ec19cb3..def44e3 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -2,6 +2,7 @@ common-obj-y = net.o queue.o checksum.o util.o hub.o
 common-obj-y += socket.o
 common-obj-y += dump.o
 common-obj-y += eth.o
+common-obj-y += net-replay.o
 common-obj-$(CONFIG_L2TPV3) += l2tpv3.o
 common-obj-$(CONFIG_POSIX) += tap.o vhost-user.o
 common-obj-$(CONFIG_LINUX) += tap-linux.o
diff --git a/net/clients.h b/net/clients.h
index 2e8feda..f75f43b 100644
--- a/net/clients.h
+++ b/net/clients.h
@@ -62,4 +62,7 @@ int net_init_netmap(const NetClientOptions *opts, const char *name,
 int net_init_vhost_user(const NetClientOptions *opts, const char *name,
                         NetClientState *peer);
 
+int net_init_replay(const NetClientOptions *opts, const char *name,
+                    NetClientState *peer);
+
 #endif /* QEMU_NET_CLIENTS_H */
diff --git a/net/dump.c b/net/dump.c
index 9d3a09e..77e2d3f 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -28,6 +28,7 @@
 #include "qemu/log.h"
 #include "qemu/timer.h"
 #include "hub.h"
+#include "replay/replay.h"
 
 typedef struct DumpState {
     NetClientState nc;
@@ -158,6 +159,11 @@ int net_init_dump(const NetClientOptions *opts, const char *name,
 
     assert(peer);
 
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        fprintf(stderr, "-net dump is not permitted in replay mode\n");
+        exit(1);
+    }
+
     if (dump->has_file) {
         file = dump->file;
     } else {
diff --git a/net/hub.c b/net/hub.c
index 7e0f2d6..91fdfb6 100644
--- a/net/hub.c
+++ b/net/hub.c
@@ -323,6 +323,7 @@ void net_hub_check_clients(void)
             case NET_CLIENT_OPTIONS_KIND_SOCKET:
             case NET_CLIENT_OPTIONS_KIND_VDE:
             case NET_CLIENT_OPTIONS_KIND_VHOST_USER:
+            case NET_CLIENT_OPTIONS_KIND_REPLAY:
                 has_host_dev = 1;
                 break;
             default:
diff --git a/net/net-replay.c b/net/net-replay.c
new file mode 100755
index 0000000..01da9ef
--- /dev/null
+++ b/net/net-replay.c
@@ -0,0 +1,68 @@
+/*
+ * net-replay.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "net/net.h"
+#include "clients.h"
+#include "qemu-common.h"
+#include "sysemu/sysemu.h"
+#include "replay/replay.h"
+
+typedef struct NetReplayState {
+    NetClientState nc;
+} NetReplayState;
+
+static ssize_t net_replay_receive(NetClientState *nc, const uint8_t *buf,
+                                  size_t size)
+{
+    return size;
+}
+
+static void net_replay_cleanup(NetClientState *nc)
+{
+}
+
+static NetClientInfo net_replay_info = {
+    .type = NET_CLIENT_OPTIONS_KIND_REPLAY,
+    .size = sizeof(NetReplayState),
+    .receive = net_replay_receive,
+    .cleanup = net_replay_cleanup,
+};
+
+static int net_replay_init(NetClientState *vlan, const char *device,
+                         const char *name)
+{
+    NetClientState *nc;
+
+    nc = qemu_new_net_client(&net_replay_info, vlan, device, name);
+
+    snprintf(nc->info_str, sizeof(nc->info_str), "replayer");
+
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        fprintf(stderr, "-net replay is not permitted in record mode\n");
+        exit(1);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        replay_add_network_client(nc);
+    } else {
+        fprintf(stderr, "-net replay is not permitted without replay\n");
+        exit(1);
+    }
+
+    return 0;
+}
+
+int net_init_replay(const NetClientOptions *opts, const char *name,
+                    NetClientState *peer)
+{
+    assert(peer);
+    assert(opts->kind == NET_CLIENT_OPTIONS_KIND_REPLAY);
+
+    return net_replay_init(peer, "replay", name);
+}
diff --git a/net/net.c b/net/net.c
index 6d930ea..e837341 100644
--- a/net/net.c
+++ b/net/net.c
@@ -809,6 +809,7 @@ static int (* const net_client_init_fun[NET_CLIENT_OPTIONS_KIND_MAX])(
 #ifdef CONFIG_L2TPV3
         [NET_CLIENT_OPTIONS_KIND_L2TPV3]    = net_init_l2tpv3,
 #endif
+        [NET_CLIENT_OPTIONS_KIND_REPLAY]    = net_init_replay,
 };
 
 
@@ -1264,7 +1265,11 @@ int net_init_clients(void)
         /* if no clients, we use a default config */
         qemu_opts_set(net, NULL, "type", "nic");
 #ifdef CONFIG_SLIRP
-        qemu_opts_set(net, NULL, "type", "user");
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            qemu_opts_set(net, NULL, "type", "user");
+        } else {
+            qemu_opts_set(net, NULL, "type", "replay");
+        }
 #endif
     }
 
diff --git a/net/slirp.c b/net/slirp.c
index 647039e..95abae2 100644
--- a/net/slirp.c
+++ b/net/slirp.c
@@ -36,6 +36,7 @@
 #include "qemu/sockets.h"
 #include "slirp/libslirp.h"
 #include "sysemu/char.h"
+#include "replay/replay.h"
 
 static int get_str_sep(char *buf, int buf_size, const char **pp, int sep)
 {
@@ -103,7 +104,11 @@ void slirp_output(void *opaque, const uint8_t *pkt, int pkt_len)
 {
     SlirpState *s = opaque;
 
-    qemu_send_packet(&s->nc, pkt, pkt_len);
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_net_packet(&s->nc, pkt, pkt_len);
+    } else {
+        qemu_send_packet(&s->nc, pkt, pkt_len);
+    }
 }
 
 static ssize_t net_slirp_receive(NetClientState *nc, const uint8_t *buf, size_t size)
@@ -267,6 +272,13 @@ static int net_slirp_init(NetClientState *peer, const char *model,
     }
 #endif
 
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        fprintf(stderr, "-net user is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return 0;
 
 error:
diff --git a/net/socket.c b/net/socket.c
index fb21e20..5070d2a 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -32,6 +32,7 @@
 #include "qemu/sockets.h"
 #include "qemu/iov.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 typedef struct NetSocketState {
     NetClientState nc;
@@ -211,7 +212,11 @@ static void net_socket_send(void *opaque)
             buf += l;
             size -= l;
             if (s->index >= s->packet_len) {
-                qemu_send_packet(&s->nc, s->buf, s->packet_len);
+                if (replay_mode == REPLAY_MODE_RECORD) {
+                    replay_save_net_packet(&s->nc, s->buf, s->packet_len);
+                } else {
+                    qemu_send_packet(&s->nc, s->buf, s->packet_len);
+                }
                 s->index = 0;
                 s->state = 0;
             }
@@ -234,7 +239,11 @@ static void net_socket_send_dgram(void *opaque)
         net_socket_write_poll(s, false);
         return;
     }
-    qemu_send_packet(&s->nc, s->buf, size);
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_net_packet(&s->nc, s->buf, size);
+    } else {
+        qemu_send_packet(&s->nc, s->buf, size);
+    }
 }
 
 static int net_socket_mcast_create(struct sockaddr_in *mcastaddr, struct in_addr *localaddr)
@@ -406,6 +415,13 @@ static NetSocketState *net_socket_fd_init_dgram(NetClientState *peer,
         s->dgram_dst = saddr;
     }
 
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        fprintf(stderr, "-net socket is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return s;
 
 err:
@@ -452,6 +468,14 @@ static NetSocketState *net_socket_fd_init_stream(NetClientState *peer,
     } else {
         qemu_set_fd_handler(s->fd, NULL, net_socket_connect, s);
     }
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        printf("-net socket is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return s;
 }
 
@@ -548,6 +572,13 @@ static int net_socket_listen_init(NetClientState *peer,
     s->listen_fd = fd;
     s->nc.link_down = true;
 
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        printf("-net socket is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     qemu_set_fd_handler(s->listen_fd, net_socket_accept, NULL, s);
     return 0;
 }
diff --git a/net/tap-win32.c b/net/tap-win32.c
index efd1c75..c0b8339 100644
--- a/net/tap-win32.c
+++ b/net/tap-win32.c
@@ -34,6 +34,7 @@
 #include "net/tap.h"            /* tap_has_ufo, ... */
 #include "sysemu/sysemu.h"
 #include "qemu/error-report.h"
+#include "replay/replay.h"
 #include <stdio.h>
 #include <windows.h>
 #include <winioctl.h>
@@ -665,7 +666,11 @@ static void tap_win32_send(void *opaque)
 
     size = tap_win32_read(s->handle, &buf, max_size);
     if (size > 0) {
-        qemu_send_packet(&s->nc, buf, size);
+        if (replay_mode == REPLAY_MODE_RECORD) {
+            replay_save_net_packet(&s->nc, buf, size);
+        } else {
+            qemu_send_packet(&s->nc, buf, size);
+        }
         tap_win32_free_buffer(s->handle, buf);
     }
 }
@@ -749,6 +754,13 @@ static int tap_win32_init(NetClientState *peer, const char *model,
 
     qemu_add_wait_object(s->handle->tap_semaphore, tap_win32_send, s);
 
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        fprintf(stderr, "-net tap is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return 0;
 }
 
diff --git a/net/tap.c b/net/tap.c
index a40f7f0..16380fb 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -39,6 +39,7 @@
 #include "sysemu/sysemu.h"
 #include "qemu-common.h"
 #include "qemu/error-report.h"
+#include "replay/replay.h"
 
 #include "net/tap.h"
 
@@ -203,11 +204,18 @@ static void tap_send(void *opaque)
             size -= s->host_vnet_hdr_len;
         }
 
-        size = qemu_send_packet_async(&s->nc, buf, size, tap_send_completed);
-        if (size == 0) {
-            tap_read_poll(s, false);
+        if (replay_mode == REPLAY_MODE_RECORD) {
+            replay_save_net_packet(&s->nc, buf, size);
             break;
-        } else if (size < 0) {
+        } else {
+            size = qemu_send_packet_async(&s->nc, buf, size,
+                                          tap_send_completed);
+            if (size == 0) {
+                tap_read_poll(s, false);
+                break;
+            } else if (size < 0) {
+                break;
+            }
             break;
         }
     }
@@ -353,6 +361,14 @@ static TAPState *net_tap_fd_init(NetClientState *peer,
     }
     tap_read_poll(s, true);
     s->vhost_net = NULL;
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        fprintf(stderr, "-net tap is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return s;
 }
 
diff --git a/net/vde.c b/net/vde.c
index 2a619fb..ca34c4b 100644
--- a/net/vde.c
+++ b/net/vde.c
@@ -30,6 +30,7 @@
 #include "qemu-common.h"
 #include "qemu/option.h"
 #include "qemu/main-loop.h"
+#include "replay/replay.h"
 
 typedef struct VDEState {
     NetClientState nc;
@@ -44,7 +45,11 @@ static void vde_to_qemu(void *opaque)
 
     size = vde_recv(s->vde, (char *)buf, sizeof(buf), 0);
     if (size > 0) {
-        qemu_send_packet(&s->nc, buf, size);
+        if (replay_mode == REPLAY_SAVE) {
+            replay_save_net_packet(&s->nc, buf, size);
+        } else {
+            qemu_send_packet(&s->nc, buf, size);
+        }
     }
 }
 
@@ -106,6 +111,13 @@ static int net_vde_init(NetClientState *peer, const char *model,
 
     qemu_set_fd_handler(vde_datafd(s->vde), vde_to_qemu, NULL, s);
 
+    if (replay_mode == REPLAY_PLAY) {
+        fprintf(stderr, "-net vde is not permitted in replay mode\n");
+        exit(1);
+    } else {
+        replay_add_network_client(nc);
+    }
+
     return 0;
 }
 
diff --git a/qapi-schema.json b/qapi-schema.json
index 2de14c4..3f2dab4 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2211,6 +2211,16 @@
     '*vhostforce':    'bool' } }
 
 ##
+# @NetdevReplayOptions
+#
+# Reads network traffic from the log in replay mode.
+#
+# Since: 2.2
+##
+{ 'type': 'NetdevReplayOptions',
+  'data': { } }
+
+##
 # @NetClientOptions
 #
 # A discriminated record of network device traits.
@@ -2233,7 +2243,8 @@
     'bridge':   'NetdevBridgeOptions',
     'hubport':  'NetdevHubPortOptions',
     'netmap':   'NetdevNetmapOptions',
-    'vhost-user': 'NetdevVhostUserOptions' } }
+    'vhost-user': 'NetdevVhostUserOptions',
+    'replay':   'NetdevReplayOptions' } }
 
 ##
 # @NetLegacy
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 59c3602..ad262d0 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -4,3 +4,4 @@ obj-y += replay-events.o
 obj-y += replay-time.o
 obj-y += replay-icount.o
 obj-y += replay-input.o
+obj-y += replay-net.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
index f8e58ea..9dacb52 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -53,6 +53,9 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_INPUT_SYNC:
         qemu_input_event_sync_impl();
         break;
+    case REPLAY_ASYNC_EVENT_NETWORK:
+        replay_net_send_packet(event->opaque);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n",
                 event->event_kind);
@@ -180,6 +183,9 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_INPUT:
                 replay_save_input_event(event->opaque);
                 break;
+            case REPLAY_ASYNC_EVENT_NETWORK:
+                replay_net_save_packet(event->opaque);
+                break;
             }
         }
 
@@ -240,6 +246,17 @@ void replay_read_events(int opt)
             replay_fetch_data_kind();
             /* continue with the next event */
             continue;
+        case REPLAY_ASYNC_EVENT_NETWORK:
+            e.opaque = replay_net_read_packet();
+            e.event_kind = read_event_kind;
+            replay_run_event(&e);
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+            /* continue with the next event */
+            continue;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 5d0aa71..eb469f0 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -51,7 +51,8 @@
 #define REPLAY_ASYNC_EVENT_THREAD      1
 #define REPLAY_ASYNC_EVENT_INPUT       2
 #define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
-#define REPLAY_ASYNC_COUNT             4
+#define REPLAY_ASYNC_EVENT_NETWORK     4
+#define REPLAY_ASYNC_COUNT             5
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -150,4 +151,22 @@ void replay_save_input_event(InputEvent *evt);
 /*! Reads input event from the log */
 InputEvent *replay_read_input_event(void);
 
+/* Network events */
+
+/*! Initializes network data structures. */
+void replay_net_init(void);
+/*! Cleans up network data structures. */
+void replay_net_free(void);
+/*! Reads packets offsets array from the log. */
+void replay_net_read_packets_data(void);
+/*! Writes packets offsets array to the log. */
+void replay_net_write_packets_data(void);
+/*! Saves network packet into the log. */
+void replay_net_save_packet(void *opaque);
+/*! Reads network packet from the log. */
+void *replay_net_read_packet(void);
+/*! Called to send packet that was read or received from external input
+    to the net queue. */
+void replay_net_send_packet(void *opaque);
+
 #endif
diff --git a/replay/replay-net.c b/replay/replay-net.c
new file mode 100755
index 0000000..0dc891a
--- /dev/null
+++ b/replay/replay-net.c
@@ -0,0 +1,191 @@
+/*
+ * replay-net.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "exec/cpu-common.h"
+#include "net/net.h"
+#include "replay.h"
+#include "replay-internal.h"
+
+/* limited by Ethernet frame size */
+#define MAX_NET_PACKET_SIZE             1560
+
+/* Network data */
+NetClientState **vlan_states = NULL;
+size_t vlan_states_count = 0;
+size_t vlan_states_capacity = 0;
+
+/* Structure for storing information about the network packet */
+typedef struct {
+    /* Offset in the replay log file where packet is saved. */
+    uint64_t file_offset;
+    /* Number of step when packet came. */
+    uint64_t step;
+} QEMU_PACKED NetPacketInfo;
+
+typedef struct NetPacketQueue {
+    /* ID of the packet */
+    uint64_t id;
+    /* ID of the network client */
+    int32_t nc_id;
+    size_t size;
+    uint8_t buf[MAX_NET_PACKET_SIZE];
+    uint64_t offset;
+} NetPacketQueue;
+
+/* Network packets count. */
+static uint64_t net_packets_count;
+/* Capacity of the array for packets parameters. */
+static uint64_t net_packets_capacity;
+/* Array for storing network packets parameters. */
+static NetPacketInfo *net_packets;
+
+
+void replay_net_init(void)
+{
+    net_packets_count = 0;
+}
+
+void replay_net_read_packets_data(void)
+{
+    net_packets_count = replay_get_qword();
+    net_packets_capacity = net_packets_count;
+    if (net_packets_count) {
+        net_packets = (NetPacketInfo *)g_malloc(sizeof(NetPacketInfo)
+                                                * net_packets_count);
+        if (fread(net_packets, sizeof(NetPacketInfo),
+                  net_packets_count, replay_file) != net_packets_count) {
+            fprintf(stderr, "Internal error in replay_net_read_packets_data\n");
+            exit(1);
+        }
+    }
+}
+
+void replay_net_write_packets_data(void)
+{
+    replay_put_qword(net_packets_count);
+    if (net_packets && net_packets_count) {
+        fwrite(net_packets, sizeof(NetPacketInfo),
+               net_packets_count, replay_file);
+    }
+}
+
+void replay_add_network_client(NetClientState *nc)
+{
+    if (vlan_states_count == 0) {
+        vlan_states = (NetClientState **)g_malloc(sizeof(*vlan_states));
+        vlan_states_count = 0;
+        vlan_states_capacity = 1;
+    } else if (vlan_states_count == vlan_states_capacity) {
+        vlan_states_capacity *= 2;
+        vlan_states = (NetClientState **)g_realloc(vlan_states,
+                                                   sizeof(*vlan_states)
+                                                   * vlan_states_capacity);
+    }
+
+    vlan_states[vlan_states_count++] = nc;
+}
+
+void replay_net_free(void)
+{
+    if (vlan_states) {
+        g_free(vlan_states);
+        vlan_states = NULL;
+    }
+}
+
+void replay_save_net_packet(struct NetClientState *nc, const uint8_t *buf,
+                            size_t size)
+{
+    if (replay_file) {
+        if (net_packets_capacity == net_packets_count) {
+            if (net_packets_capacity == 0) {
+                net_packets_capacity = 1;
+            } else {
+                net_packets_capacity *= 2;
+            }
+            net_packets = (NetPacketInfo *)g_realloc(net_packets,
+                                                     net_packets_capacity
+                                                     * sizeof(NetPacketInfo));
+        }
+
+        /* add packet processing event to the queue */
+        NetPacketQueue *p = (NetPacketQueue *)
+                                g_malloc0(sizeof(NetPacketQueue));
+        p->id = net_packets_count;
+        p->size = size;
+        if (net_hub_id_for_client(nc, &p->nc_id) < 0) {
+            fprintf(stderr, "Replay: Cannot determine net client id\n");
+            exit(1);
+        }
+        memcpy(p->buf, buf, size);
+        replay_add_event(REPLAY_ASYNC_EVENT_NETWORK, p);
+
+        ++net_packets_count;
+    }
+}
+
+static NetClientState *replay_net_find_vlan(NetPacketQueue *packet)
+{
+    int i;
+    for (i = 0 ; i < vlan_states_count ; ++i) {
+        int id = 0;
+        if (net_hub_id_for_client(vlan_states[i], &id) < 0) {
+            fprintf(stderr, "Replay: Cannot determine net client id\n");
+            exit(1);
+        }
+        if (id == packet->nc_id) {
+            return vlan_states[i];
+        }
+    }
+
+    fprintf(stderr, "Replay: please specify -net replay command-line option\n");
+    exit(1);
+
+    return NULL;
+}
+
+void replay_net_send_packet(void *opaque)
+{
+    NetPacketQueue *packet = (NetPacketQueue *)opaque;
+    NetClientState *vlan_state = replay_net_find_vlan(packet);
+
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        net_packets[packet->id].file_offset = packet->offset;
+        net_packets[packet->id].step = replay_get_current_step();
+
+        qemu_send_packet(vlan_state, packet->buf, packet->size);
+    } else if (replay_mode == REPLAY_MODE_PLAY) {
+        qemu_send_packet(vlan_state, packet->buf, packet->size);
+    }
+
+    g_free(packet);
+}
+
+void replay_net_save_packet(void *opaque)
+{
+    NetPacketQueue *p = (NetPacketQueue *)opaque;
+    p->offset = ftello64(replay_file);
+    replay_put_qword(p->id);
+    replay_put_dword(p->nc_id);
+    replay_put_array(p->buf, p->size);
+}
+
+void *replay_net_read_packet(void)
+{
+    NetPacketQueue *p = g_malloc0(sizeof(NetPacketQueue));;
+    p->id = replay_get_qword();
+    p->nc_id = replay_get_dword();
+    replay_get_array(p->buf, &p->size);
+    replay_check_error();
+
+    return p;
+}
diff --git a/replay/replay.c b/replay/replay.c
index a9268db..bcd8864 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -462,6 +462,8 @@ static void replay_enable(const char *fname, int mode)
     replay_state.current_step = 0;
     current_saved_state = 0;
 
+    replay_net_init();
+
     /* skip file header for RECORD and check it for PLAY */
     if (replay_mode == REPLAY_MODE_RECORD) {
         fseek(replay_file, HEADER_SIZE, SEEK_SET);
@@ -482,6 +484,7 @@ static void replay_enable(const char *fname, int mode)
             fread(saved_states, sizeof(SavedStateInfo), saved_states_count,
                   replay_file);
         }
+        replay_net_read_packets_data();
         /* go to the beginning */
         fseek(replay_file, 12, SEEK_SET);
     }
@@ -565,6 +568,7 @@ void replay_finish(void)
                 fwrite(saved_states, sizeof(SavedStateInfo),
                        saved_states_count, replay_file);
             }
+            replay_net_write_packets_data();
 
             /* write header */
             fseek(replay_file, 0, SEEK_SET);
@@ -593,5 +597,6 @@ void replay_finish(void)
         replay_image_suffix = NULL;
     }
 
+    replay_net_free();
     replay_finish_events();
 }
diff --git a/replay/replay.h b/replay/replay.h
index 31c02ba..419dec9 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -19,6 +19,7 @@
 
 struct QemuOpts;
 struct InputEvent;
+struct NetClientState;
 
 /* replay clock kinds */
 /* rdtsc */
@@ -119,4 +120,12 @@ void replay_init_icount(void);
 int64_t replay_get_icount(void);
 void replay_clock_warp(void);
 
+/* Network */
+
+/*! Registers net client in the replay module. */
+void replay_add_network_client(struct NetClientState *nc);
+/*! Saves incoming network packet in the replay log. */
+void replay_save_net_packet(struct NetClientState *nc, const uint8_t *buf,
+                            size_t size);
+
 #endif
diff --git a/slirp/slirp.c b/slirp/slirp.c
index 35f819a..877e1b2 100644
--- a/slirp/slirp.c
+++ b/slirp/slirp.c
@@ -26,6 +26,7 @@
 #include "sysemu/char.h"
 #include "slirp.h"
 #include "hw/hw.h"
+#include "replay/replay.h"
 
 /* host loopback address */
 struct in_addr loopback_addr;
@@ -234,8 +235,12 @@ Slirp *slirp_init(int restricted, struct in_addr vnetwork,
 
     slirp->opaque = opaque;
 
-    register_savevm(NULL, "slirp", 0, 3,
-                    slirp_state_save, slirp_state_load, slirp);
+    /* Do not save slirp state in record mode, because slirp device
+       will not be created in replay mode */
+    if (replay_mode == REPLAY_MODE_NONE) {
+        register_savevm(NULL, "slirp", 0, 3,
+                        slirp_state_save, slirp_state_load, slirp);
+    }
 
     QTAILQ_INSERT_TAIL(&slirp_instances, slirp, entry);
 

[Qemu-devel] [RFC PATCH v3 43/49] replay: audio data record/replay

[Pavel Dovgalyuk]

This patch adds deterministic replay for audio adapter. Replay module saves
data from the microphone and "end-of-playback" events.
Support of audio record and replay is implemented only for Win32 hosts.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 audio/audio.c            |   14 ++-
 audio/audio_win_int.h    |    3 +
 audio/winwaveaudio.c     |  167 ++++++++++++++++++++++++++--------
 replay/Makefile.objs     |    1 
 replay/replay-audio.c    |  228 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.c |    4 +
 replay/replay-internal.h |   14 +++
 replay/replay.h          |   21 ++++
 8 files changed, 409 insertions(+), 43 deletions(-)
 create mode 100755 replay/replay-audio.c

diff --git a/audio/audio.c b/audio/audio.c
index 9d018e9..72d7a8a 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -26,6 +26,7 @@
 #include "monitor/monitor.h"
 #include "qemu/timer.h"
 #include "sysemu/sysemu.h"
+#include "replay/replay.h"
 
 #define AUDIO_CAP "audio"
 #include "audio_int.h"
@@ -1201,7 +1202,9 @@ void AUD_set_active_out (SWVoiceOut *sw, int on)
             if (!hw->enabled) {
                 hw->enabled = 1;
                 if (s->vm_running) {
-                    hw->pcm_ops->ctl_out (hw, VOICE_ENABLE, conf.try_poll_out);
+                    hw->pcm_ops->ctl_out(hw, VOICE_ENABLE,
+                                         replay_mode == REPLAY_MODE_NONE
+                                             ? conf.try_poll_out : 0);
                     audio_reset_timer (s);
                 }
             }
@@ -1763,11 +1766,13 @@ static void audio_vm_change_state_handler (void *opaque, int running,
 
     s->vm_running = running;
     while ((hwo = audio_pcm_hw_find_any_enabled_out (hwo))) {
-        hwo->pcm_ops->ctl_out (hwo, op, conf.try_poll_out);
+        hwo->pcm_ops->ctl_out(hwo, op, replay_mode == REPLAY_MODE_NONE
+                                           ? conf.try_poll_out : 0);
     }
 
     while ((hwi = audio_pcm_hw_find_any_enabled_in (hwi))) {
-        hwi->pcm_ops->ctl_in (hwi, op, conf.try_poll_in);
+        hwi->pcm_ops->ctl_in(hwi, op, replay_mode == REPLAY_MODE_NONE
+                                          ? conf.try_poll_in : 0);
     }
     audio_reset_timer (s);
 }
@@ -1810,9 +1815,10 @@ static void audio_atexit (void)
 
 static const VMStateDescription vmstate_audio = {
     .name = "audio",
-    .version_id = 1,
+    .version_id = 2,
     .minimum_version_id = 1,
     .fields = (VMStateField[]) {
+        VMSTATE_TIMER_V(ts, AudioState, 2),
         VMSTATE_END_OF_LIST()
     }
 };
diff --git a/audio/audio_win_int.h b/audio/audio_win_int.h
index fa5b3cb..0525ae6 100644
--- a/audio/audio_win_int.h
+++ b/audio/audio_win_int.h
@@ -7,4 +7,7 @@ int waveformat_from_audio_settings (WAVEFORMATEX *wfx,
 int waveformat_to_audio_settings (WAVEFORMATEX *wfx,
                                   struct audsettings *as);
 
+void winwave_callback_out_impl(void *dwInstance, WAVEHDR *h);
+void winwave_callback_in_impl(void *dwInstance, WAVEHDR *h);
+
 #endif /* AUDIO_WIN_INT_H */
diff --git a/audio/winwaveaudio.c b/audio/winwaveaudio.c
index 8dbd145..40b9a89 100644
--- a/audio/winwaveaudio.c
+++ b/audio/winwaveaudio.c
@@ -2,7 +2,9 @@
 
 #include "qemu-common.h"
 #include "sysemu/sysemu.h"
+#include "migration/vmstate.h"
 #include "audio.h"
+#include "replay/replay.h"
 
 #define AUDIO_CAP "winwave"
 #include "audio_int.h"
@@ -47,6 +49,7 @@ typedef struct {
     int paused;
     int rpos;
     int avail;
+    int non_added;
     CRITICAL_SECTION crit_sect;
 } WaveVoiceIn;
 
@@ -124,6 +127,24 @@ static void winwave_anal_close_out (WaveVoiceOut *wave)
     wave->hwo = NULL;
 }
 
+void winwave_callback_out_impl(void *dwInstance, WAVEHDR *h)
+{
+    WaveVoiceOut *wave = (WaveVoiceOut *)dwInstance;
+    if (!h->dwUser) {
+        h->dwUser = 1;
+        EnterCriticalSection(&wave->crit_sect);
+        {
+            wave->avail += conf.dac_samples;
+        }
+        LeaveCriticalSection(&wave->crit_sect);
+        if (wave->hw.poll_mode) {
+            if (!SetEvent(wave->event)) {
+                dolog("DAC SetEvent failed %lx\n", GetLastError());
+            }
+        }
+    }
+}
+
 static void CALLBACK winwave_callback_out (
     HWAVEOUT hwo,
     UINT msg,
@@ -132,24 +153,16 @@ static void CALLBACK winwave_callback_out (
     DWORD_PTR dwParam2
     )
 {
-    WaveVoiceOut *wave = (WaveVoiceOut *) dwInstance;
-
     switch (msg) {
     case WOM_DONE:
         {
-            WAVEHDR *h = (WAVEHDR *) dwParam1;
-            if (!h->dwUser) {
-                h->dwUser = 1;
-                EnterCriticalSection (&wave->crit_sect);
-                {
-                    wave->avail += conf.dac_samples;
-                }
-                LeaveCriticalSection (&wave->crit_sect);
-                if (wave->hw.poll_mode) {
-                    if (!SetEvent (wave->event)) {
-                        dolog ("DAC SetEvent failed %lx\n", GetLastError ());
-                    }
-                }
+            if (replay_mode == REPLAY_MODE_RECORD) {
+                replay_sound_out_event((WAVEHDR *)dwParam1);
+            } else if (replay_mode == REPLAY_MODE_PLAY) {
+                /* Do nothing */
+            } else {
+                winwave_callback_out_impl((void *)dwInstance,
+                                          (WAVEHDR *)dwParam1);
             }
         }
         break;
@@ -163,6 +176,21 @@ static void CALLBACK winwave_callback_out (
     }
 }
 
+static const VMStateDescription vmstate_audio_out = {
+    .name = "audio_out",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_INT32(enabled, HWVoiceOut),
+        VMSTATE_INT32(poll_mode, HWVoiceOut),
+        VMSTATE_INT32(pending_disable, HWVoiceOut),
+        VMSTATE_INT32(rpos, HWVoiceOut),
+        VMSTATE_UINT64(ts_helper, HWVoiceOut),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static int winwave_init_out (HWVoiceOut *hw, struct audsettings *as)
 {
     int i;
@@ -173,6 +201,8 @@ static int winwave_init_out (HWVoiceOut *hw, struct audsettings *as)
 
     wave = (WaveVoiceOut *) hw;
 
+    vmstate_register(NULL, 0, &vmstate_audio_out, hw);
+
     InitializeCriticalSection (&wave->crit_sect);
 
     err = waveformat_from_audio_settings (&wfx, as);
@@ -219,6 +249,10 @@ static int winwave_init_out (HWVoiceOut *hw, struct audsettings *as)
         }
     }
 
+    if (replay_mode != REPLAY_MODE_NONE) {
+        replay_init_sound_out(wave, wave->hdrs, conf.dac_headers);
+    }
+
     return 0;
 
  err4:
@@ -262,10 +296,20 @@ static int winwave_run_out (HWVoiceOut *hw, int live)
         WAVEHDR *h = &wave->hdrs[wave->curhdr];
 
         h->dwUser = 0;
-        mr = waveOutWrite (wave->hwo, h, sizeof (*h));
-        if (mr != MMSYSERR_NOERROR) {
-            winwave_logerr (mr, "waveOutWrite(%d)", wave->curhdr);
-            break;
+        /* Only events will be simulated in REPLAY_PLAY mode,
+           no sounds will be emitted */
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            mr = waveOutWrite(wave->hwo, h, sizeof(*h));
+            if (mr != MMSYSERR_NOERROR) {
+                winwave_logerr(mr, "waveOutWrite(%d)", wave->curhdr);
+                break;
+            }
+        } else {
+            /* This header will be passed to callback function,
+               when event will be read in the log */
+            if (replay_sound_out_event(h)) {
+                break;
+            }
         }
 
         wave->pending -= conf.dac_samples;
@@ -382,6 +426,26 @@ static void winwave_anal_close_in (WaveVoiceIn *wave)
     wave->hwi = NULL;
 }
 
+void winwave_callback_in_impl(void *dwInstance, WAVEHDR *h)
+{
+    WaveVoiceIn *wave = (WaveVoiceIn *)dwInstance;
+
+    if (!h->dwUser) {
+        h->dwUser = 1;
+        EnterCriticalSection(&wave->crit_sect);
+        {
+            wave->avail += conf.adc_samples;
+        }
+        LeaveCriticalSection(&wave->crit_sect);
+
+        if (wave->hw.poll_mode) {
+            if (!SetEvent(wave->event)) {
+                dolog("ADC SetEvent failed %lx\n", GetLastError());
+            }
+        }
+    }
+}
+
 static void CALLBACK winwave_callback_in (
     HWAVEIN *hwi,
     UINT msg,
@@ -390,24 +454,16 @@ static void CALLBACK winwave_callback_in (
     DWORD_PTR dwParam2
     )
 {
-    WaveVoiceIn *wave = (WaveVoiceIn *) dwInstance;
-
     switch (msg) {
     case WIM_DATA:
         {
-            WAVEHDR *h = (WAVEHDR *) dwParam1;
-            if (!h->dwUser) {
-                h->dwUser = 1;
-                EnterCriticalSection (&wave->crit_sect);
-                {
-                    wave->avail += conf.adc_samples;
-                }
-                LeaveCriticalSection (&wave->crit_sect);
-                if (wave->hw.poll_mode) {
-                    if (!SetEvent (wave->event)) {
-                        dolog ("ADC SetEvent failed %lx\n", GetLastError ());
-                    }
-                }
+            if (replay_mode == REPLAY_MODE_RECORD) {
+                replay_sound_in_event((WAVEHDR *)dwParam1);
+            } else if (replay_mode == REPLAY_MODE_PLAY) {
+                /* Do nothing */
+            } else {
+                winwave_callback_in_impl((void *)dwInstance,
+                                         (WAVEHDR *)dwParam1);
             }
         }
         break;
@@ -421,9 +477,10 @@ static void CALLBACK winwave_callback_in (
     }
 }
 
-static void winwave_add_buffers (WaveVoiceIn *wave, int samples)
+static int winwave_add_buffers(WaveVoiceIn *wave, int samples)
 {
     int doreset;
+    int added = 0;
 
     doreset = wave->hw.poll_mode && (samples >= conf.adc_samples);
     if (doreset && !ResetEvent (wave->event)) {
@@ -435,15 +492,39 @@ static void winwave_add_buffers (WaveVoiceIn *wave, int samples)
         WAVEHDR *h = &wave->hdrs[wave->curhdr];
 
         h->dwUser = 0;
-        mr = waveInAddBuffer (wave->hwi, h, sizeof (*h));
-        if (mr != MMSYSERR_NOERROR) {
-            winwave_logerr (mr, "waveInAddBuffer(%d)", wave->curhdr);
+        /* Add buffer to replay queue in replay mode - it will be
+           loaded from log file. */
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            mr = waveInAddBuffer(wave->hwi, h, sizeof(*h));
+            if (mr != MMSYSERR_NOERROR) {
+                winwave_logerr(mr, "waveInAddBuffer(%d)", wave->curhdr);
+            }
+        } else {
+            replay_sound_in_event(h);
         }
         wave->curhdr = (wave->curhdr + 1) % conf.adc_headers;
         samples -= conf.adc_samples;
+        added += conf.adc_samples;
     }
+
+    return added;
 }
 
+static const VMStateDescription vmstate_audio_in = {
+    .name = "audio_in",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields      = (VMStateField[]) {
+        VMSTATE_INT32(enabled, HWVoiceIn),
+        VMSTATE_INT32(poll_mode, HWVoiceIn),
+        VMSTATE_INT32(wpos, HWVoiceIn),
+        VMSTATE_INT32(total_samples_captured, HWVoiceIn),
+        VMSTATE_UINT64(ts_helper, HWVoiceIn),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
 {
     int i;
@@ -454,6 +535,8 @@ static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
 
     wave = (WaveVoiceIn *) hw;
 
+    vmstate_register(NULL, 0, &vmstate_audio_in, hw);
+
     InitializeCriticalSection (&wave->crit_sect);
 
     err = waveformat_from_audio_settings (&wfx, as);
@@ -478,6 +561,7 @@ static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
     audio_pcm_init_info (&hw->info, as);
     hw->samples = conf.adc_samples * conf.adc_headers;
     wave->avail = 0;
+    wave->non_added = 0;
 
     wave->pcm_buf = audio_calloc (AUDIO_FUNC, conf.adc_samples,
                                   conf.adc_headers << hw->info.shift);
@@ -500,6 +584,10 @@ static int winwave_init_in (HWVoiceIn *hw, struct audsettings *as)
         }
     }
 
+    if (replay_mode != REPLAY_MODE_NONE) {
+        replay_init_sound_in(wave, wave->hdrs, conf.adc_headers);
+    }
+
     wave->paused = 1;
     winwave_add_buffers (wave, hw->samples);
     return 0;
@@ -570,6 +658,7 @@ static int winwave_run_in (HWVoiceIn *hw)
     LeaveCriticalSection (&wave->crit_sect);
 
     ret = decr;
+    wave->non_added += ret;
     while (decr) {
         int left = hw->samples - hw->wpos;
         int conv = audio_MIN (left, decr);
@@ -582,7 +671,7 @@ static int winwave_run_in (HWVoiceIn *hw)
         decr -= conv;
     }
 
-    winwave_add_buffers (wave, ret);
+    wave->non_added -= winwave_add_buffers(wave, wave->non_added);
     return ret;
 }
 
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index ad262d0..9fea604 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -5,3 +5,4 @@ obj-y += replay-time.o
 obj-y += replay-icount.o
 obj-y += replay-input.o
 obj-y += replay-net.o
+obj-y += replay-audio.o
diff --git a/replay/replay-audio.c b/replay/replay-audio.c
new file mode 100755
index 0000000..b6bf1a1
--- /dev/null
+++ b/replay/replay-audio.c
@@ -0,0 +1,228 @@
+/*
+ * replay-audio.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "exec/cpu-common.h"
+#include "replay.h"
+#include "replay-internal.h"
+#ifdef _WIN32
+struct audsettings;
+#include "audio/audio_win_int.h"
+#endif
+
+/* Sound card state */
+typedef struct {
+    void *instance;
+    const int event_id;
+#ifdef _WIN32
+    WAVEHDR *queue;
+#endif
+    /*! Maximum size of the queue */
+    int size;
+    /*! Current size of the queue */
+    sig_atomic_t cur_size;
+    unsigned int head, tail;
+} SoundQueue;
+
+
+static SoundQueue sound_in = {
+        .event_id = EVENT_SOUND_IN
+    },
+    sound_out = {
+        .event_id = EVENT_SOUND_OUT,
+    };
+
+#ifdef _WIN32
+/*! Spinlock for sound events processing. */
+static spinlock_t sound_lock = SPIN_LOCK_UNLOCKED;
+#endif
+
+/*****************************************************************************
+ *  Sound queue functions                                                    *
+ *****************************************************************************/
+
+/* callback functions */
+#ifdef _WIN32
+
+void replay_init_sound_in(void *instance, WAVEHDR *hdrs, int sz)
+{
+    sound_in.instance = instance;
+    sound_in.queue = hdrs;
+    sound_in.size = sz;
+    sound_in.head = 0;
+    sound_in.tail = 0;
+    sound_in.cur_size = 0;
+}
+
+void replay_init_sound_out(void *instance, WAVEHDR *hdrs, int sz)
+{
+    sound_out.instance = instance;
+    sound_out.queue = hdrs;
+    sound_out.size = sz;
+    sound_out.head = 0;
+    sound_out.tail = 0;
+    sound_out.cur_size = 0;
+}
+
+static int sound_queue_add(SoundQueue *q, WAVEHDR *hdr)
+{
+    if (q->queue + q->tail != hdr) {
+        /* state was loaded and we need to reset the queue */
+        if (q->cur_size == 0) {
+            q->head = q->tail = hdr - q->queue;
+        } else {
+            fprintf(stderr, "Replay: Sound queue error\n");
+            exit(1);
+        }
+    }
+
+    if (q->cur_size == q->size) {
+        if (replay_mode == REPLAY_MODE_PLAY) {
+            return 1;
+        }
+
+        fprintf(stderr, "Replay: Sound queue overflow\n");
+        exit(1);
+    }
+
+    q->tail = (q->tail + 1) % q->size;
+    ++q->cur_size;
+
+    return 0;
+}
+
+void replay_save_sound_out(void)
+{
+    spin_lock(&sound_lock);
+    while (sound_out.cur_size != 0) {
+        /* put the message ID */
+        replay_put_event(sound_out.event_id);
+        /* save the buffer size */
+        replay_put_dword(sound_out.queue[sound_out.head].dwBytesRecorded);
+        /* perform winwave-specific actions */
+        winwave_callback_out_impl(sound_out.instance,
+                                  &sound_out.queue[sound_out.head]);
+        /* goto the next buffer */
+        sound_out.head = (sound_out.head + 1) % sound_out.size;
+        --sound_out.cur_size;
+    }
+    spin_unlock(&sound_lock);
+}
+
+void replay_save_sound_in(void)
+{
+    spin_lock(&sound_lock);
+    while (sound_in.cur_size != 0) {
+        /* put the message ID */
+        replay_put_event(sound_in.event_id);
+        /* save the buffer */
+        replay_put_array((const uint8_t *)sound_in.queue[sound_in.head].lpData,
+                         sound_in.queue[sound_in.head].dwBytesRecorded);
+        /* perform winwave-specific actions */
+        winwave_callback_in_impl(sound_in.instance,
+                                 &sound_in.queue[sound_in.head]);
+        /* goto the next buffer */
+        sound_in.head = (sound_in.head + 1) % sound_in.size;
+        --sound_in.cur_size;
+    }
+    spin_unlock(&sound_lock);
+}
+
+void replay_read_sound_out(void)
+{
+    if (sound_out.cur_size == 0) {
+        fprintf(stderr, "Replay: Sound queue underflow\n");
+        exit(1);
+    }
+
+    /* get the buffer size */
+    sound_out.queue[sound_out.head].dwBytesRecorded = replay_get_dword();
+
+    replay_check_error();
+    replay_has_unread_data = 0;
+
+    /* perform winwave-specific actions */
+    winwave_callback_out_impl(sound_out.instance,
+                              &sound_out.queue[sound_out.head]);
+    sound_out.head = (sound_out.head + 1) % sound_out.size;
+    --sound_out.cur_size;
+}
+
+void replay_read_sound_in(void)
+{
+    if (sound_in.cur_size == 0) {
+        fprintf(stderr, "Replay: Sound queue underflow\n");
+        exit(1);
+    }
+
+    /* get the buffer size */
+    size_t size;
+    replay_get_array((uint8_t *)sound_in.queue[sound_in.head].lpData, &size);
+    sound_in.queue[sound_in.head].dwBytesRecorded = (unsigned int)size;
+
+    replay_check_error();
+    replay_has_unread_data = 0;
+
+    /* perform winwave-specific actions */
+    winwave_callback_in_impl(sound_in.instance, &sound_in.queue[sound_in.head]);
+    sound_in.head = (sound_in.head + 1) % sound_in.size;
+    --sound_in.cur_size;
+}
+
+void replay_sound_in_event(WAVEHDR *hdr)
+{
+    spin_lock(&sound_lock);
+    if (sound_queue_add(&sound_in, hdr)) {
+        fprintf(stderr, "Replay: Input sound buffer overflow\n");
+        exit(1);
+    }
+    spin_unlock(&sound_lock);
+}
+
+int replay_sound_out_event(WAVEHDR *hdr)
+{
+    spin_lock(&sound_lock);
+    int result = sound_queue_add(&sound_out, hdr);
+    spin_unlock(&sound_lock);
+
+    return result;
+}
+#endif
+
+bool replay_has_sound_events(void)
+{
+    return sound_in.cur_size || sound_out.cur_size;
+}
+
+void replay_sound_flush_queue(void)
+{
+#ifdef _WIN32
+    spin_lock(&sound_lock);
+    while (sound_out.cur_size != 0) {
+        /* perform winwave-specific actions */
+        winwave_callback_out_impl(sound_out.instance,
+                                  &sound_out.queue[sound_out.head]);
+        /* goto the next buffer */
+        sound_out.head = (sound_out.head + 1) % sound_out.size;
+        --sound_out.cur_size;
+    }
+    while (sound_in.cur_size != 0) {
+        /* perform winwave-specific actions */
+        winwave_callback_in_impl(sound_in.instance,
+                                 &sound_in.queue[sound_in.head]);
+        /* goto the next buffer */
+        sound_in.head = (sound_in.head + 1) % sound_in.size;
+        --sound_in.cur_size;
+    }
+    spin_unlock(&sound_lock);
+#endif
+}
+
diff --git a/replay/replay-internal.c b/replay/replay-internal.c
index 0c303d6..b35ec57 100755
--- a/replay/replay-internal.c
+++ b/replay/replay-internal.c
@@ -151,5 +151,9 @@ void replay_save_instructions(void)
             replay_state.current_step += first_cpu->instructions_count;
             first_cpu->instructions_count = 0;
         }
+#ifdef _WIN32
+        replay_save_sound_in();
+        replay_save_sound_out();
+#endif
     }
 }
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index eb469f0..d2b1936 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -21,6 +21,10 @@
 #define EVENT_TIME_T                1
 /* for tm event */
 #define EVENT_TM                    2
+/* for outgoing sound event */
+#define EVENT_SOUND_OUT             7
+/* for incoming sound event */
+#define EVENT_SOUND_IN              8
 /* for software interrupt */
 #define EVENT_INTERRUPT             15
 /* for shutdown request */
@@ -169,4 +173,14 @@ void *replay_net_read_packet(void);
     to the net queue. */
 void replay_net_send_packet(void *opaque);
 
+/* Sound events */
+
+/*! Returns true, when there are any pending sound events. */
+bool replay_has_sound_events(void);
+void replay_save_sound_out(void);
+void replay_save_sound_in(void);
+void replay_read_sound_out(void);
+void replay_read_sound_in(void);
+void replay_sound_flush_queue(void);
+
 #endif
diff --git a/replay/replay.h b/replay/replay.h
index 419dec9..1d32a2a 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -15,6 +15,10 @@
 #include <stdbool.h>
 #include <stdint.h>
 #include <time.h>
+#ifdef _WIN32
+#include <windows.h>
+#include <mmsystem.h>
+#endif
 #include "qapi-types.h"
 
 struct QemuOpts;
@@ -128,4 +132,21 @@ void replay_add_network_client(struct NetClientState *nc);
 void replay_save_net_packet(struct NetClientState *nc, const uint8_t *buf,
                             size_t size);
 
+/* Audio */
+
+#ifdef _WIN32
+/*! Microphone event. */
+void replay_sound_in_event(WAVEHDR *hdr);
+/*! Adds header to the queue.
+    In record mode this header is queued for saving into log.
+    In replay mode this header is queued for reading from log.
+    Returns 1 in replay mode when queue is full.
+    Otherwise returns 0. */
+int replay_sound_out_event(WAVEHDR *hdr);
+/*! Initializes queue for sound input. */
+void replay_init_sound_in(void *instance, WAVEHDR *hdrs, int sz);
+/*! Initializes queue for sound output. */
+void replay_init_sound_out(void *instance, WAVEHDR *hdrs, int sz);
+#endif
+
 #endif

[Qemu-devel] [RFC PATCH v3 44/49] replay: serial port

[Pavel Dovgalyuk]

This patch implements record and replay of serial ports.
In record mode serial port can be connected to the source of the data:
-serial tcp:127.0.0.1:23,nowait,server
In replay mode it should not be connected to any data source, but should be
initialized with null: -serial null

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 include/sysemu/char.h    |   25 ++++++++++++
 qemu-char.c              |   56 +++++++++++++++++++++++---
 replay/Makefile.objs     |    1 
 replay/replay-char.c     |  100 ++++++++++++++++++++++++++++++++++++++++++++++
 replay/replay-events.c   |   18 ++++++++
 replay/replay-internal.h |   14 ++++++
 replay/replay.c          |   14 ++++++
 replay/replay.h          |   13 ++++++
 vl.c                     |    4 +-
 9 files changed, 238 insertions(+), 7 deletions(-)
 create mode 100755 replay/replay-char.c

diff --git a/include/sysemu/char.h b/include/sysemu/char.h
index 0bbd631..a7eb578 100644
--- a/include/sysemu/char.h
+++ b/include/sysemu/char.h
@@ -85,6 +85,7 @@ struct CharDriverState {
     int is_mux;
     guint fd_in_tag;
     QemuOpts *opts;
+    bool replay;
     QTAILQ_ENTRY(CharDriverState) next;
 };
 
@@ -126,6 +127,21 @@ CharDriverState *qemu_chr_new(const char *label, const char *filename,
                               void (*init)(struct CharDriverState *s));
 
 /**
+ * @qemu_chr_new_replay:
+ *
+ * Create a new character backend from a URI.
+ * All recieved data will be logged by replay module.
+ *
+ * @label the name of the backend
+ * @filename the URI
+ * @init not sure..
+ *
+ * Returns: a new character backend
+ */
+CharDriverState *qemu_chr_new_replay(const char *label, const char *filename,
+                                     void (*init)(struct CharDriverState *s));
+
+/**
  * @qemu_chr_delete:
  *
  * Destroy a character backend.
@@ -320,6 +336,15 @@ int qemu_chr_be_can_write(CharDriverState *s);
  */
 void qemu_chr_be_write(CharDriverState *s, uint8_t *buf, int len);
 
+/**
+ * @qemu_chr_be_write_impl:
+ *
+ * Implementation of back end writing. Used by replay module.
+ *
+ * @buf a buffer to receive data from the front end
+ * @len the number of bytes to receive from the front end
+ */
+void qemu_chr_be_write_impl(CharDriverState *s, uint8_t *buf, int len);
 
 /**
  * @qemu_chr_be_event:
diff --git a/qemu-char.c b/qemu-char.c
index 956be49..046e88b 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -28,6 +28,7 @@
 #include "sysemu/char.h"
 #include "hw/usb.h"
 #include "qmp-commands.h"
+#include "replay/replay.h"
 
 #include <unistd.h>
 #include <fcntl.h>
@@ -126,6 +127,9 @@ int qemu_chr_fe_write(CharDriverState *s, const uint8_t *buf, int len)
 
     qemu_mutex_lock(&s->chr_write_lock);
     ret = s->chr_write(s, buf, len);
+    if (s->replay) {
+        replay_data_int(&ret);
+    }
     qemu_mutex_unlock(&s->chr_write_lock);
     return ret;
 }
@@ -195,9 +199,19 @@ int qemu_chr_fe_read_all(CharDriverState *s, uint8_t *buf, int len)
 
 int qemu_chr_fe_ioctl(CharDriverState *s, int cmd, void *arg)
 {
-    if (!s->chr_ioctl)
-        return -ENOTSUP;
-    return s->chr_ioctl(s, cmd, arg);
+    int res;
+    if (!s->chr_ioctl) {
+        res = -ENOTSUP;
+    } else {
+        res = s->chr_ioctl(s, cmd, arg);
+        if (s->replay) {
+            fprintf(stderr,
+                    "Replay: ioctl is not supported for serial devices yet\n");
+            exit(1);
+        }
+    }
+
+    return res;
 }
 
 int qemu_chr_be_can_write(CharDriverState *s)
@@ -207,17 +221,34 @@ int qemu_chr_be_can_write(CharDriverState *s)
     return s->chr_can_read(s->handler_opaque);
 }
 
-void qemu_chr_be_write(CharDriverState *s, uint8_t *buf, int len)
+void qemu_chr_be_write_impl(CharDriverState *s, uint8_t *buf, int len)
 {
     if (s->chr_read) {
         s->chr_read(s->handler_opaque, buf, len);
     }
 }
 
+void qemu_chr_be_write(CharDriverState *s, uint8_t *buf, int len)
+{
+    if (s->replay) {
+        if (replay_mode == REPLAY_MODE_PLAY) {
+            fprintf(stderr, "Replay: calling qemu_chr_be_write in play mode\n");
+            exit(1);
+        }
+        replay_chr_be_write(s, buf, len);
+    } else {
+        qemu_chr_be_write_impl(s, buf, len);
+    }
+}
+
 int qemu_chr_fe_get_msgfd(CharDriverState *s)
 {
     int fd;
-    return (qemu_chr_fe_get_msgfds(s, &fd, 1) == 1) ? fd : -1;
+    int res = (qemu_chr_fe_get_msgfds(s, &fd, 1) == 1) ? fd : -1;
+    if (s->replay) {
+        replay_data_int(&res);
+    }
+    return res;
 }
 
 int qemu_chr_fe_get_msgfds(CharDriverState *s, int *fds, int len)
@@ -3631,6 +3662,21 @@ CharDriverState *qemu_chr_new(const char *label, const char *filename, void (*in
     return chr;
 }
 
+CharDriverState *qemu_chr_new_replay(const char *label, const char *filename,
+                                     void (*init)(struct CharDriverState *s))
+{
+    if (replay_mode == REPLAY_MODE_PLAY && (strcmp(filename, "null")
+        && strcmp(filename, "vc:80Cx24C"))) {
+        fprintf(stderr, "Only \"-serial null\" can be used with replay\n");
+        exit(1);
+    }
+    CharDriverState *chr = qemu_chr_new(label, filename, init);
+    if (strcmp(filename, "vc:80Cx24C")) {
+        replay_register_char_driver(chr);
+    }
+    return chr;
+}
+
 void qemu_chr_fe_set_echo(struct CharDriverState *chr, bool echo)
 {
     if (chr->chr_set_echo) {
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 9fea604..c528bcb 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -6,3 +6,4 @@ obj-y += replay-icount.o
 obj-y += replay-input.o
 obj-y += replay-net.o
 obj-y += replay-audio.o
+obj-y += replay-char.o
diff --git a/replay/replay-char.c b/replay/replay-char.c
new file mode 100755
index 0000000..1f25eae
--- /dev/null
+++ b/replay/replay-char.c
@@ -0,0 +1,100 @@
+/*
+ * replay-char.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "replay.h"
+#include "replay-internal.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/char.h"
+
+#define MAX_CHAR_DRIVERS MAX_SERIAL_PORTS
+/* Char drivers that generate qemu_chr_be_write events
+   that should be saved into the log. */
+static CharDriverState *char_drivers[MAX_CHAR_DRIVERS];
+
+/* Char event attributes. */
+typedef struct CharEvent {
+    int id;
+    uint8_t *buf;
+    size_t len;
+} CharEvent;
+
+static int find_char_driver(CharDriverState *chr)
+{
+    int i = 0;
+    while (i < MAX_CHAR_DRIVERS && char_drivers[i] != chr) {
+        ++i;
+    }
+
+    return i >= MAX_CHAR_DRIVERS ? -1 : i;
+}
+
+
+void replay_register_char_driver(CharDriverState *chr)
+{
+    chr->replay = true;
+    int i = find_char_driver(NULL);
+
+    if (i < 0) {
+        fprintf(stderr, "Replay: cannot register char driver\n");
+        exit(1);
+    } else {
+        char_drivers[i] = chr;
+    }
+}
+
+void replay_chr_be_write(CharDriverState *s, uint8_t *buf, int len)
+{
+    CharEvent *event = g_malloc0(sizeof(CharEvent));
+
+    event->id = find_char_driver(s);
+    if (event->id < 0) {
+        fprintf(stderr, "Replay: cannot find char driver\n");
+        exit(1);
+    }
+    event->buf = g_malloc(len);
+    memcpy(event->buf, buf, len);
+    event->len = len;
+
+    replay_add_event(REPLAY_ASYNC_EVENT_CHAR, event);
+}
+
+void replay_event_char_run(void *opaque)
+{
+    CharEvent *event = (CharEvent *)opaque;
+
+    qemu_chr_be_write_impl(char_drivers[event->id], event->buf,
+                           (int)event->len);
+
+    g_free(event->buf);
+    g_free(event);
+}
+
+void replay_event_char_save(void *opaque)
+{
+    CharEvent *event = (CharEvent *)opaque;
+
+    replay_put_byte(event->id);
+    replay_put_array(event->buf, event->len);
+}
+
+void *replay_event_char_read(void)
+{
+    CharEvent *event = g_malloc0(sizeof(CharEvent));
+
+    event->id = replay_get_byte();
+    replay_get_array_alloc(&event->buf, &event->len);
+
+    return event;
+}
diff --git a/replay/replay-events.c b/replay/replay-events.c
index 9dacb52..362fef8 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -56,6 +56,9 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_NETWORK:
         replay_net_send_packet(event->opaque);
         break;
+    case REPLAY_ASYNC_EVENT_CHAR:
+        replay_event_char_run(event->opaque);
+        break;
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n",
                 event->event_kind);
@@ -186,6 +189,9 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_NETWORK:
                 replay_net_save_packet(event->opaque);
                 break;
+            case REPLAY_ASYNC_EVENT_CHAR:
+                replay_event_char_save(event->opaque);
+                break;
             }
         }
 
@@ -257,6 +263,18 @@ void replay_read_events(int opt)
             replay_fetch_data_kind();
             /* continue with the next event */
             continue;
+        case REPLAY_ASYNC_EVENT_CHAR:
+            e.event_kind = read_event_kind;
+            e.opaque = replay_event_char_read();
+
+            replay_has_unread_data = 0;
+            read_event_kind = -1;
+            read_opt = -1;
+            replay_fetch_data_kind();
+
+            replay_run_event(&e);
+            /* continue with the next event */
+            continue;
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index d2b1936..9bbfcde 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -38,6 +38,8 @@
 /* for async events */
 #define EVENT_ASYNC                 24
 #define EVENT_ASYNC_OPT             25
+/* for int data */
+#define EVENT_DATA_INT              26
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
 /* for clock read/writes */
@@ -56,7 +58,8 @@
 #define REPLAY_ASYNC_EVENT_INPUT       2
 #define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
 #define REPLAY_ASYNC_EVENT_NETWORK     4
-#define REPLAY_ASYNC_COUNT             5
+#define REPLAY_ASYNC_EVENT_CHAR        5
+#define REPLAY_ASYNC_COUNT             6
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -183,4 +186,13 @@ void replay_read_sound_out(void);
 void replay_read_sound_in(void);
 void replay_sound_flush_queue(void);
 
+/* Character devices */
+
+/*! Called to run char device event. */
+void replay_event_char_run(void *opaque);
+/*! Writes char event to the file. */
+void replay_event_char_save(void *opaque);
+/*! Reads char event from the file. */
+void *replay_event_char_read(void);
+
 #endif
diff --git a/replay/replay.c b/replay/replay.c
index bcd8864..7a16da3 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -600,3 +600,17 @@ void replay_finish(void)
     replay_net_free();
     replay_finish_events();
 }
+
+void replay_data_int(int *data)
+{
+    if (replay_file && replay_mode == REPLAY_MODE_PLAY) {
+        skip_async_events_until(EVENT_DATA_INT);
+        *data = replay_get_dword();
+        replay_check_error();
+        replay_has_unread_data = 0;
+    } else if (replay_file && replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_instructions();
+        replay_put_event(EVENT_DATA_INT);
+        replay_put_dword(*data);
+    }
+}
diff --git a/replay/replay.h b/replay/replay.h
index 1d32a2a..e8fd20c 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -24,6 +24,7 @@
 struct QemuOpts;
 struct InputEvent;
 struct NetClientState;
+struct CharDriverState;
 
 /* replay clock kinds */
 /* rdtsc */
@@ -149,4 +150,16 @@ void replay_init_sound_in(void *instance, WAVEHDR *hdrs, int sz);
 void replay_init_sound_out(void *instance, WAVEHDR *hdrs, int sz);
 #endif
 
+/* Character device */
+
+/*! Registers char driver to save it's events */
+void replay_register_char_driver(struct CharDriverState *chr);
+/*! Saves write to char device event to the log */
+void replay_chr_be_write(struct CharDriverState *s, uint8_t *buf, int len);
+
+/* Other data */
+
+/*! Writes or reads integer value to/from replay log. */
+void replay_data_int(int *data);
+
 #endif
diff --git a/vl.c b/vl.c
index b0127a2..52fffab 100644
--- a/vl.c
+++ b/vl.c
@@ -2569,7 +2569,9 @@ static int serial_parse(const char *devname)
         exit(1);
     }
     snprintf(label, sizeof(label), "serial%d", index);
-    serial_hds[index] = qemu_chr_new(label, devname, NULL);
+    serial_hds[index] = (replay_mode == REPLAY_MODE_NONE ? qemu_chr_new
+                                                         : qemu_chr_new_replay)
+                        (label, devname, NULL);
     if (!serial_hds[index]) {
         fprintf(stderr, "qemu: could not connect serial device"
                 " to character backend '%s'\n", devname);

[Qemu-devel] [RFC PATCH v3 45/49] replay: USB passthrough

[Pavel Dovgalyuk]

It writes all external data, returned by libusb,
to the log. This data is read in replay mode instead of calling
libusb functions.
Command line option for connecting USB device to simulator should be
specified in both record and replay modes. In replay mode only log file
is read and USB device could be not connected to the host machine.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hw/usb/host-libusb.c     |  551 ++++++++++++++++++++++++++++++----------------
 include/hw/host-libusb.h |  105 +++++++++
 replay/Makefile.objs     |    1 
 replay/replay-events.c   |   48 ++++
 replay/replay-internal.h |   21 ++
 replay/replay-usb.c      |  188 ++++++++++++++++
 replay/replay.c          |   29 ++
 replay/replay.h          |   30 +++
 8 files changed, 784 insertions(+), 189 deletions(-)
 create mode 100755 include/hw/host-libusb.h
 create mode 100755 replay/replay-usb.c

diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c
index c189147..0fde91d 100644
--- a/hw/usb/host-libusb.c
+++ b/hw/usb/host-libusb.c
@@ -40,98 +40,15 @@
 #include "monitor/monitor.h"
 #include "sysemu/sysemu.h"
 #include "trace.h"
+#include "qemu/log.h"
+#include "replay/replay.h"
 
 #include "hw/usb.h"
+#include "hw/host-libusb.h"
 
 /* ------------------------------------------------------------------------ */
 
-#define TYPE_USB_HOST_DEVICE "usb-host"
-#define USB_HOST_DEVICE(obj) \
-     OBJECT_CHECK(USBHostDevice, (obj), TYPE_USB_HOST_DEVICE)
-
-typedef struct USBHostDevice USBHostDevice;
-typedef struct USBHostRequest USBHostRequest;
-typedef struct USBHostIsoXfer USBHostIsoXfer;
-typedef struct USBHostIsoRing USBHostIsoRing;
-
-struct USBAutoFilter {
-    uint32_t bus_num;
-    uint32_t addr;
-    char     *port;
-    uint32_t vendor_id;
-    uint32_t product_id;
-};
-
-enum USBHostDeviceOptions {
-    USB_HOST_OPT_PIPELINE,
-};
-
-struct USBHostDevice {
-    USBDevice parent_obj;
-
-    /* properties */
-    struct USBAutoFilter             match;
-    int32_t                          bootindex;
-    uint32_t                         iso_urb_count;
-    uint32_t                         iso_urb_frames;
-    uint32_t                         options;
-    uint32_t                         loglevel;
-
-    /* state */
-    QTAILQ_ENTRY(USBHostDevice)      next;
-    int                              seen, errcount;
-    int                              bus_num;
-    int                              addr;
-    char                             port[16];
-
-    libusb_device                    *dev;
-    libusb_device_handle             *dh;
-    struct libusb_device_descriptor  ddesc;
-
-    struct {
-        bool                         detached;
-        bool                         claimed;
-    } ifs[USB_MAX_INTERFACES];
-
-    /* callbacks & friends */
-    QEMUBH                           *bh_nodev;
-    QEMUBH                           *bh_postld;
-    Notifier                         exit;
-
-    /* request queues */
-    QTAILQ_HEAD(, USBHostRequest)    requests;
-    QTAILQ_HEAD(, USBHostIsoRing)    isorings;
-};
-
-struct USBHostRequest {
-    USBHostDevice                    *host;
-    USBPacket                        *p;
-    bool                             in;
-    struct libusb_transfer           *xfer;
-    unsigned char                    *buffer;
-    unsigned char                    *cbuf;
-    unsigned int                     clen;
     bool                             usb3ep0quirk;
-    QTAILQ_ENTRY(USBHostRequest)     next;
-};
-
-struct USBHostIsoXfer {
-    USBHostIsoRing                   *ring;
-    struct libusb_transfer           *xfer;
-    bool                             copy_complete;
-    unsigned int                     packet;
-    QTAILQ_ENTRY(USBHostIsoXfer)     next;
-};
-
-struct USBHostIsoRing {
-    USBHostDevice                    *host;
-    USBEndpoint                      *ep;
-    QTAILQ_HEAD(, USBHostIsoXfer)    unused;
-    QTAILQ_HEAD(, USBHostIsoXfer)    inflight;
-    QTAILQ_HEAD(, USBHostIsoXfer)    copy;
-    QTAILQ_ENTRY(USBHostIsoRing)     next;
-};
-
 static QTAILQ_HEAD(, USBHostDevice) hostdevs =
     QTAILQ_HEAD_INITIALIZER(hostdevs);
 
@@ -143,6 +60,24 @@ static void usb_host_attach_kernel(USBHostDevice *s);
 
 /* ------------------------------------------------------------------------ */
 
+typedef struct USBHostTimer {
+    QEMUTimer *timer;
+} USBHostTimer;
+
+USBHostTimer usb_auto_timer;
+static int submitted_xfers = 0;
+
+static const VMStateDescription vmstate_usb_host_timer = {
+    .name = "usb-host-timer",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_TIMER(timer, USBHostTimer),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+
 #define CONTROL_TIMEOUT  10000        /* 10 sec    */
 #define BULK_TIMEOUT         0        /* unlimited */
 #define INTR_TIMEOUT         0        /* unlimited */
@@ -214,6 +149,11 @@ static void usb_host_del_fd(int fd, void *user_data)
     qemu_set_fd_handler(fd, NULL, NULL, NULL);
 }
 
+bool usb_host_has_xfers(void)
+{
+    return submitted_xfers != 0;
+}
+
 static int usb_host_init(void)
 {
     const struct libusb_pollfd **poll;
@@ -226,18 +166,35 @@ static int usb_host_init(void)
     if (rc != 0) {
         return -1;
     }
-    libusb_set_debug(ctx, loglevel);
-
-    libusb_set_pollfd_notifiers(ctx, usb_host_add_fd,
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_set_debug(ctx, loglevel);
+        libusb_set_pollfd_notifiers(ctx, usb_host_add_fd,
                                 usb_host_del_fd,
                                 ctx);
-    poll = libusb_get_pollfds(ctx);
-    if (poll) {
-        for (i = 0; poll[i] != NULL; i++) {
-            usb_host_add_fd(poll[i]->fd, poll[i]->events, ctx);
+
+        poll = libusb_get_pollfds(ctx);
+        if (poll) {
+            for (i = 0; poll[i] != NULL; i++) {
+                usb_host_add_fd(poll[i]->fd, poll[i]->events, ctx);
+            }
+        }
+        free(poll);
+    }
+
+    /* replay: changed timer for working with VM clock instead of RT */
+    if (!usb_auto_timer.timer) {
+        usb_auto_timer.timer = timer_new_ms(QEMU_CLOCK_VIRTUAL,
+                                            usb_host_auto_check, NULL);
+        if (!usb_auto_timer.timer) {
+            return -1;
+        }
+        trace_usb_host_auto_scan_enabled();
+
+        if (replay_mode != REPLAY_MODE_NONE) {
+            vmstate_register(NULL, 0, &vmstate_usb_host_timer, &usb_auto_timer);
         }
     }
-    free(poll);
+
     return 0;
 }
 
@@ -252,6 +209,7 @@ static int usb_host_get_port(libusb_device *dev, char *port, size_t len)
 #else
     rc = libusb_get_port_path(ctx, dev, path, 7);
 #endif
+
     if (rc < 0) {
         return 0;
     }
@@ -307,6 +265,7 @@ static USBHostRequest *usb_host_req_alloc(USBHostDevice *s, USBPacket *p,
     r->host = s;
     r->p = p;
     r->in = in;
+    /* allocate xfer's in REPLAY_MODE_PLAY too */
     r->xfer = libusb_alloc_transfer(0);
     if (bufsize) {
         r->buffer = g_malloc(bufsize);
@@ -320,6 +279,7 @@ static void usb_host_req_free(USBHostRequest *r)
     if (r->host) {
         QTAILQ_REMOVE(&r->host->requests, r, next);
     }
+    /* free xfer's in REPLAY_MODE_PLAY too */
     libusb_free_transfer(r->xfer);
     g_free(r->buffer);
     g_free(r);
@@ -337,12 +297,13 @@ static USBHostRequest *usb_host_req_find(USBHostDevice *s, USBPacket *p)
     return NULL;
 }
 
-static void usb_host_req_complete_ctrl(struct libusb_transfer *xfer)
+void usb_host_req_complete_ctrl(struct libusb_transfer *xfer)
 {
     USBHostRequest *r = xfer->user_data;
     USBHostDevice  *s = r->host;
     bool disconnect = (xfer->status == LIBUSB_TRANSFER_NO_DEVICE);
 
+    --submitted_xfers;
     if (r->p == NULL) {
         goto out; /* request was canceled */
     }
@@ -354,7 +315,7 @@ static void usb_host_req_complete_ctrl(struct libusb_transfer *xfer)
 
         /* Fix up USB-3 ep0 maxpacket size to allow superspeed connected devices
          * to work redirected to a not superspeed capable hcd */
-        if (r->usb3ep0quirk && xfer->actual_length >= 18 &&
+        if (usb3ep0quirk && xfer->actual_length >= 18 &&
             r->cbuf[7] == 9) {
             r->cbuf[7] = 64;
         }
@@ -370,12 +331,13 @@ out:
     }
 }
 
-static void usb_host_req_complete_data(struct libusb_transfer *xfer)
+void usb_host_req_complete_data(struct libusb_transfer *xfer)
 {
     USBHostRequest *r = xfer->user_data;
     USBHostDevice  *s = r->host;
     bool disconnect = (xfer->status == LIBUSB_TRANSFER_NO_DEVICE);
 
+    --submitted_xfers;
     if (r->p == NULL) {
         goto out; /* request was canceled */
     }
@@ -419,17 +381,17 @@ static void usb_host_req_abort(USBHostRequest *r)
     QTAILQ_REMOVE(&r->host->requests, r, next);
     r->host = NULL;
 
-    if (inflight) {
+    if (inflight && replay_mode != REPLAY_MODE_PLAY) {
         libusb_cancel_transfer(r->xfer);
     }
 }
 
 /* ------------------------------------------------------------------------ */
 
-static void usb_host_req_complete_iso(struct libusb_transfer *transfer)
+void usb_host_req_complete_iso(struct libusb_transfer *transfer)
 {
     USBHostIsoXfer *xfer = transfer->user_data;
-
+    --submitted_xfers;
     if (!xfer) {
         /* USBHostIsoXfer released while inflight */
         g_free(transfer->buffer);
@@ -467,6 +429,7 @@ static USBHostIsoRing *usb_host_iso_alloc(USBHostDevice *s, USBEndpoint *ep)
     for (i = 0; i < s->iso_urb_count; i++) {
         xfer = g_new0(USBHostIsoXfer, 1);
         xfer->ring = ring;
+        /* OK for replay too */
         xfer->xfer = libusb_alloc_transfer(packets);
         xfer->xfer->dev_handle = s->dh;
         xfer->xfer->type = LIBUSB_TRANSFER_TYPE_ISOCHRONOUS;
@@ -475,7 +438,9 @@ static USBHostIsoRing *usb_host_iso_alloc(USBHostDevice *s, USBEndpoint *ep)
         if (ring->ep->pid == USB_TOKEN_IN) {
             xfer->xfer->endpoint |= USB_DIR_IN;
         }
-        xfer->xfer->callback = usb_host_req_complete_iso;
+        xfer->xfer->callback = replay_mode == REPLAY_MODE_NONE
+                               ? usb_host_req_complete_iso
+                               : replay_req_complete_iso;
         xfer->xfer->user_data = xfer;
 
         xfer->xfer->num_iso_packets = packets;
@@ -514,6 +479,7 @@ static void usb_host_iso_free_xfer(USBHostIsoXfer *xfer, bool inflight)
         xfer->xfer->user_data = NULL;
     } else {
         g_free(xfer->xfer->buffer);
+        /* REPLAY_MODE_PLAY also allocates xfer */
         libusb_free_transfer(xfer->xfer);
     }
     g_free(xfer);
@@ -549,12 +515,19 @@ static void usb_host_iso_free_all(USBHostDevice *s)
     }
 }
 
+
+unsigned char *usb_host_get_iso_packet_buffer(USBHostIsoXfer *xfer, int packet)
+{
+    /* replay OK */
+    return libusb_get_iso_packet_buffer_simple(xfer->xfer, packet);
+}
+
 static bool usb_host_iso_data_copy(USBHostIsoXfer *xfer, USBPacket *p)
 {
     unsigned int psize;
     unsigned char *buf;
 
-    buf = libusb_get_iso_packet_buffer_simple(xfer->xfer, xfer->packet);
+    buf = usb_host_get_iso_packet_buffer(xfer, xfer->packet);
     if (p->pid == USB_TOKEN_OUT) {
         psize = p->iov.size;
         if (psize > xfer->ring->ep->max_packet_size) {
@@ -600,7 +573,7 @@ static void usb_host_iso_data_in(USBHostDevice *s, USBPacket *p)
     while ((xfer = QTAILQ_FIRST(&ring->unused)) != NULL) {
         QTAILQ_REMOVE(&ring->unused, xfer, next);
         usb_host_iso_reset_xfer(xfer);
-        rc = libusb_submit_transfer(xfer->xfer);
+        REPLAY_DATA_INT(rc, libusb_submit_transfer(xfer->xfer));
         if (rc != 0) {
             usb_host_libusb_error("libusb_submit_transfer [iso]", rc);
             QTAILQ_INSERT_TAIL(&ring->unused, xfer, next);
@@ -608,6 +581,9 @@ static void usb_host_iso_data_in(USBHostDevice *s, USBPacket *p)
                 disconnect = true;
             }
             break;
+        } else {
+            replay_req_register_iso(xfer->xfer);
+            ++submitted_xfers;
         }
         if (QTAILQ_EMPTY(&ring->inflight)) {
             trace_usb_host_iso_start(s->bus_num, s->addr, p->ep->nr);
@@ -662,7 +638,7 @@ static void usb_host_iso_data_out(USBHostDevice *s, USBPacket *p)
     while ((xfer = QTAILQ_FIRST(&ring->copy)) != NULL &&
            xfer->copy_complete) {
         QTAILQ_REMOVE(&ring->copy, xfer, next);
-        rc = libusb_submit_transfer(xfer->xfer);
+        REPLAY_DATA_INT(rc, libusb_submit_transfer(xfer->xfer));
         if (rc != 0) {
             usb_host_libusb_error("libusb_submit_transfer [iso]", rc);
             QTAILQ_INSERT_TAIL(&ring->unused, xfer, next);
@@ -670,6 +646,9 @@ static void usb_host_iso_data_out(USBHostDevice *s, USBPacket *p)
                 disconnect = true;
             }
             break;
+        } else {
+            replay_req_register_iso(xfer->xfer);
+            ++submitted_xfers;
         }
         if (QTAILQ_EMPTY(&ring->inflight)) {
             trace_usb_host_iso_start(s->bus_num, s->addr, p->ep->nr);
@@ -697,6 +676,11 @@ static void usb_host_speed_compat(USBHostDevice *s)
     bool compat_full = true;
     uint8_t type;
     int rc, c, i, a, e;
+    bool ok = false;
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        REPLAY_DATA_VAR(ok);
+        return;
+    }
 
     for (c = 0;; c++) {
         rc = libusb_get_config_descriptor(s->dev, c, &conf);
@@ -762,9 +746,12 @@ static void usb_host_ep_update(USBHostDevice *s)
         [USB_ENDPOINT_XFER_INT]     = "int",
     };
     USBDevice *udev = USB_DEVICE(s);
-    struct libusb_config_descriptor *conf;
-    const struct libusb_interface_descriptor *intf;
-    const struct libusb_endpoint_descriptor *endp;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_interface_descriptor dummy_intf;
+    struct libusb_endpoint_descriptor dummy_endp;
+    struct libusb_config_descriptor *conf = &dummy_conf;
+    struct libusb_interface_descriptor *intf;
+    struct libusb_endpoint_descriptor *endp;
 #ifdef HAVE_STREAMS
     struct libusb_ss_endpoint_companion_descriptor *endp_ss_comp;
 #endif
@@ -773,21 +760,37 @@ static void usb_host_ep_update(USBHostDevice *s)
     int rc, i, e;
 
     usb_ep_reset(udev);
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         return;
     }
     trace_usb_host_parse_config(s->bus_num, s->addr,
                                 conf->bConfigurationValue, true);
 
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
     for (i = 0; i < conf->bNumInterfaces; i++) {
-        assert(udev->altsetting[i] < conf->interface[i].num_altsetting);
-        intf = &conf->interface[i].altsetting[udev->altsetting[i]];
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            assert(udev->altsetting[i] < conf->interface[i].num_altsetting);
+            intf = (struct libusb_interface_descriptor *)
+                       &conf->interface[i].altsetting[udev->altsetting[i]];
+        } else {
+            intf = &dummy_intf;
+        }
+        REPLAY_DATA_VAR(intf->bInterfaceNumber);
+        REPLAY_DATA_VAR(intf->bAlternateSetting);
+        REPLAY_DATA_VAR(intf->bNumEndpoints);
         trace_usb_host_parse_interface(s->bus_num, s->addr,
                                        intf->bInterfaceNumber,
                                        intf->bAlternateSetting, true);
         for (e = 0; e < intf->bNumEndpoints; e++) {
-            endp = &intf->endpoint[e];
+            if (replay_mode != REPLAY_MODE_PLAY) {
+                endp = (struct libusb_endpoint_descriptor *)&intf->endpoint[e];
+            } else {
+                endp = &dummy_endp;
+            }
+            REPLAY_DATA_VAR(endp->bEndpointAddress);
+            REPLAY_DATA_VAR(endp->bmAttributes);
+            REPLAY_DATA_VAR(endp->wMaxPacketSize);
 
             devep = endp->bEndpointAddress;
             pid = (devep & USB_DIR_IN) ? USB_TOKEN_IN : USB_TOKEN_OUT;
@@ -825,45 +828,77 @@ static void usb_host_ep_update(USBHostDevice *s)
         }
     }
 
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
 }
 
 static int usb_host_open(USBHostDevice *s, libusb_device *dev)
 {
     USBDevice *udev = USB_DEVICE(s);
-    int bus_num = libusb_get_bus_number(dev);
-    int addr    = libusb_get_device_address(dev);
+    int bus_num;
+    REPLAY_DATA_INT(bus_num, libusb_get_bus_number(dev));
+    int addr;
+    REPLAY_DATA_INT(addr, libusb_get_device_address(dev));
     int rc;
+    int speed;
 
     trace_usb_host_open_started(bus_num, addr);
 
-    if (s->dh != NULL) {
+    if (s->is_open) {
         goto fail;
     }
-    rc = libusb_open(dev, &s->dh);
+    REPLAY_DATA_INT(rc, libusb_open(dev, &s->dh));
     if (rc != 0) {
         goto fail;
     }
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        /* invalid non-NULL pointer for checking conditions */
+        s->dh = (void *)0xbad;
+    }
+    s->is_open = true;
 
+    usb_host_detach_kernel(s);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_get_device_descriptor(dev, &s->ddesc);
+    }
+    REPLAY_DATA_VAR(s->ddesc.bLength);
+    REPLAY_DATA_VAR(s->ddesc.bDescriptorType);
+    REPLAY_DATA_VAR(s->ddesc.bcdUSB);
+    REPLAY_DATA_VAR(s->ddesc.bDeviceClass);
+    REPLAY_DATA_VAR(s->ddesc.bDeviceSubClass);
+    REPLAY_DATA_VAR(s->ddesc.bDeviceProtocol);
+    REPLAY_DATA_VAR(s->ddesc.bMaxPacketSize0);
+    REPLAY_DATA_VAR(s->ddesc.idVendor);
+    REPLAY_DATA_VAR(s->ddesc.idProduct);
+    REPLAY_DATA_VAR(s->ddesc.bcdDevice);
+    REPLAY_DATA_VAR(s->ddesc.iManufacturer);
+    REPLAY_DATA_VAR(s->ddesc.iProduct);
+    REPLAY_DATA_VAR(s->ddesc.iSerialNumber);
+    REPLAY_DATA_VAR(s->ddesc.bNumConfigurations);
     s->dev     = dev;
     s->bus_num = bus_num;
     s->addr    = addr;
-
-    usb_host_detach_kernel(s);
-
-    libusb_get_device_descriptor(dev, &s->ddesc);
-    usb_host_get_port(s->dev, s->port, sizeof(s->port));
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        usb_host_get_port(s->dev, s->port, sizeof(s->port));
+    }
+    replay_data_buffer((unsigned char *)s->port, sizeof(s->port));
 
     usb_ep_init(udev);
     usb_host_ep_update(s);
 
-    udev->speed     = speed_map[libusb_get_device_speed(dev)];
+    udev->speed = speed_map[REPLAY_DATA_INT(speed,
+                                            libusb_get_device_speed(dev))];
     usb_host_speed_compat(s);
 
     if (s->ddesc.iProduct) {
-        libusb_get_string_descriptor_ascii(s->dh, s->ddesc.iProduct,
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            libusb_get_string_descriptor_ascii(s->dh, s->ddesc.iProduct,
                                            (unsigned char *)udev->product_desc,
                                            sizeof(udev->product_desc));
+        }
+        replay_data_buffer((unsigned char *)udev->product_desc,
+                           sizeof(udev->product_desc));
     } else {
         snprintf(udev->product_desc, sizeof(udev->product_desc),
                  "host:%d.%d", bus_num, addr);
@@ -879,10 +914,13 @@ static int usb_host_open(USBHostDevice *s, libusb_device *dev)
 
 fail:
     trace_usb_host_open_failure(bus_num, addr);
-    if (s->dh != NULL) {
-        libusb_close(s->dh);
+    if (s->is_open) {
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            libusb_close(s->dh);
+        }
         s->dh = NULL;
         s->dev = NULL;
+        s->is_open = false;
     }
     return -1;
 }
@@ -896,11 +934,20 @@ static void usb_host_abort_xfers(USBHostDevice *s)
     }
 }
 
+static void usb_host_free_xfers(USBHostDevice *s)
+{
+    USBHostRequest *r;
+
+    while ((r = QTAILQ_FIRST(&s->requests)) != NULL) {
+        usb_host_req_free(r);
+    }
+}
+
 static int usb_host_close(USBHostDevice *s)
 {
     USBDevice *udev = USB_DEVICE(s);
 
-    if (s->dh == NULL) {
+    if (!s->is_open) {
         return -1;
     }
 
@@ -914,11 +961,20 @@ static int usb_host_close(USBHostDevice *s)
     }
 
     usb_host_release_interfaces(s);
-    libusb_reset_device(s->dh);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_reset_device(s->dh);
+    }
+
     usb_host_attach_kernel(s);
-    libusb_close(s->dh);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_close(s->dh);
+    }
+
+    /*replay_unregister_usb_device(udev);*/
+
     s->dh = NULL;
     s->dev = NULL;
+    s->is_open = false;
 
     usb_host_auto_check(NULL);
     return 0;
@@ -933,7 +989,7 @@ static void usb_host_nodev_bh(void *opaque)
 static void usb_host_nodev(USBHostDevice *s)
 {
     if (!s->bh_nodev) {
-        s->bh_nodev = qemu_bh_new(usb_host_nodev_bh, s);
+        s->bh_nodev = qemu_bh_new_replay(usb_host_nodev_bh, s, s->bus_num);
     }
     qemu_bh_schedule(s->bh_nodev);
 }
@@ -942,7 +998,7 @@ static void usb_host_exit_notifier(struct Notifier *n, void *data)
 {
     USBHostDevice *s = container_of(n, USBHostDevice, exit);
 
-    if (s->dh) {
+    if (s->is_open) {
         usb_host_release_interfaces(s);
         usb_host_attach_kernel(s);
     }
@@ -1004,57 +1060,70 @@ static void usb_host_cancel_packet(USBDevice *udev, USBPacket *p)
     r = usb_host_req_find(s, p);
     if (r && r->p) {
         r->p = NULL; /* mark as dead */
-        libusb_cancel_transfer(r->xfer);
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            libusb_cancel_transfer(r->xfer);
+        }
     }
 }
 
 static void usb_host_detach_kernel(USBHostDevice *s)
 {
-    struct libusb_config_descriptor *conf;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_config_descriptor *conf = &dummy_conf;
     int rc, i;
 
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         return;
     }
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
     for (i = 0; i < conf->bNumInterfaces; i++) {
-        rc = libusb_kernel_driver_active(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_kernel_driver_active(s->dh, i));
         usb_host_libusb_error("libusb_kernel_driver_active", rc);
         if (rc != 1) {
             continue;
         }
         trace_usb_host_detach_kernel(s->bus_num, s->addr, i);
-        rc = libusb_detach_kernel_driver(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_detach_kernel_driver(s->dh, i));
         usb_host_libusb_error("libusb_detach_kernel_driver", rc);
         s->ifs[i].detached = true;
     }
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
 }
 
 static void usb_host_attach_kernel(USBHostDevice *s)
 {
-    struct libusb_config_descriptor *conf;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_config_descriptor *conf = &dummy_conf;
     int rc, i;
 
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         return;
     }
-    for (i = 0; i < conf->bNumInterfaces; i++) {
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
+    for (i = 0; i < conf->bNumInterfaces ; i++) {
         if (!s->ifs[i].detached) {
             continue;
         }
         trace_usb_host_attach_kernel(s->bus_num, s->addr, i);
-        libusb_attach_kernel_driver(s->dh, i);
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            libusb_attach_kernel_driver(s->dh, i);
+        }
         s->ifs[i].detached = false;
     }
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
 }
 
 static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
 {
     USBDevice *udev = USB_DEVICE(s);
-    struct libusb_config_descriptor *conf;
+    struct libusb_config_descriptor dummy_conf;
+    struct libusb_config_descriptor *conf = &dummy_conf;
     int rc, i;
 
     for (i = 0; i < USB_MAX_INTERFACES; i++) {
@@ -1065,7 +1134,7 @@ static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
 
     usb_host_detach_kernel(s);
 
-    rc = libusb_get_active_config_descriptor(s->dev, &conf);
+    REPLAY_DATA_INT(rc, libusb_get_active_config_descriptor(s->dev, &conf));
     if (rc != 0) {
         if (rc == LIBUSB_ERROR_NOT_FOUND) {
             /* address state - ignore */
@@ -1073,10 +1142,11 @@ static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
         }
         return USB_RET_STALL;
     }
+    REPLAY_DATA_VAR(conf->bNumInterfaces);
 
     for (i = 0; i < conf->bNumInterfaces; i++) {
         trace_usb_host_claim_interface(s->bus_num, s->addr, configuration, i);
-        rc = libusb_claim_interface(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_claim_interface(s->dh, i));
         usb_host_libusb_error("libusb_claim_interface", rc);
         if (rc != 0) {
             return USB_RET_STALL;
@@ -1087,7 +1157,9 @@ static int usb_host_claim_interfaces(USBHostDevice *s, int configuration)
     udev->ninterfaces   = conf->bNumInterfaces;
     udev->configuration = configuration;
 
-    libusb_free_config_descriptor(conf);
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        libusb_free_config_descriptor(conf);
+    }
     return USB_RET_SUCCESS;
 }
 
@@ -1101,7 +1173,7 @@ static void usb_host_release_interfaces(USBHostDevice *s)
             continue;
         }
         trace_usb_host_release_interface(s->bus_num, s->addr, i);
-        rc = libusb_release_interface(s->dh, i);
+        REPLAY_DATA_INT(rc, libusb_release_interface(s->dh, i));
         usb_host_libusb_error("libusb_release_interface", rc);
         s->ifs[i].claimed = false;
     }
@@ -1122,7 +1194,7 @@ static void usb_host_set_config(USBHostDevice *s, int config, USBPacket *p)
     trace_usb_host_set_config(s->bus_num, s->addr, config);
 
     usb_host_release_interfaces(s);
-    rc = libusb_set_configuration(s->dh, config);
+    REPLAY_DATA_INT(rc, libusb_set_configuration(s->dh, config));
     if (rc != 0) {
         usb_host_libusb_error("libusb_set_configuration", rc);
         p->status = USB_RET_STALL;
@@ -1153,7 +1225,7 @@ static void usb_host_set_interface(USBHostDevice *s, int iface, int alt,
         return;
     }
 
-    rc = libusb_set_interface_alt_setting(s->dh, iface, alt);
+    REPLAY_DATA_INT(rc, libusb_set_interface_alt_setting(s->dh, iface, alt));
     if (rc != 0) {
         usb_host_libusb_error("libusb_set_interface_alt_setting", rc);
         p->status = USB_RET_STALL;
@@ -1177,7 +1249,7 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
 
     trace_usb_host_req_control(s->bus_num, s->addr, p, request, value, index);
 
-    if (s->dh == NULL) {
+    if (!s->is_open) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_emulated(s->bus_num, s->addr, p, p->status);
         return;
@@ -1202,7 +1274,9 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
     case EndpointOutRequest | USB_REQ_CLEAR_FEATURE:
         if (value == 0) { /* clear halt */
             int pid = (index & USB_DIR_IN) ? USB_TOKEN_IN : USB_TOKEN_OUT;
-            libusb_clear_halt(s->dh, index);
+            if (replay_mode != REPLAY_MODE_PLAY) {
+                libusb_clear_halt(s->dh, index);
+            }
             usb_ep_set_halted(udev, pid, index & 0x0f, 0);
             trace_usb_host_req_emulated(s->bus_num, s->addr, p, p->status);
             return;
@@ -1222,13 +1296,17 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
     if (udev->speed == USB_SPEED_SUPER &&
         !((udev->port->speedmask & USB_SPEED_MASK_SUPER)) &&
         request == 0x8006 && value == 0x100 && index == 0) {
-        r->usb3ep0quirk = true;
+        usb3ep0quirk = true;
     }
 
+    /* call in REPLAY_MODE_PLAY too - this function just copies data */
     libusb_fill_control_transfer(r->xfer, s->dh, r->buffer,
-                                 usb_host_req_complete_ctrl, r,
-                                 CONTROL_TIMEOUT);
-    rc = libusb_submit_transfer(r->xfer);
+                                 replay_mode == REPLAY_MODE_NONE
+                                     ? usb_host_req_complete_ctrl
+                                     : replay_req_complete_ctrl,
+                                 r, CONTROL_TIMEOUT);
+
+    REPLAY_DATA_INT(rc, libusb_submit_transfer(r->xfer));
     if (rc != 0) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_complete(s->bus_num, s->addr, p,
@@ -1237,6 +1315,9 @@ static void usb_host_handle_control(USBDevice *udev, USBPacket *p,
             usb_host_nodev(s);
         }
         return;
+    } else {
+        replay_req_register_ctrl(r->xfer);
+        ++submitted_xfers;
     }
 
     p->status = USB_RET_ASYNC;
@@ -1258,7 +1339,7 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
                             p->pid == USB_TOKEN_IN,
                             p->ep->nr, p->iov.size);
 
-    if (s->dh == NULL) {
+    if (!s->is_open) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_emulated(s->bus_num, s->addr, p, p->status);
         return;
@@ -1291,8 +1372,10 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
         } else {
             libusb_fill_bulk_transfer(r->xfer, s->dh, ep,
                                       r->buffer, size,
-                                      usb_host_req_complete_data, r,
-                                      BULK_TIMEOUT);
+                                      replay_mode == REPLAY_MODE_NONE
+                                          ? usb_host_req_complete_data
+                                          : replay_req_complete_data,
+                                      r, BULK_TIMEOUT);
         }
         break;
     case USB_ENDPOINT_XFER_INT:
@@ -1303,8 +1386,10 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
         ep = p->ep->nr | (r->in ? USB_DIR_IN : 0);
         libusb_fill_interrupt_transfer(r->xfer, s->dh, ep,
                                        r->buffer, p->iov.size,
-                                       usb_host_req_complete_data, r,
-                                       INTR_TIMEOUT);
+                                       replay_mode == REPLAY_MODE_NONE
+                                           ? usb_host_req_complete_data
+                                           : replay_req_complete_data,
+                                       r, INTR_TIMEOUT);
         break;
     case USB_ENDPOINT_XFER_ISOC:
         if (p->pid == USB_TOKEN_IN) {
@@ -1322,7 +1407,7 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
         return;
     }
 
-    rc = libusb_submit_transfer(r->xfer);
+    REPLAY_DATA_INT(rc, libusb_submit_transfer(r->xfer));
     if (rc != 0) {
         p->status = USB_RET_NODEV;
         trace_usb_host_req_complete(s->bus_num, s->addr, p,
@@ -1331,6 +1416,9 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p)
             usb_host_nodev(s);
         }
         return;
+    } else {
+        replay_req_register_data(r->xfer);
+        ++submitted_xfers;
     }
 
     p->status = USB_RET_ASYNC;
@@ -1350,7 +1438,7 @@ static void usb_host_handle_reset(USBDevice *udev)
 
     trace_usb_host_reset(s->bus_num, s->addr);
 
-    rc = libusb_reset_device(s->dh);
+    REPLAY_DATA_INT(rc, libusb_reset_device(s->dh));
     if (rc != 0) {
         usb_host_nodev(s);
     }
@@ -1425,7 +1513,7 @@ static void usb_host_post_load_bh(void *opaque)
     USBHostDevice *dev = opaque;
     USBDevice *udev = USB_DEVICE(dev);
 
-    if (dev->dh != NULL) {
+    if (dev->is_open) {
         usb_host_close(dev);
     }
     if (udev->attached) {
@@ -1445,6 +1533,40 @@ static int usb_host_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static int usb_host_post_load_replay(void *opaque, int version_id)
+{
+    USBHostDevice *dev = opaque;
+
+    usb_host_free_xfers(dev);
+    usb_host_iso_free_all(dev);
+    submitted_xfers = 0;
+    return 0;
+}
+
+static void usb_host_pre_save_replay(void *opaque)
+{
+    USBHostDevice *dev = opaque;
+
+    if (!QTAILQ_EMPTY(&dev->requests)) {
+        fprintf(stderr, "Replay: Saving VM state "
+                        "with non-empty USB requests queue\n");
+        exit(1);
+    }
+
+    if (!QTAILQ_EMPTY(&dev->isorings)) {
+        fprintf(stderr, "Replay: Saving VM state "
+                        "with non-empty USB iso rings queue\n");
+        exit(1);
+    }
+
+    /* paranoidal checking */
+    if (submitted_xfers != 0) {
+        fprintf(stderr, "Replay: Saving VM state "
+                        "with non-empty USB iso rings queue\n");
+        exit(1);
+    }
+}
+
 static const VMStateDescription vmstate_usb_host = {
     .name = "usb-host",
     .version_id = 1,
@@ -1456,6 +1578,52 @@ static const VMStateDescription vmstate_usb_host = {
     }
 };
 
+static const VMStateDescription vmstate_usb_host_interface = {
+    .name = "usb-host-interface",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_BOOL(detached, USBHostInterface),
+        VMSTATE_BOOL(claimed, USBHostInterface),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
+static const VMStateDescription vmstate_usb_host_replay = {
+    .name = "usb-host",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .post_load = usb_host_post_load_replay,
+    .pre_save = usb_host_pre_save_replay,
+    .fields = (VMStateField[]) {
+        VMSTATE_USB_DEVICE(parent_obj, USBHostDevice),
+        VMSTATE_INT32(seen, USBHostDevice),
+        VMSTATE_INT32(errcount, USBHostDevice),
+        VMSTATE_INT32(bus_num, USBHostDevice),
+        VMSTATE_INT32(addr, USBHostDevice),
+        VMSTATE_INT32(bus_num, USBHostDevice),
+        VMSTATE_BOOL(is_open, USBHostDevice),
+        VMSTATE_CHAR_ARRAY(port, USBHostDevice, 16),
+        VMSTATE_UINT8(ddesc.bLength, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDescriptorType, USBHostDevice),
+        VMSTATE_UINT16(ddesc.bcdUSB, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDeviceClass, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDeviceSubClass, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bDeviceProtocol, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bMaxPacketSize0, USBHostDevice),
+        VMSTATE_UINT16(ddesc.idVendor, USBHostDevice),
+        VMSTATE_UINT16(ddesc.idProduct, USBHostDevice),
+        VMSTATE_UINT16(ddesc.bcdDevice, USBHostDevice),
+        VMSTATE_UINT8(ddesc.iManufacturer, USBHostDevice),
+        VMSTATE_UINT8(ddesc.iProduct, USBHostDevice),
+        VMSTATE_UINT8(ddesc.iSerialNumber, USBHostDevice),
+        VMSTATE_UINT8(ddesc.bNumConfigurations, USBHostDevice),
+        VMSTATE_STRUCT_ARRAY(ifs, USBHostDevice, USB_MAX_INTERFACES, 1,
+                             vmstate_usb_host_interface, USBHostInterface),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 static Property usb_host_dev_properties[] = {
     DEFINE_PROP_UINT32("hostbus",  USBHostDevice, match.bus_num,    0),
     DEFINE_PROP_UINT32("hostaddr", USBHostDevice, match.addr,       0),
@@ -1487,7 +1655,11 @@ static void usb_host_class_initfn(ObjectClass *klass, void *data)
     uc->flush_ep_queue = usb_host_flush_ep_queue;
     uc->alloc_streams  = usb_host_alloc_streams;
     uc->free_streams   = usb_host_free_streams;
-    dc->vmsd = &vmstate_usb_host;
+    if (replay_mode == REPLAY_MODE_NONE) {
+        dc->vmsd = &vmstate_usb_host;
+    } else {
+        dc->vmsd = &vmstate_usb_host_replay;
+    }
     dc->props = usb_host_dev_properties;
     set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
 }
@@ -1508,7 +1680,6 @@ type_init(usb_host_register_types)
 
 /* ------------------------------------------------------------------------ */
 
-static QEMUTimer *usb_auto_timer;
 static VMChangeStateEntry *usb_vmstate;
 
 static void usb_host_vm_state(void *unused, int running, RunState state)
@@ -1525,70 +1696,79 @@ static void usb_host_auto_check(void *unused)
     libusb_device **devs = NULL;
     struct libusb_device_descriptor ddesc;
     int unconnected = 0;
-    int i, n;
+    int i, n, tmp;
 
     if (usb_host_init() != 0) {
         return;
     }
 
     if (runstate_is_running()) {
-        n = libusb_get_device_list(ctx, &devs);
+        REPLAY_DATA_INT(n, libusb_get_device_list(ctx, &devs));
         for (i = 0; i < n; i++) {
-            if (libusb_get_device_descriptor(devs[i], &ddesc) != 0) {
+            if (REPLAY_DATA_INT(tmp, libusb_get_device_descriptor(devs[i],
+                                         &ddesc)) != 0) {
                 continue;
             }
-            if (ddesc.bDeviceClass == LIBUSB_CLASS_HUB) {
+            if (REPLAY_DATA_INT(tmp, ddesc.bDeviceClass) == LIBUSB_CLASS_HUB) {
                 continue;
             }
             QTAILQ_FOREACH(s, &hostdevs, next) {
                 f = &s->match;
                 if (f->bus_num > 0 &&
-                    f->bus_num != libusb_get_bus_number(devs[i])) {
+                    f->bus_num != REPLAY_DATA_INT(tmp, libusb_get_bus_number(
+                                                           devs[i]))) {
                     continue;
                 }
                 if (f->addr > 0 &&
-                    f->addr != libusb_get_device_address(devs[i])) {
+                    f->addr != REPLAY_DATA_INT(tmp, libusb_get_device_address(
+                                                        devs[i]))) {
                     continue;
                 }
                 if (f->port != NULL) {
                     char port[16] = "-";
-                    usb_host_get_port(devs[i], port, sizeof(port));
+                    if (replay_mode != REPLAY_MODE_PLAY) {
+                        usb_host_get_port(devs[i], port, sizeof(port));
+                    }
+                    replay_data_buffer((unsigned char *)port, sizeof(port));
                     if (strcmp(f->port, port) != 0) {
                         continue;
                     }
                 }
                 if (f->vendor_id > 0 &&
-                    f->vendor_id != ddesc.idVendor) {
+                    f->vendor_id != REPLAY_DATA_INT(tmp, ddesc.idVendor)) {
                     continue;
                 }
                 if (f->product_id > 0 &&
-                    f->product_id != ddesc.idProduct) {
+                    f->product_id != REPLAY_DATA_INT(tmp, ddesc.idProduct)) {
                     continue;
                 }
-
                 /* We got a match */
                 s->seen++;
                 if (s->errcount >= 3) {
                     continue;
                 }
-                if (s->dh != NULL) {
+                if (s->is_open) {
                     continue;
                 }
-                if (usb_host_open(s, devs[i]) < 0) {
+                if (usb_host_open(s, replay_mode == REPLAY_MODE_PLAY
+                                     ? (libusb_device *)0xbad
+                                     : devs[i]) < 0) {
                     s->errcount++;
                     continue;
                 }
                 break;
             }
         }
-        libusb_free_device_list(devs, 1);
+        if (replay_mode != REPLAY_MODE_PLAY) {
+            libusb_free_device_list(devs, 1);
+        }
 
         QTAILQ_FOREACH(s, &hostdevs, next) {
-            if (s->dh == NULL) {
+            if (!s->is_open) {
                 unconnected++;
             }
             if (s->seen == 0) {
-                if (s->dh) {
+                if (s->is_open) {
                     usb_host_close(s);
                 }
                 s->errcount = 0;
@@ -1608,17 +1788,12 @@ static void usb_host_auto_check(void *unused)
 #endif
     }
 
-    if (!usb_vmstate) {
+    if (!usb_vmstate && replay_mode == REPLAY_MODE_NONE) {
         usb_vmstate = qemu_add_vm_change_state_handler(usb_host_vm_state, NULL);
     }
-    if (!usb_auto_timer) {
-        usb_auto_timer = timer_new_ms(QEMU_CLOCK_REALTIME, usb_host_auto_check, NULL);
-        if (!usb_auto_timer) {
-            return;
-        }
-        trace_usb_host_auto_scan_enabled();
-    }
-    timer_mod(usb_auto_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 2000);
+
+    timer_mod(usb_auto_timer.timer,
+              qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + 2000);
 }
 
 void usb_host_info(Monitor *mon, const QDict *qdict)
diff --git a/include/hw/host-libusb.h b/include/hw/host-libusb.h
new file mode 100755
index 0000000..2f5a9d7
--- /dev/null
+++ b/include/hw/host-libusb.h
@@ -0,0 +1,105 @@
+#ifndef HOST_LIBUSB_H
+#define HOST_LIBUSB_H
+
+#include <libusb.h>
+
+#include "qemu/notify.h"
+
+#define TYPE_USB_HOST_DEVICE "usb-host"
+#define USB_HOST_DEVICE(obj) \
+     OBJECT_CHECK(USBHostDevice, (obj), TYPE_USB_HOST_DEVICE)
+
+struct USBAutoFilter {
+    uint32_t bus_num;
+    uint32_t addr;
+    char     *port;
+    uint32_t vendor_id;
+    uint32_t product_id;
+};
+
+typedef struct USBHostDevice USBHostDevice;
+typedef struct USBHostRequest USBHostRequest;
+typedef struct USBHostIsoXfer USBHostIsoXfer;
+typedef struct USBHostIsoRing USBHostIsoRing;
+typedef struct USBHostInterface USBHostInterface;
+
+enum USBHostDeviceOptions {
+    USB_HOST_OPT_PIPELINE,
+};
+
+struct USBHostInterface {
+    bool                             detached;
+    bool                             claimed;
+};
+
+struct USBHostDevice {
+    USBDevice parent_obj;
+
+    /* properties */
+    struct USBAutoFilter             match;
+    int32_t                          bootindex;
+    uint32_t                         iso_urb_count;
+    uint32_t                         iso_urb_frames;
+    uint32_t                         options;
+    uint32_t                         loglevel;
+
+    /* state */
+    QTAILQ_ENTRY(USBHostDevice)      next;
+    int                              seen, errcount;
+    int                              bus_num;
+    int                              addr;
+    char                             port[16];
+    bool                             is_open;
+
+    libusb_device                    *dev;
+    libusb_device_handle             *dh;
+    struct libusb_device_descriptor  ddesc;
+
+    USBHostInterface ifs[USB_MAX_INTERFACES];
+
+    /* callbacks & friends */
+    QEMUBH                           *bh_nodev;
+    QEMUBH                           *bh_postld;
+    Notifier                         exit;
+
+    /* request queues */
+    QTAILQ_HEAD(, USBHostRequest)    requests;
+    QTAILQ_HEAD(, USBHostIsoRing)    isorings;
+};
+
+struct USBHostRequest {
+    USBHostDevice                    *host;
+    USBPacket                        *p;
+    bool                             in;
+    struct libusb_transfer           *xfer;
+    unsigned char                    *buffer;
+    unsigned char                    *cbuf;
+    unsigned int                     clen;
+    QTAILQ_ENTRY(USBHostRequest)     next;
+};
+
+struct USBHostIsoXfer {
+    USBHostIsoRing                   *ring;
+    struct libusb_transfer           *xfer;
+    bool                             copy_complete;
+    unsigned int                     packet;
+    QTAILQ_ENTRY(USBHostIsoXfer)     next;
+};
+
+struct USBHostIsoRing {
+    USBHostDevice                    *host;
+    USBEndpoint                      *ep;
+    QTAILQ_HEAD(, USBHostIsoXfer)    unused;
+    QTAILQ_HEAD(, USBHostIsoXfer)    inflight;
+    QTAILQ_HEAD(, USBHostIsoXfer)    copy;
+    QTAILQ_ENTRY(USBHostIsoRing)     next;
+};
+
+void usb_host_req_complete_ctrl(struct libusb_transfer *xfer);
+void usb_host_req_complete_data(struct libusb_transfer *xfer);
+void usb_host_req_complete_iso(struct libusb_transfer *xfer);
+unsigned char *usb_host_get_iso_packet_buffer(USBHostIsoXfer *xfer, int packet);
+
+bool usb_host_has_xfers(void);
+
+#endif
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index c528bcb..f4eef74 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -7,3 +7,4 @@ obj-y += replay-input.o
 obj-y += replay-net.o
 obj-y += replay-audio.o
 obj-y += replay-char.o
+obj-y += replay-usb.o
diff --git a/replay/replay-events.c b/replay/replay-events.c
index 362fef8..fb753aa 100755
--- a/replay/replay-events.c
+++ b/replay/replay-events.c
@@ -59,6 +59,17 @@ static void replay_run_event(Event *event)
     case REPLAY_ASYNC_EVENT_CHAR:
         replay_event_char_run(event->opaque);
         break;
+#ifdef CONFIG_USB_LIBUSB
+    case REPLAY_ASYNC_EVENT_USB_CTRL:
+        replay_event_usb_ctrl(event->opaque);
+        break;
+    case REPLAY_ASYNC_EVENT_USB_DATA:
+        replay_event_usb_data(event->opaque);
+        break;
+    case REPLAY_ASYNC_EVENT_USB_ISO:
+        replay_event_usb_iso(event->opaque);
+        break;
+#endif
     default:
         fprintf(stderr, "Replay: invalid async event ID (%d) in the queue\n",
                 event->event_kind);
@@ -162,6 +173,11 @@ void replay_add_input_sync_event(void)
     replay_add_event_internal(REPLAY_ASYNC_EVENT_INPUT_SYNC, NULL, NULL, 0);
 }
 
+void replay_add_usb_event(unsigned int event_kind, uint64_t id, void *opaque)
+{
+    replay_add_event_internal(event_kind, opaque, NULL, id);
+}
+
 void replay_save_events(int opt)
 {
     qemu_mutex_lock(&lock);
@@ -192,6 +208,17 @@ void replay_save_events(int opt)
             case REPLAY_ASYNC_EVENT_CHAR:
                 replay_event_char_save(event->opaque);
                 break;
+#ifdef CONFIG_USB_LIBUSB
+            case REPLAY_ASYNC_EVENT_USB_CTRL:
+            case REPLAY_ASYNC_EVENT_USB_DATA:
+                replay_put_qword(event->id);
+                replay_event_save_usb_xfer(event->opaque);
+                break;
+            case REPLAY_ASYNC_EVENT_USB_ISO:
+                replay_put_qword(event->id);
+                replay_event_save_usb_iso_xfer(event->opaque);
+                break;
+#endif
             }
         }
 
@@ -275,6 +302,16 @@ void replay_read_events(int opt)
             replay_run_event(&e);
             /* continue with the next event */
             continue;
+#ifdef CONFIG_USB_LIBUSB
+        case REPLAY_ASYNC_EVENT_USB_CTRL:
+        case REPLAY_ASYNC_EVENT_USB_DATA:
+        case REPLAY_ASYNC_EVENT_USB_ISO:
+            if (read_id == -1) {
+                read_id = replay_get_qword();
+            }
+            /* read after finding event in the list */
+            break;
+#endif
         default:
             fprintf(stderr, "Unknown ID %d of replay event\n", read_event_kind);
             exit(1);
@@ -295,6 +332,17 @@ void replay_read_events(int opt)
 
         if (event) {
             /* read event-specific reading data */
+#ifdef CONFIG_USB_LIBUSB
+            switch (read_event_kind) {
+            case REPLAY_ASYNC_EVENT_USB_CTRL:
+            case REPLAY_ASYNC_EVENT_USB_DATA:
+                replay_event_read_usb_xfer(event->opaque);
+                break;
+            case REPLAY_ASYNC_EVENT_USB_ISO:
+                replay_event_read_usb_iso_xfer(event->opaque);
+                break;
+            }
+#endif
 
             QTAILQ_REMOVE(&events_list, event, events);
 
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index 9bbfcde..eeab40c 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -40,6 +40,8 @@
 #define EVENT_ASYNC_OPT             25
 /* for int data */
 #define EVENT_DATA_INT              26
+/* for data buffer */
+#define EVENT_DATA_BUFFER           27
 /* for instruction event */
 #define EVENT_INSTRUCTION           32
 /* for clock read/writes */
@@ -59,7 +61,10 @@
 #define REPLAY_ASYNC_EVENT_INPUT_SYNC  3
 #define REPLAY_ASYNC_EVENT_NETWORK     4
 #define REPLAY_ASYNC_EVENT_CHAR        5
-#define REPLAY_ASYNC_COUNT             6
+#define REPLAY_ASYNC_EVENT_USB_CTRL    6
+#define REPLAY_ASYNC_EVENT_USB_DATA    7
+#define REPLAY_ASYNC_EVENT_USB_ISO     8
+#define REPLAY_ASYNC_COUNT             9
 
 typedef struct ReplayState {
     /*! Cached clock values. */
@@ -195,4 +200,18 @@ void replay_event_char_save(void *opaque);
 /*! Reads char event from the file. */
 void *replay_event_char_read(void);
 
+/* USB events */
+
+void replay_event_usb_ctrl(void *opaque);
+void replay_event_usb_data(void *opaque);
+void replay_event_usb_iso(void *opaque);
+void replay_event_save_usb_xfer(void *opaque);
+void replay_event_save_usb_iso_xfer(void *opaque);
+void replay_event_read_usb_xfer(void *opaque);
+void replay_event_read_usb_iso_xfer(void *opaque);
+void replay_event_skip_usb_xfer(void);
+void replay_event_skip_usb_iso_xfer(void);
+bool replay_usb_has_xfers(void);
+void replay_add_usb_event(unsigned int event_id, uint64_t id, void *opaque);
+
 #endif
diff --git a/replay/replay-usb.c b/replay/replay-usb.c
new file mode 100755
index 0000000..b4a54e3
--- /dev/null
+++ b/replay/replay-usb.c
@@ -0,0 +1,188 @@
+/*
+ * replay-usb.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "replay.h"
+#include "replay-internal.h"
+#include "hw/usb.h"
+
+#ifdef CONFIG_USB_LIBUSB
+#include "hw/host-libusb.h"
+
+static uint64_t replay_get_xfer_id(struct libusb_transfer *xfer)
+{
+    USBHostRequest *r = xfer->user_data;
+    USBHostDevice *host = r->host;
+
+    return ((uint64_t)host->match.vendor_id << 32)
+           | host->match.product_id;
+}
+
+static uint64_t replay_get_iso_xfer_id(struct libusb_transfer *xfer)
+{
+    USBHostIsoXfer *r = xfer->user_data;
+    USBHostDevice  *host = r->ring->host;
+
+    return ((uint64_t)host->match.vendor_id << 32)
+           | host->match.product_id;
+}
+
+void replay_req_complete_ctrl(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_CTRL,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_req_register_ctrl(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_CTRL,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_event_usb_ctrl(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+
+    usb_host_req_complete_ctrl(xfer);
+}
+
+void replay_event_save_usb_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostRequest *r = xfer->user_data;
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_put_dword(xfer->status);
+        replay_put_dword(xfer->actual_length);
+        replay_put_array(xfer->buffer, r->in ? xfer->length : 0);
+    }
+}
+
+void replay_event_save_usb_iso_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostIsoXfer *iso = xfer->user_data;
+    int i;
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        bool in = iso->ring->ep->pid == USB_TOKEN_IN;
+        replay_put_dword(xfer->status);
+        replay_put_dword(xfer->num_iso_packets);
+        for (i = 0 ; i < xfer->num_iso_packets ; ++i) {
+            /* all other fields of the packet are not used */
+            unsigned int len = xfer->iso_packet_desc[i].actual_length;
+            if (in) {
+                replay_put_array(usb_host_get_iso_packet_buffer(iso, i), len);
+            }
+        }
+    }
+}
+
+void replay_event_read_usb_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostRequest *r = xfer->user_data;
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        xfer->status = replay_get_dword();
+        xfer->actual_length = replay_get_dword();
+        size_t sz;
+        replay_get_array(xfer->buffer, &sz);
+        if (r->in && xfer->length != (int)sz) {
+            fprintf(stderr, "Replay: trying to read USB control/data buffer with unexpected size\n");
+            exit(1);
+        }
+    }
+}
+
+void replay_event_read_usb_iso_xfer(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+    USBHostIsoXfer *iso = xfer->user_data;
+    int i;
+
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        bool in = iso->ring->ep->pid == USB_TOKEN_IN;
+        xfer->status = replay_get_dword();
+        xfer->num_iso_packets = replay_get_dword();
+        for (i = 0 ; i < xfer->num_iso_packets ; ++i) {
+            /* all other fields of the packet are not used */
+            if (in) {
+                size_t sz;
+                replay_get_array(usb_host_get_iso_packet_buffer(iso, i), &sz);
+                xfer->iso_packet_desc[i].actual_length = (unsigned int)sz;
+            }
+        }
+    }
+}
+
+void replay_req_complete_data(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_DATA,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_req_register_data(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_DATA,
+                             replay_get_xfer_id(xfer), xfer);
+    }
+}
+
+
+void replay_event_usb_data(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+
+    usb_host_req_complete_data(xfer);
+}
+
+void replay_req_complete_iso(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_MODE_RECORD) {
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_ISO,
+                             replay_get_iso_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_req_register_iso(struct libusb_transfer *xfer)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        USBHostIsoXfer *r = xfer->user_data;
+        USBHostDevice  *s = r->ring->host;
+
+        replay_add_usb_event(REPLAY_ASYNC_EVENT_USB_ISO,
+                             replay_get_iso_xfer_id(xfer), xfer);
+    }
+}
+
+void replay_event_usb_iso(void *opaque)
+{
+    struct libusb_transfer *xfer = opaque;
+
+    usb_host_req_complete_iso(xfer);
+}
+
+#endif
+
+bool replay_usb_has_xfers(void)
+{
+#ifdef CONFIG_USB_LIBUSB
+    return usb_host_has_xfers();
+#else
+    return false;
+#endif
+}
diff --git a/replay/replay.c b/replay/replay.c
index 7a16da3..4151330 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -101,6 +101,16 @@ static void replay_savevm(void *opaque)
     char name[128];
     uint64_t offset;
 
+#ifdef CONFIG_USB_LIBUSB
+    if (replay_usb_has_xfers()) {
+        /* Retry of saving VM state, when USB host controller is not ready.
+           We cannot save or interrupt non-finished transfers, so
+           just wait for finishing them later. */
+        timer_mod(save_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + 1);
+        return;
+    }
+#endif
+
     offset = ftello64(replay_file);
 
     replay_save_instructions();
@@ -614,3 +624,22 @@ void replay_data_int(int *data)
         replay_put_dword(*data);
     }
 }
+
+void replay_data_buffer(unsigned char *data, size_t size)
+{
+    if (replay_file && replay_mode == REPLAY_MODE_PLAY) {
+        size_t read_size = 0;
+        skip_async_events_until(EVENT_DATA_BUFFER);
+        replay_get_array(data, &read_size);
+        replay_check_error();
+        if (read_size != size) {
+            fprintf(stderr, "Replay: read non-matching size of the buffer\n");
+            exit(1);
+        }
+        replay_has_unread_data = 0;
+    } else if (replay_file && replay_mode == REPLAY_MODE_RECORD) {
+        replay_save_instructions();
+        replay_put_event(EVENT_DATA_BUFFER);
+        replay_put_array(data, size);
+    }
+}
diff --git a/replay/replay.h b/replay/replay.h
index e8fd20c..590f9e6 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -25,6 +25,7 @@ struct QemuOpts;
 struct InputEvent;
 struct NetClientState;
 struct CharDriverState;
+struct libusb_transfer;
 
 /* replay clock kinds */
 /* rdtsc */
@@ -157,9 +158,38 @@ void replay_register_char_driver(struct CharDriverState *chr);
 /*! Saves write to char device event to the log */
 void replay_chr_be_write(struct CharDriverState *s, uint8_t *buf, int len);
 
+/* USB events */
+
+/*! Called in save mode when xfer for ctrl usb data is completed */
+void replay_req_complete_ctrl(struct libusb_transfer *xfer);
+/*! Called in play mode to register xfer
+    to be completed by reading it from the log */
+void replay_req_register_ctrl(struct libusb_transfer *xfer);
+/*! Called in save mode when xfer for usb data request is completed */
+void replay_req_complete_data(struct libusb_transfer *xfer);
+/*! Called in play mode to register xfer
+    to be completed by reading it from the log */
+void replay_req_register_data(struct libusb_transfer *xfer);
+/*! Called in save mode when iso xfer for usb data request is completed */
+void replay_req_complete_iso(struct libusb_transfer *xfer);
+/*! Called in play mode to register iso xfer
+    to be completed by reading it from the log */
+void replay_req_register_iso(struct libusb_transfer *xfer);
+
 /* Other data */
 
 /*! Writes or reads integer value to/from replay log. */
 void replay_data_int(int *data);
+/*! Macro for using replay_data_int function */
+#define REPLAY_DATA_INT(var, value) \
+        ((replay_mode == REPLAY_MODE_NONE ? var = (value) \
+            : replay_mode == REPLAY_MODE_RECORD \
+                ? (var = (value), replay_data_int(&var), var) \
+                : /* PLAY */ (replay_data_int(&var), var)))
+/*! Writes or reads buffer to/from replay log. */
+void replay_data_buffer(unsigned char *data, size_t size);
+/*! Macro for using replay_data_buffer function */
+#define REPLAY_DATA_VAR(var) \
+           replay_data_buffer((unsigned char *)&(var), sizeof(var))
 
 #endif

[Qemu-devel] [RFC PATCH v3 46/49] replay: replay_info command

[Pavel Dovgalyuk]

This patch adds support for replay_info monitor command. This command
returns the information about replay execution (replay mode and current step).

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hmp-commands.hx      |   13 +++++++++++++
 monitor.c            |   20 ++++++++++++++++++++
 qapi-schema.json     |   27 +++++++++++++++++++++++++++
 qmp-commands.hx      |   22 ++++++++++++++++++++++
 replay/Makefile.objs |    1 +
 replay/replay-qmp.c  |   29 +++++++++++++++++++++++++++++
 6 files changed, 112 insertions(+), 0 deletions(-)
 create mode 100755 replay/replay-qmp.c

diff --git a/hmp-commands.hx b/hmp-commands.hx
index d0943b1..19174f1 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1788,6 +1788,19 @@ STEXI
 show available trace events and their state
 ETEXI
 
+    {
+        .name       = "replay_info",
+        .args_type  = "",
+        .params     = "",
+        .help       = "show replay info",
+        .mhandler.cmd = do_replay_info,
+    },
+
+STEXI
+@item replay_info
+Shows information about replay process.
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/monitor.c b/monitor.c
index 5bc70a6..61fb11c 100644
--- a/monitor.c
+++ b/monitor.c
@@ -73,6 +73,7 @@
 #include "block/qapi.h"
 #include "qapi/qmp-event.h"
 #include "qapi-event.h"
+#include "replay/replay.h"
 
 /* for pic/irq_info */
 #if defined(TARGET_SPARC)
@@ -1173,6 +1174,25 @@ static void do_watchdog_action(Monitor *mon, const QDict *qdict)
     }
 }
 
+static void do_replay_info(Monitor *mon, const QDict *qdict)
+{
+    switch (replay_mode) {
+    case REPLAY_MODE_NONE:
+        monitor_printf(mon, "Replay is not enabled\n");
+        break;
+    default:
+        monitor_printf(mon, "Replay mode: %s ",
+                       ReplayMode_lookup[replay_mode]);
+        if (replay_mode == REPLAY_MODE_PLAY) {
+            monitor_printf(mon, "(%s)",
+                           ReplaySubmode_lookup[replay_get_play_submode()]);
+        }
+        monitor_printf(mon, "\n\tcurrent step: %" PRId64 "\n",
+                       replay_get_current_step());
+        break;
+    }
+}
+
 static void monitor_printc(Monitor *mon, int c)
 {
     monitor_printf(mon, "'");
diff --git a/qapi-schema.json b/qapi-schema.json
index 3f2dab4..93a7ff7 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3523,3 +3523,30 @@
 ##
 { 'enum': 'ReplaySubmode',
   'data': [ 'unknown', 'normal' ] }
+
+##
+# @ReplayInfo:
+#
+# Information about replay process
+#
+# @mode: replay mode (none, play, record)
+#
+# @submode: play submode (reverse or normal execution)
+#
+# @step: current step of record or play
+#
+# Since: 2.2
+##
+{ 'type': 'ReplayInfo',
+  'data': {'mode': 'ReplayMode', 'submode': 'ReplaySubmode', 'step': 'uint64'} }
+
+##
+# @replay-info
+#
+# Query the status of replay engine
+#
+# Returns: @ReplayInfo reflecting the status
+#
+# Since: 2.2
+##
+{ 'command': 'replay-info', 'returns': 'ReplayInfo' }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 4be4765..67dd886 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3755,3 +3755,25 @@ Example:
 <- { "return": {} }
 
 EQMP
+
+    {
+        .name       = "replay-info",
+        .args_type  = "",
+        .mhandler.cmd_new = qmp_marshal_input_replay_info,
+    },
+
+SQMP
+replay-info
+-----------
+
+Shows information about replay process. This information includes
+current execution step (number of executed processor instructions),
+replay mode (record, play or none), and replay submode (reverse or
+normal execution in play mode).
+
+Example:
+
+-> { "execute": "replay-info" }
+<- { "return": { "step": 651488674, "mode": "record", "submode": "unknown" } }
+
+EQMP
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index f4eef74..62eba1b 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -8,3 +8,4 @@ obj-y += replay-net.o
 obj-y += replay-audio.o
 obj-y += replay-char.o
 obj-y += replay-usb.o
+obj-y += replay-qmp.o
diff --git a/replay/replay-qmp.c b/replay/replay-qmp.c
new file mode 100755
index 0000000..4a0017d
--- /dev/null
+++ b/replay/replay-qmp.c
@@ -0,0 +1,29 @@
+/*
+ * replay-qmp.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "sysemu/sysemu.h"
+#include "qmp-commands.h"
+#include "qapi/qmp/qobject.h"
+#include "qapi/qmp-input-visitor.h"
+#include "replay/replay.h"
+#include "replay/replay-internal.h"
+
+ReplayInfo *qmp_replay_info(Error **errp)
+{
+    ReplayInfo *info = g_malloc0(sizeof(*info));
+
+    info->mode = replay_mode;
+    info->submode = replay_get_play_submode();
+    info->step = replay_get_current_step();
+
+    return info;
+}

[Qemu-devel] [RFC PATCH v3 47/49] replay: replay_break command

[Pavel Dovgalyuk]

This patch adds support for replay_break monitor command. This command
sets the step (measured in executed instructions) where replay should be
stopped.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hmp-commands.hx     |   14 ++++++++++++++
 monitor.c           |   20 ++++++++++++++++++++
 qapi-schema.json    |   19 ++++++++++++++++++-
 qmp-commands.hx     |   34 +++++++++++++++++++++++++++++++---
 replay/replay-qmp.c |   14 ++++++++++++++
 replay/replay.c     |   29 ++++++++++++++++++++++++++++-
 replay/replay.h     |    4 ++++
 7 files changed, 129 insertions(+), 5 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 19174f1..3eaa80e 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1801,6 +1801,20 @@ STEXI
 Shows information about replay process.
 ETEXI
 
+    {
+        .name       = "replay_break",
+        .args_type  = "step:l",
+        .params     = "step",
+        .help       = "stop replaying at the specified replay step",
+        .mhandler.cmd = do_replay_break,
+    },
+
+STEXI
+@item replay_break @var{step}
+Stops replaying at the specified @var{step}.
+
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/monitor.c b/monitor.c
index 61fb11c..1ba9096 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1184,8 +1184,12 @@ static void do_replay_info(Monitor *mon, const QDict *qdict)
         monitor_printf(mon, "Replay mode: %s ",
                        ReplayMode_lookup[replay_mode]);
         if (replay_mode == REPLAY_MODE_PLAY) {
+            uint64_t bp = replay_get_break_step();
             monitor_printf(mon, "(%s)",
                            ReplaySubmode_lookup[replay_get_play_submode()]);
+            if (bp != -1ULL) {
+                monitor_printf(mon, "\n\tbreakpoint step: %" PRId64 "\n", bp);
+            }
         }
         monitor_printf(mon, "\n\tcurrent step: %" PRId64 "\n",
                        replay_get_current_step());
@@ -1193,6 +1197,22 @@ static void do_replay_info(Monitor *mon, const QDict *qdict)
     }
 }
 
+static void do_replay_break(Monitor *mon, const QDict *qdict)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        uint64_t step = qdict_get_int(qdict, "step");
+        if (step >= replay_get_current_step()) {
+            monitor_printf(mon, "Setting break at step: %" PRId64 "\n", step);
+            replay_set_break(step);
+        } else {
+            monitor_printf(mon, "Cannot stop on the preceding step.\n");
+        }
+    } else {
+        monitor_printf(mon, "You can stop at the specific step "
+                            "only in PLAY mode.\n");
+    }
+}
+
 static void monitor_printc(Monitor *mon, int c)
 {
     monitor_printf(mon, "'");
diff --git a/qapi-schema.json b/qapi-schema.json
index 93a7ff7..e56e399 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3535,10 +3535,13 @@
 #
 # @step: current step of record or play
 #
+# @break-step: step where execution should stop
+#
 # Since: 2.2
 ##
 { 'type': 'ReplayInfo',
-  'data': {'mode': 'ReplayMode', 'submode': 'ReplaySubmode', 'step': 'uint64'} }
+  'data': { 'mode': 'ReplayMode', 'submode': 'ReplaySubmode',
+            'step': 'uint64', 'break-step': 'uint64' } }
 
 ##
 # @replay-info
@@ -3550,3 +3553,17 @@
 # Since: 2.2
 ##
 { 'command': 'replay-info', 'returns': 'ReplayInfo' }
+
+##
+# @replay-break
+#
+# Sets breakpoint at the specified step of replaying.
+# This command can be used in replay to stop execution when
+# the specified step is reached.
+# Current step of the execution can be queried with replay-info command.
+#
+# @step: step where breakpoint should be set
+#
+# Since: 2.2
+##
+{ 'command': 'replay-break', 'data': { 'step': 'uint64' } }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 67dd886..712cb5b 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3768,12 +3768,40 @@ replay-info
 
 Shows information about replay process. This information includes
 current execution step (number of executed processor instructions),
-replay mode (record, play or none), and replay submode (reverse or
-normal execution in play mode).
+replay mode (record, play or none), replay submode (reverse or
+normal execution in play mode), and breakpoint step.
 
 Example:
 
 -> { "execute": "replay-info" }
-<- { "return": { "step": 651488674, "mode": "record", "submode": "unknown" } }
+<- { "return": { "step": 651488674,
+                 "mode": "record", "submode": "unknown"
+                 "break-step": 651490000 } }
+
+EQMP
+
+    {
+        .name       = "replay-break",
+        .args_type  = "step:l",
+        .mhandler.cmd_new = qmp_marshal_input_replay_break,
+    },
+
+SQMP
+replay-break
+------------
+
+Sets breakpoint at the specified step of replaying.
+This command can be used in replay to stop execution when
+the specified step is reached.
+Current step of the execution can be queried with replay-info command.
+
+Arguments:
+
+- "step": step where breakpoint should be set
+
+Example:
+
+-> { "execute": "replay-break", "arguments": { "step": 1024 } }
+<- { "return": {} }
 
 EQMP
diff --git a/replay/replay-qmp.c b/replay/replay-qmp.c
index 4a0017d..f48b816 100755
--- a/replay/replay-qmp.c
+++ b/replay/replay-qmp.c
@@ -24,6 +24,20 @@ ReplayInfo *qmp_replay_info(Error **errp)
     info->mode = replay_mode;
     info->submode = replay_get_play_submode();
     info->step = replay_get_current_step();
+    info->break_step = replay_get_break_step();
 
     return info;
 }
+
+void qmp_replay_break(uint64_t step, Error **errp)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        if (step >= replay_get_current_step()) {
+            replay_set_break(step);
+        } else {
+            error_setg(errp, "Cannot stop on the preceding step");
+        }
+    } else {
+        error_setg(errp, "replay_break can be used only in PLAY mode");
+    }
+}
diff --git a/replay/replay.c b/replay/replay.c
index 4151330..fac5ecd 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -31,6 +31,8 @@ static char *replay_filename;
 char *replay_image_suffix;
 
 ReplayState replay_state;
+/*! Step for stopping execution at. */
+static uint64_t replay_break_step = -1;
 
 /*
     Auto-saving for VM states data
@@ -285,6 +287,19 @@ void replay_instruction(int process_events)
         } else if (replay_mode == REPLAY_MODE_PLAY) {
             skip_async_events_until(EVENT_INSTRUCTION);
             if (first_cpu->instructions_count >= 1) {
+                if (replay_get_current_step() == replay_break_step) {
+                    replay_break_step = -1;
+
+                    /* for stopping VM */
+                    if (play_submode == REPLAY_SUBMODE_NORMAL) {
+                        first_cpu->exception_index = EXCP_DEBUG;
+                        monitor_printf(default_mon, "Execution has stopped.\n");
+                        vm_stop(EXCP_DEBUG);
+                    }
+                    /* for breaking execution loop */
+                    cpu_exit(first_cpu);
+                    return;
+                }
                 ++replay_state.current_step;
                 --first_cpu->instructions_count;
                 if (first_cpu->instructions_count == 0) {
@@ -313,7 +328,8 @@ bool replay_has_async_request(void)
     }
 
     if (replay_mode == REPLAY_MODE_PLAY) {
-        if (skip_async_events(EVENT_ASYNC)) {
+        if (skip_async_events(EVENT_ASYNC)
+            || replay_get_current_step() == replay_break_step) {
             return true;
         }
 
@@ -471,6 +487,7 @@ static void replay_enable(const char *fname, int mode)
     replay_state.skipping_instruction = 0;
     replay_state.current_step = 0;
     current_saved_state = 0;
+    replay_break_step = -1;
 
     replay_net_init();
 
@@ -643,3 +660,13 @@ void replay_data_buffer(unsigned char *data, size_t size)
         replay_put_array(data, size);
     }
 }
+
+void replay_set_break(uint64_t step)
+{
+    replay_break_step = step;
+}
+
+uint64_t replay_get_break_step(void)
+{
+    return replay_break_step;
+}
diff --git a/replay/replay.h b/replay/replay.h
index 590f9e6..e737eba 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -53,6 +53,10 @@ void replay_configure(struct QemuOpts *opts, int mode);
 void replay_init_timer(void);
 /*! Closes replay log file and frees other resources. */
 void replay_finish(void);
+/*! Sets step where execution should be stopped. */
+void replay_set_break(uint64_t step);
+/*! Retrieves current breakpoint step. */
+uint64_t replay_get_break_step(void);
 
 /* Processing the instructions */
 

[Qemu-devel] [RFC PATCH v3 48/49] replay: replay_seek_step command

[Pavel Dovgalyuk]

This patch adds support for replay_seek_step monitor command. This command
loads one of the snapshots and replays the execution until the specified
step is met.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 hmp-commands.hx          |   14 ++++++++++++++
 monitor.c                |   18 ++++++++++++++++++
 qapi-schema.json         |   13 +++++++++++++
 qmp-commands.hx          |   25 +++++++++++++++++++++++++
 replay/replay-internal.h |    3 +++
 replay/replay-qmp.c      |   15 +++++++++++++++
 replay/replay.c          |   43 +++++++++++++++++++++++++++++++++++++++++++
 replay/replay.h          |    4 ++++
 8 files changed, 135 insertions(+), 0 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 3eaa80e..57ce5da 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1815,6 +1815,20 @@ Stops replaying at the specified @var{step}.
 
 ETEXI
 
+    {
+        .name       = "replay_seek",
+        .args_type  = "step:l",
+        .params     = "step",
+        .help       = "seeks the replay log to the specified step",
+        .mhandler.cmd = do_replay_seek,
+    },
+
+STEXI
+@item replay_seek @var{step}
+Seeks the replay log to the specified @var{step}.
+
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/monitor.c b/monitor.c
index 1ba9096..fdbd5a6 100644
--- a/monitor.c
+++ b/monitor.c
@@ -1213,6 +1213,24 @@ static void do_replay_break(Monitor *mon, const QDict *qdict)
     }
 }
 
+static void do_replay_seek(Monitor *mon, const QDict *qdict)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        uint64_t step = qdict_get_int(qdict, "step");
+        if (replay_seek_step(step)) {
+            monitor_printf(mon, "Seeking for step: %" PRId64 "\n", step);
+            if (!runstate_is_running()) {
+                vm_start();
+            }
+        } else {
+            monitor_printf(mon, "Cannot seek to the specified step.\n");
+        }
+    } else {
+        monitor_printf(mon, "You can go to the specific step "
+                            "only in PLAY mode.\n");
+    }
+}
+
 static void monitor_printc(Monitor *mon, int c)
 {
     monitor_printf(mon, "'");
diff --git a/qapi-schema.json b/qapi-schema.json
index e56e399..d8755a3 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3567,3 +3567,16 @@
 # Since: 2.2
 ##
 { 'command': 'replay-break', 'data': { 'step': 'uint64' } }
+
+##
+# @replay-seek
+#
+# Seeks the replay log to the specified step.
+# Loads nearest snapshot and replays the execution until
+# the destination step is reached,
+#
+# @step: destination replay step
+#
+# Since: 2.2
+##
+{ 'command': 'replay-seek', 'data': { 'step': 'uint64' } }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 712cb5b..cf642fa 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3805,3 +3805,28 @@ Example:
 <- { "return": {} }
 
 EQMP
+
+    {
+        .name       = "replay-seek",
+        .args_type  = "step:l",
+        .mhandler.cmd_new = qmp_marshal_input_replay_seek,
+    },
+
+SQMP
+replay-seek
+-----------
+
+Seeks the replay log to the specified step.
+Loads nearest snapshot and replays the execution until
+the destination step is reached,
+
+Arguments:
+
+- "step": destination replay step
+
+Example:
+
+-> { "execute": "replay-seek", "arguments": { "step": 1024 } }
+<- { "return": {} }
+
+EQMP
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index eeab40c..e529e9a 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -135,6 +135,9 @@ void skip_async_events_until(unsigned int kind);
     If the parameter is -1, the clock value is read to the cache anyway. */
 void replay_read_next_clock(unsigned int kind);
 
+/*! Finds saved state info which is nearest before the specified step. */
+SavedStateInfo *find_nearest_state(uint64_t step);
+
 /* Asynchronous events queue */
 
 /*! Initializes events' processing internals */
diff --git a/replay/replay-qmp.c b/replay/replay-qmp.c
index f48b816..04a20d5 100755
--- a/replay/replay-qmp.c
+++ b/replay/replay-qmp.c
@@ -41,3 +41,18 @@ void qmp_replay_break(uint64_t step, Error **errp)
         error_setg(errp, "replay_break can be used only in PLAY mode");
     }
 }
+
+void qmp_replay_seek(uint64_t step, Error **errp)
+{
+    if (replay_mode == REPLAY_MODE_PLAY) {
+        if (replay_seek_step(step)) {
+            if (!runstate_is_running()) {
+                vm_start();
+            }
+        } else {
+            error_setg(errp, "Cannot seek to the specified step");
+        }
+    } else {
+        error_setg(errp, "replay_seek can be used only in PLAY mode");
+    }
+}
diff --git a/replay/replay.c b/replay/replay.c
index fac5ecd..eaaee78 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -670,3 +670,46 @@ uint64_t replay_get_break_step(void)
 {
     return replay_break_step;
 }
+
+SavedStateInfo *find_nearest_state(uint64_t step)
+{
+    SavedStateInfo *first = saved_states;
+    SavedStateInfo *last = saved_states + saved_states_count;
+    while (first < last - 1) {
+        SavedStateInfo *next = first + (last - first) / 2;
+        if (next->step > step) {
+            last = next;
+        } else {
+            first = next;
+        }
+    }
+
+    return first;
+}
+
+int replay_seek_step(uint64_t step)
+{
+    if (replay_mode != REPLAY_MODE_PLAY) {
+        return 0;
+    }
+
+    /* load VM state, if possible */
+    if (saved_states_count > 0) {
+        /* find VM state to load */
+        SavedStateInfo *first = find_nearest_state(step);
+
+        if (first->step <= step
+            && (replay_get_current_step() > step
+                || replay_get_current_step() < first->step)) {
+            replay_loadvm(first - saved_states);
+        }
+    }
+
+    /* setup the breakpoint */
+    if (step >= replay_get_current_step()) {
+        replay_set_break(step);
+        return 1;
+    }
+
+    return 0;
+}
diff --git a/replay/replay.h b/replay/replay.h
index e737eba..70f8d84 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -57,6 +57,10 @@ void replay_finish(void);
 void replay_set_break(uint64_t step);
 /*! Retrieves current breakpoint step. */
 uint64_t replay_get_break_step(void);
+/*! Seeks to the specified step.
+    Loads VM state, if possible, and sets break to specified step.
+    Returns nonzero, when seeking was successful. */
+int replay_seek_step(uint64_t step);
 
 /* Processing the instructions */
 

[Qemu-devel] [RFC PATCH v3 49/49] gdbstub: reverse debugging

[Pavel Dovgalyuk]

This patch introduces support of reverse debugging through the gdb remote
protocol. Patch adds reverse-stepi and reverse-continue commands support
to qemu. Other reverse commands should also work, because they reuse these ones.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
---
 exec.c                      |    7 ++
 gdbstub.c                   |   79 +++++++++++++++++-----
 qapi-schema.json            |    2 -
 replay/Makefile.objs        |    1 
 replay/replay-debug.c       |  154 +++++++++++++++++++++++++++++++++++++++++++
 replay/replay-internal.h    |    4 +
 replay/replay.c             |    3 +
 replay/replay.h             |   11 +++
 target-arm/helper.h         |    1 
 target-arm/replay_helper.c  |    5 +
 target-arm/translate.c      |   14 +++-
 target-i386/helper.h        |    1 
 target-i386/replay_helper.c |    5 +
 target-i386/translate.c     |   12 ++-
 14 files changed, 272 insertions(+), 27 deletions(-)
 create mode 100755 replay/replay-debug.c

diff --git a/exec.c b/exec.c
index 8403cc5..f04ca43 100644
--- a/exec.c
+++ b/exec.c
@@ -1647,6 +1647,13 @@ static void check_watchpoint(int offset, int len_mask, int flags)
     QTAILQ_FOREACH(wp, &cpu->watchpoints, entry) {
         if ((vaddr == (wp->vaddr & len_mask) ||
              (vaddr & wp->len_mask) == wp->vaddr) && (wp->flags & flags)) {
+            /* Don't actually process a watchpoint, it will be processed,
+               when reverse execution stops. */
+            if (replay_get_play_submode() == REPLAY_SUBMODE_REVERSE) {
+                wp->flags &= ~BP_WATCHPOINT_HIT;
+                replay_reverse_breakpoint();
+                continue;
+            }
             wp->flags |= BP_WATCHPOINT_HIT;
             if (!cpu->watchpoint_hit) {
                 cpu->watchpoint_hit = wp;
diff --git a/gdbstub.c b/gdbstub.c
index 8afe0b7..228a36f 100644
--- a/gdbstub.c
+++ b/gdbstub.c
@@ -40,6 +40,7 @@
 #include "cpu.h"
 #include "qemu/sockets.h"
 #include "sysemu/kvm.h"
+#include "replay/replay.h"
 
 static inline int target_memory_rw_debug(CPUState *cpu, target_ulong addr,
                                          uint8_t *buf, int len, bool is_write)
@@ -313,6 +314,19 @@ typedef struct GDBState {
  */
 static int sstep_flags = SSTEP_ENABLE|SSTEP_NOIRQ|SSTEP_NOTIMER;
 
+/*! Retrieves flags for single step mode. */
+static int get_sstep_flags(void)
+{
+    /* In replay mode all events written into the log should be replayed.
+     * That is why NOIRQ flag is removed in this mode.
+     */
+    if (replay_mode != REPLAY_MODE_NONE) {
+        return SSTEP_ENABLE;
+    } else {
+        return sstep_flags;
+    }
+}
+
 static GDBState *gdbserver_state;
 
 bool gdb_has_xml;
@@ -835,7 +849,7 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
                     s->c_cpu = cpu;
                 }
                 if (res == 's') {
-                    cpu_single_step(s->c_cpu, sstep_flags);
+                    cpu_single_step(s->c_cpu, get_sstep_flags());
                 }
                 s->signal = res_signal;
                 gdb_continue(s);
@@ -863,9 +877,29 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
             addr = strtoull(p, (char **)&p, 16);
             gdb_set_cpu_pc(s, addr);
         }
-        cpu_single_step(s->c_cpu, sstep_flags);
+        cpu_single_step(s->c_cpu, get_sstep_flags());
         gdb_continue(s);
 	return RS_IDLE;
+    case 'b':
+        /* backward debugging commands */
+        if (replay_mode == REPLAY_MODE_PLAY
+            && replay_get_play_submode() == REPLAY_SUBMODE_NORMAL) {
+            switch (*p) {
+            case 's':
+                replay_reverse_step();
+                gdb_continue(s);
+                return RS_IDLE;
+            case 'c':
+                replay_reverse_continue();
+                gdb_continue(s);
+                return RS_IDLE;
+            default:
+                goto unknown_command;
+            }
+        } else {
+            put_packet(s, "E22");
+        }
+        goto unknown_command;
     case 'F':
         {
             target_ulong ret;
@@ -937,8 +971,6 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
         if (target_memory_rw_debug(s->g_cpu, addr, mem_buf, len,
                                    true) != 0) {
             put_packet(s, "E14");
-        } else {
-            put_packet(s, "OK");
         }
         break;
     case 'p':
@@ -1035,18 +1067,23 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
             put_packet(s, buf);
             break;
         } else if (strncmp(p,"qemu.sstep",10) == 0) {
-            /* Display or change the sstep_flags */
-            p += 10;
-            if (*p != '=') {
-                /* Display current setting */
-                snprintf(buf, sizeof(buf), "0x%x", sstep_flags);
-                put_packet(s, buf);
-                break;
+            if (replay_mode == REPLAY_MODE_NONE) {
+                /* Display or change the sstep_flags */
+                p += 10;
+                if (*p != '=') {
+                    /* Display current setting */
+                    snprintf(buf, sizeof(buf), "0x%x", sstep_flags);
+                    put_packet(s, buf);
+                    break;
+                }
+                p++;
+                type = strtoul(p, (char **)&p, 16);
+                sstep_flags = type;
+                put_packet(s, "OK");
+            } else {
+                /* Cannot change sstep flags in replay mode */
+                put_packet(s, "E22");
             }
-            p++;
-            type = strtoul(p, (char **)&p, 16);
-            sstep_flags = type;
-            put_packet(s, "OK");
             break;
         } else if (strcmp(p,"C") == 0) {
             /* "Current thread" remains vague in the spec, so always return
@@ -1113,6 +1150,9 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
             if (cc->gdb_core_xml_file != NULL) {
                 pstrcat(buf, sizeof(buf), ";qXfer:features:read+");
             }
+            if (replay_mode == REPLAY_MODE_PLAY) {
+                pstrcat(buf, sizeof(buf), ";ReverseStep+;ReverseContinue+");
+            }
             put_packet(s, buf);
             break;
         }
@@ -1174,8 +1214,13 @@ static int gdb_handle_packet(GDBState *s, const char *line_buf)
 
 void gdb_set_stop_cpu(CPUState *cpu)
 {
-    gdbserver_state->c_cpu = cpu;
-    gdbserver_state->g_cpu = cpu;
+    /* DEBUG interrupts are also used by replay module.
+       In some cases gdb is not connected, when replay is used.
+       This check is added to prevent faults in such cases. */
+    if (gdbserver_state) {
+        gdbserver_state->c_cpu = cpu;
+        gdbserver_state->g_cpu = cpu;
+    }
 }
 
 #ifndef CONFIG_USER_ONLY
diff --git a/qapi-schema.json b/qapi-schema.json
index d8755a3..d71fe76 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3522,7 +3522,7 @@
 # Since: 2.2
 ##
 { 'enum': 'ReplaySubmode',
-  'data': [ 'unknown', 'normal' ] }
+  'data': [ 'unknown', 'normal', 'reverse' ] }
 
 ##
 # @ReplayInfo:
diff --git a/replay/Makefile.objs b/replay/Makefile.objs
index 62eba1b..cb99839 100755
--- a/replay/Makefile.objs
+++ b/replay/Makefile.objs
@@ -9,3 +9,4 @@ obj-y += replay-audio.o
 obj-y += replay-char.o
 obj-y += replay-usb.o
 obj-y += replay-qmp.o
+obj-y += replay-debug.o
diff --git a/replay/replay-debug.c b/replay/replay-debug.c
new file mode 100755
index 0000000..8785e68
--- /dev/null
+++ b/replay/replay-debug.c
@@ -0,0 +1,154 @@
+/*
+ * replay-debug.c
+ *
+ * Copyright (c) 2010-2014 Institute for System Programming
+ *                         of the Russian Academy of Sciences.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "exec/cpu-common.h"
+#include "exec/cpu-defs.h"
+
+#include "replay.h"
+#include "replay-internal.h"
+
+/* Reverse debugging data */
+
+/* Saved handler of the debug exception */
+static CPUDebugExcpHandler *prev_debug_excp_handler;
+/* Step of the last breakpoint hit.
+   Used for seeking in reverse continue mode. */
+static uint64_t last_breakpoint_step;
+/* Start step, where reverse continue begins,
+   or target step for reverse stepping.*/
+static uint64_t last_reverse_step;
+/* Start step, where reverse continue begins.*/
+static uint64_t start_reverse_step;
+/* Previously loaded step for reverse continue */
+static SavedStateInfo *reverse_state;
+
+/*! Breakpoint handler for pass2 of reverse continue.
+    Stops the execution at previously saved breakpoint step. */
+static void reverse_continue_pass2_breakpoint_handler(CPUArchState *env)
+{
+    if (replay_get_current_step() == last_breakpoint_step) {
+        CPUState *cpu = ENV_GET_CPU(env);
+        CPUDebugExcpHandler *handler = prev_debug_excp_handler;
+        prev_debug_excp_handler = NULL;
+
+        play_submode = REPLAY_SUBMODE_NORMAL;
+
+        cpu->exception_index = EXCP_DEBUG;
+        /* invoke the breakpoint */
+        cpu_set_debug_excp_handler(handler);
+        handler(env);
+        cpu_exit(cpu);
+    }
+}
+
+/*! Breakpoint handler for pass1 of reverse continue.
+    Saves last breakpoint hit and switches to pass2
+    when starting point is reached. */
+static void reverse_continue_pass1_breakpoint_handler(CPUArchState *env)
+{
+    if (replay_get_current_step() == last_reverse_step) {
+        CPUState *cpu = ENV_GET_CPU(env);
+        /* repeat first pass if breakpoint was not found
+           on current iteration */
+        if (last_breakpoint_step == reverse_state->step - 1
+            && reverse_state != saved_states) {
+            last_reverse_step = reverse_state->step;
+            /* load previous state */
+            --reverse_state;
+            last_breakpoint_step = reverse_state->step - 1;
+            replay_seek_step(reverse_state->step);
+            /* set break should be after seek, because seek resets break */
+            replay_set_break(last_reverse_step);
+            cpu_loop_exit(cpu);
+        } else {
+            /* this condition is needed, when no breakpoints were found */
+            if (last_breakpoint_step == reverse_state->step - 1) {
+                ++last_breakpoint_step;
+            }
+            cpu_set_debug_excp_handler(
+                reverse_continue_pass2_breakpoint_handler);
+
+            reverse_continue_pass2_breakpoint_handler(env);
+            replay_seek_step(last_breakpoint_step);
+            cpu_loop_exit(cpu);
+        }
+    } else {
+        /* skip watchpoint/breakpoint at the current step
+           to allow reverse continue */
+        last_breakpoint_step = replay_get_current_step();
+    }
+}
+
+void replay_reverse_breakpoint(void)
+{
+    /* we started reverse execution from a breakpoint */
+    if (replay_get_current_step() != start_reverse_step) {
+        last_breakpoint_step = replay_get_current_step();
+    }
+}
+
+void replay_reverse_continue(void)
+{
+    if (replay_mode == REPLAY_MODE_PLAY
+        && play_submode == REPLAY_SUBMODE_NORMAL) {
+        tb_flush_all();
+        play_submode = REPLAY_SUBMODE_REVERSE;
+
+        last_reverse_step = replay_get_current_step();
+        start_reverse_step = replay_get_current_step();
+        /* load initial state */
+        reverse_state = find_nearest_state(replay_get_current_step());
+        replay_seek_step(reverse_state->step);
+        /* run to current step */
+        replay_set_break(last_reverse_step);
+        /* decrement to allow breaking at the first step */
+        last_breakpoint_step = reverse_state->step - 1;
+        prev_debug_excp_handler =
+            cpu_set_debug_excp_handler(
+                reverse_continue_pass1_breakpoint_handler);
+    }
+}
+
+/*! Breakpoint handler for reverse stepping.
+    Stops at the desired step and skips other breakpoints. */
+static void reverse_step_breakpoint_handler(CPUArchState *env)
+{
+    if (replay_get_current_step() == last_reverse_step) {
+        CPUState *cpu = ENV_GET_CPU(env);
+        CPUDebugExcpHandler *handler = prev_debug_excp_handler;
+        prev_debug_excp_handler = NULL;
+
+        play_submode = REPLAY_SUBMODE_NORMAL;
+
+        cpu->exception_index = EXCP_DEBUG;
+        /* invoke the breakpoint */
+        cpu_set_debug_excp_handler(handler);
+        handler(env);
+        cpu_exit(cpu);
+    }
+}
+
+void replay_reverse_step(void)
+{
+    if (replay_mode == REPLAY_MODE_PLAY
+        && play_submode == REPLAY_SUBMODE_NORMAL
+        && replay_get_current_step() > 0) {
+        tb_flush_all();
+        play_submode = REPLAY_SUBMODE_REVERSE;
+
+        last_reverse_step = replay_get_current_step() - 1;
+        replay_seek_step(last_reverse_step);
+
+        prev_debug_excp_handler =
+            cpu_set_debug_excp_handler(reverse_step_breakpoint_handler);
+    }
+}
diff --git a/replay/replay-internal.h b/replay/replay-internal.h
index e529e9a..ec42a94 100755
--- a/replay/replay-internal.h
+++ b/replay/replay-internal.h
@@ -88,7 +88,11 @@ struct SavedStateInfo {
 };
 /*! Reference to the saved state */
 typedef struct SavedStateInfo SavedStateInfo;
+/* List of the saved states information */
+extern SavedStateInfo *saved_states;
 
+/*! Stores current submode for PLAY mode */
+extern ReplaySubmode play_submode;
 extern volatile unsigned int replay_data_kind;
 extern volatile unsigned int replay_has_unread_data;
 
diff --git a/replay/replay.c b/replay/replay.c
index eaaee78..651653c 100755
--- a/replay/replay.c
+++ b/replay/replay.c
@@ -295,6 +295,9 @@ void replay_instruction(int process_events)
                         first_cpu->exception_index = EXCP_DEBUG;
                         monitor_printf(default_mon, "Execution has stopped.\n");
                         vm_stop(EXCP_DEBUG);
+                    } else if (play_submode == REPLAY_SUBMODE_REVERSE) {
+                        cpu_handle_debug_exception(first_cpu->env_ptr);
+                        return;
                     }
                     /* for breaking execution loop */
                     cpu_exit(first_cpu);
diff --git a/replay/replay.h b/replay/replay.h
index 70f8d84..58beed8 100755
--- a/replay/replay.h
+++ b/replay/replay.h
@@ -184,6 +184,17 @@ void replay_req_complete_iso(struct libusb_transfer *xfer);
     to be completed by reading it from the log */
 void replay_req_register_iso(struct libusb_transfer *xfer);
 
+/* Reverse debugging */
+
+/*! Initializes reverse continue execution.
+    Called by debugger module. */
+void replay_reverse_continue(void);
+/*! Initializes reverse step execution.
+    Called by debugger module. */
+void replay_reverse_step(void);
+/* Called instead of invoking breakpoint in reverse continue mode. */
+void replay_reverse_breakpoint(void);
+
 /* Other data */
 
 /*! Writes or reads integer value to/from replay log. */
diff --git a/target-arm/helper.h b/target-arm/helper.h
index 0233b64..802ff9d 100644
--- a/target-arm/helper.h
+++ b/target-arm/helper.h
@@ -531,3 +531,4 @@ DEF_HELPER_FLAGS_2(neon_pmull_64_hi, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 #endif
 
 DEF_HELPER_1(replay_instruction, i32, env)
+DEF_HELPER_0(reverse_breakpoint, void)
diff --git a/target-arm/replay_helper.c b/target-arm/replay_helper.c
index 418d548..b529ffd 100755
--- a/target-arm/replay_helper.c
+++ b/target-arm/replay_helper.c
@@ -31,3 +31,8 @@ uint32_t helper_replay_instruction(CPUARMState *env)
     replay_instruction(timer);
     return timer;
 }
+
+void helper_reverse_breakpoint(void)
+{
+    replay_reverse_breakpoint();
+}
diff --git a/target-arm/translate.c b/target-arm/translate.c
index 3407ced..7aa9d1b 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -11021,11 +11021,15 @@ static inline void gen_intermediate_code_internal(ARMCPU *cpu,
         if (unlikely(!QTAILQ_EMPTY(&cs->breakpoints))) {
             QTAILQ_FOREACH(bp, &cs->breakpoints, entry) {
                 if (bp->pc == dc->pc) {
-                    gen_exception_internal_insn(dc, 0, EXCP_DEBUG);
-                    /* Advance PC so that clearing the breakpoint will
-                       invalidate this TB.  */
-                    dc->pc += 2;
-                    goto done_generating;
+                    if (replay_get_play_submode() == REPLAY_SUBMODE_REVERSE) {
+                        gen_helper_reverse_breakpoint();
+                    } else {
+                        gen_exception_internal_insn(dc, 0, EXCP_DEBUG);
+                        /* Advance PC so that clearing the breakpoint will
+                           invalidate this TB.  */
+                        dc->pc += 2;
+                        goto done_generating;
+                    }
                 }
             }
         }
diff --git a/target-i386/helper.h b/target-i386/helper.h
index 058302b..9aacef1 100644
--- a/target-i386/helper.h
+++ b/target-i386/helper.h
@@ -219,3 +219,4 @@ DEF_HELPER_3(rcrq, tl, env, tl, tl)
 #endif
 
 DEF_HELPER_1(replay_instruction, i32, env)
+DEF_HELPER_0(reverse_breakpoint, void)
diff --git a/target-i386/replay_helper.c b/target-i386/replay_helper.c
index 7e70c78..6c8d705 100755
--- a/target-i386/replay_helper.c
+++ b/target-i386/replay_helper.c
@@ -31,3 +31,8 @@ uint32_t helper_replay_instruction(CPUX86State *env)
     replay_instruction(timer);
     return timer;
 }
+
+void helper_reverse_breakpoint(void)
+{
+    replay_reverse_breakpoint();
+}
diff --git a/target-i386/translate.c b/target-i386/translate.c
index 1843b46..d0c04a7 100644
--- a/target-i386/translate.c
+++ b/target-i386/translate.c
@@ -2536,10 +2536,14 @@ static void gen_interrupt(DisasContext *s, int intno,
 
 static void gen_debug(DisasContext *s, target_ulong cur_eip)
 {
-    gen_update_cc_op(s);
-    gen_jmp_im(cur_eip);
-    gen_helper_debug(cpu_env);
-    s->is_jmp = DISAS_TB_JUMP;
+    if (replay_get_play_submode() == REPLAY_SUBMODE_REVERSE) {
+        gen_helper_reverse_breakpoint();
+    } else {
+        gen_update_cc_op(s);
+        gen_jmp_im(cur_eip);
+        gen_helper_debug(cpu_env);
+        s->is_jmp = DISAS_TB_JUMP;
+    }
 }
 
 /* generate a generic end of block. Trace exception is also generated

Re: [Qemu-devel] [RFC PATCH v3 04/49] fdc: adding vmstate for save/restore

[Paolo Bonzini]

Il 31/07/2014 14:53, Pavel Dovgalyuk ha scritto:
> VMState added by this patch preserves correct
> loading of the FDC device state.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/block/fdc.c |   85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 files changed, 83 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/block/fdc.c b/hw/block/fdc.c
> index 490d127..7305f75 100644
> --- a/hw/block/fdc.c
> +++ b/hw/block/fdc.c
> @@ -695,10 +695,34 @@ static const VMStateDescription vmstate_fdrive_media_rate = {
>      }
>  };
>  
> +static bool fdrive_perpendicular_needed(void *opaque)
> +{
> +    FDrive *drive = opaque;
> +
> +    return drive->perpendicular != 0;
> +}
> +
> +static const VMStateDescription vmstate_fdrive_perpendicular = {
> +    .name = "fdrive/perpendicular",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT8(perpendicular, FDrive),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
> +static int fdrive_post_load(void *opaque, int version_id)
> +{
> +    fd_revalidate(opaque);
> +    return 0;
> +}
> +
>  static const VMStateDescription vmstate_fdrive = {
>      .name = "fdrive",
> -    .version_id = 1,
> +    .version_id = 2,

This is not needed when you add a subsectino.

Paolo

>      .minimum_version_id = 1,
> +    .post_load = fdrive_post_load,
>      .fields = (VMStateField[]) {
>          VMSTATE_UINT8(head, FDrive),
>          VMSTATE_UINT8(track, FDrive),
> @@ -713,6 +737,9 @@ static const VMStateDescription vmstate_fdrive = {
>              .vmsd = &vmstate_fdrive_media_rate,
>              .needed = &fdrive_media_rate_needed,
>          } , {
> +            .vmsd = &vmstate_fdrive_perpendicular,
> +            .needed = &fdrive_perpendicular_needed,
> +        } , {
>              /* empty */
>          }
>      }
> @@ -725,6 +752,14 @@ static void fdc_pre_save(void *opaque)
>      s->dor_vmstate = s->dor | GET_CUR_DRV(s);
>  }
>  
> +static int fdc_pre_load(void *opaque)
> +{
> +    FDCtrl *s = opaque;
> +    s->reset_sensei = 0;
> +    timer_del(s->result_timer);
> +    return 0;
> +}
> +
>  static int fdc_post_load(void *opaque, int version_id)
>  {
>      FDCtrl *s = opaque;
> @@ -734,11 +769,46 @@ static int fdc_post_load(void *opaque, int version_id)
>      return 0;
>  }
>  
> +static bool fdc_reset_sensei_needed(void *opaque)
> +{
> +    FDCtrl *s = opaque;
> +
> +    return s->reset_sensei != 0;
> +}
> +
> +static const VMStateDescription vmstate_fdc_reset_sensei = {
> +    .name = "fdc/reset_sensei",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_INT32(reset_sensei, FDCtrl),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
> +static bool fdc_result_timer_needed(void *opaque)
> +{
> +    FDCtrl *s = opaque;
> +
> +    return timer_pending(s->result_timer);
> +}
> +
> +static const VMStateDescription vmstate_fdc_result_timer = {
> +    .name = "fdc/result_timer",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_TIMER(result_timer, FDCtrl),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
>  static const VMStateDescription vmstate_fdc = {
>      .name = "fdc",
> -    .version_id = 2,
> +    .version_id = 3,

This is not needed when you add a subsection.

>      .minimum_version_id = 2,
>      .pre_save = fdc_pre_save,
> +    .pre_load = fdc_pre_load,
>      .post_load = fdc_post_load,
>      .fields = (VMStateField[]) {
>          /* Controller State */
> @@ -770,6 +840,17 @@ static const VMStateDescription vmstate_fdc = {
>          VMSTATE_STRUCT_ARRAY(drives, FDCtrl, MAX_FD, 1,
>                               vmstate_fdrive, FDrive),
>          VMSTATE_END_OF_LIST()
> +    },
> +    .subsections = (VMStateSubsection[]) {
> +        {
> +            .vmsd = &vmstate_fdc_reset_sensei,
> +            .needed = fdc_reset_sensei_needed,
> +        } , {
> +            .vmsd = &vmstate_fdc_result_timer,
> +            .needed = fdc_result_timer_needed,
> +        } , {
> +            /* empty */
> +        }
>      }
>  };
>  
> 

Re: [Qemu-devel] [RFC PATCH v3 06/49] serial: fixing vmstate for save/restore

[Paolo Bonzini]

Il 31/07/2014 14:54, Pavel Dovgalyuk ha scritto:
> +static bool serial_thr_ipending_needed(void *opaque)
> +{
> +    SerialState *s = (SerialState *)opaque;
> +    return (s->iir & UART_IIR_ID) != UART_IIR_THRI;
> +}

This is not correct, as you pointed out in our earlier discussion.
thr_ipending is independent from s->iir, it just happens that s->iir is
a good way to fetch the common value of thr_ipending and thus avoid
streaming the subsection in most cases.

>  const VMStateDescription vmstate_serial = {
>      .name = "serial",
> -    .version_id = 3,
> +    .version_id = 4,

As usual, this is not necessary.

Paolo

>      .minimum_version_id = 2,
>      .pre_save = serial_pre_save,
> +    .pre_load = serial_pre_load,

Re: [Qemu-devel] [RFC PATCH v3 07/49] kvmapic: fixing loading vmstate

[Paolo Bonzini]

Il 31/07/2014 14:54, Pavel Dovgalyuk ha scritto:
> +static int apic_pre_load(void *opaque)
> +{
> +    APICCommonState *s = APIC_COMMON(opaque);
> +    s->sipi_vector = 0;
> +    s->wait_for_sipi = 0;
> +    s->vapic_paddr = 0;
> +    return 0;

Is this necessary?  Or does reset already do that?

> +}
> +
>  static void apic_dispatch_pre_save(void *opaque)
>  {
>      APICCommonState *s = APIC_COMMON(opaque);
> @@ -345,12 +354,46 @@ static int apic_dispatch_post_load(void *opaque, int version_id)
>      return 0;
>  }
>  
> +static bool apic_common_sipi_needed(void *opaque)
> +{
> +    APICCommonState *s = APIC_COMMON(opaque);
> +    return s->wait_for_sipi != 0;
> +}
> +
> +static const VMStateDescription vmstate_apic_common_sipi = {
> +    .name = "apic_sipi",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_INT32(sipi_vector, APICCommonState),
> +        VMSTATE_INT32(wait_for_sipi, APICCommonState),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
> +static bool apic_common_vapic_paddr_needed(void *opaque)
> +{
> +    APICCommonState *s = APIC_COMMON(opaque);
> +    return s->vapic_paddr != 0;
> +}
> +
> +static const VMStateDescription vmstate_apic_common_vapic_paddr = {
> +    .name = "apic_vapic_paddr",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT64(vapic_paddr, APICCommonState),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
>  static const VMStateDescription vmstate_apic_common = {
>      .name = "apic",
> -    .version_id = 3,
> +    .version_id = 4,

You know what I'm going to say. :)

Paolo

Re: [Qemu-devel] [RFC PATCH v3 09/49] pckbd: adding new fields to vmstate

[Paolo Bonzini]

Il 31/07/2014 14:54, Pavel Dovgalyuk ha scritto:
>      .name = "pckbd",
> -    .version_id = 3,
> +    .version_id = 4,

Patch is okay apart from this recurring issue.

Paolo

Re: [Qemu-devel] [RFC PATCH v3 49/49] gdbstub: reverse debugging

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

On 07/31/2014 06:58 AM, Pavel Dovgalyuk wrote:
> This patch introduces support of reverse debugging through the gdb remote
> protocol. Patch adds reverse-stepi and reverse-continue commands support
> to qemu. Other reverse commands should also work, because they reuse these ones.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---

> +++ b/qapi-schema.json
> @@ -3522,7 +3522,7 @@
>  # Since: 2.2
>  ##
>  { 'enum': 'ReplaySubmode',
> -  'data': [ 'unknown', 'normal' ] }
> +  'data': [ 'unknown', 'normal', 'reverse' ] }

Should this mode be added in patch 18/49?  Also, you forgot to document it.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]

Re: [Qemu-devel] [RFC PATCH v3 10/49] rtl8139: adding new fields to vmstate

[Paolo Bonzini]

Il 31/07/2014 14:54, Pavel Dovgalyuk ha scritto:
> +static bool rtl8139_TimerExpire_needed(void *opaque)
> +{
> +    RTL8139State *s = (RTL8139State *)opaque;
> +    return s->TimerExpire != 0;
> +}
> +
> +static const VMStateDescription vmstate_rtl8139_TimerExpire = {
> +    .name = "rtl8139/TimerExpire",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_INT64(TimerExpire, RTL8139State),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
> +static bool rtl8139_timer_needed(void *opaque)
> +{
> +    RTL8139State *s = (RTL8139State *)opaque;
> +    return timer_pending(s->timer);
> +}

I think this is needed only because of the complicated logic to compute
PCSTimeout on the fly when IntrStatus/IntrMask change or are read.
But we can just eliminate it, and leave the timer always set when
TimerInt is nonzero.

rtl8139_set_next_tctr_time should now be idempotent and deterministic.
Then it can be set in rtl8139_post_load.


---------------- 8< ---------------
From: Paolo Bonzini <pbonzini@redhat.com>
Subject: [PATCH] rtl8139: simplify timer logic

Pavel Dovgalyuk reports that TimerExpire and the timer are not restored
correctly on the receiving end of migration.

It is not clear to me whether this is really the case, but we can take
the occasion to get rid of the complicated code that computes PCSTimeout
on the fly upon changes to IntrStatus/IntrMask.  Just always keep a
timer running, it will fire every ~130 seconds at most if the interrupt
is masked with TimerInt != 0.

This makes rtl8139_set_next_tctr_time idempotent (when called with
wrap_on_overflow == false, as is the case from the post_load callback).

Not-tested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

diff --git a/hw/net/rtl8139.c b/hw/net/rtl8139.c
index 90bc5ec..430eab3 100644
--- a/hw/net/rtl8139.c
+++ b/hw/net/rtl8139.c
@@ -508,7 +508,6 @@ typedef struct RTL8139State {
 
     /* PCI interrupt timer */
     QEMUTimer *timer;
-    int64_t TimerExpire;
 
     MemoryRegion bar_io;
     MemoryRegion bar_mem;
@@ -520,7 +519,7 @@ typedef struct RTL8139State {
 /* Writes tally counters to memory via DMA */
 static void RTL8139TallyCounters_dma_write(RTL8139State *s, dma_addr_t tc_addr);
 
-static void rtl8139_set_next_tctr_time(RTL8139State *s, int64_t current_time);
+static void rtl8139_set_next_tctr_time(RTL8139State *s, bool wrap_on_overflow);
 
 static void prom9346_decode_command(EEprom9346 *eeprom, uint8_t command)
 {
@@ -1282,6 +1281,7 @@ static void rtl8139_reset(DeviceState *d)
     s->TCTR = 0;
     s->TimerInt = 0;
     s->TCTR_base = 0;
+    rtl8139_set_next_tctr_time(s, false);
 
     /* reset tally counters */
     RTL8139TallyCounters_clear(&s->tally_counters);
@@ -2648,7 +2648,6 @@ static void rtl8139_IntrMask_write(RTL8139State *s, uint32_t val)
 
     s->IntrMask = val;
 
-    rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
     rtl8139_update_irq(s);
 
 }
@@ -2683,13 +2682,7 @@ static void rtl8139_IntrStatus_write(RTL8139State *s, uint32_t val)
     rtl8139_update_irq(s);
 
     s->IntrStatus = newStatus;
-    /*
-     * Computing if we miss an interrupt here is not that correct but
-     * considered that we should have had already an interrupt
-     * and probably emulated is slower is better to assume this resetting was
-     * done before testing on previous rtl8139_update_irq lead to IRQ losing
-     */
-    rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+    rtl8139_set_next_tctr_time(s, true);
     rtl8139_update_irq(s);
 
 #endif
@@ -2697,8 +2690,6 @@ static void rtl8139_IntrStatus_write(RTL8139State *s, uint32_t val)
 
 static uint32_t rtl8139_IntrStatus_read(RTL8139State *s)
 {
-    rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
-
     uint32_t ret = s->IntrStatus;
 
     DPRINTF("IntrStatus read(w) val=0x%04x\n", ret);
@@ -2881,43 +2872,25 @@ static void rtl8139_io_writew(void *opaque, uint8_t addr, uint32_t val)
     }
 }
 
-static void rtl8139_set_next_tctr_time(RTL8139State *s, int64_t current_time)
+static void rtl8139_set_next_tctr_time(RTL8139State *s, bool wrap_on_overflow)
 {
-    int64_t pci_time, next_time;
-    uint32_t low_pci;
+    int64_t delta;
 
     DPRINTF("entered rtl8139_set_next_tctr_time\n");
 
-    if (s->TimerExpire && current_time >= s->TimerExpire) {
-        s->IntrStatus |= PCSTimeout;
-        rtl8139_update_irq(s);
-    }
-
-    /* Set QEMU timer only if needed that is
-     * - TimerInt <> 0 (we have a timer)
-     * - mask = 1 (we want an interrupt timer)
-     * - irq = 0  (irq is not already active)
-     * If any of above change we need to compute timer again
-     * Also we must check if timer is passed without QEMU timer
-     */
-    s->TimerExpire = 0;
     if (!s->TimerInt) {
-        return;
-    }
-
-    pci_time = muldiv64(current_time - s->TCTR_base, PCI_FREQUENCY,
-                                get_ticks_per_sec());
-    low_pci = pci_time & 0xffffffff;
-    pci_time = pci_time - low_pci + s->TimerInt;
-    if (low_pci >= s->TimerInt) {
-        pci_time += 0x100000000LL;
-    }
-    next_time = s->TCTR_base + muldiv64(pci_time, get_ticks_per_sec(),
-                                                PCI_FREQUENCY);
-    s->TimerExpire = next_time;
-
-    if ((s->IntrMask & PCSTimeout) != 0 && (s->IntrStatus & PCSTimeout) == 0) {
-        timer_mod(s->timer, next_time);
+        timer_del(s->timer);
+    } else {
+        delta = muldiv64(s->TimerInt, get_ticks_per_sec(), PCI_FREQUENCY);
+        if (wrap_on_overflow &&
+            s->TCTR_base + delta >= qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL)) {
+            /* The value in the Timer register does not change, but
+             * we're counting up by 2^32 ticks (approx. 130 seconds).
+             */
+            s->TCTR_base += muldiv64(0x100000000LL, get_ticks_per_sec(),
+                                     PCI_FREQUENCY);
+        }
+        timer_mod(s->timer, s->TCTR_base + delta);
     }
 }
 
@@ -2965,14 +2938,14 @@ static void rtl8139_io_writel(void *opaque, uint8_t addr, uint32_t val)
         case Timer:
             DPRINTF("TCTR Timer reset on write\n");
             s->TCTR_base = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-            rtl8139_set_next_tctr_time(s, s->TCTR_base);
+            rtl8139_set_next_tctr_time(s, false);
             break;
 
         case FlashReg:
             DPRINTF("FlashReg TimerInt write val=0x%08x\n", val);
             if (s->TimerInt != val) {
                 s->TimerInt = val;
-                rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+                rtl8139_set_next_tctr_time(s, true);
             }
             break;
 
@@ -3249,7 +3222,7 @@ static uint32_t rtl8139_mmio_readl(void *opaque, hwaddr addr)
 static int rtl8139_post_load(void *opaque, int version_id)
 {
     RTL8139State* s = opaque;
-    rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+    rtl8139_set_next_tctr_time(s, false);
     if (version_id < 4) {
         s->cplus_enabled = s->CpCmd != 0;
     }
@@ -3280,8 +3253,7 @@ static void rtl8139_pre_save(void *opaque)
     RTL8139State* s = opaque;
     int64_t current_time = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 
-    /* set IntrStatus correctly */
-    rtl8139_set_next_tctr_time(s, current_time);
+    /* for migration to older versions */
     s->TCTR = muldiv64(current_time - s->TCTR_base, PCI_FREQUENCY,
                        get_ticks_per_sec());
     s->rtl8139_mmio_io_addr_dummy = 0;
@@ -3448,7 +3420,7 @@ static void rtl8139_timer(void *opaque)
 
     s->IntrStatus |= PCSTimeout;
     rtl8139_update_irq(s);
-    rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+    rtl8139_set_next_tctr_time(s, true);
 }
 
 static void rtl8139_cleanup(NetClientState *nc)
@@ -3536,9 +3508,7 @@ static int pci_rtl8139_init(PCIDevice *dev)
     s->cplus_txbuffer_len = 0;
     s->cplus_txbuffer_offset = 0;
 
-    s->TimerExpire = 0;
     s->timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, rtl8139_timer, s);
-    rtl8139_set_next_tctr_time(s, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
 
     add_boot_device_path(s->conf.bootindex, d, "/ethernet-phy@0");
 

Re: [Qemu-devel] [RFC PATCH v3 12/49] mc146818rtc: add missed field to vmstate

[Paolo Bonzini]

Il 31/07/2014 14:54, Pavel Dovgalyuk ha scritto:
> This patch adds irq_reinject_on_ack_count field to VMState to allow correct
> saving/loading the state of MC146818 RTC.
> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/timer/mc146818rtc.c |   34 +++++++++++++++++++++++++++++++++-
>  1 files changed, 33 insertions(+), 1 deletions(-)
> 
> diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
> index 233fc70..0b78d88 100644
> --- a/hw/timer/mc146818rtc.c
> +++ b/hw/timer/mc146818rtc.c
> @@ -698,6 +698,13 @@ int rtc_get_memory(ISADevice *dev, int addr)
>      return s->cmos_data[addr];
>  }
>  
> +static int rtc_pre_load(void *opaque)
> +{
> +    RTCState *s = (RTCState *)opaque;
> +    s->irq_reinject_on_ack_count = 0;

Probably not needed?

> +    return 0;
> +}
> +
>  static void rtc_set_date_from_host(ISADevice *dev)
>  {
>      RTCState *s = MC146818_RTC(dev);
> @@ -733,10 +740,27 @@ static int rtc_post_load(void *opaque, int version_id)
>      return 0;
>  }
>  
> +static const VMStateDescription vmstate_rtc_irq_reinject_on_ack_count = {
> +    .name = "irq_reinject_on_ack_count",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT16(irq_reinject_on_ack_count, RTCState),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
> +static bool rtc_irq_reinject_on_ack_count_needed(void *opaque)
> +{
> +    RTCState *s = (RTCState *)opaque;
> +    return s->irq_reinject_on_ack_count != 0;
> +}
> +
>  static const VMStateDescription vmstate_rtc = {
>      .name = "mc146818rtc",
> -    .version_id = 3,
> +    .version_id = 4,

You already know about this by now. :)

Paolo

>      .minimum_version_id = 1,
> +    .pre_load = rtc_pre_load,
>      .post_load = rtc_post_load,
>      .fields = (VMStateField[]) {
>          VMSTATE_BUFFER(cmos_data, RTCState),
> @@ -753,6 +777,14 @@ static const VMStateDescription vmstate_rtc = {
>          VMSTATE_TIMER_V(update_timer, RTCState, 3),
>          VMSTATE_UINT64_V(next_alarm_time, RTCState, 3),
>          VMSTATE_END_OF_LIST()
> +    },
> +    .subsections = (VMStateSubsection[]) {
> +        {
> +            .vmsd = &vmstate_rtc_irq_reinject_on_ack_count,
> +            .needed = rtc_irq_reinject_on_ack_count_needed,
> +        }, {
> +            /* empty */
> +        }
>      }
>  };
>  
> 

Re: [Qemu-devel] [RFC PATCH v3 15/49] softmmu: fixing usage of cpu_st/ld* from helpers

[Alex Bennée]

Pavel Dovgalyuk writes:

> MMU helper functions are called from generated code and other helper
> functions. In both cases they try to get function's return address for
> using it while restoring virtual CPU state.
>
> When MMU helper is called from some other helper function
> (like helper_maskmov_xmm) through cpu_st* function, the return address
> will point to that helper. That is why CPU state cannot be restored in
> the case of MMU fault.
>
> This patch introduces several inline helpers to load return address
> which points to the right place.
>
<snip>

OK I find it fairly hard to follow all the glue magic (not your fault
;-) we have in QEMU. However wouldn't it be simpler for the helper
pre-amble code to ensure the subject pc is updated in the CPU
environment?

Can QEMU only rectify the processor state from a TranlationBlock tc address?

-- 
Alex Bennée

Re: [Qemu-devel] [RFC PATCH v3 16/49] target-i386: update fp status fix

[Alex Bennée]

Pavel Dovgalyuk writes:

> This patch adds calls to update_fp_status() function from several
> places where FPU state is changed.
>
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  target-i386/cpu.c        |    1 +
>  target-i386/cpu.h        |    1 +
>  target-i386/fpu_helper.c |    5 ++++-
>  target-i386/machine.c    |    2 +-
>  4 files changed, 7 insertions(+), 2 deletions(-)

Hmm I wonder if this function should be renamed. It seems to be more
about ensuring the softfloat library is correctly configured according
to the emulated CPU configuration.

>
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index 6d008ab..906d7d7 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2573,6 +2573,7 @@ static void x86_cpu_reset(CPUState *s)
>          env->fptags[i] = 1;
>      }
>      env->fpuc = 0x37f;
> +    update_fp_status(env);
>  
>      env->mxcsr = 0x1f80;
>      env->xstate_bv = XSTATE_FP | XSTATE_SSE;
> diff --git a/target-i386/cpu.h b/target-i386/cpu.h
> index e634d83..42bda46 100644
> --- a/target-i386/cpu.h
> +++ b/target-i386/cpu.h
> @@ -1249,6 +1249,7 @@ void QEMU_NORETURN raise_interrupt(CPUX86State *nenv, int intno, int is_int,
>  /* cc_helper.c */
>  extern const uint8_t parity_table[256];
>  uint32_t cpu_cc_compute_all(CPUX86State *env1, int op);
> +void update_fp_status(CPUX86State *env);
>  
>  static inline uint32_t cpu_compute_eflags(CPUX86State *env)
>  {
> diff --git a/target-i386/fpu_helper.c b/target-i386/fpu_helper.c
> index 1b2900d..a8ffba9 100644
> --- a/target-i386/fpu_helper.c
> +++ b/target-i386/fpu_helper.c
> @@ -537,7 +537,7 @@ uint32_t helper_fnstcw(CPUX86State *env)
>      return env->fpuc;
>  }
>  
> -static void update_fp_status(CPUX86State *env)
> +void update_fp_status(CPUX86State *env)
>  {
>      int rnd_type;
>  
> @@ -1006,6 +1006,7 @@ void helper_fstenv(CPUX86State *env, target_ulong ptr, int data32)
>          cpu_stw_data(env, ptr + 10, 0);
>          cpu_stw_data(env, ptr + 12, 0);
>      }
> +    update_fp_status(env);
>  }
>  
>  void helper_fldenv(CPUX86State *env, target_ulong ptr, int data32)
> @@ -1055,6 +1056,7 @@ void helper_fsave(CPUX86State *env, target_ulong ptr, int data32)
>      env->fptags[5] = 1;
>      env->fptags[6] = 1;
>      env->fptags[7] = 1;
> +    update_fp_status(env);
>  }
>  
>  void helper_frstor(CPUX86State *env, target_ulong ptr, int data32)
> @@ -1158,6 +1160,7 @@ void helper_fxrstor(CPUX86State *env, target_ulong ptr, int data64)
>      }
>  
>      env->fpuc = cpu_lduw_data(env, ptr);
> +    update_fp_status(env);
>      fpus = cpu_lduw_data(env, ptr + 2);
>      fptag = cpu_lduw_data(env, ptr + 4);
>      env->fpstt = (fpus >> 11) & 7;
> diff --git a/target-i386/machine.c b/target-i386/machine.c
> index 16d2f6a..2922c97 100644
> --- a/target-i386/machine.c
> +++ b/target-i386/machine.c
> @@ -315,13 +315,13 @@ static int cpu_post_load(void *opaque, int version_id)
>      env->hflags &= ~HF_CPL_MASK;
>      env->hflags |= (env->segs[R_SS].flags >> DESC_DPL_SHIFT) & HF_CPL_MASK;
>  
> -    /* XXX: restore FPU round state */
>      env->fpstt = (env->fpus_vmstate >> 11) & 7;
>      env->fpus = env->fpus_vmstate & ~0x3800;
>      env->fptag_vmstate ^= 0xff;
>      for(i = 0; i < 8; i++) {
>          env->fptags[i] = (env->fptag_vmstate >> i) & 1;
>      }
> +    update_fp_status(env);
>  
>      cpu_breakpoint_remove_all(cs, BP_CPU);
>      cpu_watchpoint_remove_all(cs, BP_CPU);

-- 
Alex Bennée

Re: [Qemu-devel] [RFC PATCH v3 22/49] cpu: invent instruction count for accurate replay

[Alex Bennée]

Pavel Dovgalyuk writes:

> This patch adds instructions count fields to cpu structure and
> invents several functions for increasing this counter while executing
> translation blocks.
>
<snip>
> @@ -1186,6 +1188,7 @@ void qemu_init_vcpu(CPUState *cpu)
>      cpu->nr_cores = smp_cores;
>      cpu->nr_threads = smp_threads;
>      cpu->stopped = true;
> +    cpu->instructions_count = 0;
<snip>
> +    if (replay_file) {
> +        if (replay_mode == REPLAY_MODE_RECORD) {
> +            if (process_events && replay_has_events()) {
> +                replay_save_instructions();
> +                /* events will be after the last instruction */
> +                replay_save_events(-1);
> +            } else {
> +                /* instruction - increase the step counter */
> +                ++first_cpu->instructions_count;
> +            }
> +        } else if (replay_mode == REPLAY_MODE_PLAY) {
> +            skip_async_events_until(EVENT_INSTRUCTION);
> +            if (first_cpu->instructions_count >= 1) {
> +                ++replay_state.current_step;
> +                --first_cpu->instructions_count;
> +                if (first_cpu->instructions_count == 0) {
> +                    replay_has_unread_data = 0;
> +                }
> +            } else {
> +                replay_read_events(-1);
> +            }
> +        }
> +    }
> +}
> +
> +void replay_undo_last_instruction(void)
> +{
> +    if (replay_mode == REPLAY_MODE_RECORD) {
> +        first_cpu->instructions_count--;
> +    } else {
> +        replay_state.skipping_instruction = 1;
> +    }

I'm not quite following what instructions_count means here. Is it an
actual count of instructions executed? Can you add a comment to the
header where it's defined?

-- 
Alex Bennée

Re: [Qemu-devel] [RFC PATCH v3 04/49] fdc: adding vmstate for save/restore

[Dr. David Alan Gilbert]

* Pavel Dovgalyuk (Pavel.Dovgaluk@ispras.ru) wrote:
> VMState added by this patch preserves correct
> loading of the FDC device state.

This is a pretty big series, but I suspect most people don't need
most of these devices, especially for replay/reverse execution stuff.
Why don't you boil this down to a much smaller basic set that's enough
to get replay/reverse execution stuff on a simple setup; and then once
people have got it going add more of the devices in.

(Even if reading about floppy perpendicular mode is kind of fun for a 
Friday afternoon).

Dave


> 
> Signed-off-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
> ---
>  hw/block/fdc.c |   85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 files changed, 83 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/block/fdc.c b/hw/block/fdc.c
> index 490d127..7305f75 100644
> --- a/hw/block/fdc.c
> +++ b/hw/block/fdc.c
> @@ -695,10 +695,34 @@ static const VMStateDescription vmstate_fdrive_media_rate = {
>      }
>  };
>  
> +static bool fdrive_perpendicular_needed(void *opaque)
> +{
> +    FDrive *drive = opaque;
> +
> +    return drive->perpendicular != 0;
> +}
> +
> +static const VMStateDescription vmstate_fdrive_perpendicular = {
> +    .name = "fdrive/perpendicular",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT8(perpendicular, FDrive),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
> +static int fdrive_post_load(void *opaque, int version_id)
> +{
> +    fd_revalidate(opaque);
> +    return 0;
> +}
> +
>  static const VMStateDescription vmstate_fdrive = {
>      .name = "fdrive",
> -    .version_id = 1,
> +    .version_id = 2,
>      .minimum_version_id = 1,
> +    .post_load = fdrive_post_load,
>      .fields = (VMStateField[]) {
>          VMSTATE_UINT8(head, FDrive),
>          VMSTATE_UINT8(track, FDrive),
> @@ -713,6 +737,9 @@ static const VMStateDescription vmstate_fdrive = {
>              .vmsd = &vmstate_fdrive_media_rate,
>              .needed = &fdrive_media_rate_needed,
>          } , {
> +            .vmsd = &vmstate_fdrive_perpendicular,
> +            .needed = &fdrive_perpendicular_needed,
> +        } , {
>              /* empty */
>          }
>      }
> @@ -725,6 +752,14 @@ static void fdc_pre_save(void *opaque)
>      s->dor_vmstate = s->dor | GET_CUR_DRV(s);
>  }
>  
> +static int fdc_pre_load(void *opaque)
> +{
> +    FDCtrl *s = opaque;
> +    s->reset_sensei = 0;
> +    timer_del(s->result_timer);
> +    return 0;
> +}
> +
>  static int fdc_post_load(void *opaque, int version_id)
>  {
>      FDCtrl *s = opaque;
> @@ -734,11 +769,46 @@ static int fdc_post_load(void *opaque, int version_id)
>      return 0;
>  }
>  
> +static bool fdc_reset_sensei_needed(void *opaque)
> +{
> +    FDCtrl *s = opaque;
> +
> +    return s->reset_sensei != 0;
> +}
> +
> +static const VMStateDescription vmstate_fdc_reset_sensei = {
> +    .name = "fdc/reset_sensei",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_INT32(reset_sensei, FDCtrl),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
> +static bool fdc_result_timer_needed(void *opaque)
> +{
> +    FDCtrl *s = opaque;
> +
> +    return timer_pending(s->result_timer);
> +}
> +
> +static const VMStateDescription vmstate_fdc_result_timer = {
> +    .name = "fdc/result_timer",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_TIMER(result_timer, FDCtrl),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
>  static const VMStateDescription vmstate_fdc = {
>      .name = "fdc",
> -    .version_id = 2,
> +    .version_id = 3,
>      .minimum_version_id = 2,
>      .pre_save = fdc_pre_save,
> +    .pre_load = fdc_pre_load,
>      .post_load = fdc_post_load,
>      .fields = (VMStateField[]) {
>          /* Controller State */
> @@ -770,6 +840,17 @@ static const VMStateDescription vmstate_fdc = {
>          VMSTATE_STRUCT_ARRAY(drives, FDCtrl, MAX_FD, 1,
>                               vmstate_fdrive, FDrive),
>          VMSTATE_END_OF_LIST()
> +    },
> +    .subsections = (VMStateSubsection[]) {
> +        {
> +            .vmsd = &vmstate_fdc_reset_sensei,
> +            .needed = fdc_reset_sensei_needed,
> +        } , {
> +            .vmsd = &vmstate_fdc_result_timer,
> +            .needed = fdc_result_timer_needed,
> +        } , {
> +            /* empty */
> +        }
>      }
>  };
>  
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [RFC PATCH v3 15/49] softmmu: fixing usage of cpu_st/ld* from helpers

[Pavel Dovgaluk]

> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> Pavel Dovgalyuk writes:
> 
> > MMU helper functions are called from generated code and other helper
> > functions. In both cases they try to get function's return address for
> > using it while restoring virtual CPU state.
> >
> > When MMU helper is called from some other helper function
> > (like helper_maskmov_xmm) through cpu_st* function, the return address
> > will point to that helper. That is why CPU state cannot be restored in
> > the case of MMU fault.
> >
> > This patch introduces several inline helpers to load return address
> > which points to the right place.
> >
> <snip>
> 
> OK I find it fairly hard to follow all the glue magic (not your fault
> ;-) we have in QEMU. However wouldn't it be simpler for the helper
> pre-amble code to ensure the subject pc is updated in the CPU
> environment?

Then I'll need to rewrite all helper calls or change their structure
by adding code which restores the PC.

> Can QEMU only rectify the processor state from a TranlationBlock tc address?

Current guest PC is not known during execution of the TB. When memory access
exception occurs, helpers have to evaluate guest PC using the host one.
Host PC should point to the translated block and this patch eliminates reading
wrong host PC value during such recovery.

Pavel Dovgalyuk