From: Kevin Wolf <kwolf@redhat.com>
To: Pavel Dovgalyuk <dovgaluk@ispras.ru>
Cc: edgar.iglesias@xilinx.com, peter.maydell@linaro.org,
igor.rubinov@gmail.com, mark.burton@greensocs.com,
real@ispras.ru, hines@cert.org, qemu-devel@nongnu.org,
maria.klimushenkova@ispras.ru, stefanha@redhat.com,
pbonzini@redhat.com, batuzovk@ispras.ru, alex.bennee@linaro.org,
fred.konrad@greensocs.com
Subject: Re: [Qemu-devel] [PATCH 3/3] replay: introduce block devices record/replay
Date: Mon, 22 Feb 2016 12:06:44 +0100 [thread overview]
Message-ID: <20160222110644.GD5387@noname.str.redhat.com> (raw)
In-Reply-To: <000601d16bad$e93f9eb0$bbbedc10$@ru>
Am 20.02.2016 um 08:11 hat Pavel Dovgalyuk geschrieben:
> > From: Pavel Dovgalyuk [mailto:dovgaluk@ispras.ru]
> > > From: Kevin Wolf [mailto:kwolf@redhat.com]
> > > Am 16.02.2016 um 12:20 hat Pavel Dovgalyuk geschrieben:
> > > > Coroutine Replay
> > > > bool *done = req_replayed_list_get(reqid) // NULL
> > > > co =
> > > req_completed_list_get(e.reqid); // NULL
> > >
> > > There was no yield, this context switch is impossible to happen. Same
> > > for the switch back.
> > >
> > > > req_completed_list_insert(reqid, qemu_coroutine_self());
> > > > qemu_coroutine_yield();
> > >
> > > This is the point at which a context switch happens. The only other
> > > point in my code is the qemu_coroutine_enter() in the other function.
> >
> > I've fixed aio_poll problem by disabling mutex lock for the replay_run_block_event()
> > execution. Now virtual machine deterministically runs 4e8 instructions of Windows XP booting.
Are you sure that the lock was unnecessary? Solving deadlocks by
removing the lock is a rather adventurous method.
I assume that you're still replaying events from low-level functions
like qemu_clock_get_ns(). So even if you get rid of the hangs, the
result is probably not quite right.
I'm afraid this is going in a direction where my comments can't be more
constructive than a simple "you're doing it wrong".
> > But then one non-deterministic event happens.
> > Callback after finishing coroutine may be called from different contexts.
How does this happen? I'm not aware of callbacks being processed by any
thread other than the I/O thread for that specific block device (unless
you use dataplane, this is the main loop thread).
> > apic_update_irq() function behaves differently being called from vcpu and io threads.
> > In one case it sets CPU_INTERRUPT_POLL and in other - nothing happens.
>
> Kevin, do you have some ideas how to fix this issue?
> This happens because of coroutines may be assigned to different threads.
> Maybe there is some way of making this assignment more deterministic?
Coroutines aren't randomly assigned to threads, but threads actively
enter coroutines. To my knowledge this happens only when starting a
request (either vcpu or I/O thread; consistent per device) or by a
callback when some event happens (only I/O thread). I can't see any
non-determinism here.
Kevin
> > Therefore execution becomes non-deterministic.
> > In previous version of the patch I solved this problem by linking block events to the
> > execution checkpoints. IO thread have its own checkpoints and vcpu - its own.
> > Therefore apic callbacks are always called from the same thread in replay as in recording
> > phase.
>
> Pavel Dovgalyuk
>
next prev parent reply other threads:[~2016-02-22 11:06 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-02-09 5:55 [Qemu-devel] [PATCH 0/3] Deterministic replay extensions Pavel Dovgalyuk
2016-02-09 5:55 ` [Qemu-devel] [PATCH 1/3] replay: character devices Pavel Dovgalyuk
2016-02-09 5:55 ` [Qemu-devel] [PATCH 2/3] replay: introduce new checkpoint for icount warp Pavel Dovgalyuk
2016-02-09 5:55 ` [Qemu-devel] [PATCH 3/3] replay: introduce block devices record/replay Pavel Dovgalyuk
2016-02-09 10:27 ` Kevin Wolf
2016-02-09 11:52 ` Pavel Dovgalyuk
2016-02-10 11:45 ` Kevin Wolf
2016-02-10 12:05 ` Pavel Dovgalyuk
2016-02-10 12:28 ` Kevin Wolf
2016-02-10 12:51 ` Pavel Dovgalyuk
2016-02-10 13:25 ` Kevin Wolf
2016-02-10 13:33 ` Pavel Dovgalyuk
2016-02-10 13:52 ` Kevin Wolf
2016-02-11 6:05 ` Pavel Dovgalyuk
2016-02-11 9:43 ` Kevin Wolf
2016-02-11 11:00 ` Pavel Dovgalyuk
2016-02-11 12:18 ` Kevin Wolf
2016-02-11 12:24 ` Pavel Dovgalyuk
2016-02-12 8:33 ` Pavel Dovgalyuk
2016-02-12 9:44 ` Kevin Wolf
2016-02-12 13:19 ` Pavel Dovgalyuk
2016-02-12 13:58 ` Kevin Wolf
2016-02-15 8:38 ` Pavel Dovgalyuk
2016-02-15 9:10 ` Kevin Wolf
2016-02-15 9:14 ` Pavel Dovgalyuk
2016-02-15 9:38 ` Kevin Wolf
2016-02-15 11:19 ` Pavel Dovgalyuk
2016-02-15 12:46 ` Kevin Wolf
2016-02-15 13:54 ` Pavel Dovgalyuk
2016-02-15 14:06 ` Kevin Wolf
2016-02-15 14:24 ` Pavel Dovgalyuk
2016-02-15 15:01 ` Kevin Wolf
2016-02-16 6:25 ` Pavel Dovgalyuk
2016-02-16 10:02 ` Kevin Wolf
2016-02-16 11:20 ` Pavel Dovgalyuk
2016-02-16 12:54 ` Kevin Wolf
2016-02-18 9:18 ` Pavel Dovgalyuk
2016-02-20 7:11 ` Pavel Dovgalyuk
2016-02-22 11:06 ` Kevin Wolf [this message]
2016-02-24 11:59 ` Pavel Dovgalyuk
2016-02-24 13:14 ` Kevin Wolf
2016-02-25 9:06 ` Pavel Dovgalyuk
2016-02-26 9:01 ` Kevin Wolf
2016-02-29 7:03 ` Pavel Dovgalyuk
2016-02-29 7:54 ` Kevin Wolf
2016-02-15 14:50 ` Pavel Dovgalyuk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160222110644.GD5387@noname.str.redhat.com \
--to=kwolf@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=batuzovk@ispras.ru \
--cc=dovgaluk@ispras.ru \
--cc=edgar.iglesias@xilinx.com \
--cc=fred.konrad@greensocs.com \
--cc=hines@cert.org \
--cc=igor.rubinov@gmail.com \
--cc=maria.klimushenkova@ispras.ru \
--cc=mark.burton@greensocs.com \
--cc=pbonzini@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=real@ispras.ru \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).