[Qemu-devel] What is the best commit for record-replay?

[Qemu-devel] What is the best commit for record-replay?

[Igor R]

Hi,

I'm trying to use the deterministic record/replay feature, and I would
like to know which commit I should take to get it work.
In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
mentioned here:
http://lists.nongnu.org/archive/html/qemu-devel/2017-03/msg02657.html
with this patch:
http://lists.nongnu.org/archive/html/qemu-devel/2017-02/msg01316.html
My command line is:
./qemu-system-i386 -net none  -icount
shift=7,rr=replay,rrfile=replay.bin -drive
file=MyFedora386.qcow,if=none,id=img-direct -drive
driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
ide-hd,drive=img-blkreplay -monitor stdio

The replay advances for a while, but gets stuck in about 10-15 sec,
and it looks like it encounters deadlock trying to acquire rcu lock.

Is there a working commit of RR?


Thanks.

Re: [Qemu-devel] What is the best commit for record-replay?

[Pranith Kumar]

On Thu, Mar 23, 2017 at 4:05 AM, Igor R <boost.lists@gmail.com> wrote:
> Hi,
>
> I'm trying to use the deterministic record/replay feature, and I would
> like to know which commit I should take to get it work.
> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as

Can you retry with the latest rc? There were some fixes regarding rr since rc0.

Thanks,
--
Pranith

Re: [Qemu-devel] What is the best commit for record-replay?

[Igor R]

>> Hi,
>>
>> I'm trying to use the deterministic record/replay feature, and I would
>> like to know which commit I should take to get it work.
>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as

> Can you retry with the latest rc? There were some fixes regarding rr since rc0.


I've taken 2.9 release, and RR does not seem to work there.
I recorded the boot process of x86 Fedora-21 linux and the replay got
stuck almost immediately.

Re: [Qemu-devel] What is the best commit for record-replay?

[Alex Bennée]

Igor R <boost.lists@gmail.com> writes:

>>> Hi,
>>>
>>> I'm trying to use the deterministic record/replay feature, and I would
>>> like to know which commit I should take to get it work.
>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>
>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>
>
> I've taken 2.9 release, and RR does not seem to work there.
> I recorded the boot process of x86 Fedora-21 linux and the replay got
> stuck almost immediately.

What's your command line?

Does it get stuck at the same place each time?

Can you boot fine with icount but without record/replay?

--
Alex Bennée

Re: [Qemu-devel] What is the best commit for record-replay?

[Igor R]

>>>> I'm trying to use the deterministic record/replay feature, and I would
>>>> like to know which commit I should take to get it work.
>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>
>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>>
>>
>> I've taken 2.9 release, and RR does not seem to work there.
>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>> stuck almost immediately.
>
> What's your command line?
>
> Does it get stuck at the same place each time?
>
> Can you boot fine with icount but without record/replay?

Here is the exact scenario:
- Get 2.9 from git, configure it as follows: "./configure
--target-list=i386-softmmu --enable-sdl" and  make.
- Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
- Run qemu with the following command line, until login prompt:
-icount shift=7,rr=record,rrfile=replay.bin -drive
file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
ide-hd,drive=img-blkreplay -monitor stdio
- Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
ide-hd,drive=img-blkreplay -monitor stdio

Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
very early stage.


> Can you boot fine with icount but without record/replay?

Yes. I can also enable icount and recording - it also boots fine. The
problem with the replay.

Re: [Qemu-devel] What is the best commit for record-replay?

[Aleksandr Bezzubikov]

2017-05-02 15:42 GMT+03:00 Igor R <boost.lists@gmail.com>:
>>>>> I'm trying to use the deterministic record/replay feature, and I would
>>>>> like to know which commit I should take to get it work.
>>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>>
>>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>>>
>>>
>>> I've taken 2.9 release, and RR does not seem to work there.
>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>>> stuck almost immediately.
>>
>> What's your command line?
>>
>> Does it get stuck at the same place each time?
>>
>> Can you boot fine with icount but without record/replay?
>
> Here is the exact scenario:
> - Get 2.9 from git, configure it as follows: "./configure
> --target-list=i386-softmmu --enable-sdl" and  make.
> - Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
> - Run qemu with the following command line, until login prompt:
> -icount shift=7,rr=record,rrfile=replay.bin -drive
> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> ide-hd,drive=img-blkreplay -monitor stdio
> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> ide-hd,drive=img-blkreplay -monitor stdio
>
> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
> very early stage.
>
>
>> Can you boot fine with icount but without record/replay?
>
> Yes. I can also enable icount and recording - it also boots fine. The
> problem with the replay.

Hi guys,
Maybe the thread is a bit outdated, but the problem is still relevant.
I've just tried to record and replay WinXP boot process, and I've encountered
exactly the same problem as described above - record is fine, replay
gets stuck early. I use current master.
And I've discovered the second problem - recording makes initial snapshot,
but it doesn't seem to be saved to the disk - replay can't see it.

Hope you've already found the solution (as the last post was on 2 May)
and it's just got missed the mailing list.

>



-- 
Aleksandr Bezzubikov

Re: [Qemu-devel] What is the best commit for record-replay?

[Aleksandr Bezzubikov]

[+CC Pavel Dovgaluk, me]

2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov <zuban32s@gmail.com>:
> 2017-05-02 15:42 GMT+03:00 Igor R <boost.lists@gmail.com>:
>>>>>> I'm trying to use the deterministic record/replay feature, and I would
>>>>>> like to know which commit I should take to get it work.
>>>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>>>
>>>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>>>>
>>>>
>>>> I've taken 2.9 release, and RR does not seem to work there.
>>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>>>> stuck almost immediately.
>>>
>>> What's your command line?
>>>
>>> Does it get stuck at the same place each time?
>>>
>>> Can you boot fine with icount but without record/replay?
>>
>> Here is the exact scenario:
>> - Get 2.9 from git, configure it as follows: "./configure
>> --target-list=i386-softmmu --enable-sdl" and  make.
>> - Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>> - Run qemu with the following command line, until login prompt:
>> -icount shift=7,rr=record,rrfile=replay.bin -drive
>> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> ide-hd,drive=img-blkreplay -monitor stdio
>> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> ide-hd,drive=img-blkreplay -monitor stdio
>>
>> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>> very early stage.
>>
>>
>>> Can you boot fine with icount but without record/replay?
>>
>> Yes. I can also enable icount and recording - it also boots fine. The
>> problem with the replay.
>
> Hi guys,
> Maybe the thread is a bit outdated, but the problem is still relevant.
> I've just tried to record and replay WinXP boot process, and I've encountered
> exactly the same problem as described above - record is fine, replay
> gets stuck early. I use current master.
> And I've discovered the second problem - recording makes initial snapshot,
> but it doesn't seem to be saved to the disk - replay can't see it.
>
> Hope you've already found the solution (as the last post was on 2 May)
> and it's just got missed the mailing list.
>
>>
>
>
>
> --
> Aleksandr Bezzubikov



-- 
Aleksandr Bezzubikov

Re: [Qemu-devel] What is the best commit for record-replay?

[Pavel Dovgalyuk]

> From: Aleksandr Bezzubikov [mailto:zuban32s@gmail.com]
> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov <zuban32s@gmail.com>:
> > 2017-05-02 15:42 GMT+03:00 Igor R <boost.lists@gmail.com>:
> >>>>>> I'm trying to use the deterministic record/replay feature, and I would
> >>>>>> like to know which commit I should take to get it work.
> >>>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
> >>>>
> >>>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
> >>>>
> >>>>
> >>>> I've taken 2.9 release, and RR does not seem to work there.
> >>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
> >>>> stuck almost immediately.
> >>>
> >>> What's your command line?
> >>>
> >>> Does it get stuck at the same place each time?
> >>>
> >>> Can you boot fine with icount but without record/replay?
> >>
> >> Here is the exact scenario:
> >> - Get 2.9 from git, configure it as follows: "./configure
> >> --target-list=i386-softmmu --enable-sdl" and  make.
> >> - Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
> >> - Run qemu with the following command line, until login prompt:
> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> >> ide-hd,drive=img-blkreplay -monitor stdio
> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> >> ide-hd,drive=img-blkreplay -monitor stdio
> >>
> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
> >> very early stage.
> >>
> >>
> >>> Can you boot fine with icount but without record/replay?
> >>
> >> Yes. I can also enable icount and recording - it also boots fine. The
> >> problem with the replay.
> >
> > Hi guys,
> > Maybe the thread is a bit outdated, but the problem is still relevant.
> > I've just tried to record and replay WinXP boot process, and I've encountered
> > exactly the same problem as described above - record is fine, replay
> > gets stuck early. I use current master.

Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48

> > And I've discovered the second problem - recording makes initial snapshot,
> > but it doesn't seem to be saved to the disk - replay can't see it.

It is ok, because there is a mode where snapshot is created and loaded.

> >
> > Hope you've already found the solution (as the last post was on 2 May)
> > and it's just got missed the mailing list.

As I know, RR is still broken in the current version.
It was caused by the MTTCG implementation.
Alex Bennee tried to fix RR back. Alex, have you found any solution?

We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.

Pavel Dovgalyuk

Re: [Qemu-devel] What is the best commit for record-replay?

[Alex Bennée]

Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

>> From: Aleksandr Bezzubikov [mailto:zuban32s@gmail.com]
>> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov <zuban32s@gmail.com>:
>> > 2017-05-02 15:42 GMT+03:00 Igor R <boost.lists@gmail.com>:
>> >>>>>> I'm trying to use the deterministic record/replay feature, and I would
>> >>>>>> like to know which commit I should take to get it work.
>> >>>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>> >>>>
>> >>>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>> >>>>
>> >>>>
>> >>>> I've taken 2.9 release, and RR does not seem to work there.
>> >>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>> >>>> stuck almost immediately.
>> >>>
>> >>> What's your command line?
>> >>>
>> >>> Does it get stuck at the same place each time?
>> >>>
>> >>> Can you boot fine with icount but without record/replay?
>> >>
>> >> Here is the exact scenario:
>> >> - Get 2.9 from git, configure it as follows: "./configure
>> >> --target-list=i386-softmmu --enable-sdl" and  make.
>> >> - Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>> >> - Run qemu with the following command line, until login prompt:
>> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> >> ide-hd,drive=img-blkreplay -monitor stdio
>> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> >> ide-hd,drive=img-blkreplay -monitor stdio
>> >>
>> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>> >> very early stage.
>> >>
>> >>
>> >>> Can you boot fine with icount but without record/replay?
>> >>
>> >> Yes. I can also enable icount and recording - it also boots fine. The
>> >> problem with the replay.
>> >
>> > Hi guys,
>> > Maybe the thread is a bit outdated, but the problem is still relevant.
>> > I've just tried to record and replay WinXP boot process, and I've encountered
>> > exactly the same problem as described above - record is fine, replay
>> > gets stuck early. I use current master.
>
> Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48
>
>> > And I've discovered the second problem - recording makes initial snapshot,
>> > but it doesn't seem to be saved to the disk - replay can't see it.
>
> It is ok, because there is a mode where snapshot is created and loaded.
>
>> >
>> > Hope you've already found the solution (as the last post was on 2 May)
>> > and it's just got missed the mailing list.
>
> As I know, RR is still broken in the current version.
> It was caused by the MTTCG implementation.
> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>
> We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.

I think the method outlined in my RFC is the way to go, essentially the
RR mutex taking over for the what the BQL did. The RFC patch hadn't
hoisted the mutex for the additional devices so I'm just re-basing now
and I'll see if I can make the changes for Igor's test case.

--
Alex Bennée

Re: [Qemu-devel] What is the best commit for record-replay?

[Aleksandr Bezzubikov]

2017-09-19 12:30 GMT+03:00 Alex Bennée <alex.bennee@linaro.org>:
>
> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
>
>>> From: Aleksandr Bezzubikov [mailto:zuban32s@gmail.com]
>>> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov <zuban32s@gmail.com>:
>>> > 2017-05-02 15:42 GMT+03:00 Igor R <boost.lists@gmail.com>:
>>> >>>>>> I'm trying to use the deterministic record/replay feature, and I would
>>> >>>>>> like to know which commit I should take to get it work.
>>> >>>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>> >>>>
>>> >>>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>>> >>>>
>>> >>>>
>>> >>>> I've taken 2.9 release, and RR does not seem to work there.
>>> >>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>>> >>>> stuck almost immediately.
>>> >>>
>>> >>> What's your command line?
>>> >>>
>>> >>> Does it get stuck at the same place each time?
>>> >>>
>>> >>> Can you boot fine with icount but without record/replay?
>>> >>
>>> >> Here is the exact scenario:
>>> >> - Get 2.9 from git, configure it as follows: "./configure
>>> >> --target-list=i386-softmmu --enable-sdl" and  make.
>>> >> - Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>>> >> - Run qemu with the following command line, until login prompt:
>>> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>> >>
>>> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>>> >> very early stage.
>>> >>
>>> >>
>>> >>> Can you boot fine with icount but without record/replay?
>>> >>
>>> >> Yes. I can also enable icount and recording - it also boots fine. The
>>> >> problem with the replay.
>>> >
>>> > Hi guys,
>>> > Maybe the thread is a bit outdated, but the problem is still relevant.
>>> > I've just tried to record and replay WinXP boot process, and I've encountered
>>> > exactly the same problem as described above - record is fine, replay
>>> > gets stuck early. I use current master.
>>
>> Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48

Unfortunately this one doesn't work either. It seems we need just an
all-in-one fix
for the current implementation to make it work.

>>
>>> > And I've discovered the second problem - recording makes initial snapshot,
>>> > but it doesn't seem to be saved to the disk - replay can't see it.
>>
>> It is ok, because there is a mode where snapshot is created and loaded.

So it shouldn't work properly when I use 'rrsnapshot=<name>' for both
record and replay?
Then how can I enable this mode?

>>
>>> >
>>> > Hope you've already found the solution (as the last post was on 2 May)
>>> > and it's just got missed the mailing list.
>>
>> As I know, RR is still broken in the current version.
>> It was caused by the MTTCG implementation.
>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>>
>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.
>
> I think the method outlined in my RFC is the way to go, essentially the
> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> hoisted the mutex for the additional devices so I'm just re-basing now
> and I'll see if I can make the changes for Igor's test case.
>
> --
> Alex Bennée



-- 
Aleksandr Bezzubikov

Re: [Qemu-devel] What is the best commit for record-replay?

[Alex Bennée]

Aleksandr Bezzubikov <zuban32s@gmail.com> writes:

> 2017-09-19 12:30 GMT+03:00 Alex Bennée <alex.bennee@linaro.org>:
>>
>> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
>>
>>>> From: Aleksandr Bezzubikov [mailto:zuban32s@gmail.com]
>>>> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov <zuban32s@gmail.com>:
>>>> > 2017-05-02 15:42 GMT+03:00 Igor R <boost.lists@gmail.com>:
>>>> >>>>>> I'm trying to use the deterministic record/replay feature, and I would
>>>> >>>>>> like to know which commit I should take to get it work.
>>>> >>>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>>> >>>>
>>>> >>>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>>>> >>>>
>>>> >>>>
>>>> >>>> I've taken 2.9 release, and RR does not seem to work there.
>>>> >>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>>>> >>>> stuck almost immediately.
>>>> >>>
>>>> >>> What's your command line?
>>>> >>>
>>>> >>> Does it get stuck at the same place each time?
>>>> >>>
>>>> >>> Can you boot fine with icount but without record/replay?
>>>> >>
>>>> >> Here is the exact scenario:
>>>> >> - Get 2.9 from git, configure it as follows: "./configure
>>>> >> --target-list=i386-softmmu --enable-sdl" and  make.
>>>> >> - Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>>>> >> - Run qemu with the following command line, until login prompt:
>>>> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
>>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>>> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>>> >>
>>>> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>>>> >> very early stage.
>>>> >>
>>>> >>
>>>> >>> Can you boot fine with icount but without record/replay?
>>>> >>
>>>> >> Yes. I can also enable icount and recording - it also boots fine. The
>>>> >> problem with the replay.
>>>> >
>>>> > Hi guys,
>>>> > Maybe the thread is a bit outdated, but the problem is still relevant.
>>>> > I've just tried to record and replay WinXP boot process, and I've encountered
>>>> > exactly the same problem as described above - record is fine, replay
>>>> > gets stuck early. I use current master.
>>>
>>> Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48
>
> Unfortunately this one doesn't work either. It seems we need just an
> all-in-one fix
> for the current implementation to make it work.
>
>>>
>>>> > And I've discovered the second problem - recording makes initial snapshot,
>>>> > but it doesn't seem to be saved to the disk - replay can't see it.
>>>
>>> It is ok, because there is a mode where snapshot is created and loaded.
>
> So it shouldn't work properly when I use 'rrsnapshot=<name>' for both
> record and replay?
> Then how can I enable this mode?
>
>>>
>>>> >
>>>> > Hope you've already found the solution (as the last post was on 2 May)
>>>> > and it's just got missed the mailing list.
>>>
>>> As I know, RR is still broken in the current version.
>>> It was caused by the MTTCG implementation.
>>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>>>
>>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.
>>
>> I think the method outlined in my RFC is the way to go, essentially the
>> RR mutex taking over for the what the BQL did. The RFC patch hadn't
>> hoisted the mutex for the additional devices so I'm just re-basing now
>> and I'll see if I can make the changes for Igor's test case.
>>
>> --
>> Alex Bennée

Could you try:

  https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2

And report back?

--
Alex Bennée

Re: [Qemu-devel] What is the best commit for record-replay?

[Alexander Bezzubikov]

Alex Bennée писал 2017-09-19 17:25:
> Aleksandr Bezzubikov <zuban32s@gmail.com> writes:
> 
>> 2017-09-19 12:30 GMT+03:00 Alex Bennée <alex.bennee@linaro.org>:
>>> 
>>> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
>>> 
>>>>> From: Aleksandr Bezzubikov [mailto:zuban32s@gmail.com]
>>>>> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov 
>>>>> <zuban32s@gmail.com>:
>>>>> > 2017-05-02 15:42 GMT+03:00 Igor R <boost.lists@gmail.com>:
>>>>> >>>>>> I'm trying to use the deterministic record/replay feature, and I would
>>>>> >>>>>> like to know which commit I should take to get it work.
>>>>> >>>>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>>>> >>>>
>>>>> >>>>> Can you retry with the latest rc? There were some fixes regarding rr since rc0.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> I've taken 2.9 release, and RR does not seem to work there.
>>>>> >>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>>>>> >>>> stuck almost immediately.
>>>>> >>>
>>>>> >>> What's your command line?
>>>>> >>>
>>>>> >>> Does it get stuck at the same place each time?
>>>>> >>>
>>>>> >>> Can you boot fine with icount but without record/replay?
>>>>> >>
>>>>> >> Here is the exact scenario:
>>>>> >> - Get 2.9 from git, configure it as follows: "./configure
>>>>> >> --target-list=i386-softmmu --enable-sdl" and  make.
>>>>> >> - Download https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>>>>> >> - Run qemu with the following command line, until login prompt:
>>>>> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
>>>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>>>> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>>>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>>>> >>
>>>>> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>>>>> >> very early stage.
>>>>> >>
>>>>> >>
>>>>> >>> Can you boot fine with icount but without record/replay?
>>>>> >>
>>>>> >> Yes. I can also enable icount and recording - it also boots fine. The
>>>>> >> problem with the replay.
>>>>> >
>>>>> > Hi guys,
>>>>> > Maybe the thread is a bit outdated, but the problem is still relevant.
>>>>> > I've just tried to record and replay WinXP boot process, and I've encountered
>>>>> > exactly the same problem as described above - record is fine, replay
>>>>> > gets stuck early. I use current master.
>>>> 
>>>> Maybe this commit will work: 
>>>> cfb2d02be9413d45b30ed6d8e38800250b6b4b48
>> 
>> Unfortunately this one doesn't work either. It seems we need just an
>> all-in-one fix
>> for the current implementation to make it work.
>> 
>>>> 
>>>>> > And I've discovered the second problem - recording makes initial snapshot,
>>>>> > but it doesn't seem to be saved to the disk - replay can't see it.
>>>> 
>>>> It is ok, because there is a mode where snapshot is created and 
>>>> loaded.
>> 
>> So it shouldn't work properly when I use 'rrsnapshot=<name>' for both
>> record and replay?
>> Then how can I enable this mode?
>> 
>>>> 
>>>>> >
>>>>> > Hope you've already found the solution (as the last post was on 2 May)
>>>>> > and it's just got missed the mailing list.
>>>> 
>>>> As I know, RR is still broken in the current version.
>>>> It was caused by the MTTCG implementation.
>>>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>>>> 
>>>> We also trying to find a way to fix RR. It seems, that we will 
>>>> reinvent BQL for RR.
>>> 
>>> I think the method outlined in my RFC is the way to go, essentially 
>>> the
>>> RR mutex taking over for the what the BQL did. The RFC patch hadn't
>>> hoisted the mutex for the additional devices so I'm just re-basing 
>>> now
>>> and I'll see if I can make the changes for Igor's test case.
>>> 
>>> --
>>> Alex Bennée
> 
> Could you try:
> 
>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
> 
> And report back?

I've encountered 2 new things:
1) Significant performance regression during recording
2) Sometimes I can get through BIOS to the OS boot process itself
    Previously it got stuck in BIOS, the last block I had (I mean with -d 
in_asm) was

     IN:
     0x0000000000007ef6:  pushfl
     0x0000000000007ef8:  pushal
     0x0000000000007efa:  mov    $0xe,%ah
     0x0000000000007efc:  xor    %bx,%bx
     0x0000000000007efe:  int    $0x10

    So I couldn't even get to the boot process of the OS itself.
    Now it passes to the OS unpredictably, and still not very far. 
Anyway, it doesn't
    reach the point I stopped recording.

So we have a little progress here.


> 
> --
> Alex Bennée

-- 
Aleksandr Bezzubikov

Re: [Qemu-devel] What is the best commit for record-replay?

[Pavel Dovgalyuk]

> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> >
> >>>
> >>>> >
> >>>> > Hope you've already found the solution (as the last post was on 2 May)
> >>>> > and it's just got missed the mailing list.
> >>>
> >>> As I know, RR is still broken in the current version.
> >>> It was caused by the MTTCG implementation.
> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
> >>>
> >>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.
> >>
> >> I think the method outlined in my RFC is the way to go, essentially the
> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> >> hoisted the mutex for the additional devices so I'm just re-basing now
> >> and I'll see if I can make the changes for Igor's test case.
> >>
> >> --
> >> Alex Bennée
> 
> Could you try:
> 
>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
> 
> And report back?

Most of the code look reasonable.
Isn't better to lock before acting with icount in the following function?
 
static void prepare_icount_for_run(CPUState *cpu)
{
    if (use_icount) {
        int insns_left;

        /* These should always be cleared by process_icount_data after
         * each vCPU execution. However u16.high can be raised
         * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
         */
        g_assert(cpu->icount_decr.u16.low == 0);
        g_assert(cpu->icount_extra == 0);

        cpu->icount_budget = tcg_get_icount_limit();
        insns_left = MIN(0xffff, cpu->icount_budget);
        cpu->icount_decr.u16.low = insns_left;
        cpu->icount_extra = cpu->icount_budget - insns_left;

        if (replay_mode != REPLAY_MODE_NONE) {
            replay_mutex_lock();
        }
    }
}



Pavel Dovgalyuk

Re: [Qemu-devel] What is the best commit for record-replay?

[Pavel Dovgalyuk]

> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> Aleksandr Bezzubikov <zuban32s@gmail.com> writes:
> 
> > 2017-09-19 12:30 GMT+03:00 Alex Bennée <alex.bennee@linaro.org>:
> >>
> >>> As I know, RR is still broken in the current version.
> >>> It was caused by the MTTCG implementation.
> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
> >>>
> >>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.
> >>
> >> I think the method outlined in my RFC is the way to go, essentially the
> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> >> hoisted the mutex for the additional devices so I'm just re-basing now
> >> and I'll see if I can make the changes for Igor's test case.
> >>
> >> --
> >> Alex Bennée
> 
> Could you try:
> 
>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
> 
> And report back?

Please check the new series of the patches.
It includes patches from Alex and some fixes that made them working.

Pavel Dovgalyuk

Re: [Qemu-devel] What is the best commit for record-replay?

[Alex Bennée]

Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

>> From: Alex Bennée [mailto:alex.bennee@linaro.org]
>> Aleksandr Bezzubikov <zuban32s@gmail.com> writes:
>>
>> > 2017-09-19 12:30 GMT+03:00 Alex Bennée <alex.bennee@linaro.org>:
>> >>
>> >>> As I know, RR is still broken in the current version.
>> >>> It was caused by the MTTCG implementation.
>> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>> >>>
>> >>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.
>> >>
>> >> I think the method outlined in my RFC is the way to go, essentially the
>> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
>> >> hoisted the mutex for the additional devices so I'm just re-basing now
>> >> and I'll see if I can make the changes for Igor's test case.
>> >>
>> >> --
>> >> Alex Bennée
>>
>> Could you try:
>>
>>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
>>
>> And report back?
>
> Please check the new series of the patches.
> It includes patches from Alex and some fixes that made them working.

For reference that is:

  Subject: [RFC PATCH 00/26] replay additions
  From: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
  Date: Tue, 31 Oct 2017 14:24:57 +0300
  Message-ID: <20171031112457.10516.8971.stgit@pasha-VirtualBox>

--
Alex Bennée

Re: [Qemu-devel] What is the best commit for record-replay?

[Pavel Dovgalyuk]

> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
> 
> >> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> >> Aleksandr Bezzubikov <zuban32s@gmail.com> writes:
> >>
> >> > 2017-09-19 12:30 GMT+03:00 Alex Bennée <alex.bennee@linaro.org>:
> >> >>
> >> >>> As I know, RR is still broken in the current version.
> >> >>> It was caused by the MTTCG implementation.
> >> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
> >> >>>
> >> >>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL for RR.
> >> >>
> >> >> I think the method outlined in my RFC is the way to go, essentially the
> >> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> >> >> hoisted the mutex for the additional devices so I'm just re-basing now
> >> >> and I'll see if I can make the changes for Igor's test case.
> >> >>
> >> >> --
> >> >> Alex Bennée
> >>
> >> Could you try:
> >>
> >>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
> >>
> >> And report back?
> >
> > Please check the new series of the patches.
> > It includes patches from Alex and some fixes that made them working.
> 
> For reference that is:
> 
>   Subject: [RFC PATCH 00/26] replay additions
>   From: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>   Date: Tue, 31 Oct 2017 14:24:57 +0300
>   Message-ID: <20171031112457.10516.8971.stgit@pasha-VirtualBox>

That's right.

Pavel Dovgalyuk