fio replay_redirect fails when using log generated via write_iolog

fio replay_redirect fails when using log generated via write_iolog

[Jeremy Richards]

Hello fio group,

I'm an intern at Seagate. I'm trying to replay an fio log taken on
/dev/sdb on a different device, /dev/sdc. However, replay_redirect
doesn't seem to make a difference when using a file created with
write_iolog.

Here is the output of cat testfio.log:

fio version 2 iolog
/dev/sdb add
/dev/sdb open
/dev/sdb read 2861015040 524288
/dev/sdb read 2861015040 524288
/dev/sdb read 2861015040 524288
/dev/sdb read 2861015040 524288
/dev/sdb close

Here is the command I used to try and play it on /dev/sdc:

fio --name=replaytest --filename=/dev/sdc --replay_redirect=/dev/sdc
--read_iolog=testfio.log --write_iolog=runt2_fio.log

This did not work, and just ran the replay on /dev/sdb.

I tried two systems (the newer one runs a modified 3.10 kernel and
CentOS 6.8) and fio versions 2.0.5, 2.2.4, 2.2.13, 2.13, and 2.14,
without success.

It still does not redirect if you give fio a jobfile such as
jobfile.fio, where the output of cat jobfile.fio:

[global]
name=replay
filename=/dev/sdc
replay_redirect=/dev/sdc
direct=1
thread=1
read_iolog=testfio.log
[/dev/sdc]

I can get replay_redirect to work, but only when using a binary file
generated by blktrace. I started a blktrace:

blktrace -d /dev/sdb -a read

and ran the testfio.log on /dev/sdb a few times, then ran

blkparse -i sdb -o /dev/null -d sdb.blktrace.bin

Using that as the input for read_iolog as so:

fio --name=replaytest --filename=/dev/sdc --replay_redirect=/dev/sdc
--read_iolog=sdb.blktrace.bin

will successfully redirect to sdc.

Also, the --write_iolog flag will not generate a file when running a
replay. This makes sense, since you'd just end up with a copy of the
original iolog file if it's fio, but I was trying to make sure that
the binary blktrace file was the same as the fio iolog and could not
verify. This second problem probably isn't important but I'm
mentioning it for the sake of completeness.

Thank you in advance for any insight / help you can provide.
Regards,
Jeremy Richards
Intern | Seagate Technology
720-684-1858 (Desk Phone)

Re: fio replay_redirect fails when using log generated via write_iolog

[Jens Axboe]

On 10/14/2016 04:34 PM, Jeremy Richards wrote:
> Hello fio group,
>
> I'm an intern at Seagate. I'm trying to replay an fio log taken on
> /dev/sdb on a different device, /dev/sdc. However, replay_redirect
> doesn't seem to make a difference when using a file created with
> write_iolog.
>
> Here is the output of cat testfio.log:
>
> fio version 2 iolog
> /dev/sdb add
> /dev/sdb open
> /dev/sdb read 2861015040 524288
> /dev/sdb read 2861015040 524288
> /dev/sdb read 2861015040 524288
> /dev/sdb read 2861015040 524288
> /dev/sdb close
>
> Here is the command I used to try and play it on /dev/sdc:
>
> fio --name=replaytest --filename=/dev/sdc --replay_redirect=/dev/sdc
> --read_iolog=testfio.log --write_iolog=runt2_fio.log
>
> This did not work, and just ran the replay on /dev/sdb.
>
> I tried two systems (the newer one runs a modified 3.10 kernel and
> CentOS 6.8) and fio versions 2.0.5, 2.2.4, 2.2.13, 2.13, and 2.14,
> without success.
>
> It still does not redirect if you give fio a jobfile such as
> jobfile.fio, where the output of cat jobfile.fio:
>
> [global]
> name=replay
> filename=/dev/sdc
> replay_redirect=/dev/sdc
> direct=1
> thread=1
> read_iolog=testfio.log
> [/dev/sdc]
>
> I can get replay_redirect to work, but only when using a binary file
> generated by blktrace. I started a blktrace:
>
> blktrace -d /dev/sdb -a read
>
> and ran the testfio.log on /dev/sdb a few times, then ran
>
> blkparse -i sdb -o /dev/null -d sdb.blktrace.bin
>
> Using that as the input for read_iolog as so:
>
> fio --name=replaytest --filename=/dev/sdc --replay_redirect=/dev/sdc
> --read_iolog=sdb.blktrace.bin
>
> will successfully redirect to sdc.
>
> Also, the --write_iolog flag will not generate a file when running a
> replay. This makes sense, since you'd just end up with a copy of the
> original iolog file if it's fio, but I was trying to make sure that
> the binary blktrace file was the same as the fio iolog and could not
> verify. This second problem probably isn't important but I'm
> mentioning it for the sake of completeness.

It isn't supported on non-blktrace replay. But it's trivial to add, just
did:

http://git.kernel.dk/cgit/fio/commit/?id=dac34079f7d8cfb0a64e4d3f086728edd7eded6d

If you update to current -git, let me know how it works.

-- 
Jens Axboe