All of lore.kernel.org
 help / color / mirror / Atom feed
* Odd (wrong?) behavior with :<nr> suffix on readwrite parameter
@ 2022-02-07 23:14 ` Nick Neumann
  2022-02-10 22:52   ` Vincent Fu
  0 siblings, 1 reply; 4+ messages in thread
From: Nick Neumann @ 2022-02-07 23:14 UTC (permalink / raw)
  To: fio

I was playing with this feature today. It seems like a nice feature as
it will allow you to simulate a read or write of a somewhat fragmented
file. From the docs, I can do something like
--rw=randread:8
and expect fio will do 8 I/Os "before getting a new offset". So when
fio does its reads, they will occur at offset1, offset1+block_size,
offset1+block_size*2, ... offset1+block_size*7, and then at a new
random offset2, followed by offset2+block_size, offset2+block_size*2,
etc...

What I'm seeing with this feature though is that using it causes the
first offset chosen to always be 0, and it is used (plus multiples of
block size) for the first 7 (nr-1) I/Os. For example, with this
command:

sudo fio --ioengine=libaio --direct=1 --output-format=json+ --name=job
--filename=/dev/nvme1n1 --size=128MB --rw=randread:8 --iodepth=1
--bs=4K --write_bw_log=fio_0 --log_offset=1

The bandwidth log entries are (fifth column is the offset):

14, 288, 0, 4096, 0, 0
14, 36141, 0, 4096, 4096, 0
14, 34715, 0, 4096, 8192, 0
14, 34406, 0, 4096, 12288, 0
14, 34700, 0, 4096, 16384, 0
14, 27766, 0, 4096, 20480, 0
15, 34633, 0, 4096, 24576, 0
15, 20233, 0, 4096, 8093696, 0
15, 39409, 0, 4096, 8097792, 0

This seems odd, and kinda wrong - the first nr-1 ops do not happen
from a random offset as specified by the randread/randwrite value, but
instead from offset 0. When one does a random I/O operation without
the :<nr> suffix, the first operation is from a random offset, not 0.

I'm happy to find and fix the behavior, but wanted to first make sure
it isn't intentional or that I'm somehow misunderstanding it.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* RE: Odd (wrong?) behavior with :<nr> suffix on readwrite parameter
  2022-02-07 23:14 ` Odd (wrong?) behavior with :<nr> suffix on readwrite parameter Nick Neumann
@ 2022-02-10 22:52   ` Vincent Fu
  2022-02-10 23:08     ` Nick Neumann
  0 siblings, 1 reply; 4+ messages in thread
From: Vincent Fu @ 2022-02-10 22:52 UTC (permalink / raw)
  To: Nick Neumann, fio

> -----Original Message-----
> From: Nick Neumann [mailto:nick@pcpartpicker.com]
> Sent: Monday, February 7, 2022 6:15 PM
> To: fio@vger.kernel.org
> Subject: Odd (wrong?) behavior with :<nr> suffix on readwrite parameter
> 
> I was playing with this feature today. It seems like a nice feature as it will
> allow you to simulate a read or write of a somewhat fragmented file. From
> the docs, I can do something like
> --rw=randread:8
> and expect fio will do 8 I/Os "before getting a new offset". So when fio does
> its reads, they will occur at offset1, offset1+block_size,
> offset1+block_size*2, ... offset1+block_size*7, and then at a new
> random offset2, followed by offset2+block_size, offset2+block_size*2, etc...
> 
> What I'm seeing with this feature though is that using it causes the first offset
> chosen to always be 0, and it is used (plus multiples of block size) for the first
> 7 (nr-1) I/Os. For example, with this
> command:
> 
> sudo fio --ioengine=libaio --direct=1 --output-format=json+ --name=job
> --filename=/dev/nvme1n1 --size=128MB --rw=randread:8 --iodepth=1 --
> bs=4K --write_bw_log=fio_0 --log_offset=1
> 
> The bandwidth log entries are (fifth column is the offset):
> 
> 14, 288, 0, 4096, 0, 0
> 14, 36141, 0, 4096, 4096, 0
> 14, 34715, 0, 4096, 8192, 0
> 14, 34406, 0, 4096, 12288, 0
> 14, 34700, 0, 4096, 16384, 0
> 14, 27766, 0, 4096, 20480, 0
> 15, 34633, 0, 4096, 24576, 0
> 15, 20233, 0, 4096, 8093696, 0
> 15, 39409, 0, 4096, 8097792, 0
> 
> This seems odd, and kinda wrong - the first nr-1 ops do not happen from a
> random offset as specified by the randread/randwrite value, but instead
> from offset 0. When one does a random I/O operation without the :<nr>
> suffix, the first operation is from a random offset, not 0.
> 
> I'm happy to find and fix the behavior, but wanted to first make sure it isn't
> intentional or that I'm somehow misunderstanding it.

I tried running with randrepeat=0 and saw the same behavior you did. With the default randrepeat=1 fio will always use the same random number seed for IO offsets, but even with randrepeat=0, rw=randread:8 always started at offset 0.

I agree with your assessment that rw=randread:8 should start at a random offset.

Vincent

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Odd (wrong?) behavior with :<nr> suffix on readwrite parameter
  2022-02-10 22:52   ` Vincent Fu
@ 2022-02-10 23:08     ` Nick Neumann
  2022-02-15 18:04       ` Nick Neumann
  0 siblings, 1 reply; 4+ messages in thread
From: Nick Neumann @ 2022-02-10 23:08 UTC (permalink / raw)
  To: Vincent Fu; +Cc: fio

On Thu, Feb 10, 2022 at 4:53 PM Vincent Fu <vincent.fu@samsung.com> wrote:
> I tried running with randrepeat=0 and saw the same behavior you did. With the default randrepeat=1 fio will always use the same random number seed for IO offsets, but even with randrepeat=0, rw=randread:8 always started at offset 0.
>
> I agree with your assessment that rw=randread:8 should start at a random offset.
>
> Vincent

Thanks for the feedback on it. I'm slowly getting more comfortable
with the fio source base; hoping to be familiar enough to find and
quickly fix it soon.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Odd (wrong?) behavior with :<nr> suffix on readwrite parameter
  2022-02-10 23:08     ` Nick Neumann
@ 2022-02-15 18:04       ` Nick Neumann
  0 siblings, 0 replies; 4+ messages in thread
From: Nick Neumann @ 2022-02-15 18:04 UTC (permalink / raw)
  To: Vincent Fu; +Cc: fio

On Thu, Feb 10, 2022 at 5:08 PM Nick Neumann <nick@pcpartpicker.com> wrote:
> Thanks for the feedback on it. I'm slowly getting more comfortable
> with the fio source base; hoping to be familiar enough to find and
> quickly fix it soon.

This one was actually a really easy fix. I've got a PR in to fix it now:
https://github.com/axboe/fio/pull/1342

Thanks,
Nick

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-02-15 18:04 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <CGME20220208010644uscas1p132e264f9a90c2e515846440393203738@uscas1p1.samsung.com>
2022-02-07 23:14 ` Odd (wrong?) behavior with :<nr> suffix on readwrite parameter Nick Neumann
2022-02-10 22:52   ` Vincent Fu
2022-02-10 23:08     ` Nick Neumann
2022-02-15 18:04       ` Nick Neumann

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.