Using fio with adjusted parameters for use case "DB"

Using fio with adjusted parameters for use case "DB"

[Thomas Schneider]

Hello,

I want to benchmark different storage devices with an I/O workload 
generated by the "real" application: DB.

A typcial command could be:
fio --name=random-writers --ioengine=libaio --iodepth=4 --rw=randwrite 
--bs=1024 --direct=1 --size=64m --numjobs=2 runtime=120s

The question is regarding parameters size and runtime:
- What value should be used for size? Should this be the same as 
allocated disk space by DB data files? Or just a (reasonable) portion of 
this? DB data files could be up to xTB.
- Is it advisable to use runtime? If yes, would this somehow contradict 
to size, means if I use a larger size, then runtime is obviously longer.


Regards
Thomas

Re: Using fio with adjusted parameters for use case "DB"

[Vincent Fu]

On 3/7/23 09:30, Thomas Schneider wrote:
> Hello,
> 
> I want to benchmark different storage devices with an I/O workload 
> generated by the "real" application: DB.
> 
> A typcial command could be:
> fio --name=random-writers --ioengine=libaio --iodepth=4 --rw=randwrite 
> --bs=1024 --direct=1 --size=64m --numjobs=2 runtime=120s
> 
> The question is regarding parameters size and runtime:
> - What value should be used for size? Should this be the same as 
> allocated disk space by DB data files? Or just a (reasonable) portion of 
> this? DB data files could be up to xTB.
> - Is it advisable to use runtime? If yes, would this somehow contradict 
> to size, means if I use a larger size, then runtime is obviously longer.
> 

The documentation for runtime is here:

https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-runtime

Fio will write within the LBAs bounded by offset and size and stop when 
it has touched all the LBAs or when runtime is attained. Run some 
experiments with --debug=io to better understand fio's behavior.

Also take a look at the example job files for help:

https://github.com/axboe/fio/tree/master/examples

Vincent

Re: Using fio with adjusted parameters for use case "DB"

[Thomas Schneider]

Hello,

thanks for your reply.

I have discussed with DB developers which values should be used for
- iodepth and
- numjobs

With regards to iodepth I was told that a max. value could be defined in 
DB's global ini; the default is 512.
This means the DB could use this queue depth as max. but certainly no as 
average.

Question:
 From your experience, does it make sense to run IO benchmark with 
iodepth=512?


THX



Am 07.03.2023 um 16:22 schrieb Vincent Fu:
> On 3/7/23 09:30, Thomas Schneider wrote:
>> Hello,
>>
>> I want to benchmark different storage devices with an I/O workload 
>> generated by the "real" application: DB.
>>
>> A typcial command could be:
>> fio --name=random-writers --ioengine=libaio --iodepth=4 
>> --rw=randwrite --bs=1024 --direct=1 --size=64m --numjobs=2 runtime=120s
>>
>> The question is regarding parameters size and runtime:
>> - What value should be used for size? Should this be the same as 
>> allocated disk space by DB data files? Or just a (reasonable) portion 
>> of this? DB data files could be up to xTB.
>> - Is it advisable to use runtime? If yes, would this somehow 
>> contradict to size, means if I use a larger size, then runtime is 
>> obviously longer.
>>
>
> The documentation for runtime is here:
>
> https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-runtime
>
> Fio will write within the LBAs bounded by offset and size and stop 
> when it has touched all the LBAs or when runtime is attained. Run some 
> experiments with --debug=io to better understand fio's behavior.
>
> Also take a look at the example job files for help:
>
> https://github.com/axboe/fio/tree/master/examples
>
> Vincent
>

Using fio with adjusted parameters for use case "DB"

[74cmonty]

Hello,

I want to benchmark different storage devices with an I/O workload 
generated by the "real" application: DB.

A typcial command could be:
fio --name=random-writers --ioengine=libaio --iodepth=4 --rw=randwrite 
--bs=1024 --direct=1 --size=64m --numjobs=2 runtime=120s

The question is regarding parameters size and runtime:
- What value should be used for size? Should this be the same as 
allocated disk space by DB data files? Or just a (reasonable) portion of 
this? DB data files could be up to n TB.
- Is it advisable to use runtime? If yes, would this somehow contradict 
to size, means if I use a larger size, then runtime is obviously longer.


Regards
Thomas