inflated bandwidth numbers with buffered I/O

* inflated bandwidth numbers with buffered I/O
@ 2016-01-14 23:28 Dallas Clement
  2016-01-15  1:50 ` Matthew Eaton
  2016-01-15 15:45 ` Jens Axboe
  0 siblings, 2 replies; 8+ messages in thread
From: Dallas Clement @ 2016-01-14 23:28 UTC (permalink / raw)
  To: fio

Hi, I hope it's ok to ask usage questions on this mailing list.

I have been using fio version 2.2.4 on a Fedora 22 server host to test
throughput on a 10 GigE LIO iSCSI block device.  Up until this point I
have been testing with Direct I/O (direct=1) and writing directly to
the block device with filename=/dev/blah (no filesystem).  I am seeing
bandwidth measurements that look reasonable.  I next tried to repeat
this test with Buffered I/O (direct=0) and the bandwidth numbers are
way too large.  Here is a comparison of the sequential write results
for various block sizes:

bs=4k => direct=95.945 MB/s, buffered=1475.4 MB/s
bs=512 => direct=495.328 MB/s, buffered=2637.333 MB/s
bs=2048 => direct=502.093 MB/s, buffered=2663.772 MB/s

As you can see the buffered bandwidth measurements are much larger and
don't make sense given that not more than 1200 MB/s can be transmitted
over a 10 Gbps network connection.

Now it's very likely I am doing something stupid and lack
understanding about how fio works.  Someone please enlighten me!

Here is the script I have been using to collect my results for buffered I/O

#!/bin/bash

JOB_FILE=job.fio

cat << EOF > ${JOB_FILE}
[job]
ioengine=libaio
iodepth=\${DEPTH}
prio=0
rw=\${RW}
bs=\${BS}
filename=/dev/sdc
numjobs=1
size=10g
direct=0
invalidate=0
ramp_time=15
runtime=120
time_based
write_bw_log=\${RW}-bs-\${BS}-depth-\${DEPTH}
write_lat_log=\${RW}-bs-\${BS}-depth-\${DEPTH}
write_iops_log=\${RW}-bs-\${BS}-depth-\${DEPTH}
EOF

for RW in read write randread randwrite
do
  for BS in 4k 512k 2048k
  do
    for DEPTH in 4 32 256
    do
      RW=${RW} BS=${BS} DEPTH=${DEPTH} fio ${JOB_FILE}
    done
  done
done

I have also tried running this script without the filename=blah
setting on an XFS formatted block device.  I am still seeing inflated
numbers for this also.

Thanks,

Dallas

^ permalink raw reply	[flat|nested] 8+ messages in thread