fadvise_hint behavior unclear from documentation

fadvise_hint behavior unclear from documentation

[Isawan Millican]

Hi,

I was using fio and noticed that the throughput was lower than expected for
random reads to a small file; I would expect it to be entirely cached in
memory. The issue seems to be because fadvise_hint=0 is not behaving as
documented as POSIX_FADV_DONTNEED is being called.
This might be the intended bahavior as the program is flushing the pages
to purely test IO throughput

If the current behaviour is intentional, is there an existing flag that 
could
provide the desired functionality? Essentially I'd like to test the IO
throughput of various file sizes without bypassing the page cache to see the
effects of caching.

To reproduce:

Version:
     Built from source from master (commit f3057d268d)
     Output of /fio --version
         fio-3.28-18-gf3057

We can see fadvise is being used when ran with strace:

     isawan@hypermachine:~/Software/fio$ strace -f  -e fadvise64 ./fio  
--name=test-reader --ioengine=mmap --readwrite=randread --readonly 
--filename=/tmp/test --bs=4k --size=128M --numjobs=1 --runtime=30s 
--time_based --buffered=1 --allow_file_create=0 --fadvise_hint=0
     test-reader: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 
4096B-4096B, (T) 4096B-4096B, ioengine=mmap, iodepth=1
     fio-3.28-18-gf3057
     strace: Process 28297 attached
     strace: Process 28298 attached
     strace: Process 28299 attached
     strace: Process 28300 attached
     strace: Process 28301 attached
     strace: Process 28302 attached
     strace: Process 28303 attached
     strace: Process 28304 attached
     [pid 28301] +++ exited with 0 +++
     [pid 28297] +++ exited with 0 +++
     [pid 28303] +++ exited with 0 +++
     [pid 28298] +++ exited with 0 +++
     [pid 28299] +++ exited with 0 +++
     [pid 28304] +++ exited with 0 +++
     [pid 28302] +++ exited with 0 +++
     [pid 28300] +++ exited with 0 +++
     strace: Process 28305 attached
     Starting 1 process
     strace: Process 28306 attached
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0
     [pid 28306] fadvise64(6, 0, 134217728, POSIX_FADV_DONTNEED) = 0

As we see here, fadvise is being called, while the documentation states:

     fadvise_hint=str

         Use posix_fadvise(2) or posix_fadvise(2) to advise the kernel 
on what I/O patterns are likely to be issued.
         Accepted values are:
             "0": Backwards-compatible hint for “no hint”

Re: fadvise_hint behavior unclear from documentation

[Jens Axboe]

On 9/18/21 4:46 PM, Isawan Millican wrote:
> Hi,
> 
> I was using fio and noticed that the throughput was lower than expected for
> random reads to a small file; I would expect it to be entirely cached in
> memory. The issue seems to be because fadvise_hint=0 is not behaving as
> documented as POSIX_FADV_DONTNEED is being called.
> This might be the intended bahavior as the program is flushing the pages
> to purely test IO throughput
> 
> If the current behaviour is intentional, is there an existing flag that 
> could
> provide the desired functionality? Essentially I'd like to test the IO
> throughput of various file sizes without bypassing the page cache to see the
> effects of caching.

Just set invalidate=0 and fio won't invalidate the address space page
cache for the files or devices used.

You seem to have snipped most of the cmdhelp for fadvise_hint:

        fadvise_hint: Use fadvise() to advise the kernel on IO pattern
                type: string (opt=bla)
             default: 1
        valid values: 0          Don't issue fadvise/madvise
                    : 1          Advise using fio IO pattern
                    : random     Advise using FADV_RANDOM
                    : sequential Advise using FADV_SEQUENTIAL

which tells you that it's about telling the kernel if you're doing
random or sequential IO (this will impact read-ahead, for example).
Invalidation is kept separate.

-- 
Jens Axboe