Re: [fuse] Effects of opening with O_DIRECT

[Miklos Szeredi <miklos@szeredi.hu>]

On Sun, Mar 1, 2020 at 2:20 PM Nikolaus Rath <Nikolaus@rath.org> wrote:
>
> Hi,
>
> What happens if a file (on a FUSE mountpoint) is opened without
> O_DIRECT, has some data in the page cache, and is then opened a second
> with O_DIRECT?
>
> Will reads with O_DIRECT come from the page cache (if there's a hit), or
> be passed through to the fuse daemon?

O_DIRECT read will try first directly, and fall back to the cache on
short or zero return count.

>
> What happens to writes (with and without O_DIRECT, and assuming that
> writeback caching is active)? It seems to me that in order to keep
> consistent, either caching has to be disabled for both file descriptors
> or enabled for both...

This is not a fuse specific problem.   The kernel will try to keep
things consistent by flushing dirty data before an O_DIRECT read.
However this mode of operation is not recommended.  See open(2)
manpage:

       Applications should avoid mixing O_DIRECT and normal I/O  to  the  same
       file,  and  especially  to  overlapping  byte regions in the same file.
       Even when the filesystem correctly handles the coherency issues in this
       situation,  overall  I/O  throughput  is likely to be slower than using
       either mode alone.  Likewise, applications should avoid mixing  mmap(2)
       of files with direct I/O to the same files.

[...]
       In summary, O_DIRECT is a potentially powerful tool that should be used
       with  caution.   It  is  recommended  that  applications  treat  use of
       O_DIRECT as a performance option which is disabled by default.

              "The thing that has always disturbed me about O_DIRECT  is  that
              the whole interface is just stupid, and was probably designed by
              a  deranged  monkey  on  some  serious   mind-controlling   sub‐
              stances."—Linus

Thanks,
Miklos