FLUSH/FUA documentation & code discrepancy

[Philipp Reisner <philipp.reisner@linbit.com>]

Hi,

I think commit 1e87901e18 was wrong. Starting with that commit the REQ_FLUSH 
and REQ_FUA bits get stripped away if the queue does not advertise REQ_FLUSH
or REQ_FUA support.

But the REQ_FLUSH bit is also tested for when not merging requests
(blk_queue_bio()) or when it comes to the elevator (blk_flush_plug_list()).

So, since this patch the elevator reorders write requests on queues that 
do not have REQ_FLUSH or REQ_FUA set.

While on queues that have REQ_FLUSH or REQ_FUA set, the elevator does
not reorder writes across FLUSHes.

The Documentation/block/writeback_cache_control.txt file says:
--snip--
Implementation details for filesystems
--------------------------------------

Filesystems can simply set the REQ_FLUSH and REQ_FUA bits and do not have to
worry if the underlying devices need any explicit cache flushing and how
the Forced Unit Access is implemented.  The REQ_FLUSH and REQ_FUA flags
may both be set on a single bio.
--snap--

I have the impression every file system lets IO drain, and issues a
flush afterwards with the blkdev_issue_flush() function. BTW that
function turns into a non-obvious no-op as soon as the queue does not
have the REQ_FUA or REQ_FLUSH bits set. It does not look like it is
a no-op by intention.

The file systems seem to be all fine, only in DRBD we have a mode were
we depend on REQ_FUA/REQ_FLUSH requests being real boundaries for reordering
of writes. This is broken since the mentioned commit as we recently found out.

I suggest that either this commit gets reverted, or the documentation
is updated. I am ready to prepare such a patch, but I need directions
how it should be fixed.

Best regards,
 Phil