Re: [RFC] relaxed barrier semantics

[Chris Mason <chris.mason@oracle.com>]

On Wed, Jul 28, 2010 at 11:28:59AM +0200, Christoph Hellwig wrote:
> On Wed, Jul 28, 2010 at 11:17:06AM +0200, Tejun Heo wrote:
> > Well, if disabling barrier works around the problem for them (which is
> > basically what was suggeseted in the first message), that's not too
> > bad for short term, I think.
> 
> It's a pretty horrible workaround.  Requiring manual mount options to
> get performance out of a setup which could trivially work out of the
> box is a bad workaround.
> 
> > I'll re-read barrier code and see how hard it would be to implement a
> > proper solution.
> 
> If we move all filesystems to non-draining barriers with pre- and post-
> flushes that might actually be a relatively easy first step.  We don't
> have the complications to deal with multiple types of barriers to
> start with, and it'll fix the issue for devices without volatile write
> caches completely.
> 
> I just need some help from the filesystem folks to determine if they
> are safe with them.
> 
> I know for sure that ext3 and xfs are from looking through them.  And
> I know reiserfs is if we make sure it doesn't hit the code path that
> relies on it that is currently enabled by the barrier option.
> 
> I'll just need more feedback from ext4, gfs2, btrfs and nilfs folks.
> That already ends our small list of barrier supporting filesystems, and
> possibly ocfs2, too - although the barrier implementation there seems
> incomplete as it doesn't seem to flush caches in fsync.

Btrfs is going to be similar to xfs, except because of COW we have to
always pretend someone is extending the file (or filling a hole).

The short answer is that a preflush of the disk cache, followed by FUA
for commits is fine.  Btrfs explicitly waits for all the bios it sends
down without trusting other layers for silent ordering.

The long answer is the btrfs commit is basically:

wait for bio completion of a bunch of different things
write new super block pointing to new tree roots with barrier

Everything we waited for must be fully on disk before the new super
block, and the new super must be fully on disk after we wait for the bh.

I regret putting the ordering into the original barrier code...it
definitely did help reiserfs back in the day but it stinks of magic and
voodoo.

When it goes wrong, we'll only notice .000000001% of the time, and even
then it'll only be when people report some random corruption which we'll
blindly blame on either axboe or the drive.

-chris