Re: [PATCH v10 20/21] ext4: Add DAX functionality

[Dave Chinner <david@fromorbit.com>]

On Wed, Sep 10, 2014 at 07:49:40PM +0300, Boaz Harrosh wrote:
> On 09/03/2014 02:13 PM, Dave Chinner wrote:
> <>
> > 
> > When direct IO fails ext4 falls back to buffered IO, right? And
> > dax_do_io() can return partial writes, yes?
> > 
> 
> There is no buffered writes with DAX. .I.E buffered writes are always
> direct as well. (No page cache)

Yes, I know. But you didn't actually read the code I pointed out,
did you?

> > So that means if you get, say, ENOSPC part way through a DAX write,
> > ext4 can start dirtying the page cache from
> > __generic_file_write_iter() because the DAX write didn't wholly
> > complete? And say this ENOSPC races with space being freed from
> > another inode, then the buffered write will succeed and we'll end up
> > with coherency issues, right?
> > 
> > This is not an idle question - XFS if firing asserts all over the
> > place when doing ENOSPC testing because DAX is returning partial
> > writes and the XFS direct IO code is expecting them to either wholly
> > complete or wholly fail. I can make the DAX variant do allow partial
> > writes, but I'm not going to add a useless fallback to buffered IO
> > for XFS when the (fully featured) direct allocation fails.
> > 
> 
> Right, no fall back.

And so ext4 is buggy, because what ext4 does ....

> Because a fallback is just a retry, because in any
> way DAX assumes there is never a page_cache_page for a written data

... is not a retry - it falls back to a fundamentally different
code path. i.e:

sys_write()
....
	new_sync_write
	  ext4_file_write_iter
	    __generic_file_write_iter(O_DIRECT)
	      written = generic_file_direct_write()
	      if (error || complete write)
	        return
	      /* short write! do buffered IO to finish! */
	      generic_perform_write()
	        loop {
			ext4_write_begin
			ext4_write_end
		}

and so we allocate pages in the page cache and do buffered IO into
them because DAX doesn't hook ->writebegin/write_end as we are
supposed to intercept all buffered IO at a higher level.

This causes data corruption when tested at ENOSPC on DAX enabled
ext4 filesystems. I think that it's an oversight and hence a bug
that needs to be fixed but I'm first asking Willy to see if it was
intentional or not because maybe I missed sometihng in the past 4
months since I've paid really close attention to the DAX code.

And in saying that, Boaz, I'd suggest you spend some time looking at
the history of the DAX patchset. Pay careful note to who came up
with the original idea and architecture that led to the IO path you
are so stridently defending.....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com