Re: [PATCH 0/5] Enable per-file/directory DAX operations

From: Dave Chinner <david@fromorbit.com>
To: Boaz Harrosh <boaz@plexistor.com>
Cc: ira.weiny@intel.com, linux-kernel@vger.kernel.org,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	"Darrick J. Wong" <darrick.wong@oracle.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Christoph Hellwig <hch@lst.de>,
	"Theodore Y. Ts'o" <tytso@mit.edu>, Jan Kara <jack@suse.cz>,
	linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org,
	linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH 0/5] Enable per-file/directory DAX operations
Date: Thu, 24 Oct 2019 18:34:46 +1100	[thread overview]
Message-ID: <20191024073446.GA4614@dread.disaster.area> (raw)
In-Reply-To: <efffc9e7-8948-a117-dc7f-e394e50606ab@plexistor.com>

On Thu, Oct 24, 2019 at 05:31:13AM +0300, Boaz Harrosh wrote:
> On 24/10/2019 01:13, Dave Chinner wrote:
> > On Wed, Oct 23, 2019 at 04:09:50PM +0300, Boaz Harrosh wrote:
> >> On 22/10/2019 14:21, Boaz Harrosh wrote:
> >>> On 20/10/2019 18:59, ira.weiny@intel.com wrote:
> >> Please explain the use case behind your model?
> > 
> > No application changes needed to control whether they use DAX or
> > not. It allows the admin to control the application behaviour
> > completely, so they can turn off DAX if necessary. Applications are
> > unaware of constraints that may prevent DAX from being used, and so
> > admins need a mechanism to prevent DAX aware application from
> > actually using DAX if the capability is present.
> > 
> > e.g. given how slow some PMEM devices are when it comes to writing
> > data, especially under extremely high concurrency, DAX is not
> > necessarily a performance win for every application. Admins need a
> > guaranteed method of turning off DAX in these situations - apps may
> > not provide such a knob, or even be aware of a thing called DAX...
> > 
> 
> Thank you Dave for explaining. Forgive my slowness. I now understand
> your intention.
> 
> But if so please address my first concern. That in the submitted implementation
> you must set the flag-bit after the create of the file but before the write.
> So exactly the above slow writes must always be DAX if I ever want the file
> to be DAX accessed in the future.

The on disk DAX flag is inherited from the parent directory at
create time. Hence an admin only need to set it on the data
directory of the application when first configuring it, and
everything the app creates will be configured for DAX access
automatically.

Or, alternatively, mkfs sets the flag on the root dir so that
everything in the filesystem uses DAX by default (through
inheritance) unless the admin turns off the flag on a directory
before it starts to be used or on a set of files after they have
been created (because DAX causes problems)...

So, yeah, there's another problem with the basic assertion that we
only need to allow the on disk flag to be changed on zero length
files: we actually want to be able to -clear- the DAX flag when the
file has data attached to it, not just when it is an empty file...

> What if, say in XFS when setting the DAX-bit we take all the three write-locks
> same as a truncate. Then we check that there are no active page-cache mappings
> ie. a single opener. Then allow to set the bit. Else return EBUISY. (file is in use)

DAX doesn't have page cache mappings, so anything that relies on
checking page cache state isn't going to work reliably. I also seem
to recall that there was a need to take some vm level lock to really
prevent page fault races, and that we can't safely take that in a
safe combination with all the filesystem locks we need.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com