Re: [LSF/MM/BPF TOPIC] Allowing linkat() to replace the destination

["Darrick J. Wong" <darrick.wong@oracle.com>]

On Sat, Jan 25, 2020 at 08:25:46AM +1100, Dave Chinner wrote:
> On Thu, Jan 23, 2020 at 09:47:30AM +0200, Amir Goldstein wrote:
> > On Thu, Jan 23, 2020 at 9:16 AM Dave Chinner <david@fromorbit.com> wrote:
> > >
> > > On Thu, Jan 23, 2020 at 03:47:45AM +0000, Al Viro wrote:
> > > > On Wed, Jan 22, 2020 at 02:10:03PM -0800, Omar Sandoval wrote:
> > > >
> > > > > > Sorry for not reading all the thread again, some API questions:
> > > > > > - We intend to allow AT_REPLACE only with O_TMPFILE src. Right?
> > > > >
> > > > > I wasn't planning on having that restriction. It's not too much effort
> > > > > for filesystems to support it for normal files, so I wouldn't want to
> > > > > place an artificial restriction on a useful primitive.
> > > >
> > 
> > I have too many gray hairs each one for implementing a "useful primitive"
> > that nobody asked for and bare the consequences.
> > Your introduction to AT_REPLACE uses O_TMPFILE.
> > I see no other sane use of the interface.
> > 
> > > > I'm not sure; that's how we ended up with the unspeakable APIs like
> > > > rename(2), after all...
> > >
> > > Yet it is just rename(2) with the serial numbers filed off -
> > > complete with all the same data vs metadata ordering problems that
> > > rename(2) comes along with. i.e. it needs fsync to guarantee data
> > > integrity of the source file before the linkat() call is made.
> > >
> > > If we can forsee that users are going to complain that
> > > linkat(AT_REPLACE) using O_TMPFILE files is not atomic because it
> > > leaves zero length files behind after a crash just like rename()
> > > does, then we haven't really improved anything at all...
> > >
> > > And, really, I don't think anyone wants another API that requires
> > > multiple fsync calls to use correctly for crash-safe file
> > > replacement, let alone try to teach people who still cant rename a
> > > file safely how to use it....
> > >
> > 
> > Are you suggesting that AT_LINK_REPLACE should have some of
> > the semantics I posted in this RFC  for AT_ATOMIC_xxx:
> > https://lore.kernel.org/linux-fsdevel/20190527172655.9287-1-amir73il@gmail.com/
> 
> Not directly.
> 
> All I'm pointing out is that data integrity guarantees of
> AT_LINK_REPLACE are yet another aspect of this new feature that
> has not yet been specified or documented at all.
> 
> And in pointing this out, I'm making an observation that the
> rename(2) behaviour which everyone seems to be assuming this
> function will replicate is a terrible model to copy/reinvent.
> 
> Addressing this problem is something for the people pushing for
> AT_LINK_REPLACE to solve, not me....

Or the grumpy maintainer who will have to digest all of this.

Can we update the documentation to admit that many people will probably
want to use this (and rename) as atomic swap operations?

"The filesystem will commit the data and metadata of all files and
directories involved in the link operation to stable storage before the
call returns."

And finally add a flag:

"AT_LINK_EATMYDATA: If specified, the filesystem may opt out of
committing anything to disk."

(Or a prctl(PR_EATMYDATA) to make it process wide, idk.)

--D

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com