Re: [PATCH RFC] vfs: add a O_NOMTIME flag

From: Dave Chinner <david@fromorbit.com>
To: Andy Lutomirski <luto@amacapital.net>
Cc: Richard Weinberger <richard.weinberger@gmail.com>,
	Zach Brown <zab@redhat.com>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Sage Weil <sweil@redhat.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	"open list:ABI/API" <linux-api@vger.kernel.org>
Subject: Re: [PATCH RFC] vfs: add a O_NOMTIME flag
Date: Fri, 8 May 2015 12:42:59 +1000	[thread overview]
Message-ID: <20150508024259.GL4327@dastard> (raw)
In-Reply-To: <CALCETrWNDMq0nK3ac-uZweV5BKK_yWTQHH5D0YkyEu7bcONo9g@mail.gmail.com>

On Thu, May 07, 2015 at 12:53:46PM -0700, Andy Lutomirski wrote:
> On Thu, May 7, 2015 at 12:09 PM, Richard Weinberger
> <richard.weinberger@gmail.com> wrote:
> > On Thu, May 7, 2015 at 7:20 PM, Zach Brown <zab@redhat.com> wrote:
> >> On Thu, May 07, 2015 at 10:26:17AM +1000, Dave Chinner wrote:
> >>> On Wed, May 06, 2015 at 03:00:12PM -0700, Zach Brown wrote:
> >>> > Add the O_NOMTIME flag which prevents mtime from being updated which can
> >>> > greatly reduce the IO overhead of writes to allocated and initialized
> >>> > regions of files.
> >>>
> >>> Hmmm. How do backup programs now work out if the file has changed
> >>> and hence needs copying again? ie. applications using this will
> >>> break other critical infrastructure in subtle ways.
> >>
> >> By using backup infrastructure that doesn't use cmtime.  Like btrfs
> >> send/recv.  Or application level backups that know how to do
> >> incrementals from metadata in giant database files, say, without
> >> walking, comparing, and copying the entire thing.
> >
> > But how can Joey random user know that some of his
> > applications are using O_NOMTIME and his KISS backup
> > program does no longer function as expected?
> >
> 
> Joey random user can't have a working KISS backup anyway, though,
> because we screw up mtime updates on mmap writes.  I have patches
> gathering dust that fix that, though.

They are close enough to be good for backup purposes. The mtime only
need change once per backup period - it doesn't need to be
millisecond accurate. Yes, I know you needed that changed for
different reasons (avoid variable page fault latency), but it
doesn't matter for once-a-day or even once-an-hour incremental
backup scans.

Besides, anyone who cares about accurate backups is doing a backup
from a snapshot so they data and metadata is consistent across the
entire backup. And that makes worries about mmap and mtime
completely irrelevant because a snapshot freezes the filesystem and
hence cleans all the mapped pages. Once the snapshot is taken
the next mmap write will trigger a page fault and so change the
mtime and it will be picked up in the next backup scan...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com