All of lore.kernel.org
 help / color / mirror / Atom feed
* lazytime implementation questions
@ 2016-01-04  6:22 ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-04  6:22 UTC (permalink / raw)
  To: xfs; +Cc: ext4, linux-fsdevel, linux-api

Hi folks,

I've been looking at implementing the lazytime mount option for XFS,
and I'm struggling to work out what it is supposed to mean.

AFAICT, on ext4, lazytime means that pure timestamp updates are not
journalled and they are only ever written back when the inode is
otherwise dirtied and written, or they are timestamp dirty for 24
hours which triggers writeback.

This poses a couple of problems for XFS:

	1. we log every timestamp change, so there is no mechanism
	   for delayed/deferred update.

	2. we track dirty metadata in the journal, not via the VFS
	   dirty inode lists, so all the infrastructure written for
	   ext4 to do periodic flushing is useless to us.

These are solvable problems, but what I'm not sure about is exactly
what the intended semantics of lazytime durability are. That is,
exactly what guaranteed are we giving userspace about timestamp
updates when lazytime is used? The guarantees we have to give will
greatly influence the XFS implementation, so I really need to nail
down what we are expected to provide userspace. Can we:

	a) just ignore all durability concerns?
	b) if not, do we only need to care about the 24 hour
	   writeback and unmount?
	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
	   to provide durability of all metadata changes?
	d) do we have to care about ordering - if we fsync one inode
	   with 1 hour old timestamps, do we also need to guarantee
	   that all the inodes with older dirty timestamps also get
	   made durable?

I really want to completely ignore all ordering and forced
durability requirements for lazytime (i.e. implement only periodic,
optimistic and freeze/unmount writeback), but I haven't found any
documentation of what durability lazytime is supposed to provide and
can really only guess what was intended from the ext4
implementation....

Clarity would be appreciated.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* lazytime implementation questions
@ 2016-01-04  6:22 ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-04  6:22 UTC (permalink / raw)
  To: xfs; +Cc: linux-fsdevel, linux-api, ext4

Hi folks,

I've been looking at implementing the lazytime mount option for XFS,
and I'm struggling to work out what it is supposed to mean.

AFAICT, on ext4, lazytime means that pure timestamp updates are not
journalled and they are only ever written back when the inode is
otherwise dirtied and written, or they are timestamp dirty for 24
hours which triggers writeback.

This poses a couple of problems for XFS:

	1. we log every timestamp change, so there is no mechanism
	   for delayed/deferred update.

	2. we track dirty metadata in the journal, not via the VFS
	   dirty inode lists, so all the infrastructure written for
	   ext4 to do periodic flushing is useless to us.

These are solvable problems, but what I'm not sure about is exactly
what the intended semantics of lazytime durability are. That is,
exactly what guaranteed are we giving userspace about timestamp
updates when lazytime is used? The guarantees we have to give will
greatly influence the XFS implementation, so I really need to nail
down what we are expected to provide userspace. Can we:

	a) just ignore all durability concerns?
	b) if not, do we only need to care about the 24 hour
	   writeback and unmount?
	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
	   to provide durability of all metadata changes?
	d) do we have to care about ordering - if we fsync one inode
	   with 1 hour old timestamps, do we also need to guarantee
	   that all the inodes with older dirty timestamps also get
	   made durable?

I really want to completely ignore all ordering and forced
durability requirements for lazytime (i.e. implement only periodic,
optimistic and freeze/unmount writeback), but I haven't found any
documentation of what durability lazytime is supposed to provide and
can really only guess what was intended from the ext4
implementation....

Clarity would be appreciated.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 14+ messages in thread

* lazytime implementation questions
@ 2016-01-04  6:22 ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-04  6:22 UTC (permalink / raw)
  To: xfs-VZNHf3L845pBDgjK7y7TUQ
  Cc: ext4-u79uwXL29TY76Z2rM5mHXA,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
	linux-api-u79uwXL29TY76Z2rM5mHXA

Hi folks,

I've been looking at implementing the lazytime mount option for XFS,
and I'm struggling to work out what it is supposed to mean.

AFAICT, on ext4, lazytime means that pure timestamp updates are not
journalled and they are only ever written back when the inode is
otherwise dirtied and written, or they are timestamp dirty for 24
hours which triggers writeback.

This poses a couple of problems for XFS:

	1. we log every timestamp change, so there is no mechanism
	   for delayed/deferred update.

	2. we track dirty metadata in the journal, not via the VFS
	   dirty inode lists, so all the infrastructure written for
	   ext4 to do periodic flushing is useless to us.

These are solvable problems, but what I'm not sure about is exactly
what the intended semantics of lazytime durability are. That is,
exactly what guaranteed are we giving userspace about timestamp
updates when lazytime is used? The guarantees we have to give will
greatly influence the XFS implementation, so I really need to nail
down what we are expected to provide userspace. Can we:

	a) just ignore all durability concerns?
	b) if not, do we only need to care about the 24 hour
	   writeback and unmount?
	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
	   to provide durability of all metadata changes?
	d) do we have to care about ordering - if we fsync one inode
	   with 1 hour old timestamps, do we also need to guarantee
	   that all the inodes with older dirty timestamps also get
	   made durable?

I really want to completely ignore all ordering and forced
durability requirements for lazytime (i.e. implement only periodic,
optimistic and freeze/unmount writeback), but I haven't found any
documentation of what durability lazytime is supposed to provide and
can really only guess what was intended from the ext4
implementation....

Clarity would be appreciated.

Cheers,

Dave.
-- 
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
  2016-01-04  6:22 ` Dave Chinner
  (?)
@ 2016-01-05 17:36   ` Jan Kara
  -1 siblings, 0 replies; 14+ messages in thread
From: Jan Kara @ 2016-01-05 17:36 UTC (permalink / raw)
  To: Dave Chinner; +Cc: xfs, linux-ext4, linux-fsdevel, linux-api, tytso

  Hi,

On Mon 04-01-16 17:22:19, Dave Chinner wrote:
> I've been looking at implementing the lazytime mount option for XFS,
> and I'm struggling to work out what it is supposed to mean.
> 
> AFAICT, on ext4, lazytime means that pure timestamp updates are not
> journalled and they are only ever written back when the inode is
> otherwise dirtied and written, or they are timestamp dirty for 24
> hours which triggers writeback.
> 
> This poses a couple of problems for XFS:
> 
> 	1. we log every timestamp change, so there is no mechanism
> 	   for delayed/deferred update.
> 
> 	2. we track dirty metadata in the journal, not via the VFS
> 	   dirty inode lists, so all the infrastructure written for
> 	   ext4 to do periodic flushing is useless to us.
> 
> These are solvable problems, but what I'm not sure about is exactly
> what the intended semantics of lazytime durability are. That is,
> exactly what guaranteed are we giving userspace about timestamp
> updates when lazytime is used? The guarantees we have to give will
> greatly influence the XFS implementation, so I really need to nail
> down what we are expected to provide userspace. Can we:
> 
> 	a) just ignore all durability concerns?
> 	b) if not, do we only need to care about the 24 hour
> 	   writeback and unmount?
> 	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
> 	   to provide durability of all metadata changes?
> 	d) do we have to care about ordering - if we fsync one inode
> 	   with 1 hour old timestamps, do we also need to guarantee
> 	   that all the inodes with older dirty timestamps also get
> 	   made durable?

So the intended semantics is:
1) fsync / sync / freeze / unmount will write the timestamp updates even
   with lazytime. So unless crash happens, timestamps are guaranteed to be
   consistent. Also sync / fsync guarantees all changes to get to disk.
2) We periodically write back timestamps (once per 24 hours) to avoid too
   big timestamp inconsistencies in case of crash.

Otherwise there are no guarantees wrt durability. I've CCed Ted who
designed this just in case I missed something.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-05 17:36   ` Jan Kara
  0 siblings, 0 replies; 14+ messages in thread
From: Jan Kara @ 2016-01-05 17:36 UTC (permalink / raw)
  To: Dave Chinner
  Cc: xfs-VZNHf3L845pBDgjK7y7TUQ, linux-ext4-u79uwXL29TY76Z2rM5mHXA,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
	linux-api-u79uwXL29TY76Z2rM5mHXA, tytso-3s7WtUTddSA

  Hi,

On Mon 04-01-16 17:22:19, Dave Chinner wrote:
> I've been looking at implementing the lazytime mount option for XFS,
> and I'm struggling to work out what it is supposed to mean.
> 
> AFAICT, on ext4, lazytime means that pure timestamp updates are not
> journalled and they are only ever written back when the inode is
> otherwise dirtied and written, or they are timestamp dirty for 24
> hours which triggers writeback.
> 
> This poses a couple of problems for XFS:
> 
> 	1. we log every timestamp change, so there is no mechanism
> 	   for delayed/deferred update.
> 
> 	2. we track dirty metadata in the journal, not via the VFS
> 	   dirty inode lists, so all the infrastructure written for
> 	   ext4 to do periodic flushing is useless to us.
> 
> These are solvable problems, but what I'm not sure about is exactly
> what the intended semantics of lazytime durability are. That is,
> exactly what guaranteed are we giving userspace about timestamp
> updates when lazytime is used? The guarantees we have to give will
> greatly influence the XFS implementation, so I really need to nail
> down what we are expected to provide userspace. Can we:
> 
> 	a) just ignore all durability concerns?
> 	b) if not, do we only need to care about the 24 hour
> 	   writeback and unmount?
> 	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
> 	   to provide durability of all metadata changes?
> 	d) do we have to care about ordering - if we fsync one inode
> 	   with 1 hour old timestamps, do we also need to guarantee
> 	   that all the inodes with older dirty timestamps also get
> 	   made durable?

So the intended semantics is:
1) fsync / sync / freeze / unmount will write the timestamp updates even
   with lazytime. So unless crash happens, timestamps are guaranteed to be
   consistent. Also sync / fsync guarantees all changes to get to disk.
2) We periodically write back timestamps (once per 24 hours) to avoid too
   big timestamp inconsistencies in case of crash.

Otherwise there are no guarantees wrt durability. I've CCed Ted who
designed this just in case I missed something.

								Honza
-- 
Jan Kara <jack-IBi9RG/b67k@public.gmane.org>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-05 17:36   ` Jan Kara
  0 siblings, 0 replies; 14+ messages in thread
From: Jan Kara @ 2016-01-05 17:36 UTC (permalink / raw)
  To: Dave Chinner; +Cc: linux-fsdevel, linux-api, linux-ext4, tytso, xfs

  Hi,

On Mon 04-01-16 17:22:19, Dave Chinner wrote:
> I've been looking at implementing the lazytime mount option for XFS,
> and I'm struggling to work out what it is supposed to mean.
> 
> AFAICT, on ext4, lazytime means that pure timestamp updates are not
> journalled and they are only ever written back when the inode is
> otherwise dirtied and written, or they are timestamp dirty for 24
> hours which triggers writeback.
> 
> This poses a couple of problems for XFS:
> 
> 	1. we log every timestamp change, so there is no mechanism
> 	   for delayed/deferred update.
> 
> 	2. we track dirty metadata in the journal, not via the VFS
> 	   dirty inode lists, so all the infrastructure written for
> 	   ext4 to do periodic flushing is useless to us.
> 
> These are solvable problems, but what I'm not sure about is exactly
> what the intended semantics of lazytime durability are. That is,
> exactly what guaranteed are we giving userspace about timestamp
> updates when lazytime is used? The guarantees we have to give will
> greatly influence the XFS implementation, so I really need to nail
> down what we are expected to provide userspace. Can we:
> 
> 	a) just ignore all durability concerns?
> 	b) if not, do we only need to care about the 24 hour
> 	   writeback and unmount?
> 	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
> 	   to provide durability of all metadata changes?
> 	d) do we have to care about ordering - if we fsync one inode
> 	   with 1 hour old timestamps, do we also need to guarantee
> 	   that all the inodes with older dirty timestamps also get
> 	   made durable?

So the intended semantics is:
1) fsync / sync / freeze / unmount will write the timestamp updates even
   with lazytime. So unless crash happens, timestamps are guaranteed to be
   consistent. Also sync / fsync guarantees all changes to get to disk.
2) We periodically write back timestamps (once per 24 hours) to avoid too
   big timestamp inconsistencies in case of crash.

Otherwise there are no guarantees wrt durability. I've CCed Ted who
designed this just in case I missed something.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-05 22:59     ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-05 22:59 UTC (permalink / raw)
  To: Jan Kara; +Cc: xfs, linux-ext4, linux-fsdevel, linux-api, tytso

On Tue, Jan 05, 2016 at 06:36:04PM +0100, Jan Kara wrote:
>   Hi,
> 
> On Mon 04-01-16 17:22:19, Dave Chinner wrote:
> > I've been looking at implementing the lazytime mount option for XFS,
> > and I'm struggling to work out what it is supposed to mean.
> > 
> > AFAICT, on ext4, lazytime means that pure timestamp updates are not
> > journalled and they are only ever written back when the inode is
> > otherwise dirtied and written, or they are timestamp dirty for 24
> > hours which triggers writeback.
> > 
> > This poses a couple of problems for XFS:
> > 
> > 	1. we log every timestamp change, so there is no mechanism
> > 	   for delayed/deferred update.
> > 
> > 	2. we track dirty metadata in the journal, not via the VFS
> > 	   dirty inode lists, so all the infrastructure written for
> > 	   ext4 to do periodic flushing is useless to us.
> > 
> > These are solvable problems, but what I'm not sure about is exactly
> > what the intended semantics of lazytime durability are. That is,
> > exactly what guaranteed are we giving userspace about timestamp
> > updates when lazytime is used? The guarantees we have to give will
> > greatly influence the XFS implementation, so I really need to nail
> > down what we are expected to provide userspace. Can we:
> > 
> > 	a) just ignore all durability concerns?
> > 	b) if not, do we only need to care about the 24 hour
> > 	   writeback and unmount?
> > 	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
> > 	   to provide durability of all metadata changes?
> > 	d) do we have to care about ordering - if we fsync one inode
> > 	   with 1 hour old timestamps, do we also need to guarantee
> > 	   that all the inodes with older dirty timestamps also get
> > 	   made durable?
> 
> So the intended semantics is:
> 1) fsync / sync / freeze / unmount will write the timestamp updates even
>    with lazytime. So unless crash happens, timestamps are guaranteed to be
>    consistent. Also sync / fsync guarantees all changes to get to disk.
> 2) We periodically write back timestamps (once per 24 hours) to avoid too
>    big timestamp inconsistencies in case of crash.

Ok, so it's supposed to be a delayed timestamp update mechanism
without any specific ordering guarantees, not an opportunistic
timestamp update mechanism.

I can work with that.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-05 22:59     ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-05 22:59 UTC (permalink / raw)
  To: Jan Kara
  Cc: xfs-VZNHf3L845pBDgjK7y7TUQ, linux-ext4-u79uwXL29TY76Z2rM5mHXA,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
	linux-api-u79uwXL29TY76Z2rM5mHXA, tytso-3s7WtUTddSA

On Tue, Jan 05, 2016 at 06:36:04PM +0100, Jan Kara wrote:
>   Hi,
> 
> On Mon 04-01-16 17:22:19, Dave Chinner wrote:
> > I've been looking at implementing the lazytime mount option for XFS,
> > and I'm struggling to work out what it is supposed to mean.
> > 
> > AFAICT, on ext4, lazytime means that pure timestamp updates are not
> > journalled and they are only ever written back when the inode is
> > otherwise dirtied and written, or they are timestamp dirty for 24
> > hours which triggers writeback.
> > 
> > This poses a couple of problems for XFS:
> > 
> > 	1. we log every timestamp change, so there is no mechanism
> > 	   for delayed/deferred update.
> > 
> > 	2. we track dirty metadata in the journal, not via the VFS
> > 	   dirty inode lists, so all the infrastructure written for
> > 	   ext4 to do periodic flushing is useless to us.
> > 
> > These are solvable problems, but what I'm not sure about is exactly
> > what the intended semantics of lazytime durability are. That is,
> > exactly what guaranteed are we giving userspace about timestamp
> > updates when lazytime is used? The guarantees we have to give will
> > greatly influence the XFS implementation, so I really need to nail
> > down what we are expected to provide userspace. Can we:
> > 
> > 	a) just ignore all durability concerns?
> > 	b) if not, do we only need to care about the 24 hour
> > 	   writeback and unmount?
> > 	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
> > 	   to provide durability of all metadata changes?
> > 	d) do we have to care about ordering - if we fsync one inode
> > 	   with 1 hour old timestamps, do we also need to guarantee
> > 	   that all the inodes with older dirty timestamps also get
> > 	   made durable?
> 
> So the intended semantics is:
> 1) fsync / sync / freeze / unmount will write the timestamp updates even
>    with lazytime. So unless crash happens, timestamps are guaranteed to be
>    consistent. Also sync / fsync guarantees all changes to get to disk.
> 2) We periodically write back timestamps (once per 24 hours) to avoid too
>    big timestamp inconsistencies in case of crash.

Ok, so it's supposed to be a delayed timestamp update mechanism
without any specific ordering guarantees, not an opportunistic
timestamp update mechanism.

I can work with that.

Cheers,

Dave.
-- 
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-05 22:59     ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-05 22:59 UTC (permalink / raw)
  To: Jan Kara; +Cc: linux-fsdevel, linux-api, linux-ext4, tytso, xfs

On Tue, Jan 05, 2016 at 06:36:04PM +0100, Jan Kara wrote:
>   Hi,
> 
> On Mon 04-01-16 17:22:19, Dave Chinner wrote:
> > I've been looking at implementing the lazytime mount option for XFS,
> > and I'm struggling to work out what it is supposed to mean.
> > 
> > AFAICT, on ext4, lazytime means that pure timestamp updates are not
> > journalled and they are only ever written back when the inode is
> > otherwise dirtied and written, or they are timestamp dirty for 24
> > hours which triggers writeback.
> > 
> > This poses a couple of problems for XFS:
> > 
> > 	1. we log every timestamp change, so there is no mechanism
> > 	   for delayed/deferred update.
> > 
> > 	2. we track dirty metadata in the journal, not via the VFS
> > 	   dirty inode lists, so all the infrastructure written for
> > 	   ext4 to do periodic flushing is useless to us.
> > 
> > These are solvable problems, but what I'm not sure about is exactly
> > what the intended semantics of lazytime durability are. That is,
> > exactly what guaranteed are we giving userspace about timestamp
> > updates when lazytime is used? The guarantees we have to give will
> > greatly influence the XFS implementation, so I really need to nail
> > down what we are expected to provide userspace. Can we:
> > 
> > 	a) just ignore all durability concerns?
> > 	b) if not, do we only need to care about the 24 hour
> > 	   writeback and unmount?
> > 	c) if not, are fsync/sync/syncfs/freeze/unmount supposed
> > 	   to provide durability of all metadata changes?
> > 	d) do we have to care about ordering - if we fsync one inode
> > 	   with 1 hour old timestamps, do we also need to guarantee
> > 	   that all the inodes with older dirty timestamps also get
> > 	   made durable?
> 
> So the intended semantics is:
> 1) fsync / sync / freeze / unmount will write the timestamp updates even
>    with lazytime. So unless crash happens, timestamps are guaranteed to be
>    consistent. Also sync / fsync guarantees all changes to get to disk.
> 2) We periodically write back timestamps (once per 24 hours) to avoid too
>    big timestamp inconsistencies in case of crash.

Ok, so it's supposed to be a delayed timestamp update mechanism
without any specific ordering guarantees, not an opportunistic
timestamp update mechanism.

I can work with that.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
  2016-01-05 22:59     ` Dave Chinner
@ 2016-01-07  1:05       ` Theodore Ts'o
  -1 siblings, 0 replies; 14+ messages in thread
From: Theodore Ts'o @ 2016-01-07  1:05 UTC (permalink / raw)
  To: Dave Chinner; +Cc: Jan Kara, xfs, linux-ext4, linux-fsdevel, linux-api

On Wed, Jan 06, 2016 at 09:59:07AM +1100, Dave Chinner wrote:
> > So the intended semantics is:
> > 1) fsync / sync / freeze / unmount will write the timestamp updates even
> >    with lazytime. So unless crash happens, timestamps are guaranteed to be
> >    consistent. Also sync / fsync guarantees all changes to get to disk.
> > 2) We periodically write back timestamps (once per 24 hours) to avoid too
> >    big timestamp inconsistencies in case of crash.
> 
> Ok, so it's supposed to be a delayed timestamp update mechanism
> without any specific ordering guarantees, not an opportunistic
> timestamp update mechanism.

There is an optimization which ext4 has which will update related
timestamps when we write an inode table block, which is
"opportunistic", but there is no guarantee that this will happen.

This is purely optional; other file systems don't have to do this, but
it can be a win in that if related inodes are in the same 4k block,
and we need to update, say, the index file one because we are changing
i_size, but we were also doing non-allocating writes to the data file,
then we might as well write out the timestamps for the data file at
the same time, since this is "free".

						- Ted

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-07  1:05       ` Theodore Ts'o
  0 siblings, 0 replies; 14+ messages in thread
From: Theodore Ts'o @ 2016-01-07  1:05 UTC (permalink / raw)
  To: Dave Chinner; +Cc: linux-fsdevel, linux-api, linux-ext4, Jan Kara, xfs

On Wed, Jan 06, 2016 at 09:59:07AM +1100, Dave Chinner wrote:
> > So the intended semantics is:
> > 1) fsync / sync / freeze / unmount will write the timestamp updates even
> >    with lazytime. So unless crash happens, timestamps are guaranteed to be
> >    consistent. Also sync / fsync guarantees all changes to get to disk.
> > 2) We periodically write back timestamps (once per 24 hours) to avoid too
> >    big timestamp inconsistencies in case of crash.
> 
> Ok, so it's supposed to be a delayed timestamp update mechanism
> without any specific ordering guarantees, not an opportunistic
> timestamp update mechanism.

There is an optimization which ext4 has which will update related
timestamps when we write an inode table block, which is
"opportunistic", but there is no guarantee that this will happen.

This is purely optional; other file systems don't have to do this, but
it can be a win in that if related inodes are in the same 4k block,
and we need to update, say, the index file one because we are changing
i_size, but we were also doing non-allocating writes to the data file,
then we might as well write out the timestamps for the data file at
the same time, since this is "free".

						- Ted

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-07  2:21         ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-07  2:21 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: Jan Kara, xfs, linux-ext4, linux-fsdevel, linux-api

On Wed, Jan 06, 2016 at 08:05:06PM -0500, Theodore Ts'o wrote:
> On Wed, Jan 06, 2016 at 09:59:07AM +1100, Dave Chinner wrote:
> > > So the intended semantics is:
> > > 1) fsync / sync / freeze / unmount will write the timestamp updates even
> > >    with lazytime. So unless crash happens, timestamps are guaranteed to be
> > >    consistent. Also sync / fsync guarantees all changes to get to disk.
> > > 2) We periodically write back timestamps (once per 24 hours) to avoid too
> > >    big timestamp inconsistencies in case of crash.
> > 
> > Ok, so it's supposed to be a delayed timestamp update mechanism
> > without any specific ordering guarantees, not an opportunistic
> > timestamp update mechanism.
> 
> There is an optimization which ext4 has which will update related
> timestamps when we write an inode table block, which is
> "opportunistic", but there is no guarantee that this will happen.

XFS used to do that, too, before we removed all that hackery when we
moved to logging timestamp updates unconditionally a few years ago.
I'm going to have to re-instate some of that code for lazytime, I
think.

> This is purely optional; other file systems don't have to do this, but
> it can be a win in that if related inodes are in the same 4k block,
> and we need to update, say, the index file one because we are changing
> i_size, but we were also doing non-allocating writes to the data file,
> then we might as well write out the timestamps for the data file at
> the same time, since this is "free".

*nod*. Explicit, optimised clustered inode writeback (rather than
purely opportunistic clustering via delayed buffer writeback) was
added to XFS way back in early 1999. :)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-07  2:21         ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-07  2:21 UTC (permalink / raw)
  To: Theodore Ts'o
  Cc: Jan Kara, xfs-VZNHf3L845pBDgjK7y7TUQ,
	linux-ext4-u79uwXL29TY76Z2rM5mHXA,
	linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
	linux-api-u79uwXL29TY76Z2rM5mHXA

On Wed, Jan 06, 2016 at 08:05:06PM -0500, Theodore Ts'o wrote:
> On Wed, Jan 06, 2016 at 09:59:07AM +1100, Dave Chinner wrote:
> > > So the intended semantics is:
> > > 1) fsync / sync / freeze / unmount will write the timestamp updates even
> > >    with lazytime. So unless crash happens, timestamps are guaranteed to be
> > >    consistent. Also sync / fsync guarantees all changes to get to disk.
> > > 2) We periodically write back timestamps (once per 24 hours) to avoid too
> > >    big timestamp inconsistencies in case of crash.
> > 
> > Ok, so it's supposed to be a delayed timestamp update mechanism
> > without any specific ordering guarantees, not an opportunistic
> > timestamp update mechanism.
> 
> There is an optimization which ext4 has which will update related
> timestamps when we write an inode table block, which is
> "opportunistic", but there is no guarantee that this will happen.

XFS used to do that, too, before we removed all that hackery when we
moved to logging timestamp updates unconditionally a few years ago.
I'm going to have to re-instate some of that code for lazytime, I
think.

> This is purely optional; other file systems don't have to do this, but
> it can be a win in that if related inodes are in the same 4k block,
> and we need to update, say, the index file one because we are changing
> i_size, but we were also doing non-allocating writes to the data file,
> then we might as well write out the timestamps for the data file at
> the same time, since this is "free".

*nod*. Explicit, optimised clustered inode writeback (rather than
purely opportunistic clustering via delayed buffer writeback) was
added to XFS way back in early 1999. :)

Cheers,

Dave.
-- 
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: lazytime implementation questions
@ 2016-01-07  2:21         ` Dave Chinner
  0 siblings, 0 replies; 14+ messages in thread
From: Dave Chinner @ 2016-01-07  2:21 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: linux-fsdevel, linux-api, linux-ext4, Jan Kara, xfs

On Wed, Jan 06, 2016 at 08:05:06PM -0500, Theodore Ts'o wrote:
> On Wed, Jan 06, 2016 at 09:59:07AM +1100, Dave Chinner wrote:
> > > So the intended semantics is:
> > > 1) fsync / sync / freeze / unmount will write the timestamp updates even
> > >    with lazytime. So unless crash happens, timestamps are guaranteed to be
> > >    consistent. Also sync / fsync guarantees all changes to get to disk.
> > > 2) We periodically write back timestamps (once per 24 hours) to avoid too
> > >    big timestamp inconsistencies in case of crash.
> > 
> > Ok, so it's supposed to be a delayed timestamp update mechanism
> > without any specific ordering guarantees, not an opportunistic
> > timestamp update mechanism.
> 
> There is an optimization which ext4 has which will update related
> timestamps when we write an inode table block, which is
> "opportunistic", but there is no guarantee that this will happen.

XFS used to do that, too, before we removed all that hackery when we
moved to logging timestamp updates unconditionally a few years ago.
I'm going to have to re-instate some of that code for lazytime, I
think.

> This is purely optional; other file systems don't have to do this, but
> it can be a win in that if related inodes are in the same 4k block,
> and we need to update, say, the index file one because we are changing
> i_size, but we were also doing non-allocating writes to the data file,
> then we might as well write out the timestamps for the data file at
> the same time, since this is "free".

*nod*. Explicit, optimised clustered inode writeback (rather than
purely opportunistic clustering via delayed buffer writeback) was
added to XFS way back in early 1999. :)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2016-01-07  2:21 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-04  6:22 lazytime implementation questions Dave Chinner
2016-01-04  6:22 ` Dave Chinner
2016-01-04  6:22 ` Dave Chinner
2016-01-05 17:36 ` Jan Kara
2016-01-05 17:36   ` Jan Kara
2016-01-05 17:36   ` Jan Kara
2016-01-05 22:59   ` Dave Chinner
2016-01-05 22:59     ` Dave Chinner
2016-01-05 22:59     ` Dave Chinner
2016-01-07  1:05     ` Theodore Ts'o
2016-01-07  1:05       ` Theodore Ts'o
2016-01-07  2:21       ` Dave Chinner
2016-01-07  2:21         ` Dave Chinner
2016-01-07  2:21         ` Dave Chinner

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.