Re: Query about ext4 commit interval vs dirty_expire_centisecs

[Paul Richards <paul.richards@gmail.com>]

On Fri, 13 Dec 2019 at 15:59, Jan Kara <jack@suse.cz> wrote:
>
> Hello!
>
> On Tue 19-11-19 08:47:31, Paul Richards wrote:
> > I'm trying to understand the interaction between the ext4 `commit`
> > interval option, and the `vm.dirty_expire_centisecs` tuneable.
> >
> > The ext4 `commit` documentation says:
> >
> > > Ext4 can be told to sync all its data and metadata every 'nrsec' seconds. The default value is 5 seconds. This means that if you lose your power, you will lose as much as the latest 5 seconds of work (your filesystem will not be damaged though, thanks to the journaling).
> >
> > The `dirty_expire_centisecs` documentation says:
> >
> > > This tunable is used to define when dirty data is old enough to be eligible for writeout by the kernel flusher threads. It is expressed in 100'ths of a second. Data which has been dirty in-memory for longer than this interval will be written out next time a flusher thread wakes up.
> >
> >
> > Superficially these sound like they have a very similar effect.  They
> > periodically flush out data that hasn't been explicitly fsync'd by the
> > application.  I'd like to understand a bit more the interaction
> > between these.
>
> Yes, the effect is rather similar but not quite the same. The first thing
> to observe is kind of obvious fact that ext4 commit interval influences
> just the particular filesystem while dirty_expire_centisecs influences
> behavior of global writeback over all filesystems.
>
> Secondly, commit interval is really the maximum age of ext4 transation.  So
> if there is metadata change pending in the journal, it will become
> persistent at latest after this time. So for say 'mkdir' that will be
> persistent at latest after this time. For data operations things are more
> complex. E.g. when delayed allocation is used (which is the default), the
> change gets logged in the journal only during writeback. So it can take up
> to dirty_expire_centisecs for data to be written back from page cache, that
> results in filesystem journalling block allocations etc. and then it can
> take upto commit interval for these changes to become persistent. So in
> this case the intervals add up. There are also other special cases
> somewhere in between but generally it is reasonable to assume that data gets
> automatically persistent in dirty_expire_centisecs + commit_interval time.
> Note both these times are actually times when writeback is triggered so
> if the disk gets too busy, the actual time when data is completely on disk
> may be much higher.
>

Thanks for taking the time to reply!

Since automatic persisting of data occurs only after
dirty_expire_centisecs + commit_interval,
should the ext4 docs be corrected?  They currently state (for the
commit interval option):

"The default value is 5 seconds. This means that if you lose
your power, you will lose as much as the latest 5 seconds of work"