All of lore.kernel.org
 help / color / mirror / Atom feed
* Deferred inode inactivation and nonblocking inode reclaim
@ 2020-04-22 22:58 Darrick J. Wong
  2020-04-22 23:05 ` [XFS SUMMIT] " Darrick J. Wong
  0 siblings, 1 reply; 3+ messages in thread
From: Darrick J. Wong @ 2020-04-22 22:58 UTC (permalink / raw)
  To: xfs

Hi everyone,

Here's a jumping-off point for a discussion about my patchset that
implements deferred inode inactivation and Dave's patchset that moves
inode buffer flushing out of reclaim.

The inactivation series moves the transactional updates that happen
after a file loses its last reference (truncating attr/data forks,
freeing the inode) out of drop_inode and reclaim by moving all that work
to an intermediate workqueue.  This all can be done internally to XFS.

The reclaim series (Dave) removes inode flushing from reclaim, which
means that xfs stop holding up memory reclaim on IO.  It also contains a
fair amount of surgery to the memory shrinker code, which is an added
impediment to getting this series reviewed and upstream.

Because of the extra review needed for the reclaim series, does it make
sense to keep the two separate?  Deferring inactivation alone won't get
rid of the inode flushing that goes on in reclaim, but it at least means
that we can handle things like "rm -rf $dir" a little more efficiently
in that we can do all the directory shrinking at once and then handle
the unlinked inodes in on-disk order.  It would also, erm, help me
reduce the size of my dev tree. :)

--D

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [XFS SUMMIT] Deferred inode inactivation and nonblocking inode reclaim
  2020-04-22 22:58 Deferred inode inactivation and nonblocking inode reclaim Darrick J. Wong
@ 2020-04-22 23:05 ` Darrick J. Wong
  2020-04-23  6:46   ` Amir Goldstein
  0 siblings, 1 reply; 3+ messages in thread
From: Darrick J. Wong @ 2020-04-22 23:05 UTC (permalink / raw)
  To: xfs; +Cc: Amir Goldstein, Dave Chinner

Heh, only after I sent this did I think about tagging the subject line
and sending links to git branches when applicable.

On Wed, Apr 22, 2020 at 03:58:51PM -0700, Darrick J. Wong wrote:
> Hi everyone,
> 
> Here's a jumping-off point for a discussion about my patchset that
> implements deferred inode inactivation and Dave's patchset that moves
> inode buffer flushing out of reclaim.
> 
> The inactivation series moves the transactional updates that happen

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=deferred-inactivation

> after a file loses its last reference (truncating attr/data forks,
> freeing the inode) out of drop_inode and reclaim by moving all that work
> to an intermediate workqueue.  This all can be done internally to XFS.
> 
> The reclaim series (Dave) removes inode flushing from reclaim, which

https://lore.kernel.org/linux-xfs/20191031234618.15403-1-david@fromorbit.com/

--D

> means that xfs stop holding up memory reclaim on IO.  It also contains a
> fair amount of surgery to the memory shrinker code, which is an added
> impediment to getting this series reviewed and upstream.
> 
> Because of the extra review needed for the reclaim series, does it make
> sense to keep the two separate?  Deferring inactivation alone won't get
> rid of the inode flushing that goes on in reclaim, but it at least means
> that we can handle things like "rm -rf $dir" a little more efficiently
> in that we can do all the directory shrinking at once and then handle
> the unlinked inodes in on-disk order.  It would also, erm, help me
> reduce the size of my dev tree. :)
> 
> --D

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [XFS SUMMIT] Deferred inode inactivation and nonblocking inode reclaim
  2020-04-22 23:05 ` [XFS SUMMIT] " Darrick J. Wong
@ 2020-04-23  6:46   ` Amir Goldstein
  0 siblings, 0 replies; 3+ messages in thread
From: Amir Goldstein @ 2020-04-23  6:46 UTC (permalink / raw)
  To: Darrick J. Wong; +Cc: xfs, Dave Chinner, Chris Mason

On Thu, Apr 23, 2020 at 2:05 AM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> Heh, only after I sent this did I think about tagging the subject line
> and sending links to git branches when applicable.
>
> On Wed, Apr 22, 2020 at 03:58:51PM -0700, Darrick J. Wong wrote:
> > Hi everyone,
> >
> > Here's a jumping-off point for a discussion about my patchset that
> > implements deferred inode inactivation and Dave's patchset that moves
> > inode buffer flushing out of reclaim.
> >
> > The inactivation series moves the transactional updates that happen
>
> https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=deferred-inactivation
>
> > after a file loses its last reference (truncating attr/data forks,
> > freeing the inode) out of drop_inode and reclaim by moving all that work
> > to an intermediate workqueue.  This all can be done internally to XFS.
> >
> > The reclaim series (Dave) removes inode flushing from reclaim, which
>
> https://lore.kernel.org/linux-xfs/20191031234618.15403-1-david@fromorbit.com/
>
> --D
>
> > means that xfs stop holding up memory reclaim on IO.  It also contains a
> > fair amount of surgery to the memory shrinker code, which is an added
> > impediment to getting this series reviewed and upstream.
> >

+CC: Chris Mason

And here is a link to one of the famous bug reports:
https://lore.kernel.org/linux-xfs/06aade22-b29e-f55e-7f00-39154f220aa6@fb.com/
which also includes a reference to Chris's simoop reproducer.

How about bringing in some FB engineers to the task force to help with
reviewing the memory shrinker changes and with testing?
I believe it is in the best interest of the wider filesystem community to
expedite development of this solution.

Thanks,
Amir.

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2020-04-23  6:46 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-22 22:58 Deferred inode inactivation and nonblocking inode reclaim Darrick J. Wong
2020-04-22 23:05 ` [XFS SUMMIT] " Darrick J. Wong
2020-04-23  6:46   ` Amir Goldstein

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.