Re: [PATCH v6 0/7] fs/dcache: Track & limit # of negative dentries

From: Dave Chinner <david@fromorbit.com>
To: Michal Hocko <mhocko@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Matthew Wilcox <willy@infradead.org>,
	James Bottomley <James.Bottomley@HansenPartnership.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Waiman Long <longman@redhat.com>,
	Al Viro <viro@zeniv.linux.org.uk>,
	Jonathan Corbet <corbet@lwn.net>,
	"Luis R. Rodriguez" <mcgrof@kernel.org>,
	Kees Cook <keescook@chromium.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	Jan Kara <jack@suse.cz>,
	Paul McKenney <paulmck@linux.vnet.ibm.com>,
	Ingo Molnar <mingo@kernel.org>,
	Miklos Szeredi <mszeredi@redhat.com>,
	Larry Woodman <lwoodman@redhat.com>,
	"Wangkai (Kevin,C)" <wangkai86@huawei.com>
Subject: Re: [PATCH v6 0/7] fs/dcache: Track & limit # of negative dentries
Date: Thu, 19 Jul 2018 10:33:29 +1000	[thread overview]
Message-ID: <20180719003329.GD19934@dastard> (raw)
In-Reply-To: <20180717083326.GD16803@dhcp22.suse.cz>

On Tue, Jul 17, 2018 at 10:33:26AM +0200, Michal Hocko wrote:
> On Mon 16-07-18 16:40:32, Andrew Morton wrote:
> > On Mon, 16 Jul 2018 05:41:15 -0700 Matthew Wilcox <willy@infradead.org> wrote:
> > It's quite a small code change and would provide a mechanism for
> > implementing the hammer-cache-until-youve-freed-enough design above.
> > 
> > 
> > 
> > Aside 2: if we *do* do something like the above __d_alloc() pseudo code
> > then perhaps it could be cast in terms of pages, not dentries.  ie,
> > 
> > __d_alloc()
> > {
> > 	...
> > 	while (too many pages in dentry_cache)
> > 		call the dcache shrinker
> > 	...
> > }

Direct reclaim will result in all the people who care about long
tail latencies and/or highly concurent workloads starting to hate
you.  Direct reclaim already hammers superblock shrinkers with
excessive concurrency, this would only make it worse.

IOWs, anything like this needs to co-ordinate with other reclaim
operations in progress and, most likely, be done via background
reclaim processing rather than blocking new allocations
indefinitely. background processing can be done in bulk and as
efficiently as possible - concurrent direct reclaim in tiny batches
will just hammer dcache locks and destroy performance when there is
memory pressure.

How many times do we have to learn this lesson the hard way?

> > and, apart from the external name thing (grr), that should address
> > these fragmentation issues, no?  I assume it's easy to ask slab how
> > many pages are presently in use for a particular cache.
> 
> I remember Dave Chinner had an idea how to age dcache pages to push
> dentries with similar live time to the same page. Not sure what happened
> to that.

Same thing that happened to all the "select the dentries on this
page for reclaim". i.e. it's referenced dentries that we can't
reclaim or move that are the issue, not the reclaimable dentries on
the page.

Bsaically, without a hint at allocation time as to the expected life
time of the dentry, we can't be smart about how we select partial
pages to allocate from. And because we don't know at allocation time
if the dentry is going to remain a negative dentry or not, we can't
provide a hint about expected lifetime of teh object being
allocated.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com