Re: [LSF/MM TOPIC] Better handling of negative dentries

From: Matthew Wilcox <willy@infradead.org>
To: Colin Walters <walters@verbum.org>
Cc: James Bottomley <James.Bottomley@hansenpartnership.com>,
	Dave Chinner <david@fromorbit.com>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	Stephen Brennan <stephen.s.brennan@oracle.com>,
	lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org,
	Gautham Ananthakrishna <gautham.ananthakrishna@oracle.com>,
	khlebnikov@yandex-team.ru
Subject: Re: [LSF/MM TOPIC] Better handling of negative dentries
Date: Tue, 22 Mar 2022 20:37:21 +0000	[thread overview]
Message-ID: <YjozgfjcNLXIQKhG@casper.infradead.org> (raw)
In-Reply-To: <42d92c6f-28f2-459b-bc2a-13dd655dd4ae@www.fastmail.com>

On Tue, Mar 22, 2022 at 04:17:16PM -0400, Colin Walters wrote:
> 
> 
> On Tue, Mar 22, 2022, at 3:19 PM, James Bottomley wrote:
> > 
> > Well, firstly what is the exact problem?  People maliciously looking up
> > nonexistent files
> 
> Maybe most people have seen it, but for those who haven't:
> https://bugzilla.redhat.com/show_bug.cgi?id=1571183
> was definitely one of those things that just makes one recoil in horror.
> 
> TL;DR NSS used to have code that tried to detect "is this a network filesystem"
> by timing `stat()` calls to nonexistent paths, and this massively boated
> the negative dentry cache and caused all sorts of performance problems.
> It was particularly confusing because this would just happen as a side effect of e.g. executing `curl https://somewebsite`.
> 
> That code wasn't *intentionally* malicious but...

Oh, the situation where we encountered the problem was systemd.
Definitely not malicious, and not even stupid (as the NSS example above).
I forget exactly which thing it was, but on some fairly common event
(user login?), it looked up a file in a PATH of some type, failed
to find it in the first two directories, then created it in a third.
At logout, it deleted the file.  Now there are three negative dentries.
Repeat a few million times (each time looking for a different file)
with no memory pressure and you have a thoroughly soggy machine that
is faster to reboot than to reclaim dentries.