From: Eric Sandeen <sandeen@redhat.com>
To: Matthew Wilcox <willy@infradead.org>, Waiman Long <longman@redhat.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>,
Jonathan Corbet <corbet@lwn.net>,
Luis Chamberlain <mcgrof@kernel.org>,
Kees Cook <keescook@chromium.org>,
Iurii Zaikin <yzaikin@google.com>,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-doc@vger.kernel.org,
Mauro Carvalho Chehab <mchehab+samsung@kernel.org>,
Eric Biggers <ebiggers@google.com>,
Dave Chinner <david@fromorbit.com>
Subject: Re: [PATCH 00/11] fs/dcache: Limit # of negative dentries
Date: Thu, 27 Feb 2020 11:04:40 -0800 [thread overview]
Message-ID: <0e5124a2-d730-5c41-38fd-2c78d9be4940@redhat.com> (raw)
In-Reply-To: <20200226162954.GC24185@bombadil.infradead.org>
On 2/26/20 8:29 AM, Matthew Wilcox wrote:
> On Wed, Feb 26, 2020 at 11:13:53AM -0500, Waiman Long wrote:
>> A new sysctl parameter "dentry-dir-max" is introduced which accepts a
>> value of 0 (default) for no limit or a positive integer 256 and up. Small
>> dentry-dir-max numbers are forbidden to avoid excessive dentry count
>> checking which can impact system performance.
>
> This is always the wrong approach. A sysctl is just a way of blaming
> the sysadmin for us not being very good at programming.
>
> I agree that we need a way to limit the number of negative dentries.
> But that limit needs to be dynamic and depend on how the system is being
> used, not on how some overworked sysadmin has configured it.
>
> So we need an initial estimate for the number of negative dentries that
> we need for good performance. Maybe it's 1000. It doesn't really matter;
> it's going to change dynamically.
>
> Then we need a metric to let us know whether it needs to be increased.
> Perhaps that's "number of new negative dentries created in the last
> second". And we need to decide how much to increase it; maybe it's by
> 50% or maybe by 10%. Perhaps somewhere between 10-100% depending on
> how high the recent rate of negative dentry creation has been.
There are pitfalls to this approach as well. Consider what libnss
does every time it starts up (via curl in this case)
# cat /proc/sys/fs/dentry-state
3154271 3131421 45 0 2863333 0
# for I in `seq 1 10`; do curl https://sandeen.net/ &>/dev/null; done
# cat /proc/sys/fs/dentry-state
3170738 3147844 45 0 2879882 0
voila, 16k more negative dcache entries, thanks to:
https://github.com/nss-dev/nss/blob/317cb06697d5b953d825e050c1d8c1ee0d647010/lib/softoken/sdb.c#L390
i.e. each time it inits, it will intentionally create up to 10,000 negative
dentries which will never be looked up again. I /think/ the original intent
of this work was to limit such rogue applications, so scaling with use probably
isn't the way to go.
-Eric
next prev parent reply other threads:[~2020-02-27 19:04 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-26 16:13 [PATCH 00/11] fs/dcache: Limit # of negative dentries Waiman Long
2020-02-26 16:13 ` [PATCH 01/11] fs/dcache: Fix incorrect accounting " Waiman Long
2020-02-26 16:13 ` [PATCH 02/11] fs/dcache: Simplify __dentry_kill() Waiman Long
2020-02-26 16:13 ` [PATCH 03/11] fs/dcache: Add a counter to track number of children Waiman Long
2020-02-26 16:13 ` [PATCH 04/11] fs/dcache: Add sysctl parameter dentry-dir-max Waiman Long
2020-02-26 16:13 ` [PATCH 05/11] fs/dcache: Reclaim excessive negative dentries in directories Waiman Long
2020-02-26 16:13 ` [PATCH 06/11] fs/dcache: directory opportunistically stores # of positive dentries Waiman Long
2020-02-26 16:14 ` [PATCH 07/11] fs/dcache: Add static key negative_reclaim_enable Waiman Long
2020-02-26 16:14 ` [PATCH 08/11] fs/dcache: Limit dentry reclaim count in negative_reclaim_workfn() Waiman Long
2020-02-26 16:14 ` [PATCH 09/11] fs/dcache: Don't allow small values for dentry-dir-max Waiman Long
2020-02-26 16:14 ` [PATCH 10/11] fs/dcache: Kill off dentry as last resort Waiman Long
2020-02-26 16:14 ` [PATCH 11/11] fs/dcache: Track # of negative dentries reclaimed & killed Waiman Long
2020-02-26 16:29 ` [PATCH 00/11] fs/dcache: Limit # of negative dentries Matthew Wilcox
2020-02-26 19:19 ` Waiman Long
2020-02-26 21:28 ` Matthew Wilcox
2020-02-26 21:28 ` Andreas Dilger
2020-02-26 21:45 ` Matthew Wilcox
2020-02-27 8:07 ` Dave Chinner
2020-02-27 9:55 ` Ian Kent
2020-02-28 3:34 ` Matthew Wilcox
2020-02-28 4:16 ` Ian Kent
2020-02-28 4:36 ` Ian Kent
2020-02-28 4:52 ` Al Viro
2020-02-28 4:22 ` Al Viro
2020-02-28 4:52 ` Ian Kent
2020-02-28 15:32 ` Waiman Long
2020-02-28 15:39 ` Matthew Wilcox
2020-02-28 19:32 ` Theodore Y. Ts'o
2020-02-27 19:04 ` Eric Sandeen [this message]
2020-02-27 22:39 ` Dave Chinner
2020-02-27 8:30 ` Dave Chinner
2020-02-28 15:47 ` Waiman Long
2020-03-15 3:46 ` Matthew Wilcox
2020-03-21 10:17 ` Konstantin Khlebnikov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0e5124a2-d730-5c41-38fd-2c78d9be4940@redhat.com \
--to=sandeen@redhat.com \
--cc=corbet@lwn.net \
--cc=david@fromorbit.com \
--cc=ebiggers@google.com \
--cc=keescook@chromium.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mcgrof@kernel.org \
--cc=mchehab+samsung@kernel.org \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
--cc=yzaikin@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).