From: Waiman Long <longman@redhat.com>
To: Alexander Viro <viro@zeniv.linux.org.uk>,
Jonathan Corbet <corbet@lwn.net>,
Luis Chamberlain <mcgrof@kernel.org>,
Kees Cook <keescook@chromium.org>,
Iurii Zaikin <yzaikin@google.com>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-doc@vger.kernel.org,
Mauro Carvalho Chehab <mchehab+samsung@kernel.org>,
Eric Biggers <ebiggers@google.com>,
Dave Chinner <david@fromorbit.com>,
Eric Sandeen <sandeen@redhat.com>,
Waiman Long <longman@redhat.com>
Subject: [PATCH 00/11] fs/dcache: Limit # of negative dentries
Date: Wed, 26 Feb 2020 11:13:53 -0500 [thread overview]
Message-ID: <20200226161404.14136-1-longman@redhat.com> (raw)
As there is no limit for negative dentries, it is possible that a sizeable
portion of system memory can be tied up in dentry cache slabs. Dentry slabs
are generally recalimable if the dentries are in the LRUs. Still having
too much memory used up by dentries can be problematic:
1) When a filesystem with too many negative dentries is being unmounted,
the process of draining the dentries associated with the filesystem
can take some time. To users, the system may seem to hang for
a while. The long wait may also cause unexpected timeout error or
other warnings. This can happen when a long running container with
many negative dentries is being destroyed, for instance.
2) Tying up too much memory in unused negative dentries means there
are less memory available for other use. Even though the kernel is
able to reclaim unused dentries when running out of free memory,
it will still introduce additional latency to the application
reducing its performance.
There are two different approaches to limit negative dentries.
1) Global reclaim
Based on the total number of negative dentries as tracked by the
nr_dentry_negative percpu count, a function can be activated to
scan the various LRU lists to trim out excess negative dentries.
2) Local reclaim
By tracking the number of negative dentries under each directory,
we can start the reclaim process if the number exceeds a certain
limit.
The problem with global reclaim is that there are just too many LRU lists
present that may need to be scanned for each filesystem. Especially
problematic is the fact that each memory cgroup can have its own LRU
lists. As memory cgroup can come and go at any time, scanning its LRUs
can be tricky.
Local reclaim does not have this problem. So it is used as the basis
for negative dentry reclaim for this patchset. Accurately tracking the
number of negative dentries in each directory can be costly in term of
performance hit. As a result, this patchset estimates the number of
negative dentries present in a directory by looking at a newly added
children count and an opportunistically stored positive dentry count.
A new sysctl parameter "dentry-dir-max" is introduced which accepts a
value of 0 (default) for no limit or a positive integer 256 and up. Small
dentry-dir-max numbers are forbidden to avoid excessive dentry count
checking which can impact system performance.
The actual negative dentry reclaim is delegated to the system workqueue
to avoid adding excessive latency to normal filesystem operation.
Waiman Long (11):
fs/dcache: Fix incorrect accounting of negative dentries
fs/dcache: Simplify __dentry_kill()
fs/dcache: Add a counter to track number of children
fs/dcache: Add sysctl parameter dentry-dir-max
fs/dcache: Reclaim excessive negative dentries in directories
fs/dcache: directory opportunistically stores # of positive dentries
fs/dcache: Add static key negative_reclaim_enable
fs/dcache: Limit dentry reclaim count in negative_reclaim_workfn()
fs/dcache: Don't allow small values for dentry-dir-max
fs/dcache: Kill off dentry as last resort
fs/dcache: Track # of negative dentries reclaimed & killed
Documentation/admin-guide/sysctl/fs.rst | 18 +
fs/dcache.c | 428 +++++++++++++++++++++++-
include/linux/dcache.h | 18 +-
kernel/sysctl.c | 11 +
4 files changed, 457 insertions(+), 18 deletions(-)
--
2.18.1
next reply other threads:[~2020-02-26 16:15 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-26 16:13 Waiman Long [this message]
2020-02-26 16:13 ` [PATCH 01/11] fs/dcache: Fix incorrect accounting of negative dentries Waiman Long
2020-02-26 16:13 ` [PATCH 02/11] fs/dcache: Simplify __dentry_kill() Waiman Long
2020-02-26 16:13 ` [PATCH 03/11] fs/dcache: Add a counter to track number of children Waiman Long
2020-02-26 16:13 ` [PATCH 04/11] fs/dcache: Add sysctl parameter dentry-dir-max Waiman Long
2020-02-26 16:13 ` [PATCH 05/11] fs/dcache: Reclaim excessive negative dentries in directories Waiman Long
2020-02-26 16:13 ` [PATCH 06/11] fs/dcache: directory opportunistically stores # of positive dentries Waiman Long
2020-02-26 16:14 ` [PATCH 07/11] fs/dcache: Add static key negative_reclaim_enable Waiman Long
2020-02-26 16:14 ` [PATCH 08/11] fs/dcache: Limit dentry reclaim count in negative_reclaim_workfn() Waiman Long
2020-02-26 16:14 ` [PATCH 09/11] fs/dcache: Don't allow small values for dentry-dir-max Waiman Long
2020-02-26 16:14 ` [PATCH 10/11] fs/dcache: Kill off dentry as last resort Waiman Long
2020-02-26 16:14 ` [PATCH 11/11] fs/dcache: Track # of negative dentries reclaimed & killed Waiman Long
2020-02-26 16:29 ` [PATCH 00/11] fs/dcache: Limit # of negative dentries Matthew Wilcox
2020-02-26 19:19 ` Waiman Long
2020-02-26 21:28 ` Matthew Wilcox
2020-02-26 21:28 ` Andreas Dilger
2020-02-26 21:45 ` Matthew Wilcox
2020-02-27 8:07 ` Dave Chinner
2020-02-27 9:55 ` Ian Kent
2020-02-28 3:34 ` Matthew Wilcox
2020-02-28 4:16 ` Ian Kent
2020-02-28 4:36 ` Ian Kent
2020-02-28 4:52 ` Al Viro
2020-02-28 4:22 ` Al Viro
2020-02-28 4:52 ` Ian Kent
2020-02-28 15:32 ` Waiman Long
2020-02-28 15:39 ` Matthew Wilcox
2020-02-28 19:32 ` Theodore Y. Ts'o
2020-02-27 19:04 ` Eric Sandeen
2020-02-27 22:39 ` Dave Chinner
2020-02-27 8:30 ` Dave Chinner
2020-02-28 15:47 ` Waiman Long
2020-03-15 3:46 ` Matthew Wilcox
2020-03-21 10:17 ` Konstantin Khlebnikov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200226161404.14136-1-longman@redhat.com \
--to=longman@redhat.com \
--cc=corbet@lwn.net \
--cc=david@fromorbit.com \
--cc=ebiggers@google.com \
--cc=keescook@chromium.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mcgrof@kernel.org \
--cc=mchehab+samsung@kernel.org \
--cc=sandeen@redhat.com \
--cc=viro@zeniv.linux.org.uk \
--cc=yzaikin@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.