From: Johannes Weiner <hannes@cmpxchg.org>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
Jan Kara <jack@suse.cz>, Dave Jones <davej@codemonkey.org.uk>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
kernel-team@fb.com
Subject: [PATCH 3/5] lib: radix-tree: native accounting and tracking of special entries
Date: Wed, 19 Oct 2016 13:24:26 -0400 [thread overview]
Message-ID: <20161019172428.7649-4-hannes@cmpxchg.org> (raw)
In-Reply-To: <20161019172428.7649-1-hannes@cmpxchg.org>
Add an internal tag to identify special entries that are accounted in
node->special in addition to node->count.
With this in place, the next patch can restore refault detection in
single-page files. It will also move the shadow count from the upper
bits of count to the new special counter, and then shrink count to a
char as well; the growth of struct radix_tree_node is temporary.
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
---
include/linux/radix-tree.h | 10 ++++++----
lib/radix-tree.c | 14 ++++++++++----
2 files changed, 16 insertions(+), 8 deletions(-)
diff --git a/include/linux/radix-tree.h b/include/linux/radix-tree.h
index 756b2909467e..2e1c9added23 100644
--- a/include/linux/radix-tree.h
+++ b/include/linux/radix-tree.h
@@ -68,7 +68,8 @@ enum radix_tree_tags {
/* Freely allocatable radix tree user tags */
RADIX_TREE_NR_USER_TAGS = 3,
/* Radix tree internal tags */
- RADIX_TREE_NR_TAGS = RADIX_TREE_NR_USER_TAGS,
+ RADIX_TREE_TAG_SPECIAL = RADIX_TREE_NR_USER_TAGS,
+ RADIX_TREE_NR_TAGS,
};
#ifndef RADIX_TREE_MAP_SHIFT
@@ -90,9 +91,10 @@ enum radix_tree_tags {
#define RADIX_TREE_COUNT_MASK ((1UL << RADIX_TREE_COUNT_SHIFT) - 1)
struct radix_tree_node {
- unsigned char shift; /* Bits remaining in each slot */
- unsigned char offset; /* Slot offset in parent */
- unsigned int count;
+ unsigned char shift; /* Bits remaining in each slot */
+ unsigned char offset; /* Slot offset in parent */
+ unsigned int count; /* Total entry count */
+ unsigned char special; /* Special entry count */
union {
struct {
/* Used when ascending tree */
diff --git a/lib/radix-tree.c b/lib/radix-tree.c
index bb6ddfb60557..e58cff1d97ed 100644
--- a/lib/radix-tree.c
+++ b/lib/radix-tree.c
@@ -220,10 +220,10 @@ static void dump_node(struct radix_tree_node *node, unsigned long index)
{
unsigned long i;
- pr_debug("radix node: %p offset %d tags %lx %lx %lx shift %d count %d parent %p\n",
+ pr_debug("radix node: %p offset %d tags %lx %lx %lx shift %d count %d special %d parent %p\n",
node, node->offset,
node->tags[0][0], node->tags[1][0], node->tags[2][0],
- node->shift, node->count, node->parent);
+ node->shift, node->count, node->special, node->parent);
for (i = 0; i < RADIX_TREE_MAP_SIZE; i++) {
unsigned long first = index | (i << node->shift);
@@ -522,9 +522,15 @@ static int radix_tree_extend(struct radix_tree_root *root,
node->offset = 0;
node->count = 1;
node->parent = NULL;
- if (radix_tree_is_internal_node(slot))
- entry_to_node(slot)->parent = node;
node->slots[0] = slot;
+ /* Extending an existing node or root->rnode? */
+ if (radix_tree_is_internal_node(slot)) {
+ entry_to_node(slot)->parent = node;
+ } else {
+ /* Moving a special root->rnode to a node */
+ if (root_tag_get(root, RADIX_TREE_TAG_SPECIAL))
+ node->special = 1;
+ }
slot = node_to_entry(node);
rcu_assign_pointer(root->rnode, slot);
shift += RADIX_TREE_MAP_SHIFT;
--
2.10.0
next prev parent reply other threads:[~2016-10-19 17:25 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-19 17:24 [PATCH 0/5] mm: workingset: radix tree subtleties & single-page file refaults Johannes Weiner
2016-10-19 17:24 ` [PATCH 1/5] lib: radix-tree: provide node-granular interface for radix tree tags Johannes Weiner
2016-10-19 17:24 ` [PATCH 2/5] lib: radix-tree: internal tags Johannes Weiner
2016-10-19 17:24 ` Johannes Weiner [this message]
2016-10-20 22:33 ` [PATCH 3/5] lib: radix-tree: native accounting and tracking of special entries Dave Chinner
2016-10-24 16:01 ` Johannes Weiner
2016-10-19 17:24 ` [PATCH 4/5] mm: workingset: restore single-page file refault tracking Johannes Weiner
2016-10-19 17:24 ` [PATCH 5/5] mm: workingset: turn shadow node shrinker bugs into warnings Johannes Weiner
2016-10-19 18:16 ` [PATCH 0/5] mm: workingset: radix tree subtleties & single-page file refaults Linus Torvalds
2016-10-24 18:47 ` Johannes Weiner
2016-10-26 9:21 ` Jan Kara
2016-10-26 18:15 ` Johannes Weiner
2016-10-27 8:48 ` Jan Kara
2016-10-26 18:18 ` Linus Torvalds
2016-10-26 18:29 ` Johannes Weiner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161019172428.7649-4-hannes@cmpxchg.org \
--to=hannes@cmpxchg.org \
--cc=akpm@linux-foundation.org \
--cc=davej@codemonkey.org.uk \
--cc=jack@suse.cz \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).