From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
To: rcu@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, urezki@gmail.com,
paulmck@kernel.org,
"Joel Fernandes (Google)" <joel@joelfernandes.org>
Subject: [PATCH v2] rcu/kfree: Do not request RCU when not needed
Date: Fri, 4 Nov 2022 14:21:35 +0000 [thread overview]
Message-ID: <20221104142135.991888-1-joel@joelfernandes.org> (raw)
On ChromeOS, using this with the increased timeout, we see that we almost always
never need to initiate a new grace period. Testing also shows this frees large
amounts of unreclaimed memory, under intense kfree_rcu() pressure.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
v1->v2: Same logic but use polled grace periods instead of sampling gp_seq.
kernel/rcu/tree.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 591187b6352e..ed41243f7a49 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2935,6 +2935,7 @@ struct kfree_rcu_cpu_work {
/**
* struct kfree_rcu_cpu - batch up kfree_rcu() requests for RCU grace period
+ * @gp_snap: The GP snapshot recorded at the last scheduling of monitor work.
* @head: List of kfree_rcu() objects not yet waiting for a grace period
* @bkvhead: Bulk-List of kvfree_rcu() objects not yet waiting for a grace period
* @krw_arr: Array of batches of kfree_rcu() objects waiting for a grace period
@@ -2964,6 +2965,7 @@ struct kfree_rcu_cpu {
struct kfree_rcu_cpu_work krw_arr[KFREE_N_BATCHES];
raw_spinlock_t lock;
struct delayed_work monitor_work;
+ unsigned long gp_snap;
bool initialized;
int count;
@@ -3167,6 +3169,7 @@ schedule_delayed_monitor_work(struct kfree_rcu_cpu *krcp)
mod_delayed_work(system_wq, &krcp->monitor_work, delay);
return;
}
+ krcp->gp_snap = get_state_synchronize_rcu();
queue_delayed_work(system_wq, &krcp->monitor_work, delay);
}
@@ -3217,7 +3220,10 @@ static void kfree_rcu_monitor(struct work_struct *work)
// be that the work is in the pending state when
// channels have been detached following by each
// other.
- queue_rcu_work(system_wq, &krwp->rcu_work);
+ if (poll_state_synchronize_rcu(krcp->gp_snap))
+ queue_work(system_wq, &krwp->rcu_work.work);
+ else
+ queue_rcu_work(system_wq, &krwp->rcu_work);
}
}
--
2.38.1.431.g37b22c650d-goog
next reply other threads:[~2022-11-04 14:24 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-04 14:21 Joel Fernandes (Google) [this message]
2022-11-09 2:47 [PATCH v2] rcu/kfree: Do not request RCU when not needed Joel Fernandes (Google)
2022-11-10 13:05 ` Uladzislau Rezki
[not found] ` <CAEXW_YSq89xzgyQ9Tdt1tCqz8VAfzb7kSXVZmnxDuJ65U0UZ3w@mail.gmail.com>
2022-11-10 14:01 ` Uladzislau Rezki
2022-11-11 1:56 ` Joel Fernandes
2022-11-14 12:20 ` Uladzislau Rezki
2022-11-14 16:17 ` Paul E. McKenney
2022-11-14 20:54 ` Joel Fernandes
2022-11-14 21:26 ` Paul E. McKenney
2022-11-15 12:05 ` Uladzislau Rezki
2022-11-14 20:49 ` Joel Fernandes
2022-11-15 13:07 ` Uladzislau Rezki
2022-11-16 19:19 ` Uladzislau Rezki
2022-11-16 22:05 ` Joel Fernandes
2022-11-17 12:58 ` Uladzislau Rezki
2022-11-17 13:06 ` Joel Fernandes
2022-11-17 13:11 ` Uladzislau Rezki
2022-11-17 13:23 ` Joel Fernandes
2022-11-17 13:43 ` Uladzislau Rezki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221104142135.991888-1-joel@joelfernandes.org \
--to=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@kernel.org \
--cc=rcu@vger.kernel.org \
--cc=urezki@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.