From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
To: linux-kernel@vger.kernel.org
Cc: "Joel Fernandes (Google)" <joel@joelfernandes.org>,
Andrew Morton <akpm@linux-foundation.org>,
Ingo Molnar <mingo@redhat.com>,
Josh Triplett <josh@joshtriplett.org>,
Lai Jiangshan <jiangshanlai@gmail.com>,
linux-mm@kvack.org,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
"Paul E. McKenney" <paulmck@kernel.org>,
"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
rcu@vger.kernel.org, Steven Rostedt <rostedt@goodmis.org>,
"Uladzislau Rezki (Sony)" <urezki@gmail.com>
Subject: [PATCH 18/18] rcu/tree: Make kvfree_rcu() tolerate any alignment
Date: Sun, 29 Mar 2020 22:32:48 -0400 [thread overview]
Message-ID: <20200330023248.164994-19-joel@joelfernandes.org> (raw)
In-Reply-To: <20200330023248.164994-1-joel@joelfernandes.org>
Handle cases where the the object being kvfree_rcu()'d is not aligned by
2-byte boundaries.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
kernel/rcu/tree.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 311d216c7faa7..d6536374d12a9 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2827,6 +2827,9 @@ struct kvfree_rcu_bulk_data {
#define KVFREE_BULK_MAX_ENTR \
((PAGE_SIZE - sizeof(struct kvfree_rcu_bulk_data)) / sizeof(void *))
+/* Encoding the offset of a fake rcu_head to indicate the head is a wrapper. */
+#define RCU_HEADLESS_KFREE BIT(31)
+
/**
* struct kfree_rcu_cpu_work - single batch of kfree_rcu() requests
* @rcu_work: Let queue_rcu_work() invoke workqueue handler after grace period
@@ -2970,9 +2973,9 @@ static void kfree_rcu_work(struct work_struct *work)
next = head->next;
/* We tag the headless object, if so adjust offset. */
- headless = (((unsigned long) head - offset) & BIT(0));
+ headless = !!(offset & RCU_HEADLESS_KFREE);
if (headless)
- offset -= 1;
+ offset &= ~(RCU_HEADLESS_KFREE);
ptr = (void *) head - offset;
debug_rcu_head_unqueue((struct rcu_head *)ptr);
@@ -3221,7 +3224,7 @@ void kvfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
* to the original allocated memory, that has to be freed as
* well as dynamically attached wrapper/head.
*/
- func = (rcu_callback_t) (sizeof(unsigned long *) + 1);
+ func = (rcu_callback_t)(sizeof(unsigned long *) | RCU_HEADLESS_KFREE);
}
head->func = func;
--
2.26.0.rc2.310.g2932bb562d-goog
prev parent reply other threads:[~2020-03-30 2:33 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-30 2:32 [PATCH 00/18] kfree_rcu() improvements for -rcu dev Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 01/18] mm/list_lru.c: Rename kvfree_rcu() to local variant Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 02/18] rcu: Introduce kvfree_rcu() interface Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 03/18] rcu: Rename rcu_invoke_kfree_callback/rcu_kfree_callback Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 04/18] rcu: Rename __is_kfree_rcu_offset() macro Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 05/18] rcu: Rename kfree_call_rcu() to the kvfree_call_rcu() Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 06/18] mm/list_lru.c: Remove kvfree_rcu_local() function Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 07/18] rcu/tree: Simplify debug_objects handling Joel Fernandes (Google)
2020-03-30 7:00 ` kbuild test robot
2020-03-30 18:42 ` Joel Fernandes
2020-03-30 2:32 ` [PATCH 08/18] rcu/tree: Clarify emergency path comment better Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 09/18] rcu/tree: Simplify KFREE_BULK_MAX_ENTR macro Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 10/18] rcu/tree: Maintain separate array for vmalloc ptrs Joel Fernandes (Google)
2020-03-30 6:48 ` kbuild test robot
2020-03-30 9:10 ` kbuild test robot
2020-03-30 15:29 ` Uladzislau Rezki
2020-03-30 15:31 ` Matthew Wilcox
2020-03-30 15:37 ` Paul E. McKenney
2020-03-30 17:16 ` Joel Fernandes
2020-03-30 17:43 ` Uladzislau Rezki
2020-03-30 18:21 ` Paul E. McKenney
2020-03-30 2:32 ` [PATCH 11/18] rcu/tree: Introduce expedited_drain flag Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 12/18] rcu/tree: Support reclaim for head-less object Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 13/18] rcu/tiny: Move kvfree_call_rcu() out of header Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 14/18] rcu/tiny: Support reclaim for head-less object Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 15/18] rcu: Support headless variant in the kvfree_rcu() Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 16/18] rcu/tree: Remove extra next variable in kfree worker function Joel Fernandes (Google)
2020-03-30 2:32 ` [PATCH 17/18] rcu/tree: Simplify is_vmalloc_addr expression Joel Fernandes (Google)
2020-03-30 2:32 ` Joel Fernandes (Google) [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200330023248.164994-19-joel@joelfernandes.org \
--to=joel@joelfernandes.org \
--cc=akpm@linux-foundation.org \
--cc=jiangshanlai@gmail.com \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@redhat.com \
--cc=paulmck@kernel.org \
--cc=rafael.j.wysocki@intel.com \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
--cc=urezki@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).