From: Ming Lei <tom.leiming@gmail.com>
To: linux-kernel@vger.kernel.org, Alexei Starovoitov <ast@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>,
netdev@vger.kernel.org, Ming Lei <tom.leiming@gmail.com>
Subject: [PATCH 2/6] hlist: prepare for supporting bit spinlock
Date: Tue, 15 Dec 2015 19:21:00 +0800 [thread overview]
Message-ID: <1450178464-27721-3-git-send-email-tom.leiming@gmail.com> (raw)
In-Reply-To: <1450178464-27721-1-git-send-email-tom.leiming@gmail.com>
hlist_head is often used for implementing bucket header of
hash table, and one lock is often needed for adding/deleting
node in the bucket list. It can consume lots of memory if
per-bucket spinlock is used, so this patch trys to use the 1st
bit of hlist_head->first as bit lock for this purpose.
bit spinlock isn't efficient as spin lock, but the contention
shouldn't be very high for operating per-bucket hlist, so bit
spinlock should be OK.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
include/linux/rculist.h | 55 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
diff --git a/include/linux/rculist.h b/include/linux/rculist.h
index 14ec165..9fc394a 100644
--- a/include/linux/rculist.h
+++ b/include/linux/rculist.h
@@ -435,6 +435,61 @@ static inline void hlist_replace_rcu(struct hlist_node *old,
#define hlist_next_rcu(node) (*((struct hlist_node __rcu **)(&(node)->next)))
#define hlist_pprev_rcu(node) (*((struct hlist_node __rcu **)((node)->pprev)))
+/*
+ * This bit of hlist_head->first can be used as per-bucket
+ * bit spin_lock in hash table use case, and the lock bit
+ * should always be kept when hlist_node is added, deleted
+ * and other update operations.
+ *
+ * When lock bit is used, only the _lock version of hlist
+ * helpers can be used for operating the hlist.
+ */
+#define HLIST_LOCK_BIT 0
+#define HLIST_LOCK_MASK (1UL << HLIST_LOCK_BIT)
+#define hlist_get_first(h) \
+ (struct hlist_node *)((unsigned long)(h)->first & ~HLIST_LOCK_MASK)
+#define hlist_get_lock(h) ((unsigned long)(h)->first & HLIST_LOCK_MASK)
+#define hlist_make_1st_node(h, n) \
+ (struct hlist_node *)((unsigned long)(n) | hlist_get_lock(h))
+
+static inline struct hlist_head *hlist_get_head_lock(
+ struct hlist_head *head, struct hlist_head *new_head)
+{
+ new_head->first = hlist_get_first(head);
+ return new_head;
+}
+
+static inline void __hlist_del_lock(struct hlist_node *n)
+{
+ struct hlist_node *next = n->next;
+ struct hlist_node **pprev = n->pprev;
+
+ WRITE_ONCE(*pprev, (unsigned long)next |
+ ((unsigned long)*pprev & HLIST_LOCK_MASK));
+ if (next)
+ next->pprev = pprev;
+}
+
+/* The bit lock should be held before calling this helper */
+static inline void hlist_del_rcu_lock(struct hlist_node *n)
+{
+ __hlist_del_lock(n);
+ n->pprev = LIST_POISON2;
+}
+
+/* The bit lock should be held before calling this helper */
+static inline void hlist_add_head_rcu_lock(struct hlist_node *n,
+ struct hlist_head *h)
+{
+ struct hlist_node *first = hlist_get_first(h);
+
+ n->next = first;
+ n->pprev = &h->first;
+ rcu_assign_pointer(hlist_first_rcu(h), hlist_make_1st_node(h, n));
+ if (first)
+ first->pprev = &n->next;
+}
+
/**
* hlist_add_head_rcu
* @n: the element to add to the hash list.
--
1.9.1
next prev parent reply other threads:[~2015-12-15 11:21 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-12-15 11:20 [PATCH 0/6] bpf: hash: optimization Ming Lei
2015-12-15 11:20 ` [PATCH 1/6] bpf: hash: use atomic count Ming Lei
2015-12-15 11:21 ` Ming Lei [this message]
2015-12-15 11:21 ` [PATCH 3/6] bpf: hash: move select_bucket() out of htab's spinlock Ming Lei
2015-12-15 11:21 ` [PATCH 4/6] bpf: hash: convert per-hashtable lock into per-bucket bit spinlock Ming Lei
2015-12-15 22:51 ` Alexei Starovoitov
2015-12-16 2:57 ` Ming Lei
[not found] ` <CAHbLzkpW_seTrs+WgFqsriH5uhG4LMY_jiYC0iRQ-LAdJFiUjw@mail.gmail.com>
2015-12-16 6:58 ` Ming Lei
2015-12-18 6:20 ` Alexei Starovoitov
2015-12-26 8:58 ` Ming Lei
2015-12-15 11:21 ` [PATCH 5/6] bpf: hash: avoid to call kmalloc() in eBPF prog Ming Lei
2015-12-15 23:10 ` Alexei Starovoitov
2015-12-15 23:42 ` Daniel Borkmann
2015-12-16 7:13 ` Ming Lei
2015-12-15 23:21 ` Daniel Borkmann
2015-12-15 23:35 ` Daniel Borkmann
2015-12-16 0:12 ` Daniel Borkmann
2015-12-15 11:21 ` [PATCH 6/6] bpf: hash: reorganize 'struct htab_elem' Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1450178464-27721-3-git-send-email-tom.leiming@gmail.com \
--to=tom.leiming@gmail.com \
--cc=ast@kernel.org \
--cc=davem@davemloft.net \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).