From: "Björn Töpel" <bjorn.topel@gmail.com>
To: netdev@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net
Cc: "Björn Töpel" <bjorn.topel@intel.com>,
bpf@vger.kernel.org, magnus.karlsson@intel.com,
jonathan.lemon@gmail.com
Subject: [PATCH bpf-next] xsk: update rings for load-acquire/store-release semantics
Date: Mon, 20 Jan 2020 10:21:48 +0100 [thread overview]
Message-ID: <20200120092149.13775-1-bjorn.topel@gmail.com> (raw)
From: Björn Töpel <bjorn.topel@intel.com>
Currently, the AF_XDP rings uses fences for the kernel-side
produce/consume functions. By updating rings for
load-acquire/store-release semantics, the full barrier (smp_mb()) on
the consumer side can be replaced.
Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
---
net/xdp/xsk_queue.h | 31 ++++++++++++++-----------------
1 file changed, 14 insertions(+), 17 deletions(-)
diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h
index bec2af11853a..2fff80576ee1 100644
--- a/net/xdp/xsk_queue.h
+++ b/net/xdp/xsk_queue.h
@@ -39,19 +39,18 @@ struct xsk_queue {
u64 invalid_descs;
};
-/* The structure of the shared state of the rings are the same as the
- * ring buffer in kernel/events/ring_buffer.c. For the Rx and completion
- * ring, the kernel is the producer and user space is the consumer. For
- * the Tx and fill rings, the kernel is the consumer and user space is
- * the producer.
+/* The structure of the shared state of the rings are a simple
+ * circular buffer, as outlined in
+ * Documentation/core-api/circular-buffers.rst. For the Rx and
+ * completion ring, the kernel is the producer and user space is the
+ * consumer. For the Tx and fill rings, the kernel is the consumer and
+ * user space is the producer.
*
* producer consumer
*
- * if (LOAD ->consumer) { LOAD ->producer
- * (A) smp_rmb() (C)
+ * if (LOAD ->consumer) { (A) LOAD.acq ->producer (C)
* STORE $data LOAD $data
- * smp_wmb() (B) smp_mb() (D)
- * STORE ->producer STORE ->consumer
+ * STORE.rel ->producer (B) STORE.rel ->consumer (D)
* }
*
* (A) pairs with (D), and (B) pairs with (C).
@@ -220,15 +219,14 @@ static inline bool xskq_cons_read_desc(struct xsk_queue *q,
static inline void __xskq_cons_release(struct xsk_queue *q)
{
- smp_mb(); /* D, matches A */
- WRITE_ONCE(q->ring->consumer, q->cached_cons);
+ /* D, matches A */
+ smp_store_release(&q->ring->consumer, q->cached_cons);
}
static inline void __xskq_cons_peek(struct xsk_queue *q)
{
- /* Refresh the local pointer */
- q->cached_prod = READ_ONCE(q->ring->producer);
- smp_rmb(); /* C, matches B */
+ /* C, matches B */
+ q->cached_prod = smp_load_acquire(&q->ring->producer);
}
static inline void xskq_cons_get_entries(struct xsk_queue *q)
@@ -340,9 +338,8 @@ static inline int xskq_prod_reserve_desc(struct xsk_queue *q,
static inline void __xskq_prod_submit(struct xsk_queue *q, u32 idx)
{
- smp_wmb(); /* B, matches C */
-
- WRITE_ONCE(q->ring->producer, idx);
+ /* B, matches C */
+ smp_store_release(&q->ring->producer, idx);
}
static inline void xskq_prod_submit(struct xsk_queue *q)
--
2.20.1
next reply other threads:[~2020-01-20 9:22 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-20 9:21 Björn Töpel [this message]
2020-01-20 23:27 ` [PATCH bpf-next] xsk: update rings for load-acquire/store-release semantics Daniel Borkmann
2020-01-21 11:50 ` Björn Töpel
2020-03-16 18:44 ` Will Deacon
2020-03-17 19:14 ` Björn Töpel
2020-03-17 21:03 ` Will Deacon
2020-01-21 1:24 ` John Fastabend
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200120092149.13775-1-bjorn.topel@gmail.com \
--to=bjorn.topel@gmail.com \
--cc=ast@kernel.org \
--cc=bjorn.topel@intel.com \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=jonathan.lemon@gmail.com \
--cc=magnus.karlsson@intel.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).