* [PATCH net-next 0/6] net/tls: minor micro optimizations
@ 2019-10-07 4:09 Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 1/6] net: sockmap: use bitmap for copy info Jakub Kicinski
` (6 more replies)
0 siblings, 7 replies; 8+ messages in thread
From: Jakub Kicinski @ 2019-10-07 4:09 UTC (permalink / raw)
To: davem
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel, Jakub Kicinski
Hi!
This set brings a number of minor code changes from my tree which
don't have a noticeable impact on performance but seem reasonable
nonetheless.
First sk_msg_sg copy array is converted to a bitmap, zeroing that
structure takes a lot of time, hence we should try to keep it
small.
Next two conditions are marked as unlikely, GCC seemed to had
little trouble correctly reasoning about those.
Patch 4 adds parameters to tls_device_decrypted() to avoid
walking structures, as all callers already have the relevant
pointers.
Lastly two boolean members of TLS context structures are
converted to a bitfield.
Jakub Kicinski (6):
net: sockmap: use bitmap for copy info
net/tls: mark sk->err being set as unlikely
net/tls: make allocation failure unlikely
net/tls: pass context to tls_device_decrypted()
net/tls: store async_capable on a single bit
net/tls: store decrypted on a single bit
include/linux/skmsg.h | 12 ++++++++----
include/net/tls.h | 13 ++++++++-----
net/core/filter.c | 4 ++--
net/tls/tls_device.c | 12 +++++-------
net/tls/tls_sw.c | 13 +++++++------
5 files changed, 30 insertions(+), 24 deletions(-)
--
2.21.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH net-next 1/6] net: sockmap: use bitmap for copy info
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
@ 2019-10-07 4:09 ` Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 2/6] net/tls: mark sk->err being set as unlikely Jakub Kicinski
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2019-10-07 4:09 UTC (permalink / raw)
To: davem
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel, Jakub Kicinski, Dirk van der Merwe
Don't use bool array in struct sk_msg_sg, save 12 bytes.
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@netronome.com>
---
include/linux/skmsg.h | 12 ++++++++----
net/core/filter.c | 4 ++--
2 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/include/linux/skmsg.h b/include/linux/skmsg.h
index e4b3fb4bb77c..fe80d537945d 100644
--- a/include/linux/skmsg.h
+++ b/include/linux/skmsg.h
@@ -28,13 +28,14 @@ struct sk_msg_sg {
u32 end;
u32 size;
u32 copybreak;
- bool copy[MAX_MSG_FRAGS];
+ unsigned long copy;
/* The extra element is used for chaining the front and sections when
* the list becomes partitioned (e.g. end < start). The crypto APIs
* require the chaining.
*/
struct scatterlist data[MAX_MSG_FRAGS + 1];
};
+static_assert(BITS_PER_LONG >= MAX_MSG_FRAGS);
/* UAPI in filter.c depends on struct sk_msg_sg being first element. */
struct sk_msg {
@@ -227,7 +228,7 @@ static inline void sk_msg_compute_data_pointers(struct sk_msg *msg)
{
struct scatterlist *sge = sk_msg_elem(msg, msg->sg.start);
- if (msg->sg.copy[msg->sg.start]) {
+ if (test_bit(msg->sg.start, &msg->sg.copy)) {
msg->data = NULL;
msg->data_end = NULL;
} else {
@@ -246,7 +247,7 @@ static inline void sk_msg_page_add(struct sk_msg *msg, struct page *page,
sg_set_page(sge, page, len, offset);
sg_unmark_end(sge);
- msg->sg.copy[msg->sg.end] = true;
+ __set_bit(msg->sg.end, &msg->sg.copy);
msg->sg.size += len;
sk_msg_iter_next(msg, end);
}
@@ -254,7 +255,10 @@ static inline void sk_msg_page_add(struct sk_msg *msg, struct page *page,
static inline void sk_msg_sg_copy(struct sk_msg *msg, u32 i, bool copy_state)
{
do {
- msg->sg.copy[i] = copy_state;
+ if (copy_state)
+ __set_bit(i, &msg->sg.copy);
+ else
+ __clear_bit(i, &msg->sg.copy);
sk_msg_iter_var_next(i);
if (i == msg->sg.end)
break;
diff --git a/net/core/filter.c b/net/core/filter.c
index ed6563622ce3..46196e212413 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2245,7 +2245,7 @@ BPF_CALL_4(bpf_msg_pull_data, struct sk_msg *, msg, u32, start,
* account for the headroom.
*/
bytes_sg_total = start - offset + bytes;
- if (!msg->sg.copy[i] && bytes_sg_total <= len)
+ if (!test_bit(i, &msg->sg.copy) && bytes_sg_total <= len)
goto out;
/* At this point we need to linearize multiple scatterlist
@@ -2450,7 +2450,7 @@ BPF_CALL_4(bpf_msg_push_data, struct sk_msg *, msg, u32, start,
/* Place newly allocated data buffer */
sk_mem_charge(msg->sk, len);
msg->sg.size += len;
- msg->sg.copy[new] = false;
+ __clear_bit(new, &msg->sg.copy);
sg_set_page(&msg->sg.data[new], page, len + copy, 0);
if (rsge.length) {
get_page(sg_page(&rsge));
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH net-next 2/6] net/tls: mark sk->err being set as unlikely
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 1/6] net: sockmap: use bitmap for copy info Jakub Kicinski
@ 2019-10-07 4:09 ` Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 3/6] net/tls: make allocation failure unlikely Jakub Kicinski
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2019-10-07 4:09 UTC (permalink / raw)
To: davem
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel, Jakub Kicinski, Dirk van der Merwe
Tell GCC sk->err is not likely to be set.
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@netronome.com>
---
net/tls/tls_device.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index f306e4c7bf15..fcf38edc07d6 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -431,7 +431,7 @@ static int tls_push_data(struct sock *sk,
~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST))
return -ENOTSUPP;
- if (sk->sk_err)
+ if (unlikely(sk->sk_err))
return -sk->sk_err;
flags |= MSG_SENDPAGE_DECRYPTED;
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH net-next 3/6] net/tls: make allocation failure unlikely
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 1/6] net: sockmap: use bitmap for copy info Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 2/6] net/tls: mark sk->err being set as unlikely Jakub Kicinski
@ 2019-10-07 4:09 ` Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 4/6] net/tls: pass context to tls_device_decrypted() Jakub Kicinski
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2019-10-07 4:09 UTC (permalink / raw)
To: davem
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel, Jakub Kicinski, Dirk van der Merwe
Make sure GCC realizes it's unlikely that allocations will fail.
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@netronome.com>
---
net/tls/tls_device.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index fcf38edc07d6..23c19b8ff04e 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -452,9 +452,8 @@ static int tls_push_data(struct sock *sk,
max_open_record_len = TLS_MAX_PAYLOAD_SIZE +
prot->prepend_size;
do {
- rc = tls_do_allocation(sk, ctx, pfrag,
- prot->prepend_size);
- if (rc) {
+ rc = tls_do_allocation(sk, ctx, pfrag, prot->prepend_size);
+ if (unlikely(rc)) {
rc = sk_stream_wait_memory(sk, &timeo);
if (!rc)
continue;
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH net-next 4/6] net/tls: pass context to tls_device_decrypted()
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
` (2 preceding siblings ...)
2019-10-07 4:09 ` [PATCH net-next 3/6] net/tls: make allocation failure unlikely Jakub Kicinski
@ 2019-10-07 4:09 ` Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 5/6] net/tls: store async_capable on a single bit Jakub Kicinski
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2019-10-07 4:09 UTC (permalink / raw)
To: davem
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel, Jakub Kicinski, Dirk van der Merwe
Avoid unnecessary pointer chasing and calculations, callers already
have most of the state tls_device_decrypted() needs.
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@netronome.com>
---
include/net/tls.h | 7 +++++--
net/tls/tls_device.c | 5 ++---
net/tls/tls_sw.c | 2 +-
3 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/include/net/tls.h b/include/net/tls.h
index 24c37bffc961..b809f2362049 100644
--- a/include/net/tls.h
+++ b/include/net/tls.h
@@ -641,7 +641,8 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx);
void tls_device_offload_cleanup_rx(struct sock *sk);
void tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq);
void tls_offload_tx_resync_request(struct sock *sk, u32 got_seq, u32 exp_seq);
-int tls_device_decrypted(struct sock *sk, struct sk_buff *skb);
+int tls_device_decrypted(struct sock *sk, struct tls_context *tls_ctx,
+ struct sk_buff *skb, struct strp_msg *rxm);
#else
static inline void tls_device_init(void) {}
static inline void tls_device_cleanup(void) {}
@@ -664,7 +665,9 @@ static inline void tls_device_offload_cleanup_rx(struct sock *sk) {}
static inline void
tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq) {}
-static inline int tls_device_decrypted(struct sock *sk, struct sk_buff *skb)
+static inline int
+tls_device_decrypted(struct sock *sk, struct tls_context *tls_ctx,
+ struct sk_buff *skb, struct strp_msg *rxm)
{
return 0;
}
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index 23c19b8ff04e..33b267b052c0 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -846,11 +846,10 @@ static int tls_device_reencrypt(struct sock *sk, struct sk_buff *skb)
return err;
}
-int tls_device_decrypted(struct sock *sk, struct sk_buff *skb)
+int tls_device_decrypted(struct sock *sk, struct tls_context *tls_ctx,
+ struct sk_buff *skb, struct strp_msg *rxm)
{
- struct tls_context *tls_ctx = tls_get_ctx(sk);
struct tls_offload_context_rx *ctx = tls_offload_ctx_rx(tls_ctx);
- struct strp_msg *rxm = strp_msg(skb);
int is_decrypted = skb->decrypted;
int is_encrypted = !is_decrypted;
struct sk_buff *skb_iter;
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index 0b1e86f856eb..954f451dcc57 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -1495,7 +1495,7 @@ static int decrypt_skb_update(struct sock *sk, struct sk_buff *skb,
if (!ctx->decrypted) {
if (tls_ctx->rx_conf == TLS_HW) {
- err = tls_device_decrypted(sk, skb);
+ err = tls_device_decrypted(sk, tls_ctx, skb, rxm);
if (err < 0)
return err;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH net-next 5/6] net/tls: store async_capable on a single bit
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
` (3 preceding siblings ...)
2019-10-07 4:09 ` [PATCH net-next 4/6] net/tls: pass context to tls_device_decrypted() Jakub Kicinski
@ 2019-10-07 4:09 ` Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 6/6] net/tls: store decrypted " Jakub Kicinski
2019-10-07 13:58 ` [PATCH net-next 0/6] net/tls: minor micro optimizations David Miller
6 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2019-10-07 4:09 UTC (permalink / raw)
To: davem
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel, Jakub Kicinski, Simon Horman
Store async_capable on a single bit instead of a full integer
to save space.
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Simon Horman <simon.horman@netronome.com>
---
include/net/tls.h | 4 ++--
net/tls/tls_sw.c | 5 +++--
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/include/net/tls.h b/include/net/tls.h
index b809f2362049..97eae7271a67 100644
--- a/include/net/tls.h
+++ b/include/net/tls.h
@@ -136,7 +136,7 @@ struct tls_sw_context_tx {
struct list_head tx_list;
atomic_t encrypt_pending;
int async_notify;
- int async_capable;
+ u8 async_capable:1;
#define BIT_TX_SCHEDULED 0
#define BIT_TX_CLOSING 1
@@ -152,7 +152,7 @@ struct tls_sw_context_rx {
struct sk_buff *recv_pkt;
u8 control;
- int async_capable;
+ u8 async_capable:1;
bool decrypted;
atomic_t decrypt_pending;
bool async_notify;
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index 954f451dcc57..c006b587a7db 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -2391,10 +2391,11 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
tfm = crypto_aead_tfm(sw_ctx_rx->aead_recv);
if (crypto_info->version == TLS_1_3_VERSION)
- sw_ctx_rx->async_capable = false;
+ sw_ctx_rx->async_capable = 0;
else
sw_ctx_rx->async_capable =
- tfm->__crt_alg->cra_flags & CRYPTO_ALG_ASYNC;
+ !!(tfm->__crt_alg->cra_flags &
+ CRYPTO_ALG_ASYNC);
/* Set up strparser */
memset(&cb, 0, sizeof(cb));
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH net-next 6/6] net/tls: store decrypted on a single bit
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
` (4 preceding siblings ...)
2019-10-07 4:09 ` [PATCH net-next 5/6] net/tls: store async_capable on a single bit Jakub Kicinski
@ 2019-10-07 4:09 ` Jakub Kicinski
2019-10-07 13:58 ` [PATCH net-next 0/6] net/tls: minor micro optimizations David Miller
6 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2019-10-07 4:09 UTC (permalink / raw)
To: davem
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel, Jakub Kicinski, Simon Horman
Use a single bit instead of boolean to remember if packet
was already decrypted.
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Reviewed-by: Simon Horman <simon.horman@netronome.com>
---
include/net/tls.h | 2 +-
net/tls/tls_sw.c | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/net/tls.h b/include/net/tls.h
index 97eae7271a67..41265e542e71 100644
--- a/include/net/tls.h
+++ b/include/net/tls.h
@@ -153,7 +153,7 @@ struct tls_sw_context_rx {
struct sk_buff *recv_pkt;
u8 control;
u8 async_capable:1;
- bool decrypted;
+ u8 decrypted:1;
atomic_t decrypt_pending;
bool async_notify;
};
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index c006b587a7db..de7561d4cfa5 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -1523,7 +1523,7 @@ static int decrypt_skb_update(struct sock *sk, struct sk_buff *skb,
rxm->offset += prot->prepend_size;
rxm->full_len -= prot->overhead_size;
tls_advance_record_sn(sk, prot, &tls_ctx->rx);
- ctx->decrypted = true;
+ ctx->decrypted = 1;
ctx->saved_data_ready(sk);
} else {
*zc = false;
@@ -1933,7 +1933,7 @@ ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos,
tls_err_abort(sk, EBADMSG);
goto splice_read_end;
}
- ctx->decrypted = true;
+ ctx->decrypted = 1;
}
rxm = strp_msg(skb);
@@ -2034,7 +2034,7 @@ static void tls_queue(struct strparser *strp, struct sk_buff *skb)
struct tls_context *tls_ctx = tls_get_ctx(strp->sk);
struct tls_sw_context_rx *ctx = tls_sw_ctx_rx(tls_ctx);
- ctx->decrypted = false;
+ ctx->decrypted = 0;
ctx->recv_pkt = skb;
strp_pause(strp);
--
2.21.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH net-next 0/6] net/tls: minor micro optimizations
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
` (5 preceding siblings ...)
2019-10-07 4:09 ` [PATCH net-next 6/6] net/tls: store decrypted " Jakub Kicinski
@ 2019-10-07 13:58 ` David Miller
6 siblings, 0 replies; 8+ messages in thread
From: David Miller @ 2019-10-07 13:58 UTC (permalink / raw)
To: jakub.kicinski
Cc: netdev, oss-drivers, davejwatson, borisp, aviadye,
john.fastabend, daniel
From: Jakub Kicinski <jakub.kicinski@netronome.com>
Date: Sun, 6 Oct 2019 21:09:26 -0700
> This set brings a number of minor code changes from my tree which
> don't have a noticeable impact on performance but seem reasonable
> nonetheless.
>
> First sk_msg_sg copy array is converted to a bitmap, zeroing that
> structure takes a lot of time, hence we should try to keep it
> small.
>
> Next two conditions are marked as unlikely, GCC seemed to had
> little trouble correctly reasoning about those.
>
> Patch 4 adds parameters to tls_device_decrypted() to avoid
> walking structures, as all callers already have the relevant
> pointers.
>
> Lastly two boolean members of TLS context structures are
> converted to a bitfield.
All looks good, series applied.
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2019-10-07 13:58 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-07 4:09 [PATCH net-next 0/6] net/tls: minor micro optimizations Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 1/6] net: sockmap: use bitmap for copy info Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 2/6] net/tls: mark sk->err being set as unlikely Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 3/6] net/tls: make allocation failure unlikely Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 4/6] net/tls: pass context to tls_device_decrypted() Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 5/6] net/tls: store async_capable on a single bit Jakub Kicinski
2019-10-07 4:09 ` [PATCH net-next 6/6] net/tls: store decrypted " Jakub Kicinski
2019-10-07 13:58 ` [PATCH net-next 0/6] net/tls: minor micro optimizations David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).