* [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection
@ 2021-03-14 11:10 Alexander Lobakin
2021-03-14 11:11 ` [PATCH v3 net-next 1/6] flow_dissector: constify bpf_flow_dissector's data pointers Alexander Lobakin
` (5 more replies)
0 siblings, 6 replies; 11+ messages in thread
From: Alexander Lobakin @ 2021-03-14 11:10 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Jonathan Lemon, Alexander Lobakin, Eric Dumazet,
Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki,
Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich,
Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha,
Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev,
linux-kernel, bpf
This little number makes all of the flow dissection functions take
raw input data pointer as const (1-5) and shuffles the branches in
__skb_header_pointer() according to their hit probability.
The result is +20 Mbps per flow/core with one Flow Dissector pass
per packet. This affects RPS (with software hashing), drivers that
use eth_get_headlen() on their Rx path and so on.
From v2 [1]:
- reword some commit messages as a potential fix for NIPA;
- no functional changes.
From v1 [0]:
- rebase on top of the latest net-next. This was super-weird, but
I double-checked that the series applies with no conflicts, and
then on Patchwork it didn't;
- no other changes.
[0] https://lore.kernel.org/netdev/20210312194538.337504-1-alobakin@pm.me
[1] https://lore.kernel.org/netdev/20210313113645.5949-1-alobakin@pm.me
Alexander Lobakin (6):
flow_dissector: constify bpf_flow_dissector's data pointers
skbuff: make __skb_header_pointer()'s data argument const
flow_dissector: constify raw input data argument
linux/etherdevice.h: misc trailing whitespace cleanup
ethernet: constify eth_get_headlen()'s data argument
skbuff: micro-optimize {,__}skb_header_pointer()
include/linux/etherdevice.h | 4 ++--
include/linux/skbuff.h | 26 +++++++++++------------
include/net/flow_dissector.h | 6 +++---
net/core/flow_dissector.c | 41 +++++++++++++++++++-----------------
net/ethernet/eth.c | 2 +-
5 files changed, 40 insertions(+), 39 deletions(-)
--
2.30.2
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v3 net-next 1/6] flow_dissector: constify bpf_flow_dissector's data pointers 2021-03-14 11:10 [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection Alexander Lobakin @ 2021-03-14 11:11 ` Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 2/6] skbuff: make __skb_header_pointer()'s data argument const Alexander Lobakin ` (4 subsequent siblings) 5 siblings, 0 replies; 11+ messages in thread From: Alexander Lobakin @ 2021-03-14 11:11 UTC (permalink / raw) To: David S. Miller, Jakub Kicinski Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Alexander Lobakin, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf BPF Flow dissection programs are read-only and don't touch input buffers. Mark 'data' and 'data_end' in struct bpf_flow_dissector as const in preparation for global input constifying. Signed-off-by: Alexander Lobakin <alobakin@pm.me> --- include/net/flow_dissector.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/net/flow_dissector.h b/include/net/flow_dissector.h index cc10b10dc3a1..bf00e71816ed 100644 --- a/include/net/flow_dissector.h +++ b/include/net/flow_dissector.h @@ -368,8 +368,8 @@ static inline void *skb_flow_dissector_target(struct flow_dissector *flow_dissec struct bpf_flow_dissector { struct bpf_flow_keys *flow_keys; const struct sk_buff *skb; - void *data; - void *data_end; + const void *data; + const void *data_end; }; static inline void -- 2.30.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v3 net-next 2/6] skbuff: make __skb_header_pointer()'s data argument const 2021-03-14 11:10 [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 1/6] flow_dissector: constify bpf_flow_dissector's data pointers Alexander Lobakin @ 2021-03-14 11:11 ` Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 3/6] flow_dissector: constify raw input data argument Alexander Lobakin ` (3 subsequent siblings) 5 siblings, 0 replies; 11+ messages in thread From: Alexander Lobakin @ 2021-03-14 11:11 UTC (permalink / raw) To: David S. Miller, Jakub Kicinski Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Alexander Lobakin, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf The function never modifies the input buffer, so 'data' argument can be marked as const. This implies one harmless cast-away. Signed-off-by: Alexander Lobakin <alobakin@pm.me> --- include/linux/skbuff.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 483e89348f78..d6ea3dc3eddb 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3678,11 +3678,11 @@ __wsum skb_checksum(const struct sk_buff *skb, int offset, int len, __wsum csum); static inline void * __must_check -__skb_header_pointer(const struct sk_buff *skb, int offset, - int len, void *data, int hlen, void *buffer) +__skb_header_pointer(const struct sk_buff *skb, int offset, int len, + const void *data, int hlen, void *buffer) { if (hlen - offset >= len) - return data + offset; + return (void *)data + offset; if (!skb || skb_copy_bits(skb, offset, buffer, len) < 0) -- 2.30.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v3 net-next 3/6] flow_dissector: constify raw input data argument 2021-03-14 11:10 [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 1/6] flow_dissector: constify bpf_flow_dissector's data pointers Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 2/6] skbuff: make __skb_header_pointer()'s data argument const Alexander Lobakin @ 2021-03-14 11:11 ` Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup Alexander Lobakin ` (2 subsequent siblings) 5 siblings, 0 replies; 11+ messages in thread From: Alexander Lobakin @ 2021-03-14 11:11 UTC (permalink / raw) To: David S. Miller, Jakub Kicinski Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Alexander Lobakin, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf Flow Dissector code never modifies the input buffer, neither skb nor raw data. Make 'data' argument const for all of the Flow dissector's functions. Signed-off-by: Alexander Lobakin <alobakin@pm.me> --- include/linux/skbuff.h | 15 ++++++------- include/net/flow_dissector.h | 2 +- net/core/flow_dissector.c | 41 +++++++++++++++++++----------------- 3 files changed, 30 insertions(+), 28 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index d6ea3dc3eddb..46c61e127e9f 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1292,10 +1292,10 @@ __skb_set_sw_hash(struct sk_buff *skb, __u32 hash, bool is_l4) void __skb_get_hash(struct sk_buff *skb); u32 __skb_get_hash_symmetric(const struct sk_buff *skb); u32 skb_get_poff(const struct sk_buff *skb); -u32 __skb_get_poff(const struct sk_buff *skb, void *data, +u32 __skb_get_poff(const struct sk_buff *skb, const void *data, const struct flow_keys_basic *keys, int hlen); __be32 __skb_flow_get_ports(const struct sk_buff *skb, int thoff, u8 ip_proto, - void *data, int hlen_proto); + const void *data, int hlen_proto); static inline __be32 skb_flow_get_ports(const struct sk_buff *skb, int thoff, u8 ip_proto) @@ -1314,9 +1314,8 @@ bool bpf_flow_dissect(struct bpf_prog *prog, struct bpf_flow_dissector *ctx, bool __skb_flow_dissect(const struct net *net, const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, - void *data, __be16 proto, int nhoff, int hlen, - unsigned int flags); + void *target_container, const void *data, + __be16 proto, int nhoff, int hlen, unsigned int flags); static inline bool skb_flow_dissect(const struct sk_buff *skb, struct flow_dissector *flow_dissector, @@ -1338,9 +1337,9 @@ static inline bool skb_flow_dissect_flow_keys(const struct sk_buff *skb, static inline bool skb_flow_dissect_flow_keys_basic(const struct net *net, const struct sk_buff *skb, - struct flow_keys_basic *flow, void *data, - __be16 proto, int nhoff, int hlen, - unsigned int flags) + struct flow_keys_basic *flow, + const void *data, __be16 proto, + int nhoff, int hlen, unsigned int flags) { memset(flow, 0, sizeof(*flow)); return __skb_flow_dissect(net, skb, &flow_keys_basic_dissector, flow, diff --git a/include/net/flow_dissector.h b/include/net/flow_dissector.h index bf00e71816ed..ffd386ea0dbb 100644 --- a/include/net/flow_dissector.h +++ b/include/net/flow_dissector.h @@ -350,7 +350,7 @@ static inline bool flow_keys_have_l4(const struct flow_keys *keys) u32 flow_hash_from_keys(struct flow_keys *keys); void skb_flow_get_icmp_tci(const struct sk_buff *skb, struct flow_dissector_key_icmp *key_icmp, - void *data, int thoff, int hlen); + const void *data, int thoff, int hlen); static inline bool dissector_uses_key(const struct flow_dissector *flow_dissector, enum flow_dissector_key_id key_id) diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c index 2ef2224b3bff..2ed380d096ce 100644 --- a/net/core/flow_dissector.c +++ b/net/core/flow_dissector.c @@ -114,7 +114,7 @@ int flow_dissector_bpf_prog_attach_check(struct net *net, * is the protocol port offset returned from proto_ports_offset */ __be32 __skb_flow_get_ports(const struct sk_buff *skb, int thoff, u8 ip_proto, - void *data, int hlen) + const void *data, int hlen) { int poff = proto_ports_offset(ip_proto); @@ -161,7 +161,7 @@ static bool icmp_has_id(u8 type) */ void skb_flow_get_icmp_tci(const struct sk_buff *skb, struct flow_dissector_key_icmp *key_icmp, - void *data, int thoff, int hlen) + const void *data, int thoff, int hlen) { struct icmphdr *ih, _ih; @@ -187,8 +187,8 @@ EXPORT_SYMBOL(skb_flow_get_icmp_tci); */ static void __skb_flow_dissect_icmp(const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, - void *data, int thoff, int hlen) + void *target_container, const void *data, + int thoff, int hlen) { struct flow_dissector_key_icmp *key_icmp; @@ -409,8 +409,8 @@ EXPORT_SYMBOL(skb_flow_dissect_hash); static enum flow_dissect_ret __skb_flow_dissect_mpls(const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, void *data, int nhoff, int hlen, - int lse_index, bool *entropy_label) + void *target_container, const void *data, int nhoff, + int hlen, int lse_index, bool *entropy_label) { struct mpls_label *hdr, _hdr; u32 entry, label, bos; @@ -467,7 +467,8 @@ __skb_flow_dissect_mpls(const struct sk_buff *skb, static enum flow_dissect_ret __skb_flow_dissect_arp(const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, void *data, int nhoff, int hlen) + void *target_container, const void *data, + int nhoff, int hlen) { struct flow_dissector_key_arp *key_arp; struct { @@ -523,7 +524,7 @@ static enum flow_dissect_ret __skb_flow_dissect_gre(const struct sk_buff *skb, struct flow_dissector_key_control *key_control, struct flow_dissector *flow_dissector, - void *target_container, void *data, + void *target_container, const void *data, __be16 *p_proto, int *p_nhoff, int *p_hlen, unsigned int flags) { @@ -663,8 +664,8 @@ __skb_flow_dissect_gre(const struct sk_buff *skb, static enum flow_dissect_ret __skb_flow_dissect_batadv(const struct sk_buff *skb, struct flow_dissector_key_control *key_control, - void *data, __be16 *p_proto, int *p_nhoff, int hlen, - unsigned int flags) + const void *data, __be16 *p_proto, int *p_nhoff, + int hlen, unsigned int flags) { struct { struct batadv_unicast_packet batadv_unicast; @@ -695,7 +696,8 @@ __skb_flow_dissect_batadv(const struct sk_buff *skb, static void __skb_flow_dissect_tcp(const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, void *data, int thoff, int hlen) + void *target_container, const void *data, + int thoff, int hlen) { struct flow_dissector_key_tcp *key_tcp; struct tcphdr *th, _th; @@ -719,8 +721,8 @@ __skb_flow_dissect_tcp(const struct sk_buff *skb, static void __skb_flow_dissect_ports(const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, void *data, int nhoff, - u8 ip_proto, int hlen) + void *target_container, const void *data, + int nhoff, u8 ip_proto, int hlen) { enum flow_dissector_key_id dissector_ports = FLOW_DISSECTOR_KEY_MAX; struct flow_dissector_key_ports *key_ports; @@ -744,7 +746,8 @@ __skb_flow_dissect_ports(const struct sk_buff *skb, static void __skb_flow_dissect_ipv4(const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, void *data, const struct iphdr *iph) + void *target_container, const void *data, + const struct iphdr *iph) { struct flow_dissector_key_ip *key_ip; @@ -761,7 +764,8 @@ __skb_flow_dissect_ipv4(const struct sk_buff *skb, static void __skb_flow_dissect_ipv6(const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, void *data, const struct ipv6hdr *iph) + void *target_container, const void *data, + const struct ipv6hdr *iph) { struct flow_dissector_key_ip *key_ip; @@ -908,9 +912,8 @@ bool bpf_flow_dissect(struct bpf_prog *prog, struct bpf_flow_dissector *ctx, bool __skb_flow_dissect(const struct net *net, const struct sk_buff *skb, struct flow_dissector *flow_dissector, - void *target_container, - void *data, __be16 proto, int nhoff, int hlen, - unsigned int flags) + void *target_container, const void *data, + __be16 proto, int nhoff, int hlen, unsigned int flags) { struct flow_dissector_key_control *key_control; struct flow_dissector_key_basic *key_basic; @@ -1642,7 +1645,7 @@ __u32 skb_get_hash_perturb(const struct sk_buff *skb, } EXPORT_SYMBOL(skb_get_hash_perturb); -u32 __skb_get_poff(const struct sk_buff *skb, void *data, +u32 __skb_get_poff(const struct sk_buff *skb, const void *data, const struct flow_keys_basic *keys, int hlen) { u32 poff = keys->control.thoff; -- 2.30.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup 2021-03-14 11:10 [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection Alexander Lobakin ` (2 preceding siblings ...) 2021-03-14 11:11 ` [PATCH v3 net-next 3/6] flow_dissector: constify raw input data argument Alexander Lobakin @ 2021-03-14 11:11 ` Alexander Lobakin 2021-03-14 21:04 ` Vladimir Oltean 2021-03-14 11:11 ` [PATCH v3 net-next 5/6] ethernet: constify eth_get_headlen()'s data argument Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 6/6] skbuff: micro-optimize {,__}skb_header_pointer() Alexander Lobakin 5 siblings, 1 reply; 11+ messages in thread From: Alexander Lobakin @ 2021-03-14 11:11 UTC (permalink / raw) To: David S. Miller, Jakub Kicinski Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Alexander Lobakin, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf Caught by the text editor. Fix it separately from the actual changes. Signed-off-by: Alexander Lobakin <alobakin@pm.me> --- include/linux/etherdevice.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h index 2e5debc0373c..bcb2f81baafb 100644 --- a/include/linux/etherdevice.h +++ b/include/linux/etherdevice.h @@ -11,7 +11,7 @@ * Authors: Ross Biro * Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG> * - * Relocated to include/linux where it belongs by Alan Cox + * Relocated to include/linux where it belongs by Alan Cox * <gw4pts@gw4pts.ampr.org> */ #ifndef _LINUX_ETHERDEVICE_H -- 2.30.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup 2021-03-14 11:11 ` [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup Alexander Lobakin @ 2021-03-14 21:04 ` Vladimir Oltean 2021-03-15 9:38 ` Alexander Lobakin 0 siblings, 1 reply; 11+ messages in thread From: Vladimir Oltean @ 2021-03-14 21:04 UTC (permalink / raw) To: Alexander Lobakin Cc: David S. Miller, Jakub Kicinski, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf On Sun, Mar 14, 2021 at 11:11:32AM +0000, Alexander Lobakin wrote: > Caught by the text editor. Fix it separately from the actual changes. > > Signed-off-by: Alexander Lobakin <alobakin@pm.me> > --- > include/linux/etherdevice.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h > index 2e5debc0373c..bcb2f81baafb 100644 > --- a/include/linux/etherdevice.h > +++ b/include/linux/etherdevice.h > @@ -11,7 +11,7 @@ > * Authors: Ross Biro > * Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG> > * > - * Relocated to include/linux where it belongs by Alan Cox > + * Relocated to include/linux where it belongs by Alan Cox > * <gw4pts@gw4pts.ampr.org> > */ > #ifndef _LINUX_ETHERDEVICE_H > -- > 2.30.2 > > Your mailer did something weird here, it trimmed the trailing whitespace from the "-" line. The patch doesn't apply. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup 2021-03-14 21:04 ` Vladimir Oltean @ 2021-03-15 9:38 ` Alexander Lobakin 2021-03-15 19:00 ` Jakub Kicinski 0 siblings, 1 reply; 11+ messages in thread From: Alexander Lobakin @ 2021-03-15 9:38 UTC (permalink / raw) To: Vladimir Oltean Cc: Alexander Lobakin, David S. Miller, Jakub Kicinski, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf From: Vladimir Oltean <olteanv@gmail.com> Date: Sun, 14 Mar 2021 23:04:53 +0200 > On Sun, Mar 14, 2021 at 11:11:32AM +0000, Alexander Lobakin wrote: > > Caught by the text editor. Fix it separately from the actual changes. > > > > Signed-off-by: Alexander Lobakin <alobakin@pm.me> > > --- > > include/linux/etherdevice.h | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h > > index 2e5debc0373c..bcb2f81baafb 100644 > > --- a/include/linux/etherdevice.h > > +++ b/include/linux/etherdevice.h > > @@ -11,7 +11,7 @@ > > * Authors: Ross Biro > > * Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG> > > * > > - * Relocated to include/linux where it belongs by Alan Cox > > + * Relocated to include/linux where it belongs by Alan Cox > > * <gw4pts@gw4pts.ampr.org> > > */ > > #ifndef _LINUX_ETHERDEVICE_H > > -- > > 2.30.2 > > > > > > Your mailer did something weird here, it trimmed the trailing whitespace > from the "-" line. The patch doesn't apply. It's git-send-email + ProtonMail Bridge... Seems like that's the reason why only this series of mine was failing Patchwork everytime. Much thanks for finding this out! Al ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup 2021-03-15 9:38 ` Alexander Lobakin @ 2021-03-15 19:00 ` Jakub Kicinski 2021-03-17 11:02 ` Alexander Lobakin 0 siblings, 1 reply; 11+ messages in thread From: Jakub Kicinski @ 2021-03-15 19:00 UTC (permalink / raw) To: Alexander Lobakin Cc: Vladimir Oltean, David S. Miller, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf On Mon, 15 Mar 2021 09:38:57 +0000 Alexander Lobakin wrote: > From: Vladimir Oltean <olteanv@gmail.com> > Date: Sun, 14 Mar 2021 23:04:53 +0200 > > > On Sun, Mar 14, 2021 at 11:11:32AM +0000, Alexander Lobakin wrote: > > > Caught by the text editor. Fix it separately from the actual changes. > > > > > > Signed-off-by: Alexander Lobakin <alobakin@pm.me> > > > --- > > > include/linux/etherdevice.h | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h > > > index 2e5debc0373c..bcb2f81baafb 100644 > > > --- a/include/linux/etherdevice.h > > > +++ b/include/linux/etherdevice.h > > > @@ -11,7 +11,7 @@ > > > * Authors: Ross Biro > > > * Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG> > > > * > > > - * Relocated to include/linux where it belongs by Alan Cox > > > + * Relocated to include/linux where it belongs by Alan Cox > > > * <gw4pts@gw4pts.ampr.org> > > > */ > > > #ifndef _LINUX_ETHERDEVICE_H > > > -- > > > 2.30.2 > > > > > > > > > > Your mailer did something weird here, it trimmed the trailing whitespace > > from the "-" line. The patch doesn't apply. > > It's git-send-email + ProtonMail Bridge... Seems like that's the > reason why only this series of mine was failing Patchwork > everytime. Yup. Sorry for the lack of logs in NIPA, you can run git pw series apply $id and look at the output there. That's what I do, anyway, to double check that the patch doesn't apply so the logs are of limited use. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup 2021-03-15 19:00 ` Jakub Kicinski @ 2021-03-17 11:02 ` Alexander Lobakin 0 siblings, 0 replies; 11+ messages in thread From: Alexander Lobakin @ 2021-03-17 11:02 UTC (permalink / raw) To: Jakub Kicinski Cc: Alexander Lobakin, Vladimir Oltean, David S. Miller, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf From: Jakub Kicinski <kuba@kernel.org> Date: Mon, 15 Mar 2021 12:00:39 -0700 > On Mon, 15 Mar 2021 09:38:57 +0000 Alexander Lobakin wrote: > > From: Vladimir Oltean <olteanv@gmail.com> > > Date: Sun, 14 Mar 2021 23:04:53 +0200 > > > > > On Sun, Mar 14, 2021 at 11:11:32AM +0000, Alexander Lobakin wrote: > > > > Caught by the text editor. Fix it separately from the actual changes. > > > > > > > > Signed-off-by: Alexander Lobakin <alobakin@pm.me> > > > > --- > > > > include/linux/etherdevice.h | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h > > > > index 2e5debc0373c..bcb2f81baafb 100644 > > > > --- a/include/linux/etherdevice.h > > > > +++ b/include/linux/etherdevice.h > > > > @@ -11,7 +11,7 @@ > > > > * Authors: Ross Biro > > > > * Fred N. van Kempen, <waltje@uWalt.NL.Mugnet.ORG> > > > > * > > > > - * Relocated to include/linux where it belongs by Alan Cox > > > > + * Relocated to include/linux where it belongs by Alan Cox > > > > * <gw4pts@gw4pts.ampr.org> > > > > */ > > > > #ifndef _LINUX_ETHERDEVICE_H > > > > -- > > > > 2.30.2 > > > > > > > > > > > > > > Your mailer did something weird here, it trimmed the trailing whitespace > > > from the "-" line. The patch doesn't apply. > > > > It's git-send-email + ProtonMail Bridge... Seems like that's the > > reason why only this series of mine was failing Patchwork > > everytime. > > Yup. Sorry for the lack of logs in NIPA, you can run > > git pw series apply $id > > and look at the output there. That's what I do, anyway, to double check > that the patch doesn't apply so the logs are of limited use. Got it, thanks! Al ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v3 net-next 5/6] ethernet: constify eth_get_headlen()'s data argument 2021-03-14 11:10 [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection Alexander Lobakin ` (3 preceding siblings ...) 2021-03-14 11:11 ` [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup Alexander Lobakin @ 2021-03-14 11:11 ` Alexander Lobakin 2021-03-14 11:11 ` [PATCH v3 net-next 6/6] skbuff: micro-optimize {,__}skb_header_pointer() Alexander Lobakin 5 siblings, 0 replies; 11+ messages in thread From: Alexander Lobakin @ 2021-03-14 11:11 UTC (permalink / raw) To: David S. Miller, Jakub Kicinski Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Alexander Lobakin, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf It's used only for flow dissection, which now takes constant data pointers. Signed-off-by: Alexander Lobakin <alobakin@pm.me> --- include/linux/etherdevice.h | 2 +- net/ethernet/eth.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h index bcb2f81baafb..330345b1be54 100644 --- a/include/linux/etherdevice.h +++ b/include/linux/etherdevice.h @@ -29,7 +29,7 @@ struct device; int eth_platform_get_mac_address(struct device *dev, u8 *mac_addr); unsigned char *arch_get_platform_mac_address(void); int nvmem_get_mac_address(struct device *dev, void *addrbuf); -u32 eth_get_headlen(const struct net_device *dev, void *data, unsigned int len); +u32 eth_get_headlen(const struct net_device *dev, const void *data, u32 len); __be16 eth_type_trans(struct sk_buff *skb, struct net_device *dev); extern const struct header_ops eth_header_ops; diff --git a/net/ethernet/eth.c b/net/ethernet/eth.c index 4106373180c6..e01cf766d2c5 100644 --- a/net/ethernet/eth.c +++ b/net/ethernet/eth.c @@ -122,7 +122,7 @@ EXPORT_SYMBOL(eth_header); * Make a best effort attempt to pull the length for all of the headers for * a given frame in a linear buffer. */ -u32 eth_get_headlen(const struct net_device *dev, void *data, unsigned int len) +u32 eth_get_headlen(const struct net_device *dev, const void *data, u32 len) { const unsigned int flags = FLOW_DISSECTOR_F_PARSE_1ST_FRAG; const struct ethhdr *eth = (const struct ethhdr *)data; -- 2.30.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v3 net-next 6/6] skbuff: micro-optimize {,__}skb_header_pointer() 2021-03-14 11:10 [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection Alexander Lobakin ` (4 preceding siblings ...) 2021-03-14 11:11 ` [PATCH v3 net-next 5/6] ethernet: constify eth_get_headlen()'s data argument Alexander Lobakin @ 2021-03-14 11:11 ` Alexander Lobakin 5 siblings, 0 replies; 11+ messages in thread From: Alexander Lobakin @ 2021-03-14 11:11 UTC (permalink / raw) To: David S. Miller, Jakub Kicinski Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, KP Singh, Jonathan Lemon, Alexander Lobakin, Eric Dumazet, Willem de Bruijn, Kevin Hao, Pablo Neira Ayuso, Jakub Sitnicki, Marco Elver, Dexuan Cui, Vladimir Oltean, Ariel Levkovich, Wang Qing, Davide Caratti, Guillaume Nault, Eran Ben Elisha, Mauro Carvalho Chehab, Kirill Tkhai, Bartosz Golaszewski, netdev, linux-kernel, bpf {,__}skb_header_pointer() helpers exist mainly for preventing accesses-beyond-end of the linear data. In the vast majorify of cases, they bail out on the first condition. All code going after is mostly a fallback. Mark the most common branch as 'likely' one to move it in-line. Also, skb_copy_bits() can return negative values only when the input arguments are invalid, e.g. offset is greater than skb->len. It can be safely marked as 'unlikely' branch, assuming that hotpath code provides sane input to not fail here. These two bump the throughput with a single Flow Dissector pass on every packet (e.g. with RPS or driver that uses eth_get_headlen()) on 20 Mbps per flow/core. Signed-off-by: Alexander Lobakin <alobakin@pm.me> --- include/linux/skbuff.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 46c61e127e9f..ecc029674ae4 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3680,11 +3680,10 @@ static inline void * __must_check __skb_header_pointer(const struct sk_buff *skb, int offset, int len, const void *data, int hlen, void *buffer) { - if (hlen - offset >= len) + if (likely(hlen - offset >= len)) return (void *)data + offset; - if (!skb || - skb_copy_bits(skb, offset, buffer, len) < 0) + if (!skb || unlikely(skb_copy_bits(skb, offset, buffer, len) < 0)) return NULL; return buffer; -- 2.30.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-03-17 11:03 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-14 11:10 [PATCH v3 net-next 0/6] skbuff: micro-optimize flow dissection Alexander Lobakin
2021-03-14 11:11 ` [PATCH v3 net-next 1/6] flow_dissector: constify bpf_flow_dissector's data pointers Alexander Lobakin
2021-03-14 11:11 ` [PATCH v3 net-next 2/6] skbuff: make __skb_header_pointer()'s data argument const Alexander Lobakin
2021-03-14 11:11 ` [PATCH v3 net-next 3/6] flow_dissector: constify raw input data argument Alexander Lobakin
2021-03-14 11:11 ` [PATCH v3 net-next 4/6] linux/etherdevice.h: misc trailing whitespace cleanup Alexander Lobakin
2021-03-14 21:04 ` Vladimir Oltean
2021-03-15 9:38 ` Alexander Lobakin
2021-03-15 19:00 ` Jakub Kicinski
2021-03-17 11:02 ` Alexander Lobakin
2021-03-14 11:11 ` [PATCH v3 net-next 5/6] ethernet: constify eth_get_headlen()'s data argument Alexander Lobakin
2021-03-14 11:11 ` [PATCH v3 net-next 6/6] skbuff: micro-optimize {,__}skb_header_pointer() Alexander Lobakin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).