All of lore.kernel.org
 help / color / mirror / Atom feed
From: Eric Dumazet <eric.dumazet@gmail.com>
To: "David S . Miller" <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>
Cc: netdev <netdev@vger.kernel.org>,
	Eric Dumazet <edumazet@google.com>,
	Eric Dumazet <eric.dumazet@gmail.com>,
	Soheil Hassas Yeganeh <soheil@google.com>,
	Neal Cardwell <ncardwell@google.com>,
	Arjun Roy <arjunroy@google.com>
Subject: [PATCH net-next 12/20] tcp: add RETPOLINE mitigation to sk_backlog_rcv
Date: Mon, 15 Nov 2021 11:02:41 -0800	[thread overview]
Message-ID: <20211115190249.3936899-13-eric.dumazet@gmail.com> (raw)
In-Reply-To: <20211115190249.3936899-1-eric.dumazet@gmail.com>

From: Eric Dumazet <edumazet@google.com>

Use INDIRECT_CALL_INET() to avoid an indirect call
when/if CONFIG_RETPOLINE=y

Signed-off-by: Eric Dumazet <edumazet@google.com>
---
 include/net/sock.h  | 8 +++++++-
 net/core/sock.c     | 5 ++++-
 net/ipv6/tcp_ipv6.c | 5 +++--
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index cb97c448472aa5af3055916df844cbe422578190..2d40fe4c7718ee702bf7e5a847ceff6f8f2f5b7d 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -1018,12 +1018,18 @@ static inline __must_check int sk_add_backlog(struct sock *sk, struct sk_buff *s
 
 int __sk_backlog_rcv(struct sock *sk, struct sk_buff *skb);
 
+INDIRECT_CALLABLE_DECLARE(int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb));
+INDIRECT_CALLABLE_DECLARE(int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb));
+
 static inline int sk_backlog_rcv(struct sock *sk, struct sk_buff *skb)
 {
 	if (sk_memalloc_socks() && skb_pfmemalloc(skb))
 		return __sk_backlog_rcv(sk, skb);
 
-	return sk->sk_backlog_rcv(sk, skb);
+	return INDIRECT_CALL_INET(sk->sk_backlog_rcv,
+				  tcp_v6_do_rcv,
+				  tcp_v4_do_rcv,
+				  sk, skb);
 }
 
 static inline void sk_incoming_cpu_update(struct sock *sk)
diff --git a/net/core/sock.c b/net/core/sock.c
index 99738e14224c44e5aa4b88857620fb162e9c265f..c57d9883f62c75f522b7f6bc68451aaf8429dc83 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -327,7 +327,10 @@ int __sk_backlog_rcv(struct sock *sk, struct sk_buff *skb)
 	BUG_ON(!sock_flag(sk, SOCK_MEMALLOC));
 
 	noreclaim_flag = memalloc_noreclaim_save();
-	ret = sk->sk_backlog_rcv(sk, skb);
+	ret = INDIRECT_CALL_INET(sk->sk_backlog_rcv,
+				 tcp_v6_do_rcv,
+				 tcp_v4_do_rcv,
+				 sk, skb);
 	memalloc_noreclaim_restore(noreclaim_flag);
 
 	return ret;
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 1f1a89f096de9f77ab1bd2d871eb90a3f12e91e0..f41f14b701233dd2d0f5ad464a623a5ba9774763 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -72,7 +72,7 @@ static void	tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb);
 static void	tcp_v6_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
 				      struct request_sock *req);
 
-static int	tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);
+INDIRECT_CALLABLE_SCOPE int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);
 
 static const struct inet_connection_sock_af_ops ipv6_mapped;
 const struct inet_connection_sock_af_ops ipv6_specific;
@@ -1466,7 +1466,8 @@ INDIRECT_CALLABLE_DECLARE(struct dst_entry *ipv4_dst_check(struct dst_entry *,
  * This is because we cannot sleep with the original spinlock
  * held.
  */
-static int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)
+INDIRECT_CALLABLE_SCOPE
+int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)
 {
 	struct ipv6_pinfo *np = tcp_inet6_sk(sk);
 	struct sk_buff *opt_skb = NULL;
-- 
2.34.0.rc1.387.gb447b232ab-goog


  parent reply	other threads:[~2021-11-15 20:09 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-15 19:02 [PATCH net-next 00/20] tcp: optimizations for linux-5.17 Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 01/20] tcp: minor optimization in tcp_add_backlog() Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 02/20] tcp: remove dead code in __tcp_v6_send_check() Eric Dumazet
2021-11-16  2:48   ` David Ahern
2021-11-16  2:57     ` Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 03/20] tcp: small optimization in tcp_v6_send_check() Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 04/20] net: use sk_is_tcp() in more places Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 05/20] net: remove sk_route_forced_caps Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 06/20] net: remove sk_route_nocaps Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 07/20] ipv6: shrink struct ipcm6_cookie Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 08/20] net: shrink struct sock by 8 bytes Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 09/20] net: forward_alloc_get depends on CONFIG_MPTCP Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 10/20] net: cache align tcp_memory_allocated, tcp_sockets_allocated Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 11/20] tcp: small optimization in tcp recvmsg() Eric Dumazet
2021-11-15 19:02 ` Eric Dumazet [this message]
2021-11-15 19:02 ` [PATCH net-next 13/20] tcp: annotate data-races on tp->segs_in and tp->data_segs_in Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 14/20] tcp: annotate races around tp->urg_data Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 15/20] tcp: tp->urg_data is unlikely to be set Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 16/20] tcp: avoid indirect calls to sock_rfree Eric Dumazet
2021-11-15 19:16   ` Arjun Roy
2021-11-15 19:02 ` [PATCH net-next 17/20] tcp: defer skb freeing after socket lock is released Eric Dumazet
2021-11-16 14:27   ` Jakub Kicinski
2021-11-16 15:05     ` Eric Dumazet
2021-11-16 15:20       ` Jakub Kicinski
2021-11-16 15:22       ` Eric Dumazet
2021-11-16 15:27         ` Jakub Kicinski
2021-11-16 16:46           ` Eric Dumazet
2021-11-16 18:18             ` Jakub Kicinski
2021-11-16 20:45             ` David Ahern
2021-11-16 21:35               ` Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 18/20] tcp: check local var (timeo) before socket fields in one test Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 19/20] tcp: do not call tcp_cleanup_rbuf() if we have a backlog Eric Dumazet
2021-11-15 19:02 ` [PATCH net-next 20/20] net: move early demux fields close to sk_refcnt Eric Dumazet
2021-11-15 20:37 ` [PATCH net-next 00/20] tcp: optimizations for linux-5.17 Soheil Hassas Yeganeh
2021-11-15 21:40   ` Paolo Abeni
2021-11-15 21:47     ` Eric Dumazet
2021-11-16  2:06       ` Eric Dumazet
2021-11-16  4:01         ` Arjun Roy
2021-11-16 13:32         ` David Miller
2021-11-16 15:06           ` Eric Dumazet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211115190249.3936899-13-eric.dumazet@gmail.com \
    --to=eric.dumazet@gmail.com \
    --cc=arjunroy@google.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=kuba@kernel.org \
    --cc=ncardwell@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=soheil@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.