[PATCH bpf 0/2] bpf: net: Fixes in sk_user_data of reuseport_array

[PATCH bpf 0/2] bpf: net: Fixes in sk_user_data of reuseport_array

[Martin KaFai Lau]

This set fixes two issues on sk_user_data when a sk is added to
a reuseport_array.

The first patch is to avoid the sk_user_data being copied
to a cloned sk.  The second patch avoids doing bpf_sk_reuseport_detach()
on sk_user_data that is not managed by reuseport_array.

Since the changes are mostly related to bpf reuseport_array, so it is
currently tagged as bpf fixes.

Martin KaFai Lau (2):
  bpf: net: Avoid copying sk_user_data of reuseport_array during
    sk_clone
  bpf: net: Avoid incorrect bpf_sk_reuseport_detach call

 include/net/sock.h           |  3 ++-
 kernel/bpf/reuseport_array.c | 14 ++++++++++----
 2 files changed, 12 insertions(+), 5 deletions(-)

-- 
2.24.1

[PATCH bpf 1/2] bpf: net: Avoid copying sk_user_data of reuseport_array during sk_clone

[Martin KaFai Lau]

It makes little sense for copying sk_user_data of reuseport_array during
sk_clone_lock().  This patch reuses the SK_USER_DATA_NOCOPY bit introduced in
commit f1ff5ce2cd5e ("net, sk_msg: Clear sk_user_data pointer on clone if tagged").
It is used to mark the sk_user_data is not supposed to be copied to its clone.

Although the cloned sk's sk_user_data will not be used/freed in
bpf_sk_reuseport_detach(), this change can still allow the cloned
sk's sk_user_data to be used by some other means.

Freeing the reuseport_array's sk_user_data does not require a rcu grace
period.  Thus, the existing rcu_assign_sk_user_data_nocopy() is not
used.

Fixes: 5dc4c4b7d4e8 ("bpf: Introduce BPF_MAP_TYPE_REUSEPORT_SOCKARRAY")
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
---
 kernel/bpf/reuseport_array.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c
index 21cde24386db..a95bc8d7e812 100644
--- a/kernel/bpf/reuseport_array.c
+++ b/kernel/bpf/reuseport_array.c
@@ -20,11 +20,14 @@ static struct reuseport_array *reuseport_array(struct bpf_map *map)
 /* The caller must hold the reuseport_lock */
 void bpf_sk_reuseport_detach(struct sock *sk)
 {
-	struct sock __rcu **socks;
+	uintptr_t sk_user_data;
 
 	write_lock_bh(&sk->sk_callback_lock);
-	socks = sk->sk_user_data;
-	if (socks) {
+	sk_user_data = (uintptr_t)sk->sk_user_data;
+	if (sk_user_data) {
+		struct sock __rcu **socks;
+
+		socks = (void *)(sk_user_data & SK_USER_DATA_PTRMASK);
 		WRITE_ONCE(sk->sk_user_data, NULL);
 		/*
 		 * Do not move this NULL assignment outside of
@@ -252,6 +255,7 @@ int bpf_fd_reuseport_array_update_elem(struct bpf_map *map, void *key,
 	struct sock *free_osk = NULL, *osk, *nsk;
 	struct sock_reuseport *reuse;
 	u32 index = *(u32 *)key;
+	uintptr_t sk_user_data;
 	struct socket *socket;
 	int err, fd;
 
@@ -305,7 +309,8 @@ int bpf_fd_reuseport_array_update_elem(struct bpf_map *map, void *key,
 	if (err)
 		goto put_file_unlock;
 
-	WRITE_ONCE(nsk->sk_user_data, &array->ptrs[index]);
+	sk_user_data = (uintptr_t)&array->ptrs[index] | SK_USER_DATA_NOCOPY;
+	WRITE_ONCE(nsk->sk_user_data, (void *)sk_user_data);
 	rcu_assign_pointer(array->ptrs[index], nsk);
 	free_osk = osk;
 	err = 0;
-- 
2.24.1

[PATCH bpf 2/2] bpf: net: Avoid incorrect bpf_sk_reuseport_detach call

[Martin KaFai Lau]

bpf_sk_reuseport_detach is currently called when sk->sk_user_data
is not NULL.  It is incorrect because sk->sk_user_data may not be
managed by the bpf's reuseport_array.  It has been report in [1] that,
the bpf_sk_reuseport_detach() which is called from udp_lib_unhash() has
corrupted the sk_user_data managed by l2tp.

This patch solves it by using another bit (defined as SK_USER_DATA_BPF)
of the sk_user_data pointer value.  It marks that a sk_user_data is
managed/owned by BPF.

The patch depends on a PTRMASK introduced in
commit f1ff5ce2cd5e ("net, sk_msg: Clear sk_user_data pointer on clone if tagged").

[ Note: sk->sk_user_data is used by bpf's reuseport_array only when a sk is
  added to the bpf's reuseport_array.
  i.e. doing setsockopt(SO_REUSEPORT) and having "sk->sk_reuseport == 1"
  alone will not stop sk->sk_user_data being used by other means. ]

[1]: https://lore.kernel.org/netdev/20200706121259.GA20199@katalix.com/

Reported-by: James Chapman <jchapman@katalix.com>
Cc: James Chapman <jchapman@katalix.com>
Fixes: 5dc4c4b7d4e8 ("bpf: Introduce BPF_MAP_TYPE_REUSEPORT_SOCKARRAY")
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
---
 include/net/sock.h           | 3 ++-
 kernel/bpf/reuseport_array.c | 5 +++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index 3428619faae4..9fe42c890706 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -533,7 +533,8 @@ enum sk_pacing {
  * be copied.
  */
 #define SK_USER_DATA_NOCOPY	1UL
-#define SK_USER_DATA_PTRMASK	~(SK_USER_DATA_NOCOPY)
+#define SK_USER_DATA_BPF	2UL	/* Managed by BPF */
+#define SK_USER_DATA_PTRMASK	~3UL
 
 /**
  * sk_user_data_is_nocopy - Test if sk_user_data pointer must not be copied
diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c
index a95bc8d7e812..cae9d505e04a 100644
--- a/kernel/bpf/reuseport_array.c
+++ b/kernel/bpf/reuseport_array.c
@@ -24,7 +24,7 @@ void bpf_sk_reuseport_detach(struct sock *sk)
 
 	write_lock_bh(&sk->sk_callback_lock);
 	sk_user_data = (uintptr_t)sk->sk_user_data;
-	if (sk_user_data) {
+	if (sk_user_data & SK_USER_DATA_BPF) {
 		struct sock __rcu **socks;
 
 		socks = (void *)(sk_user_data & SK_USER_DATA_PTRMASK);
@@ -309,7 +309,8 @@ int bpf_fd_reuseport_array_update_elem(struct bpf_map *map, void *key,
 	if (err)
 		goto put_file_unlock;
 
-	sk_user_data = (uintptr_t)&array->ptrs[index] | SK_USER_DATA_NOCOPY;
+	sk_user_data = (uintptr_t)&array->ptrs[index] | SK_USER_DATA_NOCOPY |
+		SK_USER_DATA_BPF;
 	WRITE_ONCE(nsk->sk_user_data, (void *)sk_user_data);
 	rcu_assign_pointer(array->ptrs[index], nsk);
 	free_osk = osk;
-- 
2.24.1

Re: [PATCH bpf 2/2] bpf: net: Avoid incorrect bpf_sk_reuseport_detach call

[Andrii Nakryiko]

On Tue, Jul 7, 2020 at 6:46 PM Martin KaFai Lau <kafai@fb.com> wrote:
>
> bpf_sk_reuseport_detach is currently called when sk->sk_user_data
> is not NULL.  It is incorrect because sk->sk_user_data may not be
> managed by the bpf's reuseport_array.  It has been report in [1] that,
> the bpf_sk_reuseport_detach() which is called from udp_lib_unhash() has
> corrupted the sk_user_data managed by l2tp.
>
> This patch solves it by using another bit (defined as SK_USER_DATA_BPF)
> of the sk_user_data pointer value.  It marks that a sk_user_data is
> managed/owned by BPF.
>
> The patch depends on a PTRMASK introduced in
> commit f1ff5ce2cd5e ("net, sk_msg: Clear sk_user_data pointer on clone if tagged").
>
> [ Note: sk->sk_user_data is used by bpf's reuseport_array only when a sk is
>   added to the bpf's reuseport_array.
>   i.e. doing setsockopt(SO_REUSEPORT) and having "sk->sk_reuseport == 1"
>   alone will not stop sk->sk_user_data being used by other means. ]
>
> [1]: https://lore.kernel.org/netdev/20200706121259.GA20199@katalix.com/
>
> Reported-by: James Chapman <jchapman@katalix.com>
> Cc: James Chapman <jchapman@katalix.com>
> Fixes: 5dc4c4b7d4e8 ("bpf: Introduce BPF_MAP_TYPE_REUSEPORT_SOCKARRAY")
> Signed-off-by: Martin KaFai Lau <kafai@fb.com>
> ---
>  include/net/sock.h           | 3 ++-
>  kernel/bpf/reuseport_array.c | 5 +++--
>  2 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 3428619faae4..9fe42c890706 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -533,7 +533,8 @@ enum sk_pacing {
>   * be copied.
>   */
>  #define SK_USER_DATA_NOCOPY    1UL
> -#define SK_USER_DATA_PTRMASK   ~(SK_USER_DATA_NOCOPY)
> +#define SK_USER_DATA_BPF       2UL     /* Managed by BPF */
> +#define SK_USER_DATA_PTRMASK   ~3UL

nit: ~3UL looks like a random constant, while

~(SK_USER_DATA_NOCOPY | SK_USER_DATA_BPF)

would clearly indicate what's going on. Original PTRMASK definition
had this logical connection with NOCOPY bit, I think it's worth it to
preserve that.

>
>  /**
>   * sk_user_data_is_nocopy - Test if sk_user_data pointer must not be copied
> diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c
> index a95bc8d7e812..cae9d505e04a 100644
> --- a/kernel/bpf/reuseport_array.c
> +++ b/kernel/bpf/reuseport_array.c
> @@ -24,7 +24,7 @@ void bpf_sk_reuseport_detach(struct sock *sk)
>
>         write_lock_bh(&sk->sk_callback_lock);
>         sk_user_data = (uintptr_t)sk->sk_user_data;
> -       if (sk_user_data) {
> +       if (sk_user_data & SK_USER_DATA_BPF) {
>                 struct sock __rcu **socks;
>
>                 socks = (void *)(sk_user_data & SK_USER_DATA_PTRMASK);
> @@ -309,7 +309,8 @@ int bpf_fd_reuseport_array_update_elem(struct bpf_map *map, void *key,
>         if (err)
>                 goto put_file_unlock;
>
> -       sk_user_data = (uintptr_t)&array->ptrs[index] | SK_USER_DATA_NOCOPY;
> +       sk_user_data = (uintptr_t)&array->ptrs[index] | SK_USER_DATA_NOCOPY |
> +               SK_USER_DATA_BPF;
>         WRITE_ONCE(nsk->sk_user_data, (void *)sk_user_data);
>         rcu_assign_pointer(array->ptrs[index], nsk);
>         free_osk = osk;
> --
> 2.24.1
>