Re: [PATCH net] net: fix use-after-free when UDP GRO with shared fraglist

From: Willem de Bruijn <willemdebruijn.kernel@gmail.com>
To: Dongseok Yi <dseok.yi@samsung.com>
Cc: "David S. Miller" <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>,
	Miaohe Lin <linmiaohe@huawei.com>,
	Willem de Bruijn <willemb@google.com>,
	Paolo Abeni <pabeni@redhat.com>, Florian Westphal <fw@strlen.de>,
	Al Viro <viro@zeniv.linux.org.uk>,
	Guillaume Nault <gnault@redhat.com>,
	Yunsheng Lin <linyunsheng@huawei.com>,
	Steffen Klassert <steffen.klassert@secunet.com>,
	Yadu Kishore <kyk.segfault@gmail.com>,
	Marco Elver <elver@google.com>,
	Network Development <netdev@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	namkyu78.kim@samsung.com
Subject: Re: [PATCH net] net: fix use-after-free when UDP GRO with shared fraglist
Date: Mon, 4 Jan 2021 16:03:39 -0500	[thread overview]
Message-ID: <CAF=yD-+bDdYg7X+WpP14w3fbv+JewySpdCbjdwWXB-syCwQ9uQ@mail.gmail.com> (raw)
In-Reply-To: <1609750005-115609-1-git-send-email-dseok.yi@samsung.com>

On Mon, Jan 4, 2021 at 4:00 AM Dongseok Yi <dseok.yi@samsung.com> wrote:
>
> skbs in frag_list could be shared by pskb_expand_head() from BPF.

Can you elaborate on the BPF connection?

> While tcpdump, sk_receive_queue of PF_PACKET has the original frag_list.
> But the same frag_list is queued to PF_INET (or PF_INET6) as the fraglist
> chain made by skb_segment_list().
>
> If the new skb (not frag_list) is queued to one of the sk_receive_queue,
> multiple ptypes can see this. The skb could be released by ptypes and
> it causes use-after-free.

If I understand correctly, a udp-gro-list skb makes it up the receive
path with one or more active packet sockets.

The packet socket will call skb_clone after accepting the filter. This
replaces the head_skb, but shares the skb_shinfo and thus frag_list.

udp_rcv_segment later converts the udp-gro-list skb to a list of
regular packets to pass these one-by-one to udp_queue_rcv_one_skb.
Now all the frags are fully fledged packets, with headers pushed
before the payload. This does not change their refcount anymore than
the skb_clone in pf_packet did. This should be 1.

Eventually udp_recvmsg will call skb_consume_udp on each packet.

The packet socket eventually also frees its cloned head_skb, which triggers

  kfree_skb_list(shinfo->frag_list)
    kfree_skb
      skb_unref
        refcount_dec_and_test(&skb->users)

>
> [ 4443.426215] ------------[ cut here ]------------
> [ 4443.426222] refcount_t: underflow; use-after-free.
> [ 4443.426291] WARNING: CPU: 7 PID: 28161 at lib/refcount.c:190
> refcount_dec_and_test_checked+0xa4/0xc8
> [ 4443.426726] pstate: 60400005 (nZCv daif +PAN -UAO)
> [ 4443.426732] pc : refcount_dec_and_test_checked+0xa4/0xc8
> [ 4443.426737] lr : refcount_dec_and_test_checked+0xa0/0xc8
> [ 4443.426808] Call trace:
> [ 4443.426813]  refcount_dec_and_test_checked+0xa4/0xc8
> [ 4443.426823]  skb_release_data+0x144/0x264
> [ 4443.426828]  kfree_skb+0x58/0xc4
> [ 4443.426832]  skb_queue_purge+0x64/0x9c
> [ 4443.426844]  packet_set_ring+0x5f0/0x820
> [ 4443.426849]  packet_setsockopt+0x5a4/0xcd0
> [ 4443.426853]  __sys_setsockopt+0x188/0x278
> [ 4443.426858]  __arm64_sys_setsockopt+0x28/0x38
> [ 4443.426869]  el0_svc_common+0xf0/0x1d0
> [ 4443.426873]  el0_svc_handler+0x74/0x98
> [ 4443.426880]  el0_svc+0x8/0xc
>
> Fixes: 3a1296a38d0c (net: Support GRO/GSO fraglist chaining.)
> Signed-off-by: Dongseok Yi <dseok.yi@samsung.com>
> ---
>  net/core/skbuff.c | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
> index f62cae3..1dcbda8 100644
> --- a/net/core/skbuff.c
> +++ b/net/core/skbuff.c
> @@ -3655,7 +3655,8 @@ struct sk_buff *skb_segment_list(struct sk_buff *skb,
>         unsigned int delta_truesize = 0;
>         unsigned int delta_len = 0;
>         struct sk_buff *tail = NULL;
> -       struct sk_buff *nskb;
> +       struct sk_buff *nskb, *tmp;
> +       int err;
>
>         skb_push(skb, -skb_network_offset(skb) + offset);
>
> @@ -3665,11 +3666,28 @@ struct sk_buff *skb_segment_list(struct sk_buff *skb,
>                 nskb = list_skb;
>                 list_skb = list_skb->next;
>
> +               err = 0;
> +               if (skb_shared(nskb)) {

I must be missing something still. This does not square with my
understanding that the two sockets are operating on clones, with each
frag_list skb having skb->users == 1.

Unless the packet socket patch previously also triggered an
skb_unclone/pskb_expand_head, as that call skb_clone_fraglist, which
calls skb_get on each frag_list skb.

> +                       tmp = skb_clone(nskb, GFP_ATOMIC);
> +                       if (tmp) {
> +                               kfree_skb(nskb);
> +                               nskb = tmp;
> +                               err = skb_unclone(nskb, GFP_ATOMIC);
> +                       } else {
> +                               err = -ENOMEM;
> +                       }
> +               }
> +
>                 if (!tail)
>                         skb->next = nskb;
>                 else
>                         tail->next = nskb;
>
> +               if (unlikely(err)) {
> +                       nskb->next = list_skb;
> +                       goto err_linearize;
> +               }
> +
>                 tail = nskb;
>
>                 delta_len += nskb->len;
> --
> 2.7.4
>