From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Tadeusz Struk <tadeusz.struk@linaro.org>
Cc: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
Yonghong Song <yhs@fb.com>,
John Fastabend <john.fastabend@gmail.com>,
KP Singh <kpsingh@kernel.org>,
Networking <netdev@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
linux- stable <stable@vger.kernel.org>,
open list <linux-kernel@vger.kernel.org>,
syzbot+f264bffdfbd5614f3bb2@syzkaller.appspotmail.com
Subject: Re: [PATCH v3] bpf: Fix KASAN use-after-free Read in compute_effective_progs
Date: Mon, 16 May 2022 16:16:12 -0700 [thread overview]
Message-ID: <CAEf4BzY-p13huoqo6N7LJRVVj8rcjPeP3Cp=KDX4N2x9BkC9Zw@mail.gmail.com> (raw)
In-Reply-To: <20220513190821.431762-1-tadeusz.struk@linaro.org>
On Fri, May 13, 2022 at 12:08 PM Tadeusz Struk <tadeusz.struk@linaro.org> wrote:
>
> Syzbot found a Use After Free bug in compute_effective_progs().
> The reproducer creates a number of BPF links, and causes a fault
> injected alloc to fail, while calling bpf_link_detach on them.
> Link detach triggers the link to be freed by bpf_link_free(),
> which calls __cgroup_bpf_detach() and update_effective_progs().
> If the memory allocation in this function fails, the function restores
> the pointer to the bpf_cgroup_link on the cgroup list, but the memory
> gets freed just after it returns. After this, every subsequent call to
> update_effective_progs() causes this already deallocated pointer to be
> dereferenced in prog_list_length(), and triggers KASAN UAF error.
>
> To fix this issue don't preserve the pointer to the prog or link in the
> list, but remove it and replace it with a dummy prog without shrinking
> the table. The subsequent call to __cgroup_bpf_detach() or
> __cgroup_bpf_detach() will correct it.
>
> Cc: "Alexei Starovoitov" <ast@kernel.org>
> Cc: "Daniel Borkmann" <daniel@iogearbox.net>
> Cc: "Andrii Nakryiko" <andrii@kernel.org>
> Cc: "Martin KaFai Lau" <kafai@fb.com>
> Cc: "Song Liu" <songliubraving@fb.com>
> Cc: "Yonghong Song" <yhs@fb.com>
> Cc: "John Fastabend" <john.fastabend@gmail.com>
> Cc: "KP Singh" <kpsingh@kernel.org>
> Cc: <netdev@vger.kernel.org>
> Cc: <bpf@vger.kernel.org>
> Cc: <stable@vger.kernel.org>
> Cc: <linux-kernel@vger.kernel.org>
>
> Link: https://syzkaller.appspot.com/bug?id=8ebf179a95c2a2670f7cf1ba62429ec044369db4
> Fixes: af6eea57437a ("bpf: Implement bpf_link-based cgroup BPF program attachment")
> Reported-by: <syzbot+f264bffdfbd5614f3bb2@syzkaller.appspotmail.com>
> Signed-off-by: Tadeusz Struk <tadeusz.struk@linaro.org>
> ---
> v2: Add a fall back path that removes a prog from the effective progs
> table in case detach fails to allocate memory in compute_effective_progs().
>
> v3: Implement the fallback in a separate function purge_effective_progs
> ---
> kernel/bpf/cgroup.c | 64 +++++++++++++++++++++++++++++++++++++++------
> 1 file changed, 56 insertions(+), 8 deletions(-)
>
> diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
> index 128028efda64..9d3af4d6c055 100644
> --- a/kernel/bpf/cgroup.c
> +++ b/kernel/bpf/cgroup.c
> @@ -681,6 +681,57 @@ static struct bpf_prog_list *find_detach_entry(struct list_head *progs,
> return ERR_PTR(-ENOENT);
> }
>
> +/**
> + * purge_effective_progs() - After compute_effective_progs fails to alloc new
> + * cgrp->bpf.inactive table we can recover by
> + * recomputing the array in place.
> + *
> + * @cgrp: The cgroup which descendants to traverse
> + * @link: A link to detach
> + * @atype: Type of detach operation
> + */
> +static void purge_effective_progs(struct cgroup *cgrp, struct bpf_prog *prog,
> + enum cgroup_bpf_attach_type atype)
> +{
> + struct cgroup_subsys_state *css;
> + struct bpf_prog_array_item *item;
> + struct bpf_prog *tmp;
> + struct bpf_prog_array *array;
> + int index = 0, index_purge = -1;
> +
> + if (!prog)
> + return;
> +
> + /* recompute effective prog array in place */
> + css_for_each_descendant_pre(css, &cgrp->self) {
> + struct cgroup *desc = container_of(css, struct cgroup, self);
> +
> + array = desc->bpf.effective[atype];
../kernel/bpf/cgroup.c:748:23: warning: incorrect type in assignment
(different address spaces)
../kernel/bpf/cgroup.c:748:23: expected struct bpf_prog_array *array
../kernel/bpf/cgroup.c:748:23: got struct bpf_prog_array [noderef] __rcu *
you need rcu_dereference here? but also see suggestions below to avoid
iterating effective directly (it's ambiguous to search by prog only)
> + item = &array->items[0];
> +
> + /* Find the index of the prog to purge */
> + while ((tmp = READ_ONCE(item->prog))) {
> + if (tmp == prog) {
I think comparing just prog isn't always correct, as the same program
can be in effective array multiple times if attached through bpf_link.
Looking at replace_effective_prog() I think we can do a very similar
(and tested) approach:
1. restore original pl state in __cgroup_bpf_detach (so we can find it
by comparing pl->prog == prog && pl->link == link)
2. use replace_effective_prog's approach to find position of pl in
effective array (using this nested for loop over cgroup parents and
list_for_each_entry inside)
3. then instead of replacing one prog with another do
bpf_prog_array_delete_safe_at ?
I'd feel more comfortable using the same tested overall approach of
replace_effective_prog.
> + index_purge = index;
> + break;
> + }
> + item++;
> + index++;
> + }
> +
> + /* Check if we found what's needed for removing the prog */
> + if (index_purge == -1 || index_purge == index - 1)
> + continue;
the search shouldn't fail, should it?
> +
> + /* Remove the program from the array */
> + WARN_ONCE(bpf_prog_array_delete_safe_at(array, index_purge),
> + "Failed to purge a prog from array at index %d", index_purge);
> +
> + index = 0;
> + index_purge = -1;
> + }
> +}
> +
> /**
> * __cgroup_bpf_detach() - Detach the program or link from a cgroup, and
> * propagate the change to descendants
> @@ -723,8 +774,11 @@ static int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
> pl->link = NULL;
>
> err = update_effective_progs(cgrp, atype);
> - if (err)
> - goto cleanup;
> + if (err) {
> + struct bpf_prog *prog_purge = prog ? prog : link->link.prog;
> +
so here we shouldn't forget link, instead pass both link and prog (one
of them will have to be NULL) into purge_effective_progs
> + purge_effective_progs(cgrp, prog_purge, atype);
> + }
>
> /* now can actually delete it from this cgroup list */
> list_del(&pl->node);
> @@ -736,12 +790,6 @@ static int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
> bpf_prog_put(old_prog);
> static_branch_dec(&cgroup_bpf_enabled_key[atype]);
> return 0;
> -
> -cleanup:
> - /* restore back prog or link */
> - pl->prog = old_prog;
> - pl->link = link;
> - return err;
> }
>
> static int cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
> --
> 2.36.1
>
next prev parent reply other threads:[~2022-05-16 23:16 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-05 17:03 [PATCH] bpf: Fix KASAN use-after-free Read in compute_effective_progs Tadeusz Struk
2022-04-12 14:44 ` Tadeusz Struk
2022-04-13 4:34 ` Andrii Nakryiko
2022-04-13 17:28 ` Tadeusz Struk
2022-04-13 19:07 ` Andrii Nakryiko
2022-04-13 19:27 ` Tadeusz Struk
2022-04-13 19:49 ` Andrii Nakryiko
2022-04-15 14:13 ` [PATCH v2] " Tadeusz Struk
2022-04-20 17:07 ` Andrii Nakryiko
2022-05-13 18:38 ` Tadeusz Struk
2022-05-13 19:08 ` [PATCH v3] " Tadeusz Struk
2022-05-16 23:16 ` Andrii Nakryiko [this message]
2022-05-16 23:35 ` Tadeusz Struk
2022-05-16 23:45 ` Andrii Nakryiko
2022-05-17 18:04 ` [PATCH v4] " Tadeusz Struk
2022-05-23 21:36 ` Tadeusz Struk
2022-05-23 22:47 ` Andrii Nakryiko
2022-05-23 22:58 ` Tadeusz Struk
2022-06-02 14:37 ` Tadeusz Struk
2022-06-02 16:11 ` Andrii Nakryiko
2022-06-02 16:25 ` Tadeusz Struk
2022-06-02 16:51 ` Tadeusz Struk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAEf4BzY-p13huoqo6N7LJRVVj8rcjPeP3Cp=KDX4N2x9BkC9Zw@mail.gmail.com' \
--to=andrii.nakryiko@gmail.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=john.fastabend@gmail.com \
--cc=kafai@fb.com \
--cc=kpsingh@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=songliubraving@fb.com \
--cc=stable@vger.kernel.org \
--cc=syzbot+f264bffdfbd5614f3bb2@syzkaller.appspotmail.com \
--cc=tadeusz.struk@linaro.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).