From: Alexei Starovoitov <email@example.com>
To: Daniel Borkmann <firstname.lastname@example.org>
Cc: bpf <email@example.com>,
Network Development <firstname.lastname@example.org>,
Tejun Heo <email@example.com>,
"David S. Miller" <firstname.lastname@example.org>,
Martynas Pumputis <email@example.com>,
Andrii Nakryiko <firstname.lastname@example.org>,
Stanislav Fomichev <email@example.com>
Subject: Re: [PATCH bpf v4 1/3] bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode
Date: Mon, 13 Sep 2021 19:01:24 -0700 [thread overview]
Message-ID: <CAADnVQKHcpmUp8eDzmJ2ktdtYg8ChnqK44W0_MrAQ95Y9VXVqQ@mail.gmail.com> (raw)
On Mon, Sep 13, 2021 at 4:08 PM Daniel Borkmann <firstname.lastname@example.org> wrote:
> Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
> Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
> embedded per-socket cgroup information into sock->sk_cgrp_data and in order
> to save 8 bytes in struct sock made both mutually exclusive, that is, when
> cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
> falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
> The assumption made was "there is no reason to mix the two and this is in line
> with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
> However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
> this assumption no longer holds, and the possibility of the v1/v2 mixed mode
> with the v2 root fallback being hit becomes a real security issue.
> Many of the cgroup v2 BPF programs are also used for policy enforcement, just
> to pick _one_ example, that is, to programmatically deny socket related system
> calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
> a policy bypass for the affected Pods.
> In production environments, we have recently seen this case due to various
> circumstances: i) a different 3rd party agent and/or ii) a container runtime
> such as  in the user's environment configuring legacy cgroup v1 net_cls
> tags, which triggered implicitly mentioned root fallback. Another case is
> Kubernetes projects like kind  which create Kubernetes nodes in a container
> and also add cgroup namespaces to the mix, meaning programs which are attached
> to the cgroup v2 root of the cgroup namespace get attached to a non-root
> cgroup v2 path from init namespace point of view. And the latter's root is
> out of reach for agents on a kind Kubernetes node to configure. Meaning, any
> entity on the node setting cgroup v1 net_cls tag will trigger the bypass
> despite cgroup v2 BPF programs attached to the namespace root.
> Generally, this mutual exclusiveness does not hold anymore in today's user
> environments and makes cgroup v2 usage from BPF side fragile and unreliable.
> This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
> sock_cgroup_data in order to address these issues; this implicitly also fixes
> the tradeoffs being made back then with regards to races and refcount leaks
> as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
> programs always operate as expected.
>  https://github.com/nestybox/sysbox/
>  https://kind.sigs.k8s.io/
> Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
> Signed-off-by: Daniel Borkmann <email@example.com>
> Acked-by: Stanislav Fomichev <firstname.lastname@example.org>
> Acked-by: Tejun Heo <email@example.com>
> Cc: David S. Miller <firstname.lastname@example.org>
> Cc: Martynas Pumputis <email@example.com>
prev parent reply other threads:[~2021-09-14 2:01 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-13 23:07 [PATCH bpf v4 1/3] bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode Daniel Borkmann
2021-09-13 23:07 ` [PATCH bpf v4 2/3] bpf, selftests: Add cgroup v1 net_cls classid helpers Daniel Borkmann
2021-09-13 23:07 ` [PATCH bpf v4 3/3] bpf, selftests: Add test case for mixed cgroup v1/v2 Daniel Borkmann
2021-09-14 2:01 ` Alexei Starovoitov [this message]
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).