From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3ADFC433F5 for ; Tue, 12 Apr 2022 23:24:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229552AbiDLX0c (ORCPT ); Tue, 12 Apr 2022 19:26:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230030AbiDLX0U (ORCPT ); Tue, 12 Apr 2022 19:26:20 -0400 Received: from mail-qv1-xf2b.google.com (mail-qv1-xf2b.google.com [IPv6:2607:f8b0:4864:20::f2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43AD92DD72 for ; Tue, 12 Apr 2022 15:42:27 -0700 (PDT) Received: by mail-qv1-xf2b.google.com with SMTP id kl29so304487qvb.2 for ; Tue, 12 Apr 2022 15:42:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ls/k1QAUlIkMnUHslU5E6liMIuumFP772BJJxz79/xE=; b=DPiDRJXz0UzGjyxDDVUPIlmzxQjsYFak/YDGNCwgYzQkMGDjO+DORPcDGhOLDX8M7l OpS5vMKukyj5EcDUja+49CIyCTYZbM6C4zxLy0Z02PFW9C8gBx0gWY4EJqBLH4qKeXgU b9r9ZU1DxhtUYxprGBlyHBEz5pgmkAV742RM/hf7M5ThWkmhTsGxVNQM/LmtjQ/MiXCo Naylc2hu6ggVhula2hjzi4JekeEqsvNUi3OSm0fsJIHxkoVduUSqzfIsNcbFI5h5ar3N b4soYG8xr3ZUkrsRxwMxNCedl6ldfbWULUezC5hW7i/E/koauREDilvvXtavEWhNok59 6tGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ls/k1QAUlIkMnUHslU5E6liMIuumFP772BJJxz79/xE=; b=yaDR4u/BeGgqslR2P0MhXYq6vEcH5byaMv8P4pod8EzSSTXSK81XWKmVyHZAEinV4D K/Jssu0Ug2tpOJ6qKR1VEpOYA3DwYo+CCgxyMmoqYrYGHoL7BEakfTIPOrGED/sZ/jlr JX5n3zJIiQYLT3I77Lprvq/099cuB43Cu0iQncxNuXN4NTntNswNsOxrS7Kp0lAa0xA5 5BxsJRRq3NR2p3adR18FHoU9nlVjCuFDeT8qjmbUZ9BBKVoKE7tOL5nkEf0TLQeXded1 GM6ZB9iScPP7dRMzcYbGZo3E73/hkCWHm35fqUfH6IVksQID8lmo/exX9Tz3BVBDHs+I 5I6g== X-Gm-Message-State: AOAM530NTzvdEYqVs3XdigqXR8bs2qaLN/wTpz+Y62PHx4530BlzTpJo 2U5VhIEFrEuMT32O/g1LYV3rftzhyX6AQjBlIZkSJ+OtnaHhGg== X-Google-Smtp-Source: ABdhPJwAut+TRvd/AbpVK4y/VkWsusyvlOTrmjLWc/bQrdck9w1MP1SktWYgZPxSQVQaSYe/gw/pqmZZxFBwxh/rVjs= X-Received: by 2002:a05:6214:2343:b0:43d:684c:f538 with SMTP id hu3-20020a056214234300b0043d684cf538mr34346444qvb.58.1649803346153; Tue, 12 Apr 2022 15:42:26 -0700 (PDT) MIME-Version: 1.0 References: <20220407223112.1204582-1-sdf@google.com> <20220407223112.1204582-4-sdf@google.com> <20220408225628.oog4a3qteauhqkdn@kafai-mbp.dhcp.thefacebook.com> <20220412013631.tntvx7lw3c7sw6ur@kafai-mbp.dhcp.thefacebook.com> <20220412181353.zgyl2oy4vl3uyigl@kafai-mbp.dhcp.thefacebook.com> <20220412201948.b2jnefks5ptrt3yd@kafai-mbp.dhcp.thefacebook.com> <20220412221352.nmkj6drtmbweawhs@kafai-mbp.dhcp.thefacebook.com> In-Reply-To: <20220412221352.nmkj6drtmbweawhs@kafai-mbp.dhcp.thefacebook.com> From: Stanislav Fomichev Date: Tue, 12 Apr 2022 15:42:15 -0700 Message-ID: Subject: Re: [PATCH bpf-next v3 3/7] bpf: minimize number of allocated lsm slots per program To: Martin KaFai Lau Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Tue, Apr 12, 2022 at 3:13 PM Martin KaFai Lau wrote: > > On Tue, Apr 12, 2022 at 01:36:45PM -0700, Stanislav Fomichev wrote: > > On Tue, Apr 12, 2022 at 1:19 PM Martin KaFai Lau wrote: > > > > > > On Tue, Apr 12, 2022 at 12:01:41PM -0700, Stanislav Fomichev wrote: > > > > On Tue, Apr 12, 2022 at 11:13 AM Martin KaFai Lau wrote: > > > > > > > > > > On Tue, Apr 12, 2022 at 09:42:40AM -0700, Stanislav Fomichev wrote: > > > > > > On Mon, Apr 11, 2022 at 6:36 PM Martin KaFai Lau wrote: > > > > > > > > > > > > > > On Mon, Apr 11, 2022 at 11:46:20AM -0700, Stanislav Fomichev wrote: > > > > > > > > On Fri, Apr 8, 2022 at 3:57 PM Martin KaFai Lau wrote: > > > > > > > > > > > > > > > > > > On Thu, Apr 07, 2022 at 03:31:08PM -0700, Stanislav Fomichev wrote: > > > > > > > > > > Previous patch adds 1:1 mapping between all 211 LSM hooks > > > > > > > > > > and bpf_cgroup program array. Instead of reserving a slot per > > > > > > > > > > possible hook, reserve 10 slots per cgroup for lsm programs. > > > > > > > > > > Those slots are dynamically allocated on demand and reclaimed. > > > > > > > > > > This still adds some bloat to the cgroup and brings us back to > > > > > > > > > > roughly pre-cgroup_bpf_attach_type times. > > > > > > > > > > > > > > > > > > > > It should be possible to eventually extend this idea to all hooks if > > > > > > > > > > the memory consumption is unacceptable and shrink overall effective > > > > > > > > > > programs array. > > > > > > > > > > > > > > > > > > > > Signed-off-by: Stanislav Fomichev > > > > > > > > > > --- > > > > > > > > > > include/linux/bpf-cgroup-defs.h | 4 +- > > > > > > > > > > include/linux/bpf_lsm.h | 6 --- > > > > > > > > > > kernel/bpf/bpf_lsm.c | 9 ++-- > > > > > > > > > > kernel/bpf/cgroup.c | 96 ++++++++++++++++++++++++++++----- > > > > > > > > > > 4 files changed, 90 insertions(+), 25 deletions(-) > > > > > > > > > > > > > > > > > > > > diff --git a/include/linux/bpf-cgroup-defs.h b/include/linux/bpf-cgroup-defs.h > > > > > > > > > > index 6c661b4df9fa..d42516e86b3a 100644 > > > > > > > > > > --- a/include/linux/bpf-cgroup-defs.h > > > > > > > > > > +++ b/include/linux/bpf-cgroup-defs.h > > > > > > > > > > @@ -10,7 +10,9 @@ > > > > > > > > > > > > > > > > > > > > struct bpf_prog_array; > > > > > > > > > > > > > > > > > > > > -#define CGROUP_LSM_NUM 211 /* will be addressed in the next patch */ > > > > > > > > > > +/* Maximum number of concurrently attachable per-cgroup LSM hooks. > > > > > > > > > > + */ > > > > > > > > > > +#define CGROUP_LSM_NUM 10 > > > > > > > > > hmm...only 10 different lsm hooks (or 10 different attach_btf_ids) can > > > > > > > > > have BPF_LSM_CGROUP programs attached. This feels quite limited but having > > > > > > > > > a static 211 (and potentially growing in the future) is not good either. > > > > > > > > > I currently do not have a better idea also. :/ > > > > > > > > > > > > > > > > > > Have you thought about other dynamic schemes or they would be too slow ? > > > > > > > > > > > > > > > > > > > enum cgroup_bpf_attach_type { > > > > > > > > > > CGROUP_BPF_ATTACH_TYPE_INVALID = -1, > > > > > > > > > > diff --git a/include/linux/bpf_lsm.h b/include/linux/bpf_lsm.h > > > > > > > > > > index 7f0e59f5f9be..613de44aa429 100644 > > > > > > > > > > --- a/include/linux/bpf_lsm.h > > > > > > > > > > +++ b/include/linux/bpf_lsm.h > > > > > > > > > > @@ -43,7 +43,6 @@ extern const struct bpf_func_proto bpf_inode_storage_delete_proto; > > > > > > > > > > void bpf_inode_storage_free(struct inode *inode); > > > > > > > > > > > > > > > > > > > > int bpf_lsm_find_cgroup_shim(const struct bpf_prog *prog, bpf_func_t *bpf_func); > > > > > > > > > > -int bpf_lsm_hook_idx(u32 btf_id); > > > > > > > > > > > > > > > > > > > > #else /* !CONFIG_BPF_LSM */ > > > > > > > > > > > > > > > > > > > > @@ -74,11 +73,6 @@ static inline int bpf_lsm_find_cgroup_shim(const struct bpf_prog *prog, > > > > > > > > > > return -ENOENT; > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > -static inline int bpf_lsm_hook_idx(u32 btf_id) > > > > > > > > > > -{ > > > > > > > > > > - return -EINVAL; > > > > > > > > > > -} > > > > > > > > > > - > > > > > > > > > > #endif /* CONFIG_BPF_LSM */ > > > > > > > > > > > > > > > > > > > > #endif /* _LINUX_BPF_LSM_H */ > > > > > > > > > > diff --git a/kernel/bpf/bpf_lsm.c b/kernel/bpf/bpf_lsm.c > > > > > > > > > > index eca258ba71d8..8b948ec9ab73 100644 > > > > > > > > > > --- a/kernel/bpf/bpf_lsm.c > > > > > > > > > > +++ b/kernel/bpf/bpf_lsm.c > > > > > > > > > > @@ -57,10 +57,12 @@ static unsigned int __cgroup_bpf_run_lsm_socket(const void *ctx, > > > > > > > > > > if (unlikely(!sk)) > > > > > > > > > > return 0; > > > > > > > > > > > > > > > > > > > > + rcu_read_lock(); /* See bpf_lsm_attach_type_get(). */ > > > > > > > > > > cgrp = sock_cgroup_ptr(&sk->sk_cgrp_data); > > > > > > > > > > if (likely(cgrp)) > > > > > > > > > > ret = BPF_PROG_RUN_ARRAY_CG(cgrp->bpf.effective[prog->aux->cgroup_atype], > > > > > > > > > > ctx, bpf_prog_run, 0); > > > > > > > > > > + rcu_read_unlock(); > > > > > > > > > > return ret; > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > @@ -77,7 +79,7 @@ static unsigned int __cgroup_bpf_run_lsm_current(const void *ctx, > > > > > > > > > > /*prog = container_of(insn, struct bpf_prog, insnsi);*/ > > > > > > > > > > prog = (const struct bpf_prog *)((void *)insn - offsetof(struct bpf_prog, insnsi)); > > > > > > > > > > > > > > > > > > > > - rcu_read_lock(); > > > > > > > > > > + rcu_read_lock(); /* See bpf_lsm_attach_type_get(). */ > > > > > > > > > I think this is also needed for task_dfl_cgroup(). If yes, > > > > > > > > > will be a good idea to adjust the comment if it ends up > > > > > > > > > using the 'CGROUP_LSM_NUM 10' scheme. > > > > > > > > > > > > > > > > > > While at rcu_read_lock(), have you thought about what major things are > > > > > > > > > needed to make BPF_LSM_CGROUP sleepable ? > > > > > > > > > > > > > > > > > > The cgroup local storage could be one that require changes but it seems > > > > > > > > > the cgroup local storage is not available to BPF_LSM_GROUP in this change set. > > > > > > > > > The current use case doesn't need it? > > > > > > > > > > > > > > > > No, I haven't thought about sleepable at all yet :-( But seems like > > > > > > > > having that rcu lock here might be problematic if we want to sleep? In > > > > > > > > this case, Jakub's suggestion seems better. > > > > > > > The new rcu_read_lock() here seems fine after some thoughts. > > > > > > > > > > > > > > I was looking at the helpers in cgroup_base_func_proto() to get a sense > > > > > > > on sleepable support. Only the bpf_get_local_storage caught my eyes for > > > > > > > now because it uses a call_rcu to free the storage. That will be the > > > > > > > major one to change for sleepable that I can think of for now. > > > > > > > > > > > > That rcu_read_lock should be switched over to rcu_read_lock_trace in > > > > > > the sleepable case I'm assuming? Are we allowed to sleep while holding > > > > > > rcu_read_lock_trace? > > > > > Ah. right, suddenly forgot the obvious in between emails :( > > > > > > > > > > In that sense, may as well remove the rcu_read_lock() here and let > > > > > the trampoline to decide which one (rcu_read_lock or rcu_read_lock_trace) > > > > > to call before calling the shim_prog. The __bpf_prog_enter(_sleepable) will > > > > > call the right rcu_read_lock(_trace) based on the prog is sleepable or not. > > > > > > > > Removing rcu_read_lock in __cgroup_bpf_run_lsm_current might be > > > > problematic because we also want to guarantee current's cgroup doesn't > > > > go away. I'm assuming things like task migrating to a new cgroup and > > > > the old one being freed can happen while we are trying to get cgroup's > > > > effective array. > > > Right, sleepable one may need a short rcu_read_lock only upto > > > a point that the cgrp->bpf.effective[...] is obtained. > > > call_rcu_tasks_trace() is then needed to free the bpf_prog_array. > > > > > > The future sleepable one may be better off to have a different shim func, > > > not sure. rcu_read_lock() can be added back later if it ends up reusing > > > the same shim func is cleaner. > > > > In this case I'll probably have rcu_read_lock for > > cgroup+bpf_lsm_attach_type_get for the current shim. > yeah, depending on rcu grace period to free up cgroup_lsm_atype_btf_id > should be fine. It just needs to wait another grace period for sleepable > in the future. > > Also, just came to my mind, if it wants sleepable and non-sleepable > to be in the same cgrp->bpf.effective[] array. It may need more > thoughts on when to do the rcu_read_lock() and rcu_read_trace_lock(). Ack. I'll try to put these details into a commit message so once we get to the sleepable support we won't have to do these investigations again. > > > > I guess BPF_PROG_RUN_ARRAY_CG will also need some work before > > > > sleepable can happen (it calls rcu_read_lock unconditionally). > > > Yep. I think so. > > > > > > > > > > > Also, it doesn't seem like BPF_PROG_RUN_ARRAY_CG rcu usage is correct. > > > > It receives __rcu array_rcu, takes rcu read lock and does deref. I'm > > > > assuming that array_rcu can be free'd before we even get to > > > > BPF_PROG_RUN_ARRAY_CG's rcu_read_lock? (so having rcu_read_lock around > > > > BPF_PROG_RUN_ARRAY_CG makes sense) > > > BPF_PROG_RUN_ARRAY_CG is __always_inline though. > > > > Does it help? This should still expand to the following, right? > > > > array_rcu = cgrp->bpf.effective[atype]; > I think you are right: > > 86 ret = BPF_PROG_RUN_ARRAY_CG(cgrp->bpf.effective[prog->aux->cgroup_atype], > 0xffffffff812534bb <+155>: mov -0x10(%rbx),%rdx > 0xffffffff812534bf <+159>: movl $0x0,-0x38(%rbp) > 0xffffffff812534c6 <+166>: movslq 0x300(%rdx),%rdx > 0xffffffff812534cd <+173>: mov 0x500(%rax,%rdx,8),%rbx > > [ ... ] > > 1375 array = rcu_dereference(array_rcu); > 0xffffffff8125350d <+237>: callq 0xffffffff81145a50 > 0xffffffff81253512 <+242>: test %eax,%eax > 0xffffffff81253514 <+244>: je 0xffffffff812537a7 <__cgroup_bpf_run_lsm_current+903> > > [ ... ] > > 1376 item = &array->items[0]; > 0xffffffff8125351a <+250>: lea -0x40(%rbp),%rdx > 0xffffffff8125351e <+254>: mov %gs:0x1af40,%rax > 0xffffffff81253527 <+263>: lea 0x10(%rbx),%r12 > > [ ... ] > > 1378 while ((prog = READ_ONCE(item->prog))) { > 0xffffffff81253541 <+289>: test %rbx,%rbx > 0xffffffff81253544 <+292>: je 0xffffffff81253596 <__cgroup_bpf_run_lsm_current+374> > > > Do you know if a macro can work as expected ? > > > > /* theoretically, array_rcu can be freed here? */ > > > > rcu_read_lock(); > > array = rcu_dereference(array_rcu); > > ... > > > > Feels like the callers of BPF_PROG_RUN_ARRAY_CG really have to care > > about rcu locking, not the BPF_PROG_RUN_ARRAY_CG itself. Oh, right, they've been broken since we converted from a define to an inline function. With the define it should've been working correctly. I can move those rcu_read_lock to the callers of BPF_PROG_RUN_ARRAY_CG_FLAGS/BPF_PROG_RUN_ARRAY_CG/BPF_PROG_RUN_ARRAY. Doesn't seem like going back to the defines is the way to go.