From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D610C38BFA for ; Mon, 24 Feb 2020 22:29:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C9D8B2176D for ; Mon, 24 Feb 2020 22:29:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=yahoo.com header.i=@yahoo.com header.b="OQdbFYMw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728206AbgBXW3e (ORCPT ); Mon, 24 Feb 2020 17:29:34 -0500 Received: from sonic314-27.consmr.mail.ne1.yahoo.com ([66.163.189.153]:39917 "EHLO sonic314-27.consmr.mail.ne1.yahoo.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbgBXW3e (ORCPT ); Mon, 24 Feb 2020 17:29:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1582583372; bh=H3OMHz83zF7WyUPhKd7RtvZJPyUnWEp7eI8UJbwbuqI=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From:Subject; b=OQdbFYMwl6AM+QV+aeyBwhum8GaE8khOugVqAqTP/dzHvTkzJJ6AG7k+Wj1J+DLzsdZmbiEV8C+QWR5rkz0dPTCP/D9yvg6FXcZJZEKjbmMwZUb850g5nvb90TDPswb9iAwKIVNlRNKRBPJPyScr5QQ6SJvY6MxPPB0ZmPAJRRWou01PvQ8cZKVZpdO8ksivf9J310h+STEWExc6/9m8+RkQZXepie9sBR4ELA9yJradUojnyCceIeSQKPddrIxpIA57WdJfW/3/u7AV4CfJeCzB6uuzxiY2OzPqYKWUbY4JmWOrAvdqoIdfxrlf5vieH4T15RUn4ggZ22spNAKVjg== X-YMail-OSG: IeIf4q4VM1m5gVfspj2gWxyaMDMKz4FXY1tqWyjCAWRE4AvQINaz6V6F.OZsjcG ho63OkmwN9e2ZAiqyzFdgDjb1fHCtanajbd3qztnbjEr93uU1epmb2g_oZpwuF2wGJnI6u0zZmx2 mu.T_umZ3vbbWwHFPND2gBGCFdkS03lqogeHLHQfuxUglt7iqtrbTIUFsyBAM9ZBIFLAcrTsKQgU sI2RIu.sOhDci91ZPSE_U2CrCXg5h7x.O0SLg2AF6wLavP6OuwsOyUcwTH6Ic1P.CNWxjLHNDyX4 baKc1CA8x1.xkfTyaHjtc0igQNpleC70UHnM8Ad6rJNqbyBZGimba7QSOj5ennlJoHoOkP8hqNrU Kd95znLCEkUQ3Tn1w.hptFmdUr35b4nE3btL2Z5YB9XZX5u0zVsawSRxgTYy_LDVrhoyhtN_g6_4 d.cufhZqbsu0tzd4KIhVAIfr3BDEO8H_XS8rL.1ZFYhqEjvYoxqjSRPQtKyMTupqmuVa.K6xExA8 lueMBSkw.lnkV4Hj9GpSTUgRU9cNQbZqM0As4xa75CDf0Ds5SjwbHt.o_0AOcMASlR.7F4PrA35I 65RXJnFUbYYkSdrN0SVWep8bu_hGKSb6RFGLoBcbztnCYvXuIlG42zZT0oMhKi6Uc7HYgloduPC8 67fLMQtwBMlIyDw3cz1ENmdxdo2w_zQA20Yk1w3BmOEPTZyriBlAa9zWZAN4d51aZ0ZmqP9fqWlu PcfmheXI0X97lKy08kz19j1BAlPlqyHK.ZY4SPl4oRNQyBimO3tFWU6AfzNAA.yro2kkML8.vF18 AjG_Xwspe3CmUd.u3plEufvjf4c4q4ph1BanKwcJnJz7GVa0825wdjjfGA0st6QWqMcp9007SEjC X1ClQwhohLCitYygTRT5nJP_uAyPoVrECo7LTGf5G8yDTAqJ8eHsufdxGt.Fd59HxdBo50rrSt68 FErXW63uKPuVZI6KfrzGYiGa7ZhqbQH3xvaey6oH8dUb9kNbPZnIIXHeJ51nTePWV5XYQl3FXICG 199iPnyG3i2FAhhlX_Be6Pza1sVoy7iC.RsHlcDvuD6L6OvM.Xwa2OP_FjHf8iIMaA3Npy3y_HIW X_voxzSWv9bA_WP64hoeK6fcUeoAjdx0SqMxiTR9jpd1yQ7S5YLwCnzp9vPDaHJDTJon88Q1nNJy xXoJTjNiLf0khuEmxVmfo4BuAiAtAgCtxVM0G9N3BS.mqExLgHaiTZVaKU2_I92w9cTrIqz2JUYO 6v_G2EB9ENlHD1HvPaxNJdoZXx0wngv4rwncYfzsiY5l5JC3kiWLumIHvo8nBmdag52YL7g.LPDV HLUmq2tfTWtrVuYIqr0iwr.tTWEFeT2HIHVWnnsAF75qcJGfBIqGBRccDd_T48zkM96BAFGuDpvd JbEHQQXQq6mxs1Shz44wlO428FMisp60bsBMy8XDNF6epXXwseakRNx1AQnLJC67CdzWicfpc4jU DO6lNz8wPt5PX4vs7QQ-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.ne1.yahoo.com with HTTP; Mon, 24 Feb 2020 22:29:32 +0000 Received: by smtp405.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID bac25d1e210c43b5960a7de16d29a2af; Mon, 24 Feb 2020 22:29:31 +0000 (UTC) Subject: Re: [PATCH bpf-next v4 3/8] bpf: lsm: provide attachment points for BPF LSM programs To: Kees Cook Cc: KP Singh , Alexei Starovoitov , LKML , Linux Security Module list , Alexei Starovoitov , James Morris , bpf@vger.kernel.org, netdev@vger.kernel.org, Casey Schaufler References: <20200220175250.10795-1-kpsingh@chromium.org> <20200220175250.10795-4-kpsingh@chromium.org> <0ef26943-9619-3736-4452-fec536a8d169@schaufler-ca.com> <202002211946.A23A987@keescook> <20200223220833.wdhonzvven7payaw@ast-mbp> <20200224171305.GA21886@chromium.org> <00c216e1-bcfd-b7b1-5444-2a2dfa69190b@schaufler-ca.com> <202002241136.C4F9F7DFF@keescook> From: Casey Schaufler Autocrypt: addr=casey@schaufler-ca.com; keydata= mQINBFzV9HABEAC/mmv3jeJyF7lR7QhILYg1+PeBLIMZv7KCzBSc/4ZZipoWdmr77Lel/RxQ 1PrNx0UaM5r6Hj9lJmJ9eg4s/TUBSP67mTx+tsZ1RhG78/WFf9aBe8MSXxY5cu7IUwo0J/CG vdSqACKyYPV5eoTJmnMxalu8/oVUHyPnKF3eMGgE0mKOFBUMsb2pLS/enE4QyxhcZ26jeeS6 3BaqDl1aTXGowM5BHyn7s9LEU38x/y2ffdqBjd3au2YOlvZ+XUkzoclSVfSR29bomZVVyhMB h1jTmX4Ac9QjpwsxihT8KNGvOM5CeCjQyWcW/g8LfWTzOVF9lzbx6IfEZDDoDem4+ZiPsAXC SWKBKil3npdbgb8MARPes2DpuhVm8yfkJEQQmuLYv8GPiJbwHQVLZGQAPBZSAc7IidD2zbf9 XAw1/SJGe1poxOMfuSBsfKxv9ba2i8hUR+PH7gWwkMQaQ97B1yXYxVEkpG8Y4MfE5Vd3bjJU kvQ/tOBUCw5zwyIRC9+7zr1zYi/3hk+OG8OryZ5kpILBNCo+aePeAJ44znrySarUqS69tuXd a3lMPHUJJpUpIwSKQ5UuYYkWlWwENEWSefpakFAIwY4YIBkzoJ/t+XJHE1HTaJnRk6SWpeDf CreF3+LouP4njyeLEjVIMzaEpwROsw++BX5i5vTXJB+4UApTAQARAQABtChDYXNleSBTY2hh dWZsZXIgPGNhc2V5QHNjaGF1Zmxlci1jYS5jb20+iQJUBBMBCAA+FiEEC+9tH1YyUwIQzUIe OKUVfIxDyBEFAlzV9HACGwMFCRLMAwAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQOKUV fIxDyBG6ag/6AiRl8yof47YOEVHlrmewbpnlBTaYNfJ5cZflNRKRX6t4bp1B2YV1whlDTpiL vNOwFkh+ZE0eI5M4x8Gw2Oiok+4Q5liA9PHTozQYF+Ia+qdL5EehfbLGoEBqklpGvG3h8JsO 7SvONJuFDgvab/U/UriDYycJwzwKZuhVtK9EMpnTtUDyP3DY+Q8h7MWsniNBLVXnh4yBIEJg SSgDn3COpZoFTPGKE+rIzioo/GJe8CTa2g+ZggJiY/myWTS3quG0FMvwvNYvZ4I2g6uxSl7n bZVqAZgqwoTAv1HSXIAn9muwZUJL03qo25PFi2gQmX15BgJKQcV5RL0GHFHRThDS3IyadOgK P2j78P8SddTN73EmsG5OoyzwZAxXfck9A512BfVESqapHurRu2qvMoUkQaW/2yCeRQwGTsFj /rr0lnOBkyC6wCmPSKXe3dT2mnD5KnCkjn7KxLqexKt4itGjJz4/ynD/qh+gL7IPbifrQtVH JI7cr0fI6Tl8V6efurk5RjtELsAlSR6fKV7hClfeDEgLpigHXGyVOsynXLr59uE+g/+InVic jKueTq7LzFd0BiduXGO5HbGyRKw4MG5DNQvC//85EWmFUnDlD3WHz7Hicg95D+2IjD2ZVXJy x3LTfKWdC8bU8am1fi+d6tVEFAe/KbUfe+stXkgmfB7pxqW5Ag0EXNX0cAEQAPIEYtPebJzT wHpKLu1/j4jQcke06Kmu5RNuj1pEje7kX5IKzQSs+CPH0NbSNGvrA4dNGcuDUTNHgb5Be9hF zVqRCEvF2j7BFbrGe9jqMBWHuWheQM8RRoa2UMwQ704mRvKr4sNPh01nKT52ASbWpBPYG3/t WbYaqfgtRmCxBnqdOx5mBJIBh9Q38i63DjQgdNcsTx2qS7HFuFyNef5LCf3jogcbmZGxG/b7 yF4OwmGsVc8ufvlKo5A9Wm+tnRjLr/9Mn9vl5Xa/tQDoPxz26+aWz7j1in7UFzAarcvqzsdM Em6S7uT+qy5jcqyuipuenDKYF/yNOVSNnsiFyQTFqCPCpFihOnuaWqfmdeUOQHCSo8fD4aRF emsuxqcsq0Jp2ODq73DOTsdFxX2ESXYoFt3Oy7QmIxeEgiHBzdKU2bruIB5OVaZ4zWF+jusM Uh+jh+44w9DZkDNjxRAA5CxPlmBIn1OOYt1tsphrHg1cH1fDLK/pDjsJZkiH8EIjhckOtGSb aoUUMMJ85nVhN1EbU/A3DkWCVFEA//Vu1+BckbSbJKE7Hl6WdW19BXOZ7v3jo1q6lWwcFYth esJfk3ZPPJXuBokrFH8kqnEQ9W2QgrjDX3et2WwZFLOoOCItWxT0/1QO4ikcef/E7HXQf/ij Dxf9HG2o5hOlMIAkJq/uLNMvABEBAAGJAjwEGAEIACYWIQQL720fVjJTAhDNQh44pRV8jEPI EQUCXNX0cAIbDAUJEswDAAAKCRA4pRV8jEPIEWkzEACKFUnpp+wIVHpckMfBqN8BE5dUbWJc GyQ7wXWajLtlPdw1nNw0Wrv+ob2RCT7qQlUo6GRLcvj9Fn5tR4hBvR6D3m8aR0AGHbcC62cq I7LjaSDP5j/em4oVL2SMgNTrXgE2w33JMGjAx9oBzkxmKUqprhJomPwmfDHMJ0t7y39Da724 oLPTkQDpJL1kuraM9TC5NyLe1+MyIxqM/8NujoJbWeQUgGjn9uxQAil7o/xSCjrWCP3kZDID vd5ZaHpdl8e1mTExQoKr4EWgaMjmD/a3hZ/j3KfTVNpM2cLfD/QwTMaC2fkK8ExMsz+rUl1H icmcmpptCwOSgwSpPY1Zfio6HvEJp7gmDwMgozMfwQuT9oxyFTxn1X3rn1IoYQF3P8gsziY5 qtTxy2RrgqQFm/hr8gM78RhP54UPltIE96VywviFzDZehMvuwzW//fxysIoK97Y/KBZZOQs+ /T+Bw80Pwk/dqQ8UmIt2ffHEgwCTbkSm711BejapWCfklxkMZDp16mkxSt2qZovboVjXnfuq wQ1QL4o4t1hviM7LyoflsCLnQFJh6RSBhBpKQinMJl/z0A6NYDkQi6vEGMDBWX/M2vk9Jvwa v0cEBfY3Z5oFgkh7BUORsu1V+Hn0fR/Lqq/Pyq+nTR26WzGDkolLsDr3IH0TiAVH5ZuPxyz6 abzjfg== Message-ID: <3fb823a6-b5ec-9341-e788-91b197fc9e7f@schaufler-ca.com> Date: Mon, 24 Feb 2020 14:29:30 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <202002241136.C4F9F7DFF@keescook> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: en-US X-Mailer: WebService/1.1.15199 hermes Apache-HttpAsyncClient/4.1.4 (Java/1.8.0_241) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2/24/2020 1:41 PM, Kees Cook wrote: > On Mon, Feb 24, 2020 at 10:45:27AM -0800, Casey Schaufler wrote: >> On 2/24/2020 9:13 AM, KP Singh wrote: >>> On 24-Feb 08:32, Casey Schaufler wrote: >>>> On 2/23/2020 2:08 PM, Alexei Starovoitov wrote: >>>>> On Fri, Feb 21, 2020 at 08:22:59PM -0800, Kees Cook wrote: >>>>>> If I'm understanding this correctly, there are two issues: >>>>>> >>>>>> 1- BPF needs to be run last due to fexit trampolines (?) >>>>> no. >>>>> The placement of nop call can be anywhere. >>>>> BPF trampoline is automagically converting nop call into a sequence= >>>>> of directly invoked BPF programs. >>>>> No link list traversals and no indirect calls in run-time. >>>> Then why the insistence that it be last? >>> I think this came out of the discussion about not being able to >>> override the other LSMs and introduce a new attack vector with some >>> arguments discussed at: >>> >>> https://lore.kernel.org/bpf/20200109194302.GA85350@google.com/ >>> >>> Let's say we have SELinux + BPF runnng on the system. BPF should stil= l >>> respect any decisions made by SELinux. This hasn't got anything to >>> do with the usage of fexit trampolines. >> The discussion sited is more about GPL than anything else. >> >> The LSM rule is that any security module must be able to >> accept the decisions of others. SELinux has to accept decisions >> made ahead of it. It always has, as LSM checks occur after >> "traditional" checks, which may fail. The only reason that you >> need to be last in this implementation appears to be that you >> refuse to use the general mechanisms. You can't blame SELinux >> for that. > Okay, this is why I wanted to try to state things plainly. The "in last= > position" appears to be the result of a couple design choices: > > -the idea of "not wanting to get in the way of other LSMs", while > admirable, needs to actually be a non-goal to be "just" a stacked LSM > (as you're saying here Casey). This position _was_ required for the > non-privileged LSM case to avoid security implications, but that goal > not longer exists here either. Thanks. > -optimally using the zero-cost call-outs (static key + fexit trampoline= s) > meant it didn't interact well with the existing stacking mechanism. Exactly. > So, fine, these appear to be design choices, not *specifically* > requirements. Let's move on, I think there is more to unpack here... Right. >>>>>> 2- BPF hooks don't know what may be attached at any given time, so= >>>>>> ALL LSM hooks need to be universally hooked. THIS turns out to = create >>>>>> a measurable performance problem in that the cost of the indire= ct call >>>>>> on the (mostly/usually) empty BPF policy is too high. >>>>> also no. > AIUI, there was some confusion on Alexei's reply here. I, perhaps, > was not as clear as I needed to be. I think the later discussion on > performance overheads gets more into the point, and gets us closer to > the objections Alexei had. More below... Agreed. >>> This approach still had the issues of an indirect call and an >>> extra check when not used. So this was not truly zero overhead even= >>> after special casing BPF. >> The LSM mechanism is not zero overhead. It never has been. That's why >> you can compile it out. You get added value at a price. You get the >> ability to use SELinux and KRSI together at a price. If that's unaccep= table >> you can go the route of seccomp, which doesn't use LSM for many of the= >> same reasons you're on about. >> [...] >>>>>> So, trying to avoid the indirect calls is, as you say, an optimiza= tion, >>>>>> but it might be a needed one due to the other limitations. >>>>> I'm convinced that avoiding the cost of retpoline in critical path = is a >>>>> requirement for any new infrastructure in the kernel. >>>> Sorry, I haven't gotten that memo. > I agree with Casey here -- it's a nice goal, but those cost evaluations= have > not yet(?[1]) hit the LSM world. I think it's a desirable goal, to be > sure, but this does appear to be an early optimization. Thanks for helping clarify that. >> [...] >> It can do that wholly within KRSI hooks. You don't need to >> put KRSI specific code into security.c. > This observation is where I keep coming back to. > > Yes, the resulting code is not as fast as it could be. The fact that BP= F > triggers the worst-case performance of LSM hooking is the "new" part > here, from what I can see. I haven't put this oar in the water before, but mightn't it be possible to configure which LSM hooks can have BPF programs installed at compile time, and thus address this issue for the "production" case? I fully expect that such a configuration option would be hideously ugly both to implement and use. If the impact of unused BPF hooks is so great a concern, perhaps it would be worthwhile. > I suspect the problem is that folks in the BPF subsystem don't want to > be seen as slowing anything down, even other subsystems, so they don't > want to see this done in the traditional LSM hooking way (which contain= s > indirect calls). I get that, and it is a laudable goal, but ... > But the LSM subsystem doesn't want special cases (Casey has worked very= > hard to generalize everything there for stacking). It is really hard to= > accept adding a new special case when there are still special cases yet= > to be worked out even in the LSM code itself[2]. =2E.. like Kees says, this isn't the only use case we have to deal with. = >>>>> Networking stack converted all such places to conditional calls. >>>>> In BPF land we converted indirect calls to direct jumps and direct = calls. >>>>> It took two years to do so. Adding new indirect calls is not an opt= ion. >>>>> I'm eagerly waiting for Peter's static_call patches to land to conv= ert >>>>> a lot more indirect calls. May be existing LSMs will take advantage= >>>>> of static_call patches too, but static_call is not an option for BP= F. >>>>> That's why we introduced BPF trampoline in the last kernel release.= >>>> Sorry, but I don't see how BPF is so overwhelmingly special. >>> My analogy here is that if every tracepoint in the kernel were of the= >>> type: >>> >>> if (trace_foo_enabled) // <-- Overhead here, solved with static key >>> trace_foo(a); // <-- retpoline overhead, solved with fexit trampo= lines > This is a helpful distillation; thanks. > > static keys (perhaps better described as static branches) make sense to= > me; I'm familiar with them being used all over the place[3]. The result= ing > "zero performance" branch mechanism is extremely handy. > > I had been thinking about the fexit stuff only as a way for BPF to call= > into kernel functions directly, and I missed the place where this got > used for calling from the kernel into BPF directly. KP walked me throug= h > the fexit stuff off list. I missed where there NOP stub ("noinline int > bpf_lsm_##NAME(__VA_ARGS__) { return 0; }") was being patched by BPF in= > https://lore.kernel.org/lkml/20200220175250.10795-6-kpsingh@chromium.or= g/ > The key bit being "bpf_trampoline_update(prog)" > >>> It would be very hard to justify enabling them on a production system= , >>> and the same can be said for BPF and KRSI. >> The same can be and has been said about the LSM infrastructure. >> If BPF and KRSI are that performance critical you shouldn't be >> tying them to LSM, which is known to have overhead. If you can't >> accept the LSM overhead, get out of the LSM. Or, help us fix the >> LSM infrastructure to make its overhead closer to zero. Whether >> you believe it or not, a lot of work has gone into keeping the LSM >> overhead as small as possible while remaining sufficiently general >> to perform its function. >> >> No. If you're too special to play by LSM rules then you're special >> enough to get into the kernel using more direct means. > So, I see the primary conflict here being about the performance > optimizations. AIUI: > > - BPF subsystem maintainers do not want any new slowdown associated > with BPF Right. > - LSM subsystem maintainers do not want any new special cases in > LSM stacking Right. > So, unless James is going to take this over Casey's objections, the pat= h > forward I see here is: > > - land a "slow" KRSI (i.e. one that hooks every hook with a stub). > - optimize calling for all LSMs > > Does this seem right to everyone? This would be my first choice. The existing list-of-hooks mechanism is an "obvious" implementation. I have been thinking about possible ways to make it better, but as y'all may have guessed, I haven't seen all the coo= l new optimization techniques. > -Kees > > > [1] There is a "known cost to LSM", but as Casey mentions, it's been > generally deemed "acceptable". There have been some recent attempts to > quantify it, but it's not been very clear: > https://lore.kernel.org/linux-security-module/c98000ea-df0e-1ab7-a0e2-b= 47d913f50c8@tycho.nsa.gov/ (lore is missing half this conversation for so= me reason) > > [2] Casey's work to generalize the LSM interfaces continues and it quit= e > complex: > https://lore.kernel.org/linux-security-module/20200214234203.7086-1-cas= ey@schaufler-ca.com/ > > [3] E.g. HARDENED_USERCOPY uses it: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree= /mm/usercopy.c?h=3Dv5.5#n258 > and so does the heap memory auto-initialization: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree= /mm/slab.h?h=3Dv5.5#n676 >