From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21FF2C55178 for ; Mon, 26 Oct 2020 15:58:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DB133217A0 for ; Mon, 26 Oct 2020 15:58:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1784627AbgJZP6h (ORCPT ); Mon, 26 Oct 2020 11:58:37 -0400 Received: from foss.arm.com ([217.140.110.172]:43166 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1784512AbgJZP4v (ORCPT ); Mon, 26 Oct 2020 11:56:51 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 05B5C1042; Mon, 26 Oct 2020 08:56:50 -0700 (PDT) Received: from arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3D86B3F719; Mon, 26 Oct 2020 08:56:48 -0700 (PDT) Date: Mon, 26 Oct 2020 15:56:35 +0000 From: Dave Martin To: Catalin Marinas Cc: Topi Miettinen , Mark Rutland , Salvatore Mesoraca , systemd-devel@lists.freedesktop.org, Kees Cook , kernel-hardening@lists.openwall.com, Will Deacon , "linux-kernel@vger.kernel.org" , Jeremy Linton , Mark Brown , linux-hardening@vger.kernel.org, libc-alpha@sourceware.org, "linux-arm-kernel@lists.infradead.org" Subject: Re: BTI interaction between seccomp filters in systemd and glibc mprotect calls, causing service failures Message-ID: <20201026155628.GA27285@arm.com> References: <8584c14f-5c28-9d70-c054-7c78127d84ea@arm.com> <20201022075447.GO3819@arm.com> <78464155-f459-773f-d0ee-c5bdbeb39e5d@gmail.com> <202010221256.A4F95FD11@keescook> <20201023090232.GA25736@gaia> <20201026145245.GD3117@gaia> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201026145245.GD3117@gaia> User-Agent: Mutt/1.5.23 (2014-03-12) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Oct 26, 2020 at 02:52:46PM +0000, Catalin Marinas via Libc-alpha wrote: > On Sat, Oct 24, 2020 at 02:01:30PM +0300, Topi Miettinen wrote: > > On 23.10.2020 12.02, Catalin Marinas wrote: > > > On Thu, Oct 22, 2020 at 01:02:18PM -0700, Kees Cook wrote: > > > > Regardless, it makes sense to me to have the kernel load the executable > > > > itself with BTI enabled by default. I prefer gaining Catalin's suggested > > > > patch[2]. :) > > > [...] > > > > [2] https://lore.kernel.org/linux-arm-kernel/20201022093104.GB1229@gaia/ > > > > > > I think I first heard the idea at Mark R ;). > > > > > > It still needs glibc changes to avoid the mprotect(), or at least ignore > > > the error. Since this is an ABI change and we don't know which kernels > > > would have it backported, maybe better to still issue the mprotect() but > > > ignore the failure. > > > > What about kernel adding an auxiliary vector as a flag to indicate that BTI > > is supported and recommended by the kernel? Then dynamic loader could use > > that to detect that a) the main executable is BTI protected and there's no > > need to mprotect() it and b) PROT_BTI flag should be added to all PROT_EXEC > > pages. > > We could add a bit to AT_FLAGS, it's always been 0 for Linux. > > > In absence of the vector, the dynamic loader might choose to skip doing > > PROT_BTI at all (since the main executable isn't protected anyway either, or > > maybe even the kernel is up-to-date but it knows that it's not recommended > > for some reason, or maybe the kernel is so ancient that it doesn't know > > about BTI). Optionally it could still read the flag from ELF later (for > > compatibility with old kernels) and then do the mprotect() dance, which may > > trip seccomp filters, possibly fatally. > > I think the safest is for the dynamic loader to issue an mprotect() and > ignore the EPERM error. Not all user deployments have this seccomp > filter, so they can still benefit, and user can't tell whether the > kernel change has been backported. > > Now, if the dynamic loader silently ignores the mprotect() failure on > the main executable, is there much value in exposing a flag in the aux > vectors? It saves a few (one?) mprotect() calls but I don't think it > matters much. Anyway, I don't mind the flag. I don't see a problem with the aforementioned patch [2] to pre-set BTI on the pages of the main binary. The original rationale here was that ld.so doesn't _need_ this, since it is going to examine the binary's ELF headers anyway. But equally, if the binary is marked as supporting BTI then it's safe to enable BTI for the binary's own pages. I'd tend to agree that an AT_FLAGS flag doesn't add much. I think real EPERMs would only be seen in assert-fail type situations. Failure of mmap() is likely to result in a segfault later on, or correct operation with weakened permissions on some pages. Given the likely failure modes, that situation doesn't feel too bad. > The only potential risk is if the dynamic loader decides not to turn > PROT_BTI one because of some mix and match of objects but AFAIK BTI > allows interworking. Yes, the design means that a page's PROT_BTI can be set safely if the code in that page was compiled for BTI, irrespective of how other pages were compiled. The reasons why we don't do this at finer granularity are (a) is't not very useful, and (b) ELF images only contain a BTI property note for the whole image, not per segment. I think that ld.so already makes this decision at ELF image granularity (unless someone contradicts me). Cheers ---Dave From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E8ADC2D0A3 for ; Mon, 26 Oct 2020 15:58:22 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A02222400 for ; Mon, 26 Oct 2020 15:58:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="V1inz6E8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A02222400 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=e8g4SgiWaZ7GNYzeFXZL5/HL8dB2K1iWIYeH3SP0AZA=; b=V1inz6E88sce+fm8mK2m7Ky1j 0p1N9M0YvLHkZTED326x7pnJex3L9t0GJ9NcuObF398T6ax0E9buefGeNsigox9UuoRhvr3IwbUnB NuCPeRoDipersZ8+YD2jvova3upiCJkvHttPzFESh0Tziy2L6Xl+0fn4cwo1BrUgY104nBtkGtxAc mbNEsPCDPERiTxIxBnQ8MPRm8cXy+GjSlrlOKhIwdOiPIl/3TjWaxlEQch4RXEqHvSm6m9aTP56tk yaCr4+jfjK0pR8gaky3sXYhT41HkCVKygCcBznZQ3I26dUEENv0vNCdJ7zOudrWw6RurUm6D57i4T t7GCovHaQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kX4re-0000xI-KF; Mon, 26 Oct 2020 15:56:54 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kX4rb-0000wL-Ua for linux-arm-kernel@lists.infradead.org; Mon, 26 Oct 2020 15:56:53 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 05B5C1042; Mon, 26 Oct 2020 08:56:50 -0700 (PDT) Received: from arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3D86B3F719; Mon, 26 Oct 2020 08:56:48 -0700 (PDT) Date: Mon, 26 Oct 2020 15:56:35 +0000 From: Dave Martin To: Catalin Marinas Subject: Re: BTI interaction between seccomp filters in systemd and glibc mprotect calls, causing service failures Message-ID: <20201026155628.GA27285@arm.com> References: <8584c14f-5c28-9d70-c054-7c78127d84ea@arm.com> <20201022075447.GO3819@arm.com> <78464155-f459-773f-d0ee-c5bdbeb39e5d@gmail.com> <202010221256.A4F95FD11@keescook> <20201023090232.GA25736@gaia> <20201026145245.GD3117@gaia> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20201026145245.GD3117@gaia> User-Agent: Mutt/1.5.23 (2014-03-12) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201026_115652_116127_7C9143E7 X-CRM114-Status: GOOD ( 37.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Salvatore Mesoraca , systemd-devel@lists.freedesktop.org, Kees Cook , kernel-hardening@lists.openwall.com, Will Deacon , "linux-kernel@vger.kernel.org" , Jeremy Linton , Mark Brown , Topi Miettinen , linux-hardening@vger.kernel.org, libc-alpha@sourceware.org, "linux-arm-kernel@lists.infradead.org" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Oct 26, 2020 at 02:52:46PM +0000, Catalin Marinas via Libc-alpha wrote: > On Sat, Oct 24, 2020 at 02:01:30PM +0300, Topi Miettinen wrote: > > On 23.10.2020 12.02, Catalin Marinas wrote: > > > On Thu, Oct 22, 2020 at 01:02:18PM -0700, Kees Cook wrote: > > > > Regardless, it makes sense to me to have the kernel load the executable > > > > itself with BTI enabled by default. I prefer gaining Catalin's suggested > > > > patch[2]. :) > > > [...] > > > > [2] https://lore.kernel.org/linux-arm-kernel/20201022093104.GB1229@gaia/ > > > > > > I think I first heard the idea at Mark R ;). > > > > > > It still needs glibc changes to avoid the mprotect(), or at least ignore > > > the error. Since this is an ABI change and we don't know which kernels > > > would have it backported, maybe better to still issue the mprotect() but > > > ignore the failure. > > > > What about kernel adding an auxiliary vector as a flag to indicate that BTI > > is supported and recommended by the kernel? Then dynamic loader could use > > that to detect that a) the main executable is BTI protected and there's no > > need to mprotect() it and b) PROT_BTI flag should be added to all PROT_EXEC > > pages. > > We could add a bit to AT_FLAGS, it's always been 0 for Linux. > > > In absence of the vector, the dynamic loader might choose to skip doing > > PROT_BTI at all (since the main executable isn't protected anyway either, or > > maybe even the kernel is up-to-date but it knows that it's not recommended > > for some reason, or maybe the kernel is so ancient that it doesn't know > > about BTI). Optionally it could still read the flag from ELF later (for > > compatibility with old kernels) and then do the mprotect() dance, which may > > trip seccomp filters, possibly fatally. > > I think the safest is for the dynamic loader to issue an mprotect() and > ignore the EPERM error. Not all user deployments have this seccomp > filter, so they can still benefit, and user can't tell whether the > kernel change has been backported. > > Now, if the dynamic loader silently ignores the mprotect() failure on > the main executable, is there much value in exposing a flag in the aux > vectors? It saves a few (one?) mprotect() calls but I don't think it > matters much. Anyway, I don't mind the flag. I don't see a problem with the aforementioned patch [2] to pre-set BTI on the pages of the main binary. The original rationale here was that ld.so doesn't _need_ this, since it is going to examine the binary's ELF headers anyway. But equally, if the binary is marked as supporting BTI then it's safe to enable BTI for the binary's own pages. I'd tend to agree that an AT_FLAGS flag doesn't add much. I think real EPERMs would only be seen in assert-fail type situations. Failure of mmap() is likely to result in a segfault later on, or correct operation with weakened permissions on some pages. Given the likely failure modes, that situation doesn't feel too bad. > The only potential risk is if the dynamic loader decides not to turn > PROT_BTI one because of some mix and match of objects but AFAIK BTI > allows interworking. Yes, the design means that a page's PROT_BTI can be set safely if the code in that page was compiled for BTI, irrespective of how other pages were compiled. The reasons why we don't do this at finer granularity are (a) is't not very useful, and (b) ELF images only contain a BTI property note for the whole image, not per segment. I think that ld.so already makes this decision at ELF image granularity (unless someone contradicts me). Cheers ---Dave _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BE44C4363A for ; Mon, 26 Oct 2020 15:57:14 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id 2EA1222400 for ; Mon, 26 Oct 2020 15:57:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2EA1222400 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-20273-kernel-hardening=archiver.kernel.org@lists.openwall.com Received: (qmail 7957 invoked by uid 550); 26 Oct 2020 15:57:02 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Received: (qmail 7919 invoked from network); 26 Oct 2020 15:57:02 -0000 Date: Mon, 26 Oct 2020 15:56:35 +0000 From: Dave Martin To: Catalin Marinas Cc: Topi Miettinen , Mark Rutland , Salvatore Mesoraca , systemd-devel@lists.freedesktop.org, Kees Cook , kernel-hardening@lists.openwall.com, Will Deacon , "linux-kernel@vger.kernel.org" , Jeremy Linton , Mark Brown , linux-hardening@vger.kernel.org, libc-alpha@sourceware.org, "linux-arm-kernel@lists.infradead.org" Subject: Re: BTI interaction between seccomp filters in systemd and glibc mprotect calls, causing service failures Message-ID: <20201026155628.GA27285@arm.com> References: <8584c14f-5c28-9d70-c054-7c78127d84ea@arm.com> <20201022075447.GO3819@arm.com> <78464155-f459-773f-d0ee-c5bdbeb39e5d@gmail.com> <202010221256.A4F95FD11@keescook> <20201023090232.GA25736@gaia> <20201026145245.GD3117@gaia> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201026145245.GD3117@gaia> User-Agent: Mutt/1.5.23 (2014-03-12) On Mon, Oct 26, 2020 at 02:52:46PM +0000, Catalin Marinas via Libc-alpha wrote: > On Sat, Oct 24, 2020 at 02:01:30PM +0300, Topi Miettinen wrote: > > On 23.10.2020 12.02, Catalin Marinas wrote: > > > On Thu, Oct 22, 2020 at 01:02:18PM -0700, Kees Cook wrote: > > > > Regardless, it makes sense to me to have the kernel load the executable > > > > itself with BTI enabled by default. I prefer gaining Catalin's suggested > > > > patch[2]. :) > > > [...] > > > > [2] https://lore.kernel.org/linux-arm-kernel/20201022093104.GB1229@gaia/ > > > > > > I think I first heard the idea at Mark R ;). > > > > > > It still needs glibc changes to avoid the mprotect(), or at least ignore > > > the error. Since this is an ABI change and we don't know which kernels > > > would have it backported, maybe better to still issue the mprotect() but > > > ignore the failure. > > > > What about kernel adding an auxiliary vector as a flag to indicate that BTI > > is supported and recommended by the kernel? Then dynamic loader could use > > that to detect that a) the main executable is BTI protected and there's no > > need to mprotect() it and b) PROT_BTI flag should be added to all PROT_EXEC > > pages. > > We could add a bit to AT_FLAGS, it's always been 0 for Linux. > > > In absence of the vector, the dynamic loader might choose to skip doing > > PROT_BTI at all (since the main executable isn't protected anyway either, or > > maybe even the kernel is up-to-date but it knows that it's not recommended > > for some reason, or maybe the kernel is so ancient that it doesn't know > > about BTI). Optionally it could still read the flag from ELF later (for > > compatibility with old kernels) and then do the mprotect() dance, which may > > trip seccomp filters, possibly fatally. > > I think the safest is for the dynamic loader to issue an mprotect() and > ignore the EPERM error. Not all user deployments have this seccomp > filter, so they can still benefit, and user can't tell whether the > kernel change has been backported. > > Now, if the dynamic loader silently ignores the mprotect() failure on > the main executable, is there much value in exposing a flag in the aux > vectors? It saves a few (one?) mprotect() calls but I don't think it > matters much. Anyway, I don't mind the flag. I don't see a problem with the aforementioned patch [2] to pre-set BTI on the pages of the main binary. The original rationale here was that ld.so doesn't _need_ this, since it is going to examine the binary's ELF headers anyway. But equally, if the binary is marked as supporting BTI then it's safe to enable BTI for the binary's own pages. I'd tend to agree that an AT_FLAGS flag doesn't add much. I think real EPERMs would only be seen in assert-fail type situations. Failure of mmap() is likely to result in a segfault later on, or correct operation with weakened permissions on some pages. Given the likely failure modes, that situation doesn't feel too bad. > The only potential risk is if the dynamic loader decides not to turn > PROT_BTI one because of some mix and match of objects but AFAIK BTI > allows interworking. Yes, the design means that a page's PROT_BTI can be set safely if the code in that page was compiled for BTI, irrespective of how other pages were compiled. The reasons why we don't do this at finer granularity are (a) is't not very useful, and (b) ELF images only contain a BTI property note for the whole image, not per segment. I think that ld.so already makes this decision at ELF image granularity (unless someone contradicts me). Cheers ---Dave