From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52145C388F9 for ; Mon, 26 Oct 2020 16:32:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 04CC22224A for ; Mon, 26 Oct 2020 16:32:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ilyK4nri" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1786034AbgJZQcP (ORCPT ); Mon, 26 Oct 2020 12:32:15 -0400 Received: from mail-lj1-f195.google.com ([209.85.208.195]:45178 "EHLO mail-lj1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1786024AbgJZQcO (ORCPT ); Mon, 26 Oct 2020 12:32:14 -0400 Received: by mail-lj1-f195.google.com with SMTP id a4so10876035lji.12; Mon, 26 Oct 2020 09:32:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=81sFbAVwzGBYdqNxSKasnwihrmLk5qFx2xW7a2yOVFA=; b=ilyK4nriJpTU+ZsR5TzVNUPI2EICBAV+EtzdIfm0c8gnj9uEXsmcIlrH4esm52Sof5 Z9W3RF1ohEzkLaXmgew/evfyvbKalDRv4wGUlakWh1vwJi+Holvp1iFQavUz7IIINteZ DXfWJqJcuIjSWoOYQlpTF8qMEVNtDnTAy14ZhUAoO6a9psISAvTf4eiGo88ojEk0OVOt DoluxxDfBapo4lA0C3ZHtuZ8p4Qzmm8/zACDjcDNBsCtA/MAYCoro7MwWTB4NW5u7X6u DympMB1G1hMAQePPLbaLhpIDiugGlLowk9LLdoo0k7DAhyw+bMfwmtOEMfW0IrE3liO8 VgLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=81sFbAVwzGBYdqNxSKasnwihrmLk5qFx2xW7a2yOVFA=; b=aBD6BVbE9GaYshfcgxadgaPU+aGfpaOayN3P62A/cJUaWvnj7KHw/aYtW1wIKX8c1W mWMRFNjPdLsO/dzpTrm2h6X276k+rCHXcH174gazTUKJ3lGcIAzhDOg/bEvBrw1Zyj6K w8aNrK7mXXxSqT43lC2k4wqghadFXp4Pnnqhr9iLEdcJC2H8jhRY8qgvSue3ktmkHHFF ulnNAWMPJ8cTgxeMf/pztIwZhBsCKrnCjie6U+lFWYAAm2ycF41s5bGSJQ4fKlEqaNIg iACewGugwkBLAyyH4OYowMKeoEcInH/DueTXhPGJE2cxCQRlrTmY6fJb4nExnvWx0E4z V3RA== X-Gm-Message-State: AOAM530qw7zP4PUjFwCTG0vewLcj1tXKFBFIGxilasEniAS34LyiWIrD NelPotsLWrdJKf9AAhihR0tVWqgoPeInSQ== X-Google-Smtp-Source: ABdhPJyBBZepygrfLeS9IiJiGCwiL99FGQQQhPP2SALgITxO/hNm/8c7G2c/C1WBoQBEiojldvEEsA== X-Received: by 2002:a2e:2ac6:: with SMTP id q189mr5924430ljq.269.1603729930410; Mon, 26 Oct 2020 09:32:10 -0700 (PDT) Received: from [192.168.1.112] (88-114-211-119.elisa-laajakaista.fi. [88.114.211.119]) by smtp.gmail.com with ESMTPSA id q123sm1264618ljq.108.2020.10.26.09.32.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Oct 2020 09:32:09 -0700 (PDT) Subject: Re: BTI interaction between seccomp filters in systemd and glibc mprotect calls, causing service failures To: Catalin Marinas Cc: Kees Cook , Szabolcs Nagy , Jeremy Linton , "linux-arm-kernel@lists.infradead.org" , libc-alpha@sourceware.org, systemd-devel@lists.freedesktop.org, "linux-kernel@vger.kernel.org" , Mark Rutland , Mark Brown , Dave Martin , Will Deacon , Salvatore Mesoraca , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org References: <8584c14f-5c28-9d70-c054-7c78127d84ea@arm.com> <20201022075447.GO3819@arm.com> <78464155-f459-773f-d0ee-c5bdbeb39e5d@gmail.com> <202010221256.A4F95FD11@keescook> <20201023090232.GA25736@gaia> <20201026145245.GD3117@gaia> From: Topi Miettinen Message-ID: Date: Mon, 26 Oct 2020 18:31:47 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <20201026145245.GD3117@gaia> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 26.10.2020 16.52, Catalin Marinas wrote: > On Sat, Oct 24, 2020 at 02:01:30PM +0300, Topi Miettinen wrote: >> On 23.10.2020 12.02, Catalin Marinas wrote: >>> On Thu, Oct 22, 2020 at 01:02:18PM -0700, Kees Cook wrote: >>>> Regardless, it makes sense to me to have the kernel load the executable >>>> itself with BTI enabled by default. I prefer gaining Catalin's suggested >>>> patch[2]. :) >>> [...] >>>> [2] https://lore.kernel.org/linux-arm-kernel/20201022093104.GB1229@gaia/ >>> >>> I think I first heard the idea at Mark R ;). >>> >>> It still needs glibc changes to avoid the mprotect(), or at least ignore >>> the error. Since this is an ABI change and we don't know which kernels >>> would have it backported, maybe better to still issue the mprotect() but >>> ignore the failure. >> >> What about kernel adding an auxiliary vector as a flag to indicate that BTI >> is supported and recommended by the kernel? Then dynamic loader could use >> that to detect that a) the main executable is BTI protected and there's no >> need to mprotect() it and b) PROT_BTI flag should be added to all PROT_EXEC >> pages. > > We could add a bit to AT_FLAGS, it's always been 0 for Linux. Great! >> In absence of the vector, the dynamic loader might choose to skip doing >> PROT_BTI at all (since the main executable isn't protected anyway either, or >> maybe even the kernel is up-to-date but it knows that it's not recommended >> for some reason, or maybe the kernel is so ancient that it doesn't know >> about BTI). Optionally it could still read the flag from ELF later (for >> compatibility with old kernels) and then do the mprotect() dance, which may >> trip seccomp filters, possibly fatally. > > I think the safest is for the dynamic loader to issue an mprotect() and > ignore the EPERM error. Not all user deployments have this seccomp > filter, so they can still benefit, and user can't tell whether the > kernel change has been backported. But the seccomp filter can be set to kill the process, so that's definitely not the safest way. I think safest is that when the AT_FLAGS bit is seen, ld.so doesn't do any mprotect() calls but instead when mapping the segments, mmap() flags are adjusted to include PROT_BTI, so mprotect() calls are not necessary. If there's no seccomp filter, there's no disadvantage for avoiding the useless mprotect() calls. I'd expect the backported kernel change to include both aux vector and also using PROT_BTI for the main executable. Then the logic would work with backported kernels as well. If there's no aux vector, all bets are off. The kernel could be old and unpatched, even so old that PROT_BTI is not known. Perhaps also in the future there may be new technologies which have replaced BTI and the kernel could want a previous generation ld.so not to try to use BTI, so this could be also indicated with the lack of aux vector. The dynamic loader could still attempt to mprotect() the pages, but that could be fatal. Getting to the point where the error can be ignored means that there's no seccomp filter, at least none set to kill. Perhaps the pain is only temporary, new or patched kernels should eventually replace the old versions. > Now, if the dynamic loader silently ignores the mprotect() failure on > the main executable, is there much value in exposing a flag in the aux > vectors? It saves a few (one?) mprotect() calls but I don't think it > matters much. Anyway, I don't mind the flag. Saving a few system calls is indeed not an issue, but not being able to use MDWX and PROT_BTI simultaneously was the original problem (service failures). -Topi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE705C4363A for ; Mon, 26 Oct 2020 16:33:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 32552205ED for ; Mon, 26 Oct 2020 16:33:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="NMsV0O32"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ilyK4nri" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 32552205ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cA7QGjXBg4MTz+oMu7jvUVf2+T+dI6uZ8B+4tMyCgcQ=; b=NMsV0O32OjyCPQYCY9rWTJm7H cApZxRvcKD+umdKczOS4RzUR71tb8pdh10j34Tk+Drx2heRiXECNrYyNTf+nHGu09UDg7F/GaqKQe 8Fhc2z2IjFCMMIFKN5604zPerU/89pg+NxRnZIuCTD6YAP3B2HmxY5cPfMFaZ5NEIbCofgqvcc8p/ Ipp1Yc9a77JHa1j5jYzDp8Ih0ohaxqIUKSEf7216HFMoQkpgNt0FM7Whzm/0WdWFjY6ctBBd97D8o V+R5ujCHGeuXAOpV/eRaovkqIEUyXx6u2fKMewZ7+alLVZ9aQN6ViaO26UYy/9SZ66f45dxQvHMqK aIQQXpIyg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kX5Pq-0000SM-MB; Mon, 26 Oct 2020 16:32:14 +0000 Received: from mail-lj1-x241.google.com ([2a00:1450:4864:20::241]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kX5Po-0000RW-4b for linux-arm-kernel@lists.infradead.org; Mon, 26 Oct 2020 16:32:13 +0000 Received: by mail-lj1-x241.google.com with SMTP id m20so10908392ljj.5 for ; Mon, 26 Oct 2020 09:32:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=81sFbAVwzGBYdqNxSKasnwihrmLk5qFx2xW7a2yOVFA=; b=ilyK4nriJpTU+ZsR5TzVNUPI2EICBAV+EtzdIfm0c8gnj9uEXsmcIlrH4esm52Sof5 Z9W3RF1ohEzkLaXmgew/evfyvbKalDRv4wGUlakWh1vwJi+Holvp1iFQavUz7IIINteZ DXfWJqJcuIjSWoOYQlpTF8qMEVNtDnTAy14ZhUAoO6a9psISAvTf4eiGo88ojEk0OVOt DoluxxDfBapo4lA0C3ZHtuZ8p4Qzmm8/zACDjcDNBsCtA/MAYCoro7MwWTB4NW5u7X6u DympMB1G1hMAQePPLbaLhpIDiugGlLowk9LLdoo0k7DAhyw+bMfwmtOEMfW0IrE3liO8 VgLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=81sFbAVwzGBYdqNxSKasnwihrmLk5qFx2xW7a2yOVFA=; b=sp7S5MI3P9Pl9Ou5e8Ywzk6y7ocXvNjDJZ9j7pVGXDt2X5ssi233wBM7zBaRDHZpNl ViUzeRXmhiRa+PS0Nsuyh/NREky/UtWvQA/zFLzknbj9GvSNQIRU4V1RdvKYglaNMEys z+99rt+E+O7ETsUP9w0W46WSuAyZh2aT0PmQ6dhqpYtSxgDn6+OBZOgWdyMP+DOgGO1h TkO0+EAx/ByHPYOICLdnm38S9uSUTbvxwLqaTJ4tVECziyPWKgeWpFafR3CDAjK1DHkf n8wQ//LH6iVmeHFN0LS4gtwqO3SmGWuunTYzpph37Q2hcgd7D2RLi5Md5LSQRy+ksAnE 6MZw== X-Gm-Message-State: AOAM532dDTXkQdhPAkAWb/Uatxi6wynUoKrdDsD2zrQshncA0tICXOgW /wsb4LPw773FEYeLxa3WruI= X-Google-Smtp-Source: ABdhPJyBBZepygrfLeS9IiJiGCwiL99FGQQQhPP2SALgITxO/hNm/8c7G2c/C1WBoQBEiojldvEEsA== X-Received: by 2002:a2e:2ac6:: with SMTP id q189mr5924430ljq.269.1603729930410; Mon, 26 Oct 2020 09:32:10 -0700 (PDT) Received: from [192.168.1.112] (88-114-211-119.elisa-laajakaista.fi. [88.114.211.119]) by smtp.gmail.com with ESMTPSA id q123sm1264618ljq.108.2020.10.26.09.32.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Oct 2020 09:32:09 -0700 (PDT) Subject: Re: BTI interaction between seccomp filters in systemd and glibc mprotect calls, causing service failures To: Catalin Marinas References: <8584c14f-5c28-9d70-c054-7c78127d84ea@arm.com> <20201022075447.GO3819@arm.com> <78464155-f459-773f-d0ee-c5bdbeb39e5d@gmail.com> <202010221256.A4F95FD11@keescook> <20201023090232.GA25736@gaia> <20201026145245.GD3117@gaia> From: Topi Miettinen Message-ID: Date: Mon, 26 Oct 2020 18:31:47 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <20201026145245.GD3117@gaia> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201026_123212_289232_4E696B91 X-CRM114-Status: GOOD ( 33.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Salvatore Mesoraca , systemd-devel@lists.freedesktop.org, Kees Cook , kernel-hardening@lists.openwall.com, Szabolcs Nagy , Will Deacon , "linux-kernel@vger.kernel.org" , Jeremy Linton , Mark Brown , linux-hardening@vger.kernel.org, libc-alpha@sourceware.org, Dave Martin , "linux-arm-kernel@lists.infradead.org" Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 26.10.2020 16.52, Catalin Marinas wrote: > On Sat, Oct 24, 2020 at 02:01:30PM +0300, Topi Miettinen wrote: >> On 23.10.2020 12.02, Catalin Marinas wrote: >>> On Thu, Oct 22, 2020 at 01:02:18PM -0700, Kees Cook wrote: >>>> Regardless, it makes sense to me to have the kernel load the executable >>>> itself with BTI enabled by default. I prefer gaining Catalin's suggested >>>> patch[2]. :) >>> [...] >>>> [2] https://lore.kernel.org/linux-arm-kernel/20201022093104.GB1229@gaia/ >>> >>> I think I first heard the idea at Mark R ;). >>> >>> It still needs glibc changes to avoid the mprotect(), or at least ignore >>> the error. Since this is an ABI change and we don't know which kernels >>> would have it backported, maybe better to still issue the mprotect() but >>> ignore the failure. >> >> What about kernel adding an auxiliary vector as a flag to indicate that BTI >> is supported and recommended by the kernel? Then dynamic loader could use >> that to detect that a) the main executable is BTI protected and there's no >> need to mprotect() it and b) PROT_BTI flag should be added to all PROT_EXEC >> pages. > > We could add a bit to AT_FLAGS, it's always been 0 for Linux. Great! >> In absence of the vector, the dynamic loader might choose to skip doing >> PROT_BTI at all (since the main executable isn't protected anyway either, or >> maybe even the kernel is up-to-date but it knows that it's not recommended >> for some reason, or maybe the kernel is so ancient that it doesn't know >> about BTI). Optionally it could still read the flag from ELF later (for >> compatibility with old kernels) and then do the mprotect() dance, which may >> trip seccomp filters, possibly fatally. > > I think the safest is for the dynamic loader to issue an mprotect() and > ignore the EPERM error. Not all user deployments have this seccomp > filter, so they can still benefit, and user can't tell whether the > kernel change has been backported. But the seccomp filter can be set to kill the process, so that's definitely not the safest way. I think safest is that when the AT_FLAGS bit is seen, ld.so doesn't do any mprotect() calls but instead when mapping the segments, mmap() flags are adjusted to include PROT_BTI, so mprotect() calls are not necessary. If there's no seccomp filter, there's no disadvantage for avoiding the useless mprotect() calls. I'd expect the backported kernel change to include both aux vector and also using PROT_BTI for the main executable. Then the logic would work with backported kernels as well. If there's no aux vector, all bets are off. The kernel could be old and unpatched, even so old that PROT_BTI is not known. Perhaps also in the future there may be new technologies which have replaced BTI and the kernel could want a previous generation ld.so not to try to use BTI, so this could be also indicated with the lack of aux vector. The dynamic loader could still attempt to mprotect() the pages, but that could be fatal. Getting to the point where the error can be ignored means that there's no seccomp filter, at least none set to kill. Perhaps the pain is only temporary, new or patched kernels should eventually replace the old versions. > Now, if the dynamic loader silently ignores the mprotect() failure on > the main executable, is there much value in exposing a flag in the aux > vectors? It saves a few (one?) mprotect() calls but I don't think it > matters much. Anyway, I don't mind the flag. Saving a few system calls is indeed not an issue, but not being able to use MDWX and PROT_BTI simultaneously was the original problem (service failures). -Topi _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 163E8C388F9 for ; Mon, 26 Oct 2020 16:32:32 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id E5576206FB for ; Mon, 26 Oct 2020 16:32:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ilyK4nri" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5576206FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-20274-kernel-hardening=archiver.kernel.org@lists.openwall.com Received: (qmail 22122 invoked by uid 550); 26 Oct 2020 16:32:22 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Received: (qmail 22087 invoked from network); 26 Oct 2020 16:32:21 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=81sFbAVwzGBYdqNxSKasnwihrmLk5qFx2xW7a2yOVFA=; b=ilyK4nriJpTU+ZsR5TzVNUPI2EICBAV+EtzdIfm0c8gnj9uEXsmcIlrH4esm52Sof5 Z9W3RF1ohEzkLaXmgew/evfyvbKalDRv4wGUlakWh1vwJi+Holvp1iFQavUz7IIINteZ DXfWJqJcuIjSWoOYQlpTF8qMEVNtDnTAy14ZhUAoO6a9psISAvTf4eiGo88ojEk0OVOt DoluxxDfBapo4lA0C3ZHtuZ8p4Qzmm8/zACDjcDNBsCtA/MAYCoro7MwWTB4NW5u7X6u DympMB1G1hMAQePPLbaLhpIDiugGlLowk9LLdoo0k7DAhyw+bMfwmtOEMfW0IrE3liO8 VgLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=81sFbAVwzGBYdqNxSKasnwihrmLk5qFx2xW7a2yOVFA=; b=Cl5i2qNkeZw4sWWkGZUUSboi9+AwMs2hp/90s+ODoiWmH4CIwaJ9xUPjdMKhJ45B5p Cl1+CRQwEV5IcmCSw5RqyM67JnPvkllLHBiyawWLiuJVquZkJHbiV673b7Zp08a1fIuw ugaDAwwBF3MjtOGBlQ2UE2kSMogI28xh7FH6M4loDU203DyKwiuK84pxw79TQhDP6qbL 3ieNay9Mbaar23aI+bWudp1D+gi44vw6aHVTWQAQER1wGQ1RGl5cQHMKiDJlR3QULrpl cTbAkmHIu0MZd/yJVPmkzAs3SInDQmpRBFloW+8YHxe5/R4RLg+yVMV5iWsZuP10rFO0 uoAg== X-Gm-Message-State: AOAM532CY4YjId0iygRuyO2XF7bCCyGGXB+3BAIr8RJWJTfurw8A0rTM QtXKYrNCSJgCCkCbsGISGqg= X-Google-Smtp-Source: ABdhPJyBBZepygrfLeS9IiJiGCwiL99FGQQQhPP2SALgITxO/hNm/8c7G2c/C1WBoQBEiojldvEEsA== X-Received: by 2002:a2e:2ac6:: with SMTP id q189mr5924430ljq.269.1603729930410; Mon, 26 Oct 2020 09:32:10 -0700 (PDT) Subject: Re: BTI interaction between seccomp filters in systemd and glibc mprotect calls, causing service failures To: Catalin Marinas Cc: Kees Cook , Szabolcs Nagy , Jeremy Linton , "linux-arm-kernel@lists.infradead.org" , libc-alpha@sourceware.org, systemd-devel@lists.freedesktop.org, "linux-kernel@vger.kernel.org" , Mark Rutland , Mark Brown , Dave Martin , Will Deacon , Salvatore Mesoraca , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org References: <8584c14f-5c28-9d70-c054-7c78127d84ea@arm.com> <20201022075447.GO3819@arm.com> <78464155-f459-773f-d0ee-c5bdbeb39e5d@gmail.com> <202010221256.A4F95FD11@keescook> <20201023090232.GA25736@gaia> <20201026145245.GD3117@gaia> From: Topi Miettinen Message-ID: Date: Mon, 26 Oct 2020 18:31:47 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <20201026145245.GD3117@gaia> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit On 26.10.2020 16.52, Catalin Marinas wrote: > On Sat, Oct 24, 2020 at 02:01:30PM +0300, Topi Miettinen wrote: >> On 23.10.2020 12.02, Catalin Marinas wrote: >>> On Thu, Oct 22, 2020 at 01:02:18PM -0700, Kees Cook wrote: >>>> Regardless, it makes sense to me to have the kernel load the executable >>>> itself with BTI enabled by default. I prefer gaining Catalin's suggested >>>> patch[2]. :) >>> [...] >>>> [2] https://lore.kernel.org/linux-arm-kernel/20201022093104.GB1229@gaia/ >>> >>> I think I first heard the idea at Mark R ;). >>> >>> It still needs glibc changes to avoid the mprotect(), or at least ignore >>> the error. Since this is an ABI change and we don't know which kernels >>> would have it backported, maybe better to still issue the mprotect() but >>> ignore the failure. >> >> What about kernel adding an auxiliary vector as a flag to indicate that BTI >> is supported and recommended by the kernel? Then dynamic loader could use >> that to detect that a) the main executable is BTI protected and there's no >> need to mprotect() it and b) PROT_BTI flag should be added to all PROT_EXEC >> pages. > > We could add a bit to AT_FLAGS, it's always been 0 for Linux. Great! >> In absence of the vector, the dynamic loader might choose to skip doing >> PROT_BTI at all (since the main executable isn't protected anyway either, or >> maybe even the kernel is up-to-date but it knows that it's not recommended >> for some reason, or maybe the kernel is so ancient that it doesn't know >> about BTI). Optionally it could still read the flag from ELF later (for >> compatibility with old kernels) and then do the mprotect() dance, which may >> trip seccomp filters, possibly fatally. > > I think the safest is for the dynamic loader to issue an mprotect() and > ignore the EPERM error. Not all user deployments have this seccomp > filter, so they can still benefit, and user can't tell whether the > kernel change has been backported. But the seccomp filter can be set to kill the process, so that's definitely not the safest way. I think safest is that when the AT_FLAGS bit is seen, ld.so doesn't do any mprotect() calls but instead when mapping the segments, mmap() flags are adjusted to include PROT_BTI, so mprotect() calls are not necessary. If there's no seccomp filter, there's no disadvantage for avoiding the useless mprotect() calls. I'd expect the backported kernel change to include both aux vector and also using PROT_BTI for the main executable. Then the logic would work with backported kernels as well. If there's no aux vector, all bets are off. The kernel could be old and unpatched, even so old that PROT_BTI is not known. Perhaps also in the future there may be new technologies which have replaced BTI and the kernel could want a previous generation ld.so not to try to use BTI, so this could be also indicated with the lack of aux vector. The dynamic loader could still attempt to mprotect() the pages, but that could be fatal. Getting to the point where the error can be ignored means that there's no seccomp filter, at least none set to kill. Perhaps the pain is only temporary, new or patched kernels should eventually replace the old versions. > Now, if the dynamic loader silently ignores the mprotect() failure on > the main executable, is there much value in exposing a flag in the aux > vectors? It saves a few (one?) mprotect() calls but I don't think it > matters much. Anyway, I don't mind the flag. Saving a few system calls is indeed not an issue, but not being able to use MDWX and PROT_BTI simultaneously was the original problem (service failures). -Topi