From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF5CBC47420 for ; Thu, 1 Oct 2020 01:10:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4F024206FA for ; Thu, 1 Oct 2020 01:10:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="zm5aCTk/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4F024206FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 75AA16B005C; Wed, 30 Sep 2020 21:10:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 70B176B0062; Wed, 30 Sep 2020 21:10:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 621706B0068; Wed, 30 Sep 2020 21:10:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0081.hostedemail.com [216.40.44.81]) by kanga.kvack.org (Postfix) with ESMTP id 4DDD06B005C for ; Wed, 30 Sep 2020 21:10:35 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 128778249980 for ; Thu, 1 Oct 2020 01:10:35 +0000 (UTC) X-FDA: 77321576430.30.balls49_2b0904727197 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id E60A6180B3C8B for ; Thu, 1 Oct 2020 01:10:34 +0000 (UTC) X-HE-Tag: balls49_2b0904727197 X-Filterd-Recvd-Size: 12906 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Thu, 1 Oct 2020 01:10:34 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 12FC121D46 for ; Thu, 1 Oct 2020 01:10:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1601514633; bh=pHq5pJpYKrfc4SrVwINHvkk5fdj4v2c/IBE6vUVBA24=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=zm5aCTk/sVr/pU9rDMBUrGBdeas9sD/FX7/wNUHtwE0BtXBPIMr772YAGiG2WmRTk J0GTzBvmn3y/QcHtwU2O25ymPjuztFw00NZO0bT0edAkDMJIsctPxgu+mvcWH2TfUZ 9dZQbU7hFRksEJFtFN9zyU70wFrR4YUpV6QEeCA8= Received: by mail-wr1-f41.google.com with SMTP id e16so3739860wrm.2 for ; Wed, 30 Sep 2020 18:10:32 -0700 (PDT) X-Gm-Message-State: AOAM532fImhmx7Qv52aAHg2pYQH2L74WN/LE1fNzsJY/prDJhz3mApp4 o7BJC7/p/sWIyiKDReZTCex2PhspYT6//byJdZzRFA== X-Google-Smtp-Source: ABdhPJyeedVaH1YrWPzgVb/YjwYsrQOcnOiupnFz5QIxyB3dlHWAkhKAMveGNrJuREIyuBmXMaID9uNeO3rW5PCMqpA= X-Received: by 2002:a5d:5281:: with SMTP id c1mr5793346wrv.184.1601514631554; Wed, 30 Sep 2020 18:10:31 -0700 (PDT) MIME-Version: 1.0 References: <99B32E59-CFF2-4756-89BD-AEA0021F355F@amacapital.net> In-Reply-To: From: Andy Lutomirski Date: Wed, 30 Sep 2020 18:10:19 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v13 8/8] x86/vsyscall/64: Fixup Shadow Stack and Indirect Branch Tracking for vsyscall emulation To: "H.J. Lu" Cc: Andy Lutomirski , "Yu, Yu-cheng" , X86 ML , "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , LKML , "open list:DOCUMENTATION" , Linux-MM , linux-arch , Linux API , Arnd Bergmann , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Sep 30, 2020 at 6:01 PM H.J. Lu wrote: > > On Wed, Sep 30, 2020 at 4:44 PM Andy Lutomirski wrote: > > > > On Wed, Sep 30, 2020 at 3:33 PM Yu, Yu-cheng wrote: > > > > > > On 9/29/2020 1:00 PM, Andy Lutomirski wrote: > > > > On Tue, Sep 29, 2020 at 12:57 PM Andy Lutomirski wrote: > > > >> > > > >> On Tue, Sep 29, 2020 at 11:37 AM Yu, Yu-cheng wrote: > > > >>> > > > >>> On 9/28/2020 10:37 AM, Andy Lutomirski wrote: > > > >>>> On Mon, Sep 28, 2020 at 9:59 AM Yu-cheng Yu wrote: > > > >>>>> > > > >>>>> On Fri, 2020-09-25 at 09:51 -0700, Andy Lutomirski wrote: > > > >>>>>>> On Sep 25, 2020, at 9:48 AM, Yu, Yu-cheng wrote: > > > >>>>> + > > > >>>>> + cet = get_xsave_addr(&fpu->state.xsave, XFEATURE_CET_USER); > > > >>>>> + if (!cet) { > > > >>>>> + /* > > > >>>>> + * This is an unlikely case where the task is > > > >>>>> + * CET-enabled, but CET xstate is in INIT. > > > >>>>> + */ > > > >>>>> + WARN_ONCE(1, "CET is enabled, but no xstates"); > > > >>>> > > > >>>> "unlikely" doesn't really cover this. > > > >>>> > > > >>>>> + fpregs_unlock(); > > > >>>>> + goto sigsegv; > > > >>>>> + } > > > >>>>> + > > > >>>>> + if (cet->user_ssp && ((cet->user_ssp + 8) < TASK_SIZE_MAX)) > > > >>>>> + cet->user_ssp += 8; > > > >>>> > > > >>>> This looks buggy. The condition should be "if SHSTK is on, then add 8 > > > >>>> to user_ssp". If the result is noncanonical, then some appropriate > > > >>>> exception should be generated, probably by the FPU restore code -- see > > > >>>> below. You should be checking the SHSTK_EN bit, not SSP. > > > >>> > > > >>> Updated. Is this OK? I will resend the whole series later. > > > >>> > > > >>> Thanks, > > > >>> Yu-cheng > > > >>> > > > >>> ====== > > > >>> > > > >>> From 09803e66dca38d7784e32687d0693550948199ed Mon Sep 17 00:00:00 2001 > > > >>> From: Yu-cheng Yu > > > >>> Date: Thu, 29 Nov 2018 14:15:38 -0800 > > > >>> Subject: [PATCH v13 8/8] x86/vsyscall/64: Fixup Shadow Stack and > > > >>> Indirect Branch > > > >>> Tracking for vsyscall emulation > > > >>> > > > >>> Vsyscall entry points are effectively branch targets. Mark them with > > > >>> ENDBR64 opcodes. When emulating the RET instruction, unwind shadow stack > > > >>> and reset IBT state machine. > > > >>> > > > >>> Signed-off-by: Yu-cheng Yu > > > >>> --- > > > >>> v13: > > > >>> - Check shadow stack address is canonical. > > > >>> - Change from writing to MSRs to writing to CET xstate. > > > >>> > > > >>> arch/x86/entry/vsyscall/vsyscall_64.c | 34 +++++++++++++++++++++++ > > > >>> arch/x86/entry/vsyscall/vsyscall_emu_64.S | 9 ++++++ > > > >>> arch/x86/entry/vsyscall/vsyscall_trace.h | 1 + > > > >>> 3 files changed, 44 insertions(+) > > > >>> > > > >>> diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c > > > >>> b/arch/x86/entry/vsyscall/vsyscall_64.c > > > >>> index 44c33103a955..30b166091d46 100644 > > > >>> --- a/arch/x86/entry/vsyscall/vsyscall_64.c > > > >>> +++ b/arch/x86/entry/vsyscall/vsyscall_64.c > > > >>> @@ -38,6 +38,9 @@ > > > >>> #include > > > >>> #include > > > >>> #include > > > >>> +#include > > > >>> +#include > > > >>> +#include > > > >>> > > > >>> #define CREATE_TRACE_POINTS > > > >>> #include "vsyscall_trace.h" > > > >>> @@ -286,6 +289,44 @@ bool emulate_vsyscall(unsigned long error_code, > > > >>> /* Emulate a ret instruction. */ > > > >>> regs->ip = caller; > > > >>> regs->sp += 8; > > > >>> + > > > >>> +#ifdef CONFIG_X86_CET > > > >>> + if (tsk->thread.cet.shstk_size || tsk->thread.cet.ibt_enabled) { > > > >>> + struct cet_user_state *cet; > > > >>> + struct fpu *fpu; > > > >>> + > > > >>> + fpu = &tsk->thread.fpu; > > > >>> + fpregs_lock(); > > > >>> + > > > >>> + if (!test_thread_flag(TIF_NEED_FPU_LOAD)) { > > > >>> + copy_fpregs_to_fpstate(fpu); > > > >>> + set_thread_flag(TIF_NEED_FPU_LOAD); > > > >>> + } > > > >>> + > > > >>> + cet = get_xsave_addr(&fpu->state.xsave, XFEATURE_CET_USER); > > > >>> + if (!cet) { > > > >>> + /* > > > >>> + * This should not happen. The task is > > > >>> + * CET-enabled, but CET xstate is in INIT. > > > >>> + */ > > > >> > > > >> Can the comment explain better, please? I would say something like: > > > >> > > > >> If the kernel thinks this task has CET enabled (because > > > >> tsk->thread.cet has one of the features enabled), then the > > > >> corresponding bits must also be set in the CET XSAVES region. If the > > > >> CET XSAVES region is in the INIT state, then the kernel's concept of > > > >> the task's CET state is corrupt. > > > >> > > > >>> + WARN_ONCE(1, "CET is enabled, but no xstates"); > > > >>> + fpregs_unlock(); > > > >>> + goto sigsegv; > > > >>> + } > > > >>> + > > > >>> + if (cet->user_cet & CET_SHSTK_EN) { > > > >>> + if (cet->user_ssp && (cet->user_ssp + 8 < TASK_SIZE_MAX)) > > > >>> + cet->user_ssp += 8; > > > >>> + } > > > >> > > > >> This makes so sense to me. Also, the vsyscall emulation code is > > > >> intended to be as rigid as possible to minimize the chance that it > > > >> gets used as an exploit gadget. So we should not silently corrupt > > > >> anything. Moreover, this code seems quite dangerous -- you've created > > > >> a gadget that does RET without actually verifying the SHSTK token. If > > > >> SHSTK and some form of strong indirect branch/call CFI is in use, then > > > >> the existance of a CFI-bypassing return primitive at a fixed address > > > >> seems quite problematic. > > > >> > > > >> So I think you need to write a function that reasonably accurately > > > >> emulates a usermode RET. > > > >> > > > > > > > > For what it's worth, I think there is an alternative. If you all > > > > (userspace people, etc) can come up with a credible way for a user > > > > program to statically declare that it doesn't need vsyscalls, then we > > > > could make SHSTK depend on *that*, and we could avoid this mess. This > > > > breaks orthogonality, but it's probably a decent outcome. > > > > > > > > > > Would an arch_prctl(DISABLE_VSYSCALL) work? The kernel then sets a > > > thread flag, and in emulate_vsyscall(), checks the flag. > > > > > > When CET is enabled, ld-linux will do DISABLE_VSYSCALL. > > > > > > How is that? > > > > Backwards, no? Presumably vsyscall needs to be disabled before or > > concurrently with CET being enabled, not after. > > > > I think the solution of making vsyscall emulation work correctly with > > CET is going to be better and possibly more straightforward. > > > > We can do > > 1. Add ARCH_X86_DISABLE_VSYSCALL to disable the vsyscall page. > 2. If CPU supports CET and the program is CET enabled: > a. Disable the vsyscall page. > b. Pass control to user. > c. Enable the vsyscall page when ARCH_X86_CET_DISABLE is called. > > So when control is passed from kernel to user, the vsyscall page is > disabled if the program > is CET enabled. Let me say this one more time: If we have a per-process vsyscall disable control and a per-process CET control, we are going to keep those settings orthogonal. I'm willing to entertain an option in which enabling SHSTK without also disabling vsyscalls is disallowed, We are *not* going to have any CET flags magically disable vsyscalls, though, and we are not going to have a situation where disabling vsyscalls on process startup requires enabling SHSTK. Any possible static vsyscall controls (and CET controls, for that matter) also need to come with some explanation of whether they are properties set on the ELF loader, the ELF program being loaded, or both. And this explanation needs to cover what happens when old binaries link against new libc versions and vice versa. A new CET-enabled binary linked against old libc running on a new kernel that is expected to work on a non-CET CPU MUST work on a CET CPU, too. Right now, literally the only thing preventing vsyscall emulation from coexisting with SHSTK is that the implementation eeds work. So your proposal is rejected. Sorry.