From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33886C47420 for ; Thu, 1 Oct 2020 17:26:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A9E8520897 for ; Thu, 1 Oct 2020 17:26:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1601573204; bh=ZeucOxEGwYAwOisFa15yd0vUgz9q8YmfYAwL5fw3YaY=; h=References:In-Reply-To:From:Date:Subject:To:Cc:List-ID:From; b=GT9DPkHzHYj/mevgEge2V7FgEkLwYTiSF2s3fsMNh5tI+mB06MocW4w9W/ydUcfzN zodgMTkBtRkID0mD8F6nsg7JC9iSQd2GZz+LdWv8h+bQBXkFlR2OiNlsRqsOlEHHNd TQs48fBa8k64tOnRD+SyCxV4rEPhXezJ0GNRN3cY= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732702AbgJAR0n (ORCPT ); Thu, 1 Oct 2020 13:26:43 -0400 Received: from mail.kernel.org ([198.145.29.99]:53842 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732096AbgJAR0m (ORCPT ); Thu, 1 Oct 2020 13:26:42 -0400 Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AF55420897 for ; Thu, 1 Oct 2020 17:26:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1601573201; bh=ZeucOxEGwYAwOisFa15yd0vUgz9q8YmfYAwL5fw3YaY=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=0DoMHNw3NEBGDy+zyfTNx4WucL0t18JLri9m/nPIS7Nfby1AOn3ObVnpBj7m/fVyX VleadcapZkbv5SJori6McqCdpJ7XL3JyK1yh5kBMBTddPMQSV3eYe7/JRT/MO8Nrxz VTNBpDK+94LNjx3yvLbFTcRtOW9/2ZDBEK99T8wY= Received: by mail-wm1-f46.google.com with SMTP id y15so4008910wmi.0 for ; Thu, 01 Oct 2020 10:26:40 -0700 (PDT) X-Gm-Message-State: AOAM5330XLbAfy18w2p8eoDGJKjoDZy4aIda01Vw/LgflyNSlohe8BEI o+3+jROyXvY19+Dq1llf9NFatsLsEHDbI+ZqO3FXyw== X-Google-Smtp-Source: ABdhPJzsDHc4cgbeqbrVUch9Umv63EOwhe6DxGcuS92ZNYw+C2Bn57ahMoGL2VU45fCJ3LrAwL1irG/U+dQg1/42e5c= X-Received: by 2002:a1c:63c1:: with SMTP id x184mr1111951wmb.138.1601573199240; Thu, 01 Oct 2020 10:26:39 -0700 (PDT) MIME-Version: 1.0 References: <99B32E59-CFF2-4756-89BD-AEA0021F355F@amacapital.net> <79d1e67d-2394-1ce6-3bad-cce24ba792bd@intel.com> In-Reply-To: <79d1e67d-2394-1ce6-3bad-cce24ba792bd@intel.com> From: Andy Lutomirski Date: Thu, 1 Oct 2020 10:26:27 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v13 8/8] x86/vsyscall/64: Fixup Shadow Stack and Indirect Branch Tracking for vsyscall emulation To: "Yu, Yu-cheng" Cc: Andy Lutomirski , "H.J. Lu" , X86 ML , "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , LKML , "open list:DOCUMENTATION" , Linux-MM , linux-arch , Linux API , Arnd Bergmann , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-arch@vger.kernel.org On Thu, Oct 1, 2020 at 9:51 AM Yu, Yu-cheng wrote: > > On 9/30/2020 6:10 PM, Andy Lutomirski wrote: > > On Wed, Sep 30, 2020 at 6:01 PM H.J. Lu wrote: > >> > >> On Wed, Sep 30, 2020 at 4:44 PM Andy Lutomirski wrote: > > [...] > > >>>>>>> From 09803e66dca38d7784e32687d0693550948199ed Mon Sep 17 00:00:00 2001 > >>>>>>> From: Yu-cheng Yu > >>>>>>> Date: Thu, 29 Nov 2018 14:15:38 -0800 > >>>>>>> Subject: [PATCH v13 8/8] x86/vsyscall/64: Fixup Shadow Stack and > >>>>>>> Indirect Branch > >>>>>>> Tracking for vsyscall emulation > >>>>>>> > >>>>>>> Vsyscall entry points are effectively branch targets. Mark them with > >>>>>>> ENDBR64 opcodes. When emulating the RET instruction, unwind shadow stack > >>>>>>> and reset IBT state machine. > >>>>>>> > >>>>>>> Signed-off-by: Yu-cheng Yu > >>>>>>> --- > >>>>>>> v13: > >>>>>>> - Check shadow stack address is canonical. > >>>>>>> - Change from writing to MSRs to writing to CET xstate. > >>>>>>> > >>>>>>> arch/x86/entry/vsyscall/vsyscall_64.c | 34 +++++++++++++++++++++++ > >>>>>>> arch/x86/entry/vsyscall/vsyscall_emu_64.S | 9 ++++++ > >>>>>>> arch/x86/entry/vsyscall/vsyscall_trace.h | 1 + > >>>>>>> 3 files changed, 44 insertions(+) > >>>>>>> > >>>>>>> diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c > >>>>>>> b/arch/x86/entry/vsyscall/vsyscall_64.c > >>>>>>> index 44c33103a955..30b166091d46 100644 > >>>>>>> --- a/arch/x86/entry/vsyscall/vsyscall_64.c > >>>>>>> +++ b/arch/x86/entry/vsyscall/vsyscall_64.c > >>>>>>> @@ -38,6 +38,9 @@ > >>>>>>> #include > >>>>>>> #include > >>>>>>> #include > >>>>>>> +#include > >>>>>>> +#include > >>>>>>> +#include > >>>>>>> > >>>>>>> #define CREATE_TRACE_POINTS > >>>>>>> #include "vsyscall_trace.h" > >>>>>>> @@ -286,6 +289,44 @@ bool emulate_vsyscall(unsigned long error_code, > >>>>>>> /* Emulate a ret instruction. */ > >>>>>>> regs->ip = caller; > >>>>>>> regs->sp += 8; > >>>>>>> + > >>>>>>> +#ifdef CONFIG_X86_CET > >>>>>>> + if (tsk->thread.cet.shstk_size || tsk->thread.cet.ibt_enabled) { > >>>>>>> + struct cet_user_state *cet; > >>>>>>> + struct fpu *fpu; > >>>>>>> + > >>>>>>> + fpu = &tsk->thread.fpu; > >>>>>>> + fpregs_lock(); > >>>>>>> + > >>>>>>> + if (!test_thread_flag(TIF_NEED_FPU_LOAD)) { > >>>>>>> + copy_fpregs_to_fpstate(fpu); > >>>>>>> + set_thread_flag(TIF_NEED_FPU_LOAD); > >>>>>>> + } > >>>>>>> + > >>>>>>> + cet = get_xsave_addr(&fpu->state.xsave, XFEATURE_CET_USER); > >>>>>>> + if (!cet) { > >>>>>>> + /* > >>>>>>> + * This should not happen. The task is > >>>>>>> + * CET-enabled, but CET xstate is in INIT. > >>>>>>> + */ > >>>>>> > [...] > >>>>>> > >>>>> > >>>>> For what it's worth, I think there is an alternative. If you all > >>>>> (userspace people, etc) can come up with a credible way for a user > >>>>> program to statically declare that it doesn't need vsyscalls, then we > >>>>> could make SHSTK depend on *that*, and we could avoid this mess. This > >>>>> breaks orthogonality, but it's probably a decent outcome. > >>>>> > >>>> > >>>> Would an arch_prctl(DISABLE_VSYSCALL) work? The kernel then sets a > >>>> thread flag, and in emulate_vsyscall(), checks the flag. > >>>> > >>>> When CET is enabled, ld-linux will do DISABLE_VSYSCALL. > >>>> > >>>> How is that? > >>> > >>> Backwards, no? Presumably vsyscall needs to be disabled before or > >>> concurrently with CET being enabled, not after. > >>> > >>> I think the solution of making vsyscall emulation work correctly with > >>> CET is going to be better and possibly more straightforward. > >>> > >> > >> We can do > >> > >> 1. Add ARCH_X86_DISABLE_VSYSCALL to disable the vsyscall page. > >> 2. If CPU supports CET and the program is CET enabled: > >> a. Disable the vsyscall page. > >> b. Pass control to user. > >> c. Enable the vsyscall page when ARCH_X86_CET_DISABLE is called. > >> > >> So when control is passed from kernel to user, the vsyscall page is > >> disabled if the program > >> is CET enabled. > > > > Let me say this one more time: > > > > If we have a per-process vsyscall disable control and a per-process > > CET control, we are going to keep those settings orthogonal. I'm > > willing to entertain an option in which enabling SHSTK without also > > disabling vsyscalls is disallowed, We are *not* going to have any CET > > flags magically disable vsyscalls, though, and we are not going to > > have a situation where disabling vsyscalls on process startup requires > > enabling SHSTK. > > > > Any possible static vsyscall controls (and CET controls, for that > > matter) also need to come with some explanation of whether they are > > properties set on the ELF loader, the ELF program being loaded, or > > both. And this explanation needs to cover what happens when old > > binaries link against new libc versions and vice versa. A new > > CET-enabled binary linked against old libc running on a new kernel > > that is expected to work on a non-CET CPU MUST work on a CET CPU, too. > > > > Right now, literally the only thing preventing vsyscall emulation from > > coexisting with SHSTK is that the implementation eeds work. > > > > So your proposal is rejected. Sorry. > > > I think, even with shadow stack/ibt enabled, we can still allow XONLY > without too much mess. > > What about this? > > Thanks, > Yu-cheng > > ====== > > diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c > b/arch/x86/entry/vsyscall/vsyscall_64.c > index 8b0b32ac7791..d39da0a15521 100644 > --- a/arch/x86/entry/vsyscall/vsyscall_64.c > +++ b/arch/x86/entry/vsyscall/vsyscall_64.c > @@ -48,16 +48,16 @@ > static enum { EMULATE, XONLY, NONE } vsyscall_mode __ro_after_init = > #ifdef CONFIG_LEGACY_VSYSCALL_NONE > NONE; > -#elif defined(CONFIG_LEGACY_VSYSCALL_XONLY) > +#elif defined(CONFIG_LEGACY_VSYSCALL_XONLY) || defined(CONFIG_X86_CET) > XONLY; > -#else > +#else > EMULATE; > #endif I don't get it. First, you can't do any of this based on config -- it must be runtime. Second, and more importantly, I don't see how XONLY helps at all. The (non-executable) text that's exposed to user code in EMULATE mode is trivial to get right with CET -- your code already handles it. It's the emulation code (that runs identically in EMULATE and XONLY mode) that's tricky.