Linux-api Archive on lore.kernel.org
 help / color / Atom feed
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: "H.J. Lu" <hjl.tools@gmail.com>,
	Dave Hansen <dave.hansen@intel.com>,
	Yu-cheng Yu <yu-cheng.yu@intel.com>,
	the arch/x86 maintainers <x86@kernel.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	LKML <linux-kernel@vger.kernel.org>,
	"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	Linux-MM <linux-mm@kvack.org>,
	linux-arch <linux-arch@vger.kernel.org>,
	Linux API <linux-api@vger.kernel.org>,
	Arnd Bergmann <arnd@arndb.de>, Andy Lutomirski <luto@kernel.org>,
	Balbir Singh <bsingharora@gmail.com>,
	Borislav Petkov <bp@alien8.de>,
	Cyrill Gorcunov <gorcunov@gmail.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Eugene Syromiatnikov <esyr@redhat.com>,
	Florian Weimer <fweimer@redhat.com>, Jann Horn <jannh@google.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Kees Cook <keescook@chromium.org>,
	Mike Kravetz <mike.kravetz@oracle.com>,
	Nadav Amit <nadav.amit@gmail.com>,
	Oleg Nesterov <oleg@redhat.com>, Pavel Machek <pavel@ucw.cz>,
	Randy Dunlap <rdunlap@infradead.org>,
	"Ravi V. Shankar" <ravi.v.shankar@intel.com>,
	Vedvyas Shanbhogue <vedvyas.shanbhogue@intel.com>,
	Dave Martin <Dave.Martin@arm.com>,
	Weijiang Yang <weijiang.yang@intel.com>
Subject: Re: [PATCH v10 01/26] Documentation/x86: Add CET description
Date: Fri, 22 May 2020 18:48:08 +0100
Message-ID: <ee1b03d8-bb0e-57dc-0a6e-c82622f17067@citrix.com> (raw)
In-Reply-To: <20200522164953.GA411971@hirez.programming.kicks-ass.net>

On 22/05/2020 17:49, Peter Zijlstra wrote:
> On Sat, May 16, 2020 at 03:09:22PM +0100, Andrew Cooper wrote:
>
>> Sadly, the same is not true for kernel shadow stacks.
>>
>> SSP is 0 after SYSCALL, SYSENTER and CLRSSBSY, and you've got to be
>> careful to re-establish the shadow stack before a CALL, interrupt or
>> exception tries pushing a word onto the shadow stack at 0xfffffffffffffff8.
> Oh man, I can only imagine the joy that brings to #NM and friends :-(

Establishing a supervisor shadow stack for the first time involves a
large leap of faith, even by usual x86 standards.

You need to have prepared MSR_PL0_SSP with correct mappings and
supervisor tokens, such that when you enable CR4.CET and
MSR_S_CET.SHSTK_EN, your SETSSBSY instruction succeeds at its atomic
"check the token and set the busy bit" shadow stack access.  Any failure
here tends to be a triple fault, and I didn't get around to figuring out
why #DF wasn't taken cleanly.

You also need to have prepared MSR_IST_SSP beforehand with the IST
shadow stack pointers matching any IST configuration in the IDT, lest a
NMI ruins your day on the instruction boundary before SETSSBSY.

A less obvious side effect of these "windows with an SSP of 0" is that
you're now forced to use IST for all non-maskable interrupts/exceptions,
even if you choose not to use SYSCALL, and you no longer need IST to
remove the risks of a userspace privilege escalation, and would prefer
not to use IST because of its problematic reentrancy characteristics.

For anyone counting the number of IST-necessary vectors across all
potential configurations in modern hardware, its #DB, NMI, #DF, #MC,
#VE, #HV, #VC and #SX, and an architectural limit of 7.

There are several other amusing aspects, such as iret-to-self needing to
use call-oriented-programming to keep itself shadow-stack-safe, or the
fact that IRET to user mode doesn't fault if it fails to clear the
supervisor busy bit, instead leaving you to double fault at some point
in the future at the next syscall/interrupt/exception because the stack
is still busy.

~Andrew

P.S. For anyone interested,
https://lore.kernel.org/xen-devel/20200501225838.9866-1-andrew.cooper3@citrix.com/T/#u
for getting supervisor shadow stacks working on Xen, which is far
simpler to manage than Linux.  I do not envy whomever has the fun of
trying to make this work for Linux.

  reply index

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-29 22:07 [PATCH v10 00/26] Control-flow Enforcement: Shadow Stack Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 01/26] Documentation/x86: Add CET description Yu-cheng Yu
2020-04-29 22:53   ` Dave Hansen
2020-04-29 23:02     ` Yu-cheng Yu
2020-05-12 23:20       ` Yu-cheng Yu
2020-05-15 18:39         ` Dave Hansen
2020-05-15 21:33           ` Yu-cheng Yu
2020-05-15 22:43             ` Dave Hansen
2020-05-15 23:29               ` Yu-cheng Yu
2020-05-15 23:56                 ` Dave Hansen
2020-05-16  2:51                   ` H.J. Lu
2020-05-17 23:09                     ` Dave Hansen
2020-05-16  2:53                   ` Yu-cheng Yu
2020-05-18 13:41                     ` Dave Hansen
2020-05-18 14:01                       ` H.J. Lu
2020-05-18 14:26                         ` Dave Hansen
2020-05-18 14:21                       ` Yu-cheng Yu
2020-05-18 23:47                     ` Yu-cheng Yu
2020-05-19  0:38                       ` Dave Hansen
2020-05-19  1:35                         ` Andy Lutomirski
2020-05-20  1:04                           ` Andy Lutomirski
2020-05-29  2:08                             ` Yu-cheng Yu
2020-05-16  0:13               ` Andrew Cooper
2020-05-16  2:37                 ` H.J. Lu
2020-05-16 14:09                   ` Andrew Cooper
2020-05-22 16:49                     ` Peter Zijlstra
2020-05-22 17:48                       ` Andrew Cooper [this message]
2020-04-29 22:07 ` [PATCH v10 02/26] x86/cpufeatures: Add CET CPU feature flags for Control-flow Enforcement Technology (CET) Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 03/26] x86/fpu/xstate: Introduce CET MSR XSAVES supervisor states Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 04/26] x86/cet: Add control-protection fault handler Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 05/26] x86/cet/shstk: Add Kconfig option for user-mode Shadow Stack Yu-cheng Yu
2020-05-07 15:55   ` Dave Hansen
2020-05-07 16:59     ` Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 06/26] x86/mm: Change _PAGE_DIRTY to _PAGE_DIRTY_HW Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 07/26] x86/mm: Remove _PAGE_DIRTY_HW from kernel RO pages Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 08/26] x86/mm: Introduce _PAGE_COW Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 09/26] drm/i915/gvt: Change _PAGE_DIRTY to _PAGE_DIRTY_BITS Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 10/26] x86/mm: Update pte_modify for _PAGE_COW Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 11/26] x86/mm: Update ptep_set_wrprotect() and pmdp_set_wrprotect() for transition from _PAGE_DIRTY_HW to _PAGE_COW Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 12/26] mm: Introduce VM_SHSTK for shadow stack memory Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 13/26] x86/mm: Shadow Stack page fault error checking Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 14/26] x86/mm: Update maybe_mkwrite() for shadow stack Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 15/26] mm: Fixup places that call pte_mkwrite() directly Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 16/26] mm: Add guard pages around a shadow stack Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 17/26] mm/mmap: Add shadow stack pages to memory accounting Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 18/26] mm: Update can_follow_write_pte() for shadow stack Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 19/26] x86/cet/shstk: User-mode shadow stack support Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 20/26] x86/cet/shstk: Handle signals for shadow stack Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 21/26] ELF: UAPI and Kconfig additions for ELF program properties Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 22/26] ELF: Add ELF program property parsing support Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 23/26] ELF: Introduce arch_setup_elf_property() Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 24/26] x86/cet/shstk: ELF header parsing for shadow stack Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 25/26] x86/cet/shstk: Handle thread " Yu-cheng Yu
2020-04-29 22:07 ` [PATCH v10 26/26] x86/cet/shstk: Add arch_prctl functions for " Yu-cheng Yu
2020-05-21 22:42   ` Kees Cook
2020-05-22 17:17     ` Yu-cheng Yu
2020-05-22 17:29       ` Eugene Syromiatnikov
2020-05-22 18:13         ` Yu-cheng Yu
2020-05-21 15:15 ` [PATCH v10 00/26] Control-flow Enforcement: Shadow Stack Josh Poimboeuf
2020-05-21 15:57   ` Yu-cheng Yu
2020-05-21 18:50     ` Josh Poimboeuf
2020-05-21 19:08       ` Yu-cheng Yu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ee1b03d8-bb0e-57dc-0a6e-c82622f17067@citrix.com \
    --to=andrew.cooper3@citrix.com \
    --cc=Dave.Martin@arm.com \
    --cc=arnd@arndb.de \
    --cc=bp@alien8.de \
    --cc=bsingharora@gmail.com \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=esyr@redhat.com \
    --cc=fweimer@redhat.com \
    --cc=gorcunov@gmail.com \
    --cc=hjl.tools@gmail.com \
    --cc=hpa@zytor.com \
    --cc=jannh@google.com \
    --cc=keescook@chromium.org \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=luto@kernel.org \
    --cc=mike.kravetz@oracle.com \
    --cc=mingo@redhat.com \
    --cc=nadav.amit@gmail.com \
    --cc=oleg@redhat.com \
    --cc=pavel@ucw.cz \
    --cc=peterz@infradead.org \
    --cc=ravi.v.shankar@intel.com \
    --cc=rdunlap@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=vedvyas.shanbhogue@intel.com \
    --cc=weijiang.yang@intel.com \
    --cc=x86@kernel.org \
    --cc=yu-cheng.yu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-api Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-api/0 linux-api/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-api linux-api/ https://lore.kernel.org/linux-api \
		linux-api@vger.kernel.org
	public-inbox-index linux-api

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-api


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git