linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
To: "bp@alien8.de" <bp@alien8.de>
Cc: "david@redhat.com" <david@redhat.com>,
	"bsingharora@gmail.com" <bsingharora@gmail.com>,
	"hpa@zytor.com" <hpa@zytor.com>,
	"Syromiatnikov, Eugene" <esyr@redhat.com>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"rdunlap@infradead.org" <rdunlap@infradead.org>,
	"keescook@chromium.org" <keescook@chromium.org>,
	"Eranian, Stephane" <eranian@google.com>,
	"kirill.shutemov@linux.intel.com"
	<kirill.shutemov@linux.intel.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"fweimer@redhat.com" <fweimer@redhat.com>,
	"nadav.amit@gmail.com" <nadav.amit@gmail.com>,
	"jannh@google.com" <jannh@google.com>,
	"dethoma@microsoft.com" <dethoma@microsoft.com>,
	"kcc@google.com" <kcc@google.com>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	"pavel@ucw.cz" <pavel@ucw.cz>,
	"oleg@redhat.com" <oleg@redhat.com>,
	"hjl.tools@gmail.com" <hjl.tools@gmail.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"Yang, Weijiang" <weijiang.yang@intel.com>,
	"Lutomirski, Andy" <luto@kernel.org>,
	"jamorris@linux.microsoft.com" <jamorris@linux.microsoft.com>,
	"arnd@arndb.de" <arnd@arndb.de>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"Schimpe, Christina" <christina.schimpe@intel.com>,
	"mike.kravetz@oracle.com" <mike.kravetz@oracle.com>,
	"debug@rivosinc.com" <debug@rivosinc.com>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"x86@kernel.org" <x86@kernel.org>,
	"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
	"john.allen@amd.com" <john.allen@amd.com>,
	"rppt@kernel.org" <rppt@kernel.org>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-api@vger.kernel.org" <linux-api@vger.kernel.org>,
	"gorcunov@gmail.com" <gorcunov@gmail.com>
Subject: Re: [PATCH v7 28/41] x86: Introduce userspace API for shadow stack
Date: Thu, 9 Mar 2023 16:56:37 +0000	[thread overview]
Message-ID: <a4dd415ac908450b09b9abbd4421a9132b3c34cc.camel@intel.com> (raw)
In-Reply-To: <20230309125739.GCZAnXw5T1dfzwtqh8@fat_crate.local>

On Thu, 2023-03-09 at 13:57 +0100, Borislav Petkov wrote:
> So this all sounds weird. Especially from a user point of view.
> 
> Now let's imagine there's a Linux user called Boris and he goes and
> buys
> a CPU which supports shadow stack, gets a distro which has shadow
> stack
> enabled. All good.
> 
> Now, at some point he loads a program which pulls in an old library
> which hasn't been enabled for shadow stack yet.
> 
> In the name of not breaking stuff, his glibc is configured in
> permissive
> mode by default so that program loads and shadow stack for it is
> disabled.
> 
> And Boris doesn't even know and continues on his merry way thinking
> that
> he has all that cool ROP protection.

There is a proc that shows if shadow stack is enabled in a thread. It
does indeed come later in the series.

> 
> So where is the knob that says, "disable permissive mode"?

glibc has an environment variable that can change the loader's
behavior. There is also a compile time config for the default mode. But
this "permissive mode" is a glibc thing. The kernel doesn't implement
it per-se, just provides building blocks.

> 
> Or at least where does the user get a warning saying, "hey, this app
> doesn't do shadow stack and we disabled it for ya so that it can
> still
> work"?
> 
> Or am I way off?

I don't think so. The whole "when to enable shadow stack" question is
thornier than it might seem though, and what we have here is based on
some trade offs in the details.

> 
> I hope you're catching my drift. Because if there's no enforcement of
> shstk and we do this permissive mode by default, this whole overhead
> is
> just a unnecessary nuisance...

In the existing glibc patches, and this is highly related to glibc
behavior because the decisions around enabling and locking have been
pushed there, there are two reasons why shadow stack would get disabled
on an supporting executable after it gets enabled.
1. An executable is loaded and one of the shared objects (the ones that
come out of ldd) does not support shadow stack
2. An executable is loaded in permissive mode, and much later during
execution dlopen()s a DSO that does not support shadow stack.

One of the challenges with enabling shadow stack is you only start
recording the shadow stack history when you enable it. If you enable it
at some point, and then return from that function you underflow the
shadow stack and get a violation. So if the shadow stack will be
locked, it has to be enabled at the earliest point it might return to
at some point (for example after returning from main()).

So in 1, the existing logic of glibc is to enable shadow stack at the
very beginning of the loader. Then go through the whole loading/linking
process. If problems are found, disable shadow stack. If no problems
are found, then lock it.

I've wondered if this super early glibc enabling behavior is really
needed and if they could enable it after processing the linked
libraries in the elf. Then save the work of enabling and disabling
shadow stack for situations that don't support it. To me this is the
big wart in the whole thing, but I don't think the kernel can help
resolve it. If glibc can enable later, then we can combine the locking
and enabling into a single operation. But it only saves a syscall and
it might prevent some other libc that needs to do things like glibc
does currently, from being able to make it work at all.

In 2, the enabling happens like normal and the locking is skipped, so
that shadow stack can be enabled during a dlopen(). But glibc
permissive mode promises more than it delivers. Since it can only
disable shadow stack per-thread, it leaves the other threads enabled.
Making a working permissive mode is sort of an unsolved problem. There
are some proposals to make it work in just userspace, and some that
would need additional kernel support. If you are interested I can go
into why per-process disabling is not straightforward.

So the locking is needed for the basic support in 1 and the weak
permissive mode in 2 uses it. I am considering this series to support
1, but people may end up using 2 to get some permissive-ness. In
general the idea of this API is to push the enabling decisions into
userspace because that is where the information for making the decision
is known. We previously tried to add some batch operations to improve
the performance, but tglx had suggested to start with something simple.
So we end up with this simple composable API.


  reply	other threads:[~2023-03-09 17:03 UTC|newest]

Thread overview: 159+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-02-27 22:29 [PATCH v7 00/41] Shadow stacks for userspace Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 01/41] Documentation/x86: Add CET shadow stack description Rick Edgecombe
2023-03-01 14:21   ` Szabolcs Nagy
2023-03-01 14:38     ` Szabolcs Nagy
2023-03-01 18:07     ` Edgecombe, Rick P
2023-03-01 18:32       ` Edgecombe, Rick P
2023-03-02 16:34         ` szabolcs.nagy
2023-03-03 22:35           ` Edgecombe, Rick P
2023-03-06 16:20             ` szabolcs.nagy
2023-03-06 16:31               ` Florian Weimer
2023-03-06 18:08                 ` Edgecombe, Rick P
2023-03-07 13:03                   ` szabolcs.nagy
2023-03-07 14:00                     ` Florian Weimer
2023-03-07 16:14                       ` Szabolcs Nagy
2023-03-06 18:05               ` Edgecombe, Rick P
2023-03-06 20:31                 ` Liang, Kan
2023-03-02 16:14       ` szabolcs.nagy
2023-03-02 21:17         ` Edgecombe, Rick P
2023-03-03 16:30           ` szabolcs.nagy
2023-03-03 16:57             ` H.J. Lu
2023-03-03 17:39               ` szabolcs.nagy
2023-03-03 17:50                 ` H.J. Lu
2023-03-03 17:41             ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 02/41] x86/shstk: Add Kconfig option for shadow stack Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 03/41] x86/cpufeatures: Add CPU feature flags for shadow stacks Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 04/41] x86/cpufeatures: Enable CET CR4 bit for shadow stack Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 05/41] x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 06/41] x86/fpu: Add helper for modifying xstate Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 07/41] x86: Move control protection handler to separate file Rick Edgecombe
2023-03-01 15:38   ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 08/41] x86/shstk: Add user control-protection fault handler Rick Edgecombe
2023-03-01 18:06   ` Borislav Petkov
2023-03-01 18:14     ` Edgecombe, Rick P
2023-03-01 18:37       ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 09/41] x86/mm: Remove _PAGE_DIRTY from kernel RO pages Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 10/41] x86/mm: Move pmd_write(), pud_write() up in the file Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 11/41] mm: Introduce pte_mkwrite_kernel() Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 12/41] s390/mm: Introduce pmd_mkwrite_kernel() Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 13/41] mm: Make pte_mkwrite() take a VMA Rick Edgecombe
2023-03-01  7:03   ` Christophe Leroy
2023-03-01  8:16     ` David Hildenbrand
2023-03-02 12:19   ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 14/41] x86/mm: Introduce _PAGE_SAVED_DIRTY Rick Edgecombe
2023-03-02 12:48   ` Borislav Petkov
2023-03-02 17:01     ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 15/41] x86/mm: Update ptep/pmdp_set_wrprotect() for _PAGE_SAVED_DIRTY Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 16/41] x86/mm: Start actually marking _PAGE_SAVED_DIRTY Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 17/41] mm: Move VM_UFFD_MINOR_BIT from 37 to 38 Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 18/41] mm: Introduce VM_SHADOW_STACK for shadow stack memory Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 19/41] x86/mm: Check shadow stack page fault errors Rick Edgecombe
2023-03-03 14:00   ` Borislav Petkov
2023-03-03 14:39     ` Dave Hansen
2023-02-27 22:29 ` [PATCH v7 20/41] x86/mm: Teach pte_mkwrite() about stack memory Rick Edgecombe
2023-03-03 15:37   ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 21/41] mm: Add guard pages around a shadow stack Rick Edgecombe
2023-03-06  8:08   ` Borislav Petkov
2023-03-07  1:29     ` Edgecombe, Rick P
2023-03-07 10:32       ` Borislav Petkov
2023-03-07 10:44         ` David Hildenbrand
2023-03-08 22:48           ` Edgecombe, Rick P
2023-03-17 17:09   ` Deepak Gupta
2023-02-27 22:29 ` [PATCH v7 22/41] mm/mmap: Add shadow stack pages to memory accounting Rick Edgecombe
2023-03-06 13:01   ` Borislav Petkov
2023-03-06 18:11     ` Edgecombe, Rick P
2023-03-06 18:16       ` Borislav Petkov
2023-03-07 10:42   ` David Hildenbrand
2023-03-17 17:12   ` Deepak Gupta
2023-03-17 17:16     ` Dave Hansen
2023-03-17 17:28       ` Deepak Gupta
2023-03-17 17:42         ` Edgecombe, Rick P
2023-03-17 19:26           ` Deepak Gupta
2023-02-27 22:29 ` [PATCH v7 23/41] mm: Re-introduce vm_flags to do_mmap() Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 24/41] mm: Don't allow write GUPs to shadow stack memory Rick Edgecombe
2023-03-06 13:10   ` Borislav Petkov
2023-03-06 18:15     ` Andy Lutomirski
2023-03-06 18:33       ` Edgecombe, Rick P
2023-03-06 18:57         ` Andy Lutomirski
2023-03-07  1:47           ` Edgecombe, Rick P
2023-03-17 17:05   ` Deepak Gupta
2023-02-27 22:29 ` [PATCH v7 25/41] x86/mm: Introduce MAP_ABOVE4G Rick Edgecombe
2023-03-06 18:09   ` Borislav Petkov
2023-03-07  1:10     ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 26/41] mm: Warn on shadow stack memory in wrong vma Rick Edgecombe
2023-03-08  8:53   ` Borislav Petkov
2023-03-08 23:36     ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 27/41] x86/mm: Warn if create Write=0,Dirty=1 with raw prot Rick Edgecombe
2023-02-27 22:54   ` Kees Cook
2023-03-08  9:23   ` Borislav Petkov
2023-03-08 23:35     ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 28/41] x86: Introduce userspace API for shadow stack Rick Edgecombe
2023-03-08 10:27   ` Borislav Petkov
2023-03-08 23:32     ` Edgecombe, Rick P
2023-03-09 12:57       ` Borislav Petkov
2023-03-09 16:56         ` Edgecombe, Rick P [this message]
2023-03-09 23:51           ` Borislav Petkov
2023-03-10  1:13             ` Edgecombe, Rick P
2023-03-10  2:03               ` H.J. Lu
2023-03-10 20:00                 ` H.J. Lu
2023-03-10 20:27                   ` Edgecombe, Rick P
2023-03-10 20:43                     ` H.J. Lu
2023-03-10 21:01                       ` Edgecombe, Rick P
2023-03-10 11:40               ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 29/41] x86/shstk: Add user-mode shadow stack support Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 30/41] x86/shstk: Handle thread shadow stack Rick Edgecombe
2023-03-02 17:34   ` Szabolcs Nagy
2023-03-02 21:48     ` Edgecombe, Rick P
2023-03-08 15:26   ` Borislav Petkov
2023-03-08 20:03     ` Edgecombe, Rick P
2023-03-09 14:12       ` Borislav Petkov
2023-03-09 16:59         ` Edgecombe, Rick P
2023-03-09 17:04           ` Borislav Petkov
2023-03-09 20:29             ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 31/41] x86/shstk: Introduce routines modifying shstk Rick Edgecombe
2023-03-09 16:48   ` Borislav Petkov
2023-03-09 17:03     ` Edgecombe, Rick P
2023-03-09 17:22       ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 32/41] x86/shstk: Handle signals for shadow stack Rick Edgecombe
2023-03-09 17:02   ` Borislav Petkov
2023-03-09 17:16     ` Edgecombe, Rick P
2023-03-09 23:35       ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 33/41] x86/shstk: Introduce map_shadow_stack syscall Rick Edgecombe
2023-03-02 17:22   ` Szabolcs Nagy
2023-03-02 21:21     ` Edgecombe, Rick P
2023-03-09 18:55     ` Deepak Gupta
2023-03-09 19:39       ` Edgecombe, Rick P
2023-03-09 21:08         ` Deepak Gupta
2023-03-10  0:14           ` Edgecombe, Rick P
2023-03-10 21:00             ` Deepak Gupta
2023-03-10 21:43               ` Edgecombe, Rick P
2023-03-16 20:07                 ` Deepak Gupta
2023-03-14  7:19       ` Mike Rapoport
2023-03-16 19:30         ` Deepak Gupta
2023-03-20 11:35           ` Szabolcs Nagy
2023-03-10 16:11   ` Borislav Petkov
2023-03-10 17:12     ` Edgecombe, Rick P
2023-03-10 20:05       ` Borislav Petkov
2023-03-10 20:19         ` Edgecombe, Rick P
2023-03-10 20:26           ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 34/41] x86/shstk: Support WRSS for userspace Rick Edgecombe
2023-03-10 16:44   ` Borislav Petkov
2023-03-10 17:16     ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 35/41] x86: Expose thread features in /proc/$PID/status Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 36/41] x86/shstk: Wire in shadow stack interface Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 37/41] selftests/x86: Add shadow stack test Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 38/41] x86/fpu: Add helper for initing features Rick Edgecombe
2023-03-11 12:54   ` Borislav Petkov
2023-03-13  2:45     ` Edgecombe, Rick P
2023-03-13 11:03       ` Borislav Petkov
2023-03-13 16:10         ` Edgecombe, Rick P
2023-03-13 17:10           ` Borislav Petkov
2023-03-13 23:31             ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 39/41] x86: Add PTRACE interface for shadow stack Rick Edgecombe
2023-03-11 15:06   ` Borislav Petkov
2023-03-13  2:53     ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 40/41] x86/shstk: Add ARCH_SHSTK_UNLOCK Rick Edgecombe
2023-03-11 15:11   ` Borislav Petkov
2023-03-13  3:04     ` Edgecombe, Rick P
2023-03-13 11:05       ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 41/41] x86/shstk: Add ARCH_SHSTK_STATUS Rick Edgecombe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a4dd415ac908450b09b9abbd4421a9132b3c34cc.camel@intel.com \
    --to=rick.p.edgecombe@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=andrew.cooper3@citrix.com \
    --cc=arnd@arndb.de \
    --cc=bp@alien8.de \
    --cc=bsingharora@gmail.com \
    --cc=christina.schimpe@intel.com \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=david@redhat.com \
    --cc=debug@rivosinc.com \
    --cc=dethoma@microsoft.com \
    --cc=eranian@google.com \
    --cc=esyr@redhat.com \
    --cc=fweimer@redhat.com \
    --cc=gorcunov@gmail.com \
    --cc=hjl.tools@gmail.com \
    --cc=hpa@zytor.com \
    --cc=jamorris@linux.microsoft.com \
    --cc=jannh@google.com \
    --cc=john.allen@amd.com \
    --cc=kcc@google.com \
    --cc=keescook@chromium.org \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=luto@kernel.org \
    --cc=mike.kravetz@oracle.com \
    --cc=mingo@redhat.com \
    --cc=nadav.amit@gmail.com \
    --cc=oleg@redhat.com \
    --cc=pavel@ucw.cz \
    --cc=peterz@infradead.org \
    --cc=rdunlap@infradead.org \
    --cc=rppt@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=weijiang.yang@intel.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).