Re: [PATCH v11 25/25] x86/cet/shstk: Add arch_prctl functions for shadow stack

[Dave Martin <Dave.Martin@arm.com>]

On Tue, Sep 01, 2020 at 11:11:37AM -0700, Dave Hansen wrote:
> On 9/1/20 10:45 AM, Andy Lutomirski wrote:
> >>> For arm64 (and sparc etc.) we continue to use the regular mmap/mprotect
> >>> family of calls.  One or two additional arch-specific mmap flags are
> >>> sufficient for now.
> >>>
> >>> Is x86 definitely not going to fit within those calls?
> >> That can work for x86.  Andy, what if we create PROT_SHSTK, which can
> >> been seen only from the user.  Once in kernel, it is translated to
> >> VM_SHSTK.  One question for mremap/mprotect is, do we allow a normal
> >> data area to become shadow stack?
> > I'm unconvinced that we want to use a somewhat precious PROT_ or VM_
> > bit for this.  Using a flag bit makes sense if we expect anyone to
> > ever map an fd or similar as a shadow stack, but that seems a bit odd
> > in the first place.  To me, it seems more logical for a shadow stack
> > to be a special sort of mapping with a special vm_ops, not a normal
> > mapping with a special flag set.  Although I realize that we want
> > shadow stacks to work like anonymous memory with respect to fork().
> > Dave?
> 
> I actually don't like the idea of *creating* mappings much.
> 
> I think the pkey model has worked out pretty well where we separate
> creating the mapping from doing something *to* it, like changing
> protections.  For instance, it would be nice if we could preserve things
> like using hugetlbfs or heck even doing KSM for shadow stacks.
> 
> If we're *creating* mappings, we've pretty much ruled out things like
> hugetlbfs.
> 
> Something like mprotect_shstk() would allow an implementation today that
> only works on anonymous memory *and* sets up a special vm_ops.  But, the
> same exact ABI could do wonky stuff in the future if we decided we
> wanted to do shadow stacks on DAX or hugetlbfs or whatever.
> 
> I don't really like the idea of PROT_SHSTK those are plumbed into a
> bunch of interfaces.  But, I also can't deny that it seems to be working
> fine for the arm64 folks.

Note, there are some rough edges, such as what happens when someone
calls mprotect() on memory marked with PROT_BTI.  Unless the caller
knows whether PROT_BTI should be set for that page, the flag may get
unintentionally cleared.  Since the flag only applies to text pages
though, it's not _that_ much of a concern.  Software that deals with
writable text pages is also usually involved in generating the code and
so will know about PROT_BTI.  That's was the theory anyway.

In the longer term, it might be preferable to have a mprotect2() that
can leave some flags unmodified, and that doesn't silently ignore
unknown flags (at least one of mmap or mprotect does; I don't recall
which).  We attempt didn't go this far, for now.

For arm64 it seemed fairly natural for the BTI flag to be a PROT_ flag,
but I don't know enough detail about x86 shstk to know whether it's a
natural fit there.

Cheers
---Dave