Re: [PATCH RFC] seccomp: Implement syscall isolation based on memory areas

[Gabriel Krisman Bertazi <krisman@collabora.com>]

"Robert O'Callahan" <robert@ocallahan.org> writes:

> rr (https://rr-project.org, https://arxiv.org/abs/1705.05937) grapples
> with a similar problem. We need to intercept commonly-executed system
> calls and wrap them with our own processing, with minimal overhead. I
> think our basic approach might work for Wine without kernel changes.
>
> We use SECCOMP_SET_MODE_FILTER with a simple filter that returns
> SECCOMP_RET_TRAP on all syscalls except for those called from a single
> specific trampoline page (which get SECCOMP_RET_ALLOW). rr ptraces its
> children. So, when user-space makes a syscall, the seccomp filter
> triggers a ptrace trap. The ptracer looks at the code around the
> syscall and if it matches certain common patterns, the ptracer patches
> the code with a jump to a stub that does extra work and issues a real
> syscall via the trampoline. Thus, each library syscall instruction is
> slow the first time and fast every subsequent time. "Weird" syscalls
> that the ptracer chooses not to patch do incur the context-switch
> penalty every time so their overhead does increase a lot ... but it
> sounds like that might be OK in Wine's case?
>
> A more efficient variant of this approach which would work in some
> cases (but maybe not Wine?) would be to avoid using a ptracer and give
> the process a SIGSYS handler which does the patching.

We couldn't patch Windows code because of the aforementioned DRM and
anti-cheat mechanisms, but I suppose this limitation doesn't apply to
Wine/native code, and if this assumption is correct, this approach could
work.

One complexity might be the consistent model for the syscall live
patching.  I don't know how much of the problem is diminished from the
original userspace live-patching problem, but I believe at least part of
it applies.  And fencing every thread to patch would kill performance.
Also, we cannot just patch everything at the beginning.  How does rr
handle that?

Another problem is that we will want to support i386 and other
architectures.  For int 0x80, it is trickier to encode a branch to
another region, given the limited instruction space, and the patching
might not be possible in hot paths.  I did port libsyscall-intercept to
x86-32 once and I could correctly patch glibc, but it's not guaranteed
that an updated libc or something else won't break it.

I'm not sure the benefit of not needing enhanced kernel support
justifies the complexity and performance cost required to make this work
reliably, in particular since the semantics for a kernel implementation
that we are discussing doesn't seem overly intrusive and might have
other applications like in the generic filter Andy mentioned.

-- 
Gabriel Krisman Bertazi