From: Andy Lutomirski <luto@amacapital.net> To: linux-kernel@vger.kernel.org, Kees Cook <keescook@chromium.org>, Will Drewry <wad@chromium.org>, Oleg Nesterov <oleg@redhat.com> Cc: x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@linux-mips.org, linux-arch@vger.kernel.org, linux-security-module@vger.kernel.org, Alexei Starovoitov <ast@plumgrid.com>, hpa@zytor.com, Frederic Weisbecker <fweisbec@gmail.com>, Andy Lutomirski <luto@amacapital.net> Subject: [PATCH v5 5/5] x86_64,entry: Use split-phase syscall_trace_enter for 64-bit syscalls Date: Fri, 5 Sep 2014 15:13:56 -0700 [thread overview] Message-ID: <a3dbd267ee990110478d349f78cccfdac5497a84.1409954077.git.luto@amacapital.net> (raw) In-Reply-To: <cover.1409954077.git.luto@amacapital.net> In-Reply-To: <cover.1409954077.git.luto@amacapital.net> On KVM on my box, this reduces the overhead from an always-accept seccomp filter from ~130ns to ~17ns. Most of that comes from avoiding IRET on every syscall when seccomp is enabled. In extremely approximate hacked-up benchmarking, just bypassing IRET saves about 80ns, so there's another 43ns of savings here from simplifying the seccomp path. The diffstat is also rather nice :) Signed-off-by: Andy Lutomirski <luto@amacapital.net> --- arch/x86/kernel/entry_64.S | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index 0bd6d3c28064..df088bb03fb3 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S @@ -478,22 +478,6 @@ sysret_signal: #ifdef CONFIG_AUDITSYSCALL /* - * Fast path for syscall audit without full syscall trace. - * We just call __audit_syscall_entry() directly, and then - * jump back to the normal fast path. - */ -auditsys: - movq %r10,%r9 /* 6th arg: 4th syscall arg */ - movq %rdx,%r8 /* 5th arg: 3rd syscall arg */ - movq %rsi,%rcx /* 4th arg: 2nd syscall arg */ - movq %rdi,%rdx /* 3rd arg: 1st syscall arg */ - movq %rax,%rsi /* 2nd arg: syscall number */ - movl $AUDIT_ARCH_X86_64,%edi /* 1st arg: audit arch */ - call __audit_syscall_entry - LOAD_ARGS 0 /* reload call-clobbered registers */ - jmp system_call_fastpath - - /* * Return fast path for syscall audit. Call __audit_syscall_exit() * directly and then jump back to the fast path with TIF_SYSCALL_AUDIT * masked off. @@ -510,17 +494,25 @@ sysret_audit: /* Do syscall tracing */ tracesys: -#ifdef CONFIG_AUDITSYSCALL - testl $(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT),TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET) - jz auditsys -#endif + leaq -REST_SKIP(%rsp), %rdi + movq $AUDIT_ARCH_X86_64, %rsi + call syscall_trace_enter_phase1 + test %rax, %rax + jnz tracesys_phase2 /* if needed, run the slow path */ + LOAD_ARGS 0 /* else restore clobbered regs */ + jmp system_call_fastpath /* and return to the fast path */ + +tracesys_phase2: SAVE_REST FIXUP_TOP_OF_STACK %rdi - movq %rsp,%rdi - call syscall_trace_enter + movq %rsp, %rdi + movq $AUDIT_ARCH_X86_64, %rsi + movq %rax,%rdx + call syscall_trace_enter_phase2 + /* * Reload arg registers from stack in case ptrace changed them. - * We don't reload %rax because syscall_trace_enter() returned + * We don't reload %rax because syscall_trace_entry_phase2() returned * the value it wants us to use in the table lookup. */ LOAD_ARGS ARGOFFSET, 1 -- 1.9.3
WARNING: multiple messages have this Message-ID (diff)
From: luto@amacapital.net (Andy Lutomirski) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 5/5] x86_64, entry: Use split-phase syscall_trace_enter for 64-bit syscalls Date: Fri, 5 Sep 2014 15:13:56 -0700 [thread overview] Message-ID: <a3dbd267ee990110478d349f78cccfdac5497a84.1409954077.git.luto@amacapital.net> (raw) In-Reply-To: <cover.1409954077.git.luto@amacapital.net> On KVM on my box, this reduces the overhead from an always-accept seccomp filter from ~130ns to ~17ns. Most of that comes from avoiding IRET on every syscall when seccomp is enabled. In extremely approximate hacked-up benchmarking, just bypassing IRET saves about 80ns, so there's another 43ns of savings here from simplifying the seccomp path. The diffstat is also rather nice :) Signed-off-by: Andy Lutomirski <luto@amacapital.net> --- arch/x86/kernel/entry_64.S | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index 0bd6d3c28064..df088bb03fb3 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S @@ -478,22 +478,6 @@ sysret_signal: #ifdef CONFIG_AUDITSYSCALL /* - * Fast path for syscall audit without full syscall trace. - * We just call __audit_syscall_entry() directly, and then - * jump back to the normal fast path. - */ -auditsys: - movq %r10,%r9 /* 6th arg: 4th syscall arg */ - movq %rdx,%r8 /* 5th arg: 3rd syscall arg */ - movq %rsi,%rcx /* 4th arg: 2nd syscall arg */ - movq %rdi,%rdx /* 3rd arg: 1st syscall arg */ - movq %rax,%rsi /* 2nd arg: syscall number */ - movl $AUDIT_ARCH_X86_64,%edi /* 1st arg: audit arch */ - call __audit_syscall_entry - LOAD_ARGS 0 /* reload call-clobbered registers */ - jmp system_call_fastpath - - /* * Return fast path for syscall audit. Call __audit_syscall_exit() * directly and then jump back to the fast path with TIF_SYSCALL_AUDIT * masked off. @@ -510,17 +494,25 @@ sysret_audit: /* Do syscall tracing */ tracesys: -#ifdef CONFIG_AUDITSYSCALL - testl $(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT),TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET) - jz auditsys -#endif + leaq -REST_SKIP(%rsp), %rdi + movq $AUDIT_ARCH_X86_64, %rsi + call syscall_trace_enter_phase1 + test %rax, %rax + jnz tracesys_phase2 /* if needed, run the slow path */ + LOAD_ARGS 0 /* else restore clobbered regs */ + jmp system_call_fastpath /* and return to the fast path */ + +tracesys_phase2: SAVE_REST FIXUP_TOP_OF_STACK %rdi - movq %rsp,%rdi - call syscall_trace_enter + movq %rsp, %rdi + movq $AUDIT_ARCH_X86_64, %rsi + movq %rax,%rdx + call syscall_trace_enter_phase2 + /* * Reload arg registers from stack in case ptrace changed them. - * We don't reload %rax because syscall_trace_enter() returned + * We don't reload %rax because syscall_trace_entry_phase2() returned * the value it wants us to use in the table lookup. */ LOAD_ARGS ARGOFFSET, 1 -- 1.9.3
next prev parent reply other threads:[~2014-09-05 22:14 UTC|newest] Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-09-05 22:13 [PATCH v5 0/5] x86: two-phase syscall tracing and seccomp fastpath Andy Lutomirski 2014-09-05 22:13 ` Andy Lutomirski 2014-09-05 22:13 ` [PATCH v5 1/5] x86,x32,audit: Fix x32's AUDIT_ARCH wrt audit Andy Lutomirski 2014-09-05 22:13 ` Andy Lutomirski 2014-09-09 2:43 ` [tip:x86/seccomp] x86, x32, audit: " tip-bot for Andy Lutomirski 2014-09-05 22:13 ` [PATCH v5 2/5] x86,entry: Only call user_exit if TIF_NOHZ Andy Lutomirski 2014-09-05 22:13 ` Andy Lutomirski 2014-09-09 2:43 ` [tip:x86/seccomp] x86, entry: " tip-bot for Andy Lutomirski 2014-09-05 22:13 ` [PATCH v5 3/5] x86: Split syscall_trace_enter into two phases Andy Lutomirski 2014-09-05 22:13 ` Andy Lutomirski 2014-09-09 2:44 ` [tip:x86/seccomp] " tip-bot for Andy Lutomirski 2015-02-05 21:19 ` [PATCH v5 3/5] " Dmitry V. Levin 2015-02-05 21:19 ` Dmitry V. Levin 2015-02-05 21:27 ` Kees Cook 2015-02-05 21:27 ` Kees Cook 2015-02-05 21:27 ` Kees Cook 2015-02-05 21:40 ` Dmitry V. Levin 2015-02-05 21:40 ` Dmitry V. Levin 2015-02-05 21:40 ` Dmitry V. Levin 2015-02-05 21:52 ` Andy Lutomirski 2015-02-05 21:52 ` Andy Lutomirski 2015-02-05 21:52 ` Andy Lutomirski 2015-02-05 23:12 ` Kees Cook 2015-02-05 23:12 ` Kees Cook 2015-02-05 23:12 ` Kees Cook 2015-02-05 23:39 ` Dmitry V. Levin 2015-02-05 23:39 ` Dmitry V. Levin 2015-02-05 23:39 ` Dmitry V. Levin 2015-02-05 23:49 ` Kees Cook 2015-02-05 23:49 ` Kees Cook 2015-02-05 23:49 ` Kees Cook 2015-02-06 0:09 ` Andy Lutomirski 2015-02-06 0:09 ` Andy Lutomirski 2015-02-06 0:09 ` Andy Lutomirski 2015-02-06 2:32 ` Dmitry V. Levin 2015-02-06 2:32 ` Dmitry V. Levin 2015-02-06 2:32 ` Dmitry V. Levin 2015-02-06 2:38 ` Andy Lutomirski 2015-02-06 2:38 ` Andy Lutomirski 2015-02-06 2:38 ` Andy Lutomirski 2015-02-06 19:23 ` Kees Cook 2015-02-06 19:23 ` Kees Cook 2015-02-06 19:23 ` Kees Cook 2015-02-06 19:32 ` Andy Lutomirski 2015-02-06 19:32 ` Andy Lutomirski 2015-02-06 19:32 ` Andy Lutomirski 2015-02-06 20:07 ` Kees Cook 2015-02-06 20:07 ` Kees Cook 2015-02-06 20:07 ` Kees Cook 2015-02-06 20:12 ` Andy Lutomirski 2015-02-06 20:12 ` Andy Lutomirski 2015-02-06 20:12 ` Andy Lutomirski 2015-02-06 20:16 ` Kees Cook 2015-02-06 20:16 ` Kees Cook 2015-02-06 20:16 ` Kees Cook 2015-02-06 20:20 ` Andy Lutomirski 2015-02-06 20:20 ` Andy Lutomirski 2015-02-06 20:20 ` Andy Lutomirski 2015-02-06 23:17 ` a method to distinguish between syscall-enter/exit-stop Dmitry V. Levin 2015-02-06 23:17 ` Dmitry V. Levin 2015-02-06 23:17 ` Dmitry V. Levin 2015-02-07 1:07 ` Kees Cook 2015-02-07 1:07 ` Kees Cook 2015-02-07 1:07 ` Kees Cook 2015-02-07 3:04 ` Dmitry V. Levin 2015-02-07 3:04 ` Dmitry V. Levin 2015-02-07 3:04 ` Dmitry V. Levin 2015-02-06 20:11 ` [PATCH v5 3/5] x86: Split syscall_trace_enter into two phases H. Peter Anvin 2015-02-06 20:11 ` H. Peter Anvin 2015-02-06 20:11 ` H. Peter Anvin 2014-09-05 22:13 ` [PATCH v5 4/5] x86_64,entry: Treat regs->ax the same in fastpath and slowpath syscalls Andy Lutomirski 2014-09-05 22:13 ` [PATCH v5 4/5] x86_64, entry: " Andy Lutomirski 2014-09-09 2:44 ` [tip:x86/seccomp] x86_64, entry: Treat regs-> ax " tip-bot for Andy Lutomirski 2014-09-05 22:13 ` Andy Lutomirski [this message] 2014-09-05 22:13 ` [PATCH v5 5/5] x86_64, entry: Use split-phase syscall_trace_enter for 64-bit syscalls Andy Lutomirski 2014-09-09 2:44 ` [tip:x86/seccomp] " tip-bot for Andy Lutomirski 2014-09-08 19:29 ` [PATCH v5 0/5] x86: two-phase syscall tracing and seccomp fastpath Kees Cook 2014-09-08 19:29 ` Kees Cook 2014-09-08 19:29 ` Kees Cook 2014-09-08 19:49 ` H. Peter Anvin 2014-09-08 19:49 ` H. Peter Anvin 2014-09-08 19:49 ` H. Peter Anvin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=a3dbd267ee990110478d349f78cccfdac5497a84.1409954077.git.luto@amacapital.net \ --to=luto@amacapital.net \ --cc=ast@plumgrid.com \ --cc=fweisbec@gmail.com \ --cc=hpa@zytor.com \ --cc=keescook@chromium.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mips@linux-mips.org \ --cc=linux-security-module@vger.kernel.org \ --cc=oleg@redhat.com \ --cc=wad@chromium.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.