From: Peter Zijlstra <peterz@infradead.org> To: Sami Tolvanen <samitolvanen@google.com> Cc: linux-kernel@vger.kernel.org, Kees Cook <keescook@chromium.org>, Josh Poimboeuf <jpoimboe@redhat.com>, x86@kernel.org, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Nathan Chancellor <nathan@kernel.org>, Nick Desaulniers <ndesaulniers@google.com>, Joao Moreira <joao@overdrivepizza.com>, Sedat Dilek <sedat.dilek@gmail.com>, Steven Rostedt <rostedt@goodmis.org>, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev Subject: Re: [RFC PATCH v2 20/21] x86: Add support for CONFIG_CFI_CLANG Date: Mon, 16 May 2022 20:30:47 +0200 [thread overview] Message-ID: <20220516183047.GM76023@worktop.programming.kicks-ass.net> (raw) In-Reply-To: <CABCJKucXA2jbTc9TF1mLUsEDKu52t71tzxpnsGOXY3_ks+W4Bg@mail.gmail.com> On Mon, May 16, 2022 at 10:15:00AM -0700, Sami Tolvanen wrote: > On Mon, May 16, 2022 at 2:54 AM Peter Zijlstra <peterz@infradead.org> wrote: > > > > On Fri, May 13, 2022 at 01:21:58PM -0700, Sami Tolvanen wrote: > > > With CONFIG_CFI_CLANG, the compiler injects a type preamble > > > immediately before each function and a check to validate the target > > > function type before indirect calls: > > > > > > ; type preamble > > > __cfi_function: > > > int3 > > > int3 > > > mov <id>, %eax > > > int3 > > > int3 > > > function: > > > ... > > > > When I enable CFI_CLANG and X86_KERNEL_IBT I get: > > > > 0000000000000c80 <__cfi_io_schedule_timeout>: > > c80: cc int3 > > c81: cc int3 > > c82: b8 b5 b1 39 b3 mov $0xb339b1b5,%eax > > c87: cc int3 > > c88: cc int3 > > > > 0000000000000c89 <io_schedule_timeout>: > > c89: f3 0f 1e fa endbr64 > > > > > > That seems unfortunate. Would it be possible to get an additional > > compiler option to suppress the endbr for all symbols that get a __cfi_ > > preaamble? > > What's the concern with the endbr? Dropping it would currently break > the CFI+IBT combination on newer hardware, no? Well, yes, but also that combination isn't very interesting. See, https://lore.kernel.org/all/20220420004241.2093-1-joao@overdrivepizza.com/T/#m5d67fb010d488b2f8eee33f1eb39d12f769e4ad2 and the patch I did down-thread: https://lkml.kernel.org/r/YoJKhHluN4n0kZDm@hirez.programming.kicks-ass.net If we have IBT, then FineIBT is a much better option than kCFI+IBT. Removing that superfluous endbr also shrinks the whole thing by 4 bytes. So I'm fine with the compiler generating working code for that combination; but please get me an option to supress it in order to save those pointless bytes. All this CFI stuff is enough bloat as it is. > > > ; indirect call check > > > cmpl <id>, -6(%r11) > > > je .Ltmp1 > > > ud2 > > > .Ltmp1: > > > call __x86_indirect_thunk_r11 > > > > The first one I try and find looks like: > > > > 26: 41 81 7b fa a6 96 9e 38 cmpl $0x389e96a6,-0x6(%r11) > > 2e: 74 02 je 32 <__traceiter_sched_kthread_stop+0x29> > > 30: 0f 0b ud2 > > 32: 4c 89 f6 mov %r14,%rsi > > 35: e8 00 00 00 00 call 3a <__traceiter_sched_kthread_stop+0x31> 36: R_X86_64_PLT32 __x86_indirect_thunk_r11-0x4 > > > > This must not be. If I'm to rewrite that lot to: > > > > movl $\hash, %r10d > > sub $9, %r11 > > call *%r11 > > .nop 4 > > > > Then there must not be spurious instruction in between the ud2 and the > > indirect call/retpoline thing. > > With the current compiler patch, LLVM sets up function arguments after > the CFI check. if it's a problem, we can look into changing that. Yes, please fix that. Again see that same patch for why this is a problem. Objtool can trivially find retpoline calls, but finding this kCFI gadget is going to be hard work. If you ensure they're unconditionally stuck together, then the problem goes away find one, finds the other.
WARNING: multiple messages have this Message-ID (diff)
From: Peter Zijlstra <peterz@infradead.org> To: Sami Tolvanen <samitolvanen@google.com> Cc: linux-kernel@vger.kernel.org, Kees Cook <keescook@chromium.org>, Josh Poimboeuf <jpoimboe@redhat.com>, x86@kernel.org, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Nathan Chancellor <nathan@kernel.org>, Nick Desaulniers <ndesaulniers@google.com>, Joao Moreira <joao@overdrivepizza.com>, Sedat Dilek <sedat.dilek@gmail.com>, Steven Rostedt <rostedt@goodmis.org>, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev Subject: Re: [RFC PATCH v2 20/21] x86: Add support for CONFIG_CFI_CLANG Date: Mon, 16 May 2022 20:30:47 +0200 [thread overview] Message-ID: <20220516183047.GM76023@worktop.programming.kicks-ass.net> (raw) In-Reply-To: <CABCJKucXA2jbTc9TF1mLUsEDKu52t71tzxpnsGOXY3_ks+W4Bg@mail.gmail.com> On Mon, May 16, 2022 at 10:15:00AM -0700, Sami Tolvanen wrote: > On Mon, May 16, 2022 at 2:54 AM Peter Zijlstra <peterz@infradead.org> wrote: > > > > On Fri, May 13, 2022 at 01:21:58PM -0700, Sami Tolvanen wrote: > > > With CONFIG_CFI_CLANG, the compiler injects a type preamble > > > immediately before each function and a check to validate the target > > > function type before indirect calls: > > > > > > ; type preamble > > > __cfi_function: > > > int3 > > > int3 > > > mov <id>, %eax > > > int3 > > > int3 > > > function: > > > ... > > > > When I enable CFI_CLANG and X86_KERNEL_IBT I get: > > > > 0000000000000c80 <__cfi_io_schedule_timeout>: > > c80: cc int3 > > c81: cc int3 > > c82: b8 b5 b1 39 b3 mov $0xb339b1b5,%eax > > c87: cc int3 > > c88: cc int3 > > > > 0000000000000c89 <io_schedule_timeout>: > > c89: f3 0f 1e fa endbr64 > > > > > > That seems unfortunate. Would it be possible to get an additional > > compiler option to suppress the endbr for all symbols that get a __cfi_ > > preaamble? > > What's the concern with the endbr? Dropping it would currently break > the CFI+IBT combination on newer hardware, no? Well, yes, but also that combination isn't very interesting. See, https://lore.kernel.org/all/20220420004241.2093-1-joao@overdrivepizza.com/T/#m5d67fb010d488b2f8eee33f1eb39d12f769e4ad2 and the patch I did down-thread: https://lkml.kernel.org/r/YoJKhHluN4n0kZDm@hirez.programming.kicks-ass.net If we have IBT, then FineIBT is a much better option than kCFI+IBT. Removing that superfluous endbr also shrinks the whole thing by 4 bytes. So I'm fine with the compiler generating working code for that combination; but please get me an option to supress it in order to save those pointless bytes. All this CFI stuff is enough bloat as it is. > > > ; indirect call check > > > cmpl <id>, -6(%r11) > > > je .Ltmp1 > > > ud2 > > > .Ltmp1: > > > call __x86_indirect_thunk_r11 > > > > The first one I try and find looks like: > > > > 26: 41 81 7b fa a6 96 9e 38 cmpl $0x389e96a6,-0x6(%r11) > > 2e: 74 02 je 32 <__traceiter_sched_kthread_stop+0x29> > > 30: 0f 0b ud2 > > 32: 4c 89 f6 mov %r14,%rsi > > 35: e8 00 00 00 00 call 3a <__traceiter_sched_kthread_stop+0x31> 36: R_X86_64_PLT32 __x86_indirect_thunk_r11-0x4 > > > > This must not be. If I'm to rewrite that lot to: > > > > movl $\hash, %r10d > > sub $9, %r11 > > call *%r11 > > .nop 4 > > > > Then there must not be spurious instruction in between the ud2 and the > > indirect call/retpoline thing. > > With the current compiler patch, LLVM sets up function arguments after > the CFI check. if it's a problem, we can look into changing that. Yes, please fix that. Again see that same patch for why this is a problem. Objtool can trivially find retpoline calls, but finding this kCFI gadget is going to be hard work. If you ensure they're unconditionally stuck together, then the problem goes away find one, finds the other. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-05-16 18:31 UTC|newest] Thread overview: 174+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-05-13 20:21 [RFC PATCH v2 00/21] KCFI support Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-13 20:21 ` [RFC PATCH v2 01/21] efi/libstub: Filter out CC_FLAGS_CFI Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:42 ` Kees Cook 2022-05-14 21:42 ` Kees Cook 2022-05-16 15:44 ` Sami Tolvanen 2022-05-16 15:44 ` Sami Tolvanen 2022-05-13 20:21 ` [RFC PATCH v2 02/21] arm64/vdso: " Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:42 ` Kees Cook 2022-05-14 21:42 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 03/21] kallsyms: Ignore __kcfi_typeid_ Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:43 ` Kees Cook 2022-05-14 21:43 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 04/21] cfi: Remove CONFIG_CFI_CLANG_SHADOW Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:43 ` Kees Cook 2022-05-14 21:43 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 05/21] cfi: Drop __CFI_ADDRESSABLE Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:44 ` Kees Cook 2022-05-14 21:44 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 06/21] cfi: Switch to -fsanitize=kcfi Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:46 ` Kees Cook 2022-05-14 21:46 ` Kees Cook 2022-05-15 3:41 ` Kees Cook 2022-05-15 3:41 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 07/21] cfi: Add type helper macros Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:49 ` Kees Cook 2022-05-14 21:49 ` Kees Cook 2022-05-16 12:28 ` Rasmus Villemoes 2022-05-16 12:28 ` Rasmus Villemoes 2022-05-16 16:23 ` Sami Tolvanen 2022-05-16 16:23 ` Sami Tolvanen 2022-05-16 16:04 ` Sami Tolvanen 2022-05-16 16:04 ` Sami Tolvanen 2022-05-13 20:21 ` [RFC PATCH v2 08/21] psci: Fix the function type for psci_initcall_t Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:50 ` Kees Cook 2022-05-14 21:50 ` Kees Cook 2022-05-16 15:44 ` Sami Tolvanen 2022-05-16 15:44 ` Sami Tolvanen 2022-05-17 8:47 ` Mark Rutland 2022-05-17 8:47 ` Mark Rutland 2022-05-13 20:21 ` [RFC PATCH v2 09/21] arm64: Add types to indirect called assembly functions Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:50 ` Kees Cook 2022-05-14 21:50 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 10/21] arm64: Add CFI error handling Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:51 ` Kees Cook 2022-05-14 21:51 ` Kees Cook 2022-05-16 16:24 ` Sami Tolvanen 2022-05-16 16:24 ` Sami Tolvanen 2022-05-13 20:21 ` [RFC PATCH v2 11/21] arm64: Drop unneeded __nocfi attributes Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:54 ` Kees Cook 2022-05-14 21:54 ` Kees Cook 2022-05-16 16:28 ` Sami Tolvanen 2022-05-16 16:28 ` Sami Tolvanen 2022-05-13 20:21 ` [RFC PATCH v2 12/21] treewide: Drop function_nocfi Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:54 ` Kees Cook 2022-05-14 21:54 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 13/21] treewide: Drop WARN_ON_FUNCTION_MISMATCH Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:54 ` Kees Cook 2022-05-14 21:54 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 14/21] treewide: Drop __cficanonical Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:56 ` Kees Cook 2022-05-14 21:56 ` Kees Cook 2022-05-16 16:32 ` Sami Tolvanen 2022-05-16 16:32 ` Sami Tolvanen 2022-05-13 20:21 ` [RFC PATCH v2 15/21] objtool: Don't warn about __cfi_ preambles falling through Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:56 ` Kees Cook 2022-05-14 21:56 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 16/21] x86/tools/relocs: Ignore __kcfi_typeid_ relocations Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:57 ` Kees Cook 2022-05-14 21:57 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 17/21] x86: Add types to indirectly called assembly functions Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:58 ` Kees Cook 2022-05-14 21:58 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 18/21] x86/purgatory: Disable CFI Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:58 ` Kees Cook 2022-05-14 21:58 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 19/21] x86/vdso: " Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 21:58 ` Kees Cook 2022-05-14 21:58 ` Kees Cook 2022-05-13 20:21 ` [RFC PATCH v2 20/21] x86: Add support for CONFIG_CFI_CLANG Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 22:02 ` Kees Cook 2022-05-14 22:02 ` Kees Cook 2022-05-16 18:57 ` Sami Tolvanen 2022-05-16 18:57 ` Sami Tolvanen 2022-05-15 3:19 ` Kees Cook 2022-05-15 3:19 ` Kees Cook 2022-05-16 8:32 ` David Laight 2022-05-16 8:32 ` David Laight 2022-05-16 16:39 ` Sami Tolvanen 2022-05-16 16:39 ` Sami Tolvanen 2022-05-16 21:32 ` David Laight 2022-05-16 21:32 ` David Laight 2022-05-16 21:44 ` Peter Zijlstra 2022-05-16 21:44 ` Peter Zijlstra 2022-05-16 22:03 ` Sami Tolvanen 2022-05-16 22:03 ` Sami Tolvanen 2022-05-17 6:44 ` Peter Zijlstra 2022-05-17 6:44 ` Peter Zijlstra 2022-05-17 20:36 ` Sami Tolvanen 2022-05-17 20:36 ` Sami Tolvanen 2022-05-17 7:56 ` David Laight 2022-05-17 7:56 ` David Laight 2022-05-16 9:54 ` Peter Zijlstra 2022-05-16 9:54 ` Peter Zijlstra 2022-05-16 11:45 ` Peter Zijlstra 2022-05-16 11:45 ` Peter Zijlstra 2022-05-16 12:58 ` Peter Zijlstra 2022-05-16 12:58 ` Peter Zijlstra 2022-05-20 13:49 ` Matthew Wilcox 2022-05-20 13:49 ` Matthew Wilcox 2022-05-16 17:15 ` Sami Tolvanen 2022-05-16 17:15 ` Sami Tolvanen 2022-05-16 18:30 ` Peter Zijlstra [this message] 2022-05-16 18:30 ` Peter Zijlstra 2022-05-16 19:39 ` Sami Tolvanen 2022-05-16 19:39 ` Sami Tolvanen 2022-05-16 20:37 ` Peter Zijlstra 2022-05-16 20:37 ` Peter Zijlstra 2022-05-25 20:02 ` Kees Cook 2022-05-25 20:02 ` Kees Cook 2022-05-16 22:59 ` Kees Cook 2022-05-16 22:59 ` Kees Cook 2022-05-17 8:05 ` Peter Zijlstra 2022-05-17 8:05 ` Peter Zijlstra 2022-05-17 8:32 ` Joao Moreira 2022-05-17 8:32 ` Joao Moreira 2022-05-17 8:40 ` Peter Zijlstra 2022-05-17 8:40 ` Peter Zijlstra 2022-05-17 8:48 ` David Laight 2022-05-17 8:48 ` David Laight 2022-05-17 9:38 ` Peter Zijlstra 2022-05-17 9:38 ` Peter Zijlstra 2022-05-13 20:21 ` [RFC PATCH v2 21/21] init: Drop __nocfi from __init Sami Tolvanen 2022-05-13 20:21 ` Sami Tolvanen 2022-05-14 22:03 ` Kees Cook 2022-05-14 22:03 ` Kees Cook 2022-05-16 17:16 ` Sami Tolvanen 2022-05-16 17:16 ` Sami Tolvanen [not found] ` <CA+icZUWr+-HjMvY1VZf+nqjTadxSTDciux0Y5Y-+p_j4o7CmXg@mail.gmail.com> 2022-05-16 17:57 ` [RFC PATCH v2 00/21] KCFI support Sami Tolvanen 2022-05-16 17:57 ` Sami Tolvanen 2022-05-17 7:33 ` Sedat Dilek 2022-05-17 7:33 ` Sedat Dilek 2022-05-17 18:49 ` Nathan Chancellor 2022-05-17 18:49 ` Nathan Chancellor 2022-05-19 9:01 ` Sedat Dilek 2022-05-19 9:01 ` Sedat Dilek 2022-05-19 20:26 ` Nathan Chancellor 2022-05-19 20:26 ` Nathan Chancellor 2022-05-19 20:41 ` Sami Tolvanen 2022-05-19 20:41 ` Sami Tolvanen 2022-05-17 8:57 ` Peter Zijlstra 2022-05-17 8:57 ` Peter Zijlstra 2022-05-17 20:25 ` Sami Tolvanen 2022-05-17 20:25 ` Sami Tolvanen
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220516183047.GM76023@worktop.programming.kicks-ass.net \ --to=peterz@infradead.org \ --cc=catalin.marinas@arm.com \ --cc=joao@overdrivepizza.com \ --cc=jpoimboe@redhat.com \ --cc=keescook@chromium.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-hardening@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=llvm@lists.linux.dev \ --cc=mark.rutland@arm.com \ --cc=nathan@kernel.org \ --cc=ndesaulniers@google.com \ --cc=rostedt@goodmis.org \ --cc=samitolvanen@google.com \ --cc=sedat.dilek@gmail.com \ --cc=will@kernel.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.