From: Mark Rutland <mark.rutland@arm.com> To: "Madhavan T. Venkataraman" <madvenka@linux.microsoft.com> Cc: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v3 00/22] arm64: livepatch: Use ORC for dynamic frame pointer validation Date: Tue, 11 Apr 2023 14:25:11 +0100 [thread overview] Message-ID: <ZDVft9kysWMfTiZW@FVFF77S0Q05N> (raw) In-Reply-To: <054ce0d6-70f0-b834-d4e5-1049c8df7492@linux.microsoft.com> On Fri, Apr 07, 2023 at 10:40:07PM -0500, Madhavan T. Venkataraman wrote: > Hi Mark, > > Sorry for the long delay in responding. Was caught up in many things. > My responses inline.. > > On 3/23/23 12:17, Mark Rutland wrote: > > Hi Madhavan, > > > > At a high-level, I think this still falls afoul of our desire to not reverse > > engineer control flow from the binary, and so I do not think this is the right > > approach. I've expanded a bit on that below. > > > > I do think it would be nice to have *some* of the objtool changes, as I do > > think we will want to use objtool for some things in future (e.g. some > > build-time binary patching such as table sorting). > > OK. I have been under the impression that the arm64 folks are basically OK with > Objtool's approach of reverse engineering from the binary. I did not see > any specific objections to previously submitted patches based on this approach > including mine. This has admittedly changed over time, but the preference to avoid reverse-engineering control flow has been around for a while. For example, during LPC 2021's "objtool on arm64" session: https://lpc.events/event/11/contributions/971/ ... where Will and I expressed strong desires to get the compiler to help, whether that's compiler-generated metadata, (agreed upon) restrictions on code generation, or something else. [...] > > There's a more fundamental issue here in that objtool has to reverse-engineer > > control flow, and so even if the kernel code and compiled code generation is > > *perfect*, it's possible that objtool won't recognise the structure of the > > generated code, and won't be able to reverse-engineer the correct control flow. > > > > We've seen issues where objtool didn't understand jump tables, so support for > > that got disabled on x86. A key objection from the arm64 side is that we don't > > want to disable compile code generation strategies like this. Further, as > > compiles evolve, their code generation strategies will change, and it's likely > > there will be other cases that crop up. This is inherently fragile. > > > > The key objections from the arm64 side is that we don't want to > > reverse-engineer details from the binary, as this is complex, fragile, and > > unstable. This is why we've previously suggested that we should work with > > compiler folk to get what we need. > > > > So, what exactly do you have in mind? What help can the compiler folk provide? There are several possibilities, e.g. * Generate some simple metadata that tells us for each PC whether to start an unwind from the LR or FP. My understanding was that SFrame *might* be sufficient for this. We might need some custom metadata for assembly (e.g. exception entry, trampolines), but it'd be ok for that to be different. * Agree upon some restricted patterns for code generation (e.g. fixed prologues/epilogues), so that we can identify whether to use LR or FP based on the PC and a symbol lookup. > By your own argument, we cannot rely on the compiler as compiler implementations, > optimization strategies, etc can change in ways that are incompatible with any > livepatch implementation. That's not quite my argument. My argument is that if we assume some set of properties that compiler folk never agreed to (and were never made aware of), then compiler folk are well within their rights to change the compiler such that it doesn't provide those properties, and it's very likely that such expectation will be broken. We've seen that happen before (e.g. with jump tables). Consequently I think we should be working with compiler folk to agree upon some solution, where compiler folk will actually try to maintain the properties we depend upon (and e.g. they could have tests for). That sort of co-design has worked well so far (e.g. with things like kCFI). Ideally we'd have people in the same room to have a discussion (e.g. at LPC). > Also, there can always be bugs in the compiler implementations. I don't disagree with that. > Can you please elaborate? Are we looking for a way for the compiler folks to > provide us with something that we can use to implement reliable stack trace? I tried to do so a bit above. I'm looking for some agreement between kernel folk and compiler folk on a reliable mechanism. That might be something that already exists, or something new. It might be metadata or some restrictions on code generation. > > I'll note that at the last Linux Plumbers Conference, there was a discussion > > about what is now called SFrame, which *might* give us sufficient information, > > but I have not had the time to dig into that as I have been chasing other > > problems and trying to get other infrastructure in place. > > I will try to locate the link. If you can provide me a link, that would be greatly > appreciated. I will study their SFrame proposal. From looking around, that session was: https://lpc.events/event/16/contributions/1177/ At the time it was called CTF Frame, but got renamed to SFrame. I'm not sure where to find the most recent documentation. As I mentioned above I have not had the time to look in detail. > >> FWIW, I have also compared the CFI I am generating with DWARF > >> information that the compiler generates. The CFIs match a > >> 100% for Clang. In the case of gcc, the comparison fails > >> in 1.7% of the cases. I have analyzed those cases and found > >> the DWARF information generated by gcc is incorrect. The > >> ORC generated by my Objtool is correct. > > > > Have you reported this to the GCC folk, and can you give any examples? > > I'm sure they would be interested in fixing this, regardless of whether we end > > up using it. > > I will try to get the data again and put something together and send it to the > gcc folks. Thanks for doing so; that's much appreciated! Thanks, Mark.
WARNING: multiple messages have this Message-ID (diff)
From: Mark Rutland <mark.rutland@arm.com> To: "Madhavan T. Venkataraman" <madvenka@linux.microsoft.com> Cc: jpoimboe@redhat.com, peterz@infradead.org, chenzhongjin@huawei.com, broonie@kernel.org, nobuta.keiya@fujitsu.com, sjitindarsingh@gmail.com, catalin.marinas@arm.com, will@kernel.org, jamorris@linux.microsoft.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v3 00/22] arm64: livepatch: Use ORC for dynamic frame pointer validation Date: Tue, 11 Apr 2023 14:25:11 +0100 [thread overview] Message-ID: <ZDVft9kysWMfTiZW@FVFF77S0Q05N> (raw) In-Reply-To: <054ce0d6-70f0-b834-d4e5-1049c8df7492@linux.microsoft.com> On Fri, Apr 07, 2023 at 10:40:07PM -0500, Madhavan T. Venkataraman wrote: > Hi Mark, > > Sorry for the long delay in responding. Was caught up in many things. > My responses inline.. > > On 3/23/23 12:17, Mark Rutland wrote: > > Hi Madhavan, > > > > At a high-level, I think this still falls afoul of our desire to not reverse > > engineer control flow from the binary, and so I do not think this is the right > > approach. I've expanded a bit on that below. > > > > I do think it would be nice to have *some* of the objtool changes, as I do > > think we will want to use objtool for some things in future (e.g. some > > build-time binary patching such as table sorting). > > OK. I have been under the impression that the arm64 folks are basically OK with > Objtool's approach of reverse engineering from the binary. I did not see > any specific objections to previously submitted patches based on this approach > including mine. This has admittedly changed over time, but the preference to avoid reverse-engineering control flow has been around for a while. For example, during LPC 2021's "objtool on arm64" session: https://lpc.events/event/11/contributions/971/ ... where Will and I expressed strong desires to get the compiler to help, whether that's compiler-generated metadata, (agreed upon) restrictions on code generation, or something else. [...] > > There's a more fundamental issue here in that objtool has to reverse-engineer > > control flow, and so even if the kernel code and compiled code generation is > > *perfect*, it's possible that objtool won't recognise the structure of the > > generated code, and won't be able to reverse-engineer the correct control flow. > > > > We've seen issues where objtool didn't understand jump tables, so support for > > that got disabled on x86. A key objection from the arm64 side is that we don't > > want to disable compile code generation strategies like this. Further, as > > compiles evolve, their code generation strategies will change, and it's likely > > there will be other cases that crop up. This is inherently fragile. > > > > The key objections from the arm64 side is that we don't want to > > reverse-engineer details from the binary, as this is complex, fragile, and > > unstable. This is why we've previously suggested that we should work with > > compiler folk to get what we need. > > > > So, what exactly do you have in mind? What help can the compiler folk provide? There are several possibilities, e.g. * Generate some simple metadata that tells us for each PC whether to start an unwind from the LR or FP. My understanding was that SFrame *might* be sufficient for this. We might need some custom metadata for assembly (e.g. exception entry, trampolines), but it'd be ok for that to be different. * Agree upon some restricted patterns for code generation (e.g. fixed prologues/epilogues), so that we can identify whether to use LR or FP based on the PC and a symbol lookup. > By your own argument, we cannot rely on the compiler as compiler implementations, > optimization strategies, etc can change in ways that are incompatible with any > livepatch implementation. That's not quite my argument. My argument is that if we assume some set of properties that compiler folk never agreed to (and were never made aware of), then compiler folk are well within their rights to change the compiler such that it doesn't provide those properties, and it's very likely that such expectation will be broken. We've seen that happen before (e.g. with jump tables). Consequently I think we should be working with compiler folk to agree upon some solution, where compiler folk will actually try to maintain the properties we depend upon (and e.g. they could have tests for). That sort of co-design has worked well so far (e.g. with things like kCFI). Ideally we'd have people in the same room to have a discussion (e.g. at LPC). > Also, there can always be bugs in the compiler implementations. I don't disagree with that. > Can you please elaborate? Are we looking for a way for the compiler folks to > provide us with something that we can use to implement reliable stack trace? I tried to do so a bit above. I'm looking for some agreement between kernel folk and compiler folk on a reliable mechanism. That might be something that already exists, or something new. It might be metadata or some restrictions on code generation. > > I'll note that at the last Linux Plumbers Conference, there was a discussion > > about what is now called SFrame, which *might* give us sufficient information, > > but I have not had the time to dig into that as I have been chasing other > > problems and trying to get other infrastructure in place. > > I will try to locate the link. If you can provide me a link, that would be greatly > appreciated. I will study their SFrame proposal. From looking around, that session was: https://lpc.events/event/16/contributions/1177/ At the time it was called CTF Frame, but got renamed to SFrame. I'm not sure where to find the most recent documentation. As I mentioned above I have not had the time to look in detail. > >> FWIW, I have also compared the CFI I am generating with DWARF > >> information that the compiler generates. The CFIs match a > >> 100% for Clang. In the case of gcc, the comparison fails > >> in 1.7% of the cases. I have analyzed those cases and found > >> the DWARF information generated by gcc is incorrect. The > >> ORC generated by my Objtool is correct. > > > > Have you reported this to the GCC folk, and can you give any examples? > > I'm sure they would be interested in fixing this, regardless of whether we end > > up using it. > > I will try to get the data again and put something together and send it to the > gcc folks. Thanks for doing so; that's much appreciated! Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2023-04-11 13:25 UTC|newest] Thread overview: 113+ messages / expand[flat|nested] mbox.gz Atom feed top [not found] <0337266cf19f4c98388e3f6d09f590d9de258dc7> 2023-02-02 7:40 ` [RFC PATCH v3 00/22] arm64: livepatch: Use ORC for dynamic frame pointer validation madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 01/22] objtool: Reorganize CFI code madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 02/22] objtool: Reorganize instruction-related code madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 03/22] objtool: Move decode_instructions() to a separate file madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 04/22] objtool: Reorganize Unwind hint code madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 05/22] objtool: Reorganize ORC types madvenka 2023-02-02 7:40 ` madvenka 2023-02-18 9:30 ` Suraj Jitindar Singh 2023-02-18 9:30 ` Suraj Jitindar Singh 2023-03-06 16:45 ` Madhavan T. Venkataraman 2023-03-06 16:45 ` Madhavan T. Venkataraman 2023-02-02 7:40 ` [RFC PATCH v3 06/22] objtool: Reorganize ORC code madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 07/22] objtool: Reorganize ORC kernel code madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 08/22] objtool: Introduce STATIC_CHECK madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 09/22] objtool: arm64: Add basic definitions and compile madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 10/22] objtool: arm64: Implement decoder for Dynamic FP validation madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 11/22] objtool: arm64: Invoke the decoder madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 12/22] objtool: arm64: Compute destinations for call and jump instructions madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 13/22] objtool: arm64: Walk instructions and compute CFI for each instruction madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 14/22] objtool: arm64: Generate ORC data from CFI for object files madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 15/22] objtool: arm64: Add unwind hint support madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 16/22] arm64: Add unwind hints to exception handlers madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 17/22] arm64: Add kernel and module support for ORC madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 18/22] arm64: Build the kernel with ORC information madvenka 2023-02-02 7:40 ` madvenka 2023-02-10 7:52 ` Tomohiro Misono (Fujitsu) 2023-02-10 7:52 ` Tomohiro Misono (Fujitsu) 2023-02-11 4:34 ` Madhavan T. Venkataraman 2023-02-11 4:34 ` Madhavan T. Venkataraman 2023-02-02 7:40 ` [RFC PATCH v3 19/22] arm64: unwinder: Add a reliability check in the unwinder based on ORC madvenka 2023-02-02 7:40 ` madvenka 2023-02-23 4:07 ` Suraj Jitindar Singh 2023-02-23 4:07 ` Suraj Jitindar Singh 2023-03-06 16:52 ` Madhavan T. Venkataraman 2023-03-06 16:52 ` Madhavan T. Venkataraman 2023-02-02 7:40 ` [RFC PATCH v3 20/22] arm64: Define HAVE_DYNAMIC_FTRACE_WITH_ARGS madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 21/22] arm64: Define TIF_PATCH_PENDING for livepatch madvenka 2023-02-02 7:40 ` madvenka 2023-02-02 7:40 ` [RFC PATCH v3 22/22] arm64: Enable livepatch for ARM64 madvenka 2023-02-02 7:40 ` madvenka 2023-03-01 3:12 ` [RFC PATCH v3 00/22] arm64: livepatch: Use ORC for dynamic frame pointer validation Tomohiro Misono (Fujitsu) 2023-03-01 3:12 ` Tomohiro Misono (Fujitsu) 2023-03-02 16:23 ` Petr Mladek 2023-03-02 16:23 ` Petr Mladek 2023-03-03 9:40 ` Tomohiro Misono (Fujitsu) 2023-03-03 9:40 ` Tomohiro Misono (Fujitsu) 2023-03-06 16:58 ` Madhavan T. Venkataraman 2023-03-06 16:58 ` Madhavan T. Venkataraman 2023-03-06 16:57 ` Madhavan T. Venkataraman 2023-03-06 16:57 ` Madhavan T. Venkataraman 2023-03-23 17:17 ` Mark Rutland 2023-03-23 17:17 ` Mark Rutland 2023-04-08 3:40 ` Madhavan T. Venkataraman 2023-04-08 3:40 ` Madhavan T. Venkataraman 2023-04-11 13:25 ` Mark Rutland [this message] 2023-04-11 13:25 ` Mark Rutland 2023-04-12 4:17 ` Josh Poimboeuf 2023-04-12 4:17 ` Josh Poimboeuf 2023-04-12 4:48 ` Madhavan T. Venkataraman 2023-04-12 4:48 ` Madhavan T. Venkataraman 2023-04-12 4:50 ` Madhavan T. Venkataraman 2023-04-12 4:50 ` Madhavan T. Venkataraman 2023-04-12 5:01 ` Josh Poimboeuf 2023-04-12 5:01 ` Josh Poimboeuf 2023-04-12 14:50 ` Madhavan T. Venkataraman 2023-04-12 14:50 ` Madhavan T. Venkataraman 2023-04-12 15:52 ` Josh Poimboeuf 2023-04-12 15:52 ` Josh Poimboeuf 2023-04-13 14:59 ` Madhavan T. Venkataraman 2023-04-13 14:59 ` Madhavan T. Venkataraman 2023-04-13 16:30 ` Josh Poimboeuf 2023-04-13 16:30 ` Josh Poimboeuf 2023-04-15 4:27 ` Madhavan T. Venkataraman 2023-04-15 4:27 ` Madhavan T. Venkataraman 2023-04-15 5:05 ` Josh Poimboeuf 2023-04-15 5:05 ` Josh Poimboeuf 2023-04-15 16:15 ` Madhavan T. Venkataraman 2023-04-15 16:15 ` Madhavan T. Venkataraman 2023-04-16 8:21 ` Indu Bhagat 2023-04-16 8:21 ` Indu Bhagat 2023-04-13 17:04 ` Nick Desaulniers 2023-04-13 17:04 ` Nick Desaulniers 2023-04-13 18:15 ` Jose E. Marchesi 2023-04-13 18:15 ` Jose E. Marchesi 2023-04-15 4:14 ` Madhavan T. Venkataraman 2023-04-15 4:14 ` Madhavan T. Venkataraman 2023-12-14 20:49 ` ARM64 Livepatch based on SFrame Madhavan T. Venkataraman 2023-12-14 20:49 ` Madhavan T. Venkataraman 2023-12-15 13:04 ` Mark Rutland 2023-12-15 13:04 ` Mark Rutland 2023-12-15 15:15 ` Madhavan T. Venkataraman 2023-12-15 15:15 ` Madhavan T. Venkataraman 2023-04-03 22:26 [RFC PATCH v3 00/22] arm64: livepatch: Use ORC for dynamic frame pointer validation Dylan Hatch 2023-04-08 3:41 ` Madhavan T. Venkataraman 2023-04-08 3:41 ` Madhavan T. Venkataraman
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=ZDVft9kysWMfTiZW@FVFF77S0Q05N \ --to=mark.rutland@arm.com \ --cc=broonie@kernel.org \ --cc=catalin.marinas@arm.com \ --cc=chenzhongjin@huawei.com \ --cc=jamorris@linux.microsoft.com \ --cc=jpoimboe@redhat.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=live-patching@vger.kernel.org \ --cc=madvenka@linux.microsoft.com \ --cc=nobuta.keiya@fujitsu.com \ --cc=peterz@infradead.org \ --cc=sjitindarsingh@gmail.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.