From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43EC4C433E0 for ; Wed, 10 Mar 2021 11:34:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F199B64FD7 for ; Wed, 10 Mar 2021 11:34:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230319AbhCJLd7 (ORCPT ); Wed, 10 Mar 2021 06:33:59 -0500 Received: from foss.arm.com ([217.140.110.172]:44618 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230173AbhCJLc4 (ORCPT ); Wed, 10 Mar 2021 06:32:56 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2A6BF1063; Wed, 10 Mar 2021 03:32:29 -0800 (PST) Received: from C02TD0UTHF1T.local (unknown [10.57.52.108]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BF2773F85F; Wed, 10 Mar 2021 03:32:26 -0800 (PST) Date: Wed, 10 Mar 2021 11:32:20 +0000 From: Mark Rutland To: Segher Boessenkool Cc: Marco Elver , Catalin Marinas , Will Deacon , LKML , broonie@kernel.org, Paul Mackerras , kasan-dev , linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v1] powerpc: Include running function as first entry in save_stack_trace() and friends Message-ID: <20210310112441.GA19619@C02TD0UTHF1T.local> References: <1802be3e-dc1a-52e0-1754-a40f0ea39658@csgroup.eu> <20210304145730.GC54534@C02TD0UTHF1T.local> <20210304215448.GU29191@gate.crashing.org> <20210309160505.GA4979@C02TD0UTHF1T.local> <20210309220532.GI29191@gate.crashing.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210309220532.GI29191@gate.crashing.org> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Mar 09, 2021 at 04:05:32PM -0600, Segher Boessenkool wrote: > Hi! > > On Tue, Mar 09, 2021 at 04:05:23PM +0000, Mark Rutland wrote: > > On Thu, Mar 04, 2021 at 03:54:48PM -0600, Segher Boessenkool wrote: > > > On Thu, Mar 04, 2021 at 02:57:30PM +0000, Mark Rutland wrote: > > > > It looks like GCC is happy to give us the function-entry-time FP if we use > > > > __builtin_frame_address(1), > > > > > > From the GCC manual: > > > Calling this function with a nonzero argument can have > > > unpredictable effects, including crashing the calling program. As > > > a result, calls that are considered unsafe are diagnosed when the > > > '-Wframe-address' option is in effect. Such calls should only be > > > made in debugging situations. > > > > > > It *does* warn (the warning is in -Wall btw), on both powerpc and > > > aarch64. Furthermore, using this builtin causes lousy code (it forces > > > the use of a frame pointer, which we normally try very hard to optimise > > > away, for good reason). > > > > > > And, that warning is not an idle warning. Non-zero arguments to > > > __builtin_frame_address can crash the program. It won't on simpler > > > functions, but there is no real definition of what a simpler function > > > *is*. It is meant for debugging, not for production use (this is also > > > why no one has bothered to make it faster). > > > > > > On Power it should work, but on pretty much any other arch it won't. > > > > I understand this is true generally, and cannot be relied upon in > > portable code. However as you hint here for Power, I believe that on > > arm64 __builtin_frame_address(1) shouldn't crash the program due to the > > way frame records work on arm64, but I'll go check with some local > > compiler folk. I agree that __builtin_frame_address(2) and beyond > > certainly can, e.g. by NULL dereference and similar. > > I still do not know the aarch64 ABI well enough. If only I had time! > > > For context, why do you think this would work on power specifically? I > > wonder if our rationale is similar. > > On most 64-bit Power ABIs all stack frames are connected together as a > linked list (which is updated atomically, importantly). This makes it > possible to always find all previous stack frames. We have something similar on arm64, where the kernel depends on being built with a frame pointer following the AAPCS frame pointer rules. Every stack frame contains a "frame record" *somewhere* within that stack frame, and the frame records are chained together as a linked list. The frame pointer points at the most recent frame record (and this is what __builtin_frame_address(0) returns). The records themselves are basically: struct record { struct record *next; unsigned long ret_addr; }; At function call boundaries, we know that the FP is the caller's record (or NULL for the first function), and the LR is the address the current function should return to. Within a function with a stack frame, we can access that function's record and the `next` field (equivalent to the FP at the time of entry to the function) is what __builtin_frame_address(1) should return. > > Are you aware of anything in particular that breaks using > > __builtin_frame_address(1) in non-portable code, or is this just a > > general sentiment of this not being a supported use-case? > > It is not supported, and trying to do it anyway can crash: it can use > random stack contents as pointer! Not really "random" of course, but > where it thinks to find a pointer into the previous frame, which is not > something it can rely on (unless the ABI guarantees it somehow). > > See gcc.gnu.org/PR60109 for example. Sure; I see that being true generally (and Ramana noted that on 32-bit arm a frame pointer wasn't mandated), but I think in this case we have a stronger target (and configuration) specific guarantee. > > > > Unless we can get some strong guarantees from compiler folk such that we > > > > can guarantee a specific function acts boundary for unwinding (and > > > > doesn't itself get split, etc), the only reliable way I can think to > > > > solve this requires an assembly trampoline. Whatever we do is liable to > > > > need some invasive rework. > > > > > > You cannot get such a guarantee, other than not letting the compiler > > > see into the routine at all, like with assembler code (not inline asm, > > > real assembler code). > > > > If we cannot reliably ensure this then I'm happy to go write an assembly > > trampoline to snapshot the state at a function call boundary (where our > > procedure call standard mandates the state of the LR, FP, and frame > > records pointed to by the FP). > > Is the frame pointer required?! The arm64 Linux port mandates frame pointers for kernel code. It is generally possible to build code without frame pointers (e.g. userspace), but doing that for kernel code would be a bug. > > This'll require reworking a reasonable > > amount of code cross-architecture, so I'll need to get some more > > concrete justification (e.g. examples of things that can go wrong in > > practice). > > Say you have a function that does dynamic stack allocation, then there > is usually no way to find the previous stack frame (without function- > specific knowledge). So __builtin_frame_address cannot work (it knows > nothing about frames further up). > > Dynamic stack allocation (alloca, or variable length automatic arrays) > is just the most common and most convenient example; it is not the only > case you have problems here. I agree with those as general concerns, but I don't think that affects arm64's frame records, since their location within a stack frame is immaterial given the chaining. > > > The real way forward is to bite the bullet and to no longer pretend you > > > can do a full backtrace from just the stack contents. You cannot. > > > > I think what you mean here is that there's no reliable way to handle the > > current/leaf function, right? If so I do agree. > > No, I meant what I said. > > There is the separate issue that you do not know where the return > address (etc.) is stored in a function that has not yet done a call > itself, sure. You cannot assume anything the ABI does not tell you you > can depend on. This is in the frame record per the AAPCS. > > Beyond that I believe that arm64's frame records should be sufficient. > > Do you have a simple linked list connecting all frames? Yes. Thanks, Mark. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A24F0C433DB for ; Wed, 10 Mar 2021 11:32:56 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CA63864FD7 for ; Wed, 10 Mar 2021 11:32:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA63864FD7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4DwVLZ1DH0z3cW9 for ; Wed, 10 Mar 2021 22:32:54 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=arm.com (client-ip=217.140.110.172; helo=foss.arm.com; envelope-from=mark.rutland@arm.com; receiver=) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lists.ozlabs.org (Postfix) with ESMTP id 4DwVLC5mqPz2yjB for ; Wed, 10 Mar 2021 22:32:32 +1100 (AEDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2A6BF1063; Wed, 10 Mar 2021 03:32:29 -0800 (PST) Received: from C02TD0UTHF1T.local (unknown [10.57.52.108]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BF2773F85F; Wed, 10 Mar 2021 03:32:26 -0800 (PST) Date: Wed, 10 Mar 2021 11:32:20 +0000 From: Mark Rutland To: Segher Boessenkool Subject: Re: [PATCH v1] powerpc: Include running function as first entry in save_stack_trace() and friends Message-ID: <20210310112441.GA19619@C02TD0UTHF1T.local> References: <1802be3e-dc1a-52e0-1754-a40f0ea39658@csgroup.eu> <20210304145730.GC54534@C02TD0UTHF1T.local> <20210304215448.GU29191@gate.crashing.org> <20210309160505.GA4979@C02TD0UTHF1T.local> <20210309220532.GI29191@gate.crashing.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210309220532.GI29191@gate.crashing.org> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marco Elver , Catalin Marinas , linuxppc-dev@lists.ozlabs.org, LKML , kasan-dev , broonie@kernel.org, Paul Mackerras , Will Deacon , linux-arm-kernel@lists.infradead.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Tue, Mar 09, 2021 at 04:05:32PM -0600, Segher Boessenkool wrote: > Hi! > > On Tue, Mar 09, 2021 at 04:05:23PM +0000, Mark Rutland wrote: > > On Thu, Mar 04, 2021 at 03:54:48PM -0600, Segher Boessenkool wrote: > > > On Thu, Mar 04, 2021 at 02:57:30PM +0000, Mark Rutland wrote: > > > > It looks like GCC is happy to give us the function-entry-time FP if we use > > > > __builtin_frame_address(1), > > > > > > From the GCC manual: > > > Calling this function with a nonzero argument can have > > > unpredictable effects, including crashing the calling program. As > > > a result, calls that are considered unsafe are diagnosed when the > > > '-Wframe-address' option is in effect. Such calls should only be > > > made in debugging situations. > > > > > > It *does* warn (the warning is in -Wall btw), on both powerpc and > > > aarch64. Furthermore, using this builtin causes lousy code (it forces > > > the use of a frame pointer, which we normally try very hard to optimise > > > away, for good reason). > > > > > > And, that warning is not an idle warning. Non-zero arguments to > > > __builtin_frame_address can crash the program. It won't on simpler > > > functions, but there is no real definition of what a simpler function > > > *is*. It is meant for debugging, not for production use (this is also > > > why no one has bothered to make it faster). > > > > > > On Power it should work, but on pretty much any other arch it won't. > > > > I understand this is true generally, and cannot be relied upon in > > portable code. However as you hint here for Power, I believe that on > > arm64 __builtin_frame_address(1) shouldn't crash the program due to the > > way frame records work on arm64, but I'll go check with some local > > compiler folk. I agree that __builtin_frame_address(2) and beyond > > certainly can, e.g. by NULL dereference and similar. > > I still do not know the aarch64 ABI well enough. If only I had time! > > > For context, why do you think this would work on power specifically? I > > wonder if our rationale is similar. > > On most 64-bit Power ABIs all stack frames are connected together as a > linked list (which is updated atomically, importantly). This makes it > possible to always find all previous stack frames. We have something similar on arm64, where the kernel depends on being built with a frame pointer following the AAPCS frame pointer rules. Every stack frame contains a "frame record" *somewhere* within that stack frame, and the frame records are chained together as a linked list. The frame pointer points at the most recent frame record (and this is what __builtin_frame_address(0) returns). The records themselves are basically: struct record { struct record *next; unsigned long ret_addr; }; At function call boundaries, we know that the FP is the caller's record (or NULL for the first function), and the LR is the address the current function should return to. Within a function with a stack frame, we can access that function's record and the `next` field (equivalent to the FP at the time of entry to the function) is what __builtin_frame_address(1) should return. > > Are you aware of anything in particular that breaks using > > __builtin_frame_address(1) in non-portable code, or is this just a > > general sentiment of this not being a supported use-case? > > It is not supported, and trying to do it anyway can crash: it can use > random stack contents as pointer! Not really "random" of course, but > where it thinks to find a pointer into the previous frame, which is not > something it can rely on (unless the ABI guarantees it somehow). > > See gcc.gnu.org/PR60109 for example. Sure; I see that being true generally (and Ramana noted that on 32-bit arm a frame pointer wasn't mandated), but I think in this case we have a stronger target (and configuration) specific guarantee. > > > > Unless we can get some strong guarantees from compiler folk such that we > > > > can guarantee a specific function acts boundary for unwinding (and > > > > doesn't itself get split, etc), the only reliable way I can think to > > > > solve this requires an assembly trampoline. Whatever we do is liable to > > > > need some invasive rework. > > > > > > You cannot get such a guarantee, other than not letting the compiler > > > see into the routine at all, like with assembler code (not inline asm, > > > real assembler code). > > > > If we cannot reliably ensure this then I'm happy to go write an assembly > > trampoline to snapshot the state at a function call boundary (where our > > procedure call standard mandates the state of the LR, FP, and frame > > records pointed to by the FP). > > Is the frame pointer required?! The arm64 Linux port mandates frame pointers for kernel code. It is generally possible to build code without frame pointers (e.g. userspace), but doing that for kernel code would be a bug. > > This'll require reworking a reasonable > > amount of code cross-architecture, so I'll need to get some more > > concrete justification (e.g. examples of things that can go wrong in > > practice). > > Say you have a function that does dynamic stack allocation, then there > is usually no way to find the previous stack frame (without function- > specific knowledge). So __builtin_frame_address cannot work (it knows > nothing about frames further up). > > Dynamic stack allocation (alloca, or variable length automatic arrays) > is just the most common and most convenient example; it is not the only > case you have problems here. I agree with those as general concerns, but I don't think that affects arm64's frame records, since their location within a stack frame is immaterial given the chaining. > > > The real way forward is to bite the bullet and to no longer pretend you > > > can do a full backtrace from just the stack contents. You cannot. > > > > I think what you mean here is that there's no reliable way to handle the > > current/leaf function, right? If so I do agree. > > No, I meant what I said. > > There is the separate issue that you do not know where the return > address (etc.) is stored in a function that has not yet done a call > itself, sure. You cannot assume anything the ABI does not tell you you > can depend on. This is in the frame record per the AAPCS. > > Beyond that I believe that arm64's frame records should be sufficient. > > Do you have a simple linked list connecting all frames? Yes. Thanks, Mark. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 123D8C433DB for ; Wed, 10 Mar 2021 11:35:07 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7908D64F59 for ; Wed, 10 Mar 2021 11:35:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7908D64F59 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ek8bp/5WTEUHexrFOkr+JLiXr1U8LL+6XPHvfy1SqJQ=; b=ThtOAXmBKWkPCz0M6hoLsVsE+ aHA0SO8Ty4gfHu/WUuYn34lHf6K51UkWDbZUaZEyTL5LuZ+R27h6vxhHj4BHD9HRLFLHjsBIx7MId MCHvHrWDlrnbZU0upblbCt+p/s6py8Ni9dVkm9jt07vhd80JshNG/4ifEk2kDUFR/GayvQkQZzsZV vKiWA8/l01gWnDxttL8CxGqIZ97fEyFptgtXDMT+y8wP10k4U7vqqDQwC4ajXVsWGkCo5lSe8mnPr 2QfYXOvBsZwG0BSdODZTQRahNVWbud5wjFFJG7nTdYgzfqZhpilARXCXJ9B3iet3vNDDK19p66wU9 dGTPkZbDg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJx55-006img-M3; Wed, 10 Mar 2021 11:32:49 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJx4q-006im6-5U for linux-arm-kernel@lists.infradead.org; Wed, 10 Mar 2021 11:32:36 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2A6BF1063; Wed, 10 Mar 2021 03:32:29 -0800 (PST) Received: from C02TD0UTHF1T.local (unknown [10.57.52.108]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BF2773F85F; Wed, 10 Mar 2021 03:32:26 -0800 (PST) Date: Wed, 10 Mar 2021 11:32:20 +0000 From: Mark Rutland To: Segher Boessenkool Cc: Marco Elver , Catalin Marinas , Will Deacon , LKML , broonie@kernel.org, Paul Mackerras , kasan-dev , linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v1] powerpc: Include running function as first entry in save_stack_trace() and friends Message-ID: <20210310112441.GA19619@C02TD0UTHF1T.local> References: <1802be3e-dc1a-52e0-1754-a40f0ea39658@csgroup.eu> <20210304145730.GC54534@C02TD0UTHF1T.local> <20210304215448.GU29191@gate.crashing.org> <20210309160505.GA4979@C02TD0UTHF1T.local> <20210309220532.GI29191@gate.crashing.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210309220532.GI29191@gate.crashing.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210310_113234_275995_E9884F81 X-CRM114-Status: GOOD ( 55.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Mar 09, 2021 at 04:05:32PM -0600, Segher Boessenkool wrote: > Hi! > > On Tue, Mar 09, 2021 at 04:05:23PM +0000, Mark Rutland wrote: > > On Thu, Mar 04, 2021 at 03:54:48PM -0600, Segher Boessenkool wrote: > > > On Thu, Mar 04, 2021 at 02:57:30PM +0000, Mark Rutland wrote: > > > > It looks like GCC is happy to give us the function-entry-time FP if we use > > > > __builtin_frame_address(1), > > > > > > From the GCC manual: > > > Calling this function with a nonzero argument can have > > > unpredictable effects, including crashing the calling program. As > > > a result, calls that are considered unsafe are diagnosed when the > > > '-Wframe-address' option is in effect. Such calls should only be > > > made in debugging situations. > > > > > > It *does* warn (the warning is in -Wall btw), on both powerpc and > > > aarch64. Furthermore, using this builtin causes lousy code (it forces > > > the use of a frame pointer, which we normally try very hard to optimise > > > away, for good reason). > > > > > > And, that warning is not an idle warning. Non-zero arguments to > > > __builtin_frame_address can crash the program. It won't on simpler > > > functions, but there is no real definition of what a simpler function > > > *is*. It is meant for debugging, not for production use (this is also > > > why no one has bothered to make it faster). > > > > > > On Power it should work, but on pretty much any other arch it won't. > > > > I understand this is true generally, and cannot be relied upon in > > portable code. However as you hint here for Power, I believe that on > > arm64 __builtin_frame_address(1) shouldn't crash the program due to the > > way frame records work on arm64, but I'll go check with some local > > compiler folk. I agree that __builtin_frame_address(2) and beyond > > certainly can, e.g. by NULL dereference and similar. > > I still do not know the aarch64 ABI well enough. If only I had time! > > > For context, why do you think this would work on power specifically? I > > wonder if our rationale is similar. > > On most 64-bit Power ABIs all stack frames are connected together as a > linked list (which is updated atomically, importantly). This makes it > possible to always find all previous stack frames. We have something similar on arm64, where the kernel depends on being built with a frame pointer following the AAPCS frame pointer rules. Every stack frame contains a "frame record" *somewhere* within that stack frame, and the frame records are chained together as a linked list. The frame pointer points at the most recent frame record (and this is what __builtin_frame_address(0) returns). The records themselves are basically: struct record { struct record *next; unsigned long ret_addr; }; At function call boundaries, we know that the FP is the caller's record (or NULL for the first function), and the LR is the address the current function should return to. Within a function with a stack frame, we can access that function's record and the `next` field (equivalent to the FP at the time of entry to the function) is what __builtin_frame_address(1) should return. > > Are you aware of anything in particular that breaks using > > __builtin_frame_address(1) in non-portable code, or is this just a > > general sentiment of this not being a supported use-case? > > It is not supported, and trying to do it anyway can crash: it can use > random stack contents as pointer! Not really "random" of course, but > where it thinks to find a pointer into the previous frame, which is not > something it can rely on (unless the ABI guarantees it somehow). > > See gcc.gnu.org/PR60109 for example. Sure; I see that being true generally (and Ramana noted that on 32-bit arm a frame pointer wasn't mandated), but I think in this case we have a stronger target (and configuration) specific guarantee. > > > > Unless we can get some strong guarantees from compiler folk such that we > > > > can guarantee a specific function acts boundary for unwinding (and > > > > doesn't itself get split, etc), the only reliable way I can think to > > > > solve this requires an assembly trampoline. Whatever we do is liable to > > > > need some invasive rework. > > > > > > You cannot get such a guarantee, other than not letting the compiler > > > see into the routine at all, like with assembler code (not inline asm, > > > real assembler code). > > > > If we cannot reliably ensure this then I'm happy to go write an assembly > > trampoline to snapshot the state at a function call boundary (where our > > procedure call standard mandates the state of the LR, FP, and frame > > records pointed to by the FP). > > Is the frame pointer required?! The arm64 Linux port mandates frame pointers for kernel code. It is generally possible to build code without frame pointers (e.g. userspace), but doing that for kernel code would be a bug. > > This'll require reworking a reasonable > > amount of code cross-architecture, so I'll need to get some more > > concrete justification (e.g. examples of things that can go wrong in > > practice). > > Say you have a function that does dynamic stack allocation, then there > is usually no way to find the previous stack frame (without function- > specific knowledge). So __builtin_frame_address cannot work (it knows > nothing about frames further up). > > Dynamic stack allocation (alloca, or variable length automatic arrays) > is just the most common and most convenient example; it is not the only > case you have problems here. I agree with those as general concerns, but I don't think that affects arm64's frame records, since their location within a stack frame is immaterial given the chaining. > > > The real way forward is to bite the bullet and to no longer pretend you > > > can do a full backtrace from just the stack contents. You cannot. > > > > I think what you mean here is that there's no reliable way to handle the > > current/leaf function, right? If so I do agree. > > No, I meant what I said. > > There is the separate issue that you do not know where the return > address (etc.) is stored in a function that has not yet done a call > itself, sure. You cannot assume anything the ABI does not tell you you > can depend on. This is in the frame record per the AAPCS. > > Beyond that I believe that arm64's frame records should be sufficient. > > Do you have a simple linked list connecting all frames? Yes. Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel