From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 019A1C433EF for ; Mon, 11 Oct 2021 18:45:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D463A60F0F for ; Mon, 11 Oct 2021 18:45:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232719AbhJKSri (ORCPT ); Mon, 11 Oct 2021 14:47:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229542AbhJKSrg (ORCPT ); Mon, 11 Oct 2021 14:47:36 -0400 Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FF2CC061570 for ; Mon, 11 Oct 2021 11:45:36 -0700 (PDT) Received: by mail-lf1-x132.google.com with SMTP id u18so77186117lfd.12 for ; Mon, 11 Oct 2021 11:45:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pchvvATNbk/8sMncxiiN9k1TbKkT0wEgMkyPDBPDVzw=; b=ZncLCafilGU3tbBlGe8oY8z+FbwIimNfmXZZByooJEci9fFTSwKdArncsKw0Qshi8D KUpWqOvO9gwSUGOkYntqxbZ5uY/378EDzdvVljc7P218vvcRQl141QmwGP1NZqjygjn2 qTDD1XoKtPOGVPzZ/yu7BWYrh61VhQVN9t80SAc2PH7kkmBUw47GZFUeNtAtR2cro51C IK4Clulz6aKBfwIw+qLiCzEQ5HX7K4AEq9yqGzf6P9iQX6pfnLu5K/1vb+DEUll/JqHU wAuPIpLrX+GI+ul2LzPFqkoPCqK0SkA30vJfHL2tLuW6F7RqCJJwB26bnUTq6EHJLRTu vkDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pchvvATNbk/8sMncxiiN9k1TbKkT0wEgMkyPDBPDVzw=; b=0uGUksPlSeX5V68PYb/i8B1m7CXFD5GZEruaXNWOiRkLLj9l2eV0xTlwymKg2Ngc73 pYE4K8Qep3DibzyXDmTpz9mnsRyEIaFqfbOdZPBO17X42oAUslJ5DjJbOyTx8ZQDIiNt OzaWHQZeuRbpwcYZ7/zSKKVzFjT8IoSnL7cd576FEMChsjTMTDlG95q30FDMjuuE2plj KLgl4iqtiaProBsQL4Qc99b75+vdkf46ffSwYFXAMu6u5TtShO6Pn6WIaDrMHbUXdIDB fSESO55rTKTRiSR5jZCC7IZqhFl1gIvuT1tiUXCMDZ1B4PH4VMBexin20OgtnDrVa2UG dn1g== X-Gm-Message-State: AOAM5316JQOVdTjg2chrbFzo/ZZ5f9EIzTrAG4q5XITXIx/44HYD+K3O +Vd6iCBEoq3Y3XpRWli+HlwfSdPPgU+VP9/fxiIZDkkj7q0isg== X-Google-Smtp-Source: ABdhPJxnc6NOysL6L51/L6pRT+8yki6z66uJ0eFrosskIU82EVx0kymz+KCrsBNJrLmnl+Nto7Jm2+yNPsnf45Rc4gs= X-Received: by 2002:a05:6512:1103:: with SMTP id l3mr14202886lfg.550.1633977934045; Mon, 11 Oct 2021 11:45:34 -0700 (PDT) MIME-Version: 1.0 References: <163369609308.636038.15295764725220907794.stgit@devnote2> <163369614818.636038.5019945597127474028.stgit@devnote2> In-Reply-To: <163369614818.636038.5019945597127474028.stgit@devnote2> From: Nick Desaulniers Date: Mon, 11 Oct 2021 11:45:22 -0700 Message-ID: Subject: Re: [PATCH 6/8] ARM: clang: Do not relay on lr register for stacktrace To: Masami Hiramatsu Cc: Steven Rostedt , "Naveen N . Rao" , Ananth N Mavinakayanahalli , Ingo Molnar , linux-kernel@vger.kernel.org, Sven Schnelle , Catalin Marinas , Will Deacon , Russell King , Nathan Chancellor , linux-arm-kernel@lists.infradead.org, Nathan Huckleberry Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Oct 8, 2021 at 5:29 AM Masami Hiramatsu wrote: > > Currently the stacktrace on clang compiled arm kernel uses the 'lr' > register to find the first frame address from pt_regs. However, that > is wrong after calling another function, because the 'lr' register > is used by 'bl' instruction and never be recovered. > > As same as gcc arm kernel, directly use the frame pointer (x11) of > the pt_regs to find the first frame address. Hi Masami, Thanks for the patch. Testing with ARCH=arm defconfig (multi_v7_defconfig) Before this patch: $ mount -t proc /proc $ echo 0 > /proc/sys/kernel/kptr_restrict $ cat /proc/self/stack [<0>] proc_single_show+0x4c/0xb8 [<0>] seq_read_iter+0x174/0x4d8 [<0>] seq_read+0x134/0x158 [<0>] vfs_read+0xcc/0x2f8 [<0>] ksys_read+0x74/0xd0 [<0>] __entry_text_start+0x14/0x14 [<0>] 0xbea38cc0 After this patch: $ mount -t proc /proc $ echo 0 > /proc/sys/kernel/kptr_restrict $ cat /proc/self/stack [<0>] proc_single_show+0x4c/0xb8 [<0>] seq_read_iter+0x174/0x4d8 [<0>] seq_read+0x134/0x158 [<0>] vfs_read+0xcc/0x2f8 [<0>] ksys_read+0x74/0xd0 [<0>] __entry_text_start+0x14/0x14 [<0>] 0xbeb55cc0 Is there a different way to test/verify this patch? (I'm pretty sure we had verified the WARN_ONCE functionality with this, too.) If I change from CONFIG_UNWINDER_ARM=y to CONFIG_UNWINDER_FRAME_POINTER=y, before: # cat /proc/self/stack [<0>] stack_trace_save_tsk+0x50/0x6c [<0>] proc_pid_stack+0xa0/0xf8 [<0>] proc_single_show+0x50/0xbc [<0>] seq_read_iter+0x178/0x4ec [<0>] seq_read+0x138/0x15c [<0>] vfs_read+0xd0/0x304 [<0>] ksys_read+0x78/0xd4 [<0>] sys_read+0xc/0x10 after: # cat /proc/self/stack [<0>] proc_pid_stack+0xa0/0xf8 [<0>] proc_single_show+0x50/0xbc [<0>] seq_read_iter+0x178/0x4ec [<0>] seq_read+0x138/0x15c [<0>] vfs_read+0xd0/0x304 [<0>] ksys_read+0x78/0xd4 [<0>] sys_read+0xc/0x10 [<0>] __entry_text_start+0x14/0x14 [<0>] 0xffffffff So I guess this helps the CONFIG_UNWINDER_FRAME_POINTER=y case? (That final frame address looks wrong, but is potentially yet another bug; perhaps for clang we need to manually store the previous frame's pc at a different offset before jumping to __entry_text_start). Also, I'm curious about CONFIG_THUMB2_KERNEL (forces CONFIG_UNWINDER_ARM=y). before: # cat /proc/self/stack [<0>] proc_single_show+0x31/0x86 [<0>] seq_read_iter+0xff/0x326 [<0>] seq_read+0xd7/0xf2 [<0>] vfs_read+0x93/0x20e [<0>] ksys_read+0x53/0x92 [<0>] ret_fast_syscall+0x1/0x52 [<0>] 0xbe9a9cc0 after: # cat /proc/self/stack [<0>] proc_single_show+0x31/0x86 [<0>] seq_read_iter+0xff/0x326 [<0>] seq_read+0xd7/0xf2 [<0>] vfs_read+0x93/0x20e [<0>] ksys_read+0x53/0x92 [<0>] ret_fast_syscall+0x1/0x52 [<0>] 0xbec08cc0 Tested-by: Nick Desaulniers so likely this fixes/improves CONFIG_UNWINDER_FRAME_POINTER=y? Is that correct? > > Signed-off-by: Masami Hiramatsu > --- > arch/arm/kernel/stacktrace.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/arch/arm/kernel/stacktrace.c b/arch/arm/kernel/stacktrace.c > index 76ea4178a55c..db798eac7431 100644 > --- a/arch/arm/kernel/stacktrace.c > +++ b/arch/arm/kernel/stacktrace.c > @@ -54,8 +54,7 @@ int notrace unwind_frame(struct stackframe *frame) > > frame->sp = frame->fp; > frame->fp = *(unsigned long *)(fp); > - frame->pc = frame->lr; > - frame->lr = *(unsigned long *)(fp + 4); > + frame->pc = *(unsigned long *)(fp + 4); > #else > /* check current frame pointer is within bounds */ > if (fp < low + 12 || fp > high - 4) > -- Thanks, ~Nick Desaulniers From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39820C433F5 for ; Mon, 11 Oct 2021 18:48:36 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EEFD860E54 for ; Mon, 11 Oct 2021 18:48:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org EEFD860E54 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=q4pSpp0RmsIiqjCQMePIlu0Gh7fwZsysr43N6kvbel4=; b=avFskwRPzGdGKz nsT/BEj/FjBTmOBoQqHYXsf0p4SGILjVRONEZJAOyro33nTNRUSeWn/7k8GVSg7qC7IFTAGH8Z/2i mHaPOXLX+aXB7S5LZUkKdfSty3XZuv3R6DdgLz1U2aJO+P3OuDiHVlE2mHv+YlZMAyuxBdrVd1qpo nx1vih5O54rPBLNsbRcl1JU6HmRRcGyQkiAxt4JU3VfwUzYc+8nzmELNzR+hd3eyC3H57l8LhIQiv ofUZnFjx0UnYC0y/RCiKXnyeOlxkBxeXG+QBx5le1alR14VnGjrmI5BizI5upQxJKa5yxXD5x3BLa 5elGUu6iAZiJ0/Zxfmag==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ma0It-00APj6-Tc; Mon, 11 Oct 2021 18:45:40 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ma0Ip-00APiP-Ry for linux-arm-kernel@lists.infradead.org; Mon, 11 Oct 2021 18:45:37 +0000 Received: by mail-lf1-x12d.google.com with SMTP id t9so76031679lfd.1 for ; Mon, 11 Oct 2021 11:45:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pchvvATNbk/8sMncxiiN9k1TbKkT0wEgMkyPDBPDVzw=; b=ZncLCafilGU3tbBlGe8oY8z+FbwIimNfmXZZByooJEci9fFTSwKdArncsKw0Qshi8D KUpWqOvO9gwSUGOkYntqxbZ5uY/378EDzdvVljc7P218vvcRQl141QmwGP1NZqjygjn2 qTDD1XoKtPOGVPzZ/yu7BWYrh61VhQVN9t80SAc2PH7kkmBUw47GZFUeNtAtR2cro51C IK4Clulz6aKBfwIw+qLiCzEQ5HX7K4AEq9yqGzf6P9iQX6pfnLu5K/1vb+DEUll/JqHU wAuPIpLrX+GI+ul2LzPFqkoPCqK0SkA30vJfHL2tLuW6F7RqCJJwB26bnUTq6EHJLRTu vkDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pchvvATNbk/8sMncxiiN9k1TbKkT0wEgMkyPDBPDVzw=; b=rGcSGGwJtWS2N9zDmV28kY5ffwd9prP6etAU9obFfVM2mxmAU1+68memoHXEj19oPP 3zEAGx8v69N1/pkqrmBRIRo/3Z1Ggc0u+u+eCBdXwn/Dt+eL4Df3ncPmh/r9TZ46qi0V xJps1kQNce75Z1Gsm23MiMucyobn0LboajbYaeg9n4JVJvf/73ksKNl8QlRJ+gmIXqpi aY5AtgFs1+uoL48Xia+woH8l/i2wr40iTz54TNtiAbL737XLtpVLDZQVk29ElM8ef8iZ pX8fOT43bE0oOxh9id3wOC1gfrOFLYBfC8maGtD5Zs70CJOgePv0JwrYitXjgjumm2NU QjmQ== X-Gm-Message-State: AOAM5321jFvV4hOfBR8p3iWpqtWAVy18QpX07is3jLhjc0WdIawV+9Xo /3aNvwHjmeIewEPtjJNekXCdfjslZnqX4HMaMZTYvQ== X-Google-Smtp-Source: ABdhPJxnc6NOysL6L51/L6pRT+8yki6z66uJ0eFrosskIU82EVx0kymz+KCrsBNJrLmnl+Nto7Jm2+yNPsnf45Rc4gs= X-Received: by 2002:a05:6512:1103:: with SMTP id l3mr14202886lfg.550.1633977934045; Mon, 11 Oct 2021 11:45:34 -0700 (PDT) MIME-Version: 1.0 References: <163369609308.636038.15295764725220907794.stgit@devnote2> <163369614818.636038.5019945597127474028.stgit@devnote2> In-Reply-To: <163369614818.636038.5019945597127474028.stgit@devnote2> From: Nick Desaulniers Date: Mon, 11 Oct 2021 11:45:22 -0700 Message-ID: Subject: Re: [PATCH 6/8] ARM: clang: Do not relay on lr register for stacktrace To: Masami Hiramatsu Cc: Steven Rostedt , "Naveen N . Rao" , Ananth N Mavinakayanahalli , Ingo Molnar , linux-kernel@vger.kernel.org, Sven Schnelle , Catalin Marinas , Will Deacon , Russell King , Nathan Chancellor , linux-arm-kernel@lists.infradead.org, Nathan Huckleberry X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211011_114535_935513_9915A476 X-CRM114-Status: GOOD ( 24.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Oct 8, 2021 at 5:29 AM Masami Hiramatsu wrote: > > Currently the stacktrace on clang compiled arm kernel uses the 'lr' > register to find the first frame address from pt_regs. However, that > is wrong after calling another function, because the 'lr' register > is used by 'bl' instruction and never be recovered. > > As same as gcc arm kernel, directly use the frame pointer (x11) of > the pt_regs to find the first frame address. Hi Masami, Thanks for the patch. Testing with ARCH=arm defconfig (multi_v7_defconfig) Before this patch: $ mount -t proc /proc $ echo 0 > /proc/sys/kernel/kptr_restrict $ cat /proc/self/stack [<0>] proc_single_show+0x4c/0xb8 [<0>] seq_read_iter+0x174/0x4d8 [<0>] seq_read+0x134/0x158 [<0>] vfs_read+0xcc/0x2f8 [<0>] ksys_read+0x74/0xd0 [<0>] __entry_text_start+0x14/0x14 [<0>] 0xbea38cc0 After this patch: $ mount -t proc /proc $ echo 0 > /proc/sys/kernel/kptr_restrict $ cat /proc/self/stack [<0>] proc_single_show+0x4c/0xb8 [<0>] seq_read_iter+0x174/0x4d8 [<0>] seq_read+0x134/0x158 [<0>] vfs_read+0xcc/0x2f8 [<0>] ksys_read+0x74/0xd0 [<0>] __entry_text_start+0x14/0x14 [<0>] 0xbeb55cc0 Is there a different way to test/verify this patch? (I'm pretty sure we had verified the WARN_ONCE functionality with this, too.) If I change from CONFIG_UNWINDER_ARM=y to CONFIG_UNWINDER_FRAME_POINTER=y, before: # cat /proc/self/stack [<0>] stack_trace_save_tsk+0x50/0x6c [<0>] proc_pid_stack+0xa0/0xf8 [<0>] proc_single_show+0x50/0xbc [<0>] seq_read_iter+0x178/0x4ec [<0>] seq_read+0x138/0x15c [<0>] vfs_read+0xd0/0x304 [<0>] ksys_read+0x78/0xd4 [<0>] sys_read+0xc/0x10 after: # cat /proc/self/stack [<0>] proc_pid_stack+0xa0/0xf8 [<0>] proc_single_show+0x50/0xbc [<0>] seq_read_iter+0x178/0x4ec [<0>] seq_read+0x138/0x15c [<0>] vfs_read+0xd0/0x304 [<0>] ksys_read+0x78/0xd4 [<0>] sys_read+0xc/0x10 [<0>] __entry_text_start+0x14/0x14 [<0>] 0xffffffff So I guess this helps the CONFIG_UNWINDER_FRAME_POINTER=y case? (That final frame address looks wrong, but is potentially yet another bug; perhaps for clang we need to manually store the previous frame's pc at a different offset before jumping to __entry_text_start). Also, I'm curious about CONFIG_THUMB2_KERNEL (forces CONFIG_UNWINDER_ARM=y). before: # cat /proc/self/stack [<0>] proc_single_show+0x31/0x86 [<0>] seq_read_iter+0xff/0x326 [<0>] seq_read+0xd7/0xf2 [<0>] vfs_read+0x93/0x20e [<0>] ksys_read+0x53/0x92 [<0>] ret_fast_syscall+0x1/0x52 [<0>] 0xbe9a9cc0 after: # cat /proc/self/stack [<0>] proc_single_show+0x31/0x86 [<0>] seq_read_iter+0xff/0x326 [<0>] seq_read+0xd7/0xf2 [<0>] vfs_read+0x93/0x20e [<0>] ksys_read+0x53/0x92 [<0>] ret_fast_syscall+0x1/0x52 [<0>] 0xbec08cc0 Tested-by: Nick Desaulniers so likely this fixes/improves CONFIG_UNWINDER_FRAME_POINTER=y? Is that correct? > > Signed-off-by: Masami Hiramatsu > --- > arch/arm/kernel/stacktrace.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/arch/arm/kernel/stacktrace.c b/arch/arm/kernel/stacktrace.c > index 76ea4178a55c..db798eac7431 100644 > --- a/arch/arm/kernel/stacktrace.c > +++ b/arch/arm/kernel/stacktrace.c > @@ -54,8 +54,7 @@ int notrace unwind_frame(struct stackframe *frame) > > frame->sp = frame->fp; > frame->fp = *(unsigned long *)(fp); > - frame->pc = frame->lr; > - frame->lr = *(unsigned long *)(fp + 4); > + frame->pc = *(unsigned long *)(fp + 4); > #else > /* check current frame pointer is within bounds */ > if (fp < low + 12 || fp > high - 4) > -- Thanks, ~Nick Desaulniers _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel