From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCE6CC433B4 for ; Fri, 21 May 2021 08:45:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 97F4A61132 for ; Fri, 21 May 2021 08:45:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232785AbhEUIq6 (ORCPT ); Fri, 21 May 2021 04:46:58 -0400 Received: from perceval.ideasonboard.com ([213.167.242.64]:47030 "EHLO perceval.ideasonboard.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230442AbhEUIq6 (ORCPT ); Fri, 21 May 2021 04:46:58 -0400 Received: from [192.168.1.111] (91-157-208-71.elisa-laajakaista.fi [91.157.208.71]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C43428D8; Fri, 21 May 2021 10:45:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1621586731; bh=WIs/t3U/kddeRP0vSywJKonvx676D72juDfNaO7cqkc=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=e2Un5NxX3Gh2pQL1HLaRtlF3OdVX0EDoBOp+zNnXP0/bflRKoEcQKUvMxDWIG2mws HAASKaYBq+Z42l6/U7fjdHhT+HSlKsGHQxEtnXk43XoPfKwnjhY4rE0XPD9a2gghOq fg/mPxubxUH/wVRo+ZUtWNXLXbpakntB/0GzLMVE= Subject: Re: Random stack corruption on v5.13 with dra76 To: Tony Lindgren Cc: linux-arm-kernel@lists.infradead.org, Nishanth Menon , "Bajjuri, Praneeth" , linux-omap@vger.kernel.org References: <0f48c7e5-6acd-1143-35ef-3dea2255bec6@ideasonboard.com> From: Tomi Valkeinen Message-ID: <064a9324-cfcf-47b9-6ae3-a29085a52683@ideasonboard.com> Date: Fri, 21 May 2021 11:45:29 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-omap@vger.kernel.org On 21/05/2021 10:39, Tony Lindgren wrote: > * Tomi Valkeinen [210521 07:05]: >> On 21/05/2021 08:36, Tony Lindgren wrote: >>> * Tomi Valkeinen [210520 08:27]: >>>> Hi, >>>> >>>> I've noticed that the v5.13 rcs crash randomly (but quite often) on dra76 evm >>>> (I haven't tested other boards). Anyone else seen this problem? >>> >>> I have not seen this so far and beagle-x15 is behaving for me. >>> >>> Does it always happen on boot? >> >> No, but quite often. I can't really say how often, as it's annoyingly random. >> I tried to bisect, but that proved to be difficult as sometimes I get multiple (5+) >> successful boots before the crash. >> >> I tested with x15, same issue (below). So... Something in my kernel config? Or compiler? >> Looks like the crash happens always very soon after (or during) probing palmas. > > After about 10 reboots with your .config I'm seeing it now too on > beagle-x15. So far no luck reproducing it with omap2plus_defconfig. I think I have an easy way to see if a kernel is good or bad, by printing stack_not_used(current) in the first call to omap_i2c_xfer_irq(). There's a huge drop between v5.12 and v5.13-rc1. And interestingly, sometimes a simple printk seems to use hundreds of bytes of stack (i.e. compare stack usage before and after the print). But not always. So maybe the issue is somehow related to printk. I'm bisecting. Tomi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEC1BC433ED for ; Fri, 21 May 2021 08:49:05 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 822CB61363 for ; Fri, 21 May 2021 08:49:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 822CB61363 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yt2InBjt/NtzlQrvIk5mRJxVEp74QZLltgF9kCpqY80=; b=caoRxptdgf9ZgQ4m1cHi0+IUUp VKtW+9DJNQuZnc8oXFn+L4H+RoaHwSwET8zn3JqxY9hUzbiQdAXGKscYO5jRBLssbw7fFY/rdLfNs /9HWvZK2r8A1QbZZfVqHVoPCUzIbsgOSYt9/nPsgAkp1NmWotCCkbsbpX0Xk8sBF0ORsaR7U8g7S1 /euSavfNTT3pRt6cjDmKbuo2OKV3edTisoNxxMt48G+jOkNB+J+7HgP3rTexXYw/+QPNXO3TcwLo2 1z6z20sLHKXk+mbVW8rhejCHhysV4Cj2z5nNonY1ZczVxIpjNlJHlU39mLSxTJ786uI1Wmm/uZiEL N8ZnrtsA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk0nd-004VNZ-7H; Fri, 21 May 2021 08:46:29 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lk0mr-004V7Y-Ou for linux-arm-kernel@desiato.infradead.org; Fri, 21 May 2021 08:45:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To: Subject:Sender:Reply-To:Content-ID:Content-Description; bh=X5DcdFyzFhBSbaLIDss620CjdT8Uzk2Lalg/VKhIK2g=; b=IaCEeNxOwAIB1NBkGjEvRIguDe HHtpKoX6f9omqkUeF91dalrgwRACrNNFhgle6yEqNxTlB5IZMapRVlUbcM4sUVhtgaykGm//kBZzg pn26fi9YCb5QiaGNXeORFQXK8u+hKH0PILSiWdN32ZlyT0e55e8r2VRRRWXGgKQuUURQFTQ4FN3MT atd5m/oP7j1gDw3/svpwJTcbJsMqpDXs1HLVJWdoxHtwnGdo2IU+KRtLhGc8LdqUd4BlQEI3XaCKw EJ2e0mfYtDt+it/nbDENNbV92qfGbe3SpNr3UfilQeceHb0VOjp82xMKeHk1y7blWVctaZl7N++c+ lRV4o+TA==; Received: from perceval.ideasonboard.com ([2001:4b98:dc2:55:216:3eff:fef7:d647]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lk0mp-00GxTE-3Y for linux-arm-kernel@lists.infradead.org; Fri, 21 May 2021 08:45:40 +0000 Received: from [192.168.1.111] (91-157-208-71.elisa-laajakaista.fi [91.157.208.71]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C43428D8; Fri, 21 May 2021 10:45:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1621586731; bh=WIs/t3U/kddeRP0vSywJKonvx676D72juDfNaO7cqkc=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=e2Un5NxX3Gh2pQL1HLaRtlF3OdVX0EDoBOp+zNnXP0/bflRKoEcQKUvMxDWIG2mws HAASKaYBq+Z42l6/U7fjdHhT+HSlKsGHQxEtnXk43XoPfKwnjhY4rE0XPD9a2gghOq fg/mPxubxUH/wVRo+ZUtWNXLXbpakntB/0GzLMVE= Subject: Re: Random stack corruption on v5.13 with dra76 To: Tony Lindgren Cc: linux-arm-kernel@lists.infradead.org, Nishanth Menon , "Bajjuri, Praneeth" , linux-omap@vger.kernel.org References: <0f48c7e5-6acd-1143-35ef-3dea2255bec6@ideasonboard.com> From: Tomi Valkeinen Message-ID: <064a9324-cfcf-47b9-6ae3-a29085a52683@ideasonboard.com> Date: Fri, 21 May 2021 11:45:29 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210521_014539_344561_CF9EA09D X-CRM114-Status: GOOD ( 15.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 21/05/2021 10:39, Tony Lindgren wrote: > * Tomi Valkeinen [210521 07:05]: >> On 21/05/2021 08:36, Tony Lindgren wrote: >>> * Tomi Valkeinen [210520 08:27]: >>>> Hi, >>>> >>>> I've noticed that the v5.13 rcs crash randomly (but quite often) on dra76 evm >>>> (I haven't tested other boards). Anyone else seen this problem? >>> >>> I have not seen this so far and beagle-x15 is behaving for me. >>> >>> Does it always happen on boot? >> >> No, but quite often. I can't really say how often, as it's annoyingly random. >> I tried to bisect, but that proved to be difficult as sometimes I get multiple (5+) >> successful boots before the crash. >> >> I tested with x15, same issue (below). So... Something in my kernel config? Or compiler? >> Looks like the crash happens always very soon after (or during) probing palmas. > > After about 10 reboots with your .config I'm seeing it now too on > beagle-x15. So far no luck reproducing it with omap2plus_defconfig. I think I have an easy way to see if a kernel is good or bad, by printing stack_not_used(current) in the first call to omap_i2c_xfer_irq(). There's a huge drop between v5.12 and v5.13-rc1. And interestingly, sometimes a simple printk seems to use hundreds of bytes of stack (i.e. compare stack usage before and after the print). But not always. So maybe the issue is somehow related to printk. I'm bisecting. Tomi _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel