Re: Kernel Panic in skb_release_data using genet

From: Florian Fainelli <f.fainelli@gmail.com>
To: Florian Fainelli <f.fainelli@gmail.com>,
	Maxime Ripard <maxime@cerno.tech>
Cc: Doug Berger <opendmb@gmail.com>,
	bcm-kernel-feedback-list@broadcom.com,
	linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
	Nicolas Saenz Julienne <nsaenz@kernel.org>
Subject: Re: Kernel Panic in skb_release_data using genet
Date: Fri, 28 May 2021 09:21:27 -0700	[thread overview]
Message-ID: <6caa98e7-28ba-520c-f0cc-ee1219305c17@gmail.com> (raw)
In-Reply-To: <1482eff4-c5f4-66d9-237c-55a096ae2eb4@gmail.com>

On 5/24/21 8:37 AM, Florian Fainelli wrote:
> 
> 
> On 5/24/2021 8:13 AM, Maxime Ripard wrote:
>> Hi Florian,
>>
>> On Mon, May 24, 2021 at 07:49:25AM -0700, Florian Fainelli wrote:
>>> Hi Maxime,
>>>
>>> On 5/24/2021 6:01 AM, Maxime Ripard wrote:
>>>> Hi Doug, Florian,
>>>>
>>>> I've been running a RaspberryPi4 with a mainline kernel for a while,
>>>> booting from NFS. Every once in a while (I'd say ~20-30% of all boots),
>>>> I'm getting a kernel panic around the time init is started.
>>>>
>>>> I was debugging a kernel based on drm-misc-next-2021-05-17 today with
>>>> KASAN enabled and got this, which looks related:
>>>
>>> Is there a known good version that could be used for bisection or you
>>> just started to do this test and you have no reference point?
>>
>> I've had this issue for over a year and never (I think?) got a good
>> version, so while it might be a regression, it's not a recent one.
> 
> OK, this helps and does not really help.
> 
>>
>>> How stable in terms of clocking is the configuration that you are using?
>>> I could try to fire up a similar test on a Pi4 at home, or use one of
>>> our 72112 systems which is the closest we have to a Pi4 and see if that
>>> happens there as well.
>>
>> I'm not really sure about the clocking. Is there any clock you want to
>> look at in particular?
> 
> ARM, DDR, AXI, anything that could cause some memory corruption to occur
> essentially. GENET clocks are fairly fixed, you have a 250MHz clock and
> a 125MHz clock feeding the data path.
> 
>>
>> My setup is fairly simple: the firmware and kernel are loaded over TFTP
>> and the rootfs is mounted over NFS, and the crash always occur around
>> init start, so I guess when it actually starts to transmit a decent
>> amount of data?
> 
> Do you reproduce this problem with KASAN disabled, do you eventually
> have a crash pointing back to the same location?
> 
> I have a suspicion that this is all Pi4 specific because we regularly
> run the GENET driver through various kernel versions (4.9, 5.4 and 5.10
> and mainline) and did not run into that.

I have not had time to get a set-up to reproduce what you are seeing,
could you share your .config meanwhile? Thanks
--
Florian