Re: [Bug 216373] New: Uncorrected errors reported for AMD GPU

From: "Lazar, Lijo" <lijo.lazar@amd.com>
To: Bjorn Helgaas <helgaas@kernel.org>, Tom Seewald <tseewald@gmail.com>
Cc: regressions@lists.linux.dev, "David Airlie" <airlied@linux.ie>,
	linux-pci@vger.kernel.org, "Xinhui Pan" <Xinhui.Pan@amd.com>,
	amd-gfx@lists.freedesktop.org,
	"Kai-Heng Feng" <kai.heng.feng@canonical.com>,
	"Daniel Vetter" <daniel@ffwll.ch>,
	"Alex Deucher" <alexander.deucher@amd.com>,
	"Stefan Roese" <sr@denx.de>,
	"Christian König" <christian.koenig@amd.com>
Subject: Re: [Bug 216373] New: Uncorrected errors reported for AMD GPU
Date: Sat, 20 Aug 2022 13:22:43 +0530	[thread overview]
Message-ID: <6aad506b-5324-649e-9700-7ceaaf7ef94b@amd.com> (raw)
In-Reply-To: <20220819190725.GA2499154@bhelgaas>

On 8/20/2022 12:37 AM, Bjorn Helgaas wrote:
> On Fri, Aug 19, 2022 at 12:13:03PM -0500, Bjorn Helgaas wrote:
>> On Thu, Aug 18, 2022 at 03:38:12PM -0500, Bjorn Helgaas wrote:
>>> [Adding amdgpu folks]
>>>
>>> On Wed, Aug 17, 2022 at 11:45:15PM +0000, bugzilla-daemon@kernel.org wrote:
>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.kernel.org%2Fshow_bug.cgi%3Fid%3D216373&amp;data=05%7C01%7Clijo.lazar%40amd.com%7C958674b7d05040ddd93f08da8216107a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637965328525221166%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=otI1jfdKInPamMGrLyhFsVoGnK%2BIgL%2BcyMtBVPsV7bE%3D&amp;reserved=0
>>>>
>>>>              Bug ID: 216373
>>>>             Summary: Uncorrected errors reported for AMD GPU
>>>>      Kernel Version: v6.0-rc1
>>>>          Regression: No
>>
>> Tom, thanks for trying out "pci=noaer".  Hopefully we won't need the
>> workaround for long.
>>
>> Could I trouble you to try the debug patch below and see if we get any
>> stack trace clues in dmesg when the error happens?  I'm sure the
>> experts would have a better approach, but I'm amdgpu-illiterate, so
>> this is all I can do :)
> 
> Thanks for doing this, Tom!  For everybody else, Tom attached a dmesg
> log to the bugzilla: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.kernel.org%2Fattachment.cgi%3Fid%3D301606&amp;data=05%7C01%7Clijo.lazar%40amd.com%7C958674b7d05040ddd93f08da8216107a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637965328525221166%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=pzmtR3daPOimRkX6fohHQTDiEkp8B9aCJX5Z%2BKUcq2c%3D&amp;reserved=0
> 
> Lots of traces of the form:
> 
>    amdgpu_device_wreg.part.0.cold+0xb/0x17 [amdgpu]
>    amdgpu_gart_invalidate_tlb+0x22/0x60 [amdgpu]
>    gmc_v10_0_hw_init+0x44/0x180 [amdgpu]
> 
>    amdgpu_device_wreg.part.0.cold+0xb/0x17 [amdgpu]
>    gmc_v10_0_hw_init+0xa8/0x180 [amdgpu]
> 
>    amdgpu_device_wreg.part.0.cold+0xb/0x17 [amdgpu]
>    gmc_v10_0_flush_gpu_tlb+0x35/0x280 [amdgpu]
>    amdgpu_gart_invalidate_tlb+0x46/0x60 [amdgpu]
>    gmc_v10_0_hw_init+0x44/0x180 [amdgpu]
> 
> I tried connecting the dots but I gave up chasing all the function
> pointers.
> 

Missed the remap part, the offset is here -

https://elixir.bootlin.com/linux/v6.0-rc1/source/drivers/gpu/drm/amd/amdgpu/nv.c#L680 

The trace is coming from *_flush_hdp.

You may also check if *_remap_hdp_registers() is getting called. It is 
done in nbio_vx_y files, most likely this one for your device -
https://elixir.bootlin.com/linux/v6.0-rc1/source/drivers/gpu/drm/amd/amdgpu/nbio_v2_3.c#L68

Thanks,
Lijo