Re: [PATCH] PCI: Reprogram bridge prefetch registers on resume

From: Daniel Drake <drake-6IF/jdPJHihWk0Htik3J/w@public.gmane.org>
To: Sinan Kaya <okaya-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Mika Westerberg
	<mika.westerberg-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>,
	Linux PM <linux-pm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Wysocki,
	Rafael J"
	<rafael.j.wysocki-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
	nic_swsd-Rasf1IRRPZFBDgjK7y7TUQ@public.gmane.org,
	Keith Busch <keith.busch-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
	netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	nouveau-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
	Bjorn Helgaas <bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Andy Shevchenko
	<andriy.shevchenko-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>,
	Linux Upstreaming Team
	<linux-6IF/jdPJHihWk0Htik3J/w@public.gmane.org>,
	davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org,
	Jon Derrick
	<jonathan.derrick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
	hkallweit1-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Subject: Re: [PATCH] PCI: Reprogram bridge prefetch registers on resume
Date: Fri, 7 Sep 2018 16:06:19 +0800	[thread overview]
Message-ID: <CAD8Lp466EffdgL5zVZi-L+s78Yxa9+FryB8ch1=UAwwCPqu9Ag@mail.gmail.com> (raw)
In-Reply-To: <3b37e4fd-c793-bd52-7d70-950f846a7d5a-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>

On Fri, Sep 7, 2018 at 2:40 PM, Sinan Kaya <okaya@kernel.org> wrote:
> On 9/6/2018 10:36 PM, Daniel Drake wrote:
>>
>> +       if (pci_dev->class == PCI_CLASS_BRIDGE_PCI << 8)
>> +               pci_setup_bridge_mmio_pref(pci_dev);
>
>
> This should probably some kind of a quirk rather than default
> for the listed card as it sounds like you are dealing with
> broken hardware.

With that approach there's a sizeable list that your quirk list is
incomplete or out of date.

And when the bug bites, it's extremely cryptic. We've spent months
working on this issue and only found this magic register write mostly
through a stroke of good luck. Separately there's been a flurry of
mails around the r8169 MSI-X problem but as far as I can see nobody
suggested even looking at the values of the parent bridge prefetch
registers. And even if they did, they'd probably have said "values are
fine, nothing to see here" (exactly as we did 4 months ago when Nvidia
mentioned these registers as a possible cause - oops!).

So here I'm instead following a suggestion from Bjorn, after also
having confirmed the windows behaviour:

https://marc.info/?l=linux-pci&m=153574276126484&w=2
> Can we tell whether Windows rewrites this register unconditionally at
> resume-time?  If so, it may be more robust for Linux to do the same.
> The whole thing is black magic, which I hate, but if it's our only
> choice, it may be better to have this applied everywhere so we don't
> keep stubbing our toes on new systems that require the quirk.

Also, we just spoke to Asus BIOS engineers who told us that the BIOS
does already restore the PCI bridge prefetch registers on resume. I
guess this is why the other registers like the (non-zero) prefetch
base address lower 32 bits do have the right value on resume even
before my patch. It sounds like a more subtle bug related to register
write timing or sequence, in that case it will be harder to define who
is responsible for the breakage and hence under which conditions the
quirk should apply.

Daniel
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau