Re: [PATCH v3] PCI: aardvark: Don't touch PCIe registers if no card connected

From: "Pali Rohár" <pali@kernel.org>
To: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>,
	"Andrew Murray" <amurray@thegoodpenguin.co.uk>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Marek Behún" <marek.behun@nic.cz>,
	"Remi Pommarel" <repk@triplefau.lt>,
	"Tomasz Maciej Nowak" <tmn505@gmail.com>,
	Xogium <contact@xogium.me>,
	linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3] PCI: aardvark: Don't touch PCIe registers if no card connected
Date: Fri, 10 Jul 2020 17:44:58 +0200	[thread overview]
Message-ID: <20200710154458.bntk7cgewvxmubf4@pali> (raw)
In-Reply-To: <20200710091800.GA3419@e121166-lin.cambridge.arm.com>

On Friday 10 July 2020 10:18:00 Lorenzo Pieralisi wrote:
> I would be very grateful if you could describe what happens in HW
> when these conditions trigger - I would like to understand if this
> issue is aardvark specific or it isn't.

Hello Lorenzo! We are not sure what is the problem and where it happens.

There are more issues which happens randomly or under some specific
conditions.

I can reproduce following issue: Connect Compex WLE900VX card, configure
aardvark to gen2 mode. And then card is detected only after the first
link training. If kernel tries to retrain link again (e.g. via ASPM
code) then card is not detected anymore. To detect it again it is needed
to reset card via PERST# signal (assert PERST#, wait, de-assert PERST#).
PCI warm, hot or function reset does not help. When aardvark is
configured in gen1 mode then card is detected fine also after multiple
link training.

Above problem does not happen with Compex WLE200VX (ath9k) or Compex
WLE1216V5-20 cards.

Sometimes WLE900VX card disappear from the bus during usage. It just
stop communicates with ath10k driver and aardvark does not see link.

Another issue which happens for WLE900VX, WLE600VX and WLE1216VS-20 (but
not for WLE200VX): Linux kernel can detect these cards only if it issues
card reset via PERST# signal and start link training (via standard pcie
endpoint register PCI_EXP_LNKCTL/PCI_EXP_LNKCTL_RL) immediately after
enable link training in aardvark (via aardvark specific LINK_TRAINING_EN
bit). If there is e.g. 100ms delay between enabling link training and
setting PCI_EXP_LNKCTL_RL bit then these cards are not detected.

Also issuing reset via PERST# signal is required to detect these cards
if either board was rebooted (not started from cold power off state) or
if U-Boot touched/initialized PCIe aardvark.

WLE200VX works fine also after doing second or third link training and
also works without need to issue reset via PERST# signal.

And WLE900VX card is not detected even after resetting it via PERST#
signal if aardvark link training (LINK_TRAINING_EN bit) was enabled
prior toggling PERST#. PERST# signal is controlled via GPIO.

When I put WLE900VX card into board with uses mvebu PCI driver (not
aardvak) then card is working fine, there is no need to issue card reset
via PERST#, no need to explicitly set gen mode and card is also working
after more link training.

So basically I have no idea why it happens or where is the problem,
either in aardvark or in cards or on both places. As you can see each of
tested card has different set problems.

Today I tested card from different vendor but with same Qualcomm chip as
is in WLE900VX and I observe same behavior as from Compex WLE900VX. So
it looks like that card vendor does not have to matter, important is
wifi chip inside.

I read in kernel bugzilla that WLE600VX and WLE900VX cards are buggy and
more people have problems with them. But issues described in kernel
bugzilla (like card is reporting incorrect PCI device id) I'm not
observing.

If you have any idea how to either debug these problems or come up with
idea where could be the problem, please let me know.