Re: [PATCH 08/15] PCI: mvebu: Propagate errors when updating PCI_IO_BASE and PCI_MEM_BASE registers

From: "Pali Rohár" <pali@kernel.org>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>,
	"Lorenzo Pieralisi" <lorenzo.pieralisi@arm.com>,
	"Rob Herring" <robh@kernel.org>,
	"Krzysztof Wilczyński" <kw@linux.com>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Marek Behún" <kabel@kernel.org>,
	linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 08/15] PCI: mvebu: Propagate errors when updating PCI_IO_BASE and PCI_MEM_BASE registers
Date: Fri, 7 Jan 2022 23:28:26 +0100	[thread overview]
Message-ID: <20220107222826.cv2bzywwayjwzy3c@pali> (raw)
In-Reply-To: <20220107215504.GA406217@bhelgaas>

On Friday 07 January 2022 15:55:04 Bjorn Helgaas wrote:
> On Thu, Nov 25, 2021 at 01:45:58PM +0100, Pali Rohár wrote:
> > Properly propagate failure from mvebu_pcie_add_windows() function back to
> > the caller mvebu_pci_bridge_emul_base_conf_write() and correctly updates
> > PCI_IO_BASE, PCI_MEM_BASE and PCI_IO_BASE_UPPER16 registers on error.
> > On error set base value higher than limit value which indicates that
> > address range is disabled. 
> 
> Does the spec say that if software programs something invalid,
> hardware should proactively set the base and limit registers to
> disable the window?

No. But this patch address something totally different. Software can do
fully valid operation, e.g. try to set forwarding memory window as large
as possible. But because this driver "emulates" pci bridge by calling
software/kernel function (mvebu_pcie_add_windows), some operations which
in real HW cannot happen, are possible in software.

For example there are limitations in sizes of forwarding memory windows,
because it is done by mvebu-mbus driver, which is responsible for
configuring mapping and forwarding of PCIe I/O and MEM windows. And due
to Marvell HW, there are restrictions which are not in PCIe HW.

Currently if such error happens, obviously kernel is not able to set
PCIe windows and it just print warnings to dmesg. Trying to access these
windows would result in the worst case in crashes.

With this change when mvebu_pcie_add_windows() function fails then into
emulated config space is put information that particular forwarding
window is disabled. I think that it is better to indicate it in config
space what is the current "reality" of hardware configuration. If window
is disabled in real-HW (meaning in mvebu-mbus driver) then show it also
in emulated config space of pci bridge.

Do you have better idea what should emulated pci bridge do, if software
try to set fully valid configuration of forwarding window, but it is not
possible to achieve it (even compliant PCI bridge must be able to do
it)?

> I'm not sure I've seen hardware that does this, and it seems ... maybe
> a little aggressive.
> 
> What happens if software writes the base and limit in the wrong order,
> so the window is invalid after the first write but valid after the
> second?  That actually sounds like it could be a sensible strategy to
> prevent a partially-configured window from being active.
> 
> Bjorn

Invalid window (limit < base) means that window is disabled. And
pci-mvebu.c in its callbacks from pci-bridge-emul.c should correctly
handle it and propagates information about disablement to mvebu-mbus
driver.

After window is valid again (limit > base) then pci-mvebu.c call
mvebu-mbus to setup new mapping.