Re: [PATCH V4] PCI: rcar: Add L1 link state fix into data abort hook

[Marek Vasut <marek.vasut@gmail.com>]

On 11/19/20 6:35 PM, Lorenzo Pieralisi wrote:
>> +#ifdef CONFIG_ARM
>> +/*
>> + * Here we keep a static copy of the remapped PCIe controller address.
>> + * This is only used on aarch32 systems, all of which have one single
>> + * PCIe controller, to provide quick access to the PCIe controller in
>> + * the L1 link state fixup function, called from the ARM fault handler.
>> + */
>> +static void __iomem *pcie_base;
>> +/*
>> + * Static copy of bus clock pointer, so we can check whether the clock
>> + * is enabled or not.
>> + */
>> +static struct clk *pcie_bus_clk;
>> +#endif
> 
> Don't think you can have multiple host bridges in a given platform,
> if it is a possible configuration this won't work.

Correct, all the affected platforms have only one host bridge.

>>   static inline struct rcar_msi *to_rcar_msi(struct msi_controller *chip)
>>   {
>>   	return container_of(chip, struct rcar_msi, chip);
>> @@ -804,6 +820,12 @@ static int rcar_pcie_get_resources(struct rcar_pcie_host *host)
>>   	}
>>   	host->msi.irq2 = i;
>>   
>> +#ifdef CONFIG_ARM
>> +	/* Cache static copy for L1 link state fixup hook on aarch32 */
>> +	pcie_base = pcie->base;
>> +	pcie_bus_clk = host->bus_clk;
>> +#endif
>> +
>>   	return 0;
>>   
>>   err_irq2:
>> @@ -1050,4 +1072,58 @@ static struct platform_driver rcar_pcie_driver = {
>>   	},
>>   	.probe = rcar_pcie_probe,
>>   };
>> +
>> +#ifdef CONFIG_ARM
>> +static int rcar_pcie_aarch32_abort_handler(unsigned long addr,
>> +		unsigned int fsr, struct pt_regs *regs)
>> +{
>> +	u32 pmsr;
>> +
>> +	if (!pcie_base || !__clk_is_enabled(pcie_bus_clk))
>> +		return 1;
>> +
>> +	pmsr = readl(pcie_base + PMSR);
>> +
>> +	/*
>> +	 * Test if the PCIe controller received PM_ENTER_L1 DLLP and
>> +	 * the PCIe controller is not in L1 link state. If true, apply
>> +	 * fix, which will put the controller into L1 link state, from
>> +	 * which it can return to L0s/L0 on its own.
>> +	 */
>> +	if ((pmsr & PMEL1RX) && ((pmsr & PMSTATE) != PMSTATE_L1)) {
>> +		writel(L1IATN, pcie_base + PMCTLR);
>> +		while (!(readl(pcie_base + PMSR) & L1FAEG))
>> +			;
>> +		writel(L1FAEG | PMEL1RX, pcie_base + PMSR);
>> +		return 0;
>> +	}
> 
> I suppose a fault on multiple cores can happen simultaneously, if it
> does this may not work well either - I assume all config/io/mem would
> trigger a fault.
> 
> As I mentioned in my reply to v1, is there a chance we can move
> this quirk into config accessors (if the PM_ENTER_L1_DLLP is
> subsequent to a write into PMCSR to programme a D state) ?

I don't think we can, since the userspace can do such a config space 
write with e.g. setpci and then this fixup is still needed.

> Config access is serialized but I suspect as I said above that this
> triggers on config/io/mem alike.
> 
> Just asking to try to avoid a fault handler if possible.

See above, I doubt we can fully avoid this workaround.

[...]