Re: [PATCH 0/3] PCI: designware: Fixing MSI handling flow

[Marc Zyngier <marc.zyngier@arm.com>]

On Tue, 13 Nov 2018 23:16:33 +0000,
Gustavo Pimentel <gustavo.pimentel@synopsys.com> wrote:
> 
> On 13/11/2018 22:57, Marc Zyngier wrote:
> > It recently came to light that the Designware PCIe driver is rather
> > broken in the way it handles MSI[1]:
> > 
> > - It masks interrupt by disabling them, meaning that MSIs generated
> >   during the masked window are simply lost. Oops.
> > 
> > - Acking of the currently pending MSI is done outside of the interrupt
> >   flow, getting moved around randomly and ultimately breaking the
> >   driver. Not great.
> > 
> > This series attempts to address this by switching to using the MASK
> > register for masking interrupts (!), and move the ack into the
> > appropriate callback, giving it a fixed place in the MSI handling
> > flow.
> > 
> > Note that this is only compile-tested on my arm64 laptop, as I'm
> > travelling and do not have the required HW to test it anyway. I'd
> > welcome both review and testing by the interested parties (dwc
> > maintainer and users affected by existing bugs).
> 
> As we spoke on the conference, as soon as I get back and I've the necessary
> conditions I'll test the discussed modifications on my HW.

I just realised (at 1am!) that the first patch is awfully buggy:
- ENABLE and MASK have opposite polarities
- There is nothing initialising the ENABLE and MASK registers

I've stashed the following fix-up on top of the existing series:

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index f06e67c60593..0fa9e8fdce66 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -166,7 +166,7 @@ static void dw_pci_bottom_mask(struct irq_data *data)
 
 		pp->irq_status[ctrl] &= ~(1 << bit);
 		dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + res, 4,
-				    pp->irq_status[ctrl]);
+				    ~pp->irq_status[ctrl]);
 	}
 
 	raw_spin_unlock_irqrestore(&pp->lock, flags);
@@ -189,7 +189,7 @@ static void dw_pci_bottom_unmask(struct irq_data *data)
 
 		pp->irq_status[ctrl] |= 1 << bit;
 		dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + res, 4,
-				    pp->irq_status[ctrl]);
+				    ~pp->irq_status[ctrl]);
 	}
 
 	raw_spin_unlock_irqrestore(&pp->lock, flags);
@@ -664,10 +664,15 @@ void dw_pcie_setup_rc(struct pcie_port *pp)
 	num_ctrls = pp->num_vectors / MAX_MSI_IRQS_PER_CTRL;
 
 	/* Initialize IRQ Status array */
-	for (ctrl = 0; ctrl < num_ctrls; ctrl++)
-		dw_pcie_rd_own_conf(pp, PCIE_MSI_INTR0_ENABLE +
+	for (ctrl = 0; ctrl < num_ctrls; ctrl++) {
+		dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK +
 					(ctrl * MSI_REG_CTRL_BLOCK_SIZE),
-				    4, &pp->irq_status[ctrl]);
+				    4, ~0);
+		dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_ENABLE +
+					(ctrl * MSI_REG_CTRL_BLOCK_SIZE),
+				    4, ~0);
+		pp->irq_status[ctrl] = 0;
+	}
 
 	/* Setup RC BARs */
 	dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004);

Please let me know if this helps.

	M.

-- 
Jazz is not dead, it just smell funny.