* dwc: tegra194: issue with card containing a bridge
@ 2020-12-15 10:24 Mian Yousaf Kaukab
2020-12-15 12:15 ` Vidya Sagar
0 siblings, 1 reply; 10+ messages in thread
From: Mian Yousaf Kaukab @ 2020-12-15 10:24 UTC (permalink / raw)
To: lorenzo.pieralisi, vidyas, robh; +Cc: bhelgaas, linux-pci, linux-kernel
Hi,
I am seeing an issue with next-20201211 with USB3380[1] based PCIe card
(vid:pid 10b5:3380) on Jetson AGX Xavier. Card doesn't show up in the
lspci output.
In non working case (next-20201211):
# lspci
0001:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad2 (rev a1)
0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13)
0005:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad0 (rev a1)
In working case (v5.10-rc7):
# lspci
0001:00:00.0 PCI bridge: Molex Incorporated Device 1ad2 (rev a1)
0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13)
0005:00:00.0 PCI bridge: Molex Incorporated Device 1ad0 (rev a1)
0005:01:00.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab)
0005:02:02.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab)
0005:03:00.0 USB controller: PLX Technology, Inc. Device 3380 (rev ab)
# lspci -t
-+-[0005:00]---00.0-[01-ff]----00.0-[02-03]----02.0-[03]----00.0
+-[0001:00]---00.0-[01-ff]----00.0
\-[0000:00]-
#lspci -v
https://paste.opensuse.org/87573209
git-bisect points to commit b9ac0f9dc8ea ("PCI: dwc: Move dw_pcie_setup_rc() to DWC common code").
dw_pcie_setup_rc() is not removed from pcie-tegra194.c in this commit.
Could the failure be caused because dw_pcie_setup_rc() is called twice now in case of tegra194?
BR,
Yousaf
[1]: https://www.broadcom.com/products/pcie-switches-bridges/usb-pci/usb-controllers/usb3380
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: dwc: tegra194: issue with card containing a bridge 2020-12-15 10:24 dwc: tegra194: issue with card containing a bridge Mian Yousaf Kaukab @ 2020-12-15 12:15 ` Vidya Sagar 2020-12-15 13:25 ` Mian Yousaf Kaukab 0 siblings, 1 reply; 10+ messages in thread From: Vidya Sagar @ 2020-12-15 12:15 UTC (permalink / raw) To: Mian Yousaf Kaukab, lorenzo.pieralisi, robh Cc: bhelgaas, linux-pci, linux-kernel Thanks Mian for bringing it to our notice. Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c file on top of linux-next? and does that solve the issue? diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c index 5597b2a49598..1c9e9c054592 100644 --- a/drivers/pci/controller/dwc/pcie-tegra194.c +++ b/drivers/pci/controller/dwc/pcie-tegra194.c @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, val); } - dw_pcie_setup_rc(pp); + //dw_pcie_setup_rc(pp); clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); I took a quick look at the dw_pcie_setup_rc() implementation and I'm not sure why calling it second time should create any issue for the enumeration of devices behind a switch. Perhaps I need to spend more time to debug that part. In any case, since dw_pcie_setup_rc() is already part of dw_pcie_host_init(), I think it can be removed from tegra_pcie_prepare_host() implemention. Thanks, Vidya Sagar On 12/15/2020 3:54 PM, Mian Yousaf Kaukab wrote: > External email: Use caution opening links or attachments > > > Hi, > I am seeing an issue with next-20201211 with USB3380[1] based PCIe card > (vid:pid 10b5:3380) on Jetson AGX Xavier. Card doesn't show up in the > lspci output. > > In non working case (next-20201211): > # lspci > 0001:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad2 (rev a1) > 0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13) > 0005:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad0 (rev a1) > > In working case (v5.10-rc7): > # lspci > 0001:00:00.0 PCI bridge: Molex Incorporated Device 1ad2 (rev a1) > 0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13) > 0005:00:00.0 PCI bridge: Molex Incorporated Device 1ad0 (rev a1) > 0005:01:00.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab) > 0005:02:02.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab) > 0005:03:00.0 USB controller: PLX Technology, Inc. Device 3380 (rev ab) > # lspci -t > -+-[0005:00]---00.0-[01-ff]----00.0-[02-03]----02.0-[03]----00.0 > +-[0001:00]---00.0-[01-ff]----00.0 > \-[0000:00]- > #lspci -v > https://paste.opensuse.org/87573209 > > git-bisect points to commit b9ac0f9dc8ea ("PCI: dwc: Move dw_pcie_setup_rc() to DWC common code"). > dw_pcie_setup_rc() is not removed from pcie-tegra194.c in this commit. > > Could the failure be caused because dw_pcie_setup_rc() is called twice now in case of tegra194? > > BR, > Yousaf > > [1]: https://www.broadcom.com/products/pcie-switches-bridges/usb-pci/usb-controllers/usb3380 > ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-15 12:15 ` Vidya Sagar @ 2020-12-15 13:25 ` Mian Yousaf Kaukab 2020-12-15 15:41 ` Rob Herring 0 siblings, 1 reply; 10+ messages in thread From: Mian Yousaf Kaukab @ 2020-12-15 13:25 UTC (permalink / raw) To: Vidya Sagar; +Cc: lorenzo.pieralisi, robh, bhelgaas, linux-pci, linux-kernel On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > Thanks Mian for bringing it to our notice. > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > file on top of linux-next? and does that solve the issue? > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > b/drivers/pci/controller/dwc/pcie-tegra194.c > index 5597b2a49598..1c9e9c054592 100644 > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > *pp) > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > val); > } > > - dw_pcie_setup_rc(pp); > + //dw_pcie_setup_rc(pp); I still see the same issue with this change. Reverting b9ac0f9dc8ea works though. > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > sure why calling it second time should create any issue for the enumeration > of devices behind a switch. Perhaps I need to spend more time to debug that > part. > In any case, since dw_pcie_setup_rc() is already part of > dw_pcie_host_init(), I think it can be removed from > tegra_pcie_prepare_host() implemention. > > Thanks, > Vidya Sagar > BR, Yousaf ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-15 13:25 ` Mian Yousaf Kaukab @ 2020-12-15 15:41 ` Rob Herring 2020-12-15 19:44 ` Rob Herring 2020-12-15 20:52 ` Mian Yousaf Kaukab 0 siblings, 2 replies; 10+ messages in thread From: Rob Herring @ 2020-12-15 15:41 UTC (permalink / raw) To: Mian Yousaf Kaukab Cc: Vidya Sagar, lorenzo.pieralisi, bhelgaas, linux-pci, linux-kernel On Tue, Dec 15, 2020 at 02:25:04PM +0100, Mian Yousaf Kaukab wrote: > On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > > Thanks Mian for bringing it to our notice. > > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > > file on top of linux-next? and does that solve the issue? > > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > > b/drivers/pci/controller/dwc/pcie-tegra194.c > > index 5597b2a49598..1c9e9c054592 100644 > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > > *pp) > > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > > val); > > } > > > > - dw_pcie_setup_rc(pp); > > + //dw_pcie_setup_rc(pp); > I still see the same issue with this change. > Reverting b9ac0f9dc8ea works though. > > > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > > sure why calling it second time should create any issue for the enumeration > > of devices behind a switch. Perhaps I need to spend more time to debug that > > part. > > In any case, since dw_pcie_setup_rc() is already part of > > dw_pcie_host_init(), I think it can be removed from > > tegra_pcie_prepare_host() implemention. I think the 2nd time is making the link go down is my guess. Tegra was odd in that its start/stop link functions don't do link handling, so I didn't implement those functions and left the link handling in the Tegra driver. Can you try the below patch. It needs some more work as it breaks endpoint mode. 8<-------------------------------------------------------------------- diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c index 648e731bccfa..49bb487b16ae 100644 --- a/drivers/pci/controller/dwc/pcie-tegra194.c +++ b/drivers/pci/controller/dwc/pcie-tegra194.c @@ -907,9 +907,32 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, val); } - dw_pcie_setup_rc(pp); - clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); +} + +static int tegra_pcie_dw_host_init(struct pcie_port *pp) +{ + pp->bridge->ops = &tegra_pci_ops; + + tegra_pcie_prepare_host(pp); + tegra_pcie_enable_interrupts(pp); + + return 0; +} + +static int tegra_pcie_dw_link_up(struct dw_pcie *pci) +{ + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); + u32 val = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA); + + return !!(val & PCI_EXP_LNKSTA_DLLLA); +} + +static int tegra_pcie_dw_start_link(struct dw_pcie *pci) +{ + u32 val, offset, speed, tmp; + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); + struct pcie_port *pp = &pci->pp; /* Assert RST */ val = appl_readl(pcie, APPL_PINMUX); @@ -929,17 +952,6 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) appl_writel(pcie, val, APPL_PINMUX); msleep(100); -} - -static int tegra_pcie_dw_host_init(struct pcie_port *pp) -{ - struct dw_pcie *pci = to_dw_pcie_from_pp(pp); - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); - u32 val, tmp, offset, speed; - - pp->bridge->ops = &tegra_pci_ops; - - tegra_pcie_prepare_host(pp); if (dw_pcie_wait_for_link(pci)) { /* @@ -975,7 +987,8 @@ static int tegra_pcie_dw_host_init(struct pcie_port *pp) val &= ~PCI_DLF_EXCHANGE_ENABLE; dw_pcie_writel_dbi(pci, offset, val); - tegra_pcie_prepare_host(pp); + tegra_pcie_dw_host_init(pp); + dw_pcie_setup_rc(pp); if (dw_pcie_wait_for_link(pci)) return 0; @@ -985,25 +998,6 @@ static int tegra_pcie_dw_host_init(struct pcie_port *pp) PCI_EXP_LNKSTA_CLS; clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]); - tegra_pcie_enable_interrupts(pp); - - return 0; -} - -static int tegra_pcie_dw_link_up(struct dw_pcie *pci) -{ - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); - u32 val = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA); - - return !!(val & PCI_EXP_LNKSTA_DLLLA); -} - -static int tegra_pcie_dw_start_link(struct dw_pcie *pci) -{ - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); - - enable_irq(pcie->pex_rst_irq); - return 0; } ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-15 15:41 ` Rob Herring @ 2020-12-15 19:44 ` Rob Herring 2020-12-15 20:50 ` Mian Yousaf Kaukab 2020-12-15 20:52 ` Mian Yousaf Kaukab 1 sibling, 1 reply; 10+ messages in thread From: Rob Herring @ 2020-12-15 19:44 UTC (permalink / raw) To: Mian Yousaf Kaukab Cc: Vidya Sagar, lorenzo.pieralisi, bhelgaas, linux-pci, linux-kernel On Tue, Dec 15, 2020 at 09:41:47AM -0600, Rob Herring wrote: > On Tue, Dec 15, 2020 at 02:25:04PM +0100, Mian Yousaf Kaukab wrote: > > On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > > > Thanks Mian for bringing it to our notice. > > > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > > > file on top of linux-next? and does that solve the issue? > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > > > b/drivers/pci/controller/dwc/pcie-tegra194.c > > > index 5597b2a49598..1c9e9c054592 100644 > > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > > > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > > > *pp) > > > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > > > val); > > > } > > > > > > - dw_pcie_setup_rc(pp); > > > + //dw_pcie_setup_rc(pp); > > I still see the same issue with this change. > > Reverting b9ac0f9dc8ea works though. > > > > > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > > > > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > > > sure why calling it second time should create any issue for the enumeration > > > of devices behind a switch. Perhaps I need to spend more time to debug that > > > part. > > > In any case, since dw_pcie_setup_rc() is already part of > > > dw_pcie_host_init(), I think it can be removed from > > > tegra_pcie_prepare_host() implemention. > > I think the 2nd time is making the link go down is my guess. Tegra was > odd in that its start/stop link functions don't do link handling, so I > didn't implement those functions and left the link handling in the Tegra > driver. > > Can you try the below patch. It needs some more work as it breaks > endpoint mode. That one missed some re-init. Try this one instead 8<-------------------------------------------------------------------- diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c index 5597b2a49598..d8fed3561e91 100644 --- a/drivers/pci/controller/dwc/pcie-tegra194.c +++ b/drivers/pci/controller/dwc/pcie-tegra194.c @@ -933,14 +933,24 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) static int tegra_pcie_dw_host_init(struct pcie_port *pp) { - struct dw_pcie *pci = to_dw_pcie_from_pp(pp); - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); - u32 val, tmp, offset, speed; - pp->bridge->ops = &tegra_pci_ops; tegra_pcie_prepare_host(pp); + return 0; +} + +static int tegra_pcie_dw_start_link(struct dw_pcie *pci) +{ + u32 val, offset, speed, tmp; + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); + struct pcie_port *pp = &pci->pp; + + if (pcie->mode == DW_PCIE_EP_TYPE) { + enable_irq(pcie->pex_rst_irq); + return 0; + } + if (dw_pcie_wait_for_link(pci)) { /* * There are some endpoints which can't get the link up if @@ -998,15 +1008,6 @@ static int tegra_pcie_dw_link_up(struct dw_pcie *pci) return !!(val & PCI_EXP_LNKSTA_DLLLA); } -static int tegra_pcie_dw_start_link(struct dw_pcie *pci) -{ - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); - - enable_irq(pcie->pex_rst_irq); - - return 0; -} - static void tegra_pcie_dw_stop_link(struct dw_pcie *pci) { struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-15 19:44 ` Rob Herring @ 2020-12-15 20:50 ` Mian Yousaf Kaukab 0 siblings, 0 replies; 10+ messages in thread From: Mian Yousaf Kaukab @ 2020-12-15 20:50 UTC (permalink / raw) To: Rob Herring Cc: Vidya Sagar, lorenzo.pieralisi, bhelgaas, linux-pci, linux-kernel On Tue, Dec 15, 2020 at 01:44:21PM -0600, Rob Herring wrote: > On Tue, Dec 15, 2020 at 09:41:47AM -0600, Rob Herring wrote: > > On Tue, Dec 15, 2020 at 02:25:04PM +0100, Mian Yousaf Kaukab wrote: > > > On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > > > > Thanks Mian for bringing it to our notice. > > > > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > > > > file on top of linux-next? and does that solve the issue? > > > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > index 5597b2a49598..1c9e9c054592 100644 > > > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > > > > *pp) > > > > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > > > > val); > > > > } > > > > > > > > - dw_pcie_setup_rc(pp); > > > > + //dw_pcie_setup_rc(pp); > > > I still see the same issue with this change. > > > Reverting b9ac0f9dc8ea works though. > > > > > > > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > > > > > > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > > > > sure why calling it second time should create any issue for the enumeration > > > > of devices behind a switch. Perhaps I need to spend more time to debug that > > > > part. > > > > In any case, since dw_pcie_setup_rc() is already part of > > > > dw_pcie_host_init(), I think it can be removed from > > > > tegra_pcie_prepare_host() implemention. > > > > I think the 2nd time is making the link go down is my guess. Tegra was > > odd in that its start/stop link functions don't do link handling, so I > > didn't implement those functions and left the link handling in the Tegra > > driver. > > > > Can you try the below patch. It needs some more work as it breaks > > endpoint mode. > > That one missed some re-init. Try this one instead > > 8<-------------------------------------------------------------------- > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c > index 5597b2a49598..d8fed3561e91 100644 > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > @@ -933,14 +933,24 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) > > static int tegra_pcie_dw_host_init(struct pcie_port *pp) > { > - struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > - u32 val, tmp, offset, speed; > - > pp->bridge->ops = &tegra_pci_ops; > > tegra_pcie_prepare_host(pp); > > + return 0; > +} > + > +static int tegra_pcie_dw_start_link(struct dw_pcie *pci) > +{ > + u32 val, offset, speed, tmp; > + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > + struct pcie_port *pp = &pci->pp; > + > + if (pcie->mode == DW_PCIE_EP_TYPE) { > + enable_irq(pcie->pex_rst_irq); > + return 0; > + } > + > if (dw_pcie_wait_for_link(pci)) { > /* > * There are some endpoints which can't get the link up if > @@ -998,15 +1008,6 @@ static int tegra_pcie_dw_link_up(struct dw_pcie *pci) > return !!(val & PCI_EXP_LNKSTA_DLLLA); > } > > -static int tegra_pcie_dw_start_link(struct dw_pcie *pci) > -{ > - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > - > - enable_irq(pcie->pex_rst_irq); > - > - return 0; > -} > - > static void tegra_pcie_dw_stop_link(struct dw_pcie *pci) > { > struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); With USB3380 card PTM enabled... is the last message I see. It doesn't boot further: [ 9.124500] tegra194-pcie 141a0000.pcie: iATU unroll: enabled [ 9.130310] tegra194-pcie 141a0000.pcie: Detected iATU regions: 8 outbound, 2 inbound [ 9.138915] tegra194-pcie 141a0000.pcie: Link up [ 9.144940] tegra194-pcie 141a0000.pcie: PCI host bridge to bus 0005:00 [ 9.151849] pci_bus 0005:00: root bus resource [bus 00-ff] [ 9.157595] pci_bus 0005:00: root bus resource [mem 0x1c00000000-0x1f3fffffff pref] [ 9.165509] pci_bus 0005:00: root bus resource [mem 0x1f40000000-0x1ffffeffff] (bus address [0x40000000-0xfffeffff]) [ 9.176444] pci_bus 0005:00: root bus resource [io 0x30000-0x3ffff] (bus address [0x0000-0xffff]) [ 9.186189] pci 0005:00:00.0: [10de:1ad0] type 01 class 0x060400 [ 9.193638] pci 0005:00:00.0: PME# supported from D0 D3hot D3cold [ 9.200626] pci 0005:00:00.0: PTM enabled (root), 16ns granularity With a Nvidia GT530 It boots to the prompt: [ 9.133576] tegra194-pcie 141a0000.pcie: iATU unroll: enabled [ 9.139371] tegra194-pcie 141a0000.pcie: Detected iATU regions: 8 outbound, 2 inbound [ 10.152565] tegra194-pcie 141a0000.pcie: Phy link never came up [ 11.164571] tegra194-pcie 141a0000.pcie: Phy link never came up [ 11.172598] tegra194-pcie 141a0000.pcie: PCI host bridge to bus 0005:00 [ 11.179999] pci_bus 0005:00: root bus resource [bus 00-ff] [ 11.185998] pci_bus 0005:00: root bus resource [mem 0x1c00000000-0x1f3fffffff pref] [ 11.194120] pci_bus 0005:00: root bus resource [mem 0x1f40000000-0x1ffffeffff] (bus address [0x40000000-0xfffeffff]) [ 11.205339] pci_bus 0005:00: root bus resource [io 0x30000-0x3ffff] (bus address [0x0000-0xffff]) [ 11.215464] pci 0005:00:00.0: [10de:1ad0] type 01 class 0x060400 [ 11.224039] pci 0005:00:00.0: PME# supported from D0 D3hot D3cold [ 11.231086] pci 0005:00:00.0: PTM enabled (root), 16ns granularity [ 11.268132] pci 0005:00:00.0: PCI bridge to [bus 01-ff] [ 11.275849] pcieport 0005:00:00.0: PME: Signaling with IRQ 54 [ 11.284553] pcieport 0005:00:00.0: AER: enabled with IRQ 54 [ 11.291322] pcieport 0005:00:00.0: bw_notification: enabled with IRQ 54 [ 11.299108] pci_bus 0005:01: busn_res: [bus 01-ff] is released [ 11.305119] pci_bus 0005:00: busn_res: [bus 00-ff] is released ... However, lspci doesn't list GT530. Before the patch GT530 was working properly. BR, Yousaf ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-15 15:41 ` Rob Herring 2020-12-15 19:44 ` Rob Herring @ 2020-12-15 20:52 ` Mian Yousaf Kaukab 2020-12-17 14:58 ` Rob Herring 1 sibling, 1 reply; 10+ messages in thread From: Mian Yousaf Kaukab @ 2020-12-15 20:52 UTC (permalink / raw) To: Rob Herring Cc: Vidya Sagar, lorenzo.pieralisi, bhelgaas, linux-pci, linux-kernel On Tue, Dec 15, 2020 at 09:41:47AM -0600, Rob Herring wrote: > On Tue, Dec 15, 2020 at 02:25:04PM +0100, Mian Yousaf Kaukab wrote: > > On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > > > Thanks Mian for bringing it to our notice. > > > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > > > file on top of linux-next? and does that solve the issue? > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > > > b/drivers/pci/controller/dwc/pcie-tegra194.c > > > index 5597b2a49598..1c9e9c054592 100644 > > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > > > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > > > *pp) > > > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > > > val); > > > } > > > > > > - dw_pcie_setup_rc(pp); > > > + //dw_pcie_setup_rc(pp); > > I still see the same issue with this change. > > Reverting b9ac0f9dc8ea works though. > > > > > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > > > > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > > > sure why calling it second time should create any issue for the enumeration > > > of devices behind a switch. Perhaps I need to spend more time to debug that > > > part. > > > In any case, since dw_pcie_setup_rc() is already part of > > > dw_pcie_host_init(), I think it can be removed from > > > tegra_pcie_prepare_host() implemention. > > I think the 2nd time is making the link go down is my guess. Tegra was > odd in that its start/stop link functions don't do link handling, so I > didn't implement those functions and left the link handling in the Tegra > driver. > > Can you try the below patch. It needs some more work as it breaks > endpoint mode. > > 8<-------------------------------------------------------------------- > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c > index 648e731bccfa..49bb487b16ae 100644 > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > @@ -907,9 +907,32 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, val); > } > > - dw_pcie_setup_rc(pp); > - > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > +} > + > +static int tegra_pcie_dw_host_init(struct pcie_port *pp) > +{ > + pp->bridge->ops = &tegra_pci_ops; > + > + tegra_pcie_prepare_host(pp); > + tegra_pcie_enable_interrupts(pp); > + > + return 0; > +} > + > +static int tegra_pcie_dw_link_up(struct dw_pcie *pci) > +{ > + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > + u32 val = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA); > + > + return !!(val & PCI_EXP_LNKSTA_DLLLA); > +} > + > +static int tegra_pcie_dw_start_link(struct dw_pcie *pci) > +{ > + u32 val, offset, speed, tmp; > + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > + struct pcie_port *pp = &pci->pp; > > /* Assert RST */ > val = appl_readl(pcie, APPL_PINMUX); > @@ -929,17 +952,6 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) > appl_writel(pcie, val, APPL_PINMUX); > > msleep(100); > -} > - > -static int tegra_pcie_dw_host_init(struct pcie_port *pp) > -{ > - struct dw_pcie *pci = to_dw_pcie_from_pp(pp); > - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > - u32 val, tmp, offset, speed; > - > - pp->bridge->ops = &tegra_pci_ops; > - > - tegra_pcie_prepare_host(pp); > > if (dw_pcie_wait_for_link(pci)) { > /* > @@ -975,7 +987,8 @@ static int tegra_pcie_dw_host_init(struct pcie_port *pp) > val &= ~PCI_DLF_EXCHANGE_ENABLE; > dw_pcie_writel_dbi(pci, offset, val); > > - tegra_pcie_prepare_host(pp); > + tegra_pcie_dw_host_init(pp); > + dw_pcie_setup_rc(pp); > > if (dw_pcie_wait_for_link(pci)) > return 0; > @@ -985,25 +998,6 @@ static int tegra_pcie_dw_host_init(struct pcie_port *pp) > PCI_EXP_LNKSTA_CLS; > clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]); > > - tegra_pcie_enable_interrupts(pp); > - > - return 0; > -} > - > -static int tegra_pcie_dw_link_up(struct dw_pcie *pci) > -{ > - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > - u32 val = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA); > - > - return !!(val & PCI_EXP_LNKSTA_DLLLA); > -} > - > -static int tegra_pcie_dw_start_link(struct dw_pcie *pci) > -{ > - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); > - > - enable_irq(pcie->pex_rst_irq); > - > return 0; > } > Boot is ok with this patch. Some improvement in lspci as well: # lspci 0001:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad2 (rev a1) 0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13) 0005:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad0 (rev a1) 0005:01:00.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab) BR, Yousaf ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-15 20:52 ` Mian Yousaf Kaukab @ 2020-12-17 14:58 ` Rob Herring 2020-12-17 17:06 ` Mian Yousaf Kaukab 0 siblings, 1 reply; 10+ messages in thread From: Rob Herring @ 2020-12-17 14:58 UTC (permalink / raw) To: Mian Yousaf Kaukab Cc: Vidya Sagar, lorenzo.pieralisi, bhelgaas, linux-pci, linux-kernel On Tue, Dec 15, 2020 at 09:52:35PM +0100, Mian Yousaf Kaukab wrote: > On Tue, Dec 15, 2020 at 09:41:47AM -0600, Rob Herring wrote: > > On Tue, Dec 15, 2020 at 02:25:04PM +0100, Mian Yousaf Kaukab wrote: > > > On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > > > > Thanks Mian for bringing it to our notice. > > > > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > > > > file on top of linux-next? and does that solve the issue? > > > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > index 5597b2a49598..1c9e9c054592 100644 > > > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > > > > *pp) > > > > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > > > > val); > > > > } > > > > > > > > - dw_pcie_setup_rc(pp); > > > > + //dw_pcie_setup_rc(pp); > > > I still see the same issue with this change. > > > Reverting b9ac0f9dc8ea works though. > > > > > > > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > > > > > > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > > > > sure why calling it second time should create any issue for the enumeration > > > > of devices behind a switch. Perhaps I need to spend more time to debug that > > > > part. > > > > In any case, since dw_pcie_setup_rc() is already part of > > > > dw_pcie_host_init(), I think it can be removed from > > > > tegra_pcie_prepare_host() implemention. > > > > I think the 2nd time is making the link go down is my guess. Tegra was > > odd in that its start/stop link functions don't do link handling, so I > > didn't implement those functions and left the link handling in the Tegra > > driver. > > > > Can you try the below patch. It needs some more work as it breaks > > endpoint mode. [...] > Boot is ok with this patch. Some improvement in lspci as well: Some improvement? Meaning not completely working still? > # lspci > 0001:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad2 (rev a1) > 0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13) > 0005:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad0 (rev a1) > 0005:01:00.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab) This patch was closer to the original flow, but would not have worked if DLFE disabled mode was needed. Please give this patch a try: 8<-------------------------------------------------------- diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c index 5597b2a49598..0515897b2f3a 100644 --- a/drivers/pci/controller/dwc/pcie-tegra194.c +++ b/drivers/pci/controller/dwc/pcie-tegra194.c @@ -853,12 +853,14 @@ static void config_gen3_gen4_eq_presets(struct tegra_pcie_dw *pcie) dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val); } -static void tegra_pcie_prepare_host(struct pcie_port *pp) +static int tegra_pcie_dw_host_init(struct pcie_port *pp) { struct dw_pcie *pci = to_dw_pcie_from_pp(pp); struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); u32 val; + pp->bridge->ops = &tegra_pci_ops; + if (!pcie->pcie_cap_base) pcie->pcie_cap_base = dw_pcie_find_capability(&pcie->pci, PCI_CAP_ID_EXP); @@ -907,10 +909,24 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, val); } - dw_pcie_setup_rc(pp); - clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); + return 0; +} + +static int tegra_pcie_dw_start_link(struct dw_pcie *pci) +{ + u32 val, offset, speed, tmp; + struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); + struct pcie_port *pp = &pci->pp; + bool retry = true; + + if (pcie->mode == DW_PCIE_EP_TYPE) { + enable_irq(pcie->pex_rst_irq); + return 0; + } + +retry_link: /* Assert RST */ val = appl_readl(pcie, APPL_PINMUX); val &= ~APPL_PINMUX_PEX_RST; @@ -929,19 +945,10 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp) appl_writel(pcie, val, APPL_PINMUX); msleep(100); -} - -static int tegra_pcie_dw_host_init(struct pcie_port *pp) -{ - struct dw_pcie *pci = to_dw_pcie_from_pp(pp); - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); - u32 val, tmp, offset, speed; - - pp->bridge->ops = &tegra_pci_ops; - - tegra_pcie_prepare_host(pp); if (dw_pcie_wait_for_link(pci)) { + if (!retry) + return 0; /* * There are some endpoints which can't get the link up if * root port has Data Link Feature (DLF) enabled. @@ -975,10 +982,11 @@ static int tegra_pcie_dw_host_init(struct pcie_port *pp) val &= ~PCI_DLF_EXCHANGE_ENABLE; dw_pcie_writel_dbi(pci, offset, val); - tegra_pcie_prepare_host(pp); + tegra_pcie_dw_host_init(pp); + dw_pcie_setup_rc(pp); - if (dw_pcie_wait_for_link(pci)) - return 0; + retry = false; + goto retry_link; } speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) & @@ -998,15 +1006,6 @@ static int tegra_pcie_dw_link_up(struct dw_pcie *pci) return !!(val & PCI_EXP_LNKSTA_DLLLA); } -static int tegra_pcie_dw_start_link(struct dw_pcie *pci) -{ - struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); - - enable_irq(pcie->pex_rst_irq); - - return 0; -} - static void tegra_pcie_dw_stop_link(struct dw_pcie *pci) { struct tegra_pcie_dw *pcie = to_tegra_pcie(pci); ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-17 14:58 ` Rob Herring @ 2020-12-17 17:06 ` Mian Yousaf Kaukab 2020-12-18 10:12 ` Mian Yousaf Kaukab 0 siblings, 1 reply; 10+ messages in thread From: Mian Yousaf Kaukab @ 2020-12-17 17:06 UTC (permalink / raw) To: Rob Herring Cc: Vidya Sagar, lorenzo.pieralisi, bhelgaas, linux-pci, linux-kernel On Thu, Dec 17, 2020 at 08:58:57AM -0600, Rob Herring wrote: > On Tue, Dec 15, 2020 at 09:52:35PM +0100, Mian Yousaf Kaukab wrote: > > On Tue, Dec 15, 2020 at 09:41:47AM -0600, Rob Herring wrote: > > > On Tue, Dec 15, 2020 at 02:25:04PM +0100, Mian Yousaf Kaukab wrote: > > > > On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > > > > > Thanks Mian for bringing it to our notice. > > > > > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > > > > > file on top of linux-next? and does that solve the issue? > > > > > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > index 5597b2a49598..1c9e9c054592 100644 > > > > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > > > > > *pp) > > > > > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > > > > > val); > > > > > } > > > > > > > > > > - dw_pcie_setup_rc(pp); > > > > > + //dw_pcie_setup_rc(pp); > > > > I still see the same issue with this change. > > > > Reverting b9ac0f9dc8ea works though. > > > > > > > > > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > > > > > > > > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > > > > > sure why calling it second time should create any issue for the enumeration > > > > > of devices behind a switch. Perhaps I need to spend more time to debug that > > > > > part. > > > > > In any case, since dw_pcie_setup_rc() is already part of > > > > > dw_pcie_host_init(), I think it can be removed from > > > > > tegra_pcie_prepare_host() implemention. > > > > > > I think the 2nd time is making the link go down is my guess. Tegra was > > > odd in that its start/stop link functions don't do link handling, so I > > > didn't implement those functions and left the link handling in the Tegra > > > driver. > > > > > > Can you try the below patch. It needs some more work as it breaks > > > endpoint mode. > > [...] > > > Boot is ok with this patch. Some improvement in lspci as well: > > Some improvement? Meaning not completely working still? > > > # lspci > > 0001:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad2 (rev a1) > > 0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13) > > 0005:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad0 (rev a1) > > 0005:01:00.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab) > > This patch was closer to the original flow, but would not have worked if > DLFE disabled mode was needed. > > Please give this patch a try: Thank you for the patch! Initial results with it looks very promising. I’ll get back to you tomorrow after running a few more tests. BR, Yousaf ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: dwc: tegra194: issue with card containing a bridge 2020-12-17 17:06 ` Mian Yousaf Kaukab @ 2020-12-18 10:12 ` Mian Yousaf Kaukab 0 siblings, 0 replies; 10+ messages in thread From: Mian Yousaf Kaukab @ 2020-12-18 10:12 UTC (permalink / raw) To: Rob Herring Cc: Vidya Sagar, lorenzo.pieralisi, bhelgaas, linux-pci, linux-kernel On Thu, Dec 17, 2020 at 06:06:35PM +0100, Mian Yousaf Kaukab wrote: > On Thu, Dec 17, 2020 at 08:58:57AM -0600, Rob Herring wrote: > > On Tue, Dec 15, 2020 at 09:52:35PM +0100, Mian Yousaf Kaukab wrote: > > > On Tue, Dec 15, 2020 at 09:41:47AM -0600, Rob Herring wrote: > > > > On Tue, Dec 15, 2020 at 02:25:04PM +0100, Mian Yousaf Kaukab wrote: > > > > > On Tue, Dec 15, 2020 at 05:45:59PM +0530, Vidya Sagar wrote: > > > > > > Thanks Mian for bringing it to our notice. > > > > > > Have you tried removing the dw_pcie_setup_rc(pp); call from pcie-tegra194.c > > > > > > file on top of linux-next? and does that solve the issue? > > > > > > > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > > b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > > index 5597b2a49598..1c9e9c054592 100644 > > > > > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c > > > > > > @@ -907,7 +907,7 @@ static void tegra_pcie_prepare_host(struct pcie_port > > > > > > *pp) > > > > > > dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, > > > > > > val); > > > > > > } > > > > > > > > > > > > - dw_pcie_setup_rc(pp); > > > > > > + //dw_pcie_setup_rc(pp); > > > > > I still see the same issue with this change. > > > > > Reverting b9ac0f9dc8ea works though. > > > > > > > > > > > > clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ); > > > > > > > > > > > > I took a quick look at the dw_pcie_setup_rc() implementation and I'm not > > > > > > sure why calling it second time should create any issue for the enumeration > > > > > > of devices behind a switch. Perhaps I need to spend more time to debug that > > > > > > part. > > > > > > In any case, since dw_pcie_setup_rc() is already part of > > > > > > dw_pcie_host_init(), I think it can be removed from > > > > > > tegra_pcie_prepare_host() implemention. > > > > > > > > I think the 2nd time is making the link go down is my guess. Tegra was > > > > odd in that its start/stop link functions don't do link handling, so I > > > > didn't implement those functions and left the link handling in the Tegra > > > > driver. > > > > > > > > Can you try the below patch. It needs some more work as it breaks > > > > endpoint mode. > > > > [...] > > > > > Boot is ok with this patch. Some improvement in lspci as well: > > > > Some improvement? Meaning not completely working still? > > > > > # lspci > > > 0001:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad2 (rev a1) > > > 0001:01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9171 (rev 13) > > > 0005:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad0 (rev a1) > > > 0005:01:00.0 PCI bridge: PLX Technology, Inc. Device 3380 (rev ab) > > > > This patch was closer to the original flow, but would not have worked if > > DLFE disabled mode was needed. > > > > Please give this patch a try: > Thank you for the patch! Initial results with it looks very promising. > I’ll get back to you tomorrow after running a few more tests. Rob, thank you for your efforts! This patch fixed the issue I was seeing. FWIW: Tested-by: Mian Yousaf Kaukab <ykaukab@suse.de> BR, Yousaf ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-12-18 10:13 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-15 10:24 dwc: tegra194: issue with card containing a bridge Mian Yousaf Kaukab 2020-12-15 12:15 ` Vidya Sagar 2020-12-15 13:25 ` Mian Yousaf Kaukab 2020-12-15 15:41 ` Rob Herring 2020-12-15 19:44 ` Rob Herring 2020-12-15 20:50 ` Mian Yousaf Kaukab 2020-12-15 20:52 ` Mian Yousaf Kaukab 2020-12-17 14:58 ` Rob Herring 2020-12-17 17:06 ` Mian Yousaf Kaukab 2020-12-18 10:12 ` Mian Yousaf Kaukab
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.