* [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC @ 2021-05-11 2:16 Ansuel Smith 2021-05-11 2:24 ` Matthew Wilcox 0 siblings, 1 reply; 12+ messages in thread From: Ansuel Smith @ 2021-05-11 2:16 UTC (permalink / raw) To: Russell King Cc: Ansuel Smith, Jonathan Corbet, Ard Biesheuvel, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, linux-doc, linux-kernel, linux-arm-kernel Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. Each pci line I/O space require 0x100000 and the current limit in the specific case of ipq8064 cause the malfunction of the second and the third line that are commonly used for wifi cards. Current IO space is set to support only 0x100000 space while for ipq806x if all 3 pcie lines are connected, it's required 0x300000 of space. Update the IO_SPACE_LIMIT to permit this larger space and update the documentation for the Mapping PCI I/O space memory end to the new value of ff0fffff. Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com> --- Documentation/arm/memory.rst | 2 +- arch/arm/include/asm/io.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/arm/memory.rst b/Documentation/arm/memory.rst index 0cb1e2938823..938fe74c9020 100644 --- a/Documentation/arm/memory.rst +++ b/Documentation/arm/memory.rst @@ -53,7 +53,7 @@ ffc00000 ffc7ffff Guard region ff800000 ffbfffff Permanent, fixed read-only mapping of the firmware provided DT blob -fee00000 feffffff Mapping of PCI I/O space. This is a static +fee00000 ff0fffff Mapping of PCI I/O space. This is a static mapping within the vmalloc space. VMALLOC_START VMALLOC_END-1 vmalloc() / ioremap() space. diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h index fc748122f1e0..76f1c668df5e 100644 --- a/arch/arm/include/asm/io.h +++ b/arch/arm/include/asm/io.h @@ -197,7 +197,7 @@ void __iomem *pci_remap_cfgspace(resource_size_t res_cookie, size_t size); #ifdef CONFIG_NEED_MACH_IO_H #include <mach/io.h> #elif defined(CONFIG_PCI) -#define IO_SPACE_LIMIT ((resource_size_t)0xfffff) +#define IO_SPACE_LIMIT ((resource_size_t)0x2fffff) #define __io(a) __typesafe_io(PCI_IO_VIRT_BASE + ((a) & IO_SPACE_LIMIT)) #else #define __io(a) __typesafe_io((a) & IO_SPACE_LIMIT) -- 2.30.2 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 2:16 [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC Ansuel Smith @ 2021-05-11 2:24 ` Matthew Wilcox 2021-05-11 2:32 ` Ansuel Smith 0 siblings, 1 reply; 12+ messages in thread From: Matthew Wilcox @ 2021-05-11 2:24 UTC (permalink / raw) To: Ansuel Smith Cc: Russell King, Jonathan Corbet, Ard Biesheuvel, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, linux-doc, linux-kernel, linux-arm-kernel On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. Do you really? I mean, yes, theoretically, I understand it, the hardware supports 64kB of I/O port space per root port. But I/O port space is rather deprecated these days. My laptop has precisely two devices with I/O ports, one with 64 bytes and the other with 32 bytes. Would you really suffer by allocating 16kB of I/O port space to each root port? ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 2:24 ` Matthew Wilcox @ 2021-05-11 2:32 ` Ansuel Smith 2021-05-11 4:26 ` Ard Biesheuvel 0 siblings, 1 reply; 12+ messages in thread From: Ansuel Smith @ 2021-05-11 2:32 UTC (permalink / raw) To: Matthew Wilcox Cc: Russell King, Jonathan Corbet, Ard Biesheuvel, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, linux-doc, linux-kernel, linux-arm-kernel On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > Do you really? I mean, yes, theoretically, I understand it, the > hardware supports 64kB of I/O port space per root port. But I/O > port space is rather deprecated these days. My laptop has precisely > two devices with I/O ports, one with 64 bytes and the other with 32 > bytes. Would you really suffer by allocating 16kB of I/O port > space to each root port? We were talking about this in the other wrong patch. I also think this much space looks wrong. The current ipq806x dts have this space so it's actually broken from a long time. The only reason pci worked before was because the pci driver didn't actually check if the settings were right. New kernel introduced more checks and this problem showed up. (to be more precise, the pci port are commonly used by the ath10k wifi and the second ath10k wifi fails to init because of this problem) If you can give me any hint on how to check if the space can be reduced I would be very happy to investigate it. In the driver I notice that the max buffer is set to 2k, could be this a hint? ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 2:32 ` Ansuel Smith @ 2021-05-11 4:26 ` Ard Biesheuvel 2021-05-11 12:15 ` Ansuel Smith 2021-05-17 10:26 ` Linus Walleij 0 siblings, 2 replies; 12+ messages in thread From: Ard Biesheuvel @ 2021-05-11 4:26 UTC (permalink / raw) To: Ansuel Smith Cc: Matthew Wilcox, Russell King, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > Do you really? I mean, yes, theoretically, I understand it, the > > hardware supports 64kB of I/O port space per root port. But I/O > > port space is rather deprecated these days. My laptop has precisely > > two devices with I/O ports, one with 64 bytes and the other with 32 > > bytes. Would you really suffer by allocating 16kB of I/O port > > space to each root port? > > We were talking about this in the other wrong patch. I also think this > much space looks wrong. The current ipq806x dts have this space so it's > actually broken from a long time. The only reason pci worked before was > because the pci driver didn't actually check if the settings were right. > New kernel introduced more checks and this problem showed up. (to be > more precise, the pci port are commonly used by the ath10k wifi and the > second ath10k wifi fails to init because of this problem) > If you can give me any hint on how to check if the space can be reduced > I would be very happy to investigate it. > In the driver I notice that the max buffer is set to 2k, could be this a > hint? > Could you share the output of lspci -vv from such a system? I agree with Matthew that fiddling with the size of the I/O space range probably papers over another problem, and with the odd exception, no PCIe card used on ARM systems actually uses their I/O BARs, even when they have them. (I used to carry a PCIe serial port card to UEFI plugfests because that was the only thing that would stop working if a system configured its I/O resource window incorrectly) ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 4:26 ` Ard Biesheuvel @ 2021-05-11 12:15 ` Ansuel Smith 2021-05-11 12:30 ` Ard Biesheuvel 2021-05-11 14:54 ` Russell King - ARM Linux admin 2021-05-17 10:26 ` Linus Walleij 1 sibling, 2 replies; 12+ messages in thread From: Ansuel Smith @ 2021-05-11 12:15 UTC (permalink / raw) To: Ard Biesheuvel Cc: Matthew Wilcox, Russell King, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > hardware supports 64kB of I/O port space per root port. But I/O > > > port space is rather deprecated these days. My laptop has precisely > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > space to each root port? > > > > We were talking about this in the other wrong patch. I also think this > > much space looks wrong. The current ipq806x dts have this space so it's > > actually broken from a long time. The only reason pci worked before was > > because the pci driver didn't actually check if the settings were right. > > New kernel introduced more checks and this problem showed up. (to be > > more precise, the pci port are commonly used by the ath10k wifi and the > > second ath10k wifi fails to init because of this problem) > > If you can give me any hint on how to check if the space can be reduced > > I would be very happy to investigate it. > > In the driver I notice that the max buffer is set to 2k, could be this a > > hint? > > > > Could you share the output of lspci -vv from such a system? > > I agree with Matthew that fiddling with the size of the I/O space > range probably papers over another problem, and with the odd > exception, no PCIe card used on ARM systems actually uses their I/O > BARs, even when they have them. (I used to carry a PCIe serial port > card to UEFI plugfests because that was the only thing that would stop > working if a system configured its I/O resource window incorrectly) Here is the output of lspci -vv 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 44 Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 I/O behind bridge: [disabled] Memory behind bridge: 08000000-081fffff [size=2M] Prefetchable memory behind bridge: [disabled] Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 000000004361bc88 Data: 0000 Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (ok), Width x1 (downgraded) TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+ RootCap: CRSVisible- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- AtomicOpsCtl: ReqEn- EgressBlck- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 RootCmd: CERptEn+ NFERptEn+ FERptEn+ RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 Kernel driver in use: pcieport lspci: Unable to load libkmod resources: error -12 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter Subsystem: Qualcomm Atheros Device cafe Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0/wifi@1,0 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 53 Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ Address: 000000004361bc88 Data: 0001 Masking: fffffffe Pending: 00000000 Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (ok), Width x1 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [158 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [160 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1SubCtl2: Kernel driver in use: ath10k_pci 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 46 Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 I/O behind bridge: [disabled] Memory behind bridge: 2e000000-2e1fffff [size=2M] Prefetchable memory behind bridge: [disabled] Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000435e8688 Data: 0000 Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- RootCap: CRSVisible- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- AtomicOpsCtl: ReqEn- EgressBlck- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 RootCmd: CERptEn+ NFERptEn+ FERptEn+ RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 Kernel driver in use: pcieport 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter Subsystem: Qualcomm Atheros Device cafe Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 54 Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ Address: 00000000435e8688 Data: 0001 Masking: fffffffe Pending: 00000000 Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [158 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [160 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1SubCtl2: Kernel driver in use: ath10k_pci ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 12:15 ` Ansuel Smith @ 2021-05-11 12:30 ` Ard Biesheuvel 2021-05-11 12:37 ` Ansuel Smith 2021-05-11 14:54 ` Russell King - ARM Linux admin 1 sibling, 1 reply; 12+ messages in thread From: Ard Biesheuvel @ 2021-05-11 12:30 UTC (permalink / raw) To: Ansuel Smith Cc: Matthew Wilcox, Russell King, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > port space is rather deprecated these days. My laptop has precisely > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > space to each root port? > > > > > > We were talking about this in the other wrong patch. I also think this > > > much space looks wrong. The current ipq806x dts have this space so it's > > > actually broken from a long time. The only reason pci worked before was > > > because the pci driver didn't actually check if the settings were right. > > > New kernel introduced more checks and this problem showed up. (to be > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > second ath10k wifi fails to init because of this problem) > > > If you can give me any hint on how to check if the space can be reduced > > > I would be very happy to investigate it. > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > hint? > > > > > > > Could you share the output of lspci -vv from such a system? > > > > I agree with Matthew that fiddling with the size of the I/O space > > range probably papers over another problem, and with the odd > > exception, no PCIe card used on ARM systems actually uses their I/O > > BARs, even when they have them. (I used to carry a PCIe serial port > > card to UEFI plugfests because that was the only thing that would stop > > working if a system configured its I/O resource window incorrectly) > > Here is the output of lspci -vv > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > I/O behind bridge: [disabled] > Memory behind bridge: 08000000-081fffff [size=2M] > Prefetchable memory behind bridge: [disabled] So this a MMIO window to the endpoint ... > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] ... and the endpoint has a single *MMIO* BAR of size 2 MiB. This has *nothing* to do with port I/O, which is what you are modifying with your patch. Did you check that the problem exists without the patch, and that the patch makes it go away? > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > Address: 000000004361bc88 Data: 0001 > Masking: fffffffe Pending: 00000000 > Capabilities: [70] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 5GT/s (ok), Width x1 (ok) > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > FRS- TPHComp- ExtTPHComp- > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > AtomicOpsCtl: ReqEn- > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > Capabilities: [158 v1] Latency Tolerance Reporting > Max snoop latency: 0ns > Max no snoop latency: 0ns > Capabilities: [160 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > L1SubCtl2: > Kernel driver in use: ath10k_pci > > 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 46 > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > I/O behind bridge: [disabled] > Memory behind bridge: 2e000000-2e1fffff [size=2M] > Prefetchable memory behind bridge: [disabled] > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ > Address: 00000000435e8688 Data: 0000 > Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 > DevCap: MaxPayload 128 bytes, PhantFunc 0 > ExtTag- RBE+ > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us > ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- > RootCap: CRSVisible- > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- > AtomicOpsCtl: ReqEn- EgressBlck- > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v1] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > RootCmd: CERptEn+ NFERptEn+ FERptEn+ > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- > FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 > Kernel driver in use: pcieport > > 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Subsystem: Qualcomm Atheros Device cafe > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 54 > Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > Address: 00000000435e8688 Data: 0001 > Masking: fffffffe Pending: 00000000 > Capabilities: [70] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok) > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > FRS- TPHComp- ExtTPHComp- > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > AtomicOpsCtl: ReqEn- > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > Capabilities: [158 v1] Latency Tolerance Reporting > Max snoop latency: 0ns > Max no snoop latency: 0ns > Capabilities: [160 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > L1SubCtl2: > Kernel driver in use: ath10k_pci ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 12:30 ` Ard Biesheuvel @ 2021-05-11 12:37 ` Ansuel Smith 2021-05-11 12:46 ` Ard Biesheuvel 0 siblings, 1 reply; 12+ messages in thread From: Ansuel Smith @ 2021-05-11 12:37 UTC (permalink / raw) To: Ard Biesheuvel Cc: Matthew Wilcox, Russell King, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote: > On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > > port space is rather deprecated these days. My laptop has precisely > > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > > space to each root port? > > > > > > > > We were talking about this in the other wrong patch. I also think this > > > > much space looks wrong. The current ipq806x dts have this space so it's > > > > actually broken from a long time. The only reason pci worked before was > > > > because the pci driver didn't actually check if the settings were right. > > > > New kernel introduced more checks and this problem showed up. (to be > > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > > second ath10k wifi fails to init because of this problem) > > > > If you can give me any hint on how to check if the space can be reduced > > > > I would be very happy to investigate it. > > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > > hint? > > > > > > > > > > Could you share the output of lspci -vv from such a system? > > > > > > I agree with Matthew that fiddling with the size of the I/O space > > > range probably papers over another problem, and with the odd > > > exception, no PCIe card used on ARM systems actually uses their I/O > > > BARs, even when they have them. (I used to carry a PCIe serial port > > > card to UEFI plugfests because that was the only thing that would stop > > > working if a system configured its I/O resource window incorrectly) > > > > Here is the output of lspci -vv > > > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > I/O behind bridge: [disabled] > > Memory behind bridge: 08000000-081fffff [size=2M] > > Prefetchable memory behind bridge: [disabled] > > So this a MMIO window to the endpoint > > ... > > > > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] > > ... and the endpoint has a single *MMIO* BAR of size 2 MiB. > > This has *nothing* to do with port I/O, which is what you are > modifying with your patch. > > Did you check that the problem exists without the patch, and that the > patch makes it go away? > > Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to init as it can't access the reg. Only the first pci wifi works but the second one fails to init. By increasing the limit all comes back to normal. What I really can't understand is if the big IO space set in the ipq8064 dtsi was wrong from the start and the ath10k fails to init just because is missconfigured. Any idea how to find the appropriate max IO space for the pci? > > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > > Address: 000000004361bc88 Data: 0001 > > Masking: fffffffe Pending: 00000000 > > Capabilities: [70] Express (v2) Endpoint, MSI 00 > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 5GT/s (ok), Width x1 (ok) > > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > > FRS- TPHComp- ExtTPHComp- > > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > > AtomicOpsCtl: ReqEn- > > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v2] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > > Capabilities: [158 v1] Latency Tolerance Reporting > > Max snoop latency: 0ns > > Max no snoop latency: 0ns > > Capabilities: [160 v1] L1 PM Substates > > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > > L1SubCtl2: > > Kernel driver in use: ath10k_pci > > > > 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0 > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 46 > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > I/O behind bridge: [disabled] > > Memory behind bridge: 2e000000-2e1fffff [size=2M] > > Prefetchable memory behind bridge: [disabled] > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > > BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ > > Address: 00000000435e8688 Data: 0000 > > Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 > > DevCap: MaxPayload 128 bytes, PhantFunc 0 > > ExtTag- RBE+ > > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- > > LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us > > ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ > > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) > > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- > > RootCap: CRSVisible- > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- > > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > > FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- > > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- > > AtomicOpsCtl: ReqEn- EgressBlck- > > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v1] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > RootCmd: CERptEn+ NFERptEn+ FERptEn+ > > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- > > FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 > > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 > > Kernel driver in use: pcieport > > > > 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > Subsystem: Qualcomm Atheros Device cafe > > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 54 > > Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > > Address: 00000000435e8688 Data: 0001 > > Masking: fffffffe Pending: 00000000 > > Capabilities: [70] Express (v2) Endpoint, MSI 00 > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok) > > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > > FRS- TPHComp- ExtTPHComp- > > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > > AtomicOpsCtl: ReqEn- > > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v2] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > > Capabilities: [158 v1] Latency Tolerance Reporting > > Max snoop latency: 0ns > > Max no snoop latency: 0ns > > Capabilities: [160 v1] L1 PM Substates > > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > > L1SubCtl2: > > Kernel driver in use: ath10k_pci ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 12:37 ` Ansuel Smith @ 2021-05-11 12:46 ` Ard Biesheuvel 2021-05-11 12:51 ` Ansuel Smith 0 siblings, 1 reply; 12+ messages in thread From: Ard Biesheuvel @ 2021-05-11 12:46 UTC (permalink / raw) To: Ansuel Smith Cc: Matthew Wilcox, Russell King, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, 11 May 2021 at 14:37, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote: > > On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > > > port space is rather deprecated these days. My laptop has precisely > > > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > > > space to each root port? > > > > > > > > > > We were talking about this in the other wrong patch. I also think this > > > > > much space looks wrong. The current ipq806x dts have this space so it's > > > > > actually broken from a long time. The only reason pci worked before was > > > > > because the pci driver didn't actually check if the settings were right. > > > > > New kernel introduced more checks and this problem showed up. (to be > > > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > > > second ath10k wifi fails to init because of this problem) > > > > > If you can give me any hint on how to check if the space can be reduced > > > > > I would be very happy to investigate it. > > > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > > > hint? > > > > > > > > > > > > > Could you share the output of lspci -vv from such a system? > > > > > > > > I agree with Matthew that fiddling with the size of the I/O space > > > > range probably papers over another problem, and with the odd > > > > exception, no PCIe card used on ARM systems actually uses their I/O > > > > BARs, even when they have them. (I used to carry a PCIe serial port > > > > card to UEFI plugfests because that was the only thing that would stop > > > > working if a system configured its I/O resource window incorrectly) > > > > > > Here is the output of lspci -vv > > > > > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > > I/O behind bridge: [disabled] > > > Memory behind bridge: 08000000-081fffff [size=2M] > > > Prefetchable memory behind bridge: [disabled] > > > > So this a MMIO window to the endpoint > > > > ... > > > > > > > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] > > > > ... and the endpoint has a single *MMIO* BAR of size 2 MiB. > > > > This has *nothing* to do with port I/O, which is what you are > > modifying with your patch. > > > > Did you check that the problem exists without the patch, and that the > > patch makes it go away? > > > > > > Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to > init as it can't access the reg. Only the first pci wifi works but the > second one fails to init. By increasing the limit all comes back to > normal. What I really can't understand is if the big IO space set in the > ipq8064 dtsi was wrong from the start and the ath10k fails to init just > because is missconfigured. Any idea how to find the appropriate max IO space > for the pci? > OK, so the entire second host bridge fails to probe, because there is no virtual address space left for the I/O window. Just change the 'downstream I/O' window size in the DT to 64k (0x10000) - I assume the current size (0x100000) is a typo anyway. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 12:46 ` Ard Biesheuvel @ 2021-05-11 12:51 ` Ansuel Smith 2021-05-11 14:55 ` Russell King - ARM Linux admin 0 siblings, 1 reply; 12+ messages in thread From: Ansuel Smith @ 2021-05-11 12:51 UTC (permalink / raw) To: Ard Biesheuvel Cc: Matthew Wilcox, Russell King, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, May 11, 2021 at 02:46:49PM +0200, Ard Biesheuvel wrote: > On Tue, 11 May 2021 at 14:37, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote: > > > On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > > > > port space is rather deprecated these days. My laptop has precisely > > > > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > > > > space to each root port? > > > > > > > > > > > > We were talking about this in the other wrong patch. I also think this > > > > > > much space looks wrong. The current ipq806x dts have this space so it's > > > > > > actually broken from a long time. The only reason pci worked before was > > > > > > because the pci driver didn't actually check if the settings were right. > > > > > > New kernel introduced more checks and this problem showed up. (to be > > > > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > > > > second ath10k wifi fails to init because of this problem) > > > > > > If you can give me any hint on how to check if the space can be reduced > > > > > > I would be very happy to investigate it. > > > > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > > > > hint? > > > > > > > > > > > > > > > > Could you share the output of lspci -vv from such a system? > > > > > > > > > > I agree with Matthew that fiddling with the size of the I/O space > > > > > range probably papers over another problem, and with the odd > > > > > exception, no PCIe card used on ARM systems actually uses their I/O > > > > > BARs, even when they have them. (I used to carry a PCIe serial port > > > > > card to UEFI plugfests because that was the only thing that would stop > > > > > working if a system configured its I/O resource window incorrectly) > > > > > > > > Here is the output of lspci -vv > > > > > > > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > > > I/O behind bridge: [disabled] > > > > Memory behind bridge: 08000000-081fffff [size=2M] > > > > Prefetchable memory behind bridge: [disabled] > > > > > > So this a MMIO window to the endpoint > > > > > > ... > > > > > > > > > > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] > > > > > > ... and the endpoint has a single *MMIO* BAR of size 2 MiB. > > > > > > This has *nothing* to do with port I/O, which is what you are > > > modifying with your patch. > > > > > > Did you check that the problem exists without the patch, and that the > > > patch makes it go away? > > > > > > > > > > Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to > > init as it can't access the reg. Only the first pci wifi works but the > > second one fails to init. By increasing the limit all comes back to > > normal. What I really can't understand is if the big IO space set in the > > ipq8064 dtsi was wrong from the start and the ath10k fails to init just > > because is missconfigured. Any idea how to find the appropriate max IO space > > for the pci? > > > > OK, so the entire second host bridge fails to probe, because there is > no virtual address space left for the I/O window. > > Just change the 'downstream I/O' window size in the DT to 64k > (0x10000) - I assume the current size (0x100000) is a typo anyway. Ok, so my fear were right... someone just typo the IO when the dtsi was pushed and was wrong from all that times. Much easier and cleaner solution. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 12:51 ` Ansuel Smith @ 2021-05-11 14:55 ` Russell King - ARM Linux admin 0 siblings, 0 replies; 12+ messages in thread From: Russell King - ARM Linux admin @ 2021-05-11 14:55 UTC (permalink / raw) To: Ansuel Smith Cc: Ard Biesheuvel, Matthew Wilcox, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, May 11, 2021 at 02:51:15PM +0200, Ansuel Smith wrote: > On Tue, May 11, 2021 at 02:46:49PM +0200, Ard Biesheuvel wrote: > > OK, so the entire second host bridge fails to probe, because there is > > no virtual address space left for the I/O window. > > > > Just change the 'downstream I/O' window size in the DT to 64k > > (0x10000) - I assume the current size (0x100000) is a typo anyway. > > Ok, so my fear were right... someone just typo the IO when the dtsi was > pushed and was wrong from all that times. Much easier and cleaner > solution. Great news. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 12:15 ` Ansuel Smith 2021-05-11 12:30 ` Ard Biesheuvel @ 2021-05-11 14:54 ` Russell King - ARM Linux admin 1 sibling, 0 replies; 12+ messages in thread From: Russell King - ARM Linux admin @ 2021-05-11 14:54 UTC (permalink / raw) To: Ansuel Smith Cc: Ard Biesheuvel, Matthew Wilcox, Jonathan Corbet, Linus Walleij, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, May 11, 2021 at 02:15:21PM +0200, Ansuel Smith wrote: > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Subsystem: Qualcomm Atheros Device cafe > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0/wifi@1,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 53 > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] It seems this Atheros device does not make any use of I/O, so this device should be fine. > 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Subsystem: Qualcomm Atheros Device cafe > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 54 > Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] This device also seems to not make use of any I/O. I don't see any reason probing of these two drivers should fail if there is no or reduced I/O space. Please can you share the kernel messages so we can see exactly what is failing and how? -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC 2021-05-11 4:26 ` Ard Biesheuvel 2021-05-11 12:15 ` Ansuel Smith @ 2021-05-17 10:26 ` Linus Walleij 1 sibling, 0 replies; 12+ messages in thread From: Linus Walleij @ 2021-05-17 10:26 UTC (permalink / raw) To: Ard Biesheuvel Cc: Ansuel Smith, Matthew Wilcox, Russell King, Jonathan Corbet, Florian Fainelli, Abbott Liu, Luis Chamberlain, Palmer Dabbelt, Linux Doc Mailing List, Linux Kernel Mailing List, Linux ARM On Tue, May 11, 2021 at 6:26 AM Ard Biesheuvel <ardb@kernel.org> wrote: > I used to carry a PCIe serial port > card to UEFI plugfests because that was the only thing that would stop > working if a system configured its I/O resource window incorrectly I've been looking for a thing like that for testing, I was actually thinking that it would perhaps be a good idea to add a I/O-resource requiring device to QEMU just to be able to test this kind of thing for completeness. Yours, Linus Walleij ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-05-17 10:26 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-05-11 2:16 [PATCH] arm: Enlarge IO_SPACE_LIMIT needed for some SoC Ansuel Smith 2021-05-11 2:24 ` Matthew Wilcox 2021-05-11 2:32 ` Ansuel Smith 2021-05-11 4:26 ` Ard Biesheuvel 2021-05-11 12:15 ` Ansuel Smith 2021-05-11 12:30 ` Ard Biesheuvel 2021-05-11 12:37 ` Ansuel Smith 2021-05-11 12:46 ` Ard Biesheuvel 2021-05-11 12:51 ` Ansuel Smith 2021-05-11 14:55 ` Russell King - ARM Linux admin 2021-05-11 14:54 ` Russell King - ARM Linux admin 2021-05-17 10:26 ` Linus Walleij
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).